Development Tools

This guide covers Ada’s utility scripts, the scripts tooling container, development workflows, and version management.

Version Management 

Ada uses Semantic Versioning with Conventional Commits for automated changelog generation.

Quick Start 

# Check current version
./scripts/version.sh current

# Suggest next version based on commits
./scripts/version.sh suggest

# Generate changelog preview
./scripts/version.sh changelog

# Bump version and create tag
./scripts/version.sh bump minor "Feature Name"

See: Version Management for complete guide.

Conventional Commits 

Commit messages follow this format:

type(scope): subject

Optional body

Types:

feat: New feature → MINOR bump
fix: Bug fix → PATCH bump
docs: Documentation → PATCH bump
perf: Performance → MINOR bump
refactor:, test:, chore: → PATCH bump
BREAKING CHANGE: in body → MAJOR bump

Examples:

git commit -m "feat: add Wikipedia specialist"
git commit -m "fix(matrix): handle rate limiting"
git commit -m "docs: update getting started guide"

Validation 

Install commit message validation (optional):

ln -sf ../../scripts/validate-commit.sh .git/hooks/commit-msg

Now invalid commits are rejected:

$ git commit -m "fixed bug"
❌ Invalid commit message format!

$ git commit -m "fix: resolve memory leak"
✓ Success!

Scripts Tooling Container 

Overview 

All utility scripts should be run via the dedicated scripts Docker service, which provides:

Consistent Python environment (same as brain service)
All dependencies pre-installed
Network access to chroma, ollama, etc.
Proper module imports (brain, rag modules available)

Why Use the Container?

Problem: Running scripts directly on host causes issues:

❌ Different Python versions
❌ Missing dependencies
❌ Can’t reach Docker services (chroma, ollama)
❌ Import errors (can’t find brain/rag modules)
❌ Environment variable mismatches

Solution: Scripts container provides:

✅ Same Python 3.13 as brain
✅ All dependencies from pyproject.toml
✅ Network access via Docker Compose
✅ Clean imports with PYTHONPATH set
✅ Consistent environment variables

Usage Pattern 

Quick Way (Recommended)

# Use the convenience wrapper
./scripts/run.sh health          # Run health check
./scripts/run.sh test            # Run test suite
./scripts/run.sh migrate         # Run migration (with confirmation)
./scripts/run.sh consolidate     # Run consolidation
./scripts/run.sh shell           # Interactive Python shell
./scripts/run.sh bash            # Bash shell in container
./scripts/run.sh my_script.py    # Run custom script

Direct Way (Full Control)

# Run any script directly
docker compose run --rm scripts python /app/scripts/<script_name>.py

# Examples
docker compose run --rm scripts python /app/scripts/health_check_chroma.py
docker compose run --rm scripts python /app/scripts/migrate_chroma_http.py

Available Scripts 

Health & Testing 

Pytest Test Suite

Modern test suite using pytest with fixtures and better reporting.

# Run all tests
./scripts/run.sh test
# Or directly:
docker compose run --rm scripts pytest

# Run specific test file
docker compose run --rm scripts pytest tests/test_rag.py

# Run with more verbosity
docker compose run --rm scripts pytest -vv

# Run only tests matching a pattern
docker compose run --rm scripts pytest -k "memory"

# Skip slow tests
docker compose run --rm scripts pytest -m "not slow"

Test files:

tests/test_rag.py - RAG store retrieval tests
tests/test_prompt_builder.py - Prompt building tests
tests/test_specialists.py - Specialist system tests

Health Check Script

Comprehensive health check for Chroma database and RAG system (operational tool, not a test).

docker compose run --rm scripts python /app/scripts/health_check_chroma.py

Checks:

Chroma server connectivity
Collection existence and document count
Embedding generation
Memory/FAQ/turn query functionality
Specialist docs retrieval
Query consistency
Persona loading

Exit codes: 0 (healthy), 1 (unhealthy)

Data Management 

Migrate Chroma Collection

Migrate Chroma collection to HTTP-compatible format with fresh embeddings.

docker compose run --rm scripts python /app/scripts/migrate_chroma_http.py

What it does:

Backs up existing collection
Deletes old collection
Creates fresh collection
Re-generates embeddings for all documents
Verifies query functionality

When to use:

After upgrading Chroma
When queries return unexpected results
When switching from embedded to HTTP mode

Import FAQ

Import FAQs from JSONL file into Chroma.

docker compose run --rm scripts python /app/scripts/import_faq.py

Import Conversation Turns

Import conversation turns from CSV into Chroma.

docker compose run --rm scripts python /app/scripts/import_turns_csv.py

Load Persona

Load persona from markdown file into Chroma.

docker compose run --rm scripts python /app/scripts/load_persona.py

Consolidate Memories

Run nightly memory consolidation (creates daily/weekly summaries).

docker compose run --rm scripts python /app/scripts/consolidate_memories.py

Note

This runs automatically via memory-consolidation service at 02:00 UTC.

Backup Management 

Upload Backups to B2

Upload Chroma backups to Backblaze B2.

docker compose run --rm scripts python /app/scripts/upload_backups_b2.py

Note

This runs automatically via sqlite-backup service hourly.

Development Patterns 

Similar Tools in Other Ecosystems 

This pattern is well-established in containerized development:

Rails: docker compose run --rm web bin/rails console
Django: docker compose run --rm web python manage.py shell
Node.js: docker compose run --rm node npm test
Kubernetes: Init containers for migrations/setup

When to Use the Scripts Container 

✅ Use the scripts container for:

Running health checks
Running tests
Database migrations
Data imports/exports
One-off maintenance tasks
Interactive debugging

❌ Don’t use for:

Running the main application (use brain service)
Anything that needs to stay running (use a regular service)

Running Tests in CI/CD 

# .gitlab-ci.yml / .github/workflows/test.yml
test:
  script:
    - docker compose run --rm scripts pytest
    - docker compose run --rm scripts python /app/scripts/health_check_chroma.py

Pre-Deployment Health Check 

#!/bin/bash
# pre-deploy.sh
docker compose run --rm scripts python /app/scripts/health_check_chroma.py
if [ $? -eq 0 ]; then
  echo "Health check passed, proceeding with deployment"
  docker compose up -d
else
  echo "Health check failed, aborting deployment"
  exit 1
fi

Interactive Debugging 

# Start an interactive Python session with all dependencies
docker compose run --rm scripts python

# Then in Python:
>>> from brain.rag_store import RagStore
>>> store = RagStore()
>>> results = store.retrieve_memories("test query", k=5)
>>> print(results)

Adding New Scripts 

Create script in scripts/ directory

Import brain/rag modules as needed:

from brain.rag_store import RagStore
from brain import config

Run via scripts container:

docker compose run --rm scripts python /app/scripts/new_script.py

No rebuild needed! Scripts are mounted as volumes.

Troubleshooting 

Script Not Found 

Problem:

docker compose run --rm scripts python /app/scripts/my_script.py
# Error: No such file

Solution: Check that script exists in scripts/ directory. Path should be /app/scripts/, not ./scripts/.

Import Errors 

Problem:

ModuleNotFoundError: No module named 'brain'

Solution: Use the scripts container, not host Python. The PYTHONPATH is set correctly in the container.

Can’t Reach Chroma/Ollama 

Problem:

httpx.ConnectError: [Errno 111] Connection refused

Solution:

Ensure services are running: docker compose ps
Use service names (chroma, ollama), not localhost
Check environment variables: CHROMA_URL=http://chroma:8000

Permission Errors 

Problem:

PermissionError: [Errno 13] Permission denied

Solution: Scripts are mounted read-write. Check file permissions on host or run with appropriate user.

Architecture 

Container Configuration 

The scripts service is defined in compose.yaml with:

Base: scripts/Dockerfile (Python 3.13-slim)
Profile: tools (not started by default)
Dependencies: brain, rag, scripts directories mounted
Environment: Same as brain service
Network: Can reach chroma, ollama by service name
No ports: Not a server, just a runner

Dockerfile Structure 

scripts/Dockerfile includes:

FROM python:3.13-slim

# Install uv package manager
RUN pip install --no-cache-dir uv

# Copy project files
COPY pyproject.toml uv.lock ./
COPY brain/ /app/brain/
COPY scripts/ /app/scripts/
COPY tests/ /app/tests/

# Install dependencies (including dev extras)
RUN uv venv && uv sync --frozen --extra dev

# Set PATH and PYTHONPATH
ENV PATH="/app/.venv/bin:${PATH}"
ENV PYTHONPATH="/app:/app/brain"

WORKDIR /app

Volume Mounts 

Scripts and tests are volume-mounted for live updates:

volumes:
  - ./brain:/app/brain:ro
  - ./scripts:/app/scripts
  - ./tests:/app/tests
  - ./seed:/app/seed:ro
  - ./persona.md:/app/persona.md:ro
  - ./data:/data

This means you can edit scripts/tests without rebuilding the container.

Best Practices 

Always use scripts container: Consistent environment
Exit codes matter: Scripts should return 0 (success) or 1 (failure)
Log to stdout: Docker captures logs automatically
Handle errors gracefully: Use try/except, print clear error messages
Document usage: Add script to this documentation with description
Test in container: Don’t rely on host Python working
Use fixtures: Share common setup in tests/conftest.py
Version control: Commit scripts and tests to git

Development Workflow 

Typical Development Cycle 

Start services:
```
docker compose up -d
```
Make changes to code
Run tests:
```
./scripts/run.sh test
```
Health check:
```
./scripts/run.sh health
```
Debug if needed:
```
./scripts/run.sh shell
```
Commit and push

Adding New Features 

Write the feature code
Add tests in tests/test_<feature>.py
Run tests to verify: ./scripts/run.sh test
Add documentation to Sphinx docs
Update CHANGELOG (if applicable)
Create pull request

Debugging Issues 

Check logs:

docker compose logs brain
docker compose logs chroma

Interactive debugging:
```
./scripts/run.sh shell
```
Health check:
```
./scripts/run.sh health
```

Test specific component:

docker compose run --rm scripts pytest tests/test_rag.py -vv

Code Quality 

Linting and Formatting 

(Future) Add code quality tools:

# Format code
docker compose run --rm scripts black brain/ scripts/

# Lint code
docker compose run --rm scripts ruff brain/ scripts/

# Type checking
docker compose run --rm scripts mypy brain/

Pre-commit Hooks 

(Future) Set up pre-commit hooks:

# .pre-commit-config.yaml
repos:
  - repo: https://github.com/psf/black
    rev: 23.1.0
    hooks:
      - id: black
  - repo: https://github.com/charliermarsh/ruff-pre-commit
    rev: v0.0.260
    hooks:
      - id: ruff

Resources 

See Testing Guide for comprehensive testing guide
See API Usage Guide for API documentation
See API Reference for code-level API reference
Docker Compose Documentation
uv Package Manager