Testing

Prerequisites

• macOS host with sandbox-exec
• run tests outside any existing sandboxed session

Core Test Suite

./tests/run.sh

• ./tests/run.sh is the primary entrypoint for the macOS Bats suite.
• By default it runs the policy and surface suites.
• Tests live under tests/policy/, tests/surface/, and tests/e2e/.
• The suite is designed for macOS with sandbox-exec.

Run a specific suite:

./tests/run.sh policy
./tests/run.sh surface
./tests/run.sh e2e
./tests/run.sh all

Single-file ad hoc runs should use Bats directly:

bats tests/policy/integrations/docker.bats
bats tests/e2e/codex.bats

Some standard-suite integration checks also depend on host browser tooling. When present, CI installs agent-browser, Playwright's chromium-headless-shell download, and Google Chrome for those checks. Missing local host dependencies skip cleanly.

E2E Checks

These Bats tests drive real agent TUIs through tmux under Safehouse. Their contract is limited to startup/readiness and a basic prompt roundtrip, not full agent capability coverage. They currently cover:

• aider
• amp
• claude-code
• cline
• codex
• gemini
• goose
• kilo-code
• opencode v1 and v2 (separate installations of the same command)
• pi

CI runs this subset in the dedicated E2E TUI Tests (macOS) workflow.

Install the same agent dependencies used by GitHub Actions:

brew install bats-core parallel tmux node aider block-goose-cli
$(brew --prefix node)/bin/npm install --global \
  @anthropic-ai/claude-code \
  @google/gemini-cli \
  @kilocode/cli \
  @mariozechner/pi-coding-agent \
  @openai/codex \
  @sourcegraph/amp \
  cline \
  opencode-ai

CI installs these packages directly in the workflow job.

Install OpenCode v2 separately so it does not replace the v1 executable:

opencode_v2_prefix="$(mktemp -d /private/tmp/safehouse-opencode-v2.XXXXXX)"
npm install --global --prefix "$opencode_v2_prefix" \
  --no-fund --no-audit --min-release-age=4 @opencode/cli
export SAFEHOUSE_OPENCODE_V2_BIN="$opencode_v2_prefix/bin/opencode"
bats tests/e2e/opencode-v2.bats

The v2 tests use --standalone so the server inherits Safehouse's policy. Their isolated home and ancestor configuration files live outside the writable workdir and temporary directories to exercise configuration discovery permissions. The startup check needs no API key; the prompt roundtrip requires ANTHROPIC_API_KEY. An explicitly configured SAFEHOUSE_OPENCODE_V2_BIN must exist and report v2; invalid CI installations fail instead of skipping.

Depending on the specific test file, you may also need provider keys such as:

• OPENAI_API_KEY
• ANTHROPIC_API_KEY
• GEMINI_API_KEY

Without the required binary or API key, the corresponding test skips cleanly.