name: differential-fuzzer
description: Information about the differential fuzzer tool, how to run it and use it catch bugs in Turso. Always load this skill when running this tool

Differential Fuzzer

Always load Debugging skill for reference

The differential fuzzer compares Turso results against SQLite for generated SQL statements to find correctness bugs.

Location

testing/differential-oracle/fuzzer/

Running the Fuzzer

Single Run

# Basic run (100 statements, random seed)
cargo run --bin differential_fuzzer

# With specific seed for reproducibility
cargo run --bin differential_fuzzer -- --seed 12345

# More statements with verbose output
cargo run --bin differential_fuzzer -- -n 1000 --verbose

# Keep database files after run (for debugging)
cargo run --bin differential_fuzzer -- --seed 12345 --keep-files

# All options
cargo run --bin differential_fuzzer -- \
  --seed <SEED>           # Deterministic seed
  -n <NUM>                # Number of statements (default: 100)
  -t <NUM>                # Number of tables (default: 2)
  -c <NUM>                # Columns per table (default: 5)
  --verbose               # Print each SQL statement
  --keep-files            # Persist .db files to disk

Continuous Fuzzing (Loop Mode)

# Run forever with random seeds
cargo run --bin differential_fuzzer -- loop

# Run 50 iterations
cargo run --bin differential_fuzzer -- loop 50

Docker Runner (CI/Production)

# Build and run from repo root
docker build -f testing/differential-oracle/fuzzer/docker-runner/Dockerfile -t fuzzer .
docker run -e GITHUB_TOKEN=xxx -e SLACK_WEBHOOK_URL=xxx fuzzer

Environment variables for docker-runner:
- TIME_LIMIT_MINUTES - Total runtime (default: 1440 = 24h)
- PER_RUN_TIMEOUT_SECONDS - Per-run timeout (default: 1200 = 20min)
- NUM_STATEMENTS - Statements per run (default: 1000)
- LOG_TO_STDOUT - Print fuzzer output (default: false)
- GITHUB_TOKEN - For auto-filing issues
- SLACK_WEBHOOK_URL - For notifications

Output Files

All output goes to simulator-output/ directory:

Reproducing Errors

Always follow these steps

1. Find the seed in the error output:
   INFO: Starting differential_fuzzer with config: SimConfig { seed: 12345, ... }

2. Re-run with that seed:
   bash cargo run --bin differential_fuzzer -- --seed 12345 --verbose --keep-files

3. Check output files:

4. simulator-output/test.sql - Find the failing statement (look for -- FAILED:)

5. simulator-output/schema.json - Check table structure at failure time

6. Create a minimal reproducer

7. Create reproducer in .sqltest or in .rs always load Debugging skill for reference

8. Compare behavior manually:
   If needed try to compare the behaviour and produce a report in the end.
   Always write to a tmp file first with Edit tool to test the sql and then pass it to the binaries.
   ```bash
   # Run failing SQL against SQLite
   sqlite3 :memory: < simulator-output/test.sql

# Run against tursodb CLI
tursodb :memory: < simulator-output/test.sql
```

Understanding Failures

Oracle Failure Types

1. Row set mismatch - Turso returned different rows than SQLite
2. Turso errored but SQLite succeeded - Turso rejected valid SQL
3. SQLite errored but Turso succeeded - Turso accepted invalid SQL
4. Schema mismatch - Tables/columns differ after DDL

Warning (non-fatal)

• Unordered LIMIT mismatch - LIMIT without ORDER BY may return different valid rows

Key Source Files

Tracing

Set RUST_LOG for more detailed output:

RUST_LOG=debug cargo run --bin differential_fuzzer -- --seed 12345