Quickstart

Quickstart and configuration

Needs Ruby 3.3 or newer on Linux or macOS. RSpec only, by design.

1. Install

bundle add saboteur --group test
# outside a Bundler project: gem install saboteur,
# then run plain `saboteur run` without bundle exec

2. Turn off SimpleCov for the run

saboteur needs Ruby's Coverage for itself. Most projects already guard SimpleCov with an environment variable.

3. Run it

bundle exec saboteur run                          # whole project
bundle exec saboteur run --since origin/main      # methods changed on this branch
bundle exec saboteur run --since origin/main --test spec/billing_spec.rb  # also scope the specs
bundle exec saboteur run --subject 'Billing::Invoice#total'
bundle exec saboteur run --subject Billing        # a namespace
bundle exec saboteur run --incremental            # reuse verdicts that did not change
bundle exec saboteur run --jobs 6                 # parallel workers
bundle exec saboteur run --max-duration 10m       # stop before the next subject when the budget is spent
bundle exec saboteur pr                           # PR loop: since main, baseline x1, 10-minute budget
bundle exec saboteur audit                        # whole project, all operators, JSON and HTML reports

Every mutation run first runs the unchanged tests twice, even when the coverage cache is warm (replay of one mutant skips this): a kill only counts when the same tests pass on the unchanged code. --baseline-passes 1 makes it one run, for the fast PR loop. Since 0.7.0 the run infers the specs named after each mutated file and runs only those, baseline passes included. A mutated file with no conventional spec stops with exit 2 before anything boots and names both ways out: pass --test once per spec path (one path per flag: --test spec/billing_spec.rb --test spec/payments_spec.rb), or --all to keep the whole configured suite (the 0.6.0 behavior).

4. Configure (optional)

# config/saboteur.rb
Saboteur.configure do |c|
  c.include = ["lib/**/*.rb"]
  c.exclude = ["lib/my_gem/version.rb"]
  c.timeout_factor = 3        # deadline = 3 x the tests' recorded time
  c.max_memory_mb = 2048      # memory growth cap per mutant
  c.max_processes = 64
  c.seed = 1
end

Rails apps

The defaults are for plain gems. A Rails app needs its boot file and app/ paths:

# config/saboteur.rb
Saboteur.configure do |c|
  c.include = ["app/**/*.rb"]
  c.require = ["./config/environment"]
  c.rails = true
  c.jobs = 4                  # optional, default is 1
end

Run with RAILS_ENV=test.

Check the environment

saboteur doctor reports Ruby, coverage, integration, Rails/SimpleCov and boot-file checks. Add --suite to also run the unchanged suite twice with the chosen seed (--test PATH limits those runs to some specs). It writes no mutation reports or coverage/verdict caches; boot files and tests still have their normal side effects. It warns when an ignore subject matches no discovered method; malformed ignore files fail before target boot.

Opt-in survivor baseline

saboteur baseline check runs every selected mutant fresh and compares reviewed survivors. It does not skip accepted mutants, change verdicts or change scores. Use whole-project or --subject scope; diff, replay, incremental reuse and score thresholds are refused.

bundle exec saboteur baseline propose \
  --baseline baseline.json \
  --proposal proposal.json
# Edit survivor reasons in proposal.json:
# document.accepted[*].reason only.
bundle exec saboteur baseline accept \
  --baseline baseline.json \
  --proposal proposal.json
bundle exec saboteur baseline check \
  --baseline baseline.json

Keep the private execution evidence in the configured cache directory until acceptance. Acceptance verifies that evidence and the current input fingerprint before publishing the manifest. Caps, uncovered work, empty generation, errors, resource-limit stops, changed inputs or missing results make a check incomplete, even with no new survivors.

Check exits 0 only for a compatible, complete, clean result; 1 needs survivor review; 2 means incompatible or invalid configuration; 3 means incomplete execution or tool failure. Local evidence is not remote attestation and does not prove external state stability. The ignore file below remains a separate mechanism.

5. Accept a survivor with a reason

# config/saboteur_ignore.rb
Saboteur.ignore do |i|
  i.mutation "1a2b3c4d5e6f7a8b",
             subject: "Billing::Invoice#total",
             reason: "equivalent: amounts are always whole cents"
end

An entry without a reason is rejected. Exit codes for saboteur run: 0 no unignored survivors (survivors on the ignore list do not fail the run), or with --threshold the score is at or above it even if mutants survived; 1 an unignored survivor, or with --threshold the score is below it or missing; 2 bad configuration or a missing source or test; 3 the tool failed, or more than half of the executed mutants errored (with --threshold, any errored or resource-limited mutant); 4 the --max-duration budget stopped the run before the next subject, so the score is partial (--max-duration-ok leaves the exit to the plain verdict). saboteur run --agent is strict: it prints one JSON verdict and exits 0 clean, 1 unignored survivors, 2 invalid invocation, 3 incomplete. The JSON report goes to .saboteur/report.json.