Compare
The open-source landscape
How saboteur fits among free and open-source mutation testing tools, taken from each project's own public documentation, checked 17 October 2026. Commercial tools are out of scope and are not covered. Numbers that come from a measurement are pinned to the measured versions and dated; nothing here is a claim about tools we did not measure.
Two open-source tools for Ruby
saboteur and mutineer are both MIT-licensed Ruby mutation testers that mutate one change at a time, run the covering tests and report what survived. The facts below come from each project's README, gemspec and RubyGems page.
| saboteur | mutineer | |
|---|---|---|
| Latest gem on RubyGems | 0.8.0 (18 October 2026) | 1.6.0 (8 October 2026) |
| License | MIT | MIT |
| Ruby required | 3.3 or newer | 3.4 or newer |
| Test frameworks | RSpec only, by design | Minitest (default) and RSpec |
| Runtime dependencies | prism and fiddle | None beyond Prism and stdlib |
| Execution | In-process mutation in forked workers; Landlock sandbox on Linux 6.2+ | Fork-isolated parallel workers; optional daemon backend for Rails |
| Diff mode | --since REF; pr preset: one baseline pass, 10-minute budget | --since REF |
| Test selection | Per-example coverage map; infers the specs named after each mutated file | Coverage-guided: only test files covering the mutant's line |
| Kill verification | The unchanged suite runs before mutating starts; a kill the suite cannot confirm stays out of the score | --baseline compares a run against a prior JSON report |
| Operators | All operators on by default; opt out per operator | Tier-1 set by default; Tier-2 via --operators |
| Reports | Terminal, self-contained HTML, Stryker-schema JSON, agent JSON | Human, JSON, self-contained HTML |
| CI | GitHub Action, advisory by default | GitHub Action on the Marketplace |
| Survivor review | Ignore list with reasons and triage states; opt-in survivor baseline | --baseline regression gate |
A measured comparison exists and stays pinned to its versions: saboteur 0.1.0 against mutineer 1.6.0, measured 10 October 2026, kept in the benchmark archive with raw data. It describes those exact versions, not today's tools. A fresh re-measurement against current releases is pending.
Beyond Ruby: Stryker-style tools
The same idea exists in most ecosystems. These are the established open-source tools per language, from their own sites and repositories:
| Tool | Language | Test setup | License |
|---|---|---|---|
| mutineer | Ruby | Minitest (default) and RSpec | MIT |
| PIT | Java and the JVM | Works with Ant, Maven, Gradle and others | Apache-2.0 |
| StrykerJS, Stryker.NET, Stryker4s | JavaScript and TypeScript, C#, Scala | Test-runner agnostic | Apache-2.0 |
| cargo-mutants | Rust | cargo test | MIT |
| mutmut | Python | pytest | BSD-3-Clause |
| Infection | PHP | PHPUnit, Codeception, PHPSpec | BSD-3-Clause |
Details worth knowing, all from the projects' own pages:
- Stryker is one design across three languages with more than 30 mutators and a shared report schema, mutation-testing-elements. saboteur's JSON report conforms to that schema, so Stryker dashboards and report viewers can read it.
- PIT made mutation testing practical on the JVM by being fast and integrating with existing build tooling, and its docs recommend the same workflow saboteur's
prpreset targets: run frequently against only the changed code. - Infection reports more than 33 million installs, enforces a Mutation Score Indicator in CI with
--min-msi, and its--git-diff-linesmutates only the lines a pull request touched. - mutmut remembers completed work so runs resume incrementally, and ships an interactive terminal browser for inspecting and re-testing mutants.
- cargo-mutants is a cargo subcommand; it builds each mutant and runs the crate's tests to find code that tests reach but do not check.
Shared lineage
saboteur's design notes record the public work it builds on in docs/prior-art.md:
- Mutation testing itself: DeMillo, Lipton and Sayward (1978), surveyed by Jia and Harman (2011).
- Diff-based mutation, arid-node suppression and surfacing survivors in review: Google's mutation testing papers (Petrović et al., 2018 and 2021).
- Coarse method-body mutations: Pitest's descartes research, concept only.
- The JSON report format: Stryker's mutation-testing-elements schema (Apache-2.0), output format only.
No source code, specs or fixtures from other mutation testing tools are used.
Where saboteur puts its effort
- RSpec-only, done properly. One framework lets the coverage map tie each method to the exact examples that cover it, so each mutant reruns only those. Why no Minitest.
- Verified kills. Every fresh kill is re-checked against the unchanged code; a kill the suite cannot confirm is rejected and stays out of the score. Timeout checks, other errors and cached kills reused by
--incrementalare not re-checked. See rejected kills. - Built for pull requests. The
prpreset scopes the run to changed methods with a 10-minute budget, and the GitHub Action runs advisory by default on every pull request. - Safety controls. A hazard guard, canary values, and time, memory and process-count limits on every mutant; a Landlock sandbox on Linux 6.2 and newer.
- Made for agents. A stable agent JSON output with a documented deprecation policy.
Sources
Checked 17 October 2026:
- mutineer: github.com/davidteren/mutineer and rubygems.org/gems/mutineer
- PIT: pitest.org and github.com/hcoles/pitest
- Stryker: stryker-mutator.io and github.com/stryker-mutator/stryker-js
- cargo-mutants: github.com/sourcefrog/cargo-mutants
- mutmut: github.com/boxed/mutmut
- Infection: infection.github.io and github.com/infection/infection