| Title: | Mutation Testing |
| Version: | 0.3.0 |
| Description: | Performs mutation testing for 'R' packages to assess the effectiveness of a test suite. It mutates source files, runs package tests against each mutant, and reports which mutants were killed, survived, or timed out. Parallel execution is supported for larger code bases. The optional 'imputesrcref' package, available from https://github.com/PRL-PRG/imputesrcref, improves source locations when installed. |
| URL: | https://github.com/PRL-PRG/mutator, https://prl-prg.github.io/mutator/ |
| BugReports: | https://github.com/PRL-PRG/mutator/issues |
| License: | GPL (≥ 3) |
| Encoding: | UTF-8 |
| Imports: | jsonlite, httr, pkgload, testthat, future, furrr, callr, covr, R6 |
| Suggests: | xml2, pkgbuild, cli, pbmcapply, knitr, rmarkdown, tinytest |
| Enhances: | imputesrcref |
| LinkingTo: | testthat |
| Config/testthat/edition: | 3 |
| VignetteBuilder: | knitr |
| SystemRequirements: | C++17 |
| Config/roxygen2/version: | 8.0.0 |
| NeedsCompilation: | yes |
| Packaged: | 2026-10-20 16:40:52 UTC; pierre |
| Author: | Pierre Donat-Bouillud
|
| Maintainer: | Pierre Donat-Bouillud <pierre.donat.bouillud@fit.cvut.cz> |
| Repository: | CRAN |
| Date/Publication: | 2026-10-21 16:50:07 UTC |
Call OpenAI API
Description
Makes a POST request to the OpenAI Chat Completions API.
Usage
call_openai_api(prompt, config, max_attempts = 8L)
Arguments
prompt |
The prompt to send to the API |
config |
API configuration with key and model information Requests refused for rate limiting (HTTP 429) or by an overloaded gateway
(502, 503, 504) are retried with exponential backoff, honouring a
|
max_attempts |
Maximum number of requests, including the first one. |
Value
On success, the parsed API response. On failure, an
openai_api_error object: a list with a message describing the cause
(HTTP status plus response body, or the network error), so callers can
surface why a request failed rather than a bare NULL.
Create a prompt for equivalent mutant detection
Description
Generates a well-formatted prompt for the OpenAI API to analyze if mutants are equivalent to the original code.
Usage
create_equivalent_mutant_prompt(original_code, mutant_details)
Arguments
original_code |
String containing the original source code |
mutant_details |
List of mutant details including IDs and mutation info |
Value
A formatted prompt string for the OpenAI API
Get OpenAI API configuration
Description
Resolves the API key, model and base URL used for equivalent-mutant detection. Each field is resolved independently, with this precedence (highest first):
Usage
get_openai_config(dir = getwd())
Arguments
dir |
Directory to search for a |
Details
values set with
set_openai_config();a
.openai_configfile indir(a human-readable "field: value" file that is parsed, never executed);the environment variables
OPENAI_API_KEY,OPENAI_MODEL,OPENAI_BASE_URLandOPENAI_MAX_PARALLEL_REQUESTS;built-in defaults (model
"gpt-4", the public OpenAI base URL).
Value
A list with elements api_key, model, base_url and
max_parallel_requests (an integer cap on concurrent API requests, or
NA for no cap).
Examples
config <- get_openai_config()
names(config)
Identify equivalent mutants using OpenAI API
Description
Analyzes survived mutants to determine if they are functionally equivalent to the original code using OpenAI's language models.
Usage
identify_equivalent_mutants(
src_file,
survived_mutants,
api_config = NULL,
batch_size = 25,
workers = 1,
report = TRUE
)
Arguments
src_file |
Path to the original source file |
survived_mutants |
List of mutants that survived test execution |
api_config |
Optional API configuration (will be loaded if NULL) |
batch_size |
Maximum number of mutants sent in a single API request. Smaller batches keep each response short enough to avoid truncation (which silently drops verdicts) and let batches run concurrently. Defaults to 25. |
workers |
Number of API requests to run concurrently (requires a forking platform). Defaults to 1 (sequential). |
report |
Whether to print per-file progress and an equivalence summary
to the console. Defaults to |
Value
Updated list of survived mutants with equivalence information
Examples
src <- tempfile(fileext = ".R")
writeLines("add <- function(x, y) x + y", src)
survived <- list(mutant_001 = list(mutation_info = "x + y -> x - y"))
suppressWarnings(identify_equivalent_mutants(
src,
survived,
api_config = list(api_key = "", model = "gpt-4")
))
Generate Mutants for a Single R File
Description
Creates mutants for a single R source file by combining AST-based mutations from the C++ mutation engine with fallback line-deletion mutants.
Usage
mutate_file(
src_file,
out_dir,
max_mutants = NULL,
max_line_deletions = 5,
operators = NULL
)
Arguments
src_file |
Path to an R source file. |
out_dir |
Directory where mutant files are written. This argument is required so that the function never writes to the current working directory unless the caller explicitly requests it. |
max_mutants |
Optional cap on the number of returned mutants. If set, a random subset of generated mutants is returned. |
max_line_deletions |
Maximum number of line-deletion mutants generated
per file (a random subset of deletable lines). These complement the
AST-based statement deletions by also covering top-level / non-block lines.
Use |
operators |
Mutation operators to apply: a character vector of
operator ids and families, read left to right, where a |
Details
Mutants that cannot change behaviour are not written: those whose code is
identical to the original or to another mutant, and those that simple rules
show to be equivalent (for example paste to paste0 with a single
argument). A message gives their number per file.
Value
A list of mutants. Each element contains:
pathPath to the mutant file.
infoFormatted mutation metadata (file, source range, and details).
locMachine-readable location: a list with
file_path,start_line, andend_line(the latter twoNAwhen unavailable), andoperator_id, the id of the mutation operator that produced the mutant (e.g."rel_swap","stmt_delete").
Examples
src <- tempfile(fileext = ".R")
writeLines("add <- function(x, y) x + y", src)
mutants <- mutate_file(src, out_dir = tempfile("mutations_"), max_mutants = 1)
length(mutants)
Run Mutation Testing for an R Package
Description
Mutates all .R files under a package's R/ directory, runs the package's
tests against each mutant in parallel, and summarizes mutation outcomes.
Usage
mutate_package(
pkg_dir,
cores = max(1, parallel::detectCores() - 2),
isFullLog = FALSE,
detectEqMutants = FALSE,
mutation_dir = NULL,
max_mutants = NULL,
timeout_seconds = NULL,
config_dir = getwd(),
max_line_deletions = 0,
cran = TRUE,
fail_fast = TRUE,
isolate = FALSE,
exclude_files = NULL,
strategy = c("auto", "testthat", "tinytest", "tinytest-installed", "installed"),
coverage_guided = TRUE,
coverage_backend = c("record_tests", "per_file"),
target_margin = NULL,
confidence = 0.95,
max_show = 50L,
operators = NULL
)
Arguments
pkg_dir |
Path to the package directory. |
cores |
Number of parallel workers used for mutant test execution. |
isFullLog |
Logical; if |
detectEqMutants |
Logical; if |
mutation_dir |
Optional directory to store generated mutant files.
If |
max_mutants |
Sample that number of mutants for testing. If |
timeout_seconds |
Optional timeout in seconds for each mutant run.
If |
config_dir |
Directory searched for a |
max_line_deletions |
Maximum number of line-deletion mutants per |
cran |
Logical; if |
fail_fast |
Logical; if |
isolate |
Logical; if |
exclude_files |
Optional character vector of shell-style glob patterns
(e.g. |
strategy |
Test strategy to use. |
coverage_guided |
Logical; if |
coverage_backend |
How |
target_margin |
Optional desired half-width of the confidence interval on
the mutation score, as a proportion (e.g. |
confidence |
Confidence level for |
max_show |
Maximum number of surviving mutants to print to the console;
the remainder are summarised as "... and N more" but always remain in the
returned |
operators |
Mutation operators to apply: a character vector of
operator ids and families, read left to right, where a |
Details
The example is not run during routine automated checks because it creates and mutation-tests a throwaway package, which is too slow for that context.
Test strategy is, by default, detected automatically:
If
tests/testthat/exists, the mutant is loaded in-process withpkgload::load_all()(no installation) and its tests are run the way the package's owntests/testthat.Rharness runs them, i.e. with the same arguments (notably anyfilter) that the harness passes totestthat::test_check(), viatestthat::test_dir().Otherwise, if
inst/tinytest/exists, the mutant is loaded in-process withpkgload::load_all()(no installation) and its tests are run withtinytest::run_test_dir("inst/tinytest").Otherwise, if
tests/exists, mutator installs the mutant package with--install-testsand runstools::testInstalledPackage().
Pass strategy to override this (for example to run a testthat or tinytest
package through the slower installed-tests path for comparison).
Value
An invisible list with four components:
package_mutantsNamed list with mutant path, mutation info, status, and optional equivalence flags.
test_resultsNamed list mapping mutant IDs to statuses:
"KILLED","SURVIVED", or"HANG".timingNamed list of phase durations in seconds:
baseline,generation,test_execution, andequivalence_detection.summaryNamed list with
generated,tested,killed,hanged,survived,mutation_score,mutation_score_ci(a length-2 percentage vector, orNULLwhen no sampling occurred),confidence, andby_operator, a data frame with one row per mutation operator and columnsoperator,tested,killed,hanged,survivedandmutation_score.
Examples
pkg <- file.path(tempdir(), "examplepkg")
dir.create(file.path(pkg, "R"), recursive = TRUE, showWarnings = FALSE)
dir.create(file.path(pkg, "tests", "testthat"), recursive = TRUE, showWarnings = FALSE)
writeLines(c(
"Package: examplepkg",
"Title: Example Package",
"Version: 0.0.1",
"Description: Minimal package for a mutator example.",
"License: GPL-3",
"Encoding: UTF-8"
), file.path(pkg, "DESCRIPTION"))
writeLines("export(add)", file.path(pkg, "NAMESPACE"))
writeLines("add <- function(x, y) x + y", file.path(pkg, "R", "add.R"))
writeLines(
"testthat::expect_equal(add(1, 2), 3)",
file.path(pkg, "tests", "testthat", "test-add.R")
)
result <- mutate_package(pkg, cores = 1, max_mutants = 1, timeout_seconds = 10)
names(result)
List the Mutation Operators
Description
Returns the mutation operators that mutator knows about. Use their ids, or
the names of their families, in the operators argument of mutate_file()
and mutate_package().
Usage
mutation_operators()
Value
A data frame with one row per operator and columns id, family,
default (whether it is part of the "default" group), description and
example.
Selecting operators
operators is a character vector read from left to right. Each element is
an operator id, a family name (e.g. "relational"), "default" or "all".
An element prefixed with - removes operators instead of adding them. If
the first element is a removal, the selection starts from "default".
When operators is NULL, the option mutator.operators is used, and
"default" if that is unset.
Examples
mutation_operators()
# Default set plus the constants family, without NA retyping:
src <- tempfile(fileext = ".R")
writeLines("f <- function(x) if (x > 0) x + 1 else NA", src)
mutants <- mutate_file(src, out_dir = tempfile("mutations_"),
operators = c("default", "constants", "-na_type_swap"))
Clear session OpenAI configuration
Description
Removes any values set with set_openai_config(), reverting to configuration
taken from a .openai_config file or environment variables.
Usage
reset_openai_config()
Value
Invisibly NULL.
Examples
set_openai_config(model = "gpt-4o-mini")
reset_openai_config()
Set OpenAI API configuration for the current session
Description
Overrides the API key, model and/or base URL used by the equivalent-mutant
detection workflow. Values set here take precedence over a .openai_config
file and over environment variables. Arguments left NULL are unchanged.
Usage
set_openai_config(
api_key = NULL,
model = NULL,
base_url = NULL,
max_parallel_requests = NULL
)
Arguments
api_key |
API key string. |
model |
Model name (e.g. |
base_url |
Base URL of an OpenAI-compatible Chat Completions API, such
as |
max_parallel_requests |
Maximum number of equivalence-detection API
requests to run concurrently. Use this to stay under a provider's
per-key parallel-request limit (exceeding it returns HTTP 429). |
Value
Invisibly, the resulting configuration (see get_openai_config()).
Examples
set_openai_config(model = "gpt-4o-mini")
get_openai_config()$model
reset_openai_config()