Self-Healing Tests

Donobu can repair failing tests for you. Run your suite with --auto-heal and Donobu hands each failure to the heal agent: an AI coding agent that diagnoses the failure, fixes what a test change can fix, and leaves product bugs alone. Donobu re-runs the affected tests to verify the fix. In CI, the verified fix arrives as one pull request into your branch.

npx donobu test --auto-heal

What self-healing addresses

Self-healing targets test-side breakage: the test no longer matches the application, but the intent of the test is still valid. Typical causes are a renamed button, a restructured form, a changed page flow, a stale cached page.ai() step, or an outdated selector or assertion in your test code.

It does not change your tests to hide:

• Genuine application bugs (the test is supposed to fail)
• Problems in the environment, such as a service that is down or slow infrastructure

When the agent diagnoses a product defect or an environment problem, it changes nothing and explains the failure instead. If it edits a test for a failure it diagnosed as a product defect, Donobu puts the edit back.

How it works

Test runs → Fails → Evidence collected → Agent diagnoses and fixes → Donobu re-runs → Pull request with verified results

1. A test fails during its normal run, and Donobu collects failure evidence (see "Evidence collection" below).
2. The agent reads the evidence, the error, the test source, and the screenshots for each failure. It can also open a headless browser to look at the application as it is now.
3. The agent acts on each failure. It edits test code, asks for stale cached AI steps to be recorded again, or changes nothing when the product or the environment is at fault. The agent does not run tests itself.
4. Donobu re-runs the tests the change can affect. Every other test in an edited test file is always re-run too.
5. If tests still fail, the next round starts from the new evidence, up to three rounds by default.
6. A test counts as healed only when its last re-run passed. Healed tests are annotated self-healed in the report.

The command exits with code zero only when every test that failed has since passed.

What the agent does not handle

• Tests that failed and then passed on a Playwright retry.
• Tests that failed as declared with test.fail.

Enabling self-healing

Pass --auto-heal to the CLI:

npx donobu test --auto-heal

The heal agent needs a DONOBU_API_KEY for an account that has agent access. It runs through the Donobu API, so you do not need a separate model key.

Pull requests from CI

On a branch run in GitHub Actions or GitLab CI, the agent commits the verified fix to a branch named donobu/heal/<your-branch> and publishes it as one pull request (or merge request) into your branch. The commit message holds the agent's description of the fix, followed by Donobu's block of verified results.

Donobu does not open a pull request for these runs. The fix stays in the working tree:

• Runs triggered by a pull request or a merge request
• Runs in a GitHub merge queue
• Runs on a donobu/heal/... branch
• Runs on a developer's machine

GitHub Actions

The agent pushes the branch and opens the pull request with the GitHub CLI (gh), which GitHub's hosted runners already have. Give the test step a GH_TOKEN and give the job write permissions:

permissions:
  contents: write
  pull-requests: write

env:
  DONOBU_API_KEY: ${{ secrets.DONOBU_API_KEY }}
  GH_TOKEN: ${{ github.token }}

See CI/CD Integration for a complete workflow.

GitLab CI

Donobu pushes the branch with the job's own token, and GitLab push options open or update the merge request. You install nothing and create no token. Two requirements apply:

• Turn on Allow Git push requests to the repository under Settings → CI/CD → Job token permissions.
• The pipeline user must be allowed to push a branch and open a merge request.

The GitLab job token cannot call the GitLab API, so the merge request description is a one-line pointer. Each job's account of its fix and its verified results is in its commit message.

Turning off pull requests

Set DONOBU_HEAL_PULL_REQUEST to off to keep the fix in the working tree and the run's artifacts:

env:
  DONOBU_HEAL_PULL_REQUEST: 'off'

How the pull request behaves

• Review flags. Donobu flags a changed test in the pull request when it gained a skip, fixme, test.fail, or .only, or lost assertions.
• Changes outside test files. When the agent changed a helper, a fixture, or the configuration, the pull request says how many of the run's tests were re-run after that change and how many were not.
• Tests the fix broke. A test that passed in the first run and fails with the agent's changes is reported as failing, and the pull request says the change broke it.
• Sharded runs. A run split across several CI jobs on one commit (for example Playwright shards) publishes into one pull request. Each job adds its fix to the heal branch and its verified results to the pull request.
• Stale pull requests close. When a later run on the branch passes with nothing to heal, Donobu closes the heal pull request from the earlier commit and deletes its branch. On GitLab, Donobu deletes the branch and GitLab closes the merge request.
• Your commits win. When a person commits to the heal branch, Donobu stops publishing to it until the branch is merged or deleted.

Evidence collection

When a test fails, Donobu collects and persists:

• A viewport screenshot at the point of failure
• The full page DOM state
• An AI-generated triage analysis that explains what went wrong and suggests what may have changed

This evidence is attached to the test report and visible in Donobu Studio.

The heal agent's working files and session logs are written to test-results/.donobu-heal/. The folder is hidden, so a workflow that uploads test-results/ as an artifact leaves the logs out. GitHub's upload step skips hidden files unless include-hidden-files is set. Donobu removes credential values from these logs.

The "self-healed" annotation

After a successful heal, the test is annotated in the Playwright report:

This annotation is visible in the HTML report and in Donobu Studio. Use it to track which tests were healed over time and identify areas of the UI that change frequently.

Classic auto-heal (fallback)

When the heal agent is not available, --auto-heal takes the classic path. Donobu runs one triage call per failure, then clears the cache and re-runs the failures diagnosed as stale cached steps. The AI records a new action sequence, and Donobu writes it back to .cache-lock/. The classic path does not change your test source code.

Donobu uses the classic path when:

• No DONOBU_API_KEY is set, or the account does not have agent access
• The model endpoint cannot be reached
• The first agent session produced no decision
• The agent cannot run on the machine: Alpine Linux, an x64 processor older than about 2013, or an install that skipped optional dependencies (--omit=optional)

If anything in the agent path fails outright, Donobu puts back everything the agent changed and runs the classic path. A line in the log says which path was taken and why.

Configuration

Limitations

• Self-healing needs a configured AI provider. The heal agent needs a DONOBU_API_KEY with agent access. The classic path works with any AI provider.
• Publishing a fix, putting back a refused change, and knowing what the agent changed all need the project to be in a git repository.
• The agent's browser session has only the agent's environment. It gets no test credentials and no deploy secrets, so it cannot reach pages that need credentials from your CI secrets.
• If the application changed so much that the original intent is no longer achievable, self-healing fails and the test stays broken.