Proof of Done

Overview

A work item's Definition of Done is a list of acceptance criteria, and on Gitmoap most of them can decide themselves. Write "the retry test passes" as a criterion bound to that test, and the box is filled in by the pipeline that ran it — against a real commit, on the branch that ships — instead of by whoever remembered to tick it. What you gain is a work item that knows whether it is done, and a merge button that can refuse a change whose criteria are red.

Criteria you cannot machine-check stay ordinary checkboxes, which is the default and the common case. And a closed item reopens itself when the test that proved it fixed goes red again.

Quick start

Before you start

How it works

A criterion is a statement plus an optional binding

Each criterion carries a text — the human statement, up to 500 characters — an optional longer description, a required_for_close flag that defaults to true, and an optional binding. The binding is what turns a checkbox into a predicate: it names a test, a measurement, a coverage floor or a pipeline node, and the platform decides the criterion from the evidence a run produced rather than from anyone's opinion.

No binding means a manual checkbox, and that is deliberately the default. Most criteria on most teams are not machine-checkable — "the error copy does not say 'oops'" is a real acceptance criterion and always will be a human's call — and a tracker that demanded a test id before it would accept a line is a tracker where nobody writes the line. Bind the ones that pay for themselves and leave the rest as boxes.

A criterion's identity is stable across every edit. Reword it, re-describe it, make it optional, move it up the list — it keeps its evidence and its history. Only one edit throws that away: changing what it is bound to, which clears the evidence and puts the criterion back to unverified — or back to a plain checkbox, if you unbound it — because a different predicate has a different history. That reset is a container setting (binding_change_resets_evidence) and it is on by default.

The five states

A bound criterion's box is drawn from its state, and a person cannot tick it. Trying returns 409 CONFLICT, with a message beginning CRITERION_IS_BOUND. That refusal is the whole proposition: a box the machine owns is worth something precisely because opinion cannot reach it. Unbind the criterion if you want it back under human control.

What each kind binds to

op is one of lt, le, gt, ge, eq, and the comparison reads value op threshold — lt with a threshold of 200 passes at 187.4 and fails at 200. Coverage takes no operator, because a coverage binding is only ever a floor.

scorecard_check is the one kind you do not author. It is filed onto a work item by a service-catalog campaign, and adding one by hand is refused with 400 BAD_REQUEST naming it. It appears in the kind catalogue so a client can render a criterion that already carries one. It is also the one binding no pipeline decides: its verdict comes from the service catalog's own recomputation, not from a run — which is why it is the one criterion that can move a work item's badge without any pipeline having run.

How a result is matched to a binding

Matching happens once per run, over everything the run's evidence reported.

• A test binding with no suite matches its test id in any suite. A binding that names a suite matches only that suite. Leaving suite out says "wherever this test lives"; fill it in when two suites share a test id and you mean one of them.
• Several matches are conjunctive. A test id that exists in two suites and fails in one is a failing criterion. Anything else would let a criterion go green while the thing it names is broken.
• A coverage binding is checked against every file its glob matched. One file under the floor fails the criterion.
• A skipped test proves nothing. It is neither a pass nor a fail; it counts as an absence, exactly like a test that no longer exists.
• A node binding is read from the run itself, not from the evidence file — the node's terminal status is the verdict.

Where the evidence comes from

Test, measurement and coverage criteria are decided by a file a pipeline node publishes: per-test results, per-measurement values and per-file coverage percentages, in a form the platform reads. The gitmoap:report-ingest capability turns the report your test runner already produces — JUnit XML, or cargo-nextest / libtest JSON — into that form, and its artifact input takes several sources at once, so every test node in the pipeline can feed one evidence set. A provider you write yourself can publish the same shape directly on a port of type evidence.

A run whose repository has no bound criteria costs nothing: the evidence is not even read. Nothing about criteria slows a pipeline down until you bind one.

One bad report discards the whole run's evidence, not just its own. A report over 8 MiB, one that does not parse, or one written to a schema the platform does not recognise stops the evaluation there: nothing is read from any node of that run, and every criterion keeps the state it had until the next run. Half a report is worse than none, because a criterion that flips to failing on a truncated file is a false alarm and false alarms are what get a feature switched off — and a run that dropped only the unreadable half would leave you unable to tell which criteria the run had actually spoken to.

Shipping branches, and merge-request heads

The same evidence answers two different questions, and the answers are kept apart on purpose.

A shipping branch is the repository's default branch, plus any branch pattern the container adds in its issue settings (closing_branches) — the same set that decides whether a Fixes #412 in a commit message closes anything. A green test on a feature branch is a normal Tuesday: it never touches a criterion's state, and it never verifies or reopens an item.

Evidence is also ordered by commit ancestry, not by clock. A slow run on an old commit that finishes after a fast run on a newer one is dropped rather than allowed to overwrite the newer verdict, and a redelivered result changes nothing.

Verified, and regressed

When every required criterion on an item is satisfied — bound ones passing, manual ones ticked — a shipping-branch run marks the item verified and records the commit and run that proved it. An item with no criteria at all is never verified, and neither is one whose criteria are all optional: "nobody wrote a Definition of Done" and "CI proved the Definition of Done" are opposite claims, and at least one line has to be required before there is a claim to prove.

Only evidence moves the badge. Every row above is written when a shipping-branch run moves a criterion — or, for a scorecard_check, when the service catalog recomputes it. Add a criterion, delete one, tick a manual box or change a binding, and you have changed the answer the next such evaluation will compute; the badge follows then. The two gates below do not wait for it — both read the criteria themselves, so a box you tick counts against a close or a merge immediately.

verified is not closed. Two container settings decide what the platform does with the verdict, and they are orthogonal: auto_close_on_verified (off by default) closes the item when it verifies, and require_verified_criteria_to_close (off by default) refuses a manual close while a required criterion is unsatisfied. An item that verifies with nothing linked to it is surfaced rather than acted on — the badge turns green and the activity says which run proved it — because closing an item because a test somebody named happens to pass is exactly the over-eager automation that gets automation turned off.

The reopen is the payoff. A bug that comes back is the same bug, not a new ticket. A closed item whose required criterion goes from passing to failing on a shipping branch reopens, records both ends of the range — the last green commit and the first red one — and writes an activity naming the criterion and both runs. It is gated by auto_reopen_on_regression, which is on by default.

Stale never reopens anything

An unmatched binding is not a failure. The test was deleted, renamed or skipped, and the absence of an assertion is not a failed assertion. So the criterion goes stale, keeping the evidence it last had so you can see what it used to prove.

And stale is narrower than it first looks: a criterion only goes stale when the very pipeline that last proved it ran again and no longer reports it. A lint pipeline that legitimately reports no tests cannot stale your test criteria, and a docs-only push cannot hollow out a repository's Definition of Done. A stale criterion blocks verification, shows loudly on the item, and never triggers a reopen — a closed item springing open because somebody renamed a file would end trust in the feature in one sprint.

How to

Read a work item's Definition of Done

Every criterion comes back in checklist order, each with its state, whether it counts as satisfied, its binding in structured form, and the evidence that last decided it — the pipeline, the run, the commit, when it was observed and, for a measurement or a coverage floor, the value seen.

Add a criterion

Send the statement and, optionally, a binding. Leave the binding out and you get a manual checkbox. required_for_close defaults to true — a line in the Definition of Done is required by definition — so pass false explicitly for an aspirational one. New criteria land at the end of the list unless you name an ordinal, and an ordinal you name is taken literally: nothing below it is pushed down, so reorder the whole list rather than hand-picking positions when the order matters.

A bound criterion cannot be created already ticked — checked: true alongside a binding is refused with 409 CONFLICT rather than quietly ignored.

Reword a criterion, or stop it gating a close

Send only the fields you are changing. text and description rewrite the statement, required_for_close: false turns a gate into an aspiration that still shows and still rolls up, and leaving binding out of the body altogether keeps the criterion bound to exactly what it was bound to — which is what makes a reword free of consequences. The criterion keeps its id, its state and every verdict a pipeline recorded against it.

Making a criterion optional is the gentler alternative to deleting one: the record survives, and only its power to block a close goes away.

Tick a manual criterion

Ticking is an edit that sets checked. It applies to manual criteria only.

Bind a criterion to evidence

Send a binding on an existing criterion to bind it, a different binding to re-bind it, or "binding": null to unbind it back to a checkbox. Omit the key entirely and the binding is left alone, which is what lets you reword a criterion without disturbing what proved it.

Binding and re-binding are where evidence is lost, so it is worth being deliberate: a reword keeps history, a re-bind starts a new one.

Binding is also the one edit that can be refused for a quota. A manual criterion costs nothing against the repository's bound-criteria limit; the moment you bind it, it does, and a repository already at that ceiling answers 429 TOO_MANY_REQUESTS and leaves the criterion exactly as it was.

Emit the evidence from a pipeline

Bound criteria stay unverified until a pipeline publishes results. This is a pipeline-authoring task rather than a tracker one, and it is done once per repository.

A provider you write yourself can publish evidence directly: declare an output port of type evidence and write an evidence.json into the node's outputs directory in the shape shown under Examples. Coverage and measurement results have no ingest capability of their own, so they arrive this way.

Reorder the Definition of Done

The order is the order people read the checklist in, so it is worth keeping the setup steps above the acceptance steps. Reordering takes the full list of criterion ids in their new order — every criterion on the item, exactly once — and returns them renumbered.

Remove a criterion

Deleting changes what the item is judged on. Removing the last unsatisfied required criterion clears the way for the close gate and the merge gate straight away, and lets the next shipping-branch run mark the item verified; removing a bound criterion also frees a slot against the repository's bound-criteria limit.

Read the Proof-of-Done view

The verification view is the whole picture in one read: the item's verification state, the commit and run that verified it or the ones it regressed from, every criterion with its evidence, a summary of how many criteria are satisfied against how many are required, and — where the item was filed from a blueprint — its facets, each green when every required criterion under it is satisfied.

Investigate a regression

When an item is regressed, the platform already knows both ends of the range: the commit it was last verified at, and the commit whose evidence proved it broken. The regression read hands back the commits between them — newest first, each with its author — plus the required criteria that are red right now. One of those commits broke it.

The list is capped at 100 commits. When the range is wider, the response says so and gives you the true count alongside the newest slice, which is the end nearest the failure and the likeliest place to look. A commit whose object can no longer be read — a force-push took it away — comes back as a bare id rather than failing the whole read.

Asking for the range on an item that is not regressed is refused with 409 CONFLICT — the message begins NOT_REGRESSED — rather than answered with an empty list, because an empty range and "this item is fine" are different answers.

Look up what a kind takes

The kind catalogue is a global read: every criterion kind with its label, its help text, whether the platform evaluates it, the typed fields it takes, and a JSON Schema you can render a form from. It lists all six kinds, scorecard_check included, so a client can render a criterion a campaign filed as well as the five you author. It is the same contract pipeline capability configuration uses, and it is how a client stays correct when a kind gains a field.

Gate closing and merging on criteria

Criteria are worth writing on their own, but the two settings that make them binding live elsewhere and are off until you turn them on.

• Refuse a close while the work is unproved. Turn on require_verified_criteria_to_close in the container's issue settings, and completing an item with an unsatisfied required criterion answers 409 CONFLICT with a message beginning CRITERIA_NOT_GREEN and a count of how many are outstanding. Cancelling stays exempt, so work you have decided against can always be closed, and an item with no required criteria is never blocked.
• Refuse a merge while the criteria are red. Turn on require_issue_criteria_green in the repository's merge request settings, and every work item the merge request closes must have its required criteria green at the merge request's head commit. Items the merge request merely mentions are not read. A head commit no pipeline has run against fails this check rather than passing it: a gate that opens because it has nothing to say is not a gate. A required manual box counts as its tick, which is the same wherever the head is — so an unticked box blocks the merge even in a repository with no pipeline at all.

Both settings are inherited from the groups above, so turning one on at the organisation's root group covers every repository beneath it.

Reference

Limits

Two counts are enforced when a criterion is written, and a refused write creates nothing at all — no half-added criterion and no partially seeded checklist.

Fifteen per work item is the same on every plan, and a limit increase does not move it. It is a shape limit rather than a capacity one: an item that needs a twentieth acceptance criterion is describing more than one piece of work, and the fix is to split it and let the parts that are really their own job become child items. Reaching it answers 429 TOO_MANY_REQUESTS naming the work item.

The repository count is of bound criteria only — manual checkboxes do not count towards it. It exists because every bound criterion in a repository is re-checked against every pipeline run there, so the number is a bound on work the platform does on your behalf. Reaching it answers 429 TOO_MANY_REQUESTS naming the repository, and the same refusal comes back if you try to bind a manual criterion while the repository is at the ceiling. Delete bound criteria on work that is long closed, leave the criterion manual, or file a limit-increase request naming this limit — unlike the per-item fifteen, this one can be raised.

The rest are fixed, and they are shape rather than plan:

Going over the text limit is a 400 BAD_REQUEST when you write the criterion. Going over either evidence limit is not an error you see: the run's evidence is discarded whole — every node's, not only the offending one — and every criterion keeps the state it had, as though the run had published nothing.

Troubleshooting

Examples

A manual criterion, which is the common case. Everything but text is optional:

A criterion bound to a named test, in a named suite:

A measurement compared against a threshold, and a coverage floor over a path. The comparison reads value op threshold, so this passes while the reported checkout_p99_ms stays under 200:

An aspirational criterion that shows and rolls up but does not gate a close:

Unbinding a criterion and ticking it by hand. The first body hands the box back to a person and clears the evidence; the second ticks it. The binding is applied before the tick is checked, so sending both keys in one body works too:

The evidence a pipeline node publishes, for a provider you write yourself. Write it as evidence.json in the node's outputs directory, on a port of type evidence. A measurement states what was measured and never whether that was acceptable — the threshold lives on the criterion, so it can change without your pipeline changing:

FAQ

Related