> ## Documentation Index
> Fetch the complete documentation index at: https://docs-stg.outerlayer.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Policy and validators
> Declare what evidence a change must carry, and how strict the verdict is, from files on the base branch.
Every pull request is evaluated against a set of checks. The policy decides which checks run and how much each counts.
The checks appear on the work item page's **Checks** tab. On GitHub, they appear as one evidence comment. Each row on it is a check. GitHub also gets one `OuterLayer evidence` check run. It reads two things side by side: whether OuterLayer's own checks pass, and where a person's review stands. OuterLayer writes both only when **Post session summaries on pull requests** is on for that repository. **Settings → General** has one switch per linked GitHub repository, each off by default. See [Connect a repository](/connect-repository#what-the-connection-is-used-for).
The verdict is one of these:
* **pass**: no check that counts failed, and none is still waiting. A row at level `info`, and a row that could not be checked, never stop a pass.
* **flagged**: something needs attention.
* **waiting**: nothing failed, but a result has not arrived. The comment says whether it waits on a required result or session evidence.
A person's review is not one of the checks, so it never makes the verdict flagged or waiting. The comment shows the review in its own block above the checks.
Two places hold a repository's policy: the policy file, `.outerlayer/policy.yaml`, and the validator files in `.outerlayer/validators/`. Both are read from the pull request's base branch, so a change takes effect when it merges. If they cannot be read, that evaluation uses the defaults: no validator checks, and every key at its default. To declare a check of your own, see [Write a validator](/write-a-validator).
When a factory has a control plane, named or the default, both come from that repository instead. The default is the factory's first-linked repository when it links several and names none. A change there applies at the next evaluation. If the source cannot be read, the comment shows a row saying so. See [Share instructions across repositories](/share-instructions-across-repositories).
Two keys are never read from a control plane: `build.workflows` and `criteria.replace`. Each is read from the default branch of the repository it acts on, and its row in the table says which.
## The policy file
`.outerlayer/policy.yaml`:
```yaml theme={"system"}
validators:
code-review: off
merge_gate: on-flag
review: required
build:
workflows: allow
criteria:
replace: people
missing: warn
test_proof: any
```
| Key | Values |
| - | - |
| `validators` | Per-check level overrides. `warn` counts toward the verdict. `info` shows the row but never counts. `off` removes the row. |
| `merge_gate` | How strict the `OuterLayer evidence` check run on GitHub is: `none` (default) or `on-flag`. Any other value is a policy error, and the check behaves as `none`. See [The OuterLayer evidence check](#the-outerlayer-evidence-check). |
| `review` | `required` (default, also with no policy file) means the check's title says when a person's review is still pending. `optional` says no review is required. Neither value holds the verdict or the check open, and a person's request for changes fails the check under `merge_gate: on-flag` with either value. See [Review and merge](/review-the-work). |
| `build.workflows` | `deny` (default) or `allow`. `allow` lets a build's push token change files under `.github/workflows/`. Read from the default branch of each repository the token is for. See [GitHub App permissions and build tokens](/github-app-and-build-tokens). |
| `criteria.replace` | Who may replace an item's recorded acceptance criteria once it has a list. `people` (default): only a person; an agent session, or a key bound to no member, gets `409`. `anyone`: those callers may replace it too. Read from the default branch of the item's own repository. See [`outerlayer emit criteria`](/reference/cli-emit#outerlayer-emit-criteria). |
| `criteria.test_proof` | Where test results must come from to prove a criterion that declares `test` proof. `any` (default) takes results from any source. `ci` takes only results recorded from CI, so an agent session's own run proves nothing. See [Prove a criterion with test results](/attach-evidence#prove-a-criterion-with-test-results). |
| `criteria.missing` | What a pull request does when its item has no recorded criteria: `warn` (default), `info` or `off`. `warn` flags the verdict. `info` shows the row without flagging. `off` drops it. A repository with no policy file is never flagged for it. See [Recorded criteria in the verdict](#recorded-criteria-in-the-verdict). |
To let agent sessions replace a recorded criteria list, set `criteria: replace: anyone`.
Every key is optional. An empty file sets nothing, and every key takes its default.
An unknown key or an invalid value voids the whole file for that evaluation. The comment shows a policy error row. Your overrides drop out, and every key takes its default. The validator files still apply. Fix the file where it is read from: the base branch, or the control plane.
Two keys are read separately and survive a voided file. `build.workflows: allow` still grants workflow changes, and `criteria.replace: anyone` still lets agents replace criteria, even when another key is invalid.
A `validators:` entry that names no validator file is a policy error too, but it voids nothing: the other keys and levels still apply. Delete the entry, or add the file. A level for an id that a `signal` file declares does nothing.
Two ids belonged to checks the product used to define: `code-review-ran` and `acceptance-criteria`. A policy file that levels either keeps loading. The level is ignored while no validator file declares that id. A validator file of yours may take one, and then the level applies to it. The starter criteria judge check takes `acceptance-criteria`.
`extends` is not a key. OuterLayer has no preset to extend: every check comes from a validator file. A file that sets `extends`, including `extends: outerlayer:recommended@v1`, gets a policy error row that names the fix. Run `outerlayer init --template default`, which writes `.outerlayer/validators/code-review.yaml` if it is missing, then delete the `extends` line. In a repository whose context comes from a control plane, add the file to the source repository instead.
## The OuterLayer evidence check
`merge_gate: none`, the default, completes the check neutral, so it never fails. `merge_gate: on-flag` fails it when one of OuterLayer's checks fails, and when a person has requested changes. A missing review does not fail it.
To make GitHub block a merge:
1. Turn on **Post session summaries on pull requests** for the repository, on **Settings → General**.
2. Set `merge_gate: on-flag` in `.outerlayer/policy.yaml`.
3. Require the `OuterLayer evidence` check in the branch protection of the base branch.
The title of the `OuterLayer evidence` check says what decided it. With `merge_gate: on-flag`:
| Title | Result |
| - | - |
| Waiting on OuterLayer checks: *names* | In progress. The checks are still running. With no check to name, the title ends in "session evidence". |
| *n* OuterLayer check failing: *name* | Fails. When no failing check has a name, the title is "OuterLayer checks failing". |
| Changes requested by *name* · OuterLayer checks pass | Fails. A person asked for changes in OuterLayer, whatever `review` says. |
| OuterLayer checks pass · review pending in OuterLayer | Passes. Nobody has reviewed yet. |
| OuterLayer checks pass · approved by *name* | Passes. |
| OuterLayer checks pass · review pending: new commits since *name*'s approval | Passes. The approval predates the latest commits. |
| OuterLayer checks pass · review not required | Passes. The policy has `review: optional`. |
Two more titles are neutral whatever `merge_gate` says:
| Title | When |
| - | - |
| Evidence check could not run this refresh | The policy could not be read on the first evaluation of this commit. A later push retries it. |
| Nothing left to resolve the evidence wait | The check was waiting on session evidence that will never arrive, and no claim is live to end the wait. |
With `merge_gate: none`, every completed result is neutral and blocks nothing, but the titles are the same.
The gate applies only once the check completes. While results are pending, the check is in progress under either value. A branch protection rule that requires the check holds the merge until it completes.
The check's **Details** link opens the work item's page in OuterLayer. `gh pr checks` lists a neutral result as `skipping`. That means the check blocks nothing, not that nothing was checked.
In the evidence comment, a check with no result reads "not proven for this pull request", followed by its level. At `warn` it flags the verdict. At `info` it is shown but not counted.
A pull request nobody ran through OuterLayer is never evaluated and gets no comment. It can still get a neutral `OuterLayer evidence` check titled "Not run through OuterLayer", which says nothing was checked and never blocks a merge. It gets that check when **Post session summaries on pull requests** is on and the policy has `review: required` (the default) or a `merge_gate` other than `none`. The policy is the base branch's, or the control plane's when the factory has one. A policy that cannot be read counts as gating, so the check is posted. Otherwise OuterLayer writes nothing to it.
## Where checks come from
Every check comes from a file in `.outerlayer/validators/`. OuterLayer defines none of its own: `outerlayer init` writes the two starter checks below, and [you write the rest](/write-a-validator).
OuterLayer adds some rows itself, whatever the validator files say. They have no level and cannot be turned off:
* `criterion-unproven`, for each recorded criterion whose declared proof is not attached. See [Recorded criteria in the verdict](#recorded-criteria-in-the-verdict).
* Rows a linked issue asks for.
* Policy errors, and a row for a control plane that cannot be read.
`criteria-missing` is added too when the repository has a policy file, at the level `criteria.missing` sets. So a repository with no validator files still has its recorded criteria checked: a missing proof flags its pull requests. A person's review of the item is not a row; it has its own block.
## The code review check
`outerlayer init` writes a policy file and two checks: `.outerlayer/validators/code-review.yaml`, described here, and `.outerlayer/validators/acceptance-criteria.yaml`, described in [The criteria judge check](#the-criteria-judge-check). So does `outerlayer init --template default`. Each file is written only when it is missing, and is yours to edit. Neither command writes them outside a git repository, or in a repository whose context comes from a control plane. There, the checks live in the source repository. Without its comments, it reads:
```yaml theme={"system"}
id: code-review
kind: validation
row: "A code review result is recorded for this change"
level: warn
run:
where: ci
emit: code-review
```
The check passes when a pass is recorded under `code-review` for the work item. A session, CI or a person can record it. [Record a code review](/attach-evidence#record-a-code-review) shows how.
The row says a result is recorded, not what the review found. [Which result counts](/attach-evidence#which-result-counts) applies: the latest result on the item counts, and a missing result waits while a claim is live. Once the claim ends, a missing result fails: the comment reads "not proven" and the item page reads "fail".
To turn it off, either set `code-review: off` under `validators:`, or delete the file. Do not do both: a level for an id that no file declares is a policy error.
## The criteria judge check
`.outerlayer/validators/acceptance-criteria.yaml` is the second check `outerlayer init` writes, on the same terms: only when it is missing, and yours to edit. Without its comments, it reads:
```yaml theme={"system"}
id: acceptance-criteria
kind: validation
row: "A criteria judge result is recorded for this change"
level: warn
run:
where: host
emit: acceptance-criteria
judge:
brief: .agents/skills/outerlayer-build/references/criteria-judge.md
```
The check is a `where: host` check with a [`judge`](/write-a-validator#a-judge-on-your-host). After a build ends ok, the runner starts a fresh agent. The agent judges each recorded criterion against the test that cites it, drafted criteria included. The runner records the verdict itself, with the judge's report as its proof. A build does not record this check.
The verdict holds one [check item](#check-items) per criterion: its result, the judge's reason and the tests that cite it. The item page's **Criteria** tab shows each check item on its criterion's row. The check's sentence names every criterion that failed.
A criterion whose declared proof is an artifact, such as a screenshot, is not judged here. Its check item reads `skipped`, and its own `criterion-unproven` row asks for that artifact. A criterion with no proof and no citing test fails without the judge. Each criterion the judge does not find proven makes the check fail.
The runner judges a criterion again only when something the judge reads about it changed. That is its text, its declared proof, a test file that cites it, or the brief. Otherwise the criterion keeps the result the runner recorded last. So the same tests never get two different verdicts.
When the item records no criterion to judge, the runner records a fail with the reason `no criteria are recorded for the item`. That happens when the issue states none and none could be drafted from it. The row never reads as passed, and no judge runs. The `criteria-missing` row flags the missing list beside it.
An item built in a local session shows no `acceptance-criteria` row, because no runner records it. To have your own CI or a person record the result, declare `where: ci` with `emit: acceptance-criteria` and record it with `outerlayer emit acceptance-criteria --item 412 --result pass --link `.
The row says a judge's result is recorded, not that the criteria hold. [Which result counts](/attach-evidence#which-result-counts) applies, as for the code review check. To turn it off, set `acceptance-criteria: off` under `validators:`, or delete the file, not both.
## Recorded criteria in the verdict
An item's recorded criteria count toward the verdict of its open and merged pull requests. A pull request closed without merging is not flagged.
* **An unproven criterion** is a recorded criterion whose declared proof is not attached. The row is `criterion-unproven` and it always counts toward the verdict; no policy key lowers it. A team that does not want a proof kind does not declare it. A criterion that declares no proof never produces one.
* **No recorded list** produces `criteria-missing`, which `criteria.missing` levels. `warn` flags the verdict. `info` records the fact without flagging it. `off` drops it. Any other value is a policy error.
```yaml theme={"system"}
criteria:
missing: info
```
While a claim on the item is live, both read as pending, so a build in progress is not flagged. That covers a runner's build and a local session's claim alike. They flag once the claim ends. A merged pull request keeps the flag on its final record.
A flagged row has the effects any flagged row has:
* **Approve** stays disabled on the item page, though the item still reads **Ready for review**.
* A runner releases the build as incomplete.
* The GitHub check fails when `merge_gate: on-flag` is set.
The Checks tab adds no row for either one.
A repository with a policy file whose builds do not record criteria has its open pull requests flagged. Set `criteria.missing` to `info` or `off`, or record criteria with [`outerlayer emit criteria`](/reference/cli-emit#outerlayer-emit-criteria).
## Check items
A check can carry check items: one result for each thing it judged, such as a criterion, a file or a test. Each check item has these fields:
| Field | Required | What it holds |
| - | - | - |
| `id` | yes | What the check item is about, such as a criterion id or a file path. Unique within the check, at most 200 characters. |
| `result` | yes | `pass`, `fail` or `skipped`. |
| `criterion` | no | The acceptance criterion it is about. The **Criteria** tab shows the check item on that criterion's row. |
| `reason` | no | Why, in one to three sentences. At most 1,000 characters. |
| `refs` | no | Up to 20 places it points at, each a `path`, an optional `line` and an optional `https://` `href`. |
A check holds at most 500 check items. A check with a failing check item cannot pass. The **Checks** tab lists the failing check items under their check. An agent that amends the work reads them from `outerlayer work status --json`, under `checks`.
Any check can carry check items. The criteria judge records one per criterion. A host command writes them to the file `OUTERLAYER_CHECK_ITEMS` names. CI or a person sends them with [`outerlayer emit --items `](/attach-evidence#record-check-items).
## A factory with several repositories
A repository belongs to one factory, so one policy applies to each pull request. A factory can still hold several repositories, and a build can open a pull request in each.
Each pull request's comment and check are judged under its own repository's files. The item has one verdict, which Approve and a runner read. That verdict is computed under the files of whichever repository's name sorts last. To give every pull request of an item the same checks, name a control plane, or keep the files the same in each repository.
This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.