Oko
Oko/Docs

Command reference

oko tasks verify

Judges the recorded result through Brama and saves an independent verdict for each open task, including before a running session stops.

Invocation#

Command
oko tasks verify (--open [--hours N] | --session ID ...) [--before-stop] [--json]

Inputs and options#

  • --open selects every live session, or those touched in the last N hours when --hours N is stated; --session selects one exact session and may repeat. Ordinary verification leaves active sessions alone and lists them under working.
  • --before-stop checks results already saved by an active session, including real tool results with no assistant summary. Without an answer, a tool call alone is insufficient: its result must be recorded. It requires explicit --session selection and refuses --open. It neither interrupts nor resumes the session.
  • The pass is run by oko and prints its report for a person, or as JSON with --json: judged, current, working, unanswered, unreadable and refusal. Example: oko tasks verify --before-stop --session SESSION_ID. --db no longer exists: verdicts, outages and the judge's health live in the shared Oko database.

Output and state effect#

  • Oko asks Brama as consumer oko for `best`, the subscription with the most plan left whichever provider holds it, with no paid API fallback, so a rate-limited provider does not fail every verdict while other subscriptions stand idle. Each open task is assessed separately against its original operator request, applicable rules, recorded calls and results. The requested result must be present: a writing task needs the text, an explanation needs its content, an export needs observed data, and a code change needs its applicable real tests, documentation, commit and push. The latest turn and the conversation before it are given whole; read_transcript and search_transcript let the reviewer read any other record of the immutable snapshot. A claimed filename or registered report is a pointer, not proof.
  • A short request such as “do it” is resolved in its original conversation, not the latest topic. read_transcript answers a whole record, or the record from the offset the judge names, and search_transcript answers every matching record with its line. The tools cannot execute commands or read arbitrary files. The judge looks up as often as it needs; the route's own context ends a conversation that never decides, and the route says so by refusing. A conversation that ends without a verdict is reported under unreadable and saves no verdict; an omitted excerpt is not proof that work was never done.
  • After a harness continuation, the handoff keeps originTranscript separately from the current transcript. Verification reads the original operator requests first and appends the continuation's work into one immutable evidence snapshot. Its hash covers both recordings; continuation prompts do not renumber the original requests. Failure to read either file names both paths and saves no invented verdict. The regression journey requires real recordings through OKO_TEST_ORIGIN_TRANSCRIPT and OKO_TEST_CONTINUATION_TRANSCRIPT, with OKO_TEST_ORIGIN_SESSION and OKO_TEST_ORIGIN_TASK identifying the original request; absent inputs refuse rather than substitute a fabricated continuation.
  • The explicit verification pass uses the newest recorded answer available in either source, including paired tool results before Stop. An idle continuation does not hide later work recorded in the original session. Both sources remain readable to the independent reviewer; selecting an answer is not itself a completion verdict.
  • Writing a message does not authorize sending or publishing it; missing permission for an unrequested action cannot block the draft. A blocker is a claim, not permission to stop. The reviewer must inspect paired calls and results for the required failed operation and the permitted alternatives through its read-only transcript tools. The program rejects invented, ambiguous, unpaired or unread observations. A saved blocker contains the task ordinal and request hash, transcript hash, actual operations, cause, observed state and resumption condition. Missing observations return as work to diagnose, never as a confirmed blocker. Existing prose-only blockers are unqualified and reviewed again.
  • Each saved verdict in oko_task_verdicts in the shared Oko database includes its reason, missing items and an input binding covering the original request and saved decision. Legacy verdicts are reassessed even when the transcript has not changed. An older writer cannot preserve qualification while replacing decision fields. The ledger shows verified done, verified not done or verified blocked; a done task leaves every handover. A later operator correction outranks the verdict in both directions.
  • Verdict history follows the task identity, not whichever row occupies its current message number. Only a row recorded without a task identity can serve as numbered history for the message's own task; a separately recorded part never inherits that history. Another identified task's verdict is not a previous round of this task. CLI verification and Desktop use this same shared writer.
  • Every verdict binds the task identity, including a message's own root task. Matching text, timestamp and message number in another transcript cannot transfer its completion. Root verdicts written before identity was included are reassessed; part verdicts already carried their identity. CLI task reports and Desktop read the same admission rule. Identity binding alone does not establish an atomic evidence/acceptance snapshot.
  • Run `node --env-file=<fixture.env> tests/judge/root_identity.mjs`. Declare OKO_BIN, OKO_TEST_SOURCE_REVISION, OKO_TEST_SUCCESS_EXIT, OKO_TEST_VERDICT_SOURCE_TASK, OKO_TEST_VERDICT_TARGET_TASK and their OKO_TEST_VERDICT_SOURCE_SESSION/OKO_TEST_VERDICT_TARGET_SESSION. Both must be isolated real root tasks with the same retained text, timestamp, ordinal and correction history; the source needs a genuine applicable independent done verdict, and the target must be open with no stored verdict or outage. The journey replays that actual decision onto the target through the CLI database interface, requires the task to remain open with no applicable verdict, and removes only the fixture replay using its retained binding. It retains commands, bodies, exits, readbacks, source/binary identities and cleanup errors under .build/judge/root-identity. It never invents a transcript or a successful model answer.
  • The context built for each task lists every manual one-off repair recorded after that task’s message, as `oko oneoff list` reports them, each with its tool-use id, category, rule and outcome (ran, refused by a hook, or no result recorded). The standard tells the judge that such a command changes live state once and leaves the product unchanged: a request to fix, build or make something work is not_done when its recorded delivery is only such a command, and missing then names the product change; it is the requested result only when the operator asked for exactly that action.
  • Each route is asked once per pass: a refusal (whether Brama's envelope marks it `retryable` or not), an empty answer or prose where a verdict belongs moves to the next route, and the pass never waits, because the control routine's next pass is the wait. A dropped connection or an address where nothing listens ends the pass as an outage. When every route refused a transient refusal, the outage's sentence says the gateway called it transient, that every route refused this pass, that nothing needs authorizing and that the control pass asks again; `brama subscription list` says which credential the pool is waiting on.
  • When Brama refuses in a way no retry repairs, or cannot be reached, the pass stops with the refusal and records it per task in oko_task_judge_outages; the task report shows it as judgeOutage on a task whose evidence was recorded before the outage and after the operator last spoke, so Tama's block-stop-with-open-work treats that task as unjudged rather than open. Evidence is a commit or report from oko tasks evidence or, for work with neither (a print job, an answer), every acceptance check met with evidence through oko tasks check: the report then carries `proof.checks` with the met results, recorded when the newest of them was. A later verdict clears it. What was seen of the judge is kept in oko_judge_health and copied to judge-health.json in Oko's application-support directory, which Tama's gates read without a network call. When that record says the judge refused, the pass asks one probe first and, while it still refuses, is refused with SEDZIA NIE DZIALA and the outage written on every open task with evidence in scope.
  • `oko tasks judge --probe` asks one word-long question along the judge's own walk — `best`, then `decision-model` as one typed question on POST /v1/decisions (Brama refuses that alias on chat completions), then the models the gateway lists for oko — and stops at the first route that answers. When none does, the recorded refusal names every route asked and what it answered; a `403` is printed with the grant it lacks, because Brama admits oko's bearer only to the `call:brama#<alias>` capabilities recorded in the vault of the host that runs the gateway, so a grant minted on another host changes nothing. A probe that asked `best` alone would let a quota block on the subscription pool mark the judge down before the walk reached `decision-model`.
  • `oko tasks judge --held-by DEFECT` (repeatable, while the judge is recorded down) names the open defects that hold the outage in judge-health.json; the names stay until the judge answers again. The judge command and the SEDZIA NIE DZIALA refusal then print, for each such defect, its title, the operator's standing ruling on it from `oko rulings list`, and the command that relates a task to it (`oko tasks relate <task-id> blocked-by defect-<id>`), and say the operator already knows the outage. With none named, they print how to find or record the defect and name it. A closed or unknown defect is refused by name.
  • A verdict must be one complete JSON object with an exact declared verdict label, a nonblank reason and a string array of nonblank missing items. Done requires no missing items; not_done must name missing work. A blocker is allowed only as an object on a blocked verdict. Broken JSON, duplicate or unknown fields, partial fragments, surrounding prose and inconsistent fields are unreadable, not salvaged with text matching. A failed rules read, evidence snapshot read, verdict read or verdict write names the operation and actual error. The complete-response binding contract invalidates earlier saved verdict bindings for reassessment; no previously extracted fragment is grandfathered in. Operator statements in the transcript still take precedence.
  • Judge health writes are not best-effort success. A failed shared observation or local gate-copy write names the failed operation and cause, and whether the other write succeeded. A probe returns its own observation, not a readback that could select an older healthy row or another caller's result. Desktop's Model access screen exposes saved judge health and an explicit Probe judge action through the resident API.

Refusals and safe exits#

  • Refuses without --open or --session; --before-stop requires --session and cannot accompany --open.
  • A --before-stop pass also reads the frustration register: every moment the operator lost patience in a selected session whose cause nobody has named is listed under unexplainedFrustration, and the pass refuses with “the operator lost patience in this session and no cause is named for: <id> (<words>); …” naming `oko frustration cause <id> --cause` as what names it. Verdicts already saved in that pass remain saved; an ordinary pass does not read the register.
  • Refuses when the Brama route for consumer oko is absent (stado service directory connect brama --consumer oko), oko-model-router#token cannot be read through Skarbiec, or the oko-model-agent-auth signing identity is missing. Credential refusals include the actual failed directory lookup or secret read, its scoped consumer and the returned error. The CLI and Tasks screen use the same refusal. A Brama refusal retains its HTTP status and body and exits unsuccessfully.
  • A failed Brama directory lookup retains the exact command, exit status and Stado error. An unreachable registry is not reported as a missing route declaration. The same per-call error reaches task execution, verification and proxy startup; no shared last-error state is used.
  • If any task judgment remains unreadable, the report retains those task identities and actual model answers, sets refusal and exits unsuccessfully; the native API and Desktop return the same refusal. Valid judgments already saved in that pass remain saved. Typed decisions require a declared choice and confidence in [0,1]; absent confidence is not invented as zero. External-provider failures name the selected service address and do not prescribe Brama grants or subscription operations.

CLI reference#