Oko
Oko/Docs

Command reference

oko sessions scorecard

What each agent session and each model delivered and what it cost, measured from the registers and transcripts: commits and repairs, consent dialogs, interruptions, stalled tools, current time waiting for a model, model turns without a tool or commit since progress, ignored directives, calls and tokens.

Invocation#

Command
oko sessions scorecard [--hours N] [--socket PATH] [--json]

Inputs and options#

  • The sessions are the ones `oko sessions agents` reads: every one a live process holds and every one recorded with a role (`oko sessions role`), and with --hours every one written in the last N hours. --hours also bounds the registers: only commits, dialogs and interruptions recorded in that window are counted.
  • --socket names the broker to read terminals from, as for `oko sessions agents`; --json prints the answer as JSON, without it the same answer is printed as lines for a person.

Output and state effect#

  • Reads the delivery registers and reconciles retained Stado Cargo receipts into the charge register. Each session's card carries session, model, role, alive, state and lastWrite from `oko sessions agents`, then: commits, the `<repository>@<sha>` the session recorded with `oko tasks evidence --commit` (the shared task evidence register), and repositories, the repositories they are in; repairs, the defects whose recorded repair or fix is one of those commits, read from the defect register under its full or abbreviated sha; dialogs, the ids of the asks Tama raised with the session as subject — each one a consent dialog the session stood on; interrupts {count, seconds}, the calls `oko sessions interrupt` and the model policy pass stopped in it and the seconds they had stood, from ~/.oko/interrupts.jsonl; stalledNow, whether its running call stands past the measured norm of its shape now, with stalledSeconds and stalledNorm when it does; and for an executor, reflection {ignoredDirectives, pendingReadings}, the supervisor directives `oko reflection` read as ignored and the readings still waiting on the model.
  • Each card also carries tokens: what the session's calls sent to its models, measured from its own OMP transcript as `oko telemetry inputs` measures it (calls, context windows, context and output tokens, context per call, seconds in calls, models, the inputs that took the context and the tool calls repeated with the same arguments). A session with no OMP transcript on this machine with a measured call in the window carries tokensFailure saying so instead; a transcript that could not be measured is named under tokenFailures.
  • models sums the same figures per model over sessions that run on it: {model, sessions, commits, repairs, dialogs, interrupts, interruptedSeconds, stalledNow, circlingNow, ignoredDirectives, calls, contextTokens, outputTokens, callSeconds, modelWaitingNow, modelWaitSecondsNow, idleTurns, idleSeconds, idleAboveMeasuredNorm}. A session whose transcript names no model is summed under `model not recorded`.
  • Each card carries held: every task the session claimed in the shared ledger and has neither evidence nor a verdict of done for, as {task, claimedAt, heldSeconds, aboveNorm}. circlingNormSeconds is measured from the same ledger: for every claim whose task reached evidence after it, the time from the claim to that first evidence, the longest of them. aboveNorm is true when a claim has been held longer than that, and null when no claim ever reached evidence, in which case circlingNormFailure says so and nothing is called circling. A live session holding such a claim is listed under its model's circlingNow, printed as `holds <task> without evidence since <time>, <n> s; longer than any claim took to reach evidence: circling`, and the capacity pass of `oko sessions supervision` counts it as overloaded with that reason. Desktop shows the same claims on each session card, a warning for a circling one, the circling count per model and the norm or its failure. An unreadable claims, evidence or verdict register is registerFailures.circling and held is absent from every card.
  • An answering session carries modelWaitSecondsNow, from its newest transcript record to observedAt; missing or malformed times are modelWaitFailure. For OMP, idle {turns, since, seconds} counts measured model calls after the newer of its last tool call and last recorded commit evidence. An unreadable timestamp, commit register or transcript is idleFailure, not zero. idleNormTurns is the longest measured model-call run that ended in a tool call across these sessions; if there is no such sample, idleNormFailure names that absence and no session is classified. idleAboveMeasuredNorm marks only live sessions whose current model-only run exceeds that observed fleet norm, and each model lists those sessions. A completed run closed by commit evidence alone does not set the norm; this conservative bound avoids labeling model turns above a guessed number. These are observations, not a claim that the model or executor is at fault.
  • Each register is read once per answer. One that cannot be read is named under registerFailures {commits | dialogs | interruptions | tokens | circling: cause}, and its figures are absent from every card, never zero; a defect register that cannot be read is repairsFailure on the card. brokerFailure, holderFailure and processFailure carry the same causes `oko sessions agents` reports. The native API's sessions domain answers the same document under the scorecard action, which OkoKit reads as `OkoSessions.scorecard(hours:)`.
  • Oko Desktop → Work → Sessions → Scorecard shows the same answer through that action: a card per session with its model and role, a stalled-now chip, its commits, repositories, repairs, dialogs, interrupts, ignored directives, calls and tokens and the input that took most of its context, then the per-model sums; a register that could not be read is a warning panel naming it, and its figures are absent from the cards.
  • The Desktop scorecard shows those same waiting and progress figures per executor and model, warns when a model-only run exceeds the measured norm, and displays modelWaitFailure, idleFailure or idleNormFailure by cause. The real `tests/sessions/scorecard.rs` journey uses the built CLI and native service over live transcripts; `cargo test --manifest-path rust/Cargo.toml --test sessions-scorecard -- --ignored --nocapture` additionally requires a live answering executor and one measured OMP executor with no recorded commit. A retained report contains the source and binary revisions, CLI exit and stdout/stderr. No run is a success merely because a register or a transcript was present.
  • The same observation pass embeds report: the source-cited fleet projection documented under oko report. Desktop shows its grouped metrics and daily trend below the existing executor scorecard, with expandable citations and named missing-source refusals.
  • Without --json the card ends with ratings per model, from the report: one line per model group with its closed_items_per_hour, repair_success_rate and repair_seconds, or each one's refusal, so `oko sessions scorecard --hours 24` names which model is fast and which is good from the same cited figures.
  • Compiler charges come from retained Stado Cargo result.json findings. The scorecard, native API and `oko sessions scorecard charges` reconcile the canonical workspace's retained receipts through `stado product cargo --all --json results` before reading the charge register; a new failed build is not dependent on a manual observe call. `oko sessions scorecard charges observe <result.json>` can ingest a particular retained receipt. Each charge stores its pinned source revision on main, compiler location, Git-blamed introducing commit and matching Transcript Lake commit call with its model. A missing finding, source, transcript or model stays named as unread or pending, never guessed from Git author names; the next reconciliation retries it. The scorecard exposes records per session and model, a broken-commit rate only when commit evidence supplies its denominator, and chargeUnread/chargePending for unresolved builds. Desktop shows paths, lines, receipts and causes.
  • The ignored real compiler-charge journey in tests/sessions/scorecard/charges.rs requires a dedicated HOME with its actual commit transcript, an exact failed Stado result, the committing session and model, and a declared qualification window. It observes the CLI's persisted charge register and the native session and model cards; a missing result refuses before any register write. It is not a pass until that built-source journey and the Desktop screen are observed.
  • Verified repair credits require the current defect's whole-repair measure, a holding claim with every required test or structural and adversarial verdict, and its committed repair on main. Transcript Lake's Git commit call identifies the committing session and model; the claim's recorder is cited separately, not substituted for the committer. The declared factor table counts registered priority, build-first status, blocked dependents, class consolidation, member products, changed files, reopenings survived and slower verified peers in the same priority/gate tier; each amount names its source. An outage is cited from the defect's impact and operator-declared category, not guessed from words in a title; its priority, gate and blocked dependents determine criticality. A reopened repair loses its credit and appears under creditWithdrawn. creditPending names every missing claim, verdict, main revision, attribution or source. Per-session and per-model standing is verified credit points minus distinct compiler-charged commits, or a named refusal when charges are unreadable.
  • Use `oko sessions scorecard credits [--hours N] [--session SESSION | --model MODEL] --json` for just those source-cited credits, pending items, withdrawals and standings. The native sessions/scorecard_credits action accepts the same selectors. The unfiltered scorecard embeds those same values; Oko Desktop → Work → Sessions → Scorecard shows each factor and cited input, pending cause, withdrawal and charged standing without recalculating them.
  • The real ignored tests/sessions/scorecard/credits.rs journey requires an isolated signed-in HOME and defect/charge registers below this checkout's .build, genuine holding repair claims for a fast critical class item and a slower low-priority instance, the exact source revision and both commits' Transcript Lake calls. It checks CLI and native values, refuses contradictory selectors, reopens only the dedicated class fixture, then observes withdrawal while the instance stays credited. Its report retains source and binary identities, command exit statuses and final register state. Written tests or unobserved Desktop rendering are not a pass.
  • For example, `oko sessions scorecard --hours 1` after an hour of supervised work shows which executor recorded commits that repaired defects, which one stood on consent dialogs or interrupted calls, which model delivered most across its sessions, and how many tokens each spent doing it.

Refusals and safe exits#

  • “oko sessions scorecard: --hours takes a positive whole number, got <value>”, “oko sessions scorecard takes only flags; it was given <word>, which it would not read” and “oko sessions scorecard does not take <flag>; it takes --hours, --socket” (exit 2); the native API refuses a non-positive hours with “sessions scorecard takes hours as a positive whole number, got <hours>”.
  • A missing Stado executable, unreadable receipt index or malformed result is chargesFailure with its operation and cause in CLI, native API and Desktop; `charges` refuses it. `observe` refuses a missing or unreadable result with its path. A failed Cargo build without a pinned source or compiler findings stays chargeUnread with the failed operation and cause; missing Transcript Lake provenance stays chargePending until reconciliation finds it. A missing charge register is chargesFailure rather than a zero score.
  • Credits missing their whole-repair measure, required verdict, main ancestry, Transcript Lake commit-call author, priority, relation source or changed-file read remain creditPending with the actual cause. An unreadable defect, claim or database register becomes creditsFailure. Both --session and --model are refused together; an unknown selected owner is refused by name. A compiler charge register failure or unowned failed build makes the standing unavailable with its cause.
  • The refusals of `oko sessions agents` when the process table, session roles or model policy cannot be read (exit 1); a register the card counts from that cannot be read is not a refusal but its named failure on the answer.

CLI reference#