Exam protocol

This is the app’s contract: what tolearn does with a verdict.

It lives in the repository rather than inside a bundle on purpose. The examiner.md in a bundle is a file the LLM regenerates and the user may edit; it describes what the examining model does. What the app does is decided by the app — otherwise the tracker’s behaviour would depend on the contents of an imported folder. A disagreement between the two is resolved in favour of this protocol, and the app warns on import when a bundle’s examiner.md describes different rules.

The prompt boundary

examiner.md is two parts separated by a horizontal rule ---. Only what comes before --- is copied into the prompt. After it live the “what the tracker does” and “known limitation” sections, which address the tracker and the person. The header block at the top of the file is cut out too — it addresses the generator.

The verdict

The model’s answer ends with exactly one JSON block. The app takes the last block from the pasted text: the whole answer gets pasted, and making a person dig the block out by hand guarantees mistakes.

{
  "topic_id": "...",
  "date": "YYYY-MM-DD",
  "model": "...",
  "verdict": "pass|partial|fail|blocked",
  "hinted": false,
  "practice_accepted": true,
  "failed_checks": [],
  "per_question": [
    {"id": "q1", "result": "ok|partial|miss", "quote": "...",
     "missed": [], "signal_extension": false}
  ],
  "gaps": ["..."],
  "calibration": "...",
  "notes": ["..."],
  "next_action": "proceed|retry_failed|redo_practice|split_topic",
  "retry_after_days": 3
}

Required: topic_id, verdict, per_question. A verdict with no per-question breakdown is rejected — without it, it is no different from eyeballing a grade. A topic_id that does not match the open topic is rejected with an explicit message: that is almost always an answer pasted into the wrong topic. With other fields missing, parsing is partial — whatever is there is applied, and the list of what was missing is shown.

Verdict → status

Verdict	Status	Also
pass	passed	passed_at = the date; next_review_at per retention
partial	in_progress	only questions with result != ok go into the queue; retry after retry_after_days
fail	failed	gaps[] highlighted on the analysis screen, practice is redone
blocked	in_progress	the attempt is not recorded in statistics

blocked means the exam did not take place — the required artefact was not produced. That is not a result, so it counts neither towards the attempt counter nor towards three failures in a row.

When a topic comes back

N is the topic’s effective revalidation period: its own revalidate_after_days, or defaults.revalidate_after_days[volatility] from the header when the field is absent. The date is the base plus N calendar days, the base itself not counted: 2026-05-01 + 30 is 2026-05-31.

retention	Base	next_review_at	Why that base
by_schedule	passed_at	passed_at + N	it is human memory that fades, so count from the exam
by_use	the topic’s verified_at	verified_at + N	knowledge is held by practice; it is the subject that ages, not the memory
none	—	null	the topic does not come back

The base for by_use does not depend on the attempt, so the date is set even for a topic passed by calibration: the subject ages by verified_at regardless. For by_use the date coincides with the shelf life of the knowledge itself — a topic enters the review queue exactly when its content stops being fresh.

Three failures in a row

Three fail verdicts in a row on one topic mean the topic is too large or a prerequisite was not met. The app offers to split the topic and hands over ready-made request text for regeneration in refresh mode. It does not change the bundle itself.

Marking by hand

Any status can be set by hand, without an exam — you may have known the topic already, covered it at work, or disagreed with the model. A manual mark is recorded with source: manual and is kept apart from a passed exam in the statistics; otherwise the “passed” number stops meaning anything. The reverse holds too: resetting a status by hand does not erase the attempt history.

The shape of an attempt

An attempts[] element in progress.yaml stores the verdict verbatim plus what the app knows and the model does not — the timestamp and source: exam | manual. Alongside them sits raw, the original JSON block exactly as it arrived. It is always stored: the verdict format may change in the next version of the protocol, and the original text is the only thing the history could then be rebuilt from.

Practice and the timebox

Practice runs against a timer set by the topic’s time_box_min. The countdown lives in progress.yaml as a separate practice record rather than in attempts[]: an attempt only appears with an exam verdict, while the timebox expires earlier — sometimes across several sittings and with no exam at all.

practice:
  started_at: "2026-07-28T10:00:00+03:00"   # null — the timer is paused
  spent_sec: 900                            # accumulated, not counting the current run
  expired: true                             # the timebox has already run out

The countdown is restored from started_at, so moving between screens and restarting the app do not reset it. Pausing adds the elapsed time to spent_sec and clears started_at; a reset erases the record entirely.

Expiry blocks nothing: the timer goes negative and keeps counting, and expired stays recorded even after a pause. It is a measurement, not a prohibition — practice time says something about the topic and the plan, but not about whether you may carry on.

Known limitation

The prompt reduces sycophancy but does not remove it. The only thing that really holds the grade is executable acceptance of the practice; for theory topics a pass remains the model’s judgement. The app does not fix that and does not pretend to — it merely keeps you from skipping the steps of the protocol.