24. Agentic Runs

Chapters 10 and 19 teach you to call a model and to orchestrate agents. This chapter is the seatbelt layer: how you test, record, time-limit, remember, restrict, and diagnose those runs so they stay cheap, repeatable, and safe enough for everyday work and CI.

An ordinary function is deterministic: same input, same output. A prompt is not. The model can change its wording, spend tokens, call tools, hang, or invent a path. You therefore need extra machinery that ordinary programming never needed — not because MALDA is harder, but because the guest (the model) is unpredictable.

Interpreter and C# transpile agree on the offline examples in this chapter. JavaScript is host-only (n/a).

What you already need

Read this chapter after you can write a prompt with a schema and call evalPrompt (no API key). You do not need a live model to learn suites, journals, context, policy, AgentError, stream, or grounded.wrap. Live calls appear only when you record a cassette or you await a prompt.

A map of the chapter

Each construct answers one common question. Skim the table, then read the section that matches the problem you have.

Which command do I run?

Four CLI verbs look similar and do four different jobs. Mixing them up is the most common first mistake in this chapter.

CLI flags are summarized in 2. Tools & Tooling.

suite, context, policy, and within are contextual words, not reserved keywords. They mean something only in the declaration / statement shapes below. var context = 1; still parses. case is already a keyword (reused for suite cases vs match). expect and stream are builtins.

24.1 Eval suites

A suite is a named group of eval cases — think unit tests for prompts. Each case feeds a fixture (fake model JSON) through the same extract / validate path that await would use, then expects a boolean. No API key, no network, no surprise wording.

That is why malda run / ordinary interpret skips suites (like property): they are not part of the program’s output. Run them with malda eval. That is not malda test, which discovers property declarations and *.test.malda files — see 36. Property Testing.

24.1.1 The smallest suite

Three pieces work together:

1. A schema names the shape you want back (name: string).
2. A prompt with -> Card is only a template until you await it. Here we never await.
3. evalPrompt(instance, fixture) pretends the model returned fixture and runs the same coerce path as await.

Lookahead after case: case "title" { is a suite case; case Pattern: is a match arm. The quoted title is the name you will see in the eval report.

schema Card {
    name: string;
}

prompt extract(raw) -> Card {
    user: raw;
}

suite "extract" {
    case "offline shape" {
        var checked = evalPrompt(extract("x"), dict { "name": "Ada" });
        expect(checked.ok);
    }
}

print("ok");

Walk-through: extract("x") builds a PromptInstance (still no model). The dict { "name": "Ada" } is the fake completion. evalPrompt checks that dict against Card and returns { ok, data } or { ok: false, error }. expect(checked.ok) throws like assert when the condition is false, which fails that sample. The print("ok") after the suite still runs under malda run because the suite itself is skipped.

validate is the quieter cousin: it returns { ok, error } and does not throw on mismatch. Use expect inside cases (fail the sample). Use validate in ordinary code when you want to branch.

expect(1 + 1 == 2);
print("ok");

24.1.2 How to run it

malda eval Examples/Prompts/eval_suite_offline.malda
malda eval path.malda --baseline .malda/evals.json
malda eval path.malda --baseline .malda/evals.json --update-baseline

• First line: run every suite in the file. Exit non-zero if a case fails.
• --baseline: compare this run’s pass rates to a saved JSON file. A case whose pass rate drops fails the process (you made the prompt worse).
• --update-baseline: write the current rates into that file after a run you trust. Do this on purpose, not to hide a regression.

24.1.3 Optional case decorators

These sit on a case (or the suite) when you later move from fixtures to live samples:

• @samples(N) — under MALDA_REPLAY, replay N recorded variants; without replay, N live samples. N <= 1 runs once.
• @threshold(0.9) — exit non-zero when the case pass rate is below the threshold (90% here). Useful when a live model is allowed to fail a rare sample.
• @judge(prompt gradeHelpfulness) — after a passing body, evaluate the judge expression (a prompt is called with the case title). A result object with ok: false fails the sample. The judge is a second prompt that grades the first.

Same file: Examples/Prompts/eval_suite_offline.malda. Agent snippet: docs/llm/few-shot/43_eval_suite.malda.

24.2 Cassettes

A cassette is a tape of LLM traffic: every Chat (HTTP, GGUF, client bridge) can be recorded to a JSONL file and played back later. Think of a VCR. Record once while you have a key and a network; replay forever in class and CI without spending tokens or depending on the vendor’s mood.

Secrets (Authorization, API keys, obvious secret-shaped strings) are redacted on record, so the file is safer to commit than a raw log. How to record: tests/cassettes/README.md.

24.2.1 Record, then replay

MALDA_RECORD=tests/cassettes/name.jsonl   malda Examples/path.malda
MALDA_REPLAY=tests/cassettes/name.jsonl   malda Examples/path.malda
MALDA_REPLAY_STRICT=1                    malda Examples/path.malda

1. MALDA_RECORD — run the program for real. Each Chat is appended to the JSONL file.
2. MALDA_REPLAY — the next run looks up each Chat in that file and returns the saved answer.
3. MALDA_REPLAY_STRICT=1 — a lookup miss is a hard error. Without it, a miss journals cassette_miss and the host calls the live model. That is the footgun: CI that “replays” can still spend money.

24.2.2 What makes two calls “the same”

The cassette key is prompt hash + canonical args + model + mode (A/B/C) + schema hash. Change the prompt text, the arguments, the model name, the tool mode, or the schema, and you get a new key — a replay miss. That is intentional: you should not silently reuse an answer that was recorded for a different question.

Examples/Agents/agent_governance_golden.malda is an offline interpret/transpile pair (no cassette). Examples/RalphWiggum/RalphWiggum.malda stays n/a (too-large).

24.3 Run journal

The run journal is an in-process flight recorder. One list collects prompt, tool, cap, hop, and workflow-step events for the current process. After a run you can ask “what happened?” without scraping stdout.

Workflow SQLite remains the durability store for long-lived instances (see 23). Successful steps are also mirrored here so trace.journal() and workflow events share a substrate. The journal is not a cassette: it does not replay model answers; it only remembers that a call occurred.

24.3.1 Read it after an offline prompt

schema Card {
    name: string;
}

prompt extract(raw) -> Card {
    user: raw;
}

var checked = evalPrompt(extract("Ada"), dict { "name": "Ada" });
var events = trace.journal();
print(events.length > 0);

evalPrompt is enough to land a journal event — you do not need a live Chat. Each event is an object with ts, span, kind (prompt / tool / cap / hop / step), name, optional promptHash / model / tokens / cost / ms / repairs / error, and ok.

• trace.span("name", fn) — nest events under that span name (a labeled region, like a folder in the log).
• trace.lastUsage() — last recorded token/cost/ms object, or null if nothing has been billed yet.
• .usage on a prompt result is metadata. An own object key named usage (schema field) wins — if your schema has a field usage, that field is the value, not the billing metadata.

Export: MALDA_TRACE=jsonl|otlp plus optional MALDA_TRACE_FILE. That is not the existing malda trace CLI, which still reads session .malda-trace.jsonl files. Same file: Examples/Prompts/run_journal_offline.malda.

24.4 within scopes

The statement within (30s) { … } is a wall-clock kitchen timer. When the timer rings, the runtime journals a timeout and throws Timeout: exceeded within (…ms). Cancellation flows into LLM Chat and cap consume, so a stuck model call or a slow tool is not left running forever in your process.

Units are a number plus ms, s, or m (converted to milliseconds). A bare within (30) is a parse error — write 30s or 30000ms. Optional budget(tokens: N, tools: N, cost: N?) pushes a nested resource budget like @budget.

24.4.1 Statement vs decorator

The decorator @within(ms) remains the per-function / per-think deadline. It is not a statement and takes milliseconds only. See 10. Prompts and 9.7 Decorators.

within (10ms) {
    sleep(50);
}

print("should-not-print");

This block is supposed to fail. On expiry the runtime journals a timeout and throws; the print is never reached. Same file: Examples/Prompts/within_sleep.malda (interpret and C# both fail — a pair on error identity, not stdout).

24.4.2 Racing several branches

When you start more than one timed piece of work, these builtins pick a winner:

• race(tasks…) — first completed branch, even if that branch is a timeout.
• firstOk(tasks…) — first branch that is not a Timeout variant (skip the ones that ran out of time).
• all(tasks…) waits for every branch. A failed branch becomes an AgentError Upstream slot; the call does not rethrow the first error. You inspect the slots yourself.
• cancelWorkflow(id) flips a durable-workflow instance to cancelled; the body polls that status between statements. That is a workflow control, not a within timer — see 23.4.

24.5 Declared context

A chat is a growing list of turns. Models have a token limit; if you keep every message forever, later calls get slower, cost more, and eventually fail. A context Name { … } declaration is a schema-like mini-DSL for the notebook’s house rules: how many turns to keep, what never to drop, and how to shrink the rest.

new Name() constructs a journaled object with .add and .turns. You still decide when to send those turns to a prompt; the context object only stores and trims them.

context Session {
    budget: 800 tokens;
    pin: systemFacts;
    retain: last 2;
    evict: oldest;
}

var s = new Session();
s.add("user", "hi");
s.add("user", "there");
print(s.turns.length >= 1);

s.add("user", "hi") appends a turn { role, content }. One-argument s.add("hi") is the same as role user. After each add, if the list is longer than the retain window (plus pinned slots), the runtime journals a context.compact event and evicts.

WF1006: compaction / context mutation inside a workflow body must sit in a step. Replay would otherwise add the same turn twice. Imported or unknown callees stay Info, same as WF1005. See 23. Durable Workflows.

This object does not attach to Agent or Conversation. agent.think() trims its own history with MALDA_AGENT_CONTEXT_BUDGET_TOKENS; the budget: field here is not that env var. Same file: Examples/Prompts/context_turns.malda. Agent snippet: docs/llm/few-shot/45_context_turns.malda.

24.6 Program-boundary policy

Chapter 12.6 gives a tool a key (a capability token) so it cannot invent a path. A file-level policy { … } is the building rule on top: even if a token was minted, consume can still be refused. Violation journals a cap event and throws ToolDenied: ….

Policy is enforced on cap consume regardless of which tools the host registered. It is the last gate before filesystem, network, shell, or MCP work actually happens.

policy {
    fs:    readOnly under "./work";
    net:   allow "api.example.com", "*.internal";
    shell: deny;
    mcp:   allow "local";
}

A policy that only denies shell is enough to see the declaration parse and run:

policy {
    shell: deny;
}

print("ok");

24.6.1 The four verbs

24.6.2 Policy vs @effects vs tokens

Three layers that are easy to collapse into one word “capability”:

They are not the same mechanism. A pure function cannot call cap.run. A token cannot unlock a path the policy denies. A policy cannot replace a token — consume APIs still reject strings and forged dicts. How-to for tokens: 12.6.

24.7 AgentError

When a prompt or tool fails, older APIs give you a string error. Strings are easy to print and hard to match on. AgentError is a built-in sum type — a tagged reason with fields — pre-registered so you do not declare it yourself:

AgentError = Refused(reason) | Unparsable(raw, attempts) | SchemaMismatch(schema, error)
           | BudgetExceeded(kind, limit) | ToolDenied(tool, reason) | Timeout(ms)
           | Upstream(status, body)

Existing { ok, error } strings stay. Throws stay throws — a within expiry still raises; it does not become a quiet result object. Sites that already return a result object also attach agentError as the variant. validate mismatch → SchemaMismatch.

schema Card { name: string; }
var r = validate("Card", dict { "email": "x" });
print(r.ok);

schema Card { name: string; }
var r = validate("Card", dict { "email": "x" });
print(str.indexOf(string(r.agentError), "SchemaMismatch") >= 0);

Read r.agentError for the tagged variant. Checking only the string error is still valid and still misses the tag — you cannot tell SchemaMismatch from Timeout by reading English text. See also 4. Data Types for match on sum types.

24.8 Stream and for await

stream(value, schema?) turns a value into an array of chunks so a UI can update as text arrives. Each chunk is { text }, or a schema-aware partial object when you pass a registered schema name. for await (var x in expr) iterates that collection.

Today the builtin materializes the array first; C# emit is foreach over it. You can still write the loop that a streaming UI will keep when live tokens exist. You do not need an LLM to learn the syntax: passing a string chunks that string offline.

var n = 0;
for await (var chunk in stream("hello world from malda")) {
    n = n + 1;
}
print(n > 0);

The incremental JSON prefix parser closes partial objects/arrays and yields the largest prefix that validates — so a schema-aware stream can show { "name": "Ad" } before the closing quote arrives. LlamaCpp has no live token stream — chunks are carved from the finished text (the known gap at the top of this chapter).

24.9 Grounded sinks

A retrieval-augmented answer should remember where it came from. grounded.wrap(value, citations?) attaches citation metadata on the runtime value — a sticky note, not a new match kind. You still print the text; you can also read .citations.

Interpolation and str.* union citation sets. Mixing grounded and ungrounded text marks the result partial (some pieces have sources, some do not).

var g = grounded.wrap("Ada", [{ "source": "wiki", "id": "1" }]);
var s = $"Hello {g}";
print(s.citations.length >= 1);

Walk-through: g is { value, citations, sourced }. Inside $"Hello {g}" the wrapper interpolates as its value "Ada" and the resulting string keeps the wiki citation. There is no flat grounded() alias and no Grounded keyword. GraphMemory can return the same wrapper from memory.ask / query(..., { grounded: true }) — see 20.3.15.

@requiresCitations on a function (and marked sinks such as io.writeFile / HTTP response) errors when a string argument has no citations. Use it on the last hop before a user-visible answer or a file write, so an ungrounded hallucination cannot slip out. match / validate do not see a new type — citations ride along as data.

24.10 Check loop

malda check is the spellcheck of the toolchain: it diagnoses and does not execute. Use it in an agent loop or before you run a source file. --fix then applies a short list of mechanical edits — the ones that are unambiguous, not “rewrite my program”.

malda check app.malda --json
malda check app.malda --fix
malda prompts --diff .malda/evals.json

--json diagnostics may include suggestedFix (did-you-mean). Not every gotcha is auto-fixed — if the tool is unsure, it only warns. Flat aliases and parseJson stay until a dedicated 2.0 MAJOR; --fix / gotchas are the prep.

Diagnostic sources you will see in the JSON:

• malda-gotcha — io.input EOF loops, AnsiConsole.markup without a newline, unused validate, arr.append, parseJson homograph, unknown borderStyle.
• malda-tools — @Tool / @MCPTool with no type hints and no schema argument.
• malda-agents — static team topology (literal graphs / plans only).

Optional @version("2026-09-01") on a prompt is an informational pin. malda prompts --diff compares prompt hashes between eval baseline files so you can see that the template changed even when the eval still passes.

24.11 A first path through the chapter

If you are writing your first agentic program, do these in order. Each step stays offline until you choose otherwise.

1. Write a schema and a prompt (chapter 10). Call evalPrompt with a fixture dict. Confirm checked.ok.
2. Move that check into a suite / case and run malda eval.
3. Print trace.journal() so you can see the prompt event.
4. If the program uses tools, add a policy that denys shell (and readOnly the work folder). Mint tokens as in 12.6.
5. Wrap slow work in within (30s) { … }.
6. When you finally await a live model, record a cassette and replay it with MALDA_REPLAY_STRICT=1.
7. Before you ship the file, run malda check --json (and --fix if the mechanical edits look right).

That path uses every construct in this chapter except streaming and grounded citations, which you add when the UI or the RAG project needs them.