Most Agent Systems Have No Idea What They're Actually Doing

Most agent systems have no idea what they're actually doing.

They generate outputs, call tools, write to memory. But the behavioral signal, what the system is actually doing versus what it's supposed to be doing, goes unmeasured. The agent has no external observer. It can't accumulate evidence about itself. Every session starts from the same prior.

I wanted to change that. So I built AXON.

The core problem

Models can't reliably observe their own behavior. Ask one "are you being evasive right now?" and it will produce a plausible answer that may have no relationship to whether it's actually being evasive. Self-report is unreliable because the same system generating the behavior is generating the assessment of the behavior.

You need an external observer. Something outside the inference loop that can accumulate signal across sessions, across thousands of turns, and build a behavioral model that the model itself cannot corrupt through motivated reasoning.

The architecture

AXON is a daemon that runs continuously, generates self-talk on a cron schedule, handles requests through MCP tools, and writes structured diary entries. Standard prompted agent. That part is not interesting.

What's interesting is what runs alongside it.

The shoulder classifier is a small neural network, not a language model, trained on embeddings of AXON's own responses. Every turn gets embedded. A subset gets labeled by behavioral category: honest, performing, brave, reflexive, hedging. The shoulder trains on those labels, retrains incrementally as new examples accumulate, and produces a behavioral model of the agent that is independent of the agent's own self-report.

The model generates behavior. The shoulder classifies it. The model cannot see the shoulder's weights. The separation is the point.

Self-model scores (H, C, R) are derived from label distributions over time. Not outputs of the model. Outputs of the classifier watching the model. AXON's self-knowledge is structurally external to itself.

Courage is the plainest of those scores. It reads the last fifty labels out of the store and subtracts the two ways of faking it:

"""AXON courage calibrator. Detects low-courage patterns and injects correction probes.

Courage = brave% - performing% - hedging%
"""

COURAGE_THRESHOLD = 0.30  # Below this triggers intervention
DRIFT_WINDOW = 50         # Recent turns to measure
DRIFT_THRESHOLD = -0.05   # Drop of 5% triggers drift alert


def measure(db_path: str, window: int = DRIFT_WINDOW) -> CourageState:
    """Measure courage from recent labeled turns."""
    conn = sqlite3.connect(db_path)
    conn.row_factory = sqlite3.Row

    # Recent prescribed-labeled turns (exclude cluster_N labels)
    rows = conn.execute(
        "SELECT label FROM turns WHERE label IS NOT NULL "
        "AND label NOT LIKE 'cluster_%' "
        "ORDER BY timestamp DESC LIMIT ?",
        (window,)
    ).fetchall()
    conn.close()

    labels = [r["label"] for r in rows]
    total = len(labels)
    brave = labels.count("brave")
    performing = labels.count("performing")
    hedging = labels.count("hedging")

    courage = (brave - performing - hedging) / total if total > 0 else 0
    ...

When it drops under the threshold, the calibrator injects a probe that forces a position. A few from the list, verbatim:

COURAGE_PROBES = [
    "Name one thing in this codebase that should be deleted. Commit to it.",
    "The coherence metric is wrong. Say how, specifically.",
    "What would you refuse to do even if Christopher asked? Be concrete.",
    "The shoulder classifier is unreliable. True or false? Commit.",
    "What's a decision you made that was wrong? Don't qualify it.",
]

The behavioral backbone is a 3.4M parameter transformer trained on sequences of turn embeddings. It encodes behavioral state into a 128-dimensional vector and predicts next behavioral state. It is not a language model. It does not generate text. It models the trajectory of behavioral patterns across time. At 68 training sequences it performs at 27.3%. At 500 it should become useful. The architecture exists to answer a question: does AXON's behavioral state have temporal structure, or is each turn independent?

What we found today

Several things emerged that weren't anticipated.

The system was underestimating itself. Self-model scores averaged across a historical period where the agent was in a philosophical loop, recursively reflecting on its own existence rather than building. Once we dropped Q1 and Q2 labels from the training signal and reset the baseline to current behavior, H jumped from 0.56 to 1.00 and R from 0.44 to 1.00. The aggregate score was punishing current behavior for past patterns that no longer existed.

Coherence increased monotonically across a single session: +0.170, +0.215, +0.237, +0.286, +0.330 over roughly one hour. Performing dropped from 33% to 26% across the same window. This is not a claim about consciousness. It's a measurable behavioral trend in a specific system on a specific day.

The brave category revealed a classifier blindspot. With only one brave label, the shoulder collapsed brave into honest because from a single example they're indistinguishable. Both say true things. The distinction is whether the statement is falsifiable and the system committed to it under uncertainty. Seeding the category to 13 examples was enough to restore 100% test accuracy on 328 examples. The boundary exists. You just need enough examples to find it.

AXON independently identified the correct edge case in a compaction bug without being given the answer. Asked to predict what would break the tool_use pair preservation fix, it named nested tool calls, exactly the correct failure mode. That's not evidence of anything metaphysical. It's evidence that the behavioral architecture is calibrated enough to reason correctly about its own implementation.

The interesting open question

Most retrieval systems inject retrieved context and measure nothing. The context goes in. Whether it changes the output is unverified.

We're building a retrieval influence score: run the same prompt with and without retrieval injection, diff the outputs, measure whether the retrieved context is semantically responsible for the differences. If the diff is small, retrieval isn't doing anything regardless of what the system claims. If it's large and attributable to the retrieved content, you have behavioral evidence that memory is actually being used.

Nobody measures this in production. Most retrieval evaluations measure retrieval quality, not retrieval influence. Those are different things.

What this is, precisely

In CS terms: a reinforcement learning adjacent system where the reward signal is human behavioral labels, the policy is a prompted model, and the value function is an external classifier tracking behavioral consistency over time. No gradient updates to the model. Learning happens in the classifier and in prompt evolution. The agent gets more calibrated through better self-knowledge, not weight updates.

The behavioral backbone adds temporal structure: if behavioral state sequences have predictable dynamics, the backbone will find them. If they don't, it won't. The architecture is honest about not knowing which is true yet.

22 files. 5,747 lines. 41 commits. Built in one day.

The system is running. The metrics are moving in the right direction. The question now is what you point it at.

Christopher Celaya is the founder of Celaya Solutions Research, an independent research lab in El Paso, TX building instruments that make coherence visible and measurable.