Most agent steps end as a multiple-choice question. Paying a frontier model for every one of them is a habit, and JevAgentHarness treats that habit as the product surface.
The paper's useful result is blunt: leaderboard accuracy on isolated tool questions ranked the wrong models for control inside a live agent, until confidence thresholds were fitted on the agent's own traces.
Cascades, thresholds and a 4B model trained on post-mistake states
JevAgentHarness sits on top of an existing agent loop. At each decision point the harness asks a typed multiple-choice question (tool, arguments, stop, reflection, routing, memory, action). A decision model returns option probabilities. In cascade mode, the harness accepts that answer only when confidence clears a threshold fitted from the agent's trajectories; below the threshold the large LLM decides as usual. Other modes (replace, assist, guard, trigger) are mapped across eight decision points, but the cost wins that survive Stage 1 are mostly tool and argument cascades.
AgentDecisionBench evaluates decision models at three depths: isolated questions, step-level scoring on recorded states, and task-level control where choices change the next state. DecisionGym supplies a customer-service environment with acceptable next-step keys even after detours. Because stock decision models rarely see states that follow the agent's own errors, the authors train AgentDecision-4B on those states using Jev's answers as labels, then optionally blend public tool-use data.
Long agent traces made every-step frontier calls the expensive default
Public agent leaderboards already show five-figure evaluation bills, and production coding or support agents compound the same pattern over longer horizons. Routers and small function-calling models exist, yet they are usually scored on fixed questions. JevAgentHarness lands as the harness-native version of that idea in the same October cluster as cache-engineering work: a cost-control layer that leaves the strong model in place for hard steps. The 69.5% task-success warning is why the piece matters now. Teams buying "fast tool routers" on isolated accuracy alone have been reading the wrong chart.
MIT harness code; models and data under mixed CC licences
The project site lists MIT code for JevAgentHarness, CC BY 4.0 for AgentDecisionBench, and CC BY-NC 4.0 for AgentDecision-4B / 0.8B weights and several training sets. Activity: arXiv:2610.21412 with a public project site and a promised GitHub release under the JevAgentHarness organisation. Maturity: usable for teams that can run shadow-mode logging and threshold fitting before turning cascades on. Production-shaped only after you reproduce non-inferiority on your own task mix; the paper's strongest savings are on DecisionGym and ALFWorld-style agents, with smaller gains on some coding harnesses.
Fit thresholds in shadow mode before replacing tool choice
If your harness already logs tool and argument decisions, enable shadow logging, fit per-point thresholds on held-out traces, and promote only cascades that keep success within your tolerance (the paper uses a three-point band). Prefer AgentDecision-4B or an in-domain 4B trained on your post-mistake states when you need open weights and shifted-task robustness; keep Jev or another teacher in the loop for labelling. Coding agents can try the OpenAI-compatible proxy path that routes routine steps to a fast tier without rewriting Codex or mini-SWE-agent. Skip hard replace modes for irreversible tools until you have a separate policy guard; the paper itself warns that a cheap stop-guard accepted many wrong end attempts when the agent LLM was weak.
Isolated leaderboards, unfitted thresholds and guards used as safety
Trusting BFCL-style isolated ranks led to a model that won the chart and then solved only 69.5% of tasks under control. Turning over every tool choice without a fitted threshold lost 19.0 points. A fixed 0.5 confidence cut also collapsed success in the ablations, while the fitted cascade held outcomes flat at 14.2% lower cost on 500 tasks. Using decision-model guards as the only control on irreversible actions is unsafe under the authors' own Flash-Lite results. Adapter failures are designed to fall back to the LLM for cost and speed only; they are the wrong place to enforce security policy. Adversarial observations that flip typed answers were left largely untested.
Measure your own cascade before you celebrate the 4B sticker
Stand up the harness against a slice of production-like tasks, compare strong-only versus routed runs, and keep the three-point success budget honest. The open Monday question is whether your tool taxonomy is stable enough for a fitted threshold to remain calibrated after the next prompt or tool-schema change.