We halved the model's commands. Accuracy did not change.
Auspex ran almost twice as many commands as it needed. A simple budget fixed it.
When Auspex diagnoses a fault, the model runs read-only commands on the network: read the topology, check the VLANs, ping a host. Every command takes time. In testing we saw the model run far more of them than it needed.
What was wrong
Our first guess was missing information. Maybe the model kept looking because it did not understand the network. So we gave it a map of the topology up front. Nothing changed. The real problem was simpler: the model had no sense of when to stop.
The fix
We added a budget to the loop that runs the model. It has two thresholds:
- At 20 commands, the model gets a short reminder: name the most likely cause. If the evidence is enough, finish. If not, make one more call.
- At 35 commands, the run stops.
Each threshold fires only once. In early tests, repeating the reminder pushed the model off track.
The result
We measured on 89 exam questions. The model used 48% fewer commands, solved exactly the same number of faults, and no run hit the hard limit anymore. The change shipped in auspex-cli v0.3.3. To get the old behaviour back, set AUSPEX_TOOL_BUDGET=0.
The lesson
Not every problem with a model needs more training. Sometimes the fix is in the program around it. A small rule saved half the work and cost nothing in accuracy.