The Model That Thought Too Much
4 min read
I built a worker that scored job postings against my preferences. It returned zero matches. Every day. Not "no good jobs" zero — broken zero. Nothing was getting scored at all.
The model wasn't wrong. It was thinking too much.
What Actually Happened
I'd built the scorer on qwen3.5 running in thinking mode. It's a strong reasoning model, and that's exactly why it failed. In thinking mode, the model spends a chunk of its token and time budget on an internal reasoning trace before it produces an answer. For open-ended analysis, that trace is the entire value — it's the model showing its work and getting to a better conclusion.
My task was not open-ended analysis. It was: score this posting 0–100 against this brief, return JSON. Structured output. The model would dutifully think its way through a long reasoning trace and hit the call timeout before it ever emitted the JSON. So I had a worker that fetched perfectly, deduped perfectly, and scored nothing — because the scorer was busy reasoning when I needed it to just answer.
The Generalizable Failure
This is a trap worth naming because it doesn't look like a failure when it happens.
Thinking and reasoning models silently sabotage structured-output tasks — scoring, classification, JSON extraction, anything where you want a fast, shaped answer. They don't error. They don't crash. They consume the budget on a trace and return late, empty, or truncated. The task fails, but it fails quietly, which is the worst way for anything to fail. You stare at "0 results" and start debugging your fetch logic, your dedup, your threshold — everything except the actual culprit, which is that you handed a throughput task to a model built for deliberation.
The fix was not a better prompt. It was not a longer timeout. It was matching the model to the job.
The Decision Framework
Here's the rule I now apply without thinking about it:
Match model class to task class.
- Open-ended analysis, synthesis, judgment, multi-step reasoning → use a reasoning model. Let it think. The trace is the value.
- Structured output, scoring, classification, extraction, high-throughput shaped answers → use a fast, non-thinking model. You want the answer, not the deliberation.
I swapped the scorer from qwen3.5 to gemma4:e4b — a fast model tuned for structured JSON at throughput — and scores started coming back in a fraction of the time. Same pipeline, same prompt, right model class. Fixed.
Two operational habits fall out of this, and I'd push both onto anyone shipping LLM workers:
Pass think:false wherever the runtime supports it. If a model can run in thinking mode, assume it will unless you tell it not to. For structured tasks, turn the trace off explicitly. Don't leave it to the default.
Live-smoke-test every new worker before you merge. This is the one that actually bit me. My CI mocked the LLM call — which means the test passed green while production returned nothing, because the mock never reproduced the thinking-mode timeout. The pipeline was "tested." The pipeline was broken. A mocked LLM tells you your plumbing is connected; it tells you nothing about whether the model can actually do the task inside your real time budget. The only thing that catches a thinking-mode failure is running the real model against a real input and looking at the real output before you ship.
The Real Lesson
The mistake wasn't picking a bad model. qwen3.5 is excellent. The mistake was picking a model for its reputation instead of for the job in front of it.
"AI product" maturity, in my experience, is less about knowing which model is best and more about knowing which model is best for this task. A reasoning model and a fast structured model aren't better and worse — they're different tools, and using the deliberation tool for the throughput job gives you a worker that fails green, passes CI, and quietly does nothing every morning at 8 AM.
Pick the model for the task. Turn off thinking when you don't want thinking. And never trust a mocked LLM to tell you a worker works — make the real model produce a real answer before it ships.
If you're building agent workers and want to talk model strategy, I'm at reed@grainlabs.io.