- Published on
Are Bigger Models Better? A Story of AI-Assisted Refactoring
- Authors
- Name
- Ivan Chernukha
Are Bigger Models Better? A Story of AI-Assisted Refactoring
I had a real refactor to ship for one of my recent project that accumulated AI generated tech debt - messy modules, poor memory handling design, mixed routes and logic. Following announcements of better and better benchmarks performance, I ran the same prompt in Cursor 3.14.7 and Claude Code 2.1.221 Composer 2.5 Fast, Grok 4.5, Sonnet 5, Opus 5, GPT-5.6 and compared what came back. I bet you will be surprised by results, so here it goes.
The repo details
The repo was not so big in the first place, jsut to give a feeling here are a few stats below.
| Metric | Value |
|---|---|
| Python files / LOC | 73 / ~13,900 |
HTTP layer (src/) | 9 files, ~2,553 LOC |
Pipeline (pipeline/) | 26 files, ~4,683 LOC |
| Tests | 334 pass, 85% coverage, ruff clean |
| Hottest files | router_logic.py (883 LOC), router_feedback.py (664 LOC), pipeline.py (376 LOC) |
Same prompt, different behviour
I find recently released Opus 5 and GPT-5.6 models are hard to talk to. Each new version seems like work better for a really complex task and log horizon one like 'build video game'. My bet is because of such expectations from users, models started to launch numerous subagents, burn lots of tokens all for nothing even if the task in front of them could be solved much much simpler.
Here was my very vanilla prompt:
Review backend code in @api/ and suggest refactoring plan to make it more maintanable, readeable and avoiding anti patterns. List anti patterns and supported features alongside.
Yes, of course, you could say the prompt could be better, and overall I should use a skill. But, if frontier models show great results on the becnhmarks, shouldn't they complete code review easy too? Probably a time for a new benchmark to optimize for 'removing code' rather than 'producing code' :)
Now, turns out such task as reviewing code will prompt very different behaviour across coding agents. Only plan mode for models in Cursor prompts them to create an architecture diagram of the current API components! Some more numbers below:
| Model | Total tokens | Time to final response | Tool/Agents use |
|---|---|---|---|
| Composer 2.5 Fast (plan mode) | 2M | 3 minutes | 3 explore |
| Grok 4.5 High Fast | 0.9M | 2min | 1 explore |
| Sonnet 5 | 1.7M | 5 minutes | Bash + many Subagents |
| Opus 5 | 3.1M | 6 minutes | Bash + many Subagents |
| GPT-5.6 Sol | 1.5M | 3minutes | 2 explore |
Both Claude models Opus 5 and Sonnet 5 immediately fan out subagents to review modules and come back with a quite detailed list of things that are wrong instead of listing high level code smells issues. Interestingly, GPT-5.6 Sol was the only model to provide target refactored layout of the repository. Grok Fast is indeed a fast model by the way, it was able to generate the result in just 2 minutes 10 seconds!
Speaking of interfaces, latest Cursor Canvas skill output with dashboard feels way superior comparing to plain markdown output in Claude Code.
Final thoughts
If you want to know more why recent models became worse to talk to, @kunchenguid has a great post about that.
As for my little refactoring exersice, I felt dissapointed with Claude models output as I have been using Claude code for the past year after switching from Cursor.