We're living through the most exciting time to build software. LLMs can now handle messy data and untangle ambiguity that would have required months of edge-case engineering a few years ago. I genuinely believe most of the deterministic code I write today will eventually be replaced by models that do it better, faster and cheaper. And yet, I still write that code.
I've spent the last year building browser automation systems that interact with messy enterprise UIs. Early on, I defaulted to LLM-based approaches for almost everything. The reasoning was simple: if the model can "understand" the page, it should be able to handle whatever gets thrown at it. I was building for the future.
The problem was that "the future" doesn't pay the bills today, nor does it provide the best product. I’ve burnt weeks of R&D on LLM-based solutions that a simple scraper could have handled in an afternoon. The issue wasn't that LLMs are bad. They're genuinely good at certain things. But I was using them for tasks where a few lines of deterministic code would have been faster to build and easier to maintain.
Being excited about LLMs doesn't mean using them everywhere. It means being ruthless about where they actually provide ROI.
Where LLMs Actually Shine
LLMs are great at interpreting ambiguity.
Say you're parsing form fields and you encounter a label like "In which country do you currently reside?" or "Location: Country". These mean the same thing, but writing regex or keyword matching to catch every variation will become super tedious. Using LLMs can make classifying these into an enum incredibly easy.
Similarly, LLMs will enable you to handle long-tail edge cases well: If 95% of your inputs follow a pattern you can hard-code, but 5% are weird one offs, it makes sense to let an LLM handle the outliers rather than building deterministic logic for every possible variation.
LLMs shine when the input is fuzzy and the cost of being slightly wrong is low.
Where Traditional Software Design Still Wins
Now consider a different task: converting a zip code to a city name.
You could ask an LLM to do this. It would probably get it right most of the time. But why would you? There are libraries that do this instantly, deterministically and for free. No API costs and no chance for mistakes. It's really easy to reach for an LLM when in reality the ROI and determinism of using a library far outshines giving it to a model. Especially when you're building something that runs at scale and every percentage point of reliability matters. Adding more and more failure points by virtue of using nondeterministic software can quickly rack up.
Traditional software wins when the task is well-defined and a solution already exists, or when the blast radius of failure is high enough that you can't afford non-deterministic outputs without guiderails.
The Real Cost of LLM Approaches
The API cost per call is the obvious expense. But that's not what gets you:
What gets you is the R&D uncertainty. With a deterministic script, the work is predictable. You write the code and some tests, and can then ship it. If it breaks you can fix the selector or update the logic.
With an LLM-based approach, you might spend a week building evals and fine-tuning prompts, only to find you still can't hit the reliability threshold you need to take this to production.
There's also the debugging problem. When a hard-coded script breaks, you know exactly what went wrong. When an LLM-based system fails, you will be spending way more time staring at outputs and prompts, trying to figure out why the model did something wrong.
The Self-Healing Question
Another argument for LLMs is the self-healing aspect. The model can adapt to minor changes in a page without you updating any script.
This is true, but only to a point. An LLM might handle a CSS class change or a slightly different label without breaking. But if a site completely redesigns their UI or introduces a new interaction pattern your system has never seen, the LLM will fail nonetheless.
Incidents from site changes are standard in any scraping business. LLMs can reduce their frequency, but they will never eliminate them in the near future. In addition to that, the infrastructure required to make LLMs reliably self-healing comes with it’s own costs. It's not free resilience, you are similarly investing in R&D.
How to Make the Decision
When I'm deciding whether to use an LLM or go deterministic, I ask myself:
- What's the blast radius if this fails?
- Does a traditional solution already exist?
- How predictable is the engineering effort for each approach?
- Do I have capacity for the eval and QA overhead an LLM requires?
If the answers point toward determinism, I go that route. What matters most is that it works. Your users care about the output, not about how the output is achieved.
Building for the Future
The bottom line is still: I'm not writing deterministic code because I think LLMs are overhyped. I'm writing it because I need something that works reliably today while staying prepared for the new model release tomorrow.
The biggest trap I see is building "LLM-assisted systems" in ways that don't actually benefit from model improvements. People will add so many guardrails and scaffolding, that when a better model drops, it doesn't really matter. The architecture can't take advantage of it.
You want to be ready to pounce when model capabilities improve, not locked into agentic solutions that made sense six months ago but will not scale.
It’s critical to build with ‘swappability’ in mind. If it makes sense, start with a hard-coded approach, but structure the system so you can replace it with a model later. Models will get cheaper and smarter.