Web DevelopmentI’m calling it now, the adoption of AI agents into software development will be one of the most costly mistakes in the field’s history. Agents cannot program, and it’s taking longer and longer to realize that they can’t. They are a highly sophisticated statistical model designed to mimic the distribution of programming. The output is broken, but in a way that’s getting harder and harder to detect. Which is exactly what you’d expect from an increasingly accurate statistical model.
if it's broken in a way that can't be detected, is it actually broken?
all software is broken in some way. if the rate of bugs generated by llm and the severity of those bugs drops below the rate you would expect from a human programming team, then llm is offering something competitive.
This. They're trained on human data, and people can't seem to grasp that they act with roughly similar truthiness as actual humans. Humans often make just as many mistakes.
Also, unlike say, writing articles, like 75% of coding is copy.pasting blindly from elsewhere anyway.
But you're forgetting the key difference that makes it so much worse - we can fix human mistakes especially if we can talk to the human to figure out how. With an llm we have no external reference, only poorly designed code where the comments are there to guide the writing, not describe what was written. So it's much harder to debug an output, and the llm cannot be trusted to clean it up either.
A human can be held responsible. A machine cannot. If the machine writes bad code, and someone gets injured or killed because of it, who takes responsibility?
I state again: a machine cannot be held responsible.
It is never the coder that is responsible, it is the one who makes the code available to use. Often with humans, they are one and the same. With machines, they are not.
You can totally fix AI-written code with AI. You tell it something is wrong, it tries to fix it.
I did a recent experiment with AI writing a document format converter and that's exactly what I did. It wrote some code, I checked the output, found a formatting issue or similar, asked it to fix it, repeat. It works unreasonably well and with Fable the final code isn't even bad.
You can fix problems, if you know they are there and there is a model of that problem being fixed.
You can't fix problems you don't know are there, or do not have modeling.
That's true for humans too surely? How would anyone fix problems they don't know exist?
No one paying the bills cares
This alarm's being rung for over a year now, so "calling it now" means finally reading the writing on the wall
yeah I was kinda like. calling it?
Agents cannot program
This is just factually incorrect. Difficult to get past a false assertion of this magnitude.
They are a highly sophisticated statistical model designed to mimic the distribution of programming.
I thought we had got over the stochastic parrot nonsense by now.
You can totally have objections about the ability of AI to program - how good it is, poor failure modes, high cost, technical debt, knowledge debt, broken social contracts, etc. All valid.
But if you're still in the "It's just a next word predictor! It can't really think!" stage of denial, even now... Sorry you're an idiot.
