66
So im having a "issue" that might not be an issue? Ill explain.
Most of work is going full AI. We have around 20 developers. Its about 3/4 that are using AI to do ALL the work with Claude. Earlier in the year, everyone got a ton of AI credits and the amount of "Finished" products have popped out in mass. The amount of projects we have had is skyrocketing.
Now for the problem. Half the projects have terrible bugs. Pages that go no where, bugs that are VERY hard to find in the code (because its filled with code smells). And sometimes the site/program/etc... will just stop working for no reason because they decided to use some old version of react instead of anything modern. Because AI.
Even the ones that work will often be very difficult to debug if its just something the AI did or something the user did.
So a LOT of AI projects are getting abandoned...for more AI remake projects. I cant event call it iterations because no one learned anything making the last project.
Now once in a while I do use AI. It was part of my research back almost 20 years ago. But its mostly for one offs like creating a command line argument in python or give the the reference page for xyz feature. But I dont have it write the entire codebase for me. Seems like a recipe for disaster. And ive seen it bork a couple of peoples machines before.
I dunno am I the old man yells at clouds now?
I cant event call it iterations because no one learned anything making the last project.
You got the right point.
Why I haven't even started using AI ☝️
Its a strange feeling. I kinda get it, its easy to turn off your brain. But thats KINDA why I get paid. To think right? To solve the issue whatever it is.
These coding LLMs are essentially like a vacuum that sucks your brain dry. Anything you do with it is used to train: those comments you leave in a PR, the instructions you give within the chat thread, the manual fixes you make to the broken output, etc. So you lose ability, while the models gain ability.
The logical extreme here is that, in the future, the only way humanity will be able to code/write/design/whatever is through LLM products, because everyone forgot and stopped learning.
Whether or not you think that's progress, I think everyone can agree that what is definitely bad is that those models are centralized and controlled by a small group people (who also happen to be the worst people). It is the ultimate tech bro monopoly strategy, to win complete control over 100% of the high skill sectors of our economy. Any child not born to a billionaire family circle will only be able to make a living from physical labor.
the only way humanity will be able to code/write/design/whatever is through LLM products, because everyone forgot and stopped learning
Not very extreme from what I see.
It regularly happens for a lot of things to smaller extents (who can make a computer to run their code? I'll hypothetically give you RISC V source and lots of money for materials and see if you can manage without giving said money to the company that creates fabrication machines) and AI just reduced the bar for the user, in a way that enough people might just want to give up trying to understand how stuff works.
Use it to skip the part where you type it all out yourself, and go straight to the part where you review the draft code.
Its not a 'do my job for me' imo its predictive text on roids.
Yeah. It's not why I became a programmer, to tell something else to program for me. Might as well find another job in that case.
Our job is to think of logic patterns that solve problems, not necessarily to manually type them out ourselves.
Its valid that the typing part means something to you, im just saying it is possible to integrate it without turning your own brain off.
The reason why I like the typing part is because it forces me to really think about my code. Often enough I realise that the model I had in mind for my code is not perfect and that it makes sense to change it now, so that future development gets easier. If I would have outsourced the typing to the AI, its possible, that I would have not realised that. This is especially important, when thinking about how I want to integrate a subsystem into the rest of my codebase, because that is nothing I would outsource to AI.
I enjoy the typing part as well, yes.
You are most emphatically not.
AI is as good as the human in the loop. Two rules would help in the long run: accountability (AI proposes, dev accepts, therefore dev gets the 2AM call when it hits the fan) and understanding (thou shalt not commit code thou cannot explain). If you have the power to push these, you have a fighting chance. Otherwise ask Claude to polish your CV.
It's a really mixed bag. I was a bit of an AI laggard for work stuff, while most of my org went all-in. Some stuff worked, most stuff didn't. Things that didn't work:
- Everyone is an engineer now (lol)
- Replacing engineers
- "Software factories"
- "Agentic product design"
- Intern + Claude = senior
But some stuff did work, and most of it was our more senior and technical staff's usage of claude. I'll admit that after I saw my peers hop on board and truly accomplish impressive (functional) things, I dabbled more and I think I have a rule of thumb that helps me. If I'm already a SME, claude may turn me into a super dev. I can smell bad smells, stop it early, and generally know what decisions I need to make up front. If I don't already know what I want, I NEVER let claude drive. Letting claude make decisions is a BAD IDEA.
So honestly I think it depends on who's using it and how.
My problem is that it's killing the next generation. When used properly, AI is replacing junior work. In a decade, all the current seniors will be a decade older and burning out and there won't be any new ones to take their places.
Oh I agree with that as well. I have no idea how I would be a junior on this industry now, nor do I even know how to manage juniors anymore given management's expectations. We're gonna have a real dearth in some years and I don't know what's gonna happen.
In a decade, all the current seniors will be a decade older and burning out and there won’t be any new ones to take their places.
That's by design. Nvidia, OpenAI, Anthropic, etc will fill that void.
But some stuff did work, and most of it was our more senior and technical staff’s usage of claude.
Almost as if the senior staff sees it as a tool in a toolbox instead of a subcontractor.
Its pretty good at SQL that already exists! And it of course heavily depends on the model. And theres the rub. I think we spend 1.5x for devs now cause of the tokens haha.
Claude can do some crazy stuff thats for sure. But I recall there was code generators that did entire apps before the AI boom. And they were more reliable but more static. And it keeps making like .75 of what you ask, then you have to spend twice as long to debug. At least in my opinion. And worse is that no one can read the code so they dont know what it does. The devs just go, well ill ask cluade to do it instead of taking responsibility. (old man noises)
I also think org rot is real and is a different problem. Or at least... related but distinct.
Corpo leadership is completely brainrotted by promises of LLM sentience and firing half the company and I have no idea how to manage upwards on that front.
I think the sloperators I see at my company are people who weren't fantastic in the first place. If you were already a "rockstar" or a "10x" dev, claude probably makes you a 12-15x dev. But if you're just a person who works hard, and you aren't at a point where you understand when and why it's OK to break rules, I think your output is gonna suffer and make your team suffer as a result.
I have no idea how i would be a junior in this environment but I know as a senior/staff, claude has let me do more than I could before, and my defect rate is not higher than pre-LLM. Mostly because I'm a SME and I bring my judgement and humanity to the design, and claude is just there to generate diffs for me to read and nudge.
Sorry for the barrage of comments, I've just been thinking and talking about this a lot recently and I tend to think out loud.
So, here's where I think I'm gonna lose you. These days I read diffs but I so rarely read entire files. I generally know how pieces of things fit together, and I generally know how the systems are architected, the tech stacks, and I can still read functions and syntax... but I wouldn't say I "know how it works" at a code level at this point. Once I accepted that it's OK to know the high level, and verify the diff/PR level, it's OK to not have 100% awareness on implementation, using claude became a lot less... smelly to me.
It's not an easy thing to let go of, but the reality that I've experienced is that generally, claude can write good, secure functions. From a purely mechanical perspective, it'll probably do a better job than I would on syntax, knowing the nooks and crannies of libraries, etc. But I review the diffs to be sure. However, I, the human, know what I want and generally how it should be done.
I'd say that learning where to not bother with claude and where it works well was a bit of a thing to figure out, but I really don't spend hours debugging claude slop. Cause honestly in qualified hands, it doesn't really write slop. It does write slop if you let it, or ask it to take on too much scope, but... "work this ticket for me, pay attention to x y z and don’t change a b or c" is well within a reasonable model's capabilities.
I'd never pressure anyone into using LLMs and I have large ethical issues with the industry, but the idea that claude only slops doesn't match with my lived experience in the past 6 months. I suspect when the finances of this whole thing go boom, KIMI and other open source models are gonna be able to run on rented GPUs for relatively cheap. Deepseek's (reported) profit margins on inference seem to pay for the hardware (inc. GPU) in months... $1.6M per card in 5 years for revenue (not profit).
Obviously these models are not intelligent and cannot be, but they can be very powerful code generators.
If my processes fuck up i think its something like 2 mill in transactions that fuck up. Haha. So yeah...
I mean I've never had data loss, pre or post LLM land so... but yes, risk tolerance is your own.
Honest question, in your opinion, how experienced was the development team prior to AI being introduced? What level of code quality could be expected? And what was the average velocity for development projects?
I have substantial development experience prior to AI and after. I also have spent the last eight years at of the the lagest technology companies advising a wide variety of our customers on implementation.
I can tell you that both my own experience and what I see at customers is that if you already produce low-quantity code and/or are not truly development minded, then the AI tools will largely reflect that. They are tools and require proper direction just like a human. If I give it a three paragraph prompt to create some app it will probably do it and it might even work, but it likely won't be reliable as you have experienced. However, if I put on my software architect hat and give it a detailed instruction file which coding standards, architectural patterns, frameworks, etc. It will usually produce a pretty good product.
As the age old computer adage goes: Garbage in, garbage out.
Honest question, in your opinion, how experienced was the development team prior to AI being introduced?
At work you have to have a college degree and at least 5 years of experience. Generally thats the case. I think at most we have one person 20ish years of experience. all the way to two years straight out of college.
What level of code quality could be expected?
Well we had CI/CD, testing,etc... in prior projects. Now its just a toss up what does what. Its better now actually haha. Because docker and other containerization tools mostly rather than AI.
And what was the average velocity for development projects? No idea. I can say we accelerated like mad after beginning of this year. So 10x projects at least.
I get that giving it a VERY specific detailed prompt can get better responses...but at that point, I can just prompt myself and call it good. I am the AI/worker at that point.
What is your question exactly? Do you guys test and review the code? It sounds like you don't.
We do PRs.
This might help, but tell Claude to do Test Driven Development [TDD] which will require each feature/function/etc.to have a test case and a pass before it can exit the worktree before a PR.
Also look into graphify which does a bit of "explain what this does" and makes things very easy to describe the code in a systematic way. AST is here again.
I'm having success with an agentic orchestration model that doesn't move on unless they pass the tests. And if they fail they hand off to a smarter model. I have to guide it and sometimes ask basic questions about why we're trying to load docker locally when I have guthub actions to do it, but if you know that much you can just poke it with a stick occasionally. The graph is updated and I have records for design decisions and reports on failure as well as tests to pass.
This is definitely a different approach and I can easily see how some people don't get it; and it's probably tied to doing things the hard earned way. Less developer and more TPM in terms of mindset.
I think throughput [lines of code, commits, PRs, beans, etc] is a worthless metric, velocity means speed + direction. A poorly guided version is just scattered all over, make it work and that's it, until the next round of changes.
But it's definitely a different game now.
Ok and what kinds of review comments do people leave? How often are PRs rejected? Do you have CI running your test suite? Why do you think so many bugs are getting through? Usually you need a strong culture of people demanding rigorous testing methodology in order for bugs to get caught early in the dev lifecycle.
You’re not yelling at clouds. But AI isn’t going anywhere, least of all in software dev. If anything I’d suggest learning and exploring patterns like spec-driven dev and how best to build skills and agents.
I lead engineering for my company and it’s taken a tremendous amount of work to scaffold for quality. Everyone runs loops and ships much more quickly. But, like I said, that takes a lot of work and isn’t easy to capture in a one-off comment.
Lots of cool shit to learn, see, and do though!
Well given that my research was literally AI...Its a prediction engine in most cases. Hell I give feedback on hugging face regularly. I used to use hugging face to create specific models like the command line one a bit ago. Specific models for one specific thing. It was fun for a while, but then I got burnt out because everyone just distills without understanding what they are distilling.
I had a fantastic time making a bullshit simulator using startreck jargon. that was fun.
