A week later and after many hours in front of the computer, my project is a complete failure and a great success.
Let me explain.
The Failure
First the failure – the project is almost in the same place it was a week ago. The agents and I started working on a small UI interaction where the user selected (clicked) a checker with a valid move on the board, which should have de-highlighted the other checkers with available moves and changed the highlight of the selected checker. Simple, right?
We had a board. We had checkers. We had dice. The player could roll the dice and after this the checkers with valid moves were highlighted. All this was implemented by the AI Agent with multiple prompts, instructions, and a detailed description of the game of backgammon. Things were going well.
And then we got stuck. No matter how much detail I added to the prompts or how many tests we added, the agent was unable to implement the interaction.
That is when I decided to take a look under the hood and see if there is anything I can do to help. Until now I let the AI run without looking at the code in depth, just a look here and there, not really digging into the code as I would in a PR at work.
Boy, what a mess! Variables named i, j, k, s; Unused variables and methods all over the place; And then came the worst part – the AI Agent had implemented the game model and logic in two different places – once in the UI and the second in the model – and each implementation was being used by a different part of the application.
Tried doing some manual “intervention” to salvage the project, but the code was so bad that this will require a lot of work and beats the whole point. I’m thinking of throwing out all game logic implementation and leaving only the UI which looks good enough (much better than if I would have written it, even if I knew how to write it. I’m really bad at UI). We’ll see.
Great Success – Lessons Leaned
Now let’s talk about the great success. I’ve learned TONS during this process:
- My prompt writing capabilities have improved tremendously! I’m also learning new workflows to create the prompts, like asking another AI Agent to write a prompt and having long conversations with the AI Agent before starting implementation to make sure the Agent understand what I want it to do.
- It is VERY important to set up the project basics before writing the first line of code: architectural decisions, languages, frameworks, test frameworks, CI pipelines, code conventions, etc. And, obviously, these can be done with the help of AI Agents.
- Similar to (2), a good set of system prompts is a MUST HAVE for the AI Agent to perform correctly. This prompt (or many prompts) provides instructions that are given at the start of every session, explaining it how plan the task, system design guidelines, and general information for the AI agent that are relevant for all tasks. There are a number of repos with sample for this – awesome-ai-system-promts stands out with many start (3.5K at the moment), and kind mention also to my past co-worker who pointed me in this direction and also maintains his own PromptEngineering repo.
- My TypeScript and HTML/CSS reading abilities are much better than they were 1 week ago :-).
Last Thoughts
I’ve heard many people saying that AI Agents are like junior developers. This comparison does a disservice to the junior developers (and all developers in general). Humans are constantly learning, adapting, and improving through experience and feedback. In contrast, AI agents operate on static models that aren’t retrained after each task. Having memory banks and self-improvement helps but doesn’t mitigate the issue.
Most developers I’ve worked with don’t repeat the same mistake five times without progress. When I offer feedback, they internalize it and apply it moving forward. With AI agents, replicating that kind of learning is still tedious, imprecise, and resource-intensive.
Despite the frustrations and setbacks, this “experiment” with AI agents has been awesome. My vacation just ended so I’ll have less time to experiment with this personal project, but I’m definitely going to apply all the learnings to my daily work and see how it goes.
But don’t worry, the journey continues…