Fresh agents catch more bugs

After working on some unit testing today, I think one of the key things sceptical engineers are missing is that a model can act completely differently depending on its context.

This is why getting a fresh AI to write unit tests catches a lot of bugs it made earlier. It's using a different part of its brain, kinda like how a tester will catch things an engineer won't.

The big mistake people make when starting out is assuming the agent applies all its knowledge all of the time. It doesn't. They think that because it missed something, it's dumb in that area. It does have jagged intelligence (good at some things, terrible at others), but a lot of the time it could catch the thing it stuffed up if you gave it the chance.

Just like us, it can only do so much at one time. Getting it into a different mode helps it think about the problem in a different way.

As models get better they can apply more of their knowledge at one time, but they're still not applying everything.

Writing unit tests also forces the agent to pull more information out of the code. This is feedback it wouldn't otherwise have got, and your next agent will thank you for it.