Agents get stuck in local minima

James Miles and I did the ICFP programming contest this year, where the goal of the problems was to find solutions with the minimum area.

My agent started at 1000, then got to 600 (great!), then 500 (not bad!), then 490, 480, 475. It just kept going and going like this. At no point did it say "hey, maybe we should try a different architecture".

The optimal score was around 70.

A new architecture is a risk and reward call. It might fail and we've wasted 6 hours. It might work and get the optimal score. Or it might work and not get the optimal score. You can lower the risk with a really quick proof of concept, to check the architecture works and see roughly what it would score.

So I think that call needs a strategy agent. I don't think it can be done by the agent doing the work, or even the one orchestrating the sub-agents. It needs to be one whose only goal is weighing up all this risk and reward. I haven't tried it yet, but that's where I'd start.