I’ve always run two AIs side by side.
I’ll ask the same question to both, compare the answers, and pick the one I like better. The other gets dropped. Then I keep going with follow-up questions against the winner until I land on a final answer.
It works, but it’s manual. I’m the judge every time.
Stuck on an 8 hour transit with nothing better to do, it hit me: this is exactly what you do when you evaluate a prompt. You get a model to generate a result, then use another model to judge it. So why not let the AIs judge each other instead of me doing it by hand every time?
So I built AI Thinktank.
What it does
Give it a topic. One AI produces an initial result. Then two or three other models take turns critiquing and refining that result - arguing it out between themselves - until they converge on a final answer.
What it doesn’t do (yet)
- Right now it only runs on each model’s built-in knowledge. No web search, no other tools. That’s next.
- I have only tested this with Gemini models. Testing with Anthropic models is next.
Try it
You’ll need to bring your own API keys to run it.
./J