← Back to the Research Program
Language & ReasoningMiddle researchExample brief
Do Different AI Chatbots Agree?
The research question
If I ask several AI chatbots the same question, do they give the same answer?
Abstract
I asked three chatbots the same set of questions and compared answers. They agreed on facts but disagreed on opinions and tricky questions.
Background
Different chatbots are trained differently. I wanted to see how often they actually agree.
What I did
I asked three chatbots ten questions — five factual and five open-ended — and compared their responses.
What I found
They mostly agreed on clear facts, but gave noticeably different answers on opinion and ambiguous questions.
What's next
I would like to test whether they agree more when the question is worded very precisely.
Takeaway
AI answers are not one single 'truth' — comparing sources matters, even with AI.