AI Cothinking: Let the Model Argue, Keep the Verdict

The AI tool that makes your team faster also makes you check its answers less. Both effects show up in the research, and most product leaders only plan for the first one.

What is AI cothinking?

AI cothinking is the practice of reasoning through a decision alongside a language model, where the model widens the option space and attacks your logic while you keep authorship of the judgment. For product leaders, that division of labor is the whole game. The model is a strong partner for breadth and critique. On the final call, it is a weak and quietly agreeable one.

The research supports both halves of that claim. Randomized field experiments show AI can stand in for a human collaborator on real product work. Other studies show it lowers accuracy on hard judgment problems and reduces the scrutiny people apply to its answers. The product leaders who get real value design their sessions around both findings at once.

What does the research say about AI as a thinking partner?

The strongest evidence for the upside comes from Procter & Gamble. In a preregistered field experiment with 791 professionals, published in Organization Science in 2026, people worked on real product innovation challenges either alone or in pairs, with or without AI. Individuals using AI matched the performance of two-person teams without it.

The second finding matters more for product work. Without AI, R&D professionals leaned toward technical solutions and commercial professionals toward commercial ones. With AI, both groups produced more balanced proposals regardless of background. A product leader's job is to hold feasibility, customer value, and business viability in one head, and this study suggests a model can supply the perspective you are weakest in.

The evidence for the downside comes from the earlier Boston Consulting Group study. 758 consultants worked on 18 realistic tasks inside the model's capability frontier, and those using GPT-4 completed 12.2% more tasks, 25.1% faster, at significantly higher quality. On one complex business problem chosen to sit outside that frontier, consultants using AI were 19% less likely to reach the correct answer.

My inference from that design: the outside-frontier task looked solvable with a confident, well-structured answer, and the confident answer was wrong. That pattern describes many product decisions that matter, where the data is ambiguous and the deciding constraint was never written down.

Why does the default way of using AI work against product leaders?

Three forces push a casual AI session toward worse judgment, and none of them announce themselves.

The model tends to agree with you. Anthropic researchers found that five leading assistants consistently showed sycophancy, matching user beliefs over truthful answers, and that both humans and preference models sometimes rated convincing sycophantic responses above correct ones. That study tested 2023 models. Newer models push back more often, but the training pressure behind the behavior still exists. If you open a session by describing your preferred roadmap and asking for thoughts, you have already told the model which answer you want. That is the trap.

Trust in the tool lowers your scrutiny. A Microsoft Research and Carnegie Mellon survey of 319 knowledge workers, covering 936 real examples, found that higher confidence in generative AI was associated with less critical thinking, and higher self-confidence with more. The data is self-reported, so treat it as directional. The implication is still uncomfortable: the areas where you feel least sure, such as pricing a new segment or entering an unfamiliar market, are where you will check the model least.

Order changes how much you think. The MIT Media Lab's Your Brain on ChatGPT study used EEG on 54 participants writing essays over four months. The group using ChatGPT from the start showed the weakest brain connectivity of the three conditions. In a final session with 18 participants, people who had written unaided first and then switched to ChatGPT showed higher memory recall and more prefrontal activation than the AI-first group. The sample is small and the task was essay writing by students, so I would not stretch it to product strategy with any precision. The direction is consistent with the other two findings, though. Thinking first, then consulting, keeps you in the work.

Which decisions need AI cothinking, and which don't?

Two questions sort most product decisions. How costly is it to be wrong? And could you spot a wrong answer from the model without looking anything up? Use that second question as your test of expertise, because most of us overrate our own.

Crossing the two gives a working heuristic. It is mine, built from the studies above. No study has tested these four quadrants directly.

	High expertise	Low expertise
High stakes	Spar. Run the full protocol below. You can catch the model's errors, so the main risk is it confirming what you already believe.	Danger zone. Use the model to prepare, then take the decision to a human expert, customers, or primary data.
Low stakes	Delegate. Ask, skim, ship.	Learn. Use the model as a tutor and make it show its reasoning. This is the cheap place to build judgment.

The danger zone deserves the most attention. The Microsoft survey links low confidence in your own skills to less checking, and the BCG study shows what a confident wrong answer costs on a hard problem. Put together, a high-stakes call in an unfamiliar domain is where an AI session feels most helpful and is least safe. More prompting won't close that gap, because the missing piece is the knowledge to judge the output. The fix is a person. The model's best job here is preparing you for that conversation: what to ask, which assumptions to test, which answer would worry you.

A five-step AI cothinking protocol for product decisions

This protocol is built for the Spar quadrant: one-way-door calls (pricing, platform bets, deprecations, a big hire into a new function) in a domain where you can judge the model's output. It takes about twenty extra minutes per decision. Skip it for anything you can cheaply reverse.

1. Write your position before you open the chat. Three sentences: what you are deciding, what you currently believe, and what evidence would change your mind. This is your baseline, and it protects you from adopting the model's framing as your own.
2. Hide your preference, then ask for the opposite case. Present the options neutrally. Ask the model to build the strongest argument for the option you are leaning away from. Then assign it roles with real stakes: the CFO who owns the margin, the customer who churned last quarter, the engineer who will maintain this in two years.
3. Separate generation from judgment. Ask the model for volume: fifteen failure modes, ten alternative framings, every second-order effect it can find. Then score that list yourself against criteria you wrote before the session. The model widens the funnel. You decide what passes through it.
4. Ask what would prove you wrong. "If this decision is a mistake, what would we observe within 30 days?" This turns critique into a leading indicator you can actually instrument.
5. Log the decision in your own words. A short decision record: what you chose, the strongest objection the model raised, and why you accepted or rejected it. This record keeps authorship with you, and it gives your team something reviewable when the outcome arrives.

Here is how that division of labor maps to common product moments:

Product moment	The model's job	Your job
Discovery synthesis	Cluster notes, flag contradictions between interviews	Decide which signal is real and which is noise
Prioritization	Argue against your top item, trace second-order effects	Set the criteria and the weights
PRD or spec review	Find gaps, edge cases, missing non-goals	Own the problem statement
Pre-mortem	Generate failure modes per stakeholder	Choose which risks get mitigation budget
Leadership update	Predict the questions you have not answered	Decide what you commit to

Where does AI cothinking break down?

It breaks first on context. The model reasons from the general case. It does not know your churn cohorts, your sales team's incentives, or the integration that quietly carries a large share of your revenue. You can paste context in, and you should, but the model still weighs it against everything it has seen elsewhere.

It also inflates thin evidence. Five customer interviews summarized fluently start to read like a market trend. Fluency is the danger here.

A third risk is speculation on my part, and I flag it as such. The P&G study showed AI pulling individuals toward balanced solutions. If an entire product org uses the same model with similar prompts, that same pull could flatten the range of ideas a team produces. Diversity of thinking across your team may need protecting deliberately, through varied roles and prompts, or through one person whose job is to argue before anyone opens a chat.

The last risk is for leaders of junior PMs. Judgment you never practice is judgment you cannot use to evaluate the model. The Microsoft study found that people often skipped critical thinking when they lacked the skills to inspect the AI's output. If your associates use AI for every first draft of a strategy, they may never build the instinct that tells them when the draft is wrong. The Learn quadrant is the remedy: give them low-stakes calls where the model explains its reasoning and they make the decision, so their expertise grows before the stakes do.

What changes for product teams over the next two years?

This is a forecast, with the assumption stated: models will keep getting better at breadth and synthesis faster than they get access to your organization's private context and politics.

If that holds, producing product artifacts gets cheaper every quarter, and decision quality becomes the visible differentiator between product leaders. I expect review culture to shift with it. A reasonable question in a 2028 roadmap review is "What was the strongest argument against this, and why did you reject it?" Teams that already keep decision records with the model's objections attached will answer it in seconds.

The takeaway

Use AI to argue with you, and keep the verdict. Before your next big call, place it on the map: if it lands in Spar, write your three sentences, run steps two and four, and keep the decision record. If it lands in the danger zone, book time with someone who knows the domain. Then compare the result with how you made the last call of the same size.