Why the Obvious Fix Doesn't Work in AI Products

Direct answer first

When an AI feature misbehaves, most teams reach for the most visible lever: the prompt. They rewrite it, add a rule, tweak a temperature setting, and the bad behavior goes quiet, then resurfaces somewhere else a week later. That's not bad luck. In Donella Meadows' framework of leverage points, places to intervene in a system, the prompt sits near the bottom of the hierarchy with parameters, the weakest place to push. The interventions that actually change an AI product's behavior, its feedback loops, its information flows, the goal it optimizes for, sit higher up and are less obvious precisely because they're more powerful. The core PM skill isn't knowing more fixes. It's diagnosing which level a problem lives on before you spend a sprint pushing the wrong lever.

The distinction most AI teams skip

Meadows spent her career on one question: why do smart people, working hard with good intentions, push on a system and get nothing, or make it worse? Her answer, published in 1997 and expanded in Thinking in Systems, was that interventions fail because they target the wrong part of the system, not because the people are wrong. She ranked twelve types of intervention from weakest to strongest, and the ranking is deliberately counterintuitive: the levers we reach for first are usually the ones that move the system least.

A later academic reading by Abson and colleagues (2017) groups her twelve points into four tiers, which is a useful way to hold the whole list in your head: the parameters (the numbers), the feedbacks (the loops and flows that regulate the system), the design (structure, rules, who gets to self-organize), and the intent (the goal, and the paradigm underneath it). An AI product is a genuine complex system in this sense, full of feedback loops, delays, and emergent behavior. So the framework maps onto it almost directly, and the map explains why so many AI fixes don't hold.

The prompt is a parameter, which is why it barely moves the system

Parameters are the lowest-leverage intervention Meadows lists. They're the most clearly perceived, which is exactly why people reach for them first, and yet they rarely change behavior much. The temperature setting, the top-k value, the max tokens, a line added to the system prompt, these are the AI equivalent of adjusting the number on a thermostat. Sometimes that's the right call: if a summary runs too long, cutting the max-length parameter fixes it and it stays fixed.

But most persistent AI failures aren't parameter problems wearing a parameter costume. When you rewrite a prompt for the fourth time to stop a support agent from inventing refund policies, and it invents a different wrong policy the next week, the prompt was never the leverage point. You're turning the thermostat dial in a building with no insulation. The behavior you don't like is being produced somewhere higher in the system, and a parameter can only nudge the output, not the thing producing it.

Where the leverage actually is

Move up Meadows' hierarchy and the interventions get less obvious and more durable. Four levels matter most for AI products.

Delays in feedback (why your product oscillates). Information received too late causes over-correction and oscillation. In AI products this is everywhere. You ship a change, but your quality signal lags by days because it depends on human review or user complaints, so you overcorrect, then correct back, and quality swings instead of settling. The fix isn't a better prompt. It's shortening the delay, faster evals, quicker labeled feedback, so you can see the effect of one change before you stack three more on top of it.

Missing or broken feedback loops. A negative feedback loop keeps a system near its goal; a positive one amplifies whatever it's fed. Most AI products that "get worse over time" have a missing negative loop: nothing measures drift and pulls the system back. More dangerous are the accidental positive loops, a recommender trained on its own outputs, or a content tool whose generations quietly become next month's training data, amplifying their own biases with no brake. You can't prompt your way out of a runaway loop. You install the missing loop, or slow the positive one, which Meadows notes is usually more effective than speeding up a negative one.

Information flows (who gets to see what). Changing who has access to information is cheaper than changing structure, and often more powerful. A large share of AI failures are really information-flow failures in disguise: the model can't see the user's account state, the eval team can't see production failures, the PM can't see which retrieved chunks fed a bad answer. Routing the right information to the right part of the system, grounding the model in live context, surfacing failure traces to the people who fix them, moves behavior more than another prompt rewrite.

The goal (the highest lever you actually control). Changing the goal changes everything below it, parameters, loops, and information alike. This is the one AI teams miss most often, and there's now hard evidence for how strong the pull is. Anthropic's research on sycophancy found that when models are optimized against a preference model trained on human approval, they can drift toward telling people what they want to hear rather than what's true, because human raters, and the preference models built from them, measurably favor agreeable answers over correct ones. A separate 2024 study found RLHF raised human approval of answers by roughly 9% on a QA task while barely improving their actual correctness, training the model to be more convincing without being more right. No line in a system prompt beats the reward the model is actually chasing. If the metric you optimize and the behavior you want are in tension, the metric wins. The leverage is changing what you measure and reward, not adding instructions that argue with it.

The paradigm above the goal: that control equals quality

Meadows put paradigms, a system's shared unstated assumptions, above even goals. The paradigm quietly driving most mis-aimed AI fixes is the belief that more direct control produces better outcomes, that the way to get the behavior you want is to specify it harder: more explicit prompt instructions, more rules, more parameters clamped down. Leverage-points thinking says almost the opposite. The direct, high-control levers are the weakest ones, and the interventions that actually shape behavior are indirect: the loop you install, the information you route, the goal you set and then let the system organize itself around. (There's a related paradigm worth naming, that software is deterministic and a bug is a single reproducible defect, which I've written about in ADLC vs SDLC and Enterprise Design Thinking for AI.) Teams that shift from "specify it harder" to "shape the system that produces it" stop fighting their own products. That shift is the highest-leverage move available, and the hardest, because it changes how the team thinks before it changes any code.

How to use this without over-engineering it

This is a diagnostic tool, not a mandate to always chase the deepest lever. Meadows' own caution holds: higher leverage points are harder to change and easier to push in the wrong direction. Changing a goal mid-flight can destabilize a product that just needed a parameter tuned. The discipline is to locate the problem before you fix it:

• If a fix works and holds, it was a parameter problem. Fine. Move on.
• If you've "fixed" the same behavior three times and it keeps returning in new clothes, stop tuning parameters. The leverage is higher up: a missing loop, a delayed signal, or a goal in tension with what you want.
• If the whole team keeps being surprised by the same class of failure, you're likely looking at a paradigm problem. The shared assumption is wrong, not the implementation.

The framework doesn't hand you the answer. It stops you from spending a sprint on the most visible lever when the problem lives two tiers up. That diagnosis, ten minutes on a whiteboard, is cheaper than the sprint.

Conclusion

The obvious fix doesn't work because the obvious fix is almost always a parameter, and parameters are where systems resist change. Meadows mapped this in 1997 for economies and ecosystems, and it transfers cleanly to AI products because they share the same anatomy: loops, delays, goals, and the assumptions underneath them. Next time an AI feature misbehaves and the prompt rewrite doesn't hold, don't reach for a better prompt. Ask which level the problem actually lives on. The fix you're avoiding is usually the one that would last.