In short
- An expert leaving the room does not take a secret with them. What goes missing is the conditions under which their answer would have been different.
- Structured elicitation is the interview that turns a recommendation into criteria, lines and exceptions.
- A model is only repeatable if somebody without the expert's background can apply it and defend the result.
- Test it backwards against decisions you already made before anyone uses it on a customer.
- Models go stale quietly. Give one person the job of reviewing it, and write down what changed and why.
Why does expert judgment stop working when the expert leaves the room?
Not because the expert is wrong. But because the reasoning was never written down anywhere.
A founder or a senior engineer can say why an approach fits a customer, and be right. Then they step out for a budget meeting, and the same question comes up in a technical review nobody prepared for. The team repeats the conclusion. It cannot rebuild the reasoning behind it, so the conversation drops back to a feature list.
What sits underneath the judgment is pattern recognition, an assumption nobody said out loud, and a tradeoff that was never argued in public. The expert says it fits because of the data volume and the team size. But which volume, exactly? What does a bigger team change? What happens to someone sitting right between two options? None of that is written down anywhere, which is why none of it travels past the person who worked it out.
So the cost moves. Deals stall because nobody inside the account can check the recommendation, and delivery redoes discovery because it does not trust what sales passed along. A new hire spends a month absorbing something that could have been a page.
What is structured elicitation, and what does it actually ask?
It is the interview that gets the reasoning out. The move is away from "what do you recommend" and towards a much narrower set of questions, asked in order.
So frame the decision first, and frame it small. Not "how do we advise customers" but the one call the expert actually makes: this architecture or that one, this pricing shape, this sequence of work. Then take that decision apart. What does the expert check first? What comes after it? Then write the logic as something you could hand over: when this is true and that is not, do this. Then ask how sure they are, and where they are guessing.
None of this is an attempt to reduce a person's experience to a formula they would not recognise. It makes the judgment visible so it can be argued with. A rule that names what it looked at and where the line sits is a rule someone else can apply, and a rule a stakeholder can push back on. A recommendation with no stated conditions is just authority, and authority does not survive the room you are not in.
The question most elicitation sessions miss.Experts state the recommendation and stop. Close every session by asking what would make them give the opposite answer. That one question is what draws out the lines and the exceptions, and without it you have written down a preference rather than a rule.
What makes a model repeatable rather than just written down?
Three tests. The logic is visible, somebody other than the expert can apply it, and it holds up when a stakeholder pushes on it hard. Most attempts pass the first and fail the other two.
And visible logic means the path is on the page, not just the answer. This is where drawing it helps, because a decision tree or a matrix will not let you hide a step. Every node is something you check. Every branch is a condition. The end of the line is what you do.
But the criteria also have to be things a person without the expert's background can actually check. "Assess whether they are ready" is not a criterion, it is a re-run of the original problem. "Whether one named person owns this" is a criterion. So is anything somebody could look up in your own systems in a couple of minutes. If applying the model needs the expert's eye, you have written a description of the judgment rather than a replacement for it.
- Use criteria somebody can check, not qualities they have to sense
- Write down where each line sits, and say so in a number when the thing is countable
- Say what the model assumes, and where you are less sure of it
- Name the situations the model does not cover, instead of pretending it covers them
- Leave a way for the team to report back when it breaks
How do you test a model before your team uses it on a customer?
Run it backwards over decisions you already made. Pull a set of real past cases, enough to cover the awkward ones as well as the obvious ones, and put each through the model. Does it land where you actually landed? Where it disagrees, one of two things is true, and you need to know which one: the model is wrong, or the original decision was.
Then hand it to other experts, separately, and say nothing about what you expect. One builder, two independent readers. Where they disagree about a line, that is not a problem with the model. That is the model finding a real disagreement your company had been carrying quietly, and writing it down is worth more than picking a side.
Finally run it forward on something live, and watch who uses it. Can a team produce a recommendation from it that they are willing to defend in front of the customer? Did using it show up a question nobody had thought to ask? Skipping this step is how a model that reads well in a workshop turns into something the field quietly stops using.
Which visual format fits which kind of decision?
So the format is not decoration. A model that lives in a document does not survive a sales cycle, because nobody opens a document in a meeting. A drawing gets put on the table.
Decision trees suit logic that runs in order, where each answer settles the next question. They are hard to misread, and every node gives a stakeholder somewhere specific to object, which is useful rather than annoying. Matrices suit a judgment made on two or three things at once, because a person can point at where they sit and see what follows. A sequence works when the recommendation is staged and the order carries the reasoning.
Scroll the table sideways to see every column.
| Format | Suits | What it gives you | Where it breaks |
|---|---|---|---|
| Decision tree | Logic that runs in order, one answer deciding the next question | Hard to misread; every node is somewhere specific to push back | Gets unwieldy fast once branches multiply or the inputs are continuous |
| Matrix | A judgment made on two or three things at once | A person can point at where they sit and read off what follows | Needs discrete categories; falls apart past three dimensions |
| Sequence | Staged recommendations where the order carries the reasoning | Shows what happens when, and what depends on what | Needs more explaining; wasted on a single decision |
| Flowchart | Genuinely branching logic with loops and several routes through | Holds real complexity without lying about it | Overwhelms a reader quickly; needs careful labelling and testing |
Whatever you pick, it has to clear two bars. Simple enough to talk through in a meeting, and complete enough to decide from without the expert. If somebody has to ring the expert to interpret the drawing, the drawing is not finished.
One example, and the point is what it left behind. At an industrial equipment manufacturer the whole sales process still ran through the owner. So the judgment got separated into three decisions.
Opening → Setting → Closing
During Opening somebody contacted the enquiry and decided whether another conversation made sense. During Setting the team worked out what the buyer needed and whether the fit was there, and only qualified buyers reached Closing. Difficult or unusual calls could still need the owner. But the team could now see what to handle before they brought him in, which is the whole test.
At Strategem this is the work. We do it for an expert whose exceptions keep coming back to them, for a founder the team keeps pulling into the same sales moment, and for a champion who has to carry the argument internally.
How do you stop a working model from going stale?
Nothing about a model announces that it has stopped being true. The market moves, the customers change, and the drawing keeps looking just as authoritative as the day it was right.
So put a review in the calendar rather than waiting to notice. Once a quarter, or twice a year, pull a sample of the decisions people actually made with it. Did the team believe the output? Where did it miss? What has changed outside that the model has not caught up with? And give it one owner, because a model that everybody is responsible for maintaining is a model nobody actually maintains.
Then write down what you changed and why. If a line moved because the old one turned out to be wrong, record that, and if you added a criterion because the market shifted, say which shift it was. Without that trail the model slowly turns back into something only its author understands, which is the problem you started with.
Where do these attempts usually fail?
The failures repeat, and most of them are recognisable early.
- Too simple. The model reads cleanly and then breaks on the first awkward customer. Say what it assumes and where it does not apply, rather than smoothing that over.
- Assumptions nobody said. The expert skipped something because it was obvious to them. Have someone outside the domain read it and ask why at every step.
- Criteria nobody can check. "Assess the culture" is not a criterion. If you cannot see it or count it, you cannot hand it over.
- Papered-over disagreement. Two experts disagree, so somebody picks one and moves on. Write the disagreement down instead; the model is stronger for admitting where opinion splits.
- No testing. Backwards over old cases, forwards on a live one. Both, before the field sees it.
- Treating it as finished. A model that never changes is a model on its way to being wrong.