Teaching models to reason like we do

Suppose someone hands you a year of approved and rejected expense reports and asks you to write the reimbursement policy. There is no policy yet. There is only the stack.

You read. The first twenty reports tell you little. By the fiftieth, patterns start to hold. Meals under some amount go through. Meals above it need a note. Anything with alcohol draws a second look. A receipt photographed at an angle still counts, but a credit-card summary with no itemization does not. You did not begin with these rules. You pulled them out of the pile.

Then someone proposes a rule: reimburse any meal that has a receipt. You test it. It fails on the alcohol cases. It fails on the hotel folio that bundled a room-service charge with a spa charge. You know it fails because you can hold the proposed rule against specific reports you remember. At that point you are no longer finding a rule. You are judging one.

Those are two kinds of thinking, and they tend to run in that order.

Reading the pile to find the rule is induction, which reasons from specifics up to a generalization. Testing the proposed rule against a case is deduction, which reasons from a rule down to whether it holds. People move between the two without noticing the switch. A doctor induces a likely diagnosis from a cluster of symptoms, then deduces which single test would confirm or rule it out.

The reasoning asymmetry

Now hand the same stack of expense reports to a language model and ask it to propose the policy. It will do in seconds what took you an afternoon, and it will often do it well. Generalizing a rule from a set of examples is close to what these models are built for.

Ask the model to do the second half, and the picture changes. Hand it the proposed rule and a genuinely awkward case, the folio with the bundled spa charge, and its answer gets less reliable, especially where the case cuts against what it has seen most often.

When the two abilities are separated carefully, this gap shows up in the research. Models can be strong inductive reasoners and markedly weaker deductive ones, with the weakness sharpest on cases that run counter to their training distribution (Cheng et al., "Inductive or Deductive? Rethinking the Fundamental Reasoning Abilities of LLMs"). I have come to think of this as the reasoning asymmetry: a model may be a fast and capable proposer of rules and an unreliable judge of them.

The asymmetry is easy to miss, because an induced rule reads as convincing whether or not it is right. The model can mistake a quirk of the sample for a real rule. It can overfit to noise. It can produce two near-duplicate categories that a person would merge on sight. The output looks the same in each case, which is what makes a confident wrong rule more expensive than an obvious gap. The gap you notice. The confident error you build on.

Where each half belongs

If a model is strong at induction and weak at deduction, the design question follows from the asymmetry. Give the model the pass it is good at, and route the deductive pass to whoever can do it, which for now is usually a person who knows the domain.

This is the reverse of the common framing, in which the model does the work and a human approves it. Approval treats the person as a rubber stamp on the model's judgment. The division that fits the asymmetry treats the person as the one doing the harder reasoning the model cannot yet do. The model reads the pile and proposes. The person tests the proposal against knowledge that never appeared in the pile.

Two systems for analyzing call-center conversations were built on that division, and the shape held in both.

In one, a model reads a large sample of conversations and proposes a set of categories for why customers are getting in touch. The interface then hands a domain expert a set of deductive operations. Merge two categories the business treats as one. Split one that is doing too much. Rename a category into the company's own vocabulary. Delete one the business does not track. Each edit is the expert testing the model's induced rule against something the model could not see. The categories become numbers anyone will report on only after that pass.

In the other, a model reads a sample of one workflow's calls and proposes its core flow as a graph of phases and steps. The expert reorders phases, merges steps that are really one, and marks a branch that fires only for some accounts. The model induces. The person deduces. Only then does the system apply the confirmed flow across the full volume.

The detail that decides whether the check happens

The deductive pass only works if the person can actually do it, which makes the sample size a design decision rather than a model setting. It flips depending on who reads the examples.

When no person reads the individual cases, a large sample is correct, because representativeness comes from the raw size of a random draw. A system may sample thousands and let the person check only the resulting structure.

When a person has to audit every case in the loop, a small sample is correct. Thirty to fifty examples is a pile someone can read before they decide to trust the proposed pattern. Thousands is not. Asking for a deductive check while handing over a stack too large to read produces the appearance of verification and none of its substance.

The division as a checklist

The same structure applies to most features where a model finds a pattern in a mess of data. Compressed, it runs as a chain: read the pile, propose the rule, check the rule, apply the rule at scale. The first and last steps belong to the model and to deterministic code. The middle step, the check, belongs to the person.

Expanded, it is four questions worth answering before building.

1. What is the pile, and how large a sample can the model read to propose from?
2. What is the rule the model should propose, stated concretely enough to be wrong?
3. Who runs the deductive check, and is the evidence small enough for them to actually run it?
4. What happens to a check that fails, and where does that disagreement go?

Reading the symptoms backward

When an AI feature feels wrong, the asymmetry gives a way to find which half was misassigned.

If the output looks confident and specific and turns out wrong often, the model was probably handed a deductive call it cannot make, treated as a final authority rather than a proposer.

If the system is accurate but cannot produce anything until a person has read everything first, the person was probably handed the inductive pass, the slow reading the model should have done.

If a pattern looks clean in the demo and breaks on real volume, there may be no deductive pass at all, an induced rule applied at scale with nothing between induction and measurement.

If experts keep overriding in the same place, the model's induced rule likely diverges from theirs on a real branch, and the override is worth capturing rather than discarding, because it marks exactly where the two kinds of reasoning came apart.

Where this goes

The division is not permanent. As models get better at the deductive pass and at taking in context that today lives only in a person's head, the check can move toward the model, and the sample a person must read can grow. The design worth building now is the loop that lets the human's role shrink as the model earns it, which means capturing every failed check as the signal that would let the system need the person less.

For now, most of the work is smaller than it sounds. Give the model the half it is good at, give the person the half it is not, and make the person's half cheap enough that they actually do it.