We know more than we can explain

For a long time, expertise was associated with understanding. Increasingly, those two things are beginning to separate.

Ask an experienced physician how they recognized a diagnosis. Ask a firefighter why they entered one building instead of another. Ask a designer why a workflow immediately felt wrong. The answers are often surprisingly unsatisfying.

"It just didn't look right."

They know. Explaining how they know is considerably harder.

For decades, software largely reduced our dependence on this kind of intuition. Processes became explicit. Rules became workflows. Decisions became documented. Good systems were expected to behave predictably enough that understanding could be taught.

Intelligent systems are quietly reversing that direction.

---

Traditional abstraction hides implementation. We do not need to understand combustion to drive a car. We do not need to understand operating systems to send a message. AI introduces a different kind of abstraction. It increasingly hides reasoning — not simply what happened, but why it happened.

That difference is subtle. Its consequences are not.

When reasoning becomes opaque, expertise begins to change. People stop building complete mental models of the system beneath them. Instead, they build something softer. A sense of where the system performs well. Where it becomes unreliable. Which kinds of requests consistently succeed. When to intervene. When to let it continue.

Their knowledge becomes less explicit and more embodied. Less procedural. More intuitive.

---

That intuition should not be mistaken for superficiality. It is earned through repetition. Through experimentation. Through hundreds of small calibrations between expectation and outcome. The expertise is real. It is simply difficult to articulate.

In many cases, people know considerably more than they can explain.

One of the most unusual properties of modern AI is that it enables people to become remarkably effective without ever developing a complete explanation of the systems they are working with.

---

This creates an unusual challenge for design. For years, we assumed better products would require less learning. Increasingly, they may require a different kind of learning instead — not memorizing features, not mastering interfaces, but developing reliable intuition about systems whose internal reasoning remains only partially visible.

Explanations help. Documentation helps. But neither fully solves the problem.

The goal is not complete understanding. That may no longer be realistic. The goal is helping people calibrate their confidence. To recognize where the system is dependable. Where it is uncertain. And perhaps most importantly, where their own mental model no longer matches reality.

For decades, expertise meant knowing how a system worked. Intelligent systems suggest a quieter definition. Perhaps expertise is becoming less about explanation and more about calibration — less about possessing perfect knowledge, more about maintaining reliable judgment despite never having it.

The future of expertise may not belong to the people who understand intelligent systems best. It may belong to those who know how much confidence to place in what they cannot fully explain.

---

The observation that practitioners know more than they can tell draws on Michael Polanyi's concept of tacit knowledge, developed in The Tacit Dimension (1966). The firefighter and physician examples echo Gary Klein's research on naturalistic decision-making, documented in Sources of Power (1998). Both remain surprisingly relevant to how people are learning to work alongside intelligent systems.