Testing for Expert Judgment

“The most courageous act is still to think for yourself. Aloud.” — Coco Chanel

Expert judgment is the ability to make predictions and avoid problems in a given domain.

How can you test the judgment of somebody on your team?

You can observe them over time, or you can accelerate the process by asking them about a topic they are passionate about.

In The First 90 Days: Proven Strategies for Getting Up to Speed Faster and Smarter, Michael Watkins writes about how to test a person’s capacity for expert judgment.

Make sure You are Assessing Judgment

Bright people can have lousy judgment.

Via The First 90 Days:

“Make sure you are assessing judgment and not technical competence or raw intelligence. 

Some very bright people have lousy judgment, and some people of average competence have extraordinary judgment. 

It is essential to be clear about the mix of knowledge and judgment you need from key people.”

Test for Making Predictions and Avoiding Problems

Expert judgment is about translating insights into effective action.

Via The First 90 Days:

“One way to assess judgment is to work with a person for an extended time and observe whether he or she is able to (1) make sound predictions and (2) develop good strategies for avoiding problems. 

Both abilities draw on an individual’s mental models, or ways of identifying the essential features and dynamics of emerging situations and translating those insights into effective action. 

This is what expert judgment is all about.”

How To Speed Up Testing for Expert Judgment

Test expert judgment by asking about a domain somebody is passionate about outside of work.

Via The First 90 Days:

“One way is to test people’s judgment in a domain in which feedback on their prediction abilities will emerge relatively quickly. … Ask individuals whose judgment you want to evaluate about a topic that they are passionate about outside work. 

It could be politics or cooking or baseball; it doesn’t matter. 

Challenge them to make predictions: ‘Who do you think is going to do better in the debate?’ ‘What does it take to bake a perfect soufflé?’ ‘Which team will win the game tonight?’ 

Press them to commit themselves — unwillingness to go out on a limb is a warning sign in itself.  Then probe why they think their predictions are correct. 

Does the rationale make sense? 

If possible, follow up to see what happens.  

What you are testing is a person’s capacity to exercise expert judgment in a particular domain.  Someone who has become an expert in a private domain is likely to have done so in his or her chose field of business too, given enough passion about it. 

However you do it, the key is to find ways, beyond just waiting to see how people will perform on the job, to probe for the hallmarks of expertise.”

Key Take Aways

Here are my key take aways:

• Expert judgment is the ability to make predictions and avoid problems.   When you delegate, you need to have somebody that you trust can make good predictions to avoid problems and make good decisions based on sound judgment.
• Technical expertise is not the same as sound judgment.  Having technical expertise in a domain is not the same as demonstrating sound judgment.
• Find ways to test for expert judgment.  Find ways to test for expert judgment beyond waiting to see how people perform on the job.
• Test people by asking them for predictions.  You can speed up the process by asking somebody about an area outside of work and seeing how well they make predictions.

Expert judgment is the stuff that great leaders and made from.