The Wizard of Oz Treatment

Will Thalheimer got suspicious and decided to check it out. His blog entry People remember 10%, 20%…Oh Really? is a case study of the need for appropriate skepticism and due diligence, especially when we borrow from others. The case in point is the ‘common knowledge’ of learning retention related to the media used for learning.

we may not be able to trust the information that floats around our industry. It tells us that even our most reputable people and organizations may require the Wizard-of-Oz treatment—we may need to look behind the curtain to verify their claims.

The fact that our field is so easily swayed by the mildest whiffs of evidence suggests that we don’t have sufficient mechanisms in place to improve what we do. Because we’re not able or willing to provide due diligence on evidence-based claims, we’re unable to create feedback loops to push the field more forcefully toward continuing improvement.

The phenomena is similar to the social game where a statement is whispered to one person who then whispers that to another. By the time the statement gets all the way around the room, it is likely to be very different from the original. This is entertaining as a game but when the same thing happens to research results it can be a problem.

Will points out a number of warning signs that should alert users of information to proceed with caution. He also suggests that those who toss out information, especially information that is a research product, know the source and understand its quality.

Comments are closed.