DFY: Don’t fool yourself, and you are the easiest person to fool.

 

Drag the words into the correct boxes

 

lack of preregistration1 of 11 draggables.
precise numbers2 of 11 draggables.
statistical inference3 of 11 draggables.
translating a measure to a lower level4 of 11 draggables.
incomplete reporting of a study5 of 11 draggables.
lack of random assignment6 of 11 draggables.
cherry picked results7 of 11 draggables.
lack of replication8 of 11 draggables.
selection of studies9 of 11 draggables.
calculations and results10 of 11 draggables.
a measurement11 of 11 draggables.
DFY: Don’t let
fool you: It relies on a statistical model. Our model includes the assumption of random sampling, which is often not achievable. If it isn’t, use judgment before applying a conclusion to a particular population.

DFY: Don’t let
fool you. Only experimental research, based on careful random assignment, can justify a causal conclusion. Be cautious about any causal conclusion you read—is it based on experimental research?

DFY: Don’t let
fool you. The measure is not the same as the underlying construct. Consider how well the measure operationalizes the construct. Consider its reliability and validity.

DFY: Don’t let
fool you. Think about what the numbers represent: Does it make sense to manipulate them? Don’t average the flavors!

DFY: Don’t let
fool you. Such a translation loses information, and should be avoided if possible. Prefer higher levels, where that’s possible. In other words, prefer ratio to interval to ordinal to nominal.

DFY: Don’t let
fool you into believing they must mean something. When using a mean or other statistic that requires interval scaling, consider whether the equal-interval assumption is justified.

DFY: Don’t let
fool you. Usually we need replications before we have confidence in a research finding.

DFY: Don’t let
fool you. All relevant studies must be available for us to have the full story. If publication bias means that some relevant studies are unavailable, even meta-analysis may give a biased result.

DFY: Don’t let
fool you. Fully detailed reporting is needed, to avoid being misled.

DFY: Don’t let
fool you. Don’t be fooled by seeing a face in the clouds. Distinguish carefully between planned and exploratory analysis.

DFY: Don’t let
fool you. Without preregistration, aspects of the study or its analysis may be missing from what’s reported.