QuIRKy Questions to Consider
"Deploying numbers skillfully is as important to communication as deploying verbs."
-Max Frankel, The New York Times Magazine
QuIRKy Questions
(Based on Neil Lutsky's 10 Foundational Quantitative Reasoning Questions.)
What do the numbers show?
Don't settle for weasel words like "some" or "many" when precise numbers are available. "Many" people don't suffer from AIDS in the US-over 1 million do.
But don't just settle for any number. Consider whether a particular figure is the right number. Interrogate numbers just as you interrogate texts.
When writing introductions or conclusions to papers, consider how you might use a few well-chosen numbers to establish a context or document the importance of the phenomenon discussed. This is a powerful use of numbers even in papers that are not inherently quantitative. For example, if you are writing a paper that discusses the nature and causes of psychogenic pain, it might help to tell the reader how common (or uncommon) the disorder really is.
How representative is that?
"For instance is no proof."
-Yiddish proverb
Stories are compelling. But anecdotes can also be misleading. Ask yourself whether a case is typical, and provide evidence to your reader assessing how representative your example is. When reporting averages, consider whether there are different subgroups or notable extreme scores that would be useful to report.
Compared to what?
Is $1 million a lot of money? If it's a salary figure, it puts you in the top 1/2 of 1 percent of US tax filers. But it's only 1/4,000,000 of the US federal budget.
Numbers (especially really big or really small numbers) need context. It often helps to compare them to other better-known figures.
How is the variable defined and measured?
Are you shocked to learn that the fraction of kids with autism has increased 12-fold in the last 20 years? You might not be when you find out that the official definition of autism has been broadened twice in that time and that the increase is matched by decreases in reports of other mental disorders.
What's the size of the effect?
Sometimes research uncovers effects which, while real, are very small. You won't be surprised to find out that the researchers often fail to emphasize the small effect size. So you need to ask yourself: What is the practical significance of the effect you are citing? Would an effect this large substantially change the world or would it be almost imperceptible?
Is the association really causal?
Because A and B occur together doesn't mean A causes B. B might cause A. Or C might cause both A and B. Take care when making causal claims. And press yourself to think of alternatives to your explanation for why a relationship might exist between two things.
Controlling for what?
One way we can discern whether an observed relationship is causal is to control for other possible causes. What was controlled for in the study you are reading? What factors were not controlled for but may be important?
What's the source of the numbers?
Consider whether the people reporting the figures are credible or if they might have a bias. Also note whether the number comes from a single study or is the result of an entire literature—that is, a collection of studies.
Want Even More QuIRKy Questions?
Read the full list of Professor of Psychology Neil Lutsky's 10 Foundational Quantitative Reasoning Questions.
Other Resources
Help Writing with Numbers
Drop by the Write Place and ask them for help using numbers to strengthen your arguments.
Help Finding Data
Stop by the Research/IT desk in Gould Library for help locating numbers to use in your work.
Help Working with Data Files and Statistical Tools
Talk to Paula Lackie in ITS for technological help in accessing and exploring more complex data sources.