Will drinking Red Bull and smoking cigars boost our productivity at work?
Will sleep deprivation increase the number of mistakes we make?
This post addresses these questions, as well as how math-myopia affects love for metrics and statistics about sports, dieting, work injuries and so forth.
If you read German, check out our series on political campaigning and the usefulness of polling (US presidential election).
[su_box title=”Table 1: Keep the numeracy problem in mind” box_color=”#86bac5″ title_color=”#ffffff” radius=”5″ width=”px 700″ ]
1.1 Making a good guess: Avoid base rate neglect
When we combine two pieces of information, we tend to ignore one of them completely. This phenomenon is called “base rate neglect”. For example, the base rate tells us how many people are affected by bowel cancer (6 out of 100), or how many have a fatal injury at work (i.e. 142 deaths per year in the UK – a rate of 0.46 deaths per 100,000 workers).
Knowing the base rate helps put things in perspective.
1.2 Our culture makes it acceptable to say, “I do not do numbers.”
Imagine a study that presents a test that is 75 percent accurate. In 25 percent of the cases where the test predicts a self-reported injury at work will happen, it does not. This is called a false positive.
What is the chance that the person has a work-related accident? Intuitively we might say that in 75 percent of cases a fatal accident will occur. However, the correct answer is, ‘we do not know’ – unless we have the base rate.
To illustrate, if we test 100 people and 4 come out positive, 3 were rightfully identified to likely have a work-related accident and 1 was wrongfully identified. But wait! Of the 96 others, 24 (= 25 percent) will have a false positive. This means we predict a non-fatal work injury, but they will not have one.
Unfortunately, in a culture where Prince Harry can publicly state that he may not have the math skills to be an air ambulance helicopter pilot, he is likely to ignore the base rate…
The base rate is a good way to start if we want to forecast something or put test results in perspective (see Tables 2 and 3 below).
Below we illustrate this a bit more with an example based on a May 2015 Financial Times article, which nicely illustrates how things can be misconstrued by journalists.
To reduce this risk, we must go to the trouble and check the numbers.
[su_custom_gallery source=”media: 1986″ limit=”7″ link=”image” target=”blank” width=”531px” height=”130px”]
Great headline. Unfortunately, the FT journalist fails to refer us readers to the original study from which she got these numbers.
I left a comment, asking author and Employment Correspondent Sarah O’Connor for help.
[su_custom_gallery source=”media: 1984″ limit=”7″ link=”image” target=”blank” width=”441px” height=”234px”]
My comment (see above image) did not get published, nor did I get an answer from the journalist regarding my query.
So as a paying subscriber, should I trust these claims? Might it be wiser to go and check?
You guessed, I continued digging myself. An interesting journey that took me 28 minutes…
I found an article from the Daily Mail (see image below).
But it referred to an earlier article by Mail staff referring to a study by Uppsala University (Sweden) researchers.
References? None whatsoever!
[su_custom_gallery source=”media: 1987″ limit=”7″ link=”image” target=”blank” width=”492px” height=”368px”]
So where did the Financial Times’ (FT) journalist find the information if not from the Daily Mail?
The Huffington Post managed a link to the FT article from which it had copied. In other words, to avoid copyright infringement the journalist had done a fast re-write. The content was the same as in the FT article using different wording.
[su_custom_gallery source=”media: 1988″ limit=”7″ link=”image” target=”blank” width=”471px” height=”337px”]
So the Huffington Post failed to add substance to what was posted in the FT. What now?
You might suggest as a good next step to go and check whether the sponsor of a study might offer the full report. That is what we did.
Unfortunately, the sponsor’s website did not make it easy – it failed basic usability requirements. After some digging we found something, but it did not link to the original or complete report either.
You got that right, the sponsor did not provide the full report. Just a bit of information and nothing more. Real bummer.
[su_custom_gallery source=”media: 1985″ limit=”7″ link=”image” target=”blank” width=”490px” height=”349px”]
Maybe a search with different keywords could help? Read on and find out. Incidentally, why not subscribe to this blog’s newsletter right now?
[su_box title=”Table 2: 6 things we must do to make sure the numbers add up” box_color=”#86bac5″ title_color=”#ffffff” radius=”5″ width=”px 700″ ]
2.1 Go search for the original (see organic search data in image below)
This got me the link to the Rand Corporation. Here something was written up about the study. But once you have the original study, read it carefully, including the method section.
2.2 Read the method and result section carefully
To illustrate, I recently read:
Murray, Sarah (2016-02-26). Frustrated US workers go it alone. Freelancing. Work is becoming more flexible but less secure. Financial Times, FT Executive Appointments. Employment Global Best Practice. Retrieved, Feb. 27, 1026 from http://www.ft.com/intl/cms/s/0/6030a0dc-d0b2-11e5-92a1-c5e23ef99c77.html
I went to the original study, Freelancing, and found another study where the authors do not provide a methodology section.
After reading such a study, make sure the following four questions are answered and if not, don’t make decisions based on such work:
2.2.a — How was the sample selected? No information given in the report!
2.2.b –What kind of survey was used? No information given!
2.2.c — Were participants interviewed using an online survey? No information!
2.2.d — Was a combination of landline and cellphone random digit dialing samples used to get responses through interviews? No information!
If the report does not provide information regarding questions 2.2.a to 2.2.d, should I trust it?
Put differently, as a shareholder or tax assessor, would you trust the company’s financial statements if answers to such auditor questions were missing?
Certainly not, so why should you trust such numbers for an opinion poll? I rest my case!
[su_custom_gallery source=”media: 1999″ limit=”7″ link=”image” target=”blank” width=”413px” height=”498px”]
A short description is offered and at the bottom a link to the report. Another page opens with another description about the study. Eureka – I can finally download the report.
[su_custom_gallery source=”media: 1989″ limit=”7″ link=”image” target=”blank” width=”489px” height=”268px”]
The report itself is very interesting. On page 11 it introduces the reader to the concept of abseenteeism and presenteeism.
– Absenteeism refers to the measure of days absent from work
– Presenteeism refers to the measure of reduced productivity while at work (e.g., due to headache, flu, etc.).
On page 12, it goes on to say, “The instrument consists of six questions with a recall time frame of seven days. The questions ask whether the respondent is employed; the number of hours missed from work; the number of hours actually worked; and the degree to which the respondent feels that a health problem has affected productivity while at work and affected their ability to do daily activities other than work. WPAI-GH outcomes are expressed as impairment percentages, where higher percentages indicate greater impairment and lower productivity. We use the following three work-related impairment percentages calculated on the basis of the WPAI-GH scale
– Per cent work time missed due to ill-health (absenteeism),
– Per cent impairment while working due to ill-health (presenteeism),
– Per cent overall work impairment due to ill-health (absenteeism and presenteeism).”
Hafner, Marco; van Stolk, Christian; Saunders, Catherine, L; Krapels, Jochim; Baruch, Ben (May 22, 2015). Health, wellbeing and productivity in the workplace. A Britain’s Healthiest Company summary report. Retrieved, May 31, 2015 from http://www.rand.org/content/dam/rand/pubs/research_reports/RR1000/RR1084/RAND_RR1084.pdf
What is the problem with the Rand study?
The survey depends on how well subjects recollect facts from last week. But do seven days in a person’s year accurately reflect the status of their health? Additionally, does it make a difference if we collected these data in July, October, December or February of the year we studied?
Finally, large companies are over-represented in this sample. Moreover, companies with under 50 employees – over 70 percent of British firms – could not participate.
So is this a great study? It is very interesting, but the journalists’ interpretation of these data far exceeds what the authors infer from their own data.
By the way, there is research that is far better suited than the above to learning how sleep deprivation can affect job performance or studying math.
[su_custom_gallery source=”media: 1993″ limit=”7″ link=”image” target=”blank” width=”445px” height=”179px”]
Uppsala University to the rescue
Olga E. Titova, et al., (2014) Associations of self-reported sleep disturbance and duration with academic failure in community-dwelling Swedish adolescents. Sleep Medicine doi: 10.1016/j.sleep.2014.09.004 Retrieved May 31, 2015 from http://www.researchgate.net/…… (click on citation to get study link since it is too long to post here).
The study included 20,000 adolescents aged 12 to 19. This longitudinal study was conducted from 2005 to 2011. About 30 percent of participants reported regular sleep problems.
The study found that if you have less than seven hours of sleep, data indicate an increased risk of failure in school.
The group also found in a previous study that going without a night of sleep increased toxic substances in the brain. Possible increased risk of Alzheimer’s, Parkinson’s and multiple sclerosis was reported. Other prior research shows how the brain uses sleep to cleanse itself.
2015-07-20 – One night of sleep loss can alter clock genes in your tissues
2015-07-13 – Sleep loss makes memories less accessible in stressful situations
Three studies show that teens should decrease screen time before going to bed
Bottom line: Show me the data…
[su_box title=”Table 3: Does the story meet 2 critical benchmarks?” box_color=”#86bac5″ title_color=”#ffffff” radius=”5″ width=”px 700″ ]
1. Does the article provide the reader with a link to the original study discussed?
Every journalist does some research before writing their story. If the original material is available online, why not reference it? Saves your reader time and gives them a chance to read up on this interesting topic.
Hence, the printed article should provide a link to the original source(s). At the very least, it should refer to the online version of the article where links to the original sources are provided.
2. Does the study report provide the reader with a method section?
Explain succinctly how you did the study, such as:
“The analysis in this report is based on telephone interviews conducted February 18-21, 2016 among a national sample of 1,002 adults, 18 years of age or older, living in the continental United States (501 respondents were interviewed on a landline telephone, and 501 were interviewed on a cellphone, including 312 who had no landline telephone)….
A combination of landline and cellphone random digit dialing samples were used… Interviews were conducted in English and Spanish. Respondents in the landline sample were selected by randomly asking for the youngest adult male or female currently at home. Interviews in the cell sample were conducted with the person who answered the phone, if that person was an adult 18 years of age or older.”
Great read: PEW – information about our survey methodology
For those like Prince Harry, who claims, “I can’t do maths,” the above paragraph can be a lifesaver.
If you read an article like the one for this post, better check the number to see if the headline by the journalist can be justified from the study’s results. Very likely it cannot, so putting decreasing amounts into content seems to only viable strategy left.
And to answer our question in the title: No study shows drinking #RedBull boosts job performance.
However, it does increase your daily sugar intake significantly, which is probably not a good thing.
Join the conversation
- Do you have an example of how mathematics phobia is affecting basic mastery of mathematics skills?
- Do you have a good example of a sponsored study that addresses some of the issues outlined here?
- How do you make good guesses about things that affect your decision-making (i.e. invest my money here or there…)?
Of course, I will answer you in the comments. Guaranteed.
Can we trust these numbers?
Interesting reads point out that trust is learned more than inherited. Trust is socially received and transmitted.