Effect Size: Evaluating the efficacy of an intervention (e.g., medical treatment) or phenomenon – Dale Mortimer, M.D.
Effect Size
Effect size is a standardized, scale-free quantitative measure of the relative size of the effect of an intervention or strength of a phenomenon. Measuring effect size is a simple way of quantifying the difference between two groups, and it has many advantages over the use of tests of statistical significance alone. Effect size is not the same as statistical significance: statistical significance tells how likely it is that a result is due to chance, and effect size tells you how important the result is. Effect size emphasizes the size of the difference. Alas, few textbooks, research methods courses, computer packages or “medical experts” even address the concept of effect size because, with few exceptions, while the treatment intervention may be statistically more significant than placebo (– and, thus, can be touted as "evidence–based"), the reality is that effect sizes for many medical treatments are abysmally low (- thus, in Dr. Mortimer's humble opinion, those who advertise as practicing “evidence–based medicine only” should be fleered). Below are effect sizes for selected interventions/treatments
Proposed Intervention or Treatment and its Effect size
Placebo or ineffective treatment (e.g., the odds of correctly guessing the flip of a coin): 0.00
School–based substance abuse education to prevent substance use [e.g., Drug Abuse Resistance Education (DARE)]: 0.00 [i.e., this intervention was found to be ineffective.]
National Youth Anti–Drug Media Campaign to reduce uptake of cannabis/ illegal drugs by children and adolescents: 0.00 [i.e., this intervention was found to be completely ineffective.]
Raising the prices of cigarettes and alcohol to limit adolescent uptake of these products: 0.00 [i.e., this intervention was found to be completely ineffective.]
Additive–free diet’s effect on children’s hyperactivity (now known as ADHD): 0.02 [i.e., this intervention was found to be ineffective in almost all ADHD children]
Individualized school instruction: 0.10 [– Well, everyone does a little bit better with individualized tutoring, don't they!]
Aspirin for the prevention of cardiovascular disease (e.g., heart attacks and strokes): 0.12 [i.e., the prophylactic benefit of aspirin for the prevention of cardiovascular disease is very low.]
Statins for the prevention of cardiovascular events (heart attacks and strokes): 0.15 [i.e., the prophylactic benefit of aspirin for the prevention of cardiovascular disease is very low.]
Substance abuse–prevention program for adolescents delivered over the Internet: 0.17 [i.e., this intervention is close to completely ineffective]
Treatment programs for juvenile delinquents (adjudicated youth): 0.17 [i.e., this intervention is close to completely ineffective]
The difference in height between 15 year old and 16 years old American girls: 0.20 [i.e., it is quite difficult to distinguish the age of a 15 year old girl from a 16 year old girl based solely on their heights.]
[Anything with an effect size of 0.20 or less is considered significant but trivial (or low) – that is, if two groups' mean averages don't differ by at least 0.2 standard deviations, then the difference is trivial, even if it is statistically significant.]
––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
Atypical antipsychotic medications (e.g., olanzapine, quetiapine, risperidone, sertindole) for the treatment of schizophrenia: 0.25
Cholinesterase inhibitors (donepezil, galantamine, rivastigmine) for dementia treatment: 0.26
Alpha–2 agonists (clonidine, guanfacine) for the management of tics: 0.31 [– well, OK, but haloperidol and risperidone are usually much more effective for managing tics in any particular patient.]
Practice test–taking in schools: 0.32
Higher vs lower levels of C–reactive protein in those with bipolar disorder: 0.39
Antidepressant (e.g., fluoxetine, sertraline, paroxetine) use in the treatment of generalized anxiety disorder: 0.39
Therapy for test–anxiety in anxious students: 0.42
Haloperidol and atypical antipsychotic medications (e.g., olanzapine, quetiapine, risperidone, sertindole) for the treatment of acute mania in bipolar disorder: 0.44
Mainstreaming vs special education (for primary age, disabled students): 0.44
Antipsychotic/antidepressant Lurasidone (Latuda) effect size in children and teens with bipolar I depression (compare with the effect size of coenzyme Q10 below): 0.45
Height difference between 14 year old and 18 year–old American girls: 0.50 [i.e., a four–year age difference in girls' age is correlated with a difference in height that is usually obvious.]
Effect size of strong serotonin–reuptake inhibitors (i.e., fluoxetine, sertraline, paroxetine, citalopram, fluvoxamine, escitalopram) in the treatment of either obsessive compulsive disorder or major depression: 0.50
Relaxation training used for treating medical symptoms: 0.52
High school–based substance abuse education focused on cannabis use: 0.58
Neuroleptics (anti–psychotic medications such as risperidone) for the management of tics: 0.58
Non–stimulants (e.g., desipramine, nortriptyline, atomoxetine) used in the treatment of attention–deficit/hyperactivity disorder (ADHD): 0.62
Targeted interventions for at–risk students (students at risk of failing school): 0.63
Antidepressants to prevent major depressive disorder relapse in adults: 0.64
––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––––
Anything with an effect size of 0.80 or higher is considered Highly Significant (– that is, the effect is so large or consistent enough that you may be able to see it “with the naked eye.”)
The difference between heights of 13– and 18–year old girls in the U.S: 0.80
The difference in IQ between holders of the Ph.D. degree and typical college freshmen: 0.80
Giving students an extra hour of time to revise their exam responses: 0.86
Efficacy of anti–oxidant coenzyme Q10 for bipolar depression after 8 weeks of treatment: 0.87
Immediate–release stimulants (e.g., dextroamphetamine, methylphenidate) used in the treatment of attention–deficit/hyperactivity disorder (ADHD): 0.91
Antipsychotic medications used to prevent relapse in schizophrenia: 0.92
Long–acting stimulants (e.g., Vyvanse, Mydayis, Concerta, Focalin XR) used in the treatment of ADHD: 0.95
One standard deviation difference between measured groups: 1.00 [– per the definition of effect size]
Lithium for the treatment of bipolar disorder in children and teens: 1.06
Proton pump inhibitors (e.g., omeprazole) used to treat reflux esophagitis (GERD): 1.39
The difference in height between men and women in Britain (e.g., this difference is so large that you can see the difference with the naked eye): 1.72
Two standard deviations between measured groups: 2.00 [– by definition of effect size]
Interferon for the treatment of hepatitis C: 2.27 [– thus, Interferon is one of the most effective treatment interventions in all of medicine]