layout: true --- name: xaringan-title class: inverse, left, middle .pull-left[ # .center[Common misconceptions in statistics] ## .center[Adrian Barnett, QUT] ### .center[23 June 2022] [<svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M459.37 151.716c.325 4.548.325 9.097.325 13.645 0 138.72-105.583 298.558-298.558 298.558-59.452 0-114.68-17.219-161.137-47.106 8.447.974 16.568 1.299 25.34 1.299 49.055 0 94.213-16.568 130.274-44.832-46.132-.975-84.792-31.188-98.112-72.772 6.498.974 12.995 1.624 19.818 1.624 9.421 0 18.843-1.3 27.614-3.573-48.081-9.747-84.143-51.98-84.143-102.985v-1.299c13.969 7.797 30.214 12.67 47.431 13.319-28.264-18.843-46.781-51.005-46.781-87.391 0-19.492 5.197-37.36 14.294-52.954 51.655 63.675 129.3 105.258 216.365 109.807-1.624-7.797-2.599-15.918-2.599-24.04 0-57.828 46.782-104.934 104.934-104.934 30.213 0 57.502 12.67 76.67 33.137 23.715-4.548 46.456-13.32 66.599-25.34-7.798 24.366-24.366 44.833-46.132 57.827 21.117-2.273 41.584-8.122 60.426-16.243-14.292 20.791-32.161 39.308-52.628 54.253z"/></svg> @aidybarnett](http://twitter.com/aidybarnett) [<svg aria-hidden="true" role="img" viewBox="0 0 496 512" style="height:1em;width:0.97em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M165.9 397.4c0 2-2.3 3.6-5.2 3.6-3.3.3-5.6-1.3-5.6-3.6 0-2 2.3-3.6 5.2-3.6 3-.3 5.6 1.3 5.6 3.6zm-31.1-4.5c-.7 2 1.3 4.3 4.3 4.9 2.6 1 5.6 0 6.2-2s-1.3-4.3-4.3-5.2c-2.6-.7-5.5.3-6.2 2.3zm44.2-1.7c-2.9.7-4.9 2.6-4.6 4.9.3 2 2.9 3.3 5.9 2.6 2.9-.7 4.9-2.6 4.6-4.6-.3-1.9-3-3.2-5.9-2.9zM244.8 8C106.1 8 0 113.3 0 252c0 110.9 69.8 205.8 169.5 239.2 12.8 2.3 17.3-5.6 17.3-12.1 0-6.2-.3-40.4-.3-61.4 0 0-70 15-84.7-29.8 0 0-11.4-29.1-27.8-36.6 0 0-22.9-15.7 1.6-15.4 0 0 24.9 2 38.6 25.8 21.9 38.6 58.6 27.5 72.9 20.9 2.3-16 8.8-27.1 16-33.7-55.9-6.2-112.3-14.3-112.3-110.5 0-27.5 7.6-41.3 23.6-58.9-2.6-6.5-11.1-33.3 2.6-67.9 20.9-6.5 69 27 69 27 20-5.6 41.5-8.5 62.8-8.5s42.8 2.9 62.8 8.5c0 0 48.1-33.6 69-27 13.7 34.7 5.2 61.4 2.6 67.9 16 17.7 25.8 31.5 25.8 58.9 0 96.5-58.9 104.2-114.8 110.5 9.2 7.9 17 22.9 17 46.4 0 33.7-.3 75.4-.3 83.6 0 6.5 4.6 14.4 17.3 12.1C428.2 457.8 496 362.9 496 252 496 113.3 383.5 8 244.8 8zM97.2 352.9c-1.3 1-1 3.3.7 5.2 1.6 1.6 3.9 2.3 5.2 1 1.3-1 1-3.3-.7-5.2-1.6-1.6-3.9-2.3-5.2-1zm-10.8-8.1c-.7 1.3.3 2.9 2.3 3.9 1.6 1 3.6.7 4.3-.7.7-1.3-.3-2.9-2.3-3.9-2-.6-3.6-.3-4.3.7zm32.4 35.6c-1.6 1.3-1 4.3 1.3 6.2 2.3 2.3 5.2 2.6 6.5 1 1.3-1.3.7-4.3-1.3-6.2-2.2-2.3-5.2-2.6-6.5-1zm-11.4-14.7c-1.6 1-1.6 3.6 0 5.9 1.6 2.3 4.3 3.3 5.6 2.3 1.6-1.3 1.6-3.9 0-6.2-1.4-2.3-4-3.3-5.6-2z"/></svg> @agbarnett](http://github.com/agbarnett) [<svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M172.2 226.8c-14.6-2.9-28.2 8.9-28.2 23.8V301c0 10.2 7.1 18.4 16.7 22 18.2 6.8 31.3 24.4 31.3 45 0 26.5-21.5 48-48 48s-48-21.5-48-48V120c0-13.3-10.7-24-24-24H24c-13.3 0-24 10.7-24 24v248c0 89.5 82.1 160.2 175 140.7 54.4-11.4 98.3-55.4 109.7-109.7 17.4-82.9-37-157.2-112.5-172.2zM209 0c-9.2-.5-17 6.8-17 16v31.6c0 8.5 6.6 15.5 15 15.9 129.4 7 233.4 112 240.9 241.5.5 8.4 7.5 15 15.9 15h32.1c9.2 0 16.5-7.8 16-17C503.4 139.8 372.2 8.6 209 0zm.3 96c-9.3-.7-17.3 6.7-17.3 16.1v32.1c0 8.4 6.5 15.3 14.8 15.9 76.8 6.3 138 68.2 144.9 145.2.8 8.3 7.6 14.7 15.9 14.7h32.2c9.3 0 16.8-8 16.1-17.3-8.4-110.1-96.5-198.2-206.6-206.7z"/></svg> Median Watch](https://medianwatch.netlify.app) [<svg aria-hidden="true" role="img" viewBox="0 0 512 512" style="height:1em;width:1em;vertical-align:-0.125em;margin-left:auto;margin-right:auto;font-size:inherit;fill:currentColor;overflow:visible;position:relative;"><path d="M476 3.2L12.5 270.6c-18.1 10.4-15.8 35.6 2.2 43.2L121 358.4l287.3-253.2c5.5-4.9 13.3 2.6 8.6 8.3L176 407v80.5c0 23.6 28.5 32.9 42.5 15.8L282 426l124.6 52.2c14.2 6 30.4-2.9 33-18.2l72-432C515 7.8 493.3-6.8 476 3.2z"/></svg> a.barnett@qut.edu.au](mailto:a.barnett@qut.edu.au) ] .pull-right[  ] --- ## Pie charts are terrible <img src="figures/pie_charts.jfif" width="62%" style="display: block; margin: auto;" /> From https://twitter.com/MaxCRoser/status/857389434756505600 --- ## Truly terrible example <img src="figures/pie_terrible.jpg" width="80%" style="display: block; margin: auto;" /> --- class:inverse ## 3D charts are even worse <img src="figures/3d_pie_1.png" width="80%" style="display: block; margin: auto;" /> --- class:inverse ## 3D charts are even worse <img src="figures/3d_pie_2.png" width="80%" style="display: block; margin: auto;" /> --- ## Bar charts should not be used for continuous data <img src="figures/pbio.1002128.g001.png" width="80%" style="display: block; margin: auto;" /> Weissgerber et al, DOI: 10.1371/journal.pbio.1002128 --- ## Outliers should not be deleted <img src="slides_files/figure-html/unnamed-chunk-6-1.png" style="display: block; margin: auto;" /> --- ## Outliers should not be deleted .pull-left[ <img src="slides_files/figure-html/unnamed-chunk-7-1.png" style="display: block; margin: auto;" /> ] .pull-right[ <img src="figures/rothwell.jpg" width="88%" style="display: block; margin: auto;" /> (from BBC) ] Data from [Spyrou et al](https://www.nature.com/articles/s41586-022-04800-3#MOESM1) --- class:inverse <!--- two columns ---> .pull-left[ ## The most exciting phrase to hear in science, the one that heralds new discoveries, is not “Eureka” but “That’s funny...” Isaac Asimov (1920 - 1992) ] .pull-right[ <img src="https://upload.wikimedia.org/wikipedia/commons/archive/3/34/20100906191953%21Isaac.Asimov01.jpg" width="320" height="464"/> ###### Image from Wikipedia, Phillip Leonian ] --- # Too many numbers ## "The number of deaths in the last ten years was 1,456,231.'' ## "The percentage working was 44.23% and the percentage retired was 12.14%.'' --- # Too many numbers ## "The number of deaths in the last ten years was 1.4 million.'' ## "The percentage working was 44% and the percentage retired was 12%.'' Guidelines here: DOI: [10.1136/archdischild-2014-307149](https://adc.bmj.com/content/100/7/608) --- class: inverse .pull-left[ ## “Lack of mathematical education does not become more evident than by excessive precision in numerical calculation" Gauss (1777.241 - 1855.145) ] .pull-right[ <img src="https://upload.wikimedia.org/wikipedia/commons/e/ec/Carl_Friedrich_Gauss_1840_by_Jensen.jpg" width="320" height="464"/> ###### Image from Wikipedia ] --- # Continuous data do not have to be normal <img src="slides_files/figure-html/unnamed-chunk-9-1.png" style="display: block; margin: auto;" /> --- # Continuous data do not have to be normal <img src="slides_files/figure-html/unnamed-chunk-10-1.png" style="display: block; margin: auto;" /> --- # Continuous data do not have to be normal <img src="slides_files/figure-html/unnamed-chunk-11-1.png" style="display: block; margin: auto;" /> --- class:inverse # A common misconception "It is widely but incorrectly believed that the t-test and linear regression are valid only for Normally distributed outcomes. This belief leads to the use of rank tests for which confidence intervals are very hard to obtain and interpret and to cumbersome data-dependent procedures where different transformations are examined until a distributional test fails to reject Normality." We only need to worry about **extreme** non-Normality #### DOI: [10.1146/annurev.publhealth.23.100901.140546](https://pubmed.ncbi.nlm.nih.gov/11910059/) --- class:center, middle <img src="figures/screw_meme.png" width="88%" style="display: block; margin: auto;" /> --- # Rarely need non-parametric tests <img src="figures/time_difference.png" width="88%" style="display: block; margin: auto;" /> --- # Testing for a difference between groups .pull-left[ ## Parametric t-test * Difference of 6.3 days * 95% confidence interval: 2.5, 10.0 days * p-value = 0.001 ] .pull-right[ ## Non-parametric Wilcoxon test * W = 20428 * p-value = 0.0003 ] --- # Do not categorise continuous data <img src="https://raw.githubusercontent.com/agbarnett/talks/master/skepticon/figures/malnutrition.jpg" width="75%" style="display: block; margin: auto;" /> Known as "Dichotomania" --- class: inverse # Measuring malnutrition <!--- https://scopeblog.stanford.edu/2015/05/29/study-finds-arm-circumference-is-accurate-measure-of-malnutrition-in-children-with-diarrheal-illnesses/ --->  ###### Image from Stanford Medicine (study was not by Stanford) --- ## Lab vs real life <img src="figures/psa_journey1.jpg" width="62%" style="display: block; margin: auto;" /> --- ## Lab vs real life <img src="figures/psa_journey2.jpg" width="62%" style="display: block; margin: auto;" /> --- ## Lab vs real life <img src="figures/psa_journey3.jpg" width="62%" style="display: block; margin: auto;" /> --- ## Lab vs real life <img src="figures/psa_journey4.jpg" width="62%" style="display: block; margin: auto;" /> --- ## Do not use p-values and statistical significance <img src="figures/pval_journey.jpg" width="50%" style="display: block; margin: auto;" /> False positive probability = 9 / (9 + 12) = 43% --- class:inverse ## Do not use p-values and statistical significance .pull-left[ ## "Significance tests are popular with non-statisticians, who like to feel certainty where no certainty exists" (Yates and Healy 1964) DOI: [10.2307/2344003](https://shibbolethsp.jstor.org/start?entityID=https%3A%2F%2Fidp.qut.edu.au%2Fentity&dest=https://www.jstor.org/stable/2344003&site=jstor) ] .pull-right[ <img src="figures/Yates_Fisher_Cochran.jpg" width="60%" style="display: block; margin: auto;" /> Yates, Fisher and Cochran (Barry Eagel, Wikimedia Commons) ] --- ## Statistical significance has created huge biases in the literature <img src="figures/z_values.png" width="80%" style="display: block; margin: auto;" /> --- ## Don't use matching .pull-left[ Matching is a very strong assumption, needs a very strong match Widely applied inappropriately to match patients and hence eliminate confounding Can easily create selection biases Often reduces power ] .pull-right[ <img src="figures/cards.jpg" width="60%" style="display: block; margin: auto;" /> ] --- ## Understand differences in risk <img src="figures/risks.png" width="60%" style="display: block; margin: auto;" /> Adapted from DOI: [10.1016/S0140-6736(10)62296-9](https://pubmed.ncbi.nlm.nih.gov/21353301/) --- # Use research checklists <img src="figures/checklists.png" width="89%" style="display: block; margin: auto;" /> [goodreports.org](https://www.goodreports.org/) --- class:inverse # Same statistical mistakes ad nauseam <img src="figures/push_back.png" width="85%" style="display: block; margin: auto;" /> [Andrew Althouse](https://discourse.datamethods.org/t/reference-collection-to-push-back-against-common-statistical-myths/1787) --- class:center, middle, inverse ## Work slowly and carefully ## Publish fewer papers ## Take time to learn the methods you use ## Check your work, try to break your models --- ## “Do you want to be credible or incredible?” <img src="figures/simine.png" width="55%" style="display: block; margin: auto;" /> [Association for Psychological Science](https://www.psychologicalscience.org/observer/do-we-want-to-be-credible-or-incredible) --- .pull-left[  ] .pull-right[ <img src="figures/david.jpg" width="73%" style="display: block; margin: auto;" /> ]