Lies, Damned Lies, and Statistics

Many people like to divide science into hard and soft. Hard sciences are the ones where you can directly apply the scientific method whereas soft sciences don’t lend themselves well to the scientific method. For example, physics is generally considered a hard science since you can replicate the results of previous experiments with new experiments. Sociology, on the other hand, doesn’t lend itself well to the scientific method because the results of previous experiments often can’t be replicated by new experiments. As if to acknowledge that fact sociologists tend to rely heavily on statistics.

In our modern world where science is the new god you can’t make an argument without somebody demanding to see your scientific evidence. While such demands make perfect sense in debates about, say, physics, they don’t make much sense when it comes to social issues because you can create statistics that prove whatever you want. Case in point, a research project found that one in every 24 kids in the United States has witnessed a shooting. However, the statistic was created through a survey with a question worded in such a way to guarantee a predetermined result:

It all started in 2015, when University of New Hampshire sociology professor David Finkelhor and two colleagues published a study called “Prevalence of Childhood Exposure to Violence, Crime, and Abuse.” They gathered data by conducting phone interviews with parents and kids around the country.

The Finkelhor study included a table showing the percentage of kids “witnessing or having indirect exposure” to different kinds of violence in the past year. The figure under “exposure to shooting” was 4 percent.

[…]

According to Finkelhor, the actual question the researchers asked was, “At any time in (your child’s/your) life, (was your child/were you) in any place in real life where (he/she/you) could see or hear people being shot, bombs going off, or street riots?”

So the question was about much more than just shootings. But you never would have known from looking at the table.

That survey was then picked up by the Center for Disease Control (CDC( and the University of Texas (UT) who further twisted the research:

Earlier this month, researchers from the CDC and the University of Texas published a nationwide study of gun violence in the journal Pediatrics. They reported that, on average, 7,100 children under 18 were shot each year from 2012 to 2014, and that about 1,300 a year died. No one has questioned those stats.

This is how statistics is often used to create a predetermined result. First a statistic is created, oftentimes via a survey. The first problem with this methodology is that surveys rely on answers given from individuals and there is no way to know whether or not the people being surveyed are being truthful. The second problem is that survey questions can be worded in such a way as to all but guarantee a desired result. Once the results from the survey have been published then other researchers often take them and use them inappropriately to make whatever point they want, which is what happened in the case of the CDC and UT. Finally, you have a bunch of people making arguments based on those questionable statistics used erroneously by organizations that share their agenda.