Lies, Damned Lies, and Statistics

Miguel at Gun-Free Zone posted an interesting article ran in the LA Times in 2007:

Police discourage autopsies that might reveal a higher homicide rate in their jurisdiction, and pressure doctors to attribute unnatural deaths to health reasons, usually heart failure, the group alleges. Odds are, it says, that people are getting away with murder in Japan, a country that officially claims one of the lowest per capita homicide rates in the world.

“You can commit a perfect murder in Japan because the body is not likely to be examined,” says Hiromasa Saikawa, a former member of the Tokyo Metropolitan Police security and intelligence division. He says senior police officers are “obsessed with statistics because that’s how you get promotions,” and strive to reduce the number of criminal cases as much as possible to keep their almost perfect solution rate.

Japan’s annual police report says its officers made arrests in 96.6% of the country’s 1,392 homicides in 2005.

But Saikawa, who says he became disillusioned by “fishy” police practices and in 1997 left the force in disgust after 30 years, claims that police try to avoid adding homicides to their caseload unless the identity of the killer is obvious.

This article brings up a problem with reported numbers, they’re often massaged in order to make an issue look better than it really is. Numbers can be massaged in many different ways. Japan apparently tries to avoid labeling deaths as homicides, likely in an attempt to make the country appear safer by keeping the number of reported homicides very low. This isn’t the only case of such shenanigans being used to argue a case.

Consider Chicago’s policy of reporting homicides that occur indoors separately from those occurring outdoors. Chicago’s police justify the separate categorization for indoor homicides by claiming that police were unable to intervene whereas police intervention is possible outdoors. In truth the separate categorization was likely done in order to make homicide numbers look lower.

Number massaging occurs elsewhere. The United States is often cited as having the highest infant mortality rate of any developed nation. On paper this statistic makes it appear as though the healthcare industry in the United States is woefully lacking, especially when compared to nations that have state run healthcare systems. When you dig into each country’s methodology for calculating those numbers notable differences arrise:

A 2006 report from WHO stated that “among developed countries, mortality rates may reflect differences in the definitions used for reporting births, such as cut-offs for registering live births and birth weight.” The Bulletin of WHO noted that “it has also been common practice in several countries (e.g. Belgium, France, Spain) to register as live births only those infants who survived for a specified period beyond birth”; those who did not survive were “completely ignored for registration purposes.” Since the U.S. counts as live births all babies who show “any evidence of life,” even the most premature and the smallest — the very babies who account for the majority of neonatal deaths — it necessarily has a higher neonatal-mortality rate than countries that do not.

When two countries have different methods of calculating infant morality rates comparing official statistics from those two countries will give unreliable results.

One must keep in mind that reported numbers are potentially inaccurate. Inaccuracies may arise due to manipulating numbers before they’re reported (as Japan apparently does by failing to report many homicides as homicides) or by manipulating numbers after they’re reported (as Chicago does by categorizing indoor and outdoor homicides separately). When making arguments based on reported numbers one must use caution. Without knowing how those numbers were generated they become potentially worthless. A country’s homicide rate may be reported as low but you must know how those homicide numbers are generated to know for sure.