I’m a skeptical man by nature but I tend to be more skeptical of what are traditionally labeled soft sciences such as psychology and sociology. My stronger than average skepticism stems from several factors.
First, and probably most importantly, experiments in these fields can’t isolate variables. When you’re experimenting on humans, one variable is the life experiences of the subjects of your experiment. Different people have different life experiences, which can lead them to act differently under the same circumstances.
Second, the subject of experiments in fields like psychology tend to act differently when they’re the subject of an experiment. This tendency isn’t unique to humans. Ravens and chimpanzees act differently when they know that they’re being watched.
Third, most experiments involving human subjects suffer from selection bias. Professors have a ready pool of humans to experiment on, western undergrads, and utilize them for most experiments. Anybody with even the most basic observation skills will notice that undergrad students tend to behave differently than, say, elderly individuals.
Now I have a fourth reason for my skepticism. It turns out that the findings of many psychological experiments are, to put it nicely, rather dubious:
The Zimbardo prison experiment is not the only classic study that has been recently scrutinized, reevaluated, or outright exposed as a fraud. Recently, science journalist Gina Perry found that the infamous “Robbers Cave“ experiment in the 1950s — in which young boys at summer camp were essentially manipulated into joining warring factions — was a do-over from a failed previous version of an experiment, which the scientists never mentioned in an academic paper. That’s a glaring omission. It’s wrong to throw out data that refutes your hypothesis and only publicize data that supports it.
Perry has also revealed inconsistencies in another major early work in psychology: the Milgram electroshock test, in which participants were told by an authority figure to deliver seemingly lethal doses of electricity to an unseen hapless soul. Her investigations show some evidence of researchers going off the study script and possibly coercing participants to deliver the desired results. (Somewhat ironically, the new revelations about the prison experiment also show the power an authority figure — in this case Zimbardo himself and his “warden” — has in manipulating others to be cruel.)
The problem of manipulation isn’t unique amongst so-called soft sciences. The scientific method generally assumes that the experimenter is unbiased but what happens when the experimenter wants a specific outcome? Oftentimes, they can setup the experiment or manipulate the results in such a way that they can create their desired outcome. This is especially easily to do when the subjects of an experiment are manipulable humans. A little coercion can result in desired behavior.
I’m happy that these issues are finally being scrutinized more thoroughly. But I’m curious what the fallout will be. Science has become a religion to many people. People tend to react negatively when they learn that their priests have been lying to them and that their gods are not actually gods. Part of my worries that the backlash of this scrutiny could be a reflexive opposition to science by the masses but then the other part of me remembers that most fans of science aren’t actually scientifically minded anyways.