How the folks at The Skeptics’ Guide to the Universe got it all wrong. And why James Randi’s Million Dollar Paranormal Challenge remains an impediment to progress.
If you read Fringe-ology, or heard me interviewed on the radio, you know I think the value of paranormal stories often lies in what our reactions to such tales reveals about us.
In researching subjects like telepathy, UFOs, ghosts and consciousness, for instance, I found that the people most vocal about the paranormal tend to resist the unexplained. That statement probably sounds counter-intuitive, but allow me to explain: To the most ardent believers, a UFO report is evidence of alien visitation. To passionate skeptics, a UFO report is evidence of how readily people misidentify Earth-bound phenomenon as exotic technology. The truth, of course, is that an Unidentified Flying Object is just that—unidentified—and nothing more. It should go without saying that a UFO, once identified, ceases to be a UFO.
In these terms, skeptics and believers should be able to rally around a common set of cases. After all, listen to alien-visitation advocate Stanton Friedman or arch-skeptic James Randi speak on the subject and both will acknowledge the vast majority of UFO reports can be explained as mis-identified but well-known phenomena. Only a small handful of cases remain unexplained. But even when the data runs out, skeptics and believers go on arguing for their point of view. Friedman advances the E.T. hypothesis. Randi, or the like, explains that these unsolved cases will likely be answered by Earth-bound explanations—and in any case there is no evidence that aliens have visited; and so this is not a possible explanation we should take very seriously; and—may I add?—harrumph.
In my opinion the conversation about any of these unresolved cases should conclude in the exact geolocation where we really sit: at the edge of a mystery, which might be solved by any number of possible explanations—from the prosaic to the powerfully strange—and which cuts right to the heart of one of the deepest existential questions humankind has yet to answer: Are we alone in the universe? But as I mentioned at the outset, mystery—even if it is reality—is less appealing to the blinkered believer or dogmatic skeptic than the answer they have presupposed.
This is a natural phenomenon—as natural as aliens (if they exist). We all bring some bias to the conversation. We view new information through the lens of what we already believe. And we tend to notice or give greater weight to information that confirms our beliefs, or to interpret the data in some way that leaves us feeling more secure about our worldview.
This phenomenon, dubbed confirmation bias, operates within us all—and overcoming it requires a willful effort. In politics, we either assume our favorite candidate will withstand every scandal or we just flat-out ignore any salacious information that emerges. But we are far less gracious toward our political opponents. Consider: To the farthest right republicans, every facet of Democratic President Barack Obama’s biography seems to hint at his real status as a secret Muslim who hates America. To democrats situated most dramatically left, republican president George W. Bush was a dry drunk with daddy issues who allowed 9/11 to happen so he could invade Iraq.
I argue, in Fringe-ology, that such biases prove particularly destructive in conversations about the paranormal. But I now hold a more intimate understanding of how confirmation bias operates. Because in mid-January, I got to see how I look through the eyes of skeptics disinterested in looking past their own worldviews.
The occasion was a podcast produced by the good people over at the The Skeptics’ Guide to the Universe. On Jan. 19, the team there compounded lead host Steven Novella’s off-point rebuttal to my critique of James Randi’s Million Dollar Challenge with a litany of what I can only guess were heart-felt, bias driven mistakes.
I’ll take you through the segment, point-by-point, interjecting as needed. Where I can, I identify the speaker. But there are enough voices chiming in (five) that I occasionally can’t make out who is talking. In those instances, I simply identify the speaker as “SG”, for Skeptics’ Guide. (My comments are preceded by “Volk.”)
Steven Novella: So Evan, more people are attacking our beloved Million Dollar James Randi paranormal challenge.
Evan Bernstein: Yeah, the latest is by a fella named Steve Volk. Anyone ever heard of Steve Volk?
Rebecca Watson: Nope.
Bernstein: No. Of course you haven’t. Because he writes about the paranormal.
Volk: There are already a couple of interesting things here. But the most intriguing to me is that Bernstein presumes his skeptical co-hosts wouldn’t have heard of me precisely because I write about the paranormal.
I would’ve thought skeptics might actually consider it part of their beat (in journalistic terms) to familiarize themselves with what people in the paranormal community are writing. But Bernstein and his cohorts casually dismiss the idea. Back to the show…
Bernstein: He floats along unnoticed in the paranormal soup that’s out there with thousands of other like-minded authors. He’s a freelance writer, a journalist, and he’s made appearances on radio shows, usually in the Pennsylvania area, Philadelphia, Pittsburgh and he was recently on Joe Rogan’s podcast—
Bernstein: He’s been on Coast to Coast so maybe there are a few people who’ve heard of him. In any case, he wrote a book about the paranormal, and it’s called: Fringe-ology: How I Tried to Explain Away the Paranormal—and Couldn’t.
Volk: He misses some small things. The subtitle uses the word “unexplained,” not “paranormal.” I freelance, but I am on staff at Philadelphia magazine. Also, I’m guessing he misspoke but he seems to be saying most of the radio I’ve done has been in the Pennsylvania area. For what it’s worth, I’ve actually done comparatively few media appearances in Pennsylvania, and I’m terribly grateful to have received a lot of national exposure, including the recent Joe Rogan appearance, two interviews on the nationally syndicated Coast to Coast AM and segments on NPR’s Radiolab and MSNBC’s Dylan Ratigan show. But who cares? These guys are just warming up.
Bernstein: He saw a ghost as a kid and he hasn’t been regular ever since. That’s the gist of the book. And if you think about it, it’s kind of an argument from ignorance, right? “I can’t explain it, therefore it must be the paranormal.”
Volk: The problem, for anyone who knows the first thing about what I’ve written or said, is that the only people arguing from ignorance here are the hosts of Skeptics’ Guide.
For starters, I’ve never seen a ghost. I’ve also never claimed to see a ghost. The Family Ghost Story, as I call it in Fringe-ology, involves noises we could never explain. I had my own, fragmented memories of these sounds. But I never, as Bernstein suggests, conclude these bumps in the night represented an actual ghost. In fact, I write the opposite: “…to leap from an unexplained sound to the existence of disembodied life forms is, well, too great a leap to make, based on the evidence at hand.”
I even spend a great deal of the chapter looking into some ambitious materialist explanations for ghost stories, including infrasound and electromagnetic waves.
I can’t say I was entirely surprised that The Skeptics’ Guide screwed this up so thoroughly. I spent a lot of time researching just how mired the modern skeptical movement can be in its own conceits. And while this is sheer speculation on my part, I can guess at how Bernstein arrived at his mistaken notions. I suspect that whoever researched me looked at the description of Fringe-ology on my website or my Amazon bio and saw that I wanted to “investigate an old family ghost story.” Interpreting this phrase according to their own pre-existing biases The Skeptics’ Guide flew off into a host of pejorative associations. Then they moved on to my blog post.
Bernstein: Mr. Volk has a blog called “Steve Volk: The Generalist,” and let’s see…on January 9th he posted a blog titled “The Joke of the James Randi Challenge: In Defense of Sheldrake.” He dubbed Randi the “cranky elf” of the skeptical movement.
Watson: I think Randi would accept that, actually.
Watson: I think he would like it.
Bernstein: Volk lets us know in this blog post, he’d like to address what is likely the worst, least credible thing that Randi promotes, which is his long running challenge in which he vows to give one-million dollars to anyone who can prove paranormal claims in a controlled test. He says the Challenge has been muddled by the very boundaries of science* [I write that the Challenge muddles the boundaries of science, but Bernstein can be forgiven for the mis-quote]…allowing Randi-ites—I guess that’s us—to say paranormal claims don’t hold up to scientific scrutiny, while conceding—when pressed—that the Challenge isn’t science. I think Volk entirely misses the point here.
Bernstein: The Challenge is an effort to design a statistically sound protocol for any given extraordinary claim, which greatly minimizes bias and/or fraud.
Volk: Actually, I think the statistical thresholds on the Challenge are set primarily to minimize the chance of luck intervening. But let’s keep moving…
Watson: He seems to want the million dollars to be given to anybody whose results are statistically significant in a scientific sense. Like, if you were to do this, and publish it in a journal, that should get the million dollars. But he forgets the fact that the whole point of the Challenge is to find proof that the paranormal exists and that proof needs to go beyond a reasonable doubt…
Volk: Again, there are a few problems here. Watson suggests I want some paranormal researcher to collect the million. But I don’t want that. I have also never written or said so. Further, the point of the Challenge is not to prove or disprove the paranormal. She is simply, like so many others (as I will soon demonstrate) attributing a power and purpose to the Challenge that it does not hold. Incredibly, she also directly contradicted the nominal leader of the Skeptics’ Guide, Steven Novella, whose written response to me states: “The purpose of the challenge is not to design and run scientific experiments, and it is not to scientifically prove or disprove the existence of the paranormal or any particular supernatural phenomenon.”
Still, he lets Watson get away with this misstatement. Did he miss it? Or is he simply disinclined to criticize one of his own?
Novella: Let’s look at this another way. Like, if you wanna use a p-value of .05, which is a traditional cut off for statistical significance in a scientific publication, that means Randi would be giving a million dollars to every 20th applicant of the million dollar challenge, which is obviously not practical. So essentially, what Volk says is that [research of the sort conducted by Dean Radin or Rupert Sheldrake] would require years of research and thousands of trials. Again, completely missing the point. The point of the Million Dollar Challenge is not to conduct publishable scientific research. It’s not to look for subtle effects that would require lots of trials…
Volk: Um, actually, that is my point. In other words, Novella seems to miss this entirely, but he and I agree. What I write in my initial post, “The Joke of the James Randi Challenge,” and a subsequent follow-up, “Randi’s Minions and the Million,” is that the MDC is not set up to serve any meaningful scientific aim. There are no papers coming out of it. And the results applicants need to generate in order to achieve victory are far beyond what scientific researchers like Radin and Sheldrake claim to be able to produce. Street corner psychics may claim inerrancy, but the Sheldrakes of the world are looking for much smaller, subtler effects. My criticism is that, too often, paranormal researchers conducting actual science are subjected to questions about why they don’t apply for Randi’s million; or the entire field of parapsychology is somehow cast under suspicion because no one has won Randi’s prize.
To illustrate this, I had—in the very post these skeptics are so ineffectively critiquing—used the new atheist thinker Sam Harris as an example. In his own appearance on the Rogan podcast, Harris claimed that there is “something fishy” about the failure of a scientist like Sheldrake to apply for the million.
In response to Harris, I wrote:
“…There is nothing ‘fishy’ about the disinclination of a scientist like Sheldrake to participate in the Challenge. Statistical significance is built through sheer repetition. In fact, achieving a proper ‘sample size,’ testing an effect enough times, is bedrock science. Conversely, failing to obtain a representative sample size is a hallmark of the Randi Challenge. Scientists like Sheldrake, Dean Radin or Daryl Bem conduct studies that require dozens of people (or more) and take weeks or months or even years to perform. Randi puts on events that occur in a fraction of the time, generally over an evening or afternoon. In conclusion, Randi’s protocols simply won’t allow Sheldrake to conduct real science.”
But amazingly, Novella never mentions Harris in his written critique of my initial post. And during this subsequent Skeptics’ Guide podcast, he and his peeps never address the Harris portion of my argument—my central exhibit—either. Readers familiar with my book, or simply alert to the problems associated with group dynamics, might already see what’s happening here. But allow me to explain: Me and the good people at Skeptics’ Guide are talking past each other. They accuse me of “missing the point” of their “beloved” challenge while I keep waiting for them to even make the faintest move toward addressing the point I actually made. The issue, as I see it, is that I am writing from the perspective of someone sympathetic to parapsychology researchers. That marks me as “outside” their group—skeptics—and so they start ascribing to me all kinds of visions (he saw a ghost), motivations (I want Randi to give away his million) and thoughts (can’t explain it, must be a ghost), with a staggering 100-percent inaccuracy rate. Further, because I am defending the group they seek to defeat, I can’t be granted any sort of win—or even traction.
Again, I am speculating. But I can only guess that they essentially back up my chief point—The Challenge isn’t science—but avoid delving into the Harris portion of my argument because they never wanted to string three particular words together: “Volk is right.”
To make matters still worse (for them), by the time this Skeptics’ Guide aired, I had already rebutted Novella’s initial printed response.
Here’s a quick excerpt*:
*Greg Taylor, at the Daily Grail, has done a bang-up job of addressing the Challenge’s shortcomings. And he has already published a response to Novella of his own. He notes that Harris is far from the only interested party who is unclear on the Challenge’s range and authority. “New Scientist asked Daryl Bem if he would apply for Randi’s million dollars,” writes Taylor. “We even find the appearance of this trump card in the scientific literature. For instance, in the Wagenmakers et al. response regarding Bem’s famous experiments (‘Why Psychologists Must Change the Way They Analyze Their Data: The Case of Psi’)—an editorial that Novella himself described as ‘the best thing to come out of Bem’s research’—we find this mention about the chances of precognition being a real phenomenon: ‘[T]here is no real-life evidence that people can feel the future (e.g., nobody has ever collected the $1,000,000 available for anybody who can demonstrate paranormal performance under controlled conditions, etc.)’”
In other words, there is an odd pattern here, in which really smart people with scientific backgrounds wind up confused about just what James Randi’s Challenge is all about. Given that the resulting confusion affords the Challenge greater standing, skeptics might even be seen to hold a keen self-interest in allowing delusions over the Challenge’s boundaries to persist.
Intriguingly, Harris remarks, in his interview with Rogan, that he had met with Randi and discussed the Challenge. Yet he walked away with a muddled view. And frankly, there are moments in their mid-January podcast when the hosts of The Skeptics’ Guide also seem similarly confused: As I write above, Watson falsely stated that the point of the Challenge is to prove the paranormal exists; additionally, another host (I couldn’t make out which one) said “[Volk] doesn’t get that there has to be a protocol and it has to be difficult or you’re not proving anything”—again falsely raising the idea that the MDC is about proving paranormal phenomena exist; and even Novella himself slips into referring to the Challenge as “the study” and “these studies.” But, as he himself argued, the MDC is not the sort of test that might result in a peer-reviewed scientific publication or even stand as a preliminary experiment; so the word “study”, with all its scientific connotations, is inappropriate.
Slips of the tongue, if that’s what they were, are understandable. And it may seem as if, here at the end, I am being particularly hard on The Skeptics’ ragged use of language. But whether such self-contradictory statements were intentional (I doubt it) or accidental (more likely) is, to me, almost beside the point. What intrigues me is that this segment of Skeptics’ Guide functions less as a critique of my argument than as a kind of unintended support: The Challenge is too often confused with what it isn’t: science. Thanks for that, guys. But even though I am giving myself the win here—hey, maybe I’m a little biased too—I emerge from this entire exchange more disturbed than happy.
The skeptical movement purports to guard the boundaries of science. The slogan employed by the Skeptics’ Guide is “Your escape to reality.” But in this encounter, reality was the victim. And I can only wonder if the population at large is as skeptical as it needs to be of a modern skeptical movement that has so much trouble seeing past its own worldview.