Bad Intentions and Psi (& Skeptiko No.100)

Skeptiko is a pro-parapsychology podcast which attracts a mix of ire and exasperation from the skeptical community due to its misleading title and the tendency of its host, Alex Tsakiris, to promote the show as being an agenda free exploration of ‘controversial science’. A quick listen to any episode of Skeptiko will confirm to an impartial listener that the host, far from being ‘agenda-free’, possesses a painfully evident bias which infusses the entire character and tone of the show. Tsakiris tends to fawn over guests who are on to promote forms of parapsychology he supports and is openly hostile to- the point of occasionally berating- skeptical guests.

In light of the above, and to try and redress some of the bias inherent in the show, I have decided that I will try and offer some skeptical commentary on the Skeptiko shows as Alex adds them and I get time to go through them. It is unlikely that this will have any impact on Alex or the fans of his show, but it might mean that someone who listens to Skeptiko without an agenda and then does a google search on the topic could come across an alternative perspective rather than falling further down the Skeptiko rabbit hole.

So without further adieu, here are my thoughts on ‘Skeptiko episode 100: Dr. Garret Moddel Brings Psi Research to University of Colorado Classroom‘.

The show begins as it often does with Alex extolling the academic credentials of his guest. This is not something unusual in the podcast world, or interviews more generally, but for Alex it frequently serves as the basis for later appeals to authority. In these oft repeated appeals Alex’s general argument is that if some smart academics promote parapsychology then obviously there MUST be something to it. The fact that smart academics across the world promote a whole host of dubious theories including things like intelligent design and aids denialism makes this a less than persuasive argument but that doesn’t seem to ever give Alex much pause for thought… as we shall see later.

For now turning to the ‘meat’ of the podcast, the first thing to say is that Dr. Garrett Moddel is certainly a smart guy and he has probably forgot more about physics than I will ever know. He is a legitimate scientific researcher working in quantum engineering and in the interview he comes across as a reasonable man who seems to have a genuine interest in exploring psi. None of this however, means that his arguments or the studies he presents are compelling and indeed I find that there a host of rather transparent problems besetting his theoretical positions and the parapsychology research he promotes.

An immediate warning sign that his judgement on this topic might be off is that he states early into the show that he was ‘blown away’ when he started looking at the existing research on psi. Now this is, in some respects, a subjective call but frankly, I would love to know what specific evidence it was that ‘blew his mind, as the evidence for psi which is most often cited by psi proponents almost always shows very small effect sizes. And these small effects also frequently disappear when replications are attempted or  better controls are employed. Dr. Moddel seems to freely admit these points at various points during the interview by acknowledging that psi researchers often have problems replicating findings and that psi effects in experiments seem to be very ethereal. Getting blown away by small effects that frequently hover around the border of statistical significance and which disappear with surprising regularity with replications, or when tighter controls are introduced, seems to me to be a strange position for an experienced researcher to take. But that was just the start of the surprises…

Dr. Moddel also discussed how he went about setting up his course in psi research at the university of Colorado. Here aside from criticising the universities apparent initial reluctance to approve the course he also provided details about what the course entails and some of the impressive projects created by students of the course. In this portion of the discussion there were a number of issues that jumped out at me. The first is that after Dr. Moddel explained his vision for the course:

The way I had designed the course, it was discussing psi research and a point-counterpoint discussion throughout the course between skeptical and proponents’ perspectives.

He then goes on to state that for the course “the two textbooks that I’m using are Dean Radin’s Entangled Minds, which is just a wonderful, wonderful book describing psi research and then also Chris Carter’s book on Parapsychology and the Skeptic“. This will immediately set off alarm bells to any skeptic as Dean Radin is one of the most prominent pro-psi advocates today and he is not noted for having anything approaching a balanced position on psi research.

Instead, Radin is one of the strongest advocates for the position that psi phenomena have already been proven and mainstream researchers are ignoring the amazing results because of their conservative bias and prejudices. Using his book as one of the two textbooks for the course might be understandable however if the second textbook provided students with the alternative skeptical perspective and covered some of the common methodological issues which beset psi research. Chris Carter’s book is not the kind of book to provide this information.

Chris Carter, although sharing the same name is not the same individual responsible for the X-Files, I previously made the mistake of assuming they were! However, while he is not famous pro-paranormal TV show creator, his position and clear bias in relation to psi can be seen remarkably clear in a previous episode of Skeptiko during which he declared to Alex that the “so-called skeptics are basically defending an outmoded philosophy of science that’s turned into an ideology”. He also explained how skeptics are motivated to challenge research due to their preconceptions “that these sort of phenomena don’t make any sense and challenge their world-view… they’re going to do anything they possibly can to dismiss evidence that challenges their preconceptions”. This betrays a fundamental lack of understanding of the skeptical position and it also indicates quite clearly how unsuitable Chris Carter’s book is for providing any sort of balance to Radin’s perspective.

So if these two books are the ‘core texts’ for the course I think the chances that the skeptical position gets a fair hearing in Dr. Moddel’s class are rather low. A text book discussing how to avoid flaws in statistical analysis or how to design properly controlled experiments would be a lot more valuable for would-be parapsychology researchers than two pro-parapsychology treatises and this becomes even more evident when the topic moved on to the student’s experiments…

The first experiment mentioned in the interview was described as follows:

One of them involves guessing hidden Zener cards. Zener cards are cards that are of five different types having a circle, square, wriggly line, a plus, and a star. What the student did is he got 100 Zener cards, 20 of each, shuffled them, turned them upside-down, and had the subject sit across the table from him so that the cards were blocked and they couldn’t see them because they were upside-down and also because he had a laptop display between the cards and the subject. He then raised one card at a time and had the subject guess which of the five they thought it was.

He went through the entire stack this way. Then he did this with a number of different subjects. He did it with 12 subjects and tabulated the results.

The actual ‘study’ can be found here.

Now reading through the study it seems that the experiment was designed to test the ability of the respondent to use psi to identify what was located on a card that could not be seen from their position. It was not designed to test whether they read the answer from the experimenter’s mind, as the experimenter was also supposed to be blinded to what was on the card. The blinding procedure used was as follows: the cards were shuffled, placed face down, then the experimenter picked up a card and held it up facing away from them and towards the subject.  A laptop on the desk between the experimenter and the subject blocked the subject’s view of the card and then after they guessed it was set face down on the table.

This is an ok procedure but it would be easy to improve. For example, if a person’s psi ability means they can ‘see’ through a laptop you would imagine that they could also ‘see’ through a coloured envelope so why not just limit all chances of accidental detection from the experimenter AND the subject by randomising the cards and getting a volunteer (not the experimenter) to place them in numbered envelopes then carry out the experiment as before. This might seem like an insignificant amendment but when you consider that the research is attempting to test whether a person can use psi rather than any other more mundane signal to gain information then controls and blinding become extremely important!

For example, imagine that the experimenter unconsciously noticed blurred reflections in the laptop screen as he held up the card. That introduces a potential influence that could lead to significant results that have nothing to do with the subjects possessing psi ability so it is essential that such influences be ruled out.

However, there is a bigger problem with the study- the extremely small amount of participants! For such a simple experiment 12 participants is really a very small sample and it also dramatically increases the chances that you will get a statistically significant result which would disappear with a larger amount of participants. However, instead of taking steps to remedy this problem the student then exasperated the problem by breaking the 12 respondents into four different groups meaning that there were now only three subjects in each group! The groups were described as follows:

1) Those who believe and were presented with pro-psi arguments; 2) those who don’t believe and were presented with pro; 3) those who do believe and were presented with anti-psi arguments; and 4) those who don’t believe and were presented with anti-psi arguments.

And the results?

… for the three groups that had a negative input regarding psi, either they didn’t believe or they were told that it was rubbish; they were pretty close to random in their guesses as to what these cards were. However, the believers who were presented with pro-psi arguments got 27% of the cards correct, whereas by random chance, they would have been expected to get 20% correct.

Finding that one group out of the four got a result that was 7% above random chance is hardly surprising. In fact when you have lax controls and a selection of incredibly small groups it is virtually guaranteed that somewhere in your data something will come out as statistically significant.  And it is also worth noting that there is no indication that the experimenter was blinded to which group they were testing. This is important because numerous studies have demonstrated that experimenter’s bias has the potential to significantly influence results by causing them to treat groups differently due to their beliefs/expectations. This again would have been very easy to control for by simply making another experimenter responsible for the grouping or, to be even more thorough, blinding all the experimenters until the study was completed by using randomised envelopes.

As per Dr. Moddel’s point that ‘by random chance they would have been expected to get 20% correct’ well yes and no. It is true the odds of selecting the correct card are 1/5 but that does not mean that we would expect everyone to get exactly 20% anymore than we would expect that 12 people flipping coins 100 times in a row to all get heads 50 times. We would expect some to be over 50 and some to be under and we would also expect that there will occasionally be some outliers! This is exactly what Dr. Moddell and his student found. They got three outliers in their twelve results and the two positive outliers happened to lie in their pro-psi, pro-psi argument group. Now this could be due to the fact that this group’s positive intentions meant they were able to use a psi ability to improve their guesses or it could be due to random chance.

Unfortunately we will never know because the only way to find out if the effect was robust would be to replicate the study with more participants but Dr. Moddell is against both replicating studies and increasing the number of subjects involved. By doing either of these he argues that the experimenters enthusiasm and interest would drop and that could cause the effect to disappear because it is reliant on both subjects and experimenters being enthusiastic. That’s one potential explanation… but the other much simpler explanation is that the effect will disappear because it is a random anomaly that only exists due to poor controls and small sample size. The boredom objection also doesn’t really hold up because it would be a simple matter to get a group of different group of students to replicate the study with a different group of participants. Hence, their boredom should be no greater than the original experimenter as they would be doing the study for the first time.

Discussing the dangers of boredom Dr. Moddell asserts that:

When you do these experiments, if you’re enthusiastic and interested, you put your intention and interest into it and they work. In fact, that’s probably why, as you’ll see, most of the experiments that my students carry out, work because they’re young and enthusiastic and probably most important, they only do one set of trials. They do enough so that it’s statistically significant, but they don’t do it to the point of boredom.

But what he seems to completely ignore is that getting biased researchers to do small one off studies is also the best way to get biased experimental results which report a chance random effect that is statistically significant but which is, in reality, meaningless. And if psi phenomenon really is so incredibly fleeting that it only amounts to things like improving people’s ability to guess the predict the correct card to 7% above chance as long as no-one in the vicinity has any doubt in psi and those involved haven’t heard anything negative about psi recently then I have to wonder how Dr. Moddell can justify statements like this:

It is truly mysterious and it’s appearing that the answer to understanding consciousness is going to involve psi phenomena. That psi is not just a little pimple on the large experience that we have in life, but instead, psi is one of the major phenomena that controls almost every activity and every thought that we’re involved with.

If psi disappears in the presence of skeptics, repeated measures or simply if enthusiasm dips due to boredom then how can it also be a ‘major phenomena that controls almost every activity’. It seems Dr. Moddell want’s to have his cake and eat it by simultaneously claiming that psi is a huge fundamental force and that, at the same time, it only produces small effects that disappear with the slightest hint of a bad vibe in the area.

At one point after explaining another experiment involving people using their intention to increase the reflection of light in which the experimenter again appears to have been unblinded and a very small sample is used Alex enthusiastically exclaimed:

But I think it speaks to a couple of things. One, clearly for anyone who has any doubts, you’re a guy who knows how to set up and run an experiment like that, or at least oversee someone else doing it. You have all the credentials and you’re a Fellow of the Optical Society of America. I don’t even know what that means.

The typical appeal to authority is here but the point that Alex misses is that Dr. Moddell’s ability to run a Physics experiment is not in question nor is his ability to employ scientific equipment properly. What is in question is whether he has the experience to design properly controlled experiments with human participants and to implement proper controls to rule out experimenter bias and/or control for non-psi effects. Physicists and electrical engineers do not typically spend much of their time during their studies learning about the unique challenges involved with conducting research with human subjects and facts such as Dr. Moddell being a Fellow of the Royal Optical Society bear absolutely no relevance to his ability to conduct good parapsychology research. Mistaking academic expertise in one area for expertise in parapsychology and how to conduct good experiments with humans is a mistake constantly made on Skeptiko discussions and it seems to me that the various expert’s lack of relevant expertise is often the key source for the problems in parapsychology research.

There is much more that could be said about dodgy statistics and poor controls but I think the final experimental example Dr. Moddell provided illustrates a large number of the problems that beset the approach to research that he and his students follow. In this case he described how students in the class tried to use ‘associative remote viewing’ to see into the mind of the experimenter, draw an image of what they saw and thus predict the future. That sounds strange (and it is) but stick with me. The experimenter was supposed to have decided on two different images with one being associated with the stock market going up the next day and one with the stock market going down the next day. He would then show the students one of the images at the end of the following day depending on what the stock market had done.

So to be clear… in this experiment the students were collectively supposed to be able to not only read the mind of the experimenter but also use the image they saw to predict his actions in the future and thus predict the activities of the stock market! Dr. Moddell claimed that using this method his class was able to correctly predict the future activity of the stock market seven times in a row. Setting aside for now the bizarre nature of the claim the first point to note is that since the students were not informed of the two images the experimenter chose a ‘judge’ was included in the experiment and this judge took the collected drawings and then decided on which of the two images the students drawings most resembled.

This sounds like an incredibly subjective judgement and it is also absolutely crucial that the ‘judge’ had no idea which image was associated with which outcome as otherwise his knowledge of what was going on with the stock market could easily influence the result. Furthermore, it is also crucial that the experimenter was kept unaware of the final outcome of the judges deliberations as otherwise they could unconciously alter which image refers to which outcome. Absolutely stringent controls would be needed here as otherwise the prediction would have been a self fulfilling prophecy and yet I can’t help being skeptical that this would have been the case.

Furthermore, although getting the correct result seven times is unlikely it is certainly not as amazing as Dr.Moddell and Alex seem to believe. After all the stock market is not something which each day has a 50/50 chance of going up or down. In some periods the stock market might be stable and growing and in others such as a recession it is quite obvious that its going to be falling more often than its growing. It may have also been the case that of the two images used one was more likely to represent a positive outcome than the other and hence the judge who was probably not an unbiased observer may have been unconciously effected in how he interpreted the students drawings.

In short for experiments like this one, which claim incredible effects (reading minds and predicting the future stock market), you really want to make sure that you have super tight controls in place and that the effect is actually there i.e. multiple replications and repeated independent samples. Dr. Moddel’s theoretical positions forbids things like replications however and also provides a nice escape hatch that if skeptics cannot replicate his results then it is their mindset which is responsible. We are therefore left with nothing but claims of extraordinary results which are never replicated and which are in all likelihood down to poor controls, inadequate samples and experimenter bias. Dr. Moddel acknowledged for instance that when he continued to try and reproduce the effect it disappeared.

I continued using this effect with a few of us afterwards, and we found that as we continued to do it, a little less formally than we had done it with the class, that the effect started going haywire.

This could be as Dr. Moddel believes because those involved became to interested in the outcome but it is also exactly what we would expect to see if the original experiments were statistical flukes or being influenced by some problem with the experimental setup.

The bottom line is it all sounds very flakey and despite possesing clear credentials as a physicist Dr. Moddel (and his students) do not seem to be very capable of designing or implementing well controlled parapsychology experiments. Instead what we get is yet another series of extraordinary claims from a random academic with a suite of poor evidence from individual studies to back them up and a host of dubious explanations to account for why skeptics or even other researchers will likely not be able to replicate their results.

The final point I would mention and one which Alex seemed to pay little attention to during the interview is that Dr. Moddel while stating his endorsements of various ‘alternative’ research topics also alludes to what appears to be an endorsement for aids denialism:

So the topics that we consider are psi phenomena, of course. UFOs, advanced energy concepts, so zero point energy, what used to be called cold fusion is now called low energy nuclear reactions, just second law of thermodynamics issues. We also look at cosmology and astronomy, issues there.

We look at phenomena such as AIDS and HIV. There are alternative theories as to how AIDS is caught. In fact, I think pretty soon you’re going to see mainstream changing around and realizing that the connection between HIV and AIDS that was originally supposed is not quite right.

Then advanced propulsion techniques. Even cryptozoology. There are a number of different topics that are pursued.

So to return to my original point and one Alex and fans of Skeptiko’s approach would do well to heed. Someone being an expert in one field does not mean they are therefore experts in completely unrelated fields. Dr. Moddel’s comments about aids denialism should indicate that he might not be particularly good at assessing medical research and likewise when he claims that “somebody with an open mind, looking at the literature, has got to accept that these psi phenomena exist” I remain skeptical.

Advertisements

17 comments

  1. Hello,

    Interesting stuff…

    OK, one mistake: the Chris Carter who wrote the “Parapsychology and the Skeptic” is NOT the creator of X-Files. They have the same name, but they’re not the same guy. The Chris Carter who wrote the parapsychology book was on Skeptiko wayyyyyy before. He’s strongly anti-skeptic. So your point about the textbooks is very valid. I had exactly the same reactions when I heard that. But nothing to do with X-Files…

    With skepticality,

    Like

  2. Thanks for the heads up I’ll fix the article shortly (and acknowledge my mistake!). The quotes I attributed to him were his quotes from the old Skeptiko episode so I got the right guy but I just also credited him with making the X Files!

    Edit: Fixed now.

    Like

  3. I am on the pro-psi camp, but most of what you say makes a lot of sense.

    I too, was shocked at Prof Moddel’s endorsement of the wacky HIV denialists (those that are still alive). In my opinion he hemorraged credibility with that statment.

    I will petitition the SSE (society for scientific exploration) to remove this has a topic for discussion. It is dangerous and without ANY support whatsoever.

    Like

  4. Hi Chris… wanted to respond to some of this but not sure how far I will get… still chomping thru the first paragraph:

    “Skeptiko is a pro-parapsychology podcast which attracts a mix of ire and exasperation from the skeptical community due to its misleading title”

    — this always gets me… there are a lot of ways to be “skeptical”… I hope I’ve given listeners plenty to be skeptical about.

    “… promotes the show as being an agenda free exploration of ‘controversial science’.”

    — I don’t get the quote on ‘controversial scinece’???

    “A quick listen to any episode of Skeptiko will confirm to an impartial listener that the host, far from being ‘agenda-free’, possesses a painfully evident bias which infusses the entire character and tone of the show.”

    – of course this is a subjective opinion, but my willingness to engage skeptics and proponents is not. Where do you go to get both sides of these issues?

    “Tsakiris tends to fawn over guests who are on to promote forms of parapsychology he supports and is openly hostile to- the point of occasionally berating- skeptical guests.”

    — that’s silly… who have I berated?

    Alex

    Like

  5. Hello Alex, of course all of the above is my subjective opinion and I also suspect that you will find much above you disagree with but my goal was to provide an alternative interpretation so that much is to be expected.

    As you may well have other points to make I’ll respond to the ones you’ve made above so as to avoid a mammoth reply. The numbers correspond to your replies under the quotes.

    1. You do provide plenty of material to be skeptical about but the issue is that the title skeptiko creates the impression that your show is from a skeptical position. You may argue that you have an alternative skeptical viewpoint, but since you are generally pro-psi and pro-alternative views, the skeptiko title is a bit like an alternative medicine podcast calling itself mainstream medico.

    2. The quotation marks were because I took the quote from your site and also because I believe there is some debate over just how accurate it is to term a number of the subjects you address as science.

    3. I was originally attracted to Skeptiko because I really liked the idea of a show that engaged both sides and encouraged scientifically informed debate and collaboration. I was severely disappointed after listening to a few shows as it became incredibly obvious that your show was essentially a pro-psi show which occasionally interviews skeptics but typically in a rather confrontational manner. I am not alone in this opinion. Take a look on any skeptical message board and you will see the same sentiment voiced. I like the idea of Skeptiko a lot but I think the clear bias means that its goal is never reached.

    4. Maybe you can’t hear it Alex but you honestly can come across as extremely dismissive and hostile when you disagree with your guest. Didn’t you recently have a run in with Mark Edward? And before that some of the exchanges with Ben Radford were painful to listen to! Also, look at the title of your most recent episode! It’s hardly respectful towards your interviewee!

    Hope that fleshes out my perspective a bit better.

    Like

  6. Glad to hear your perspective… but kinda pointless to debate this kinda stuff.

    Here’s one we can nail down:

    “3. I was originally attracted to Skeptiko because I really liked the idea of a show that engaged both sides and encouraged scientifically informed debate and collaboration. I was severely disappointed after listening to a few shows as it became incredibly obvious that your show was essentially a pro-psi show which occasionally interviews skeptics”

    – give me a count of Skeptiko’s skeptical/proponent guests… then give me a similar count from ANY other show.

    Like

  7. I was wrong to say occasionally as you are correct you do frequently interview skeptical guests. Or at least, you have been quite balanced over the last 20 podcasts I looked at. The numbers however do not really do not really make the bias in the interviews any less disproportionate.

    Like

  8. Chris,

    I’m not sure I agree with your bias comments. The interview with Dr Woerlee was an examplar of balanced interviewing. Indeed Dr Woerlee was invited back to make his points against Dr Long’s rebuttal of Woerlee’s first interview. This doesn’t sound like bias to me. We all have our own ways of seeing the world, but that doesn’t mean it necessarily has to impact on how we nehave towards dissenting opinions.
    Alex has behaved far more admirably than 99% of skeptics out there.

    Michael Duggan Ph.D

    Like

  9. Michael,

    Alex deserves kudos for inviting Dr. Woerlee back on and the interview wasn’t too bad, I agree. The issue is how Alex chose to present and frame the interview. So the title of the episode is “101. Near-Death Experience Skeptics Running Out of Excuses” and the sub heading reads “Series of interviews with leading near-death experience skeptics show no plausible medical explanation for afterlife experiences.”

    Neither of which is true or an accurate portrayal of Dr. Woerlee’s position. Instead, that is a clear representation of Alex’s biased position on NDE. Now whether you feel his bias is justified or not is another matter but it is clearly there and it has a big impact on how he interacts with his guests. I listen to a lot of podcasts and all the hosts have their own opinions and often they disagree with their guests but I never get the same clear feeling of bias as I do with Skeptiko episodes.

    Right now time to go to sleep. Oh and Michael glad to see you noticed the aids denialism too! I think it’s something we can all agree is complete bunk!

    Like

  10. Chris, interesting post, but I have to say it’s kinda long. I got bored with it around the part where you were talking about boredom. 😉 In my blog, if I have a really long post coming up, I break it down into 2 or more parts to try to make them more digestible. I suggest you try to shorten or break into multiple posts in the future.

    Like

  11. Please don’t break the articles up into parts! We don’t need mid-season cliffhangers on the Internet! 🙂 Breaking them up makes it harder to print or save, and all the ones I’ve read so far have been worth doing both. 🙂

    I’m glad you’re listening so I don’t have to. Alex’ debate with Mr. Radford (if my recollection it was him is correct) in which he declares over and over, practically yelling, that a psychic helped solve a case because one police officer called her “helpful” even though she never found the body or told them to search anywhere they weren’t already searching was enough to get me to pull the earbuds out of my ears as listening to the lawn mower was less grating. 😉 If Mr. Radford had been physically present during that episode I would have feared for his safety. That was the last episode I ever listened to. If there’s a particularly compelling guest on I might glance at a transcript.

    I’d love to see the tables turned and have someone like the writer of Ebon Musings on (http://www.ebonmusings.org/atheism/ghost.html) and put Alex on the hotseat and challenge him to defend the mountain of evidence against dualism, as opposed to his ambushing scientists about one or two anecdotes about NDEs.

    Like

  12. I came upon this blog, but I must say I am very happy to have had the opportunity to read and make my comments
    I consider this article very interesting and rich in content, would also be very nice to discuss issues that are present in this paper.
    They still agree with what you said and I believe we can always find new ideas for improvement.
    Thank you very much to enable the vision of my post.Se have the desire to see the contents of my site, you are welcome, welcome advice on how to make it better. My intention is to become the largest in the world.

    Dott.Lorenzo Dall’Olio
    Usui and Karuna Reiki Master
    http://www.inforeiki.it

    Like

  13. You wrote: “This betrays a fundamental lack of understanding of the skeptical position and it also indicates quite clearly how unsuitable Chris Carter’s book is for providing any sort of balance to Radin’s perspective.”

    Have you actually read my book? I think that my book is the most comprehensive attempt ever to understand the skeptical position. I back up everything I say with hard evidence. All skeptical arguments are closely examined.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s