In one of my recent posts I had some fun with the topic “…the six things I want our kids to know about science” … of course there are more, but it was a good start. Among the comments I received on the post, one from a friend posed a troubling question, which could be restated as, “OK wise guy, how DO we help our kids figure out what to believe with all this seemingly conflicting and confusing sciency stuff?” In re-reading my rants on this subject I realize that while I have done a passable job of laying out the challenge of figuring out how to decide what to believe, I’ve done a crap job in providing any answers to the question how. This led to some rapid self- reflection, asking myself the question, “OK wise guy, how do YOU figure out what to believe with all this seemingly conflicting and confusing sciency stuff?” …fie on self-reflection. But the effort did allow me to see that I do have a bit of method to my madness, outlined here.
My first step in assessing the validity of a claim is pretty much always the same… I take a really, really close look at the source. I feel this is the best place to start, and while taking a bit of time and effort it can often yield immediate results. If the source does not pass the smell test, then move on. This exercise breaks down into two broad categories, qualification and motivation.
Part one: Assess if the source is qualified to make the claim
For me, this one is pretty straight forward... Do they have the expertise, track record and just plain legitimacy to make claims? If I don’t immediately recognize the source of a claim, such as a major university, research institute, etc, a quick internet check can be very revealing. A couple of my favorite examples here are when I Google mapped the location of a very sciency sounding institute that was putting out climate change denial lies, and it turns out they are housed in a small out-building on a dirt road in rural Oregon; or when a legitimate sounding, oft quoted source on the danger of GMOs turned out to be a group of osteopaths in Kansas who did no research at all, but just passed along bogus information. I’ve done these searches many times and it’s really amazing how much you can learn about the sources of the nonsense that gets put out, and then picked up, by the media, bloggers, interest groups or your Facebook friends. You don’t have to be a scientist to figure this out…just consider the source and use your judgment.
BONUS Tactic - If you still have lingering doubts about a source, look into how they get their funding, who is on their Board of Directors and, if they are a nonprofit, have a quick look at their tax returns (not hard, go here http://www.guidestar.org) … lots of insight to be gained doing this.
Part Two: Assess the motivation of the source making the claim
This one seems straight forward, but you have to be careful. When examining the motivation of a source, it is easy to fall into the trap of, “Well, of course they would say that, it is in their self-interest”. Self-interest can take many forms, such as a financial stake in the outcome, academic rewards, pressure to conform to an ideological perspective and just plain status/ego/hubris. The tricky problem here though is that many perfectly valid sources have self-interest in the claims they make, or at least can be accused of having so. Therefore, self-interest should never be an immediate disqualification of the validity of a claim. But it does play a role, and needs to be viewed in the context of the other factors. In the recent effort to label GMOs in California, the claims made by the large food industry were attacked in part for their self-interested motivation to maintain the status quo…fair enough. But you had to also understand that many pro labeling folks also had significant self-interest at stake, in this case the growth of non GMO food markets. So yes, once again you’ll have to exercise some judgment, and weigh all the factors. My next stop is the data itself.
This is probably the hardest part for us nonscientists, because if you ever try to look at the actual data underlying any claim, unless you are an expert in the field, you quickly realize that you really don’t fully understand what is being presented…in fact, that’s why we rely on trusted sources to interpret and explain the meaning of the data. So my approach is to assess the preponderance and longevity of the data from qualified sources supporting the claim. Examples…the data to support the claims of human caused climate change is overwhelming and longstanding, while that to support the risks of GMOs is essentially nonexistent. I could go on, but the point is that we need to weigh a number of factors in assessing claims. Sadly, there is no easy or simple approach, but it’s certainly worth the effort.
Yikes, I’ve reached my self-imposed word limit, having no doubt lost most readers already (…sorry gang). I’ll close for now, but to keep you on the edge of your seats, next time I’ll weave in systemic causation, epistemic closure and the limits of science on how they factor in to our beliefs…consider this part one of two…nothing but wacky fun on this blog… [Sic].
So, what do you believe?
