For more detailed and statistically sound analysis, check out Science and Engineering Indicators 2008, which I posted about last year. It states “[...] many Americans do not reliably distinguish between scientific and nonscientific knowledge claims.” See my linked post for the quote and a few stats if you don’t want to go immediately to the full report.

Hat tip to Decrepit Old Fool’s “A depressing score“, but there are no comments there. He picked it up from Greg Laden’s “I am, like, really smart and stuff“, where you’ll find plenty of commentary.

So, is the sky really falling? Today is one of those days when I’m feeling too old and tired to rant about any of this … after all, what’s a few hundred years of darkness ^[1] now and then in the broader sweep of history?

image: Hannes Grobe, Erlenmeyer flask hg.jpg, Wikipedia

1. yes, I did read the Wikipedia article; can you say snark?

Science is the testing of ideas.

This is subject to ever-increasing refinement (or contention), as you can see from his post and its comments. You might consider the “scientific method” as the first-level refinement:

• observe
• hypothesize
• predict
• test

That’s basically the inner loop of the scientific method. You define a question based on your curiosity or a problem to be solved, then organize what you already know about it to bootstrap the loop. You form a hypothesis that predicts a testable result. You perform experiments or make observations that generate data which you analyze and interpret to determine the accuracy of your hypothesis. You refine your hypothesis or generate a new one. Rinse. Repeat. Meanwhile, you publish your results, and other scientists try to repeat your experiments

It’s important to keep a couple key concepts in mind. First, the prediction should be falsifiable; that is, you must make a prediction that can be tested for failure. If hypothesis H implies prediction P, you can’t “prove” H by observing that P is true — P could be true for entirely independent reasons (like the old “keeping the elephants away” joke). However, if H implies P, and testing P “fails”, then you know that H is incorrect. As Einstein said (paraphrased), “No amount of experimentation can ever prove me right; a single experiment can prove me wrong.”

Second, your experiments should be repeatable, so that they can be independently verified. That is, there must be evidence that your results weren’t just a fluke, a result of an error in procedure or analysis, wishful thinking, or deliberate deception. Can you say “cold fusion“?

This methodology is also useful beyond scientific research — the observe-hypothesize-predict-test loop is the best approach to both trouble-shooting and tuning systems. Of course it doesn’t take into account the malevolent stochastic uncertainty of certain operating systems *cough*Windows*cough* that makes you throw up your hands in desperation and reboot … or wipe the disk and start over from scratch.

45% – Lasers work by focusing sound waves. (False)
56% – Antibiotics kill viruses as well as bacteria. (False)
76% – Does the Earth go around the Sun, or does the Sun go around the Earth? (Earth around Sun)
55% – How long does it take for the Earth to go around the Sun: one day, one month, or one year? (One year)

I’m not even going to get into the evolutionary or cosmological questions. The report also states:

The large numbers of Americans who regard astrology as at least somewhat scientific is an indicator that many Americans do not reliably distinguish between scientific and nonscientific knowledge claims. Available national data cannot differentiate those who misapply what they think are scientific criteria from those who in some respects reject conventional scientific criteria, even though they are familiar with them.

– “Science and Engineering Indicators, published by the National Science Board, provides a broad base of quantitative information on the U.S. and international science and engineering enterprise.”