What Scientific Ideas Are Ready for Retirement?

Two of the many fine essays in this year’s Edge question have direct bearing on the way developmental vision, vision therapy, and vision rehabilitation are practiced in the real world.  The truths they hold are self-evident.
CarrNicholas Carr:  Anti-anecdotalism

We live anecdotally, proceeding from birth to death through a series of incidents, but scientists can be quick to dismiss the value of anecdotes. “Anecdotal” has become something of a curse word, at least when applied to research and other explorations of the real. A personal story, in this view, is a distraction or a distortion, something that gets in the way of a broader, statistically rigorous analysis of a large set of observations or a big pile of data. But as this year’s Edge question makes clear, the line between the objective and the subjective falls short of the Euclidean ideal. It’s negotiable. The empirical, if it’s to provide anything like a full picture, needs to make room for both the statistical and the anecdotal.

The danger in scorning the anecdotal is that science gets too far removed from the actual experience of life, that it loses sight of the fact that mathematical averages and other such measures are always abstractions. Some prominent physicists have recently questioned the need for philosophy, implying that it has been rendered obsolete by scientific inquiry. I wonder if that opinion isn’t a symptom of anti-anecdotalism. Philosophers, poets, artists: their raw material includes the anecdote, and they remain, even more so than scientists, our best guides to what it means to exist.

KleinGary Klein: Evidence-Based Medicine 

Any enterprise has its limits and boundary conditions, and science is no exception. When the reach of science moves beyond these boundary conditions, when it demands respect and obedience that it hasn’t earned, the results can be counter-productive. One example is Evidence-Based Medicine (EBM), which is the scientific idea that I think we should retire.

The concept behind EBM is certainly admirable: a set of best practices validated by rigorous experiments. EBM seeks to provide healthcare practitioners with treatments they can trust, treatments that have been evaluated by randomized controlled trials, preferably blinded. EBM seeks to transform medicine into a scientific discipline rather than an art form. What’s not to like? We don’t want to return to the days of quack fads and unverified anecdotes.

But we should only trust EBM if the science behind best practices is infallible and comprehensive, and that’s certainly not the case. Medical science is not infallible. Practitioners shouldn’t believe a published study just because it meets the criteria of randomized controlled trial design. Too many of these studies cannot be replicated. Sometimes the researcher got lucky and the experiments that failed to replicate the finding never got published or even submitted to a journal (the so-called publication bias). In rare cases the researcher has faked the results. Even when the results can be replicated they shouldn’t automatically be believed—conditions may have been set up in a way that misses the phenomenon of interest so a negative finding doesn’t necessarily rule out an effect.

And medical science is not comprehensive. Best practices often take the form of simple rules to follow, but practitioners work in complex situations. EBM relies on controlled studies that vary one thing at a time, rarely more than two or three. Many patients suffer from multiple medical problems, such as Type 2 diabetes compounded with asthma. The protocol that works with one problem may inappropriate for the others. EBM formulates best practices for general populations but practitioners treat individuals, and need to take individual differences into account. A treatment that is generally ineffective might still be useful for a sub-set of patients. Further, physicians aren’t finished once they select a treatment; they often have to adapt it. They need expertise to judge whether a patient is recovering at an appropriate rate. Physicians have to monitor the effectiveness of a treatment plan and then modify or replace it if it isn’t working well. A patient’s condition may naturally fluctuate and physicians have to judge the treatment effects on top of this noisy baseline.

Sure, scientific investigations have done us all a great service by weeding out ineffective remedies. For example, a recent placebo-controlled study found that arthroscopic surgery provided no greater benefit than sham surgery for patients with osteoarthritic knees. But we also are grateful for all the surgical advances of the past few decades (e.g., hip and knee replacements, cataract treatments) that were achieved without randomized controlled trials and placebo conditions. Controlled experiments are therefore not necessary for progress in new types of treatments and they are not sufficient for implementing treatments with individual patients who each have unique profiles.

Worse, reliance on EBM can impede scientific progress. If hospitals and insurance companies mandate EBM, backed up by the threat of lawsuits if adverse outcomes are accompanied by any departure from best practices, physicians will become reluctant to try alternative treatment strategies that have not yet been evaluated using randomized controlled trials. Scientific advancement can become stifled if front-line physicians, who blend medical expertise with respect for research, are prevented from exploration and are discouraged from making discoveries.

7 thoughts on “What Scientific Ideas Are Ready for Retirement?

  1. Dear Lenny,

    Thank you for your post. It is obvious that EBM has many flaws. It is the rule until it conflicts with the doctor’s own clinical insights/bias. The CITT study is a perfect example of this phenomena. Despite the joint efforts of optometry and ophthalmology, there are still critics of the concept that an office based vision therapy program is the best choice in treating convergence insufficiency. As you are aware, Dr. David K. Wallace wrote a rebuttal to the study, which was imbedded in the original article. He felt that a more intensive home based program would solve the problem. I do not know the specifics but it could be a sexier pencil or more cycles at a higher rate of speed but approaches, which have had no scientific basis. Ironically, some insurance companies are NOW requiring that patients undergo four to six weeks of unsuccessful home based vision therapy before covering an in office program. How many patients want to undergo a sham program before receiving the proper treatment? I guess that we would need to ask Dr. Wallace and his band of ophthalmologic skeptics that question.

    The number game has already become an integral part of our health care system. With EMR, we have Big Brother/Big Sister looking over our shoulder. Drug companies have found away around the research dilemma. They publish WHAT is helpful and they bury WHAT is counter productive. In other words, they hid the evidence. They use ghost writers and television commercials to promote their drugs. Ironically, many of the newer drugs are not better than the older drugs except that they are more expensive. And then the drug companies increase the drug prices across the board. They also keep the time frames narrow and test only one to one relationships in the effects of their drugs regimens. My final comment to this on going catastrophe is HOW many of these patients are taking another or another or many other drugs to treat multiple medical conditions for long periods of time or for a life time, which are not considered in the original research because it makes the equation too difficult to prove.

    Thanks again for sharing these opinions/perspectives

    PS The recent controversy in the areas of cholesterol and high blood pressure guidelines is another example of this INSANITY


  2. Spot on, Rich. I fully agree that hypocrisy with EBM is no more evident than related to CITT. I’ve noticed a trend lately that when submitting claims for VT on behalf of patients diagnosed with CI, the carrier writes a letter back that they will only pay for office-based therapy if 12 visits of home-based therapy has been tried first. Can you think of another treatment for which an insurance company would deny a claim because placebo therapy wasn’t tried prior to the therapy for which the claim is being submitted?

  3. And where is the research, which has shown the efficacy of this approach. The bottom line is that the new health care system from an insurance perspective is to charge higher premiums, increase the number of tests and procedures which are no longer covered, raised the level of deductions before insurance coverage kicks in and then claim that we have the best health care system in the world. The problem is that our system is poorly rated compared to many countries, which would be considered third world.

    It is time to get back to the basics. It is about patient care. It is about improving the quality of our patient’s lives. It is not about being bullied by insurance companies, pharmaceutical companies and medical device companies. The latter two groups have been fined billions and billions of dollars, which have not impacted their bottom line. The use of robots as surgeons has also been a disaster. Maybe tomorrow, I will wake up and the nightmare will be over. Thanks again for focusing on these important issues, which we are confronting every day in our flawed and corrupt health care system.


  4. Len, thanks for posting these articles. The very idea of imposing EBM is so fraught with danger to patients that as a well informed patient, I break out in a sweat whenever the topic comes up. The late Charles Margach once said that the most useful (and often most scietific) research were individual case studies that provided guidelines by which you could predict the outcome of a particular treatment in a particular patient. In my own case, EBM said that my throat cancer was best treated by use of radiation and Cisplatin, a 30 year old drug. This despite clear evidence that that drug is extremely detrimental to the health of older patients. Another drug, which is much easier on older people had shown results equal to Cisplatin, but with less damage to kidneys, hearing and other delicate organ tissues. The process by which I could have insisted on that new drug would have significantly delayed treatment and in the face of many oncologists, I began with the old, and incidentally, much cheaper Cisplatin. The issue you and Richard illuminate here has such a profound impact on quality of care–heck, even availability of care–that I hope the debate becomes a public issue. Allowing the pseudo science of EBM to persist also suggests that new, less traumatic treatments will not be available because they have not been supported yet by a double blind, controlled study. Thanks again for posting this.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s