Is Evidence Based Practice Making Us Stupid?

[Barrett Dorko]

I’ve been reading and writing a number of things lately, and, predictably, none of these things has reached the level of coherency in the writing that I require before I put it out there.

Having read Brain Rules recently I knew I couldn’t multitask like this (part of rule #4) and expect to get much done, but I’m of the opinion that flooding yourself with information in this manner will help you make connections between seemingly unrelated ideas that you wouldn’t ordinarily see. At least, this is how it works for me.

I was pointed toward an article in the current issue of Atlantic Monthly titled Is Google Making Us Stupid? by Nicholas Carr that I’ve gone on to read very carefully. As does often happen, it came along at precisely the right moment and I want to use it to build various aspects of this thread. You can find it here.

Let’s begin with this:

Carr cites a change in Nietzsche’s writing that can be traced directly to his acquisition of a typewriter in 1882. His prose became tighter, almost as if he were writing telegraphs instead of essays. When his friends pointed this out he said, “You are right, our writing equipment takes part in the forming of our thoughts.”

Please consider this, read the article when you’ve the time and see if you can guess where I’m going next.

More soon.

[oljoha]

Hey Barrett, is that Brain Rules book worth buying?

[Barrett Dorko]

Ole,

No, not really.

See how I got your buddy Nietzsche into the first post?

[Diane]

Have you seen this Barrett? I found it here.
Argh.

[oljoha]

Originally Posted by Barrett Dorko

See how I got your buddy Nietzsche into the first post?

Yes - I like Nietzsche.

[Jason Silvernail]

I think this post from EIM belongs here.
EBP, Deep Models, and Scientific Reasoning

[Barrett Dorko]

Diane,

Your references are precisely on point. An interesting list of comments accompanying the article as well.

Carr's contention in the originally referenced Is Google...? article is that the way we now find and familiarize ourselves with information has lead to a literal change in the way we read, think and, consequently, act. If we surf the web enough - seeing snippets here and then there, and then quickly over there - we will often find ourselves unable to read any tome from cover to cover as we carefully consider the deeper implications of its message. Some regular visitors here might already be aware that this has happened to them.

In short, the machine has changed us by virtue of the way it thinks, not the way we think.

[Mary C]

The medium is the message???

Marshall McLuhan I believe

[nari]

Television is also a culprit in encouraging a short attention span. Lots of information thrown at you a few seconds at a time; constant scene-changing. At least with the net and hard copy newspapers we have a choice: we can linger and contemplate, but kids are not likely to do that, anyway.

So if we don't take time to connect the dots properly, we become ambulant dictionaries perhaps, with little or no ability to understand anything other than on a superficial level, and that is dependent on how the modality serves it up to us.

Nari

[EricM]

I had thought that it was either a) the length of time since I was in school, or b) the fact I'm getting older, that was responsible for the noticeable decline in my attention span. I'm not sure I'm relieved to learn of this other contributer, but it does make sense. I rarely read any more than about 3/4 of any post on the first run through. I also find it increasingly difficult to compose these posts. I'd like to be more thoughtful and thorough but am too frequently distracted by something else to take the time to accomplish even this small a task. You've no idea how many pots of rice I've burned on the stove in the last year! As much as I'd like a remedy, I'm not so sure I'd follow through with anything.

Some regular visitors here might already be aware that this has happened to them.

You got me.

[Diane]

I haven't noticed any change in the depth to which I can sink into a topic, and I feel my ability to write has actually improved...mostly by writing lots....
But there has been one effect I've noticed, which I don't know is a bad thing or a good thing yet, which indicates to me how deep into my brain the internet has sunk and how it has taken over pathways: it is in dreaming. I rarely dream about my physicality anymore - i.e., walking down a street, encountering other people in my dreams in an ordinary physical way the way one does in waking life. Instead it's like I watch dreams on a screen, and can change them by opening a new window, or shut them off if I don't like them.

[Barrett Dorko]

Well, you guys are already way ahead of where I thought we’d be at this point. Jason saw that this is yet another run at the weaknesses inherent to the current notions surrounding evidence based thinking as it has evolved in our profession, and he’s exactly right about that. I’d recommend looking at the link he provided. This thread might even be linked to that eventually.

Steven Novella, a neurologist, recently wrote a bit about his own problems with EBM on his blog, Neurologica. Here he writes at length in response to what he refers to as a “crank” who is wildly critical of his skeptical view:

The goal of EBM, ironically, is to focus on evidence for efficacy - not mechanism of action. In fact…that is my primary criticism of EBM, that it does not adequately consider mechanism of action when dealing with highly improbable therapies.

So, as you can see, those of us who have been speaking critically of this specific aspect of EBM thinking have a powerful ally.

[Jon Newman]

Well I'll tell you one thing. Surfing the net and writing quick discussion board posts DOES NOT prepare you for the writing portion of the GRE. I know. Twice.

[nari]

Jason's link is certainly relevant.
The problem with EBP (which is well recognised by those who are wary of its consequences) is the risk that a patient has to 'fit' into a certain objective category and therefore the therapist will perform this technique.
Many patients do fit into categories such as the CPR for manips, but what about individual differences? Take them away and you have recipe-induced thinking which tends to make us technically correct but not much else. And as we know, that is not how therapy should be.

Eric, I do the same thing; some posts I skip altogether and return a day or so later when my brain is in an academic mood. Maybe that is laziness, and/or only a remote interest in the thread especially when a link is a zillion pages long.

What I find myself doing is going to Barrett's threads/posts first; and I think this is because I can understand them immediately, and they then assist in delving into other posts, sometimes. No offence intended to those who usually write almost dopamine-inducing posts.

Nari

[Luke Rickards]

I wonder, is it EBM itself that makes us stupid, or does it just distract us?

Perhaps a closer analogy would be, "Does abstract-only reading make us stupid?"

[Barrett Dorko]

In this podcast a few weeks ago Susan Jacoby decries the effect that Google has upon our basic knowledge. Knowing that I can simply bring up a list of The Bill of Rights may obviate my need to know why and how they exist. Then I go ahead and vote for someone who is perfectly happy to violate them without understanding the context of this behavior.

Similarly, following the dictates of “best practice” by proceeding with treatment demonstrated as effective precludes my need to explain anything to the patient, family or referral source. It is simply anti-intellectualism as applied to therapeutic practice, and, in a very real sense, unreasonable – because reasoning isn’t a part of the process.

What keeps running through my mind is this: EBP massively retards the progression of practice in light of new evidence from neuroscience, and when these two collide (EBP and neuroscience), someone’s going to lose.

So far, it appears to be us.

[Jon Newman]

Good point Luke. It seems we're getting back toward what the practice of EBM (or evidence based allied health ) requires of those practicing it. Beyond simple abstract reading, I don't think knowing guidelines, CPRs, etc would count as EBM either. Perhaps a Venn Diagram would have two circles with EBM circle being almost, but not entirely, overlapping the Medicine circle. Those in the non-overlapping portion of it may practice EBM but not Medicine. I've seen the term "evidence only practice" and similar phrases but I'm not sure who to credit for that.

Still, on Steven Novella's take there is an inadequate consideration of mechanism of action (I'm not sure why he limits the criticism to highly improbable therapies). This won't be addressed by reading whole journal articles versus abstracts if the journal articles don't address mechanism. Perhaps Novella limits his criticism to improbable therapies because the more probable therapies already have a mechanism of actione established elsewhere. Regardless, in order to fund their work researchers have to ask question worth answering in the eyes of those funding the work. Mechanism of action may not be the right question. Or the question is being asked out of order.

[Mary C]

What I find frustrating is the reductionism. It seems they over-simplify. Or maybe they over-generalise. There's a lack of consideration for all the other factors that I have to take into account when faced with the reality of an individual patient.

[matchupitchu]

Good afternoon

First I would like to apologize for my english level and hope that my post will be understandable!

I've already noticed these difficulties when reading books and novels, mainly by lack of attention. I can't concentrate enough anymore for long time readings because my mind overflows with hundreds of various thoughts at the same time (like popups...) and at the end, it's difficult to remind the gist of what I read....
However, beeing aware of this phenomenon might help dealing with it? How can we switch off the "Google mode" in our brain in order to refind our capabilities of concentration? Is there a way of "defragmenting" our inner "hard drive"??

[Luke Rickards]

Mechanism of action may not be the right question. Or the question is being asked out of order.

Jon, that is exactly the argument. The first three minutes of this discussion between Dawkins and Randi is very relevant here.

Often, but not always, there is a difference between the hypothetical mechanism (theory) and the actual mechanism. Dawkins explains that the most appropriate way to proceed is to firstly show whether the phenomenon predicted by the theory exists or not, ie "does it work"? Just because something should work, doesn't mean that it does. Trying to understand how something works, or whether it works the way we think, may be a waste of time if it doesn't actually work. After it is established that there is a phenomenon, then understanding the mechanism can come closer to the fore.

Understanding the therapeutic mechanisms behind physical treatments is notoriously difficult due to the complexity of interactions between multiple local and systemic systems. Nevertheless, I would say that studies which fail to at least attempt an address of mechanisms in the discussion are a minority. We might also consider that research which rules out possible mechanisms, rather than seeking positive proof, might also be included as applying EBP, such as the multitude of studies challenging the validity and clinical relevance of impairments detected via manual diagnosis (many can be found in the Red File). This process is even built into the RCT, which pits everything against the 'gold standard' mechanism present in the outcomes of all treatments - placebo.

It was Nic Lucas who coined the term "evidence-only practice", and I think it is important to differentiate it from evidence-based practice here. As Sackett states, EBM does not advocate cookbook medicine. It is supposed to be a reasoned process based on an understanding and integration of a number of sources, including professional experience and the considerations of the patient. Based on this, I don't actually see it as exclusive of mechansims or ethical communication with the patient. That doesn't mean I think there are no inherent limitations to EBM. I suggested 'distracting' above because the danger is that efforts to demonstrate that there is a reproducible phenomenon, ie outcome studies, can easily be regarded as higher yielding in justifying the activity, and research budget, of applied sciences (PT, osteo etc), and the final step of discovering the mechanisms (basic science) is then forgotten.

[Luke Rickards]

It seems we're getting back toward what the practice of EBM requires of those practicing it

Jon, just another point related to this - basing one's practice on evidence requires the ablity to determine if data actually offers evidence or not. The internal validity of a study is not ensured just because it is published. There is actually a lot of poor quality research out there and if you don't have the skills to identify it (see here) then the evidence you base your practice on might not be evidence at all.

[Jon Newman]

Great references Luke. I'll be sure to plumb the depths rather than skim the surface of those.

[Barrett Dorko]

The things we hold, manipulate, visually consider and remember can and will alter us, whether we know it or not. This is the primary premise behind Carr’s article and it still haunts me.

I may be off base here, but isn’t that also the basic premise behind NDT?

It’s all about external stimulation and the brain and not about external stimulation and the effect on, say, the connective tissue.

[Luke Rickards]

That might be true, Barrett - so would basing clinical practice on the evidence supporting it make one stupid?

[EricM]

Matchupitchu asked:

How can we switch off the "Google mode" in our brain in order to refind our capabilities of concentration?

By first becoming aware of the problem, assuming ownership of it and becoming personally accountable to yourself. The same would apply to anyones interpretation of evidence as well.

[matchupitchu]

Originally Posted by EricM

By first becoming aware of the problem, assuming ownership of it and becoming personally accountable to yourself. The same would apply to anyones interpretation of evidence as well.

Thank you Eric, I think we should consider internet just for what it is, just a tool, that's all!
I became aware of this problem and the probable link with internet "addiction" by reading Nicholas Carr's article (the full text and in full conscience, uh that was hard!).

[Luke Rickards]

Matchu,

The trouble, as Barrett has drawn attention to, is that using the tool changes us, whether we realise it not. There's some great discussion of this in the Blakeslee's book - The Body has a Mind of its Own (see here). As Eric suggests, our best defense against the repercussions of this, if they are undesirable, might be awareness and responsibility. With regards to the main content of the thread, I guess we need to ask the question, "Is the tool responsible for current disturbing trends in clinical practice actually EBP, or is it some other tool mistakenly wearing an EBP label?"

[Diane]

I very much liked Alan Dove's reply to the article I posted about in post #4:

The main problem is that extrapolating from Google Analytics to model-free science leaves out science’s key contribution.

Scientists don’t just see and predict correlations, they develop robust organizing principles, called theories, which provide vastly greater intellectual leverage than any merely correlative model. Theories are tools to accomplish a particular purpose, yes, but good theories also become something more, with uses that may extend far beyond what their original developer intended.

A Google-style system might (eventually) be able to take all of the variations in finch beaks, beetle wings, and pigeon plumage and predict that species in general will vary within certain parameters. But if you want to do away with theories, that’s not nearly enough. The same machine would also have to construct a complete but succinct summary of Darwinian natural selection and speciation. Then, if you really want to prove the point, have the machine use that summary - and nothing else - to explain something that seems to conflict with it. Male nipples, for example.

The same is also true in every other hard science. Don’t let the engineers tell you otherwise.

My bolds.
Alan Dove is a science writer. Here is a link to his site.

[EricM]

Steven Shapin's The Sociology of Scientific Knowledge is an interesting read whose topic seems relevant.

The political vulnerability of one of the few sociological specialties that, so to speak, “studies up,” that aims to interpret a culture far more powerful and prestigious than itself, and that offers accounts at variance with that culture’s official myths, is only now being made manifest. As the Chinese proverb has it, he who rides on the back of the tiger may wind up inside.

Perfect description of the risks inherent to the purpose of this thread.

[Diane]

This book is probably worth a read. I watched the author interviewed by Steven Colbert.

[Jon Newman]

Thank you Eric, I think we should consider internet just for what it is, just a tool, that's all!--Matchu

I think one of the important points of this thread is that there is no such thing as "just a tool." Using the tool changes your synapses. Whether it is for the better or worse or whether it makes us stupid, is open to debate.

I think addiction is an issue for some and not for others but addiction or not, it changes us. It is part of our environment. It is part of our culture. Even if you don't use it the people you interact with do and that will affect you. Resistance is futile.

[gilbert]

Jon

I disagree with the "Resistance is Futile" part. The rest of your post is right on.

I think our 'resistance', or the attitude and the choices we make about whether and how we use the medium makes a huge difference. Personally I experienced noticeably reduced attention span and more difficulty with comprehension after starting to watch TV during college (I was blessed to grow up almost TV-free). I also find that the "Google" level of information tends to be a lot of "surface" type facts about a topic and learning this way often misses out on deeper understanding as well as a whole layer of experiential learning that is just not possible via computer.

Of course, in spite of the limitations SS is a tremendous learning tool, and much appreciated!

[EricM]

For my part in changing anyone's synapses with links to articles like the one above, I accept full responsibility.

[Jon Newman]

Gilbert I admire your optimism. I think it's important to clarify that I think resisting the process is futile and that I'm not suggesting some sort of fatalism. I think resisting the process would be like resisting the second law of thermodynamics.

I agree that understanding the process and using it for good (the determination of which is the hard part) may give us some of the control of which you speak. And if it doesn't, it least gives us the illusion of it and sometimes that's enough.

[gilbert]

Well, I don't think it's futile. Of course the current Google culture affects us, but once we are aware of its effects we can DECIDE to some extent how much we want to let it influence us. After all if something is making us stupid, we can choose that we don't WANT that to happen. Self-preservation or something like that....(BTW, I don't watch TV any more)

On the Second Law of Thermodynamics, I don't especially get the connection, but remember that although the overall tendency toward entropy in the universe can never be violated, it is quite successfully "resisted" at a local level by living things which can become more ordered.

[Barrett Dorko]

It has occurred to me that I wrote something about thoughts generated by the acquisition of data without context but I’ll be darned if I can find the old column (titled Running to the Delta Quadrant) from a defunct magazine where it appeared. What to do…what to do…

Wait! Google! Of course!

I knew that it concerned an episode of Star Trek Voyager and I knew the plot, so a simple search resulted in my finding The Voyager Conspiracy; a complex examination of the consequences of downloading too much information without understanding enough about its meaning. It just happens to star Seven of Nine, my favorite character. Pure coincidence.

In yet another brilliant performance Seven decides to familiarize herself with the entire history of the spaceship’s journey and the behavior of its inhabitants. But this sudden wealth of information leads her to believe that a vast conspiracy is afoot and that she has been the unwitting target. Poor Seven!

Anyway, this is a wonderful example of how important context becomes when behaviors are critically observed, and what Google and EBP don’t offer enough of its clearly defined context. I’ve written about that here. This is a story I still tell my patients when discussing their sensation of pain.

And all of this after Jon says Resistance is futile. How cool is that?

[matchupitchu]

I agree with Gilbert but in my point of view, "resistance" might be more passive (awareness, acceptation and assimilation) than active (struggle). If the tool does change us (and I accept that), its first function is to serve us not to control us. That's what I meant by "It's just a tool!"

Originally Posted by Luke Richards

"Is the tool responsible for current disturbing trends in clinical practice actually EBP, or is it some other tool mistakenly wearing an EBP label?"

What are these disturbing trends?

[Diane]

On the Second Law of Thermodynamics, I don't especially get the connection, but remember that although the overall tendency toward entropy in the universe can never be violated, it is quite successfully "resisted" at a local level by living things which can become more ordered.

It is thought the second law is not so much resisted by open systems such as life so much as life is a dupe for the second law by helping decrease environmental gradients toward entropy. I'm kind of in the "resistance is futile" group.

The only way to not have one's brain be googlized is probably to stay off the internet entirely... but then one would have to put up with all the boredom and ignorance life can offer instead. My own wishful thinking makes me think that because I started letting my brain be googlized at a relatively advanced age, maybe it won't get AS googlefied as ones that start out at a tender age, like 5 or 6. I say 'wishful thinking' because there is such abundant neuroplasticity available all the way to the grave.

Seven of Nine was always my fave too. (Used to marvel that she could sleep standing up and not get swollen ankles or hemorrhoids.)

[EricM]

The bloggers are catching up with us. Consider this post from Mindblog, (one the longest from Bownds in a while).

[Jon Newman]

Hi Gilbert,

I don't actually think we're in any real disagreement here.

On the Second Law of Thermodynamics, I don't especially get the connection

I used the second law as a simple example of inviolable process. I could have used other examples.

Barrett, exactly and "very cool", to answer your question. In a single post you recapitulated the thread.

[gilbert]

Diane

One small question:

Was everyone bored and ignorant prior to Google?

[nari]

In pre-Google days one used libraries, radio and television for entertainment and learning; Google certainly made a lot more information available quickly without travelling to a library.
However, as with books, it is what one does with information that counts. It's fairly useless unless it metamorphises into validated knowledge, and that is a process that eludes many. EBP probably intended to do that, but it got waylaid in the clinical reasoning process used by some.

Nari

[Diane]

Was everyone bored and ignorant prior to Google?

Probably not. Some people read, and for the rest there was TV. Or sports on TV. I read (but not as much as I do now) science, and had a love-hate relationship with TV. There certainly was some deep and only vaguely sensed need in me that was stimulated/fulfilled by getting on the internet. It's been the only addiction I've ever actively maintained, because so far it hasn't become boring.

[Barrett Dorko]

From Carr’s Atlantic Monthly article:

The kind of deep reading that a sequence of printed pages promotes is valuable not just for the knowledge we acquire from the author’s words but for the intellectual vibrations those words set off within our own minds. In the quiet spaces opened up by the sustained, undistracted reading of a book, or by any other act of contemplation, for that matter, we make our own associations, draw our own inferences and analogies, foster our own ideas. Deep reading, as Maryanne Wolf argues, is indistinguishable from deep thinking.

I think that this is pretty good. It may give us a clue to the antidote for the Googlemind.

Read deeply.

[physio-philosopher]

In "Overview of Clinical Trials in Medical Rehabilitation," Fuhrer highlights the difference between "pragmatic" and "explanatory" trials. Though they are not entirely mutually exclusive, the former seeks to determine whether an intervention works (or works better than another intervention), whereas the latter seeks to test theory or elucidate underlying mechanisms for treatment response. Explanatory trials are more selective, interventions are more narrowly defined, and they often use a placebo control group. He notes the atheoretical (i.e, pragmatic) character of many rehab studies and proceeds to highlight the features of a good treatment theory:

1) the functional problems on which the intervention is intended to impact
2) characteristics of individuals that make them appropriate for the intervention
3) critical features of the intervention that are ostensibly responsible for intended outcomes
4) elements and contingencies in the causal chain connecting interventions with outcomes
5) expected changes in recipients' status

Personally, I foresee significant difficulty in satisfying these criteria in most research situations due to the enormous variability that exists across individuals, and I look very skeptically on the commonly used notion of "homogenous" groupings.

[Diane]

Here is the thread on academic width as opposed to depth. I think it belongs here, because I think googlization of the brain has resulted in a breach of academe.

[gilbert]

Barrett

Your last post was perfect and I agree with the instruction to 'Read Deeply'

Apologies if the back and forth about the internet took the thread away from the main point, which is related to EBM (and so-called EBM) and how it affects our thinking and practice. This is something that concerns me.

My feeling is that there is a definite tendency toward "evidence only" practice in physiotherapy, with many who are only willing to consider RCTs and Systematic Reviews. This does, in my opinion, dumb down our profession. I fully agree that clinical practice needs to be informed and guided by research, but we also need to be able to reason and to form our own conclusions based on our knowledge of the basic sciences and our experience observing, touching, and treating patients. This points back to earlier discussions of the Deep Model, and the importance of Theory to guide clinical practice.

-Gilbert

[nari]

I agree with all that Gilbert wrote.

EBP/M can stifle creativity, the injection of one's own deep clinical reasoning into clinical practice. Also the ability to look for flaws in studies/RCTs can be inhibited. If very few or no flaws are detected, then the studies becomes useful for further reference or to counter erroneous perceptions.

Nari

[Jon Newman]

The kind of deep reading that a sequence of printed pages promotes is valuable not just for the knowledge we acquire from the author’s words but for the intellectual vibrations those words set off within our own minds.--Barrett citing Carr

I recently audited a course that required weekly quizzes largely focused on minutiae (and some trivia) in the assigned reading. The last assingned reading of the course was available as an audiobook and I decided I would "read" the book on my way to work and back each day. My quiz performance notably dropped.

I also note that I print things I find online that require my close attention for one reason or another.

While I think it is certainly possible to deeply attend to the spoken and digital word, turning that sequence of printed pages seems to trigger a different neural network within me.

[Luke Rickards]

turning that sequence of printed pages seems to trigger a different neural network within me.

Jon, the same thing. If I'm serious about any piece of writing, it now gets printed out before I look at it.