Healthcare, etc. has a great post deconstructing what it means to say a treatment works:
What exactly does it mean when we say that a treatment works? Do we mean the same thing for all treatments? Are there different ways of assessing whether and how well a treatment works? I am sure you’ve guessed that I wouldn’t be asking this question if the answer were simple. And indeed, the answer is “it depends.”
What I am talking about is examining outcomes. I did a post a couple of years ago here, where I use the following quote from a Pharma scientist:
“The vast majority of drugs – more than 90 per cent – only work in 30 or 50 per cent of the people,” Dr Roses said. “I wouldn’t say that most drugs don’t work. I would say that most drugs work in 30 to 50 per cent of people. Drugs out there on the market work, but they don’t work in everybody.”Here is that word “work” again. What does this mean? well, let’s take such common condition as heart disease. What does heart disease do to a person? Well, it can do many things, including give him/her such symptoms as a chest pain, shortness of breath, dizziness and palpitations, to name a few. These symptoms may have at least two sets of implications: 1) they are bothersome to the individual, and in this way may impair his/her enjoyment of life, and 2) they may signal either a present or a future risk of a heart attack. Why are heart attacks important? Well, they are important because one may kill the person who is having it, or one (or several) may weaken the heart to the point of a substantial disability and thus a deterioration in the quality of life. So, there certainly seems to be a good rationale to prevent heart disease either from happening in the first place or from at least worsening when it’s already established.
Now, what’s available to us to prevent heart disease? Well, some think that lowering one’s cholesterol is a good thing. OK, let’s go with that. What is the sign that the statins (cholesterol-lowering drugs) “work”? What would it look like if it was about lowering the cholesterol? Say, your total cholesterol is 240. You go on a statin and in 6 months your total cholesterol is 238. Your cholesterol was lowered, it worked! Well, yes, but if you are asking what this 2-point drop really accomplishes, you are beginning to understand the meaning of “work.” So, just intuitively we can say that there needs to be a certain, perhaps “clinically significant,” drop in the total cholesterol in order for us to say that the drug “worked.”
Great! Now we are sidling up to the real issue: What constitutes a “clinically significant” drop in cholesterol? Is it some arbitrary number that looks high enough? Probably not. How about some drop that correlates to a drop in the risk of the actual condition we are trying to impact, heart disease? Say, a 40-point drop, or getting to below 200, may be the right threshold for the “works” judgment. Ah, but there is yet another question to ask: How often does this type of a drop lead to a reduction in heart disease? Is it always (not likely), or is it the majority of the time (rarely) or at least some of the time (most likely in clinical medicine)? And what portion of that time do we consider satisfactory — 60%? 40%? 20%? 2%?
There’s more. Read the rest here.