Blinded Knee Doctors

An experiment is called a double-blinded study when both the subjects and the researchers are unaware who is receiving the treatment and who is receiving the placebo.

a) Why is it important for this study to be double-blinded?

b) Describe a treatment vs. control study in which blinding is not possible for the subject.

c) Is it possible for the researcher to be blinded in your example for Part b? If so describe how this could be accomplished. If not explain why it would not be possible.

 

• Understand the advantages of a comparative experiment. Be able to identify the basic features of a randomized comparative experiment.

a) Several of the measurements (for example, level of pain) by both patient and doctor are subjective and may be influenced by knowledge of whether or not the patient was a member of the treatment group.

b) One example would be a study of the effectiveness of a physical therapy or exercise program such as swimming or the use of an exercise bike. In this case the patients would be aware of the treatment and therefore any variables measured requiring their input may be affected by their knowledge of their participation in the treatment.

c) For the examples given in Part b one could blind the researcher who performs the assessment by having someone else perform the therapy or surgical procedure. This would eliminate possible bias in the evaluation of the treatment.

The researchers reported that one patient was removed from the study due to "intolerance of the preoperative study medication (1 patient who was receiving a placebo)" and "slightly more patients who took a placebo reported side effects potentially caused by the study medication" than did patients who received the NSAID. Do these results say anything about the importance or effectiveness of the blinding in this study?

• Understand the advantages of a comparative experiment. Be able to identify the basic features of a randomized comparative experiment.
  

The fact that reported side effects were more common in the control group may indicate that the procedure for blinding was effective and/or the NSAID may in fact limit possible side effects due to other factors present in the story (e.g. codeine pills and the anesthesia used during the operation).

The researchers state:

“The fact that all patients enrolled in the study were not followed the full 8 weeks could potentially bias the results if subjects were lost to followup, depending on which study medication was taken, the NSAID or the placebo.”

Do you agree with this statement? If so, how might this affect the results? If not, why would this fact not bias the results?

• Be able to identify problems with poorly designed experiments.

One possible reason for not participating in the follow-up exams might be that the patients felt the knee had healed enough that further examination was unnecessary. If the majority of these patients came from the control group their absence might exaggerate negative results for the remaining control group members. Conversely, if a majority of these patients were from the treatment groups it might lessen the positive results of the remaining treatment subjects.

The patients were asked to assess the level of pain they were experiencing at each of the exam times. A pain score was based on the patients' marks on a 10-cm line with a range from 0 (no discomfort) to 100 (agonizing pain).Is the pain score a valid measurement of the amount of pain felt by the patients? Explain your answer.

 

• Be able to identify potential sources of bias.

The validity of the scale is questionable since pain is a highly subjective quantity and it would be difficult to assume that similar scores for different people would indicate the same amount of pain. However, repeated measurements over time could serve as a valid indicator of increases or decreases in the level of pain for all of the subjects.

Examine the table of mean pain scores.

a) At which exam time(s) would it be desirable for there to be little or no difference between the three groups? Does this appear to be the case? What does this say about the randomization used to determine the three groups?

b) For one exam time and group use the information provided to describe the likely distribution of the pain scores. Is it plausible to assume the data is normally distributed? Explain your answer.

 

• Understand three basic techniques fundamental to well-designed experiments: randomization, replication, and blocking. Understand that the first is used to decrease bias and that the last two help to increase precision.
• Be able to assess important features of the shape of a distribution by looking at a histogram (single or multiple peaks, symmetric or skewed, the existence of outliers).

 

a) It is important that there is little or no difference between the three groups at the start of the study to eliminate some of the possibility that a factor other than the treatment is responsible for differences observed later in the study. In this case we see that Group C had slightly higher pain scores than Groups A and B at the time of enrollment but there is considerable overlap of all three groups within one standard deviation. If the pain score was the only variable of interest for the study this would suggest the randomization was successful in creating three similar groups of patients.

b) As one example consider Group B on the day of surgery where the mean is 41, the SD is 27, and the range is from 5–89. Comparing these numbers it is evident that all of the observations lie within two standard deviations of the mean, which is near the middle of the range. This implies that the distribution is somewhat symmetric and has few outliers.

As a second example consider Group B at one week after surgery, where the mean is 26, the SD is 23, and the range is from 2–84. This time the distribution does not appear to be symmetric due to the presence of extreme values near the high end of the scale.

Although neither description exactly describes a normal distribution one might consider the data to be from a truncated (tails removed) normal distribution due to the limits imposed by the use of the pain scale. A histogram of the individual scores would help us determine if this is a reasonable assumption.

Suppose you only had information on the pain scores at the exam times listed in the table. For each research question below choose one of the suggested relative comparisons and explain why it may be the best choice for addressing the question.

a) Research Question: Is the NSAID beneficial as a preoperative treatment?

i. Comparison of the pain scores at day of surgery among Groups A, B, and C.
ii. Comparison of the change in pain scores from day of enrollment to day of surgery among Groups A, B, and C.
iii. Comparison of the pain scores at day of surgery between Group A and the combined grouping of Groups B and C.
iv. Comparison of the change in pain scores from day of enrollment to day of surgery between Group A and the combined grouping of Groups B and C.

b) Research Question: Does NSAID aid in the speed of recovery immediately after surgery?

i. Comparison of the pain scores one week after surgery among Groups A, B, and C.
ii. Comparison of the change in pain scores from one day after surgery to one week after surgery among Groups A, B, and C.
iii. Comparison of the pain scores one week after surgery between Group C and the combined grouping of Groups A and B.
iv. Comparison of the change in pain scores from one day after surgery to one week after surgery between Group C and the combined grouping of Groups A and B.

• Be able to conduct a formal hypothesis test for

a) a population mean,
b) a population proportion,
c) association between two categorical variables.

Be able to state the relevant hypotheses, compute the test statistics, and compare the value of the test statistic to the appropriate distribution. Be able to draw a formal conclusion in the language of the problem.

a) One way to assess the benefit of the treatment would be to see if there was a difference in the reduction in pain from time of enrollment up to the time of surgery. Also, since both Groups B and C received the same treatment (i.e. a placebo) up to this point we are not concerned about any differences in these two groups. Therefore the comparison given in (iv) may be the best choice of those provided to address this research question.

b) The property of interest, speed of recovery, may be addressed using the differences in pain scores from one time to another. This time both Groups A and B received the same treatment (i.e. the NSAID) following surgery. However, their treatment differences before surgery may be confounded in the results if we combine these two groups. Therefore the comparison given in (ii) may be the best choice of those provided to address this research question.