Rapid critical appraisal of an RCT Step 3

Step 3: What do the results mean?

In the article by Spink, M.J. et al., the difference in the results between the treatment (intervention) group and the control group tell you how large or small the effect of the intervention is. Such an effect may be due to the intervention or treatment (in this case the multifaceted intervention package) or simply due to chance. The falls rate for each group was calculated and compared. The ratio (0.64) equates to a 36% reduction in falls rate for the intervention group compared to the control group (see page 4).

Click to reveal the answers.

Could the effect have been due to chance?

The probability that a result is due to chance is normally assessed statistically by means of the p-value and the confidence interval (CI).

Question 1	In regard to the falls rate ratio 0.64, the 95% confidence interval is quoted as 0.45 to 0.91 (Table 2, page 5). What does that suggest?
Answer	The 95% Confidence Interval (CI) is narrow and does not include 1. We can therefore be confident that there is a 95% chance that this range of values includes the real value.

Question 2	In regard to the falls rate ratio 0.64, the p value (probability that the result is purely due to chance) is 0.01 (Table 2, page 5). What does that suggest?
Answer	The p value is less than 0.05 so is said to be statistically significant - that is, a very low chance that the result is due to chance.

What measure was used and how large was the treatment effect?

The true measure of effect of the treatment or intervention is often expressed as the Relative Risk (RR).

Take another look at Table 2, page 5 where Relative Risk (RR) is calculated for the second and third outcome measures: 0.85 and 0.63 respectively. RR tells us how many times more likely it is that an outcome (for example one or more falls) will occur in the intervention group compared to the control group.

This means for example that people in the intervention group were 0.85 times more likely to fall more than once in a 12-month period than people in the control group. The intervention was therefore protective and decreased the likelihood of a fall. However, statistical analysis reveals that this result is not statistically significant.

Question 3	How can you tell that this result (0.85) is not statistically significant?
Answer	The 95% confidence interval includes 1 and the p-value is greater than 0.05.

p-value: The probability that any particular outcome would have arisen by chance

Confidence interval (CI): An interval within which the ‘true’ value is expected to lie with a given degree of certainty, for example 95%

Relative Risk (RR): The risk of the outcome in the treatment group relative to that in the control group.