Significance

This has been a tough idea for a lot of you, so I will take another stab at it:

Statistical Significance (critical value): The likelihood that the differences between index scores or scaled scores are not due to chance. If a discrepancy is statistically significant, that means that compared to their overall performance (e.g. the mean of 10 subtests). performance on a certain item is uniquely different. It is reflective of actual different levels of performance between skills, not just a typical range of variability. For example, a 10 on Vocab and a 12 on Comprehension are different from one another, but NOT to a statistically significant degree. You would not be concerned about this difference nor try to figure out more about it. However, if there was a 10 on Vocab and an 18 on Comprehension, that would be a statistically significant difference, indicating that there is something fundamentally different about a person's performance on those tests. Which, especially because those tests are on the same index, is of great interest to us.

Clinical Significance (base rate): Once something pops up on the radar of statistical significance, we want to find out how common or rare that kind of difference is. If the difference is really common, it concerns us less because it is not likely to interfere with functioning. However, if a difference falls below 5% of the population, we know that there is something really unique about that, and that those abilities are relative strengths or weaknesses. People with these differences are performing very differently from the way potentially millions of other people would.

Satistical Significance is determined by comparing the difference between two scores (e.g. between the VCI and PRI, or between Matrix Reasoning and the mean of 10 subtests) and then determining if that difference is greater than the listed critical value. If it is, it is statistically significant.

Clinical Significance is determined by looking at the base rate. If a difference is greater than the base rate listed for 5% or less than the population, we get excited about that.

Example: on page 239, Table B.5, a person could have a difference on Block Design of 3 points and be statistically significant (critical value), but a difference of 3 occurs in 15% of the population (base rate). It is low, but not that concerning.

For the assignment, do your best to determine which differences are clinically noteworthy and show me your thoughts about what may be happening for Linda - what skills or abilities are easy or hard, etc.

As always, bring your questions to class and we'll hash it out!

EV