"Where the data deviated significantly from the normal distribution, we used standard statistical techniques to make the distribution of the values closer to that of a normal curve. We then standardized the value of these indicators about its mean." www.usnews.com/usnews/edu/grad/rankings/about/04method.php
I don't know what all those words mean, but I'm pretty clear that they mean a bell curve was "created" whether it existed or not. So I decided to see how close to a bell curve the data was in the first place. I started by creating graphs of the distributions of three US News variables -- academic reputation, employment rates nine months after graduation, and bar passage rate. Here's what I saw:
mean (average)
2.5
median (midpoint) 2.3
mode (most frequent) 2.0
In a "proper" bell curve, the mean, median and mode are all the same. Here, the values are clearly skewed to the low end. I assume that forcing a normal distribution onto these numbers will minimize values at the high end and maximize those at the low end -- i.e., by pretending 2.5 is the midpoint, you push the curve to the left.
mean (average)
92.5%
median (midpoint) 94.6%
mode (most frequent) 97.5%
Here you have an absurd effect. The mean is much lower than the other two numbers, because three schools (out of 177) had very low employment rates. The curve shifts to the right; all high-end values become magnified, and low-end values reduced. I think this causes false high scores for schools with very high employment rates. But I'm am illiterate when it comes to math; I'm just looking at the picture and imagining.
mean (average)
80.3%
median (midpoint) 81.7%
mode (most frequent) 81.5%
This graph is actually pretty close to a bell curve. Moving the center down to the mean instead of leaving it where it is increases differences at the high end a bit, but not as much as the skewing on the other two "curves."
I think it makes the percentage weights they report false. If you decrease the differences in reputation, where a few schools stand out above the crowd, you give it less than the 25% weight it's supposed to have. If you increase the minute differences in employment, you give it more than the 12% (.6 x .2) weight it's supposed to have. Schools with high reputations suffer, schools with high employment rates benefit.
If anyone really knows a lot about statistics, and can send me a reference in plain English explaining whether my intuition is right, I'd appreciate it. But please don't argue opinions without support.