Camera Ratings - Part 2

2006.03.16

A rating is found at the end of nearly every digital camera review we read. Sometimes it is a score, like 8 out of 10, sometimes it is an adjective, like good and sometimes it is a number of stars, like 4 1/2 stars. No matter the way it is described, it is a rating and hopefully some thought has gone into producing it.

In Camera Ratings - Part 1, we said that rating are relative. Well, here is another way in which they are relative. Take the common 0-to-10 scale for example, 0 should be the worst, 10 should be the best and 9 should be almost that. The catch is that a great number of sites never give less than 8, or its equivalent. In such case, 8 is the worst. That is how we translated some results as poor.

Take DP Review, a great technical review site which is considered one of the most demanding. From 1999 to 2005, it has rated 6 below average cameras, 10 average, 27 above average, 75 recommended and 71 highly-recommended. Since 2003, there has been no average or below average rating at DP Review. So since 2003, every camera has been better than average!

Obviously, that is not statistically sound. There are a few improbable explanations for this: they refuse to review poor cameras, they know which cameras are good and only ask to review those or they don't publish reviews of poor cameras. Some more probable explanations: they give good ratings to please manufacturers who send them cameras or they decided to give good ratings because most cameras give acceptable results anyways.

On the neocamera site, we decided to fill out the scale from poor to excellent, with the goal of having average cameras, well, average. If you count the number of cameras for each rating, there are still more good and excellent cameras than poor ones. So average is still not the exact middle. It turns out that keeping the average camera average is somewhat tricky. By occasionally updating camera ratings, we hope to keep average-performing cameras at an average rating. Neocamera is probably the first digital camera site to do this. Time will tell if this improves the usefulness of the site.

To be continued

Part 3 will cover what goes into camera ratings and what does not.

• Review