How do five current full-frame DSLRs compare when put through a battery of objective lab tests for image quality? DxOLabs.com has conducted these tests, based on the Raw image data, to determine ISO accuracy, low-light high ISO image quality performance, color accuracy and depth, and dynamic range. Today, we're going to geek out, pixel peep, and go mind-numbingly high-tech to find out the answers.
Yes, I know all these cameras are good. After all, they are full-frame cameras and cost a bundle, and we would expect nothing but high-quality images. But there are differences which these test results tease out.
We'll look at the Canon 1Ds Mark III and 5D Mark II, the Nikon D3 and D700, and the Sony Alpha 900. (When DxOMark completes its testing of new cameras, such as the Nikon D3x, we'll add that as well). Not surprisingly, these five cameras earned DxOMark's top five overall image-quality ratings of all cameras they have tested so far. (The other full-frame DSLRs, the Canon EOS 1Ds Mark II, EOS 5D and EOS 1Ds, are older but with the exception of the EOS 1Ds were near the top of the list as well.)
Intangible factors
Before we get to the test results, let's talk briefly about non-technical, intangible factors.
The shape of the camera, control layout, firmware features and menu items and navigation all play into which of these cameras is best for you. You may need a camera with a super-fast burst rate and don't care about anything else, in which case the Nikon D700 and D3, which offer the fastest burst rates, would likely be your first and second choices. You may require lots of high-resolution blow-up prints, in which case the Sony or either Canon might be your first choice.
You may already be heavily invested in the Canon or Nikon system, which will also weigh on your buying decision (although many pro shooters have been known to switch when given a compelling enough reason).
Then there's price. Can we expect a $1,000 camera to produce image quality that rivals that of a $3,000 camera?
While it's not within the scope of this article to compare each camera's intangibles, it is important to read impartial user reviews to find out how the camera feels in the hand, how easy or difficult it is to operate its controls and navigate and use its menu items.
What was measured, and what it means
DxOMark conducts measurements based on the Raw image, which is the unadulterated image file as captured by the sensor (a jpeg, on by comparison, has already gone through some processing which may include grain reduction or color compensation, before you even get to see it.) DxO Analyzer, which has become an industry standard for camera image quality testing, was used in determining the results. DxO conducts its tests using specifically designed test targets under consistent conditions in a lab environment.
While DxO also offers tonal range, color sensitivity and color response data on its site, we and interperate the following test results:
ISO Accuracy: The actual measured ISO was compared against the manufacturer's ISO settings. Revelation: None of the ISO settings were accurate! Using this information you can set your camera's exposure compensation to make up for differences between the "official" and the actual ISO speeds.
Low-Light High ISO: Each camera sensor was tested for its signal-to-noise ratio (SNR) to determine how much digital noise (also referred to as grain) would be produced under low light at each of its ISO settings. The "Low Light ISO" rating reflects the highest ISO at which digital noise is considered acceptably low (to simplify things, any SNR reading over 30dB is considered acceptable).
Color Depth: Each camera is tested based on how accurately it can reproduce the 24 colors of the Gretag Macbeth Color Checker chart, and the results are displayed on a Lab map. The Lab map displays the results as circles known as "noise covariance ellipses." These ellipses, by their size and shape, show each of the 24 colors and how well the camera sensor can distinguish them from adjoining colors. The larger the ellipses, the more "color blind" the camera is (this could show up in real-world photos as color banding). Smaller ellipses mean the camera will record color more faithfully.
Dynamic Range: One of the big complaints regarding digital photography is that digital sensors have less exposure latitude, or dynamic range, than film. This is important when you are trying to capture a wide range of light, from bright highlights to shadow details, in a single shot. DxO has determined that an exposure latitude of at least 9 stops is necessary and that less than 9 stops will produce unacceptable results. 11 stops or higher rivals print film. Dynamic ragne changes as ISO is adjusted so we show not just each camera's optimal dynamic range, but also the point at which its dynamic range drops below 9 stops.
Test results
Note: All test results and graphics courtesy DxOMark.com. Used with permission.
Canon 5D Mark II
Price: Approx. $2,700
Size: 6x4.5x3"
Weight: 28.6 oz.
ISO accuracy
The 5D Mark II's actual, measured ISO was consistently approximately ½ stop lower than what the camera's ISO settings indicated. ISO 100 was measured at ISO 73, ISO 200 was ISO 143, ISO 400 was 285, and ISO 800 was measured at 564, and so on. The highest claimed ISO, 25,600, actually measured at ISO 15,110.
Low-Light High ISO Performance
 Measuring the 5D Mark II's signal-to-noise ratio at 18% mid-gray level, the camera's highest ISO at which image quality is considered to have an acceptably low amount of noise was ISO 1815. This means you can shoot at up to ISO 1600 (camera setting) and get good results. Overall, the camera's signal-to-noise ratio was outstanding from ISO 100-400, and very good through ISO 1600. As expected, its highest ISO produced high the most noise.
Color Depth
 The 5D Mark II's color depth rated a 23.7 on a scale of 1-25, an excellent result.The Lab map (shown) shows very small noise covariance elipses, which indicates the camera is picking up daylight colors very accurately with virtually no "color blindness".
Dynamic Range
 The 5D Mark II's best dyamic range is 11.9 stops when shooting at ISO 100. From ISO 100-400, the dynamic range remains fairly steady, but then drops off dramatically starting at ISO 800. At the super-fast speeds, it dips below 8 stops and by ISO 25,600 it is around 5.5 stops. A 9-stop range is considered the narrowest acceptable dynamic range.
Canon 1Ds Mark III
Price: Approx. $6,550
Size: 6.1x6.3x3.1"
Weight: 42.5 oz.
ISO accuracy
As with its sister camera the 5D Mark II, the Canon 1Ds Mark III's ISO accuracy is close, with the actual ISO coming in consistently at around half a stop lower than the indicated speed. ISO 100 was actually measured at 73. ISO 200 was 144, ISO 400 was 285, ISO 800 was 578, ISO 1600 was 1171, and ISO 3200 was ISO 2166.
Low-Light High ISO Performance
Measuring the 1Ds Mark III's signal-to-noise ratio at 18% mid-gray level, the camera's highest ISO at which image quality is considered to have an acceptably low amount of noise was ISO 1663, which effectively means image quality will be good at ISO 1600. Signal-to-noise ratio starts at ISO 100 (actual: 73) at 38.5DB and decreases steadily throughout the sensitivity range. The best image quality was at ISO 100-400.
Color Depth WINNER!
The 1Ds Mark III's color depth is 24 out of 25—the best performance of any full-frame camera tested by DxOLabs. The camera is able to distinguish all colors as indicated by the Lab map (shown, at ISO 100 and select luminance of 30) with virtually no "color blindness." 
Dynamic Range
The 1Ds Mark III's best dynamic range is 12 stops at ISO 100, and remains above 9 stops through ISO 1600 and only dips slightly, to 8.2 stops, at ISO 3200.
Nikon D3
Price: Approx. $4,100
Size: 6.3x6.2x3.4"
Weight: 2.7 lb.
ISO accuracy
The Nikon D3's ISO range begins at 200; the actual ISOs were consistently close—around 1/3 of a stop less than the indicated speed. ISO 100 was measured at 161, ISO 400 was 326, and ISO 800 was 635. These numbers remained consistent until ISO 25,600, when ISO measured 16,134. That's still fast.
Low-Light High ISO Performance
The D3's highest ISO at which digital noise is acceptable was tested to be ISO 2290, which means ISO 1600 for practical purposes. But since ISO 3200 is actually 2512 (which is pretty close) you may be able to squeeze fairly reasonable results at that speed as well (especially if you use noise reduction software). Signal-to-noise ratio remained above 30db through the ISO 1600 setting but was unacceptable (around 24db) by ISO 6400.
Color Depth
The D3's color depth is impressive. The noise covariance ellipses on the Lab Map (shown) indicate that the camera is able to faithfully capture the widest possible range of colors. Its color depth rating of 23.5 puts it close to the top of all digital cameras currently on the market in this area of image quality.
Dynamic Range
The D3's dynamic range, 12.2 out of 15, is excellent and edges out both Canon DSLRs. However, while it does very good at ISO 200-800, it begins to fall faster by ISO 1600 and drops below the minimum 9 stops by ISO 6400. Nevertheless, getting 9 stops of exposure latitude at ISO 3200 is impressive.
Nikon D700
Price: Approx. $2,700
Size: 5.8x4.8x3"
Weight: 2.19 lbs.
ISO accuracy
As with the D3,the Nikon D700's actual ISO sensitivity readings are within 1/3 of a stop of the indicated speeds for all of the practical speeds, but by the time you reach the top ISO ratings the actual sensitivity falls off. At ISO 200, the D3's actual ISO sensitivity measures at 162, while at ISO 400 it is 327, at ISO 800 it is 651 and at ISO 1600 it is 1277. By ISO 25,000, however, it is "only" 14,085.
Low-Light High ISO Performance WINNER!
The highest ISO at which the D700 offers acceptably low digital noise is 2303 (closest setting: ISO 1600), the highest rating of any camera tested. At this speed, Signal-to-Noise ratio is around 30dB. The best performance, not surprisingly, is at ISO 200,where the Signal-to-Noise ratio is 39.4dB.
Color Depth
With an overall color depth rating of 23.5 out of 25, the D700's ability to capture full color is virtually identical to that of the D3, so its color depth is impressive. The tiny noise covariance ellipses on the Lab map (shown) indicate that the D700 can clearly distinguish an extremely wide variation of colors.
Dynamic Range
The D700's 12.2-stop dynamic range is among the highest and the latitude remains above 11 stops through ISO 800. At ISO 1600, latitude is a decent 10.2 stops, and at 3200 it is still acceptable at 9.1 stops. At speeds beyond that, dynamic range quickly deteriorates. 
Sony Alpha 900
Price: Approx. $3,000
Size: 6.2x4.6x3.25"
Weight: 29 oz.
ISO accuracy
The Sony Alpha 900's actual ISO sensitivity is consistently around 1/3 stop less than the indicated ISO—with one notable exception. At the ISO 100 reading, the actual measured speed is ISO 119, a 1/5-stop overexposure! At ISO 200, the measured speed is 151, at ISO 400, it is 303, and is statistically consistent right up to the top speed of ISO 6400.
Low-Light High ISO Performance
The highest ISO at which the Alpha 900 produced images with acceptably low grain was ISO 1431, which effectively means the highest acceptable setting is ISO 800, although at ISO 1600 you could probably push it if you use noise reduction software after the fact. The Alpha 900's low-light high ISO performance was similar to that of the four-year-old Canon EOS 1D Mark II and was lower than the other cameras in this shoot-out.
Color Depth
The Alpha 900's color depth rating of 23.7 is one of the best performances of any DSLR and is comparable to all of the cameras tested for this story. The noise covariance ellipses in the Lab map (shown) indicate the camera can distinguish almost all colors with virtually no "color blindness."
Dynamic Range
The Alpha's 12.3-stop dynamic range at ISO 100 is the widest latitude of any full-frame digital SLR, but it steadily declines as the ISO setting goes up, and it drops under 9 stops (which is the border between acceptable and unacceptable image quality) by ISO 1600.
Mason Resnick is the Editor of the Adorama AIRC Learning Center.
© 2009 Adorama Camera, Inc.
|
