Lens testing: focusing in manual focus with live view at 10x.
Testing a lens can be fun an informative, but it is not that easy: you have to be very careful if you want to get reliable results, and you need a lot of patience. In this article, I have explained how I review a lens, why I give more importance to some aspects rather than others. This page is useful both to understand better my reviews, and to help you if you want to do your own tests!
Even though the basis of my review process are always the same, in these years I have made many small improvements to make the reviews more precise, useful and informative. The information in this page describes the review system I use from late 2009.
Specs are useful to get an overview of the lens. Some notes: there are mainly two types of AF, the "Ultrasonic AF motor" (very fast and silent) and the old "AF motor", that is slower and it does not have full time manual focus. In the comparison table, I always list the type of AF actually used by the lens, regardless of the manufacturer's claim: many manufacturers are not very honest about this aspect, and they call "Ultrasonic" even lenses that use the cheap "micro-USM" AF motor, that is identical to the old non-USM AF motor. For example, the Canon 300mm f/4 L IS USM has the true ultrasonic AF motor, while the Canon 70-300mm f/4-5.6 IS USM has the cheap non-USM AF motor.
The production field lists the year of introduction of the lens and, in case of lens that are out of the market, the year of discontinuation. The date can be useful if you are evaluating the purchase of the lens, because old lenses are more likely to be replaced soon by a new version than recent lenses. The prices are in United States dollars and they are based on the B&H prices (www.bhphotovideo.com).
Description (built quality, autofocus and stabilization)
In this paragraph I describe the main aspect of the lens, and I compare its features and built quality with other similar lenses. I don't do in-depth tests of autofocus or image stabilization because it is nearly impossible to make a "scientific" evaluation of these aspects; my consideration are based on my experience with these lenses. (when I testing some lenses, I try to use them as much as possible, to get an idea of their AF and IS).
The postage stamps test: Sharpness, contrast, chromatic aberration
Sharpness is one of the most important aspect of image quality, for this reason I carefully test sharpness (and contrast) of every lens I test, and I publish 100% crops. These crops are useful even to evaluate chromatic aberration, too, but this is not as important: CA is one of those things you can correct with PS, while you can't do much for sharpness. (sharpening helps a bit, but it won't turn you 70-300 in a 300 2.8!)
To test the sharpness, I use a 24x36 centimeters test target covered with postage stamps that have fine detail (for long lenses), while for wide-angles I prefer to use a much wider test target, that is about 3 x 4.5 meters. It does not make sense to test a wide angle with a small test target because the majority of times you will use it for wide views, so I think a 3 x 4.5 meters target provides a more accurate result. Standard zooms and tele instead are often used even for relatively small subjects (portraits, birds) so an approximately A4 test target is ok.
Of course the test target must be flat and it must be perfectly parallel to the sensor! I focus carefully using live view and 10x magnification, and I take the photos using a stable tripod, mirror lock up and self timer, to avoid any vibration that may invalidate the results.
I always repeat this test at least two times, to be sure there are not focus errors or other imprecisions. Currently I use two cameras for the tests: a Canon 1Ds III (fullframe, 21 megapixels) to review the majority of FF lenses except teles, and the Canon 7D (APS-C, 18 megapixels) to review APS-C lenses and teles. I prefer to use the 7D for long lenses because the majority of times these lenses will be used on a crop body, and usually when you take a photo with a tele the corner performance does not matter much, while the sharpness in the center is important.