[Leica] lens testing, the suitability factor.

[Erwin Puts <imxputs@knoware.nl>]

The practical method for lens evaluation that has been proposed on this
list recently, has a long history. It has been in vogue since the thirties
and has found extensive descriptions in all kinds of articles since then.
In fact such a practical test is not a lens test, but a system test (as it
involves subject, lens, camera,film, exposure, development  etc). This
means that many variables will influence the system result. If for instance
one lens has 100 lines/mm and another one 200 l/mm (optically a very big
difference), but the system uses a film with 100l/mm (which is mostly the
case) then the resulting resolving power of the object at the film plane
will be 50 and 67 l/mm. Thus only really big differences in optical
performance will be detected by this kind of testing, IF all is done
properly and consistently AND the object to be photographed is a suitable
one AND the focus setting is at its optimum position. Many of these
conditions are not met in practise.
Still this practical method has one big advantage: it shows the limit of
acceptability  of a certain system for this individual photographer, given
his/her needs and experience. and systemsetup.
 It makes sense that a photographer should be happy with a systems
performance and not worry too much if (s)he should consider another lens to
use or buy.
This method is mostly opposed to the analytical method of testing lenses
(not systems) which is based on a variety of equipment and methods and
training. These analytical tests are mostly electronic now. Most industry
tests shy away from visual inspections as too unreliable.  The most popular
electronic one is the MTF test. This test comes in two flavours.  The
reading of the areal image of a slit by a scanning device and the
subsequant computer analysis of the contrast transfer and edge contrast.
Again this method has many versions. The PopPhoto, Photodo, Chasseur's
d'Image and BAS and ColorPhoto are some examples.
The second version is the computer generated MTF: this works very
(simplified) as follows: he optical aberrations of the lens  will deform
the wavefronts of the incoming light bundles as reflected from the object
point. These deformations will be measured as optical path differences
which after much number juggling and Fourier Transforms will generate the
OTF. from which is derived the MTF. This one is the most accurate and most
revealing of a lens performance. (Leica, Zeiss use this version). So the
MTF of Leica can not be compared to the ones of Photodo etc.
This MTF is not complete: it gives no info about flare, distortion, colour
correction, close up performance etc. It is also difficult to interprete
without a thorough optical background.

The best and most complete testing cycle of a lens would be composed of
three different methods, that complement each other: an analysis of MTF
graphs of the second type, a study of an aeral projection of a test pattern
to show distortion, curvature of field, astigmatism etc and practical field
testing to show colour correction, flare, close up performance, out of
focus behaviour etc. This combination of results should be weighed and
reported upon in a meaningful and understandable way for a practical
photographer.

The system test, however conducted can never be a substitute for the
analytical testing.It is a matter of choice which one a photographer should
use as a basis for buying lenses. A simple dichotomy will not work.
The practical and analytical methods both have merits for photographers.
One should be carefull not to try to prove one to be superior above the
other. They are too different for a direct comparison.

BTW: we talked about QC and lenses some time ago. Here are some figures
from the Japanese Camera and Optical Instrument Inspection and Testing
Centre.  Mass production of lenses require a statistical approch to
testing. How is this done. Assume a production run of 3000 to 5000 lenses.
Then the Institute requires that a batch is choosen at random. This batch
must be 85 lenses. These 85 are thoroughly tested. If 10 defects are found
the whole production run is accepted. At 11 the run is rejected. If the
total number is 181 to 500, the sample is 40 items and the rejection number
is 5 (passed by 4).
These numbers are for Japanese lenses. It does show that this Institute
accepts a certain failure rate (sometimes even of 10%) when lenses are mass
produced. When lenses are hand assembled the same laws apply. More rigorous
inspection and testing will reduce the failure rate, but can never
eliminate it.


Erwin