[Leica] subjective versus objective testing procedures

[Erwin Puts <imxputs@knoware.nl>]

DonjR43198@aol.com wrote, at least in part

> As far as lens quality is concerned, I prefer the BAS type report as opposed
> to a subjective comment.  Further BAS explains the type of testing it does
> and then reports the tests in numerical figures rather than a subjective
> statement that anyone may make.
>
> Numerical figures based on scientific testing procedures carry weight and are
> something to be considered while subjective comments are nothing more than
> someone's opinion.  I do enjoy reading the opinion of everyone regarding
> quality, etc., of the Leica equipment since that is the only 35 mm equipment
> that I have owned and used since 1957 save for on excursion in the Nikon camp
> for a couple of months.  However, I do distinguish between subjective opinion
> and scientific testing.

Dear DonjR43198,

you are absolutely right in distinguishing between scientific testing and
subjective opinion.
If the world we live in, would be a simple one, we could just stop at this
point. In reality, we have a more complicated world. We have in fact bad
scientific testing and  correct subjective opinion.
I am intimately acquainted with the BAS testing procedure as I am with the
Photodo and PopPhoto testing method. While it may sound impressive that all
three base their findings on a kind of MTF testing, you might be willing to
realise that MTF testing now is as error prone as the classical resolution
tests are that were popular and authoritive  some years ago. You might
realise that taking a picture of a pattern of alternative black and white
lines ( a socalled grating) and measuring the "blackness" of the  dark
stripes of the grating and the "whiteness" of the white stripes and then
computing the relative intensity of both at several spatial frequencies
could be interpreted as a simple MTF test. This is the test BAS did in the
past. It is a far cry from the real thing: the MTF analysis as generated
from the optical design programs, that base their analysis on the correct
approach: the Optical Transfer Function as generated from the Point Spread
Function. This OTF has two components, the Phase Transfer Function and the
Modulation Transfer Function. It is a bit too far to explain this all to
you in a short message.
Be assured however that the BAS approach is a very crude way and in many
ways an erroneous one to analyse the performance of a lens.
I may refer you to the most authoritive magazine in the world (the British
Journal of Photography) that states quite explicitly that a lens report
carefully worded in terms that working photographers can understand is
worth more than one simple merit figure.
To give you one example: the BAS test of the pre-asph 2/35 gives a simple
merit figure of 92% for optical performance. Would you be able to relate
that to the MTF graphs? The Elmarit-M 2,8/135  gets 92%, as do the Summilux
1,4/75 and the Summicron 2/90.  The 2,8/90 and the 4/135 get 94%. Would you
be able to explain to a prospective user the difference in performance?
You presume my testing to be subjective because I use words in stead of
numbers? Let me then tell you that my testing is based upon an optical
bench with very accurate figures, the real MTF graphs from the factory and
practical picture taking in the field.
Now my testing indicates that there is a big difference between the imagery
of the Elmarit 2.8/135 and the Summicron 2/35 (both 92%). BAS does not tell
you that. I may be wrong, but my reports by their very nature of being
descriptive can be refuted. It is impossible to refute the number 92%.
Would I have to prove that 91% would be better?
I do assume that a report that tells you that at full aperture a certain
lens does record very fine detail with fuzzy   edges on axis rapidly
falling off to the outer zones and a another lens which at full aperture is
capable of recording extremely fine detail with  a high contrast edge over
most of the image field is more informative that the merit figure 92% in
both cases.

I am afraid that you are mixing up two things. Producing figures is not in
itself constituting a scientific approach and trying to explain in words
what can be noted in tables of measurements  is in itself not a subjective
approach.

Maybe you are aware of Wittgenstein the Austrian philosopher or Feinmann,
the American physicist.Both made extensive use of words to convey the most
exact of thinking based on explicite logic and the most demanding series of
figures.
If you really are of the opinion that any  number is worth more than a
statement in words, then please believ e the BAS figures.
Look especially at the number for the Elmarit-M 2,8/21 (the non ASPH
version) at 96% and the Summicron-M 2/50 also at 96%. Please be so good and
tell me where the differences between these two lenses is supposed to be.
This scientific testing proves to you  that both lenses have the same
optical performance.
I beg to differ and question these numbers.

A number in itself is not science and a word is not in itself the result of
a subjective impression.

Erwin