[Leica] Re: subjective versus objective testing procedures

[Jim Brick <jimbrick@photoaccess.com>]

This is precisely and exactly why Erwin's lens reports are so very very
valuable to "real photographers"... us. Because they are in terms that we
understand and in words that describe the very details that we deal with on
a day to day basis. A number (e.g.; 92%) is completely and totally useless.

Want to know how a lens really performs, in actual use, read Erwin's lens
reports.

Want to confuse someone with hocus-pocus numbers, read numerical lens reports.

Thanks again Erwin,

Jim


At 01:10 AM 5/15/99 +0200, you wrote:
>DonjR43198@aol.com wrote, at least in part
>
>> As far as lens quality is concerned, I prefer the BAS type report as opposed
>> to a subjective comment.  Further BAS explains the type of testing it does
>> and then reports the tests in numerical figures rather than a subjective
>> statement that anyone may make.
>>
>> Numerical figures based on scientific testing procedures carry weight
and are
>> something to be considered while subjective comments are nothing more than
>> someone's opinion.  I do enjoy reading the opinion of everyone regarding
>> quality, etc., of the Leica equipment since that is the only 35 mm equipment
>> that I have owned and used since 1957 save for on excursion in the Nikon
camp
>> for a couple of months.  However, I do distinguish between subjective
opinion
>> and scientific testing.
>
>Dear DonjR43198,
>
>you are absolutely right in distinguishing between scientific testing and
>subjective opinion.
>If the world we live in, would be a simple one, we could just stop at this
>point. In reality, we have a more complicated world. We have in fact bad
>scientific testing and  correct subjective opinion.
>I am intimately acquainted with the BAS testing procedure as I am with the
>Photodo and PopPhoto testing method. While it may sound impressive that all
>three base their findings on a kind of MTF testing, you might be willing to
>realise that MTF testing now is as error prone as the classical resolution
>tests are that were popular and authoritive  some years ago. You might
>realise that taking a picture of a pattern of alternative black and white
>lines ( a socalled grating) and measuring the "blackness" of the  dark
>stripes of the grating and the "whiteness" of the white stripes and then
>computing the relative intensity of both at several spatial frequencies
>could be interpreted as a simple MTF test. This is the test BAS did in the
>past. It is a far cry from the real thing: the MTF analysis as generated
>from the optical design programs, that base their analysis on the correct
>approach: the Optical Transfer Function as generated from the Point Spread
>Function. This OTF has two components, the Phase Transfer Function and the
>Modulation Transfer Function. It is a bit too far to explain this all to
>you in a short message.
>Be assured however that the BAS approach is a very crude way and in many
>ways an erroneous one to analyse the performance of a lens.
>I may refer you to the most authoritive magazine in the world (the British
>Journal of Photography) that states quite explicitly that a lens report
>carefully worded in terms that working photographers can understand is
>worth more than one simple merit figure.
>To give you one example: the BAS test of the pre-asph 2/35 gives a simple
>merit figure of 92% for optical performance. Would you be able to relate
>that to the MTF graphs? The Elmarit-M 2,8/135  gets 92%, as do the Summilux
>1,4/75 and the Summicron 2/90.  The 2,8/90 and the 4/135 get 94%. Would you
>be able to explain to a prospective user the difference in performance?
>You presume my testing to be subjective because I use words in stead of
>numbers? Let me then tell you that my testing is based upon an optical
>bench with very accurate figures, the real MTF graphs from the factory and
>practical picture taking in the field.
>Now my testing indicates that there is a big difference between the imagery
>of the Elmarit 2.8/135 and the Summicron 2/35 (both 92%). BAS does not tell
>you that. I may be wrong, but my reports by their very nature of being
>descriptive can be refuted. It is impossible to refute the number 92%.
>Would I have to prove that 91% would be better?
>I do assume that a report that tells you that at full aperture a certain
>lens does record very fine detail with fuzzy   edges on axis rapidly
>falling off to the outer zones and a another lens which at full aperture is
>capable of recording extremely fine detail with  a high contrast edge over
>most of the image field is more informative that the merit figure 92% in
>both cases.
>
>I am afraid that you are mixing up two things. Producing figures is not in
>itself constituting a scientific approach and trying to explain in words
>what can be noted in tables of measurements  is in itself not a subjective
>approach.
>
>Maybe you are aware of Wittgenstein the Austrian philosopher or Feinmann,
>the American physicist.Both made extensive use of words to convey the most
>exact of thinking based on explicite logic and the most demanding series of
>figures.
>If you really are of the opinion that any  number is worth more than a
>statement in words, then please believ e the BAS figures.
>Look especially at the number for the Elmarit-M 2,8/21 (the non ASPH
>version) at 96% and the Summicron-M 2/50 also at 96%. Please be so good and
>tell me where the differences between these two lenses is supposed to be.
>This scientific testing proves to you  that both lenses have the same
>optical performance.
>I beg to differ and question these numbers.
>
>A number in itself is not science and a word is not in itself the result of
>a subjective impression.
>
>Erwin