Re: Emotive lenses

[dmorton@cix.compulink.co.uk (David Morton)]

In-Reply-To: <2.2.16.19961005112953.4bcf1cc8@postoffice3.mail.cornell.edu>
Charles E. Love, Jr. writes:

> As for resolution tests, I agree.  What do you think of Popular
> Photography's efforts to improve lens testing with their method they 
> call
> "Subjective Quality Factor?"  It seems to be an effort to respond to the
> concern that the old resolution and contrast measurements were too 
> narrow.

Firstly allow me to preface what I'm about to say by mentioning that I 
have made my living in years past as a journalist working for several 
different UK IT magazines and one national newspaper. That's not the line 
of work I'm currently in (for personal reasons I don't propose to go in 
to here), though I am being pestered gently to return by a couple of 
editors who became friends.

I also spent 13 years as a research engineer at the BBC's Engineering 
Research Dept, most of those years were spent in the 'Image Scanning' 
section where I worked on HDTV cameras and film scanners. The group of 
which I was a member received an Emmy citation for our work on film 
scanning.

I have never read the US version of Popular Photography, and so I can't 
comment on their assessment methods, which may be exemplary for all I 
know.

However the motivation behind product evaluation techniques used in some 
sections of the popular press is - in my experience - not to produce a 
scientifically sound test.

Without descending to extreme cynicism (though frankly I'm tempted), the 
editorial requirement is frequently for a test which is a) cheap, b) 
quick (goes with (a)), and c) which uses some unique measurement 
technique which cannot easily be emulated by the competition, so 
preventing any informed criticism of the results.

Ideally the test should only actually *fail* those lenses made out of 
re-cycled beer bottles in a a lock-up garage in the Far East, so ensuring 
that the reviews upset the smallest possible number of advertisers or 
potential advertisers (these are the publication's source of revenue, the 
cover price of the magazine barely covers the print cost).

Any test which attempts to derive a single figure of merit for something 
as complex as a photographic lens is doomed from day one. More worryingly 
it is open to cynical abuse in the choice of the weights chosen for the 
different elements (pardon the pun) of the test. It is quite possible to 
chose those weights in such a way as to favour one manufacturer or group 
of manufacturers over another. You can weight benchmarks to favour one 
CPU over another, the same technique is just as applicable to MTF tests 
and their like.

dmorton@cix.compulink.co.uk      |
david@cassandra.compulink.co.uk  |  "The loss of an old man
(+44) 181 450 5459               |  is like the destruction
                                 |  of a library"
Kilburn, London, England         |