RE: [Leica] Value of test reports?

[Paul Chefurka <Paul_Chefurka@pmc-sierra.com>]

So, if:

1. Magazines and other reviewers do not divulge important aspects of their
testing methodology,
2. Reviews of the same item in different magazines cannot be compared due to
methodological differences,
3. Manufacturers will typically not reveal weaknesses in their lenses to
mere customers,
and
4. Most of us don't know our butts from third base about how to test a lens;

How on earth are we to make any informed buying decisions, in your view?  It
sounds to me like we're left with Dan Cardish's approach:  shoot some
pictures, take a look and decide which one we like best.

Given that, how can any of us justify spending the money that the 35/1.4
ASPH commands?   We have no definitive way of telling if it's really any
better than the similarly specified Nikkor that sells for a fifth of the
price.  We can't trust any supposedly "authoritative" source to be giving us
the straight goods, and our own meagre efforts are way too loosely
controlled to tell us anything beyond the fact that the lens didn't fall off
the camera during the exposure.

It seems to me that the "statistical validity" approach is the only real one
we have to rely on.  Not the testing of a sufficient number of lens samples,
but the gathering of a sufficient number of opinions, at least some of which
were claimed to be the result of rigorous investigation.  Even if we don't
know the exact testing methods, or the degree of rigour, we can at least
look for broad agreements - given a sufficient number of sources.

How would you make up your mind if you were me, and wanted to spend all your
time using lenses and not testing them?

Paul