M6 problem survey

[Eric Meyer <74415.1305@compuserve.com>]

To:  >INTERNET:Leica-users@mejac.palo-alto.ca.us

Dennis Painter wrote:
  >> Are you a statistican?

     Am I a statistician?  Why ask only me?  But no, I just know enough
about it to be a little less dangerous.
     The issue of bias had already been mentioned, such as would arise
if those who've had problems are more likely to respond to a survey on
problems than those who haven't -- as indeed I expect they are.  I
merely observed that nobody sounded particularly aware of a second
factor involved in evaluating any survey's results: how closely any
sample is likely to reflect the frequencies of the whole population
depends on the sample size.  The smaller it is, the less confident one
can be of its similarity to the whole.  And twenty-some units out of
several hundred thousand is a very small sample indeed.
     While reasoning from known probabilities is relatively familiar and
straightforward, the inverse problem, trying to estimate unknown
probabilities from observed frequencies, is not, and calls for some
rather sophisticated mathematics.  Were anyone here adept at this, they
would draw from this survey conclusions of the form: "There is an X%
likelihood that the real problem frequency with these Leicas lies in a
range of Y-Z%".  And I'm quite sure that the confidence levels (X%) for
any reasonably narrow range would turn out to be low, for such a small
sample.  Since I can't demonstrate that myself, you are welcome to try
to dispute it.  But regardless, the fact remains that claims of such
simplistic form as "you have a Y% chance of getting a problematic M6"
just don't follow from survey samples, period -- only from complete data
on an entire population.  From a sample all you can get is confidence
intervals, so the task of evaluating the legitimate implications of this
survey's results requires technical competence that has not yet been
found here.

  >> the intent was to get a feeling of the experiences of the
  >> listmembers regarding this.

     I'm sure that the survey was a well-meant attempt to improve on a
purely anecdotal approach to the quality issue.  But whatever "feeling of
experiences" it may have conveyed shouldn't be translated into specific
statistical claims that it can't support.
     (And for that matter, is it even plausible that Leica could have
stayed in business for 11 years with M6s actually experiencing a 50-75%
overall repair rate in their first few years of use, and that the first
hint of this would only appear now, here?)


and Detlef Beyer wrote:
  >> Also unjustified is: the qualitity of Leica...
  >> or can you proof the quality?

     A strange question.  I have nothing to prove about Leica quality
myself, because I'm not one of those who have been making statistical
claims about it.  I've merely asked those who have been doing so to
desist, since their purported conclusions don't follow from the survey
data.  A larger, more carefully implemented, and properly analyzed survey
could still be useful; it would be even better, of course, if Leica made
their actual repair history public.  It may be awhile before either of
these occurs, but we can be doing other things with our time.

                                  -- Eric Meyer.