Audio Equipment Testing - Difficulty of Testing

Difficulty of Testing

It is difficult, but very important, to match sound levels before comparing systems, as minute increases in loudness—more than 0.15 dB or 0.1 dB—have been demonstrated to cause perceived improvements in sound quality.

Listening tests are subjected to many variables, and results are notoriously unreliable. Thomas Edison, for example, showed that large audiences responded favorably when presented both live performances by artists and reproductions by his recording system, which today would be regarded as primitive in quality.

Similarly, results of component evaluation between various listeners or even the same listener under different circumstances cannot be easily replicated or standardized.

Similarly, the acoustic behavior of the listening room—the interaction between loudspeakers and the room's acoustics—and the interaction between an electromechanical device (loudspeaker) and an electronic device (amplifier) are subjected to many more variables than between electronic components. Thus the "difference" in sound quality between amplifiers is actually the ability of an amplifier to interface well with loudspeakers or a lucky combination of loudspeaker, amplifier, and room that works well together.

The introduction of switching apparatus, with either metal connection (mechanical switches) or electronic processing (solid-state switches), may, some believe, obscure the differences between the two signal sources being tested.

Skepticism advocate James Randi has offered a $1 million dollar prize to subjectivists who can prove their most dubious claims through scientific blind testing. The prize remains unclaimed.