Audio Signal Processing and Coding

12.4 SUBJECTIVE EVALUATIONS OF TWO-CHANNEL STANDARDIZED CODECS

The influence of site and subject dependencies on subjective listening tests can potentially invalidate direct comparisons between independent test results for different algorithms. Ideally, fair intercodec comparisons require that scores are obtained from a single site with the same test subjects. Soulodre et al. conducted a formal ITU-R BS.1116-compliant [ITUR94b] listening test that compared several standardized two-channel stereo codecs [Soul98], including the MPEG-1 layer II [ISOI92], the MPEG-1 layer III [ISOI92], the MPEG-2 AAC [ISOI96a], the Lucent Technologies PAC [John95], and the Dolby AC-3 [Fiel96] codecs over a variety of bit rates between 64 and 192 kb/s per stereo pair. The AAC algorithm was tested in the main complexity profile, but with the dynamic window shape switching and intensity coding tools disabled. The MPEG-1 layer II codec was tested in both software simulation and in a real-time implementation (“ITIS”). In all, 17 algorithm/bit rate combinations were examined in the tests. Listening material was selected from a library of 80 items deemed critical by experts, and ultimately the two most critical items were chosen for each codec tested. Mean difference grades were computed over the set of results from 21 expert listeners after three of the original subjects were eliminated from consideration due to their statistically unreliable performance (t-test disqualification).

The test results, reproduced ...