Test Bench 1.8: Changelog

What's Changed?

Since getting the Brüel & Kjær (B&K) 5128-B, we've begun updating our sound methodology. Starting with the fundamentals, we've updated our target curve for use with the B&K and made a couple of adjustments to frequency-response-related tests to better reflect the role of sound preference. All in all, these changes are somewhat small, but they set the basis for larger changes we plan to make to our sound methodology in a future update.

Test Group	Description
Sound Profile	We've updated our target curve to match our test rig. 'Normalization Frequency' comparison lets you see what frequency we use to normalize our response.
Raw Frequency Response	We've also updated our target curve here.
Bass/Mid-Range/Treble Profile: Target Compliance (Previously called 'Bass/Mid/Treble Accuracy')	We've updated this group of test names to better reflect the role of preference in relation to our target curve. 'STD Error' is now 'RMS Deviation from Target.'
Peaks/Dips	We use 1/1 octave smoothing from the Sound Profile test as the Peaks/Dips baseline.
Neutral Sound	Peaks/Dips now comprise 16.0% of the score of this usage.

New Tests

Our target curve

Our updated target curve is similar to our previous target and still resembles Harman's target, but we've made a few adjustments that correspond to our test rig (which is different than the one Harman used for their study):

Added the diffuse field response of the B&K
Tilted the response 6 dB
Added 4.3 dB bass shelf (@105Hz, Q .707)
Applied 1/6th octave smoothing

If you want to get into the weeds, we've got a whole R&D article outlining our process for defining a target curve. In this abridged article, however, we want to stress that this target approximates a sound that most people will find balanced in a pair of headphones, not the absolute measure of good sound quality. There's always an element of preference involved in interpreting frequency response.

In addition to the update to our target curve, there have been a couple of interrelated changes:

We've merged our two targets (one for over/on-ears and another for IEMs) into one target for all headphones. While we originally had an IEM target with a 5dB bass boost, informed by studies published in Audio Engineering Society¹^,², the differences in bass response between different ear simulators are still not well understood. As a result, there isn't strong enough evidence to suggest that an IEM target should be more v-shaped.
No more human passes for over-ear headphones when it comes to raw frequency response measurements. This is because most people won't move their headphones in an extreme way that would create a significant drop in bass. We reseat over-ears on our rig's head five times to capture small variations. For in-ears, we reseat the buds three times. However, we still do human passes when it comes to the Frequency Response Consistency test to showcase the effect of hair, glasses, or head shape on the results.
Frequency response measurements are now made at 94 dB instead of 90 and 100 dB to align more with industry standards.

Sound Profile

To give headphones the best chance to score well, we have a 'Normalization Frequency' value, which allows us to better level the frequency response to our target.

Test Bench 1.7	Test Bench 1.8

If you're using the graph tool, you'll also notice a yellow bar in the high-treble range. This bar highlights the range when you hover over the top of it and lets you know that the highlighted measurements are more precarious. This is because the human ear's morphology varies, which affects the treble response.

Bass/Mid-Range/Treble Profile: Target Compliance

Although they're three different tests, they represent an exploded view of the sound profile as it progresses through each range. We've updated the name of these tests so that it's clearer that we're measuring the response's deviation from our target rather than accuracy to a perceived 'neutrality.'

Test Bench 1.7 bass response of Beyerdynamic DT 770

Test Bench 1.8 bass response of Beyerdynamic DT 770

We've also renamed 'STD Error' to 'RMS Deviation from Target' for the same reason. The test is still the same, though. RMS is short for root mean square or quadratic mean. The scoring for 'RMS Deviation from Target' is more favorable to small deviations to reflect a range of values that are still near perfect.

Peaks/Dips

Since we've made changes to our sound methodology, we also need to update tests that use a pair of headphones' frequency response as a baseline. In Peaks/Dips, the resulting one octave smoothed frequency response becomes the new baseline. As a result, there's more leniency for frequency response variations that are considered by design. At the same time, it allows us to be more discriminant about what is considered a peak or dip.

Neutral Sound

We've added Peaks/Dips to the score components of this usage. Peaks/Dips is a performance-based test; regardless of your sound preference, a bad performance in this test will negatively affect sound quality across all users.

Let Us Know What You Think!

We're just getting started with our updates to sound. If there's something you want us to consider, investigate, or add to our test benches, let us know!

Recent Changes

10/24/2024: We've added a section regarding smoothing the Peaks/Dips baseline.

57 Headphones Updated So Far

We are retesting popular models first. So far, the test results for the following models have been converted to the new testing methodology. However, the text might be inconsistent with the new results.

50 Headphones Planned To Be Updated

We are also planning to retest the following products over the course of the next few weeks: