Hi,

These are all good points. I would say the FRC and peaks/dips should play a bit larger role since how much a headphone may vary from the average measures as well as how much it varies from its own curve gives one a better idea of whether one will hear something similar to the average measured response.

Cheers, Tom

Mar 21, 2022 08:19 AM By thomaslmusic55

280

Hi, These are all good points. I would say the FRC and peaks/dips should play a bit larger role since how much a headphone may vary from the average measures as well as how much it varies from its own curve gives one a better idea of whether one will hear something similar to the average measured response. Cheers, Tom

Peaks and Dips is literally just FR with a weight matrix. I understand why it’s a metric, but it really doesn’t matter; anything with good Bass/Mid/Treble accuracy scores will also have good Peaks & Dips scores.

Hi,

I guess I’ll agree to disagree since I find measurements of consistency/variance to be useful. I will admit that 5 measurements is an extremely small sample size for FRC and that in and of itself may not make the tests that useful as a metric.

As someone said in your other thread, . one can always define their own personalized score and just use the information that they think is most relevant. I personally use one that just looks at bass/mid/treble/imaging/FRC/dips+peaks and ignore soundstage completely.

Finally, one could also say that “neutral” is not the right term for this score anyhow. There isn’t anything “neutral” per se about their Harman derived target. There are plenty of folks who would say the bass shelf increase e.g. is not neutral. In the end, Rtings has to decide what they mean by this overall score and come up with a satisfying term/definition for it. :)

cheers,

–tom

Mar 21, 2022 05:20 PM By thomaslmusic55

280

Hi, I guess I’ll agree to disagree since I find measurements of consistency/variance to be useful. I will admit that 5 measurements is an extremely small sample size for FRC and that in and of itself may not make the tests that useful as a metric. As someone said in your other thread, . one can always define their own personalized score and just use the information that they think is most relevant. I personally use one that just looks at bass/mid/treble/imaging/FRC/dips+peaks and ignore soundstage completely. Finally, one could also say that “neutral” is not the right term for this score anyhow. There isn’t anything “neutral” per se about their Harman derived target. There are plenty of folks who would say the bass shelf increase e.g. is not neutral. In the end, Rtings has to decide what they mean by this overall score and come up with a satisfying term/definition for it. :) cheers, –tom

To be perfectly clear, I find the measurements of all of these things incredibly useful. RTings is arguably the best when it comes to having a diverse set of data points. It’s just problematic that they use these things to determine a score for Neutral Sound when they have no bearing on such a metric.

While yes, anyone can make their own rating, most people won’t. Most just take the numbers on the screen at face value, and because of that, the Neutral Sound score as calculated by the site is incredibly important for people’s decision making.

I’m one of those who think that Harman is mostly neutral aside from the bass boost being a touch too big, however because of bass SPL only happening around the ears instead of firing at the body, many feel like a flat signature from 300Hz-5Hz sounds bass light because you’re not getting the tactility of air moving and hitting your body… and that’s to say nothing of equal loudness contours’ effects on perceived SPL of bass frequencies.

Thanks again for the very detailed feedback. I totally understand the point you are making about soundstage being too heavily weighted in a score about neutrality.

While readjusting the scores and keep improving them overtime with feedback like yours is something we will always seek to do, we are internally thinking about broadening what is considered “accurate” since scoring headphone with one definition of accuracy or neutrality will always cause issues with others that do not agree with that definition. For example, at the moment the neutral sound score is the only usage score that is 100% about sound (only includes sound measurement metric) making it the score that people associate with “good sound” for some while for others it’s about how truly neutral the sound is. This one example of disconnect in interpretation will always stay no matter how we change the weight of the current neutral score usage so we need to consider other approaches to make it more clear and useful for more people.

Different target curve, more sound oriented usage scores, redefining what we want to expose with our neutral sound score are things we have to explore to try and fix those definition issues to provide better interpretation of what we measure, but also cover a broader spectrum of sound preference and if those ideas sparks suggestions or concerns we would be very happy to hear them as we shape our mind on how to approach those issues.

As for the graph readability issues, those are all good proposition that we will look into implementing when we start working on the next test bench as I agree that it suffers a lot from readability issues. Having community feedback like yours really helps guide the direction of what needs to be improved to push the quality of our reviews forward so thanks a lot.

Mar 24, 2022 01:50 PM By  Alexandre Meunier

Thanks again for the very detailed feedback. I totally understand the point you are making about soundstage being too heavily weighted in a score about neutrality. While readjusting the scores and keep improving them overtime with feedback like yours is something we will always seek to do, we are internally thinking about broadening what is considered “accurate” since scoring headphone with one definition of accuracy or neutrality will always cause issues with others that do not agree with that definition. For example, at the moment the neutral sound score is the only usage score that is 100% about sound (only includes sound measurement metric) making it the score that people associate with “good sound” for some while for others it’s about how truly neutral the sound is. This one example of disconnect in interpretation will always stay no matter how we change the weight of the current neutral score usage so we need to consider other approaches to make it more clear and useful for more people. Different target curve, more sound oriented usage scores, redefining what we want to expose with our neutral sound score are things we have to explore to try and fix those definition issues to provide better interpretation of what we measure, but also cover a broader spectrum of sound preference and if those ideas sparks suggestions or concerns we would be very happy to hear them as we shape our mind on how to approach those issues. As for the graph readability issues, those are all good proposition that we will look into implementing when we start working on the next test bench as I agree that it suffers a lot from readability issues. Having community feedback like yours really helps guide the direction of what needs to be improved to push the quality of our reviews forward so thanks a lot.

I agree that soundstage should be weighted less. But not so low that it’s at 5%

I strongly agree with the bit about soundstage weighting. To me, it only makes sense to go for headphones with good passive isolation for accurate sound. But these will inevitably have a low soundstage score, since its basically inversely proportional to passive isolation.

Which is better for critical listening, “soundstage” or isolation? I would think the answer is obvious.

Maybe soundstage is more desirable for casual/leisure listening. But I can’t see how it would ever be desirable for critical listening.

Jun 11, 2022 02:36 AM By eiglow_

140

I strongly agree with the bit about soundstage weighting. To me, it only makes sense to go for headphones with good passive isolation for accurate sound. But these will inevitably have a low soundstage score, since its basically inversely proportional to passive isolation. Which is better for critical listening, “soundstage” or isolation? I would think the answer is obvious. Maybe soundstage is more desirable for casual/leisure listening. But I can’t see how it would ever be desirable for critical listening.

Noted! As you know, we always compile feedback, so we’ll add this to the thread pertaining to this suggestion. There’s lot of useful information here, so we’re hopeful it’s going to lead to some positive changes to our testing/reviews. Thanks again for sharing your thoughts with us!

Hello everyone,

I just wanted to reach out and say thank you for leaving such detailed comments. Our team has been reading through your comments and we have an article we’d like to share with you: Improving Headphone Testing. It’s based on comments such as yours, what the future of testing headphones for us can look like, and how we can improve. However, it’s not possible without your input. If you have any feedback or comments, we’d love to hear from you once more. 🙂