Test Bench 1.7

Changelog

Oct 03, 2024 10:25 PM By liminalsunset

1

It would be nice if the updates could be extended to the Transparency mode of headphones with this feature, as it’s becoming increasingly common in headphones. Perhaps a recording and a graph of the neutrality of the headphones in their Transparency mode compared to not wearing headphones would be very helpful as there can be significant quality differences between headphones.

Hi Liminalsunset, thanks for the suggestion, we’ve had a few request on this and it’s something we’d like to work on. No dates yet but it’s on the pile. In the meantime, there is very interesting info here, that could be of interest to you. https://critias.ca/otc/doku.php?id=en

It would be nice if the updates could be extended to the Transparency mode of headphones with this feature, as it’s becoming increasingly common in headphones. Perhaps a recording and a graph of the neutrality of the headphones in their Transparency mode compared to not wearing headphones would be very helpful as there can be significant quality differences between headphones.

Got it! Thanks for the update.

Jun 16, 2024 12:34 AM By turtis123

24

Hello. Why is Breathability score not present for most headphones even though it is a part of rating calculations and actually makes a difference for headphone selection in warm environments?

Our breathability equipment broke a little while ago and we’ve been unable to fix it. So, we have no way to test it at this time. Sorry abour that!

Hello. Why is Breathability score not present for most headphones even though it is a part of rating calculations and actually makes a difference for headphone selection in warm environments?

Apr 08, 2024 09:43 AM By  Andrew Morantz

Thank you both for the comments! These are all good points, and it’s part of the reason why we added audio samples to the Common Scenarios and Voice Handling tests. We want to allow you to directly compare headphones side-by-side. Here is an example of the WH-1000XM5 and Focal Bathys compared together like erk_ mentioned. I can see an argument for a bigger gap in scoring between the two, however there is a noticeable difference in scoring and the audio samples reflect this. I think it’s fair to say that the Focal Bathys have good ANC, but are a clear step down from the best-of-the-best. Probably more in line with what you mentioned, there is a more noticeable difference between the two headphones in the Voice Handling audio samples (likely because the Bathys have worse attenuation in the mid range, where most voice frequencies are). As I previously mentioned, we considered scoring this section, but ultimately decided against it. However, it’s something we can look at in a future update. As for the small gap in scoring between headphones, this is due to the choices we made when deciding on which headphones to include in the first batch of our testing. Many of these headphones happen to be high-performing ANC headphones, so the scores are expected to be reasonably high. I should also note that we have only tested a small portion of ANC headphones on the market with this new methodology and, with time, we will have more headphones updated and a wider range of scores. I hope that answers your questions. I’d love to hear if you have any other thoughts on the new tests!

Would be nice to have an A-weighted version of the ANC score. Basically adapt the score to where we are most sensitive to sound. It would also give a more realistic impression of how people would perceive the effectiveness in an environment where all frequencies are similar in noise level.

Apr 05, 2024 04:02 AM By erk_

53

I own several of the newer of the newer ANC headphones and using something like the WH-1000XM5 vs a Focal Bathys in an office with people having a discussion close by is like night and day. One lets voices through quite clearly unless you pump the volume up quite a lot, the other drowns out voices almost completely. Scoring on the new system is quite close, but their real world performance isn’t even remotely close when it comes to ANC performance.

Thank you both for the comments!

These are all good points, and it’s part of the reason why we added audio samples to the Common Scenarios and Voice Handling tests. We want to allow you to directly compare headphones side-by-side. Here is an example of the WH-1000XM5 and Focal Bathys compared together like erk_ mentioned. I can see an argument for a bigger gap in scoring between the two, however there is a noticeable difference in scoring and the audio samples reflect this. I think it’s fair to say that the Focal Bathys have good ANC, but are a clear step down from the best-of-the-best.

Probably more in line with what you mentioned, there is a more noticeable difference between the two headphones in the Voice Handling audio samples (likely because the Bathys have worse attenuation in the mid range, where most voice frequencies are). As I previously mentioned, we considered scoring this section, but ultimately decided against it. However, it’s something we can look at in a future update.

As for the small gap in scoring between headphones, this is due to the choices we made when deciding on which headphones to include in the first batch of our testing. Many of these headphones happen to be high-performing ANC headphones, so the scores are expected to be reasonably high. I should also note that we have only tested a small portion of ANC headphones on the market with this new methodology and, with time, we will have more headphones updated and a wider range of scores.

I hope that answers your questions. I’d love to hear if you have any other thoughts on the new tests!

Apr 05, 2024 01:15 PM By  Andrew Morantz

Hi erk_, Thanks for the feedback! We decided on not scoring our Voice Handling test because letting in the sound of voices could be seen as an advantage for awareness. You should, however, still be able to directly compare how headphones perform with voices specifically using the audio samples in the Voice Handling test. That being said, I can understand why scoring based on attenuation could be useful, and I’ve taken note of your suggestion. For the noise recordings in the Common Scenarios test, we played the recordings at the volume they were recorded at in order to give the most accurate playback. That volume is different for each recording since they are different scenarios. If you check the tooltip for each recording, the no-headphone recording (made using speakers surrounding our head simulator) is available. Since everyone’s devices/headphones/speakers are different, we decided to use the recordings without any adjustments and allow users to adjust their system volume to where the no-headphone recordings are realistic. I’m not sure if that clarifies our reasoning for the recording volume but I’d be interested to hear what you think and if that’s close to what you were suggesting.

The problem is I would have to set my headphones to well over 100db to get the same voice level as I would get in an office or on public transportation with the same headphones. Setting them to 93db at 1khz tone gave me significantly less volume on the voices than I would experience in a regular office setting or on public transportation with the headphones tested. The raw recording at 90db on my headphones is actually closer, than the damped recording with headphones at 95db, to what I would hear through the Bathys if people are having a conversation close by in the office. Not sure if the microphone has low sensitivity or if the voice is lower than most conversations.

My main use cases for ANC headphones is reducing noise on public transportation (including announcers, people talking etc.) and not hearing peoples conversation in the office. Most of my colleagues in the office have ANC headphones to block out voices as well to block out ongoing conversations while listening to music at a moderate volume. The ability to block out voices is the #1 need for me in an ANC headphone and voices are also in the frequency area where the human hearing is most sensitive. If I need awareness I would set them to passthrough or transparency mode :)