Bellonz

Been using these for a while now. They have an incredible AI noise cancelling mic, great for calls cause you can keep them in all day without being cut off.

One comment I have with your test bench is that the voice isolation/noise performance results aren’t valid anymore because many truly wireless buds have a voice pickup unit that uses bone conduction to help drive a AI algorithm to filter background noise. For example if you look on YouTube for the spectacular performance of the Air Pods 2 Pro with voice isolation or the Galaxy Buds 3 in very loud environments they are completely silent with only your voice being heard. But on rtings tests that isn’t reflected because you’re only getting acoustic conduction via your speaker to the earphones mic.

It needs testing on a real person who’s mouth moves. Otherwise the test is invalid. Off the top of my head the ones that do this are: Air Pods Pro 4, Air Pods Pro 2 these two only with iOS. Galaxy Buds 3 and Sony LinkBuds Donughts

I’ve been using the previous version of this with a micro USB since 2016. It’s an amazing travel sound bar. I find airbnbs have cheap tvs with terrible built in speakers. Most Bluetooth speakers just don’t have the form factor to fit centrally under the tv. This does and you get a real left, right, center sound stage.

The bluetooth pairing and physical buttons are a breeze to connect up to the tv or apple TV I travel with.

It’s rattlegate. https://forums.macrumors.com/threads/airpods-pro-rattlegate.2233658/page-31?post=30505679

I’m on my second pair. It’s a design flaw and apple have Replacement program

Thanks Shea

The other issue I have found is that the noise isolation only applies to actual cell voice phone calls. It doesn’t apply to voice recording memos or recording a WAV/mp3 via a bluetooth connection and PC. Very tricky to test! But worth mentioning and worth a subjective showing of how a voice call sounds on a mobile in a noisy environment, perhaps by calling your own voice mail and obtaining a recording that way.

Voice pickup isn’t so important in things like Microsoft teams which have machine learned noise isolation. It’s like magic and can completely isolate background noise from any mic.

It still has a use case for mobiles though. If you’ve not tried it, search for your Aftershockz opencomm. Insane boom mic that doesn’t protrude and has crazy noise isolation. That combined with Teams means recently I had an actual Jack hammer going off outside my office and was able to continue my presentation with no issues

Upon using Teams outside my company I realized it’s a license that enables the AI noise supression. There’s also things like “Krisp” that enable it if your oganisation doesn’t support it.

Wasn’t aware of that. It won’t impact the measurements though because of the high SPL the speaker generates to measure the isolation.

Interestingly enough I know that the APP’s automatically turn on ANC when in transparency mode and there’s an unexpected loud noise. Clanking plates together in the gym always sets the bud nearest to the noise to full ANC. Others have reported the same on reddit too

I agree - this is something that needs to be talked about because it really relates to comfort and why buds like the Ear Pods/ Air Pods are so popular - because they minimize it. Most people hate the “sealed off” or “under water” feeling.

In my experience the Air Pods Pro do a remarkable job in their transparency mode of minimizing it. Next comes the Bose 700s. A good proxy of it from the measurements rtings do take would be poor noise isolation. The Sound Sport Free, for example, have minimal occlusion effect because they’re open. As good as the Ear Pods / Air Pods but with decent bass delivery.

Something akin to this might be to use something like the etymoic deep insert IEMs as to what’s possible. Not sure if they isolate more than regular foam ear plugs though

Sep 08, 2020 08:48 PM By Loyden

1

And for devices connected wirelessly, maybe you could add the dBA / volume % ? You could even add some indicators for dBA and their “safe time” equivalent. E.g. (NIOSH data) : 70 dBA : safe 82 dBA : 16 hours 85 dBA : 8 hours […]

That would actually be extremely useful - especially for non Apple products which already account for that.

Just listened myself and I agree. I wonder if the test bench isn’t able to cope with AI filtering as well as real world situations. Upcoming headsets have accelratometers to determine when a user is speaking. The test bench won’t cope with that. But a manual test would.

The speech in the subway test sounds fine to me - doing a great job at blocking out noise. Yes it does get garbled with the ton of noise. But I wonder if in a natural real world situation that the user would then increase their own volume (talk louder) and the algorithm would then cope better with more signal.

Other respected reviewers/analazers have said that the Galaxy Buds are, objectively, one of the best sounding IEM’s ever made. crincile and oratory1990. It follows the Harman target exactly out the box.

I believe the problem is that audiophilla marketing has made fans of sound automatically think certain brands sound better. All sound perception is psychoacoustic, so this has some validity. The sound you are perceiving isn’t just the raw driver response coupling to your ear drum. It’s made up of an emotional aspect in your brain.

Headphones are minimum phase devices, so there’s no phasing issues to worry about - although bass extension is a rory. The frequency response tells everything you need to know in the time domain. If you correct the FR it’ll sound like a better headphone. There’s a project for doing exactly that, AutoEQ. It has done wonders to my harsh sounding DT990 and every set of headphones I own.

I like the analogy you made about the calibration. That’s so true! We’re comparing the out the box sound on these headphones and factory tuning. While we don’t have an objective reference like Rec 709/Gamma 2.4 in headphone world, we know that the Harman target is close to it.

Rtings is great for determining which headphone will be suitable for correction. The things that matter are frequency response consistency. If you can’t guarnatee similar FR on re-seats, it’s not a headphone suitable for EQing. Bass extension is also important. If a headphone doesn’t have sufficient bass an EQ boost might cause distortions.

If you’re at all interested in this like I said look up AutoEQ and oratory1990 wiki on reddit.

I have a BT5.0 audio dongle by Avantree, the DG60. That’s pure audio - so when you hook it up via USB WIndows sees it as a sound card. With that dongle I also get the higher quality recording profile that goes up to 16khz. The 700s are 100% recording for me at higher quality with both the DG60 and the dongle I linked to.

To be clear - I listened to the audio through an Audacity recording on both and it’s clear 5 has wide band audio and 4 doesn’t.

Here’s the Windows property page: https://imgur.com/a/n0I8LLi

Edit: Just tried the BT 4 dongle again, weirdly windows paired to the Bose without me having to re-pair it and it showed up as 16khz too. The last time I did it it was definitely 8khz.

Thanks for posting this - all very odd and likely not BT5 as claimed but they’ve frankenstined some BT audio chipset like from the DG60 to a regular windows one.

Hey, good info! This sounds a lot like HDMI to me. Are TV’s HDMI 2.1? Well they support a load of the features but not full 48gbps bandwidth, unless it’s the LG C9.

The B variant has BT5.0 features (https://www.realtek.com/en/products/communications-network-ics/item/rtl8763b).

The big thing I got from the BT5 dongle was moving from the lower recording quality, to higher. I actually made a recorded in Audaciy on the two dongles and clearly the BT5 one has the 16khz profile. Again verified in Windows settings. BT4 simply didn’t record high quality.

The other advantage of BT5 is no stuttering with the Bose 700 connected to multiple devices and truly wireless connectivity like to my Air Pods Pro will be improved.

After my own test I concur - when you use the Platronics USB dongle it makes it really obvious and isolates the issue to BT profile selection.

No I meant the WH XM3. I did the same thing as you and compared the graphs from 0db. You have to weight the ANC to what bothers us most - brains pick up human voices - in the range of 80-180hz the APPs with mem-foam cancel more. In the treble range the over ears do more - as a function of their form factor with passive isolation.

The dummy head also has none of the nuisance factors in real world cases - like hair, odd jaw shapes, and glasses that could prevent a tight seal. I find any memory foam tips seal far better than any over-ears on my own head. I’ve A vs B’d the APPs and WHXM3, WFXM3 and Bose 700 against white noise and cafe noise in my 7.1.4 theater. The APPs are significantly better for me. The crazy thing is you can even combine the APPs with the WHXM3’s because they have spacious ear cups - so you can wear them over the APPs!

You might be interested in this video: https://www.head-fi.org/threads/anc-is-more-complicated-than-it-sounds-advanced-anc-headphone-measurements.934609/#post-15669815

It’s a proposal on how to weight ANC performance on how humans hear. Rtings has a form of that via a weighting system but this one is a little more complex.

Another site has the measurements up. It’s on a GRAS coupler too, so very high quality: https://www.0db.co.kr/REVIEW_0DB/1370151

Just use google translate - but the graphs are there.

No reduction in ANC. Slight difference in bass, likely to give less ear drum suck effect.

More interestingly, on that site the cheap memory foam tips are also measured - they give a massive increase in ANC performance. So much so that the APP’s now clearly out perform the WH XM3s.

Been working at home and on the road since 2011. I’ve found the best headset for me is the Platronics Edge. The in ear hook design works great. It leaves your ear open so you feel natural during calls. It also has an excellent noise gate processor. The UC version also has a USB dongle which means soon as you plug it into whatever laptop/desktop you use - it’ll automatically set it to the right output and you don’t need to pair.

There’s a Voyager 5200, which is similar. Rtings will soon have a review up - outstanding noise gate processor - good as the Bose 700 in many respects. But as a glasses user I found the over ear hook design a little annoying and I couldn’t just take it off and put it on quickly for calls. The biggest thing with the Edge is soon as you put it into your ear it answers the call.

The Bose 700 is great for extreme situations outside though. Think of doing a conf call in a cafe or subway station.

The one that’s really surprised me is the After Shockz Air. The beauty of those is that they leave your ear completely open. You can wear them all day and do other things in the house while you’re working. but soon as a call you’re waiting for comes in you’re ready to answer.

For anyone interested a YTer has compared them. It’s obvious it’s improved. https://www.youtube.com/watch?v=g8d0UDCF3N4&t=331s

He doesn’t say how he was measuring it though. You can clearly hear it though. I suspect he uses a MiniDSP HaTS that the other bud is on.

Mine just got the update.

I also cleaned all the speakers and mics with blue tac. That’s completely solved my buzzing/ratling problem with foot steps and body noises.

The biggest change I can feel is that they’ve stopped ear drum suck. It feels like the Bose 700 now, they’ve probably improved the filter coming out from ANC to non ANC frequencies. It’d be very good if rtings could re-test once the back log from Covid has cleared!

They sound fantastic now, like on day 1 of my purchase.

Unlikely it’s an error - more likely that rtings just measured with the transparency mode on. Last time they probably measured with both transparency and ANC off.

A poor fit will most likely show up in the lower bass range for the frequency response graphs too. That’s not evident here.

Does anyone know if a factory reset reverts firmware back to factory state also?

Thanks Ian

I suspect Apple have tried to minimise the ear drum suck effect, much like Bose did with the 700 series. There’s a relationship between how much bass you can cancel and how quickly you slope out of that that impacts this effect. See Brent Butterworths article on it.

A shame they don’t give the option to choose the highest cancelling. Mine have also auto updated despite barely ever having them connected to my iPhone. They really didn’t do a great job on a plane ride on the new firmware.

Thanks Ian, can you please show the graphs before and after the update?

Thanks for that link - glad they finally recognised the issue.

Ian - any change on getting those measurements posted? Will be very interesting now to compare different firmware versions since Apple are working on the issue themselves.

Thanks so much Ian. Have a great Christmas break 😊

Thanks so much Ian. Were there differences in the FR? From oratory’s measurements on Reddit I understand that the Airpods Pro EQ based on ear canal length. Guess that would be hard to test without different dummy heads.

Wonder what causes the mass hysteria at ANC performance on firmware updates? I’m sure it’s some unknown psychoacoustics. Completely different user basis that Bose/Sony have to Apple and there’s mass reports.

Man it’s hard to be rtings - you contribute the best objective data on the market and continually improve. But you get comments like this.

IMHO you shoudln’t need to faf around with uncomfortable earphone fitting techniques that are potentially dangerous (https://medium.com/@70M70M/apple-hides-b7be9492c758). In an age where the airpods pro can provide class leading noise cancellation from a comfortable earbud design the rest of the market seems so far behind.

Also the AutoEQ project already has pre computed EQ’s to various targets available for any headphones on rtings

NFC works great for this. It’ll even switch the headphones on when they are off. It’s a shame Windows doesn’t support it though

It is confusing how to disable it - you need to go into the Bose Music App and then select ANC off from there.

Critical listening tends to mean horrible sharp sibilant treble that’s confused for detail.

I have these - they are the best sounding truly wireless by far that I’ve tried. They’re extremely open too - like airpods so have that open back feel.

I’m a headphone insider - I think paywalls are a more transparent way of funding a site than ads. The types of consumers that will consult rtings for reviews will also use cash back sites, discount codes etc. which may mean advert revenue won’t be there.

The paywall means that rtings will be able to fund their business while maintaining an unbias view. That’s sorely lacking in todays internet and similar to consumer reports of old.

Very impressed with the initial work on sound bars - hoping you add smart speakers to it! Anachoic data for frequency response is available on many sources for loud speakers - but smart speakers is much harder to get a hold of.

Rtings ratings are flat relative to a target curve. There’s an established body of research on the Harman Target curve that shows listener preference for a curve that replicates a loud speaker in a real room, including boundary gain. That includes a generic HRTF

It’s simple to shape your own preference now days with EQ. AutoEQ is a great example of that.

There’s a new mark 2 out in January. Willn have the same chip as the XM3, so slightly improved ANC

From the Amazon reviews that seems to be a common issue. A shame, I’m sure if you got a good seal with comply and the like they’ll be by far the best noise cancelling on the market

The thing to look for is a front vent that equalises pressure.

An engineer on that reddit thread wrote

You‘ll see the presence of a front venting in measurements, when the bass response is dropping at the subbass even when the seal in the coupler is perfect.

For example the WI-1000x has the front vent - that’s why it rolls off at around 20hz.

I read the Thomas Farago’s entire article on Medium - very enlighting and quite frighting. As a user of IEM’s I started to worry.

There’s a brief thread on reddit that mentions some of the design principles that engineers use to avoid this situation.

Because insert-type earphones create a physical seal, where the volume of air between the earphone and the eardrum is sealed from the outside. This is why insert-type earphones have much better bass response (because they can make use of the pressure-chamber effect), but the downside is that whenever you move the earbud only slightly, pressure is excerted onto the eardrum, which can be quite dangerous (it’s not lethal, but it can be dangerous)

Is this a problem? No. Any engineer worth his salt will use venting holes in the front volume as part of the earphone design. This is a small hole that physically connects the volume between earphone and eardrum with the outside world. The way this is done is by adding a small hole in the shell in front of the earphone’s diaphragm (the part that moves to create sound pressure). This hole allows for the equalization of static pressure, meaning that at very low frequencies (below 20 Hz, not audible) the air pressure is released into the outside rather than being pressed against the eardrum.

Care must of course be taken so that only inaudibly low frequencies are affected, this is done by tuning the physical dimensions of the venting hole. Ideally it would be long (more than a few millimeters), thin (cross-sectional area of far less than a square millimeter) and sufficiently damped. Damping can be done by applying an acoustic mesh over the vent, or by making the vent so thin that the friction of the air against the walls of the vent will provide enough damping.

Front vents do pose an engineering challenge and especially a manufacturing challenge, and they do increase the cost of production, which is why on some cheaper earphone designs they are omitted.

You will also find a lot of true wireless earbuds without front venting holes (because true wireless earbuds are typically designed by electronics companies rather than by acoustics companies).

From what I’ve read from owners the better noise cancelling could be due to clamp force - the newer model clamps on your head tighter.

The real innovation this year is that ear drum suck has been addressed for those that experience it.

Hi Ian,

Just off the bat - it wasn’t an either or with me for the WI-1000x vs the XM3’s. Both serve their purpose so I didn’t make a wrong purchasing decision based off rtings. Really love all the work you do.

It’s really hard to conform to the objective methodology that you (rightly) conform to with some aspects of headphones. I know that on reddit Sam in your team was toying with the idea of purchasing a new coupler - that’ll be a method of it.

In the meantime it’d be great if you could add a “personal impressions” section. You have all the great measurements that are repeatable and comparable. But I think you’re running into difficulties with upcoming technology like speaker virtualization, surround sound over headphones and in-ear isolation because ultimately there’s no real objective measurement you can make on those as it’s very personal. But because your reviewers have listened to hundreds of headphones and are really experts their opinion carries more weight for me. Than say, CNET or Engadget who aren’t used to taking measurements.

For inear noise cancellers I’d just like a section that goes into how your reviewer tried various situations in real life and which he found best. You could do this in various locations - the Montreal metro gets pretty loud I know! And North America has those horrifically loud trucks that are regulated against in Europe.

At home I’m lucky enough to have a 5.2.4 surround system with dual subs that have an f3 of 6hz. So in order to test noise cancelling I play various sounds off YouTube like airplane noise, busy coffee shop, traffic etc. I put my reciever on all channel stereo so that the sound comes from all around to try and mimic a natural environment where sound comes from everywhere. In those situations it’s very easy to A vs B two headphones and see which one has better isolation performance.

You don’t necessarily need a large setup for that type of test too. Because of pressure vessel gain and boundary re-enforcement, small sealed subs get to very high SPL and very deep in small rooms. One SVS SB2000 in a 4x3.2m office I have gets to that f3 at 6hz.

Your reviewer could then post their impressions after those two types of tests.

For over ears I think this isn’t needed because the entire cup covers the ear so the ear canal shape and seal is a non issue.

The London Underground/Metro system regularly goes over 95db for long stretches. It can certainly lead to hearing damage - https://www.bbc.co.uk/news/uk-england-london-42791299

I’ve found the rtings isolation test doesn’t match up to my perception. The WI-1000x is shown as far less isolating than the XM3’s. So I got the XM3’s - but they were far worse. Almost did nothing on the London Underground except for bass.

The inner ear fit of the WI-1000x with the hybrid sony likely passively isolates a ton of noise for me that the dummy head doesn’t replicate. And it’s astonishing just how good they are in extreme environments. So much so that I use them when operating power tools - far better than foam earplugs even.

Hi Sam,

Audeze mentioned they are beta testing the 3d effect without head tracking. So should be released in the future

The AutoEQ project posts Panoramic EQ’s and you can use the first five indpendently of the other 5 for EQ’s that only have 5 bands. Unfortunately the A50 only has 3 bands available for a Q setting so it isn’t as simple to translate the Auto EQ Panoramic EQ to the Astro as I initially thought.

Yes! I found exactly the same issue on mine - when I was cleaning up the house it was very annoying - even with the auto mode.

You can’t really review any HRTF synthesised surround sound systems because it’s so personal. Creative has the Super XFI out, which maps your ears/head to compute a custom HRTF that should work very well.

But I do get why rtings don’t review the surround sound. It would be nice to hear about the feature in the main review more though. Especially as the industry is gearing up towards more virtual surround systems - Dolby has their own headphone too.

Thanks for the response Sam - interesting info. I’ve read Floyd Toole’s book too on the circle of confusion.

I was very disappointed with my DT990 sound. Any of my Sony’s (WI-1000x or WH-1000) sound so much better and it’s because of this sharp sound and much better bass.

The only that’s improved it is HeSuVi’s EQ preset for the DT990 that makes them sound much more like regular headphones.

Critical listening headphones aren’t for everyone I guess.

You can already try hundreds of HRTF presets including Dolby Atmos/DTS:X and Direct Sound ones with HeSuVi. https://sourceforge.net/projects/hesuvi/. Depending on which you have personal preference for it can really increase the perception of moving the music outside your head and into the room.

Creative has a product - the Super X-Fi that takes pictures of your ears/head and computes a custom HRTF. I think the future of headphone sound is pretty much having a flat response down to 10hz, and low distortion. Then DSP can move the signal outside your head. Rather than worrying about the actual headphones interaction with the pinna.