Audeze Maxwell Performance Investigation

Goal of our Reviews

Our goal at RTINGS.com is to empower users with the information they need to make a proper buying decision. At the same time, we strive to be transparent and truthful about our work; we're never satisfied with where we are. We constantly aim to improve and have higher standards. However, we know we have a lot of work ahead to deliver better, more accurate measurements. That's why it's important to hear directly from people in the community. Getting your feedback, especially when it's critical, is vital to helping us implement changes that benefit users. We're thankful to users as well as Audeze for pointing out our flaws and challenging us to do better.

Our original measurements and scores.

After our retest: the new measurements and scores.

In the past, Audeze has even helped us with this goal. When we tested the Audeze Mobius in 2018, they provided constructive feedback on the headset’s virtual soundstage capacities and measured battery life. This information allowed us to expand our methodology and cover more features that benefit users. In that same vein, feedback on the Maxwell—not only from the manufacturer but from the community too—can have a positive outcome. We take your feedback very seriously, and we're always ready to learn more and improve. We’re also eager to continue the conversation with Audeze and hope they’ll continue in good faith.

 Timeline of our unit

• February 1st: Ordered our unit from Audeze's website.
• March 20th: Received the product. Users can see its status in the review pipeline.
• March 23rd: Audeze reached out to us and provided us with test notes. While we always look forward to manufacturer engagement with our work, we have our own methodology for testing headphones that allow our results to consistently show user experience across reviews. That said, our test methods are quite similar to those proposed by Audeze.
• April 4th: Testing started. We noticed phase mismatch present in our unit and that non-Bluetooth wireless latency was higher than expected. However, these flaws didn't qualify the unit as broken—the Maxwell performed very well in most other tests. If a product doesn't perform as it should but isn't clearly broken, we refer to the out-of-spec policy. It's the manufacturer's responsibility to define what is out of spec for their products as they're the ones who know how their products should perform.
• April 11th: Published early access results to insiders.
• April 19th: Audeze contacted us again after seeing our public early access review, asking how we obtained the phase response and latency results. They also provided their own test settings for achieving their results. In our response, we provided follow-up measurements and notified them of our out-of-spec policy. We left it to them whether they thought the product was performing correctly or not.
• April 24th: We published our full results.
• April 25th: Audeze chose not to declare their unit out of spec and believed our unit works correctly. We continued to look into our results to ensure the most accurate data possible.

 

Latency

Latency is the amount of delay introduced by the headphones. When there’s a lot of latency, it can cause your audio and visuals to fall out of sync. We don’t expect most people to have issues with latency under <75 ms, but some people are more sensitive to latency than others; some users may find 40 ms too high, and others won’t be bothered by it at 80 ms.

How the Maxwell Performs 

ASIO drivers allow audio software to get direct access to the audio device, bypassing the normal audio pathway. However, these drivers are typically used for audio editing to reduce latency and aren’t what's typically used by most everyday users. As a result, we don’t use the built-in ASIO driver as the sound output, as we don’t expect most users to use this while gaming. However, we know that this method doesn’t allow us to measure the system’s latency, which is something Audeze rightly highlighted.

 

We also tested latency using Audacity, a software similar to Reaper, and miniDSP E.A.R.S. The manufacturer notes that their results fall around 20 ms. We achieved the following results using this setup:

• MiniDSP E.A.R.S.: 163ms
• Our Latency Rig: 96ms

With that in mind, when testing these headphones with more typical and common drivers, the latency varies significantly. This variation exists regardless of the software used to test (Reaper or Audacity), the computer used (we tested on 4 PCs with different specs), and the equipment (we tried both miniDSP E.A.R.S and and XLR mic array).

All of our passes. We also added the SteelSeries Arctis Nova 7 Wireless as a comparative unit. This headset allows us to check how much latency was originally tested, any variation in latency, and how large the variance in latency is between the SteelSeries and Audeze headsets. As you can see in this screenshot, there's a lot of variation.

Based on the following retests, latency ranged from 4 ms, which is lower than the manufacturer reports, to 52 ms on a different PC. It's nowhere near what we originally measured. While using the same USB port on the same PC, replugging the dongle could also result in a fluctuation of ~15-25 ms.

This contrasts with our original test results, which were performed three times using Reaper software, resulting in our final averaged results of 85 ms via non-Bluetooth wireless and 43 ms via wired USB. These three measurements were among the high side of our testing and aren't representative of their typical latency. As a result, we updated the measurements in our review to be more representative.

Our new measurements show non-Bluetooth wireless latency is ~34 ms and wired USB is ~18 ms. This means that latency depends on your setup and that some people may have higher or lower latency than others. By learning how latency fluctuates depending on the device and by unplugging/replugging the dongle into the same port, we improved and re-evaluated our test. We thank Audeze for pointing out these flaws in our methodology and pushing us to explore new, more accurate ways of measuring latency.

Although we want to be able to quickly adapt and change our tests, that's not always possible. It takes a lot of time, research, feedback, and tweaking to add or update a test since we want to ensure that it meets high standards and is replicable across products. These are the limitations of what we do. However, we're keen on continuing the conversation and improving as we go.

The previously mentioned setup differs from how we test Bluetooth latency on mobile devices. While we measure latency on PCs using a local click-track feedback loop, Android and iOS devices are measured using Youtube. At the moment, we can't account for test rig latency. We recognize the limitations of using this kind of methodology and are working on updating our tests to reflect a more realistic and consistent user experience.

Phase Response 

Phase response is one test within our larger Imaging test. This test measures the amount of difference between the left and right driver's phase response. We also use a weighted filter that emphasizes audible ranges like the low-mid over harder-to-hear ranges like the mid-to-high treble. Well-matched in phase response ensures a stable stereo image, which is especially important for gaming. Conversely, mismatch can cause inaccuracies in the stereo image. While our imaging test is specific to our unit, this test can be a larger indicator of quality control and ergonomics. It's also worth pointing out that planar magnetic drivers are more susceptible to imaging issues in general as their drivers are more complex compared to more common dynamic drivers.

How the Maxwell Performs

We test headphones on our HMS II rig using REW software. We play a sine sweep in the left and right driver separately, and we repeat this process across five different reseats to ensure consistent results. Keep in mind that we do these five different reseats throughout all of our sound tests. This is primarily because each person can experience sound differently, depending on the shape and size of their auditory features (like their pinna shape) and how they prefer to wear their headphones. The value of these five passes is fully demonstrated in Frequency Response Consistency as it allows us to highlight the variance in frequency response depending on fit, positioning, and seal. While wear and variance don't factor in the same way when it comes to imaging, running five tests acts like a fail-safe and helps us catch any irregularities in driver matching.

During our original testing, we noticed a difference in impulse between the L/R drivers, and we had to perform several passes to ensure a similar impulse response. By repeating these passes, we also observed that over 80% of these passes had phase mismatch. In addition, this phase mismatch is audible subjectively and causes sounds like male voices to be recessed in the track. The difference is visible when you compare all ten (five left and five right) phase passes:

When simplified by using only two sets of passes, the difference is even clearer:

Two phase response passes of the left and right driver. 

A pattern emerges: a peak around the 300Hz range is only present in our right driver measurements. To ensure that this peak wasn't introduced by our test equipment, we did a pass with the headphones placed backward by putting the right driver on the left ear and vice versa. The result: the peak is still present in the right driver but is now being registered by the left ear. This helps us rule out our equipment as the cause of this mismatch.

Although we noticed this peak while testing and writing our review, phase mismatch was the primary indicator that something could be wrong with the headphones, as they otherwise performed very well across the board. However, we're confident in our phase response results. Even when stepping outside of our usual methodology, we can replicate the mismatch with the right driver. We've also noticed several other users note channel balance issues. Their comments help us establish whether the issue could be strictly related to our unit or if there's a potential for other users to encounter driver mismatch. While some of these comments don't directly correlate with what we experienced, they suggest that the issue isn't limited to only our unit.

 

Our unit has phase mismatch, and this may be due to the complex nature of the drivers themselves. It's also possible that you may experience this with your own unit, but we don't know how widespread or isolated this issue is or if you'll notice mismatch if your unit is affected. However, if you buy these headphones, please let us know your experiences regarding how they perform.

Conclusion

We're confident in our phase response measurements, and we were able to eliminate our test setup as the cause of this mismatch. We've also learned from criticism regarding how we measure latency and found new ways to measure latency that align with how users will use and experience the product.

We're grateful to everyone who has given us feedback, and we'll work to improve this. If you're interested in keeping up the conversation, join us in discussing the future of our headphones testing in this article: Improving Headphone Testing: Measurements vs. Preferences and More! - RTINGS.com. Also, if the idea of empowering consumers with information on products excites you, consider joining the team to help us with our mission by visiting our careers page! 

- The RTINGS.com Headphones Team