See the previous 0.8 changelog.
What's Changed?
With an increasing number of robot vacuums boasting some form of hazard avoidance system, we thought it was high time to put manufacturer claims to the test. We've updated our general maneuverability and obstacle adaptability testing to better assess the performance of various hazard avoidance systems.
| Test Group | Description |
|---|---|
| Our Verdict | New Multi-Surface Household usage Got rid of Bare Floors, Low-Pile and High-Pile Carpet usages New Performance Usages: Debris Pickup: Hard Floor, Debris Pickup: Carpet, Debris Pickup: Pet Hair, Obstacle Handling |
| Household Adaptability | A new test that serves as a more accurate assessment of unit maneuverability in an average room |
| Obstacle Avoidance | A new test to see how consistent and reliable vacuums are in spotting and avoiding smaller obstacles |
| Height Clearance | A new test to measure how well a vacuum can fit under obstacles |
| Threshold Clearance | A new test to measure how tall of an obstacle or threshold a vacuum can cross |
Our Verdict
We now include a Multi-Surface Household usage, which is meant to provide an overview of how well-suited a vacuum is to cleaning a whole-home environment with a mix of surface types. A brief explainer: For this composite usage scoring, we tend to weigh debris pickup on hard floors more than pickup on carpets, as we expect most users to use their robot vacuums on the former more often than the latter. We've also incorporated the new Obstacle Handling test scores into the existing usage boxes.
The new Performance Usages provide a quick overview of essential unit performance metrics at the top of the review, so you can get a snapshot of our own assessment of the product without having to dig too deep into the results.
| Test Bench 0.8 | Test Bench 1.0 |
|---|---|
 |  |
A comparison between Our Verdict sections for Test Bench 0.8 and Test Bench 1.0.
New Tests
Household Adaptability
This test reimagines our old Maneuverability test. This new test provides a more real-world-accurate picture of the sorts of environments a robot vacuum will have to navigate in, with a broader mix of larger furniture and the introduction of a full-length mirror. A reflective surface like that can confound models with LIDAR sensors.
| Test Bench 0.8 | Test Bench 1.0 |
|---|---|
 |  |
A comparison between the test environments for the Maneuverability test from Test Bench 0.8 and the Household Adaptability test from Test Bench 1.0.
Obstacle Avoidance
This new test is all about assessing how consistently a robot vacuum can spot, identify, and react to hazards. We use a few items to simulate ground clutter: a pair of shoes, socks, a USB cable, and even fake pet waste, with a duplicate of each item being placed on a carpeted surface. We score performance using a rubric that takes into account the number of times a vacuum touched an obstacle, avoided an obstacle completely, or got caught on an obstacle in such a way that required a tester to intervene.
Since obstacle avoidance systems don't always deliver consistent performance, testers run this test three times and collate results from all three runs to provide a more representative result.
Height Clearance
This height clearance test is run in a boundary box. A set of six slats is positioned above the unit, each slat being placed 0.5 cm (0.2") lower than the next. They range in position from 10.5 cm (4.1'') to 8.0 cm (3.1") from the ground. Vacuums are scored based on how far they can travel before their top surface touches a slat.
Threshold Clearance
This is another fairly simple maneuverability test. Vacuums are placed within a boundary box, with a set of interchangeable wooden barriers that get progressively higher; the Low Threshold component of the test uses barriers measuring 1.3 cm (0.5") and 1.9 cm (0.75") in height, while the High Threshold component uses thresholds measuring 2.5 cm (1.0") and 3.8 cm (1.5") in height. Vacuums are scored based on which point they can't proceed any further.
Let Us Know What You Think!
We're just getting started with our updates to testing obstacle handling performance. If there's something you want us to consider, investigate, or add to our test benches, let us know!
20 Robot Vacuums Updated So Far
We are retesting popular models first. So far, the test results for the following models have been converted to the new testing methodology. However, the text might be inconsistent with the new results.
- Dreame L10s Ultra
- Dreame L40 Ultra
- Dyson 360 Vis Nav
- Narwal Freo X Ultra
- Roborock Qrevo
- Roborock Qrevo Curv
- Roborock Qrevo Pro
- Roborock Qrevo S
- Roborock S7 MaxV
- Roborock S8 MaxV Ultra
- Roborock S8 Pro Ultra
- Roborock S8+/S8
- Shark PowerDetect RV2820ZE
- bObsweep Dustin
- eufy L60
- eufy Omni C20
- eufy S1 Pro
- eufy X10 Pro Omni
- iRobot Roomba j7/j7+
- iRobot Roomba j9+
14 Robot Vacuums Planned To Be Updated
We are also planning to retest the following products over the course of the next few weeks:
- Roborock Q5 Max+
- Roborock Q5/Q5+
- Roborock Q7 Max+/Max
- Shark AI Ultra Robot [AV2501AE, AV2501S, AV2511AE, RV2502AE]
- Shark IQ 2-in-1 [RV2410WD, UR2410WD]
- Shark Matrix Plus
- Shark Matrix Robot [RV2300S, RV2310AE]
- Wyze Robot Vacuum
- iRobot Roomba S9
- iRobot Roomba i1
- iRobot Roomba i3
- iRobot Roomba i4
- iRobot Roomba i7
- iRobot Roomba i8+
Comments
Test Bench: Obstacle Avoidance, Clearance and Adaptability: Main Discussion
What do you think of our article? Let us know below.
Want to learn more? Check out our complete list of articles and tests on the R&D page.