Test Bench 1.3: Changelog

What's Changed?

With this iteration of the vacuum test bench, we've completely overhauled our floor performance tests to more clearly illustrate debris pickup performance. We've listed the changes made to our test methodology below.

Test Group	Change
Hard Floor Pick-Up	New test on hard floors using mixed debris types.
High-Pile Carpet Pick-Up	New test on high-pile carpeting using mixed debris types.
Low-Pile Carpet Pick-Up	New test on low-pile carpeting using mixed debris types.
Pet-Hair Pick-Up	New test on low-pile carpeting using dyed pet hair.

Here, we'll examine these new changes and how they can help you find the right vacuum for your needs.

New Tests

Performance (Hard Floor, Low-Pile & High-Pile Carpet)

Our previous method of testing performance on hard floors was done by measuring separate amounts of cereal and rice; we'd only test performance with one debris type at a time. After making three back-and-forth passes across the testing surface, we'd measure the difference in weight between the test surface before and after cleaning. This relatively constrained testing scenario had little to do with real-world vacuum usage, where you're far more likely to encounter a mix of debris types in larger amounts. The relatively small amount of debris also overly exaggerated differences between similarly-performing vacuums.

For our new testing methodology, we measure out a larger quantity of debris, comprised of cereal, sand, and rice, and spread it onto the testing surface as a mix. Two back-and-forth passes are then made. Unlike the previous testing method, we keep the vacuum head on the ground instead of lifting it. This allows us to make a closer assessment of vacuum head performance, rather than motor performance, for which we already have airflow and suction performances; several vacuums, such as the Miele Complete C3, come with heads that have inconveniently placed bristles that push around larger amounts of debris, instead of allowing the vacuum to pass over it. Instead of determining the vacuum's performance score based on the delta in weight between its empty and filled dirt compartment, we've opted to look at the surface before and after it's been cleaned and assign a subjectively determined score.

OLD

Miele Complete C3 Hard floor test on TB 1.2

Bare floor performance test for the Miele Complete C3 on Test Bench 1.2; this allowed us to lift the vacuum to pass over larger debris. This process gave more powerful machines (like the Miele) an advantage, even if their floorheads didn't allow them to pass over bulky material.

NEW

Miele Complete C3 Hard Floor Test TB 1.3

Bare floor performance for the Miele Complete C3 on Test Bench 1.3. We've included both a video of the test being conducted and a photo of the surface after it's been cleaned. Notice how the Miele's SBD 365 floorhead prevents it from passing over larger debris.

We've also eliminated the 'Performance Over Time' aspect of the performance test; in the vast majority of cases, we noticed little in the way of performance hit as a vacuum's dirt compartment or dirtbag was filled with debris, so we deemed this aspect of the test unnecessary.

Instead of including a bar graph showing an estimated percentage of debris picked up, we've replaced that with a video of the performance test being conducted and a photo of the surface after cleaning it. This new, more robust testing methodology, combined with the clearer, more intuitive manner of displaying performance results, is intended to make it easier to determine what vacuum best suits your needs.

Robot vacuums are tested slightly differently. We set up the test rig as usual, but the vacuum is given as much time as it needs to clean the testing area until it stops. This test is conducted in this way to evaluate raw debris pickup performance independently from a vacuum's ability to provide adequate area coverage.

Performance (Pet Hair)

Our pet hair pickup performance testing methodology has also been similarly redesigned to provide a clearer, more real-world relevant example of performance. We now test for pet hair pickup on low-pile carpeting, including a video of the surface being cleaned and a photo of the surface after we've made the requisite single pass.

OLD

Kenmore Intuition Pet Hair Test on TB 1.2

Pet hair pickup test for the Kenmore Intuition Bagged on Test Bench 1.2.

NEW

Kenmore Intuition Pet Hair Test on TB 1.3

Pet hair pickup test for the Kenmore Intuition Bagged on Test Bench 1.3

Previous test bench iterations allowed for multiple passes over the same area. However, since most people in day-to-day use will want to spend a lot of time cleaning the same small area repeatedly, we've chosen to evaluate single-pass performance.

Let Us Know What You Think!

As always, our test methodology is a work in progress, and we depend on your feedback to improve it. Our methodology can always benefit from more meaningful tests. If you have comments, questions, or suggestions about this or any future updates, reach out to us in the forums.

36 Vacuums Updated So Far

We are retesting popular models first. So far, the test results for the following models have been converted to the new testing methodology. However, the text might be inconsistent with the new results.

60 Vacuums Planned To Be Updated

We are also planning to retest the following products over the course of the next few weeks: