Amazon Warehouse Automation Increases Concerns over Job Loss and Product Selection Inaccuracy
Update 02/03/2023: Amazon is looking at ways to get rid of barcodes!
This might sound like science fiction, but according to Amazon, the Robotics team has developed a new camera system that can identify and match items against the ones listed in the inventory system without scanning barcodes. And it apparently does its job with 99% accuracy! 😲
In a Science blog post, the eComm giant explains that robotic arms are not good at locating and scanning barcodes, especially those that are attached to oddly-shaped items or hard-to-reach areas. Such challenges typically result in mispicks (customers receiving the wrong item) or shipping delays.
To increase warehouse efficiency, Amazon aims to automate identification of items in its fulfillment centers using Multimodal Identification (MMID). This process uses two or more “modalities of information” like text, sizes and shapes instead of barcodes.
For example, when a camera monitors an item moving along the conveyor, it takes the item’s dimensions and appearance to see if they match the reference images stored in Amazon’s database. Once identified correctly, a robotic arm will then pick it up to be put in the correct bin to continue with the next step.
Initially, the match rates ranged between 75% and 80%. The system faced challenges when it couldn’t distinguish the difference between two different colors of Echo Dot – blue or gray.
But after extensive experiments, the system can now put confidence scores to its ratings and flag potential mismatches. Currently, the match rates are at 99%, thanks in part to “Amazon’s inventory systems that know where each product is at each step of the fulfillment process,” thereby reducing the need for barcodes.
According to AI researchers, “the algorithm does not need to match an item against Amazon’s entire catalog of hundreds of millions of products — currently an impossible task. Each item comes from a particular tote, and each tote contains a few dozen products. So, the algorithm only has to match an item against the contents of a single tote.”
Fortunately, this process happens in the early stages of the fulfillment process so any mismatch or mispick can be easily corrected by “recycling the incorrect item back into the system to its correct location” without causing any disruption, says Robotics AI specialist, Doug Morrison.
🤖 Over the past few months, Amazon has been introducing a series of innovative robotic systems to streamline its fulfillment process and make work safer (and faster) for employees.
During the first half of 2022, the tech giant rolled out two stationary robotic arm systems, Robin and Cardinal, to automate roughly half of its fulfillment process, such as grabbing a boxed-up item from the conveyor and scanning the label so that it can be sorted and transported (via mobile robots) to the correct loading dock.
The other half? Retrieving products from shelves and putting them in boxes, a repetitive task typically performed by pickers, which now Amazon aims to automate with the launch of its newest robotic system, Sparrow. 🤔
Potential Impact of Sparrow on Workers
Introduced on November 10th, Sparrow is reportedly capable of recognizing and handling 65% of all pre-packaged goods sold on Amazon.com. That’s around 230 million out of 353 million products that the entire marketplace is supposedly carrying.
If the current iteration of Sparrow can already handle that much inventory, it’s only a matter of time before it can finally do the work of more than one million workers that Amazon employs to pick, stow, and pack 5 billion packages it delivers annually.
💯 Therefore, Amazon workers are right to be worried about:
The eComm giant describes Sparrow as a major technological advancement to support its workers.
However, employees fear that while it could help them meet their productivity quotas, it could also oust them from their job, especially at a time when Amazon is facing multiple issues in its warehouses.
- Excess capacity amid slowing ecommerce growth. After nearly doubling its warehouse footprint in the last two years, Amazon is scaling back its logistics operations by delaying, shutting down, or subleasing more than 60 logistics centers to cut costs.
- Increasing unionization activity. In a report posted by Yahoo in April 2022, Morgan Stanley analyst Brian Nowak speculated that Amazon might double down on its automation program in response to unionization activity. The analysis came after Amazon workers at a logistics facility in Staten Island voted to unionize, which could drive up labor costs (as opposed to robots that can’t demand higher wages or stage protests). The Wall Street analyst estimated that for every 1% of Amazon’s workforce that’s unionized, the company’s annual operating expenses would climb by $150 million.
“If unionization efforts did begin to spread rapidly, it may cause Amazon to increase the pace at which it invests and integrates robotics and labor alternatives into its warehousing efforts,” Nowak said.
- Dwindling pool of workers. Vox’s tech news arm, Recode, reported in June that Amazon is facing a looming labor crisis. Leaked internal documents from mid-2021 show the company could run out of workers in two years if it doesn’t do “a series of sweeping changes” in its “hiring practices, productivity quotas, attendance policies and unequal enforcement of rules,” some of the biggest factors that contribute to Amazon’s high turnover.
According to The Guardian, the eComm giant was losing 150% of its workers annually before the pandemic. By comparison, the retail sector’s annual average turnover in 2021 was 64.6% and in transportation, warehousing and utility, the turnover was only 49%. Amazon is reportedly looking at increasing wages and warehouse automation to delay the impending labor shortage by a few years.
Or, the company could continue business as usual and rely more on robotics to fulfill orders instead of hiring more employees, ultimately wiping out a significant number of their workforce.
“You can’t compete with the robots. They want you to compete with the robots. They want all the employees to compete with them. But who can win against a robot?” a former Amazon warehouse worker, Mohamed Mire Mohamed, told Business Insider.
In a patent filed by the retail giant in 2020, Amazon said that Sparrow’s suction-grip “hand” is designed to replace workers who “pick items from inventory, place items in totes, remove items from totes, place items into bins, remove items from bins, and place items into boxes for shipping.”
But Amazon has denied speculations that its new robotic arm will replace human workers, saying that it’s designed to work alongside its fulfillment line, not against them. 🤔
“Working with our employees, Sparrow will take on repetitive tasks, enabling our employees to focus their time and energy on other things, while also advancing safety. At the same time, Sparrow will help us drive efficiency by automating a critical part of our fulfillment process so we can continue to deliver for customers,” an Amazon spokesperson said in a press release.
Higher Productivity Quotas
A robot identifying and selecting 65% of Amazon inventory for packing is without doubt extremely efficient. However, this could also lead to managers raising performance quotas significantly, requiring employees to do repetitive tasks over long shifts.
For instance, pickers at one warehouse said they had seen their quota grow from 100 to 400 items per hour, increasing their risk for burnout and injury.
While robots can’t get sick like their human counterparts, that doesn’t mean they don’t have flaws.
Impact of Production Inaccuracy on Sellers
Sparrow may help speed up Amazon’s pick and pack process, but issues around production accuracy (e.g., mispicks) should also be addressed, as they could impact sellers and customers in terms of receiving the right order on time.
For example, Amazon’s existing robotic arm system is built with scanners to be able to read labels on products and sort them by ZIP code. If it fails to scan your product correctly, however, it could be placed into a cart where it doesn’t belong, and potentially get lost.
Or, if your product is buried under a pile of objects of varying sizes and shapes, the robot may also fail to detect and pick it up.
⚠️ Amazon themselves are not oblivious to these problems, even admitting to the fact that robots will make mistakes in production as seen during their experiments with Robin.
In this Amazon Science blog post, Bhavana Chandrashekhar, a software development manager at Amazon Robotics & AI, reveals “sometimes, the differences between one package and another are hard to see, even for humans. You might have a white envelope on another white envelope, and both are crinkled so you can’t tell where one begins and the other ends,”
To increase Robin’s success rate, Chandrashekhar’s team “gathered thousands of images, drew lines around features like boxes, yellow, brown and white mailers, and labels, and added descriptions.” This way, when Robin picks up and scans an object, it can quickly compare what it sees with thousands of sample data to find the closest match and stow it away in the right bin.
They also took note of errors, added them back to the training deck, and retrained Robin to improve its accuracy.
The robotics team developed and equipped the robot with a quality assurance system to oversee how it handles packages.
Charles Swan, a senior manager of software development at Amazon Robotics & AI, explains that if Robin detects a problem like dropping a package or placing two parcels (instead of one) onto a sortation robot, it will try to fix it on its own or call for human support if it cannot.
“If Robin finds and corrects a mistake, it might lose some time. However, if that error wasn’t addressed at all, we might lose a day or two getting that product to the customer.”
With Sparrow expected to handle the early stages of the fulfillment process, it’s not hard to imagine Amazon using a more refined training model to perfect its newest robotic arm system before deploying it at scale. One misstep could quickly trigger some sort of domino effect in its fulfillment process.
For instance, to minimize mishandling issues, Amazon is planning to equip its robotic arms with more advanced perception features like the ability to detect deformable products (plastic bottles in mailers) to avoid crushing them with so much pressure from the suction cups and getting them delivered to customers in damaged condition.
Amazon is currently testing Sparrow in a warehouse in Texas.
The company expects a broader rollout in 2023. And when that happens, its fulfillment centers are going to become a lot less labor intensive. Conversely, it could also reduce the need for warehouse workers.
All of the above are the best-laid plans of Amazon and its robotics team. But as for the true test, that comes out in the wild when Amazon releases Sparrow more broadly into warehouses across the country.
And while humans can also make mistakes, 65% accuracy in identification is not nearly as close as what human workers might be able to achieve, at least at this stage. This is to say nothing of any potential damage by robot arms the products may sustain on their way to your customers’ doorstep.
However, this could solve a lot of problems for Amazon and for sellers in terms of faster delivery and, thus, greater sell-through for Amazon. Amazon warehouses working more efficiently could mean another moratorium on restock limits as warehouse space is utilized more efficiently.
Sellers should follow along as this develops and brace themselves for when it launches more broadly. Even as confident as Amazon sounds and as hopeful as they may be, the jury is out on whether this will improve things or become another challenge for sellers to overcome.