What Reviews Tell Us About How Walmart Is Really Doing

• Like many retailers, Walmart has struggled with supply chain issues. Those issues show up in customer reviews of Walmart stores and out-of-stock items.
• We tracked stock-related mentions in reviews from 101 Walmart stores over a 10 month period. We discovered review volumes increased as consumer satisfaction went down.
• This exercise shows, at scale and over time, how reviews offer unique insights about businesses, their strengths and problems – to whomever is interested.

In its May earnings call, Walmart acknowledged the impact of COVID on the company's supply chain and store inventories. At the time, Walmart expressed confidence that they had solved these issues with increased inventory and order lead times:

We continue to monitor industry challenges related to transit and port delays, and our merchants have taken steps to mitigate the challenges, including adding extra lead time to orders. The fundamentals of the US business continue to improve, and we're confident we have the strategy, structure and people in place to serve customers and reach our goals this year and beyond.

In their August 19 earnings call, Walmart admitted to ongoing out-of-stock issues and announced the addition of more cargo leasing and freight capacity. Yet, despite its multiple efforts to fix these inventory challenges, we're still seeing a steady stream of inventory related review complaints coming in at a high rate.

This made us curious as to whether we could identify and track inventory issues at Walmart (or any other store for that matter) via their review corpus.

Tracking Reviews from 101 Walmart Stores

With the help of GatherUp* we began tracking reviews from 101 Walmart stores across the United States. Using its natural language processing and the autotagging features, we created tags to filter reviews based on review comments about stock levels. In total we uncovered 9 words and phrases in the reviews we used as a proxy for stock levels: bare, empty, nothing, out of, restock, shelf, shelves, stock, stock levels, stocked.

We established a date range from November 2019, before the pandemic, through this month. Across these 101 stores, Walmart accrued a total of 185,990 reviews during this period. Walmart has an incredibly high natural review rate; it works out to about 80 reviews per location per month.

Of those ~185,000 reviews, just over 7,500 reviews mentioned stock-related issues. To get a sense of consumer sentiment about stock levels we plotted the average ratings per month over the time period.

What We Found: Review Sentiment

Reviews at scale can offer significant insights, particularly when they occur naturally and in sufficient volume.

In the case of Walmart we see relatively high satisfaction ratings in reviews about stock levels from November, 2019 through January, 2020. In the immediate wake of the pandemic, sentiment subsequently declines until May, 2020 and then shows a partial recovery through October, 2020. Since then, Walmart has seen a regular and slow but steady erosion in sentiment vis-a-vis Walmart's stock levels.

This view of Walmart's inventory problems dovetails nicely with company announcements. It also indicates that Walmart, coming into the busy season, still has a ways to go to achieve inventory levels expected by Walmart customers.

What We Found: Review Quantity

We postulated that the volume of reviews would increase as consumer satisfaction declined. That is exactly what we saw. You can observe a huge spike in review volume peaking in July.

The relationship between inventory problems and quantity of reviews mentioning stock levels becomes even clearer when the graphs are plotted together:

As inventory problems became apparent in March of 2020, the shock and awe felt by consumers led to a flurry of increasingly negative inventory related reviews. Last summer, as problems were being addressed and as consumers began to adjust, we saw ratings improve and review volumes drop. However, since April of this year, the charts are once again diverging, indicating increased consumer dissatisfaction with Walmart stock levels.

Reviews: A Source of Business Intelligence

As a proof of concept, this exercise shows how reviews can provide information about businesses, their strengths and problems -- at scale and over time. The strong correlations between rating sentiment, review volume and alignment with company statements point to the potential value of this method of analysis.

Unlike almost any other form of company information and tools like consumer surveys, the ongoing and frequent consumer opinions offered in reviews provide timely and tangible insights. To the analyst's delight and perhaps Walmart's dismay, this information is equally available to Walmart, competitors and analysts.

Reviews can also be leveraged to understand how well companies like Walmart are transitioning to the new online-offline commerce reality, how well employees are providing customer service and whether stores collectively or individually are maintaining corporate standards.

A Final Thought

Walmart is one of the best companies in the world at logistics and inventory management. The story told by management and reviews should give us all pause. One of the world's most sophisticated retailers is still experiencing problems getting adequate inventory onto shelves. We are 19 months into the pandemic and Walmart has not mastered the realities of COVID and inventory management.

COVID and changing demand have put great strains on the global supply chain. We need to assume COVID is not the last disruption we will see.

Stay tuned for more, as we explore the nexus between company performance reviews and business intelligence insights.