Retail & Neo4j: Personalized Promotion & Product Recommendations

Today’s retailers face a number of complex and emerging challenges.

Thanks to lower overhead and higher volume, online behemoths like Amazon can deliver products faster and at a lower price, driving smaller retailers out of business.

In order to compete, retailers need a new approach – and the fresh technologies that go along with it.



In this new series on Neo4j and retail, we’ll break down the various challenges facing modern retailers and how those challenges are being overcome using graph technology. This week, we’ll start with personalized promotions and product recommendations.

Neo4j Powers Personalized Promotion & Product Recommendation Engines

Delivering real-time recommendations to online shoppers is a proven way to maximize revenue. It improves the customer experience and increases sales.

However, shoppers expect finely tuned product recommendations and react poorly to one-size-fits-all or uninformed recommendations (e.g., “I’ve already bought that. Why are they showing it to me again?”). To be effective, recommendations must be personalized based on the individual consumer’s preferences, shopping history, interests and needs – in addition to what’s already in their current shopping cart.

Real-time recommendations require data products that connect masses of complex buyer and product data (and connected data in general) to gain insight into customer needs and product trends.

This cannot be achieved with relational database (RDBMS) technology; the SQL queries are too complex and take too long to provide recommendations in real time. The same goes for big data processing technologies like Hadoop and Spark. These technologies work well for something like email recommendations – which are delivered once a day – but they are not real time.

The Challenges with Traditional Systems

Data stored in relational databases and other silos is too disconnected and slow for real-time recommendations.

By design, graph database quickly query customers’ past purchases, as well as instantly capture any new interests shown in their current online visit, both of which are essential for making real-time recommendation engines.

Because relationships are treated as first-class entities in a graph database, retailers can connect customers’ browsing history with their purchasing history, as well as their offline product and brand interactions.

This enables a real-time product recommendation algorithm to utilize a customer’s past and present choices to offer personalized promotion recommendation. No offline pre-compute is necessary, eliminating the associated delay.

With a graph database, a retailer’s entire inventory, supply chain, customer data and other systems are all connected.

Furthermore, in order to counter dynamic pricing from the likes of Amazon, retailers need the ability to change pricing and promotions at any level of a product hierarchy in real time. For example, they must be able to mark down all 60-inch televisions by 10% for the next two hours if the right economic and competitive factors indicate such a move is necessary.

Similarly, retailers must be able to implement competing promotions. They might reduce all smartphone prices except Apple iPhones due to Apple’s strict pricing guidelines.

Real-time promotions such as these involve complex rules that are made simple when handled by a graph database like Neo4j. The database may hold millions of relationships that have only one parent node. With a graph database, retailers can change one relationship type rather than a thousand products and all their prices.

Case Study: Walmart

Walmart became the world’s largest retailer by understanding its customers’ needs better than any competitor. An important tool in achieving that understanding is the Neo4j Graph Database.

Walmart’s Brazilian ecommerce group wanted to understand the behavior and preferences of its online buyers with enough speed and in enough depth to make real-time, personalized “you may also like” recommendations. However, Walmart quickly recognized that it would be difficult to deliver such functionality using traditional relational database technology.

“A relational database wasn’t satisfying our requirements about performance and simplicity, due to the complexity of our queries,” said Marcos Wada, software developer at Walmart.

To address this, Marcos’s team decided to use Neo4j, the leading graph database. Matching a customer’s historical and session data is trivial for graph databases, enabling them to easily outperform relational and other NoSQL data products.

Walmart deployed Neo4j in its remarketing application run by the company’s ecommerce IT team based in Brazil, and it has been using Neo4j in production since early 2013. Neo4j enables Walmart to understand online shoppers’ behavior, as well as the relationship between customers and products.

As a result, the retailer has also been able to up- and cross-sell major product lines in core markets.

“With Neo4j we could substitute a complex batch process that we used to prepare our relational database with a simple and real-time graph database. We could build a simple and real-time recommendation system with low latency queries,” Marcos said.

Case Study: Top-Ten Retail Company

One Top 10 US-based, bricks-and-mortar retailer turned to Neo4j after its burgeoning online operation was almost overwhelmed by the volume of customer traffic it attracted on Cyber Monday 2015.

The company was running its site on an IBM DB2 relational database, and on Cyber Monday 2015, it offered an across-the-board 15% discount to site visitors. While the retailer had pulled in more customers than any other bricks-and-mortar rival – one of the project’s target metrics – the price paid was unacceptable: The site’s checkout function kept working that day, but 90% of customer traffic was delayed.

As one senior company executive said: “We pushed a lot of guests to the site and we were very successful in terms of volume. But the reality was we got significantly more traffic than we ever projected, and we couldn’t handle it. We protected checkout so the site functioned. But we disappointed way too many guests, and that’s never okay, period.”

The biggest bottleneck was the crucial but complex personalized promotions process, where the company invites shoppers to add last-minute extras to their online cart. To flash up exactly the right recommendations requires software that can instantly analyze the shopper’s cart contents and their buying history, and dig through 15-30 layers of data – such as promotion types, qualifying manufacturers, product names and categories – all in real time.

This proved beyond a conventional relational database like DB2. So, the retailer considered Neo4j, which is optimized to rapidly carry out such complex searches among masses of connected data.

The company already knew its biggest rival, Walmart, had turned to Neo4j to provide the best web experience for its customers (see case study above), so in mid-2016 the company rolled out both a new Neo4j-based front-end and backend to its website, transforming the company’s real-time personalized promotions engine and online cart promotion calculations.

Neo4j now processes 90% of the retailer’s 35 million-plus daily transactions – which involve between three and 22 hops across different layers of data – in 4 milliseconds or less. And during Q4 2016 – the vital Christmas retail period – the company’s digital sales rose 34% to a record high, helped by the friction-free Neo4j solution.

Conclusion

An effective product recommendation engine can’t be half-baked or only partially efficient. Either recommendations (or promotions) are timely and relevant or they convince your would-be customers that your ecommerce site only offers stale, pre-computed suggestions.

The only way to craft truly personalized promotions or product recommendations – that consider not only past buying history and current session data – is to use graph technology to power your recommendation engine. As the leader in the space, Neo4j is the graph technology of choice.

In the coming weeks, we’ll take a closer look at other ways retailers are using graph technology to create a sustainable competitive advantage, including customer experience personalization, ecommerce delivery service routing, supply chain visibility, revenue management and IT operations.





Catch up with the rest of the retail and Neo4j blog series:

• Customer Experience Personalization
• Ecommerce Delivery Service Routing
• Supply Chain Visibility & Management
• Pricing & Revenue Management
• Network & IT Management for Retailers