Dora Petrella
How We Built a Machine Learning-Based Recommendation System (And Survived to Tell the Tale)
#1about 5 minutes
Defining the business need for product recommendations
A recommendation system for substitute products is needed across multiple touchpoints to prevent lost sales from out-of-stock items.
#2about 2 minutes
Analyzing the limitations of the existing recommender
The previous system, based on the Jaccard coefficient, produced low-quality recommendations, particularly for new or unpopular items.
#3about 5 minutes
Using the Prod2Vec algorithm for recommendations
The Prod2Vec algorithm, adapted from Word2Vec, learns product relationships by analyzing co-occurrence within user session context windows.
#4about 2 minutes
Improving predictions with Meta-Prod2Vec and metadata
Incorporating product metadata like category and brand into the model (Meta-Prod2Vec) significantly improves recommendation quality for long-tail items.
#5about 2 minutes
Implementing the end-to-end MLOps pipeline
The production system uses dbt for data transformation, a Vertex AI pipeline for model training, and Elasticsearch for efficient vector similarity search.
#6about 3 minutes
Evaluating model performance with offline and online metrics
Offline metrics like NDCG confirmed model quality, while mirror traffic analysis showed a 45% increase in product recommendation coverage.
#7about 3 minutes
Visualizing product relationships with embedding projector
Using TensorFlow's Embedding Projector tool reveals how the model groups similar products into distinct clusters in a high-dimensional space.
#8about 3 minutes
Adopting pragmatic baselines and automated data analysis
Key project takeaways include using simple business-logic baselines for benchmarking and automating exploratory data analysis within the ML pipeline itself.
#9about 1 minute
Understanding the project team and final timeline
The project was completed in nine months by a cross-functional team of data engineers, data scientists, and software developers.
Related jobs
Jobs that call for the skills explored in this talk.
Matching moments
02:56 MIN
Real-world examples of machine learning in e-commerce
Data Science in Retail
Unlock full access
Log in or set up an account to access this feature and more.
01:54 MIN
Real-world applications and key takeaways
Machine learning 101: Where to begin?
Unlock full access
Log in or set up an account to access this feature and more.
05:15 MIN
How AI powers e-commerce from logistics to discovery
Intelligence Everywhere: The Future of Consumer Tech
Unlock full access
Log in or set up an account to access this feature and more.
07:54 MIN
Demo of a unified model and business monitoring dashboard
Deployed ML models need your feedback too
Unlock full access
Log in or set up an account to access this feature and more.
02:19 MIN
Future ideas for personalized vacation planning
Hacking Your Vacation: Using Data for Fun
Unlock full access
Log in or set up an account to access this feature and more.
05:57 MIN
Adopting a holistic AI strategy across business functions
Fireside Chat with Werner Vogels, VP & CTO, Amazon.com & Daniel Gebler, CTO at Picnic
Unlock full access
Log in or set up an account to access this feature and more.
01:29 MIN
Overview of the data and machine learning tech stack
Empowering Retail Through Applied Machine Learning
Unlock full access
Log in or set up an account to access this feature and more.
10:46 MIN
Navigating the machine learning project lifecycle
Intelligent Automation using Machine Learning
Unlock full access
Log in or set up an account to access this feature and more.
Featured Partners
Related Videos
58:21Data Science in Retail
Julian Joseph
37:24Deployed ML models need your feedback too
David Mosen
21:01Hybrid AI: Next Generation Natural Language Processing
Jan Schweiger
29:24MLOps - What’s the deal behind it?
Nico Axtmann
48:01What non-automotive Machine Learning projects can learn from automotive Machine Learning projects
Jan Zawadzki
22:41Empowering Retail Through Applied Machine Learning
Christoph Fassbach & Daniel Rohr
35:16How AI Models Get Smarter
Ankit Patel
44:40DevOps for Machine Learning
Hauke Brammer
Related Articles
View all articles
.gif?w=240&auto=compress,format)
From learning to earning
Jobs that call for the skills explored in this talk.
Nomitri
Berlin, Germany
DevOps
Gitlab
Docker
Ansible
Grafana
+6
Hilo By Aktiia
Lausanne, Switzerland
Intermediate
Machine Learning
Amazon.com, Inc
Senior
PyTorch
Machine Learning