 |
Recsperts - Recommender Systems ExpertsRecommender Systems are the most challenging, powerful and ubiquitous area of machine learning and artificial intelligence. Author: Marcel Kurovski
Recommender Systems are the most challenging, powerful and ubiquitous area of machine learning and artificial intelligence. This podcast hosts the experts in recommender systems research and application. From understanding what users really want to driving large-scale content discovery - from delivering personalized online experiences to catering to multi-stakeholder goals. Guests from industry and academia share how they tackle these and many more challenges. With Recsperts coming from universities all around the globe or from various industries like streaming, ecommerce, news, or social media, this podcast provides depth and insights. We go far beyond your 101 on RecSys and the shallowness of another matrix factorization based rating prediction blogpost! The motto is: be relevant or become irrelevant! Expect a brand-new interview each month and follow Recsperts on your favorite podcast player. Language: en Genres: Mathematics, Science, Technology Contact email: Get it Feed URL: Get it iTunes ID: Get it |
Listen Now...
#29: Transformers for Recommender Systems with Craig Macdonald and Sasha Petrov
Episode 30
Wednesday, 27 August, 2025
In episode 29 of Recsperts, I welcome Craig Macdonald, Professor of Information Retrieval at the University of Glasgow, and Aleksandr “Sasha” Petrov, PhD researcher and former applied scientist at Amazon. Together, we dive deep into sequential recommender systems and the growing role of transformer models such as SASRec and BERT4Rec.Our conversation begins with their influential replicability study of BERT4Rec, which revealed inconsistencies in reported results and highlighted the importance of training objectives over architecture tweaks. From there, Craig and Sasha guide us through their award-winning research on making transformers for sequential recommendation with large corpora both more effective and more efficient. We discuss how recency sampling (RSS) reduces training times dramatically, and how gSASRec overcomes the problem of overconfidence in models trained with negative sampling. By generalizing the sigmoid function (gBCE), they were able to reconcile cross-entropy–based optimization results with negative sampling, matching the effectiveness of softmax approaches while keeping training scalable for large corpora.We also explore RecJPQ, their recent work on joint product quantization for item embeddings. This approach makes transformer-based sequential recommenders substantially faster at inference and far more memory-efficient for embeddings—while sometimes even improving effectiveness thanks to regularization effects. Towards the end, Craig and Sasha share their perspective on generative approaches like GPTRec, the promises and limits of large language models in recommendation, and what challenges remain for the future of sequential recommender systems.Enjoy this enriching episode of RECSPERTS – Recommender Systems Experts.Don’t forget to follow the podcast and please leave a review.(00:00) - Introduction (04:09) - About Craig Macdonald (04:46) - About Sasha Petrov (13:48) - Tutorial on Transformers for Sequential Recommendations (19:24) - SASRec vs. BERT4Rec (21:25) - Replicability Study of BERT4Rec for Sequential Recommendation (32:52) - Training Sequential RecSys using Recency Sampling (40:01) - gSASRec for Reducing Overconfidence by Negative Sampling (01:00:51) - RecJPQ: Training Large-Catalogue Sequential Recommenders (01:21:37) - Generative Sequential Recommendation with GPTRec (01:29:12) - Further Challenges and Closing Remarks Links from the Episode:Craig Macdonald on LinkedInSasha Petrov on LinkedInSasha's WebsiteTutorial: Transformers for Sequential Recommendation (ECIR 2024)Tutorial Recording from ACM European Summer School in Bari (2024)Talk: Neural Recommender Systems (European Summer School in Information Retrieval 2024)Papers:Kang et al. (2018): Self-Attentive Sequential RecommendationSun et al. (2019): BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from TransformerPetrov et al. (2022): A Systematic Review and Replicability Study of BERT4Rec for Sequential RecommendationPetrov et al. (2022): Effective and Efficient Training for Sequential Recommendation using Recency SamplingPetrov et al. (2024): RSS: Effective and Efficient Training for Sequential Recommendation Using Recency Sampling (extended version)Petrov et al. (2023): gSASRec: Reducing Overconfidence in Sequential Recommendation Trained with Negative SamplingPetrov et al. (2025): Improving Effectiveness by Reducing Overconfidence in Large Catalogue Sequential Recommendation with gBCE lossPetrov et al. (2024): RecJPQ: Training Large-Catalogue Sequential RecommendersPetrov et al. (2024): Efficient Inference of Sub-Item Id-based Sequential Recommendation Models with Millions of ItemsRajput et al. (2023): Recommender Systems with Generative RetrievalPetrov et al. (2023): Generative Sequential Recommendation with GPTRecPetrov et al. (2024): Aligning GPTRec with Beyond-Accuracy Goals with Reinforcement LearningGeneral Links:Follow me on LinkedInFollow me on XSend me your comments, questions and suggestions to marcel.kurovski@gmail.comRecsperts WebsiteDisclaimer:Craig holds concurrent appointments as a Professor of Information Retrieval at University of Glasgow and as an Amazon Scholar. This podcast describes work performed at the University of Glasgow and is not associated with Amazon.
