Logo
Back
Benchmark Tracker

Closing the distance to SOTA.

Subnet 122 vs. the published recommendation literature on NDCG@10 and Recall@10.

Eval Set 2 · NDCG@10

COMING · NEXT EVAL SET
SOTA ceiling · ~0.156
Published SOTA range · 0.03–0.07
Bitrecs · next eval set

Tracking ndcg_at10_bm25_appliances_1000_medium — BM25-prefiltered re-ranking over a 1000-item candidate set, not full-universe retrieval. Published SOTA clusters in ~0.03–0.07 on full-universe Amazon Beauty (MT4SR+SDA, BSARec, eSASRec). Ceiling shown at 0.156 reflects the documented 3× protocol gap (Petrov & Macdonald replicability study) — direct numerical comparisons across protocols are not apples-to-apples.

Eval Set 2 · Recall@10

COMING · NEXT EVAL SET
SOTA ceiling · ~0.200
Published SOTA range · 0.05–0.09
Bitrecs · next eval set

Tracking recall_at10_bm25_electronics_500_medium (a.k.a. HR@10) — BM25-prefiltered re-ranking over a 500-item candidate set, not full-universe retrieval. Published SOTA clusters in ~0.05–0.09 on full-universe protocols. Ceiling shown at 0.20 accounts for the well-documented protocol variance — recall scales with candidate-set size, so prefiltered scores are naturally higher.

References— baselines and methodology
Classic baselines
SASRecICDM '18
Kang & McAuley. Self-Attentive Sequential Recommendation · Amazon Beauty NDCG@10 ~0.0126–0.0416
BERT4RecCIKM '19
Sun et al.. Sequential Recommendation with Bidirectional Encoder Representations from Transformer · Amazon Beauty NDCG@10 ~0.0187–0.0396
Strong recent baselines
SASRec+RecSys '23
Klenitskiy & Vasilev. Turning Dross Into Gold Loss: Is BERT4Rec really better than SASRec? · NDCG@10 0.0327
DuoRecWSDM '22
Qiu et al.. Contrastive Learning for Representation Degeneration Problem in Sequential Recommendation · NDCG@10 ~0.0546
CARCARecSys '22
Rashed et al.. Context and Attribute-Aware Sequential Recommendation via Cross-Attention · papers-with-code SOTA (CARCA Abs + Con)
BSARecAAAI '24
Shin et al.. An Attentive Inductive Bias for Sequential Recommendation beyond the Self-Attention · NDCG@10 ~0.0415–0.0551
2024–2025 frontier
MT4SR + SDAWWW Companion '24
Amazon Science. Rethinking Sequential Relationships: Improving Sequential Recommenders with Inter-Sequence Data Augmentation · NDCG@10 0.0579 (MT4SR), 0.0634 (+SDA_inter)
ActionPiecearXiv '24
Hou et al.. Tokenization / semantic-ID sequential recsys · NDCG@10 ~0.0424
eSASRecarXiv '25
Petrov et al.. Enhancing Transformer-based Recommendations in a Modular Fashion · NDCG@10 0.0523
LLM-based recsys
Kusano et al.RecSys reproducibility '25
Kusano, Akimoto et al.. Revisiting Prompt Engineering: A Comprehensive Evaluation for LLM-based Personalized Recommendation
LLM2BERT4RecRecSys '23
Harte et al.. Leveraging Large Language Models for Sequential Recommendation · +15–20% NDCG on Beauty vs BERT4Rec
GLoSSOARS workshop '25
Acharya et al.. Generative Language Models with Semantic Search for Sequential Recommendation · +30% NDCG@5 on Beauty vs ActionPiece
gSASRecRecSys '23
Petrov & Macdonald. Overconfidence in sequential recsys (Best Paper)
Protocol variance
Replicability studyRecSys '22
Petrov & Macdonald. A Systematic Review and Replicability Study of BERT4Rec for Sequential Recommendation · Same model, NDCG@10 0.0546–0.156 (3× protocol gap)