Predicting Custom-Feed Returns for New Bluesky Posts: A Prospective Study

2026-08-13

AI Summary

This study addresses the cold-start routing challenge for new posts within Bluesky’s custom feeds by proposing a ranking task tailored for independently operated streams. Through prospective data collection and delayed annotation strategies, we construct a large-scale benchmark dataset and employ a LambdaRank model to predict the relevance between new posts and specific feeds. Experimental results demonstrate that this approach achieves state-of-the-art performance on the test set, attaining a Capped Recall@10 of 0.7361 and an NDCG@10 of 0.6127. These findings indicate significant improvements in new content distribution accuracy, establishing a reliable paradigm and evaluation benchmark for cold-start recommendations in decentralized social platforms.

Abstract

The conventional approach to cold-start recommendation addresses new users or newly introduced items. Bluesky custom feeds create a different setting: independently operated feeds filter content from a shared public stream. In this setting, newly published posts are the cold-start objects, while the feeds serve as candidates. We propose a cold-start routing task in which a newly ingested public post is the query and all rankable feeds in the monitored panel are ranked according to whether each will subsequently return it. We build a still-evolving collect-first, label-later benchmark dataset. The collected dataset covers a fixed panel of 5,000 monitored feeds and contains 17.804 million public posts, 1.865 million observable post--feed return records, and 625,083 valid feed polls. The labels record whether a post is observed among a feed's AppView Top-50 results in at least one poll during the 24 hours after publication. The current experiments use two disjoint 24-hour test folds, each paired with a 24-hour training window and separated by a 24-hour outcome-availability gap. Evaluation is conditional on the 602,186 test posts that have at least one positive observed label and satisfy the metric eligibility criteria; these posts account for 9.04% of all 6,661,658 test posts. Across the two folds, LambdaRank achieves the best equal-fold mean values among the evaluated models: 0.7361 for capped Recall@10, 0.6127 for NDCG@10, and 0.7749 for Hit@10.

Problem

• Cold-start recommendation
• Custom feeds
• Content routing
• Bluesky

Innovation

• Cold-start Routing
• Bluesky Custom Feeds
• Collect-first Label-later Benchmark
• Post-feed Return Prediction
• LambdaRank