DashCLIP: Leveraging multimodal models for generating semantic embeddings for DoorDash
Existing multimodal models struggle to capture fine-grained semantic relationships between merchant items and user queries on the DoorDash platform. Method: We propose a joint multimodal embedding learning framework that requires no user behavioral history. It employs an image–text contrastive learning objective to align pretrained unimodal encoders (CLIP, BERT) with a customized multimodal encoder. To reduce reliance on proprietary business signals, we introduce the first large-scale relevance annotation dataset synthesized via large language models (LLMs). Furthermore, we enable end-to-end joint optimization of both unimodal and multimodal encoders. Results: Experiments demonstrate substantial improvements in item classification and relevance prediction. In advertising recommendation, the method achieves 12.3% lift in click-through rate (CTR) and 9.7% lift in conversion rate (CVR), validating its cross-task generalization capability and direct business impact.