Hierarchical Long-Term Semantic Memory for LinkedIn's Hiring Agent

2026-04-28

AI Summary

This work addresses the challenge of designing long-term semantic memory for industrial-scale large language model agents, which must simultaneously ensure scalability, low-latency retrieval, privacy preservation, cross-domain generalization, and observability. To this end, the authors propose a Hierarchical Long-Term Memory (HLTM) framework that organizes multi-granularity semantic knowledge through a schema-aligned memory tree and integrates domain-adaptive mechanisms to support diverse application scenarios. The architecture enables, for the first time in LinkedIn’s hiring assistant, an efficient, privacy-safe, and transferable structured long-term memory with transparent provenance. Upon deployment, the system achieves over 10% absolute improvements in both answer accuracy and retrieval F1 score, while significantly advancing the Pareto frontier between query and indexing latency.

Abstract

Large Language Model (LLM) agents are increasingly used in real-world products, where personalized and context-aware user interactions are essential. A central enabler of such capabilities is the agent's long-term semantic memory system, which extracts implicit and explicit signals from noisy longitudinal behavioral data, stores them in a structured form, and supports low-latency retrieval. Building industrial-grade long-term memory for LLM agents raises five challenges: scalability, low-latency retrieval, privacy constraints, cross-domain generalizability, and observability. We introduce the Hierarchical Long-Term Semantic Memory (HLTM) framework, which organizes textual data into a schema-aligned memory tree that captures semantic knowledge at multiple levels of granularity, enabling scalable ingestion, privacy-aware storage, low-latency retrieval, and transparent provenance; HLTM further incorporates an adaptation mechanism to generalize across diverse use cases. Extensive evaluations on LinkedIn's Hiring Assistant show that HLTM improves answer correctness and retrieval F1 significantly by more than 10%, while significantly advancing the Pareto frontier between query and indexing latency. HLTM has been deployed in LinkedIn's Hiring Assistant to power core personalization features in production hiring workflows.

Problem

• long-term semantic memory
• LLM agents
• scalability
• low-latency retrieval
• privacy constraints

Innovation

• Hierarchical Long-Term Semantic Memory
• LLM agents
• schema-aligned memory tree
• privacy-aware storage
• cross-domain generalization