PeerCoPilot: A Language Model-Powered Assistant for Behavioral Health Organizations

2025-11-19

AI Summary

Behavioral health organizations (PROs) frequently lack sufficient resources and personnel to concurrently address clients’ multifaceted needs—including mental health treatment, substance use disorder interventions, and social determinants of health (e.g., housing, employment). To address this gap, we introduce PeerCoPilot: the first retrieval-augmented generation (RAG)-based large language model assistant designed specifically for peer support specialists. It integrates a rigorously vetted, localized resource database comprising 1,300+ evidence-informed services and enables natural-language interaction to generate personalized recovery plans, define measurable goals, and match clients to appropriate supports. Our key contribution lies in the domain-specific adaptation of RAG—enhancing factual accuracy, clinical relevance, and trustworthiness. In a real-world evaluation at CSPNJ (serving >10,000 individuals), human assessments by 15 peer providers and 6 clients yielded >90% adoption rate. PeerCoPilot is now deployed at scale and undergoing continuous expansion.

Technology Category

• Humans and AI: Planning and Decision Support for Human-Machine Teams
• Application Domains: Humanities & Computational Social Science
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Behavioral health conditions, which include mental health and substance use disorders, are the leading disease burden in the United States. Peer-run behavioral health organizations (PROs) critically assist individuals facing these conditions by combining mental health services with assistance for needs such as income, employment, and housing. However, limited funds and staffing make it difficult for PROs to address all service user needs. To assist peer providers at PROs with their day-to-day tasks, we introduce PeerCoPilot, a large language model (LLM)-powered assistant that helps peer providers create wellness plans, construct step-by-step goals, and locate organizational resources to support these goals. PeerCoPilot ensures information reliability through a retrieval-augmented generation pipeline backed by a large database of over 1,300 vetted resources. We conducted human evaluations with 15 peer providers and 6 service users and found that over 90% of users supported using PeerCoPilot. Moreover, we demonstrated that PeerCoPilot provides more reliable and specific information than a baseline LLM. PeerCoPilot is now used by a group of 5-10 peer providers at CSPNJ, a large behavioral health organization serving over 10,000 service users, and we are actively expanding PeerCoPilot's use.

Problem

• Assists peer providers with daily tasks in behavioral health organizations
• Helps create wellness plans and step-by-step goals for service users
• Locates reliable organizational resources to support behavioral health goals

Innovation

• LLM-powered assistant for peer providers
• Retrieval-augmented generation with vetted resources
• Supports wellness plans and step-by-step goals