Towards Automated Scoping of AI for Social Good Projects

2025-04-28

AI Summary

AI for Social Good (AI4SG) faces critical bottlenecks in cross-domain problem scoping and scarce domain-expert resources. Method: This paper proposes a Problem Scoping Agent (PSA) powered by large language models (LLMs), the first systematic application of LLMs to early-stage AI4SG problem scoping. PSA integrates knowledge-enhanced prompt engineering with multi-source scientific literature retrieval and synthesis to enable end-to-end generation of structured technical proposals from ambiguous societal needs. Contribution/Results: We introduce a dual-evaluation framework—combining human blind review and AI-based assessment—to rigorously evaluate proposal quality. Empirical results demonstrate that PSA-generated proposals match expert-level performance in completeness, feasibility, and innovation, while reducing problem-scoping time and human effort significantly. PSA thus provides a scalable, methodology-driven foundation for accelerating AI4SG project initiation.

Technology Category

• Application Domains: Humanities & Computational Social Science
• Multiagent Systems: Distributed Problem Solving
• Humans and AI: Planning and Decision Support for Human-Machine Teams

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

Artificial Intelligence for Social Good (AI4SG) is an emerging effort that aims to address complex societal challenges with the powerful capabilities of AI systems. These challenges range from local issues with transit networks to global wildlife preservation. However, regardless of scale, a critical bottleneck for many AI4SG initiatives is the laborious process of problem scoping -- a complex and resource-intensive task -- due to a scarcity of professionals with both technical and domain expertise. Given the remarkable applications of large language models (LLM), we propose a Problem Scoping Agent (PSA) that uses an LLM to generate comprehensive project proposals grounded in scientific literature and real-world knowledge. We demonstrate that our PSA framework generates proposals comparable to those written by experts through a blind review and AI evaluations. Finally, we document the challenges of real-world problem scoping and note several areas for future work.

Problem

• Automating labor-intensive problem scoping in AI4SG projects
• Addressing scarcity of dual-expertise professionals in AI initiatives
• Generating expert-level proposals using LLMs for societal challenges

Innovation

• LLM-based Problem Scoping Agent (PSA)
• Generates expert-level project proposals
• Integrates scientific and real-world knowledge