From Pre-labeling to Production: Engineering Lessons from a Machine Learning Pipeline in the Public Sector

2025-11-03

AI Summary

Public-sector deployment of machine learning systems faces dual challenges: technical—such as extreme class imbalance and data drift—and organizational—including bureaucratic data access, unversioned datasets, and absent governance feedback loops. This paper reframes ML pipelines as “civic infrastructure,” integrating LLM-assisted pre-annotation, multi-stage routing classifiers, and controllable synthetic data generation to build an auditable pipeline with data provenance, real-time monitoring, and human-in-the-loop validation. Its core contribution lies in prioritizing institutionalized data engineering—not merely model optimization—to ensure transparency, reproducibility, and accountability. Empirical evaluation on Brazil’s Brasil Participativo platform demonstrates significant improvements in system sustainability and public trust. The framework establishes a transferable engineering paradigm for responsible public AI governance. (149 words)

Technology Category

• Machine Learning: Ethics, Bias, and Fairness
• Humans and AI: Human-in-the-loop Machine Learning
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Security and Privacy: Data transparency and provenance

Abstract

Machine learning is increasingly being embedded into government digital platforms, but public-sector constraints make it difficult to build ML systems that are accurate, auditable, and operationally sustainable. In practice, teams face not only technical issues like extreme class imbalance and data drift, but also organizational barriers such as bureaucratic data access, lack of versioned datasets, and incomplete governance over provenance and monitoring. Our study of the Brasil Participativo (BP) platform shows that common engineering choices -- like using LLMs for pre-labeling, splitting models into routed classifiers, and generating synthetic data -- can speed development but also introduce new traceability, reliability, and cost risks if not paired with disciplined data governance and human validation. This means that, in the public sector, responsible ML is not just a modeling problem but an institutional engineering problem, and ML pipelines must be treated as civic infrastructure. Ultimately, this study shows that the success of machine learning in the public sector will depend less on breakthroughs in model accuracy and more on the ability of institutions to engineer transparent, reproducible, and accountable data infrastructures that citizens can trust.

Problem

• Addressing accuracy and sustainability challenges in public sector ML systems
• Overcoming organizational barriers like bureaucratic data access and governance gaps
• Engineering transparent and accountable ML pipelines as civic infrastructure

Innovation

• Using LLMs for pre-labeling data
• Splitting models into routed classifiers
• Generating synthetic data for training