Sabi\'a-4 Technical Report

2026-03-10

AI Summary

This work addresses the limitations of large language models for Brazilian Portuguese in legal reasoning, long-context processing, instruction following, and agent-based tasks by proposing a four-stage training framework. The approach comprises continual pretraining on Portuguese and Brazilian legal corpora, extension to 128K-token context length, multi-domain supervised fine-tuning, and preference alignment. This methodology substantially enhances model performance across legal text generation, multi-turn dialogue quality, and tool invocation capabilities. The resulting model achieves state-of-the-art results across six evaluation dimensions—Brazilian Portuguese dialogue, legal knowledge, long-context comprehension, instruction adherence, standardized examination performance, and agent proficiency—while maintaining an excellent cost-performance trade-off.

Technology Category

• Natural Language Processing: (Large) Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

This technical report presents Sabi\'a-4 and Sabiazinho-4, a new generation of Portuguese language models with a focus on Brazilian Portuguese language. The models were developed through a four-stage training pipeline: continued pre-training on Portuguese and Brazilian legal corpora, long-context extension to 128K tokens, supervised fine-tuning on instruction data spanning chat, code, legal tasks, and function calling, and preference alignment. We evaluate the models on six benchmark categories: conversational capabilities in Brazilian Portuguese, knowledge of Brazilian legislation, long-context understanding, instruction following, standardized exams, and agentic capabilities including tool use and web navigation. Results show that Sabi\'a-4 and Sabiazinho-4 achieve a favorable cost-performance trade-off compared to other models, positioning them in the upper-left region of the pricing-accuracy chart. The models show improvements over previous generations in legal document drafting, multi-turn dialogue quality, and agentic task completion.

Problem

• Brazilian Portuguese
• legal language models
• long-context understanding
• agentic capabilities
• cost-performance trade-off

Innovation

• Brazilian Portuguese
• long-context extension
• legal language modeling
• agentic capabilities
• preference alignment