GitLake: Git-for-data for the agentic lakehouse

2026-07-09

AI Summary

This work addresses the absence of Git-like data versioning in existing lakehouse architectures, which hinders collaborative development among multiple agents and human-in-the-loop review workflows. To bridge this gap, the paper introduces Git semantics into lakehouse systems by extending Apache Iceberg to support lakehouse-wide commits, branches, and merges—elevating single-table snapshots to a unified versioning model. This enables agents to operate on isolated branches while ensuring atomic, cross-table changes during release. The core abstractions are formally modeled and verified using Alloy to guarantee semantic correctness. The proposed system has been deployed in production, demonstrating the feasibility and effectiveness of the collaborative workflow it enables.

Abstract

We present GitLake, a Git-for-data design for an agent-first lakehouse. The system lifts single-table Iceberg snapshots into lakehouse-wide commits, branches, and merges, letting agents work on isolated branches while humans review and publish changes. Pipelines run on temporary branches and publish through a final merge, so all outputs become visible atomically or none do. Finally, we report production lessons as well as correctness insights from a preliminary Alloy model of our core abstractions.

Problem

• Git-for-data
• agent-first lakehouse
• data versioning
• branching and merging
• atomic publishing

Innovation

• Git-for-data
• agent-first lakehouse
• atomic merge
• data versioning
• Iceberg snapshots