The Promise and Reality of Continuous Integration Caching: An Empirical Study of Travis CI Builds

2026-01-27

AI Summary

This study addresses the impact of continuous integration (CI) build duration on developer productivity and investigates the underexplored adoption and efficacy of caching as a build acceleration technique. Through a large-scale empirical analysis of 513,384 builds across 1,279 GitHub projects on Travis CI—combining log analysis, pull request intervention experiments, and developer feedback—the work reveals that CI caching is adopted by only 30% of projects, largely due to developers’ limited awareness and the perceived maintenance complexity. Notably, nearly half of previously non-adopting projects accepted caching configurations when proposed via pull requests, with approximately one-third achieving significant speedups. However, widespread issues of cache redundancy and staleness were also observed, underscoring both the feasibility and necessity of optimizing CI caching practices.

Technology Category

• Constraint Satisfaction and Optimization: Solvers and Tools
• Reasoning under Uncertainty: Causality
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

Continuous Integration (CI) provides early feedback by automatically building software, but long build durations can hinder developer productivity. CI services offer caching mechanisms to speed up builds by reusing infrequently changing artifacts, yet little is known about how caching is adopted in practice and what challenges it entails. In this paper, we conduct a large-scale empirical study of CI caching in Travis CI, analyzing 513,384 builds from 1,279 GitHub projects. We find that only 30% of projects adopt CI caching, and early adoption is strongly associated with project maturity, such as more dependencies, more commits, and longer CI lifespans. To understand why many projects do not adopt caching, we submitted pull requests enabling caching in non-adopting projects, and nearly half were accepted or merged. Developer feedback suggests that non- or late adoption mainly stems from limited awareness of CI caching support. We also examine cache maintenance and identify five common activities, performed by 24% of cache-enabled projects. Although one-third of projects see substantial build-time reductions, cache uploads occur in 97% of builds, and 33% of projects contain stale cached artifacts. Finally, our analysis of reported caching issues shows developers mainly struggle with corrupted or outdated caches or request broader caching features. Overall, CI caching does not help all projects, needs ongoing maintenance, and is more complex in practice than many developers expect.

Problem

• Continuous Integration
• Caching
• Build Performance
• Developer Adoption
• Cache Maintenance

Innovation

• Continuous Integration
• Caching
• Empirical Study
• Build Performance
• Developer Practices