Git infrastructure for agent-scale development

GitHub is rebuilding its Git architecture while GitHub keeps running — no maintenance window, no version change, no disruption to existing workflows. Pushes grew 4.9x year-over-year (690M to 3.35B monthly). Pull request merges grew 4x. GitHub Actions ran 3.26B times in September 2026 — 4x YoY. The new design decouples storage from compute (authoritative data lives in Azure Blob Storage, lightweight cache workers serve requests), minimizes coordination (only reference updates need agreement), and delivers up to 35x higher write throughput with independent read scaling.

Agent-scale development demands a new Git architecture

Agents commit after nearly every action. Thousands of agents on a single branch. CI pipelines fetching the same branch tips thousands of times per minute. This is not human-scale development — it is a different architecture entirely.

"Agentic software development is driving the next architectural shift. With developers and agents working concurrently in repositories that receive millions of commits a day, these workloads demand a different Git architecture." — Brian Celenza, Principal Software Engineer, GitHub

GitHub's existing architecture has served developers well for years. Every repository is stored by Spokes, which keeps a full copy on the local disks of several fileservers, five by default. Those fast local disks let Git operations read native repository data with low latency, and the extra copies provide redundancy while spreading reads across fileservers. When a push updates a reference, a three-phase commit protocol uses a quorum to ensure that CI, the web UI, and API clients see a consistent repository state. That pairing serves a billion repositories today.

However, the mechanism GitHub uses for durability is the same one it uses for scale. The copies on disk are the source of truth, so adding read capacity means adding another durable replica. Every replica participates in every write, so a push is only as fast as the slowest replica in its set. The net effect: adding replicas to absorb read load makes writes slower.

For most repositories, this tradeoff works well. At the highest activity levels, it becomes a ceiling: adding read replicas adds overhead to writes, losing a replica reduces read capacity, and losing quorum stops writes entirely.

Apache-2.0 inspired principles guide the redesign

GitHub is rebuilding while GitHub keeps running — no maintenance window, no version change. The architecture is running GitHub's production workloads during the rebuild — no disruption to existing workflows.

1. Minimize coordination

Not all coordination is equal. The critical path of a push is the reference update — the part that truly needs agreement. Object storage, validation, and secret scanning can proceed in parallel. GitHub is redesigning so the critical path shrinks to the reference update itself — everything else runs independently.

"Coordinate only what needs agreement. The part of a push that truly needs agreement is the reference update itself. Storing the underlying objects, validating object connectivity, and secret scanning are much more work, but most of it can happen in parallel to other writes. That shrinks the critical path of a push to the small step that needs coordination, so the rest of the work no longer delays the acknowledgment." — github.blog

2. Decouple storage from compute

Current architecture: repository copies on local disks serve as both durable storage and the layer answering Git requests.

New architecture: authoritative data lives in Azure Blob Storage — durability and replication at Azure scale, already battle-tested. A lightweight compute layer optimizes for throughput at lowest latency.

  • Reads scale without adding durable copies. Read capacity comes from lightweight cache workers. The authoritative copy lives in object storage underneath — CI fan-out, agent fleets, and large clones can all be served without adding work to every push.

  • Let each layer do one job. Authoritative repository data lives in Azure Blob Storage, which already provides durability and replication at Azure scale. The compute layer is optimized for throughput at the lowest latency.

  • Recover faster from failures. When storage and compute are coupled, losing a host reduces both capacity and durability, and recovery means rebuilding a full repository copy. When they are separate, losing a compute worker is closer to a cache miss: a replacement worker can start serving requests right away and fill its cache from durable storage as traffic arrives.

  • Match capacity to demand. Compute workers can be added or removed as traffic changes instead of provisioning for peak load in advance. A repository going through a burst of activity, like a release or a new agent fleet coming online, can get extra capacity for the burst. Once it passes, that capacity goes away.

3. Keep people in control

The system preserves branch protections, required reviews, audit logs, and repository visibility. Maintainers still review contributions. Security teams still investigate suspicious access. On-call engineers still have dependable automation and enough observability to understand why a deployment failed.

As agents take on more work, the people who own the code can still review, understand, and approve it.

"Keep people in control of their code. If the system isn't helping the people and organizations who use it, and isn't under their control, it isn't worth building. As agents take on more of the work, the people who own the code can still review, understand, and approve it." — github.blog

Watch for these architectural shifts

1. Storage-compute separation across the industry

GitHub's approach will likely influence other Git providers and internal architectures. Expect to see:

  • Object storage backends for Git (Azure Blob, S3, GCS)
  • Background maintenance workers operating independently of serving path
  • Cache layers between compute and storage for read scaling

2. Reduced coordination for writes

Watch for:

  • Minimal quorum requirements — only reference updates need agreement
  • Parallel processing of independent operations (object validation, secret scanning)
  • Background compaction that doesn't block serving

3. Agent-aware workflows

As agent development matures, expect:

  • Branch management optimized for concurrent agent operations
  • Merge conflict resolution designed for agent-driven changes
  • CI/CD pipelines specifically tuned for agent fleets

4. Embedded search in repositories

Infino's approach — embedding BM25 and vector indexes in Parquet files alongside repository data — hints at a trend: Git repositories with native search capabilities. Instead of separate code search infrastructure, search becomes a first-class property of the repository itself.

Current limitations and risks

At the highest activity levels, the current Spokes system becomes a ceiling: add replicas → writes slow down. Lose a replica → read capacity drops. Lose quorum → writes stop entirely.

For most repositories, this tradeoff works well. At the highest activity levels, it becomes a ceiling: adding read replicas adds overhead to writes, losing a replica reduces read capacity, and losing quorum stops writes entirely.

Related reading and sources

  • Infino: The Retrieval Engine for Agent-Scale Search — Infino's approach to collapsing search stacks into one Parquet table, built by the OpenSearch team at AWS
  • Infino architecture — deep dive into the superfile format, manifest-tracked tables, and cache path design