Oriel — AI & technology worth noticing, every day.

GitHub's new Git storage architecture

Web page · https://github.blog/engineering/architecture-optimization/inside-githubs-new-git-storage-architecture

First seen by Oriel on .

AI summary

GitHub describes Project Rubicon, a Git storage architecture that separates reference coordination, durable object storage, serving, and maintenance to support agent-scale repository traffic.

  • Internal testing supports 5,000 concurrent agents and 7,200 pushes per second on one repository while maintaining consistently low read latency.
  • Spokes routes requests to disposable virtual replicas whose local disks are rebuildable caches, while Azure Blob Storage holds Git objects and Ref-CAS stores references and pack manifests.
  • Ref-CAS uses quorum-committed compare-and-swap transactions that lock only touched references, enabling tens of thousands of reference operations per second and cheap unchanged reads.
  • Separate compaction nodes lease repository shards, merge packs directly in durable storage, and keep pack counts effectively flat as pushes increase; garbage collection uses dedicated workers similarly.
  • A single-blob reference log reached only the mid-hundreds of writes per second, while segmented blobs improved writes but made reads check every segment.

#git storage #azure blob storage #distributed systems #agentic development

Generated by gpt-5.6-sol. Check the original for details.

Metric trends

Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained

Points

Score reported by the source.

  • Hacker News: 236, latest observation ; +232 over observed period since
  • Lobsters: 29, latest observation ; +29 over observed period since

Comments

Comment count reported by the source.

  • Hacker News: 73, latest observation ; +73 over observed period since
  • Lobsters: 7, latest observation ; +7 over observed period since

Aliases

SourceExternal IDURL
Hacker News50397099https://github.blog/engineering/architecture-optimization/inside-githubs-new-git-storage-architecture/ Read discussion
Lobstersy3hvmehttps://github.blog/engineering/architecture-optimization/inside-githubs-new-git-storage-architecture/ Read discussion