GitHub's new Git storage architecture
First seen by Oriel on .
AI summary
GitHub describes Project Rubicon, a Git storage architecture that separates reference coordination, durable object storage, serving, and maintenance to support agent-scale repository traffic.
- Internal testing supports 5,000 concurrent agents and 7,200 pushes per second on one repository while maintaining consistently low read latency.
- Spokes routes requests to disposable virtual replicas whose local disks are rebuildable caches, while Azure Blob Storage holds Git objects and Ref-CAS stores references and pack manifests.
- Ref-CAS uses quorum-committed compare-and-swap transactions that lock only touched references, enabling tens of thousands of reference operations per second and cheap unchanged reads.
- Separate compaction nodes lease repository shards, merge packs directly in durable storage, and keep pack counts effectively flat as pushes increase; garbage collection uses dedicated workers similarly.
- A single-blob reference log reached only the mid-hundreds of writes per second, while segmented blobs improved writes but made reads check every segment.
#git storage #azure blob storage #distributed systems #agentic development
Generated by gpt-5.6-sol. Check the original for details.
Metric trends
Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained
Points
Score reported by the source.
- Hacker News: 236, latest observation ; +232 over observed period since
- Lobsters: 29, latest observation ; +29 over observed period since
Comments
Comment count reported by the source.
- Hacker News: 73, latest observation ; +73 over observed period since
- Lobsters: 7, latest observation ; +7 over observed period since