Jonathan Creamer
How a Small Team Shrank a Microsoft Monorepo by 94%
#1about 2 minutes
The scale of Microsoft's monorepo problem
A monorepo with 20 million lines of code grew from a manageable 2GB to an unworkable 150GB, prompting an investigation into its exponential growth.
#2about 3 minutes
How automated changelog tooling bloated the repository
The versioning tool Beach Ball generated thousands of changelog files, causing a separate versioning branch to swell to an enormous 130GB.
#3about 9 minutes
Discovering a Git hashing algorithm bug from 2006
A Git expert found that an old hashing algorithm only used the last 16 characters of a filename, causing collisions that prevented proper diffing of changelog files.
#4about 4 minutes
Implementing the new path walk algorithm to fix Git
The solution was a new "Path Walk" algorithm for `git push` and `git repack` that uses the full file path to avoid hash collisions and ensure correct diffing.
#5about 2 minutes
Applying the fix with new Git config and repack commands
Developers can enable the new algorithm for pushes via a `git config` setting and shrink local clones using the `git repack --use-path-walk` command.
#6about 2 minutes
Using the new `git survey` command to find large files
A new built-in command, `git survey`, was created to help developers identify large files, blobs, and binaries in their repository history.
#7about 3 minutes
Best practices for managing large repositories
Beyond the specific fix, general best practices like not checking in binaries and avoiding thousands of files in a single folder are crucial for repository health.
#8about 6 minutes
The broader impact on the open source community
The new algorithm has shown significant size reductions for other large monorepos like Chromium, and the fix is being upstreamed to benefit the entire Git community.
Related jobs
Jobs that call for the skills explored in this talk.
Matching moments
06:19 MIN
How a small team fixed tech bankruptcy incrementally
Shipping Quality Software In Hostile Environments
Unlock full access
Log in or set up an account to access this feature and more.
04:13 MIN
How Git and GitHub created corporate open source
The Future of Open Source
Unlock full access
Log in or set up an account to access this feature and more.
03:30 MIN
How GitOps fosters team growth and experimentation
GitOps keeps focus on apps, not on infrastructure
Unlock full access
Log in or set up an account to access this feature and more.
01:44 MIN
Managing community feedback and contributions
DB UX Design System – How we’ve open sourced our largest inner source project
Unlock full access
Log in or set up an account to access this feature and more.
15:02 MIN
Answering audience questions on GitHub Actions
Lights, Camera, GitHub Actions!
Unlock full access
Log in or set up an account to access this feature and more.
03:01 MIN
How GitHub evolved from a Git wrapper to a social platform
Coffee With Developers - Kyle Daigle, COO of GitHub
Unlock full access
Log in or set up an account to access this feature and more.
07:32 MIN
Deconstructing the overloaded git commit command
Coffee with Developers - Scott Chacon on growing GitButler and the future of version control
Unlock full access
Log in or set up an account to access this feature and more.
07:04 MIN
A case study of a company in tech bankruptcy
Shipping Quality Software In Hostile Environments
Unlock full access
Log in or set up an account to access this feature and more.
Featured Partners
Related Videos
37:05Coffee with Developers - Scott Chacon on growing GitButler and the future of version control
Scott Chacon
24:19Git for Code Reviews
Johannes Haux
42:13Coffee With Developers - Kyle Daigle, COO of GitHub
Kyle Daigle
43:39Reusing apps between teams and environments through Containers
Adrian Kosmaczewski
42:29Improving Developer Happiness with GitOps
Lars Hesel Christensen & Basil Brunner
26:46Keep your code refactorable
Gerrit Stapper
47:42GitOps: The past, present and future
Roberth Strand
32:29GitLab CI pipelines for a whole company
Martin Beránek
Related Articles
View all articles
From learning to earning
Jobs that call for the skills explored in this talk.
Devoteam
HTML
Microsoft Access