Brian Harry, a Microsoft executive behind Team Foundation Server and Azure DevOps, writes about developer tools and Microsoft's Git work.
HN links it mainly for the posts on moving the Windows codebase into one huge Git repo and the VFS for Git extensions that made it workable.
> the complete editing history of the Linux kernel (probably the largest Git repo ever created)
Linux is definitely not the largest git repo ever created [0]. The big corporate monorepos are definitely larger; I know MS has moved Windows to git, and itself claims it to be the largest ever created (~300GB as of 2017, per [1]). Google and Facebook both eschew git, though.
Finding data on the largest open repos is more difficult. The largest classes of projects are those that develop in monorepos that implement critical operating system [2] functionality, browser engines, and compiler implementations. The shortlist I'd make comes out to these projects (in no particular order):
* gcc
* LLVM
* Mozilla
* Chromium
* Linux
* OpenJDK
I haven't finished downloading all of these repos (my disk is begging me to stop right now), but it looks Linux is larger than gecko-dev by a very thin margin (so a putative gecko-dev that included comm-central with its CVS history as well would easily outstrip Linux), and Chromium seems to be an order of magnitude over both.
[0] To be clear here, I'm mostly thinking in terms of primarily textual repositories. Repositories with large binary assets are clearly not relevant for your means.
[1] https://devblogs.microsoft.com/bharry/the-largest-git-repo-on-the-planet/, although https://news.ycombinator.com/item?id=14411724 claims that the 300 GB measures the size of the checked-out directory on disk, not the putative size of a full .git…
microsoft: in 2017, the Windows git repo contained ~3.5 million files & used 300GB when checked out as a git repo [1], In 2020, the Office git repo contained ~3 million files [2].
facebook: In 2011, the main site had around 9.2 million lines of code "excluding numerous backend services" [3]. In 2014 the android codebase was 4 million lines of code and the main site without backend code was 62 million lines of code [3]. In 2014, when facebook were talking about scaling mercurial, the main source repo was described as "many times larger than even the Linux kernel, which checked in at 17 million lines of code and 44,000 files in 2013". Around 2018 (?) facebook started using "eden", a fork of mercurial [5].
If we make the working assumption that expected lines-of-code / file of 2013 linux kernel is representative of all repos, then that gives around 386 lines-of-code / file.
loc(microsoft) >= loc(windows in 2017) + loc(office in 2020) ≈ 386 loc / file * (3.5m + 3m) ≈ 2.5 billion lines of code
The linked blog posts from microsoft & facebook talking about scaling version control systems are fairly interesting!
[1] https://devblogs.microsoft.com/bharry/the-largest-git-repo-on-the-planet/
[2] https://devblogs.microsoft.com/devops/introducing-scalar/
[3] https://www.quora.com/How-many-lines-of-code-is-Facebook
[4] https://engineering.fb.com/core-data/scaling-mercurial-at-facebook/
[5] https://github.com/facebookexperimental/eden
I have not analyzed the full potentials and benefits of Diversion but I would not agree with the statements you made about the Git. I think you should not focus on Git in your pitch.
>>it was built for a very different world in 2005 (slow networks, much smaller projects, no cloud)
Slow network: why is this a negative thing? If something is designed for a slow network then it should perform well in a fast network.
Mush small project: I do not agree. I can say that it was not designed for very very large projects initially. But many improvements were made later. When Micorosoft adopted Git for Windows, they faced this problem and solved it. Please look at this https://devblogs.microsoft.com/bharry/the-largest-git-repo-on-the-planet/
No cloud: Again I would not agree. Git is distributed so should work perfectly for the cloud. I am not able to understand what is the issue of Git in the cloud environment.
>>In our previous startup, a data scientist accidentally destroyed a month’s work of his team by using the wrong Git command
This is mostly a configuration issue. I guess this was done by a force push command. IFAIK, you can disable force push by configuration.
To follow up on this, for anyone who might have seen it, a scan of the Word app for Android indicates that it contains
* Google Firebase Analytics
* HockeyApp
The former I've seen in about 90% of apps I've downloaded from the Google Play Store. The latter claims to be "the best way to collect live crash reports, get feedback from your users, distribute your betas, and analyze your test coverage." Oh, and Microsoft owns it. [1]
Neither seems particularly unusual to me. I wonder what has caused this much outrage:
* Assumptions about the nature of the "tracking" ("specializes in predictive profiling" leaves a lot to the imagination)
* Assuming that other apps aren't doing the same thing
* A general expression of anger at all apps that have embedded scripts like these
[1] https://devblogs.microsoft.com/bharry/microsoft-acquires-hockeyapp/
Git was once fast for my use too. Then I returned to Microsoft to work on building Windows. Even after 2.5 years of improvement,[1] "git is slow" is still one of the most common complaints about the Windows engineering system.
On my Skylake+SSD desktop, it takes over ten seconds to switch branches with git checkout, several of which are used just to calculate the "your branch is ahead of origin/your-branch-name by N commits" value.
We are making many great strides here at Microsoft to improve git performance, such as with VFS for Git which my Windows repo clone uses. But with hundreds of gigabytes in a unversioned download of the source tree, there is only so much we can do.
[1] since https://devblogs.microsoft.com/bharry/the-largest-git-repo-on-the-planet/
Microsoft had issues git’s ability to scale. According to this[1] they found and submitted solutions to allow git to scale to their needs and worked with the git team. It sounds like Facebook wasn’t partnering with them to solve the problem? The article mentions Microsoft had repos with 6m files and says things now take seconds.
I’m not personally dealing with repos of this size, but it sounds like this is a problem of the past, thanks to Microsoft. I have a hard time believing Facebook would be a bigger beast than Windows.
[1] https://devblogs.microsoft.com/bharry/scaling-git-and-some-back-story/