Software development may often seems simple and straightforward. At the very least, it may seem as though software engineers have a decent idea of the costs of developing some software system. However, as Frederick Brooks notes in No Silver Bullet: Essence and Accidents of Software Engineering, it can quickly become a business nightmare. Unfortunately, the nature of software means that it will likely never have drastic reductions in development costs from a single advancement in technique or technology. Brooks explains how software is inherently complex because ideal software should not include repeated implementation (think DRY: Don’t Repeat Yourself, make a subroutine that can be called more than once). Additionally, software revolves around arbitrary human constructs that are subject to constant change rather than nature. Plus, software is invisible, and intangible, making it more difficult to visualize and understand. Brooks also outlines several potential “silver bullets” to make software engineering more efficient, but given that he wrote No Silver Bullet in the 1980’s, I believe examining current working software engineering solutions alongside Brook’s speculations is worthwhile.
When I think about massive software projects with daunting software engineering problems, the first company that comes to mind is Google. According to Why Google Stores Billions of Lines of Code in a Single Repository by Rachel Potvin and Josh Levenberg, 95% of the software used by its developers worldwide is stored in a single repository. At first, it sounded insane that a company with so many applications and thousands of developers working on those applications across the globe are all committing code to one massive repository. But, Google couldn’t become the giant that they are today if their developers spent most of their time sorting through thousands of merge conflicts. Google has mastered time-sharing, but not in the sense that Brooks described. Brooks claimed that in the time it takes to compile and execute a program, a developer can forget what they were working on. Therefore, as technology got better, it would take less time to compile a program, meaning developers would remember where they left off, which increases productivity. Even though nearly every program can be compiled virtually instantaneously today, time-sharing is still something to continue perfecting. As applications get larger, and more and more developers are assigned to work in a single repository, a single developer’s code base may be several commits behind the current version. Today, a developer may not forget what they were working on in the time it took to compile their code, but the application they were working on may have changed while they were working. Software such as Git and Subversion have made it possible for developers to work on a single repository at the same time, but it’s not flawless.
For instance, in Kode Vicious, a reader asked George Neville-Neil when they should give up cherry-picking changes into a branch and merge. Neville-Neil responded by saying that most developers end up in nightmarish cherry-picking situations because they rarely merge whatever code they are working against into their own code. Therefore, for moderately sized repositories, the issue of being far behind a remote code base can be fixed by getting in the habit of merging often. However, Google’s repository is far from moderately sized. Even a developer with perfect merging habits could be far behind the latest version with each commit due to the sheer size and scope of Google’s repository. Google overcame this issue by using trunk-based development coupled with high programming standards. Trunk-based development is where all developers commit to a single trunk, which is similar to the master branch in git. There are no long-running branches, meaning fewer conflicts because there is only one true code base and no confusion over which branch to merge into a local branch and vice versa. Google also implements automatic testing before a commit is merged and enforces strict code review so that bad changes are unlikely to be merged into the trunk.
In my opinion, the most interesting about Google’s time-sharing advancement, trunk-based development using Piper and CitC, is that it is still no “silver bullet” for Google’s massive repository. It’s simply a better engineering solution compared to what was available decades ago. The cost to maintain code integrity is high in trunk-based development compared to branch-based development. Also, Google has to control permissions on files that could be damaging to their repository, such as certain configuration files, making open source contributions difficult in their trunk-based development environment. In fact, Google’s Android and Chrome teams use Git versioning and are not a part of Google’s massive repository because open source collaborations are essential to those applications. Despite impressive innovations in software engineering since the time No Silver Bullet was written, it seems as though Brooks’ claim that there will never be a “silver bullet” has yet to be disproven 30 years later.