Unity Build

Unity builds, a.k.a. unified build or jumbo builds, is a technique that consists in concatenating several C or C++ files in one before invoking the compiler. This generally leads to faster compilation time in part because it aggregates the cost of parsing the same headers over and over.

It is one of the many approach one can use to reduce C++ software compilation time, alongside precompiled headers and C++20 modules.

It is not an obscure technique: it's officially supported by CMake through CMAKE_UNITY_BUILD (see https://cmake.org/cmake/help/latest/variable/CMAKE_UNITY_BUILD.html).

At some point, the Chromium project supported doing jumbo builds: https://chromium.googlesource.com/chromium/src.git/+/65.0.3283.0/docs/jumbo.md, even if it got rid of it afterward https://groups.google.com/a/chromium.org/g/chromium-dev/c/DP9TQszzQLI

It is also supported by the internal build system used at Mozilla. And the speedup is there, a unified build (the default) runs twice as fast as an hybrid build (--disable-unified-build) on my setup. As a side effect, in pre-LTO area, this also led to better performance as it makes more information available to the compiler.

Wait, did I write hybrid build and not regular build? As it turns out, under --disable-unified-build some parts of Firefox are still built in unified mode, because they require it, probably for historical reason.

It's great to be able to do a unified build. It's not great to have a codebase that does not compile unless you have a unity build: static analyzers are not used to work on non-self contained sources (see https://github.com/clangd/clangd/issues/45), unity build implies a slight overhead during incremental builds. What makes it worse is that developer start to rely on unified build and get lazy in the way they develop.

Jumbo Build Creeps

In the following we assume a.cpp and b.cpp are jumbo built as

// jumbo.cpp
#include "a.cpp"
#include "b.cpp"

Let's have fun while collecting some of the cases found in the Firefox codebase while removing the unify build requirement.

// a.cpp
#include <iostream>
void foo() {
  std::cout << "hello" << std::endl;
}

// b.cpp
void bar() {
  std::cout << "hello" << std::endl;
}

// a.cpp
#include <iostream>
static void foo() {
  std::cout << "hello" << std::endl;
}

// b.cpp
void bar() {
  foo();
}

// a.cpp
#include <iostream>
#define FOO 1

// b.cpp
#ifdef FOO
  #define BAR
#endif

// a.cpp
#include <iostream>
template <typename T>
void foo(T const& arg) {
  std::cout << arg << std::endl;
}

// b.cpp
#include <iostream>
void bar(int i) {
  foo(i);
}

// foobar.h
#ifndef FOOBAR_H
#define FOOBAR_H
#include <iostream>
template <typename T>
void foobar(T arg) {
  std::cout << arg << std::endl;
}
#endif

// a.cpp
#include "foobar.h"
template <>
void foobar<int>(int arg) {
  std::cout << "int: " << arg << std::endl;
}

// b.cpp
#include "foobar.h"
void bar(int i) {
  foobar(i);
}

// a.cpp
#include <iostream>
using namespace std;

void foo() {
  cout << "hello" << std::endl;
}

// b.cpp
#include <iostream>
void bar() {
  cout << "hello" << std::endl;
}

// foobar.h
#ifndef FOOBAR_H
#define FOOBAR_H
#include <iostream>
void foobar() {
  std::cout << "hello" << std::endl;
}
#endif

// a.cpp
#include "foobar.h"
void foo() {
  foobar();
}

// b.cpp
#include "foobar.h"
void bar() {
  foobar();
}

// foobar.h
#ifndef FOOBAR_H
#define FOOBAR_H
struct foo {
static const int VALUE;
};
#endif

// a.cpp
#include "foobar.h"
const int foo::VALUE = 1;

// b.cpp
#include "foobar.h"
static_assert(foo::VALUE == 1, "ok");

// foobar.h
#ifndef FOOBAR_H
#define FOOBAR_H
void foo(int * ptr);
#endif

// a.cpp
#include "foobar.h"
void foo(const int * ptr) {
}

// b.cpp
#include "foobar.h"
void bar(const int * ptr) {
    return foo(ptr);
}

// a.cpp
static int foo = 0;

// b.cpp
#include "a.cpp"

// foobar.h
struct Foo {};

// a.cpp
#include "foobar.h"
void a(Foo&) {}

// b.cpp
void b(Foo&) {}

About the Firefox codebase

The removal of REQUIRES_UNIFIED_BUILD across the Firefox codebase was tracked under https://bugzilla.mozilla.org/show_bug.cgi?id=1626530. Since I focused on this, I've landed more than 150 commits, modifying more than 800 sources files. And it's now done, no more hard requirement of unified build, back to a normal situation.

Was it worth the effort? Yes: it prevents bad coding practices, and static analysis is now more useful compared to what it could do with unified builds.

And we're sure we won't regress as our CI now builds in both unified and non-unified mode!