An Introduction to Monorepos: Simplifying Code Management

Hello everyone! It's been a long time since I have written an article. Today, I was revisiting my tech talk and decided to explain about monorepos in this blog. Here's the link to my first tech talk if you'd rather watch the video. I'm excited to delve into the world of monorepos today and help you understand its significance in modern software development.

What is a Monorepo?

A monorepo, short for a monolithic repository, is essentially a single repository that houses code for multiple projects. These projects can either be related or they can be completely distinct. Think of it as a unified storage space for various projects, including npm packages, within an organization.

Companies like Google, Facebook, Twitter, Microsoft, Airbnb, and even our team at CK-12 Foundation, use monorepos to streamline their code management processes.

Benefits of using a Monorepo

One of the primary advantages is improved collaboration and visibility among developers. With all the code stored in a single repository, developers can easily collaborate and share code across different projects.
With a monorepo, there is more accountability since every project is accessible to all the contributors and monorepos lend themselves to security features.
This collaborative environment often leads to increased development speed as changes made in one part of the repository can seamlessly reflect across multiple applications.

Creating a Monorepo: Tools and Approach

Several tools can assist in setting up a monorepo. Some prominent ones include:

Lerna, NX by Nrwl
Turbo by Vercel
Rush JS by Microsoft
Basil by Google
Pants, previously used by Twitter.

In this article, I'll focus on Lerna, a powerful tool for creating and managing JavaScript-specific monorepo packages.

Let's take a look at how you can initialize a monorepo using Lerna:

npx lerna init

This command sets up the package structure and generates the essential lerna.json file, offering configuration options for your monorepo. We make use of the lerna-schema default that is supplied. Any of these attributes can be overridden to change them to suit your needs.

// lerna.json
{
  "$schema": "node_modules/lerna/schemas/lerna-schema.json",
  "version": "1.0.0"
}

// package.json
{
  "workspaces": ["packages/*"],
  "dependencies": {
    "lerna": "^7.1.5"
  }
}

Here, packages/* indicates that all the projects reside within the packages folder, a convention but customizable as needed.

Lerna is no longer responsible for installing and linking the dependencies in your repo. Instead, we make use of a desired package manager of choice to use its workspaces feature. Here, I am using yarn workspaces.

Lerna and its Functionality

Lerna simplifies package management within a monorepo structure. Despite its recent unmaintained status in April 2022, Nrwl, the creator of NX, took stewardship of Lerna, by integrating its advantages into the tool. Lerna excels in handling versioning, publication, and conducting parallel builds efficiently.

Incorporating NX features into Lerna significantly enhances its capabilities, especially with caching for optimized builds. Utilizing caching in Lerna involves configuration setup in the nx.json file:

// nx.json
{
  "tasksRunnerOptions": {
    "default": {
      "runner": "nx-cloud",
      "options": {
        "caching": true,
        "cacheableOperations": ["build", "test"],
        "cacheDirectory": "dist"
      }
    }
  }
}

This configuration outlines cacheable operations (such as build and test) and specifies the directory where the cache will be stored (dist in this example).

Demonstration: Setting Up a Monorepo with Lerna

I showcased a live demo of setting up a monorepo using Lerna. Through commands like npx Lerna init, I illustrated the creation of package structures and the lerna.json file's significance. This file allows customization for the packages within the monorepo, defining workspaces, and package dependencies.

Let's take a snippet from the demo showcasing the directory structure and how Lerna creates symbolic links between packages. Here, app-atoms, app-header, and app-videos represent individual packages within the monorepo, interconnected through dependencies.

app-atoms is a component library that is utilized across all the other packages in this monorepo. We have components for Buttons, Images, various Typography and Input in this package.

app-header package contains the code for the global header component that is utilized within app-videos. The overall app's login state is being maintained within this package.
app-videos can be considered the main section of this application. It includes a list of the various video cards and features distinct routes to view each video card in detail.

Demo of the application

Main section

Route to access individual video details
Logged in user playlist

Leveraging NX Advantages with Lerna

From Lerna version 6.5 onwards, NX's caching capabilities were incorporated into Lerna. I showcased how caching optimizes the build process, significantly reducing build times by utilizing the cache for unchanged components.

// nx.json

{
  "tasksRunnerOptions": {
    "default": {
      "runner": "nx/tasks-runners/default", // "nx-cloud" if connected to cloud
      "options": {
        "cacheableOperations": ["build", "test"]
      }
    }
  },
  "targetDefaults": {
    "build": {
      "outputs": ["{projectRoot}/dist"],
      "dependsOn": ["^build"]
    },
    "test": {
      "dependsOn": ["build"]
    }
  }
}

Here, the {projectRoot}/dist helps nx understand that the project build is available within the /dist folder to maintain the cached versions.

Moreover, nx looks for successful builds of the dependencies before executing the build for the current project when "dependsOn": ["^build"] is mentioned. The remaining packages wouldn't be built at all if one of their dependent packages failed.

Build status

First build - All the packages are built in the order. It takes 4.90 seconds to complete.
Cached build - As there is no changes made to the package from its previous build, the cached versions are picked up. Build time reduces to 0.43 seconds.
Now, we make some changes to app-videos package and run yarn build on the monorepo. Only the affected package is built and the rest are taken from cache.
Similarly, when we make a change to app-header and perform yarn build, both app-header and app-videos are re-built. This is because app-videos has app-header as one of its dependencies.

NX Cloud and Project Graphs

Nx Cloud is a set of cloud-based development tools and services provided by Nrwl. An additional property of accessToken would be added along with cacheableOperations while using nx-cloud.

Further, integrating NX Cloud offers enhanced capabilities, such as remote caching and CI pipelines. The demonstration highlighted how NX Cloud facilitates seamless integration and efficient management of monorepo projects.

This is the nx-cloud interface. Observe that miss against each of these packages. They indicate that the cache has been missed.

This is the cached version of the build on nx-cloud. local indicates that the cached version is available and has been utilized in this case.

The NX project graph feature offers a visual representation of project interdependencies, aiding new team members in understanding project structures quickly.

Considerations and Downsides of Monorepos

While monorepos offer numerous benefits, including streamlined collaboration, they come with challenges.

Monorepos enables handling large amounts of data and commits every day. Testing the whole repo on every commit becomes infeasible.
GitHub has limited notification settings that are not best suited for a snowslide of pull requests and code reviews.
A broken master branch affects everyone working in the monorepo. This can be seen as either disastrous or as a good motivation to keep tests clean and up to date.

Conclusion: Embracing Monorepos Efficiently

In summary, monorepos are a powerful approach to code management, fostering collaboration and efficiency. They're not without challenges, but with proper tooling and understanding, teams can harness their benefits effectively.

Thank you for joining me on this exploration of monorepos today. Remember, continuous learning and adaptation are key in the ever-evolving landscape of software development.

Feel free to refer to my Github repo and show your support by staring it.

I hope you were able to take away something new and useful from this article! If you did, please drop a like on this post. 🙂

You can connect with me on Twitter, LinkedIn, and GitHub ✨

Peace ✌