Story of Leveraging Custom Buildpacks in Heroku

15/8/2023
4-minute read

Heroku buildpacks are collections of scripts that are used for compiling apps on Heroku’s platform. Essentially, they prepare the code for execution by installing dependencies, setting up the environment, and so on. Heroku utilizes them to transform your pushed code into a slug, which can then be executed on a dyno.

The beauty of Heroku lies in the fact that it automatically detects the appropriate buildpack for your app. However, there comes a time when the defaults don’t cut it, and that’s when you need to create your own custom buildpack. For further understanding, the Heroku documentation has a brilliant in-depth guide on buildpacks.

The Birth of a Custom Buildpack

The creation of a custom buildpack wasn’t an arbitrary decision. Our organization had several specific use cases that required a bit more than the default settings.

Use Case 1: Installing Custom Packages

The first instance was when we had to install custom packages that weren’t covered in the default buildpack. After creating our custom buildpack, we could easily add a script to install the needed packages. Below is an example:

# In your bin/compile file
echo "-----> Installing custom package"
curl -L https://mypackage.url | tar xz -C .heroku

Use Case 2: Managing Robots.txt for Non-Production Environments

The second scenario was to replace the robots.txt file for our non-production environments. This was crucial to prevent search engines from indexing them. With the custom buildpack, we were able to add a script to replace the robots.txt:

# In your bin/compile file
if [ "$HEROKU_ENV" = "staging" ]; then
    echo "User-agent: *" > public/robots.txt
    echo "Disallow: /" >> public/robots.txt
fi

Use Case 3: Writing Files from the ENV content

At times, we had to create .pem files from the content stored in environment variables. Here’s a simplified script to achieve that:

# In your bin/compile file
echo "-----> Creating PEM file from ENV"
echo "$PRIVATE_KEY" > /app/config/private_key.pem

Use Case 4: Managing Multiple Procfiles

Lastly, we had a scenario where we needed to manage multiple Procfiles and control which one to choose based on the environment. The custom buildpack made this an easy task:

# In your bin/compile file
if [ "$HEROKU_ENV" = "staging" ]; then
    cp Procfile.staging Procfile
fi

Chapter 3: The Custom Buildpack Saga Continues

The creation of custom buildpacks had a profound impact on our team. Not only did it resolve our immediate needs, but it also paved the way for further customization. Today, we use them for a multitude of tasks like setting up specific configurations, running custom tasks before deployments, and so much more.

Epilogue: An Invitation

If you’re also facing scenarios that require more than the default Heroku settings, don’t hesitate to create a custom buildpack. It might seem daunting at first, but it’s a rewarding process that opens a world of possibilities.

Thank you for joining me on this journey, my fellow engineer. It was a pleasure sharing my story, and I hope it inspired you to take the plunge into the realm of custom buildpacks. Until we meet again in another adventure, happy coding!

Certainly! I will share a simple custom buildpack for a Python application that includes a few of the tasks I described earlier.

Creating the Buildpack Directory Structure

A typical buildpack consists of three essential scripts: bin/detect, bin/compile, and bin/release, all stored within the bin directory. Let’s create our directory structure:

$ mkdir my_custom_buildpack
$ cd my_custom_buildpack
$ mkdir bin
$ touch bin/{detect,compile,release}

Make the scripts executable:

$ chmod +x bin/*

Writing the bin/detect script

This script detects whether your buildpack can be used to build the app. In our case, it will check for the existence of a requirements.txt file to verify it’s a Python application.

# bin/detect
#!/bin/sh

if [ -f requirements.txt ]; then
  echo "Python"
  exit 0
else
  echo "No requirements.txt found" 1>&2
  exit 1
fi

Writing the bin/compile script

This is the script where the majority of the build process occurs. Here, we’ll install Python, install our requirements, and create a .pem file from an environment variable.

# bin/compile
#!/bin/sh

BUILD_DIR=$1

echo "-----> Installing Python"
# We'd usually download and install Python here.

echo "-----> Installing requirements"
pip install -r $BUILD_DIR/requirements.txt

echo "-----> Creating PEM file from ENV"
echo "$PRIVATE_KEY" > $BUILD_DIR/private_key.pem

Writing the bin/release script

This script provides metadata back to the runtime.

# bin/release
#!/bin/sh

echo "--- {}"

The {} in the release script provides an empty YAML object, as the script expects a YAML formatted output.

After creating your buildpack, it’s time to use it. Commit it to a Git repository and use the repository’s URL as your buildpack URL:

$ git init
$ git add .
$ git commit -m "First commit"
$ git remote add origin <your-git-repository-url>
$ git push -u origin master

Now you can use it in your app:

$ heroku buildpacks:set https://github.com/user/my_custom_buildpack

Remember to replace <your-git-repository-url> with the actual URL of your Git repository. This is a basic example to get you started. Depending on your app’s complexity, you can add more functionality in the compile script, like installing custom packages or managing multiple Procfiles based on environment variables.

heroku buildpacks