Using Environment Variables with the Cloud Native Buildpacks lifecycle

Using Cloud Native Buildpacks (CNB) in your GitLab CI pipelines without Docker and pack CLI is common practice. Especially if you leverage shared Kubernetes runners without access to the host's Docker daemon. In this case you need to use the CNB lifecycle directly. But what if you need to define environment variables for the build process?

Cloud Native Buildpacks – blog series

Part 1: Goodbye Dockerfile: Cloud Native Buildpacks with Paketo.io & layered jars for Spring Boot
Part 2: Cloud Native Buildpacks / Paketo.io in GitLab CI without Docker & pack CLI
Part 3: Using Environment Variables with the Cloud Native Buildpacks lifecycle

I'm a big fan of Cloud Native Buildpacks (#CNB) since years 😍 I wrote about how to say goodbye to your Dockerfiles already in 2020 and used them in nearly every project since then. But in modern CI/CD pipelines installments, you'd often don't have access to the host's Docker daemon (like it had been common using priviledged mode or Docker socket mounts). And this is a great thing taking security aspects into account and is implemented by our codecentric Cloud Managed GitLab per default.

Using Buildpacks in such a scenario requires you to omit pack CLI and instead use the CNB lifecycle directly, as I wrote about in this post. The deprecated Kaniko adressed a similar issue, but you still had to maintain Dockerfiles.

An updated .gitlab-ci.yml to use the CNB lifecycle without Docker

Since I wrote the last post in 2021 there was a slight change in the direct usage of the CNB lifecycle in GitLab CI, since it needs some variables to be defined before execution that weren't needed back then. In order to prevent errors like failed to get platform API version; please set 'CNB_PLATFORM_API' to specify the desired platform API we now need to define the variables CNB_USER_ID, CNB_GROUP_ID and CNB_PLATFORM_API like it is shown in the fully working .gitlab-ci.yml here:

This pipeline is derived from the 2021 article and defines two stages. The test stage runs all tests using Maven with the maven-test job. The second stage container-image features the build-publish-image job that showcases the direct usage of the CNB lifecycle without Docker or pack CLI. So far so good.

What's the issue with defining environment variables for the lifecycle?

Last week I had the requirement to use a simple environment variable with the CNB lifecycle, since the customer needed a JDK inside the run image (which is generally not a recommended thing to do, but the software needed to validate stuff using the JDK tooling). The default Bellsoft Liberica buildpack has a parameter BP_JVM_TYPE that you can set to JDK. This will configure the JVM type that is provided in the run image to use a JDK instead of the default JRE. So exactly what the customer needed. Would I be able to use pack CLI, the command would be simple and the environment variable could be defined using --env like this:

But simply running export BP_JVM_TYPE=JDK before the lifecycle execution doesn't work, as I expected it in the first place. The run image produced by the CNB build process still featured a JRE instead the needed JDK.

To verify if a JDK is used in the CNB build run image, you can simply use a docker run -it and override entrypoint of Spring Boot run image:

You might need to authenticate to your GitLab's registry for this to work. I got to know the GitLab CLI with glab auth login which I found quite handy here.

Now inside the container you can verify, if it uses a JDK or JRE as the runtime by running

If the output contains the java.home = /layers/paketo-buildpacks_bellsoft-liberica/jdk the JDK is successfully configured (which was the goal for my customer). If the output contains java.home = /layers/paketo-buildpacks_bellsoft-liberica/jre, the BP_JVM_TYPE variable hasn't been correctly set (which was the case for me).

The solution: Create the environment variables using files

I had a hard time finding a solution. But luckily an accepted answer on stackoverflow from 2022 brought me near the solution. So right back into our .gitlab-ci.yml we need to create a different platform directory in the cnb user home:

This is because directly using echo "JDK" >> platform/env/BP_JVM_TYPE gives a bash: /platform/env/BP_JVM_TYPE: Permission denied error, because the directory platform is already created inside the builder image. I guess that the builder images somehow changed and now contains the folder platform already (and we don't have - and don't want - root permissions inside the container).

Having this directory in place we can now create the environment variables as described in the buildpack platform spec documentation in the form of

Each file SHALL define a single environment variable, where the file name defines the key and the file contents define the value.

So inside the ~/platform/env directory we create a file BP_JVM_TYPE using the following command:

Now you should see a file BP_JVM_TYPE with the content JDK inside ~/platform/env.

It is extremely important to use the -n parameter with echo, since it removes the trailing newline! This was the thing that drove me nuts. I ran into an odyssey, since there is a bug in the buildpack implementation tracked in this issue, where the newline is causing parameters to NOT WORK with the lifecycle (many thanks to Daniel Mikusa for providing the hint). I can only advise you to double check the contents of the files you create like ~/platform/env/BP_JVM_VERSION in an editor:

wrong (containing newline):

correct (NO newline):

Run the CNB lifecycle using -platform

With the BP_JVM_TYPE file in place containing our value without a trailing newline, we are now ready to run the CNB lifecycle. Therefore execute the /cnb/lifecycle/creator command with an appended -platform /home/cnb/platform. But again be careful, since the directory needs to be passed WITHOUT the trailing /env!. Otherwise it won't use the environment configuration correctly:

That's it already! I would have loved to find such a blog post like this when I started out to "simply" create a run image featuring a JDK with Buildpacks :)

Here's the fully working .gitlab-ci.yml again, featuring the definition of the environment variable BP_JVM_TYPE=JDK:

Just to show the usage, i also added the BP_JVM_VERSION variable to change the JVM version.

The GitLab CI log output (cropped for this post) also looks good, since it finally contains the message A JDK was specifically requested by the user, however a JRE is available. which indicates, the environment variable is successfully picked up by the Bellsoft Liberica buildpack:

Let's finally verify if the JDK is used the run image:

The output now contains the java.home = /layers/paketo-buildpacks_bellsoft-liberica/jdk which indicates that our pipeline worked the way we wanted it to!

Optional: Developing the solution locally

If you're like me you might not like to create a lot of git commits in GitLab, wait for the pipelines to be triggered and run into errors there (the dreaded "staring on pipelines" problem).

But there's a simple solution. Clone the project you want to build to your local machine (in my case this is a Spring Boot based project), cd into it and use the paketo.io builder paketobuildpacks/builder-jammy-base simply directly as a local Docker container:

This mimics the usage in GitLab, where your Java project is mounted into the container and is based on that exact image.

Now in order to make the CNB lifecycle run successfully inside your local container, you need to to the things we do in the GitLab CI pipeline also:

For the authorization data you might need to create a new access token in your GitLab profile.

You might also run into permission denied errors later while building your image inside the container that relate to your container not having the right permissions for the target/classes directory used in the Maven build. Solutions to that are described in this great writeup.

Wrap up

Cloud Native Buildpacks still remain a fantastic solution to handle our container build requirements. This applies just as much for scenarios with shared (Kubernetes based) GitLab runners without Docker daemon access. Using the CNB lifecycle directly is slightly less documented compared to the default way using pack CLI and Docker, therefore I tried to shed some more light on this important scenario.

For more alternatives I can recommend a great overview about options to replace the deprecated Kaniko by my colleague Felix Dreißig. I think we need to drink a beer in order to get Cloud Native Buildpacks on his list as option 8 :)