Using Environment Variables with the Cloud Native Buildpacks lifecycle

Using Cloud Native Buildpacks (CNB) in your GitLab CI pipelines without Docker and pack CLI is common practice. Especially if you leverage shared Kubernetes runners without access to the host's Docker daemon. In this case you need to use the CNB lifecycle directly. But what if you need to define environment variables for the build process?

Cloud Native Buildpacks – blog series

Part 1: Goodbye Dockerfile: Cloud Native Buildpacks with Paketo.io & layered jars for Spring Boot
Part 2: Cloud Native Buildpacks / Paketo.io in GitLab CI without Docker & pack CLI
Part 3: Using Environment Variables with the Cloud Native Buildpacks lifecycle

I'm a big fan of Cloud Native Buildpacks (#CNB) since years 😍 I wrote about how to say goodbye to your Dockerfiles already in 2020 and used them in nearly every project since then. But in modern CI/CD pipelines installments, you'd often don't have access to the host's Docker daemon (like it had been common using priviledged mode or Docker socket mounts). And this is a great thing taking security aspects into account and is implemented by our codecentric Cloud Managed GitLab per default.

Using Buildpacks in such a scenario requires you to omit pack CLI and instead use the CNB lifecycle directly, as I wrote about in this post. The deprecated Kaniko adressed a similar issue, but you still had to maintain Dockerfiles.

An updated .gitlab-ci.yml to use the CNB lifecycle without Docker

Since I wrote the last post in 2021 there was a slight change in the direct usage of the CNB lifecycle in GitLab CI, since it needs some variables to be defined before execution that weren't needed back then. In order to prevent errors like failed to get platform API version; please set 'CNB_PLATFORM_API' to specify the desired platform API we now need to define the variables CNB_USER_ID, CNB_GROUP_ID and CNB_PLATFORM_API like it is shown in the fully working .gitlab-ci.yml here:

1stages:
2  - test
3  - container-image
4
5variables:
6  # see usage of Namespaces at https://docs.gitlab.com/ee/user/group/#namespaces
7  REGISTRY_GROUP_PROJECT: $CI_REGISTRY/$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME
8
9maven-test:
10  stage: test
11  image: maven:3-eclipse-temurin-25
12  script:
13    - mvn test
14
15build-publish-image:
16  stage: container-image
17  # For a detailed description on how to use Paketo.io Buildpacks in GitLab CI
18  # with Kubernetes executor & unprivileged Runners (without pack CLI & docker)
19  # see https://stackoverflow.com/questions/69569784/use-paketo-io-cloudnativebuildpacks-cnb-in-gitlab-ci-with-kubernetes-executo
20  image: paketobuildpacks/builder-jammy-base
21
22  # As we don't have docker available, we can't login to GitLab Container Registry as described in the docs https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#using-the-gitlab-container-registry
23  # But we somehow need to access GitLab Container Registry with the Paketo lifecycle
24  # So we simply create ~/.docker/config.json as stated in https://stackoverflow.com/a/46422186/4964553
25  before_script:
26    - mkdir ~/.docker
27    - echo "{\"auths\":{\"$CI_REGISTRY\":{\"username\":\"$CI_REGISTRY_USER\",\"password\":\"$CI_JOB_TOKEN\"}}}" >> ~/.docker/config.json
28    # As described in https://buildpacks.io/docs/for-platform-operators/tutorials/lifecycle/#set-environment-variables
29    # we need to set some variables like the platform api version in order to prevent errors like this:
30    # failed to get platform API version; please set 'CNB_PLATFORM_API' to specify the desired platform API version
31    - export CNB_USER_ID=$(id -u) CNB_GROUP_ID=$(id -g) CNB_PLATFORM_API=0.14
32
33  script:
34    - /cnb/lifecycle/creator -app . -cache-image $REGISTRY_GROUP_PROJECT/paketo-build-cache:latest $REGISTRY_GROUP_PROJECT:latest

This pipeline is derived from the 2021 article and defines two stages. The test stage runs all tests using Maven with the maven-test job. The second stage container-image features the build-publish-image job that showcases the direct usage of the CNB lifecycle without Docker or pack CLI. So far so good.

What's the issue with defining environment variables for the lifecycle?

Last week I had the requirement to use a simple environment variable with the CNB lifecycle, since the customer needed a JDK inside the run image (which is generally not a recommended thing to do, but the software needed to validate stuff using the JDK tooling). The default Bellsoft Liberica buildpack has a parameter BP_JVM_TYPE that you can set to JDK. This will configure the JVM type that is provided in the run image to use a JDK instead of the default JRE. So exactly what the customer needed. Would I be able to use pack CLI, the command would be simple and the environment variable could be defined using --env like this:

1pack build your.gitlab.contaner.registry.com/yourgitlabproject/microservice-api-spring-boot:latest \
2              --builder paketobuildpacks/builder-jammy-base \
3              --path . \
4              --env "BP_JVM_TYPE=JDK"

But simply running export BP_JVM_TYPE=JDK before the lifecycle execution doesn't work, as I expected it in the first place. The run image produced by the CNB build process still featured a JRE instead the needed JDK.

To verify if a JDK is used in the CNB build run image, you can simply use a docker run -it and override entrypoint of Spring Boot run image:

1docker run -it --rm  --entrypoint launcher your.gitlab.contaner.registry.com/yourgitlabproject/microservice-api-spring-boot:latest bash

You might need to authenticate to your GitLab's registry for this to work. I got to know the GitLab CLI with glab auth login which I found quite handy here.

Now inside the container you can verify, if it uses a JDK or JRE as the runtime by running

1java -XshowSettings:properties -version | grep java.home

If the output contains the java.home = /layers/paketo-buildpacks_bellsoft-liberica/jdk the JDK is successfully configured (which was the goal for my customer). If the output contains java.home = /layers/paketo-buildpacks_bellsoft-liberica/jre, the BP_JVM_TYPE variable hasn't been correctly set (which was the case for me).

The solution: Create the environment variables using files

I had a hard time finding a solution. But luckily an accepted answer on stackoverflow from 2022 brought me near the solution. So right back into our .gitlab-ci.yml we need to create a different platform directory in the cnb user home:

1mkdir -p ~/platform/env

This is because directly using echo "JDK" >> platform/env/BP_JVM_TYPE gives a bash: /platform/env/BP_JVM_TYPE: Permission denied error, because the directory platform is already created inside the builder image. I guess that the builder images somehow changed and now contains the folder platform already (and we don't have - and don't want - root permissions inside the container).

Having this directory in place we can now create the environment variables as described in the buildpack platform spec documentation in the form of

Each file SHALL define a single environment variable, where the file name defines the key and the file contents define the value.

So inside the ~/platform/env directory we create a file BP_JVM_TYPE using the following command:

1echo -n "JDK" >> ~/platform/env/BP_JVM_TYPE

Now you should see a file BP_JVM_TYPE with the content JDK inside ~/platform/env.

It is extremely important to use the -n parameter with echo, since it removes the trailing newline! This was the thing that drove me nuts. I ran into an odyssey, since there is a bug in the buildpack implementation tracked in this issue, where the newline is causing parameters to NOT WORK with the lifecycle (many thanks to Daniel Mikusa for providing the hint). I can only advise you to double check the contents of the files you create like ~/platform/env/BP_JVM_VERSION in an editor:

wrong (containing newline):

correct (NO newline):

Run the CNB lifecycle using -platform

With the BP_JVM_TYPE file in place containing our value without a trailing newline, we are now ready to run the CNB lifecycle. Therefore execute the /cnb/lifecycle/creator command with an appended -platform /home/cnb/platform. But again be careful, since the directory needs to be passed WITHOUT the trailing /env!. Otherwise it won't use the environment configuration correctly:

1/cnb/lifecycle/creator -app . -platform /home/cnb/platform $REGISTRY_GROUP_PROJECT:latest

That's it already! I would have loved to find such a blog post like this when I started out to "simply" create a run image featuring a JDK with Buildpacks :)

Here's the fully working .gitlab-ci.yml again, featuring the definition of the environment variable BP_JVM_TYPE=JDK:

1stages:
2  - test
3  - container-image
4
5variables:
6  # see usage of Namespaces at https://docs.gitlab.com/ee/user/group/#namespaces
7  REGISTRY_GROUP_PROJECT: $CI_REGISTRY/$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME
8
9maven-test:
10  stage: test
11  image: maven:3-eclipse-temurin-24
12  script:
13    - mvn test
14
15build-publish-image:
16  stage: container-image
17  image: paketobuildpacks/builder-jammy-base
18
19  before_script:
20    - mkdir ~/.docker
21    - echo "{\"auths\":{\"$CI_REGISTRY\":{\"username\":\"$CI_REGISTRY_USER\",\"password\":\"$CI_JOB_TOKEN\"}}}" >> ~/.docker/config.json
22    - export CNB_USER_ID=$(id -u) CNB_GROUP_ID=$(id -g) CNB_PLATFORM_API=0.14
23    # Configuring Environment variables for the paketo lifecycle is a bit more sophisticated
24    # As described in https://stackoverflow.com/a/79766353/4964553 & the lifecycle spec https://github.com/buildpacks/spec/blob/main/platform.md#user-provided-variables
25    # we need to create a single file for each variable, where the file name defines the key and the file contents define the value
26    # WE NEED TO USE -n with echo, otherwise we have a trailing newline in the environment variable configuration files, that breaks the parameters usage (https://github.com/paketo-buildpacks/libpak/issues/416)
27    - mkdir -p ~/platform/env
28    # Using a different JDK than default 21
29    - echo -n "25" >> ~/platform/env/BP_JVM_VERSION
30    # Using a JDK instead of a JRE at runtime (not recommended for default, but used inside the PMS currently for validation, that relies on Java Bytecode)
31    - echo -n "JDK" >> ~/platform/env/BP_JVM_TYPE
32
33  script:
34    # We need to add the -platform /home/cnb/platform parameter in order to let the lifecycle use our defined environment variables
35    - /cnb/lifecycle/creator -app . -platform /home/cnb/platform -cache-image $REGISTRY_GROUP_PROJECT/paketo-build-cache:latest $REGISTRY_GROUP_PROJECT:latest

Just to show the usage, i also added the BP_JVM_VERSION variable to change the JVM version.

The GitLab CI log output (cropped for this post) also looks good, since it finally contains the message A JDK was specifically requested by the user, however a JRE is available. which indicates, the environment variable is successfully picked up by the Bellsoft Liberica buildpack:

1...
2===> BUILDING
3target distro name/version labels not found, reading /etc/os-release file
4Paketo Buildpack for CA Certificates 3.10.4
5  https://github.com/paketo-buildpacks/ca-certificates
6  Build Configuration:
7    $BP_EMBED_CERTS                    false  Embed certificates into the image
8    $BP_ENABLE_RUNTIME_CERT_BINDING    true   Deprecated: Enable/disable certificate helper layer to add certs at runtime
9    $BP_RUNTIME_CERT_BINDING_DISABLED  false  Disable certificate helper layer to add certs at runtime
10  Launch Helper: Reusing cached layer
11Paketo Buildpack for BellSoft Liberica 11.4.0
12  https://github.com/paketo-buildpacks/bellsoft-liberica
13  Build Configuration:
14    $BP_JVM_JLINK_ARGS           --no-man-pages --no-header-files --strip-debug --compress=1  configure custom link arguments (--output must be omitted)
15    $BP_JVM_JLINK_ENABLED        false                                                        enables running jlink tool to generate custom JRE
16    $BP_JVM_TYPE                 JDK                                                          the JVM type - JDK or JRE
17    $BP_JVM_VERSION              25                                                           the Java version
18  Launch Configuration:
19    $BPL_DEBUG_ENABLED           false                                                        enables Java remote debugging support
20    $BPL_DEBUG_PORT              8000                                                         configure the remote debugging port
21    $BPL_DEBUG_SUSPEND           false                                                        configure whether to suspend execution until a debugger has attached
22    $BPL_HEAP_DUMP_PATH                                                                       write heap dumps on error to this path
23    $BPL_JAVA_NMT_ENABLED        true                                                         enables Java Native Memory Tracking (NMT)
24    $BPL_JAVA_NMT_LEVEL          summary                                                      configure level of NMT, summary or detail
25    $BPL_JFR_ARGS                                                                             configure custom Java Flight Recording (JFR) arguments
26    $BPL_JFR_ENABLED             false                                                        enables Java Flight Recording (JFR)
27    $BPL_JMX_ENABLED             false                                                        enables Java Management Extensions (JMX)
28    $BPL_JMX_PORT                5000                                                         configure the JMX port
29    $BPL_JVM_HEAD_ROOM           0                                                            the headroom in memory calculation
30    $BPL_JVM_LOADED_CLASS_COUNT  35% of classes                                               the number of loaded classes in memory calculation
31    $BPL_JVM_THREAD_COUNT        250                                                          the number of threads in memory calculation
32    $JAVA_TOOL_OPTIONS                                                                        the JVM launch flags
33    Using Java version 25 from BP_JVM_VERSION
34  A JDK was specifically requested by the user, however a JRE is available. Using a JDK at runtime has security implications.
35  BellSoft Liberica JDK 25.0.0: Reusing cached layer
36...

Let's finally verify if the JDK is used the run image:

1docker run -it --rm  --entrypoint launcher your.gitlab.contaner.registry.com/yourgitlabproject/microservice-api-spring-boot:latest bash
2java -XshowSettings:properties -version | grep java.home

The output now contains the java.home = /layers/paketo-buildpacks_bellsoft-liberica/jdk which indicates that our pipeline worked the way we wanted it to!

Optional: Developing the solution locally

If you're like me you might not like to create a lot of git commits in GitLab, wait for the pipelines to be triggered and run into errors there (the dreaded "staring on pipelines" problem).

But there's a simple solution. Clone the project you want to build to your local machine (in my case this is a Spring Boot based project), cd into it and use the paketo.io builder paketobuildpacks/builder-jammy-base simply directly as a local Docker container:

1docker run -it --rm -v "$PWD":/usr/src/app -w /usr/src/app paketobuildpacks/builder-jammy-base bash

This mimics the usage in GitLab, where your Java project is mounted into the container and is based on that exact image.

Now in order to make the CNB lifecycle run successfully inside your local container, you need to to the things we do in the GitLab CI pipeline also:

1mkdir ~/.docker
2
3echo "{\"auths\":{\"your.gitlab.contaner.registry.com\":{\"username\":\"yourUserName\",\"password\":\"your_gitlab_access_token\"}}}" >> ~/.docker/config.json
4
5export CNB_USER_ID=$(id -u) CNB_GROUP_ID=$(id -g) CNB_PLATFORM_API=0.14

For the authorization data you might need to create a new access token in your GitLab profile.

You might also run into permission denied errors later while building your image inside the container that relate to your container not having the right permissions for the target/classes directory used in the Maven build. Solutions to that are described in this great writeup.

Wrap up

Cloud Native Buildpacks still remain a fantastic solution to handle our container build requirements. This applies just as much for scenarios with shared (Kubernetes based) GitLab runners without Docker daemon access. Using the CNB lifecycle directly is slightly less documented compared to the default way using pack CLI and Docker, therefore I tried to shed some more light on this important scenario.

For more alternatives I can recommend a great overview about options to replace the deprecated Kaniko by my colleague Felix Dreißig. I think we need to drink a beer in order to get Cloud Native Buildpacks on his list as option 8 :)