6. Building Container Images

Overview

Questions

• How do I create my own Apptainer images?

Objectives

• Understand the different Apptainer container file formats.
• Understand how to build and share your own Apptainer containers.

So far you’ve been able to work with Apptainer from your own user account as a non-privileged user.

This part of the Apptainer material requires that you use Apptainer in an environment where you have administrative (root) access.

There are a couple of different ways to work around this restriction.

		pros	cons
Install Apptainer locally on a system where you do have administrative access (then then copy to HPC).		Building a container locally first is great for testing.	Not possible for many people. Local machine must have same architecture as HPC. Container image might be quite large and take a long time to copy.
Build your container from within another container		No root access required.	A bit contrived. Requires already having a built container with Apptainer installed.
Use a 'remote build service' to build your container		Convenient. Just one command to run.	Requires access to a remote build service. Build image must still be downloaded over network. Not currently available for Apptainer
Simulate root access using the --fakeroot feature		Convenient. Just an added flag.	Not possible with all operating systems. (On NeSI, only our newer nodes with Rocky8 installed have this functionality.)

We’ll focus on the last option in this part of the course - Simulate root access using the --fakeroot feature.

Let's build our first container image

• Check the content of my_container.def file

Bootstrap: docker
From: ubuntu:20.04

%post
    apt-get -y update && apt-get install -y python3

%runscript
    python3 -c 'print("Hello World! Hello from our custom Apptainer image!")'

• Build command follows the format of apptainer build --fakeroot name_of_the_image_file.sif name_of_the_definition_file.def

$ apptainer build --fakeroot my_container.sif my_container.def 

Check the image

• Once built, We can test our new container by running.

$ ./my_container.sif

Inspect the image

$ apptainer inspect my_container.sif

org.label-schema.build-arch: amd64
org.label-schema.build-date: Thursday_25_April_2024_14:43:40_NZST
org.label-schema.schema-version: 1.0
org.label-schema.usage.apptainer.version: 1.3.0
org.label-schema.usage.singularity.deffile.bootstrap: docker
org.label-schema.usage.singularity.deffile.from: ubuntu:20.04
org.opencontainers.image.ref.name: ubuntu
org.opencontainers.image.version: 20.04

Known limitations

This method, ( i.e using fakeroot), is known to not work for all types of Apptainer/Singularity containers.

If your container uses RPM to install packages, i.e. is based on CentOS or Rocky Linux, you need to disable the APPTAINER_TMPDIR environment variable (use unset APPTAINER_TMPDIR)

Few more examples

samtools-1.3.1 with HTSlib-1.3.2

Bootstrap: docker
From: ubuntu:xenial
Stage: spython-base

%labels
    MAINTAINER NeSI/GA
    description="Image for use with tools that require samtools, htslib, or tabix"

%post
    apt-get update -y && apt-get install -y \
    apt-utils \
    bzip2 \
    gcc \
    make \
    ncurses-dev \
    wget \
    zlib1g-dev

    ##############
    #HTSlib 1.3.2#
    ##############
    HTSLIB_INSTALL_DIR=/opt/htslib

    mkdir -p /tmp
    cd /tmp
    wget https://github.com/samtools/htslib/releases/download/1.3.2/htslib-1.3.2.tar.bz2 && \
    tar --bzip2 -xvf htslib-1.3.2.tar.bz2

    mkdir -p /tmp/htslib-1.3.2
    cd /tmp/htslib-1.3.2
    ./configure  --enable-plugins --prefix=$HTSLIB_INSTALL_DIR && \
    make && \
    make install && \
    cp $HTSLIB_INSTALL_DIR/lib/libhts.so* /usr/lib/

    ################
    #Samtools 1.3.1#
    ################
    SAMTOOLS_INSTALL_DIR=/opt/samtools

    mkdir -p /tmp
    cd /tmp
    wget https://github.com/samtools/samtools/releases/download/1.3.1/samtools-1.3.1.tar.bz2 && \
    tar --bzip2 -xf samtools-1.3.1.tar.bz2

    mkdir -p /tmp/samtools-1.3.1
    cd /tmp/samtools-1.3.1
    ./configure --with-htslib=$HTSLIB_INSTALL_DIR --prefix=$SAMTOOLS_INSTALL_DIR && \
    make && \
    make install

    mkdir -p /
    cd /
    rm -rf /tmp/samtools-1.3.1

    #######
    #tabix#
    #######
    ln -s $HTSLIB_INSTALL_DIR/bin/tabix /usr/bin/tabix
    ln -s /opt/samtools/bin/* /usr/local/bin/

%environment
    export HTSLIB_INSTALL_DIR=/opt/htslib
    export SAMTOOLS_INSTALL_DIR=/opt/samtools

%runscript
    cd /
    exec /bin/bash "$@"

%startscript
    cd /
    exec /bin/bash "$@"

seqtk - Using conda to install an application within a container image

• All of us have built conda environment for some applications. One of the downsides of conda is it's appetite to generate a large number of files per environment. One of the best solutions for this is to build the conda environment within the container image. This means the number of files consumed will be equal to 1 . Plus, it will make the sharing much more easier

Bootstrap: docker
From: ubuntu:xenial

%environment
    export LC_ALL=C
    export LC_NUMERIC=en_GB.UTF-8
    export PATH="/opt/miniconda/bin:$PATH"

%labels
        MAITAINER NeSI/GA
        LICENSE MIT

%help
    Container for seqtk
    Seqtk is a fast and lightweight tool for processing sequences in the FASTA or FASTQ format. It seamlessly parses both FASTA and FASTQ files which can also be optionally compressed by gzip.
    https://github.com/lh3/seqtk

    Version: 1.3
    Package installation using Miniconda3-4.7.12
    All packages are in /opt/miniconda/bin & are in PATH
    Default runscript: seqtk

    Usage:
        seqtk_v1.3.sif --help
        or:
        singularity exec seqtk_v1.3.sif seqtk --help


%runscript
    #default runscript: seqtk passing all arguments from cli: $@
    exec /opt/miniconda/bin/seqtk "$@"

%post

    #essential stuff but minimal
    apt update
    #for security fixe:
    #apt upgrade -y
    apt install -y wget bzip2

    #install conda
    cd /opt
    rm -fr miniconda

    #miniconda3: get miniconda3 version 4.7.12
    wget https://repo.anaconda.com/miniconda/Miniconda3-py39_24.3.0-0-Linux-x86_64.sh -O miniconda.sh

    #install conda
    bash miniconda.sh -b -p /opt/miniconda
    export PATH="/opt/miniconda/bin:$PATH"
    #add channels
    conda config --add channels defaults
    conda config --add channels bioconda
    conda config --add channels conda-forge

    #install seqtk

    conda install -y -c bioconda seqtk=1.3

    #cleanup
    conda clean -y --all
    rm -f /opt/miniconda.sh
    apt autoremove --purge
    apt clean