Maintaining Open Source Singularity Since 2017

SCS Login

Cross-architecture Containers with SingularityCE & PRO

By Dave Trudgian

Jul 27, 2022 | How To Guides, SingularityCE Updates

After a long period mostly standardized around the 64-bit x86 architecture, HPC systems, developer laptops, and desktops are becoming more diverse. ARM and POWER HPC systems are deployed on-premise, and ARM systems can also be constructed using cloud resources. On the desktop, Apple’s M1 / M2 CPUs bring ARM to a wide range of users.

Given this situation, you may need to build a container on a machine with a different CPU architecture than the system on which you will use the container. Sylabs’ Singularity Enterprise supports remote builds, using build hosts of different architectures. This ensures quick and easy full-speed builds are accessible to customers who have deployed Singularity Enterprise on-premise.

If you cannot use Singularity Enterprise or need to work with more exotic architectures, you can build containers using emulation of the required architecture. This requires a running x86_64 Linux system or virtual machine, with SingularityCE or PRO installed. Some patience is also needed. Emulation is slow, but it works.

Enable qemu-static binfmt_misc

We can set up our machine to transparently run and build containers for a different architecture by using qemu-static and binfmt_misc.

QEMU is a powerful emulator for a wide range of architectures and machines. The qemu-static binaries are static builds of emulators for different architectures.

binfmt_misc is a Linux kernel feature that allows associating executable files with an interpreter or other program that should run them

The multiarch/qemu-user-static project provides a simple way to bring up-to-date qemu-static binaries on your system, and register them with binfmt_misc, so your system can then handle other architectures. 

multiarch/qemu-user-static supports Singularity. To set up your system, you’ll need to run the container as the root user, as detailed in the project’s README.

$ sudo singularity run docker://multiarch/qemu-user-static --reset -p yes

Now your system can run POWER, ARM, or other containers. It’ll be slow due to the emulation, but it’ll work!

# Host is x86-64
$ uname -m
x86_64

# Pull a POWER (ppc64le) container from the SCS library
$ singularity pull --arch ppc64le ppc64le-ubuntu.sif library://ubuntu:20.04

# Run the container - it's ppc64le!
$ singularity exec ppc64le-ubuntu.sif uname -m
ppc64le

Building a Cross-architecture Container

Now, let’s try to build a container for a POWER system on our x86_64 host.

Create a definition file called ppc64le-build.def – we’ll start from the ppc64le-ubuntu.sif image we already have locally, install a package, and set up a runscript:

Bootstrap: localimage
From: ppc64le-ubuntu.sif

%post
	apt -y update
	apt -y install curl

%runscript
	echo
	echo "Hello from $(uname -m). The weather is ..."
	curl -q https://wttr.in

Now we can build the container as usual. The qemu-static binfmt_misc handling will automatically emulate a ppc64le system to perform the build. Because emulated builds are slow, now is a good time to go and make a cup of tea or coffee!

$ sudo singularity build ppc64le-build.sif ppc64le-build.def

INFO:	Starting build...
INFO:	Verifying bootstrap image ppc64le-ubuntu.sif
INFO:	Running post scriptlet
+ apt -y update
Hit:1 http://ports.ubuntu.com/ubuntu-ports focal InRelease
Get:2 http://ports.ubuntu.com/ubuntu-ports focal-security InRelease [114 kB]
Get:3 http://ports.ubuntu.com/ubuntu-ports focal-security/main ppc64el Packages [705 kB]
Get:4 http://ports.ubuntu.com/ubuntu-ports focal-security/main Translation-en [277 kB]
Fetched 1096 kB in 7s (160 kB/s)
Reading package lists... Done
...
Processing triggers for libc-bin (2.31-0ubuntu9) ...
INFO:	Adding runscript
INFO:	Creating SIF file...
INFO:	Build complete: ppc64le-build.sif

Let’s run our container…

$ singularity run ppc64le-build.sif

Hello from ppc64le. The weather is ...
Weather report: Vidor, Texas, United States

 	\  /   	Partly cloudy
   _ /"".-. 	+89(98) °F
 	\_(   ).   ↗ 5 mph
 	/(___(__)  9 mi
            	0.0 in
...

Summary

qemu-static and binfmt_misc make it easy to build containers with SingularityCE and PRO that target other architectures where a native build host is not available. The multiarch/qemu-user-static project even provides a containerized setup process, which itself is compatible with Singularity.

If you need to build large, complex containers, you’ll still definitely benefit from a native build host or cross architecture remote builds using Singularity Enterprise. Otherwise, some patience and emulation will go a long way.

Notes

If you don’t wish to, or cannot run a container as the root user to set up qemu-static & binfmt_misc, then your Linux distribution may provide a package that accomplishes the same thing. e.g., https://packages.debian.org/sid/qemu-user-static

Note that distribution packages may provide an older version of the QEMU binaries, so it’s possible you may hit bugs or performance issues not present in the containerized setup.

Join Our Mailing List

Categories

Related Posts