At Sylabs, our mission is to drive development of the open source Singularity project and expand the surrounding ecosystem positioning Singularity as the de-facto choice for HPC and Enterprise Performance Computing (EPC) workflows. .
Today, we’re happy to share with you a preview of one of the products we have been working on: the Sylabs Remote Build Service, a cloud-based service (It’s not the most creative name, I know, but as a team of software developers, you’ll have to forgive us for our lack of marketing prowess!)
Singularity enables many different workflows by giving you the opportunity to use compute resources where privileges are limited. But if we step back and think about the process of building an image, elevated privileges are sometimes necessary. Specifically, if you want to build an image from a recipe or definition file, you need root access.
Right now, Singularity users need a Linux workstation or Virtual Machine where they have elevated privileges to build and test their container images. In many HPC and EPC environments, however, there are a couple of potential issues with this approach:
Sylabs Remote Build Service addresses these challenges by removing the need for users to have elevated privileges, and potentially allowing users to build Singularity containers on non-Linux workstations.
Singularity 3.0 development is continuing at a fast pace, and new and exciting use cases are already popping up. For instance, using the the Remote Build System, it should be possible to build Singularity images using a Mac or Windows workstation; please stay tuned!
One of the goals of the Remote Build Service is for it to integrate seamlessly with Singularity so that current Singularity users don’t have to modify their workflow. In fact, with the addition of a single flag on the command line, the build can be done remotely:
$ singularity build --remote test.sif test.defThis causes Singularity to make a request to the Remote Build Service, which completes the build remotely. During the build process, output of build is streamed back to the console, so that the user can monitor its progress. Assuming the build completes successfully, the built SIF image file is transferred back to the user’s workstation, from which point it can be executed with Singularity.
Singularity and Sylabs have been focussed on security from day one. The astute Singularity user will note that building images within a managed service still requires elevated privileges. It simply shifts the location where elevated privileges are utilized.
This shift allows the Remote Build Service to implement appropriate levels of isolation between the components performing the builds with elevated privileges, and the rest of the infrastructure. As a system administrator, you get a turn-key solution that empowers users to build Singularity images, providing you with centralized auditing and monitoring of Singularity builds occuring on-site.
Do you have a use case for the Sylabs Remote Build Service? Interested in learning more? We’d love to hear from you for topics to cover in future Lab Notes, or joining community discussions.
Slack Channel or get an invitation at: Sylabs website.
Enabling Portable and Secure Computing Environments for High-Performance Workloads.As part of their ongoing efforts to streamline workflows, enhance productivity, and save time, engineers, and developers in enterprises and high performance computing (HPC) focused...
Bills of materials or BOMs are nothing new, but they are new in the software industry. They have been part of many supply chains for decades. When you purchase a car or a pharmaceutical device, the manufacturer of those products maintains a list of...
There are many different approaches that can be taken when building software. At one end of the spectrum is the extreme caution and conservatism that’s appropriate, for example, of safety critical code used in vehicles or in real-time operating systems. At the other...