Who is Sylabs? We are often asked about who Sylabs is, and what we are working on for both the open source community, and for those who require professional support of their HPC container workflow environments. Sylabs was created in 2017 to provide support,...
ALBANY, Calif., Oct. 8, 2018 — Sylabs, a leading provider of container runtime solutions for Enterprise Performance Computing (EPC) and High-Performance Computing (HPC) workloads, announced today that it has released a new version of its innovative container software: Singularity 3.0. This release focuses on providing users with access to an expanded suite of resources, including Remote Builder, Container Library, and Key Management services. Singularity 3.0 also features several platform enhancements, including Singularity Image Format (SIF), cryptographically signed and verifiable container images, base code rewrite into Go/C, enhanced instance support, and networking management for containers. These new enterprise-class features further differentiate Singularity as the premier container runtime solution, enabling your company to seamlessly and efficiently tackle today’s most demanding Artificial Intelligence (AI), machine/deep learning, and advanced analytic workloads.
“Singularity has quickly become the container platform of choice by academia and commercial computing centers,” said Sylabs CEO, Greg Kurtzer. “Singularity’s core features and the introduction of Singularity 3.0 make it the ideal container technology for AI, machine/deep learning, compute-driven analytics, and data science—areas that we characterize as enterprise performance computing, or EPC. These applications carry data-intensive workloads that demand HPC-like resources but satisfy the enterprise’s need to create predictable container environments for a substantially growing number of AI-driven use-cases. We expect to see a substantial uptake in the already existing 25,000 users spread around the world at major universities and government institutions as a result of Singularity 3.0.”
An Expanded Ecosystem
Singularity 3.0 simplifies the process of moving applications, workloads, and compute environments across disparate infrastructures by providing access to Sylabs’ growing container ecosystem:
- Remote Builder Services – Enables users to safely build containers in secure, isolated environments without the need for root access to that system. With no elevated administrative privileges required, system administrators receive a turnkey solution that streamlines container creation for all Singularity users. This service is available on-premises or in the cloud.
- Container Library Services – Delivers a repository to host Singularity containers. Users can manage, store, and share containers in public and private directories, or share private containers with other users via a generated link. Available as a cloud service, or for on-premises deployment, the Container Library optimizes the management and storage of containers within your organization.
- Key Management Services – Ensures container signing and validation services to Singularity and the Container Library. These key-signing and verification services eliminate the risk of unknowingly downloading and running compromised containers, greatly enhancing an organization’s ability to restrict the types of containers that are allowed to run on a cluster.
Singularity–Into the Enterprise
Singularity solves the challenges of security, flexibility, and reproducibility found in many container technologies today. Lenovo chose to mitigate these challenges by embracing support of Singularity to help build part of their Lenovo Intelligent Computing Orchestration (LiCO). Singularity ensures LiCO users have access to container technology that streamlines HPC application delivery for all enterprise AI use cases. “Singularity is a foundational element of our LiCO software strategy to harness the power of open source and make AI simple for our enterprise customers,” said Scott Tease, Executive Director of HPC & AI business, Lenovo Data Center Group. “Singularity builds on what is already an extremely efficient and easy-to-use container runtime solution, differentiating their platform with exciting new features designed to further optimize enterprise use-cases.”
Lenovo and Sylabs have collaborated for over two years around a shared vision of producing lightweight HPC containers that are flexible, secure, and reproducible, while also delivering performance that is equal to native OS performance. Click here to read more about how Singularity containers provide enterprise-grade solutions alongside Lenovo, including how Lenovo demonstrated that MPI applications running in Singularity containers outperformed other container applications and KVM environments.
A Flexible File Format
New with Singularity 3.0, the Singularity Image Format (SIF) is now the default packaging mechanism for each container runtime. SIF brings cryptographic signing and validation to Singularity and underpins the functionality behind an expanded ecosystem of resources. Every Singularity container can now have cryptographically signed and evolvable overlays to enable a controls-compliant workflow throughout the container runtime process. This provides unparalleled flexibility to Singularity users by enabling them to package applications and their dependencies—including definitions, configurations, metadata, security keys, and more—into a single file that is cryptographically verifiable to ensure security, reproducibility, portability, and mobility.
With Singularity, organizations can now streamline the development of critical applications, shifting investment into innovation while simultaneously increasing flexibility and security throughout the application development and implementation processes. Distributors and users now have access to a container development and runtime format that establishes a level of trust, security, and ease-of-use that is unavailable with other container formats.
Introducing Singularity + GoLang
Effective workload management continues to be a key differentiator in improving efficiency in HPC, compute-drivenanalytics, and AI. To further extend the functionality of our container software, Singularity 3.0 introduces enhancements to our base code, much of it rewritten in Go. As most container tools are written in this programming language, Singularity will be able to leverage a variety of new features that would have taken much longer to implement with the more traditional C, Python, and BASH architectures. This change ensures that Singularity moves to a modern, compiled architecture, while also eliminating run-time dependencies and ensures a consistent experience across all platforms. As many of the major projects in the container ecosystem are written in Go, this transition promotes an improved level of interoperability with existing projects, enabling Singularity to be more widely accessible to a greater number of users.
Join Our Mailing List
Now that we have readied the WSL2 environment with Singularity and the relevant CUDA libraries, it’s time to run the sample Keras workflow.
Signing the Container The Singularity 3.0 family introduced the ability to create (and manage) PGP keys to sign and verify containers. This provides a trusted method for Singularity users to share containers and ensures a bit-for-bit reproduction of the original...