When it comes to orchestrating containers, Kubernetes is a popular choice. In Seattle this week, members of the Sylabs team are taking in KubeCon + CloudNativeCon North America 2018 to immerse themselves in this critical technology and the ecosystem that surrounds it. On the most fundamental of levels, our conversations at the event are resonating particularly well with those seeking to make use of Kubernetes for their compute-intensive workloads. And because many of these conversations have been with representatives of private-sector companies, we’re anecdotally validating the case for Enterprise Performance Computing (EPC) via Kubernetes.
As this relatively recent post indicates, we arrived at this orchestration requirement completely independently. Briefly, from the Singularity Community we perceived the need to address requirements for a complex use case that blends computational demands typical of EPC with those of services-oriented computing. Kubernetes is ideal in addressing the services-oriented requirements, as real-time streaming and data pipelining into compute-focused services characterise this hybrid use case.
In November we declared the intention to integrate Singularity containers with Kubernetes at SC18. Then, and less than three weeks ago, we introduced a new open source project aimed at satisfying this need for native integration through the implementation of a Kubernetes Container Runtime Interface (CRI) ‘shim’ – a gRPC server which serves Kubelet requests by interoperating with the Singularity runtime.
When we announced Singularity CRI on November 26, our proof of concept implementation of image and runtime services passed 10 of 71 tests that serve to validate any CRI implementation. With KubeCon focusing attention on Kubernetes this week, we thought it was the perfect time to let everyone know that the development efforts over the past few weeks have resulted in our POC implementation now being valid for 52 of those 71 tests! Of course, we can’t yet predict when all tests will be validated, but this five-fold improvement in a matter of days is extremely encouraging – encouraging because those with EPC use cases will soon have a container alternative in Singularity that presents a simpler, faster, and more secure container runtime for compute-driven workloads via Kubernetes.
Our efforts regarding the Singularity CRI have been received with considerable enthusiasm, and have already generated significant interest. For example, ensuring the Singularity-Kubernetes integration ultimately includes support for the now generally available Container Storage Interface (CSI) has appeared as an add-on requirement. Rest assured that the CSI will be factored into the integration roadmap beyond Singularity CRI, and that you can expect more on this capability in the New Year from us.
As we stated when we announced Singularity CRI:
We are releasing this code while it remains under active development for your review, feedback, and hacking pleasure. You should not expect a working implementation at this time, but a clear indication of the direction this project is heading. Obviously, this POC is not intended for general consumption, but targets developers having interest in contributing to Singularity, Kubernetes, or similar technologies.
Despite the rate of progress, there remains ample opportunity to contribute to the implementation of this native integration between Singularity and Kubernetes. To conclude this post, we’d like to issue an appeal to those attending KubeCon to review our CRI implementation, provide us with your feedback, and pleasurably hack support for Singularity containers into your Kubernetes projects that emphasize compute-driven workloads. We look forward to delivering this integration with the direct involvement of the open source communities around Singularity, Kubernetes, and beyond.
[Source for featured image is here. Use licensed via Creative Commons.]
