Clik here to view.
PARIS — Let’s say a machine learning researcher has read a research paper and wants to test it with a Python-based GPU in a PyTorch environment. She asks her engineering team to access a Jupyter notebook with two GPUs and all her libraries.
The engineering team tells her it will take three days. They will have to source the GPUs, create a stack, and then give access to JupyterHub.
“This is exactly what DevOps experienced 10 years ago,” said Janakiram MSV, an independent analyst and frequent contributor to The New Stack, in a conversation at KubeCon + CloudNativeCon Europe this month.
“So now the whole idea is, how do we accelerate this and enable enterprise IT to bring infrastructure to a point where it is readily available for ML researchers, engineers and developers, where they can quickly translate their ideas into code?”
New personas reflect large language models’ (LLMs) impact on the cloud native community and raise questions about identity and Kubernetes’s role. Is Kubernetes even needed for data scientists to get their models into production?
“At NVIDIA’s GTC conference, they announced something called Nim,” said Sanjeev Mohan, an independent analyst, in a recording for an upcoming episode of The New Stack Makers. “NVIDIA inference microservices Nim is a Docker container orchestrated by Kubernetes. This only happened last week when we were at KubeCon, and NVIDIA GTC was going on at the same time.”
The stateful nature of data and the pace of change make the transition challenging for Kubernetes to become deeply data-centric. Never has data played such an essential part in the Kubernetes community. Never before has the Kubernetes community needed to adapt in such a way to the new demands brought by generative AI, model development, integration, deployment and management.
Without standard ways for data models to deploy on Kubernetes, the work ahead will require the community to accommodate the new data personas with new hardware integrations and projects.
Kubernetes and LLMs
But really, what is Kubernetes’s role in AI? The persona question brings this matter to the forefront. Kubernetes is a control plane — yes, that makes sense. Since 2014, it has served as an application architecture for DevOps.
So a question that Mohan poses becomes more relevant: Is Kubernetes for AI or is AI for Kubernetes?
At KubeCon, we saw a lot of how Kubernetes is for AI, serving as a control plane. NVIDIA was on stage in the keynote, and they talked about dynamic resource allocation to allocate fractional GPUs. That way you can save costs. So that is Kubernetes for AI. All that development is going very well, Mohan said, We will see more and more of Kubernetes becoming the control plane for general AI.
But how does AI make Kubernetes much more powerful, for example, with the use of LLMs? Mohan asked.
“I have not seen a lot on that side, maybe by next, next KubeCon or the one after that we will start seeing a higher integration,” he said.
OpenAI serves as a testament to Kubernetes, which the company uses to launch and scale experiments.
How will Kubernetes become more default for all AI workloads without having the data scientists and data engineers have to think about how to configure it, to use whatever hardware GPUs most efficiently, asked Sudha Raghavan, a senior vice president at Oracle, at KubeCon.
Raghavan, also on a panel at KubeCon, talked about a world where work on a per-workload basis gets easier, where engineers can configure out-of-the-box templates, understanding that these are the patterns of AI workloads that are yet to come and have predefined templates for them.
So any data scientists who want to do experiments don’t have to learn it on their own but rather can pick up on what the Cloud Native Computing Foundation can provide within its ecosystem for the AI and ML community.
Arun Gupta, vice president and general manager for open ecosystems at Intel, said on a panel the cloud native community’s responsibility is bridging the gap. There must be empathy for the customer, who is the data scientist. A new Cloud Native AI paper addresses these sorts of challenges, he said.
Lachlan Evenson, principal product manager at Microsoft, said on the same panel as Gupta that a new persona in the Kubernetes community also includes AI engineers, who sit between the data scientist and the infrastructure engineer or platform engineer.
Evenson noted in the panel discussion that the AI engineer needs to understand not only all the terminology that comes with the AI world but also how to use these distributed systems at scale and build these new platforms.
The Promise of Extensibility
The Kubernetes founders made Kubernetes stateless, and then later built stateful technologies to integrate with its distributed platform.
“So that speaks to not only this community but the extensibility that we build into that platform through his community,” Evenson said. “And it’s not easy, you have to be one of these big companies. And I agree with what everybody else has said, we need to make it easier to run for everybody.
“We need to give out open source alternatives and open source platforms, so that companies that are looking to start investing and understanding how AI can impact their business, that they can take models and not worry about data governance or security concerns and start to tinker with them on their local environments and get familiar with them. “
The post Kubernetes and AI: Are They a Fit? appeared first on The New Stack.
At KubeCon Europe, we heard a lot about the current and future relationship of AI and Kubernetes, the orchestrator originally built to be stateless.