The Kubernetes community is application — and infrastructure-centric. It’s all about DevOps, developers, operations and improving the developer experience. Data science? It’s historically moved slower, and not something that the Kubernetes community and Cloud Native Computing Foundation have much bothered about.
But now we see a new persona emerge with the AI engineer with a far faster pace of advancement.
All the things that come with data are about state, which means that if a database were to go down, unlike an application, it’s not possible to wake up a node and restart the database, said Sanjeev Mohan, an independent analyst, in this episode of The New Stack Makers.
In such a scenario, transactions may be in motion. “What happens to those transactions?” Mohan asked. “Your database could be in an inconsistent state if you just automated some of these tasks. So that is a reason why it has taken a long time for the data community to come on board with Kubernetes.”
The AI Engineer Factor
Historically data scientists moved at a different pace than developers. Developers, Mohan said, want to test features and see how they run. Data scientists trained their models, wrote the algorithms, and tested them.
But today, Mohan said, data engineers have picked up the pace tremendously. They are constantly trying new AI applications or workloads.
“So as a result, Kubernetes now has an important role to play in helping the data scientists,” he said. “Maybe it’s not even the data scientist, actually, it is the new role of AI engineer, who is more aware of what resources are being provisioned and how these resources are being consumed.”
Large language models (LLMs) are expensive to train, and GPUs are even more expensive to use, Mohan noted. So cost and resource utilization become critical in the AI space.
Kubernetes has an exciting role because it can help provision, monitor and observe these resources. If there’s a problem, you can shut it down or restart it. Saving costs becomes really important.
In time, as LLMs become increasingly commoditized, the needs will change to inference.
“I’m very interested in AI agents,” Mohan said. “But how do you build an AI agent? You need some framework. Microsoft has a framework, OpenAI has a framework. And now guess what? There are dozens of frameworks.
“How do you distinguish? How many frameworks do you need? So my focus for the next year or so is going to be on the business side. How do you align the business use case into the AI space?”
Check out the entire episode for more on Kubernetes’ burgeoning role in AI and data science.
The post How Kubernetes Faces a New Reality with the AI Engineer appeared first on The New Stack.
Kubernetes now has an important role to play in helping data scientists, according to Sanjeev Mohan, an independent analyst, in this episode of The New Stack Makers.