In this episode, we sit down with Nilesh Agarwal, co-founder of Inferless, a platform designed to streamline serverless GPU inference. We’ll cover the evolving landscape of model deployment, explore open-source tools like KServe and Knative, and discuss how Inferless solves common bottlenecks, such as cold starts and scaling issues. We also take a closer look at real-world examples like CleanLab, who saved 90% on GPU costs using Inferless.

Whether you’re a developer, DevOps engineer, or tech enthusiast curious about the latest in AI infrastructure, this podcast offers insights into Kubernetes-based model deployment, efficient updates, and the future of serverless ML. Tune in to hear Nilesh's journey from Amazon to founding Inferless and how his platform is transforming the way companies deploy machine learning models.

Subscribe now for more episodes!

Show Links:

OpenShift 4.17 is GA https://www.youtube.com/live/DvKHwz-c11c?si=6Zap6hk_GsQfdX2m
Policy SBOM from Styra: https://www.styra.com/blog/introducing-policy-sbom/
NVIDIA GEForce NOW runs on KubeVirt https://thenewstack.io/now-nvidia-scaled-its-cloud-services-with-kubevirt/
CBT feedback https://thenewstack.io/kubernetes-advances-cloud-native-data-protection-share-feedback
CNCF KUBEEDGE Grad https://www.devopsdigest.com/cncf-announces-kubeedge-graduation?utm_source=tldrdevops
Palumi Operator 2.0 https://www.pulumi.com/blog/pulumi-kubernetes-operator-2-0

Inferless LInks:

https://www.inferless.com/blog/cleanlab-saves-90-on-gpu-costs-with-inferless-serverless-inference
https://www.inferless.com/blog/how-spoofsense-scaled-their-ai-inference-with-inferless-dynamic-batching-autoscaling
https://www.inferless.com/
https://docs.inferless.com/introduction/introduction
LinkedIn - https://www.linkedin.com/in/nilesh-agarwal/
X- https://x.com/nilesh_agarwal2
Medium Blog https://nilesh-agarwal.medium.com/

Inference in Action: Scaling Al Smarter with Inferless 55:17 Share

Kubernetes Bytes

Shownotes Transcript

Inference in Action: Scaling Al Smarter with Inferless