We're sunsetting PodQuest on 2025-07-28. Thank you for your support!
Export Podcast Subscriptions
cover of episode Kubernetes, AI Gateways, and the Future of MLOps // Alexa Griffith // #294

Kubernetes, AI Gateways, and the Future of MLOps // Alexa Griffith // #294

2025/3/7
logo of podcast MLOps.community

MLOps.community

AI Deep Dive AI Chapters Transcript
People
A
Alexa Griffith
D
Demetrios
Topics
Alexa Griffith: 我在软件工程领域的职业发展受益于选择具有实际商业价值的问题，这让我能够参与许多开源项目，例如Envoy AI Gateway和KServe。在早期使用Airflow和Kubernetes的经历中，我发现Airflow经常崩溃，无法处理大量的并发工作负载。随着时间的推移，出现了许多新的工作流程工具，例如Kubeflow和Argo Workflows。在AI/ML领域，基础设施工程师和AI/ML工程师在工具和技术偏好方面存在差异，为了更好地服务AI/ML工程师，我们努力将Kubernetes的底层细节抽象化。最近，我在KubeCon上介绍了Envoy AI Gateway这个新的开源项目，它是在Envoy Gateway的基础上构建的，增加了针对大型语言模型的特性，例如基于令牌的速率限制。Knative Serving是一个帮助运行无服务器应用程序的开源工具，它构建在Kubernetes之上，KServe是一个简化AI模型运行的抽象层，它提供易于使用的配置和统一的API。Envoy AI Gateway也提供统一的API，可以访问不同云平台和本地部署的模型。大型语言模型带来了新的挑战，例如模型大小和冷启动时间，KServe正在努力解决这些问题，例如模型缓存。Celery是我对开源的首次贡献，它与Airflow有关。如果真的想参与开源项目，我建议从一些简单的任务开始，例如good first issue，并积极参与社区活动。在工作中参与开源项目，并将其与实际业务价值联系起来，是最好的学习方式。在评估项目价值时，要考虑其用户数量、盈利能力和实际用途，避免在无用功能上浪费资源。与超级用户建立联系，可以帮助发现痛点并改进产品。 Demetrios: Alexa在职业生涯中受益于选择具有实际商业价值的问题，这让她能够参与许多开源项目。我很好奇Alexa在使用Airflow和Kubernetes时遇到的最大难题是什么。我很好奇Alexa在使用Airflow和Kubernetes时遇到的最大难题是什么。在AI/ML领域，基础设施工程师和AI/ML工程师在工具和技术偏好方面存在差异。Envoy AI Gateway可以作为抽象层，连接不同的模型端点。大型语言模型带来了新的挑战，例如模型大小和冷启动时间。你如何看待传统机器学习用例和新的LLM AI用例之间的差异以及它们在平台中的融合？

Deep Dive

Shownotes Transcript

Claude Plays Pokémon - A Conversation with the Creator // MLOps Podcast #294 with Alexa Griffith, Senior Software Engineer at Bloomberg.

Join the Community: https://go.mlops.community/YTJoinIn) Get the newsletter: https://go.mlops.community/YTNewsletter)

// Abstract

Alexa shares her journey into software engineering, from early struggles with Airflow and Kubernetes to leading open-source projects like the Envoy AI Gateway. She and Demetrios discuss AI model deployment, tooling differences across tech roles, and the importance of abstraction. They highlight aligning technical work with business goals and improving cross-team communication, offering key insights into MLOps and AI infrastructure.

// Bio

Alexa Griffith is a Senior Software Engineer at Bloomberg, where she builds scalable inference platforms for machine learning workflows and contributes to open-source projects like KServe. She began her career at Bluecore working in data science infrastructure, and holds an honors degree in Chemistry from the University of Tennessee, Knoxville. She shares her insights through her podcast, Alexa’s Input (AI), technical blogs, and active engagement with the tech community at conferences and meetups.

// Related LinksWebsite: https://alexagriffith.com/)

Kubecon Keynote about Envoy AI Gateway https://www.youtube.com/watch?v=do1viOk8nok)


Catch all episodes, blogs, newsletters, and more: [https://go.mlops.community/TYExplore](https://go.mlops.community/TYExplore))

Join our slack community [[https://go.mlops.community/slack](https://go.mlops.community/slack))]

Follow us on X/Twitter [[@mlopscommunity](@mlopscommunity))][[https://x.com/mlopscommunity](https://x.com/mlopscommunity))] or LinkedIn [[https://go.mlops.community/linkedin](https://go.mlops.community/linkedin))] 

Sign up for the next meetup: [[https://go.mlops.community/register](https://go.mlops.community/register))]

MLOps Swag/Merch: [[https://shop.mlops.community/](https://shop.mlops.community/))]

Connect with Demetrios on LinkedIn: [/dpbrinkm](https://www.linkedin.com/in/dpbrinkm/))

Connect with Alexa on LinkedIn: [/alexa-griffith](https://www.linkedin.com/in/alexa-griffith/))