Starting from a simple application which can be deployed in every machine running Docker, we will go through all steps required to transform the simple app into a Kubernetes native streaming application. We will explain the theory and then exemplify the learnt concepts to define a recipe for running streaming applications on Kubernetes. We will focus both on cultural and technical tricks to help you successfully adopt streaming applications at scale.

At the end of the talk, you will have a comprehensive view regarding all platform building blocks and application requirements needed to successfully run a streaming application on Kubernetes.

Spoiler: you will hear several times the words Apache Kafka, Kafka Streams and Strimzi.

Francesco Nobilia is an enthusiastic engineer focused on building the next generation of a self-service and cost-effective streaming data platform. Event-Drive addicted. Apache Kafka fun. Kafka Summit and Meetup speaker. Currently, he is Principal Engineer at Nutmeg.

Jeremy Frenay is a platform engineer at Babylon Health.
He has been leading the Babylon's Data Operations efforts since late 2017, building the Kafka based data infrastructure, the automation and the tooling required to support teams of software and data engineers working on data integration pipelines in a cloud-native environment.

#19 DoK community: Towards a K8s Native Streaming Application // Francesco Nobilia & Jeremy Frenay 01:13:52 Share

Data on Kubernetes Community

Shownotes Transcript

#19 DoK community: Towards a K8s Native Streaming Application // Francesco Nobilia & Jeremy Frenay