871: NoSQL Is Ideal for AI Applications, with MongoDB’s Richmond Alake
01:13:11 Share

This is episode number 871 with Richmond Alake, staff developer advocate at MongoDB. Today's episode is brought to you by the Dell AI Factory with NVIDIA and by ODSC, the Open Data Science Conference.

Welcome to the Super Data Science Podcast, the most listened to podcast in the data science industry. Each week, we bring you fun and inspiring people and ideas exploring the cutting edge of machine learning, AI, and related technologies that are transforming our world for the better. I'm your host, John Krohn. Thanks for joining me today. And now, let's make the complex simple.

Welcome back to the Super Data Science Podcast. Today, we've got the tremendously gifted writer, speaker, and machine learning developer, Richmond Alake, on the show. Richmond is staff developer advocate for AI and machine learning at MongoDB, a huge publicly listed database company with over 5,000 employees and over a billion dollars in annual revenue.

With Andrew Ng, he co-developed the deeplearning.ai course Prompt Compression and Query Optimization that has been undertaken by over 13,000 people since its release last year. He's delivered courses also on platforms such as Coursera, Datacamp, and O'Reilly. He's authored over 200 technical articles with over a million total views, including as a writer for NVIDIA. He previously held roles as an ML architect, computer vision engineer, and web developer at a range of London-based companies.

He holds a master's in computer vision, machine learning, and robotics from the University of Surrey in the UK. Today's episode will appeal most to hands-on practitioners like data scientists, ML engineers, and software developers, but Richman does a stellar job of introducing technical concepts, so any interested listener should enjoy the episode.

In today's episode, Richmond details how NoSQL databases like MongoDB differ from relational SQL-style databases, why NoSQL databases like MongoDB are particularly well-suited for developing modern AI applications, including agentic AI applications, how Mongo incorporates a native vector database, making it particularly well-suited to RAG, retrieval augmented generation,

Why 2025 marks the beginning of the multi-era that will transform how we build AI systems. And Richman provides his powerful framework for building winning AI strategies in today's hyper-competitive landscape. All right, you ready for this fun and informative episode? Let's go. ♪

Richmond, welcome to the Super Data Science Podcast. We're filming live in person in London in your beautiful MongoDB office. Thank you for having me. Thank you for coming here. It's a pleasure to have you in London. It's nice to do this in person. It's been nice. I haven't been in London long. I did a flight just yesterday. And so obviously with the jet lag from New York, it was, you know, I couldn't fall asleep until kind of 4 a.m.

And then the craziest thing was where I'm staying. I already told Richmond this, but they were doing fire alarm testing in the morning that started at 10 a.m., which I get is fine for most people. But when you've just come from New York and you're going to bed at 4 a.m., I was like, oh, my God, this is crazy. And it was like in my room, like the fire alarm in my room starts going off at 10 a.m.,

Since I've gotten to the MongoDB office, it's been a very nice day. Beautiful space here. You can see a little bit of the background, maybe even a little bit of the Shard, which is a kind of iconic London building in the background.

So Richmond, you previously were on the show in episode number 685. That came out in June two years ago, June 2023. And in that we focused on building real-time machine learning applications. We are going to talk a little bit in the episode about building machine learning applications. But all of that now is going to be colored by the perspective that you've developed over the past year and a bit as staff developer advocate for AIML at MongoDB. So

Tell us about how that came about. What is a staff developer advocate role, and how did you get into it? Yeah, that's a very good question. And again, massive pleasure to be on the show. And it feels like a lot has happened. It's only been two years, right, since the last time I was here. But yeah, a lot has happened in that time. So obviously, the major thing is I'm at MongoDB as a staff developer advocate, and I've

When I came on your show the last time, I think I spoke about some of the external writing that I do outside of my day job. So back then I was a machine learning architect at a consultancy company. And I did a lot of, after my nine to five, I did a lot of writing and I did a lot of teaching as well. So it turns out,

It's a full-time job. That's a full-time job. So it's called developer advocacy, and people like me reside within the developer relations of certain organizations. So what I used to do for fun is actually now my full-time job. So I get to teach our customers, both externally and internally, on the latest and greatest on AI. And I do that in various forms. So it could be writing content.

It could be going on podcasts. It could be speaking directly to our customers as well over at MongoDB. And we have a lot of customers that are building very interesting AI application. Now, the staff level of it is just amount of experience and the expectations

that's expected of you as well. That was my next question. So being a staff developer advocate, it's not like that's an internal facing role. It's like when somebody gets a staff machine learning engineer role. It's a level of seniority. Yes, exactly. Cool. So let's talk about MongoDB specifically. So I have built startups that leverage MongoDB in the background for a decade now.

And so I can speak about it kind of really vaguely, which is this, the kind of the key difference for people who are already familiar with SQL. SQL is a structured table. So it looks like when you see like an Excel spreadsheet or a Google sheet where you have specific columns with names and a certain number of rows, that is similar to how SQL tables are structured. A NoSQL database like Mongo

doesn't have that structure. So, Phil, listen. It has a structure, but it's not the tabular structure that you get in relational database. And you point on one thing, which is MongoDB has been within the space for maybe over a decade now. And we've been supporting startups and large enterprise organizations with

just application development. I'd also say really quickly is that Mongo is the NoSQL company. Like when I think in my own vector representation in my brain, like a meaning representation, NoSQL and Mongo, they occupy the same location in my internal vector space. That's very good. And I'm

I'm hoping after this conversation, you're going to see MongoDB as the database for your AI application. And you can make that. And that takes up another, I guess, space within the latent space of your brain. Nice. Yeah. And so, sorry, I interrupted you as you were about to explain MongoDB.

how NoSQL database differs from a relational database like SQL. Yeah, so relational database are used data stored in that tabular format. So you have rows and columns and relational database has been, they've been within the whole space for, since the 80s.

So they work well for the use cases and some of the applications that are built. Well, MongoDB is a database that is based on the document data model, which essentially you're storing data in key value pairs.

One thing that resonated with a lot of startup developers back in the early days of MongoDB was how easy it was to get started with MongoDB and how when you use the MongoDB database, it reflects how the developers think.

which is in that JSON-like structure. And for me, it was a very, very massive moment in my career. And I don't think I touched on it the last time I was on this podcast, but I was a full-stack web developer about maybe five, maybe seven years ago. But I used to hate programming.

But I fell in love with programming because MongoDB made it easier for me to think about the stack. So I was a front-end developer, but when I understood, when I took MongoDB on the data layer and I saw that I can represent the data in the application layer in the same way within the data layer, which is that JSON structure I was talking about, that document data model,

Everything clicked. It changed my career and I became a Mean Stack developer, right? A full stack developer using the Mean Stack to build web application. Yeah, and so MEAN, M-E-A-N, it's an acronym where

each of those letters represents a different layer in the stack, and the M is MongoDB, right? It's the base of the stack. And the E was Express, and A was Angular, and N was Node.js. And it was actually Angular.js, then you have the Angular. So before Angular that we have now, there was another library called Angular.js, which was a whole much harder to work with. Mm-hmm.

One of the things that I remember as to why we went with Mongo databases way back, yeah this would be about 2014, we were making database decisions, so 11 years ago. And one of the reasons why we went with Mongo and this kind of NoSQL, JSON-like structure, key value pairs that you're describing, is that it allows you to have way more flexibility. So a given part of your application of your website

you might not know for sure long term what information you're going to have on that page or in that part of the application. And so MongoDB makes it really easy instead of needing to plan in advance and be really rigid about how some data are structured or some part of your application is structured, you can just say, "Okay, today we're going to have this key. It's going to be called this. It's going to be populated with this kind of data.

And if in the future you change that, that's okay. Yeah, yeah. And where you hit the nail on the head, firstly, is that you hit the nail, you said it very well. And we're seeing the same thing within AI today. So the AI space is moving fast.

incredibly fast. I think that's an understatement, right? It felt like yesterday ChatGPT came out and it was just two years ago when ChatGPT came out in November 2022. But one thing that we do at MongoDB is we have that document data model which allows you to have a flexible schema, right? Which means that as your understanding of the requirement of your application evolves, your data layer can evolve with it.

But at the same time, once you're stable, once you have a stable understanding of the data representation that you want, the way data is represented within the application layer, you can actually lock your schema in MongoDB. So it gives you that fixed structure once you're going to production, but you get the flexibility once you're experimenting, which...

is what a lot of people are doing with AI today. So they need that flexible data structure. They need what MongoDB brings. Makes perfect sense. Something that I'm kind of thinking off the top of my head, if you were building some kind of application that leveraged LLMs today, you might be experimenting with calling different LLM models. Sometimes you're calling an OpenAI API. Sometimes you're calling Cohere. Yes.

Other times you have your own LLAMA-based model running on your own infrastructure. And so you might have, depending on which LLM you're calling, even if your user experience is remaining relatively consistent, under the covers you might have different fields. Like maybe when you're calling your LLAMA LLM, it has different outputs. Maybe it has some additional field that you don't have when you're calling the OpenAI API. And so Mongo allows you to very flexibly

under the hood, just be like, okay, we're going to store that extra information. Whereas if you were trying to do everything in the tabular way, you would have had to have already had that column in place for that LLM that you ended up using later on. Yes, and we have... And one thing is...

Again, we have a lot of evolving and experimentation happening within AI workload today. And having a rigid schema, having to do schema migration just because something's changed in your application space that you didn't realize at the beginning, it causes a lot of pain for developers. And in the age of AI that we're in today, speed is a competitive advantage.

Enabling your developer, developer productivity is a competitive advantage. So if you have a flexible database and you have MongoDB, it allows developers to move a lot quicker. And we also realized that we also wanted these LLMs to give us structured output. So you have the structured output in OpenAI or JSON mode is what most of the LLMs have. And that resonates with what we were talking about earlier, which is

What resonated for developers like yourself 10 years ago was, hey, JSON makes a lot of sense in the application layer. Now it makes sense in a data layer. MongoDB right now, if you look at it, we have LLMs enabled in JSON mode. Now your LLMs, it's just a way to think of the whole AI stack in a unified manner.

And you think about JSON in your application layer, you think about JSON in your data layer, you think about JSON in your LLM. It just makes everything easier. Nice. That's a great soundbite. Maybe we'll have to convert that into a YouTube short or something. I love it. It's perfect.

So we've now covered what NoSQL databases are, how Mongo is kind of the archetypal NoSQL database, and how it varies from SQL relational databases. And there's ways that people can get access to MongoDB for free right away.

There's a MongoDB Community version, which I'll provide a link to in the show notes, but that is free. It also may not have the latest features that MongoDB proper offers. So it sounds like you're going to kindly provide me with a link to MongoDB Atlas, which is a MongoDB Cloud platform that's free to start.

and that allows you to get going with MongoDB. Yes, exactly. So in the show notes, there's going to be a link for you to create a free MongoDB account, and you can just try MongoDB for your AI application. So storing your operational data, your metadata, and also, which we're going to talk about, storing your vector data as well.

Yes, that is super exciting. In fact, I think we should talk about that next. But I want to also quickly say it sounds like this has almost been like a big, long infomercial so far because I'm such a big Mongo fanboy. Obviously, you're a Mongo developer advocate, so you are as well. But I am not being paid to say any of this. Mongo has in no way sponsored this episode of the podcast. This is just genuine passion for...

about this tool. Exactly. And I think that's what Mongo does, right? Because it resonates deeply with developers building these applications. And we understand how developers think. And now in this AI era, we also understand how LLMs are providing outputs. It just makes building an application fun. That's what Mongo... No one wants to be a database administrator. No developer wants to be a database administrator. So MongoDB lets the database... It just makes it fun.

makes it fun for the developer. And yeah, one thing, sorry to interrupt, but one thing about Fanboy and I wanted to point out is a couple of years ago I used to write a lot about MongoDB and there's some stuff seven years ago on Quora, me telling people to go use MongoDB. So it's actually, it's like a full circle moment for me to come back and have an impact.

This episode of Super Data Science is brought to you by the Dell AI Factory with NVIDIA, helping you fast track your AI adoption from the desktop to the data center.

The Dell AI Factory with NVIDIA provides a simple development launchpad that allows you to perform local prototyping in a safe and secure environment. Next, develop and prepare to scale by rapidly building AI and data workflows with container-based microservices, and then deploy and optimize in the enterprise with a scalable infrastructure framework. Visit www.dell.com slash superdatascience to learn more. That's dell.com slash superdatascience.

For sure. That's great. I'm glad you're getting to enjoy that. So MongoDB Atlas, tell us about why somebody-- what advantages you get working with a cloud system like MongoDB Atlas

versus trying to set up maybe your own MongoDB running on your own infrastructure? Yeah, so with MongoDB Atlas, you get a managed offering of MongoDB, right? And one of the key things is you can use MongoDB on any of the large cloud providers today. So AWS, Azure, and Google Cloud across everything.

in several different regions. So we have a multi-region, multi-cloud infrastructure. And the database layer is also distributed within MongoDB. So you get all of this. Again, what I was saying is no developer wants to be a database administrator. You get all of this within MongoDB Atlas alongside other products such as streaming. And we have dedicated search nodes, which we can talk about when we're talking about the vector data

the vector data capabilities of the MongoDB database. So that's why essentially you would. Nice. Yeah, let's talk about those vector databases right now. So I already made a joke earlier about how, so if you are not already kind of an AI developer, then you may not be aware of vector spaces, but essentially,

vectors are a high dimensional space, so it's very easy to imagine a three dimensional space, 'cause that's the kind of visual world that we live in.

And so if you imagine a three-dimensional space, you can take any kind of object. And so it was a big thing around 2011, 2012 to be taking words and putting them into a vector space. So there was this very famous algorithm Word2Vec by Thomas Mikulov. And that was an algorithm for taking words in natural language and using

some really interesting properties related to words, which is that the meaning of a word tends to be the average of the words around it, which is a trippy thing to think about, but ends up being true because you can train these computer science algorithms to say, "Okay, just based on the words around, without any labels in your data, you can just take a large amount of natural language

and use something like Word2Vec to create kind of like a word cloud in however many dimensions you want where the closer you are in that space, the more similar the meaning of the words are. So the British versus the American spelling of a word would typically end up in exactly the same location because whether you say "honor" with or without a "u", it has the same meaning.

And then synonyms would be close by. You could end up with a region of database languages. So you'd have NoSQL and MongoDB would be very close together in a well-trained space.

close by you'd find SQL and MySQL and kind of relational databases. And then in another part not too far away you might have Angular and React, you know, programming languages. In a completely different area of the 3D space you might have like dates and times. And so just you end up having meaning represented in this three-dimensional space. But with a vector space you could have a hundred dimensions or a thousand dimensions.

And so that allows you to have a lot more granularity in that space where moving along one of those dimensions represents some kind of meaning to the way the data are stored. And so anyway, vector databases have been hugely important over the past decade, starting mostly with words but now whole documents. So for RAG, for example, really important for retrieval augmented generation.

you typically take, let's say you have a million documents, you could convert each of those documents into a location into some high dimensional vector space

And then you could in real time, based on some query that somebody asks, you could retrieve the right relevant documents based on the semantic meaning of the query. So yeah, vector databases are huge these days for applications like RAG. And I did not know until we were preparing to do this interview that Mongo offers a vector database as well. Because I always think of Mongo as just that JSON structure

But yeah, you have vector databases now as well. Tell us about that. Yeah. So with MongoDB, we work with a variety of customers in

Our roots started, obviously, with storing data within a database. So we saw most of our customers over a decade ago using search engines on top of MongoDB. And to make it easier, because that's what we're focused on, which is our customer needs, we brought search into the database. So we had Atlas search. Then we also noticed that a lot of our customers were dabbling within use cases that were related to semantic search.

with their vector data. And most of these folks were bolting on if extensions or maybe another database to implement vector search. But we brought that into MongoDB. So now you have one database where you can do your normal...

lexical search, but also you can do vector search as well. And you can actually combine both and you have hybrid search. So in a situation where you would have...

And I've seen this in some cases, two free databases. With MongoDB, you have one database that can power your entire AI application. So that is the beauty of it. It makes a lot of sense. You could imagine if you were starting from scratch and you knew that, you would probably think that seems easier than getting some separate vector database and having to be confident that the linkages between them are always going to work because you can end up having version issues where

where that vector database has some kind of update where it no longer brings in the field like you'd expect from Mongo or Back. Whereas when Mongo's doing all of it, there's probably people testing to make sure that everything's working properly. Exactly. Spot on. Nice. All right, so let's now talk about something that's an extension of this idea of vector databases and AI and MongoDB. You recently wrote a blog post on AI stacks.

And it's actually right now at the time of recording, if you Google the term AI stack, your blog post comes up as the number one hit. So I'll have a link to that in the show notes. We talked a little bit earlier in the episode about things like the mean stack, which was this idea of back end all the way through to front end technologies for the developer.

Is an AI stack somehow related to that kind of thing? Yes. So we said the main stack was, well we didn't say, but we know that the main stack is a composition of a bunch of tools and libraries to build web applications. So the AI stack is a composition of tools and libraries to build an AI application. One thing I would say is the AI stack

It's different in terms of how you visualize it, depending on the persona you're talking to. So when I'm talking to developers and when you see the, when you look at the article, when I'm talking to developers, there are more layers in the AI stack than when I explain the AI stack to like a C-suite or VP-like person. And that's because I feel developers need to really go deeper.

We really need to dive deep into what is making this the AI application today and understand the composition. But for some CEOs and VP-like folks, they don't need to know the intrinsic detail. They need to know the high-level information. So just to make the point is most of VPs or high-level VPs

execs within companies who describe the AI stack as you having the application layer, you have your data layer, and you have your compute layer. Very easy. So application would be-- sorry, you have your application layer, you have your tooling layer, then you have your compute layer. So application would be any of the products you see today. So cursor, a very popular IDE that is powered by AI, would lie in the application layer.

then in your tooling layer, you have folks like MongoDB or any of the tools that enable the application layer. Then in your compute, you have like NVIDIA. But when I'm talking to developers, I double click into that and we talk about the other layers of the stack. I'm not going to remember everything now, but programming language is very important.

When you're developing this AI stack, AI application, the languages you select is very crucial because not all libraries that you're going to be using further in the stack are written in all the languages you have available. Some are just Python, or maybe some are just JavaScript, right? There's some that are evolving to have both now. But your programming language is crucial.

you have your model provider, you have the data, your database, which would be MongoDB, then you have your model provider. So you have to have several layers to that stack when I'm talking to developers. I tend to dive deep into that.

Nice, yeah. So that if people want to kind of have a framework for describing the different tiers, like you said, from the bottom all the way to the top, from your compute layer all the way through to your application layer, this AI stack framework that you provided. And again, we'll have a link in the show notes to make that a piece of cake for people to follow along with. Another thing that you published recently

is something called MemoRiz. So it's like memories, but it also has, is that like Riz? Yeah, that's like Riz. That is me coding late in the night and just coming up with a funny name for like a library. That's like a Gen Z term. It just means like good?

That was my attempt at trying to be young. Please, some Gen Z listener, provide us with information. Yeah, I thought it was funny. I don't know if people think it's funny. But really, that was an interesting project that I dabbled in. And it was mainly trying to solve a bunch of issues I was seeing around some of our customers building AI application. And that is, it had everything to do with memory. Right?

So as we're going into, or we're already in this agentic era,

The way memory becomes important, data becomes important, and the way you structure, store, and retrieve memory actually affects the reliability and the performance of your AI systems, of your agentic systems. So memory is becoming an important aspect of our AI application today. So we had RAG in 2003, in 2004. Vector databases made RAG a popular... In 2023, 2024.

In 2023. What did I say? 2003, I think. Well, 2023 and 2024, vector databases made RAGA very popular mechanisms within AI application. But as we move into late 2024 and 2025, we have more the form factor of agentic systems emerging

more prevalent within the space. And agentic rag became a thing that people were implementing. But now what we're seeing is there are a bunch of companies and open source libraries that are focused on how data is structured.

within and retrieved within this agentic system and the different scoring mechanism that goes beyond the relevance, right? There's a bunch of research papers on this and one company that comes to mind will be MemGPT. The company behind it is Letta. There's another company called, there's another paper called Hipporag.

And the main problem they're trying to solve is how do you store memory efficiently within a storage layout, but retrieve it efficiently at the right time for your agentic system. So memories was my attempt to solve some of the problems I was seeing around memory management in agentic systems.

What kinds of issues do people run into when you want to have an agentic system? Is it common to run out of memory when you're trying to develop some rag application? What are the common issues that people run into with memory? In respect to the, is it common to run out of memory? It's common to run out of context window, right?

Obviously, we've got a large context window now, 1 million context window. How deep does your pocket go is a joke I make to folks that are trying to stuff 1 million worth of tokens at every inference school. Oh my goodness. But one of the issues or the problems I saw relating to that context window is that

Within the agentic space, the LLMs are now capable of tool use, right? Which is tool use is when you provide tools

an LLM, a collection of JSON schema tools that it can use to complete an objective. So this would be JSON schema of maybe Python functions or microservices within your system that the LLM is aware of and now can actually select the right tool to use, select the right parameters, and then you invoke it on your system. That is tool use, essentially, or function calling, whatever you want to call it.

One thing is if you're using some open AI models on the documentation, the guidance is you only put between 10 to 20, um,

JSON schema of these tools within the context window at a time because obviously you have limited context window and the more tools you put in, the harder it gets for the LLM to pick. So in the library memories, what I implemented was a design pattern that we call MongoDB as a toolbox.

Essentially, what you're doing is you're using MongoDB, and this is where the MongoDB affinity for JSON comes very useful again, that document data model, is we store all the tools you can have within your system. It could be a thousand tools. You can store it within MongoDB and have a vector representation in MongoDB as well alongside your metadata of the tool. The vector representation could be

the function and some of the Google Docs string of the tool that describes when the tool is used represented in numerical format. And what you can do is just before you make a call to the LLM, you can actually make a call to your database to get the right tools. And that way you can maybe send maybe two or three tools.

In today's ever-changing AI landscape, your data demand more than the narrow applications and single model solutions most companies offer. Domo's AI and data products platform is a more robust all-in-one solution for your data. With Domo, you and your team can channel AI and data into innovative uses that deliver measurable impact. While many companies focus on narrow applications or single model solutions, Domo's all-in-one platform brings you trustworthy AI results,

secure AI agents that connect, prepare, and automate your workflows, helping you and your team gain insights, receive alerts, and act with ease through guided apps tailored to your role. And you get the flexibility to choose which AI models to use. Domo goes beyond productivity. It transforms your processes, helps you make smarter, faster decisions, and drive real growth.

Data can be hard. Domo is easy. Learn more today at ai.domo.com. That's ai.domo.com. Nice. That makes a lot of sense. Yeah, it's an interesting design pattern. And it allows your agentic system to scale, right? So now you can have as much tools as you want.

and you're bypassing the limit. - Nice, and it's a great tie together of right from the very beginning of how JSON structures that MongoDB kind of inherently recapitulates with the key value pairs, how that's always as a developer, since you discovered it, it's made developing a breeze, turned you into a full stack developer, and now it's allowing you to come up with clever ways of using LLMs to be calling tools as well. Nice.

Speaking of agents, many folks have been saying, including probably myself at times, that 2025 is the year of the agent, of agentic AI. But before we started recording, you said to me that it's the year of multi. Tell us about that.

2025 is the year of agents in different perspective, right? And I think in the mainstream commercial perspective, I was, and a lot of people within the space have been screaming about agents before this year, last year, even some people in 2023. Now it's mainstream, it's commercial, now everyone is aware of it. But I think it's our job as leaders within the space and being at the forefront and helping our customers to be several months ahead.

of our customers if we're able to provide good services to them. So agents, they're good, they're nice, and we should all be building them and learning about them. But I think the 2012 is the multi-era of AI. So what I mean by that is multi-agent architecture is going to be very popular. They're going to be very popular. We're going to have multimodal embeddings. Now you have embeddings that it's not just

Text, they're images, videos, audio, all captured using one embedding. Multilingual embeddings that can understand different languages. And you're going to have multiple retrieval mechanisms within your agentic system. So vector search is not all you need.

Vector databases were popular, but you can't just rely on vector representation to retrieve the right information from your database to give to your LLM. You need more various different search mechanism. And we're seeing a lot of people move into this memory management space I was talking about.

Multi-agent, multi-modal, multi-lingual, multiple retrieval mechanisms. There's multi-step reasoning as well. So 2025 is the multi-era for AI. Yeah, we're taking up some of the foundations that were honed over 2023 and 2024 and scaling them up in a multi-way.

You're absolutely spot on. Looking beyond 2025, what are your predictions for agents beyond this year? Predictions with AI are... They're very funny to me because everyone's always wrong. No one could have predicted we'll be here maybe three, four years ago, right? If I was to make a sound prediction, it's not going to be wild. I think voice agents are going to be... They're going to rise in popularity, right?

I have a very strong opinion that voice agent might not take off as much people think it would. They're not going to be the dominant way that we engage with agents. But voice agents, everyone is going to want to implement voice within their AI application because

Because just experiment, like I said, the experimentational nature of AI in general. I see memory management becoming an important topic. And I met Andrew Ng. He's an AI pioneer. And the first thing he said is, "Sat. Richmond, how do you model memory within a genetic system?" This was last year, April.

And he's way ahead in the space than I am. And I see a lot of people starting to focus on memory management and modeling memory within a genetic system because that's how humans work. We don't have just one form of retrieval mechanism within our brain. We have several different forms of memory, recalling memory. So a genetic system will need the same as well.

It's perfect that you mentioned Andrew Ng because my very next question for you is related to that. So Andrew Ng, it's an understatement to say that he's one of the most important people in data science and AI, one of the most known. And he was on this show in episode number 841 a few months ago.

And you had a course with him. So Andrew founded a platform called Deep Learning AI, which is a learning platform that probably a lot of our listeners are aware of and have maybe taken courses from there. And you did a generative AI course in Deep Learning AI with Andrew Yang. How did that happen? Well, it happened from my manager actually just...

contacted, I think, Deep Learning AI directly and just proposed we do a course, right? Because MongoDB is, we believe that we are one of the best, we are the best database out there for AI application. And we wanted to

spread the word. And so yeah, they were interested in what we had. They were fans of MongoDB. So Coursera in the early days was built on MongoDB. So they were interested in what we had to say. And funny enough, what we had to say in that course back then, it was released in June last year, is even more relevant today. Because we don't just talk about RAG.

We talk about RAG within a Gen-Tek system, but we talk about a pin point, which is having large context to give within LLM. So we talk about a topic which is prompt compression. So the course is prompt compression and RAG within AI systems, or something along the lines. The link will be in the show notes.

But yeah, so Andrew was excited about the course. We were excited to deliver the course. And it was a pleasure to actually meet him in person. Funny story. Andrew knew about MetaMask.

me well the digital the digital version about me uh the digital version of me before actually meeting me because back when i used to write for fun um i wrote a lot on medium and andrew has this one of the best um videos on reading research papers on youtube uh he was teaching the stanford class where and he describes to the students how to read research papers so um you

The extent of my education is to the master's level, but I still had to read a lot of research paper and I still read a lot of research papers today. So I found that video very useful around, it would be five years ago, and I wrote a Medium article on it, like breaking down and creating a framework or notion on how people can use it. And he saw that article. No kidding. Yeah. That's cool. The article went viral as well on Medium and all the good stuff.

And yeah, he just mentioned, and we spoke about the article as well. I was like, wow, you wrote the article? I was like, yeah, I did. So he met the digital version of Richmond before he met the real version. Yeah, me too. I was talking to pixels of you the entire preceding episode. They were just Richmond pixels a few seconds behind the real Richmond moving somewhere else in the world.

Nice. So what kind of listener of my show should go and take your Gen AI course in Deep Learning AI? Who's a target of that? I think most software developers should dabble in what people are building today. So if you're a software developer, go take the course.

It's very easy to break things down from first principles of, hey, this is what relational looks like, but this is what MongoDB is. And this is MongoDB within AI. And this is RAG. And this is a problem that you're probably going to face at a different level. So I break it down. So I really think most software developers should take that course.

Nice. All right, so now let's zoom out a bit. We've been focused mostly in this episode so far on specific AI tools, functionality, features that people could be using.

Let's talk now more broadly about AI strategy in general. So you are a leader in this space. You are leading teams and you're mentoring broadly within MongoDB, outside of MongoDB, on how people can build successful AI teams and products. So you're a great person to speak to about this. How can one build an effective AI strategy in such a competitive environment where there's so many

people that are trying to build something similar? Yeah, that's a good question. And it's one that I had to think a lot about in a short amount of time, by the way, as an AI moves very quickly. And I'm sure there's a bunch of AI leaders and enterprise AI leaders that are having sleepless nights, just trying to keep up with coming up with an AI strategy and the

Showing return on investment on AI initiatives, the year is also the year of show me the money within AI. But AI strategy within the space is very difficult. But one thing that I've seen that's worked is laying the groundwork of your strategy in pillars and objective truths.

So one thing that I, when I'm coming up with a strategy or if I'm thinking of how am I going to put a strategy to win a certain, for any mission or task I'm given, I always think about first principle and I look for pillars that I can base my tactics on and I look for objective truth within those pillars. So for example, if...

A pillar would be, let's say I was trying to, we're talking about developers and engineers. Let's say I was trying to think of an AI strategy that is around developers. What I would do is look at what pillars are around that developer persona, right? So one thing we know is developers live on GitHub. So GitHub would be a pillar.

then I'll think to myself, what are the objective truths of GitHub, right? So what do developers actually do on GitHub? What is something within GitHub that would never change? And if it does change, then that means there's a massive platform shift.

So those are the way I think about things. I look at a persona, I look at other attributes, I try to form the pillars, try to form objective truth within a pillar, try to form some convictions and assumptions that I'm making, and I try to form some tactics from all of this information.

Excited to announce, my friends, that the 10th annual ODSC East, the Open Data Science Conference East, the one conference you don't want to miss in 2025, is returning to Boston from May 13th to 15th. And I'll be there leading a hands-on workshop on agentic AI.

ODSC East is three days packed with hands-on sessions and deep dives into cutting-edge AI topics all taught by world-class AI experts. Plus, there will be many great networking opportunities. No matter your skill level, ODSC East will help you gain the AI expertise to take your career to the next level. Don't miss out. The early bird discount ends soon. You can learn more at odsc.com slash boston. That's odsc.com slash boston.

That makes a huge amount of sense. I really like that idea. I don't think I was doing it in such a structured way before. I was doing it kind of more in a NoSQL way in my head. Where I was... But I do... There are those kinds of things. Even though we live in this very fast-moving space, there are absolutely these kinds of pillars and these things that don't change. GitHub is a really good example. And for me, even in my career...

Going back to when I was doing my PhD between 2007 and 2012, even back then I was like, wow, the world is changing fast. Where can I find solid ground in my career? And when other people in my... I did a neuroscience PhD. And so some people are growing cell tissue cultures. Other people are doing recordings from the brains of ferrets. You have very specific skills. And I was like, well...

The amount of data that are being stored on the planet, that's going to keep increasing. Compute to be analyzing those data is going to get cheaper and cheaper. And so I was like, I should probably learn statistical computing and machine learning. That's going to be a useful thing. So there are these kinds of

mega trends or pillars that even in this very fast moving space you can feel secure around. - Yeah, it just keeps you grounded. I did the same thing when I was a web developer and I wanted to, I was looking for the next step in my career, looking for the next challenge and I thought to myself, what's an interesting problem that I can get involved in today that will be relevant for a very, very long time? And there is no more interesting problem than the replication of human intelligence.

So I said, okay, I'm going to go to university and get a master's in AI.

And that's going to be relevant. It might take different shapes and forms, but it's going to be relevant until we maybe get to AGI or some form of super intelligence. Yeah, it's interesting. Something that I've been thinking recently, I don't think I've said this out loud yet, so it'll be interesting to hear your thoughts on this, given your strong AI background in terms of education as well as commercial applications. Something I've been thinking recently is even when AGI comes about,

I think that the way that the system thinks, it'll still be quite a bit different from the way that we do. And so I think more and more, like we're seeing today where some kinds of coding problems or math problems, you can get a multi-step reasoning system to be able to be at or near PhD level math students or chemistry students on these kinds of problems, computer science students.

But there's all kinds of problems that are more difficult to package into that way. It'll be more difficult to train AI systems in that way. And so it seems to me like there's going to be places where human intelligence still has strengths, where we're complemented by these AI systems in the same way that

you know, you would rarely do long division today. You would pick up your calculator and have it do it for you. There's going to be more and more things that we can use a computer to solve for us. But I... Even when... Yeah, I don't know. I mean, I guess the idea in many ways of an AGI system, and it depends on how you define it, is that it can theoretically do any of the kind of thinking that humans do. But it still seems like there's going to be relative strengths. And so, yeah, it's just something I've been thinking about lately, that even...

There was a time a few years ago where I was like, "Wow, an AGI system or an artificial superintelligence system, does this pose an existential threat to humans?" And there's still absolutely some percentage chance that that's true. But more and more, for some reason, in a way that I'm maybe struggling to articulate really well, and you might be able to articulate better than me.

It doesn't scare me. It seems to me like we're going to be this natural complement to each other. One thing I do understand is when two intelligent systems meet, it could either be chaos or it could be collaboration. Right, right. One of the two Cs, whichever one it is. Right now, we're creating artificial intelligence, and right now it's collaborative. That's great. And I hope it keeps going that way. But even if you look at humans, right,

We are the same form of intelligence within humankind, but we still have collaboration and chaos. I think there's going to be a point where, hopefully not, but there is a probability where it switches to that chaos state. Who knows? But another C that I think that AI won't be able to replicate is curiosity. I really think that humans are one of the forms of intelligence that are just...

so curious about the environment. And we still have space exploration. We've not even left this planet yet. So we're still at the beginning of this journey. So I think the curiosity of space...

The universe is vast and we're not going to send AI to go explore the universe for us because one thing that we want to do is experience as well. We want to add the human experience into it as well. So I know we send a bunch of satellites to probe space, but they send the information back to us so we can experience and learn from it.

Curiosity is something that AI will probably never take from us. They might take what we define as work today. And I look forward to that. I don't want to work. But like I said, the things I used to do for fun, now I do it as a job. So I think human beings in a few years will have a choice. Most human beings will have a choice or the opportunity to do most things that they consider fun and get paid for it.

It might be a... Well, I'm not a doomerism, but I do think that we have to consider a possible probability of there being a chaos state between our form of intelligence and the artificial form of intelligence. Yeah, and it would be nice if... And I realize this is like...

just extremely naive, but it would be nice if we could have all of the key players in AI development like the US, China, UK, if everyone could be working together towards the same goals instead of being in opposition. Yeah, and we are working together towards the same goals. It is interesting how the research community continues to operate

Collaboratively. Yeah. Despite, you know, sanctions. Exactly. Innovation will find a way. Nice. There's another good sound bite. Nice. All right. So cool. That was an interesting tangent that we went on there. Another thing that I'd like to talk about with respect to AI strategy. So we just were talking about how to build an AI strategy in a competitive environment with your pillars.

Another framework that you've developed alongside your pillars is something that you call the Arena framework. Tell us about that. So the Arena framework is for the strategy aspect, right? It's something that I think about when I'm trying to formulate a strategy and it came off thinking about

What stood the test of time, right? And well, not entirety of time, but some form of time. And you know the Colosseums where gladiators used to fight, right? Those have stood the test of time. You can see some of them today. And the architectural buildings around them are very solid. So you look, the forms of pillars, it's a circle and they have a bunch of pillars and the pillars have segments. So you can think about that as your AI strategy where your tactics

get to battle it out in the arena, right? The arena is the market, wherever you want to call it, the free market. So formulating your strategy, one, with pillars and having segments of objective truth, understanding your resources and formulating your tactics creates very good strategy and tactics that you can put to work in the arena.

Nice. So then in terms of the team aspect of it, I think that was going to be your next question, right? So you have your strategy, you have your tactics, now you have a team. And I think the AI space moves relatively quickly. And I think for those of you that are leading teams, I think you need to be aggressive in terms of your pursuit of innovation. You need to be aggressive in terms of the release and development.

Going into the market. So being aggressive, centralized, coordinated, resourceful, efficient, and data-driven. Accreed is the short form of it. I think those principles, and if you embody those principles, you'll be very, you'll be successful in a fast-moving space. Because if you're aggressive, you're

You will approach most problems with your full potential. If you're centralized, you'll be focused. If you're coordinated, you'll be in harmony with your environment and with the resources in your environment. And if you're resourceful, you're going to use everything within your environment. If you're efficient, you're going to use it to the best of your capabilities without expending costs such as time or money. And you're data-driven, which allows you to prove your results, approve your assumption, and refine.

That was a long aspect of the way I'm thinking about things. But I think about this...

Maybe more often than not. Nice. You have some sound principles for AI strategy there. Obviously a lot to take in. Maybe it'll be a future course or book. Maybe it'll be a future course. We do have, I wanted to quickly mention before the episode ends that you have actually, in addition to all of this other work that you've been doing since we last spoke, there's a couple other things that you published that our listeners might be interested in. So in the O'Reilly platform, you published a computer vision course, which actually is

orthogonal, more or less, to all the MongoDB stuff we've been talking about today. Yeah, so computer vision is what I studied or was my strong point in my master's. So I've been doing a bunch of teaching, like I said, while I was a machine learning architect or computer...

My first role in AI was a computer vision engineer, actually. Then I became a machine learning architect. Now I'm a developer advocate for AI and ML. So I released a computer vision course in collaboration with O'Reilly last year, October 2024. And it just touches on the foundations of computer vision. So object detection, pose estimation. We're looking at different techniques. And I'm explaining the models. There's actually a...

And it's actually a practical course. So there's about four hours or maybe two hours of it got edited down. But there's about two hours of me coding using PyTorch and showing you how to build neural networks from scratch. Then we train the neural networks and we do some transfer learning. So it's a bunch of like the good parts

Where to say, but the good old stuff is not generative AI, it's computer vision. But it's really good. Yeah. I mean, it's interesting with how much we can end up in so many episodes, including today's episode. We ended up talking so much about generative AI and AI agents.

which is very much what people are talking about these days. It's new, it's cool, but absolutely things like computer vision are not going to go away. It's not a generative thing. It's not necessarily an AI agent thing, although it can obviously be tied into a system with either of those kinds of things. Complement it, provide an AI system with the ability to see and navigate the world or make decisions, come to conclusions based on information in its surroundings.

Cool. And then the very last thing, so I always end the episode asking the guests for a book recommendation, and you are still going to have to do that. But before you do that, I'm also going to mention that with a number of other co-authors, it's about 10 of you, I didn't count the exact number, you co-authored a book published by PACT in 2024.

called "Building AI-Intensive Python Applications." And that's a MongoDB book, effectively. Yes. So, MongoDB, when I joined this company, there were so many amazing people residing within MongoDB that are very creative, they're very intelligent and eager to learn.

and also eager to contribute just to the general intelligence of this space. So that's why you see that there is a lot of authors in that book, because everyone is so eager to contribute in a space that moves very fast.

There's a lot of learners here. We're in the trenches. We're in the arena. We're learning from our customers. We're helping our customers. And we brought that learning into a book that we feel resonates with a lot of developers today. And that's what we're doing. So we build learning materials that

Hopefully it's done the test of time, maybe two years, three years. Nice, yeah. And so, yeah, so I guess if people want to be learning more about the kinds of things that we were talking about in today's episode, they've obviously got the Deep Learning AI course on Gen AI. We'll have a link to that.

But if you would prefer to be doing learning from a book, I'll also have a link in the show notes to your Building AI Intensive Python Applications book. Excellent. All right, so then that brings us, Richmond, to my penultimate question, which is, do you have a book recommendation for us? Yes, can I do two? You can. Okay, excellent. So one, there's a book by Seth Godin called This Is Strategy.

Caveat, PS, it is not strategy. The book is actually not strategy. The book is actually a bunch of riffs from Seth Godin. He's a very popular marketing person. So it's a bunch of riffs, really. You can see it's a bunch of long-form tweets that is placed as a book. And some people might not like that form of structure, but he talks a lot about

formulating and building strategy within marketing and sales environment. And it's very relevant. I'm within a marketing organization within MongoDB, but I think developers should also read those books as well. And it just contains a bunch of just gems. It's not really structured, so people hate that, but I love it.

Yeah, so that's my first book recommendation. This is Strategy by Seth Godin. The second one is Crucial Conversation. And I think a lot of that's a popular book. It got recommended to me a few months ago because one thing I found myself was within my career was

Having a lot of conversation with different types of people. MongoDB is a massive company. We have over 5,000 intelligent people working here. And the AI space is growing. It was just...

A few years ago, it was just, it felt so small, like a close-knit community. Now, everyone's in this space, but people like yourself and other influencers and experts within the space, we still have that small group, but the AI space is growing. I communicate with different forms of people for different purposes. Crucial conversation allows me to speak in a way that gets me what I want. And that sounds very simple, but...

It's a very hard thing to do where you can have a conversation with someone and actually communicate your objective and not get lost in emotions or in a moment and make sure you come up with your desired outcome

Why not making enemies as well? And doing this in the most effective times, in the most crucial times. So that book really helped me with that. And I've only read about half of it. What was the book again? Crucial Conversations. Crucial Conversations. Nice. Very cool. I think it's a very popular book. And this strategy is, I think, also a fairly popular book as well. Yeah, yeah, yeah, yeah.

Nice. All right, Richmond. I was going to ask, do you have any book recommendation for me? That's so interesting. Actually, I'm currently on the flight yesterday. I was reading a great book from, do you know Cole Neusbaum or Naflik? Have you ever heard of her? No. She wrote a book years ago, maybe a decade ago, called Storytelling with Data.

Yeah, I've seen... It's hugely popular. It's crazy. Like, I don't know how many copies she's sold, but based on the number of Amazon reviews, it's like thousands and thousands or tens of thousands, I can't remember, of Amazon reviews. And so it seems safe to say that there's been hundreds of thousands of copies of that book sold. Now, I actually haven't read that book. I'm sure it's a great book. Lots of people have read it. But...

At the time of recording, I'm in the midst of preparing for a big opening keynote. You and I were talking about this. I don't know if it will have passed now at the time of recording this episode, at the time of publishing this episode, but on March 19th, I'm doing this opening keynote at the RVA Tech Data and AI Summit in Richmond, Virginia. And I've done keynotes before.

But I had Cole Neusbaumer, Naflik on the show. I don't know the episode number off the top of my head, but maybe a year ago or two years ago, as she was announcing the release of her latest book called Storytelling With You.

And so it's a guide to effective presentation and it kind of reminds me of this Crucial Conversations, like part of that is a big part of this book that she's, what she's talking about in this book is how you can use a presentation to switch people's way of thinking to your way of thinking. And so I

Yeah, I've been really engrossed by this book so far. She's an amazing communicator, and so of course it's a great book. She's this huge multi-selling, multi-best-selling author, and so unsurprisingly, it's a really tight book, really well-written, and she makes great use of story within this Storytelling With You book as well. So I'm enjoying it a lot so far. I still haven't finished it, but it's been...

Great, it's been perfect. If you're listening to the show, or you specifically, Richard, meant to ask for this recommendation, so now you're getting it. If you're preparing for a talk, I'd kind of already, you know, I'd sketched down a couple of pages of ideas of things that I wanted to do, and I'm a month out from doing the talk at the time of recording, and I'm like,

And so starting on this book is perfect because it has tons of exercises. And so I'm writing lots of things down as I go with, okay, that's helpful for defining my audience and for developing the materials. And I'm really excited. I think it's going to allow me to have my best talk ever. Nice. One, I'm not going to be in Richmond, Virginia, but I'm hoping the talk is recorded and I get to see it. One thing I wanted to mention was

Soft skills are very crucial within AI or within any tech role. Most developers should look beyond the technical capabilities and look at soft skills as well. And I actually wrote an article a few years ago, maybe four years ago or three years ago, on NVIDIA called Data Scientists Should Learn Storytelling. Yeah, I think that's what I phrased the title, but it talks about basically storytelling with data.

It's a short article, maybe 2,000, 3,000 words, with a bunch of imagery, and it talks about the same thing, right? Using data to convey ideas to the right audience. And I think I provided a framework as well as to how people should approach it. But that is all to say it was a relevant topic then, or she wrote a book 10 years ago. So I think that is something, again, that is a pillar, right?

that you can base your career around, which is effective communication, right? For sure. I used to, when I first started hosting this show five years ago now, four years ago, a very common question that I would ask guests was, what do you look for most in people that you're interviewing? Because I figured a lot of our audience would be interested in hearing what are the key skills that people are looking for. And communication was the first thing people were looking for probably most of the time.

Exactly. And even talking to LLMs, communication is a big thing. Prompt engineering. It's all about communications, man. Yeah, we had a funny episode recently with Varun Godbole. It was published, I don't know, I don't have in front of me. One of the interesting things about doing an episode live like this is I don't want to just be sitting behind my computer

But usually when I'm recording remotely, I have my computer right there so I can look up. This is the episode number. It's the same thing. Cold News Boundary, Netflix episode. I don't have the episode number memorized. Varun Godbole, same thing. And it was a really funny thing. Varun Godbole spent 10 years as a software developer at Google. And in the final years, he was a core developer on Gemini. So, you know, he's an LLM engineer. And...

And he had this really funny moment in the episode. I think it's one of the funniest maybe things that's ever happened on the podcast was he was talking about how he became a better prompt engineer as a result of going to relationship therapy with his partner.

And I gave him a hard time because I was like, that is such a Google developer thing that the thing, the main outcome from your relationship therapy was you became a better developer. Better at talking to machines. Oh, wow. That's actually good. I feel that was very good. Love that. Nice. All right. So final question for you, Richmond. How can people follow you after the show?

Yes. LinkedIn is the platform of choice for me. You can follow my LinkedIn. Also, I publish a lot of articles within MongoDB. We do a lot of articles here as well. So there will be a link to my profile within MongoDB.

Yeah, LinkedIn is the most popular way to follow me. And yeah, and also I do really encourage some developers and developers listening to try out MongoDB for their AI application. And there'll be a link below where you can actually get a free MongoDB Atlas cluster and try it out. Bring your vector data in, bring your metadata in, and let's build a future together.

Nice. Great message there at the end, Rishman. Well said. You're such a great speaker. It was awesome having you on the show and to meet you here in person. Yes. Thank you so much. And yeah, maybe in a couple of years we can check in again and see how you're doing. Thanks for having me.

Nice, what an excellent episode with Rich Manolache. In it, he covered MongoDB's Document Data Model, which uses a JSON-like structure, making it ideal for AI applications because it offers a flexible schema during experimentation, while allowing the schema to be locked in during production. He also talked about vector search capabilities and how they're now built directly into MongoDB, enabling hybrid search combining both lexical and vector search in a single database.

He talked about how 2025 is the beginning of the multi-era that encompasses multi-agent architectures becoming mainstream, multi-step problem-solving models, multi-modal embeddings handling text, images, video, and audio, multilingual capabilities, and multiple retrieval mechanisms beyond just vector search.

He also provided the ARENA framework for AI strategy, which involves building on solid pillars with objective truths, following the ACRED principles: aggressive, centralized, resourceful, efficient, and data-driven, and testing tactics in the ARENA of the market. He also talked about how memory management is becoming increasingly critical for AI systems, requiring sophisticated approaches to storing and retrieving information efficiently.

As always, you can get all the show notes, including the transcript for this episode, the video recording, any materials mentioned on the show, the URLs for Richmond's social media profiles, as well as my own at superdatascience.com slash 871.

And I can't believe it's finally here. If you'd like to connect in real life as opposed to online tomorrow on March 19th, I'll be giving the opening keynote at the RVA Tech Data and AI Summit in Richmond, Virginia. There's a ton of great speakers. So check it out. You don't have much time to get involved. So if you live in the Richmond area, that will be tomorrow. The RVA Tech Data and AI Summit. It'd be awesome to meet you there.

Thanks, of course, to everyone on the Super Data Science podcast team, our podcast manager, Sonia Breivich, media editor, Mario Pombo, partnerships manager, Natalie Scheisky, researcher, Serge Massis, our writer, Dr. Zahra Karshey, and our founder, Kirill Aramenko. Thanks to all of them for producing another fun and informative episode for us today, for enabling that super team to create this free podcast for you. We are deeply grateful to our sponsors. You can support this show by checking out our sponsors links, which you can find in the show notes.

And if you'd like to sponsor the show, you can get the details on how to do that at johncron.com slash podcast. Otherwise, share the episode with people who'd like it. Review the episode on your favorite podcasting platform. Subscribe. Feel free to edit our videos into shorts to your heart's content. But most importantly, just keep on tuning in. I'm so grateful to have you listening and hope I can continue to make episodes you love for years and years to come. Until next time, keep on rocking it out there. And I'm looking forward to enjoying another round of the Super Data Science Podcast with you very soon.