AI for Low-Code for AI
07:58 Share

Ready to break down some research? This paper, AI for Low Code for AI, explores how visual programming and natural language processing can work together to build AI systems, even for those without a traditional coding background. I'm joined today by an expert in AI who is going to help us understand the ins and outs of this study, as well as their prototype tool called Low Coder. So could you start by telling us a little about how Low Coder works?

Lowcoder aims to bridge the gap between having a concept for an AI system and the technical know-how to implement it. It does this by offering two main interfaces. There's the visual side, Lowcoder VP, where you can drag and drop blocks representing different AI operators. Think of it like building with digital Legos to create a visual representation of your AI pipeline.

On the other side, there is the natural language interface, lowcoder.nl. This allows users to describe the next step in their pipeline and get suggestions for the corresponding code, which can then be viewed and edited as blocks in the visual editor.

These two sides of low-coder complement each other by addressing the limitations of either modality on its own. Interesting. A hybrid approach to low-code AI development. Let's talk about the natural language interface, low-coder NL. It's powered by AI, right? The paper mentions that the natural language interface uses a fine-tuned Code T5 model. Could you explain what that means? That's correct.

Low-Coder-Anal uses a technique called fine-tuning, which is a way to adapt a pre-existing AI model to a specific task. The CodaT5 model the researchers chose was originally trained on a huge amount of general programming code, which means it already understands a lot about code structure and syntax.

To make it work for low-code RNL, it was then further trained on a dataset of natural language queries and Sklearn operations, which the paper describes as being extracted from Kaggle notebooks.

This process helps the model learn to associate specific words and phrases with their corresponding code, so it can accurately suggest Sklearn operations when a user enters a natural language query. That's clever. There are a lot of different AI models out there. Everything from massive, large language models like GPT-3 to...

smaller, more specialized models. The paper discusses exploring various models such as different sizes of Code T5 and Code Gen. What did the researchers learn about selecting the right sized model for this kind of tool? The researchers discovered that selecting the right size model involved a trade-off between accuracy and efficiency.

Larger models like CodeT5-Large were more accurate, but required more time to generate suggestions, which isn't ideal for a low-code tool where fast feedback is important. Ultimately, they found that a mid-sized model, CodeT5-Base, struck a balance between accuracy and speed, making it a better fit for low coder. Why is efficiency so important for a low-code tool like low coder?

Well, imagine having to wait for several seconds every time you ask low-coder NL for a suggestion. That would really disrupt the flow of building and experimenting, wouldn't it?

Quick responses from the AI make the process much more interactive, which aligns well with the goal of a low-code tool. You mentioned that the paper discusses user studies to evaluate the effectiveness of low-coder, and specifically low-coder in L, which is the natural language interface. Could you tell us a bit about how the studies were designed and who participated in them? Absolutely.

As detailed in the paper, 20 participants from a tech company with a wide range of experiences with AI, from complete novices to seasoned practitioners, were recruited. The studies were split into two conditions: one where participants had access to the AI-powered natural language interface, and a keyword-only condition where they didn't.

In each condition, participants were given several tasks that involved building AI pipelines to process data and train a classifier to make predictions, and were observed to see how the two conditions impacted their workflow. The paper shows that low-code R&L was very helpful for discoverability,

Could you elaborate on that? Were there any differences between those with more AI experience and novices, particularly in their ability to discover less common or specialized operators? One of the big wins was in discoverability, especially for operators that participants weren't previously familiar with. In the version of the tool with the natural language interface, people were able to find these unknown operators in 75% of the tasks.

Compare that to just 32.5% when they had to rely on things like web searches or scrolling through a long list of options. This suggests that low-code RnL was particularly good at helping users translate their intent, deal with missing data, for example, into the correct technical operation, even if they didn't know the specific operator name.

Interestingly, this was true for both novices and more experienced participants, although novices seemed to benefit more. That's a good result. The paper indicates that LowCoderNL helped participants iteratively refine the pipelines they built. Can you talk about some examples of this? The paper highlights the iterative nature of machine learning and how LowCoder was designed to support that.

The natural language interface allowed participants to quickly discover and add new operators to their pipeline, while the visual interface made it easy to understand, modify, and experiment with different combinations. Additionally, the visual editor also provides immediate visual feedback on how the pipeline is transforming data at each step, along with scores and error messages.

Together, these features created a highly interactive experience and supported a workflow of exploration and refinement. Let's talk about that. The visual programming interface, LowCoder VP, how did the visual aspect of the tool complement the natural language interface and contribute to the overall user experience? The visual interface is the other half of LowCoder's hybrid approach.

The visual block metaphor provides a clear, unambiguous view of the pipeline. This can be especially helpful when the AI suggests an operator or set of hyperparameters that the user isn't familiar with, enabling them to make sense of the suggestions and tweak things as needed.

The visual editor also provides immediate visual feedback on how the pipeline is transforming data, which proved to be very helpful for participants. The paper also describes some challenges participants faced during the study.

Were there particular pain points for novices or did more experienced AI developers face their own set of challenges? One of the key challenges for participants, particularly novices, was not knowing what they wanted to achieve even before considering how to implement it in the tool, which really highlights the experimental nature of machine learning.

Some participants also mentioned that they had difficulty forming effective natural language queries in low-code R&L, noting that the model sometimes failed to suggest the correct operator when they used non-technical language. This was a good discussion on the key points of AI for low-code for AI. Thank you.