App Intents
55:52 Share

What's up, folks? Welcome to another episode of the Compile Swift Podcast. This week, we are diving into some very Swifty-related things. We are going to be talking about app intents. But first of all, Mr. Jeff, how are you doing? Yeah, just been focusing on trying to get a major feature release out for Bark and I

The feature that I'm working on right now is Apple Wallet Export. And so one thing that that requires is actually doing a little bit of both front-end work and back-end work. And so I've been doing some Swift on the server with Hummingbird and building that out and having that all deployed. And so now I've got all of the back-end work done. I've got it.

Mostly more or less hooked up to my front end. And now I need to do the unfortunate part that everybody hates me for of putting it behind a paywall. So, you know, not letting people have that feature for free, unfortunately. Yeah.

We should definitely talk about maybe Hummingbird and that sometime. Folks, you may remember, I'll put a link in the show notes, our good friend Yannis, I did an episode back in those days, I interviewed him, and we should definitely talk about Hummingbird. But I am curious, how are you going to put that into production? Presumably on some kind of Linux box or something?

I have a single VPS hosted by a service called Hetzner that I run, and I basically just have it in a Docker container, and I push that up and have it running on my server. Cool. You know, actually...

I was thinking about this the other day and I should ask you, is there a good Docker container that we can just spin up that has, you know, Swift on the server and all of that stuff set up, ready to run? There is some great Docker containers for doing Swift development. This was actually one of my first viral videos ever was me posting about how to do Swift on a non-Mac device.

uh, system by taking advantage of Docker containers and GitHub code spaces. And so there are some really nice Docker images for doing Swift development and also for deploying Swift code. They have a little one of each. And so, yeah, it's very easy to get started using Swift, not really able to use all of Apple's frameworks. You can use Swift, you can use foundation, you can use the built-in stuff, but, uh,

Being able to use that on Linux or really from any machine because you can run Docker and you can run that as a virtual image and run Linux there. So yeah, being able to do Linux development inside of these Docker images. Yeah, the Swift language development.

whatever you want to call them, they do a good job of providing some nice Swift Docker images. Cool. We should talk about that sometime because, you know, I've played around with them from a development perspective, but I've always wondered and never looked into it because I've never needed to. But, you know, oh, is there, you know, what's a nice, good, secure, more than anything, production one. So maybe we'll do an episode on that sometime as well.

Yeah. So this week, I think I have finally wrapped up the update for Job Finder Tracker with the initial update.

resume features and a bunch of other updates I had a weird bug that you know when you had those bugs that you say to yourself is it me or is this a genuine bug and you can never configure it out so I've pushed it to test flight and it was a weird it's that weird scenario where you're using cloud kit and that and you wonder like okay is it taking a couple of seconds to pull down the data or

Or is something really wrong? So, you know, maybe we'll talk about that sometime. But I'm hoping to get that out this week.

Maybe next week. Depends if I get a chance to make the videos. So we'll see. But either way, it's like, gosh, I wrote some code. I feel proud of myself. All right. Yeah. All right. Let's get into the meat of this then. So I'll give a quick rundown of what app intents are from a high level. And then I know you've actually done some stuff.

dare I say, production work with App Intents. So, you know, it's definitely going to be interesting there. But for those who don't know, I would imagine most developers have at least heard of them, right? Because Apple likes to talk about them. But App Intents. So what an App Intent is, in a very broad way, is it gives you the ability to, for users in different ways, and I'll list a few in a moment,

to get into your applications or pull data out, basically interact with your application without necessarily having to open the application every time. So for example,

If you want to use App Intents, you would use them for things like, you'll see them on Spotlight Access. Widgets is probably a very popular one. Siri is another one I am sure everybody has encountered at this point as far as a user. Control Center. The one that I'm perhaps most interested in is the shortcut actions.

because of all the possibilities that opens up for automated workflows between your apps and other apps and all of those kind of things. And along with that, of course, if you really want to and you have a need, you've got the good old action button that, as you said before we started, is really the run a shortcut feature.

kind of button for the most part. And we're not talking about the new one, right? We're not talking about the camera control one. We're talking about the button... Not the camera control one. Yeah, not that one. Not yet. We're talking about the button that's been around for, gosh, what is it? Two or three versions at this point of the iPhone? At least two, right? Two. Two? It came out with the iPhone 15. Okay. All right. So, I mean, at a high level, that's what app intents are. They're a way to...

interact with your application, let other people interact with them. Like I say, widgets and Siri are probably the most well-known ones as far as you've used them and probably haven't realized what it is, but they're super useful. But, you know, Jeff, we're going to dive into the details of how you create them and so on. But, you know, go into a little more about how you've used them and why we think

This is, you know, this is kind of the thing going forward, right? Hey folks, if you like what you're hearing in this podcast and you want to help this podcast to continue going forward and having great guests and great conversations, I invite you to become a Patreon supporter. You can go to patreon.com forward slash compile swift, where you will get ad free versions of the podcast along with other content.

So I would say that app intents are basically a way of exposing a single atomic action that your app can do to the system. And in so doing that, you can kind of take these actions and combine them with other actions that your app does or other actions that other apps do or other actions, in fact, that the system does and use them to kind of tie a bunch of things together and

And, yeah, I had originally started out working with the most obvious, most visible form of app intents, which is, of course, using them in the Shortcuts app. And what the...

Biggest benefit and also the biggest curse of doing app intense with shortcuts is that, of course, shortcuts is very much a huge power user feature. And so there's not a lot of benefit to focusing on shortcuts as your app intense target because shortcuts

that's something that really is only going to be used by a very small portion of your users. However, as I've explored more into App Intents and as I've explored more into what they're capable of, it turns out that there are a lot of... App Intents is kind of, at least in my mind, and I think in a lot of developers' minds, tied to shortcuts. And shortcuts is very much a pro-user feature. But it turns out that while...

App Intents being designed for shortcuts is kind of a way to think about it. The actions can actually be exposed in ways that are way more obvious and way more presented to the user. And

it makes sense to focus on building app intents because those other more exposed ones are there. And so for that, that is stuff like app intents. That is stuff like the new custom controls in iOS 18. That is stuff like focus filters. That one's kind of more of a power user feature. Um,

Or live widgets. The shortcut, the spot, sorry, the spotlight actions, the live widgets. Yeah. All of these things. And so while you as a developer probably want to think about it in terms of what it could be as a single item in shortcuts, because that is the right mental model for designing these atomic actions. It turns out that each of these actions is more usable by the users than

in ways outside of shortcuts. I think maybe the definition here of like a really good use of an app intent is when the user doesn't realize they're using one, right? You know, it meaning that, yes, you know, I ask Siri something and Siri for once actually accurately replies because I've got an app intent that Siri recognizes, right?

That is, yeah, as we're recording today, Apple released the release candidate for iOS 18.1, which is the one that's really going to start to be able to take advantage of some of the Apple intelligence app intents and that work that is being done there. But for years now, two or three at least, you've had the ability to expose your app intents to Spotlight. So when the user is searching for something in their app,

You might have some kind of action that your app can take appear in the search list, even if the user didn't look for that app in particular. And so that's kind of what I'm getting at with this idea of like design for shortcuts because that's the right mental model, but then use that to.

Kind of get a foothold in these other areas that users are more likely to run across. For example, in my app Bark, I've got a intent that just shows you a given barcode. Type in the name of the barcode and it passes that barcode to you.

Now I'm not taking my own advice yet. That's because I had a tight deadline on releasing bark, but yeah, so I have this shortcut action in there, but what would make sense is what if I have that exposed as a spotlight action as well? And all I have to do is really just kind of take that existing app intent and make it a spotlight action. And suddenly you can now search for your barcodes in spotlight, never launch my app, never have to see my app, just stop.

go into Spotlight, search for your barcode name, tap it, boom, there's an image of your barcode. That is something that I think would be really useful to users in a way that, okay, well, you can bring up a barcode as part of a shortcut, maybe is not as useful to the vast majority of users. That is a good example right there because let's say, for example, I've got the barcode for some concert ticket, right?

you know, yes, I could go into the app and get it and da-da-da-da-da, but I've got a couple of options via the intent here. You're enabling me to either, you know, I can put it as a widget on my home screen. I get to the thing. I'm just going to tap it. Boom, there you go. Or, you know, if I want to risk talking to Siri in public, I don't know why anybody would want to do that, but you can ask Siri and those kind of things. But the idea being that Siri

the intent is going to get me right down to that one specific thing that I want from the application. Much like, I mean, this is perhaps a bad example, but the weather widget, right? My location. Well, you know, it's probably, I don't know, but in the background, maybe sending in my zip code or my GPS coordinates, pulling that back out via, you know, an app intent and getting the weather. That kind of thing is what we're talking about, right? Yep.

So, yeah, like I said, the most well-known thing that app intents are used for, well, at least until 18.1 and Apple Intelligence, is shortcuts. But I think I've got a partial list of a lot of the other things that it can be used for. And maybe I know you've also done your own independent research. Maybe you have some that I've missed. All right. The ones that I know of are...

Being able to configure a widget. So when you long press a widget and you have the background, you can like change some things about the widget. So like with the weather one, it might be what city are you looking at? Yep. That's all handled by App Intents. For the interactive widgets that we got last year, two years ago, I don't remember. That's all handled by App Intents where when you tap on something in a widget and it takes an action, that's an App Intent.

You have the new custom control center items that we got this year. When you tap on those, those do something based on an app intent. Keeping along the same line, you also have interactive live activities. When you've got a live activity on your screen, you want it to do something. That's all app intents.

We have the one that we've been discussing for a while now, the spotlight actions. That is the ability to see something that is an action or in some cases just an actual item in your app that is an app intent. Focus filters. So when you have the...

Something in your app that changes depending on a user's currently selected focus, whether it's like their bedtime focus or their driving focus or work or personal or whatever. That's a good one. Exposing those preferences to users is all an app intent. It's very similar to the configuring a widget app intent.

And then the ability to interact with some of Apple's hardware stuff. We mentioned this earlier. This is mostly just running shortcuts, but you can obviously have just a single action shortcut that is with the action button. And one that I think a lot of people miss is the squeeze function on the Apple Pencil Pro. Oh, good point. You can actually tie that directly to a shortcut.

and have it do different things. So that's my list of things that app intents can do that are not shortcuts. No, you've got it. Did I miss anything? No, you've got it. You've got a couple on there that I missed. Yeah. And now I actually want to call this episode Intent to Squeeze. Just...

See where that takes us, you know. No, actually, I've completely forgot about the pencil one. Yeah, I think a lot of people did. Yeah, that's cool. Did you use that one when you... I have not, no. The black highlighter Apple Pencil integration is entirely just handling the Apple Pencil squeeze interaction directly. It's not using Apple Pencil. Gotcha. Okay. Okay.

Now, clear something up for me here, because in my mind, I'm thinking that an app intent is really also a replacement for, you know, some things like, well, indirectly deep links, right? Not to imply that it is, but it's kind of like an executable deep link in some ways.

Is that fair?

And he has a article called Action-Centered Design that isn't really about app intents, but it is kind of about app intents, which is saying that rather than designing your app as a set of UI, rather design your app as a set of actions that a user could take and then build the UI over those actions. What does the user interface for performing this thing look like?

And once you have that set of actions, it's a set of like nouns and verbs. And once you have that set of actions, those map very neatly to app intents. And so all of the nouns and verbs that your app can do or use, build an app intent for those.

All right here it is the one thing that I cannot do without every day and that is my coffee. Anyone that knows me or anyone that's listened to it in my podcasts or anything else knows that I absolutely cannot operate without my coffee and I love good coffee. So here's the deal I'm going to give you one free bag of coffee by going to peterwidom.com forward slash coffee and

There is a wonderful company out there that follows the fair trade practices, helps out a lot of independent roasters of all sizes, and the operation is simple. What you do is, you're going to go to peterwhitam.com forward slash coffee, you sign up there, you get a free bag of coffee sent to you, yes, in return. They say thank you to me by giving me some coffee, but that's not the reason I'm doing this. The reason I'm doing this is because I have found...

So many good coffees that I just would never have come across, heard about, or experienced without this service. Trade Coffee is just fantastic. You know, there are plenty of places out there. We all know them that supply coffee, good coffee.

You can go to the store, get the coffee, but there is nothing better than discovering new independent roasters and supporting them, discovering new flavors of coffee, new grinds for you can set it up. It's very smart. You tell it the kind of coffee you like. And over time, it gets better and better as it trains in on your selections and your choices and gives you exactly the coffee you're looking for and recommending new ones that will be very similar to

Every time I get a new packet of coffee, I go through and afterwards I try the coffee. I go through the service and I say, look, I loved this coffee. I thought this coffee was okay. Or I say, look, this was really not for me. And every time I do that, it makes the service a little more accurate on the next selection for me.

So again, just go to peterwhitam.com forward slash coffee. Get your free bag of coffee today. If you're a coffee lover, you're going to really appreciate this service. I have been using it for years at this point and thoroughly recommend it. So if I'm thinking about my app Bark, for example, I can...

add a new barcode. I can scan a picture and get barcodes out of it. I can open a barcode in the app. I can add a location to a barcode. I can add a time to a barcode. I can export something to Apple Wallet. These are all of the different individual actions my app might take. And I should probably have an app intent for every single one of them. And then based on that, be able to

say what are the other use cases for those app intents. And so for example, adding a location, that might be something that's really only useful in shortcuts, but show me a barcode that's useful in shortcuts. That's useful in spotlight search. That's useful as a live activity button. That's useful as a lot of things. And so what can I do?

Once I have all of these actions listed, like what are the various ways that somebody might want to interact with them? And I think that it makes a lot of sense to have something like that.

uh, be the basis for how you design when you use app intents. Okay. Yeah. I, I'm going to go read that article myself because as I was watching one of the Apple developer videos today to remind myself how all this stuff works, you know, Apple very much talks about it in that context, you know, verbs and so on. And afterwards I'm sitting here and I was thinking to myself, okay, my job find a tracker app. What app intents would I want to have?

And I started scribbling down a list. And very quickly, like you said, you realize, oh, some of these make sense for widgets. Some of them make sense for opening up interactions for potentially other developers. You know, things like the share sheet and all these kind of things as well. And I think what I realized was,

it's all too easy to get carried away, right? You don't need to offer like the essential, essentially a visual API for everything in the app. You just need to look at them and go, what are the key things someone would want to essentially do quickly without necessarily diving into an app for, right? And that's how, once I started thinking about that, I found my list got an awful lot shorter to what are the things I actually would want to do, right?

And what are the things I don't care about? I think I disagree. Okay. Cool. In this case. And kind of to back up with the other article there that we linked to, there's also – this is an older WWDC video. I don't remember. I'll have to go find it and what year it was in. Okay. Apple engineer Michael Gorbach.

and I have no idea if I pronounced that name right, so sorry, Michael. Talking about like really, honestly, you really should have every single action that your app can do exposed as an app intent and being able to

effectively allow users to remix your app makes it so that there is way more of an ability for users to interact with your app. And so that was kind of getting back to what I was talking about with Bark in the first place is like, I shipped a very limited number of shortcuts for my first version of Bark and

But I do know that in a future release, I have like 10 more that I want to do because I want every tiny piece of the app available in a way that users can kind of quickly do something. Whatever individual thing they might want to do in the app, they are able to do it without having to launch the app. I think actually I've been using Bark as the example because that's my new hotness. And so it's what's most front of mind.

I've got a user for Black Highlighter who is very excited about the different things that Black Highlighter could do and all of the ways that I could expose the functionality of Black Highlighter through App Intents. And being able to quickly say like, oh, go grab the latest screenshot I just took and erase every single phone number in it and then...

a certain color and then save it. And then now I've taken this image and now I'm going to put it into the rest of my shortcut, which just like post it to Twitter, for example. And yeah,

What I have in Black Highlighter in shortcuts, especially before I started interacting with this user, was a lot more limited. And they've been sending me ideas and just like the number of things that it's kind of exposed to me like, oh, yeah, no. Like if I get way more granular and way more thorough in what Black Highlighter is doing, it makes the app way more powerful in a way that I don't really have to...

complicate my user interface for and so it allows for all of these individual actions all of these individual things that a lot of them are in fact already in the app but you know take a bunch of taps or take a bunch of of uh steps to do and if i expose them as app intense in a very granular fashion users are basically able to build their own interface for it

And that is a very powerful way. And yeah, this is shortcuts. This is very much a power user feature, but yeah,

You can imagine the same sort of thing happening with Spotlight, for example. Like, oh, I need to do X thing very quickly. Let me just type in the action that I would be performing. And now it's available. It's done for me and I don't have to think about it. And I think that's a very good way for users to be able to interact with your app without having to interact with your app.

And kind of just have people realizing how important your app is to them without having to go through the hassle of like, oh man, I had to like open this app and I've got to like wait for it to do this thing and I got to get to this page and then I got to go do that. Being able to instead just have every single thing that your app could do more exposed to the system.

gives them the benefit of your app without you having to build out special features explicitly. So it is interesting. I had never thought about it from the perspective of empowering a user to remix my app and essentially chain together app intents to perform some series of actions that

and use the app in a way that I'd never envisioned or, you know, the end result, something like that. The reason I thought, though, that it was a bad idea to expose too many things, and maybe this is just my experience and I've been using the wrong apps, I don't know, is I hate it when I go to, like, set up some widgets and I'll select an app and I'm just inundated with a thousand variations of the widgets.

And it's like, look, I know what I want. I don't want this endless scrolling list. But now that you've said it,

I do like the idea, especially if I twist the context and say, okay, if I was a power user and I was using shortcuts, I can essentially chain together a whole series of actions to make an app do something that the creator never dreamed of, but has empowered me to do it. So I'll give you that one. Yeah. All right. And I think that there is something even in that, that kind of is more like what is the power of app intents versus what are the power of widgets and

You can imagine a world in which you're talking about an issue where the developer has provided a bunch of different widgets. What if they provided a smaller number of widgets that because of App Intents were far more customizable? I think that would have made a better experience where you are using App Intents to take a more limited user interface and

and make it more customizable as opposed to saying, well, we're going to think of the top 10 things that a user might want and display 10 different widgets as opposed to having a single widget that is customizable in 30 different ways. Yeah. Yeah. And in fact, you know, shout out to cricket in the chat room here who points out, you know, we can spend a lot of time planning and building these interfaces and

But as things like AI and that come along and you have this more, hopefully, natural conversation with your devices that then take these things and perform actions, the idea of, okay, you know, I should provide you a UI is

But let's have this smart AI in the background use my intents to take what you're asking me to do and run it for you, essentially with no UI at that point, right? No, but that's a good point. Yeah. In that same case, you know, kind of like going back to the widget example or going back to just in line with what Cricket is saying is...

Having all of this in your actual UI means that you've made your UI more complicated for everyone. Whereas giving people building blocks to build their own UI means that you're able to build a much simpler UI for specifically the thing that you want to be able to do. And also just struck me too, you don't have to worry about the UI across platforms.

So I can just as easily talk to my watch as I can talk to an app on a desktop, a phone, a tablet, and so on. So that's a real valid point there.

Okay, so let's now dive into some code here and give folks a way to get started with this and some ideas on how to use it. Of course, we will put links in the show notes for everything. Thoroughly recommend, as always, go over to the portal, watch the videos because they're very informative and actually you will get started with this really quickly. Quite surprised how quickly it happens.

Okay, so I'm going to give you a very simple example here based on the videos. This is one for my app. All this does is create an app intent, expose it to the system, and you tap on it, it opens your app, right? Yeah, okay, no great joy there, but this is the simplest form of an app intent I could find. So, you know, you're going to open up a new Swift file, right? You're going to create a struct, give it some name. In my case, I called it add job.

It's of type app intent. There are a few different types depending on what your goal is, and we'll get into some of those. But that gives me an empty struct. In there, I've just got a very simple static let title with a localized string so that you can see it in the system, right? It's essentially a title is what it is, right? So you know what you're looking at as the user is going to use it.

and then I've got a function to perform. It's an async throws with some intent result. In there, all I do is I'm returning that result and it's got one extra line for some navigation within my app.

Then at the end of it, in the struct, I've just got a static let openAppWhenRun set to true because I'm not passing in any configuration and things like that. It's like just run the app. Now, worth noting, if you was doing something extra passing in data or something along those lines, you wouldn't need to put that line because obviously it's going to open your app.

But that is all you need to do to get a very basic working app intent that's exposed to the system. What I like about that is Apple has taken all of the pain out of it, right? None of this having to register in my configuration for my app to say, you know, hey, expose this to the system or here is a list of the things. You just create this struct.

that's it it's done so uh that's that's my example of a very basic struct there but jeff you want to you want to dive in and make it more interesting

Time for a break. Hey, everybody. It's Peter Whittem here from the CompileSuite podcast. I want to tell you about Setapp. Setapp is a service that provides a subscription fee of just $10 a month, and you get access to over 200 Mac applications, and it's also available now on iOS as part of that deal.

I use the service because it just has a ton of really good first-rate apps that I use all the time. For me, it's invaluable as a developer to have access to tools for things like APIs, for planning projects, writing emails, writing documentation, and you get all of these things, including database apps, all of that kind of stuff right there on the Setapps service for just $10 a month. You can use as many or as few applications as you need.

If you're interested in checking this out, go to peterwhitam.com, P-E-T-E-R-W-I-T-H-A-M.com forward slash set app, S-E-T-A-P-P. And you can see the details there. And it's got a link that you can go over and start using the service and see how it works out for you. I strongly recommend this to every Mac user. Break time over.

So yeah, the basic app intent, as you described, is just a Swift struct with a certain protocol and a few default parameters. Now, what you can do is you can also add additional properties with property wrappers on them that Apple provides. And those allow you to add additional configuration to your intent.

To go back a little bit, I talked about adding parameters. Once you add parameters, you can also add a parameter summary, which is basically the UI that is exposed to shortcuts. It's the sentence that you see in shortcuts. And so, like, for example, I have my open barcode one here.

What you provide is actually a string that has references to those parameters as key paths in the parameter summary. And so that is what gives you the UI that you see in shortcuts where you have English text or whatever language text and

And then you have these little spots in that sentence and you can fill those spots in either by picking something directly or by referencing the result of an earlier shortcut is fine.

That is what fills that in. And so that is the parameter summary, this entire sentence that you have with references to slots that you might put in there. For example, you have the app parameter property wrapper, and that allows you to take in additional parameters from users. So if you're thinking in shortcuts, for example, you might want to pass in the name of a thing that you're dealing with, or you might want to pass in the size of an image that you want to export or something like that.

And you can take in a bunch of different values. These are your kind of standard property list values. You got your ints, your strings, your files, URLs, all of those kinds of things. And those are the default system ones. You can additionally expose...

data from within your app to the app intense framework as well. And that's called an app entity. And these are the nouns in the nouns and verbs I described earlier. So for example, in my app bark barcodes are a app entity and I need to expose what are all of the barcodes that I have in my app out as app entities. And that allows me then to say, okay, well,

have an app intent that gives me a list of all of my barcodes. Or if a user wants to have a parameter that opens a specific barcode, then shortcuts can show that list of barcodes to them where they tap open a given barcode. They can tap on that little spot where the barcode is and it'll give them the list of barcodes and then they can select which one they want it to open. Yeah.

Yeah, you can essentially, it's like querying the data in your app, right? Yeah, exactly. You do, it's in fact what is required to do an app intent, or sorry, is what is required to do an app entity is defining an app entity query. That is a type you implement that allows you to return all of the entities that your app is using.

Cricket has a question about can you still define a model? Yes, question mark. Can you define a model as an app entity if it's not stored on device? And if you do have an app intent that lists these, can you have a fetch inquiry? Yes, you can have an app entity that lists

does the querying and I believe it does also have say like network access so that you can you can tell it to like go and like oh I need to hit my server and go do that what is more complicated and this is the part where I don't get into like I don't really know how this works is

past a certain number of entities. And so you can think of, for example, like Netflix's catalog here. If Netflix was building out a shortcut and they're like, watch movie and you have every movie that could possibly in Netflix's catalog, that is a different system that you use to integrate with those. And I don't 100% know how that works. And so I think that that's where you get into this level of like,

App entity queries are meant to be more for limited sets of items that are specific to the user. So, for example, a better example here would be a weather app. You would include the entities that a user has probably previously searched for. You would not include every city in the world.

Um, that is a different system that you have to go through. And I don't know much about how that system works. And so, okay. Yeah. You can do, uh, stuff a little bit more, um,

And I think you can call out to a server at that part. I haven't done that specifically. So I'm pretty sure there's no sandbox. You just have a limited amount of time, basically. That's what I was going to say. But then the results that you return, you can do whatever you want to there. And it's a matter of doing a deep search or something that uses this other system. Yeah, because I was...

And I think, you know, because at first I wasn't sure that I understood. But if you're talking about can I have, can I run an app intent that makes some remote API call, waits for a response, comes back, processes, whatever that may be.

and then present it to the user as a selection or whatever you're going to do on the return of that app intent. I was going to comment, I got to believe there's probably, first of all, an acceptable user amount of time to wait. And then secondly, there may even be

you know, a predefined expiry time, right? And app intent, I would imagine Apple was not going to let you run it forever, right? So if you're bringing back a lot of data or a slow server, a bad connection, anything like that, I got to believe there's going to be an automatic cutout in there somewhere.

I think it's on the order of 10 seconds. Okay, that seems reasonable. It's reasonable enough to go hit a server and do something. It's not enough to do a ton of processing. Yeah, and as a user, would I wait 10 seconds for an app intent to do something? I don't know that I would.

Um, cause I'm impatient, you know, but, but I think if he was looking at doing something like that, I actually wonder maybe that's more of a stop and think about, you know, if that's a goal in the app, how you're doing it in the app, maybe. Right. Because yes, I'd be concerned about that user interaction level and time. Sorry to respond to this question. Uh, there is a limit. It's fairly generous. Um, and yeah, I,

I have not done it, but I'm 99.9% sure that you can make network calls. And basically anything that your app can do, you can do. You can say, you can mark your app intent as running in your app's

environment, I guess. And so it's basically launching your app and then you can do whatever your app normally does. And so it's not sandboxed in a way that, for example, a lot of extensions are where it's like, oh, you've got a very limited memory environment and you're not able to make network calls and you're not able to do whatever. App entities can basically do anything an app can do.

I wonder if there's, and yes, same thing. I've not looked into this, but I almost hope from a security standpoint, maybe there's a requirement to whitelist what you're going to be allowed to call from the app so that you don't want someone to try and use it maliciously or something like that, right? I don't know.

I mean, it's sandboxed like your normal app is sandboxed. And so... Right. And that's what I mean. Sorry, not just from the app intent, but your app in general. If your app's going to call out and you're going to expose that...

security practices definitely in play. Yeah. Just in case. And so I think it's the case, like, you know, if your user has not granted you location permissions within your app, then you're not going to have location permissions within your app. Right. Absolutely. Yeah. And so, yeah, that is like one of the things that, you know, like Bark currently doesn't

have photo permissions at all. Uh, that is a feature that will be coming down in the future. This part's not in the thing and just, I can talk about future plans. Uh,

But one thing that you can do right now if you really want to have access to your photo library is you can literally just have shortcuts reference your photo library. It goes and gets your photo. It passes that as just an already grabbed image to Bark. And then that is a way for you to use your photo library in Bark is just by having it – by having shortcuts handle it basically. Yeah.

Yeah. And then that way I have not, I haven't had to build out like, oh, I need to know, I need to have photo permissions access. I haven't had to build that out. I can kind of just like let shortcuts handle. Okay. Here's, it's got all of the photo permissions access and now it's going and grabbing that photo and it's just giving me that one photo to my app intent. Yeah. It comes back to, like we were saying earlier, right? The beauty of shortcuts and being able to essentially just chain together. Yeah.

useful items yeah absolutely yeah

So yeah, we talked about app entities. And so yeah, the way an app entity is defined, or sorry, the way an entity query is defined is it says you can look up entities by a given identifier. And so then you return any entities that match an identifier. And then you also have the ability to say suggested entities, which is like a small limited set of the entities that you think are most relevant to the user right now.

And so that's, that's the difference between the entity query system and this other system that I'm talking about that is related to core spotlight. I don't know. Yeah. That's the part that I really don't know the answer to on that one is, is how you do the like far more in depth exposing of, of a bunch of data. But the entity queries have, yeah, you can look up something by a specific identifier and you can give a set of suggested ones, but you,

can't really do uh you know like deep search type stuff right and and that's you know sort of alluding back to what i was saying earlier about you know overdoing this the the thing to re to remember here is just like you know when you're creating an api or something like that you know the the more of these you create they may well be useful but it's also the more that you have to manage

and hope that they don't break in a future system. And if they do, guess what? You got to go fix them all, right? But again, I think that's why it's one of those, you know, hopefully like everything at this point, folks think about these ahead of time and plan it out a bit, right? Don't just put them in there because, oh, I should put that in there. You know, what's the reason for it, right? You know, things like that.

Okay, so that's kind of a level one of app intents right there. And that probably satisfies a lot of use cases for a lot of developers out there and what they want to do and exposing their apps. But there's a whole other level that we can dive into here where you can customize them to be exactly, you know, meet the requirements that you want and work in the way that you feel is appropriate for your app.

So let's go ahead. You've definitely got more experience with that side of things than I do. So you want to go ahead and dive in there? Yeah. So once you have your basic app intent, you can adopt other protocols for specific use cases for your intents. And so we've talked about the various use cases for app intents here. We can dig into one of them. So adding customizability to a widget.

You take your existing app intent widget exactly as we talked about defining it. And you can now add a new protocol called widget configuration intent. And what that does is it kind of just takes all of the parameters, parameter summaries, the titles, everything that you've defined. And you just say, okay,

This could also be used to configure a widget. And so all of those parameters that you've provided, they now show up in a widget when you tap on the little thing and you go to like customize and then it shows your widgets name.

All of those parameters that you've already identified as your regular app entity, those are now in this widget configuration. Simply by adopting the widget configuration intent protocol, handling any additional requirements it has. And I think for widget configuration intent, there isn't anything additional. I'm looking at the docs right now and I don't see anything. So yeah, you have your basic app intent. You say...

It's now also a widget configuration intent. And then you assign that to your widget. When you define your widget, you can say like, oh, configured by blah, blah, blah. And then you pass in your app intent there. So you could have an app intent that is doing something in shortcuts and say, by the way, this is also a widget configuration intent. And then suddenly you've got the same code kind of handling two different cases. The one that I saw in the Apple video that they gave a quick example of, I think,

brought it home real quickly, which was, you know, and interestingly, because they make mention of it at first, you didn't see it. And it's like, oh, you got to hit the little disclosure triangle, right? And then it drops down in the shortcuts, for example, you know, essentially your options and you can click on it. And this one, for example, was essentially a dropdown list you could pick from, right? But the configuration side, correct me if I'm wrong, everything has to be like most of this stuff,

excluding external data that we briefly spoke about, has to be configured in the app when you ship it, right? Because the system's got to be able to read through and essentially know what your app is wanting to expose so that when you install the app, it configures all of that in the system, right?

Yeah, I mean, that's what the app entity queries that we were discussing earlier are. Yeah, okay. Is that you are telling the app, hey, here's how my data is exposed to the system and here's what might go in here. And so, for example, you can have one of your parameters in your widget configuration intent be your app entity. And you say, like, I need you to take...

in this type of item and the user needs to select an element in this list and then they you could have that be part of your widget for example so again we go back to the idea of whether widget

you can say that your app entity is city and in your widget configuration intent, it would have that app parameter, uh, property wrapper. And then you can have, for example, var city, colon city, uh,

And city is all defined by this app intent query, yada, yada, yada. And then when you get to your widget and you tap on it and you go to the edit, on the backside, you have like a city row. And then you tap on that and it populates out a menu and you can pick your city from that menu.

That would be how this would work. And so you can imagine that exact same entity being used in shortcuts where you have a sentence there that is like, show me the weather for city. And then you tap that city. It picks the...

item there and then what that does is it returns some text for example yeah uh that that i think was an example that they put not for the weather specifically but sort of you know in the videos they had an example of how how you would put it in there like that and i think the the great thing about that is once you see it you see them do it in the video and they show you the code you

you get it really quickly. It's like, oh, this, you know, like I was saying earlier, you know, originally I struggled with deep links and things like that. But the way that they get this across in the video, it just makes sense and infinitely less code to deal with and configuration unless you choose to make it complicated.

I mean, by all means, right? Over-engineer it if you want to, but not recommend it. Yeah. Design of App Intents kind of does lead it to not getting overcomplicated because you want it to be kind of these very minimal, very self-contained pieces of...

And so you want your app entities to really just kind of do one thing very quickly, get in, get out, that's it. If you want it to get more complicated, then what you're doing is you're chaining multiple intents together and

And you can even do that in App Intents directly. App Intents can call out to other App Intents, or you can leave it to the user and they're responsible for chaining them together in something like shortcuts. Yeah. I mean, my recommendation is, you know, like you mentioned at the very beginning, maybe provide the blocks to do that, but...

Save yourself the headache. Give the user the ability to do what they want. Number one, you don't have to try and pregress it unless there's a very specific feature in your app that would benefit from, oh, if I set these things up for the user, a bunch of configuration and stuff as well. Fine, go for it.

Right. But again, it's all maintenance for you down the road. Should you want to change something or Apple makes you change it? So, for example, let me describe some of the app intents that I have in Black Highlighter. I have one app intent that is, OK, give me an image. Give me some words to redact and I'll spit you out an image. And another...

and intent that I have is literally just open an image in black highlighter. Just give me an image, go launch the app with that image in there. Now, something that a user can do is they can actually just chain those two images together and say like, I want to open this image in black highlighter, but I have a default set of things that I want always redacted. So I'm going to go ahead and have it do the default things and then open the image. The user can go and they can build those in

on their own. However, one thing that I can do as the developer to kind of make this easier is I can actually go provide a hidden parameter, hidden parameters, a little, little, uh, a bit of a misnomer. It's just kind of behind a dropdown and have a checkbox there that is open one run. And what I can then do is me as the developer, I can check this open when run property and,

It's just a Boolean. If that's true, I can just call my open intent directly from my redact intent. And so I can say, go ahead, grab this image, do the redactions, and then launch the app. And the way that I've done that under the hood, I'm just calling the same open intent that the user could call directly. And it's just a way of providing convenience to them by going directly from redact.

scan, redact, open, as opposed to the user having to have the scan, redact, open intents as three separate items. Okay, that's interesting because that answers a question that I was wondering, which is, you know, what I was going to make the statement that at some point, if an app intent feels like it's getting overly interactive or complicated, that you should at that point say,

Okay, now I should pop open the app and you should really do this in the app. But what you're describing there is something I hadn't thought of, which is, hey, let me do the bit that, you know, had the ability to do the easy bit. And then if you want to do this extra complicated bit, we'll open the app and pass this off to it. Right. So you got the best of both worlds.

Yep. Yeah. And I think that is the benefit of like really decomposing these down to their smallest individual pieces allows you to a build your own intense out of other intense, but then also allow users to use those intense in ways that mix and match them in whatever way they want to. Yeah. Yeah. Got it. Yeah.

I guess it's kind of like that writing functions, right? Write a small function that does one thing. Don't do it the way I do it. All right, folks. So I feel like we've covered App Intense here. Certainly enough to get you started and interested. And you should definitely be interested because this, you know, with the AI from Apple and, you know, hopefully the improvements from that lady one day.

It's going to be a thing going forward, right? App Intents. So the sooner you get on board and at least become knowledgeable, the better. With that said, Jeff, where can they find you? Where can they find your apps with all of these working App Intents? As always, everything is at CuckooType.com. There you go. And as always, I'm at CompileSwift.com. That's it, folks. See you in the next episode.