Microsoft AI, model routing, and the move toward portable workflows

Introduction to the episode and guest background

Mark Smith 0:05

The intelligence age is not coming. We're already living in it. I'm Mark Smith, and each week I speak with the people building, leading, and navigating a world being reshaped by artificial intelligence. We explore AI technology, the changing nature of work, and what it takes to build a meaningful career in the intelligence age. This is the intelligence age.

Microsoft's AI model selection and multi-provider approach

Mark Smith 0:25

Today I'm here with Joris from Microsoft, and uh he's been heavily in the Microsoft AI scene now for the last four years. So, Eurists, to get started, tell us a bit about what's been top of mind for you this last three months, I suppose. Because it's like we can't talk beyond that these days.

Joris de Gruyter 0:48

Yeah, three months is like two years or something. I don't know. Uh what's gosh, I mean, I think you and I have talked a lot about early. I think let's go a little bit further back though. Like I think early of the year was a big, big moment for most people. Like all the coding stuff, all the coding agents, you know, codecs and cloud code, all those things, GitHub Copilots. I think it it came a long, a long way. All of a sudden it was like a big deal. And then I think the progression from there into using those agents, if you will, for non-coding tasks, I think is was a big revelation for a lot of people, but it requires quite a bit of technical knowledge, at least, to do that. And then moving on to like co-work and uh you probably from a Microsoft perspective saw the the autopilot and the scouts announcements. So I think that is probably sort of a move of getting that productivity that developers have been seeing into the hands of just information workers. And I think that's sort of I feel like that transition is sort of in its infancy still, but it's it it's there, right? And I think those of us, I think you can do a lot already with cowork, including the cowork in and co-pilots, but I think you still need to be somewhat savvy to get that going. And I think that that progression, I think, is is the big thing that's happening right now, in my opinion. Have you used cowork much? On and off. Um it kind of depends. Like I I feel like from my my work perspective, you know, when I was a uh a developer and a and a PM at Microsoft, I felt like I had a lot more sort of emails and routine and going through bug triage, those types of things. Uh I feel like my days these days are a lot more scattered. Like it's it's hard to sort of define what my Um absolutely um so my day looks like. So then it becomes a bit more difficult to sort of have these sort of routines and things. Um, so I'm still sort of more using the um the coding agents to do ad hoc stuff. Like I basically I my my favorite tool is still playwright, the playwright MCP, and basically I'm doing stuff in the browser, and then at some point I'll like call in GitHub Copilot and be like, okay, look at my screen, and now let's let's get to work from here on out. So I'm more ad hoc at this point. So I've I've I did some cowork thing for a bit, but I I I'm doing less of it now. It kind of depends sort of what my my workload is like.

Mark Smith 3:22

See, I jumped into it heavily at the start before um Microsoft switch it to a consumption model. And then I just it said, hey, it stopped. And so I didn't do anything. And then about a week or so ago, I jumped back to back in and had a look at it, and I'm like, because I always thought it was just a reskin of anthropic, you know, cowork, which I think it was in the early days. But looking at it the other Um absolutely um so my day, it seems to me that Microsoft have built it as a Microsoft first-class product, and it's not just tied to anthropic anymore. I can see, because I can select uh, you know, the open AI models in there as well. So that tells me micro and and also just so many things are easier. Like it used to be where were my skills? Now I can clearly go in there and create a new skill and see exactly, you know, where it is. So it seems much uh easier. Oh, sorry, it seems easier to use now. And also one of the big pushbacks I'd heard in market was people were concerned with the anthropic side of things, is that all anthropic compute is not happening in Azure. It's all happening on AWS, and Microsoft is just piping it through. Although they have signed an interlock agreement that it cannot be used to train models, so you still get that protection that you get with M365, which is super important. But it seems to me that Microsoft's has gone beyond that now and allowing it much, you know, really to once again surface the models that Microsoft provide across the board rather than locked into a single provider. So I assume, particularly in the EU, that gives much better choice.

Joris de Gruyter 5:01

Yeah, yeah, like retention policies and training and that sort of thing. Foundaries, yeah. Yeah.

Mark Smith 5:07

Yeah. Interesting. Um are you using the the Maya models much yourself? May I?

Joris de Gruyter 5:15

Um I am personally not. I to be fair, like I said, at this point, at least my my main goo to go-to system is is GitHub Copilot. Interesting. And um big because like I said, my my work is partially technical. I use Playwright, I need my browser locally, so I need a system that can which with co-worker for cloud-based, which has advantages, but it has disadvantages as well when I'm doing stuff local. Um so to be fair with GitHub Copilot, I basically sort of stick with whatever default. I started a lot of new projects. I don't really have tons of long-running sessions. Most of what I do is just a few weeks at a time. Um, and then I'll just switch to whatever the default is. And and obviously, from a GitHub Copilot perspective, I think it's sort of a whatever default comes up is probably a mix of whatever's cheap to run, but also sort of latest and greatest, sort of that mix. And it's fine. Like I think I was a huge fan of Opus 4.6, but then you know, I feel like these models always sort of, oh, the new one comes out and it's slightly better at this, and then the next one is sort of go back and forth. So I feel like I'm not, especially on the frontier side, I'm not too concerned anymore about what model. I mean, they all sort of have their tendencies, I guess. Yeah. But all in all, um, it doesn't take too long to get used to. And if you're just focusing on very specific things, you don't have to worry too much about intricacies of of how each model does its thing. So so yeah, I don't have a favorite model right now, I would say. I feel like because they keep like, you know, progressing. There's always a newer one, and then this one's best. So it's like, okay, well, I'm trying to I'm trying to stay neutral and agnostic as much as possible.

Mark Smith 7:00

See, that's why I've gone with with OpenClaw across the board, because it allows me to keep all my data in context, but it leads allows me to flip in and flip out of any model at any time. Like, and and you know what this year has taught me is that it and at the start of the year, you know, anthropic was the flavor of the month. Everybody was like, Oh, I'm like hard out on anthropic and making big commitments to that ecosystem, and then you know, OpenAI released their uh updated codex version, and all of a sudden that's just cranking and and amazing. And so I'm I've really committed now to not storing anything in a single provider, but really allowing me to plug and play. And as I say, I use um I use Chinese models as well. Um, I'm testing everything and and you know, I'm I'll get to the point where tasks will be selected based uh sorry, the models will be selected based on the task or activity. But at the moment, it's definitely a case of I want to make sure my data layer is all mine, connected, etc. And then a model is used while it's fit for purpose and then switched out. Because I, as you say, I don't feel that there's too much improvement now happening between the models. They're they're all good enough, I suppose,

Skills, Markdown, and portable AI workflows

Mark Smith 8:28

is what I'm saying.

Joris de Gruyter 8:29

Yes, agreed. Sorry, I think your point about skills and it's I think that's that's really the key, right? That's the thing that is transferable between the different agents to different models. Yes. I mean, maybe the skills they'll behave slightly differently, but for the most part, if they're well written, I think that's that's really your your IP, if you will, right?

Mark Smith 8:48

Totally, totally. And it's funny, right? Because ultimately everything coming down to markdown files, which are highly transparent, highly readable. Um, but it's it's also portable, right? It allows you to move. And I was really impressed with Microsoft about five months ago, maybe now, maybe four months ago, they in SharePoint they introduced MD, Markdown, as a first-class citizen. And you can now, you've got a markdown editor within SharePoint by default, and a markdown reader. And I just like the way that surprised me from Microsoft, because you know, typically wanting to lock in um uh the word format in so many uh situations, but there's something about not having any um formatting, you know, as in uh slowing things down, just being uh data that's irrelevant as you get to a state where AI is is reading more than perhaps humans are reading, and you want to just give it that conciseness and the context without the formatting.

Joris de Gruyter 9:55

Yeah, I wish I wish uh OneNote could work with with Markdown. I I'm a huge OneNote nerd. It's one of those things where I mean, if you go back not that far ago, it's sort of the preferred note-taking app in the Microsoft ecosystem. I feel like just because it was easy, it was quick, it was like you could format, but you didn't have to worry too much about the formatting either. It was great. Um and so yeah, I I I agree. I'm having this issue myself with like, okay, I want more in Markdown, but I like also the fact that OneNote is sort of on my phone, on my computer, on my Linux machine, my Mac. I have, I mean, I'm a I have all of the OSs, um, and it just synchronizes, I don't have to worry about it. And and if you want to look look at a tool like Obsidian or whatever, which I think is a default for a lot of people, it doesn't really have a good syncing story out of the box, even if you get their subscription. And I feel like there's a gap. So whoever's listening to this, like if

Notes, SaaS bloat, and data cleanup

Joris de Gruyter 10:58

you're an entrepreneur, uh hit me up. I'll I'll I'll beta test your MD version of OneNote for sure.

Mark Smith 11:05

Yeah.

Joris de Gruyter 11:05

Yeah.

Mark Smith 11:06

You're so right. Like I've got Obsidian running on my desktop purely because I exported Evernote, which I'd had for since Um absolutely um so my day dot. And they are like, and this is another struggle I'm having, I suppose, with the traditional world I came from, in that everything was SaaS, right? And then what you've noticed over time is SaaS products continue to increase in cost, as in they keep putting the subscription up. But there's the value is so incremental and it's not value anymore. It's just them trying to extend beyond their purview of what was their solution into every other category. Like right now, I use Calendly for setting calendar appointments. And all of a sudden it starts telling me, hey, I can do the recording of those. And I'm like, no, no, you're a scheduling tool. Stick within your lane. I only want you to do that. I don't need you to now be something else. Just be the one thing, please. Um, and I found with Evernote, it just became bloated, slower in time, yet less usable. And the subscription cost just kept going up. So I just did this massive export. Obsidian was the only tool that I could see that I could do a port and make it readable. But right now I've got eight gigs of files sitting there in Obsidian, and half of it's just rubbish. And one of the things that I did for years, maybe almost 20 years now, I've been collecting all my data. I would back up my PST file and Outlook, I would, you know, I would, and I've got just gigs and gigs. And what I found is that because I was like, one Um absolutely um so my day AI will come along and I'll be able to make sense of it all. But what I realized, that data was important at that period of my life. I'm not there anymore, and I don't care about any of that now. Because the way I thought then is not the way I think now. And what I'm finding is that I'm using AI just to work out what needs to go to the trash. Like, let's just delete it. And I mean, I went into one of my Outlook accounts, my MVP Outlook account that I used. There are 33,000 emails in there. And once I had finished, there was no sent emails, there was no deleted emails, there was nothing. It was, I just cleaned them. Like, I don't need any of that. I don't need 10-year-old emails, you know, like why? And I just deleted the whole thing, started fresh because it's my main GitHub integration point in a what does I do want to know anything that's firing on GitHub, right? Because something could be flapping, something could be, you know, burning credit, and I want to know about that. And now, you know, taking that one account, integrating it also with AI, it now can pick up anything that's going wrong in GitHub for me and come up with a solution, a fix, you know, research fix, et cetera. And so, but I'm just finding that, yeah, it's interesting you say about um OneNote. Um because I just I I haven't used it for two, two to three years, and I used to use it uh solidly. I've got so much OneNote content. The other one then I got into for was um what was Microsoft's next product it bought out for markdown kind of files and stuff. Um I mean personally I use VS Code a lot, but uh they came up with loop, loop, loop, is what I'm thinking of. And like I really got into loop until I built a CLI integration to um between um between OpenClaw and um the Microsoft Graph. And it's my only uh what it's it's my only I think it's no, I've started a second open source project on on GitHub, but it's my my first public one. And it's been running for now nine months, and it does all my integration between it, so it's really robust. I use Go as the as the compiled, and which I regret it in a way, because it takes so much resources every time to compile it, but it's the most secure way of doing it, so that there's no risk of any leakage. And I thought if I'm integrating with Microsoft, I need to be the most secure possible for people to trust it. But what happened when I got to Loop, Loop has got no real API. It allows you to read it, but you cannot update or edit, which is exactly what I want to do via API. And so Loop is disappearing out of my world now as well, just because I feel if you don't have something with API access, what's the point?

Joris de Gruyter 15:54

Like Yeah, I I think um I think you and I are very similar in the sense that we're obviously we've been in the Microsoft ecosystem for a long time. I mean, I work there

Vendor lock-in and open standards

Joris de Gruyter 16:04

now, but uh I was just as long outside of Microsoft and consulting and stuff as well. Um but we're not necessarily tied to it, right? In the sense that I'm I'm not a huge fan of a vendor lock-in, um, which is which is a problem for anything with software, right? I I have an iPhone, but I have an M365 subscription, and my all of my my photos are getting synced to OneDrive. And the main reason for that is because I want them on my Windows PC, but also want them on the Mac, right? And sort of usually by process of emanation, you end up with just one service that's left that can actually somewhat reliably do it across everything, and that's a problem. And I think that's where we're going now as well with all these AI ecosystems, and I think that's where the MD file is interesting, right? I mean, skill files is very specific, but there's a lot of just pure data that you want to store, notes, all that kind of stuff. And I said earlier with OneNote, I want it on my phone because I'll take a note or I'll like take a screenshot of something and like share it into a OneNote file, that sort of thing. It syncs to my all of my machines. Um, but I also want to be able to have it show up and say, co-work in the cloud. So how do I now get it there? Right? And I feel like that obviously is a vendor lock-in issue that has been brewing. And I think we're it's it's gonna get worse in a sense that now it's no longer Google, Apple, and Microsoft, but you got OpenAI, and you got Anthropic, and you got uh God knows what other company. And you know, we need, I guess what I'm saying is we need some more open standards for this stuff. And I'm glad at least it's MD files, but now how to get that across the different spaces, yeah.

Mark Smith 17:54

So so it's interesting. Two months ago, I think it was Google came out with a standard for data structure, which is all MD-based with a little bit of YAML up front for metadata, but all MD files. Um and they've created a whole standard around it, uh, I think called OSF from memory. And it allows for portability across any ecosystem. Um and that ISV tool of actually based the structured layer on that for portability. So ultimately, uh, although this is exclusively built for M365, it could work in Google Workplace over time. Um if I ever decided to a port that way. Um it's interesting what you raised there, though, about vendor lock-in. I think that's almost gonna become a dirty word for anybody. Nobody wants vendor lock-in these days, and it's harder for a vendor to get lock-in because the minute you perceive that they're trying to do that, there's almost like a counter-resistance to look for alternatives. And I wonder if it's gonna go down to we saw everybody migrate to the cloud, and now, and are we gonna see a migration back to your own infrastructure? I'm not gonna say off-cloud, but so use the cloud for it's needed, but then use personal infrastructure. And, you know, Jensen Wong, I think, is is hard if you will, you know, with this hugging face acquisition, things like that, it's hard going that way and going, hey, hang on a second, you could run your own little cluster, you know, dare I say it, back in the cupboard again, like we used to have servers in the past. And and the the because that retention, right? That, you know, I mean, Microsoft, Asacha came out a couple of months or about a month or six weeks ago and said about, you know, um, about you training the models. You pay for it, and then you're giving your alpha, your IP to this organization. And do you really want to do that? Now, I know Microsoft, I mean, the lock-in there was, hey, you're in your own tenant, etc. We don't train on your models. And that's that's that's a good Microsoft story and has been on Microsoft Story for the last 20 years or so, or as long as Azure has been around. Um but I think that more and more companies are gonna look for this idea

The SaaS shift and custom integrations

Mark Smith 20:28

of can I run on-prem my AI, my compute, et cetera, and keep my data local.

Joris de Gruyter 20:36

Yeah, I think two things. One is uh because I said vendor lock-in, but at the same time, like there's been all this talk about the SaaS pocalypse, right? Yeah, which I don't I don't necessarily subscribe to, but I I do think sort of the era of of certain vendor lock-ins is gonna be gone, right? Because you can you don't need an army of developers to make your own integration. Right? As long as the API exists, if you want to resist the vendor lock-in, you can probably create your own. I mean, you just created an ISE, but I I I I so I feel like the SAS pocalypse, I don't think we're we're at least not short term, who knows, long term. I don't think anyone could see that, but I don't see that disappearing. But I'm I am imagining that sort of vendor lock-in is gonna be a lot different just because you can easily

Local AI models and hardware economics

Joris de Gruyter 21:33

write a line of business app, an integration point, um, you know, if you know what you're doing. Um so that's one thing. I think on the on the local, yeah, I think the the the open source models or open weights, I should say, I guess, um are getting very good, right? I mean they're at this point GPD four level, which you know, GPD four obviously was one of the first major ones, but And they're beyond that already. And I feel like I I could see, I mean, why wouldn't you? I think at this point it's just a price point, especially with hardware prices going way up. Um you sort of have to wonder, like, is this worth it? I bought a a local, I just mentioned this right before we started. I bought a a new GPU. I went cheap because you know, like I said, everything's gotten so expensive. Yeah, but I got an Intel uh a B70 Pro, which is 32 gigs of VRAM. Um and it's only a a thousand US dollars. Uh I say only because for that amount of RAM is actually cheap. Um but obviously, you know, Intel, the drivers aren't that great. It's it's not as fast as as NVIDIA. But at this point, I feel like the the amount of VRAM you have is more important to be able to run it. Uh I mean, I'm not gonna get hundreds of tokens per second, but even a couple dozen is enough to and so I'm playing around. I've I've I've been loading open code, just just trying it out and seeing what what can be done, right? And it's kind of amazing what these local models can do. So I can imagine as as a company for certain workloads, it may be worthwhile to invest, right? And assuming that the models get better without getting too much larger or requiring too much more VRAM, then your hardware at least is gonna also last a long time. So I could see that. I mean, you mentioned open claw and you have like use different models with that. I could see that being a thing as well, right? Where like, okay, certain things, I don't need you know, a model in the cloud. I could just run this locally, it's super fast. I know if you saw the new the the Apple, the Mac minis and the Mac Cs is gonna I was like heavily leaning into local inferencing. Um I yeah, I could I could see that being worth it. Now I I looked at a machine, one of the apples, the I think the one I wanted ended up like over 5,000 US dollars, yeah, which that's a lot of money. That um, you know, I could get a pretty good subscription for that amount of money. But that's that's just that's just me, right? If you're if you're a company and yeah, so I think we'll see how that equation uh progresses, especially

Apple's AI strategy and voice interfaces

Joris de Gruyter 24:16

with hardware prices, they're probably not gonna come down for a while, but no uh it's ra RAM is the biggest issue, right?

Mark Smith 24:21

RAM is the one that's killing everybody at the moment, uh a RAM shortage. Well, you know, just don't want to touch on Apple, because you said you're in the Apple ecosystem as I am from an iPhone. I watch that type of an iPad perspective, but I don't um after leaving IBM, I don't have a a MacBook anymore. Um I feel that Apple have really dropped the ball in the last five years. Ooh, this is where I'm gonna have to disagree, so keep going, keep going. And I tell you, uh I'll tell you why. Siri was the you know the Jarvis of of its time, if you like, and it has just totally failed. I I I use Siri for one thing, and that is when my kid goes, can you play a song? and I don't know, you know, I ask it and it plays it on Spotify. It's the only thing I use Siri for, because my expectation working with all the other tools is so much higher than what it can deliver. I feel that it had the opportunity to lean way heavier into the hardware from an AI story perspective way sooner. And because of the security they offer, the lockdown, etc., of having your own private AI or private data that you could use with AI off-cloud. And, you know, their um Apple intelligence just was a total cluster. It was a nothing thing. It was a, you know, um, I feel that I'm hoping the new CEO is gonna turn the bus around because at the moment I feel like they're a yester yeah company. Their hardware from an Apple perspective, you know, I'm on the latest iPhone that currently exists. But if I look at the last three, four years, the reason I upgrade is only two. One, it's got the old phone's got slow, and two, the camera tech has got so much better on the new phone. Those are the only two reasons for moving. But I feel that it could be the most amazing AI tool, but there's this, and and and even if you take a look at Microsoft and Google in the same category, big, large companies become risk-adverse like no tomorrow. They've got an ecosystem that's established. It's easier to milk that cow. Um, and an economics term is the whole concept of you milk your your, you know, the the cow is the the thing that people come back, you know, the cow likes to be milked, right? It gives it relief. It that that type of thing. But there's no the innovation can never be like what you're seeing in in other areas. And this is why I think, you know, if you said six months ago, I'd say, hey, Gemini was on a and up, and then what's happened to Gemini? It's it's it's not stayed in lockstep with the with the change. Um and I just think these big companies struggle to be the new.

Joris de Gruyter 27:09

I think so uh there's there's there's two parts to any good AI system, right? One is the model, and two is uh what we're now calling the harness, right? Um and I think when Apple first announced the Apple intelligence, which of course did not materialize, um I don't think the models were good enough. I mean, think about it. What we've been talking about so far, we're talking about co-work and Claude Code and Codex. These are all recent developments that we think are finally getting to a place where so I think Apple honestly, I think they jumped a gun. I felt like they they were pressured to introduce something, and then I think once they got going, they realized like this is not up to our standards at all. I mean, think about what happened to like an Amazon Alexa as well. Um one of the problems with that system, right, was I mean it was limited, but worked pretty well. It was one of the first, you know. But the problem becomes is people get so annoyed with the things it can't do. And then once it's entrenched, yeah, you have I don't know how many millions of users that have it, but how do they discover what's changed? Like, I try to ask it, like uh, so we're heavy um home automation users. I have a home assistant, we have tons of like lights and stuff in the house. Some of it is automated, but a lot of it is just pure voice control for us. Yes, um, and you know, for the for the longest time, you couldn't ask it to to turn on or off multiple lights at once, right? And it was such a frustration, and then you have to like my parents come over, you have to thought, well, you have to ask in in succession, right? Like, what that's stupid. Yes, I know it's stupid, right? And then somebody at work, I was talking about this, is like, yeah, and like, no, you can ask for multiple lights. It's well, what are you talking about? It's okay, yeah. They introduced that last year. Like, I have no idea, but that's the problem you run in with these things, right? Especially when we're having these, what we're now calling super apps, right? Person goes in, you try to do something, it doesn't work, or it does a shoddy job, and you immediately form an opinion. Are you gonna, as a user, are you gonna retry this every month to see if it got better? No, no, no, no. But Apple can't email you and say, oh, we just fixed this thing you tried last month, right? And I think they could. So going back to the Apple story is I think they realize that the models were just not good enough to keep the consistency that Apple has. I mean, we've seen that, right? We've seen a lot of these systems hallucinate, all that kind of stuff. I can't imagine that Apple would want. I mean, obviously they had their summary issues, and that was that was kind of fun to watch. But all in all, I think, you know, I think they're right. Like, and and we'll see what happens when they do finally release it. I guess it'll be end of this year, early next year, whatever. We'll see what happens. But I think we're only gotten to the point where the models and our understanding of building software for the models has progressed enough that I think now it would be possible to build something that is somewhat reliable and and good enough for their standards. That's my opinion.

Mark Smith 30:19

But but you know, there's a lot around the edges that they could have been working on. Like right now, I want to do most of my interface with AI via voice, two-way voice. Just absolutely struggles. Like, why haven't they gone and created a voice keyboard? You know, that concept, why like vit Whispers Whisperflow does. Like they could have built that in. Right now, the screen reader on an on an iPhone is six-year-old technology and it still sounds like a robot in the robot chamber, you know. And I'm like, come on, people have sorted voice now for for the last you know, three years ago. It's like it's it's it's it's it's done now, yet I'm still having to, you know, hit three fingers on the phone in the right way to rec read the sentence and and it's just an awkward thing and it hangs quite a bit as well. There and it's an accessibility issue. What the freak? This is not an accessibility tool. This is a first-class tool now, as a way people are interfacing with these. Get this these like you don't have to get the AI right, just get these supporting things that make my phone much more valuable to me as a an AI tool by getting those those things right.

Joris de Gruyter 31:32

But the fact that nobody has it. I feel like Google is also now just finally starting to do interesting things. I I have an Android um not as my main phone. I have it for testing apps and stuff. I haven't really played with with um the sort of integrated Gemini stuff, but I also think they only recently started like trying to build stuff. Like, good example. My my daughter just started school today. Uh, it's the first Um absolutely um so my day of school here. She's got uh sports, and I was trying to look up all the schedule for like things she has to do, and it's a website and it says at what other school they're playing and what times. Like, I just want to take a freaking screenshot and say, add all of this to my calendar. You can't do that. Like, I think that's actually one of the scenarios that Apple has demonstrated, I think, on, or at least I think they did. But those are things to me that as a former software developer and somebody's working in AI today, is like that is not hard to make. Like, please, those are the like the silly small things that I wish we could do right. Um, yeah, and you could probably actually to be fair, you should you may be able to take that take a screenshot and then share it to like Chat

Home Assistant and AI troubleshooting

Joris de Gruyter 32:43

GPT or Gemini or Copilot, and it could probably do it. Yeah, uh, but building that into the phone seems like a no-brainer. I agree.

Mark Smith 32:51

Yeah, and I I think Apple still nail the hardware, best in class. That they nail that right. They get their even their cloud services great. The thing that lets them down is their software. They need to be better on the software side, they need to advance a lot more. What worked, what people are satisfied with in the past, I think they need to come a long way fur forward and move the dial in that respect. You mentioned um home assistant. I'm I'm a big home assistant fan. In fact, I moved on from a Raspberry Pi and now run mine on a full dedicated computer, um, a mini computer, because I've got so much going on. And I rebuilt mine from the ground up with AI. As in as an I'm talking about from bare metal up with AI, my home assistant integration, etc. And it has been amazing. I agree. You know, it and now it does remediation work. It looks for new drivers, it's always maintaining it in an optimal state. It notifies me the minute a backup fails, and I have three different backup locations. It's um, but like complex, you know, um, like you're saying, multiple lights and things that happen. Um I've got these big roller doors on my barn that have uh, you know, three meter by three meter windows in in uh that they they come up each morning, but they do it 10 minutes before sunrise, so I get to see the sunrise happen. And then 10 minutes after sunset, they automatically and there's so much of this particularly multi-part configuration now. AI and home assistant is just epic. And if there's a gap, it goes and fixes it, goes and finds who created that GitHub, you know, for that, and and says, oh, they've got a bug here, here, let me fix it and apply it to you. I'm just like, man, this is phenomenal.

Joris de Gruyter 34:42

Um, I have not I have not integrated the two yet. I have so I also have a dedicated PC, actually. It's a Linux machine because I run Plex for multimedia. Um I don't have I don't have cable anymore, but I have an antenna on the roof, and I have a TV card on the machine that's hooked up, so I can watch TV, like whatever's on the air, like the major channels, I could get on Plex basically, which is great on this machine. And then it has a VM that has home assistant on it, uh, a couple of things like that. Um and so, but I was having issues. I moved this, this was a Windows 10 machine, but with the licensing and with uh that getting out of support, I've like finally made the step to move it all into Linux. Yes. Um, and I had issues with the VM. Like every time the machine rebooted, which obviously doesn't happen a lot, but when it did, there's always some some weird problems. And same thing, I was like diagnosing, and I finally just put Claude to work. Uh, and it's like, okay, here's what I'm seeing, here's what I'm doing. It's like, oh, have you tried this? Okay, get me the logs of this, try this command. And then it ran for a month or so, and then the issue reoccurred, and then I went back to the session. It's like, okay, it's the issue is back. It's like, oh, then it probably was this, and so we and now it's fixed, completely fixed. I had a similar issue on my Windows PC. It was a bizarre networking problem that came out of nowhere, probably a driver problem. But I had it was like I was having packet loss, and I could not figure out. I thought it was my network, but turned out it was only my computer, which is the best PC in the house, it's hardwired. It's like, how is this possible? And same type of thing, I'm uh wit's end, fired up Claude, and it's like, okay, well, tried this, and then yeah, it fixed it. I'm like, man, this this would have taken me weeks of troubleshooting and trying to figure out and reading. So, yes. So I I have not hooked straight into home assistant, but I am using it for a lot of sort of home networking and IT support that obviously us as as geeks uh we're pretty good at, but um, sometimes it goes beyond our knowledge phase, or it would take too much time. So it's a huge, huge time saver.

Mark Smith 36:44

And that's the thing, it would take too much time, I feel, to do it manually. I have a robotic lawnmower that um uh Chinese, but it's called Momotion Luba AWD 5000, so about 5,000 square meters of lawn it covers on my property. And uh about two months ago, somebody in Sweden let their 18-month toddler out with their mower. And um and when it does a reverse function, it doesn't have the LIDAR sensors on the reverse of it, and so it backed onto this kid. That and so what they did is globally did a firmware update and screwed everybody's reverse function. And so it made a, in this case, a $7,000 MOA one take what's called tank turns, you know, basically could carved out these things in the soil. Well, because it had an API and there was a home assistant integration to it, they had removed it from the app, the experience to do what's called a multi-point turn, but they hadn't removed it from the API. And so my work round, once again, with Copile, sorry, so I used um Codex for this, was able to go, hang on a second, we've got this covered. And uh rather than launch from the app anymore, I only launched from Home Assistant and and kept that feature um that I needed. So I mean, things like that, you just wouldn't have the access in the old world of pro you wouldn't have the time. You wouldn't be just like, I'm gonna do a weekend.

Joris de Gruyter 38:11

It's a time thing, yeah, for sure. Yeah, yeah. I have uh my thermostat has been freaking out. It can't connect to the Wi-Fi, or it's connected but it can't work, and then every so often it reboots randomly, which I think is because there's something. Uh, and I know I've changed settings in my network, and I don't know what I changed to cause it. It's the only device on my network is completely misbehaving. And similarly, I have sort of a Wi-Fi that you can put in monitor mode so you can see all the signals coming by basically. And so I've already been talking to Claude about it, and I'm just basically gonna give it access to the machine and be like, okay, here's here's my my router, here's the Wi-Fi monitoring tool, go figure it out. Because it was asking me too many questions. Like, okay, typing this back and forth is gonna take too much time. It's time to give it access straight up. Yeah, those are the things that I mean, it's it's funny. I was talking to my my parents uh the other Um absolutely um so my day. My mom had like one of those pop-up issues in the browser.

AI subscriptions and model choice

Joris de Gruyter 39:06

You know how that happens. They clicked on something, and then I think it was an extension or something that got installed. Um, and she was talking to me about AI, and I was telling her the story. It's like, you know, there's things that I can't fix either, or at least I maybe could, but who knows how long it's gonna take me if I can find the right article on wherever online. And yeah, so we're getting to the point where we can also solve problems that you are even beyond our technical expertise. It's yeah. Yeah. So how many subscriptions do you have, Mark? For AI subscriptions.

Mark Smith 39:41

Okay, so obviously.

Joris de Gruyter 39:42

We can't count Foundry because that's kind of like multiple models. So let's let's part that.

Mark Smith 39:46

So full version of Copilot and co-work, uh, sorry, yeah co-work uh from Microsoft Ecosystem. I got a full license to um GitHub Copilot. Uh I've got max subscription to anthropic max subscription to open AI, and then I have nine API subscriptions, base subscriptions. So those are for um all the other models, if you like, that are not in that category that I just covered. And I'm talking about European models I access that way. I think I got three, four maybe different Chinese providers. Yeah, four, four, uh four in that. Um and yeah, so I I'm I'm using a lot. Um, I wouldn't say that I'm token maxing as in because one thing that Codex has done is just giving you think you're gonna run out, and then they're like, we're just we just reset your limits. And I'm like, yes. Because I tell you, I have been in the position where I have maxed out, and it's a horrible feeling. Yes. When like I've bought more credit on top of the max subscription just so I could keep going.

Joris de Gruyter 40:54

Yeah, I don't have any of the max ones because obviously I I have the benefit of getting at work at least getting tons of stuff. Um, but I do have at home here, I do have subscriptions. I have I pay like the minor version of Gemini, um, which I've been using for research.

Mark Smith 41:12

I think Gemini I have sorry, I have Gemini as well. I've got that subscription I've got that script. Yeah.

Joris de Gruyter 41:17

Yeah, I think Gemini is great for research. Makes sense. Google's a search company, they have huge libraries of the internet, so it's great to do that kind of work. Uh obviously their image generator is great. I don't use that a ton, but when

Microsoft Scout and enterprise access

Joris de Gruyter 41:29

I do, I I like the banana model for sure.

Mark Smith 41:32

I tell you what, I reckon Maya's image generation is now the best in the world. That's my experience.

Joris de Gruyter 41:38

Is it?

Mark Smith 41:38

Yeah, I I it's better than open AIs. Okay. Yep. That's my and I've been through a lot of image generating journeys.

Joris de Gruyter 41:49

Yeah. Uh and then I have a Claude subscription, which is also probably just a minimum one. Um and then I do have a personal GitHub Copilots Pro subscription. I think that's the only one that I have a bit higher uh limits on. And that's mainly because you know I can't use the one from work for too many personal things or I'll get questions being asked. Makes sense.

Mark Smith 42:13

Have you scout at all?

Joris de Gruyter 42:14

Microsoft Scout? Um I have, but I'd rather refrain from talking about it because I don't know what features are released or not released.

Mark Smith 42:23

So I'm Yeah, no, I as an I just asked because I got a mark coming on the show. He's is uh the the Okay, nice. Put it this way. Since the new podcast has started, I've only asked three Microsoft people to come on the show. You're one, he's one, and and Katie who did the co-pilot uh adoption program in Microsoft, which I've got late this afternoon. Otherwise, I'm I'm not uh indexing heavily unless like I know somebody like yourself who who's across the board on so much. So no, I was just wondering, as in because the blocker for me is because it has this weird requirement to have an enterprise GitHub subscription. And I've got a pro, and I'm like, why an enterprise? It's just it seems an arbitrary click of the ticket, and that's not needed. There's nothing in I know of um uh of um open claw that requires an enterprise version

Secure model routing with Foundry

Mark Smith 43:20

of and when you I don't know if you've ever done this, when you start changing versions, even going from a company GitHub to uh personal GitHub Pro, etc., it stuffs up a lot, like you've got to do a lot of logistics and changes.

Joris de Gruyter 43:35

Yeah, enterprise is definitely sort of very gated, uh, which I guess is is the selling point. Yeah, I I have opinions and thoughts on Scout, but I will refrain from because I have I have no idea. I mean, I may even be saying things that are not true, so I'd rather not at all. Especially if you have the man himself coming on, then that's perfect. Yeah, yeah. I I do know that it's um the people that have used it internally, I I think it's become very popular.

Mark Smith 44:03

So yeah. Well, the thing is, if if if it's anything like, and I and this is interesting from the open claw community, as in Peter, the you know, the founder of OpenClaw, he speaks very highly of the relationship with Microsoft and the work that's happening inside Microsoft with Scout. Um, but then I hear recently I saw an article actually where um it looks like Microsoft is building their o their own claw ish tool that is not on OpenClaw but uses the full Microsoft stack. I tell you what, there's some low-hanging fruit, I feel, and I'm not saying you'd hand this on from Microsoft. The routing tool in Foundry could be a massive money earner for Microsoft if they invested more into it. As in having all the models to go to task-based routing to really make that a massive feature, that really the community understood what you could do with that. I think that's a an untapped um opportunity for Microsoft to be the model provider with all the security that Microsoft provides around your data, and then have a real sophisticating routing tool that's not, oh, I want to select that model. All right, no, based on the task, it says this I'm gonna use out of this pool of tools this. Um, but I'm gonna keep your data safe. I'm gonna do all the things that Microsoft provides boundary-wise, etc. Um, I think, yeah, massive untapped opportunity right there to go large. Like it should be the open router equivalent or the even the hugging face equivalent at that type of scale, but inside Microsoft's secure system, I think it needs somebody that looks at as its own product line almost and engineers it accordingly to be best in class against open router and hugging face. Although they're slightly different tools, I think the foundry router could be absolutely that at scale.

Joris de Gruyter 46:00

Yeah, that's that's that's interesting. I yeah, I think for for enterprises though, uh it's probably already an issue anyway, but sort of the cost prediction is is that's still a big issue. And I think when you start and do sort of automated routing, I I assume that would be a challenge as well. Like, but yeah.

Mark Smith 46:17

It's doable. It's a challenge, right?

Joris de Gruyter 46:20

Yes.

Mark Smith 46:21

Joris, it's been awesome talking to you. Thank you so much for coming on the show. Um, it's always a pleasure. Yes, sir. Same. Love being here. Thank you. You've been listening to the intelligence age on my tablet. The intelligence age is already here. The question is, what we choose to do with it.