Home
Sign Up
Stéphane Boisvert from Principal WP
Sep 21, 2026
Stéphane Boisvert from Principal WP
15
30
00:00
37:59
1×
SUBSCRIBE FOR UPDATES
Subscribe
Listen on
Apple Podcasts
Apple Podcasts
Spotify
Spotify
YouTube
YouTube
Transcript
0:00
91% adoption of AI usage, and only 6% of those companies saw an actual, like, benefit- Yeah... like, to the bottom line. I think most people are trying to make a faster horse.
0:12
Let's give this tool a small ticket to, to see if it works, and I was like, "No, give it your hardest ticket." I haven't been this excited about the state of things for- Yeah... at least a decade.
0:22
Maybe even since, like, Ajax. This is Not Just Faster Horses, conversations about deploying AI in media companies at the intersection of product technology and people.
0:33
I'm Ben May, founder of The Code Company, and today I'm joined by Stefan Boisvert from Principal WP.
0:40
Stefan has spent 15 years, uh, working inside enterprise WordPress from WordPress VIP, CTO of XWP, and is now consulting as Principal WP.
0:51
Um, Stefan is now consulting with, uh, media and engineering teams about deploying AI inside their operation. Thanks for joining me, Stefan. It's a pleasure to be here, Ben.
1:01
So you've watched this play out across a lot of different contexts, in-house at a product company, client services business, and now as a free agent. Mm-hmm.
1:10
Um, what are you seeing at the moment in terms of adoption across AI, uh, across these different sort of cohorts o- of people? I think it depends on your definition of adoption, right?
1:21
So I think everyone is doing something with AI, right? I think everyone thinks that they're doing things. Many thing... Many people think they're doing the right things, but it is, it's incredibly inconsistent.
1:34
It's incredibly inconsistent, and this is not just a media and publishing thing, right? Like our other clients as well. Yeah. Yeah. We're seeing across the board
1:44
it's not a uniform adoption, and some people you see are getting real gains, but most aren't, right?
1:50
There's, there's been a bunch of surveys done, right, and I think McKinsey's the biggest one, and they looked at everyone, and there was about, like, 91% adoption of AI usage, and only 6% of those companies saw an actual, like, benefit- Yeah...
2:06
like, to the bottom line, right? Wow.
2:09
Um, so as, as engineering has got faster but necessarily delivery hasn't kept up with that, how, uh, y- how are you sort of trying to help people in figuring out what that bottleneck is, where the bottleneck has now moved to?
2:21
I think most people are trying to make a faster horse, right? And I, I think it's logical, right? They're, they know the concept of, like, we have developers, we have our steps, right?
2:33
And the old process was basically sort of like gating things so that you didn't have too much stuff going to developers. You only had the right stuff going to the developers. Mm-hmm.
2:43
The, only the right thing got coded, right? Because code was expensive. Yeah. And I don't think people have changed the mindset.
2:49
Some of them are trying to be, like, they're trying to add AI in the product owner or for project management, right? Writing a better brief. Y- yeah, exactly. Yeah. Right? Yeah.
2:58
They're trying to, or to dri- write a faster one. Mm. Right? But most people haven't taken this concept of coordination. That I think is the biggest one. Mm-hmm.
3:09
And, and coordination sounds a bit lofty, but what it really is is just that instead of, instead of doing things the, the same way as before, you're just like one step at a time, et cetera, you basically interact, like integrate the AI across everything.
3:24
So you rethink how it means to work with a stakeholder, right? I'll, I'll give an example. I'm sure you've had this before. Your, your team's working on a ticket, right?
3:36
And they're like, "Hey, uh, SEO team, we're, we're making a change to how the category landing page is, so we need just your thoughts on, you know, if this is okay." Crickets, right? Yeah.
3:48
And, and then sometimes you'll follow up or something, but then the engineer just does the thing, deploys it to production- Yep... everything breaks, and you go back and then you need to revert everything, right?
3:59
And what you need to think about is not just getting the SEO team inside Jira, but it's making it so that there's an AI, there's an SEO AI- Mm-hmm... that knows what the SEO team cares about, right?
4:12
And then tries to reply, tries to answer by itself, but when it can't, it goes where the SEO team is, right? And that could be in their Slack channel. Yeah. That could be in their MS Team. That could be an email- Yeah...
4:22
if that's what they care about, right? Yep. And then just sends them exactly the targeted info so they can reply, "Yes," "No," or, "This is the thing," right? Let me jump into it. Yeah. Exactly.
4:31
It's, it's really about rethinking the patterns completely. Yeah. And you've worked beyond just media, um, you know, across your different roles and, and even where you're working now.
4:42
W- what are some of the more unique things, I guess, inside media companies, um, when it comes to sort of product and engineering, um, uh, in terms of, yeah, what are the problems they're trying to solve, things that come up often over and over again, even outside AI, I guess, and where AI can help?
4:56
Hmm. I think one of the, the trickiness is people haven't, again, made the switch to you can have now your revenue team be able to prototype things. Mm.
5:08
Be able to push changes to the agents that will then test things out, that will do things for them, that will do m- most of the work for them, and then they can pass it to a developer to review. Mm. Right?
5:21
Or to a developer to maybe rework, but that gives them like, especially for revenue, for ad ops, for, uh, design, for any of those, right, for editorial or for, uh, sponsored content, right? They can create new blocks.
5:33
They can do all of that, show it, work with it, with the client before getting the developer to implement the real version. Mm. Right?
5:40
So I think those are the kinds of things that media publishers are, are starting to, to tune into, right? Yeah. Starting to empower their designers, starting to empower the others- Yeah...
5:49
without putting all of those gates in front. And I'd even zoom out even a bit more and be like, it's a industry that is sort of under existential turmoil- It is. [laughs]... uh, forever. Yeah.
5:58
It's not just new, but it's definitely now. Yeah. Um, to me, it's like it gives you this huge opportunity to do stuff that you've never been able to do before. That's exactly it.
6:06
Um, but again, not just by copying and pasting snippets of code out of- No... ChatGPT. No. No. Uh, it's gotta be a bit more transformative. You need, you need the orchestration pipeline for that. Mm. Right?
6:15
You need to have a plan and a vision, and you need to map it out about here's all the ways things are gonna integrate into our pipeline, right? Yeah.
6:22
So everyone's on this journey and, and I think engineers are naturally predisposed to wanting to, you know, use more tech and to be more in, um, you know, adventurous and innovative and all this sort of stuff.
6:31
Have you got any examples or, or, or cases you've seen sort of this, this flip happen from, um, the, you know, the old way of working to, you know, the, the new way of working?
6:40
Um, what did, what did that look like or how, how do you think about that as, like, from transformative perspective? I think it's interesting because one of the assumptions you made, that engineers are ready to jump on.
6:51
I don't- They're not all [laughs] Not all. Engineers also have opinions about things. Yeah, yeah. Yeah. [laughs] And, and part of the challenge is that many of them tried using AI when it was Sonnet 3.5. Yeah. Right?
7:03
Or, like y- Just a garbage machine, basically. Like- Exactly, right? Yeah. Like, you could, you could get, like, a fun, like, Excel formula or something out of it. Yeah. Right? But...
7:12
And I think that, uh, that's changed. Yeah. Right? So- It's no longer Will Smith eating spaghetti or whatever. Like- [laughs]... we've moved beyond that sort of phase of- Exactly Yeah. Exactly.
7:20
And so I think what I find with a lot of people, is disbelief or discredulity, right? Especially people who don't know me or something, they're like, "Mm, no, what you're saying, what you're...
7:30
Like, I think you're a charlatan," right? Yeah. And even people who do, they don't believe me until they see the results themselves. Mm-hmm. And I'll give this example.
7:38
I, I was working with one of the lead engineers at, at a media client, right? And they were extremely skeptical. Mm-hmm. Right? And I understand it, right?
7:46
And so they were like, "Okay, let's, let's give this tool a small ticket to, to see if it works." And I was like, "No, give it your hardest ticket." Mm. "The hardest ticket you have," right?
7:56
Which is, like, custom live blog integration into Apple News Live, right? Which anyone who's worked with Apple News knows how horrible that is a platform. It's, it- From WordPress to Apple News is a bad...
8:05
Yeah It's, it's never good. Yeah. Right? And, um, you know, the pipeline started asking it questions and, and she was like, "Oh, wait, this is a really good question. I hadn't thought of that." Mm. Right?
8:15
And then answering these questions, and defining, and f- like, fixing up the requirements and the scope. And then it built the whole functionality. Yeah.
8:23
And, and I, I just remember being on a call after, and they kept telling me, like, it built the whole thing, and I was like, "Yeah, I know." Yeah. [laughs] And they were like, "No, no, but, like, but it, it works."
8:32
I was like, "No, I, I..." Yeah. [laughs] It's not a cardboard cutout. Yeah, yeah. Um, 'cause you talked about that this week at WordCamp US. Mm-hmm. Your, um, your pipeline- Yeah... which is sort of your agent.
8:41
When you demo that to people- Mm-hmm... and, like, the example you just gave then- Yeah... who in the, who in that room is getting excited, but also I guess who's getting nervous, uh- Yeah...
8:49
around, around seeing that sort of thing? Yeah. I think initially engineer gets nervous, 'cause I think they assume that it's a slot machine. Mm. Right? Like, that, that all, every- Or it's more a job killer. Yeah, yeah.
9:01
Yeah. One or the other, right? I think... And that's interesting, because when I started, I did think that maybe some companies who'd hire me, they would want to reduce headcount. Mm.
9:12
And every time it was the exact opposite. Yeah. What they're looking for is just more output. Yeah. They've already lost the people. Yeah. They've already had to cut the budgets recently- Yeah, yeah...
9:20
in the last couple of years. Yeah. So that's, that's no longer an option. Yeah. But what they need is more output from the same amount of people, right? Yeah. Yep.
9:27
And I think engineers worried that this is actually just gonna cause more work because they're gonna rev- be reviewing and fixing mistakes, and fixing a bunch of stuff.
9:35
You just hired 20 juniors, now I have to manage all of this mess. Exactly. Yeah. Exactly, right? And, and I understand, right? And so you need to bring them along, and you need...
9:42
For, for engineers, you need to show them the pudding. Yeah. Right? Like, that's where the proof is.
9:46
Um, but who gets excited is usually people who haven't always been able to be at the front of the features request list, right? Yeah.
9:56
So if you have a product manager, or the design team, or the, like, sponsored editorial team, or all of those, right?
10:05
They now see this as like, oh wow, I can, I can prototype and I can get things production ready for the developers to review and to push out. Yeah. Right? And I can do all of this cycling- Mm...
10:17
without needing them, right? Yep. I can get a whole bunch of things, get some prototypes, make a bunch of tweaks, get, get the output, and then when I'm ready, make it ready for the engineers to review. Yeah.
10:30
Have you, uh, uh, thought... Ha- had any, you know, revelations around what measurements, which always leads to then ROI, uh, in these conversations? Mm.
10:38
It's, it's this weird paradox we're in where no one's just gonna turn it off. We're not just gonna be like, "We're not spending that money anymore on tokens." Mm-hmm. "We're gonna go back doing it by hand." Mm-hmm.
10:47
But at the same time, you would imagine there's gonna be someone who says, "Well, hang on. You're spending 50K on tokens this year." Mm-hmm. "What did we get out of that?" Yeah.
10:55
How, how do you manage that sort of pushback or, or, you know, genuine question if it's, it's a new line item? Yeah. I think it, you know, for, for...
11:03
So for us, we're still doing the, the tracking, the analysis so that we can put out a case study where we have- Mm... those numbers. Yeah. Those six numbers, right?
11:11
But we wanna make sure that we have enough time to get enough data, right? But what we have already from people is their perspective.
11:19
We ask them, like, "Compared to before, like, how much more work are you getting done?" Yeah. Right?
11:25
"Compared to before, how much more comfortable are you with the safety, the security, the, the performance of the code it's outputting?" Yeah. Right? Compared to that.
11:33
And, like, our results, people either pick a lot more or night and day. Yeah. Right? Those, those are the two highest ones. Which is still subjective, but they're, they, they're very- It's, it's incredibly subjective...
11:41
but they're very clear. They're very clear though. Yeah. Yeah. It's- It's not the opposite... but it's very clear. It's not, it's not- Yeah... a, like, moderately more, or like somewhat more.
11:46
They're not hesitating, yeah. No, they're not hesitating. Yeah. Right? They've seen that, that output.
11:51
And I think there will be soon a sort of reckoning of ROI, because I think a lot of people are using AI in a way, and they're not getting an ROI, right? They're just kind of slopping around, right?
12:02
And, and potentially then setting that back a year or two internally because like, "Well, we tried AI last year." Yeah, exactly. We... Yeah. And it cost too much.
12:08
We spent too much money and, and- Where do you see WordPress as, as a CMS and a content authoring system, its strengths, its, its sort of... How does it hedge off, you know, AI and vibe coded solutions? So
12:20
I don't think it's gonna hedge off vibe coded solutions, because I think vibe coded solutions are targeted to people who they only have three, four pages. Right. They don't have a massive amount of content, right?
12:33
And I think that- WordPress needs to understand that, right? That it is no longer the easy- Yeah... CMS. I, I think it's okay that WordPress loses a little bit of market share. Yeah. It's, it's just gonna...
12:45
It's happened with Wix, it's happened with all these other things. Yeah. It's gonna happen again. Yeah.
12:48
But again, there's probably people who do that work, but that, I think, is just, that's just, you know, the, the bar of open source to me is rising. Yeah. The threshold of where it makes sense. Exactly.
12:56
It still exists, but that bar is still ri- And it's been rising the whole time I've been in doing digital. Yeah. You know?
13:01
Like, you couldn't buil- you'd build a WordPress site because Wix didn't exist, then all of a sudden- Yeah... Wix came about, and like- Yeah... just go set up your local fish and chip shop on a- Yeah... Wix website.
13:10
No, exactly. Like, yeah. Your restaurant website, right? Yeah, that's right. Like WordPress, and I think w- we may need to accept that, right? Yeah. And accept the difference that now WordPress is actually going to be
13:20
like mostly for enterprises. Yeah. Mostly for large companies that have a lot of content, that have a lot of things. It's no longer what we thought it was before. Yeah. Right? Like, for SMEs or, or- Yeah...
13:30
that sort of thing, right? Yep. Um, what about in terms of, I guess, like the, the tooling and how you interact with WordPress? Mm. Do you see it ended up just being like a data backout, a back end?
13:40
Where, where do you think WordPress sits in that sort of mix? Well, I think it, it will keep sitting in the middle, right?
13:44
I do think, I do see some value in some MCP connectors, because it does allow you to, for example, pull things out and send it to someone in Slack, so that- Yeah...
13:54
they can review something without needing to log in, right? Yep. So that you can output this page so that, um, you can, you know, create a PDF to send it to your sponsor- Right...
14:05
to get them to approve the sponsored content, right? Yeah. So, so I think there will be a lot of, of connectors with WordPress, and I think part of the appeal of WordPress is that it's so well-documented. Mm.
14:17
And there's so... Like, there's so much functionality built in, and it's easy to get the AI to know what the right way of doing something with WordPress is. Yeah.
14:27
Now, it is a bit more challenging than some other platforms, because the training data set for WordPress skews to bad code. Yes. [laughs] All the things to avoid. Yeah. The, the anti-patterns. Yeah. Yeah.
14:37
It, there, it's everywhere, right? Yeah. So I think even for, for us, we're seeing that a lot, is that compared to other platforms, you need a lot more guardrails around- Right...
14:45
WordPress, because by default, Claude is going to build what's in the training data set, and the training data set tells it to do things that will not scale at, at any- Everything's in functions PHP. Yeah. Yeah.
14:57
Yeah, things like... Well, even if it is, like often it's writing just things in PHP templates or something. Yeah. It's not creating blocks the right way. It's not using the latest version of blocks.
15:04
I've, I've been fiddling around with it and, and it did, it created some blocks, but it just put like chunks of HTML in the block. HTML, e- exactly.
15:09
But by default, by default- I mean, okay, well, that's kind of cool, but that's not what [laughs] I- It's, it's completely useless... that's what I wanted it. Right. It's not at all.
15:13
So I, again, some fine-tuning and some guardrails. Yeah. Now it's doing- Yeah... what it needs to do. Yeah. So yeah, you have to really- You really need to fine-tune it... know what you're trying to design. Exactly.
15:20
Yeah. And I think that is something like if you do just start using Claude just straight out of the box to do WordPress stuff, you're gonna get weird results. Yeah.
15:28
You need to have this, this library of knowledge of WordPress. Yep. Right? And you need to have it so that the AI uses that while it's working. Yep. And I think most people don't do that. Yeah. Um,
15:41
in terms of just on WordPress for ano- for a minute, w- with AI doing security reviews and things like that- Mm... how do you, how do you see that playing out?
15:49
Is that a, a, a net positive or a, a risk to manage, um, that, you know, we're, we're in definitely the bumpy period right now, where- Yeah...
15:56
where a lot of, as AI's hitting this like incredibly good inflection point of like finding stuff. Mm-hmm. I mean, we saw it with Firefox and like Mythos. Yeah. Like
16:05
25-year-old piece of software, they found hundreds of bugs that had- Yeah... been manually reviewed forever, right? Yeah. And, and, and we're probably just starting to have that now with WordPress.
16:11
I think we've had four maintenance releases. Yeah. Um, with some, you know, not, not the worst, but not, not, not nothing either, like some decent bugs, uh, security things in there, yeah. Some very, some huge bugs.
16:21
Yeah. And if you read how they work- Yeah... uh, like I've been in the WordPress game for a long time. Yeah. And I was impressed- Yeah... by the, the, just how it did things, right? Yeah.
16:32
How it, how it figured out how to call- Connected two random things- Exactly... to jump a do- yeah. It, it, you know, it, it's frankly impressive. Yeah. Now, I do think it's overall a good thing. Mm-hmm. Because
16:45
before you probably had some either government affiliated or pseudo government affiliated- Yeah... organizations that had the capacity to find those bugs. Hundreds of people in a room attempting to- Yeah...
17:02
to backdoor stuff. Yeah. Hundred of, of PhDs with- [laughs]... you know, uh, who- Locked in a, in a, in a computer lab somewhere. Yeah. Yeah. In, in a bunker somewhere. Yes. Right? Put under duress, yeah. Yeah.
17:12
[laughs] And so there, there's probably people who knew about these. Yeah. Right? Yeah. And yes, it's easier for people to find them. Yeah. But most, it's much, much easier for people to find them and flag them. Yeah.
17:28
But there's, there's now, if one person finds it and doesn't flag it, there's still someone else that there's a good chance someone else is gonna find the same thing- Yeah...
17:35
relatively quickly with the same models, right? Yeah. So, so I think overall it's a good thing. Yeah.
17:40
And then again, like you, you've probably seen as much as us, like especially the enterprise, it is not rare to come in and they're running like a 15-year-old version of.NET that's not even supported. No.
17:49
And it's like, well, you can either be working with something open source that is, it's all being sort of done in the live. Yeah. Um, and this stuff is moving fast. Yeah. And it, I think- Yeah...
17:55
I think we'll, we'll get over this, this hedge of like everything's caught up enough and it's got through the backlog of stuff. Yeah. Yeah. Uh, and, and I don't think that will be, this won't be the case forever. Yeah.
18:04
It'll just be f- a, a, a thing, but again, if you're using some unsupported 15-year-old thing- Oh, no. No... no one's even testing that. No.
18:10
You're- So there's probably all these vulnerabilities that you- They will find those vulnerabilities- And no one's fixing them... incredibly quickly. Yeah, exactly. Yeah.
18:15
No, if you're running something, I think, that's custom made- Yeah... um, the odds of you getting hacked have just become- Yeah... immensely higher. Yeah. Immensely higher. How do you, how do you...
18:26
I mean, this probably s- comes out of your, slightly out of your current lane, but in terms of evaluating, if you were building out a new product today- Mm-hmm...
18:33
and, and, you know, gonna vibe code a back end or- Mm-hmm... or use open source components and things like that, how would you advise people to sort of be thinking about that paradigm?
18:40
If they're, if they're getting told, you know, I, I had- Coffee with someone a couple of weeks ago, and they're like, "Well, my IT manager is like wants to rebuild HubSpot, because he could- just give me two weeks, I'm gonna go out and build HubSpot- Mm-hmm...
18:50
because we're spending too much with HubSpot." Mm-hmm. Now, you and I know that's a terrible idea. Mm-hmm.
18:53
But like, but if you're a, a leader, executive who doesn't really understand that, how do you sort of think through that? [laughs] Like, it sounds too good to be... You mean I can save 50 grand on HubSpot- Yeah...
19:02
if, if I let this guy go off and build like- Mm-hmm... HubSpot for two weeks? Like- Yeah... yeah. I mean, if it's only two weeks, sure. [laughs] Let your, let your engineer try. Yeah.
19:10
And then you'll get data on if you can trust that engineer- Yeah... [laughs] and what they're saying. That's right. Um, so, so I'd say, you know, make the calculus and maybe give it a go. Yeah.
19:21
But don't close your HubSpot account right away. [laughs] That's right. Maybe try- Yeah... using both at the same time for a while. Yeah, that's right. Um, because I think...
19:30
The CMS is obviously at risk as well, because- Sure... when you, when you boil it down, be like, "Well, it's just- it's just an article editor." Yeah, exactly. Like, you're just, it's just post and table.
19:36
Yeah, yeah, yeah. Like, you know, I forget. No, I've... And, and we've seen so many people over the years try to build their own, right? Yeah. And, and I think, like,
19:45
there's gonna be this, this, this attempt to do that, right? Yeah. And I think at some point it may become that the ROI makes sense, right? Yeah. At some point, maybe that's possible. I think for now, it really depends.
19:59
If you're just using HubSpot to, to be a place where your form leads come in- Yeah... and then someone copy-pastes the email from HubSpot to, you know- You know, you're using 5% of the functionality. Exactly, yeah. Yeah.
20:10
If you're... If there's software, if there's a SaaS, you're using 5% functionality. I think there's gonna be, yeah, re- huge redistribution of, like, SaaS companies. Yeah. Yeah, exactly.
20:17
'Cause there are people who are locked in, "Oh, I wanna do conversion tracking." Yeah. "Well, you need to be on our enterprise plan." Yeah. "I don't use anything else- Yeah... but I just wanted that one feature." Yeah.
20:25
Um, but then the other, other end of the spectrum's, like, I pay Calendly $45 a year. Yeah. Am I gonna go build a whole data warehouse for my, my calendar booking? Yeah. Yeah. You know.
20:34
Well, but the interesting thing is, like, there's a lot of cross... So Google now launched their calendar booking. Yep. Right? Which I did jump off, and I don't pay the full...
20:44
But on the vibe coding one, there's- No, no, exactly... yeah. And that's, and, and I think that is a- another, a different difference- Mm...
20:49
right, is that there's gonna be a lot of products that are gonna have things that before you had a bunch of little side things. Yeah. And everyone's trying to do this, and it's not super new. Like- Yeah...
20:59
Zoom has these, like, notes and docs and spreadsheets and slides. Every- everyone has a... Yeah, yeah. Everyone has this super app now. Yeah. Right? Yeah. That they want to just get you into the thing, right?
21:08
Every app now does meeting transcription. Yeah. Right? Like, a- and so, yeah, as part of, like, anything, right? Yeah, exactly. So I think that's gonna happen. Yeah.
21:19
But I, I think if, if the software is not your main business concern- Mm... I still think it's a dangerous game to play. And for a media business, it's building on top of the, you know, established framework.
21:31
Build, build, y- and again, using AI and all these things. Yeah. Because again, media companies, you've always gotta make trade-offs. You, you don't have to make as many- Yeah... now.
21:37
Like, build all this, you know, unique stuff to your business model that- Yeah, yeah... gets audience, gets traffic- Yeah... gets, you know, revenue, whatever those things are. Yeah.
21:44
Um- I think that's the, that's the real way to focus on, right, is- Yeah... how can we, you know, as much as like, ugh, this is- It gives you some constraints to, like, narrow it in.
21:51
Like, 'cause otherwise the surface area is infinite. Is infinite, yeah. Now you're like, "Well, okay, here's our lane, and we can build on top of that- Yeah... and, and be disciplined." Yeah. To some...
21:59
'Cause again, if you lose engineering talent, someone else has gotta pick it up. And if some, it's... Again, I, [laughs] I'm sure you've done these as well.
22:05
You come in to do a review of someone's work, like, "Yeah, we had two guys. They built their own thing." Yeah. "They've both left, and now no one knows what it is- Yeah... or how it works." Yeah.
22:12
And like AI, I think, is just gonna multiply that- Yeah... uh, with, with the creation- Dramatically... of stuff. Yeah. Yeah. Yeah. Um, back to, to media, I guess. If you were...
22:20
You've done a lot of sort of in-house, out, uh, and, and agency side. If you were dropped into a new business that was starting up, blank canvas- Mm-hmm...
22:28
how are you thinking about product and engineering teams now, um, in, in terms of the ratios of, of headcount?
22:36
What is stuff that you want, uh, happening inside the tent versus outsource or agentic or, or whatever that is? Um, yeah, paint a picture of a new, of a new product and engineering team.
22:46
I think what your product and engineering team should be now is fully just focused on building the systems that the rest of the organization uses to l- to update the site, to update the functionality, to build new functionality.
23:00
Your product and engineering team should maybe do review of stuff before it goes out. Mm-hmm.
23:05
But basically, they should be building the framework that the rest of the organization uses so that now you just have your, your, uh, advertising team. They will type in their things in, in a ticket.
23:20
The agents, the systems they've built will start asking questions, will start trying to pick it apart, try creating the requirements.
23:26
Then the system will do the research, will figure out, how does this work with the rest of the system, right? And then, uh, bring back the requirements as it understands it to that stakeholder, right?
23:36
Do all of this work, like, in the background, and then build out the specs. Mm. Build out the whole plan of how is this gonna be implemented, right? And the engineer is no longer doing the specs. Mm.
23:46
They're building the system that does the specs, and they're ro- reviewing the system that does the specs, and they're improving that system, and they're continuously monitoring and, and doing that work.
23:56
It's now systems thinking, right? Yeah. And it's really about you shouldn't have any developer writing code that is for that functionality. They should only be writing code that is for the pipeline, the system.
24:11
Some people call it the software factory. Right. Right? And I think you call it comp- the compounding factor, like- Exactly... everything you're doing is, like, chipping away at the- Exactly... IP of that, that company.
24:19
Exactly. It keeps compounding, right? Because you need to, to learn from every AI run. Yep.
24:24
You need to keep improving your system based on that functionality, 'cause it's gonna make mistakes, but it should on- only ever make that mistake once. Right.
24:31
And, and you know, everyone talked about that for inside their company, right? I want all my engineers to reuse things that we built for other projects- Oh... to reuse functionality. For, for, for 20 years.
24:40
Yeah, exactly. Exactly. Never happened. No, never happened. This is incredibly hard, and, like, the cognitive load, like- The cognitive load is impossible...
24:45
I'll just remember every project we've ever done and just share that- Yeah... with everyone every week. Yeah. Yeah. But now- Yeah... you can have that.
24:51
You can create reference files for every single piece of functionality. Yeah. Right? For everything. And you can have AI look through- Hundreds of pages- Yeah...
25:00
of the things you've done before, and how it worked and h- why it works or how it sh- can be tweaked, how it can't be tweaked, that sort of thing. Yeah. Incredibly quickly.
25:08
How are you thinking about junior engineering and fitting into that pipeline of talent, um, as people are coming in? I think right now it's probably a really awkward time in, in the education system. Mm-hmm.
25:18
Uh, engineer- like, a lot, everywhere it's probably- Mm-hmm... there's probably got a, a group of people right now who are going through this where the people who are teaching or training haven't even figured it out yet.
25:27
Yeah. Um, but, you know, fast-forward a little bit, where do you think the, the, the new, you know, junior of 2030 is gonna come from, and how is that sort of pathway into an org?
25:36
'Cause historically you'd, you'd be given the crap work. You'd be like, "Lis, here's like this really annoying job no one wanted." Mm-hmm. "You go work on it for a few weeks-" Mm-hmm...
25:42
"and we'll, we'll PR it," and things like that. Mm-hmm. Um, have you had any, any brainwaves on, on that yet or? You know, if I think about engineering, right?
25:51
'Cause we're all gonna get too old one day and we can't- Yeah, yeah. [laughs] You know, it's good.
25:54
All us people born in the '80s, it's, it, you know, it, it's gonna, it's gonna, we're gonna have to figure out how do you top up the funnel. Yeah.
26:00
[laughs] I think if you think about engineering as providing solutions to things, right? I think that kind of thinking is still there, and to me, it's gonna be a lot more about systems thinking. Mm. Right?
26:14
About, like, where, where are the sinks, where are the drains, where, you know, how do things move through a system, right?
26:23
And I think that sort of overview thinking and that ability to, to spot where things are gonna have, like, challenges, and then you use software to, to fix those challenges, to implement those- Yeah...
26:36
to, to tweak it to make sure that your system works. So I think it's gonna be a lot less about understanding a design pattern, right, in, in the way that we learned about them- Yeah...
26:46
right about, like, some singletons or et cetera. Yeah. But it's gonna be thinking about design patterns about how do system components work- Mm... and how do they integrate with the rest of the organization. Yeah. Right?
26:57
And how do they integrate with messy humans. Yeah. Yeah, ex- and messy work, and messy processes- Yeah... and people who misprocess.
27:04
Yeah, the analogy I've been thinking of more is, not to tie it back to another automotive res- reference, uh, but the, like, looking at how these cars are being built in China, these production factories, no one is, like, sitting screwing, like, bolts in cars anymore.
27:15
No. They're all operating the robots that are building- Yeah... the cars. Yeah. And, and, and much like- That's exactly it...
27:20
and much like, like, the, the, the, w- you know, the, it's such a perfect metaphor of, like, how much quicker it is to build code. These Chinese, like, car factories are building a car a minute- Yeah...
27:29
are coming off the lots. Yeah. Uh, like, could you imagine a world in any point that where you'd be like, you're gonna assemble a car a minute. Yeah. You would need half a, half a country to, like- Yeah...
27:37
build that much stuff. Um, so that to me is like, it's, it's taking that knowledge of the people again. There's probably this generation of people who built cars by hand, or assembly line workers- Yeah...
27:47
who have helped train the robots, but th- those robots still need management and optimization of processes- Yeah... and, like, make sure the wheel- They do... doesn't go on backwards, you know? Everything, yeah.
27:55
Um, that- Yeah... it's an evolving process- Exactly... and, and learning from that. Yeah. So it's a, I think we're, like, behind the automotive industry in that sense of- Yeah...
28:02
like, that's where I see the utopia of, uh, product engineering, like, those ideas can- Yeah... proliferate out. Exactly. What do you, what have s- teams stalled on the most going to the change management part of it? Mm.
28:11
Where do they get stuck when you're sort of working with them and seen the amazing demo, they know where they're trying to get to. Mm-hmm.
28:16
What are the sort of biggest hurdles as you're sort of thinking about rolling out? Hmm.
28:21
I, you know, there's this, this saying that like, you know, all happy families are alike and all unhappy families are unhappy in their own way. Yeah.
28:30
I don't think there's a single way that, that, that, you know, teams have been having challenges or anything like that, right? I think it's often almost on an individual basis, and
28:41
different people will have different challenges with it. Maybe they have, like, some existential angst about this, right? And so therefore they're, they're resisting.
28:50
Maybe they, again, they worked with it in the past and it's not working. Maybe their setup is wrong and they just keep getting the wrong output, right?
28:57
And I think for this sort of change management and this sort of thing, it's easy to want to try to, like, bring people in a big room, do the thing, show them the thing, and then go off and do it, right? Yeah.
29:09
And that, that's not gonna work. Some big, some big bang in the ceremony and yeah. Yeah, exactly. Right? I mean, you're just like, "Voila. We've now, I've now-" If here is the new... Yeah, that's right. Yeah.
29:16
I've, I'm, come down from the mountain with the tablets. That's right. And- It's working, yeah. Yeah. Yeah. But what we've found is that it requires one-on-one work. Yeah.
29:25
Even small groups, people don't, like, don't wanna talk about the challenges or, you know, et cetera. Yeah. And those one-on-one sessions with people where we, we take their exact problem and we work through it, right?
29:37
And we walk them through the thinking of how to work through it, right? Yeah. And we, we have them do the work, right? We're like, "Okay, here's how you can do it. Like, try this. Try..." You know?
29:46
"Okay, what do you think? Okay, let's poke at it a bit more." That's what actually the change management is. Yeah. And I think,
29:52
and I think it's the, the part that people want to cut or they assume that we'll get them to do it on their own and they will learn it on their own, et cetera.
30:00
I mean, that's like the most fundamental in anything to do with, like- Yeah... digital transformations or- Yeah, yeah... even just building a website five years ago is the change management is such- Yeah...
30:08
a understated and so important to get it right. Yeah. Um, especially when people are like, yeah, their identity is tied up in all of this and they- Exactly... yeah.
30:15
And for a lot of people, part of it is almost a therapy session- Yeah, yeah... to talk about what their job now is. Yeah. Right? Part of it is to ensure that they feel that this is not gonna lose their job. Yeah.
30:29
To give them, to show them, "Oh, your job is changing." Yeah. "Your job is changing. There's still a job, and it's gonna be an even funner, in my view, role."
30:36
And your, and your skills are, like, way more in demand to be able to do this stuff well. Yeah. Yeah. Exactly, right?
30:40
Like, 'cause I think the people who will do it well, it, there's, there's, they're just gonna escalate in terms of- Yeah... like, their, their value and potential for, for organizations. Yeah.
30:50
When you're out selling the utopia to people- Mm-hmm... what, what do, um, what do leadership teams or, or, or whoever you're sort of talking to, what are they, what are the misconceptions?
30:58
'Cause I know there's this sort of- Mm... I've forgotten the name of the, the phenomenon, but, like, the further away you get away from the work- Mm-hmm...
31:04
the greater your expectations of, like, AI's impact, you know, CEOs who are LinkedIn all day. Yeah. This is gonna be 500% increase- Yeah... down to the guy who does the work- Yeah... it's like a 10% increase. Mm-hmm.
31:14
Um, like, so there's that, that sort of cognitive gap of- Mm... expectations. But yeah, what do people get wrong expectations or, or where it sort of takes the most time to bring them around? I think
31:26
it's that people sometimes assume that if you fix the engineering part, then things are gonna be faster. Mm-hmm. But like I know some engineers, right? They're, they're just idling. Mm. Right?
31:40
Half the day now is just idling. Right. Right? They're just waiting for things, or there's nothing to do. There's, you know... And, and I think it's like y- they didn't think about how they need...
31:52
They're, they're, they just built a faster horse. Yeah. Right? Yeah. The, the horse is faster now. Yeah. You still can only bring one person. You don't have a truck.
31:59
What do, what do I do, uh, at 11:00 AM when I've done all my jobs for the day? Yeah, yeah. I just go really fast. Yeah. Yeah. So and, and I think that's, like that's the mindset.
32:08
You need to think about it from an organizational point of view, right? Yeah. I think... And, and I...
32:12
Most of these tools that AI companies are building, or most of these frameworks, right, there's so many that, that empower engineers. There's Superpowers, there's GSD, there's Beam, the Spec Kit, et cetera.
32:22
They're all focused on individual- Mm... individual performance improvement. Yeah. Right? And, and you see it in the stats. Yeah. Right?
32:29
Like I don't have them with me, but the, the amount, the speed that people finish things- Right... has gone incredibly up. Yeah. But the time to PR review has gone up. Yeah. The number of bugs have gone up.
32:42
The, the amount of time, like mean time to revert, all of those things, they haven't improved, right? Mm-hmm. And you need to basically be thinking about how you bring in AI from an organizational lens- Yeah...
32:54
for it to actually take effect. I, I think that's the thing that people miss, is that they're giving people individual tools, and it doesn't solve the organizational problem, right? Yeah.
33:02
And that's why it's a pipeline and not just a tool one person uses. Yeah. When... How do you coach people through, I guess, the attachment to like human in the loop?
33:11
I think there's always a human in the loop, it just, you need to define what the loop is. Mm. And the loop, to me, keeps getting bigger and bigger.
33:20
I think once you start, right, when you get started, you're gonna have a human in the loop at basically every AI step. Yeah. Right? And then you're gonna have that human, right...
33:29
What people, I think, misunderstand from human in the loop is that they think that that human in the loop is sort of a verification gate, and that all it does is just says, "No, this is bad, and let me fix it." Yeah.
33:40
Right? That's not- The, the screwed up AI, yeah. Yeah. Yeah. But, but that's not it. The human in the loop fixes the AI. Mm. Right? It will be like, "Oh, that's not right. Let's tweak this." Yeah. Right?
33:51
Let's tweak that. Oh, okay, now it's right. Okay, now they've improved it. Mm. Right? So they should never be fixing things- Yeah... just themselves. Well, 'cause AI never learns. I mean- Exactly. It will never learn...
34:01
I'm not a teacher, and I would be a terrible teacher, but- Yeah... that's the problem. Like, you don't just go fix the kid's homework. No. You tell them how to do it- Exactly... themselves and think through it. Yeah.
34:08
Yeah. And that's, that's why, like, so every step you human in the loop until, oh, there's- Nothing to do... there's nothing more anymore. Yeah. We've, we've, we've, we've got it. Okay, so now we move to bigger loop.
34:18
Yeah. To a bigger loop, to a bigger loop. Yeah. Right? And, um- That's a really great distinction of separating those things out, yeah. Exactly, right?
34:26
Because I think people, they, they, they keep thinking like, "Oh, I'm reviewing this PR." It's like the hero, hero complex or something, and like, "Well, only I can fix the PR." Exactly. Yes, exactly.
34:33
And just- Only I am able- [laughs]... to do this. No human could... No, no robot could fix this- No... little bit of code. Yeah. Could possibly- Yeah. I could not possibly teach a P- a, a, a robot- Yeah, yeah, yeah...
34:42
certain rules to apply to do this, or a certain guideline. Same thing that'll happen 500 more times in my life. Yeah. Yeah. Exactly. Yeah.
34:49
Uh, open source obviously has a, a he- culture of, like sharing code and things like that. Mm-hmm. Um, and, and, and like internal, you know, companies, media companies- Mm... the big ones will open source their stuff.
34:58
Mm-hmm. Agencies, freelancers. Mm-hmm. Um, versus figuring out what is actually, yeah, I'm not gonna give you my, you know, uh, IP.MD file. Yeah. 'Cause that, that's where my brain lives, you know? Yeah.
35:09
I think it's a very good question. I think what is going to change in the near term is that sharing code is not gonna be enough, right? In that sharing this open source library, um, sure, that's, that's nice and all.
35:26
But what's gonna be really important is sharing the context, sharing the skills, sharing the what... so that this code gets used by an agent well. Mm-hmm.
35:37
And I think that's gonna make a difference between if you're just open sourcing things or if you're open sourcing them with the context, with the... Maybe you already have built-in reviewers inside- Mm...
35:48
your plugin that you ship. Yeah.
35:49
So that when someone wants to build functionality on top of it, there's a reviewer that knows how to review the functionality you've built, knows the like, oh, you know, uh, Codex, Codex Sol often makes this type of mistake, so I'm gonna make a check for it.
36:03
Or like, these models make that kind of... Right? List them all out, have the reviewer built into that.
36:07
And I think that's gonna make the difference in terms of like open sourcing things, is what's open sourced and what's open sourced with the functionality, with the context, with the reviewers, with all of that for that.
36:18
Yeah. And an interesting thing that we've been thinking about is building that for open source plugins. Mm. Right?
36:24
Having a repository where we have all of these context files and all of these skills for open source plugins. Yeah.
36:31
So that if someone wants to build on top of it, and they don't need to try to figure it out, they don't need to get the agent to try to figure it out.
36:36
Yeah, how many million people are gonna build the same sort of skills to do translation- Exactly. Yeah, yeah... or like some, something, something nice to have in the back of WordPress- Yeah...
36:43
or, or doing nuances or, or whatever- Yeah... correctly. Yeah. Um, but you also don't want, like these random snippets. You want it s- sort of packaged and- Yeah... con- um, contextualized. And I think it...
36:51
And I think you can do that for WordPress, with the WordPress skills, right? Yeah. My colleague Mikey's on the core AI team, right? Mm-hmm. And they do that. And we've committed many improvements to those skills. Mm.
37:00
Right? And those are the skills for core WordPress. Mm. But I think you need to do that exact same thing with your plugin, with your open source library. Yeah.
37:07
So it doesn't just ship with that, it ships with exactly how it works and how agents can use it. Yeah. If there's one word I could use to describe how I think things are now, it's eventful. Mm. Right?
37:21
And [laughs] that's gonna include some good things and some bad things. Yeah. So I think the, the good parts, right, is that there's a lot of potential right now. Maybe even since like Ajax. Yeah. [laughs] Like Web 2.0.
37:36
Web two, yeah. Web 2.0, and you could like- Yeah... get a jQuery like- That was, that was incredible. Yeah... you know, mind-blowing. Yeah. Yeah, yeah. When, when jQuery fixed- Yeah...
37:42
um, being able to do things in IE6 and Firefox and- Yeah, yeah, yeah... yeah. Yeah, I think there's a lot of like- It's this kind of data stuff... shared trauma ar- around that early days of the internet. Yeah.
37:50
But I think it's happening all over again, but just- Yeah... much faster and much different- Yeah... and a much bigger surface area.
