Episode 255: How Shopify Is Leveraging AI at Scale with Vanessa Lee
Join us in this episode as Melissa Perri sits down with Vanessa Lee, VP of Product at Shopify, to explore the groundbreaking work being done in AI integration and agentic commerce. Vanessa sheds light on Shopify's AI assistant, Sidekick, which aims to revolutionize how merchants manage and grow their businesses.
Discover how Shopify is pushing the boundaries of e-commerce with cutting-edge technology and a commitment to empowering entrepreneurs.
Through engaging anecdotes and insights, Vanessa shares how Sidekick is not just a tool but a strategic partner for merchants, helping them navigate the complexities of entrepreneurship. We delve into the collaboration with OpenAI and how Shopify is leveraging AI to create seamless, innovative shopping experiences.
Want to understand how AI can transform product management and e-commerce? Tune in to this episode for an enlightening conversation that explores the future of commerce and the evolving role of product managers.
You’ll hear us talk about:
05:19 - Shopify's Agentic AI Assistant
Vanessa explains how Sidekick, Shopify's AI assistant, was designed to be a co-pilot for merchants, proactively assisting with tasks like product creation and payment setup to help entrepreneurs succeed in their journey.
36:38 - Partnership with OpenAI
Discover how Shopify partnered with OpenAI to integrate commerce functionalities into ChatGPT, making merchants' products discoverable and enabling direct checkouts within the AI platform.
22:34 - Practical Advice for PMs Working with AI
Vanessa offers valuable advice for product managers looking to enhance their skills in generative AI and LLMs, emphasizing the importance of creativity and experimentation in developing innovative AI applications.
Episode resources:
Vanessa on LinkedIn: https://www.linkedin.com/in/vlaurenlee/
Check the localization report: https://lokalise.com/library/data-reports/localization-revenue-report-2025/
Check our courses: https://productinstitute.com/
Follow/Subscribe Now:
Facebook | Instagram | LinkedIn
Episode Transcript:
[00:00:00] Vanessa Lee: I'd say the beginning of this year, we were truly hill climbing, building an assistant, and for anyone out there who has tried to do this, it is the easiest thing to demo and the hardest thing to build in terms of production because when it comes to conversation. Like the user could ask anything.
[00:00:19] And it is so difficult to succeed and provide good answers across all the topics that could be asked. It's so easy to make a demo and so hard to get right. And so I think that's what we really learned. And one thing that was a big unlock for us which people don't normally get to see, and I think it's such a good demonstration for how creative PMs have to be in this day and age is how we do our evaluations.
[00:00:41] We'd create this judge that is like ever evolving, right? You're basically embodying all of your product intuition about how you want this agent to behave, and you're putting it in this LLM Judge.
[00:00:50] ~So, that LLM judge, that we just trained,~ we would take that and we would actually then give it to the entire development team. And so every time any developer sits down to build on Sidekick, and every time they say, okay, I iterate the prompt, I add a new tool or I'm testing a new model, they would then run it against a bunch of simulated conversations.
[00:01:07] And then they would have the LLM grade the conversations against that candidate build of sidekick. And so we do this every time anyone sits down to build on sidekick. ~We do this many times. ~We also run this LLM Judge on live conversations when we launched to see how is this trending based on what users are actually asking.
[00:01:22] So this ~LLM judge~ becomes the tool that you use to guide the team. ~It becomes even more ~it's so much more than a spec, ~which is so fascinating,~ because it becomes the measurement of whether or not we are achieving what we want as a product team in this agent, which is fascinating.
[00:01:33] PreRoll: Creating great products isn't just about product managers and their day to day interactions with developers. It's about how an organization supports products as a whole. The systems, the processes, and cultures in place that help companies deliver value to their customers. With the help of some boundary pushing guests and inspiration from your most pressing product questions, we'll dive into this system from every angle and help you find your way.
[00:02:02] Think like a great product leader. This is the product thinking podcast. Here's your host, Melissa Perri.
[00:02:11] Melissa Perri: Hello and welcome to another episode of the Product Thinking Podcast. Today I'm excited to have Vanessa Lee with us. Vanessa is the VP of product at Shopify, where she leads innovative efforts in agentic commerce and AI integration. I'm thrilled to dive into how Shopify is setting the pace in agentic commerce and what lessons we can learn from their journey. Welcome, Vanessa. It's great to have you here.
[00:02:31] Vanessa Lee: It is good to be here, Melissa. Thanks for having me!
[00:02:33] Melissa Perri: Yeah, I'm so excited about this topic. You guys have been really full-blown, going crazy at Shopify with really cool new products.
[00:02:41] Vanessa Lee: Yeah, we've been a bit loud about it from all respects so, this is a meaty topic to dive into for sure.
Vanessa's Career Journey
---
[00:02:46] Melissa Perri: I love it. So can you tell us a little bit about how did you become the VP of product at Shopify? What do you oversee and what was your journey to this moment?
[00:02:55] Vanessa Lee: Yeah. I've been a founder a couple times so, and I'm engineer by by education. And I did a couple startups. I worked at Microsoft for a little, like a hot minute. And then when I joined. Shopify. I actually joined as a senior product manager, and so I have seen every level of product management at Shopify.
[00:03:14] I worked on a few different things at Shopify. I started first building our developer platform and essentially an our developer ecosystem in 2017. And then I worked on our online store, which is what shopify is known for, it's our bread and butter. And then I worked also on a, some of our AI initiatives, so like things like sidekick and what we do with generative ai, like media generation.
[00:03:39] And then so recently I've gotten a chance to pair with, to be honest, like most of Shopify, which has been really fun. And yeah, so it's been a journey. It's been almost nine years, which is wild.
[00:03:50] Melissa Perri: Wow. That's awesome. And then so what have been like some of the key moments when you look at Shopify's journey for you that define how you think about product management?
[00:04:02] Vanessa Lee: Oh, I feel like product management is different in every organization. It's probably the one craft that swings wildly depending on the culture of the, of the company. Us in particular, I'd say also a very specific flavor of product management, but one that I personally couldn't imagine not working in, so we are a very in the details product management culture. Like I think it flows from Toby just all the way through our entire company. We are the type of product managers who just absolutely love building and being in the details. And our team is actually quite light when you look at ratios between PM and engineering.
[00:04:43] So I think that kind of sets the tone for how much we expect PMs across the board, even all the way up to director. I have a bunch of product directors who are way more in the weeds than you would expect a product director to be. But I think if you love building and you're super curious Shopify's PM culture really speaks to those types of product managers.
[00:05:04] Melissa Perri: And you are working right now on one of the hot button topics, I feel
[00:05:09] Vanessa Lee: I feel like it's like the hot button topics of all of tech. Yes.
[00:05:12] Melissa Perri: So I feel like it's funny it's like almost a game now. It's every time you say AI, I do something,
[00:05:17] Vanessa Lee: Take a shot. Yeah.
Shopify’s Agentic AI Assistant
---
[00:05:19] Melissa Perri: But I know Shopify's been doing a lot of work around agentic AI. Can you tell us what you've been working on lately and what's making you really excited about this?
[00:05:27] Vanessa Lee: Yeah, we are the, Toby and Harley and folks like to say, we are the entrepreneurship company, which means our goal is to make sure that merchants, our merchants succeed. And we believe that there's more entrepreneurs in the world than than that there are out there today that are just waiting to find their passion.
[00:05:45] And so one of the things that we've really started out, and now I believe a couple years, we put out this essentially this vision for what became sidekick which is our AI assistant, to make sure that merchants have someone by their side to help them succeed. So if you think about AI in that respect, this is like a co-pilot or an assistant that you can chat with that knows Shopify better than any other assistant out there, knows your business better than any assistant out there and that knows, how to do things on your behalf. So like it's able to proactively create products and collections for you, help you set up your payment methods. Basically anytime there's a hurdle in entrepreneurship, we wanted sidekick to be there to help you through it, knowing how lonely entrepreneurship can be at times.
[00:06:35] And so that was probably our biggest focus. In the early days with ai, outside of just smaller projects that we had around, product description generation or image generation we really envisioned a lot more than just the one shot generation. We also envisioned this like assistant this entity besides you.
[00:06:54] And yeah, sidekick really kicked it off. And then I'd say lately there, there's been a lot of advancements that we've, we've seen over how sidekick can go also beyond just someone you talk to and now moving into an assistant that is a lot more proactive that folks turn to in a lot more of a general sense versus just creating products, creating collections.
[00:07:14] So I say a lot of it in the early days were, was around our soul, our heart and soul as a company, which is how do we make more entrepreneurs successful?
[00:07:22] Melissa Perri: And with Sidekick, I think people could understand it a little bit and think maybe it's just like a chat bot answering but it goes beyond that. What are some. Use cases people will turn to Sidekick for, and how does it work?
[00:07:35] Vanessa Lee: I think the best well sidekick is just embedded inside of Shopify. And I think one of the things that we're still working on is really how sidekick will evolve over the next couple years in terms of how it changes the UI dynamic on how's someone you know, using Shopify day in and day out to run their business experiencing sidekick.
[00:07:55] But to be honest, sidekick is actually, from a UI perspective, pretty simple today. It really is just embedded inside the ED admin. But what was magical is that over the last couple years is in the last, I'd say the last, 2025 in particular we saw Sidekick really take off, and I think that's a mixture of two things.
[00:08:12] So one, we have a audience, a merchant base who is super hungry for this companion. Entrepreneurship is a very lonely journey. It's not easy. The cards are often stacked against you. And so we had people who were just really hungry for help. And so that's that was one piece was we had just this user base who was ready for sidekick to come and help them.
[00:08:36] And then we also just hit it in terms of foundations, sidekick really improved in terms of answers and what it was able to do. So sidekick's able to do things that you would expect it to be able to do. In Shopify, things like creating a product, creating a collection, creating a discount for you, helping you navigating Shopify's admin easier. But now we've started to see people really leverage sidekick for, I'd say like higher order tasks. Things like, Hey, tell me about my sales over the last year. What trends are you seeing? What should I be doing in the next year to help me grow? What's an attainable goal that you would see for my type of business in my industry considering, I had x number of dollars of revenue in the last month?
[00:09:22] And I thought there was like one really key conversation that's such a good example. There was this like merchant who just went back and forth with sidekick on like a Friday night, a hundred over a hundred times.
[00:09:35] And you really saw them start to like just even talk to sidekick, truly as if it was another member of their team. And I think that was incredible because it meant sidekick for so long, we had worked on the foundations for a long time and we finally broke through as to, okay, now we're really providing useful answers for our merchants.
[00:09:54] And that journey was not an easy one. But yeah, I'd say like merchants. Use us a lot for higher order tasks, which was also a surprise. It's like anything like a data analyst an advisor, a financial advisor understanding what marketing tactics they should explore. I'd say that those are some of the surprising, most useful use cases for sidekick.
[00:10:17] Melissa Perri: I love that it's basically an e-commerce expert
[00:10:20] Vanessa Lee: Yes.
[00:10:21] Melissa Perri: embedded into the product, which is really changes the user experience for people, right? Who are not, let's say e-commerce expert. They're small business owners that know their brand or, design clothes, but they're like, how do I actually sell this?
[00:10:33] So I love that. You're giving them that whole experience in there too.
[00:10:36] Vanessa Lee: Yeah. Yeah. It's it's been such an amazing, probably one of the most rewarding products I've had a chance to work with to build in my career. Like even just watching merchants, they're like, yeah, I just, I wanna talk to sidekick all the time. And the fact that sidekick just knows so much about their business is I think what unlocks a lot of that versus any other assistant that they could reach for that is around the internet.
[00:11:01] Melissa Perri: Yeah, and it has all the data. So how did you. did you come up with Sidekick? Tell us a story about like, how you built this, how it got started, what that process was like.
How Sidekick Was Built
---
[00:11:12] Vanessa Lee: Yeah, so this is, if you go back, you'll see Toby's original post, I think this was, I can't remember the exact date, but I think it was in 2023. And he really set the aspirations for what we wanted this to be. I think like originally we, even in Toby's video, I don't think you see exactly where we're going, but he already painted the picture pretty far into the future at that point.
[00:11:36] So sidekick really kicked off then. I think once we saw LLMs start to kick off, chat gpt, there was a moment I think that everyone who was working on, especially SaaS based software knew, okay, this is a, this is a moment to pay attention to, and I'd say that was like. I'd say at that year, 2023, we were all just finding our way.
[00:11:59] Like at that point 2024, we started to be like, okay, you start to see companies truly come out and say, okay, we're gonna start to have assistance and assistance inside of our platforms, and we're gonna give that a go. And so really for a while, I'd say, since Toby sent his video out.
[00:12:15] And up until even, I'd say the beginning of this year, we were truly hill climbing, building an assistant, and for anyone out there who has tried to do this, it is the easiest thing to demo and the hardest thing to build in terms of production because when it comes to conversation. Like the user could ask anything.
[00:12:39] And it is so difficult to succeed and provide good answers across all the topics that could be asked. It's so easy to make a demo and so hard to get right. And so I think that's what we really learned. And one thing that was a big unlock for us which people don't normally get to see, and I think it's such a good demonstration for how creative PMs have to be in this day and age is how we do our evaluations.
[00:13:04] So when you're building an assistant it's such a intricate system of what models am I using? What context am I providing the model? Are we using several models who are, that are working together? What data is feeding that model? Is it a fine tune model? Is it a model that you're getting off the shelf?
[00:13:24] And then also how does it integrate with the platform? There's so many variables in that. And every time you change any of those variables, which your developers are doing all the time, every time they do a pr for a new iteration on the prompt, they are changing a variable. How do you know that it's not just like, how do you test it, right?
[00:13:43] When we build software, up until this point, you have unit tests. You would you have your set of tests that you know, okay, it works. How do you test when it's not? It's not a deterministic product. And so what we did was we also had the cold start problem. Everyone who build an, who builds an assistant has that problem where they just don't have the data yet.
[00:14:04] So we had no data, we had no example conversations. And so one thing that we did, which I think is so ingenious and I can, I definitely can't claim credit, the team really did this. I watched folks build a simulator for a merchant.
[00:14:17] Melissa Perri: Oh.
[00:14:18] Vanessa Lee: We would take all these topics and basically ask LLMs to help us generate all of these topics of, give us like... a thousand questions that merchants might ask in their journey. And we would do it over and over again for different verticals of merchants, different maturity of a merchant, a scaling merchant of a merchant who's just starting. And then we'd take all of those topics, and then we'd feed it to another LLM that we have prompted to be a merchant.
[00:14:46] So we said you are a merchant. Yes, you are a merchant. You are a brand new merchant. You are working out of your garage. You are aged, 30 years old. You like answers that are simple. You goal is to build your, I don't know, your your lip gloss business out of your garage. And then we would do, we'd do many merchant profiles.
[00:15:06] We'd give them all of these corresponding topics and then we'd have them talk to a candidate of sidekick, like our first build to sidekick. And we'd get all of this synthetic conversation. We'd get like these LLMs talking to LLMs, which is also hilarious. 'cause you'd be surprised watching LLMs talk to each other.
[00:15:24] They're just. So willing to please that like they just kept going on and on. So there's a lot of work you have to do to actually get that to simulate, to be a real topic like humans do. But we get these, we got to a point where we'd be able to have these thousands upon thousands of synthetic conversations.
[00:15:40] And then we would take them and we would actually start to mark them as a group of humans. So they're like, let's say we had five humans and PMs do this all the time. I think this is like one of those things that PMs don't realize when they build these products. But the way you actually spec out what you want your agent to do is actually through this grading process.
[00:16:00] Because you said, Hey, this conversation, the agent went sidekick, went too long did not answer the question, and created a product instead of creating a collection. Which is what would've been the right answer. I'll grade this. and listen, Melissa, you're also a PM on the team. You would also grade the same conversation and we would say, okay, what we care about for this product is like on five dimensions.
[00:16:21] It's did it serve the merchant's goals? Did it follow our safety standards? Did it actually use the right tools? And was it grounded in the right way? Did it have the right tone? And so we would define all of these dimensions, then me and you would grade this particular conversation.
[00:16:37] And funny enough, even humans don't agree with each other all the time. Okay? And so we would take this overlap and we'd say, okay Melissa and I on average we have a agreement overlap of about 60%, which means 60% of the time we would mark the same conversation the same way across these dimensions.
[00:16:56] And then we take that and we say, okay, we're going to build an LLM Judge. So this is like not proprietary. I think this is just fascinating 'cause it's, if you read papers out there, you will find this. This is not, we are not the only ones to do this, but I don't think people tell this story enough.
[00:17:11] We take an LLM judge and we actually say: Can we prompt this judge so that it would also overlap at 60%? We want it to be able to slot into the team and be like, Vanessa's not here today, so you know, this LLM is gonna judge and the way that they judge basically simulates Vanessa enough. So once you have this LLM Judge tuned means you prompted, you iterated on it and you've ran it a couple times a against all of the same markings, and it turned out it would grade it the same way me and you would.
[00:17:40] With the right overlap, we call it Cone Kappa. And then from there we would say, okay, now we have our judge. That process that I just called to you, we do that all the time. So every time there is a new feature in Shopify, every time there, we believe that actually let's revisit. We don't think that merchants are starting to change their behavior with sidekick.
[00:18:00] And we need to start to create essentially new ground truth sets for our judge because, people are asking now about marketing tactics. We haven't graded marketing tactics before. So me and you would do that again, and then we'd create this judge that is like ever evolving, right? So this is almost the new version of a spec, when you think about pm. You're basically embodying all of your product intuition about what you want this, how you want this agent to behave, and you're putting it in this LLM Judge. ~And what, go ahead. Go ahead. Yeah, I know I'm talking a long time 'cause I could go on for this forever.~
[00:18:31] Melissa Perri: I think it's so fascinating 'cause you're getting to what I was gonna ask about this, which is. It, it's a different tool set and a different way of working, I think with PMs for PMs, to work with more generative LLMs and agents training agents. Tell us a little bit about, now it's not a spec, is it evals? Like what do we have to learn?
[00:18:50] Vanessa Lee: Yes.
[00:18:51] Melissa Perri: What are we doing here? With that a different tool set. Yeah.
[00:18:54] Vanessa Lee: So, that LLM judge, that we just trained, we would take that and we would actually then give it to the entire development team. And so every time any developer sits down to build on Sidekick, and every time they say, okay, I iterate the prompt, I add a new tool or I'm testing a new model, they would then run it against a bunch of simulated conversations.
[00:19:17] And then they would have the LLM grade, the conversations against that candidate build of sidekick. And so we do this every time anyone sits down to build on sidekick. We do this many times. We also run this LLM Judge on live conversations when we launched to see how is this trending based on what users are actually asking.
[00:19:36] So this LLM judge becomes the tool that you use to guide the team. It becomes even more it's so much more than a spec, which is so fascinating, because it becomes the measurement of whether or not we are achieving what we want as a product team in this agent, which is fascinating.
[00:19:53] Melissa Perri: Yeah. That's really cool. So it's pretty much like designing the agent, like how you want it to think.
[00:20:00] Vanessa Lee: Yes.
[00:20:01] Melissa Perri: And then embedding that into it, and then it's basically you make the product for it though, too.
[00:20:06] Vanessa Lee: Yeah, you're basically imparting all of your product intuition into an LLM to then shape yet another LLM, which is so mind-boggling. But this is the type of creativity that I think when teams truly hit an acceleration with AI or building AI products, this is the type of creativity you need. You do not find data just lying around. You do not manually okay, let me test this over and over again. That's not gonna scale. And so I think that this is one of the fascinating stories that I love telling because it's just such a amazing story of ingenuity.
[00:20:44] Melissa Perri: Yeah, it's a, it's really cool. It's extremely creative about how you handle that lack of data that we were talking about at the beginning. And I get that question from newer PMs or people launching products so much where they're saying, Hey, if I have no data, how do I even start this? How do I look at it?
[00:21:00] And you guys figured out a whole system on how to generate it.
[00:21:04] Vanessa Lee: And I think everyone should set it that way. LLMs, they have so much to offer in terms of capabilities. I know we are all used to chatting with agents and using them to help us learn faster. And I think that's. those are all great applications of ai and we definitely do that a lot internally where we expect, gut check, sense check ideas with with agents, especially as they've gotten better, especially this year.
[00:21:27] But I also think that LLMs we often forget are useful tools in creating data and cleaning data. And that's what with how the team has built sidekick and probably at this point, like this is not, this is tribal knowledge, I think for the teams who have built agents at scale. But I think for people who are just getting started, you don't get to see that every day. And I think that is that's such a valuable lesson.
[00:21:50] Melissa Perri: I am excited to share that I'll be joining Product Weekend in New York City on November 14th, an incredible event powered by Lokalise.
[00:21:57] In the last edition of May, I had the chance to meet a few people from the Lokalised team and some of their clients, and it completely changed how I think about localization.
[00:22:05] It's not just about translation. It can be a real growth driver. In fact, some companies attribute up to half of their revenue growth to localization. Lokalise helps teams do this at scale with over 3000 companies using their AI powered localization platform to speed up translation, ensure quality, and deliver better experiences worldwide.
[00:22:25] Head over to lokalise.com to learn more and check out their localization revenue report to understand more about how localization can help scale your business.
Practical Advice for PMs Working with AI
---
[00:22:34] Melissa Perri: For people out there who are, trying to get better at doing product management when it comes to generative AI and using LLMs, what types of new skill sets would you advise them to look into to be able to do these things correctly? Like, where do they start? What should they learn?
[00:22:51] Vanessa Lee: Yeah. To be honest this part is, is hard. I think these stories and why I love telling this one, I tell it internally at Shopify all the time because to be honest, even at a company as large as Shopify this knowledge that I'm telling you and telling you is not uniform. But I think the biggest advice I have is definitely play. Everyone plays with AI. I feel like that's not a new thing. But also see how you can use AI in more creative scenarios. I think that's when you start to just get better ideas. So like we started using AI even before sidekick. We went through that loop. Simultaneously I had another team who was building our product taxonomy, which is a different application of AI where we take LLMs and different ML models and we try to basically get a better idea of the products that merchants are selling so we can help them find the right channels to sell those products. And every product is very different and every merchant, holds their product data in Shopify differently as well. So sometimes you'd be like, okay, is this a garden hose or is this a lipstick?
[00:23:57] It's funny because we were able to use LLMs to give us product attributes and suggest it to merchants to be like, alright, it's a garden hose. We can guess from the description and the title and the image, but also garden hoses should typically have a length and it should have a color, and it should have, these are all the attributes that you should fill in so that it can actually be sold properly when you connect it to marketplaces, SEO, et cetera.
[00:24:22] And we used LLMs also in that respect to help merchants basically accurately label all of the attributes of their product, which now have also helped merchants sell more accurately when they go to connect to other channels. So these things where you're like these, this is not like a normal application.
[00:24:40] I'm not just chatting with an LLM, I'm using it on a specific goal. I think once you get that muscle going, then you'll draw a lot more creative conclusions on other applications you could use it for. But I think you just gotta start.
[00:24:52] Melissa Perri: Yeah. You have to see what it's capable of doing.
[00:24:54] Vanessa Lee: Yeah.
[00:24:55] Melissa Perri: I guess there's the limitations part of it too, that we talk about with generative AI and LLMs. Is there anything that you would call out to say Hey, this is not well suited for it, or Here's some limitations that we found that we try to steer clear of.
[00:25:08] Vanessa Lee: To be honest. We use LLMs, so ubiquitously across Shopify. I like, I'm trying to think of an application that wouldn't be good. I think that's actually funny enough that's more challenging nowadays than finding applications which AI and LLMs are good for. To the extent where I'm gonna have to go back to you 'cause I actually don't know what that would be. I think everything in Shopify has been completely redefined and retouched in some way or other by LLMs and AI.
[00:25:40] Melissa Perri: Oh, that's cool. That's also saying like they're getting better and they're super smart at this point to, to lean on them.
[00:25:47] Vanessa Lee: Yeah I, I recently had to onboard very quickly to our inventory system, for example. And I wanted to be helpful to them. And there's this funny story where I'm sure Toby would be fine if I told everyone this, but me and him were both going back and forth on the design of Shopify's inventory system and how do we make that better?
[00:26:06] It's a really important part of Shopify, and we started passing each other essentially like chats that we had with AI about okay, what if it worked like this? And if we had virtual inventory groups and then we had a ledger and we actually use LLMs quite a lot in how we learn and how quickly we can ramp up in areas that we, we either want a deeper understanding of or we could be new to. So I think that's it's a fun story. Everyone uses it. There's no ego in Shopify about not using AI to help us do our jobs. That's for sure.
[00:26:38] Melissa Perri: cool. That's really cool. And you're operationalizing AI at a massive scale at Shopify, if it's touching everything, you're, you have it across the platform. What do you think, it's such a complex challenge. What are, what gives Shopify a unique edge in overcoming doing this at scale? Being able to actually implement AI across such a large platform.
[00:27:03] Vanessa Lee: ~I think our ethos, we've. ~I think thanks to just a lot of curiosity from Toby and from a lot of folks here, we've been willing to be cutting edge on stuff that may or may not have been ready. We were super early on AR VR we were super bullish on QR codes, before COVID. And I think that even on things like stablecoin and then now AI, I think there's ~just a big belief. ~There's a big belief that if we are on the cutting edge, then our merchants will also be on the cutting edge. And so we're always bullish on okay, let's do a 3D model and make it possible for someone to upload a 3D model in their online store so that you could see the details of the stroller. Like I remember doing that and that was way before I think merchants even understood like, how do I get a 3D model, of my products made. But I think we always trusted that if we did our jobs and made it easy on our side of the platform, that over time a lot of these bets will pay off.
[00:27:58] I think AI was just one of those where very early we could tell, okay, this is a bet. Oh this is. More than this bet is very likely to hit. And I, and then I think we just really encouraged everyone to play with it and to use it and to not be afraid of it. And I think that comes all the way, from Toby all the way down.
[00:28:18] And I think that's really changed the way that we use AI and then how we build with AI as well.
Democratizing Agentic Commerce
---
[00:28:24] Melissa Perri: When you think about the vision and the future for Shopify, and especially when it comes to age agentic commerce,
[00:28:31] How do you see that playing out? Can you tell us a little bit about what agentic commerce actually means as well and what. Where do you think this is, this is going in the future.
[00:28:42] Vanessa Lee: Yeah I think, if you Google Agentic commerce, you'll come up with many results. So agentic commerce I think was is really like this big umbrella term about how are we going to change shopping in the advent of agents. And it goes everywhere from how are we gonna change how we discover products to buy, and how we live our lives. For example the ones that I'm personally excited about are things like, helping people spend less time searching. I wanna be able to say, Hey, it's my five-year-old's birthday, this year. Help me plan it and help me procure everything.
[00:29:14] Okay, great. That looks good. Purchase. And that just saves so much time. And so I think that there's just this advent of okay, there is an assistant that is helping you with all aspects of your life, including the parts that perhaps require a bridge into physical goods, right? Purchasing things.
[00:29:33] How do, how does AI play in that space? This is something that we've been excited about for a long time to be honest. 'cause we do think that is humans are anything but very smart and lazy beings, which is what makes us great tool makers. And so I, we trusted that this would happen. So we started planning for this a while ago, a couple years ago.
[00:29:52] Even with the taxonomy, the product taxonomy work that we, that I told you about, like understanding what products are what, so that we could. Serve up to an agent. Okay? This is a garden hose. It's good. It has great reviews. It's green, it's eight feet, right? If we could understand our merchant's products better, then we could better help integrate them in a future, not even that far in the future, where agents we're helping to actually purchase on behalf of users.
[00:30:19] So there's the discovery portion of this, and then there's the actual execution and checkout. We have one of the most amazing checkout infrastructures. I think that has been our bread and butter. We've had some of the largest flash sellers in the world come to Shopify because of what we can handle at scale. And then also we're the only checkout platform that have millions of merchants on a single platform.
[00:30:45] Melissa Perri: So cool. I like I love that I can just go to a random clothing website or something. I'm like, oh, let me check out. And it's do you wanna log into your shop account? And I'm like, yes. I don't wanna create 18,000 other accounts. That's amazing. I remember the first time I saw it, I was like, this is brilliant.
[00:31:02] Vanessa Lee: Yeah, it's so good. And funny story inside story that was a hackday project inside of Shopify a long time ago, which was clearly go like a great idea that has then just absolutely just exceeded all of our expectations. It's been such a great thing for merchants to, to have that and such a great, like everyone that I talked to is oh, I love. Like logging with Shop, shop checkout is the best. So we have this great checkout, but checkout is so much more than just a payment, like when you log in with shop, you don't need to put in your payment credentials, you have your address field. But our checkout, what has been so fascinating is over the last half decade we've invested in making checkout modular.
[00:31:45] So our checkout, if you are a merchant, let's say you're someone really large, I have a lot of GMV, I'm all birds. I can use Shopify's checkout and I can customize it. I can say, you know what? I need to show you a like, you know, in Canada right now, we have a strike on our postal service. I need to tell you, I need to tell all my Canadian customers, Hey, there might be delays in shipping, or our shipping costs are different, or whatever.
[00:32:09] So this is Allbirds all the way to a furniture store where they need to do a white glove delivery. And so they're going, Hey Melissa, when do you want this chair delivered? And so we have all of these extension points, we call extension points in our checkout. But what's great with us investing in that for the last five years is now we're at this point where our checkout is a single platform that works across millions of businesses that in a dime, and we launched this in August as a checkout sheet. You can integrate into your agent and it can come up and be styled to be, let's say your background is pink in your agent. You can style it to feel native to your agent and it will work across millions of businesses.
[00:32:50] And I think that is just something that is like incredible that people don't normally think about when they think of agentic is like, how do you check out? And so that's something we've invested in for the last half decade. So it's like everything from discovery all the way to checkout I think is like something that you'll start to see more agents adopt over the next couple years.
[00:33:08] Melissa Perri: And then when you think about AI too, to commerce especially, with small brands compared to like large global brands, how do you think it differs? How does your product philosophy differ to serve the one person entrepreneur versus the large global brands like we're talking about Allbirds, like huge
[00:33:26] Vanessa Lee: Yeah.
[00:33:26] Melissa Perri: huge sneaker company.
[00:33:27] Vanessa Lee: Yeah I think that there are, I think the one thing that we always try and do at Shopify is we always try and make sure that we are democratizing access to customers and helping all entrepreneurs succeed. Whether it's a small merchant or a really large merchant, I do see differences in terms of how they reach customers and consumers.
[00:33:45] We wanna make sure that's pretty uniform, so we're gonna do our job to make sure that even the two person, company trying to build something is going to show up in all the right places, no matter what channels happen. We had social media on the rise and now we have agentic. This is new and not new at the same time for us. We've seen so many rises in, search being the biggest thing when we first came out as a platform in like 2010 being the primary way, then social media. And so this is just the next wave and every time we try and make sure that we're at the forefront. We see a large difference in terms of how AI is leveraged to be honest by those folks. Which has been fascinating. Yeah. I think that there's just this scrappy resilience that smaller entrepreneurs have, which is fascinating. And so they actually are some of the most hungry users of our AI tools. So like they're generating media using sidekick for their ads, they're generating media to help them with their product images.
[00:34:47] They're super into leveraging sidekick to change all of their SEO tags to be more optimized. And so you see them use it a lot. For larger merchants, I think it's a lot more like what that specific person is doing. So this is like internal to the organization. A marketer will use sidekick for their specific job. Okay, tell me about the data of the store. But we actually see more uptick, I'd say a lot of uptick in smaller merchants. And then when it comes to really large merchants, it's really on specific tasks, right? Like specific tasks around data at. Analysis and trying to understand, what should my marketing tactic be coming up?
[00:35:25] So they're different. I think there's more holistic usage coming from smaller merchants and for larger merchants you just see it more in okay, they have a very particular role that they're using sidekick for.
[00:35:36] Melissa Perri: That's fascinating. Do you think that has to do with, the, you were talking about the scrappy nature of the entrepreneurs, but. They don't have a whole team around it. And do you think it's more about like the specific roles that people play in larger companies. Where it's like, how do I use this to help me rather than how do I look at it globally?
[00:35:52] Vanessa Lee: A hundred percent. A hundred percent. Like that one entrepreneur needs to do all the things, right? Whereas that marketing manager is trying to do their best job in their role.
[00:36:02] Melissa Perri: In marketing.
[00:36:03] Vanessa Lee: In marketing. And the fascinating thing is how many, I'd say like executives at larger merchants, they have been using it more holistically.
[00:36:12] Melissa Perri: Interest.
[00:36:12] Vanessa Lee: Yeah. 'cause then they're able to oversee like a larger area. Okay. Like help. Help me brainstorm what what customer segments should I go after next quarter that we haven't tapped into yet. Build them for me. So there's, there is some stuff that we're seeing, but yeah, for sure the early entrepreneurs they are, they're a different breed of a person. They're just so stubborn and just scrappy and resourceful, and we see that in the way they use sidekick for sure.
[00:36:38] Melissa Perri: That's really cool. And then you have been partnering with Open AI's agentic Commerce Protocol. Can you talk a little bit about that partnership and what, how that came about, how you're using it?
[00:36:50] Vanessa Lee: Yeah, I mean we've worked closely with OpenAI for years, not just as, a partner, but also as a customer, to be frank, because we use a lot of their models. We use a lot of models from a lot of different partners, but they for sure are one of them. And last year actually we started talking about, okay, how do we get how to get product discovery in. And so earlier this year we started getting together to talk about, okay, how do we bring commerce to Chat GPT? And right. I think this is just announced we're super excited to start to roll this out. But essentially we partnered with them to make sure that our merchant's products are discoverable and that they're able to check out directly inside of Chat GPT.
[00:37:29] And so that, that's. Yeah, that's been super fun to, to work with 'em on. And they're, I think that commerce will start to take over, not just it's gonna become ingrained in every agent. And so I think you will start to see a lot of activity across the board. But we are super pumped to be working on this with OpenAI.
[00:37:46] Melissa Perri: When I saw the launch about this, I got really excited 'cause I was like, man, I am using chat GPT to brainstorm all these gift ideas and
[00:37:53] Vanessa Lee: Yes.
[00:37:54] Melissa Perri: then they're just linking out to it. I'm like, I just wanna buy it here. I don't wanna create like 18,000 accounts.
[00:37:59] Vanessa Lee: A hundred percent. Yeah. And I think that's part of why we decided, okay let's work together on this and let's start to work on how do we make that possible for so many consumers?
[00:38:07] Melissa Perri: Do you think that's also going to change the way that, that people think about marketing their shops as well, because now it's going to be like how do you show up in these searches? What, what do small business or merchants or anybody really large businesses too, have to think about now about how we talk about our products and what we offer and how they're marketed to different people.
[00:38:28] Vanessa Lee: I think that is probably the top question that I get. Not just about this particular integration with CHE EPT, but also at large I think merchants. Could smell this. Having gone through SEO then social media, like they are very, they're very in tune with where consumers are changing behavior.
[00:38:48] Yeah. I do think that will change. So a lot of that has been. How do we help merchants make sure that their product data is correct? 'cause a lot of them the models currently, they're just starting to get awareness on, okay what are these products that I can search for? Can I search for it accurately?
[00:39:05] Am I pulling the feet versus 10 feet? Because that's the type of nuance you get with agents, right? You're like, Hey I'm trying to set up a sprinkler system in my front yard. It's about 10 feet long. What should, what's a good garden hose for me? And it'll. Give you, it'll give you the exact one that you want.
[00:39:23] But in order for that to be the case, it needs to know that this product is eight feet, this product is 10 feet. And so we really started with the foundations there. It's like, how do we create a standard product taxonomy? Now from there, every merchant's gonna be like, how do I make sure that I stand out?
[00:39:36] And this is where we as a platform have tried to make sure that all of the stuff, product descriptions, reviews about products are all within Shopify so that it can then be served like in one shot, in one integration with all of these partners. We always say, I think one of the superpowers of being on Shopify is the fact that, we're millions of merchants and so we can go and talk about these partnerships with folks like OpenAI, Microsoft, all these, like all these partnerships that we've brokered over the years.
[00:40:06] Because we represent millions of merchants and it's one integration that they can build, and then all of these all of these businesses can get access to it. And so I do think that we will invest a lot more in, okay, how do we make sure that you continue to show up properly? And how do you want your brand to show up?
[00:40:22] We launched something in quietly called like the knowledge base app, where you can install that application. You can see what questions are being asked of your products, of your store, and you can provide grounding data so you can start to understand, okay, I wanna be like, I am, I do not have any tariffs or duties on my product because that's a question that I get asked a lot, so I wanna make sure that's clear.
[00:40:44] So we are experimenting, but I think we'll continue to see more of it in the next six months.
Future Trends in PM and Career Advice
---
[00:40:49] Melissa Perri: And when you think about the future of product management, what do you think are some emerging trends that are gonna have a significant impact on it and are like our whole discipline in the next couple years?
[00:40:59] Vanessa Lee: Oh, I think I am so excited to see where product management will be. Even inside Shopify, we started a vibe coding interview, pairing interview. Yes, because what I want product managers and Shopify to do, and I think it's so exciting for the industry at large all product managers, is we can now prototype stuff that we would typically have to rely on the dev team being free.
[00:41:28] And we can just tinker with it and we can understand how it feels. And that process, that feedback loop is so tight. And so I think that, that is such a superpower. Of course, like engineers, not all of this code is actually ready to be actually put into production. But even that alone, being able to gut check ideas is huge.
[00:41:47] And I expect PMs to do this all the time to be able to communicate better. PMs don't, it's fascinating, I always say this, we don't actually do any real work. Our work is actually in communication and helping to make sure that everyone is building the right thing at the right time. But that goes, building the right thing is not an easy thing to do.
[00:42:07] You can't just throw requirements on, there are a million ways to solve any problem. And in being able to do a prototype quickly, you can quickly see. Is it working out? Is it actually how I want it to feel? Especially for like very high interaction products where you wanna understand like, does this gonna, is this gonna feel right?
[00:42:24] You think about like the swiping gesture, that was introduced by, Tinder and others, and then now that's very native in a lot of different UIs. But that you don't you wouldn't come up with that without having this tactile experience and feedback loop. And I think that PM's now getting the ability to do that, independently and then now taking, okay, team, here's a more clear, crystallized view of what I'm thinking means that their teams can now be more effective when it bringing it to production. So I just expect this to be a huge force in function for us to move faster as a whole discipline.
[00:43:00] Melissa Perri: Yeah. I like, I am super excited about the whole vibe coding thing. I am worried because I think people are gonna put junk code into their systems.
[00:43:07] Vanessa Lee: Yes, do not merge your PRS without someone reviewing it for sure.
[00:43:11] Melissa Perri: But I love the fact that we can now prototype, test with the users, test with ourselves, just like communicate things to so much more easily instead of just waiting and trying to translate it through developers, and it's okay, we can just, look at it like you're saying and then see if it's feeling right.
[00:43:28] If it's not feeling right, get in front of people. Like it, it's huge to me that we can now do this.
[00:43:33] Vanessa Lee: Hundred percent It is. It is magic.
[00:43:36] Melissa Perri: Amazing. Vanessa, my last question for you, if you could give one piece of advice to your younger self, what would it be?
[00:43:42] Vanessa Lee: I think one of the things that I realized, and I wish I realized this earlier, is that not only just as a pm it's what led me to, I think, PM and staying in this role for this long is instead of chasing titles or, like particular companies, think about how you wanna spend your time and then go find the place where you're gonna get to do that.
[00:44:08] And so if you really enjoy collaborating, go find the place where you're gonna get to collaborate with people the most. If you enjoy building, then go find the place that you're gonna get hands-on building. I think sometimes I talk to a lot of PMs and PM has grown very quickly into quite, uh, and when I started at PM wasn't that much of a thing, it was just getting started. I think a lot of people coming into PM because it's starting to become something that is known to be a good job. But I think PM is actually quite difficult. For anyone who's been in the industry, it is a slog. It is 80% slog, 10% the best, 10% the worst.
[00:44:46] And, I think you have to know why you love it. So if you love talking to people, then okay, then maybe you're being, maybe you'd be in a PM environment where you're having to talk to a lot of customers. But I think it's how you spend your time because I think. More than anything PM is actually being a source of energy and courage to your teams.
[00:45:09] And I think to have that and to make sure you have a good career doing it, you have to know which environments you are gonna be able to supply that energy because you get the energy from essentially building. So I'd say have a really good idea of what gives you energy so that you can give it back to your teams.
[00:45:27] Melissa Perri: I think that's great advice and man, great advice for people out there too, thinking about, do I wanna work at a small company, a large company, like what types of company too because I've seen so many people like, let's say from enterprise companies go, oh, I really wanna work on a startup. And then they get in there and they're like, like that, this one. So it's honing in on those factors. I think that's fantastic advice.
[00:45:49] Thank you so much Vanessa, for joining us. If people wanna learn more about you and your work, can they go?
[00:45:54] Vanessa Lee: You can find me on X. I'm not there all the time, but I do look and I do pay attention. Yeah.
[00:45:59] Melissa Perri: Alright.
[00:46:00] Vanessa Lee: Thank you for having me.
[00:46:01] Melissa Perri: Yeah, thanks for joining us. So we'll put the links to reach out to Vanessa on X and also to Shopify and more information about their agentic commerce on our show notes at productthinkingpodcast.com. Thank you so much for listening to the Product Thinking Podcast. We'll be back next week ~with another amazing guest.~ And in the meantime, if you have any questions for me, go to dearmelissa.com and let me know what they are. I answer them every single week. And make sure that you like and subscribe so that you never miss an episode. We'll see you next time.