[Music] [Bea Krug] Hi, everyone. Thank you for joining today's session about Adobe Mix Modeler and Michael Kors' journey with it throughout the last year. My name is Bea Krug. I'm a Solutions Consultant at Adobe based in Germany, and I'm very happy that I have a partner for today's session.
[Manuel Neto] I'm Manuel Neto, and I'm a Global VP of Analytics and Data Science for Capri Holdings, which is where Sausage Mutant Michael Kors buy our bags.
Cool. So what will we talk about today? We will talk a little bit about today's challenges for marketers and how Adobe Mix Modeler can help with them as well as Michael Kors' journey with the tool.
So when I was four years old, my siblings and I thought it was a great idea to doodle all over our wallpaper in our room. And because my parents hated it, they made us tear it down and put up new wallpaper. And the question that they had was, what made us do it? So was it that we didn't have any papers left? Was it that we got this really cool new pens we wanted to use? Was it my sister who just wanted us to do some mischief? Or maybe it was just a mixture out of all of these different factors. But whatever it was in the end, all of these different factors together made us convert to wallpaper criminals. And that's very similar to the question marketers have nowadays. Right? What different touch points, what activities actually help our customers to convert? What make them take that purchasing decision in the end? And it's not an easy question to answer. Right? Marketers are struggling to prove the incremental impact of marketing activities and use that to then prioritize the right channels to drive business outcomes.
Marketing budgets are tight. We want to make sure that every marketing dollar we spend is spent wisely. And it's harder than ever because consumers nowadays have more than 70 core digital touch points. And the customer journey is so complex, and it spans across multiple channels from display to website to email. There's lots going on. And all of these different touch points obviously are associated with cost. And it's very hard to tell in which way these touch points actually contribute to the conversion in the end.
Because this is so complex, basic KPIs don't really provide a comprehensive picture of how marketing drives value. And marketers are struggling to ope-rationalize true measures of marketing success. If they want to do it, there's a lot of data involved. There's many stakeholders they need to involve. And often, companies are lacking the required resources or skills to actually provide meaningful insights. So, Manny, would you agree with this? Or do you have other challenges in mind that I might have forgotten here? I have many challenges in mind, but I would 100% agree. I think that's more and more, we're being demanded to deliver, with less budget potentially or staying flat year over year. But the way consumers are navigating today and the more different pathways they're taking before essentially they even consider purchasing from us or converting from us, it just makes it really, really much harder for us to, essentially, attribute those correctly. So having tools like yours, has made us a significant difference for us because we are able to get closer to the pathways, the consumer journey, and therefore, make sure our dollars are working a bit harder for us, to lead them to conversion as well. So that's been very exciting. Yeah. That's what we also found, or what different research found. Right, so marketers are struggling to prove the efficiency of their marketing activities. And they often turn to media mix modeling, to answer questions around the impact of the marketing activities, but they find it lacking. And they're looking for better or faster media, mixed modeling or for some kind of incremental lift testing. So, data is very important. Right? I'm sure you would agree with it. I know we talked about it a lot and what do you think of data and marketing, right, Manny? I would 100% agree. I think we've moved away from data's and nice to have to data has to be the foundation on to drive everything forward. And I'm a huge fan of the word data informed and data viewing decision making as in moving away from just data down to data decisions and then going ahead and making sure there's solid information to supply that. And that's essentially what we've been doing a lot with your tool as well. It's the enablement of, how is data being leveraged to amplify the understanding that we have of our customers, to amplify the understanding of what we have of their pathways, and then apply our strategies on top of it. So it's been quite transformational for us. Oh, that's great to hear. Yes. So we talked about, there's so many challenges to face, but luckily, we are still optimistic that we can face all these challenges. Right, If you just use the right tools, if you use AI, if you use data, all the things you've already mentioned, then we can actually go ahead and boost productivity to make sure our marketing activities are actually successful, even despite budget cuts. I know budgets are tied everywhere. Right? Would you agree with that? What kind of tools do you think are very relevant nowadays? Which tools do marketers need? I would agree. I would say that more than ever, we need to be closer to the consumer, not only from, high level data perspective such as who they are, what they buy. Those now are basic expectations. You need to know what they buy. You need to know where they buy and how they buy, but also understanding what are the touch points that they tend to engage with us the most, what are the channels they come to talk to us. And then tailor their their navigation, tailor their own customize really, their approach so that they feel even though it's a mass production, opportunity strategy, they're feeling individualized. They feel like they're receiving special love or special treatment. And that's when the innovative approaches come to mind when AI comes to play because it's no longer just enough to leverage what we know of. MMM, for example, that has been used for the last, say, 6, 7, 10 years. Yeah. It's really about how is MMM being applied or are there other elements that MMM is, being augmented with such as AI, and what are the modeling really the foundation of the models being created. And the really interesting we partnered really closely is which attracted me to you guys as well is the fact that, the way the model is being built and created, not always in partnership with us, but it also uses a much more advanced approach and which even entirely has required some educational effort for us because people were just not necessarily super familiar because it's very advanced. This is exciting tools that we need. No longer expecting the basic, get that and blow it up with AI and everything else available these days. Yeah. Yeah. We need innovative approaches. Right? What we used to do doesn't work anymore. When you think about how traditional workflows look like, when we look at traditional workflows to measure marketing success, they're often quite convoluted. Right? They're quite complex. If a marketer has a question, for example, "What's the ROI for this cross selling campaign that I did, for shoes for customers who usually just buy?" Let's say, handbags. Then usually, to answer such a question, the marketer has to go to the data science team. The request goes to some backlog because, obviously, teams are busy. They have so much other requests as well. So it takes a while for them to actually go ahead and do something about the request. They have to gather the data manually. They have to manually build a model, train a model, iterate on it. And then once they have an output, they can deliver that to the marketer, and the marketer can take the output to take a decision. But from the start of that initial request to the answer, there's a lot of time that passes. Right, and so by the time that the marketer actually gets the answers that they were looking for, the campaign might already be over, maybe the next campaign has already started. So the answers that they were looking for to actually improve and to optimize comes too late. And this is also why we at Adobe, created Adobe Mix Modeler. Right? This workflow was just not good enough anymore. It doesn't deliver meaningful insult, results in a timely manner. So we built a tool, Adobe Mix Modeler, that enables marketers to measure incremental impact of marketing activities and optimize planning, holistically across all channels. Paid, earned, owned, you name it, wherever you are. And with this tool, we are disrupting the measurement space. We are bringing AI-powered measurement and planning together in an interface that was purpose-built for marketers. And I really mean we are disrupting this space because we have automated so much of this process, of this workflow that I've showed before, from data gathering, and data harmonization to model prep, to exploring different scenarios, to taking decisions on budget allocation, campaign execution and measurement, and to optimize post quarter reviews. So there's a lot of goodness that we bring together. We automate and make sure that you save a lot of time. Right, so you get results not in months or even years to get your results really when you need them in a very timely manner. And how we do it is by our unified methodology that measures incrementally at both touch point and aggregate level, while we're also ensuring fully consistent results with it. So the tool is built on Adobe Experience Platform, where there's a lot of different applications as well, but we're using that same platform. And we're making sure that it is future-proofed by also measure each channel at the most best available granularity.
We are taking data from all across the place. Right? From online, from offline. We have customers bringing in data, from online channels like, Meta or Google, from connected TV, but also from offline channels like, radio, direct mail, linear TV. You name it. Whatever you think of, you can bring it, you can measure it. And we're making sure that everything is actionable. Right, that you have the data available at the right time to then take this to improve your decisioning.
How does it work? What does it look like a little bit under the hood? So we have created a proprietary and patent pending, methodology technology based on the needs of our own Adobe Marketing Team. So we're also our own best customer, let's say. And how it works is you bring in your data both on an aggregate level or an event level. So you can choose to bring in summary level or event level data. It's up to you, if you bring in either or both. You can bring in data from paid channels, from owned channels. And also very important, you can bring in data that is, just factors into your activities. Right, so internal factors like, sales data, HR data, or you can bring in also external factors like, the stock index or industry trends, this kind of information. And all of this aggregate level data goes into marketing mix modeler where a multivariate regression engine then determines the incremental impact of nonevent level data. But you can also bring in your event level data, right, from your walled gardens, from your website, whatever you have, and we bring it into a multi-touch attribution model where a supervised machine learning attribution engine then learns the incremental impact of event level channels by comparing all customer paths.
And we are unifying this bottom up and top down approach by, using this AI technology that we have built. And this means that the aggregate outputs from our marketing mix modeling model also inform the multi-touch attribution model and vice versa.
What you then get as a result is a complete view of all the incremental marketing channel performance. You can use this to then go ahead and plan for your activities. Right? You can develop. You can compare multiple budget plans to make them sure you take the right decisions, and you can take in flight optimizations because you have everything available, in real-time or as soon as you bring in the information and run your models, run your plans. You can use that data then to take the right decisions to optimize campaigns that are still running and, obviously, everything that comes in the future.
So there's a lot of good things you can do with it, and I know that Michael Kors has already started doing a lot of these good things. So I'm very happy to have Manny here today. And you can talk a little bit maybe about all the great results you had so far, all the experiences you had, and just walk us a little bit through your journey so far.
So it's been a very interesting yet rewarding journey. We've created, we've leveraged all of this magic you described on this screen and done, I think, the first outputs after 80 plus iterations, to again, get really closer to what's the best iteration, what's the best acceptable rate, and etcetera, has been really rewarding in getting us to essentially as closer to the journey of the consumer as possible. So it's been quite an interesting journey. We're finally going live, and I can't wait. I'm very excited.
Very nice. Do you want to talk us a little bit through, what you have learned so far from everything you've done? I think it falls into a couple of buckets. One of them being, what I call the shocking realities. And I've been knowing and that normally requires a couple of of sweets when we're having conversations and candy as we're having conversations. So it has been really eye-opening perspective of, "Hey. We knew this worked. We didn't realize this channel works as much as we thought it did. So we didn't realize how reliant we are on a particular journey." And so you find that knowledge for us. And then in other scenarios, it has been more like, "Wait. Hang on, I invest X amount of dollars in this particular location, this particular channel, and this is actually something that doesn't work anymore at all." And often hard realities, but always welcome realities because, again, it helps us do better, help us optimize better. But it has also helped us consolidate is what we've always thought existed is what we call synergies. So understanding that it's not looking at channel isolation, it's channel A plus channel B. It works much closer, much stronger together. And if you move investment from A to away or you put too too much on B, that synergy may break or maybe damage. So understanding how the synergies work has been really key for us because they help us understand how one channel or multiple channels impact each other and how it's important to have a balanced approach towards that. I think, ultimately, we're embarking. We're about to start a journey of change management. So leveraging all of that information to understand, hey, moving from an outdated attribution method to now embarking to this in a mixed model journey and soon a multi-touch journey, it requires a way to change the way we look at things, educate the orders on how that what that means for us, the new data is coming along, and then hopefully leading us to skyrocket the returns that we're hopefully will get. So very exciting times. It's been a change management journey, a realization journey. And now embarking to the final tests and learns then full on shortly. Yeah. So you mentioned a change process, change management from your experience, I guess it doesn't only apply to changes you have internally, but also changes you can have with partners you work with. Right? So could you maybe elaborate a little bit of, how this changes the way that you work with agencies and how that looks like for you today now that you've implemented yet another tool that you're using? So it's actually gotten as much closer if I could even say that because we're already close. But our agency's partner has been really great in the sense of-- Embark with us from understanding how the model was built. So which in itself turned into education process because what I was saying earlier, like, there's a lot of, this day, of things that you guys bring to the table, they are brand new, not only for us as a company, for the industry. So when you have to forget how things were made because there's a much stronger, faster way to do it, it does require some educational perspective, but they've been phenomenal in embracing their educational element of it and then taking it to apply. So it has a strength from the perspective of, here's how the cookie is being made, here's the ingredients of the cake. So we've been unpacking all of that in a lot of detail and lot, and then creating a road map, not only for implementation and testing, which we have done, which again got us even closer to understanding how we're going to be doing that, but also from a perspective of the educational. So it's been huge for us that the partners in the agency understand what kind of change management is required. But before we even start applying those change management, that there is an educational approach of, like, look, here's what KPI used to mean with this methodology that's a little bit outdated. Now here's what this KPI will start to mean with the new methodology that we're applying with MMM. So it's being really key to partner close together, not to your point, not only internally, but also to pretty much all partners, particularly our agency friends who are going to be, ope-rationalizing this a little bit more closely, but also from both educational as well as application perspective. And that'll show the results.
Yeah. So change seems to be the constant here, in the conversation because I know you've already done a lot. You have implemented Adobe Mix Modeler throughout the last year. There were changes that you've already started with, but I know you've got more planned, more lined up. So maybe, could you walk us a little bit through what you've planned for the future? What changes do you still have in store? So we're going globally very shortly. So that's very interesting. I think, we started with North America, and now that we have a very strong foundation, we're going to be going globally. We're going to be launching new ways and new models, and so we launched with model A, and we're going to be amplifying that with model B, C, and D, which is something fun we're able to do to as well as being able to compare the different ways of looking to model, which consequence amplifies the planning and the scenario planning.
We are also internally revisiting what success looks like. And so I think moving away from historical potential KPIs that, again, our eyes have been a little bit more opened, and we'll be hopefully continuously open as we refresh models, as we update models. But by consequence, that means, potentially, the way we're looking into historical KPIs may have to change or may have to have a different weighting toward them. So this is also helping us look into what does success really look like and what should we optimizing against, and so it's been quite fun. So a long way of saying, there is a global world domination.
We're going to start like a mad scientist going all over the world. We're going to be building more iterations, even though I said 80 plus the beginning. Yes, we're likely going to reach double that, which is exciting, and we're going to be doing new iterations, really. And then also, we have begun a process of, "Hey. It used to be yellow. Now it's going to be green," kind of conversations, and it's been quite exciting.
Okay. So you started out in the US. Is there something you expect to go differently in the EMEA markets? Yeah. So EMEA has a little bit of limitation to data due to the privacy laws. Right, so by consequence, we're going to have some of those challenges/opportunities because less data means the model has to work harder. So I'm excited for that. But I'm expecting or hoping we had some of the shocking revelations that we had here as well. And if we don't have them, I need to put them in. And it was getting. So I'm excited about them. I think we're going to learn that particularly because of the more privacy roles being a bit more aggressive over there. I think we're going to have a little bit more shocking truth in the sense of what really works versus what doesn't work because the dollars have to work 10 times harder over there because we don't know sometimes where there's a black hole of the audience who is receiving it. So this is definitely going to help shine some light to black hole. And then I'll sip some tea, and excited about seeing how they materializes for us.
Cool. Thank you so much for your time. We're almost at the end, but I would want you to leave our audience maybe with a couple of last tips and tricks.
So I'll give you one which I personally had to overcome is, don't be scared when things look a little bit too good to be true or when the unknown could create some uncomfortable scenarios. So I've done quickly that data sometimes is not all unicorns and fairies, but it's best to know the reality of what it's telling you, particularly in the market that we live today when market dollars have to work 10 times harder for you, then live with a fairy world that's not essentially real. So accept the uncomfortable, live with the uncomfortable and more than uncomfortable to become an opportunity for you, and build from there. It's been key for us. We've had some challenging conversations. We've had some hard conversations based on the data. But in the end, all these conversations have been fundamental for us to keep moving forward stronger than before. So good luck. And add me on LinkedIn if you need some extra content and help. Yay. Thank you so much for your time. It was a pleasure having this conversation with you. And I'd say for our audience, have fun at Adobe Summit. Thanks for having me. Bye. Bye, everyone. [Music]
