[Music] [Bob Conklin] Our session today, B2B Marketing Measurement: From AI to Ultimate. We're going to hit this topic of B2B marketing measurement from a bunch of different angles, and it's almost like a three in one combo session here.

I'm Bob Conklin, with the Product team at Adobe for B2B apps. So that's things like Marketo and Marketo Measure, Marketo Engage, our CDP B2B Edition, etcetera. I'm joined by Li Gao from our product organization as well, and Kimberly Galitz, Marketing Ops at F5.

And like I said, we're going to be taking you through this at a bunch of different altitudes. There's going to be something for everyone here. I promise that. So our agenda is-- I'm going to start off with a little bit just, kind of, level setting on attribution and B2B marketing measurement, where we've been, a little bit of where we're going. Li's going to come up, he's going to talk about some of our new innovations that we've been rolling out over the last year or so, and then Kimberly's going to wrap it up with best practices, you know, from a practitioner's perspective. Kimberly's also, you know, the leader of the user group for Marketo Measure, so brings quite a bit of experience. You may have seen her speak before. She does a lot of these types of sessions. So like I said, we're going to hit it from all of the angles. I think this is tagged as, kind of, an intermediate level session, but we'll have some, kind of, basic stuff. And then when Li's up here, we're going to go a little deep into the machine learning algorithms and things for those of you that want to know the details. We did a session last year and got a lot of pretty technical questions about how's the AI? How's this work? So we're going to try to explain as best we can, kind of, when we get to Li's section.

So start off with a little bit of history. What is marketing attribution? Start in the beginning. Well, it's the leading technique of measuring the impact of marketing tactics. I say especially in B2B 'cause in B2C you have other techniques that are, kind of, coming back into favor again. But this is about B2B. So we're going to focus on B2B. When I say impact, I mean not clicks, not lead creation, I mean, things like pipeline and revenue and the metrics that matter, ROI. When I say tactics, I mean campaigns, channels, and content. So our task as marketers is we're going to help to, you know, drive growth for the company. We better be measuring, are we helping, you know, grow revenue or not? That's what revenue attribution is all about.

So how it works, highest level, right, you capture as many touchpoints as you can, you catalog those, you cross-reference that with your funnel, stages, so you can see the relationship between interactions with buyers and when things moved along and progressed in your funnel. Number two, you have to, maybe the hard part, figure out which of all of those interactions which were the important ones that helped create that revenue in the end or helped create that conversion partway through the funnel.

And then lastly, you know, you're tracking these touchpoints. A touchpoint is an interaction like someone came to your website, interacted with this piece of content, or a touchpoint is they attended your webinar, or a touchpoint is they clicked on your ad, right? That's kind of the foundation of all of this is actual interaction with customer, with customers with all of your channels.

So the last step is you take all of those insights and you put them to use in your business. So one of the conversations always with attribution is defense versus offense. Attribution, kind of, first came on the scene, I would have to say, is primarily it felt like a defensive thing where marketing was saying, "No, really, really salespeople and CEO, we really provide value here. We're not just making this up. All of this engagement we're driving actually means something." Unfortunately, that's still important. You still need to articulate the value of marketing. But the more important thing with attribution is to be on the offense and to use the insights to help guide what channel should we be using in this part of the funnel, which campaigns are actually driving revenue, which ones aren't, what content is actually leading to pipeline and revenue, and what content isn't, right? Seems like basic questions, hard things to get at in B2B.

You know, so why do we do this? To provide, you know, the essential insight we need to know what marketing tactics to deploy, where to spend our money, you know, to get the most out of our limited time and money. We have a customer of ours, a CMO, and her quote was, "I don't have the money to waste money and I don't have the time to waste money," which I thought was a great quote because you know that spending all of your time working on the tactics aren't actually moving the needle. Nobody has time for that. Drive growth with sales, right? When I talk about being on offense, it's not marketing versus sales, it's marketing with sales. I have a quote coming up in a second from a customer of ours that talks about that. And yes, it's about quantifying the value of marketing and all of the tactics. And then lastly, as this becomes more of a general practice within B2B marketing, it's just showing your competence as a marketing leader that you are actually measuring the impact of all of these tactics and investments on the business. So attribution insights answer a bunch of really important questions. You know, what's the revenue and ROI impact of this campaign? Is this channel better than that channel from an ROI perspective? Which blog post is leading to revenue? Which one isn't? You know, how do we compare channels to each other? They could be very different channels. How are events contributing versus email or chat? This, you know, attribution can be done to track sales like or sales activities. What types of BDR engagements are working? Meetings, right, phone calls, all of those things. Events versus webinars. What's driving stage velocity? What tactics are working, you know, in a specific account? We have a session tomorrow that Cisco, a customer of ours is presenting, and she talks about one of the first things she did with attribution insights was to actually just look at one opportunity and say, "Look, we have all of the data of all of the touchpoints that fed this one opportunity." This one large win that they had at Cisco, and she used that as the way to, kind of, describe attribution, explain it to other people in the company that, "Hey, look, you can see this whole deal happening, the buying group coming together, all of the touchpoints leading all the way through the funnel to the end." So lots of questions, lots of important questions that are unanswered without this type of insight.

We do research all of the time. We have something called the state of marketing automation report. Last year, we found this similar stat to this. You know, everyone's telling us this is really important. We need to get this done. We're thinking about how to incorporate AI here. Here's the quote from JP Morgan where the gentlemen there is saying, you know, "It's amazing how you can improve the relationship with sales when you're using numbers that they care about like pipeline and revenue." This attribution, this measurement thing, as you know, is particularly difficult in B2B.

Now I know I'm painting an extreme picture of opposites here, but let's say a very transactional consumer purchase, right? We have this consumer, she's interacting with us 6 times, it could be different channels and in a matter of days or weeks their purchase is made for $50 and the deal is done. So if we're a consumer go-to-market, we're going to look that. We're trying to figure out how those six interactions led to that revenue creation, right? That's the problem attribution is solving. In B2B, it's more like this. That's an, yeah, ugly, isn't it? We all try to, like, portray the B2B journey, and it's never pretty 'cause now you have, you know, a 10-person buying group, and you have a sales cycle or a buying cycle that's going on for months, or quarters, or some of you probably longer, and because the sales cycle's in buying cycle is so long, you have to care about all of these stages and the movement of things through the funnel over time. And you can't just be like, "Oh, we have two weeks left in the quarter and we're behind. We'll dump a couple, dump 10 million in advertising and make the quarter." In B2B, you can't do that. The buying cycles are too long. You have to manage the funnel really closely. So now guess what you're doing? You're trying to figure out which of those 100, and how many do I have there? 140, you know, which ones were the important ones that drove revenue and how much impact did they have and how much influence did they have? This stuff is extremely hard. This is one place where AI really comes in and really is changing everything.

The three things you need to do, great attribution, data. You need a complete picture of the touchpoints. Intelligence. You need to be able to see the patterns in the data to understand which touchpoints were the most important. And then automation, you know, as marketing ops people, you know, have done a lot to automate marketing engagement, but often reporting, often attribution measurement is still very manual.

And so there's this idea of how do we operationalize this like we've done with actual engagement, the engagement side of marketing. So to go through, kind of, these continuum. So data completeness, it starts with single touch attribution, right, where you're saying we're going to give 100% revenue credit to the interaction that happened right before we opened the opportunity, right? This is the most basic form of attribution.

The good news is you're kind of taking the step first step on a path, you know, towards more complete attribution. For B2B folks, usually the next step after this is they measure all of the campaign touchpoints. So we're usually looking at CRM campaigns, we're looking at like Marketo programs, and we're tracking that, kind of, engagement.

And that's better. Now we're in the land of multi-touch attribution. We're tracking multiples, but we still have a lot we're not tracking. What we really want to do is get to every trackable touchpoint. Not every touchpoint is trackable, but we want to get to every trackable touchpoint to have as complete a picture as possible so that's something worst driving for as a vendor. Second, intelligence, right? So attribution that most people know about is rules based.

So a rule could be, give 20% of the deal revenue to the touchpoint that most closely preceded the creation of the opportunity. That's an example of a rule.

Even though, some of the rules based models that are out in the market, it's not like they were completely made up. They come from aggregate data of what's happening in the industry. But this is, kind of, the starting point for attribution. The next thing you can do is you can say let's just not always assume that number is 20%.

Let's, you know, if we look at your data as your company and look at all your history, we can start to get at a more precise view of what should that percentage be. We should only give 10% credit to that touchpoint that preceded Opp creation. We call that a hybrid approach. That's something we've had in our Marketo Measure product for six years, I think, for quite a while.

And now really the new and the best way, you know, to do this is to, you know, give a little more control over to the AI machine learning, where it's looking at all of the data, and it's assigning revenue attribution directly to touchpoints. You don't have this whole thing about stage waiting and 20% here and 30% there, and that's just gone because you're allowing the machine learning to find the patterns in immense amounts of data. That example I gave earlier had the 140 touchpoints, right? That is what we see, and we, you know, our software, you know, looks at a lot of different opportunities out there. That's pretty average for a B2B opportunity when you're actually tracking everything. All of the website visits, all of the ad clicks, all of the BDR conversations, all of the webinar attendees, all of the email opens, right? This stuff adds up when you're dealing with buying groups in B2B. And then the last thing I mentioned is what I call do-it-yourself spreadsheets and tiers, which is the, you know, hacking it out yourself method, and moving from that to channel-specific reports like only measuring campaigns, to finally going to something that's more automated. Again, this, kind of, thing that we're pushing for is we want this to be as operationalized as driving engagement to customers. The measurement thing has just been way too DIY for too long. It doesn't have to be at all. So we have a product, Adobe Marketo Measure, the artist formerly known as Bizible. This is, you know, been the leading product in B2B attribution for a while, and it does the things we've talked about. It quantifies pipeline, revenue, and ROI impact, of campaigns, channels, and content, and it gives marketers the insights they need to understand the relationship between their tactics and pipeline and revenue. And it's all done in a very automated way. And that's all good.

But our enterprise customers were asking us for more. And the first thing was they said, "Hey, I love that, you know, this product Marketo Measure originally Bizible. It was built with a really tight connection to Salesforce and Microsoft Dynamics, and that's cool." But I don't use those CRMs. I use a different CRM. Or, you know, Marketo Measure was built to, kind of, link directly to one CRM instance. I have seven CRM instances. And I have a Marketo here and an Eloqua there. And I've got all kinds of stuff.

Our, you know, Marketo Measure was not able to take in and aggregate that, kind of, data. So they asked for that. Second, they asked for this type of AI that I just mentioned, where we're not going through a rules-based model. We're not trying to do stage waiting or any of that stuff. We just want to use machine learning to do the attribution.

And then lastly, they asked for, you know, more automation. The less work that marketing has to do, the better here because we all know, you know, as marketers, we're not pressed to do measurement. We're pressed to get campaigns out the door and drive engagement, and there's never enough time for good reporting. There's never enough time for good measurement, even though it's super important. It's just got to be automated or it does not get done. So for all of those reasons, we created a product called Adobe Marketo Measure. We're using Adobe Experience Platform to get the data on the front-end to add that flexibility on the front-end for data collection. We're using something called attribution AI. These are all Adobe Assets that we've, kind of, grabbed onto in the product team, to give us that full AI/ML modeling in the middle. And we've done a bunch of work with our reporting at the end. So we have a new offering called Marketo Measure Ultimate. Like I said, AI is, kind of, at the center of it, which is going to be what Li is going to focus on next. And with that, I will hand it to Li. [Li Gao] All right. - Thanks, Bob. - Yep.

All right. So Bob has already, shared a bunch of questions about what insights you might be able to get. But essentially, we're here to help you to answer one single most important question, like, what did work, and what didn't work, right? That's what we're here for. So I'm here to share, how we actually do it.

First, when we talk about attribution, especially B2B attribution, I would like to have all of us think about, you know, where the data fits into the buckets, right? Essentially, when you think about attribution, you think about the conversion events, like, what was the business outcome, right? It could be a revenue event. It could be something else you care, right? The other thing is the touchpoints, what people did, the engagements, you know, we had with our customers. And for B2C, those two buckets are good enough, right? You have touchpoints. You have conversion events. You're trying to find the data correlation between those two data buckets. It's not sufficient enough for B2B because for B2B, you need to find out who are the people relevant to your conversion events. That is where buying committee comes into between, and you need to find those people. And from there, you then can find the touchpoints relevant to the conversion event. Then you can run the app B2B attribution. That is actually, that really where part of the complexity resides for the B2B attribution 'cause answering that question, who are the people involved is actually very, very difficult, right? So I just want to highlight two perpetual challenges for B2B attribute here. First one is a buying committee. Who are the people, right? This is actually very hard. Today, we largely use our contact as a buying committee, but in the future, and then one of my slides are going to show, we'll have thoughts on how we're going to improve over that in the future. The second thing challenge is about lookback window, right? So when you think about it, collect all of the touchpoints and use that for the conversion event analysis, like, how far do I go back to find all of the touchpoints relevant, right? And that, by the way, is still a very much a challenge today, how we're going to indicate where is the starting point. Right now, we will allow you to set a lookback window based on your, you know, heuristics experience about, you know, for your industry, for your product, how far you usually should go back about the entire, you know, purchasing lifecycle. So we allow you to set that. But in the future, you can imagine maybe we should also ask AI to help us to do some of those things as well. Those are two perpetual challenges I want to share that is relevant for B2B attribution. And I want to also briefly talk about the holy grail of attribution. To me, I think, attribution is fundamentally a data problem, like, there's no better way than just asking the data, you know, get the reading, get the pattern out of the data, and ask the data to help you to answer that question.

Okay? So yeah, as Bob shared, customers have been asking us a lot about how we can help them to collect more touchpoints. Because if you don't have touchpoints in place, then you're going to miss out giving them the credit they deserve, you know, if you don't even have them in place, right? So we leverage AEP to help us to accomplish that so we can get more data and easier into the system. Number two, as we just talked about, you know, let's remove the human bias as much as possible using the AI/ML to drive that real attribution decision, like, given one touchpoint, how much did it help us to close the deal, right? We shouldn't be setting a rule. We should ask the data and let the data tell us about it. And looking at this architecture diagram, you can see for the AEP in the middle, it really has a lot of components and services. For the Marketo Measure Ultimate solution, we don't use all of those. We don't use most of those services out there 'cause they were designed for and built for CDP application. But we do use the data ingestion. We do use the data lake to store data, and then we use a special destination connector to activate those data into Marketo Measure Ultimate. And as you can see, Marketo Measure Ultimate is one of those applications built on the top of the AEP or linked to integrated with the AEP to deliver the values and the values, you know, for our customers.

Okay. There's another diagram to show us how the entire end-to-end Marketo Measure architecture looks like. As you can see, we ingest data from multiple places. And we have a direct ad platform connection to read the campaign cost data and also do the auto-tagging, you know, to help you to improve the tracking position, right? And we have the JavaScript to collect the data from your website and make them the web touchpoints directly, right? The most important thing we did with Ultimate is we replaced all of those direct connections to a CRM or Marketing Automation with AEP. Now you can connect, you know, as many of CRMs available, as many of the Marketing Automation available in other data sources. For example, if you have a webinar system, you have an offline event management system, you can bring those data in as touchpoints a lot easier as well, right? The other thing highlighted in red there as well in attribution, Bob shared we had the rule-based hybrid approach as well. Now we are also adding Attribution AI as a full automated AI/ML approach for you to do the attribution in the ML fashion, right? Once we run through attribution, data will be sent to data warehouse, then we report right there, right? You can use our application discover dashboards to get the out-of-the-box insights. You can also take that data out from your data warehouse and do your customer reporting.

Okay? Here's a couple of, you know, really just quick screenshots about some of the insight you may get out-of-the-box from the application. This is a revenue attribution.

And next one is a return on investment analysis. And I just want to highlight that we also recently introduced the concept of a realized ROI, which allows you to track the lineage from your investment over the time and how that turned out to be the investment. We use a touchpoint to accomplish that. So this helps you to answer one question, like, "Oh, I invested $1 million, you know, last Q1. How did that help us? What's the return on investment for that $1 million?" So realize how I will help you to answer that question.

That's the engagement dashboard. It helps you to visualize and measure the engagement at a person level, and it will be adding account and opportunity level as well.

And does the velocity help you to track and then, you know, measure and compare how the velocity looks like over stage by stage? So you, kind of, get a better understanding on where the things goes well and where things, kind of, get stuck and you need to improve on that.

Okay? So again, just want to emphasize the values we're getting from the AEP here, which is, there's a large source connector catalog, 60 plus, that allows you to really bring the data from all different data systems into the AEP system, then activating to measure for attribution. It allows you to do flexible object mapping, flexible field mapping, and also allows you to do data transformation via calculated fields if you needed to. So with that, I'm going to just quickly run a short demo on how that ingestion looks like.

Okay. So here, you know, I'm looking at the source catalog here, right? And the sources there, there are many, many, different source connectors available. Here, I'm using Azure Blob Storage. Simply, there's a file stored in the cloud. And what am I going to? I'm going to ingest that and make a dataset so that we can then use them for the attribution.

Okay? Here, I'm going to pick opportunity dataset here, and you will see some sample data shown here. And the next step is you're going to tell the system whether you want to ingest that into an existing dataset. You want to create a new dataset set for it, right? In this case, we're going to just create a new dataset for it.

Okay? And you need to tell the system what data this is, right? In this case, we know this is an opportunity. But anyway, so the system allows you to do object mapping right here. So you can basically tell the system what the data is and basically mapping any source object to a target object.

So we are picking the opportunity here, and we enable the partial ingestion. So that's some, you know, technical details. We can get to the next step.

And the next step here is also to do the field mapping. Basically, now you're telling the system from the source what the field should be mapped to the target, right? And here, you would basically need to have an understanding of your data model, then figure out what is actually what. Then you will be doing the mapping here. And you will be using, the navigator and find the right field and do that mapping. Or if you have the mapping done already elsewhere, you can just import that, which I did that beforehand. I will ingest that, import that from another mapping flow that already done and all of the errors were resolved. They were mapped correctly. You can run the Validate here, and it will tell you everything looks correct. And also, if you do need to transform the field, you would be able to go to the calculated field, and then you can use, longest stuff of functions available and transform your fields to the shape you want, okay? All right. So once you do the field mapping, you go to next stage. You're going to set up the cadence. You can run it, just once. You can do it on a cadence, right? So like, you know, daily, weekly, whatever, then it's going to ingest the data incrementally based on your cadence you indicated, right? Okay. So in this case, we're just going to click on Next, and we're going to Finish, and this will just complete your dataset ingestion. It will then create a dataset in the data lake, then you can use that activated to the measure for activation.

Okay? All right.

Okay, so I'm switching over.

Okay. So next, let's talk about the Attribution AI. So we are now enabling the real full AI/ML attribution with Attribution AI. It enables us to do a bunch of use cases we were not able to do before. Number one, flexible conversion event. So now you are able to specify what kind of conversion event you want us to attribute against for, right? Like, it doesn't have to be your Opportunity Close Won deal. It can be something else. It can be a particular opportunity stage, for example, Opportunity Qualification or Opportunity Creation. If you only, you would like to focus on only on the top of the funnel, like, creating an opportunity. If that's what you want to do, you can also use this to accomplish this. So that's one opportunity, flexible conversion event. The next thing is we allow you, within one instance setting up for set up for the AI/ML attribution, you can specify multiple segment models because you have different markets, and different markets may have complete different behaviors that you would like to really to be separate models to trend and to learn from those data and then be able to score them for attribution differently. For example, if you think about the US new business market, if you are having, you know, a SaaS business, you have a new business versus renewal, and you know their lifecycle is going to be different, right? You know, US new business, like Japan renewal in this example, they could be totally different markets, and you want to create separate models, separate trend, and using them separately to score the conversion events in those markets, right? We always, always have a default model that would include all of the data in case any data that, you know, does not fall into those buckets we define, they will always be scored by the default model.

Okay. We also enabled incremental model. That means imagine if you even don't invest anything on your marketing investments, you can still sell something. That might be because your brand equity, your partnership, or your location advantage, whatever that is. We call that baseline effect. So using the statistical machine learning model, it will be able to tell you not just on total impact, but also are able to strip out the baseline effect and give you the net impact from the marketing investment, right? So you will get both scoring attribution from the system. You know, we call this influenced model, will add up to 100 points. The other one will add to somewhere. You know, be it 60 or 70. It varies. It really varies by opportunity, by your conversion event. But it will tell you what is the marketing impact for that incrementally without the baseline effects, okay? So with that, before we jump into the system flow, I just want to quickly show you how to configure, Attribution AI model, okay? So as I indicated, you come to this Attribution AI instance, configuration within the Marketo Measure application. You can create a new instance. Let's just call this Qualify.

And then you will be able to pick either opportunity event or the leading event for your conversion event. In this case, we're going to say, "Yeah. Let's use opportunity stage that is a qualification, and that's my conversion event." Qualification. It's, kind of, hard to read, but that's okay, right? And then you're going to create a data segment, as we indicated. You could create multiple data segments, so the data will be partitioning into different data segments, and then we'll train them separately.

Okay, let's see.

We're going to give it, lookback window here. We're going to say, for that market, it's going to be a year and a half.

Okay.

Okay. So there you go. We just created a data segment, and this will enable us to train separately for this market, right? It can create more models. With many models within this single instance, you can create up to 10. So we're going to just stop here, and then we can go to Next.

Yeah. Sorry.

Okay.

Oh, okay. Let me just remove that. One sec.

Okay. So we also need to give a conversion name here. That's why it's holding up.

Qualification. Okay.

Okay, next step, we're going to need to group the touchpoints. The system asked us to group touchpoints into no more than 40, right? So if we start with a channel or sub-channel, if you have a high number, than 40 sub-channels or channels, you're going to have to regroup them a little bit to make that, the total number less than 40, right? Last page here, you would be looking at telling system how much data you want to use to train the AI/ML model, and we default that to two years, but you can use shorter amount of time. You can use a year and a half. You can use a year or six months. It's really up to you to select how much did you want to train the model, right? And stage selection here, those were the some of the milestones stage you may want to use to give a hint to the AI/ML system to tell the system, some stage probably more important than the other. So when those touchpoints are closer those milestones you may want to consider a higher weight, we don't tell the system how much but you can give a hint to the system, right? It's up to three. Lastly, is the job title ranking. You can indicate to the system about what job title is more important than the other. Again, that's a hint. It's optional. For example, you could say VP is really more important than the CMO, and CMO is more important than a director. For example, use those data to give the system a little bit of hint, how they weight or overweight the touchpoints, okay? Right. Going. Once you do that, you will click Finish button. Then we are creating, AI instance here that will be trained. We run, weekly training, retraining, and we score on incremental basis on updated basis, okay? All right. Now going back, I just want to spend a minute or two to talk about really how the AI actually works, okay? So first, we actually, the Attribution AI would do a data stitching and a pre-process, right? We identify the conversion events and the touchpoints. Then we stitch a touchpoint to a conversion event through the buying committee. And right now, we just use that, kind of, context as buying committee. But over time, this will evolve into other concepts as well.

We then do the data filtering based on the training window and lookback window. So anything outside of those window, we're not considered, right? Now the fun part is on model training. How do we do it, right? We call the algorithm discrete-time survival model, right? So first, we're able to generate the positive path and negative path, like, certain things converted, certain things did not convert. We use them for comparison, right? Then we generate features. The features will be based on a combination of a touchpoint group. We talked about it earlier, the time lag to the conversion event. And also, if you have touchpoint stage selected, that's optional, it will be part of your feature as well. So those feature will basically buckets. We put the touchpoints into those buckets, and we use them to run through the statistical regression, you know, computation, right? Then the next is we separated those data by the model training, by those conversion events based on the data segment definition we just saw a couple of minutes ago.

Lastly, we run a logistic regression. We generate the coefficients for each of those features we just talked about and also the intercept for the baseline effect so you can have the influence versus the incremental model.

Once we have that, we can go into the scoring, right? And we do similarly. First, we generate the positive path only. We don't score negative path. We only score the positive path, which is conversion event. Then we generate features. You know, we have same variables coming for the features, plus also the contact title here as well. So notice a difference here. We don't use a contact title for training, but we do bring that in as a variable for scoring. And lastly, we're going to score the touchpoints based on the coefficients we just talked about, right? This is similar to the Shapley value calculation, in terms of the algorithm, if you guys want to know the internal how the math actually works. Lastly, then we normalize the value. We're going to generate the final score with and without baseline effects that give us both incremental, as well as influenced model, right? The value will always be between 0 to 100. That's how we normalize it.

And finally, we would produce, you know, some of the model insights as a model efficiencies, touchpoint effectiveness, etcetera, right? So that's the entire, really how the system flow works for the Attribution AI, machine learning algorithm, okay? Lastly, just want to spend a minute, you know, where we're going with the B2B attribution, right? I think, really, the B2B attribution, as we talk about, one of the key challenges is to identify, who are those people relevant for your conversion event? And that has been always, always been difficult, right? So today, we largely use that, kind of, contact. But for a larger account, that could be a little bit too broad. So we think the future is we leverage the new Buying Group concept we introduced, over the Summit. And then we used the Buying Group concept to drive a better precision on how you associate touchpoints to your conversion event. And more importantly, we use Buying Group to indicate, really the product separation on the touchpoints, right? Once you have Buying Group, we can say this Buying Group is for this product, and that Buying Group is for a different product. And you also have the separation by conversion event. Now you can put them together. You're going to be able to run attribution by different product solutions or product groups. So I think for large enterprises, this is one of the most asked for things, and we will be going down that path with Buying Group able to make that happen. The second thing is, we understand that data may come into multiple places. So really, it's best for the system to use the best available information to drive attribution not just one, but whatever the best. So we're going to be introducing a hierarchy of the data available in the system. For example, could you try to use Buying Group first? If it does not exist, we can use opportunity contact role. And if it doesn't exist, it's going to fall back on the account contact, for example, right? There's going to be, multiple levels of attribution, Buying Group association we're going to be trying out there, right? With that, this is the end of, you know, how things work. I'm going to hand it over to Kimberly about how what are the best practices. [Kimberly Galitz] Thank you.

Okay. Hi, everybody. I'm going to bring it back to a little bit more basics, way less technical than we had. So a little bit about my story. Bob mentioned the spreadsheets and tiers thing. That's literally where I started with attribution. I don't know if there are more tiers or spreadsheets, but definitely had that going for me. So I started with needing a way to track any touch. First touch just needed to know where our leads were coming from and what was happening with them. We did not have that. I was manually exporting reports from Salesforce every day. So that's what got me started on this path.

As soon as I got there, I realized I needed more. So luckily, at that time, it was visible, but they started offering multi-touch. So we moved into a multi-touch world where we were tracking more than just the first initial engagement. What else were they doing before the opportunity creation? After that, we wanted to know what they were doing while they were talking to the sales people. So what were they doing between opportunity creation and the end of that deal? Hopefully, closed one deal. So we evolved to a full path in custom modeling at that point, and then I started switching companies and using it in different places and using the BI tools. So taking that data and exporting it into a place where everybody uses BI tools that-- Oh, sorry, everybody that uses BI tools is in the single source of truth. So we started to put that data there so that we could build reports and dashboards for our users, our field marketers, and our C-suite as well. And now we're moving into what Bob and Li were talking about, where we need intelligent models, not just rule-based models. So I've Marketo Measure, who started with single-touch moved to multi-touch, and now are on to AI.

So why Attribution AI? So as our need for that data and that insight grows, we really need a tool that's equally important. So we can do as much as we can as humans, but what can we do on the back-end in that machine? And that breaks away from that rule-based human bias. So from First Touch to Full Picture, including that understanding, the buying group is obviously, as we mentioned, different depending on your product, depending on the renewal, depending on the region. So that's important to understand. And when you're constrained into just one rule, it's a little bit harder to gauge those insights. So the timing, the channels they engage with it all impacts revenue. So you want to know what the different buying groups interact with at different stages, and then our target audiences change. They change as our business changes. So being able to be three steps ahead of that so that you're measuring what's out there and how marketing can pivot to reflect what your audience actually wants, that's going to be key.

So one thing that the ultimate measurement plan offers is not just that one thing, like, not just that rule-based thing, not just that first-touch, it actually offers insights from the beginning. What are you looking for? So this is kind of-- If you don't have attribution today or you do and you're struggling, this is, kind of, my framework for how you can be successful. So I like to ask, what is it you're really trying to look for? What engagements in your marketing tech stack and your marketing mix are you are you tracking? Are you missing tracking on anything? If you are, can Marketo Measure solve that for you or an attribution tool solve that for you? Are you ready for that? That's the other thing you have to ask yourself. And then what are you currently measuring and reporting on? And then those KPIs that your dashboards have today, what are they looking at? What is your C-suite looking at? And then you don't want to forget about your marketers. So what is your field team looking at, your events team looking at? They're not always going to be looking at what your C-suite is looking at. They want to know really granular level data, so you need to meet them where they are and give them that insight. And that is what you can do inside of Marketo Measure. And then your audience, so who are you targeting? Do you know who you're targeting? Are you confused? Like I said, your audience can change. So do you understand the types of job titles, functions, roles that your buying group consists of? We can break all of this data down in Marketo Measure and understand which job titles and roles and functions interact at different points in the funnel, and that's how we know which marketing content they should be engaging with at that time.

So attribution helps in a lot of ways, obviously, but the rules and models provide a framework for measurement and understanding. So it's not just one tool that solves all your problems, but it gives you that framework to set yourself up for success in reporting. Like I mentioned, I started at a really small company with Marketo Measure. I've evolved over the years, and now we're using it in a place that's a multibillion dollar enterprise company. So having that framework from the get-go really got me to understand what I needed as I evolved down that path, and I know what people kind of need. So that framework is really what you want to understand while you're getting started and as you're evolving as a company. The data helps your team make decisions. So how can you inform ways that they can use their budget wisely? We often have people asking for more money, and in an economy like this, that's really tough. How can you help your marketing teams understand how their budget is really being impactful in the funnel, and how is it helping your customer? Are they getting what they need? Are you missing a really key piece of content that they could use to renew? Those are the type of insights you'll get here. Multi-touch gives you that deep, like I said, deep funnel insight, so you can understand, like, all of those touchpoints, that picture that Bob had. Let's collect them all and then really understand how they're spread out across that journey. And then, of course, it's dynamic. So obviously, with AI, it's even more dynamic and flexible. But with that strategy, you can pivot at any time.

One of the key things in Marketo Measure that I love is, you can just change things on the back-end and your data recalculates. I don't know in any other world where that happens, so it's really awesome that you can just use that flexibility to your advantage as your business changes.

So regardless, like I said, where you are in the journey, I have tips for everyone if you're just getting started using multi-touch or just want to hear how people do it.

So what type of measurement are you using today? I would say if you're using first or last touch, can you look at some campaigns and people in those opportunities? Just kind of look at what they've been doing in your marketing automation platform or wherever you have access to that data. They might be doing a lot more. And if you can point out places that they're doing things and you aren't tracking them, you can bring that to whoever and say, like, I really think we need to think about a multi-touch approach 'cause we're really missing a big chunk of this data.

That covered number two with number one, but, yep, same thing. If you want to move to multi-touch, so what reporting problem are you trying to solve? Like I said, do you think you're missing something? Are you looking for deeper insights into the funnel? Are you partnering really closely with sales and need to understand what that marketing sales mix looks like? If that's the case, multi-touch attribution really helps with that. You can track sales touchpoints, you can track all of the marketing touchpoints, and you can join them all together. And then they're spread out in that attribution shape that we were talking about earlier.

If you already have multi-touch, so I've talked to a lot of people that use attribution tools, and then they get really stuck when they see the data. It's really overwhelming. There's a lot of information there, but how do you piece it together and understand what you're looking at? So if you have a data issue, is it, like, a data cleanliness issue? Do you think things are wrong? Do you just not really understand what to do with what you have? A lot of times, I've worked with teams where there's, like, an events team or a field marketing team, and they see these dashboards and, like, I don't really know what to do with it. So if you are able to sit them down and get them some framework, get them a dashboard they can clone, get them a report they can use, and help them understand how their events were impactful. When it's their data and their heart and soul that went into it, they want to know how it did. They will listen to you. So that's really important to get alignment because then those people are going to be your cheerleaders moving on.

And also, attribution is not like a set it and forget it thing. Like I said, it's dynamic. Your business is dynamic. Nothing stays static. So you really need to evolve with it, the tool and your business. So go back and look at your rules, understand what they're capturing. I like to say look at it every six months. You can do it by quarter, but it's definitely worth checking on what you're looking at.

So regardless of where you are today, like I said, alignment is really key. So find your hype people because they're the ones that are going to help you either gain better use of the tool or teach people how to use it. And then once you have that understanding of the tool and how to use your marketing budget, everybody really likes this tool. And then understanding the data is key. So a lot of times, people don't understand the terminology, like, what's a buyer touchpoint? What does that mean? What does W-Shape mean? What does weight mean? What is a milestone? What are all these words we're using? Having a just a one-on-one training to help people understand those buzzwords really, kind of, takes some of that anxiety away when people are trying to start using these tools. I found it's really helpful, and then they go off, and they're the owners of that report and that data. And they really are the owners, and they start to be the data stewards at that point. So like I said, never set it and forget it, and then gather insights and feedback from your team if they feel like they're missing anything. If they're struggling to gain insights on a campaign or an event, help them through that.

Marketo Measure is actually really flexible, and when you can go into those reports and actually see what's going on, so walk them through that, I suggest starting with the most basic reports, like channel level, campaign level, and just looking at touchpoint data date by date, by quarter, something like that, and you really start to get a hold of what you're looking at.

So that's it from me. A little bit more is I actually am the co-leader of the Marketo Measure user group. So if you're not in it today, it is virtual, so you can join. We hold sessions at random. I can't say I hold to a quarterly schedule, but I do my best. But feel free to join. We're happy to, a lot of times we just jump on and ask each other questions. So it's super collaborative, and we're happy to work through some problem solving.

And I'm going to hand it back to Bob.

Thank you, Kimberly.

Thank you. [Music]