[Music] [Miray Vu] Hello, everyone. Welcome to our session, Marketo Engage Data Hygiene Strategies today. I am Miray Vu. I'm a Senior Marketing Operation Manager at Qualcomm. And as you may guess what my role is, I'm responsible for one and only Marketo.
[Alex Greger] Alex Greger, Solution Architect at Perficient, supporting Qualcomm and all their marketing operation strategy and execution.
So who are we? You're looking at, between the two of us, five-time champion class, both of us user group leaders. Both of us have renewed our architecture master within Marketo Engage architecture and once was MCSA and about 20 years of experience in Marketo Engage. So between the two of us, we've seen quite a few instances. We've done a lot of cleaning of houses within Marketo instances.
Okay. Let's start. What if we tell you that with a process change or just implementing a new process, you potentially can see up to 95% error decrease, and you may save two hours from manual work. It's amazing. Right? Well, I don't know you all, but if someone showed me this picture two years ago, I'm probably going to stop doing whatever I'm doing, and tell me what's your secret sauce? Because I was losing a sleep about the sync error, and we were really dealing with big issues that how we going to fix it? What we going to do? So today, we're going to talk about a lot of data hygiene, data cleaning, or what's still our challenges and how we actually find a solution. We definitely going to show you this picture one more time, but first, let's start about the data. Data is important, but data integrity is everything. Why data integrity is very, very crucial for, especially, marketing? When you have a clean, optimized data, you can start to building a great foundation for your marketing. This foundation brings ease for your campaigns. For example, if you are using the targeted or personalized strategies for your campaigns and you know that you have a clean data to start with, you are one step ahead. But if you don't have that clean data and you need to step back, work with your, maybe, several teams, make sure that your data is clean, and then you can put your strategies together, which, it's more time, more money, more effort. It's a domino effect. And then you also give option to your marketer to give their decision by supported data when you have the clean data strategies. Another effect is the compliance and legal consideration. The compliance is getting more strict every day, and we need to-- Especially, you are working with the global companies, we know that it's very important. It's not only just the Marketo anymore. It's like the most of the MarTech stats needs to apply those compliance rules. I have good news and bad news. So the good news is compliance is same, similar for every companies, every industries. We know the GDPR countries is not changed by company by company. Right? But if you have a bad data in your system, even though you apply those compliance process in your database, you cannot really see the result because you started with the bad data. But if you have the clean optimized data, you are, again, one step of the game. You set up your compliance, and you forget it. You're done with this part. You know that you don't need to face with any legal issues. And with all these effects going to help to improve your ROI and, eventually, revenue.
Let's look at how we can achieve that data integrity in general. I usually bring the science in my presentation. I have a science background, but my heart is live with marketing. So I really like to take the approach for three-step approach. First of all, methodology. We need to build systematic and repeatable process in order to achieve that data integrity. Standardized data entry from beginning till the end. Automated workflows, this is a process, it's going to keep running, and then we need to regularly check our system if it's working or not, it's with the audits. The second part is the validation. We put the process together. It's great, but are we doing the right thing? We are in the right path? So that comes to validation. We can definitely build some data validation rules. We can use some reporting function, but to me is the best part is the getting the real-time feedback from our stakeholders. Are they seeing any effects for the whole data cleaning process? And the last part is the optimization. Optimization is very important because data cleaning process is not easy to put together but you really don't want to keep redoing it when you need to change one part of that optimization or part of that data hygiene process. So we need to give our staff a room to improve our process based on the business needs, because whatever we put two years ago, I guarantee is not going to work two years later.
Let's look at the whole Marketo ecosystem, how the Marketo Database starts. I like to call this the art of person creation because in Marketo, everything is tied to person objects, so we see all the data out on the person. So there is a way that you can create a person. I believe that there's 14 or 16 of them, but I really put here the most important ones, like the most common used one, the List Import, Form Fill, Web API, CRM Sync, Event Partners. These are the way that you can create the person in Marketo, same way you can update the person after it's created. So if you don't have any data hygiene process in your Marketo system, you're going to end up this world of data mess. If you guys have a kid, you probably heard it million times, how they grow up so fast. Like closing your eyes and open your eyes, they grow. Same thing, if you don't have the data hygiene process in your system, your data problem grow so fast you cannot even imagine, how fast it's going to be.
So how we can actually work around it and make sure that we have clean data strategies? Same thing like the three-step approach that I want to build it this one in Marketo. This is very high view of the data hygiene cleaning system. First of all, we want to put the data hygiene campaign as a centralized location, centralized data hygiene campaign process. I see some people, some companies going to decentralized version, which to me, is just endangering to issues. You cannot really keep up with the demand when you are using to one or two campaigns. But when you centralize, you have option to actually control anything, and it's to make the process easier.
And your centralization campaign is going to contain the normalization of values and if you have any data enrichment, you can also edit that data enrichment under these campaigns. The second part is important. We are going to run our data hygiene campaign process periodically, not once person created. The person creation is definitely the first point, but we need to also understand that this is a whole process. When person go through the updated field or any value that we are looking for, they need to re-run our data hygiene campaigns. And the last part is the optimization. Validation of the data quality, definitely important. For our cases, we really work closely with our sales operation team to understand if we are in the correct path. And also, again, we give our software room to expand our data hygiene campaign based on the business needs instead of recreating every time.
Now I want to start talking about Qualcomm data hygiene challenges. So in Qualcomm, we have two really main issue bring up to the whole data hygiene challenge. The first one is the Marketo Salesforce Sync. So you probably hear a lot of time Marketo and Salesforce one-to-one sync, which is true, but in our cases, this is not really true because we are very selective that who is going to sync for Marketo the Salesforce. It's not only the hand raiser, but we have additional rules and filters on those hand raisers as well. So if it's going to give you some idea, we used to sync less than our 1% of our Marketo database to Salesforce.
We were excluding B2C communities. We are great communities. We support developer, Snapdragon Insiders. So these people, we market pretty much every day but we don't want them to be in Salesforce. And then we don't want the employees to be in the Salesforce. We also exclude the free email providers, not just Gmail or Yahoo. We actually determine up to a little bit over 6,000 free email providers. We were excluding those email domains as from Salesforce as well. So this is the first part. The second part is extensive data normalization field. What that means? If you have the data hygiene campaign running, you usually see that the country, state is very common practices, lead source, maybe, a couple of other fields. But in Qualcomm, we require more process happen before this thing happen. For example, one of them do not sync Salesforce. That's how we excluding those people that I talked before, like the communities, free email providers. And another one is marketing qualification source. This is not a lead source. This is totally different, and it's based on the personal activities we determine the certain values, and this also needs to happen before this thing happen. Top of that, we have data enrichment tool that it's not sits in the Marketo, but we integrated with Marketo, which we are sending to qualified people through the Webhook to our data enrichment tool. The tool do the magic, send it back through the web API. In our experiences, it takes 8 to 10 minutes, and this also needs to happen before the sync happens. So there we go. We have so many data hygiene challenges. And I'm going to put this one in a timeline, when we actually start to seeing issues because don't get me wrong, we had a great data hygiene process before. I was very proud of it. Okay. We don't really need to worry about our data cleaning, but I was wrong. So first we implemented Marketo, 2016. We really didn't have any CRM sync until 2019, which at that point, we start to seeing that very selective sync standing from Marketo to Salesforce. Because the sync start to happening, we build our version one, 2020, Lead Source, Country, State normalization, which perfectly worked for us. Between 2020 and 2024, as you see, we optimized our process, which we included the duplication management. We added to that Do Not Sync Salesforce process. We added to Marketing Qualifications Source field update. These are great. And 2023, we decided to use the CRM Campaign object. We had the object, but we never truly using it. First, we work on the Campaign Naming Convention. If you guys at the Keynote yesterday, I really like the saying that taxonomy is so sexy.
We worked very hard to make our campaign naming taxonomy, apply different business units so we can actually use it and get the great data. We built the hierarchy. We built our hierarchy by the three-layer because our plan was using the Marketo measure for the future. So when we're building everything, we try to look for the future to make sure that we are capturing a lot of stuff. So for 2023, once everything's settled for the campaign, CRM Campaign object, we just start to syncing one or two campaign. If there's a important webinar happening or it's the event happening, then we sync them. You can start to sync some sync issues, but it wasn't really high degree. And 2024, one of them is needed, one of them actually tried to push. We need to showcase our marketing efforts. We need to show how important to campaign object because not only the showing to marketing, but also sales are going to take a lot of advantage from the campaign data. So we decided to sync pretty much one-to-one campaign, like whatever we create at Marketo, we start to create it in Salesforce except the operational ones. It was beautiful, right? But this bit, it come with the best, so we did not think about this is going to be big issue for the sync wise until the Executable Campaigns, which Alex is going to talk about a lot, our strategy, how we build it, what was our solution is. But before that, I want to also show this picture to, how much lead sync happened in 2024. As you see, we started very little. 2022 and 2023, you see little bit increase. But 2024, that's jumped up, I don't know, three, four times already. So some of you might going to think that, "So what's wrong with that? It's good. You are sinking more leads. You might be going to create more opportunity." But that's sinking more leads come with data quality decline, increased manual efforts, reduce effectiveness and create confusion. And it's not only my team, I already losing so many sleepless night like, "What's going to happen? How are we going to fix this one?" But our sales operation team the same way because the fields that create the issues actually tied our lead roading, so the sales get the wrong lead in their queues. But it's definitely a big mess, and we find ourselves here. We were trying to finding a losing game. And Alex.
Awesome. So Qualcomm was having this exponential issue of really trying to show the increase in the marketing efforts, which was leading to an increase in lead intake, which was then leading to an increase in the sync issues and errors. So where do we start? We really, like everything, started with what's the root cause, taking an audit, looking deep in, and saying where are these errors coming from, how are the errors occurring, what fields are affecting this, looking at those key insights and understanding very quickly that we had a race condition happening. People were trying to sync and tried to be updated or normalized simultaneously. Some were getting through successfully. Others were not. That's where Executable Campaigns really came into play. With Executable Campaigns, you can really control that order of operations on when things occur, especially when you have downstream dependencies. So that's where our nirvana came in. But before we get into how we designed that strategy, we wanted to take a step back. We wanted to ask you, with a raise of hands, how many of you actually utilize Executable Campaigns today? Can you show me with a raise of hands? Awesome. About 25%, maybe, 30% of the room. Very cool. So we're going to take a quick step back and show what is a Executable, why we chose an Executable Campaign solution for this. So if you've used Request Campaigns in Marketo, Executable is really an evolution on that. Same idea within a Smart Campaign. You have the audience or smart list that you're designing who you want to go through that flow. You have then the flow steps that take you through what actions you really want them to take. One of the big changes with an Executable Campaign is it's immediately live and ready to be called upon when you create it. There's no scheduling. There's no activation like you would for a batch or a trigger Smart Campaign. The next aspect is it really acts as a child campaign of whoever or whatever workflow is calling it. And so Miray will get into that a little bit later, but there's a lot of advantages of how you can utilize that parent-child relationship. And then, finally, how are Executable Campaigns really leveraged? They're in those common operational tasks. Today, our use case highlights the data hygiene aspects. They can be used for life cycle modeling, scoring, interesting moments, any really operational task where you have key dependencies, where you need something updated changed prior to another task or another element taking place. That's where the big advantage of it in running in series or sequential processing comes into play, really making sure that the first step completes before the next step continues on, very different than the Request Campaign type of Smart Campaign that we'll get into in a minute. So how do you create an Executable Campaign? Just like any other Smart Campaign, you would go, you would name it. You'd provide a description, and then there's a little checkbox down at the bottom. It's as simple as that to create your executable, and then you can see that and define that in your Marketo tree by the little light bulb with a gear icon in front of it, differentiating it from a Request Campaign that has the little arrow, trigger campaign that's the lightning bolt, or any batch.
So back to why we chose Executable Campaigns for this process. The key difference there between request and executable is that in parallel versus in-series processing. So on the left there-- Will it work? Cool. It's working. With the Request Campaign, you go through a flow step with Step 1, Step 2, and then you have a call of a Request Campaign. This runs in parallel. Let's use country normalization as an example. You're processing records. You want them to go out and maybe have US, U.S.A, something like that, and you want it to normalize. So you call out to that normalization process, and then simultaneously, it's going to continue on to that next step in the process. The issue there, again, is you may have dependencies there. So the advantage to the executable is going through Step 1, Step 2 processes that could be add to list, send an email. And then your Step 3 of that normalization or data hygiene process that you want to call out, it goes out to normalize that country value. It may have a dependency to then look at your state value and say, "Okay, based in X country, US, for example, I now want to update the state or ensure the state is normalized." So it's going to do that. Then it's going to come back to your parent workflow and continue on to that process. So the advantage there is highlighted here. So on the top right, you can see the Request Campaign. You can see within the red box, we have a Request Campaign out for data standardization. We then immediately see it try to sync to a Salesforce campaign, and you can see highlighted there, there's an error that the country is erroring out because it's not a normalized country. So you see highlighted on the bottom left the Executable Campaign process. The bottom green arrow shows the start of that executable process. It goes through them between the two green arrows is all the key data hygiene workflows that we want it to hit. You can see country being updated, state updated. And then once it's complete, we then move into the top green arrow there that starts to sync that record to the Salesforce campaign and successfully syncs them. So this is really allowing us to really control that order of operations.
So let's get back to Qualcomm and the exponential increase in these errors and how we resolved our marketing operations headaches through daily updates and changes that we were making. The first thing we started with is an audit. That's where everybody starts to look at something and understand what's happening. We looked at data value changes, persons created and Request Campaign workflows. Then we started to categorize those. Are they priority one issues? Are they priority two? Where do they fall in dependency order to getting somebody synced or the usage downstream that is needed from an audience or a workflow perspective? Then once we had those organized by importance and understood that, we also made notes to Miray's note earlier to the existing hygiene process. I was running very smoothly until it didn't, and so it was a two-year extent. The business had updated, transformed over time. So there were notes there of operational updates that we can make optimizations changes as we rebuild this process.
Next, we didn't want to boil the ocean. We had this issue on a day-to-day basis, and we wanted to get something out the door that alleviated some of our time as well as ensure that our sales and sales enablement team were getting the data and speed to lead for routing and records in front of our sales team as soon as possible. So with that, we really defined those key orders of operations that were essential for the routing and getting somebody into that Salesforce process. And then something that we learned in our research process from a separate mug that was related to execution campaigns was really providing operational notes for yourself. This has been a great advantage for us. The instance has got a very large instance with many records flowing through it, lots of activity happening. And so what this allows you to do is quickly see where those data normalization processes are happening. It also helped us see from a metrics perspective, the velocity in which somebody was moving through this process. We didn't want to slow down the speed to lead. We didn't want to bottleneck that process in any way. So this allowed us to really analyze that and make sure that we weren't causing major issues in getting those records in front of sales in a slower process.
Next was really designating our organizational tiered process. With Executable Campaigns, you have that ability to nest them, parent-child, again, type of relationships. So really, designing and understanding where our central calls will be, what are our orchestrators versus what are our task or units or workhorses that are actually updating the key fields that we need. Aligning with our stakeholders in sales ops and really drawing this out, really allowed us to understand from a high level view all the different dependencies, all the different fields we were working with. One thing we did want to call out is, with Executable Campaigns, you cannot use Webhooks or Wait Steps. As Miray noted at the beginning, data enrichment was part of our process, and so we needed a Webhook to go out to third-party, match a record and pull in additional data points. We couldn't do that through a single executable, so we daisy-chained the process together with an Executable and a Request Campaign so we could still control the order of operations there without losing that aspect, but still maintain the request and the enrichment needs.
The next thing we wanted to note was Sandbox Testing. So if you're all familiar with our Marketo Sandbox, it's very different than an AEM Sandbox or a CDP or a Salesforce Sandbox. It is not an exact replica of your PROD instance. That sometimes can cause issues when you're designing something that's a little more complex and as critical as a lead processing or lead management type of workflow. So we wanted to call that out here in our design process so you're aware and share that knowledge that we weren't able to really replicate the volume or the scale that we were seeing in PROD, just because of the difference in instances. The other aspect was the Marketo Sandbox runtime is not as optimized as a PROD runtime. And so when we were going through our regression testing and different use cases and processing them, there were some assumptions that we had to make when it came to the speed in which somebody was moving through that process. We actually brought in Marketo Support at one point and had them validate for us that it wasn't an issue with the processing. It was more of a Sandbox versus PROD element. And then moving all of this into PROD. So from design to actual slow rollout, this took about a quarter, three months, and really, we had it fully designed and built in our sandbox. If you aren't aware, you can connect your two different instances together. So something complex like this that has many different Smart Campaigns, many different flows, when you're dealing with country standardization, you got several hundred countries, so you got several hundred choices. Bringing that on one screen and trying to build it or replicate it in the other, we're all human. We make errors. So it's much easier to connect those instances together and just import that program. So Marketo support can connect to your instances and allow you to import that process. Then we did a slow rollout. We really started with where people were created and where the hand raisers were immediately happening, MQL stage, contact sales stage, really ensuring that those process are tightly updated. And we had really good insight and view on them before moving it across the rest of the instance. Then we updated our program templates. So as our team cloned new programs, a webinar, a gated asset, the executable hygiene process was built into that, and it slowly then expanded across the instance. So that's our last step is, again, we really started with an MVP of those key fields that help validate a record before they get into Salesforce. With the ability within Executable Campaign features and functions and how we built the program in a modular fashion, it was easy for us to build on processes as we got new use cases or as the business expanded. And so one of the fast follows was a sales-prioritized lead process, allowing somebody to indicate a record that should be immediately sent to sales, and we needed that inserted into our process. The next was the unqualified records, understanding feedback and suppression from this data hygiene process and adding that on so they wouldn't continue through the process and be fed back to sales. And then some simple lead scoring qualification processes. Miray will touch on some of the iterations and optimizations we've made over time as well as we get into the future slides. So back to the outcomes that we saw. They really speak for themselves. We saw 95% decrease in those errors that were occurring on a day-to-day and campaign by campaign process and the headaches that we were seeing on a day-to-day basis, about two hours of re-syncing records for a campaign, updating, standardizing that data, and then processing them again was set at more of edge case two hours a month that we were spending. So saved the team a lot of time and headaches. So I reach out to all of you in understanding if you're seeing similar issues, I highly recommend looking at Executable Campaigns. If you're on a regular day-to-day basis updating records, re-syncing records for your campaigns, or if your team is doing this and spending a lot of manual effort there, are running into similar race conditions that we were seeing, I highly recommend looking at Executable Campaigns for your data hygiene process.
So we wanted to touch on a few lessons learned along the way, the first one being, not everyone had to go through the process every time. We had a lot of standardization from pick list values and forms, for example, that people were coming in with standard values already, and we didn't need them to go through. It just slowed down their process. So adding filters and choices to the Executable process allowed them to skip over when they already met the needs of the standard data values and allowing them to more quickly get through that process, get synced and get in front of sales.
A couple other lessons learned along the way were, there are specific use cases we didn't want to feed through this process. Executable Campaigns are great, but depending on how you architect them and the volume you're sending through them, because they work in a sequential manner, they can really slow down your instance or bring your instance to a halt if you're not careful. So operational list imports, for example, hundreds of thousands of records that you may be cleaning up or migrating from one instance or an acquisition or something to your central instance, you want to suppress those from going through a process like this to really ensure that they don't bog down your instance or cause a big backlog of some sort. The other one we touched on earlier was the Sandbox versus PROD. So with Executable Campaigns, we really saw a difference in our testing and our processes that we didn't really know or understand when going into the process. And then finally, really, the limited use cases in the architecture of designing this out with our stakeholders, ensuring we knew what exact fields were being implicated. There are many other fields that are in play but they're less important. They can be managed by batches or other requests that don't have dependencies on getting these records synced in a timely manner.
So Miray's going to touch on how we plan to use executables in the future.
Thank you, Alex. Yes. What is the future of the Executable Campaigns for Qualcomm? As Alex mentioned, we didn't want to boil the ocean first. We started small. We put our data hygiene strategies in the Executable Campaign, and it works wonder. And for the near future, we want to expand that data hygiene definitely for adding to several more fields. We might want to remove some fields for the later depending on what's business needs, but this is our near future use cases. And then we definitely want to bring life cycle and scoring an interesting moment operational campaigns under the Executable Campaign. You might going to ask why especially for those two? Because they are using-- Like, when you create the interesting moment or scoring campaign, how will they use the token strategies? And with Executable Campaign, you can actually carry over the token from parent campaign to your child's campaign. So your child campaign looks like using the trigger token and trigger web page under the Executable, and there's no tokens. But again, because of the parents is actually have all the token values you can't see on the description. We built this one in our sandbox. Again, it's our very near term. We want to put them in the Executable Campaign to eliminate that racing condition, put to that important information in front of the sales without the back and forth and also scalable using the scalable alerts option.
What are some our key takeaways? As I mentioned before, data is important, but data strategy is everything. Build your strategy first on the paper. You might not need to use actual paper paper. You might want to use the build whiteboard or any tool that's on your computer, but don't start to build it on Sandbox or PROD because the processes itself is very large, giant process, and it's easy to miss one step. When you build some in front of you and see whole process, you can easily point it out, "Oh, this needs to be changed," before you start to building and testing it. And the second one is, make decision on the best solution. I am a very fan of the Executive Campaign. I mentioned several times it's literally saved my days and sleeps. But just because Executable Campaign worked for me, it doesn't mean that it's going to work for you. You might not need it that very high level of, very complicated data hygiene strategies. Request Campaign might going to be answer your needs, or you might have a more complicated issue than my issue. So Executable Campaign, it might be a part of your solution, but you might need to edit additional, like the solution to your issues. So it's very important to know what is best for you and for your environment. And the last one is testing. You guys probably hear every presentation. The testing is our world. The testing is going to give you great insight and actually, testing is going to continue in the process itself. It's not just going to end for the one testing.
And just move it. Yeah. And then a couple key takeaways specifically for our team was really those operational efficiencies or headaches on a daily basis of manual efforts being reduced. Really seeing that clean data being synced on a regular basis, reducing those errors by 95%. The way it was built in a modular manner allowed us to scale with the business complexities that were changing. Qualcomm is still in this digital transformation that we're going through, and so their business is changing every day just like your business, and so the modular aspect allowed us to make updates and changes without needing to update hundreds of campaigns that were related to data updates, data normalizations or calls to a different central place. And then finally, ensuring that that data stays normalized throughout its life cycle. This shouldn't be-- I think Miray touched on this earlier. This shouldn't be a one-time process that you push somebody through. Data changes throughout its life cycle and it's constantly needed, not just from a sync to Salesforce perspective, but also from an audience management perspective. Integration, if Qualcomm has CDP, so that drives a lot of the audiences in their CDP as well. So ensuring that that data is maintained within your Marketo instance allows it then to be federated out to those other platforms cleanly, and ensuring that integrity.
So that's our presentation for today. If you aren't aware by now, there's a survey. Please take that survey. You might have Starbucks or headphones. The last thing we wanted to touch on is the Champion Program and User Groups within Marketo. Both of us are major advocates of both programs. We've been a part of it. It's helped us both in our knowledge and peer-to-peer sharing as well as our career. The one on the right is the Champion Program if you want to learn more about it. The one on the left is joining your local user group, or there's several virtual user groups out there as well.
Any questions? And we have some T-shirts to give away, guys, for the questions.
[Music]
