[Music] [Jason DeFlorin] Good morning, everybody. Really appreciate you guys all coming today. I know Thursday morning's always a tough morning here at Summit, so thank you very much. So David and I obviously going to talk a little bit about the CDP and our AGO journey that we took at US Bank. It was a long path, but it was exciting.
As you guys know, I'm Jason DeFlorin. I've been at US Bank, I tell everybody forever, 25 years. I don't even remember life prior to US Bank. And I've got to do a lot of different things. And I will honestly say getting to work with David and the rest of the team here on our AJO and AEP launch has been probably the highlight of my career. We've done some amazing things and we're super proud of what we did. So I will turn it over to David to introduce himself. [David Barnes] So I'm David Barnes, and I'm a little loud. I am the product owner for Adobe Experience Platform, Real-Time CDP, AJO. And what am I missing? Target? Oh, am I officially target now? Okay. I don't know a thing about it.
And also offer decisioning, which we see as a separate product from AJO.
And one of the things we've been doing all week is talking to other clients. We did a one to many session yesterday and one of the-- First of all, that means I love getting questions. So I can't wait for the Q&A at the end. And so write down, be engaged. But one of the things we-- It's obvious is that every one of us has a lived different MarTech experience, right? I talked to one client who does all their work. They do pre-prep. They use Google Analytics, and they do all this identity prep and then load it, right? Another client doesn't even use the UI. It's all APIs. Another client just got their data prepped after five years of trying to get data and they're just starting to load. So we all have a different experience. You're going to hear our experience, right? And you're going to hear some things that didn't go well for us. I think you'll recognize a few of them, if not all of them. And we're not going to necessarily tell you how to fix them. We're going to tell you how we fix them or how we dealt with them and give you some of our tips on how to approach the situation, the tools and Adobe, how to manage Adobe. So you're going to hear all of this from us today, and I'm going to need the pointer.
Okay.
Yes, there we are. Oh, and by the way if you don't get your question answered, please reach out on LinkedIn. If you're not a vendor, I may respond. I will respond.
And so for the vendors in here, I apologize, I never respond. So what we're going to talk about today is AEP is a complex system, right, and all the tools on top of it. And AEP is definitely an adventure. You will go places you did not expect to go. It's not a linear journey and we'll illustrate that in a second. But you need to be prepared. The more you know and think about things ahead of time, the better. And you may not be told the things you should have thought about ahead of time. So the more you can learn that from conversations like this or ask your Adobe team, they'll try to give you, but they don't often live it like we do. So I think peer-to-peer is really important.
Find the right guides. That's what I was just talking about. There's some paid expertise. We'll talk about this in more detail. Paid expertise, Adobe expertise. But again, these kind of community conversations that happen naturally at Summit or that are planned are really helpful. And then finally, you will run into issues. You will need to deal with them. And we're going to give you some examples of situations we dealt with.
Okay. So wanted to give you a little bit about our situation. We started out, like most people, we had Adobe Analytics, we had Target, we had those tools, but we had Audience Manager and we had authenticated based messaging. We were using-- And I think you guys will get these slides. But okay, you can take pictures if you want. But we were doing a lot of authenticated space marketing using campaign as the messaging signal, and we would do an API call into campaign, and we basically broke the system. We were doing too many calls. Campaign couldn't keep up. And so we had to move to-- We were advised to move to Real-Time CDP.
The second use case, I think most of you had, or a lot of you have had, was just personalizing from site behavior using Audience Manager either on the website or in paid media.
We're going to give you a little bit about Next Best Action. That's our brand name. I've heard other people use the same brand name. I've heard different names for our authenticated based marketing. This is when someone shows up, they log in online banking and mobile, which is a critical part of our advertising because at that moment, you are a known customer and we have messages for you in that moment. We want to maximize that messaging. And AEP is exceptional with that. It combines the digital information of a digital app user with the offline data of a digital app user because you've authenticated. So it's really the sweet spot for us of AEP and Real-Time CDP and offer decisioning is that authenticated based marketing, having messages in online banking, mobile. We also do use that same capability in ATMs and in branch and call center.
So where we are now, we have adopted AEP, Real-Time CDP, what they call offer decisioning. That's now migrating to HAO decisioning. And if anybody wants to talk about that, reach out afterwards or ask in Q&A. And we're also doing more omnichannel where we're doing emails, direct mail, and paid media out of the tool. But again, our-- Well, we'll talk about that in a second.
Okay. So the goal here, and we tried to do this theme of a journey. This is where we're headed.
Okay, let me see. Okay, see if I can touch one button at a time. So we talked about the goal. Our primary goal is this authenticated on us that we call NBA.com. Very close second. Replacing Audience Manager with a more integrated experience. And then we wanted to get into more omnichannel triggered email. And that's the theme of our presentation, we ran into some issues, multi ID. Some of you may recognize that one. We didn't have Web SDK. We're not going to talk about that a lot, but I highly recommend Web SDK. If you don't have it, it will be a constant stumbling block. But we're not going to focus on that one. We have duplicate deliveries. Again, we're going to go into depth on these and array data personalization. But we wanted to answer some questions. We should have. We didn't actually do all this cool stuff at the bottom we're recommending. We should have been more intentional.
And there's a person in the room that actually built this thing. So, Jill, you'll recognize this. You can correct afterwards anything I say that's wrong. But we started out by defining our use cases and I mentioned the bulk of them and the channels. We had a data strategy, but it wasn't as conscious as maybe it needs to be.
But it was effective. And I'll describe that in a second.
Realtime data has become an issue. We probably should have focused on it more and that would've helped us prioritize Web SDK. And then finally, we didn't get CJA at the beginning. And that's starting to bite us. We're having a hard time proving value without having a well-defined analytics platform.
And I need to stay away from that. Okay. So these are these questions. Your map will be different. Like I said earlier, everybody's experience is different. Your map will be different. These are the questions we either asked or should have asked more clearly because they've been a part of our detours.
We were focused on marketing. This platform is great for servicing. It's great for holistic customer experience, but we prioritize revenue and marketing. That both limited us and made it a little easier to focus.
AJO, though, is a really good tool for both. And we're just getting there for combining both. So our focus was on marketing. I already mentioned our primary targets were authenticated space, which we call NBA and web personalization.
Identity management. And again, we did this-- I would frame our decision differently now than we did three years ago, four years ago.
We tried to do everything in the tool.
A lot of people don't. But our identity management, we took identities that already existed. We did not do any cleansing prior to loading. And we let the tool do its own identity management. We also got data from existing data marts.
They were extracted for us, so formatted, but they weren't cleansed. So all of our data was from existing sources. We loaded it, we let the tool ingest it natively. That's very efficient. We'll talk about that again some more later if we don't run out of time.
This real-time issue keeps biting us. And again, I don't think we prioritize it enough. We're getting pressure to do real time for everything. But the only place where I think it matters where we're really focused is on same visit personalization. And if you don't have Web SDK, when you move from Audience Manager to AEP, you lose what you had because with Audience Manager for site behavior remarketing, you had same visit instantaneous segmentation and personalization. If you don't have Web SDK, you lose that when you go to AEP. You have 15 minutes delays on average. So that was a big-- It bit us and I wish we'd been more focused in the beginning. And finally, we tried to do some analytics on around audiences and Power BI. It's been awkward. We're still very reliant on clickstream data for any kind of BI.
And we're piloting CJ right now, but it would've been nice if we'd done that sooner.
Okay.
There is no way as a new user that you are going to know what's about to happen to you, right? I mean, that's a given.
You just can't.
It's just too much. It's too complicated. You need someone who's been there before to help you at least design what you're doing.
We had a consultant, one consultant, primarily who's been with us for four years. We have him for 10 hours a week and he knows our system backwards and forwards. We call him our Adobe whisperer because he's integrated into Adobe. He's not an Adobe employee, but he's contracted through Adobe. He has access to their Jira boards, their engineers, their product people. He has been stunningly valuable.
But you're going to have to become an expert. We have funded most of our work through our own teams. We've trained up people, we've learned the tools. We have conversations with Adobe to push them to improve. And I'll talk about that a bit more too. But that's been driven from us. We do not rely mostly on contractors. We had this one and maybe some other help, but particularly this one, who helped us at the beginning and has helped us with big issues, like really complicated stuff. But we've become the experts. And you can do that by listening, having conversations, meeting with peers. I really believe that other customers have experiences you will not get from Adobe because we live it in a way Adobe does not live it. And we run into things Adobe does not in their environment, in their testing, and their design work. AI Assistant's really useful. I will say I don't know of any institution that's actually gotten approval to use it since they put out the GenAI writer. Maybe if someone's in here has actually gotten their org to let them use AI Assistant, I'm impressed. We're about to get approval. We were part of the beta and the alpha. So I know it's a great tool. I love it, but I haven't been allowed to use it since June of 2024 and I can't wait until I get it back. It's a great tool.
And then ask your Adobe team questions. That's what they're there for. If you need extra expertise, we get it. I hope that anyone can get it, but ask for it if you need it.
And hopefully, we'll have time to talk a little bit about that. And you got to have good employees because the thing is you're going to have to learn this. It's not always logical and intuitive. You need people who are smart, can learn things, and are mentally flexible.
A good communicator's really nice. But you can't always have everything. But you got to have people who are mentally flexible and can learn this stuff because it's new to all of us, right? There's very few. There's not 10 years of a legacy experience here. It's all growing rapidly and changing rapidly. It's very different every year. Ultimate Success. We rely heavily on our Ultimate Success team.
I will say Adobe engineering can be obtuse. And when you submit a ticket, you often get bizarre answers. And it helps to have ultimate success to interpret and to push back and to dig. So we have found-- I have found that team to be invaluable and irreplaceable in terms of communicating with engineering on tickets. That's my number one value. There's other values, but that's-- I think I could not replace them for that. And then as much as you can, alphas, betas, and client advisory boards, the more you work with Adobe and give them feedback and input, and the more you give them, the more you get back. You get road mapping. You get influence on their path, where they're headed. We wouldn't be where we are as US Bank without the engagement that people like Jason and I and others have had on these alphas, betas, and client advisory boards. We're on five of them I think, four or five. - So anyway, that's critical. - Yep. I just want to emphasize David's point about becoming the expert because I'll tell you, I think that was hands down our best strategy and we would not have been successful with that. And investing in our team, it allowed us to push back on Adobe, actually have intelligent conversations with engineers. So it was vital that we were becoming the experts and not just relying on our third party. We used them to educate us. They teach us to fish. I'll say this to everybody who will listen, that is to me, I thought was our reason, number one reason for success is, and then like I said, Adobe was their-- Samir, the person that David talking about. He was a great educator of us and embraced that. But that to me was the differentiator for us.
And we would not be here right now if we did not. Yeah, totally agree with that.
Okay, now let's get into some specifics. We've thrown a lot of-- - Hello. - Okay. We've thrown a lot of theory, thoughts, me yapping at you. But let's give you some specific examples.
For me to get excited about this presentation, I wanted to talk about the things that we were annoyed about in the moment.
I actually had names earlier, like where you're going to stub your toe and things like that. So they let me have this title, which is Managing the Unexpected. But these are the things that we did not know about that have been real blockers. They caused us significant pain for a while and then we resolve them, but in different ways. So we want to talk about resolution strategy when you run into something like this. So the big ones for us, multi identity-- - I don't know. - Okay. Hello. I don't know how many of you have run into shared device multi ID. If you're active users, I'd be shocked if you haven't. And it caused us not to be able to market to anyone whose profile had been collapsed because when your profile collapse-- Well, let me not get into the details. We have a detail page. The second one's duplicate signals. We sent five postcards to the same person in five days when we went live the first time with direct mail. And I'll talk a bit about that.
Personalizing from an array, we do a lot of deep links for credit cards and we're starting to do it for checking where you click through and end up in a specific site for that card. If you have two or three cards, it's one card. And personalizing from an array has been a challenge. And then we've-- I told you earlier we used this very direct data loading process, which worked great for three years. And then we had two major data fails three months ago, and we we're starting to deal with that.
Okay, these are going to be wordy slides and I apologize.
Multi ID, if I and my wife both log in to the same device, it has two people identities, my identity and my wife's identity and one ECID. And the way Adobe handles identity, if you have two identities on the same record, those become the same profile, right? And so when I log into that device, my wife logs in, both of our IDs get combined and all of our attributes get written on top of each other because each record can only have one attribute at any-- One value for each attribute. So when my data is loaded and then my wife's data is loaded, it would overwrite mine, right? And that's unmarketable because you don't know whose data came in first. You don't know what you're looking at. Segmentation. Could market them. So this was a huge issue for us.
And the way we dealt with it is we escalated it. We told Adobe it's an issue. They said, "Ah, it's a big-- It's going to take us a long time." We raised this in January 2022 and it was a big engineering investment for them and nothing happened for a year. And we kept complaining. And finally, we escalated and their head of product, Sandeep and Sandy, their top engineers and top product people talked to us about it. And they finally committed, "We're going to get this fixed." And we worked directly with their engineering. We were the first clients on the new tool. So if you're not on it, I think it's in limited release. It may not be GA yet, but they now have a shared device blocker where you pick an identity that cannot be brought together. You have a primary ID. In our case, it's our customer ID, our internal key and shared device. If you have two logins, it swaps them in and out in a way that makes sense. And you'll have to talk to Adobe about the exact process. But it fixed it, they fixed it and we're on it and it works beautifully. But it took us escalating and prioritizing and they had to hear it from a lot of clients before they put the money into fixing it, right? So when you give feedback to Adobe, it alters their prioritization and their roadmap. They put money where people complain. So I highly recommend if something's bothering you, talk to Adobe, make sure they know it bothers you. Make sure they know and you care. It helps to give them use cases and value. And we started throwing money at them, like this is how much we're losing because we can't do this. But communicate.
And we're going to show you just a few tricks just so you-- We're going to expose you to some code just for fun. So before we got the full solution, Jason actually went out and mastered a custom destination capability called Destination SDK. So when we sent emails, and we use Salesforce Marketing Cloud. So when we sent signals to Salesforce Marketing Cloud, he built out the destination so it would hunt for a multi ID. Then Adobe implemented a multi ID blocker. And if you had multi IDs and you turn that on, it just wouldn't even see them. You couldn't see profiles that were multi IDs. So they wouldn't segment, they wouldn't distribute. And then finally, they did the solution we're happy with, which is you can't even have a multi ID profile. So I'm going to let Jason talk to you about some of the coding he did.
All right. Yep. So we're going to talk through this and if you guys have some questions afterwards, jump in there because a lot of this is going to be specific to US Bank. So let me see if I get the pointer working here for you. So really this logic we built here. So I don't know how familiar everybody is with Destination SDK, but I'll tell you it was hard. When we went through this, it was hard to find somebody in Adobe who was familiar with Destination SDK. So it was back to my earlier point about learning the tool and that we were forced to learn this. And thanks to our Ultimate Success partners, they were able to help us find some folks on the tech side that were able to kind of partner with us to build this out. Really what happens when you're delivering to the Destination SDK in here, the identity and identity profile, the identity map, all identities are in there. So we walk that through and you can see we do the loop in here looking to see, as David had referenced, we have that unique ID at US Bank for Jason DeFlorin. I have one unique ID in there. All my products roll up, all my information rolls up underneath that. So we walk through that identity map table and if we see two of those, we don't have a multi identity situation. So we kick them out. And the reason-- We did actually a couple different things. We actually did a batch implementation with query services to do this. But it can happen in real time. Like I said, if I took David's phone right now and I logged in, instantaneously, we'd be combined and then we could send an email based on that triggering event. So this happens in real time. So that's what allowed us to kick them out so we did not pass it on to. And this example was going to our marketing cloud partners to send the email out. So we had to do the same code if you will in two different locations, one dealing with the batch just from volume standpoint. But then we also had to handle it in real time. And this is kind of how we achieved that.
Maybe.
Oh, there you go.
Okay, so the second issue we wanted to talk about or second challenge we ran into, which is still live. This one is not fixed. The previous one, fixed. Done.
So I don't know how many of you have ever worked in paid media, worked with something like LiveRamp and you're constantly sending signals to a platform, right? I have an identity in LiveRamp, and all of a sudden, there's a new cookie, LiveRamp can associate it and it keeps sending signal to the downstream platform because you want all the signals out there, right? So Adobe applied basically that logic to all their destinations, which works great if you just want to be ready when someone shows up. So that's paid media, authenticated space, and unauthenticated space. But if you're sending an email, I don't want to send an email every time my identity changes. If it's direct mail, I don't want to send a direct mail signal every time my identity changes. And by change, I just mean I brought in a new identity into the AEP identity graph. It's not that my email changed, it's not that my address changed. So the way we load data, we load all of our attributes every day. We never went to incremental. We didn't have a motivation. And there are reasons to not go to incremental for us.
But every time we'd load a file-- Not every time. Sometimes, and we never actually learned the trigger, but sometimes it would resend to the destination even though the customer did not reenter the segment. The segment realized date is still in the past, it's not today, but a new signal is sent to the destination. And again, we had one customer get five postcards on five straight days and we couldn't have that. So first thing we did was Jason put a blocker in Destination SDK and he's going to show you how he did that.
There are other ways to do it too for direct mail where we cannot be wrong. So the good news about email is Salesforce could also block it, right? They could say, "You can't re-enter this journey." So they were cleaning up some of our mess. We still didn't want to send. We were actually at one point sending people five times a week or so and we were swamping Salesforce. So we had to block it ourselves. But in direct mail, you just don't want to send two pieces of paper to the same person. So we built something where we write out the fact we sent it to you and remove you from the segment immediately after destination. That's not scalable, right? And they built something in AJO where you can look at the stream if you have a send stream in AJO, you can use that to suppress from direct mail. We have not mastered that. We're looking at it.
But this is our number one enhancement request to Adobe right now. Create a flag or a capability where I can say for this campaign, this delivery destination for this audience, I can only send each person once, each profile once. And after that, I cannot send them again for X period.
And I'd love for you guys to also ask for that if anyone cares because the more of us who ask for that, the sooner we'll get it. Anyway, you can just-- You can keep it.
Okay. We got a couple things going on here that we'll talk through. Obviously, the Destination SDK is very similar to what we talked about before. And we put it in, obviously, the Destination SDK for the realtime capabilities. Obviously, maybe six months ago, six, nine months ago, Adobe did add some augmentation to their destination delivery on batch that you can replicate this logic there and Ryan from my team has done that. So we can all do it at the same logic, which was not available when we built this in Destination SDK, but you can build it in a batch destination now too. Though, that's really complicated and-- It is. Yep, it-- He did that by writing a query, checking a query, writing a query off the query, checking that query. And I think it was like 20 queries deep before you could get one field built in the batch that works for this. Yeah. So it's viable but not viable. So, yeah, all this stuff goes back to about being an expert on the platform, right? So you can challenge those, and I'll tell you guys I'm very fortunate that we have a very good team. They're way smarter than I am. So I'm just up here talking, thankfully. But we do basically very similar to what we did here at the Destination SDK, we had very similar logic here around the segment here. Very similar, there's the identity map that gets sent over to the Destination SDK. So there's the segment map membership gets passed over there. We're doing a check here to make sure we find the right segment. You can see the segment ID up here so we know which one we're looking for. But obviously, as David had mentioned earlier, a lot of this stuff is not really very scalable. If you want to start doing hundreds and thousands of destinations, this is obviously not going to work. But it got us off the ground, which was the optimal and gave Adobe more time to fix some of the issues here. But we're going to walk through that and David had mentioned, here's where we're doing that check here on time comparing it. So if we know we need to kick them out or not that, "Hey, we already saw you come through today, we're going to boot you out and not let you come through again." So we handle that. And then over here, this is kind of our newer evolution of the same kind of problem that we're solving here at AJO. And as David said, we had been on Kitewheel originally, was a tool since acquired by CSG and I forget they renamed it now, but we migrated off that onto AJO. But in here, when you enter the Journey, we're actually taking in setting a flag that you're in the journey. So we can do this condition check here and you see, oh, already in Journey, then we're going to kick you out. We do need to do a little time check here too because you can get back into the Journey if you've been out for long enough. Then we want to get you back in here, but then I'll let you through. So it's just a simple check here that we kind of manage. But I'm also going to tell you it gets complicated because A great example I love is I could have-- I think this might actually be an onboarding. I could actually have two checking accounts I'm opening and be going through the onboarding journey twice. So we had to add additional logic in a few of these flavors to be able to handle that situation, to let multiple ones get in here and which ones you not want to. But the nice thing though is that we do have that ability to kind of take that on and be able to kind of kick them out. So we're really happy that that I go back to maybe add before about David's point with AEP because we originally did not have AJO. We had Kitewheel. And I think it became clear to us that we really needed AJO, and I'll tell you, I was probably frustrated a little bit with Adobe that they did not do a better job of selling us AJO at the beginning because I really think you really need them coupled together to really achieve what we were looking for. So I do highly recommend it if you do one to buy the other.
So I don't know how many of you have this issue. It was a big issue for us. And again, you can only have one attribute per profile identity. If you have a checking account, a credit card, and a mortgage for the same customer, you have to have an array because you can't have three distinct account identities and data sets. So you build an array. The arrays work great in segmentation. It is really, really easy to find a segment where one of the array elements meets a condition. But then let's say I have three credit cards and I want to have a message in the mobile app that says, "Click through to get your balance transfer offer." But I only have that offer on one of the three cards. So somehow I have to identify which of the three cards and therefore which of the three-- We use the last four of card number because any more than that is not allowed. But of the last four of the card number, which one of those was the balance transfer for so I can click through on the right one? So we've had to add a lot of logic in a lot of different places to find elements. And you're essentially rebuilding the segment to get to the right field.
And we're going to show you a little bit of that.
Yep. So, as David said, obviously kind of set the table there. Our products, obviously, as a bank, you can have quite a few products. So we do load that data in as an array in here. And so what we're doing in here, so now as David said, this segmentation works beautifully, right, that you got your second product in your array, got you qualified for the segment. What we do now because at some point in time, we're going to want to send that information that's in that array to the destination. So what we're doing in here is when you enter the journey, we're basically replicating that segment logic in there. And this is for our buddy Carrie in the back. I see where these are-- Our business partner that's been asking for this forever. So what we did, we replicate that logic here in AJO. And basically, we're now taking and going to grab that pointer so we know, oh, it was the right item too. We got a set that said what got you into this journey. And then we hold onto it. So when you get to the point in the journey when you hit the condition and we need to deliver it, that-- Oh, yep, it's time to send this to Salesforce, maybe direct mail, whatever the destination is. Now we know exactly ID, so we'll go walk the array looking for that ID and then we'll be able to grab that. In Carrie's case, the last four of your credit card to pass it onto the email to do another layer of personalization. But that's how we have been able to kind of get around that. So there is some, like I said, batch has that ability inside of AEP with the query services to do it. Now it's just a little bit more complex, but it's been very beneficial. And this gets into once again why I feel that AJO and AEP are really married together to really deliver everything you need to. In these kind of scenarios, you have to have the AJO as a part of your tool belt. Otherwise, you just won't be able to deliver what you need to. At least it was our findings.
There are reasons when you want to deliver straight from the CDP. This is a side note that you're-- Since you're selling AJO for them. I do want to point out that I was talking to one client. I get invited to talk to peers a lot and it's fun. I love it. But one of them was saying they had built 50 journeys in AJO and they had another 800 to build, and how are they going to do that? And I said, "Well, wait, why do you need 800 journeys? Why don't you just trigger it straight out of the CDP if they meet the condition, if it doesn't need a sequential touchpoint?" And they said, "Oh, you can do that?" So I was-- We all have a different lived MarTech experience. Just remember that. But that was fascinating to me that they were so obsessed or so focused on AJO, they'd forgotten that the Real-Time CDP is really good at delivering single message audiences to all channels. So it's the two together that are most efficient.
Okay, this is a new one for us. Like I said, we've tried to use the tools the way they were built. We load the data straight to profile. In many cases, we don't do checks on the data. And that worked for us for three years. We really didn't have major issues. And then they made a change to one of our source data systems. There was a marketing data area and they changed how they were building the data. And what that did was it increased risk. They made it possible for the data to calculate simultaneously in two ways and therefore create conflicts like collisions in the data. So one day, it was December 31st of 2024 that we got bad mobile usage data and we had an email campaign. If you've been to the mobile information page, but you have not been on the mobile app in the last 90 days, we send you an email. All of a sudden, we sent 700,000 emails to people who had been using the mobile app, including a lot of our internal product people and managers who then immediately contacted us and pointed out that we had just done this. Because the underlying data, which we did not create, someone else created, was calculated badly. They ran it at the wrong time, which hadn't even been allowed a couple months earlier. And then they did the same thing again two weeks later. So we got major pushback. It wasn't the same thing, it was a variation on the thing, but it had the same effect. So these people got this email twice that they weren't supposed to get in a one-month period. We are now building out more checks. So we're going to do-- And we're still building this because it's a fairly big lift. That's the great thing about just taking the data, trusting the data and loading the data is you don't have to do all this. But we're going to start doing. And we were doing record count validations but not before we loaded. We're going to now start doing record count validations. The record counts haven't varied by more than-- And they shouldn't vary by more than half a percent or a tenth of a percent per day. And I'm talking customer attributes, total customer base shouldn't be varying too much.
Anything that doesn't have a null should not have a null. That's a big one because the first issue was all nulls. That's what caused the issue. And it's easy to know when a field should have nulls or not. So that's an easy check. We're going to check every field that should never have nulls and see if they suddenly have nulls. And then we've loaded, what, 600,000 attributes. I mean, somewhere in that range.
We can't check every one of them every day for variance, but we're going to pick like this mobile usage one, the one that's bit us twice. We're going to check every day and make sure it doesn't vary by more than a set limit. And if any of those things happen, we stop it, we check it. We don't load it automatically. So this is something we're in the process of building right now and this one's on us. This is not Adobe. This is our bad, but it's important to the validity and the trustworthiness of the database, right, of the entire ecosystem. Yep, one thing I might add to that because requests we do have into Adobe that came up as a part of the cab that Dave was referencing earlier is kill switches, right, that we can-- If we see these data issues to be able to shut off a journey or delivery. Everything's in real time, right? So they qualify, if we have a data issue, we need to be able to pull the plug. So we're really looking forward to leaning in them a little bit to get some kill switches that allow us to pull the plug on a journey, our destination delivery instantaneously and at scale, right? So if we do it today, we have to go through and do it one at a time and then we got to remember to turn them all back on. So it's very, very cumbersome. And I know they are working at it. There was somebody from the AJO team back there that I know they're working on it for the Journey piece. You hadn't told me this. So Jason's in the AJO cab and I'm in the Real-Time CD cab. We had the exact same request, only our request, we wanted AI to say we normally send 5,000 to this destination every day and now we're trying to send 700,000. Stop, don't send anything because you've got a big queue and it's too big. - Yep. - And the tool should be able-- And then it would kill and we'd check and then we'd turn it back on. So yeah, it's funny they heard it from both cabs. - Yeah. - That's really neat. Okay, so I'm sure that one will get prioritized.
Okay, so in summary, the more you know ahead of time before you start developing, the better your outcome's going to be. I know that's kind of perfectionistic. You're not going to know everything you're going to hit. We all have a different lived MarTech experience. Make sure that you are working with people who have been here before.
Again, I cannot imagine going into this without someone who'd done it before. At least talk to a lot of people who've done it before.
Discuss your use cases with Adobe, meet with peers.
Again, different lived MarTech experience. Make sure it's relevant to you. Focus on the stuff that you think will be relevant to you. But the more you know ahead of time, the better. And I can't say that too many times. And always talk to Adobe. And we've said this so many times. The more feedback, the more constant communication you have with them, the better the tool will be for all of us and the better it will be for you.
And I think that's it until-- Yep.
[Music]
