[Music] [Preeti Patel] Hello, everyone. Welcome to the session.

Embracing GenAI: A gamechanger for customer experience. First, I'll start with a few introductions. I'm Preeti Patel. I'm Senior Partner Technology at TCS Interactive. Joining me today are three illustrious leaders who've done a ton of amazing work across marketing, digital transformation, and customer experience.

Greg Stuart here, who's nodding his head, is the CEO of MMA.

MMA is the leading body focused on architecting the future for marketing organizations. You have over 800 companies globally and your board includes CMOs from AT&T, CVS, Walmart, et cetera, execs from Facebook and Google as well.

Greg has held positions as CEO, CMO, and CRO at Sony Online Ventures, Y&R, and other large companies. He's an early leader in adapting technologies to marketing. He helped establish the standard measurement for advertising called viewability. [Greg Stuart] This feels like too long a-- Even I'm bored. Well, you well deserve it. This is my favorite. When you're old enough you get to-- You've done enough stuff, right? This is my favorite. Greg co-created MTA, which is Multi-Touch Attribution. Multi-Touch Attribution. Yeah. We didn't call it that in 2001, but that's what it was, right? Yeah. Rachel Mercer, who-- It's still goddamn hard to do.

[Amit Manurkar] I can vouch for that. Yeah. No shit. It's really just ridiculous. I don't know what we didn't fix that, but well. Okay. Sorry. Rachel Mercer, who I've had the fortune of working with, is an award winning strategist and design and product genius who spent her career leading teams to create groundbreaking new products, services, and experiences. She's named as a gamechanger by Forbes, one of the most creative people in the industry by Business Insider and a disruptor by The Drum.

She's currently Head of Customer Experience, Design and Product Innovation at JPMorgan Chase. Prior to this, she was a cofounder at Proto and Head of Strategy at RJ where she led clients like Ally, Verizon, Beam Suntory, and Unilever. [Rachel Mercer] Hi.

Amit Manurkar, he's an avid traveler with a passion for traveling and combines his passion with his work with a decadelong career in travel and hospitality, currently VP of Digital Transmission and MarTech at Marriott International, leading the charge with building a cohesive and futureproof ecosystem and personalization at scale.

- Amit is-- - That's what we are trying to do. Yes.

Amit has also helped support great growth at Etihad Airways and Nestle. So welcome to the panel, everyone. - Thank you. - Thanks for having us.

We can leave now, right? [inaudible]. Just wrapped up the session. That's it. It's a packed room, and everyone's here for one reason. We all know that customer experience is a gamechanger and is a leading driver for growth. For years, we've talked about the promise of delivering consistent, real-time, hyper-personalized, intelligent customer experience. And now with GenAI, this promise is turning into a reality. At TCS, we run several mission critical customer experience projects, and we're seeing how GenAI is reframing how brands understand their customers, connect with their customers across the funnel, information synthesis, and insights generation.

GenAI has also been known as the biggest disruptor and the biggest hype.

And brands are investing a lot, but it's challenging to balance the value and the risk. And it's critical to understand the challenges, the opportunities, and how do you kind of balance the short term and the long term.

Greg, Rachel, and Amit have deep insights and very different perspectives, so we look forward to an interesting conversation. There will be time at the end for questions, and we look forward to them as well.

So I'll start with Greg. Greg, we're seeing a shift with marketing strategy being tied increasingly closely to customer experience. Yeah. What are your thoughts and insights about that? So, actually, I can add a little bit to that. So let's set the ground and sort of cover customer experience before we start to add even tech to it. So MMA has been, over the last six, seven years, doing a series of marketing organizational on strategy design work, understanding how do capabilities align to strategy and does that produce optimal performance? This is the studies that have been done with both by Global Board, which is mentioned as the CMOs of some of the biggest companies out there and a team of academics.

What's interesting about that research, the thing that caught us-- Well, first off, let me just level set. There are three basic marketing strategies, and only three, everybody.

We need to focus so that other people start to understand our business. Those are brand, obviously transactional, what we often call, direct to consumer, and experience. And what we have found through our research is some early indicators that customer experience is the number one driver of company performance today. Not brand, not where I grew up. I grew up in brand. I worked in Procter and Gamble business for years. It's not that. It's not to say the brand isn't important and it can't be a driver. And it depends, in part, what your strategy is. That's a decision that you, CMO, CEO, CFO, all need to make. But this interesting dynamic of customer experience being the single driver of the best financial performance for companies really caught us off guard early in our work. And I want to be careful because it is early, and it does relate to some companies. It will vary a little bit by sector and so on, but it's a pretty important point. In fact, I said in fact, I can name names on this. I said to the CMO of General Motors about three years ago when she was my board chair. I said, I think that-- And I'm a nonprofit trade associate if you didn't get that, right? So I have nothing to sell.

But the goal there, as I said, I think that my job, the job at the MMA, is to migrate brand CMOs because that's how most of them got there to be customer experience CMOs. She didn't even hesitate. She said, "That is my journey." So I don't know where you all come from. I assume you're all customer experience oriented here, but just to reinforce, that is the right thing to do. The chart that's showing up here is a breakdown of-- And I don't want to get off on this too far. We'll just sort of touch on it. But it's 72, and the work that the academics did with the Global Board, we defined there were 72 marketing capabilities required in every organization to deliver both either customer value, which is on the right, or firm value, which is on the left. They're all required. If you want to write me, I'll send this to you. Or if you look for-- I think you'll find it probably. If you look on Google, you'd probably find this. But Greg at MMA Global, and I'll give that again, but happy to send it to people. But it's basically mapping out those capabilities and what capabilities are aligned to each marketing strategy. And, again, getting that right is what drives performance. And I'll explain and talk more about that later. But it was a pretty big undercover for us. Okay. Great. And, Rachel, what growth opportunities and strategies are you seeing in this space? I mean, I can't always speak for everything that we're doing across JPMorgan Chase, but I love the research that Greg and his team have done in different and showcasing that customer experience really is the core of it. Customers are always judging you based on their best experience, their worst experience, and their last experience. And so for us especially, we're oftentimes looking at servicing and complaints as being one place where Generative AI and our service agents can start to show up together. I think we're also seeing that it's very helpful in much more like responsive spaces. So, I think Alaska Airlines has a really great use case around this recently where everybody is asking questions around if their plane is or is not a Boeing plane lately. And that is something that's starting to absolutely overwhelm their customer service lines. And I think that they have a really good internal engine where it's identifying when they get emails answering that question and sending it back. I think there are places where I always like to send her on pain points because I'm a design thinker at heart. So that's where I always think that this can lead.

Okay. Great. And, Amit, what about the trends in hospitality and travel? So, again, customer experience is at the heart of what we do, how we do it, why we do it, right? So from that perspective, I think I wouldn't even call it trend. I think, like, this has been kind of how we work. Yeah. And this is how hospitality, specifically Marriott, has always done things, right? I mean, putting customer experience at the front and center of everything we do, and I guess that is the reason why we have probably such a large loyal customer base who love to stay at our hotels, and we are glad to have you and sort of kind of we do our best to provide you kind of the best service that we possibly can. And within that technology, specifically AI, more specifically Gen AI, has a big role to play in it on continue to optimize that experience further, kind of make it seamless, kind of not just be intuitive but also potentially predict, hopefully right most of the time in terms of what exactly you are looking for, not just based on your past behavior but also based on what you need today, right, which oftentimes is not the same as what you wanted three months ago or a year ago. And that's where I do see plenty of opportunities, specifically within GenAI space to kind of bring that to forefront. Awesome.

And, Greg, because you have the year of a lot of CMOs and you're kind of hearing their pain points and their point of view as well, what are they saying? What are their-- What do they see as the biggest opportunities? What are the biggest challenges? Yeah.

I don't think they know what the answers are yet. I mean, we just basically reframe it into sort of three different things. And in fact, we have another event coming up here called Possible in Miami pretty soon. We're going to release some of this work. But we basically found, and there's been some other research done around this, is that there's basically three core areas, right? Efficiency, performance boost, and then game changing on the business. And so we've tended to bucket ideas, and the MMA is currently running a series of experiments in each one of those areas to try to validate what's the real opportunity. Is it big? Is it small? What do we not get? What don't we get? What's the variation? When does, under what conditions, would academics do pretty well? Like under what conditions is it true and what conditions is it not true? And so we'll be releasing that. I'll talk maybe here a little while about some personalization of ads that we did that sort of gave us some insight to that. I will say, though, it was funny. I had a board call yesterday with, again, some of the world's biggest CMOs. They just have to be involved aboard. And I was surprised at-- I heard them a year ago sort of indicate importance of AI. And yesterday, I heard a completely amped up version of that.

And it was like, what are we doing and how fast are we getting there kind of thing. So I'm not sure what's happened in the last month or two, but something has shifted for these big enterprise CMOs. So I'm not sure if you're all feeling that, but I suspect that there's a whole new element of this that's going to come to fruition here in corporations. Yeah. - Do you agree? - Absolutely. And I think in that way, like, AI is acting sort of very differently from kind of the previous, I would say, technology advancement just in terms of the adoption. And I think all of that is driven by some very tangible opportunities that we see ahead of us, right? Yeah. Like as you mentioned, like it's a journey from, like, efficiency to kind of the game changing. I sort of put it as AI as a sort of a copilot to coworker, right, kind of that spectrum. And I think most of the companies are still sort of in that copilot space. Yeah. But I think quite a few of them are running sort of pretty fast on kind of making AI as the coworker, like, truly sort of invent new business model or, like, do things completely differently than how they have been doing, right? So it's not just about improving the efficiency by a few percentage point, but oftentimes, it's eliminating a set of wholesale processes like that can just be done by AI, things like that. Can I ask the audience-- Can I ask the audience question? Can I just do that? - Sure thing. - Yeah, okay. How many-- Listen, this is maybe a biased group here, but I'm curious. So the question would be, how many of you think AI is the most important thing that's going to happen in your lifetime? And how many of you think that, "Meh, it's like meta. It's a little bit of a flash in the pan?" Okay. So are you ready? So how many of you think that AI is a little bit of a flash, it's probably going to die off, it's not going to be that big of a deal? How many of you think it's the most important thing you're going to see in your lifetime? Okay. Just checking. I assumed so. Good. I just want to make sure that you're not ignorant. Okay.

Yeah. No, there wasn't a single hand for the first question, which was good.

Amit, it's interesting you say that because we keep hearing the term digital concierge mentioned a lot. So concierge kind of assisting you across various tasks, across various-- Across the life cycle, if you will.

Rachel, can you talk a little bit about the strategies and the tools? I know you're passionate about tools and operationalizing. Sure. So I think there's two sort of stories here. Like, one, I was a lifelong consultant before I came to JPMorgan Chase. I loved the language that Amit used earlier around AI really being a partner to humans. It's not necessarily like a one for one replacement for employees or anything like that. Where I do think there's some like very interesting work being done is, there's a consultancy called Redscout that basically uses it to create a fully integrated sort of like customer that anyone can talk to, which is a really powerful thing, right? Like how many of us actually get to go out and talk to our customers each and every day, right? The ones who drive our customer experience are those best, worst, and lasting experiences. So having the ability to sort of "talk to a customer at any point in time" is a really amazing thing. I'm also seeing other consultancies sort of take either the four Cs or sort of the four perspectives, whether it's like a strategist, a technologist, a designer, use that to sort of ingest or shorten the discovery period that they have with their clients in order to get to concepting and therefore user testing a lot faster, right? Because like AI is very good at hallucinating things, editorializing things, but it is not necessarily a replacement for true insights from your customers. So using it to sort of shorten some of those processes to make it more accessible or to better validate your processes is something that's personally very exciting for me to see.

Greg. By the way, I really like that word hallucinating instead of predicting. Yeah. I'm sure I'm going to use it somewhere. I mean, I think it's important. I'm sorry I'll go off on a tangent for just a second. But I think it's important to talk about what AI is good and not good at. - Exactly. - Right? Like, it is very good at editing and editorializing. But we think that it is the same as a computer that if we were to ask ChatGPT create two numbers for me and then help me understand what the bigger number is, it's actually going to make a lot of guesswork within the sort of world that you created for it. But it's not actually very accurate in a lot of those hard coded things that you would expect like an Excel spreadsheet or it is then Logic Tree to be able to do. So exciting when you think about it as brands being like these shared hallucinations.

But again, like something maybe not to be trusted in terms of like immediately, I don't know, ingesting contracts or something like that that has much more serious implications. I heard an interesting quote. It said, GenAI is-- Hallucinations with GenAI is not a bug, it's a feature. If you ask a human being the same question twice, you will not get the same answer. I thought it was super interesting, and eventually, it's actually one of the key features and abilities of GenAI as it'll unravel.

I'm confused by my family making up stuff all the time, so maybe it's very similar to that. I don't know. I never do that.

So, Amit, I know you're doing a lot of work practically and tactically. If you could talk a little bit about the building blocks...

Your journey, the key ingredients? Yeah. I mean, I-- Again, I feel like we are still sort of on, like, day one of year one of AI, right? So, like, I wouldn't say, like, we have done anything significant. I don't think we have even thought about doing anything significant just yet. Now that doesn't mean like we're not doing anything. But I think it is important, and, Rachel, like the point you made, right, I mean, it's just, yes, we all want to adopt new technology, new ways of working, but there has to be a level of discipline, right? We do need to, sort of constrain ourselves.

We do need to keep doing things, but we also need to know whether we are doing right things. Are we doing them right way? Are they performing as well as we expect them to perform? And I think that is where a lot of our focus has been, so kind of as we-- Well, actually, let me take a step back. So one thing, I guess, we have learned is I think it makes a lot of sense to do sort of the POCs and pilot and kind of learn from those and kind of continue to accumulate learnings. But as you are kind of trying to then do the real thing, like, instead of kind of scoping yourself to pilot, like, always scope yourself to the real deal, right? I mean, how will you take this to full scale production? Because oftentimes, things work very well in a POC and piloting, and everything looks green. All the problems start to happen when you try to scale it up. And so I think if you can start with where you're going to scale it and then sort of work backwards so that everything you are doing in terms of your cost analysis, in terms of what it will actually take to pull it off from an organizational perspective, you are aware of that, and that is then what is informing your sort of the step one. So that is being sort of kind of our approach, not just to AI but to kind of a lot of things, but it has specially helped with this fairly new technology.

Yeah. That's helpful. We do hear about a lot of brands trying them as pilots first before they kind of go for scale and doing it in the right way with, as you said, kind of having the bigger picture in mind and planning for that as you kind of set up these pilots is super important.

Greg, can you talk to us about some of the personalization work that you do? Yeah. We have, we-- So MMA as an orientation is-- Listen, our mission statement, as mentioned, is architecting the future market, and I often say that we're here to save marketing from marketers is the mission statement. And what I mean by that is that there's a lot of things that we believe to be true that aren't true, and there's a lot of things that we just don't know about in marketing. And MMA operates in their series of think tanks to try to provide guidance to markers about where we think real value creation is. Okay. So one of the studies we've done is we actually did create an AI per se driven personalization using contextual signals. So creative personalization, not full GenAI, just to be clear. There's very few marketers, I think, are willing to do that in the wild at this point. However, we did sort of have a very contained series of creative units. In fact, you might have-- Yeah. Pull that next thing up. This is the study from-- No. Go back to the-- Show the ads there if you can. Okay. So these are the ads that were created. They're really not that special, to be very frank. This is for Kroger's. This is an actual campaign that ran. And what we did is we chose different color backgrounds, different headlines. There's a different element, like, they had their Krogi. They have a thing called a Krogi. It's a little character. And so they used different elements, and what we did is the machine learning, it was machine learning, not Generative AI, just to be clear. And then we basically said personalize the ad using purely contextual signal. When we started this exercise, we were just trying to find out if cookies went away, could we still personalize ads. That was the original thesis of the research.

So the machine created 72 versions of the ad, and then go to-- You have that another slide? Yeah. This is the different signal it picked up and the decisions that it was making like that.

And what we found is an increase in performance versus what Kroger's was doing now, so how they were optimizing currently to this new AI machine learning approach. Let me call it clear. I'll tell you what it was. Machine learning approach was plus 259% in performance. Blew us away. In fact, when the researcher called me, it was done by a guy named Rex Briggs, a pretty famous guy. He said, I don't believe, he goes, "Greg, the results are unbelievable." He told me, I said, "I don't believe it" because that's what he said, it's unbelievable. He goes, "I don't believe it." He spent another 80 hours and redid all the analysis, went back to the core data, and came back and said it's absolutely true. We've now run three other-- Well, we've run four other studies at this point to try to understand what is the value of machine learning driven as an element of AI. Like I said, I don't have a brand who will do. If you want to do Generative AI with us, we would be happy to execute and experiment with you and set it all up. And what's interesting about this is, just so it's a whole new way for marketing. One of the big learnings from the board and the members around this is that what's happening there is it's optimizing using-- And I'm not a technologist at this level, but it's using, one hot encoding with K modes clustering, which is an approach within optimizing. I see some people nodding their heads. They know what that is, so we can let them explain that later. But that's what's going on, and the performance of this just really blew us away. We didn't expect anything near this. On average, the campaigns have done 195%. Yeah, you can show the summary of all the campaigns. One of the campaigns, though, didn't work, by the way, which was interesting, and it's all right. It was the GM1. We don't think they personalized. Either the company that we used in that case didn't really have AI like they said they did. We don't know. Or they just didn't personalize enough is what we think happened.

But all of those are much closer to the purchase point than GM? So, we were still moving them. Basically, it was a digital KPI. You're correct. In the case of Kroger's, it was any visit to five ecommerce pages. So how Kroger's looked at it, the GM 1 was 2. I don't think it went all the way to car configurator, but it went at least to some pages on the website. But you're right. And we don't-- We actually now are rerunning-- We're actually doing experiments now where we run all the way through to sales and for the companies that we can do that with. But I'll tell you, having done a lot of experiments with a lot of research and a lot of different approaches, I've never seen anything as strong as this. Unbelievable. Insanely so. Yeah. But I mean, this brings up an, like, interesting point, like the self-combining now the power of GenAI with the machine learning. - Right. - Right? I mean, like, you saw on the screen how many or, like, say, 50 different variation of the content. With Gen AI, I can produce 5,000, like, at fraction to no cost. And then we can iterate through what actually works, right? And so I think it just opens completely new paradigm. What happens when it was very cost prohibitive till now because it's like you're going to an agency and you will spend all this money. And so, like, you have to be very thoughtful about how many variations you're going to have and all of that, and it just that constraint entirely goes away. Yeah. Well, I mean, it's interesting you say that. So we have done another experiment that's not gone public yet where they actually produced 700 versions of the ad. That was the combination could have been done, and it failed.

And so our thesis, we don't know. We're still picking up signal in different places. What we think happened, it was just too much personalization if there is such a thing. So I don't know. We're going to have to do-- I mean, I put that out there as a thesis to be explored, but we just don't know, which is interesting to that point. No. I mean, this is-- But still the production GenAI would make it so much easier to do. - You could run it more cleverly. - Exactly. And maybe I think that is where sort of some of that counterbalances because, like, I think we all like to think that we really, really know our customers. That's really not true, right? We are guessing most of the times and oftentimes we get it wrong. And if you then hyper-personalize it based on that wrong information, then what's going to happen is exactly what you mentioned, right? And that's where maybe I think with the with the GenAI aspect, I think we have the ability to maybe sort of create more content and, like, may not be that targeted, especially when we are not sure whether that's the right level of-- And I agree with you that we don't know. Like, when we did an audio one. In fact, the monday.dot-- Yeah, monday.com was an audio one, and we tested actually different voices, female, male voice. And if you AB test split that, the female voice performed 20% better. So any marketer worth their salt would have just run the female voice everywhere, right? That's what we would have done. Okay. We ran both to see what happened. What happened and I have no judgment about this. I'm just giving you the data as we understand it today. Basically, across the southern states, Utah and Michigan, the male voice performed better.

We think our thesis is patriarchially oriented culture societies, so preponderance of that, greater than you might find in other parts of the world or the US. That's what we think. That was our thesis. The male voice always performed better after 10 o'clock at night.

So I think to your point, like, there's so much of this you just can't guess. We have another one for an auto, for a non automotive for a fuel company, whatever. And it performed really well. Formula 1, we couldn't have seen it coming. When Formula 1 was active, the performance of the ads went up dramatically.

So I just think we're going to-- I think that you're right to your point, like, we don't know, we're guessing. Yeah. And we still may be guessing, but the machine is probably going to be better guessing than we can, would be my guess. And it will be a lot more efficient than us in that. I mean, all right. So I'm going to apologize because I told you guys I was going to be a little bit spicy. But I have, like, a completely opposite perspective on this. I do think that there is, like a very, taking things to like hyper-personalization, testing 150 different variations, like we'll get you some efficiencies so far, but I'd really like to see like long term brand value and growth sort of come off of the back of that. I think there is, we, as consumers, witness this sort of in everyday, right? Like if any of us go on amazon.com and are searching for an umbrella, we have no idea what a good umbrella is because we know all the reviews are fake. We know all the brands are made up. And all of the images are AI generated as well, right? So context and brand has suddenly become meaningless in this context. So like, as marketers, if you are going to trust your brand in some sort of generative context, I would make sure to again think of yourselves as sort of the editors and partners alongside of this. I think a great example sort of showcasing the need for curation is Netflix. Netflix saw huge drops in engagement. They arguably know exactly what you want to watch all the time. But they actually saw much greater success and engagement in their platform when they introduced things like the top 10, or people like you also watched things like this. So that people wouldn't spend all night trying to figure out a movie they're trying to watch but instead could get right to doing the thing that they came to Netflix to do. By the way, I would agree with you, just FYI. However, I will say we've actually have developed and adapted the methodology to work against brand. Right. So if you're interested in doing a study to understand brand, we've not yet picked up somebody who has. I think maybe there's some in the works, so we might have some information later this year around that. But you're right. I'd like to understand those differences because as I know from the MTA work I've done in the past, if you ask me what's the optimum marketing mix, I'd say what's your objective because for brand, the mix is completely different than the optimal mix for sales. And most people-- I see somebody nodding their heads. Most people don't understand that. It's like we're so unclear about how marketing works and some of what we need to do to become really good at this.

It's just the state of where we are at some level. - Yeah. - Yeah. I'm so glad we're talking about marketing measurement and experimentation and optimization. We see a lot of opportunity with the combination of AI and GenAI. So at TCS, we have an AI driven product called TwinX, as you guys are aware of, which uses digital twins. So twins of your customers, your products, messaging channels across multiple dimensions to experiment at scale. So forget about the days of AB testing. I mean, that's way gone, but experiment at scale in a virtual environment, understand what is working, and then validate and implement in a live environment. So just thinking about the opportunity there, especially across marketing and customer experience is mind blowing. But I mean, there's a lot of experiments going on. I'm really interested to see where people will kind of test which use cases are they going to kind of go for first and then how do they scale that out.

Rachel, I wanted you to kind of let us know about, like, how do you deal with a data security privacy compliance? Is that something that comes up? Yes. I mean, I think the good news for maybe any other consultant here is that how we just went through 10 years of data modernization and data lakes, if we continue to have AI, GenAI projects, we are going to continue to have data modernization and sort of lake creation. I do think, I mean, JPMorgan Chase is very strict with PII and making sure that anything that ingests customer data is sort of appropriately used. We also have to make sure that we're extra aware of anything that is going to drive risk and fraud. So not to place judgment on Cash App, but Cash App recently, for example, runs a lot of their, like, card replacement processes and things like that through an AI model, which are easily sort of mimicked. Like, as soon as you open something up, anyone else is going to try it. So the way that we've been thinking about it within Chase is, again, always as sort of like a partner to servicing. Like, everyone, 20 years ago when the ATM first came out was completely panicked that the ATM was going to absolutely replace branches and replace tellers, and no, what we have been working through with teams is that it's just going to broaden the scope of what you offer. And Chase continues to invest massively in growing its branch footprint because we understand the value that it offers to customers and we certainly offer supplemental experiences there. So a lot of the experiments that the team is doing is around how can AI sort of partner with our servicing team around a variety of experiences. We have the third largest booking engine in the United States for travel booking, so it's working with travel agents in that way to addressing customer service complaints. And it's been a very fun time for sure. Great. Yeah. I think you bring a good point. I mean, with AI just like before any technology before this, right, I mean, going all the way back to, like, the personal computers to Internet and everything. I think it's just helping us kind of advance up the value chain, right? And I think there are just so many mundane tasks that sort of our teams do, which I don't know if people truly enjoy doing those or not. But I mean, if you can use AI in a way that sort of takes away some of that, then that gives us the opportunity to sort of deploy these amazing resources to do kind of something that is of a much higher value. And I think that then also ultimately translates to, kind of, we call all our employees associates, which I'm just going to say like the associate experience, right? We all want to do bigger, better things, and I think AI is sort of one of the levers in sort of getting to that.

I mean, I forget the economist who said it. I think it was Keynes who said that his grandchildren weren't going to work. So I'm ready for that to happen with AI so we work little less.

And, Amit, can you talk a little bit about the roadblocks and the challenges or any other implications that people should be aware of? There are so many. Where do we start? But, again, I think we talked about some of the-- At least my perspective on how you should do it and all of that. I think the other thing to maybe consider is the current cost structure, right, which I think for a lot of the use cases and I'm just going to make up an example, but, say, if I wanted to create 100 personalized experience for you when you visit marriott.com, like, if you want information on a specific destination. And if I want it to be real time, it's cost prohibitive to do it today, right, and so I think the cost factor and, like, people look at the token cost, right, whatever is the running market value, like 30 cents per 1,000 token or something like that. But once you start to convert that into number of pages or content and multiply that by the monthly or annual traffic, like, you will quickly realize I guess not every idea is implementable today. And so when you want to implement something, then you got to take care of building some of that.

And that's why I said, like, you need to consider that scale in mind, right? Because if you just ran a POC about how can I deliver completely personalized experience using AI, it will work flawlessly, but then you will never be able to scale it. But if you think about scale first, then hopefully you will come up with an approach that provides that stopgap that has like, for example, caching and things like that built into it. So I would say like just the technology implementation of AI, I think that's-- You really need to think through that at scale.

Hopefully, somebody creates an AI driven calculator that gives you those optimizations. I'm sure they do exist today. And we-- I used it to come up with these cost examples.

Yeah. I mean and I'm sure like as we move forward just like with any other technology piece, like that cost will continue to go down in a very significant way, but I think we are still far away from that.

And, Greg, a hot topic is the CMO and CIO, CTO collaboration-- Yeah. Is so important to make these types of initiatives work. What have you seen, are you seeing in various brands and organizations? So through some of the marketing org research I mentioned earlier, we've sort of come up with a phrase that marketing is no longer managing a function, it's managing a coalition, and that it's much more integrated to the business. We also have a chief digital officer board and their integration. I find a lot of-- Some CMOs are hesitant to get into sort of the technology side of things. A little bit unfortunate, I think that's going to be to their own detriment over time as would be my guess. And you see a lot of-- That's why I think some companies have installed sort of marketing or customer oriented CDOs, chief digital officers.

I don't know as to how that relationship works out. We're actually about to present a bunch of research that we've done within the company to try to understand those connections. And so it's just-- It's a very complex-- I don't think the two sides talk to each other, I don't know. I could kind of go on and on, but it's a complicated relationship at this point, it feels like. - Yeah. - Yeah. I mean, that's a complex one because it's the art and science coming together.

- When they speak different languages. - Yeah. Yeah. - I've heard some-- - They have different objectives. They have different sort of orientations, and that's where a CEO at some point or the board needs to pull the company together in those regards, so. Yeah. I've heard some people say that it's almost like you need a translation service or a translator that kind of helps bridge that gap. But I truly feel that brands that figure that out will be ahead of the others.

Yeah. Let me ask you both. Sort of a funny question, though, to that regard, what you just said there. So listen, competitive advantage is accomplished from sort of being ahead of everybody else. If everybody else is doing it and you're doing it, then there's no advantage.

So I know there's an argument for being safe...

But there's a stronger argument for being more aggressive about it, right? I mean, you're complicated because you're financial services, so the risks are unbelievably big. Yeah, we have to be safe with all your money. Yeah. I took some back away from you guys. - Yeah. Yeah. - I was a little concerned. No, not for that. I'm just kidding. No, I'm just kidding. But don't you think that there's sort of some risk to not sort of being more aggressive? Like, why wouldn't your companies put pedal to the metal, let's go. We have one of them, the Kroger's example up there. I don't know if I can say this. Don't tell him I said anything. They're actually implementing Silicon Valley innovation theory, which the 70, 20, 10. And, honestly, they're the only marker I've heard really doing that at this kind of scale. They literally set aside 10%. So they've done far more experience than just this one we do with them here and other things of that the MMA has and other experiments in other areas. But I don't really hear that a lot. And I'm kind of surprised that marketing driven companies aren't oriented that way...

Because they fall behind to miss that opportunity, isn't that your-- Isn't that the company's responsibility to shareholders? I mean, the company does have like a center of excellence who's dedicated to it. It's just extremely controlled and who is allowed to use it and not use it. Obviously, no, nothing that's feeding back into the publicly available algorithm or anything like that. And only like a certain amount of analysis. Again, ideas that I would get excited about is like we have a Chase Insights Platform, right? Any research report that anyone has done in the consumer bank lives in a portal that you can go and like trawl through and find surveys, research, whatever else. I would love to make that something that's more accessible so you could, instead of having to get exactly the right query to pull from that database from, like, there are certainly, like, internal functions that we could apply things to, but I just don't-- I don't begrudge Chase for being cautious with these tools because we don't want anyone accidentally putting in either like analytical investment data or customer data that could put things at risk.

And again, I feel like we all are inherently risk managers, right, I think. One of my bosses once told me like as you move up your career, like, you are becoming more and more like of risk manager, right? That's sort of becomes your primary job. And I feel like the AI risk is, like, no different than any other risk, and it needs to be managed as such, right? I think I don't think there is a reason to sort of shy away from it, but at the same time, I think recognition of the risk and then having a mitigation approach around it, I do feel that is critical independent of what industry you belong to, right? Because, again, you do have-- There is a real danger of disrupting customer experience, and as you disrupt customer experience, it does have impact on conversion. It does have impact on brand value and those are not things that you can easily win back.

And so I do feel like you have to be thoughtful and you have to be very methodical as you sort of go in. I mean, all of us wants to go fast, but we do need to understand the risk profile and then act accordingly. I also think there's a strategic risk in sort of running straight into the sameness. Like, I don't know about you all, and again, like, most AI generated art that I see nowadays looks the same. Like, we can all identify exactly where it's come from. But where it gets interesting is if you get to an interesting prompt, right? I would like to see Ukiyo-e Batman. And so something that I've never seen before, but it's built off of two, like, very differentiated references. So from, like, a creative or a marketing perspective, if you have, like, an extremely distinctive heritage as a brand like Coca-Cola or if you have, as a designer, like, a very deep set of references, then you may be able to build out things that are more differentiated. But I think most folks are engaging at like a very shallow sort of prompt driven level. And that, I think, starts to, I think, generate a lot of things that start to sound the same unless you're getting, again, like, very good at prompt generation or things like that.

Yeah. That makes complete sense. The distinction comes from the person sitting behind the AI kind of who's bringing in the creativity and the capability. And I think it's so interesting that we are still sort of in the early stages of AI adoption, but like how quickly everything is becoming commoditized, right? I mean, you look at, like, major LLMs, they are-- Again, I'm not saying they are exactly at the same level, but they're going to reach parity. - Yeah. - Right. There will be very little differences. They all are operating off of, like, the same public dataset. So, like, that is getting commoditized. Underlying hardware is getting commoditized. And so I guess we need to now start defining those niches and then, like, truly invest in those niches, right? I mean, be it like the financial data, be it the guest activity data, right? And I think that's really, I guess, how we can create that competitive edge and the sea of, like, sameness that we are getting into.

Yeah. Wonderful. I think we're out of time. Thanks so much, everyone. It was wonderful having you guys here, and thanks to the audience. Thank you, guys.

[Music]