2.5x
growth in testing program
$1B
across incremental revenue and loss prevention during the pandemic recovery
Expanded testing mindset to value negative results as powerful insights
Marriott uses these solutions:
“Always start with a question, not the data. It's about understanding what the data means for Marriott’s customer experience and what question we're trying to answer around it, and then bridging the two.”
Susan Bloomberg
Vice President of Personalization and Product Performance, Marriott International
Data is the new destination
For a company that began as a nine-seat A&W root beer stand, Marriott has come a long way. As the world’s largest hotel company, it now offers more than 1.1 million rooms across 30 brands in over 110 countries. The secret? Constantly changing to meet the needs of their customers. Today, customers can engage with Marriott however they please. They could be booking a room, planning their vacation, or checking into the hotel. Marriott offers these experiences across its website, in app, in person, or through a combination of touchpoints. For Marriott to continue delivering top-notch hotel experiences, it needed to better understand how customers were engaging. This meant changing the way data was measured and interpreted. And bringing in the right expertise to make it happen.
A self-proclaimed Math Nerd — “capital M, capital N,” as she said — Susan Bloomberg was up for the challenge. As she took on the role of vice president of personalization and product performance, she brought that passion with her. Now, Bloomberg is enthusiastically sharing her background in data with Marriott.
Repacking goals into different suitcases
Following Bloomberg’s lead, Marriott has checked in to a new mindset. “We’ve changed the way that teams think about data. That means thinking about how it represents the user experience, not just data for the sake of data.”
To achieve its new mindset, Marriott needed to change the way its teams operated. While different teams still own various technology solutions, they now collaborate around common goals. “We are starting to integrate, which has been a very difficult problem for the teams to solve,” said Marriott’s Bloomberg. “Adobe’s consulting teams have been helping us on the Audience Manager, Target, and Campaign integrations for email. And to start to drive dynamic content in emails using our Adobe tools.”
Recently, Marriott created a powerful real-time connection between Adobe Target and Adobe Audience Manager. “When someone logs into the site, we can update their profile to reflect what we know about them,” said Bloomberg. “If we see someone new on the site, but then they log in, we can build out that profile for some really great experiences.”
But as with any change, getting teams to embrace data takes time. For example, Marriott recently adjusted the links on its website’s global navigation. At first, one team questioned how the changes might limit opportunities for their part of the business. "We looked at how many clicks from the global navigation reach into that part of the website,” said Marriott’s Bloomberg. “From there, I helped them see we could replace it with something more impactful for the customer.” Now, Marriott allows customers to easily find just what they’re looking for, whether it’s exploring vacation destinations or finding the right stay with one of Marriott’s 30 brands.
Testing travels the extra mile
Marriott has customers booking multiple times a year. So it might seem like the booking experience is very efficient. In fact, this is what Marriott told Bloomberg when she first started. But she wanted to explore this assumption. “Sometimes when you're a company like Marriott, people will put up with what they have to in order to book. But just because we’re doing well doesn't mean we're doing the best that we can. So, we’ve changed from working based off what we think we know about the customer and what they want, to measuring what they think and want.”
For Bloomberg, this means making sure experiments are set up for learning — not just results.
Recently, the search team wanted to test an entirely new filter experience that would allow customers to find and book the right stay more seamlessly. When they asked Bloomberg and her team to compare it against the default experience, she used it as a learning opportunity. “We could have done an A/B test, but then we wouldn't know which pieces of what they changed actually made an impact,” said Bloomberg. “We had to break it down into four tests, and they were really frustrated with us. But I think at the end of the day, they realized why we were doing it that way.” The team used Adobe Target to see how the different improvements they had made to each experience impacted how customers used the new search filter. And as a win for Bloomberg, they started to understand the value of using the right test to understand the customer experience better.
Bloomberg and her team used Adobe Target to test how different improvements to each experience impacted how customers used the new search filter.
Despite these successful learning moments, some teams are still resistant to testing. That’s why Marriott’s Bloomberg is changing the definition of what makes a test successful. “I actually still consider it a win when tests show a negative impact on revenue, as long as there is some statistically significant learning that comes out of it. Sometimes learning what our customers don't want from us is just as, if not more, impactful than learning where we got it right.”
In addition to measuring when feature changes might add to revenue, Bloomberg has introduced loss prevention to measure the revenue Marriott might have lost if a feature had been launched without being tested.
“It encourages teams to take more risks in experiments if they don't feel they're only being measured on the wins,” said Bloomberg.
Bloomberg has since built up an entire experimentation program to support the company’s data needs. When first joining in 2019, in the team ran 30 tests the entire year. Since changing the culture of testing, the team ran more tests in the first two quarters of 2021 than they did in all of 2019.
“Sometimes learning what our customers don't want from us is just as, if not more, impactful than learning where we got it right.”
Susan Bloomberg
Vice President of Personalization and Product Performance, Marriott International
Thinking like a data storyteller
While gathering the right data is important, it’s not everything. Marriott needed to understand its data in ways that were approachable. But traveling in the customer’s shoes is easier said than done. Now, Bloomberg encourages teams to take on a more creative framework.
“I do think of myself as a data storyteller, not necessarily a data analyst. Analytics sound scary to some people, but it's really just using data to tell and support the story.”
This storytelling framework is especially helpful when data doesn’t tell the full picture. Instead of viewing open questions as obstacles, Marriott now embraces a different perspective on finding solutions. “Data is not perfect, especially digital data,” said Bloomberg. “And one of the things I love about data not being quite so perfect is that it gives you some space to add art to the science.”
For Marriott, that means asking the right questions. “Take conversion rates,” said Bloomberg. “It sounds really simple, but in a very complex ecosystem like Marriott.com, customers come for more than just booking at a hotel. They come for account management.” When conversion rates fluctuate, teams now have the tools to uncover if it’s because more customers are opting for digital check-in or they’re just using the site to find information that will lead to booking in the future — like finding a specific hotel’s phone number or address on the homepage.
Instead of just looking at the numbers, Marriott’s Bloomberg has also shown the hospitality giant how to think like its customers — weary from a day of business travel or looking to recharge after exploring the streets of a new city. And then apply that mindset to improving experiences like mobile check-in.
“When customers arrive using their mobile key, we want to know what makes that experience easy for them,” said Bloomberg. “The data is there to help us figure out if the experience is working. Are people using this feature? If so, we’re looking to the data to see what they like and don't like.”
This mindset has pushed Marriott to test different features and functionality to offer quick information about hotel amenities, local dining options, transportation, or local attractions — at just the right moments to fuel a travel story that customers will keep telling.
Seeking the next destination, in real-time
The best stays make you feel at home. For Marriott, bringing that feeling to customers means knowing what experiences to bring them before they anticipate it. That’s why Bloomberg and her team are excited for what’s next: unlocking real-time, personalized experiences from the time customers book their stay to when they settle into their journey — whether it’s finding the perfect location, unpacking their suitcases, or heading out to shake off their jetlag.
“We’re focused on what we’re building for our customers next. Trying to anticipate those needs, not just for today's success, but for tomorrow and for the future.”
And the future looks bright. So far, Bloomberg’s experimentation program has measured over $250 million in incremental gross revenue for Marriott. They're on track for one billion dollars across incremental revenue and loss prevention, all from testing experiences that either worked or alerted teams to further improvements.
As for the company’s humble beginnings, guests can still get a treat with Marriott on Root Beer Float Day to celebrate where it all started.
More customer success
https://main--bacom--adobe.hlx.page/fragments/customer-success-stories/contact-footer
