Data analytics

A: When you think about analytics, the analogy I give is someone who is out panning for gold. Gold is insight, essentially. You're looking to find something interesting in the data that you can do something with. As a data analyst, you're the individual who is out there panning for gold. Analytics is your pan and your tool to sift out all the unnecessary silt — the unnecessary data — to find the gold within the silt.

We develop tools and technology to find meaningful patterns in data. They’re designed to break down data as part of the process to find meaningful patterns and insight. Analytics is meant to be done at an aggregate level.

You can take analytics down to as granular a level as you want to go, but the real benefit of analytics comes from scale. If you have a lot of data and you're looking at analytics to sift that data, you're trying to find meaningful patterns at scale. For a business, if I can find something that optimises one individual, that's good, but at the end of the day, if you can find an insight that optimises a $5 million market segment, that's really what you're looking for. And analytics is the process to do that, to sift through any amount of data.

A: There's no analytics without data. If you want to draw a distinction, you might categorise it as if someone is looking for data analytics, they're probably looking for analytics tailored to a specific type of data or dataset.

Data analytics might be the actual practice and use of applying technology to sift through data, whereas analytics in and of itself is the broad term that encompasses strategy and what the big vision of analytics could be.

A: You start with storage and infrastructure. Whatever data you collect has to be stored somewhere. The infrastructure component needs to be as robust as possible, so whatever different types of data that you're going to ingest, it can handle it. That's the first level.

Once you have the storage component, then you're going to fill it with data, but you have to have the data connected together and integrated. So the next thing that you need is connectivity — being able to connect disparate datasets together through some kind of identifier, some kind of deterministic or probabilistic way. And integrated data is different than mashing up data in something like Excel where you still take two separate datasets. That’s not something you can base analysis on. Integrated data is the important note there.

Once you have that, the next piece is processing or what we call “ETL” — extract, transform and load. Ultimately, what you're doing is you're giving that data shape and meaning. You're turning the raw data into an usable format, whether you're putting it in cubes or sessionising that data. You're taking the raw stuff and putting it together in an usable format.

Then you've got the analysis or the modelling layer on top of that, where you're actually using some kind of UI. Potentially, you could be using some kind of language like R or Python. But now that you've collected it, you've integrated it and you've transformed it, you're analysing it. And then the next portion is visualisation and interpretation.

Even when you look into Adobe Analytics, you can go in and break tables of data down, then put them into visuals so that they're more digestible, understandable and shareable. You're taking data and you're turning it into information, which in and of itself still isn't necessarily valuable. But once you get it to data visualisation, to the insight level, that's when it turns into knowledge.

Ultimately, there needs to be an output of analytics. It could be a segment or an audience that you've identified that you can send into a tool like Adobe Target and run different A/B tests with. It could be a predictive score. When you acquire new customers on the site, based on their traits or behaviour, you can assign a score to them. And then that lets the system know how much time or effort to invest in each individual. 

Connecting the outputs of analytics into systems of action, whether it's for content optimisation or ad optimisation, is really the last piece. Just being able to slice and dice data and have something pop up in the UI is not useful. You have to take some kind of action on it. An insight in and of itself has a shelf life because data just gives you a snapshot in time. Your data and insights will expire and after that point it’s no longer beneficial to pursue the opportunity that you found.

A: Data science and analytics have evolved quite a bit. Let's just look at digital interactions, for instance. Pre-digital, analytics was run mostly on CRM or ERP data, trade behaviour, trade data, basically. Once we had the Internet come out and then mobile, we gained online behavioural data that organisations can look at to understand trait-based behaviour.

The big change is the digitisation of different channels and that's happening at a much more rapid clip than in years past. We had web data, originally, with website data and web analytics. Then mobile apps, other mobile or other digital channels started to come in and now you're seeing things like voice assistant, voice analytics and speech analytics. We have location or spatial analytics that are coming into play. You're seeing just different things that are turning into data that can be analysed that we never would've thought of before.

The other thing that's changed is centralising all this data together and normalising this data so that we can have a more holistic view of the customer journey and customer engagement. This is something that's almost the holy grail for businesses today because they've only been optimising in certain channels.

Companies have been optimising web, but that's not how we interact any more. Let's say I'm buying a new truck and I go to the Ford website. I might do some research on the desktop, then I might be in bed later watching Netflix and then even later I might pull out the iPad and start watching videos. The next day, I might be on my phone at the barber shop, flipping through things. It's a multi-device journey that has to be connected, so that is really where we're at today.

Analytics is typically a rearview mirror look at a business. You're looking at things that have already happened and analytics, when it started out, was a descriptive type of tool or discipline. Over the last few years, we've started to look straight ahead as we're driving instead of in the rearview mirror. This is where we start getting into predictive analytics, where we take the historical and then we project out an expected range of values and we start making predictions on the next best action.

At the moment, we’re in the very early stages of prescriptive analytics where, essentially, what businesses are looking for is for the technology and the analytics tools to just tell them what to do.

A: There are two things. One is smarter technology, meaning more accessible AI and machine learning that augments what an analyst does. The other is more automation and interoperability standards between tools so that data can more easily be shared and flowed between systems.

Currently, organisations are not quite where they want to be. If you look across all the analytic tools and vendors and processes out there, it’s possible to stitch together something that gets down to prescriptive, one-to-one marketing. However, many of the necessary requirements aren't developed within organisations today.

Even with existing tech, organisations probably don't know how to actually act on multichannel journeys and predictive analytics in a cost-effective way. We also definitely need more AI and business intelligence built in to help surface insights and then to help take action on that insight.

At the end of the day, if analytics automatically statistically determines high-value segments and pumps out predictive scores, those should just automatically get ingested by something like Adobe Target and then it should run an A/B test. And then if there's a winner of that test, the people that manage that campaign don't need to go into a room and debate it. The system should just know to go with the winning test, to configure another test and to start testing additional segments that have been found.

Until companies gain access to better automation and analytics technology, they can start to improve by integrating the next channel. Let's say that you've been a web-only retailer and you start seeing that mobile traffic is increasing. The next channel of data might be your mobile app data and you can begin integrating it into your current analysis.

A: Analytics should be viewed as a constantly evolving cycle. You collect data, which is constant. You're always collecting data in real-time. That data is going to update and it's going to be dynamic.

When you think about what that virtual cycle looks like, you're collecting data. You're going through and you're analysing it — you're interpreting it and you're deriving insight. Those insights go out and get consumed again by some other customer-touching technology like A/B testing.  

You run a test and based on that test, there's going to be new data that comes in from people behaving on the website. If you do two different banners on a homepage, there's going to be not just activity on that page but downstream activity pushed through that site. And as you collect that data that comes in, you need to reanalyze it.

And as you find interesting segments, it's just a continual process of refinement where you're continually just sifting out more of that silt. You dip the pan back in and just keep sifting out. It gets finer and finer and you're just going to refine your analysis as you go on. And the idea is that you're going to get higher and higher efficiencies.

When you start with analytics, all of a sudden you establish a baseline and you may quickly see huge returns. As you refine them more and more, your conversion rates and improvements will likely come down some. But because you're operating at such a high level, even if it's a half a per cent improvement in conversion, that will drastically impact revenue down the line.

A: One problem companies run into is that they typically have stakeholders that have some kind of agenda and who will tease and finagle and wrangle data until it finally says something that supports what they need, even though that might not necessarily be true. This can easily happen when marketing managers want to show the effectiveness of their campaigns by attribution.

Attribution technology assigns credit to various marketing activities for conversion. For instance, if you send a bunch of emails out, people click them, add stuff to the basket and then buy. Because customers clicked from your email, that email is going to get credit for the revenue. If someone clicks an ad and buys, the display ad is going to get credit for that revenue. So for most organisations, you're siloed off into functional groups that have budget for those line items. If you run an attribution model and let's say that paid search gets a bunch of credit, the CMO will likely request providing additional funds to the paid search group. 

What typically happens is you have all the other channel managers fighting and start building different types of models to say what they want them to say. And so you may or may not actually get the truth from that because people are over-twisting or over-wrangling data for their own purposes. 

Companies can also suffer from a lack of maturity and understanding about how to analyse data. They may have a lot of individuals that aren't trained in analytics and now they've been thrown into roles where they're trying to do something with it. It’s also typical for businesses to not have enough data scientists — it’s a job that’s in high demand and there just aren’t enough of them to go around.

From a data collection standpoint and data storage standpoint, one of the biggest mistakes is people deciding to collect data and deal with it later. You’re not taking the full business strategy into account — and ultimately, data and analytics becomes a cost for the business with little return. 

From the connectivity side, a lot of organisations are centralising or trying to centralise data. Many companies decide to dump everything into a data lake. The problem here is from a connectivity standpoint. They aren't normalising the data or standardising the data. What that means is you actually can't integrate it very well, even though it resides in the same place. You can't quickly derive insights. It then takes a large amount of time to cleanse and normalise and standardise the data to answer one business question. And if there’s a lack of interactivity, a company can’t adequately visualise the data or apply intelligence and automation. 

From the ETL and the processing standpoint, a lot of organisations are really stuck in legacy paradigms. For example, with mobile, the average engagement or session is typically two minutes or less. That does not relate to or work in the old paradigm of a session being a 30-minute window on the web. 

The last problem is connecting your insights to systems of action. Companies struggle to take action across experiences, cobbling together technology and customised builds to try and retransform data yet again.

A: Data mining is very much a data-modelling type of capability. This is where you're doing a lot more programming, for instance. You're using different statistical programmes, database systems and tonnes of programming in between each. Data mining really is the realm of the data scientists, whereas data analytics is much more democratised. Data mining is much more raw.

A: We usually talk about three main categories or sources of data. There's first-party data, which is your owned data, gathered when someone goes to your website, for example. Then you've got second-party data, which is data sharing between you and a partner. For instance, an airline and a credit card company do some kind of joint offer. They're going into an agreement to share data back and forth to help both their businesses because they have customers in common.

Third-party data is anything that you can purchase in markets. You can go and purchase those big datasets and bind them all together to augment your first-party data. Companies can also access publicly available data, such as data from the government and weather data, among other things.