A: One of the challenges of a recommendation engine is collecting and analyzing all the information on a massive scale. Being able to deal with literally billions of user interaction events across a catalog that may have millions of SKUs. And being able to determine and serve up the right recommendations in real time. It's relatively easy to do speedy requests if you're just doing a few customers an hour. It's also relatively easy to come up with millions of results if you have all day to do it. It's hard to do both at the same time. The engineering challenge is really in addressing speed and scale simultaneously.
Also, a lot of the computational techniques that are available to make sense of these millions of data points grow exponentially more expensive when you add the million and first data point. For example, if you’re building a matrix that's one million by one million rows, then every incremental product that you add is adding a million rows to that matrix.
Another challenge is the cold-start problem. If I have a new user, you don't know very much about their preferences. If I have a new item, I don't know how that item relates to other items.
Companies can also run into problems if they don’t analyze the data they receive. The most-purchased item may not be the best item to recommend. There's definitely a lot of work that goes into identifying the best models that work reliably — that aren't led astray by different customer behavior that ends up creating a poor recommendation because it’s simply reflective of overall popularity. That type of recommendation isn’t adding a lot of value because customers likely would have bought the popular items anyway.
Another potential problem that we could run into is just irrelevant recommendations — things that we're showing that customers aren’t interested in. That could have an adverse impact on the customer experience.
A final interesting problem is when you optimize your recommendations around the wrong metric. If you haven't thought about what metric you want to optimize around, you could end up showing recommendations that actually work against our business interests. For example, if you consistently recommend lower-profitability products, you could actually dilute the overall profitability of your business.