Machine learning can surely have immense impact on generating insights from the immense amount of data we create today, but is it ready to replace human-insights just yet?
I was having an interesting conversation recently with a product manager with extensive experience in data and analytics, who was vehemently against the idea of throwing resources at machine learning algorithms. True we were talking specifically about e-commerce applications, but it piqued my interest enough to come back and do a little bit of research myself.
For the purposes of this short-read, I refer to Machine Learning simply as a way to allow computer systems to “learn” based on data, without being explicitly programmed.
I had always imagined using machine learning to drive insights that would help me price products accurately, or target the right message for audiences, or help double our gross margins in low-margin categories (because why not?). The reality however is that most such attempts fail to take into account all the factors that influence the outcome. If you have ever seen e-commerce product recommendations at work, you know how they can provide some inane suggestions – and companies like Amazon have thrown an immense amount of firepower at it already.
The truth though is that sometimes humans can just do better. In the end, if your Category Manager can point out why last month’s top product isn’t selling any more without a pause for consideration, do you really need to teach your program how to do the same thing after looking at thousands or millions of data points? How do you account for the external market knowledge that there’s a new model out now that makes your top product less exciting this month?
There is a reason why most traders on Wall Street still don’t trust their funds to ML programs. There is a reason why Facebook still uses humans as the final authority on weeding out hate speech and fake news. Until we find a way to programmatically collect and analyse institutional knowledge that exists within the company, we should leave the insight generation to humans. We will also need to find a way to avoid us humans passing on our ideologies and biases to ML and AI programs.
At some point, we will be able to find a way to integrate the outside world to machine learning, allowing self-learning networks to imbibe information from external influences, combine them with realms of internal data, and provide comprehensive insights for business decision-making. Until then, don’t cut the people that are in charge of generating these insights.