The Traps of Big Data

Aug 28, 2021 | AI and Big Data, Text analytics

With all the buzz around big data nowadays, it is easy to get carried away with collecting as much data as possible. People think to themselves that with more data they will be able to better improve their business making decisions. This is true… but to an extent and with caveats. Over at Semeon, we pride ourselves in our ability to make use of data for the companies we work with. This has been our mission, and since our inception, we have been able to help brands such as P&G, Honda, Toyota, Pepsi, Sephora and many more leverage their data for actionable insights. 

Along our journey, we have encountered a countless number of firms who have fallen into some of the traps of big data. Some of the major pitfalls we see are as follows:

  1. Vanity Metrics

This is the number one problem we see with companies today: they focus on data that doesn’t matter. It takes time, but it is important to determine what data matters to your company. What data is going to help increase user retention? What data is going to help you better understand the needs of the customer? What data is measurable and consistent? Focus on quality over quantity. Too many companies love the idea of having data that they forget what the purpose is for. The purpose is to create insights that allow you to make better, more informed decisions.

  1. Un-Actionable Insights

This leads right into our next point: actionable insights. This problem is deeply correlated with vanity metrics but requires its own mention due to its importance. Many companies we have interacted with collect data for data’s sake. Perhaps this data can be analyzed ex-post but it is much more effective to know why it is being collected, ex-ante. If you can’t summarize what the data is being used for in a few lines, it is time to reconsider whether or not it is providing any use to the company.

  1. Lack of Statistical Significance

Insights can’t help you if your results aren’t valid. When you find trends and interpret data there is the question of whether your results are statistically significant. This can be figured out with little knowledge of statistical analysis and is necessary to confirm trends. Using data sets that are small in size can lead to a misalignment of strategic priorities based on information that isn’t true.

  1. Not Leveraging it enough

I’m sure we don’t have to explain to you the importance of data. Chances are, if you are in the workforce, you have either dealt with analyzing data firsthand or at least contributed to data entry. Economist Intelligence Unit reported that 60% of the professionals they quizzed feel that data is generating revenue within their organizations and 83% say it is making existing services and products more profitable. The uses of it are seemingly limitless and continue to be improved upon, yet still, there are copious amounts of companies not leveraging it to their full capabilities. As the famous manager, Peter Drucker says, “What gets measured, gets managed”. Our suggestions to companies that use data is to evaluate what else they could be measuring. Is there anything in the company that needs to be improved upon? If so, is it measurable? Will measuring it interfere with workflow? These are just some of the questions companies should be asking themselves on a regular basis.

  1. Misinterpreting it

“It is much easier to maintain and bolster views that we currently hold than it is to create and develop new ones” – Daniel Kahneman.

Confirmation bias is one of the biggest dangers to big data. It is a cognitive bias in which one possesses a hypothesis and actively seeks out data to support it – ignoring the data that reject it. The worst part is that people do it unconsciously. Over at Semeon, we have been able to statistically show some companies that their firmly held assumption were false.

We have also seen a decent amount of firms mistake correlation for causation. Just because data suggests two events are interconnected, it doesn’t mean that one definitively caused the other, even if it makes logical sense.


We hope you enjoyed the article! If you are interested in receiving a free consultation call on how to improve your company’s use of data, we are more than willing to assist and offer insight. Contact us by clicking the “Try Semeon” button to the left of this article.

Check out the link below to learn more about our platform. Or give us a call at 1800-630-6000.

Related Articles