Collect all data possible

In “Alchemy, the Philosopher's Stone and the Ten Capital Sins of businesses” I describe the ten capital sins of businesses when it comes to collecting and using data. The second capital sin is collecting irrelevant or excessive data.

Star Trek fans may remember the first Star Trek full-motion picture (1979). In it, the Enterprise needs to face the threat of an alien entity that is later discovered to be the Voyager VI space probe (V’ger) sent from Earth hundreds of years earlier. It had amassed so much knowledge, following the 20th-century mantra, "collect all data possible", that it had become intelligent and even conscious.

Many pundits love the analogy that data is the 21st-century analogue of oil and, according to them, the more you have of it, the richer you are. But even the price of oil, as it happened at the onset of the Covid-19 crisis in April 2020, can dip below zero. On the 20th of April 2020, newspapers around the world published the following news: "US oil prices turned negative for the first time on record on Monday after oil producers ran out of space to store the oversupply of crude left by the coronavirus crisis, triggering a historic market collapse which left oil traders reeling." Storing has its cost, and if you cannot use or sell what you are storing, the costs will outweigh the benefits. What is true for oil is even truer for data. While data storage costs have dropped significantly, they are not zero. This is the second cardinal sin of which many companies are guilty: collecting irrelevant or excessive data.

Some businesses fall into the trap of collecting vast amounts of data without a clear purpose or strategy. This can lead to data overload and make it difficult to extract valuable insights. It can also be expensive without providing any benefit and hamper or slow down data retrieval and backups. It is important to focus on collecting relevant data that aligns with business objectives and can provide actionable insights.

In "How to handle big data," I describe the four Vs of data: Volume, Veracity, Variety, and Velocity, and describe how companies should focus on all of them to improve their performance and the ability to generate valuable information and insights "that cannot be improved by volume alone, which, in fact, by itself can produce a negative linear impact on firm performance."

Having a clear strategy and developing clear KPIs is necessary to understand what data is necessary and useful versus what is redundant and unnecessary. By collecting too much data, extracting useful insights becomes akin to the search for the classic needle hidden in the large haystack of data.

Data is important, if used correctly, and similar to the Philosopher's stone, it can transform seemingly useless information into gold in the form of higher business performance and profits. However, similarly to how not every frog will turn into a prince when kissed, not all data is created equal, and collecting irrelevant and excessive data will not help any business thrive. The 20th-century (still true in the 21st century) mantra, as mentioned by Spock, may be "collect all data possible", but no one should expect that this will lead to (business) intelligence. In fact, V'ger may come back with a vengeance.