Are you in touch with your dark side?

Generative AI offers organizations the opportunity to create economic value out of their data but a significant volume of that data – so-called “dark data” – goes unused, constituting a massive environmental problem. Take this short quiz to test your understanding of the dark data dilemma and read on for three tips on managing your data in more profitable and sustainable ways.

What exactly is dark data?

a. The information you hold on individuals that is subject to data protection rules.

b. Documents, emails, images, and videos the organization has amassed over time.

c. Information you have acquired but which has no formal record regarding its source.

Answer: (b). It sounds sinister but dark data, as defined by Gartner, simply refers to “the information assets organizations collect, process and store during regular business activities, but generally fail to use for other purposes, such as analytics, business relationships and direct monetizing.”

What volume of data is unused and becomes dark?

a. 10%

b. 30%

c. 95%

Answer: (c). Staggeringly, less than 5% of the huge data reservoirs that organizations collect are used, while the rest sit untapped in vast lakes. Given that data centers now account for 2-3% of global greenhouse gas emissions, this presents a huge challenge to meeting carbon-reduction goals.

Why does data “go dark”?

a. Because someone decided it would be a good idea to store all the information that passed through the organization in a vast data warehouse but didn’t label it appropriately.

b. Because people who could exploit its potential don’t know it’s there or don’t understand its value.

c. Because it’s stored in different departments, leading to data silos, fragmentation, isolation, and a general lack of visibility.

Answer: All of the above – and much more. Many factors contribute to the build-up of dark data. Other significant causes include a lack of robust data governance, failure to integrate legacy systems, changing business priorities, organizational data illiteracy, and the accumulation of ROT (data that is redundant, obsolete, or trivial).

Managing your dark data: three recommendations for business leaders

Evaluate your AI use cases not just for their potential business value, but also for their environmental impact. Consider the carbon footprint of your AI applications and opt for alternatives that use less data (data minimalism) and less energy.

Adopt a structured approach to categorize your data and implement stringent policies for its lifecycle management, including archival and deletion protocols.

Rethink how you store the data and, most importantly, how you design and maintain your data pipelines (the process whereby raw data is captured from various data sources and then ported to data storage, such as a data lake or data warehouse, for analysis).

José Parra Moyano

Professor of Digital Strategy

José Parra Moyano is Professor of Digital Strategy. He focuses on the management and economics of data and privacy and how firms can create sustainable value in the digital economy. An award-winning teacher, he also founded his own successful startup, was appointed to the World Economic Forum’s Global Shapers Community of young people driving change, and was named on the Forbes ‘30 under 30’ list of outstanding young entrepreneurs in Switzerland. At IMD, he teaches in a variety of programs, such as the MBA and Strategic Finance programs, on the topic of AI, strategy, and Innovation. He is also the Program Director of Leading Teams in the AI Era Sprint.