The Power State of Dark Data.

Have you ever heard of “Dark Data”?

It may sound like we are in The Matrix movie, but it’s really a simple term.

Let’s say for example, that you work in a company. And one day you realized that there are a lot of data that can’t be used, analyzed, or even found because of certain reasons or circumstances.

Well, all of that data is called “ Dark Data”.

Dark data is data that the organization is unaware of or unable to find or even fail to use.

Well, I guess that wasn’t hard, right?

What’s hard is that, despite the effort from companies on gathering data, it seems like most of the organizations still aren’t using data properly as soon as they get it.

According to Splunk’s State of Dark Data Report, a lot of organizations have a lot of dark data. The report was a joint effort between Splunk and TRUE Global Intelligence. The companies surveyed over 1,300 business managers and IT leaders.

Can you imagine that over 60% of companies believe that half of their data is dark?!

Not only that, but a third of them believe that over 75% of their data is dark.

75%? That’s huge.

I am wondering how we get all these are incredible and huge numbers.

“Data is hard to work with because it’s growing at an alarming rate and is hard to structure and organize. So, it’s easy for organizations to feel helpless in this chaotic landscape,” (Tim Tully), chief technology officer at Splunk.

I did some research and realized that there are high challenges which are faced by these organizations:

1. The huge data volume.

2. The lack of skills and resources.

3. Not having the right motivation from employees to gain more data skills.

4. Bad management & leadership.

In the report, they mentioned that 69% of employees said that they want to continue doing what they’re already doing, regardless of the impact on the business or their own career.

In simple words, all these employees will have a very hard time in structuring and organizing the data. Unless if they were working in a data-driven role.

Personally, I believe that we live in a world with very huge and complicated data. And our world is getting more complex every single day.

Different Regions, Different Attitudes About Dark Data

There are huge differences between different countries and regions towards dark data. Whether they were opinions, actions, or even education.

Let’s take China for example.

Nowadays, you can see that people in China are so confident in their abilities about AI, even if their current adoption of AI isn’t that much higher than the global average.

But they believe that they can handle their dark data in manageable and efficient ways more than the other countries or regions.

Even for countries like France, Japan, and Germany, respondents said positive answers about 25 percent less than respondents from other countries did.

Data is so valuable. I hope that companies will start to think about their Dark data strategy.

Because by the end of the day, there's a huge power state of dark data and we must bring it into the light.

Everyone should be part of the solution.