The turn of the century saw a boom in data collection, where businesses were hungry for insight and placing increased value on the business-critical information that could put their organisations above the rest.
Hand in hand with new technologies, this heralded a new, golden age of data insight – businesses across every industry could now track their ins and outs, retailers could better predict purchasing habits, and doctors could keep hold of more detailed medical records.
But now, as we’ve seen from the most recent cyber attacks on high-profile Australian organisations, data is incredibly precious, with many businesses realising too little too late that having all the data they want can be a double-edged sword; the more data there is, the harder it is to manage, and the more at risk organisations are of a cyber attack. If there is data missing, stolen or corrupted, the whole system collapses.
In a world where everything we touch turns into data, enterprises will benefit by adopting an Autonomous Data Management (ADM) approach to monitor the vast amount of information within their systems beyond what any human team can manually manage.
Drowning in dark data
Think of a time when you’ve shared a critical document with colleagues. You make a copy of your original document to share with two team members, but it also needs to be shared with someone in Legal and someone in Finance – that’s six copies of the same document being stored on your company’s servers.
Years down the track, this same document is still being stored on your company’s servers, but it is no longer being used to derive insights or decision-making. This is what we call ‘dark data’, where a business accumulates massive amounts of data that exceed its ability to maintain or analyse.
In fact, Veritas research has found that 33 per cent of data stored by Australian organisations is dark, while a whopping 68 per cent is considered redundant, obsolete or trivial (ROT), well above the global average of 50 per cent. Dark data growth has soared with more organisations working in a remote and highly distributed environment in recent years.
This overload of unused information and data-driven decision-making can be problematic for a number of reasons. Not only can dark data create significant waste in employee productivity when searching for old documents, but dark data can also contain highly sensitive information which, if compromised, can lead to significant legal, financial and reputation tarnish.
On top of this, the servers that store duplicates of this dark data create enormous volumes of carbon pollution due to the amounts of electricity required to run them. Veritas calculated that, in 2020 alone, businesses’ storage of dark data was estimated to have contributed 5.8 million tonnes of CO2 waste to the Earth’s atmosphere.
It is critical that IT leaders take their data into their own hands to avoid such risks and maximise the benefits of full data control. After all, with this dark data piling up so quickly, you’d have to be superhuman to sort through it all – so then, what is the solution?
Reducing the data burden through ADM
In a rush to fast-track digital transformation and hybrid working models due to COVID-19, IT leaders worked quickly to implement the technology infrastructures necessary for business continuity, leaving security as an unfortunate afterthought.
Veritas has called this phenomenon the ‘vulnerability lag’, with research revealing there is a two-year lag between the deployment of new applications and having the appropriate protection strategies in place to secure them – that’s two years of being vulnerable to ransomware and two years of leaving the door open to potential compliance breaches.
Not only that, but Veritas research has found that, on average, Australian businesses would need to hire 27 full-time staff and spend US$2.3 million to close these gaps within 12 months.
In a tough talent pool, organisations must lean on solutions such as artificial intelligence (AI) and machine learning (ML) to augment the skills of an existing IT team and process greater amounts of information at speed. This is called Autonomous Data Management (ADM), which sees cloud technology platforms learn data management practices and automatically apply them to new data sets.
This is historically a manual, time-consuming task, relying on an IT member to tell the system where data needs to be stored, how it is used, and when it can be deleted. But with the capabilities of ADM, proactive data decision-making, storage and protection can take place autonomously and transparently, without the need for human intervention.
ADM enables enterprises to unlock full cloud benefits, such as operational scale and agility, whilst also allowing for
- A reduction in storage space by optimising the way data is held and deleting the data that isn’t needed
- A reduction in the amount of power required to store data and associated CO2 emissions with the deduplication of data
- Enhanced security by eliminating room for human error and the risk of downtime
- Increased operational efficiencies by minimising manual labour
With the threat of cyber criminals and security breaches not going away anytime soon, businesses must act now to manage their data efficiently and securely or risk significant IP loss. This new era of data management through ADM is an opportunity for businesses to put businesses back in control of their data and restore the powers of big-data-decision making.
Keep up to date with our stories on LinkedIn, Twitter, Facebook and Instagram.
