Big Data, Big Challenges

Given the right data knowledge and strategic partnerships, the possibilities to generate significant corporate revenue from big data are massive and it can be used to assist cash flow forecasting too, although there are many challenges along the way before this potential is fully realised.

Firstly, it takes time and resources to mine the appropriate data, and analytics capabilities require a sizable investment. Secondly, given the immense amount of data regularly generated by corporations and financial institutions, most organisations don’t even know where to begin in taking full advantage of all of the information they gather. Thirdly, although there is much talk about combining structured data with more fluid unstructured information gathered from the cloud, social media, video and mobile devices, actually finding staff with the knowledge to do the job right can prove challenging.

Indeed, executives are faced with a major data management challenge in creating any value in mushrooming data volumes, and most lack the knowledge to fully utilise the data they hold.  There is the prospect of better financial planning and analysis (FP&A) at treasuries, but only if the data management challenge is met.

Where to Start?

Companies need to decide exactly what they want to get out of business data and what data is deemed important.  Bear in mind that about 2.5bn gigabytes of data is produced each day – and 90% of the stored unstructured data in the world has been created in just the past two years. That’s a lot of data to sift through, so any business should know exactly what it wants before it starts digging.

Once companies understand this important data they need to become more analytical in their approach to data management, to ensure that only useful information is kept and combined with existing data to provide insights.

In addition, as information is pouring in from a number of places effective deletion and de-duplication policies should be in place. Finally, companies must secure all mobile and static media that store data. 

The last two points – the secure erasure of end-of-life data and effective security of live data on all forms of media – are important to emphasise as they are growing areas of concern. There are a number of risks related to the storage of unencrypted media, particularly when in transit between company sites and long-term storage facilities. Over the past three years the negative press and costs associated with losing customer data has made banks more vigilant in trying to remove the risks before they can cause damage. 

Nowadays, the banking sector and most enterprise organisations have managed to eliminate this risk from their primary backups and most recent archives by transferring data via a secure wide area network (WAN) to their storage facilities and then archive to encrypted tape. The big headache that organisations are encountering – and often turning to professional services for help – is related to the older tape archives.

Companies have the option of either restoring all their data to disk-based storage, within their secure data centre, or restoring and then backing-up again to encrypted tape. Then there is the headache of degaussing – the process of decreasing or eliminating a remnant magnetic field – and knowing how to quality control the erasure process. Very few companies are able to fully destroy data on magnetic media in-house. Even fewer can prove that degaussing in-house has been successful without third party assistance.

A major bank recently used Kroll Ontrack to validate its degaussing process and its team of engineers successfully recovered the data. It turned out that the bank’s existing degausser was not powerful enough and created the wrong shaped magnetic field for it to penetrate multi-platter workstation disk drives.

If a company is just using offsite data destruction for unencrypted media, then the media is at risk during transit and from the quality controls of the destruction company. Degaussing onsite reduces this risk, but again, the system has to be powerful enough to destroy all data.


The benefits of harnessing big data are immense. The more intelligence that firms have on their customers, the greater the opportunity to cross-sell and build a bigger business. But there are ways to go about tackling the data – and knowing which data to use and how to safeguard it should be on the top of the agenda for any organisation seeking to make data one of its greatest assets. 


Related reading