How to Derive Maximum Value from Big Data

For organisations it is a battle to turn insights from Big Data into faster, smarter and more compliant decisions. A simple statistic tells the story of why this isn’t so easy. Research group Forrester estimates that most businesses are only able to derive real value from approximately 12% of the data available to them. A lot of effort is put into combing through the immense variety and volume of data to separate the “wheat from the chaff,” or what data is noise, instead of into what data can drive analytic insights and actions.

The velocity with which Big Data accumulates and ages means that analytic insights can only impact business performance to the extent that insights can be quickly brought into operation and drive actions. While the effort to extract business value from Big Data is underway, a recent study of 300 business decision makers in the UK revealed that the financial services sector scored lowest across the board when it comes to Big Data capabilities.

In particular, a recently-published report from Opinium Research found that many UK organisations in the banking sector are struggling with compliance, lack of data analytics infrastructure and flexibility. Although 56% of financial decision makers confirmed Big Data’s potential to drive revenue streams and better understand customers, almost half revealed that their IT infrastructure can’t move fast enough to facilitate better use of such data.

The financial services industry is under greater scrutiny than any other sector and the regulatory requirements are not going to change any time soon. However, infrastructure shouldn’t be the thing that is holding its members back from making the most of industry-changing analytics – especially not in a world where you can easily access tools and infrastructure via the cloud. So, how can the financial sector embrace data analytics?

 1. Analyse all relevant data

Up to 80% of the Big Data available to businesses is text, speech, video and other unstructured data, so should not be ignored. A growing number of automated techniques for transforming these inputs into numerical representations can be used with statistical analysis to discover predictive features that can be combined with findings from traditional, structured data into predictive models. In one recent project, FICO demonstrated that a risk scorecard imbued with text-extracted insights lifted predictiveness by 8% over a traditional scorecard. In another project, analysing notes from sales inquiries, the addition of text insights enabled a scorecard to identify 3% more leads resulting in sales.

Across the wide variety of available data, extracting more value can require adopting a diverse set of analytic techniques, such as named entity extraction. Unstructured data analytics can increase the predictive accuracy of models, but, for banks and other financial institutions, the problem of Big Data adds to an already precarious balancing act between getting more analytic models into production to address a widening spectrum of business challenges, and ensuring these models retain sharpness over time. Even if an organisation solves the daunting challenge of finding the occasional needle in the Big Data haystack, there is considerable pressure on analytic modellers to figure out how to put this data to use. 

 2. Make insights impact operations

To achieve real business value, you have to be able to operationalise the results of all this analysis. Although this seems obvious, far too many projects are left gathering dust or encounter delays because it is too hard to leverage findings where they could provide value. The opportunity cost to the company – from all the suboptimal decisions made in the interim – can be immense.

Wise selection of data is critical. What looks wonderful in the lab may not be available or may be too expensive to obtain for in day-to-day business operations. Industry regulations can affect where and how data can be used. Also, most analytics require extensive calculations to be made from the raw data to turn it into useful variables. All this has to happen in efficient, automated ways to make accurate insights available fast enough to drive operational decisions and actions.

Analytic development teams must carefully consider how their models will be published and used by operations teams. Models that rely on manually intensive data processing steps, for instance, can cause problems at implementation. The quality of scripts and how well they’ve been documented may determine whether recoding is required for deployment. Such issues can have far-reaching effects, especially in regulated areas like lending and insurance underwriting, where they make it difficult to explain and defend data-driven decisions to auditors and customers.

Technology advances are helping businesses avoid these problems and speed up analytic lifecycle processes. A key reason for this improvement is the widespread use of business rules management, enabling applications to execute or access analytic models as part of making decisions. As a result, implementations increasingly revolve around deployable analytic libraries – sets of models, characteristics and business rules codifying additional logic needed for production. These streamlined methods not only reduce time to operational value, but also make analytic work easier to share and reuse for multiple purposes. In addition to being able to deploy analytics quickly, having a mechanism in place to enforce best practices in model lifecycle management helps avoid development and implementation delays. It also reduces the time and cost of regulatory compliance.

 3. Embrace analytic diversity

R, Python, Hive, Groovy, Scala, MATLAB, SQL, SAS… there are many new tools that the data analysis teams can take advantage of and new innovations are being developed all the time. Analytic teams will inevitably need to use multiple development methods to deliver the insights that business needs.
To get multiple types of analytic models to work together in an efficient development structure and robust production environment, you need a flexible infrastructure that embraces diversity. Fundamental requirements include the ability to operationalise models authored by a wide range of tools, by supporting extensible libraries, web services and standards such as the Predictive Model Markup Language (PMML). Centralised lifecycle management should extend across models, business rules and analytic assets from any source.

 4. Big Data means model creep

A bank or insurer group may have hundreds, even thousands, of models at work across its business lines and regions, many crucial to the organisation’s profitability and risk exposure. As the use of predictive and other models proliferates, however, only a small percentage of these businesses have instituted structured programs of model management to determine whether their critical models are remaining current and valid in setting the parameters of risk control decisions and identifying profitability opportunities.

With Big Data straining the model management foundation even further, organisations are faced with hiring new analytic talent to meet the demand for better, faster model development, which may not be possible due to budgetary restrictions or the increasing scarcity of such resources. What’s more, there is no guarantee their modelling infrastructure can stand up to regulatory scrutiny, help identify model degradation, or deploy new models on a timely basis.

When a financial institution relies heavily on the output of hundreds of models for analysing and making important financial decisions about customers, the accuracy of those models is essential for customer growth and profitability. As an example, one large retail bank serving approximately one out of every six Americans employs more than a thousand analytic models for making a wide array of customer decisions but wanted to gain greater control, be able to manage models and demonstrate regulatory compliance in the use of modelled decisions more efficiently, thereby driving more profitable, customer-centric decisions.

This leading retail bank adopted FICO’s scalable platform, designed to help firms maximise the performance of their analytic models and to increase efficiency across the entire enterprise and the complete lifecycle of models. Through automation and shared access to services, this bank has been able to bridge otherwise disconnected workflows and operations from development all the way through to model replacement and decision strategy. It can now monitor and measure models and their impact on comprehensive documentation for supporting regulatory compliance and profitability. In addition, the bank is integrating attribute lineage management into their paradigm, putting impact analysis at their fingertips to monitor and manage downstream effects of attribute change.

Big Future

The value of Big Data to business is easy to understand, but it’s not as easy to extract customer insights from immense stores and incoming streams of data in an actionable form, and in time to make a difference. Fortunately, a reliable set of best practices for Big Data analytics is proving itself in industries and markets around the world, and creating Big Data analytics no longer requires making a huge investment in expensive infrastructure and specialised skills thanks to cloud-based solutions By leveraging cloud services, companies can let a dedicated third party securely handle the underlying systems and services, paying just for the capacity and services they need.


Related reading