Banks, pensions, and investment firms are challenged by an unprecedented glut of data. While every financial institution aspires to be “data-driven,” the reality is that many firms are still burdened by legacy technology, data silos, and manual processes that inhibit their ability to effectively leverage those rapidly growing volumes of data. The velocity at which data is generated presents further challenges. In most cases, useful information has an extremely short shelf life and needs to be extracted from raw data in near real time to provide value.
Accurate, timely, and consistent data is critical to making informed investment and operational decisions. Financial data spans many domains, from pricing and reference data to benchmarks, positions, and investable cash. Organisations employ large teams of specialists to manage and validate these disparate data sources in the hope of flagging and remediating suspicious data.
Today’s capital markets generate data at a pace that makes it difficult for teams to keep up, even at larger institutions. Finding the proverbial needle in a haystack is an apt metaphor for data validation. There are very few actual errors, but those that go undetected can create significant problems if consumed by downstream systems.
Artificial Intelligence (AI) and machine learning offer innovative and viable alternatives to improving the current rules-based approaches employed in data validation workflows. While many of the algorithms underpinning AI have existed for decades, three relatively recent advances have enabled the proliferation of new applications that banks and investment firms are creating for both internal and client-facing use cases.
First, the cloud has made it possible to harness massive computational resources on-demand. Organisations no longer need to build and maintain their own server farms to support these initiatives. Cloud computing lets firms leverage best-in-class cyber security, resiliency, and scalability offered by leading vendors like Amazon, Microsoft and Google.
Second, a new generation of cloud-native data management platforms like Snowflake eliminate time-consuming data movement and make it possible to assemble the massive and complex data sets required by AI algorithms in a matter of minutes. The ability to build AI models in-database rather than running structured query language (SQL) extracts significantly accelerates the productivity of data science teams and their ability to add greater value across the modelling process.
Third, open-source software has democratised access to powerful high-level programming languages like Python and Julia, enabling data scientists to focus on data exploration and model construction instead of time-consuming, error-prone coding.
Together, these transformational tools enable teams to rapidly build robust AI models trained on massive volumes of data and place those models into production to provide operations and investment professionals with actionable and differentiated insights in close to real time.
At State Street, our data science and engineering teams are deploying a form of artificial intelligence known as deep learning to flag anomalous and/or suspicious data, which is escalated to human specialists for investigation and remediation. By continuously learning from new examples across data domains, our AI is transforming the art and science of data validation.
The results of our initial prototype are compelling. While traditional rules-based validation methods flagged over 31,000 data exceptions across a six-month period (of which there were only 250 true exceptions), our AI identified only 4,000 exceptions while discovering one hundred percent of the true exceptions. The massive reduction in false positives means significantly less work for human experts tasked with investigating each exception as a potential error.
As a technology-led organisation, State Street believes that initiatives like this will enable our institutional clients to place greater trust in the veracity of their data when making investment decisions, extract new operational efficiencies by eliminating low-value activities, and more successfully differentiate their products and services to end investors.
State Street is a Gold Sponsor of IMpower 2023. Find out more about the 2023 agenda here >>
