Big data analytics for market risk management

Discover Societe Generale's solution for big data analytics for market risk management with Opensee.

October 21, 2021
Share to


The ability to calculate market risk accurately is a key function for financial institutions and, as a universal bank, Societe Generale collects a huge amount of data on a daily basis. Societe Generale needed a solution that could help solve its data management and analytics capabilities, particularly around the new capital requirement calculations mandated by the Fundamental Review of the Trading Book (FRTB) regulatory framework. Banks now need to calculate the amount of capital they must hold to absorb losses from market risk at a much more granular level, which necessitates the management of huge volumes of data.

By implementing Opensee’s innovative data analytics solution, Societe Generale now has no limits on the size of the datasets. In addition, the platform empowers the bank’s business users to autonomously perform any aggregation, calculate ever-growing datasets and manage exponential data growth more efficiently, at a minimal cost without compromising on performance or volume. Having the right tools to access and leverage the most granular level of information and most relevant historical data, they can make advanced business decisions while optimising the bank’s resources, such as capital, credit lines, market risk, exposures, liquidity and so on.

“By implementing Opensee’s transformative technology, Societe Generale has been able to better analyse its risks, delivering solutions faster to end users.” Kevin Bruyère, Deputy Head of Risk on Capital Markets Activities, Societe Generale

Societe Generale Market Risk Department (RISQ/RMA)

Societe Generale is one of the leading European financial services groups. Founded in 1864, the bank has been playing a vital role in the economy for over 150 years. With more than 148,000 employees based in 76 countries worldwide, Societe Generale accompanies 32 million clients throughout the world on a daily basis. Based on a diversified universal banking model, the Group combines financial strength with a strategy of sustainable growth.

Societe Generale Corporate & Investment Banking (SGCIB) is the fourth largest investment bank in the European Economic Area and is present in more than 75 countries around the world. The mission of the Global Market Risk team (RISQ/RMA), headquartered in Paris, is to contribute to the development of the SG Group's activity by facilitating the objectives of the Business Lines while maintaining independent oversight through risk evaluation and monitoring.

Societe Generale’s Requirements

FRTB and the Data Volume Challenge

During the last financial crisis it was apparent that the regulatory capital for market risk was not adequate enough to cover these risks, therefore, the Basel Committee on Banking Supervision (BCBS) created a new framework called the Fundamental Review of the Trading Book (FRTB). As a result, large financial institutions have had to face a number of adjustments in their methodologies, systems and technologies for calculating the capital charge for market risk. In particular, a new Expected Shortfall (ES) risk metric is to be employed for internal market risk models (replacing the Value at Risk (VaR) method), in addition to other revisions covering the model approval process, taking account of market liquidity of positions, backtesting requirements and many more. Because substantially larger data sets are needed to calculate risk to satisfy FRTB requirements, banks have been forced to significantly rethink their data management strategies.

Banks can choose between two methods to calculate capital under FRTB: a standardised approach (SA) or an internal models approach (IMA). There are, however, many new complexities in calculating IMA: beyond the requirement of aligning trading desks and risk management pricing, the increase in data volume (transactions and historical data) is substantial. There are also data management challenges, such as the use of proxy data and managing rules across multiple jurisdictions, with full auditability and data versioning throughout. One strategic question for banks is to choose between the SA or IMA. As the methodology needs to be applied at the trading desk level, results on both approaches need to be analyzed at the most granular level to avoid shortcuts that may call into question the viability of the decision. Banks, therefore, need to be able to simulate, as well as quickly adapt to new situations. This requires more flexibility from data management setups: ability to quickly scale while operating either on-cloud or on-premise, with easy-to-use APIs and adapters for many data types.

“We knew a tsunami was coming.”

Similar to other major international banks confronted with having to find adequate IT systems to meet the FRTB rules, Societe Generale had the challenge of dealing with a massive increase in the volume of data, both in terms of storage and analysis. The underlying hardware infrastructure of the incumbent in-memory technology was becoming inordinately expensive to maintain when faced with “tsunami levels” of increasing volumes of data. Due to the unsustainable nature of having to constantly purchase costly new servers providing terabytes of RAM, Societe Generale was actively on the lookout for an innovative technical solution to provide an alternative to manage the challenge of producing and analysing FRTB indicators.

We knew we were facing a tsunami of data and we needed to have a solution to analyse the shortfall and manage the huge volume of data. The ability to accurately calculate market risk is a key function for financial institutions and, as a universal bank, Societe Generale collects a huge amount of data on a daily basis. By using Opensee, we have gained between one to two years in developing a solution because they took care of managing all the low-level details.Philippe Vibien, Global CIO Finance, Risk & Market Data (GBIS), SGCIB

Faster and less costly database management system

Societe Generale had been contemplating various options in order to lower the infrastructure costs linked to their technology and solve its limitations. As Societe Generale has historically preferred to build its own solutions, internal developments were considered and a prototype solution had been built using ClickHouse (which is one of the components of the Opensee solution). ClickHouse is a fast, open-source database management system that allows generating analytical data reports in real time using SQL queries, whose column-oriented storage allows it to run on faster, top-performing (and less expensive) commodity hardware. In the end, Societe Generale decided to choose a technology provider that had the experience of dealing directly with ClickHouse and which had created an entire solution around it, as well as the knowledge and financial expertise of building relevant calculations around specific use cases. Another key requirement for the bank was that the solution would need to have flexibility for data storage (to be either on-premise or cloud-based) in order to cater for differing data confidentiality obligations.

Opensee Solution Strategy

The Opensee’s innovative data analytics solution was chosen by Societe Generale’s RISQ/RMA department because it helped tackle the bank’s data challenges at scale. The Opensee platform empowers the bank’s business users to autonomously perform any aggregation, calculate ever-growing datasets and manage exponential data growth more efficiently, at a minimal cost without compromising on performance or volume.

Specifically, Opensee was chosen for the following three main reasons:

1. Speed: The Opensee platform is based on the high-performance, open-source database ClickHouse;

2. Financial services expertise: In addition to the technological benefits, Opensee provides financial services expertise that provides pre-configured procedures (or calculators) for custom aggregations, which help build internal or regulatory metrics (such as expected shortfall calculations up to the full calculation of FRTB SA or FRTB IMA). This allows business users to easily plug into dedicated functionalities, easing up constraints on developers;

3. Ability to scale: Due to Opensee’s proven capability in coping with various technical challenges around extremely large and/or in homogenous datasets, and the inevitable need for increasingly larger data warehouses, Societe Generale intends to roll out the solution across other risk platforms.

About Opensee

The Opensee platform democratises self-service analytics of all stored data. Users can explore all underlying data in real time and build user-defined functions to improve their response to regulations, such as FRTB, using targeted analytics. The platform can also be used to access all trade and order data to improve market intelligence and execution, or to analyse customer behaviour based on all related data.

In technical terms, the platform uses open source components, is written in the functional programming language Scala, and based on ClickHouse, an open-source big data, analytical column-oriented database to which the company has added a multitude of functionalities that allows non-data scientists to extract, aggregate, analyse and visualise data interactively. It also includes generic APIs to integrate with bank information systems and ingest large volumes of data at high speed, as well as a low-code API that allows users to write code in Python and interrogate data autonomously. The platform can be deployed on premise, in private or public clouds, or in a hybrid environment, with scaling supported by commodity hardware. The technology also offers advanced data management, such as a unique git-like versioning system, in which users can perform a full audit of any change (user-based changes, what-if, collaborative versioning, etc.).

Key Quantitative and Qualitative Results

With Opensee, Societe Generale now has no limits on the size of the datasets. Opensee has allowed Societe Generale to not only improve how it leverages data to make advanced risk decisions and optimise resources, but to also significantly decrease infrastructure costs.


  • By giving financial institutions’ business users the autonomy to dive deeper into their data, Opensee solves the complex problem of providing more data to users, saving up to 90% on infrastructure costs
  • Societe Generale can now instantly and interactively analyse an unlimited number of years’ worth of data, rather than 3 days’ worth which was previously the case, thereby allowing them to manage risk more efficiently
  • The Market Risk team is now able to produce specific reports requested by top management or the regulator in a very short time frame to make quick decisions. Thanks to user-defined functions, model risk analysts can - on demand - simulate the impact of a new model in a time frame of 3-36 hours, as opposed to an average of 2-3 months for previous cycles
  • In addition, Opensee simplifies dealing with the outperforming, but complex, distributed database technology of ClickHouse by managing any technical issues that may arise or subsequent releases of the technology. Due to Opensee’s expertise with the underlying technology, time to market has been accelerated by 24-36 months.

Working with Opensee

Opensee was one of the first cohorts of Societe Generale’s flourishing Global Markets Incubator (GMI) programme and had successfully completed two proof of concepts on running calculations for both the liquidity risk profile and the risk-weighted assets (RWA’s) on market activities of the bank. Opensee’s progression to other departments within Societe Generale such as Market Risk is testament to its success in pushing the boundaries of self-service data analytics for financial institutions.

“Opensee brought a cutting-edge solution that combines the transformative use of technology with financial markets business experience. Its tools allow us to access and leverage the most granular level of information and most relevant historical data.” Claire Calmejane, Group Chief Innovation Officer, Societe Generale

“It was really key to have a partner like Opensee to help our development team. Opensee’s financial expertise allows them to have a view over all the necessary use cases, which helps us to capitalise on many areas and really saves us time.” Gary Elezovic, Head of Technical Architecture for Finance Risk & Market Data, GBIS

“Opensee are always available when you need them and have provided us with excellent advice. Our department is one of the first to work with Opensee, and we are pleased to report that the market risk project is sticking to our requirements. I look forward to continuing to work at this level of cooperation in the future.” Yann Le Bars, IT Architect for Finance Risk & Market Data

Future roadmap

Looking forward, Societe Generale plans to leverage the competitive edge that Opensee provides by managing big data via multi-dimensional cubes with low-cost supporting facilities, and expanding it across other areas within the bank.There are a number of further use cases expected to be launched with Opensee, such as credit risk.

Collaborative Success

Opensee is thrilled to be in the position of providing an innovative technology solution to Societe Generale that delivers new ways to manage huge volumes of data across several parts of the business - all in an economical way. As roll-out of Opensee’s data analytics solution continues across other parts of their business, Opensee looks forward to continuing to build on the successful implementation in what has grown into a highly collaborative relationship with Societe Generale.

“Our mission is to help users realise their data potential by providing the ability to leverage more data than they originally thought possible, by unlocking technological and cost barriers.” Stephane Rio, Founder & CEO of Opensee

Other news

Put Opensee to work for your use case.

Get in touch to find out how we can help with your big data challenges.

Get a demo