Holistic Trade Surveillance: A New Paradigm in Securities Market Compliance

In the present-day regulatory regime, financial firms invariably fall short of regulators’ expectations, despite spending large amounts on compliance. This shortage results in heavy penalties and reputational loss, which has a far-reaching impact on business. In some cases, it can even amount to the suspension/revocation of licenses or it can attract recurring fines if remediation efforts are found inadequate.

Regulators are shifting to demonstrable compliance for market oversight, which means that financial firms have to:

  • Be proactively compliant to the applicable regulatory provisions,
  • Be ready with accurate reporting data for submission at all times,
  • Possess a robust violation detection system that meets regulators’ expectations,
  • Keep effective and repeatable end-to-end compliance processes in place, and;
  • Be able to demonstrate the effectiveness of the systems and processes frameworks used for compliance with proper evidences.

Since financial firms have to track and comply with numerous provisions across multiple asset classes, markets, trading venues and geographies, the cost of compliance becomes higher than the cost of doing business itself. However, non-compliance is simply not an option due to soaring penalties that can not only erode profits significantly, but also cause irreparable damage to the reputation of the firm.

A Case for Holistic Trade Surveillance

Trade surveillance has recently undergone a big change driven mainly by increased regulatory rigor, coupled with other factors, such as a complex trading environment and the convergence of different forms of financial crimes, which make anomaly detection an extremely challenging task.

Take the recent example of mirror trades going undetected in a leading European global bank that were being used to conveniently launder money from Russia to other parts of the world. The question here is: Could this have been dealt within the realm of trade surveillance or in anti-money laundering (AML) isolation? Or,did the bank need to be aware of the “big picture” of potential violations in order to prevent them from happening? The answer is an obvious “yes” and in favor of the latter. This is what has madea holistic approach in surveillance a timely matter.

What does holistic surveillance entail?

Holistic surveillance presents an integrated approach to trade surveillance by leveraging data from various structured and unstructured sources and drawing meaningful insights/inferences using analytics-based tools that would not have been visible in isolation. For example, an alert suggesting a potential insider trade entered by a trader coupled with a communication between the trader and the insider who passed on the price-sensitive information based on which the said trade was entered. Without this trace of communication, it would be difficult to conclusively prove that the trade was based on insider information done with a mala fide intention of making profits. This is where holistic surveillance steps in to add value through investigative and preventive capacities. It broadly involves analysis of the following information and data elements:

  • Trading the data and pattern of participating entities (such as client, trader, broker, etc.)
  • Trade-related communication amongst entities
  • Information disseminated in the market during the period in question
  • Social media information that can be linked with the trading behavior
  • HR and behavioral data of traders and employees of the organization

Holistic Surveillance: A Typical Data View

With the advent of digital technologies such as big data, analytics, mobility, cloud and social media, today’s surveillance has the opportunity to make full use of all available data sources (structured or unstructured). The following figure provides a view of data available to be leveraged for performing holistic surveillance.

Figure 1: Holistic Surveillance Data View
Figure 1: Holistic Surveillance Data View

Key Challenges in Holistic Surveillance

Given the enormity and variety of data available for analysis, it is very important that the right set of data is picked up for analysis; otherwise, the results will be misleading or even inconclusive. The following are some key challenges faced when handling data for holistic surveillance:

  • Relevant communication information on trade is often available in isolation and at first, it may not appear to be linked with the trade violation. For example, to trace an insider trading related communication, one has to scan large volumes of telephonic conversations, chat/emails texts, etc. Many times these turn out to be an exercise in futility and do not provide any clue. Therefore, they need to be worked upon patiently given the low “effort-to-result” ratio.
  • It is a challenging task to achieve lexicon optimization in order to keep the number of e-communication alerts to reasonable levels, so that a focused set of “communication of interest” (i.e., communication that is related to the trade violation under investigation) can be targeted. A large number of false positive communication alerts dilute the effectiveness of communication data in providing insights to trading behavior.
  • It is cumbersome to join different pieces of structured and unstructured data and create a cross-market, cross-asset class and cross-communication channel view of orders/trades depicting trading behavior, which gives the appearance that linking them, is an onerous task.
  • Deciphering the extremely unstructured voice communication, which generally carries maximum information on any violation, is a difficult task.
  • Managing regulators’ expectations with a proactive and preventive approach, as well as safeguarding against penalties.

How Communication Data Helps in Trade Surveillance

The table below highlights a few examples of detectable communication anomalies that act as indicators to potential violations in securities trading.

Trading Violation  Scenarios Trade Data Anomalies Detectable Communication Anomalies Acting as Indicators
Insider trading - Trading on the basis of unpublished price sensitive information known to the trader, which will give him an unfair advantage over the market
  • Trade happening before the dissemination of price sensitive news in the market, will have profit potential once the news is discounted by the market
  • Suspicious call/chat amongst trader(s) and client(s)
  • Social media postings before the dissemination of news in the market
  • Employee sharing or receiving insider information
Media abuse - Dissemination of false or misleading market information through various media channels with the intention to earn profits
  • Generally, no triggers on the trading side as this is purely information driven. However, trade data can be analyzed for evidences once the alert is generated based on the communication data
  • False postings in the media from entities linked with traders, customers and/or employees
  • Suspicious chat and/or voice communication with the intention of creating rumors
Front running - Entering into a trade with advance knowledge of a block transaction that will influence the price of the underlying security
  • Orders placed just before the large block orders, which enter into a profit position once the block trade hits the market
  • Suspicious call/chats between the trader (handling block trade) and his client
Circular trading – Trading amongst a closed group of entities with an ulterior motive (e.g., to falsely create liquidity or volume while there is no genuine change in ownership of the security)
  • A large number of identical or near-match trades happening between members of a colluding group having an overall nullifying effect but creating an impression of high liquidity and volumes.
  • Suspicious communication between traders and a colluding group of clients
  • Social media pictures/videos establishing an association among colluding groups
  • Historical suspicious communication data among colluding entities suggesting association
Mirror trading – Trading in two markets (generally cross-border venues) by booking identical buy and sell transactions in a given security with the intention of laundering money from one geography to another
  • Placement of one-sided trades in large volumes having little business acumen supporting the decision
  • Corresponding reverse trading positions in the same security in another geography not appearing to be backed by any sound trading strategy
  • Linkages among traders in the affected branches of the banks
  • Suspicious communication among the trading desk employees of different geographies of the bank
  • Suspicious off-market hour communication between traders and clients

Emerging Trends in Holistic Surveillance

While challenges are aplenty in leveraging the full potential of holistic surveillance, the following trends are emerging as it further evolves:

  • Behavioural analytics is being increasingly used to study the behavior patterns of traders, such as profiled information and also the patterns exhibited by other traders of a peer group.
  • Greater focus is placed on entity linkages/relationships mapping to effectively deploy predictive analytics on communication clusters between entities of interest.
  • Risk-based communication profiling is being done across entities (employees, customers etc.) to judiciously weed out low-risk communication categories and save efforts on them. 
  • Cognitive/machine learning models are increasingly used for continuous improvement in the quality of trade, communication, employee surveillance alerts and to sustain a high level of accuracy.
  • Investigators are using extensive communication and voice analytics to optimize a large number of voice/communication alerts in order to focus on the right alerts and to reduce investigative efforts.
  • Holistic surveillance is set up by financial firms as an integrated platform addressing multiple use cases1 in order to mutualize cost leveraging, common analytics capability and data repository.

Holistic Surveillance Transformation Roadmap for Financial Firms

Given the above-mentioned trends and regulatory compulsions, it is imperative for every organization involved in trading to embark on this initiative as quickly as possible. However, good planning will provide them with an integrated solution for several use cases. The ideal road map for holistic surveillance can include the following:

Figure 2: Holistic Surveillance Transformation Roadmap
Figure 2: Holistic Surveillance Transformation Roadmap

Holistic Surveillance Program Implementation Approach and Best Practices: Key Elements

For a financial institution to succeed in a holistic surveillance initiative, the following practices become very important:

  • All relevant use cases to be addressed by the initiative should be listed out and cross-leveraging possibilities should be explored. For example, trade communication can be leveraged for trade surveillance, AML and for employee surveillance needs. Similarly, communication data can be leveraged for many purposes besides trade surveillance, such as anti-fraud, AML, internal threat management and conduct risk management, etc.
  • The data lake for holistic surveillance should be created taking into account the design considerations of an integrated repository that can handle all kinds of data and accommodate information (structured or unstructured) pertaining to all the targeted use cases of the program. Besides this, the data lake should be scalable and should be able to accommodate new data needs due to the calibration of analytics-based models.
  • The system should be robust and advanced enough to make effective use of voice data as it has maximum information on violations, but is difficult to explore and correlate. A case in point would be to effectively convert voice data to text and then use it for analytics.
  • The system should be capable of risk profiling traders and deploying a risk-based approach, as it helps a great deal in optimizing efforts on trader surveillance by quickly ‘zeroing on’ to the ‘communication of interest.’
  • The holistic surveillance system should be able to effectively use machine and cognitive learning to continuously upgrade surveillance models rendering them effective at all times and reducing false positives.
  • The framework should be capable of establishing and analyzing entity linkages in order to explore the right set of communication data for the trade alerts that are going to be investigated. An advanced visualization-based linkage analysis tool can be used to achieve this.
  • Digital technologies used for holistic surveillance should be effectively able to leverage a wide variety of data sources (such as employee HR data, personal trading, limit monitoring, trade and communication alerts, past case history, etc.) and establish material relationships among them. There should preferably be a single data lake for all trade and communication-related information feeding into the analytics layer.
  • It is advisable to perform a detailed analysis at the initial stage itself, in order to ascertain the quality of data available for holistic surveillance. Appropriate data quality improvement measures should be taken to bring the data to meaningful levels for surveillance before using them further.
  • If a communication surveillance system is already in place, then it is important to gauge the quantum of false positive communication alerts. If it is high, it has to be fixed before making use of the alerts generated.
  • A detailed exercise needs to be done to explore all structured and unstructured data sources and to identify the ones that will be relevant for the platform and weed out the unnecessary ones.
  • Data retention compliance requirements, such as trade-related and voice and electronic communication requirements, need to be identified and provided for. This will be an important compliance requirement.
  • Entity relationships mapping with risk profile information must be enriched in an ongoing basis as much as possible, so that it supports identifying the right set of data (especially unstructured data) for analysis.

Looking Into the Future

The true power of surveillance lies in leveraging digital technologies with the relevant set of cross- channel information for creating a next generation surveillance framework. Such digital transformation not only increases the financial firm’s capability to meet the stringent compliance requirements of the regulators, but it also improves the overall customer experience significantly, thereby bringing in the competitive advantage needed to survive in the market.

Given the above scenario, the future of surveillance will be shaped by financial institutions who:

  • Make a smooth shift from a siloed approach to a holistic view across securities trading and communication related to the same (i.e., communication related to the trades made by the traders, clients, etc.) to detect any violation
  • Adopt a risk-based approach to trading/communication activity and individuals
  • Cross leverage data to generate and investigate alerts in order to arrive to conclusive evidence (i.e., communication triggered alerts substantiated through trade data and vice versa) 
  • Deploy state-of-the-art digital technologies to analyze the identified data

The following are other key developments that will cause metamorphic changes in holistic surveillance: 

  • Communication channels are likely to proliferate further with newer forms of digital communication coming into existence
  • The cost of digital data (structured and unstructured) storage and retrieval is going to fall and computing power will see a further boost to process complex events and perform analytics in real-time or near real-time
  • The share of unstructured data in overall investigations is going to increase with lexicon-based text analysis, cognitive computing and deep learning
  • AML, anti-fraud, internal threat management, market abuse and other trading violations will converge and will have a lot of interplay between them making holistic surveillance an absolute necessity
  • Electronic audit trails and usage patterns/statistics will have a larger role to play in investigations
  • The role of prescriptive analytics will increase in holistic surveillance and it will turn it into a rule-based intervention applicable on the fly


In the end, we can safely conclude that while technology providers will continue to bring in advancements to detect anomalies at the earliest time, regulators will strive to give more accountability on those they believe will improve compliance levels. On the other hand, rogue traders will resort to more sophisticated abusive trading strategies to outsmart the existing detection technology and regulatory norms. It will be an ongoing race for superiority in the time to come.

Sunil Kumar, consulting partner, TATA Consultancy Services Limited, Mumbai, India, sunil.kumar1@tcs.com

  1. “Use Case” is an industry term used in IT solutioning. It means a specific scenario, which is derived from a problem/pain area that a bank wants to address.

Leave a Reply