Market Data Archive & Playback

Market Data linked to Profits

The growth of program, algorithmic, and electronic trading has provided opportunities to maximize profits by gaining an understanding of market behavior. This understanding is developed thru the analysis of quoting and trading behavior of market participants by examining market data. Several companies have initiated projects to capture, archive and access intra-day and historical market data to validate trading models, understand and mitigate portfolio risk and to satisfy compliance requirements.

Excluding options, the volume of U.S. equity market data messages from exchanges and ECNs routinely exceed 2 billion messages per day, and are growing ever higher. Many of these messages are generated from full depth “order book” feeds such as ARCA and INET which complicate the task of determining the overall liquidity that existed in the market for an issue at a given point in time. To do so, requires that data be analyzed from market open up until the time of inquiry, the application is required to “walk the book”.

While a relational data base provides the robust flexibility that would be optimal for working with market data, the complexity of retrieving a consolidated view of liquidity for a point in time has typically eliminated these systems from consideration and customers are attempting to solve this problem with large in memory databases such as KX and Vhayu, sacrificing flexibility for performance.

Mining your Market Data

The head of West Highland professional services has in the recent past, led a project that developed technology enabling the retrieval of the consolidated order book information from a SQL server data base containing over 1 billion records in .5 seconds. This platform agnostic technology was recognized as the 2007 Innovator of the year by “Windows on Wall Street” and a panel presentation was made to industry analysts at Microsoft’s 2007 Tech Ed conference. Additionally, Microsoft has done a case study on the achievement and this study soon will be added to their product archives.

West Highland’s professional services personnel has experience as a data vendor, providing historic market data to “High Frequency” traders and quantitative modelers at both sell side and buy side firms. Their services help create efficiencies in how market data is stored and accessed, maximizing the value that end users derive from these systems. As data rates grow ever higher, an efficient implementation is essential to maximizing a replay system’s usefulness and reducing the ongoing cost of supporting a mission critical market data archive.

The Engagement

West Highland is prepared to leverage its experience on their customers’ behalf in whatever manner best meets the customer’s needs. West Highland professional services will help execute any (or all) of the following project tasks.

Requirements definition

 Understand business objectives
 Identify the users
 Identify the data they will be accessing
 Document what they will be doing with the data
 Identify how they will access the data
 Identify how the data will be presented
 Identify any exchange or data source reporting requirements
 Understand the timeliness of the data requirements
 Understand system availability requirements
 Understand budgetary restraints
 Target delivery date
 Document requirements

Selecting the solution

 Buy vs. Build analysis
  • Identify off the shelf solutions (if they exist)
  • Estimate development costs
  • Estimate implementation costs
  • Estimate on-going support costs
  • Provide implementation timeline
 Executive presentation to decision makers

Implementing the soution

 Project Plan
 Identify and gain commitment from project resources
 Staff augmentation (should there be a shortage of available client resources)
 Execute the project

Engaging West Highland’s services for a market data archive/replay project ensures that customers benefit from the knowledge gained thru providing these types of solutions. And with market data rates and data storage costs sky rocketing, an objective, vendor agnostic understanding of technology choices and their fully loaded costs is essential to the success of these types of projects.


Support Services  |  Professional Services  |  Direct Feeds  |  Capacity Management  |  Data Warehousing  |  Technology Audit  |  Contact


Copyright © 2001-2012 West Highland LLC. ALL RIGHTS RESERVED