The Lab

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.
Comment: Published by Scroll Versions from this space and version 1.5.96

...

Pricing

Subscription Management

A Venue pricing Session is established with the standard UserRequest (LogOnUser) / UserNotification (LoggedOn) message exchange.

On successful connection, if available, SecurityStatus messages will be delivered to the client, providing details of the supported instruments. These messages may also be sent intra-session, to notify the Client User of events such as tradeability changes, date roll events, etc.

Aside from this, CLOB Pricing sessions are managed via MarketDataRequest messages issued by the Client User to the Venue, with streaming MarketDataIncrementalRefresh messages being delivered until such time as the request is terminated.


The basic market data sequence is outlined below:

Gliffy Diagram
displayName4.0 Schema_MarketDataRequest_CLOB.01
name4.0 Schema_MarketDataRequest_CLOB.01

Subscription Response

Note
titleSubscription Response

The client will receive an immediate response to an invalid MarketDataRequest.

However, the client may not always receive an immediate MarketDataIncrementalRefresh in response to a valid MarketDataRequest stream subscription:

  • There may be an interval of several minutes before the first MarketDataIncrementalRefresh is received by the client.
  • The venue may never respond.

A Venue's response actually depends on the instrument being traded, time of day and venue-specific behaviour.

The client may wish to implement a timeout mechanism to unsubscribe and then either:

  • Mark the instrument as unavailable.
  • Resubscribe. Generally this is of very little value and is not recommended.

SecurityStatus Messages

Security Status SecurityStatus messages communicate static attributes of each instrument supported by the Venue and are delivered on successful connection to a CLOB Pricing session.

...

The usage of specific fields is elaborated below:

  • ContractMultiplier - This is aka Lot SIze. whilst there are a few FX venues to allow the client to trade standardised contracts (e.g. LMAX), this is primarily present for trading Futures on venues such as the CME. For most venues, the contract multiplier returned will be 1.0.
  • MinQty - This is the minimum allowed Order size for the Venue. given that the venue may be trading in terms of contracts, the client may safely  assume that this quantity is expressed in ‘nominal’ units (e.g. contracts) and not in total units.

MarketData Messages

Incremental Processing

Whisperer delivers market data in the form of incremental updates, wherever possible.

...

EndMarker Messages

For CLOB Pricing sessions, EndMarkers are sent to the Client in two contexts:

  • SecurityStatus
  • MarketDataIncrementalRefresh

SecurityStatus EndMarkers

On session establishment, an EndMarker message is sent by Whisperer to mark the end of the initial set of SecurityStatus messages. This serves as an indication to the Client that the full list of all available instruments, and their status has been published.

Note
titleAlways Published at Start of Session

The EndMarker message is always published on session establishment for CLOB MarketData feeds. Where venues do not provide any  initial SecurityStatus info, we will publish the EndMarker immediately.

The EndMarker message may thus be used by the Client as a reliable "ready" signal to initiate market data subscriptions.


Note
titleNot Published Intra-Day
EndMarker messages are not utilised for intra-day SecurityStatus updates, (e.g. for ValueDateRoll),  clients are expected to respond to these as soon as they are published.

MarketDataIncrementalRefresh EndMarkers

Venues typically publish market data in a conflated fashion (as opposed to a real-time stream of individual events), and it is important for Client systems to be able to recognise the end of each conflation interval. Where these notifications are available, they will be published in the form of EndMarker messages to the Client User, whenever any MarketDataRequest subscriptions have been established.

Under normal trading circumstances, EndMarkers may also be published for incremental refresh conflation intervals when nothing has changed. As such they have the appearance of a Venue heartbeat, but it must be remembered that they serve an entirely different purpose.

NOTE: For UDP/Multicast feeds, different EndMarkers may be published for each data channel, Incremental vs Trades Feeds for example. This is Indicated via MDBookType.

MarketData Messages

Snapshot vs Incremental Updates

Whisperer uses a single MarketDataIncrementalRefresh message  for all market data updates.

The MDFlags.IsSnapshot flag is used to indicate how the message relates to the previous state:

  • TRUE  - This message must fully replace any book that exist prior.
  • FALSE - This message is to be applied to the local book, which must be retained.

This flag is guaranteed to be set such that the local book is always correct. In particular it should be noted that:

  • On starting the subscription, the local book is regarded as empty. For the the first update the client receives, all MDEntries will specify NoMDEntries[*].MDUpdateAction = New, and MDFlags.IsSnapshot = TRUE.
  • Snapshots may be delivered at any time in the life of the subscription. This is particularly important for connections to UDP multicast feeds where dropped packets must trigger book/session recovery.

Incremental Processing

When MDFlags.IsSnapshot = FALSE, standard incremental processing is expected. The client must reference each Snapshots always fully replace any book that might exist prior.Standard incremental processing is expected, with clients making direct reference to NoMDEntries[*].MDUpdateAction and applying apply the New/Change/Delete actions action against each associated the associated MDEntryID. It is important essential that the NoMDEntries repeating repeating groups are are processed in the exact order given.

...

  • All    - MDEntryID and MDUpdateAction are always provided.
  • Delete  - by definition the entry is no longer valid, so there is no need to redeliver the state of the outdated entry.
  • New    - all other available fields will always be provided.
  • Change - typically, only the fields that have changed will be provided.
Info
titlePerformance considerations

For Change events, the following principles apply:

  • If the venue also delivers unchanged fields in addition to those that trigger the update, these will also be supplied.
  • In the typical case where the venue only delivers the modified fields, Whisperer does not re-populate the previously published fields.

MDEntryTypes

The following MDEntryTypes are used to manage the state of the Book:

All other MDEntryTypes represent monotonic attributes of the market data feed, for example Trade Ticker events, statistical measures (High, Low, Close etc).

Empty Book Handling

An empty book may occur in one of two ways:

  • Explicit publication of an EmptyBook snapshot
  • The implicit result of the incremental deletion of the last remaining entry(ies).

When clearing the book, only entries that form the book should be cleared. All other MDEntryTypes should be left alone.

Expand
titleExpand example
First MarketDataIncrementalRefresh
MessageProcessed view
IsSnapShotTrue
MDUpdateActionMDEntryTypeMDEntryIDPrice
NewBid11.12
NewTradingSessionHighPriceMF:TSHP1.15

The Book will contain one entry, and the TradingSessionHighPrice lives outside the book.

MDEntryTypeMDEntryIDPrice

Book

Bid11.12
Statistical Entries
TradingSessionHighPriceMF:TSHP1.15


Second MarketDataIncrementalRefresh
MessageProcessed view
IsSnapShotTrue
MDUpdateActionMDEntryTypeMDEntryIDPrice
NewBid11.12
NewBid21.14

The Book will contain two entries, and the TradingSessionHighPrice is still active as it lives outside the book.

MDEntryTypeMDEntryIDPrice

Book

Bid11.12
Bid21.14
Statistical Entries
TradingSessionHighPriceMF:TSHP1.15


Third MarketDataIncrementalRefresh
MessageProcessed view
IsSnapShotFalse

MDUpdateActionMDEntryIDMDEntryIDPrice
Delete111.12
Delete221.14

As both entries from the Book are deleted, leaving an implicit empty book.

MDEntryTypeMDEntryIDPrice

Book

---
Statistical Entries
TradingSessionHighPriceMF:TSHP1.15


Fourth MarketDataIncrementalRefresh
MessageProcessed view
IsSnapShotTrue
MDUpdateActionMDEntryTypeMDEntryIDPrice
NewBid31.14
ChangeTradingSessionHighPriceMF:TSHP1.16

Note that the TradingSessionHighPrice has remained in place after processing each of the previous messages because it lives outside the Book

MDEntryTypeMDEntryIDPrice

Book

Bid31.14
Statistical Entries
TradingSessionHighPriceMF:TSHP1.16


Fifth MarketDataIncrementalRefresh
MessageProcessed view
IsSnapShotTrue
MDUpdateActionMDEntryTypeMDEntryIDPrice
NewEmptyBookEMPTYBOOK1.12
NewBid21.14

The explicit EmptyBook entry clears the Book, but the TradingSessionHighPrice remains in place.

MDEntryTypeMDEntryIDPrice

Book

---
Statistical Entries
TradingSessionHighPriceMF:TSHP1.16


Sixth MarketDataIncrementalRefresh
MessageProcessed view
IsSnapShotFalse
MDUpdateActionMDEntryTypeMDEntryIDPrice
NewOffer11.14

The New Offer entry implicitly deletes the EmptyBook entry.

MDEntryTypeMDEntryIDPrice

Book

Offer11.14
Statistical Entries
TradingSessionHighPriceMF:TSHP1.16


Book Depth

Where clients subscribe for custom book depths against a given Venue, it is important to understand that entries that are pushed below the requested depth by more competitive orders will not necessarily result in an explicit Delete being delivered by the Venue for Whisperer to process.

...

  • OrderDepth - MDEntryID provides a unique identifier of that individual Order, as and when the order is modified it's ID will change. Additional linkage information may also be provided by the venue so the market data consumer can track each individual order through it's entire lifetime. Whisperer currently delivers this in the NoEntryPassthruFields block, if available.
  • PriceDepth - here the MDEntryID represents a specific side/price level and so could be used to give the correct book ordering. Typically it will be a string representation of either the price itself, or the level within the book (position number).


Info
titleMDEntryID Uniqueness

The MDEntryID is guaranteed to be unique through the MDUpdateAction lifecycle for the parent subscription only:

  • Previously deleted IDs may be re-used in the lifetime of the subscription.
  • Previously used IDs may be used across subscriptions and sessions.

In order to ensure a consistent processing model across both book types, the client may implement an efficient equivalent of the following:

  1. Process the individual updates contained within the market data message.
  2. Collect the orders by price level. This is redundant for a PriceDepth book, which is already aggregated.
  3. Sort prices for each side of the book best to worst. I.e.  The Client always sorts the book by price for each side (not MDEntryID).
  4. For each price level at or above the subscribed MDBookDepth, sort MDEntryIDs smallest (oldest) > largest (newest). Again, this is redundant for a PriceDepth book, which is already aggregated. I.e.  the MDEntryID is a secondary key for OrderDepth books, to sort multiple orders at a give price level.
Info

The presence of any entries outside the subscribed MDBookDepth is unintended behaviour.

Book Integrity

...

  • Whisperer ensures that books will never exceed any specific requested depth - when  when implicit deletes are detected, the explicit Delete MDEntry is placed before the New that triggers it.
  • Two MDEntries in a PriceDepth book may momentarily have the same price - this typically occurs when multiple MDEntries on one side of the book change their prices up or down, such that the first MDEntry moves to a price level occupied by a different MDEntry that has yet to have it's price modified.

Orders

The Orders workflow is always normalised and consistently implements the state transition model as outlined in the diagram below. Where Venue events are absent, synthetic equivalents are generated by Whisperer in their place. These messages are easily discernible by reference to the TradingFlags.IsSynthetic field.

Info
titleExecutionAck
It's worth remembering that while  all inbound messages will adhere to the same state transition model, there will necessarily be Venue-specific variation in whether and how outbound ExecutionAck messages are handled.

Order State Transition Diagram

UDP/Multicast MarketData Feeds

As well as the EndMarker behaviour outlined above, the client needs to be aware of the following unique attributes of UDP Market Data feeds:

  • Line Arbitrage - the MDSubFeedType specifies the UDP market data source for venues which utilise A/B circuits.
  • Gap Detection - in the event that a UDP message gap occurs, this is reported to the Client User immediately it is detected for each affected subscription, via the publication of a EmptyBook MarketDatIncrementalRefresh message with MDFlags.IsSnapshot and MDFlags.GapDetected both set to TRUE.
  • Gap Recovery - the gap will be resolved by the Gateway utilising the venue's standard Late-joiner logic, and the next message will be a fresh Snapshot (typically, but not always, representing the full book). Thereafter Incrementals will be published as normal.

Orders

Order Submission and Modification

Gliffy Diagram
displayName
Gliffy Diagram
name4.0 Schema_ESP_OrderState.3

Order Submission and Modification

gliffy
Order.5
name4.0 Schema_ESP_Order.5

Order Cancellation

Gliffy Diagram
displayName4.0 Schema_OrderCancel.02
name4.0 Schema_OrderCancel.02