Replay Service

Summary

The Enterprise Replay Service allows a client to replay messages from any of their venue sessions.

Messages are available for replay immediately after they are sent to the client.

The current week's messages are available for replay. If you reset sequence numbers during the week, you will need to use a time-based query to replay messages from before the reset. It is possible, depending on the nature of the sequence number reset, that after thereset messages will not be available for replay.

Currently this service is only available to SBE protocol clients.
Message field information can be found in the SBE schema message definition.
Only Execution Reports are replayable.
Only Execution Reports sent to the client from the MarketFactory servers are replayable.
Supports multiple concurrent connections.

Versions

There are 3 key schema versions involved in a Replay Session - Replay Service Server, Client, Original/Replayed Message.

Example Schema Versions
Example Scenario	Replay Service	Client	Original/Replayed Mesage	Notes
Newer Client	18654	18656	18656	You can often use a client to connect to a replay service running an older schema version. This is true as long as the schema Session messages (i.e. Logon) and the Replay specific messages (i.e. ApplicationMessageRequest) are unchanged/backwards compatible.
Older Client	18656	18654	18654	You can often use a client to connect to a replay service running a newer schema version. This is true as long as the schema Session messages (i.e. Logon) and the Replay specific messages (i.e. ApplicationMessageRequest) are unchanged/backwards compatible.
Older Replayed Message	18656	18656	18212	The payload of the replayed message (RawMessage field of ReplayedMessage) can be encoded using any schema version - it depends on the version of the gateway your pricing/orders clients are using to generate the original messages. That version might be older or newer than that used by the Replay Service or your Client. In this example, your Client is using schema version 18656 but the replayed messages are encoded with schema version 18212. To read the ReplayedMessage you will a Decoder from schema version 18654. However to decode the original messages stored as raw bytes in the ReplayedMessage you will need to use Decoders generated using schema 18212 (or compatible).

Client Session Management & Logon

Logon to the Replay Service is identical to any other gateway - see https://confluence.marketfactory.com/x/IIFZAg for details.

The Replay Service will never gap fill or require the client to gap fill. It will instead match the expected sequenece numbers provided in the Logon Message.

Any enabled user can be used to access the replay service even if the user is concurrently connected to other services.

Logon Message Notes. Does not cover all fields - refer to the SBE schema for other notes.
Field	Comments	Example
venuename	Replay service identifier provided by MarketFactory.	client_replay
nextExpectedMsgSeqNum	The replay service does not track sequence numbers across sessions. It will match the sequence numbers of the client logon message. It is preferred to set the Logon Message msgSeqNum and nextExpectedMsgSeqNum to 1.	1
sessionType	Must be 'Replay'	Replay

Logon Example

logonEncoder
  .venuename("client_replay")
  .username("testuser")
  .password("user1password")
  .nextExpectedMsgSeqNum(1)
  .heartBtInt((short) 30)
  .sessionType(SessionType.Replay)
  .text("");

Replay Protocol Summary

After a successful logon

1) Submit an ApplicationMessageRequest to start the replay.

2) Receive zero or more ReplayedMessage messages.

3) Receive 1 ApplicationMessageReport message at the end of the replay. Every request will be terminated with an ApplicationMessageReport.

4) To start another replay, stay connected and repeat steps 1-4 above; or disconnect.

Only Execution Reports sent to the client from the MarketFactory servers are replayable.
Once a replay is complete, a new one may be started.
Do not send a UserRequest message.
ApplReqID needs to be unique .

ApplicationMessageRequest Field Notes. Does not cover all fields - refer to the SBE schema for other notes.
Field	Comments	Example
ApplReqID	Unique request identifier. Request with duplicate ApplReqID will be rejected.	120392
RefApplID	Gateway service name with type to replay message for. Format - venue_name.session_type.	lmax.maker.exchange.Orders
ApplMessageTypes	Replays original messages with a type matching one of the provided values. Note: Only 'ExecutionReport' is supported and must be included in this field. Any other requested message types will be ignored.	.applMessageTypes().clear().executionReport(true)
ApplBegSeqNum	Replays original messages with a seqNum greater than or equal to this value. If provided, the ApplEndSeqNum must also be provided. Must be less than the ApplEndSeqNum. Inclusive.	352
ApplEndSeqNum	Replays original messages with a seqNum less than or equal to this value. If provided, the ApplBegSeqNum must also be provided. Must be greater than the ApplBegSeqNum. Inclusive.	400
ApplStartTime	Replays original messages with a sendingTime greater than or equal to this value. If provided, the ApplEndTime must also be provided. Must be less than ApplEndTime. Epoch nanos.	1625718249000000000
ApplEndTime	Replays original messages with a sendingTime less than or equal to this value. If provided, the ApplStartTime must also be provided. Must be greater than ApplStartTime. Epoch nanos.	1625718259000000000

Common Replay Failure Reasons

ApplicationMessageReportEncoder Failed ReportType	Text value	Action
ResendFailure	Failed. Duplicate active requestID.	Use different ApplReqID for each request.
	Failed. Filter only supports ExecutionReports. Include ExecutionReports in your request.	Add ExecutionReport to their request. All other requested message types will be ignored.
	Failed. Invalid request. Reason: "XXXX"	The request is invalid. The specific reason will be provided in the 'Reason' text.
	Failed. Unknown Venue.	The venue requested venue/service cannot be found in the configuration. Ensure the client provides the correct name.

Acceptable ApplicationMessageRequest parameter combinations

Note: All 4 values must be explicitly set in each request. If not 'Set' to a value then explictily set the appropriate 'Null' Value.

applBegSeqNum	applEndSeqNum	applStartTime	applEndTime	Query
Set	Set	Null	Null	Replay all Execution Reports with sequence numbers equal to and between the provided sequence numbers.
Null	Null	Set	Set	Replay all Execution Reports between the provided start and end times.
Set	Set	Set	Set	Replay all Execution Reports between both the time and sequence numbers.

Example ApplicationMessageRequest to start a sequence number based replay.

ApplicationMessageRequest Example

applicationMessageRequestEncoder
  .applReqID(System.currentTimeMillis())
  .applReqType(ApplReqType.StartRetransmission);
  final var noApplIDsEncoder = applicationMessageRequestEncoder.noApplIDsCount(1);
  noApplIDsEncoder.next();
  noApplIDsEncoder
    .refApplID("fidessa_orders.Orders")
    .applUsername("Test")
    .applMessageTypes().clear().executionReport(true);
  noApplIDsEncoder.applBegSeqNum(1);
  noApplIDsEncoder.applEndSeqNum(10000);
  noApplIDsEncoder.applStartTime(ApplicationMessageRequestEncoder.NoApplIDsEncoder.applStartTimeNullValue());
  noApplIDsEncoder.applEndTime(ApplicationMessageRequestEncoder.NoApplIDsEncoder.applEndTimeNullValue());

Example decoding original message in ReplayedMessage - Not Safe for Production

public void onReplayedMessage(MessageHeaderDecoder messageHeader, ReplayedMessageDecoder replayedMessage) {
 
    LOG.info(format("[Replay] Received replayed message [%s %s]", messageHeader, replayedMessage));
 
    final ByteBuffer rawBuffer = ByteBuffer.allocateDirect(16000);
    final UnsafeBuffer raw = new UnsafeBuffer(rawBuffer);
 
    replayedMessage.getRawMessage(raw, 0, replayedMessage.rawMessageLength());
 
    final MessageHeaderDecoder header = new MessageHeaderDecoder().wrap(raw, 0);
    final int msgId = header.templateId();
 
    switch (msgId) {
        case ExecutionReportDecoder.TEMPLATE_ID: {
            final ExecutionReportDecoder erd = new ExecutionReportDecoder().wrap(raw, header.encodedLength(), header.blockLength(), header.version());
            System.out.println(erd);
            break;
        }
        default: {
            System.out.println("Unhandled message: " + msgId);
        }
    }

Replay Service

Summary

Versions

Replay Protocol Summary

Common Replay Failure Reasons

ApplicationMessageReportEncoder Failed ReportType