The SBE protocol standard allows for a degree of backward compatibility, to allow message consumers to be using older versions of the message schema than message publishers, and vice versa. The following sections outline the rules and logic required to ensure that backward compatibility problems are reduced to a minimum.

Obviously this backward compatibility is only of use if the added types, fields, or messages are not actually required for the implemented business processes. If the new features are required then the client must of course implement the necessary changes to support the new schema in full.

Constraints

The following rules ensure backward compatibility:

Fields may be added to either the root of a message or to a repeating group, but in each case, they must be appended to end of a block.
Existing fields cannot change data type or move within a message.
Message and repeating group byte alignment may not change.
A repeating group may be added after existing groups at the root level or nested within another repeating group.
A variable-length data field may be added after existing variable-length data at the root level or within a repeating group.
Message header encoding cannot change.
In general, metadata changes such as name or description corrections do not break compatibility so long as wire format does not change.

Any message template that has changed in an incompatible way may be assigned a new template "id" attribute. However:

In order to maintain consistent message normalisation standards, MarketFactory will occasionally have need to make schema changes that break backward compatibility rules.

In such circumstances, the client will be provided with detailed information relating to these changes and may need to update to the current Whisperer schema.

Versioning

Schema Version

Different versions of the Whisperer SBE schema are identified via the version attribute of the <messageSchema> root element, e.g. version="18367". This version applies to the schema as a whole, not to individual elements and is sent in the message header so the client can determine which version of the message schema was used to encode the message.

The version number is in fact an Epoch date value, and can be converted into a calendar date by use of the following bash one-liner:

echo $(date -d @$((version*86400)) +%F)

Version Compatibility Scenarios

The message decoder compares the schema version in a received message header to the version that the decoder was built with.

Decoder Version = Received Version - All fields are known to the decoder and may be parsed, no further analysis is required.
Decoder Version < Received Version - All fields known to the decoder may be parsed but it will be unable to parse any added fields. An older decoder may also encounter unexpected enumeration values used within the received message. In this case, the application layer must determine how serious an error this is.
Decoder Version > Received Version - Only the fields of the older schema version may be parsed. This information is not directly available in the message received off the wire, but is available in the Decoder's local schema as the sinceVersion metadata field attribute. If sinceVersion is greater than received schema version, then the field is not available.

Version Metadata

Version metadata is maintained at a more granular level within the schema, via the following optional attributes:

sinceVersion - Documents the version of a schema in which a type was added
deprecated - Documents the version of a schema in which a type was deprecated. Updated applications should not publish deprecated messages or values, but declarations may remain in the message schema during a staged migration to replacement message layouts.

These metadata field values will always be less than or equal to the version of the parent message schema.

These attributes are supported in the following elements:

<type>
<composite>
<ref> (unused)
<enum>, <validValue>
<set>, <choice>
<message>

NOTE: <group> elements do not support these attributes.