Major/Minor Version

Different versions of the Whisperer SBE schema are identified via the version attribute of the <messageSchema> root element, e.g. version="18655". This version applies to the schema as a whole, not to individual elements and is sent in the message header so the client can determine which version of the message schema was used to encode the message. The schema version number encodes a major.minor version scheme where the individual components are obtainable via the following bash one-liners:

   Major=$((version >> 9))  # breaking changes
   Minor=$((version & 511)) # incremental changes

Major - Breaking Changes

In order to maintain consistent message normalisation standards, MarketFactory will occasionally have need to make schema changes that break backward compatibility rules. Typically these will be planned releases, with long lead-times.

In such circumstances, the client will be provided with detailed information relating to these changes in Pending Client System Impacts and may need to update to the current Whisperer schema.

Changes in the major version represent a breaking change that will require users of previous schema versions to make code changes.Typically, these will be of a straightforward house-keeping nature, including:

Removal of deprecated messages and components (impacting message layout), datatypes and enumeration values.
Reordering of message components.
Changing wire representations of individual fields (e.g. String lengths).
Renaming of fields, enumeration values and messages (typically semantic types and wire values will remain unchanged).

Minor - Incremental Changes

The SBE protocol standard allows for a degree of backward compatibility via Template Extension rules, to allow message consumers to be using older versions of the message schema than message publishers, and vice versa. The following sections outline the rules and logic required to ensure that backward compatibility problems are reduced to a minimum.

Obviously this backward compatibility is only of use if the added types, fields, or messages are not actually required for the implemented business processes. If the new features are required then the client must of course implement the necessary changes to support the new schema in full.

The following Template Extension rules ensure backward compatibility:

Fields may be added to either the root of a message or to a repeating group, but in each case, they must be appended to end of a block.
Existing fields cannot change data type or move within a message.
Message and repeating group byte alignment may not change.
A repeating group may be added after existing groups at the root level or nested within another repeating group.
A variable-length data field may be added after existing variable-length data at the root level or within a repeating group.
Message header encoding cannot change.
In general, metadata changes such as name or description corrections do not break compatibility so long as wire format does not change.

Any message template that has changed in an incompatible way may be assigned a new template "id" attribute.

Version Compatibility Scenarios

The message decoder compares the schema version in a received message header to the version that the decoder was built with.

Decoder Version = Received Version - All fields are known to the decoder and may be parsed, no further analysis is required.
Decoder Version < Received Version - All fields known to the decoder may be parsed but it will be unable to parse any added fields. An older decoder may also encounter unexpected enumeration values used within the received message. In this case, the application layer must determine how serious an error this is.
Decoder Version > Received Version - Only the fields of the older schema version may be parsed. This information is not directly available in the message received off the wire, but is available in the Decoder's local schema as the sinceVersion metadata field attribute. If sinceVersion is greater than received schema version, then the field is not available.

Version Metadata

Version metadata is maintained at a more granular level within the schema, via the following optional attributes:

sinceVersion - Documents the version of a schema in which an element was added
deprecated - Documents the version of a schema in which an element was deprecated. Updated applications should not publish deprecated messages or values, but declarations may remain in the message schema during a staged migration to replacement message layouts.

These metadata field values will always be less than or equal to the version of the parent message schema.

These attributes are supported in the following elements:

<type>
<composite>
<ref> (unused)
<enum>, <validValue>
<set>, <choice>
<message>

NOTE: <group> elements do not support these attributes.