David Orchard: “In general, an implementation must be conservative in its sending behaviour and liberal in its receiving behaviour.”

Breaking changes

As a value-add, our ProductSearch service includes in the search results a field indicating whether or not the product is currently in stock. The service populates this field using an expensive call into a legacy inventory system – a dependency that is costly to maintain. The service provider would like to remove this dependency, clean up the design, and improve the overall performance of the system – preferably without imposing any of the cost of change on the consumers. In speaking to the consumers’ owners, the provider team discovers that none of the consumer applications actually do anything with this value; though expensive, it is redundant.

Unfortunately, with our existing setup, if we remove a required component – in this case, the InStock field – from our extensible schema, we will break existing consumers. To fix the provider, we have to fix the entire system: when we remove the functionality from the provider and publish a new contract, each consumer application will have to be redeployed with the new schema and the interactions between services thoroughly tested. The ProductSearch service in this respect cannot evolve independently of its consumers: provider and consumers must all jump at the same time.

Our service community is frustrated in its evolution because each consumer implements a form of “hidden” coupling that naively reflects the entirety of the provider contract in the consumer’s internal logic. The consumers, through their use of XSD validation and to a lesser extent, static language bindings derived from a document schema, implicitly accept the whole of the provider contract, irrespective of their appetite for processing the component parts.

David Orchard provides some clues as to how we might have avoided this issue when he alludes to the Internet Protocol’s Robustness Principle. He said, “In general, an implementation must be conservative in its sending behaviour and liberal in its receiving behaviour.” We can augment this principle in the context of service evolution by saying that message receivers should implement “just enough” validation: that is, they should only process data that contributes to the business functions they implement and should only perform explicitly bounded or targeted validation of the data they receive – as opposed to the implicitly unbounded, “all-or-nothing” validation inherent in XSD processing.

Schematron

One way we can target or bound consumer-side validation is to assert pattern expressions along the received message’s document tree axes, perhaps using a structural tree pattern validation language like Schematron. Using Schematron, each consumer of the ProductSearch service can programmatically assert what it expects to find in the search results:

<?xml version=”1.0” encoding=”utf-8” ?>

<schema xmlns=”http://www.ascc.net/xml/schematron”>

  <title>ProductSearch</title>

  <ns uri=”urn:example.com:productsearch:products” prefix=”p”/>

  <pattern name=”Validate search results”>

    <rule context=”*//p:Product”>

      <assert test=”p:CatalogueID”>Must contain CatalogueID node</assert>

      <assert test=”p:Name”>Must contain Name node</assert>

      <assert test=”p:Price”>Must contain Price node</assert>

    </rule>

  </pattern>

</schema>

Schematron implementations typically transform a Schematron schema such as this into an XSLT transformation that the message receiver can apply to a document to determine its validity.

Notice that this sample Schematron schema makes no assertions about elements in the underlying document for which the consuming application has no appetite. In this way, the validation language explicitly targets a bounded set of required elements. Changes to the underlying document’s schema will not be picked up by the validation process unless they disturb the explicit expectations described in the Schematron schema, even if those changes extend to deprecating or removing formerly mandatory elements.

Here then is a relatively lightweight solution to our contract and coupling problems and one that doesn’t require us to add obscure meta-informational elements to a document. So let’s roll back time once again and reinstate the simple schema described at the outset of the article. But this time round, we’ll also insist that consumers are liberal in their receiving behaviour and only validate and process information that supports the business functions they implement (using Schematron schemas rather than XSD to validate received messages). Now when the provider is asked to add a description to each product, the service can publish a revised schema without disturbing existing consumers. Similarly, on discovering that the InStock field is not validated or processed by any of the consumers, the service can revise the search results schema – again without disturbing the rate of evolution of each of the consumers.