Why open standards and interoperability are subtly different

I've been pondering on my experience in Open Standards - especially in the WSRM/WSRX world. In particular I've been thinking about the difference between an Open Standard and an Interoperable standard. It seems to me that there is a general assumption out there that these two things are the same. In an ideal world they would be, but I think there is a gap:

• Independent implementations
• Specification completeness
• Testing

My own criteria for interoperability is that there are "tested independent implementations". This means that different groups have looked at the spec, worked through it and written code. And then they have taken these implementations and tried them against each other.

Let me give a simple example from the WSRM testing. In WSRM there is the option for a client to request an acknowledgement. An acknowledgement (ack) is designed to let the client know which messages have been received and which haven't. The ability to request acks is key to the working of the protocol. However, acks can also be sent by the server without a request. And we built in the option to allow the server to send a "nack" as well as an ack. For example, an ack might say "I have received messages 1-5 and 6-100", but the nack can specify: "I'm missing message 6 - send it now".

During the testing we came across the following situation: the client was asking for an acknowledgement and the server was sending a nack. The client needed to understand the whole sequence state (which the ack gives) but couldn't because it was only being told one data point (I don't have message 6). The result was a problem. Now, we could have simply said - this is a problem with the client - it needs to deal with it. However, the reality is that this - while legal behaviour of the spec - was not something the designers envisaged. The fact is that it is important for the client to be able to request a complete ack. So we solved this by improving the spec. If the server is initiating it can send a nack, but when responding it must always send the full ack.

Of course its possible we might have caught this just by reviewing the spec. But we didn't. The original team who wrote the spec didn't think of this. The WSRX committee that reviewed it didn't spot it. None of the 5 implementers of the spec spotted it. It was only at the interop testing that we noticed it. And it was only because we had independent implementations and interoperability testing that we ended up with a complete specification.