Monday, 3 December 2007

A new kind of (SOA) Registry

For a while I've been thinking that the SOA registry space has been a little overcomplicated. The first problem is the inheritance of the UDDI initiative. UDDI has a number of significant problems, but the biggest problem is that it has skewed the collective view of what a SOA registry and repository should look like.

The second problem is the lack of a really nice, simple, successful open source initiative in this space. When I talk to customers, even hardened Open Source supporters, they tell me they are looking at proprietary systems such as HP/Systinet and Software AGs Infravio.

So we thought, lets look at this again. Our view on OSS is to start with what is really needed and work from there, maybe adding some extras here and there. So what is essential in an SOA registry:
  • You need to be able to store and retrieve stuff, both with a remote API and with a Web browser. Of course its also nice to have a local embeddable Java API.
  • You need to be able to search, categorize and catalogue the stuff - and wouldn't it be nice if you could also tag and comment on that stuff.
  • Its essential that there is security, access control and versioning on every bit of stuff you put in there, so you know that this is a robust and trusted source of information.
  • The system should track dependencies between items, and should be able to manage those dependencies as items are versioned - this is really important for SOA governance.
  • There needs to be ways of importing, exporting and copying stuff, as well as managing the lifecycle - from sandbox to test to staging to production to deprecation.
You notice I called the things you store and search and read from the registry/repository "stuff". In the SOA case they are mainly XML documents (WSDLs, WS-Policies, etc) but they might also be Word or Excel documents. And in fact, fundamentally they are just resources.

So now let's look at this problem from the Resource Oriented perspective instead. Let's think about UDDI and where it went wrong - why do I need a SOAP call to update a resource, when a POST or PUT can do? At least UDDIv3 allows me to read resources using HTTP, but is there any reason to have any SOAP call to read a resource? And why do I need unique identifiers to be TModels when the humble URI/URL makes a perfectly good unique identifier?

So fundamentally the approach we have taken is to build a registry/repository based on REST concepts. And as we looked at the REST space, we kept noticing how close the Atom Publishing Protocol (APP) is to our needs, so we've made that the public remote API to access the repository. Of course, if you are just browsing the registry, you only need a browser - APP is mainly there to support updating resources.

Of course, using Atom and APP gives some really nice benefits too - like being able to subscribe a feed of new resources that meet your search criteria.

If you want to join in, you can subscribe to registry-dev, or you can download the code, have a play around and submit some JIRAs. Join the fun!

10 comments:

Jim said...

Paul,

Agree with your UDDI analysis and I don't think there is ever going to be huge support for UDDI specification. I'm working for a telco standards group at the moment that is debating the merits of recommending a public/shared UDDI repository. I'm definitely interested in taking a look at the REST based registry/repository.

Miko Matsumura said...

I think that registry repository will be a fairly easy component to develop in an open source way.

Hopefully a project like this would eventually become the core of all registry repositories, given that it is sufficiently standards-compliant that it can meet organizational needs.

I would certainly be open to discussing federation with such a registry repository and Software AG Centrasite (a product with both Infravio and Software AG roots).

This would be like many enterprises and their use of Eclipse as a platform for creating all kinds of tooling.

The reason why I propose such a thing is because there's registry repository validation which is a facet of governance, but we have found that building governance applications on top of a standards-based (in our case JAXR provider 1, UDDI and ebXML Registry Repository) is what most large organizations need, not just the core registry repository functions.

This would include workflows, approvals and ways to federate disparate IT constituencies. RSS, REST and other lightweight content models are excellent of course.

The economic value of the registry repository platform so far has been in enabling enterprises to govern not just design time validation, but the whole lifecycle. I'm sure that makes a ton of sense to you as I know you have very strong runtime understanding...

Miko

Paul Fremantle said...

Miko

I think the idea of a standardized model for a Registry/Repository is very appealing - and I'd even like to see an open source project to build out a framework for validation and governance too. I completely agree about the value of this.

Paul

Norm said...

Paul,

I like your regrep implementation and the whole REST vs ebRIM catalog has been generating a lot of discussion in the geospatial software world.

A couple of questions though;

ebRIM allows object associations which is really powerful (e.g. service->dataset, dataset->owner) - I don't see how this is possible with REST apart from using xlink:href or a path such as /owner/dataset which isn't very intuitive (I can't see explicit associations with a REST registry)

JEE spec mandates a JAXR client, so that components can access an XML registry - how do you envisage this in a case of a REST registry and being spec compliant?

Registry looks really good though and I am going to start to look into it!

Miko Matsumura said...

We are seeing more and more appreciation of the ebRIM/JAXR metamodel these days. It's not so much that there's interop inherent in it (although there is), it's more that this end-user driven standard really encodes for a lot of use cases that happen to come in handy. I agree that UDDI is far from universal but a lot of vendor interop uses it today...

J. Apps, B. Cameron, S. Siebold said...

William Henry of IONA (http://www.ipbabble.com/ posted a note on doing this quite some time ago. I cannot find it right now, but will look further when I have time.
I do like the 'simple' nature of this, something just about everyone should be able to understand and use.
Cheers, John

Paul Fremantle said...

The first release of this code is now available: check out http://wso2.org/downloads/registry

Paul

william_henry said...

Here is what I wrote a couple of years ago:
http://www.ipbabble.com/2006/01/rss_for_soa_based_services_reg.html

I had some follow-up with some code here:
http://www.ipbabble.com/2006/01/rss_demo_code.html

Cheers,
William

Miko Matsumura said...

I like that your RSS approach isn't coming from a fundamentalist anti uddi perspective but a very pragmatic one...kudos for a fresh view and it seems you thought of it already a while ago!

Miko

Alex said...
This post has been removed by a blog administrator.