Simple scenarios for the ESB

Tomorrow I will be doing a joint webinar with Asankha Perera and Ruwan Linton about the WSO2 ESB. We will be showing how to handle some standard scenarios:

• Converting between legacy file formats and message-based systems
• XML transformation and message augmentation
• The "pushme-pullyou": using polling techniques to link database applications with live systems

The aim of this webinar is to give attendees an insight into how to handle some common scenarios with the ESB and also give a headstart on your own projects.

If you missed the introductory webinar I ran two weeks ago, that is being repeated on Thursday 3rd July. Apologies in advance to US folks heading off on their holidays!

You can sign up to both events here.

SOA Governance and Open Source Governance

SOA Governance is one of those great titles - it makes it sound like something incredibly difficult is happening. And the reality is - that in a large organization where SOA has grown up without controls - it may be difficult. But that doesn't mean it has to be. The basic principles are simple.

What are those basic principles? Open Source development principles!

The reason SOA governance is important is that SOA is inherently empowering. It is about enabling teams to take ownership of their own business services. Of course the problem is that you can't empower Bob without empowering Joe. That means that Bob has got to trust Joe, and Joe has to trust Bob.

Now Open Source developers are naturally dispersed teams who are empowered, and who trust each other. In Apache we talk about Meritocracy - which literally means giving the governance to the people who have merit. But it isn't just based on having wonderful smart people - its also based on processes and systems.

The main systems in place for OSS governance are simple:

• A shared repository with correctly managed access rights, accessible via command line, tools or website
• Versioning of every change so broken code can be rolled back
  
• Notification of code changes through publication
  
• A shared forum or mailing list for discussion
• A good build system that manages dependencies
• Validation of code through unit tests
  
• A well defined release process

These simple aspects create a framework for a community to govern multiple components across different geographies and teams. And it works. If you want proof, simply look at Apache's productivity - or Codehaus - or better still - WSO2.

What does this mean for SOA? Let's take each of those aspects and see how it maps to our SOA Governance solution - the WSO2 Registry:

• Shared repository: this is what the Registry is. It is a shared repository, with fine grained access control, and can be viewed as a website or via command-line/programmatic tools
• Every change to a resource is fully versioned, and you can checkpoint a directory at any time, capturing the exact state of all resources in that tree
• Notification of changes - you can subscribe to any aspect of the registry and repository using Atom, or extend the system using well-defined plugpoints to use email or other publication systems
• Even better than a mailing list, each resource has its own place for comments. You can subscribe to all comments across the system, but each comment is closely tied to the resource it applies to. This maintains the history of the discussion directly against the resource.
• Dependency management - both automatic (this WSDL uses this Schema) or simply added by the developers, this allows impact analysis - which WSDLs are affected when I change this Schema
• Validation - we now have build in WSDL validation and WS-I compliance testing, and the system can be extended to support any level of validation of resources (e.g. WADL validation or Policy checking)
• Release process - you can simply define a lifecycle for resources (or different lifecycles). These can be as simple as promote/demote, or can build in rules to invoke different users to authorize.

These simple models are the basis of a properly managed and well-defined SOA Governance process - and its available now, completely Open Source under the clean Apache License. Download the WSO2 Registry 1.1 release now!

Google Analytics finding my trackbacks

I just installed Google Analytics on this blog - so be warned - I'm tracking you!

Anyway, its telling me the referrer URL. And from that I'm spotting trackback's on my blog entries. I just noticed I got picked up by a W3C blog written by Karl Dubost - picking up my blog entries. I'm pleased, because this was a serendipitous outcome of getting web analytics that I wasn't expecting.

Progress buys IONA

It seems that Iona is no longer in limbo - Progress Software are buying Iona for around $100m in actual value.

Back in April 2007 I joked about the perennial problem: you wait ages for a bus, and then three turn up. In Iona's case they had Artix, ServiceMix and C24. Now Progress has to somehow deal with the ESB capabilities of Sonic, Artix, ServiceMix and C24. Its going to be interesting to see how they position four ESBs to customers. Or is it five? Or maybe six? Sorry I forgot that James Strachan often says people consider ActiveMQ an ESB. That would make seven!

I can only imagine they are going to have to cut some of that technology.

NIO buffering in Synapse

Asankha has written a blog entry on how the Synapse ESB uses a constant memory buffer to effectively throttle incoming messages so that the system can handle large messages in constant memory. Worth reading - and a great idea from Oleg.

Every cloud has a silver lining

WSO2 just published an update to our ESB performance testing.

If you recall, we published two sets of ESB performance testing last year. At the time, this was the only example of a openly published ESB benchmark. Since then, everyone has updated their ESBs, so it seemed like time to redo the benchmarking.

You can read the details here, and I suggest you do, because there are always interesting details that any summary cannot capture.

Before I talk about the results, I think its interesting that in a year, no-one else has published an open benchmark. So while any vendor sponsored benchmark is likely to be suspect, the fact is that we are the only people willing to openly benchmark ESBs and publish the code, and the results - whatever they are.

The results? Well, we still maintain a healthy lead on the other Open Source ESBs we tested. Not just a lead on transactions per second, but also in stability and zero-message-loss. For example, for the most basic proxy/virtualization, on average we are 2x as fast as ServiceMix and 4x as fast as Mule.

On the other side, the proprietary ESB we tested has improved significantly from last year. In fact, for the basic proxy function, the proprietary ESB has a very significant advantage.

I'd really like to see more ESB benchmarking. Firstly, I think it helps improve the whole marketplace. Secondly, since an ESB is often adding an extra hop to every interaction, ESB performance is key.

Finally, the silver lining? Well, if there is a cloud - it has to be how much we got beaten by the proprietary ESB on the proxying large messages. And if there is a silver lining - at least we can't be accused of fixing the results!

The Unreliable Internet

A great article from Michael Galpin on WSRM and WSAS.

Apache Synapse is an ESB!

When we started the Apache Synapse project, there were a lot of vendors who showed interest in the initiative. One of those was Sonic, and Dave Chappell was the ESB spokesperson for Sonic at the time. He said a lot of things at the time that were marketing-speak, and they have remained, dutifully searchable, on the Internet for ever more.

Now I thought this was all past history - dating back to August 2005 - nearly three years ago. But I noticed this being quoted on a message board recently.

I want to set the record straight:

• This positioning of Synapse as not an ESB was clearly motivated by vendors who already were selling ESBs and didn't want it to step on their toes.
• This predates any coding on the project, and we the coders were not influenced by it - we wanted to - and did - build the most performant Open Source ESB.
• Apache Synapse is an ESB.
  

Code commits as Art

This video, showing the commit history of the Apache HTTPD server, is incredible!

code_swarm - Apache from Michael Ogawa on Vimeo.

Here is the project that produced it:
vis.cs.ucdavis.edu/~ogawa/codeswarm/

Music - "Ultravista" by SkinMechanix (magnatune.com/artists/skin_mechanix - used with CC by-nc-sa license)

Introducing WSO2 ESB 1.7 Webinar

I'm giving a webinar on the WSO2 ESB 1.7 on Tuesday June 17th at 9am Pacific, 12pm Eastern, 5pm UK, 6pm Europe (all summer time).

Register here:
https://wso2.on.intercall.com/confmgr/event_description.jsp?title=Introducing+WSO2+ESB+1.7&type=regrequired&eventid=68185

Here is the blurb!

Introducing WSO2 ESB 1.7

The WSO2 ESB is an ultra fast, light-weight and versatile Enterprise Service Bus powered by the Apache Synapse ESB. In this session, Paul Fremantle, CTO of WSO2, will introduce the new features and capabilities of the WSO2 ESB. The session will recap on the overall approach and benefits of the WSO2 ESB solution and the underlying Apache Synapse project, and then go into the added functionality and benefits of the 1.7 release. The session will include details of the newly released support for FIX, AMQP and also discuss the improvements in performance and stability.

Introduction to OpenID

Prabath Siriwardena who built the OpenID support in our Identity Server has put together the best introduction to OpenID that I've seen.

Interop with Microsoft

Yesterday, in front of an audience of thousands at Microsoft Teched for IT Professionals, Jonathan Marsh, WSO2's Director of Mashup Technologies presented how PHP and Java can interoperate with .NET WCF. The Stocktrader application is available for download now, as is WSAS 2.3 which we released yesterday.
I recommend you watch the video! If you are impatient, the interop part comes just about 20 minutes into the session.

Sanjiva has more to say about the value of interop on his blog.

Eating Plastic Bags

This is a great article. Amazing work by a scientist to find bacteria that can eat plastic bags without giving off CO2. Daniel Burd obviously needs wider recognition. He spent months carefully nurturing the bacteria that fed best on plastic bags and ended up with a 42% elimination in just 6 weeks.

The best thing about this story: Daniel is just 16 years old!

Apache Synapse 1.2 and WSO2 ESB 1.7

Apache Synapse has just released version 1.2 of the Open Source ESB, and WSO2 has released the WSO2 ESB 1.7 based upon Synapse. Although these are "only" point releases, there is a great deal of cool stuff in here:

• Support for the FIX financial industry protocol (thanks to the QuickFIX/J team for a great project!)
• Support for the high-performance Hessian binary web services transport
• Improved JMS support including support for AMQP via Apache QPid
• Non-blocking streaming of large messages, at high concurrency, in constant memory thanks to the non-blocking HTTP transport, based on the Apache HTTPCore NIO project (kudos to Oleg and the Apache HTTPComponents team)
  
• Graceful restart for continuous availability clusters
• Support for a global pre-mediation applied to all services

In addition, the WSO2 ESB 1.7 now ships with the same core registry available in our WSO2 Registry project. This gives you two choices: either simply use the registry built into the ESB, or set up a full registry including tagging, rating, comments and point the ESB at it. Or start with the in-built registry and move up to the full version when you need it.

I'd like to say congratulations to both the Synapse and WSO2 ESB teams on the releases. Well done, folks!

SD Times 100

SD Times has listed WSO2 as one of their 100 software development companies of 2007.

CTO 25

On Monday it was announced that I have been listed as one of Infoworld's top 25 CTO's of 2008.
I'm pretty overjoyed about this, and I wanted to take the time to thank the WSO2 team. We are a small company and this award reflects as much on the company as on me. When I was being interviewed by Eric Knorr, I told him this is a team effort, and I explained the nature of the Open Source model and the fact that we have a strong culture of challenging each other, both on the open source lists and face-to-face. Eric replied - "Of course - everyone always says 'its a team effort', but in your case I believe you - that the Open Source model really makes a difference".

I strongly believe - and long may it continue - that WSO2 doesn't suffer from "Drive By Architecture" syndrome. What is "Drive By Architecture"? It's when a senior exec throws a few technical thoughts out to the engineering team, and they are uncritically taken as gospel. And I think you can clearly see that the model we have works - we have one of the youngest teams in the industry but they constantly deliver amazing results. So, once again - thanks!

When does the political situation get so bad that even geeks start to notice!!

I don't normally write about politics. In fact, although I shouldn't admit it, I'm normally fairly apathetic. Yes, I vote (when I'm in the country), but that's about my limit.

However, it seems like Britain is secretly become a police state without us noticing. I think this can be squarely blamed on the 9/11 and 7/7 terrorist incidents and the aftermath, but what's worrying is that the first uses of these new laws are nothing to do with terrorism.

The first offender is the Regulation of Investigatory Powers Act (RIPA), which was designed to control circumstances in which people could be bugged, followed and surveilled. Instead it seems to have opened the floodgates. This is a law which has been justified on grounds of combating terrorism, and its being used by local councils in England to follow and photograph dog-walkers (in case they don't scoop the poop) and 3 year olds (in case the parents lied about their address to get their infant into a better school).

Now, whenever there is a debate about a new law affecting human rights the discussion goes "but what if a really bad fascist government came to power?". These examples prove that it doesn't need a fascist government, it just needs any government to be given the power before they misuse it.

The next offender is the Serious and Organized Crime and Police Act (SOCPA). It seems that Animal Protesters are now Serious criminals (I thought Serious and Organized Crime meant the Mafia, Yakuza, and Yardies!). SOCA (section 145) makes it an offence for an individual to attempt to persuade a third party not to honour their contractual obligations to an “animal research organisation”. The way this now works is that if you contact anyone more than two times in a way that might distress them (e.g. saying that you think what they are doing is wrong), then this is harrassment. Now that would normally be a civil offense, but this has now been magically transformed into a criminal offense.

The result - a peaceful protest could end you in jail with a criminal record.

And then the final straw of stupidity - you can't board a plane at Heathrow T5 (video) if you wear a transformers t-shirt. Honestly. I know it seems hard to believe.