Event Stream Processing in Synapse

There's a lot of buzz around Esper at the moment. It seems pretty cool to me. I hear that BEA has dual-licensed it and is shipping it as part of (all of?) their new Event Server, so I took a quick look at making it worth together with Synapse.

So what is Esper and how can it work with Synapse?
Basically, you Esper will look for patterns in a set of messages. When it spots a pattern, it creates an event. Because Esper supports XML and XPath through DOM, it fits perfectly with Synapse. Without writing any code you can configure it to look at SOAP or XML messages as they flow through Synapse. When a query "hits" it sends a new message into Synapse. That message can then be sent out or logged. The message can either be an exact copy of one of the messages that came into Synapse or it can be a set of tag/values extracted by Esper.

Esper uses a query language just like SQL except called EQL. Because there aren't any tables, instead you define the root element of the XML and give that a name. You can also pass in a schema. Now you can do selects on that just like it was a table.

It took me less than a day to get it working. In fact, the first message worked pretty soon. It took me longer to get XPath working. I suspect there is a bug in Esper and I've raised a JIRA. I wasted a bit of time on that. I also had to learn Esper, but that turned out pretty simple. I certainly haven't got all the features working, and I only tried a really simple example.

Basically what I did was create a Synapse class mediator. You can add it to your synapse.xml like this:

<class name="org.fremantle.esper.EsperMediator">
<property name="Configuration"
value="c:/synapse-1.0/repository/conf/esper.conf.xml"/>
<property name="statement"
value="select symbol from StockQuoteEvent"/>
<property name="EventToAddress"
value="http://localhost:9999/soap/EventListener"/>
</class>

When an event hits that matches the statement, the event will be re-injected into Synapse with the To address as specified. You can then either mediate it or have a rule to send it out. If you specify "select *" then the whole message will be sent on.

If you specify parts using an XPath, then I simply create a little XML that captures those:
e.g.
<event xmlns="http://fremantle.org">
<entry key="symbol">IBM</entry>
</event>


The Esper configuration that we pointed to above needs to define the XML objects that you are going to see. That allows you to query on them.

<esper-configuration>
<event-type alias="StockQuoteEvent">
<xml-dom root-element-name="getQuote"
default-namespace="http://services.samples/xsd"/>
</event-type>
</esper-configuration>

The code is a bit of a hack at the moment. Ideally we will make a couple of changes to Synapse to make it cleaner. These were already in discussion, but this has pushed them up my priority list.
I'd also like to create a XMLMediatorFactory so this can be plugged in a little more neatly - for example having the Esper config inlined in Synapse.xml. And I'd like to make it more robust and support things like patterns and some of the more advanced Esper features.

Despite that, I think this is enough to get going and play with it. I'm really pleased. The combination of the Synapse performance with Esper's flexibility is a cool combination.