WSO2 Summer School ESBs and SOA
View more presentations from wso2.org.
Friday, 24 July 2009
Sunday, 19 July 2009
Kindle vs 1984
Everyone has reacted with outrage over the idea that Kindle owners had 1984 removed from them. They say that in the "analogue" world this would be equivalent to Barnes and Noble breaking into your house and taking books back.
Of course the real question is whether Amazon had agreed the rights to sell this book. If they had, then they have broken users terms of service, which gives users a permanent right to own this book on their Kindle. On the other hand, if Amazon were not authorized to distribute this, then this is exactly like the analog world. If you buy stolen goods, you have to return them, whether or not you bought them in good faith. So basically its just the same as the analogue world.
Of course the real question is whether Amazon had agreed the rights to sell this book. If they had, then they have broken users terms of service, which gives users a permanent right to own this book on their Kindle. On the other hand, if Amazon were not authorized to distribute this, then this is exactly like the analog world. If you buy stolen goods, you have to return them, whether or not you bought them in good faith. So basically its just the same as the analogue world.
Saturday, 18 July 2009
Learning Enterprise Architecture lessons from being a parent
I apologize to readers of this blog who are childless, because this analogy isn't going to necessarily grab you, but I encourage you to read on, because I think there is an important point I'm going to make.
When my wife was expecting our first child, we focussed a lot on the due date. It was the be all and end all of our lives. We drew up birth plans, we went on hospital visits, we talked about it for hours. In the end, all the planning went out the window. Without going into details, it didn't go very well. In fact it went about as badly as it could - with the exception of Anna being born happy and healthy. And very shortly afterwards we realized a really important lesson. The birth wasn't the important thing. The birth was over and done with within 24 hours. What really mattered was the ongoing situation. Bringing up a child is much harder than giving birth, no matter how hard the birth is.
So what's this got to do with SOA and Enterprise Architecture? I've recently been involved in reviewing some good and bad SOA projects (not WSO2 projects I hasten to add). And one thing that characterizes some of the bad projects is the focus on the due date. A focus on delivering the baby. Get it out there. Not a focus on building a good project environment. Not a focus on how this project is going to live and evolve and grow beyond day 1, but simply deliver on the project scope and be done.
The problem with this is that designing an Enterprise Architecture isn't like designing a toy. If you design a toy, its a one-off. You create the design, ship to manufacturing and thats it. EA and SOA are much more evolutionary activities. You need to think about how to keep this "baby" alive and growing and healthy and learning. For example you need to know how:
I'd like to give a great example. One of our customers is a big purveyor of digital content to mobile devices, and they have our ESB in a continuous available mode. They are currently having peak loads of more than 1000 messages a second and the load is constantly increasing as they add new services into the system. They add services in continuously. The system configuration is updated more than once in some days, with zero downtime and heavy load. This is the kind of living system that is successful and agile and really delivers on the promise of SOA.
So if you are in a project that is 3 months from delivering I have two pieces of advice. Firstly, think how you want the world to look like in 4 months time. And make sure that is what it looks like now. Put your system in place right now, even if its not yet working. Make the production delivery look like just another milestone on a continuous plan. Treat it like an average birthday (the ones where the day before isn't very different to the day after) rather than the birth day!
When my wife was expecting our first child, we focussed a lot on the due date. It was the be all and end all of our lives. We drew up birth plans, we went on hospital visits, we talked about it for hours. In the end, all the planning went out the window. Without going into details, it didn't go very well. In fact it went about as badly as it could - with the exception of Anna being born happy and healthy. And very shortly afterwards we realized a really important lesson. The birth wasn't the important thing. The birth was over and done with within 24 hours. What really mattered was the ongoing situation. Bringing up a child is much harder than giving birth, no matter how hard the birth is.
So what's this got to do with SOA and Enterprise Architecture? I've recently been involved in reviewing some good and bad SOA projects (not WSO2 projects I hasten to add). And one thing that characterizes some of the bad projects is the focus on the due date. A focus on delivering the baby. Get it out there. Not a focus on building a good project environment. Not a focus on how this project is going to live and evolve and grow beyond day 1, but simply deliver on the project scope and be done.
The problem with this is that designing an Enterprise Architecture isn't like designing a toy. If you design a toy, its a one-off. You create the design, ship to manufacturing and thats it. EA and SOA are much more evolutionary activities. You need to think about how to keep this "baby" alive and growing and healthy and learning. For example you need to know how:
- To do updates into the live system
- To test changes in a widely distributed system
- To monitor the live system - both at the technical and the business level
- To accomodate many different parties each updating and controlling their own part
- To backoff changes that break something
- etc
I'd like to give a great example. One of our customers is a big purveyor of digital content to mobile devices, and they have our ESB in a continuous available mode. They are currently having peak loads of more than 1000 messages a second and the load is constantly increasing as they add new services into the system. They add services in continuously. The system configuration is updated more than once in some days, with zero downtime and heavy load. This is the kind of living system that is successful and agile and really delivers on the promise of SOA.
So if you are in a project that is 3 months from delivering I have two pieces of advice. Firstly, think how you want the world to look like in 4 months time. And make sure that is what it looks like now. Put your system in place right now, even if its not yet working. Make the production delivery look like just another milestone on a continuous plan. Treat it like an average birthday (the ones where the day before isn't very different to the day after) rather than the birth day!
Friday, 17 July 2009
WSO2 WSAS 3.1
Continuing my blogs about the latest Carbon releases, here are the details about WSO2 Web Services Application Server 3.1. Its based on Carbon 2.0.
Before I go into the details of the enhancements, I'd like to explain a little about what WSAS is. The question we often get asked is: "Is WSAS a competitor to a J2EE application server?". The answer is both yes and no. Yes, WSAS offers some of the same capabilities as a J2EE application server. It provides a managed runtime for hosting code, in this case Java Services. You can also deploy AJAX applications and REST services. You can write services as Pure Java (POJO), EJBs, JSR181 annotated classes, or Axis1 and Axis2 services.
Why No? Well, WSAS can be run on top of any J2EE server, and our customers use it on top of JBoss, Tomcat, WebSphere and WebLogic. We aren't trying to compete on hosting J2EE applications - what we do compete on is hosting services.
You can use those servers to host Services, so why would you choose WSAS? WSAS offers a huge amount of value add because we focus solely on what makes a good service container. Let me give you some simple examples:
What is special about WSAS 3.0 and now 3.1 is the ability to plug in new features using OSGi. Here is a simple example. Suppose you have a deployed working service, but you need to support a different XML format as well. Of course you could rewrite the service with logic to handle this. Or you could write an Axis2 handler if you are an Axis2 guru. But with WSAS 3.1 you have a cleaner, simpler and more effective approach.
Using p2 (the Equinox provisioning model), you can go to the web repository and install the mediation feature. This pulls down the OSGi bundles from the web, validates all the dependencies are in place, and adds the new capability into the system. Now when you go into WSAS you will have a new option in the UI to enable Mediation against any service. Now you can use XSLT or even simple Javascript/E4X to write a conditional transform that ensures all messages match the existing service logic. Its like having a mini-ESB embedded into your service container, but without the overhead of managing it.
p2 is the Eclipse Update Manager, except based in the core OSGi container. This means that it gives Carbon, and WSAS, the ability to control how features are deployed (and undeployed) in a very clean, effective way. It really does bring properly componentized middleware.
In addition to the new p2 capability, WSAS 3.1 also has one big ease-of-use feature - auto-wrapping JARs as OSGi. If you want to deploy a JAR you simply copy it into the right directory and we take care of OSGi for you.
Finally, this release is based on Axis2 1.5, which has a huge number of enhancements and bug-fixes over Axis2 1.4, as well as the associated Rampart and Sandesha2 releases.
Before I go into the details of the enhancements, I'd like to explain a little about what WSAS is. The question we often get asked is: "Is WSAS a competitor to a J2EE application server?". The answer is both yes and no. Yes, WSAS offers some of the same capabilities as a J2EE application server. It provides a managed runtime for hosting code, in this case Java Services. You can also deploy AJAX applications and REST services. You can write services as Pure Java (POJO), EJBs, JSR181 annotated classes, or Axis1 and Axis2 services.
Why No? Well, WSAS can be run on top of any J2EE server, and our customers use it on top of JBoss, Tomcat, WebSphere and WebLogic. We aren't trying to compete on hosting J2EE applications - what we do compete on is hosting services.
You can use those servers to host Services, so why would you choose WSAS? WSAS offers a huge amount of value add because we focus solely on what makes a good service container. Let me give you some simple examples:
- Graphical security configuration: we have the most comprehensive yet simple approach that allows you to easily configure and manage permissions, roles, WS-Security encryption, signature. There is a built-in STS and full support for WS-Trust and SecureConversation.
- Full XKMS service key management server.
- TryIt: we offer a built-in, instant dynamic test client for any WSDL service
- Full support for HTTP, SOAP, JMS and multi-transports with zero-coding. Your services are automatically exposed via HTTP POST/GET, SOAP, JMS all with a single codebase.
- Caching, throttling, and statistics on every service, with zero coding
What is special about WSAS 3.0 and now 3.1 is the ability to plug in new features using OSGi. Here is a simple example. Suppose you have a deployed working service, but you need to support a different XML format as well. Of course you could rewrite the service with logic to handle this. Or you could write an Axis2 handler if you are an Axis2 guru. But with WSAS 3.1 you have a cleaner, simpler and more effective approach.
Using p2 (the Equinox provisioning model), you can go to the web repository and install the mediation feature. This pulls down the OSGi bundles from the web, validates all the dependencies are in place, and adds the new capability into the system. Now when you go into WSAS you will have a new option in the UI to enable Mediation against any service. Now you can use XSLT or even simple Javascript/E4X to write a conditional transform that ensures all messages match the existing service logic. Its like having a mini-ESB embedded into your service container, but without the overhead of managing it.
p2 is the Eclipse Update Manager, except based in the core OSGi container. This means that it gives Carbon, and WSAS, the ability to control how features are deployed (and undeployed) in a very clean, effective way. It really does bring properly componentized middleware.
In addition to the new p2 capability, WSAS 3.1 also has one big ease-of-use feature - auto-wrapping JARs as OSGi. If you want to deploy a JAR you simply copy it into the right directory and we take care of OSGi for you.
Finally, this release is based on Axis2 1.5, which has a huge number of enhancements and bug-fixes over Axis2 1.4, as well as the associated Rampart and Sandesha2 releases.
Friday, 10 July 2009
WSO2 ESB 2.1
The latest batch of Carbon releases is out. I'll be blogging about some of the others soon, but I'll start with the latest ESB 2.1. This release is really about two major themes:
The first major enhancement I want to talk about is the transaction support that we built into ESB 2.0. Transactions are a very interesting subject, especially in the context of an SOA. I think there is a huge misunderstanding of the benefits and disadvantages of transactions in many people (obviously I don't mean you (: ). One of the disadvantages is that ACID transactionality doesn't work well in widely distributed request-response systems. However, there is a specific pattern of transactions that is very valuable in an ESB. You have to think of this as a message-based system, so even if there is a request-response interaction going on, you consider each interaction as asynchronous and separate. In this model, the message can come into the ESB over a transactional queue, and possibly leave over another transactional queue. And all the operations the ESB performs locally can be part of that transaction. For example, you might update several database and then continue. This model still doesn't imply any end-to-end transaction. But it does mean that the message is only forwarded if everything in the ESB space has continued correctly. This is the model that we added to Synapse and the WSO2 ESB in the last release. One of the design decisions that we took with Synapse was to not encumber the core processing model with transactions, but to allow it to be layered on. This means that the core engine is screamingly fast, but still allows full transactions to be enabled.
The second major enhancement I want to talk about is the event-based architecture support we added into the ESB. In the 2.1 release we have done significant performance and hardening of this. It is based loosely on WS-Eventing, but in fact, like everything in the ESB, it is completely multi-format and multi-transport. Effectively this allows the ESB to be a pub-sub broker where the subscribers can use a variety of transports and formats.
The third major enhancement that came in 2.0 was the significant improvement to the UI and the range of things that can be configured via it. This included full control of transports, role and group based authentication and authorization configuration, and much better sequence and proxy editors.
In addition we have now started supporting the Drools/Rules mediator, which means that it is very easy to add rule-based mediation into your ESB flows. We plan a lot more Rule-based integration into Carbon over the next few months.
There are a host of other improvements and additions but I think its a good time to talk about the Carbon improvements now.
What does Carbon really bring to the ESB? It brings the ability to plug in other components like BPEL (Business Process Server), Registry/Governance, Service Hosting (WSAS), and more. But how do you install those? In the last Carbon release, these were simply ZIP files you had to download and dump into the right place. But the big Carbon change we have done is to add support for p2 provisioning. p2 is the Eclipse Update manager, recast as a core part of the Equinox OSGi stack. This means that we now have a well defined set of features, with a feature repository, and the command-line p2 tools to install and uninstall them into any Carbon-based product.
Congratulations to the whole Carbon team on this release and kudos to the ESB team for the great product they've built. Download it now - and let us know how we can make it even better!
- Updating the core Carbon platform
- Tightening up the significant additions that went into ESB 2.0
The first major enhancement I want to talk about is the transaction support that we built into ESB 2.0. Transactions are a very interesting subject, especially in the context of an SOA. I think there is a huge misunderstanding of the benefits and disadvantages of transactions in many people (obviously I don't mean you (: ). One of the disadvantages is that ACID transactionality doesn't work well in widely distributed request-response systems. However, there is a specific pattern of transactions that is very valuable in an ESB. You have to think of this as a message-based system, so even if there is a request-response interaction going on, you consider each interaction as asynchronous and separate. In this model, the message can come into the ESB over a transactional queue, and possibly leave over another transactional queue. And all the operations the ESB performs locally can be part of that transaction. For example, you might update several database and then continue. This model still doesn't imply any end-to-end transaction. But it does mean that the message is only forwarded if everything in the ESB space has continued correctly. This is the model that we added to Synapse and the WSO2 ESB in the last release. One of the design decisions that we took with Synapse was to not encumber the core processing model with transactions, but to allow it to be layered on. This means that the core engine is screamingly fast, but still allows full transactions to be enabled.
The second major enhancement I want to talk about is the event-based architecture support we added into the ESB. In the 2.1 release we have done significant performance and hardening of this. It is based loosely on WS-Eventing, but in fact, like everything in the ESB, it is completely multi-format and multi-transport. Effectively this allows the ESB to be a pub-sub broker where the subscribers can use a variety of transports and formats.
The third major enhancement that came in 2.0 was the significant improvement to the UI and the range of things that can be configured via it. This included full control of transports, role and group based authentication and authorization configuration, and much better sequence and proxy editors.
In addition we have now started supporting the Drools/Rules mediator, which means that it is very easy to add rule-based mediation into your ESB flows. We plan a lot more Rule-based integration into Carbon over the next few months.
There are a host of other improvements and additions but I think its a good time to talk about the Carbon improvements now.
What does Carbon really bring to the ESB? It brings the ability to plug in other components like BPEL (Business Process Server), Registry/Governance, Service Hosting (WSAS), and more. But how do you install those? In the last Carbon release, these were simply ZIP files you had to download and dump into the right place. But the big Carbon change we have done is to add support for p2 provisioning. p2 is the Eclipse Update manager, recast as a core part of the Equinox OSGi stack. This means that we now have a well defined set of features, with a feature repository, and the command-line p2 tools to install and uninstall them into any Carbon-based product.
Congratulations to the whole Carbon team on this release and kudos to the ESB team for the great product they've built. Download it now - and let us know how we can make it even better!
Thursday, 9 July 2009
To ESB or not to ESB
Ross from Mule has written an excellent blog entry on when an ESB makes sense. I think its very pragmatic advice.
The first point I want to make is that there is a huge difference between Open Source and Proprietary models here. Firstly, Open Source ESBs are usually pulled into projects rather than pushed onto projects. WSO2 certainly doesn't have salesmen taking people out for nice drinks and persuading them they have to have an ESB. This is a point that Ross makes very clearly and strikes a strong chord with me.
I've recently evaluated a project where I feel like the technology has been used because its there. Maybe its been used to justify the choice of an expensive software stack, but fundamentally the ESB had been thoroughly misused. In particular, there was not a single interface that could be used by more than one client or flow.
The second observation I have is about the ownership of the logic in the ESB, and the placement of function. The way that this ESB was developed was "monolithically". In other words a single team had the change management of the whole ESB, and the way it was run involved a multi-week long process to get even the most minor changes. This kind of model goes against the grain of an agile SOA and also makes the development teams struggle with the whole ESB model. By contrast we have a customer who has a model where changes can be (and actually are) made to the production system on a daily or even hourly basis, with zero downtime. This is on a system that handles peak loads equivalent to 85million messages/day.
We are working a lot on this issue of how to manage multiple concurrent ESB configurations and I think this will be a key improvement to the overall ESB approach.
The final comment I'd like to make about "to ESB or not" is that another thing we are often asked for is the ability to embed "ESB-like" function in a runtime. This is another good design pattern that can avoid problems: embed the transformation directly into the target system thereby simplifying point-to-point or even ESB-based architecture. This is exactly the model that Carbon allows, where mediation and transformation can be selectively added to Data Services, Business Processes, Service Hosting, etc.
The final point is about the responsibility of software suppliers to software users. Its always possible to misuse software. One of the very first experiences I had of ESBs was a large company building way too much business logic into their ESB layer and then beating up the supplier about it! In that case I think it was fair - the supplier had sat back watching the revenues come in and hadn't given the customer straight talk about how the product should best be used. This is the flip side of Open Source - we don't always get involved in the customer's solution until they are a long way down the path to production. But I'm always very straightforward - if I believe that the ESB is being misused or shouldn't be used, I'd rather lose the support revenue and gain the customer's trust by telling him or her straight.
The first point I want to make is that there is a huge difference between Open Source and Proprietary models here. Firstly, Open Source ESBs are usually pulled into projects rather than pushed onto projects. WSO2 certainly doesn't have salesmen taking people out for nice drinks and persuading them they have to have an ESB. This is a point that Ross makes very clearly and strikes a strong chord with me.
I've recently evaluated a project where I feel like the technology has been used because its there. Maybe its been used to justify the choice of an expensive software stack, but fundamentally the ESB had been thoroughly misused. In particular, there was not a single interface that could be used by more than one client or flow.
The second observation I have is about the ownership of the logic in the ESB, and the placement of function. The way that this ESB was developed was "monolithically". In other words a single team had the change management of the whole ESB, and the way it was run involved a multi-week long process to get even the most minor changes. This kind of model goes against the grain of an agile SOA and also makes the development teams struggle with the whole ESB model. By contrast we have a customer who has a model where changes can be (and actually are) made to the production system on a daily or even hourly basis, with zero downtime. This is on a system that handles peak loads equivalent to 85million messages/day.
We are working a lot on this issue of how to manage multiple concurrent ESB configurations and I think this will be a key improvement to the overall ESB approach.
The final comment I'd like to make about "to ESB or not" is that another thing we are often asked for is the ability to embed "ESB-like" function in a runtime. This is another good design pattern that can avoid problems: embed the transformation directly into the target system thereby simplifying point-to-point or even ESB-based architecture. This is exactly the model that Carbon allows, where mediation and transformation can be selectively added to Data Services, Business Processes, Service Hosting, etc.
The final point is about the responsibility of software suppliers to software users. Its always possible to misuse software. One of the very first experiences I had of ESBs was a large company building way too much business logic into their ESB layer and then beating up the supplier about it! In that case I think it was fair - the supplier had sat back watching the revenues come in and hadn't given the customer straight talk about how the product should best be used. This is the flip side of Open Source - we don't always get involved in the customer's solution until they are a long way down the path to production. But I'm always very straightforward - if I believe that the ESB is being misused or shouldn't be used, I'd rather lose the support revenue and gain the customer's trust by telling him or her straight.
Friday, 3 July 2009
Thursday, 2 July 2009
Subscribe to:
Posts (Atom)
