SOA: The Human Angle

Most discussions on SOAs assume an architectural model in which services are provided by software and these services are in turn consumed by clients (e.g. user interfaces) or are orchestrated by a process engine. A good example of this type of thinking has been recently published by a set of authors from IBM in IEEE IT Pro. In their article, “S3: A Service-Oriented Reference Architecture” they describe a metamodel which:

“ … captures architectural building blocks … as well as the relations among these blocks and among layers, interaction patterns, options, and architectural decisions.”.

However , although most of these SOA models talks about business processes they give scant attention to the role of humans within those process. For instance, at one stage in a credit process we might need someone to approve the loan. Effectively this means that the approver is providing a service – deciding on loans using their experience and intelligence. In the IT Pro article we come across this statement:

“An SOA ... would support that process by creating a composite service that choreographs the information flow among a set of services and human actors”

But that’s it. How the human actors are actually integrated into the SOA is not explained. This is fairly typically for most of the literature in this area which assumes an architectural model such as:



I was therefore interested to see that the BPEL4People draft specification has been recently published by a consortium of companies with the intent of getting this ratified by OASIS as a WS-* specification. BPEL4People is a set of extensions to the BPEL specification that address scenarios in which humans are involved in business processes. How it does this from an architectural point of view I’d like to explore in my next entry.

German government specifies SOA

The German federal government has released an IT Architecture for the administration. Surprisingly (or perhaps not, depending on your PoV) it's service oriented.

Currently it is only in German, but often such documents are translated into English and published on the same web site later.

More on ESBs

Just to muddy the waters in trying to explain what an ESB is (and why we need one) I came across this (dated 2005) presentation from Jim Webber (you have to display it in presentation mode to see it all). Jim doesn't hold much with ESB vendors and believes that the "bus" should just be SOAP and WS-Addressing.

The slide set reminds of a colleague (I wish I could remember his name) who once said "it is almost criminal to let a vendor determine your architecture".

Point-Point, Hub-and-Spoke, ESB

My colleague Mark Masterson asks in his blog about the “fitness of the classical "hub and spoke" architecture in a modern enterprise systems landscape” and “what, exactly, is the purpose of an enterprise service bus (ESB)?”

As he points out there is a lot written about this, but nothing seems to be conclusive. When confronted by such situations I attack them by asking “want do we really want” and then work backwards to “what have we got”.

So, the first question is; do we want a “hub and spoke” architecture? Well, this was proposed as an answer to the point-to-point interconnection style. Continuing on with this ruthless analysis (so beloved of Dr. House) let’s ask the seemingly dumb question; do we need a point to point interconnection style?

Point to point came about because people needed to transfer data between applications. As time went on, more and more applications were written and, with them, more and more point to point interconnections were produced. This lead to a nightmare IT landscape which nobody could understand, let alone maintain (I once did an application inventory for a client in which the interfaces were also documented – the result filled four sheets of DIN A0 and made for interesting wallpaper!).

To solve this the hub-and-spoke architecture was proposed, which lead to EAI tools etc. etc.. But that was then and this is now and I think we have to ask again: why point-to-point?

In the brave new world of SOA we don’t have applications which need to pass data between themselves. Instead we have autonomous services. These are arranged in a hierarchy (although not a strict one) . Services are consumed by other services (e.g. composite services call atomic services) or, at the top nodes, by service consumers such as user interfaces. If we start building systems to implement this architectural style, we soon start to realize that certain functionality is required time and time again. We need to implement small processes which call services in predefined sequences, data needs to be aggregated from many sources or needs to be distributed to other services. The data needs to be transformed, enriched, split up. In addition we need build low level coordination between the services, so that we can implement concepts such as transactions and security. We also need to ensure that data is really transfered to the services (i.e. reliable delivery) etc..

All of this is what we really need to implement an SOA. But instead of implementing this all ourselves, we buy these things from somebody who has packaged it all together and given it a name - ESB.

Using this analysis means that the original question of fitness of a hub-and-spoke system is a question that does not need to be answered as a hub-and-spoke architecture is irrelevant for a modern systems landscape if an SOA style is used.

Designing Interfaces and REST

I've blogged in the past about how the design of interfaces in distributed systems is an important, but often overlooked topic. Instead the current debate seems to be around the technology used - SOAP, REST - rather that the interface design itself.

I was therefore interested to come across this blog entry from Joe Gregorio in which he applies REST technology to the design of the Apache DayTrader benchmark. What I find striking about this entry is how much care he has given to the interface design, reducing the original, and somewhat ad-hoc interface specification, to simpler principles based on a few entities and operations.

Tools for Globally Distributed Development (GDD)

Working for a company that routinely spreads its development out through the world I was interested to come across this report from Ryan Bagueros on how a small software development company can set up the tool support for GDD on the cheap by using tools available on the web. As he says:

"With some careful setup, any group of developers can compete with the budgets used by multinational software development firms to distribute their application engineering ..."

Of course, these tools do not need to be used on the scale of global development, but can be used on smaller scale developments where the developers do not all sit together.

Using BPMN

As an architect I often have to sketch the processes to see how everything goes together. Over the last year I been increasingly using BPMN to do this, either on paper or with Visio. Unfortunately there doesn't seem to be much out there in the way of books which explain how BPMN should be used . Instead, I use a couple of tutorials written by Stephen White from IBM. Those with patience can use the textual version, for the impatient the slide show is the best.

As to sketching processes in Visio a number of stencils (with various levels of completeness and features) are available on the bpmn.org site.

Developing rich internet apps - what to use

A quick round up of what seem to be the current major development libraries and/or environments for developing RIAs.

Flash is the well-known browser plug-in from Adobe as well as the development platform. Adobe have also announced in the mean time - but not released - Apollo as a cross platform run-time that handles Flash, Flex, PDF, plain HTML or Ajax based user interface. Effectively a type of universal client instead of the browser.

DoJo is an open source javascript library. As well as the usual UI components, one feature I find interesting are components to store data on the local drive. I first saw this feature used in TiddlyWiki.

script.aculo.us is also a javascript library. It is based on the Prototype javascript library.

Google Web Toolkit (GWT). The GWT lets developers write code in Java which is then converted to JavaScript apparently apparently freeing the developer for learning JavaScript (Java developers are assumed here, not a bad assumption for most enterprise development work). This has been nicely written up in a IBM developer world article.

Are there any more?

Loosely coupled interfaces

I posted a short while back an entry about designing generic service interfaces. By chance I happened to come across this (rather old) blog entry by Carlos E. Perez in which he proposes extending speech acts to define loosely couple interfaces. He bases it on a paper by researchers at Stockholm University in which they try to provide a universal framework for human communication in an economic domain, i.e. the business world (the paper is still marked as work-in-progress, so I guess it hasn't been peer reviewed). To do this they use a a triple:

(intention, action, object).

The values each element can take on are:

	Intention		Action
	propose		create
	accept		change
	reject		cancel
	declare		none
	query
	reply
	assert

So, to create a new customer record the triple could be:

(propose, create, aCustomer)

the service could then return with:

(accept, create, aCustomer)

or, in the case of an error could return:

(reject, create, aCustomer)

You can have a lot of fun thinking up new combinations. One thing to notice is that services themselves don't seem to have names. Instead, one could envision a dispatcher which takes each received triple and then finds the correct service implementation.

If this would work out in practice, I do not know. But it seems to aid support to my argument that the REST vs SOAP debate is about which technology should be used, and not about designing interfaces to achieve loosely coupled services. How to achieve this should be the real debate.

Quantum Computers

Really?

The next n-tier architecture

It seems that the now traditional three tier architecture may well be slowly coming to an end. For specialised high performance applications, the bottlenecks between the tiers have already been pushing developers in this area to turn to specialised data caching platforms such as Tangosol, or to unified application platforms such as Intersystems Ensemble. In the data caching area the architecture seems to be consolidating to a cached data pool which is accessible from many machines in a cluster. This trend is now being followed by vendors of "traditional" application servers such as BEA and IBM. For SOAs this has the advantage that each service accesses the same data in the data pool, rather than having to aggregate data from different data sources (see diagram). The application platform deals with all the hard problems of data caching such as data consistency, cache actualization etc.


Not so long ago I saw a presentation from GigaSpaces who also provided a cached data pool. What is interesting about the product architecture is that they have revived the old, and hardly adopted, JavaSpaces concept. The data pool (or space) is a set of Java objects which are effectively in a shared memory accessable from each machine in a cluster. Each object can be accessed by providing a "template" which matches the contents of the objects and returns one of more of the matching objects. Templates can also be set as triggers, so that creation of new objects with particular data characteristics can notify another Java object. If you're still interested, a good tutorial on Java Spaces can be found here.

Building Virtual SOAs

Doug Kaye has written about how he used Amazon web services to build the infrastructure for his application, i.e.

• elastic compute cloud to run the apps
• simple storage services to store the content
  
• simple queue services to connect the apps.

And talking about building applications, Coghead (which has recently gone into limited beta) lets users define screens which can trigger user defined orchestration processes and data mappings. Just about what every ESB/EAI/SOA tool can do except that this uses a SaaS model.

Using these tools, what stops you defining a "virtual" SOA, in which the Coghead defined UI triggers the orchestration which calls up web services implemented using the Amazon virtual infrastructure?

On Generic Interfaces

In his comment on my last post , Steve Vinoski warns against getting into “pseudo-religious technical debates like "REST vs. SOAP”. I agree.

However, for an enterprise architect implementing (or planning to implement) an SOA, the decision still has to be made as to what technology should be used – a decision which could have a considerable influence on the costs and success of any SOA initiative. Such a decision needs to be based on arguments which corporate management understands, rather than hearsay about what apparently works or does not.

The difficulty in making such architectural decisions is not on seeing what simply works. SOAP works, so does REST. Way back in the mists of time (i.e 1999) I worked on a project which used a REST style of communication between the client (an applet - remember those) and the server. I made the decision to use a simple HTTP interface as no other alternative existed then (according to Don Box, the SOAP spec only shipped in 4Q1999). It actually worked surprisingly well (and is still working to this day). REST can work in an enterprise environment.

The problems start when we come to architecture in the large. In large architectures other factors come into play, such as how to manage the interfaces in the face of changing requirements. As Steve mentions in his article "... some of the architects and developers I know who work on large SOA systems ... figured out the uniform interface constraint on their own, without ever hearing of REST". Developing generic interfaces does help, but how generic does an interface need to be?

In the REST model the generic interface supports the four interaction verbs described in the HTTP spec, i.e. GET, POST (equivalent to create), PUT (equivalent to update) and DELETE. However, taking one extreme it is possible to reduce POST, PUT and DELETE to one verb - MAINTAIN - and split the data payload into a before image (BI) and an after image (AI). Specifying the BI and leaving the AI empty is equivalent to a DELETE. Specifying the AI and leaving the BI empty acts like a create. Specifying both is an update, with the advantage that the system can check if the data has changed in the meantime by looking at the BI. Apart from database interfaces I haven't seen anything written about this technique in creating SOAs. I have, however, seen one system which implemented it in a web service context and with SOAP.

As the above example shows, you can still provide generic interfaces with the SOAP/WSDL combination as well as with REST. It seems to me that the SOAP vs. REST debate is arguing about the wrong things. It should about how to design interfaces rather than the technology used for their implementation.

REST vs. SOAP Reloaded

It seems that Google have decided to drop its SOAP interface in favour of a REST style interface. This again raises the REST vs. SOAP debate. I was therefore interested in reading this article from Steve Vinoski of IONA Technologies. Although he tries to "explain REST from the viewpoint of someone steeped in SOA..." I had the sneaking feeling that he came down on the side of REST, which is even more surprising when you realise that IONA was one of the original exponents of CORBA and has been heavily pushing SOAP over the past years.

Where are the composite app standards?

In reading about composite applications, I was surprised to see how little work in terms of open source and open standards is being done in this area. I found the following standards:

• XForms for web based forms
• BPEL for orchestration
  
• WS-CAF from OASIS has only released one part of 3 planned standards.
  
• and of course the whole raft of web services standards
  

Missing are standards for:

• Screen flow, i.e. controlling the order of screens presented to the user.
• Work flow in terms of transferring work between people (the so called BPEL4People standard has yet to be published and maybe is not what we want.
• Screen layout.
• Packing all the bits of a composite app together so that it can be shipped (similar to web apps today).
  

All this is available in propriety products, which makes me surprised to find that nobody is working on standards in this area and even more surprised that nobody is working on an open source version of a composite application framework.

Will the real composite app please stand up

In trying to get to grips with what is meant by a composite application I came across this blog entry from Jeff Calow of IBM. He writes that the usual definitions of composite applications all deal with automation, but then asks about the user interface aspects. I would like to extend this and assert that a composite application consists of three optional aspects:

• A user interface which brings together in one place everything the user needs to carry out a task. This could be a portal or a scripted front end as found in many mashups . Rich clients are another possibility.
  
• A business process execution environment that not only handles the automation of processes, but also deals with user centric aspects such as workflow and screenflow.
  
• An integration infrastructure that allows data to be transformed from one format to another.
  

IMHO any self respecting composite application framework should support all three aspects.

More on BPM and OO

Keith Harrison-Broninski starts to ask in his blog why process modelling is based on a series of activities rather than business objects. He believes that we need an additional object based modelling layer for high level objects and their interactions.

I’ve tried to show in a previous post that BPM and object orientation are, at a deep level, conceptually similar. Rather than yet another layer which represents yet another set of concepts, we really need a unified model which lets modellers (and the underlying software) do what the business wants without having to constantly bamboozle them with different representations.

Reuse and SOA

In the last week I have heard and read the usual lament that software reuse is hard and has been a dismal failure. As many claim that the main value preposition for SOA is based on reuse, the argument is that it also will be a dismal failure.

Here I have to take a heretical POV and contend that software reuse is maybe hard, but that it exists, has been an enormous success and forms the mainstay of software development today.

Let’s look at UIs. Nobody would develop a UI today without a set of reusable components. What about basic programming. One only needs to look at how rich and extensive the J2EE or .NET class libraries are and how much their (re)use is part of standard software development practices is to see that reusability is the norm rather than the exception.

But what about the bit in the middle, the so called business logic? The enormous success of ERP companies such as SAP is surely adequate proof that reusable business logic is a reality.

The only thing that stands out here is that reusable components are not developed in-house by IT-Shops (as David Chappell pointed out when he asked about in-house reuse programs), but are developed by vendors who:

a) invest heavily into their products and

b) have an industry wide view point instead of a restricted company view point.

Perhaps this is clear indicator to where we can expect the majority of reusable business services to come from.

Difference between OO and BPM Part II - Inheritance

I tried to show in a previous post that the difference between the brave new world of BPM and the old traditional world of object orientation is really not so different. I left off wondering about inheritance and how that manifests itself in the BPM world.

Now every OO programmer knows what inheritance is. Or do they? It seems that inheritance is a slippery concept. Way back in 1996, Bertrand Meyer (the inventor of Eiffel) published an article in IEEE Computer in which he listed 12 different types of inheritance. It's no wonder that the usual advice given to an OO developer is to keep the inheritance hierarchy shallow – if you don’t, you’ll get swallowed up in the complexity.

Judging by the amount of academic papers written on this theme, it seems that process inheritance is also a complex topic. One which caught my eye, was a paper by Guangxin Yang as part of his Ph.D.. In it, he models process as a set of activities, each of which can trigger events which, in turn, cause messages to flow to other activities. In this way he can allow inherited processes to add new activities, add events, enable/disable triggers and also to change the semantics of the activities so they fit more closely to the business requirements. Although these seem to be powerful ideas, Yang’s paper doesn’t seem to have attracted much attention, only getting 2 citations according to Google Scholar.

So far, process inheritance does not seem to have reached commercial products, although this would seem to be a very useful feature to have. For instance, a typical business process could be defined and customer specific extensions could then be made to the inherited process in the same way that specific features are added to inherited classes in the OO paradigm. Some readers may recognise this. It is how most ERP systems are implemented today, only the “inheritance” mechanism is implemented by programming user exits and/or adding new customer build activities. The disadvantage here is that these changes are specific and propriety to the ERP system being implemented. If we want to implement processes over a heterogeneous environment (i.e. a lot of different applications or over the web) then inheriting from a process running in an external process engine would be an obvious solution. And with the rise of web services, composite apps a.k.a mashups, it wouldn’t surprise me if we start to see inheritance mechanisms creeping into the feature sets of process engine vendors.

Speeding up SOAP requests in a SOA

In a previous post I mentioned how efforts are under way to inprove the speed of XML processing. Another approach is taken in this article I came across in which the author shows how SOAP requests in a SOA can be cached in a middle tier.