Sunday, April 13, 2008

JBI Misses the Mark

Every now and again I get into a discussions about why I decided not to adopt JBI for Mule. Admittedly, the topic has cropped up less and less over the past 2 years since people realize that just because something is called a standard doesn’t make it the best solution. However, now and again there is a die-hard fan that is there to push JBI as the only way. It’s been a few years since I read the JBI specification, but here goes.

When JBI (Java Business Integration a.k.a. JSR-208) first started cropping up it generated some interest since it was an attempt to standardize an area of application development where there were a lot of moving parts and complex problems to solve. Of course I looked at the spec early on since Mule was a platform that operating in the integration/ESB space.

My initial reaction was mixed since I felt there was scope for standardization in integration but the scope of JBI seemed to intrude into the problem space much further than was required. Integration problems are varied, complex and different for every organization because the technologies, infrastructure and custom applications in organizations are always different. Furthermore, they are immutable. This is a key point since many proprietary integration solutions (EAI brokers, ESBs, SOA suites) assume that an organization is either creating a green-field application or can rip-and-replace pieces of their infrastructure to make way for the Vendor X way of doing things.

Mule was designed around the philosophy of “Adaptive Integration”. What this means for Mule users is that they can build best-of-bread integration solutions because they can choose which technologies to plug together with Mule. It also means that they can leverage existing technology investment by utilizing middleware that was purchased in the past from other vendors. When talking about integration or SOA I think the piece with the most value is the glue between systems. With that said it is vitally important that this glue is as flexible as possible in order to be useful for a wide range of integration and SOA scenarios. This is one area I think JBI went wrong.

JBI attempts to standardize the container that hosts services, the binding between services, the interactions between services, and how services and bindings are loaded. It sounds like a great idea, right? The problem occurs when the APIs around all these things makes assumptions about how data is passed around, how a service will expose its interface contract and how people will write services. These assumptions manifest themselves as restrictions, which as we know is very bad for integration. These assumptions include –

  • Xml messages will be used for moving data around. For some systems this is true, but for most legacy systems, Xml didn’t exist when they were built and they use a variety of message types such as Cobol CopyBook, CSV, binary records, custom flat files, etc.

  • Data Transformation is always XML-based. Transformation is hugely important for integration, so to assume all transforms will be XML and use the standard javax.xml.Transform library was a huge limitation.

  • Service contracts will be WSDL. This might seem like a fair assumption, but again it’s very XML-centric and WS-centric too. We know that back and middle office integration is no place for Web Services. What about other ways of defining service contracts such as Java Annotations, WADL or Java interfaces?

  • No need for message streaming. There is no easy way to stream messages in JBI. It just wasn’t a consideration. Some vendors have worked around the API to support streaming, but that defeats the purpose of having an API.

  • You need to implement a pretty heavy API to implement a service. This means the guys writing your services need to understand more about JBI than is necessary. Mule has always believed that services can be any object such as a POJO, EJB Session bean or proxy to another component. This means it does not force developers to know anything about Mule logic, only write logic relevant to the business problem at hand. It’s worth noting that vendors have also found workarounds in JBI to allow developers to deploy POJO services, but it quickly starts looking like JBI is working against them.

  • It’s not actually that clear what a service engine is in JBI. JBI seems like a container of containers, but what about services? Do I need to host an EJB container inside JBI and then have an EJB Session bean as my service? Do I write a service engine per service? It seems both may be valid, but I never thought either were optimal.

There are other issues as well. The JBI specification is obfuscated with lots of new jargon. For a new developer they sheer amount of knowledge required to get started with JBI is a little overwhelming. This could be said for any middleware but I think JBI is one of the worst for new developers to grasp.

The JBI specification does talk about roles of the different types of users that will interact with a JBI system. This is a good approach, but in reality it was difficult to understand primarily because it seems JBI was designed without really thinking about how the developer would work with it. Why? Well, when you get a load of vendors (though not Oracle or BEA) to sit around and design an integration system, they will design a system around the way they see the world. This is exactly what vendors did before JBI too, and we were often not happy with the products we were given. JBI seems to be a “standard” written by middleware vendors for middleware vendors.

This “vendor view” of the world is one of the main reasons Open source has done so well. Traditionally, Open Source has been written by developers much closer to the problem being tackled. These developers can deliver a better way of solving the problem using their domain knowledge, experience and the need for something better. This was the ultimate goal Mule and given the success of the project I believe that goal has been realized with the caveat that things can always be improved (which we continue to do).

Also I think the far-reaching scope of JBI affects its re-usability across vendors. By their nature, vendors need to differentiate themselves in order to compete. Because JBI attempts to define how everything should work vendors have to build in features and workarounds that go beyond the specification to differentiate their service container. This breaks the re-use story since, if I use a JBI Binding Component in one container doesn’t mean it will behave the same way in another container.

This leads me to my final point, which is that one of the big selling points of JBI and standards in general is to promote re-use. But I don’t think we’ve seen much re-use from JBI. Where is the library of Service Engines and Binding Components? I know Sun have started porting their J-Way connectors to JBI, but nobody supports them. If you look at every JBI implementation each has written their own JMS, FILE, HTTP, FTP etc Binding components… not exactly what I’d call re-use.

One thing that concerns me about the JCP is that when something is released through this process people label the resulting work a standard. This is a little strange to me since I think of a standard as something that is defined when the problem domain being addressed is fully understood. For example, TCP/IP is a real standard because it states a well-defined protocol for exchanging information over a network. Its scope is very clear, the protocol is precise and it deals with a single problem. JMS on the other hand is not really a standard, it’s a standard API defined ultimately by two vendors (IBM and TIBCO) who made sure that the standard suited their needs. Not necessarily the needs of Java messaging. This is why JMS seems to have some quirks to it and the API is heavy (though improved in JMS 1.1).

The fact that JCR-312 is called JBI 2.0 not JBI 1.1 means that the forces behind JBI 1.0 realize the flaws and are looking to make amends. To all those that badgered me in the past about adopting JBI in Mule may now retract some of their overzealous statements since JBI 1.0 already looking obsolete. I am glad I did not pollute Mule with JBI.

I had some good conversations with Peter Walker (co-lead) about joining the expert group for JSR-312. I appreciated the community spirit of Peter reaching out but I felt the direction wasn’t compelling enough. It seemed to be going in a similar direction. We talked about OSGi and SCA support and that these may be weaved into the specification. While I think this is a good idea I don’t think it’s up to a standards committee to define how to build a server. Again, the scope seemed too broad.

So, should we just give up? No, of course not. But I think we should re-align our sights to target the particular areas that make sense to be standardized. From my experience there are two areas that would make sense to standardize; configuration and connectors. This is because these are the pieces that either the user or other systems need to interact with.

Service Component Architecture (SCA) addresses the XML configuration piece and I do think this is a step in the right direction (I’d like to see the same done for programmatic configuration). The issue with SCA right now is people still think it’s a bit Web Services centric and the configuration is quite verbose. Another major issue is that SCA hasn’t received the adoption required to be a front-runner.

On the Connectors front we do have , but anyone who’s read the JCA spec may wonder how on earth a bunch of smart people could come up with such a recalcitrant API.

I’d like to see a compelling connector architecture come out of the JCP process. Focusing on this specific area may promote middleware vendors and other independent software houses to build re-usable connectors that we could all use.


Yuen-Chi Lian said...

I had tried a few times to look for articles or discussions on Mule vs. ServiceMix, and usually the argument revolves around JBI compliance.

When I was reading "Enterprise Service Bus" by David Chappell, I found that XML is being over-emphasized, e.g. XML as canonical format, XPath and XSD for filtering and validation. I could not agree entirely, although I could agree that XML is an essential and standard technology in SOA. It still remains as an option; tightly coupling SOA/ESB with XML is an oxymoron.

From what you described, it looks like JBI (and thus ServiceMix) shares the same "idealogy" with the book.

- yc

douglas dooley said...


Good post, leave it 2 u 2 make me think that I am washed-up on this topic, as u have done your homework on JBI and made a compelling case against it being a 'religious' decision...

I come from a background on Sun's app server which included an effort to standardize EJB components enough to make them portable and re-usable across app server platforms...perhaps u recall Theory Center before BEA bought them, kind of like that....

Unfortunately, I am still waiting for that independent marketplace of components to come to fruition, but it does not mean that dream has died, even considering we are going on EJB v. 3.1, and the onslaught of negative EJB-noise from the Spring base...

It is a long and arduous process to get disparate vendors of all sizes and focus to agree on a spec. that suits their needs and strategies...however, if we have learned anything about how to make this a non-.Net world, or even a non-Windows client app. world, it is that the community has to come together on standards...

The problem may be that that delivers too much power to Sun, in the form of the only currently working community: the JCP...but OSS, in the form of Glassfish and MySQL offer a model that is superior than to say WebSphere and DB2, and allows vendors, like Mule, to sleep better at night that their business will not be swallowed by a proprietary extension...

At present, u may b right that this list of binding components:

is not supported across platforms, but it is a start, and I just dont see anything else that comes close to providing some assurances to customers that they will not be locked-in, a la EAI...

In summary, it is one thing to slam JBI, it is quite another not to have a standards-based integration story or leadership role, u guys would run circles around the openESB and JBoss teams on JBI, i am still, even after diligently reading your 4/13/8 post, trying to figure out why exclude when u can include and compete head-head on that many more RFPs, where the customer has experience in standards, whether imperfect or not...

with that said, all the best, to you and Dan Diephouse who I got to have a beer with some months back, i acknowledge u have a 1st-rate team, and what appears to be a solid product-line, just keep your eyes open to all opportunities, and don't forget to keep watching Sun and JBoss on this issue, though middleware app server vendors, they have a line on standards that will be compelling to some of your customers...

Ross Mason said...

yuen-chi, The right word is "idealogy". There is nothing "ideal" about integration which is why assumptions about the environment are a bad thing. It's fine to dream about how things may be one day, but with the amount of legacy we have in this young industry already, standards needs to focus on smaller pieces of the problem as not to force assumptions in areas that need not be affected.

Ross Mason said...

The JBI decision was a pragmatic one looking at the problem at hand. We have embraced many standards in Mule, but it would have been detrimental to the Mule platform to adopt JBI at it's core. I would recommend to any company choosing and ESB/SOA technology to try it first to make sure it suited their needs. For those companies that choose technology via RFP's alone, they will probably choose IBM. BTW there is interesting fact somewhere that the RFP process yields alarmingly low follow-up business for vendors i.e somewhere south of 5%.

jlorenzen said...

It was difficult getting my entire response in this comment, so I replied in my blog. Please see my response here and I await your feedback.

eKorpRat said...

I agree that many of the people within the JBI industry (and its followers) now are following both religion and technology - that's sort of central to the role of technology evangelists.

The problem I have had, however is that the implementations have fell short of the mark. JBI promises to be a good implementation (once 2.0, 3.0, etc) are implemented. But still just one way to skin the cat.

I have worked with JBI implementations, and have found they have missed the mark not on the technology choices, but on a very basic principal - KISS.

Several of the implementations out there (I won't name names) have become the bane of my existance.

You see as an application integrator/system integrator the issues come down to reliability, fault tollerance, ease of integration (XML or not), reusability, and yes a migration from legacy to SOA.

It's not so much that JBI misses the mark, its that the level of maturity is just not there. And the attention given to JBI as "technical" feature somehow eludes me. Its not a feature, its just underlying guts.

Real integrators need products which address our core issues: reliability, robustness, repeatability, and productivity.

Everything else, is just technical chat at lunch.

Raul said...

As far as I know, the content of a NormalizedMessage in the NMR is of type javax.xml.transform.Source, which means that it can take the form of a or a javax.xml.transform.stax.StAXSource.

Why do you say that Streaming wasn't taken into consideration? Does Mule differ in the way it represents streams?