Thursday, February 26, 2009

Gmail Fails, Cloud Computing Fails

While I don't go to the MotleyFool for my tech news, they ran an article this week about the Gmail Fail (or #gfail on Twitter). The Fool was in praise of Gmail because even though the service was down for a few hours some interfaces such as IMAP for the iPhone kept working (though this wasn't true for me). To me the fact that Gmail went down and affected people in Europe and the US was a big blow for the cloud exuberance we're currently experiencing. One of the benefits around cloud computing is being able to scale and utilise redundant machines if other machines fail. In this instance we had millions of users across continents lose email. What's worse it wasn't just the free Gmail service that failed but also their premium service. This means that the premium Google Apps service missed their SLA and Google have had to compensate (we got an email from them yesterday).

One can't help but see this as a blow for the Cloud since it shows how vulnerable we can be when we lose service. It also tells us that the one company who showed the world how to scale is not immune to severe outages. This incident didn't cost Google much, but most mission critical architectures that I have worked on would have suffered a 3 hour outage in a crippling fashion.

Does SalesForce go down all at once too? What assurances do we have in the cloud other than SLAs that can be easily broken? It is going to take a long time before companies will trust public SaaS services for anything remotely mission critical, they may never overcome that hurdle.

Wednesday, February 25, 2009

How SOA paved the way for Cloud

Though some think SOA failed and others see it only as a partial success, this has been because of the approach to SOA not its underlying principles. SOA (or what I’d rather call Service Orientation, but that doesn’t fit our obsession with TLAs) is responsible for beginning a new evolution in the way we build software. This change is as fundamental as the shift to OO programming but essentially a much bigger jump - OO was adopted one developer at a time, but SOA requires teams and departments to make the culture, technology and mind shift together. Where SOA has been most successful is evolving the way we build applications to the point where new deployment models become possible; enter the cloud.

There isn’t anything terribly new about the cloud. Virtualization (VMWare, XEN, Parallels, Hyper-V) has been around for a while, application scale-out solutions have existed in the forms of Grid computing (GigaSpaces, GemStone, Globus, GridGain, Paremus), Map Reduce (Hadoop) and distributed caching (Coherence, GigaSpaces). We could build clouds in the past, but there was a lot of work involved. Infrastructure as a Service (IaaS) was coined in 2006 and Amazon has been at the forefront of this movement. What changed recently was the ability to pull all of this stuff together in an easy way and the utility pricing model, driven by Amazon, has put all this technology in the reach of a far wider user community that couldn’t afford to play before.

In order for cloud to work, we also needed to change the way we built applications to run in the cloud. SOA has paved the way here, shifting our focus from building isolated applications that communicate using point-to-point integration to thinking of these applications in terms of services. Services are discrete pieces of logic that have a well-known contract and well understood function. This enables consumers to invoke services and execute their functions. The fundamental shift here is that you must think about how others will interact with your code. This is quite a big jump since most developers can happily develop in a vacuum and just get things working. In a service oriented application, your consumers may be your co-workers, other departments, the whole company, business partners or the whole world. The fact that other people will be using your services adds additional burden since interfaces need to be defined with consideration to hiding sensitive data, Security, versioning, data types, interface languages and a consistent API across services. SOA has been conditioning us to do this. For many organizations moving to the cloud is much easier because the heavy lifting has already been done. Applications that have been componentized into services are much better suited to the cloud, by their nature components are loosely coupled and can be hosted with location independence.

However, Moving your services to the cloud isn't necessarily a plug-and-play task. The cloud changes some of the rules for applications, I'll cover this in a follow up blog.

Tuesday, February 24, 2009

Twitter is Changing Community Dynamics


It seems the mirco-blogging platform is being used for a lot more than just posting links and banal commentary on the days events.

I am an avid user of TripIt for organizing my frequent travel schedules. TripIt works by allowing users to forward their travel arrangement emails to TripIt, which will then parse the emails and create a travel schedule for you. Genius. A couple of weeks ago I booked a flight from Malta to London, moments later I received my confirmation email and dutifully forwarded to TripIt. I know from previous attempts that TripIt does not parse emails from AirMalta, I raised this as an issue with TripIt a while back (I do between 10-20 flights out of Malta each year) and I still forward the confirmations just in case they added support in the meantime. As usual my confirmation email was rejected, so I tweeted my minor annoyance. Then something interesting happened, TripIt tweeted me back saying they’d like to fix the issue. Within the hour they had added support for AirMalta confirmations.

It occurred to me that the model of community support was changing. I had tried the normal channel support channel with TripIt but got no response, but now they had found me.

About a day later I tweeted that GMail IMAP support should expose an “Archive” folder so that clients such as the iPhone could archive mail. Within a few minutes I had a friend from Google raise the issue with the Gmail team and a couple of other responses from my, erm, Twuniverse tell me about a work around. There was definitely a pattern emerging. Rather than me having to sign up to a community and ask for help, I was being offered help.

I am usually a good community member, I join, I ask questions, I leave comments, I raise issues. However, these days I do not have the time to be actively involved with any more than two communities. With the shift of social media and the plethora of communities out there it has become impossible to be part of the 10 percent that actually contribute in some way to more than a couple of communities. The Twitter community model says that I don’t need to join a community to get assistance; I can write a 140 character cry for help and post it to Twitter. Forward thinking services such as TripIt will be trawling the sea of Tweets and can reach out to me. I’d call it the ‘message in a bottle’ community model.

The question is whether this model will scale if Twitter goes mainstream. Given the possible influx of requests it may become difficult to service those requests. However, large companies are having success with this approach. ComCast employee Famous Frank aka @comcastcares has demonstrated that it is possible to support a large user base one tweet at a time (there is a good article about ComCast in Wired this month). Famous Frank set out to help change the tarnished image of ComCast support and now has a team of people offering support via Twitter.

One reason communities are the way they are is so that users can talk with other users and self-serve by trawling the archives. The Twitter model doesn’t make it easy to view a useful history of the resolution. Also, with the restriction of 140 characters it will sometimes be necessary to switch to email or phone to resolve an issue. Finally, offering support like this means that a support issue is not created and tracked, which may be problem for some companies.

It seems there is an opportunity here to integrate Twitter into more traditional forms of support and community. Twitter can still be used as the communication channel, but the conversation can be logged and tracked in other tools to provide a ticket and searchable history. Surely this a job for Atlassian?

You can follow me on Twitter @rossmason

Monday, February 23, 2009

A New Chapter for MuleSource

It’s been an exciting couple of months for MuleSource. We began the year with 2 blowout quarters behind us. Last week we announced that Mark Burton, the former executive vice president of Sales from MySQL has joined the MuleSource board of directors. Mark brings a wealth of sales operating experience from one of the most successful open source companies to date. Mark has an excellent understanding of the different sales dynamics an open source company faces and has an uncanny ability to communicate his ideas on sales operations and strategy that makes you wonder why you never thought of them yourself.


The big news for MuleSource this week is that Greg Schott has joined us as CEO. Greg comes from SpringSource where was senior vice president of Marketing. He’s no stranger to open source and has a extraordinary understanding of open source business models and strategy. Greg also has a proven operation track record where he served as vice president of corporate development at Agile Software (acquired by Oracle). And served as vice president of marketing and vice president of operations at DG Systems. It's been a pleasure getting to know Greg and truly believe he is the right person to lead the team to build on the success of Mulesource. I am really looking forward to working with him.

Here is to the exciting next chapter of MuleSource, thank you for reading.

Friday, February 20, 2009

Pragmatic SOA Governance

I recently wrote an article that appeared in eBizQ on the topic of SOA governance. In this article, I argue that the primary reason that many pundits have declared “SOA is dead” is that the traditional “top-down” approach to SOA and governance have failed. Vendors have for too long evangelized a “big bang” re-architecture of development processes, using their tools to enforce new behaviors from developers, in order to realize the benefits of SOA. These approaches have failed because they make the average developer’s life more difficult, rather than simpler. Development teams, already stretched for time and resources, are generally reluctant to do extra work in the pursuit of some abstract notion of “service reuse.” Instead, modern governance approaches that are getting traction today actually help developers by integrating with existing tools and streamlining, rather than obstructing, familiar processes.

Read the entire article here.

What I didn’t talk about in the article was some of the real-world activity I have seen from MuleSource customers. The way our customers use Mule Galaxy - our SOA governance platform and registry - validates this new pragmatic approach to governance. For example, one user has incorporated Galaxy into their software development lifecycle, integrating their IDE environment using Galaxy’s REST API, making it easy for developers to find services. This user also stores services in the Galaxy repository, allowing them to enforce policies around their services, including WS-I BasicProfile and a number of security policies. Another user has service metadata defined within Java classes via custom annotations – using Galaxy and a couple of lines of Groovy code, they are able to automatically extract this metadata, publishing it in Galaxy for others to find and reuse. Finally, a third user pushes builds from their continuous integration system into Galaxy using the Apache Ant build tool, then using Galaxy to track the lifecycle information of particular application builds and services. Once a new service was pushed into staging or production, Galaxy’s application deployment feature (Mule NetBoot) could detect the change and automatically download and start that service.

In today’s world, it’s clear that the traditional top-down philosophy for SOA is outmoded and outdated – a new approach is needed for today’s organizations to see real value. The implications on SOA governance are clear – the focus should be on making things easier for everyone*, not about virtuous architecture; on improving existing organizational structures and processes rather than wholesale re-engineering; on implementing pragmatic tools at the work-group level (e.g., using open source tools) rather than embarking on massive enterprise-wide software implementations. Whether you are calling it SOA, composite applications, mash-ups, or any other buzzword of the day, only when the everyday IT professional sees the benefits of a new approach to development will the enterprise see value overall.

To see a live demonstration of Mule Galaxy, view this 7 minute screencast.

* Historically the role of the developer as a user of an SOA registry has always been overlooked. Martin Fowler wrote an interesting post on Humane Registries that touches on ways in which the right information can be tracked from a developers perspective to make the tracking of services more useful to developers (yes, Mule Galaxy supports tracking developer metadata outlined by Martin).

Monday, February 9, 2009

Android Predictions


Android is getting momentum with more phone vendors gearing up to release Android phones this year. While I don’t think Android is at par with the iPhone yet, it is a much more compelling platform for innovation. The openness of Android is a blessing and a curse. The community around Android will flourish since developers and startups will be more comfortable investing in an open platform. Open also means that the platform is subject to abuse and without regulation will suffer from low quality software. These factors and more will foster a new community of mobile vendors. Here are a few predictions-

Distributions


We’ll see a host of Android distributions (a la RedHat) crop up in the next 1-2 years. These distributions will provide a supported software suite with consistent behaviour and look and feel. As the Android Market gets flooded with applications it will be difficult for users to select the best applications. Having a consistent distribution managed and supported by a single vendor will be compelling for the mass market.

Certified Markets


Along the same lines as distributions, there is need to have a third-party assess applications for their stability, quality, behaviour and user interface. While the community can do much of this, we’ll see new Android markets that offer a certification process. There is always money in certification.

Enterprise Offerings


A common criticism of the iPhone and the reason that the Blackberry has done so well is that the Enterprise needs secure, robust messaging and support. This means integrating with Microsoft Exchange and Zimbra, but also providing secure groupware tools for communication and collaboration. I suspect there are folks already working no this. If you are, I’d be interested in hearing about it.

Entertainment


One thing that bugs me about Android platform is that it has the potential to be as good or better than the iPhone for entertainment. It’s going to require co-ordination with the phone manufacturers to enable Android to compete in this area. There are already applications cropping up to fill the lack of video support on the G1, but what is more important to me is that the phone manufacturers add simple features such a headphone socket. One killer feature of the iPod was the dock, since it grows an ecosystem of hardware vendors too. I hope the next generation of Android phones look to replicate this.

Device Diversity


The focus on Android is primarily on phone devices but it is clear that the operating system could be suitable for other types of devices such as tablets, netbooks and internet/media devices. In fact vendors have started hinting at more Android-powered devices such as the Archos Internet tablet.

Friday, February 6, 2009

Cloud: Defining the Undefinable

Given that the definition of cloud is still somewhat fuzzy and the terminology is already spiraling (yes we have Private clouds, federated clouds and heavenly cloud cakes), I decided (for my own understanding), to map out where IaaS, PaaS and SaaS sit. The bit I was struggling with was that PaaS has multiple layers. Big thanks to Alexis for helping to organize my thoughts on this.



The cloud itself is just a category that groups more concrete concepts together.

The next step would be to drilling down into these layers and defining what services are required for each one.

Making The Same Mistakes With Cloud

I’m watching the rise of the cloud with some disappointment. It was inevitable but still I hoped that we would take a more pragmatic approach to defining the cloud. The term cloud is being bounded around by almost all vendors, analysts and many technologists, but following the forums and blogs it’s clear we have very different views what the cloud means. I do find it amazing that so many companies are anchoring themselves to a concept so indefinable. This reminds me of how we all jumped on the SOA bandwagon earlier in the decade and I fear we’re going to make the same mistakes.

Like SOA, the concept of “cloud” is built upon a set of interesting ideas and principles that make a lot of sense. The underpinnings of SOA state that we should architect systems in terms of services to enable reuse, promote decoupling and enable systems to be agile enough to respond to the ever-changing needs of the business. At its simplest form, the cloud focuses on the way we deploy software and handle the need of on-demand scalability.

As we have seen with the SOA is Dead rejoicing, good principles can be vastly overshadowed by hype, leaving early adopters disillusioned. I blame this hype pattern on the vendors making wild promises about their technology (there is no silver bullet. Ever.) And to some degree the analysts who need to look into the future and set benchmarks that vendors feel compelled to reach. This doesn’t mean all vendors and analysts are evil, just that they help create market forces bigger than themselves, which vendors cannot resist in a competitive climate (case in point, listen to Larry Ellison: What the Hell is Cloud Computing?).

I am not in the “fuck the cloud” camp, but I do think they naysayers raise relevant arguments that cannot be ignored. We’re going to see an absolute mass of cloud “offerings” of all types of in the coming year with every vendor and his fluffy friend jumping on the hype wagon. Before talking about technical solutions, security and vendors I would urge consumers to consider the following questions:


  1. What are your expectations of cloud computing?

  2. What the key drivers are for considering a cloud solution in your organisation?

  3. What is the business value you hope to achieve and how will you measure it?

  4. What is the total cost? (Cloud is usually not cheaper, but sometimes easier to build out scalable infrastructure)

  5. How will a shift to the cloud affect your users, operational staff and developers?

  6. How will you manage the change so those affected feel empowered and see real value? (Change is always resisted, but will be accepted if value is demonstrated)

  7. How will it affect your existing infrastructure? How will you partition and manage it?


I am a huge advocate of taking a pragmatic approach to technology adoption. To me this means starting small and building on a proven track record. It’s good to have a long-term strategy, but break that strategy up into small manageable units of work that build on top of each other to enforce the value of the strategy. In the current economic climate we are already seeing customers doing this naturally. Big Bang projects are so 1999.