On Web 2.0 Mashups

One would think that sites like Weather.com and friends would be the first choice for implementing such a feature. Only thing is that they were strongly against being mashed-up Web 2.0 style on the client – they had enough scalability problems of their own. Interestingly enough (or not), these partners were quite happy to publish their weather data to us and let us handle the whole scalability issue.

Implementation 1.0

The current implementation was fairly straightforward – client issues a regular web service request to the GetWeather webmethod, the server uses the user’s IP address to find out their location, then use that location to find the weather for that location in the database, and return that to the user. Standard fare for most dynamic data and the way most everybody would tell you to do it.

Only thing is that it scales like a dog.

Add Some Caching

The first thing you do when you have scalability problems and the database is the bottleneck is to cache, well, that’s what everybody says (same everybody as above).

The thing is that holding all the weather of the entire globe in memory, well, takes a lot of memory. More than is reasonable. In which case, there’s a fairly decent chance that a given request can’t be served from the cache, resulting in a query to the database, an update to the cache, which bumps out something else, in short, not a very good hit rate.

Not much bang for the buck.

If you have a single datacenter, having a caching tier that stores this data is possible, but costly. If you want a highly available, business continuity supportable, multi-datacenter infrastructure, the costs add up quite a bit quicker – to the point of not being cost effective (”You need HOW much money for weather?! We’ve got dozens more features like that in the pipe!”)

What we can do is to tell the client we’re responding to that they can cache the result, but that isn’t close to being enough for us to scale.

Look at the Data, Leverage the Internet

When you find yourself in this sort of situation, there’s really only one thing to do:

Leverage the Internet

The internet is the biggest, most scalable data serving infrastructure that mankind was lucky enough to have happen to it. However, in order to leverage it – you need to understand your data and how your users use it, and finally align yourself with the way the internet works.

Let’s say we have 1,000 users in London. All of them are going to have the same weather. If all these users come to our site in the period of a few hours and ask for the weather, they all are going to get the exact same data. The thing is that the response semantics of the GetWeather webmethod must prevent intermediaries from caching so that users in Dublin and Glasgow don’t get London weather (although at times I bet they’d like to).

REST Helps You Leverage the Internet

Rather than thinking of getting the weather as an operation/webmethod, we can represent the various locations weather data as explicit web resources, each with its own URI. Thus, the weather in London would be http://weather.myclient.com/UK/London.

If we were able to make our clients in London perform an HTTP GET on http://weather.myclient.com/UK/London then we could return headers in the HTTP response telling the intermediaries that they can cache the response for an hour, or however long we want.

That way, after the first user in London gets the weather from our servers, all the other 999 users will be getting the same data served to them from one of the intermediaries. Instead of getting hammered by millions of requests a day, the internet would shoulder easily 90% of that load making it much easier to scale. Thanks Al.

This isn’t a “cheap trick”. While being straight forward for something like weather, understanding the nature of your data and intelligently mapping that to a URI space is critical to building a scalable system, and reaping the benefits of REST.

What’s left?

The only thing that’s left is to get the client to know which URI to call. A simple matter, really.

When the user logs in, we perform the IP to location lookup and then write a cookie to the client with their location (UK/London). That cookie then stays with the user saving us from having to perform that IP to location lookup all the time. On subsequent logins, if the cookie is already there, we don’t do the lookup.

BTW, we also show the user “you’re in London, aren’t you?” with the link allowing the user to change their location, which we then update the cookie with and change the URI we get the weather from.

In Closing

While web services are great for getting a system up and running quickly and interoperably, scalability often suffers. Not so much as to be in your face, but after you’ve gone quite a ways and invested a fair amount of development in it, you find it standing between you and the scalability you seek.

Moving to REST is not about turning on the “make it restful” switch in your technology stack (ASP.NET MVC and WCF, I’m talking to you). Just like with databases there is no “make it go fast” switch – you really do need to understand your data, the various users access patterns, and the volatility of the data so that you can map it to the “right” resources and URIs.

If you do walk the RESTful path, you’ll find that the scalability that was once so distant is now within your grasp.

The question of how web-based (or 3rd party) consumers can work with pub/sub based services comes up a lot.

Many developers are used to implementing web services exposing methods on them like GetAllCustomers.

When moving to pub/sub and other more loosely coupled messaging patterns, developers look to implement the same pattern, opting for something like duplex GetCustomersRequest and GetCustomersResponse. The reasoning is simple and straightforward – it is difficult to push data over the web to consumers.

However, there are still ways to disconnect the preparation of the data from its usage thus gaining many of the advantages of pub/sub.

By employing REST principles and modelling our customer list as an explicit resource, web-based consumers would simply perform regular HTTP GET operations on the URI to get the list of customers.

The resource itself could be a simple XML file – it wouldn’t need to be dynamic at all.

You can get all the scalability benefits of pub/sub for web based consumers. All you need is a bit of REST

NServiceBus Tutorial

I gave a full day tutorial on nServiceBus and we had a full house! The tutorial was about 90% how to think about distributed systems, and 10% mapping those concepts onto nServiceBus. I made an effort to cram about 3 days of a 5 day training course I give clients into one day, but I think I was only about 85% successful. People didn’t have the time needed to let things really sink in and ask questions, but the lively forums and skype conversations available will probably do the trick.

Jim Webber after looking at the unit testing features of nServiceBus had this to say:

“Oh my God – you’ve created testable middleware! It’ll never catch on. The vendors won’t have it.”

To which I replied that several vendors were already coming on board with their own implementations of transports and saga persistence. I have absolutely no intention, desire, or (quite frankly) the ability to write an enterprise-class middleware runtime. All I hope to do with nServiceBus is to make it so that developers use what’s out there in one, middleware-product-agnostic way that will make their code more robust and flexible.

MEST & Mark – REST & Stefan

It was also great finally meeting the head MESTian, Mark Little, who also happens to work for Redhat as SOA Technical Development Manager and Director of Standards in the JBoss division. It was interesting to see the difference between how I went about messaging in nServiceBus (full peer-to-peer including pub/sub) whereas most of the Java world has the messaging infrastructure handled by something database-like in a deployment/networking kind of perspective. If that’s the way things are done, then I can definitely appreciate the advantages of Space-Based Architectures.

And I even got to steal Stefan Tilkov’s RESTful ear for an hour or so before I had to jet back home. It looks like we MESTians and RESTians can be one big happy family. I’m guessing that our despise of WS connects us all at a deeper level

Core Design Principles

I also gave a talk about core design principles, “Intentions & Interfaces – making patterns concrete”, and it went over very well especially considering that that was the first time that I gave that talk. You can find the slides here. From the feedback I heard after the talk, I think many people were surprised how many different parts of a system can be designed this way, and how flexible it is without making the code any more complex. The message was this:

Make Roles Explicit

Despite its simplicity, that leads to IEntity, IValidator<T> where T : IEntity, (which I wrote about a year ago – generic validation) and with a bit of Service Locator capabilities, you can add a line of code to your infrastructure that will validate all entities before they’re sent from the client to the server.

It leads to IFetchingStrategy<T> for improved database loading performance (also a year old – better DDD implementation and the NHibernate implementation).

It’s also how nServiceBus does message handling – IMessage, IMessageHandler<T> where T: IMessage, ISaga<T> where T : IMessage.

San Francisco?

Just a quick shout to my readers in the San Francisco area, if you’d be interested in hearing these talks/tutorials, give the organizers of QCon a shout and they’ll bring me out. That’s actually what got me to London – one of the attendees of a talk I gave at Oredev in Sweden last November missed my tutorial there so he put in a request and that did it. (Thanks Jan, I appreciate it!)

If you’re in a different part of the world and you’d like to have me give one of these talks, or other ones (I have a fair amount of material on Domain Models/DDD and Occasionally Connected Smart Clients), I’d be happy to make the trip and see you there as well.

Download

Download via the Dr. Dobb’s site

Or download directly here.

Additional References

• Podcast: “REST vs Web Services Debate”
• REST Definition
• Roy Fielding’s Thesis: “Architectural Styles and the Design ofNetwork-based Software Architectures”
• ETag Definition

Want more?

Check out the “Ask Udi” archives.

Got a question?

Send Udi your question to answer on the show.

Q: Could you tell you your thoughts or preference for a distributed or centralized ESB?

DON: there is no such thing as a centralized ESB.

This is the problem with a lot of the products that call themselves ESBs. They are centralized brokers which may be clustered for availability. But they are in no way an implementation of the Bus Architectural Pattern. Please check this before cutting a check to your vendor.

Also, understand that if you do security related things in your ESB, possibly as a part of your routing rules, that if the security infrastructure is centralized that means your ESB is too. Even if it really was distributed to begin with.

Buyer beware.

DevHawk (aka Harry Pierson) writes:

As you might guess then, I’m not a fan of Astoria. I believe the sweet spot for so called “data services” will be read only (because they don’t need transactions, natch). I’m sure there are some read/write scenarios Astoria will be useful for, but I think they will be limited – at least within the enterprise.

After catching up to Pablo Castro (the man behind Astoria) at DevTeach, and chatting with him and Tim Mall about Astoria, I heard something interesting. In order to provide business context for updates, you will be able to use something like a WS-Action header.

So, it’s based on REST, but for updates, it’s like WS-*. Hmm… Best of both worlds, or worst? What do you think?

If we were to let clients handle transaction management, we would end up in a situation where database server resources would be tied up by clients over slow connections, or transactions timing out when those clients had some connectivity issues or just crashed themselves. It will be practically impossible to build a scalable system this way.

The problem is that we’re left with no way to tie multiple reads, inserts, updates, and deletes together. Once again, there will be cases where clients will be unable to complete their unit of work because of failures. The unfortunate result is that our database may get into an inconsistent state.

Luckily, I ran into a post on the MSDN blogs showing that it is possible to fix these conflicts. It just requires that you write your most complex logic in SQL, a language designed for set manipulation. Apparently, even though the logic runs in the database, it doesn’t perform that well either. I hope you’re either really good at this kind of coding, don’t need to scale, or can have an inconsistent database if you’re planning on using Astoria for any kind of transactional work.

This is not to say that using Astoria for REST style GET actions is bad. On the contrary, I love the scalability of GET, well, over HTTP anyway. I just don’t think that we need a new technology for it – HTTP GET works just fine, both as an API and an imp

> A while ago I tried to explain the difference between SOA and REST is
> that SOA is based on “MEST” – MESsage Transfer. The main thing is the
> message – which is a statement of intent, and includes the relevant
> data for that intent. For instance, a message like
> ChangeCustomerAddressMessage is itself the intent; the data it
> carries, the customer ID and the new address completes the picture.

Ahh. This puts your name in context for me. I did some reading on MEST some time ago. I think that there are probably a few misunderstandings on both sides of the fence still about what the other is all about. I have recently been reading “Software Factories”, ISBN 0-471-20284-3. It takes the view of a service being a software component. Code usable in different software environments that has enough self-description to be accessed from multiple languages and runtimes. This sets up services as platform-independent objects.

I think I understand this evolution and the corresponding object patters that are encoded into WS standards. Essentially, SOA attempts to make objects more accessible. REST is quite a different beast. Its primary function is to allow communication to evolve.

> The main difference between this style and REST is that where REST is
> focusing on one resource, SOA/MEST style accept that fact that as a
> result in the change of address, the customer may now be in a “rich
> neighbourhood”, and his customer rep now has over X people in rich
> neighborhoods, so the customer rep’s status changes. This cascading of
> changes I believe is done in orchestration/choreography type code outside of the resource in REST.
> Such complex business logic is best implemented using OO models which
> span objects/entities/resources.

I think this is a common misunderstanding. Each resource demarcates a subset of an application’s state. Resources are likely to overlap. When one resource is updated, it is likely that other resources will also change. This is not well ennunciated by REST proponents who are far more focused on the other side of the exchange, that between client and server. In REST, your ChangeCustomerAddressMessage sent to object A would be a PUT of a customer address record over the top of the old one, at resource http://example.com/customer1.

I see REST as a tweak to the object model. Instead of having a base-class that defines methods and data to be transferred, REST decouples the definition of methods and data. The intention is that the set of methods, the set of data definitions, and the set of objects
(resources) can evolve independently of each other. This tweak does not change existing capabilities of the object model. Objects work pretty-much as before, but their interface to clients and the definition of their interface is modified.

The effect of the decoupling is that methods can be understood separately to data and to objects. For example, I can determine what a GET or PUT request is going to do to a resource regardless of the object the method is applied to. As an intermediatary I can operate as a caching proxy, or apply security policy appropriately. I see this like the jump to java beans. Some programmers were downright offended that they had to name their methods with “get” and “set” in order to get the architecture working, however the advantages are now so clear that the issue simply doesn’t come up anymore.

REST is about building a communicatons infrastructure that scales to a large number of participants who want different things from their architecture over time. At a technical level there are significant performance advantages to be gained by caching and other features, but at a social level it makes the definition of new interfaces simpler and allows them to change over time without disrupting the architecture as a whole. New methods can be introduced over time. New content types can be introduced over time. Old features can be supported simultaneously to new.

> It is this model of message/document based communication that isn’t
> very well supported by WSDL. Support for publish/subscribe semantics
> is also vary light in the WS world. This is the main reason why I see
> SOA as something outside of WS. I don’t see this as a “nod” to REST
> but a coherent architectural style that been employed long before the web.

Support for pub/sub is also pretty light in the REST world, something I have mind to do something about

> As for scalability, it’s very easy to scale the publishing of static
> html pages, it is an entire other thing to scale complex business
> logic that may have huge working sets of data. To tell you the truth,
> I’d be interested in seeing the design of such a system in terms of
> REST, specifically in how transactions are handled.

Transactions in REST are usually handled by demarcating all of the state that will be updated in a single resource, and PUTting a representation of that state to the resource. REST is usually applied at a chunky enough level that this makes sense. There is always the expectation in REST that objects or RDBMS or some other back-end technology does the work of making the updates.

There are other ways to handle transactions in REST, but they usually don’t come up. Again, it is a matter of scale and the level of abstraction. REST is a client-facing communications model. It isn’t necessarily useful for communication between services within the same tier. It may also be that the tier that presents a REST world view hides further tiers that use distributed object technologies.

> Anyway, that’s where I’m coming from. I think that the communications
> semantics of SOA – send a message, are just about as simple as it
> gets. It is, if anything, more minimalistic than REST.

I think that you are right about minimalism, at least after a fashion.
With WS-* or even corba, it is easy enough to make two programs work like one in an enclosed environment. REST is really about a larger-scale communications system where not all of the participants are part of the same organisation. It is about forging agreement on the larger scale, not an easy thing to do. However, I think that REST scales down better than SOA scales up. It is no harder to define special-purpose XML formats than it is to define a base-class, although defining all appropriate resources is probably starting to make life too difficult.

My organisation integrates disparate systems. It is about 50% of the work we do on a job. In this environment REST is a valuable tool in making things work with each other over long periods of time, and exposing the data of one system to another for viewing and update.

Orignal message:
> I’m quite interested in expanding my knowledge of SOA best practices.
> As I noted in the entry you refer to that my list is derived from a
> presentation by Sun Microsystems, the slides of which may be found > href=”http://au.sun.com/events/developer/downloads/download.html?s_dLi
> nk=SOA _JBI_BPEL.pdf”>here. I often hear that SOA is an
> architectural style rather than a name for the WS-* stack, however it
> is hard to come to terms with what those principles are given the
> variety of views on the subject.
>
> It seems that many SOA proponents who are familiar with REST take a
> number of their particular constraints from REST itself. If those
> individuals are to be taken as the litmus test, SOA is generally a
> superset of REST that allows for less uniformity. In other words it
> looks like a version of REST that doesn’t scale up to such big network
> sizes. Vendors seem to see SOA differently, the message-bus focus I
> referred to in my entry.
>
> I would also like to note that the main focus of the title was on the
> principle difference that I percieve there to be between SOA and REST:
> the nature of the uniform interface. Even if you take your web
> services and try to construct a uniform interface, you do so as a
> base-class that encapsulates both verbs and content types. REST
> decouples these. I think the same is possible as document-oriented
> processing under web servcies, however this again seems to be a nod to
> REST rather than an alternative to REST practice.
>
> Do you have a specific list of constrains you yourself like to point
> to as the meaning of SOA?

The goal of Microsoft Codename Astoria is to enable applications to expose data as a data service that can be consumed by web clients within a corporate network and across the internet. The data service is reachable over HTTP, and URIs are used to identify the various pieces of information available through the service. Interactions with the data service happens in terms of HTTP verbs such as GET, POST, PUT and DELETE, and the data exchanged in those interactions is represented in simple formats such as XML and JSON.

To me, this sounds like an unarchitected mashup of REST and the irrelevant part of SDO.

Please, Microsoft, just stop. You’re going the wrong way. We really don’t need yet another data access strategy from you.

Patrick Logan is still waiting to see anything concrete come out before weighing in. While Sam Gentile has given it the “New and Notable” stamp. Fabrice Marguerie will be investigating as well. And Paul Gielens also finds the REST path worthwhile.

But I’ve got to say, I’ve been against these “data services” from day one. The REST style is most applicable for large, chunky resources – while this seems to be targeting single tables in the database. Look at this discussion on REST vs SOA for some examples.

I am skeptical but will keep watching. But, what with all these fine bloggers giving me the low-down, I’m sure we’ll be finding out the important part soon – you know, the part Microsoft doesn’t put online

Resources:

Wikipedia definition of REST
Ask Udi a question on SOA
Subscribe to the Ask Udi podcast

Get it via the Dr. Dobb’s site here.

Or download directly here.

Want more? Go to the “Ask Udi” archives.

The tenets of Service Orientation as put forth by Microsoft include one about “autonomy”. The tenet states that “Services should be autonomous”. After some digging, I found out that the intent of the authors was “The teams that develop different services should not be dependent on each other”, or in shortened form “Autonomous Teams”. This revelation was surprising to me since the real meaning of the tenet was less profound than what I had imagined – autonomous computing.

The idea of autonomous computing has been around for some time and presents a view of the world in which computing units cooperate to achieve global goals yet are not dependent even on the existence of other computing units to function. (If you’re envisioning tiny robots playing soccer, you’re not far off.)

So when I first saw the autonomy tenet, I was thinking of autonomous services: services so loosely coupled that the correct functioning of a service would not be dependent on the correct functioning of other cooperating services. Services loosely coupled in time as well as in code. Obviously this would mean that if Service A needed to cooperate with Service B, and Service B was not even available, Service A would continue to function, and live up to its service-level-agreement. But before we start drifting off into the outer reaches of business-IT alignment, let’s bring this down to earth.

Before we get into a detailed analysis of the how, let’s first agree on the why. Despite being a historical trend, architectures these days tend to be more loosely coupled than before. Loose coupling being a good thing that enables us to better manage the complexity inherent in large software projects. The practical test of loose coupling in a system is changing the public interface of a class and seeing how much of the system doesn’t compile any more (dynamic languages aside). Service Orientation brings us tenets that, when followed, lead to more loosely coupled architectures than if we actively did not follow them. I think that we can agree then that if we could somehow achieve the loose coupling in time mentioned above, without paying an arm and a leg, that would move our architectures another step forward.

Looking back on the evolution of the field of distributed computing, we can see that, over time, less and less things are being assumed. It is now well understood that anything that goes over the network takes much longer than those calls that stay on the same machine, yet once systems were built that abstracted the network communication into looking just like local calls. The performance of those systems was matched only by their lifetime. With the advent of autonomous computing, the assumption that the called service is available and will respond in a timely fashion is called into question. In the real world, servers crash and network equipment goes up in smoke – we can no longer take for granted that communication will always be available, and that its quality will be good enough. In essence, this marks the end of synchronous RPC/RMI. The following code just won’t cut it in this brave new world:

localhost.service1 s1 = new localhost.service1();
orderReply = s1.HandleOrder(orderRequest);

If the service is unavailable, what will happen to our order request? Will it just get lost?
If the service takes a long time to respond, will our server tie up resources for the same amount of time? If this happens under peak load, might it cause our server to crash?

Performing the above code on a different thread won’t make any difference, autonomous services means the end of Request/Response as we know it.

“No Request/Response between services?!”, you ask incredulously.

The simple answer is “yes”, but there is another level of meaning to it. If you have two software entities that between them you just HAVE to have request/response communication, then they should be in the same service. This is where the real architectural guidance comes in.

In component-orientation and object-orientation, the division of the solution into the right number of parts, with each part having the right amount of responsibility was a kind of black magic passed from master to apprentice. Getting the boundaries right was paramount, but difficult. A number of litmus tests are used to catch the gross errors, and the rest is just gut. So too, the request/response test helps us catch gross errors in service boundary demarcation.

The interesting thing that happens after separating our services out this way is that we often end up with services that mirror the way the business side is structured. Voila, business-IT alignment with your hands closed and one eye tied behind your back! Well, it’s one step in the right direction anyway.

This leaves us with the original types of one-way communication (fire-and-forget, pub-sub, etc) and with one kind of two-way communication: duplex. Duplex is really just two one-way communications (A to B, then B to A) that are correlated. First, I send a message to you, mark it with an id number, and save that number. At some future point in time, you get the message, process it, and send a message back with its own id number. But, you’ll have to put my original id number on the message too, so that I’ll know that your message is a response to mine. At some even more distant point in the future, I get a message from you, look at it, and see that it is the long-awaited response to the request I sent way back when.

If I had to sum up the difference autonomous services bring to the styles of communication used between services, I’d say this: You get a message, look at it, and figure out what it means and what you should do. This isn’t an infrastructure issue. There application level timeouts to deal with (If I don’t get a response back in 3 days, then notify the supervisor), and long-running workflows to manage (next whitepaper ).

If there is one thing to pay attention to in this whole “autonomous services paradigm” it is that the focus has shifted from between services to within a single service. In parting, I want to let you know that systems can be, and are being, built this way. It works. It better than works. Systems created this way are more robust to failures (seeing as they’re designed for failures makes it less impressive) and easier to manage. Give it a try. You didn’t really think that SOA would fizzle away into a bunch of WS specs, did you?