Udi Dahan   Udi Dahan – The Software Simplist
Enterprise Development Expert & SOA Specialist
 
  
    Blog Consulting Training Articles Speaking About
  

Data Duplication and Replication

Tuesday, August 28th, 2012.

agent_smith_replicationOccasionally I’ll get questions from people who have been going down the CQRS path about why I’m so against data duplication. Aren’t the performance benefits of a denormalized view model justified, they ask. This is even more pronounced in geographically distributed systems where the “round-trip” may involve going outside your datacenter over a relatively slow link to another site.

CQRS

As his been said several times before by many others, it’s not the denormalized view model that defines CQRS.

One of the things that sometimes surprising people after going through my course is that in most cases you don’t need a denormalized view model, or at least, not the kind you think. Yes, that’s right: MOST cases.

But I don’t want to get too deep into the CQRS thing in this post – that can wait.

SOA

The big thing I’m against is raw business data being duplicated between services.

Data that can be expected to be accessible in multiple services includes things like identifiers, status information, and date-times. These date-times are used to anchor the status changes in time so that our system will behave correctly even if data/messages are processed out of order. Not all status information necessarily needs to be anchored in time explicitly – sometimes this can be implicit to the context of a given flow through the system.

For example, the Amazon.com checkout workflow.

In that flow, if you provide a shipping address that is in the US, you are presented with one set of options for shipping speed, whereas an international address will lead you to a different set of options.

Assuming that the address information of the customer and the shipping speed options are in different services, we need to propagate the status InternationalAddress(true/false) between these services in that same flow. In this case, there isn’t a need to explicitly anchor that status in time.

But what’s so bad about duplication of data between services?

The danger is that functionality ultimately follows raw business data.

You start with something small like having product prices in the catalog service, the order service, and the invoice service. Then, when you get requirements around supporting multiple currencies, you now need to implement that logic in multiple places, or create a shared library that all the services depend on.

These dependencies creep up on you slowly, tying your shoelaces together, gradually slowing down the pace of development, undermining the stability of your codebase where changes to one part of the system break other parts. It’s a slow death by a thousand cuts, and as a result nobody is exactly sure what big decision we made that caused everything to go so bad.

That’s the thing, it wasn’t viewed as a “big decision” but rather as just one “pragmatic choice” for that specific case. The first one excuses the second, which paves the way for third, and from that point on, it’s a “pattern” – how we do things around here; the proverbial slippery slope.

So what’s with the word “Replication” in the title of this post?

While data duplication between services is very dangerous, replication of business data WITHIN a service is perfectly alright.

Let’s get back into multi-site scenarios, like a retail chain that has a headquarters (HQ) and many stores. Prices are pushed out from the HQ and orders are pushed back from the stores according to some schedule.

We know that we can’t guarantee a perfect connection between all stores and the HQ at all times, therefore we copy the prices published from the HQ and store them locally in the store. Also, since we want to perform top-level analytics on the orders made at the various stores, that would be best done by having all of those orders copied locally at the HQ as well.

We should not view this movement of data from one physical location to another as duplication, but rather as replication done for performance reasons. If there were some magical always-on zero-latency network that existed, we wouldn’t need to do any of this replication.

And that’s just the thing – logical boundaries should not be impacted by these types of physical infrastructure choices (generally speaking). Since services are aligned with logical boundaries, we should expect to see them cross physical boundaries – this includes SYSTEM boundaries (since a system is really nothing more than a unit of deployment).

I know that you might be reading that and thinking “What!?” but there isn’t enough time to get into this in any more depth here. You can read some of my previous posts on the topic of SOA for more info here.

Cross-site integration without replication

There are some domains where sensitive data cannot be allowed to “rest” just anywhere. Let’s look at a healthcare environment where we’re integrating data from multiple hospitals and care providers. While all of these partners are interested in working together to make sure that patients get the best care, which means that they need to share their data with each other, they don’t want any of THEIR data to remain at any partner sites afterwards (and are quite adamant about this).

In these cases, the decision was made that performance is less important than data ownership. Personally, I don’t agree with this mindset. The fact that data is “at rest” in a location as opposed to “in flight” does not change ownership. It could be stored in an encrypted manner so that only a certain application could use it, resulting in the same overall effect, but this is an argument that I’ve never won.

People (as physical beings) put a great deal of emphasis on the physical locations of things. It’s understandable but quite counterproductive when dealing with the more abstract domain of software.

In closing

By virtue of the fact that we don’t duplicate raw business data between services, that means that the regular data structures inside a service already look very different from what they would have looked like in a traditional layered architecture with an ORM-persisted entity model.

In fact, you probably wouldn’t see very many relationships between entities at all.

Going beyond that, you probably wouldn’t see the same entities you had before. An Order wouldn’t exist the way you expect; addresses (billing and shipping) would be stored (indexed by OrderID) in one service whereas the shipping speed (also indexed by OrderId) would be in another, and the prices may well be in yet another.

It is in this manner that data does not end up being duplicated between services, but rather is composed by many services whether that is in the UI of one system, the print-outs down by a second system, or in the integration with 3rd parties done by a third system.

If performance needs to be improved, look at having these services replicate their data from one physical system to another – in-memory caching is one way of doing this, denormalized view models might be though of as another (until you realize there isn’t very much normalization within a service to begin with).

And a word from our sponsor 🙂

For those of you on “rewrite that big-ball-of-mud” projects looking to use these principles, I strongly suggest coming on one of my courses. The next one is in San Francisco and I’ve just opened up the registration for Miami.

For those of you on the other side of the Atlantic, the next courses will be in Stockholm in October and in London this December.

The schedule for next year is also coming together and it will include South Africa and Australia too.

Anyway, here’s what one attendee had to say after taking the course earlier this month:

I wanted to thank you for the excellent workshop in Toronto last week. I spent the better part of the weekend reflecting over what was presented, the insights we learned through the group exercises, and how my preconceptions of SOA have changed. By the end of the course, all the tidbits of (usually) rather ambiguous information that I’ve collected from various blogs, books, and other sources, finally coalesced into something more intelligible – one big A-HA moment if you will. Overall, I found the content of the workshop to be incredibly enlightening and it left me feeling invigorated and excited to learn more.
– Joel from Canada

Hope you’ll be able to make it.

If travel is out of the question for you, you can also look at get a recording of the course here.

One final thing

If your employer won’t foot the bill for these, please get in touch with me.
I wouldn’t want you not to be able to come just because you’re paying out of pocket.

There are very substantial discounts available.

Contact me.

  
If you liked this article, you might also like articles in these categories:


Check out my Best Content If you've got a minute, you might enjoy taking a look at some of my best articles.
I've gone through the hundreds of articles I've written over the past 6 years and put together a list of the best ones as ranked by my 5000+ readers.
You won't be disappointed.

Subscribe to my feedIf you'd like to get new articles sent to you when they're published, it's easy and free.
Subscribe right here.

Follow me on Twitter Follow me on Twitter @UdiDahan.



Something on your mind? Got a question? I'd be thrilled to hear it.
Leave a comment below or email me, whatever works for you.

32 Comments

  1. Steve Sheldon Says:

    Once an Order has been placed, the cost of an item is fixed in time and space.

    How do you manage that if I keep going back to this product service to get prices?


  2. ĂĽgyviteli szoftver Says:

    Steve: save it as a fixed value with the OrderItem.


  3. Kristoffer Garcia Says:

    I’ve been following your blog for awhile, and I think I get what you are saying about apps that are composed from bits of UI hosted within services, but I have doubts about how this affects “queries”. For example, what about screens that are supposed to show a grid of data where the data is owned by two different services (say …Customers and Orders?). I guess one could look at this a couple of ways, 1) Customers and Orders should not be in separate services and the question is invalid, 2) Customer service and Order service replicate the necessary data (so I guess this would be one of those exceptional cases?), 3) The service that hosts the grid in some way gets the necessary data from the other service just for the purposes of displaying the grid (which would be a violation of the service boundary/autonomy?), 4) The grid belongs to a third “OLAP-ish/BI-ish” “service”, where data from the other services is replicated and combined (so another exceptional case?). This might not be a good scenario to use as it assumes that Customer and Order should be in diff services which might not be correct? But hopefully you get the gist of my question and propose a better scenario in the answer if you get to it.


  4. Steve Sheldon Says:

    ĂĽgyviteli, but that’s data duplication. 🙂

    Kristoffer, I know what you are talking about… and I think what we’re encountering is a reporting concern. A lot of apps I work on have a strong reporting element to them, which causes a great deal of confusion to everyone involved.

    I think this problem is what CQRS is trying to address. I think Udi is right, but at the same time might be confusing things as well. Perhaps it’s a question of how the data is being used by the system.


  5. Aaron Blohowiak Says:

    You should not use time between computers for any business logic; NTP, drift, leap seconds &etc ultimately make using “time” your sequencing basis a very poor choice. Alternatives include having a sequencing service or getting a little more advanced with vector clocks (distributed sequencing..)

    You will want a full audit trail of currency conversion factors, price changes and when things were purchased. Additionally, there will invariably be package deals, discount codes &etc. Some of these latter things can be modeled as functions of the bill of sale, but then you get into things where only certain product category + price combinations create eligibility for discounts.

    Kristoffer: 4) is the way to go. Eventually, most systems end up with some form of ETL for crunching/splciing.


  6. udidahan Says:

    Steve,

    The service which owns the pricing is the one that can hold on to the history of prices indexed by time (or with vector clocks if you require high precision as Aaron mentioned).


  7. udidahan Says:

    Kristoffer,

    If you need to compose data from multiple services into a single grid, you can use frameworks like Knockout.js and Backbone.js as shown in this blog post:

    http://blog.hansenfreddy.com/2011/07/07/tabular-data-composition-in-the-browser-for-soa/

    Which shows a shopping cart (a much better example than customers and orders).


  8. Dennis van der Stelt Says:

    Steven,

    ServiceA might hold products and their details, but ServiceB holds their price and calculates totals. So ServiceB never needs to go back, once a product comes in as an order (or as something that needs to be billed) you receive just the primary identifier and are able to calculate total price.


  9. Alberto Says:

    Udi,
    your post is very interesting for me, but a discussion was born in my team. When you talk about “replication WITHIN service”, what do you mean ? Why you wrote WITHIN in upper case ? You think Data Replication must be implemented as SOA service or in other way?


  10. Jay Flowers Says:

    Udi,
    In your InfoQ video you promote the idea of caching the price data by other services. Here you seem to say that is a bad idea. Have I misunderstood or have you changed you mind?

    Thanks,
    Jay


  11. udidahan Says:

    Jay,

    That older video was one in which I was illustrating the use of the pub/sub model in the bus architectural style to move the discussion of SOA in the right direction without yet introducing the concepts of composition-based UIs (as it would have taken up a LOT of space to explain that).


  12. Paul Davies Says:

    Surely some duplication is required? If a few different applications have a different view of a Customer, then perhaps the data relating to that customer will be distributed across those various applications/services. But what if a customer leaves? It is likely one system would handle this and publish an event to inform the others. These applications would need to update their records because this information would affect business decisions. Surely then this particular customer state has been replicated?


  13. udidahan Says:

    Paul,

    An application is not a service (at least not in the way I do SOA). Applications and systems are really just processes/executables that host code that can come from multiple services. The service owns the code – not the app.


  14. Paul Davies Says:

    Udi, I have been watching your distibuted systems course (I recently contacted you about this – very informative from what I’ve watched so far.)

    In the section about Temporal Coupling, you have a slide that shows Service B publishing updated customer info, and Service A storing the data because it needs the data in order to decide whether to make the customer prefered. Surely this data has been replicated? Not trying to pick holes – just making sure I understand everything correctly!


  15. udidahan Says:

    Paul,

    That video introduces the basic concept of temporal coupling without getting too deep into larger architectural issues but yes, there would have been duplication of data in that case.


  16. Paul Davies Says:

    Thanks for that Udi!


  17. Jon Says:

    Hi Udi,

    Has your thinking evolved since this post or am I misunderstanding your position?:

    http://www.udidahan.com/2008/12/13/self-contained-events-and-soa/

    In that post you say that events can contain all the data and services store a local copy?

    Thanks,
    Jon


  18. udidahan Says:

    Jon,

    The thing I rail against in that post is all services having to go to a central database to get the data that they need. You might have noticed that I emphasized that the data going through the events were primarily IDs, and not so much the raw business data.

    My thinking has evolved in that I’ve seen many more real world examples where this approach works becoming more convinced in its advantages.


  19. Paul Davies Says:

    Udi,

    I understand data does not need to be replicated across services just for the sake of displaying that data to the user – as you advocate, UIs are composed of data from many services.

    But, refering again to the Temporal Coupling section of your Distributed Systems course, would it be fair to say that data from Service B can be replicated into Service A if Service A needs to make a business decision based on it?

    Paul


  20. udidahan Says:

    Paul,

    If there is a need for strong consistency in business logic on that data, then stale data (the kind you end up with when data gets duplicated) is out of the question. This would mean that service boundaries are wrong.


  21. Wendell Says:

    Udi,

    It appears that setting service boundaries such that raw data is not duplicated can create services with very different business activities. In the example of book prices, the original idea was that Marketing service included the logic of setting book prices (which was published), and the billing service subscribed to those price updates and used them to calculate a specific total for an order and bill the customer. If your thinking has now evolved to keep prices all within one service, then we have the very different and separate business activities of setting prices and billing customers within the same service. That seems to be contrary to the idea of “business-IT alignment” and removes the benefit of having different teams handling these 2 very different business activities. We end up with a very big service, encompassing every possible activity that requires knowledge of book prices.


  22. udidahan Says:

    Wendell,

    There is a special service I call IT/Ops which is in charge of integration (like with 3rd party credit card systems) which is able to pull in information from multiple other services.

    This resolves the tension of data duplication.


  23. Neylor Ohmaly Says:

    If ServiceA and ServiceC needs some data of the ServiceB to make a decision:

    * This data should be published in an event ?
    * The services boundaries are wrong and the logic of the ServiceA, ServiceB and ServiceC should be in a single service?
    * The ServiceA and ServiceC could query the ServiceB for the data they need?


  24. udidahan Says:

    Neylor,

    If the data structure is extremely stable, then it is OK to share it between services. In these cases, querying is permissible as is using an event.

    Data whose structure is more volatile should not be shared – meaning the service boundaries may need to be adjusted.


  25. Maninder Batth Says:

    IMO, it is okay for an “event” to be “published” across bounded contexts and for that “event” to be persisted. An “Event” is not always the same as “raw business data”. Consider finance and marketing departments. Finance is responsible for providing a base pricing for a product. It does that by taking into account rules such as
    1. Investment costs into creating the product
    2. R&D costs to keep it competitive over its life-cycle
    3. Lifetime of the product etc

    Marketing on the other hand is responsible for determining a “dynamic” rate for a product, which is function of many dimensions such as
    1. Base price as determined by finance
    2. Customer loyalty, such as for how long customer has been using the company’s product
    3. Customer’s investment level, such as how many products and what service levels are consumed by customer. Some customers may be paying large annual fee for best SLA and using family of products
    4. Current competitor prices etc…

    Important thing to note is that even at business level, marketing and finance are two separate department, performing their core business independently.

    In such a case, whenever base pricing is changed by finance, the marketing gets a PriceChanged Event and it applies its various rules to re-evaluate their existing promotions, prices and calculate prices for future orders. The “PriceChanged” event may contain basic information such as Price Id, new price and old price.

    For marketing to be effective, it is critical for it to know when base pricing chances and perform its business operations. Should it store the “PriceChanged” event I would say YES as the event is a business contract, which serves to satisfy auditing.


  26. udidahan Says:

    Maninder,

    Without having a concrete problem in front of me, what I’d say is this:

    Marketing could be responsible for the mark-up/mark-down from the base price. That potentially could be represented as a percentage, rather than a dollar-value.

    Also, don’t confuse the departments within which people sit with service boundaries. It could very well be that there is a deep collaboration between departments that should be identified as its own business capability / service.


  27. Maninder Batth Says:

    Udi,

    With regards to your suggestion
    ” Since services are aligned with logical boundaries, we should expect to see them cross physical boundaries – this includes SYSTEM boundaries (since a system is really nothing more than a unit of deployment).”

    What I understood from the above, is that when applications require shared data, rather than duplicating the raw data to their local stores, they should deploy an instance of a the service responsible for that data.

    In other words, a service that is aligned with business, but physically deployed in many applications that need it. One way to do it would be through some “component” based service design, where the service component could be deployed with various applications. Other than this approach, it would not be possible to achieve the SLA that applications are trying to achieve by duplicating raw data in their stores.

    But this approach poses challenges.

    1.There needs to be some way to document, which applications across the enterprise are dependent on what reusable services

    2. When a service revision is released, co-ordinate deployment across dependent applications in a certain timeline.

    This does not sound agile. What are your suggestions?


  28. udidahan Says:

    Maninder,

    You’ve got it!

    The only thing that I’d say is that it is allowed for the components of a service deployed in a given process to locally cache any data they need, which is fundamentally the same as replicating the data.

    To your questions – yes, documentation is important. We’re working on a tool to simplify this with NServiceBus, but it’s not just about documentation. You use this tool to build your services and compose your applications. That way the documentation is always up to date:

    http://particular.net/servicematrix

    Your second point also requires a more mature devops organization and supporting infrastructure.

    Just because something is hard, doesn’t mean it’s not agile 🙂


  29. Plamen Boudinov Says:

    Hello Udi,

    Thanks enormously for the profound insights that you have brought on me:)

    I understand the ‘Micro’services isolation principle, aiming for vertical stripes, where a service business logic can span multiple processes and system. And the principle not to publish data, but operations as service events.

    However how do we proceed when we have some 3rd party systems, like a CRM, or a delicate Loan Management System that should be isolated from web tier.
    We have a web portal where a customer registers, so at some point his information must be replicated to the CRM, to be used by a CRM related ‘service’. What is this replication in this case, is it hosted by a IT/Ops service?
    And the replication between web tier and Loan management system I guess is service-internal replication, which is perfectly fine?

    Thanks a lot in advance!


  30. udidahan Says:

    Plamen,

    You’ve got it – the 3rd party systems (CRM, Loans, etc) are physical boundaries – and getting data to/from them is an integration concern which falls under the responsibility of IT/Ops.


  31. jesus Says:

    Hi Udi,

    What happens if we duplicate raw data like the quantity of a line item, ‘Billing’ for order total calculation and ‘Shipping’ to know how many items to ship.

    Is there a better way?


  32. udidahan Says:

    Jesus,

    Billing and Shipping (and several other things whose names end in “ing”) are processes that often are the result of the composition of multiple logical services rather than being services themselves. This is how we can avoid data duplication.

    That being said, if the semantic meaning of a piece of data is very stable (like an identifier), then the risk around it being shared across multiple services goes down substantially.

    Quantity is neither here nor there, as it may end up evolving to have units as well (2 pounds, 3 kilograms, 4 megabytes, etc), so I wouldn’t feel entirely comfortable sharing it across service boundaries.


Your comment...



If this is your first time commenting, it may take a while to show up.
I'm working to make that better.

Subscribe here to receive updates on comments.
  
   


Don't miss my best content
 

Recommendations

Bryan Wheeler, Director Platform Development at msnbc.com
“Udi Dahan is the real deal.

We brought him on site to give our development staff the 5-day “Advanced Distributed System Design” training. The course profoundly changed our understanding and approach to SOA and distributed systems.

Consider some of the evidence: 1. Months later, developers still make allusions to concepts learned in the course nearly every day 2. One of our developers went home and made her husband (a developer at another company) sign up for the course at a subsequent date/venue 3. Based on what we learned, we’ve made constant improvements to our architecture that have helped us to adapt to our ever changing business domain at scale and speed If you have the opportunity to receive the training, you will make a substantial paradigm shift.

If I were to do the whole thing over again, I’d start the week by playing the clip from the Matrix where Morpheus offers Neo the choice between the red and blue pills. Once you make the intellectual leap, you’ll never look at distributed systems the same way.

Beyond the training, we were able to spend some time with Udi discussing issues unique to our business domain. Because Udi is a rare combination of a big picture thinker and a low level doer, he can quickly hone in on various issues and quickly make good (if not startling) recommendations to help solve tough technical issues.” November 11, 2010

Sam Gentile Sam Gentile, Independent WCF & SOA Expert
“Udi, one of the great minds in this area.
A man I respect immensely.”





Ian Robinson Ian Robinson, Principal Consultant at ThoughtWorks
"Your blog and articles have been enormously useful in shaping, testing and refining my own approach to delivering on SOA initiatives over the last few years. Over and against a certain 3-layer-application-architecture-blown-out-to- distributed-proportions school of SOA, your writing, steers a far more valuable course."

Shy Cohen Shy Cohen, Senior Program Manager at Microsoft
“Udi is a world renowned software architect and speaker. I met Udi at a conference that we were both speaking at, and immediately recognized his keen insight and razor-sharp intellect. Our shared passion for SOA and the advancement of its practice launched a discussion that lasted into the small hours of the night.
It was evident through that discussion that Udi is one of the most knowledgeable people in the SOA space. It was also clear why – Udi does not settle for mediocrity, and seeks to fully understand (or define) the logic and principles behind things.
Humble yet uncompromising, Udi is a pleasure to interact with.”

Glenn Block Glenn Block, Senior Program Manager - WCF at Microsoft
“I have known Udi for many years having attended his workshops and having several personal interactions including working with him when we were building our Composite Application Guidance in patterns & practices. What impresses me about Udi is his deep insight into how to address business problems through sound architecture. Backed by many years of building mission critical real world distributed systems it is no wonder that Udi is the best at what he does. When customers have deep issues with their system design, I point them Udi's way.”

Karl Wannenmacher Karl Wannenmacher, Senior Lead Expert at Frequentis AG
“I have been following Udi’s blog and podcasts since 2007. I’m convinced that he is one of the most knowledgeable and experienced people in the field of SOA, EDA and large scale systems.
Udi helped Frequentis to design a major subsystem of a large mission critical system with a nationwide deployment based on NServiceBus. It was impressive to see how he took the initial architecture and turned it upside down leading to a very flexible and scalable yet simple system without knowing the details of the business domain. I highly recommend consulting with Udi when it comes to large scale mission critical systems in any domain.”

Simon Segal Simon Segal, Independent Consultant
“Udi is one of the outstanding software development minds in the world today, his vast insights into Service Oriented Architectures and Smart Clients in particular are indeed a rare commodity. Udi is also an exceptional teacher and can help lead teams to fall into the pit of success. I would recommend Udi to anyone considering some Architecural guidance and support in their next project.”

Ohad Israeli Ohad Israeli, Chief Architect at Hewlett-Packard, Indigo Division
“When you need a man to do the job Udi is your man! No matter if you are facing near deadline deadlock or at the early stages of your development, if you have a problem Udi is the one who will probably be able to solve it, with his large experience at the industry and his widely horizons of thinking , he is always full of just in place great architectural ideas.
I am honored to have Udi as a colleague and a friend (plus having his cell phone on my speed dial).”

Ward Bell Ward Bell, VP Product Development at IdeaBlade
“Everyone will tell you how smart and knowledgable Udi is ... and they are oh-so-right. Let me add that Udi is a smart LISTENER. He's always calibrating what he has to offer with your needs and your experience ... looking for the fit. He has strongly held views ... and the ability to temper them with the nuances of the situation.
I trust Udi to tell me what I need to hear, even if I don't want to hear it, ... in a way that I can hear it. That's a rare skill to go along with his command and intelligence.”

Eli Brin, Program Manager at RISCO Group
“We hired Udi as a SOA specialist for a large scale project. The development is outsourced to India. SOA is a buzzword used almost for anything today. We wanted to understand what SOA really is, and what is the meaning and practice to develop a SOA based system.
We identified Udi as the one that can put some sense and order in our minds. We started with a private customized SOA training for the entire team in Israel. After that I had several focused sessions regarding our architecture and design.
I will summarize it simply (as he is the software simplist): We are very happy to have Udi in our project. It has a great benefit. We feel good and assured with the knowledge and practice he brings. He doesn’t talk over our heads. We assimilated nServicebus as the ESB of the project. I highly recommend you to bring Udi into your project.”

Catherine Hole Catherine Hole, Senior Project Manager at the Norwegian Health Network
“My colleagues and I have spent five interesting days with Udi - diving into the many aspects of SOA. Udi has shown impressive abilities of understanding organizational challenges, and has brought the business perspective into our way of looking at services. He has an excellent understanding of the many layers from business at the top to the technical infrstructure at the bottom. He is a great listener, and manages to simplify challenges in a way that is understandable both for developers and CEOs, and all the specialists in between.”

Yoel Arnon Yoel Arnon, MSMQ Expert
“Udi has a unique, in depth understanding of service oriented architecture and how it should be used in the real world, combined with excellent presentation skills. I think Udi should be a premier choice for a consultant or architect of distributed systems.”

Vadim Mesonzhnik, Development Project Lead at Polycom
“When we were faced with a task of creating a high performance server for a video-tele conferencing domain we decided to opt for a stateless cluster with SQL server approach. In order to confirm our decision we invited Udi.

After carefully listening for 2 hours he said: "With your kind of high availability and performance requirements you don’t want to go with stateless architecture."

One simple sentence saved us from implementing a wrong product and finding that out after years of development. No matter whether our former decisions were confirmed or altered, it gave us great confidence to move forward relying on the experience, industry best-practices and time-proven techniques that Udi shared with us.
It was a distinct pleasure and a unique opportunity to learn from someone who is among the best at what he does.”

Jack Van Hoof Jack Van Hoof, Enterprise Integration Architect at Dutch Railways
“Udi is a respected visionary on SOA and EDA, whose opinion I most of the time (if not always) highly agree with. The nice thing about Udi is that he is able to explain architectural concepts in terms of practical code-level examples.”

Neil Robbins Neil Robbins, Applications Architect at Brit Insurance
“Having followed Udi's blog and other writings for a number of years I attended Udi's two day course on 'Loosely Coupled Messaging with NServiceBus' at SkillsMatter, London.

I would strongly recommend this course to anyone with an interest in how to develop IT systems which provide immediate and future fitness for purpose. An influential and innovative thought leader and practitioner in his field, Udi demonstrates and shares a phenomenally in depth knowledge that proves his position as one of the premier experts in his field globally.

The course has enhanced my knowledge and skills in ways that I am able to immediately apply to provide benefits to my employer. Additionally though I will be able to build upon what I learned in my 2 days with Udi and have no doubt that it will only enhance my future career.

I cannot recommend Udi, and his courses, highly enough.”

Nick Malik Nick Malik, Enterprise Architect at Microsoft Corporation
“You are an excellent speaker and trainer, Udi, and I've had the fortunate experience of having attended one of your presentations. I believe that you are a knowledgable and intelligent man.”

Sean Farmar Sean Farmar, Chief Technical Architect at Candidate Manager Ltd
“Udi has provided us with guidance in system architecture and supports our implementation of NServiceBus in our core business application.

He accompanied us in all stages of our development cycle and helped us put vision into real life distributed scalable software. He brought fresh thinking, great in depth of understanding software, and ongoing support that proved as valuable and cost effective.

Udi has the unique ability to analyze the business problem and come up with a simple and elegant solution for the code and the business alike.
With Udi's attention to details, and knowledge we avoided pit falls that would cost us dearly.”

Børge Hansen Børge Hansen, Architect Advisor at Microsoft
“Udi delivered a 5 hour long workshop on SOA for aspiring architects in Norway. While keeping everyone awake and excited Udi gave us some great insights and really delivered on making complex software challenges simple. Truly the software simplist.”

Motty Cohen, SW Manager at KorenTec Technologies
“I know Udi very well from our mutual work at KorenTec. During the analysis and design of a complex, distributed C4I system - where the basic concepts of NServiceBus start to emerge - I gained a lot of "Udi's hours" so I can surely say that he is a professional, skilled architect with fresh ideas and unique perspective for solving complex architecture challenges. His ideas, concepts and parts of the artifacts are the basis of several state-of-the-art C4I systems that I was involved in their architecture design.”

Aaron Jensen Aaron Jensen, VP of Engineering at Eleutian Technology
“Awesome. Just awesome.

We’d been meaning to delve into messaging at Eleutian after multiple discussions with and blog posts from Greg Young and Udi Dahan in the past. We weren’t entirely sure where to start, how to start, what tools to use, how to use them, etc. Being able to sit in a room with Udi for an entire week while he described exactly how, why and what he does to tackle a massive enterprise system was invaluable to say the least.

We now have a much better direction and, more importantly, have the confidence we need to start introducing these powerful concepts into production at Eleutian.”

Gad Rosenthal Gad Rosenthal, Department Manager at Retalix
“A thinking person. Brought fresh and valuable ideas that helped us in architecting our product. When recommending a solution he supports it with evidence and detail so you can successfully act based on it. Udi's support "comes on all levels" - As the solution architect through to the detailed class design. Trustworthy!”

Chris Bilson Chris Bilson, Developer at Russell Investment Group
“I had the pleasure of attending a workshop Udi led at the Seattle ALT.NET conference in February 2009. I have been reading Udi's articles and listening to his podcasts for a long time and have always looked to him as a source of advice on software architecture.
When I actually met him and talked to him I was even more impressed. Not only is Udi an extremely likable person, he's got that rare gift of being able to explain complex concepts and ideas in a way that is easy to understand.
All the attendees of the workshop greatly appreciate the time he spent with us and the amazing insights into service oriented architecture he shared with us.”

Alexey Shestialtynov Alexey Shestialtynov, Senior .Net Developer at Candidate Manager
“I met Udi at Candidate Manager where he was brought in part-time as a consultant to help the company make its flagship product more scalable. For me, even after 30 years in software development, working with Udi was a great learning experience. I simply love his fresh ideas and architecture insights.
As we all know it is not enough to be armed with best tools and technologies to be successful in software - there is still human factor involved. When, as it happens, the project got in trouble, management asked Udi to step into a leadership role and bring it back on track. This he did in the span of a month. I can only wish that things had been done this way from the very beginning.
I look forward to working with Udi again in the future.”

Christopher Bennage Christopher Bennage, President at Blue Spire Consulting, Inc.
“My company was hired to be the primary development team for a large scale and highly distributed application. Since these are not necessarily everyday requirements, we wanted to bring in some additional expertise. We chose Udi because of his blogging, podcasting, and speaking. We asked him to to review our architectural strategy as well as the overall viability of project.
I was very impressed, as Udi demonstrated a broad understanding of the sorts of problems we would face. His advice was honest and unbiased and very pragmatic. Whenever I questioned him on particular points, he was able to backup his opinion with real life examples. I was also impressed with his clarity and precision. He was very careful to untangle the meaning of words that might be overloaded or otherwise confusing. While Udi's hourly rate may not be the cheapest, the ROI is undoubtedly a deal. I would highly recommend consulting with Udi.”

Robert Lewkovich, Product / Development Manager at Eggs Overnight
“Udi's advice and consulting were a huge time saver for the project I'm responsible for. The $ spent were well worth it and provided me with a more complete understanding of nServiceBus and most importantly in helping make the correct architectural decisions earlier thereby reducing later, and more expensive, rework.”

Ray Houston Ray Houston, Director of Development at TOPAZ Technologies
“Udi's SOA class made me smart - it was awesome.

The class was very well put together. The materials were clear and concise and Udi did a fantastic job presenting it. It was a good mixture of lecture, coding, and question and answer. I fully expected that I would be taking notes like crazy, but it was so well laid out that the only thing I wrote down the entire course was what I wanted for lunch. Udi provided us with all the lecture materials and everyone has access to all of the samples which are in the nServiceBus trunk.

Now I know why Udi is the "Software Simplist." I was amazed to find that all the code and solutions were indeed very simple. The patterns that Udi presented keep things simple by isolating complexity so that it doesn't creep into your day to day code. The domain code looks the same if it's running in a single process or if it's running in 100 processes.”

Ian Cooper Ian Cooper, Team Lead at Beazley
“Udi is one of the leaders in the .Net development community, one of the truly smart guys who do not just get best architectural practice well enough to educate others but drives innovation. Udi consistently challenges my thinking in ways that make me better at what I do.”

Liron Levy, Team Leader at Rafael
“I've met Udi when I worked as a team leader in Rafael. One of the most senior managers there knew Udi because he was doing superb architecture job in another Rafael project and he recommended bringing him on board to help the project I was leading.
Udi brought with him fresh solutions and invaluable deep architecture insights. He is an authority on SOA (service oriented architecture) and this was a tremendous help in our project.
On the personal level - Udi is a great communicator and can persuade even the most difficult audiences (I was part of such an audience myself..) by bringing sound explanations that draw on his extensive knowledge in the software business. Working with Udi was a great learning experience for me, and I'll be happy to work with him again in the future.”

Adam Dymitruk Adam Dymitruk, Director of IT at Apara Systems
“I met Udi for the first time at DevTeach in Montreal back in early 2007. While Udi is usually involved in SOA subjects, his knowledge spans all of a software development company's concerns. I would not hesitate to recommend Udi for any company that needs excellent leadership, mentoring, problem solving, application of patterns, implementation of methodologies and straight out solution development.
There are very few people in the world that are as dedicated to their craft as Udi is to his. At ALT.NET Seattle, Udi explained many core ideas about SOA. The team that I brought with me found his workshop and other talks the highlight of the event and provided the most value to us and our organization. I am thrilled to have the opportunity to recommend him.”

Eytan Michaeli Eytan Michaeli, CTO Korentec
“Udi was responsible for a major project in the company, and as a chief architect designed a complex multi server C4I system with many innovations and excellent performance.”


Carl Kenne Carl Kenne, .Net Consultant at Dotway AB
“Udi's session "DDD in Enterprise apps" was truly an eye opener. Udi has a great ability to explain complex enterprise designs in a very comprehensive and inspiring way. I've seen several sessions on both DDD and SOA in the past, but Udi puts it in a completly new perspective and makes us understand what it's all really about. If you ever have a chance to see any of Udi's sessions in the future, take it!”

Avi Nehama, R&D Project Manager at Retalix
“Not only that Udi is a briliant software architecture consultant, he also has remarkable abilities to present complex ideas in a simple and concise manner, and...
always with a smile. Udi is indeed a top-league professional!”

Ben Scheirman Ben Scheirman, Lead Developer at CenterPoint Energy
“Udi is one of those rare people who not only deeply understands SOA and domain driven design, but also eloquently conveys that in an easy to grasp way. He is patient, polite, and easy to talk to. I'm extremely glad I came to his workshop on SOA.”

Scott C. Reynolds Scott C. Reynolds, Director of Software Engineering at CBLPath
“Udi is consistently advancing the state of thought in software architecture, service orientation, and domain modeling.
His mastery of the technologies and techniques is second to none, but he pairs that with a singular ability to listen and communicate effectively with all parties, technical and non, to help people arrive at context-appropriate solutions. Every time I have worked with Udi, or attended a talk of his, or just had a conversation with him I have come away from it enriched with new understanding about the ideas discussed.”

Evgeny-Hen Osipow, Head of R&D at PCLine
“Udi has helped PCLine on projects by implementing architectural blueprints demonstrating the value of simple design and code.”

Rhys Campbell Rhys Campbell, Owner at Artemis West
“For many years I have been following the works of Udi. His explanation of often complex design and architectural concepts are so cleanly broken down that even the most junior of architects can begin to understand these concepts. These concepts however tend to typify the "real world" problems we face daily so even the most experienced software expert will find himself in an "Aha!" moment when following Udi teachings.
It was a pleasure to finally meet Udi in Seattle Alt.Net OpenSpaces 2008, where I was pleasantly surprised at how down-to-earth and approachable he was. His depth and breadth of software knowledge also became apparent when discussion with his peers quickly dove deep in to the problems we current face. If given the opportunity to work with or recommend Udi I would quickly take that chance. When I think .Net Architecture, I think Udi.”

Sverre Hundeide Sverre Hundeide, Senior Consultant at Objectware
“Udi had been hired to present the third LEAP master class in Oslo. He is an well known international expert on enterprise software architecture and design, and is the author of the open source messaging framework nServiceBus. The entire class was based on discussion and interaction with the audience, and the only Power Point slide used was the one showing the agenda.
He started out with sketching a naive traditional n-tier application (big ball of mud), and based on suggestions from the audience we explored different solutions which might improve the solution. Whatever suggestions we threw at him, he always had a thoroughly considered answer describing pros and cons with the suggested solution. He obviously has a lot of experience with real world enterprise SOA applications.”

Raphaël Wouters Raphaël Wouters, Owner/Managing Partner at Medinternals
“I attended Udi's excellent course 'Advanced Distributed System Design with SOA and DDD' at Skillsmatter. Few people can truly claim such a high skill and expertise level, present it using a pragmatic, concrete no-nonsense approach and still stay reachable.”

Nimrod Peleg Nimrod Peleg, Lab Engineer at Technion IIT
“One of the best programmers and software engineer I've ever met, creative, knows how to design and implemet, very collaborative and finally - the applications he designed implemeted work for many years without any problems!”

Jose Manuel Beas
“When I attended Udi's SOA Workshop, then it suddenly changed my view of what Service Oriented Architectures were all about. Udi explained complex concepts very clearly and created a very productive discussion environment where all the attendees could learn a lot. I strongly recommend hiring Udi.”

Daniel Jin Daniel Jin, Senior Lead Developer at PJM Interconnection
“Udi is one of the top SOA guru in the .NET space. He is always eager to help others by sharing his knowledge and experiences. His blog articles often offer deep insights and is a invaluable resource. I highly recommend him.”

Pasi Taive Pasi Taive, Chief Architect at Tieto
“I attended both of Udi's "UI Composition Key to SOA Success" and "DDD in Enterprise Apps" sessions and they were exceptionally good. I will definitely participate in his sessions again. Udi is a great presenter and has the ability to explain complex issues in a manner that everyone understands.”

Eran Sagi, Software Architect at HP
“So far, I heard about Service Oriented architecture all over. Everyone mentions it – the big buzz word. But, when I actually asked someone for what does it really mean, no one managed to give me a complete satisfied answer. Finally in his excellent course “Advanced Distributed Systems”, I got the answers I was looking for. Udi went over the different motivations (principles) of Services Oriented, explained them well one by one, and showed how each one could be technically addressed using NService bus. In his course, Udi also explain the way of thinking when coming to design a Service Oriented system. What are the questions you need to ask yourself in order to shape your system, place the logic in the right places for best Service Oriented system.

I would recommend this course for any architect or developer who deals with distributed system, but not only. In my work we do not have a real distributed system, but one PC which host both the UI application and the different services inside, all communicating via WCF. I found that many of the architecture principles and motivations of SOA apply for our system as well. Enough that you have SW partitioned into components and most of the principles becomes relevant to you as well. Bottom line – an excellent course recommended to any SW Architect, or any developer dealing with distributed system.”

Consult with Udi

Guest Authored Books



Creative Commons License  © Copyright 2005-2011, Udi Dahan. email@UdiDahan.com