Why you should be using CQRS almost everywhere…
Sunday, October 2nd, 2011.
… but differently than the way most people have been using it.
I think I’ve just about drove everybody crazy now with my apparent zigzagging on CQRS.
Some people heard about CQRS first from one of my presentations and got all excited about it. Then I did some blogging which further drove people to CQRS (as did Greg Young and some others). As CQRS was just about to hit its stride with the Early Adopters, I started pushing a more balanced view – CQRS not as an answer, but as one of many questions. More recently I’ve pushed more strongly back against CQRS saying that it should be used rarely.
So what’s the missing piece?
If you’re in the Domain-Driven Design camp (as many doing CQRS are), then it’s Bounded Contexts.
If you’re in the Event-Driven SOA camp (a much smaller camp to be sure), then it’s Services.
The problem is the naming, because the DDD guys have their kinds of services which do not fit the definition for Service of the Event-Driven SOA approach.
Let me propose the term Autonomous Business Component for the purposes of this blog post to describe that thing which is both a DDD Bounded Context (have the shared BC part of the acronym) and an SOA Autonomous Services. Resulting in the nice short form: ABC (and everyone knows you need to have a good acronym if you want something to catch on).
What does this have to do with CQRS?
Nothing just yet. Well, at least, nothing directly to do with CQRS.
Although some proponents of CQRS have stated that it can and should be used as the top-most architectural pattern, both myself and Greg Young (arguably the first two to talk about it and the two who ultimately collaborated on naming it – and now Google knows we didn’t means “cars”) always recommended it as a pattern to be used one level down.
Although Greg and I have had many long discussions on the topic and do agree very much about what the overall structure should look like, I’ll try to avoid putting words in his mouth from this point on.
Before talking more about ABCs, let’s discuss the principle upon which they rest: The Single Responsibility Principle (SRP).
What does SRP have to with CQRS?
Many developers are familiar with SRP and have seen good results from using it. What we’re going to do is take this principle to the next level.
In Object Orientation (OO), data is encapsulated in an object. A good object does not expose its data to other objects to do with as they wish. Rather, it exposes methods that other objects can invoke, and those methods operate on the internal data.
SRP would guide us to not have the same data exist in two objects. For example, if we saw the customer’s first name as an internal data member of two objects, we’d be right to question that kind of duplication and move to refactor it away. However, when we see two systems doing the exact same thing – somehow that gets excused.
“Of course we need to be able to see the customer’s first name in the front-end website as well as in the back-end fulfillment system. How could we NOT have the customer’s first name in both those code-bases?”
And there’s the catch.
Who said that a system should be a single code-base?
But what about integration?
Although many times we do need to integrate existing systems together, sometimes we have the ability to change those systems. More importantly, when going to create a new solution, we can avoid getting ourselves into the problems that integration tries to solve.
Integrating with a system that cannot be changed can be done also by composing multiple ABCs, but that’s a topic for another post.
It is better to think of integration as a necessary evil – kind of like regular expressions and multi-threading; things to be avoided unless absolutely necessary.
“If you have a problem that you decide to use a regular expression to solve, you now have 2 problems.” Or so the saying goes. With multi-threading, you have a non-deterministic number of problems to solve.
If you thought you had duplicate responsibilities with 2 systems operating on the same data, how will introducing a 3rd code base (also known as “integration”) help? Remember that Single Responsibility Principle – our goal is to get it down to one.
OK, so how do ABCs do that?
In order for us to get back into alignment with SRP, that would require us to have responsibility for a single piece of data exist in one code base. Note that SRP makes no statements about how many physical places a given code base can be deployed to. Nor does it state that only a single technology can be in play – code that emits HTML can be packaged at design time together with rich-client code in the same solution.
If an ABC is responsible for a piece of data, it is responsible for it everywhere, and forever. No other ABC should see that data. That data should not travel between ABCs via remote procedure call (RPC) or via publish/subscribe. It is the ultimate level of encapsulation – SRP applied at the highest level of granularity.
This results in systems which are the result of deploying the components of multiple ABCs to the same physical place. The ABC which owns the customer name would have the necessary web code to render it in the e-commerce front-end and in the shipping back-end for printing on labels. This would mean that practically every screen in any UI is a composite of widgets owned by their respective ABCs.
This is ultimately what keeps the complexity of each ABC’s code base to a minimum.
But why not just use CQRS as the top-level pattern? ABCs are weird.
Imagine trying to create a single denormalized view model for the entire Amazon.com product page – product name, price, inventory, editorial review, customer comments, other products that customers viewed, other products that customers bought, etc.
Pretty complex, right?
How much duplication would you have for the page shown after you add an item to a cart? Once again, you need to show other products that customers bought, their names, images, prices, and inventory.
And then on the home page – items you might be interested in, names, images, prices.
And that’s only in the front-end system.
It’s not just the duplication, but how complex the code is for each one.
Instead of the duplication that top-level CQRS would bring you, consider an ABC responsible for products names and images that has just about the same view model composed on each of the above screens. The same with another ABC responsible for price.
You may be thinking that this would result in more queries to get the data to show on a page, and you’d be right. But it isn’t necessarily a classical N+1 Select problem, as the queries are bounded to the number of ABCs. Secondly, consider the ability to have well-tuned caching at the granularity of an ABC – something that would be much more difficult when dealing with everything as a single monolithic view model. In short, not only will it not be a performance problem, often it will actually improve performance.
OK – that explains “everywhere”, what about “forever”?
Forever is where things get interesting – or more accurately, when they get interesting.
Let’s talk about things like invoices.
One of the requirements in this area is that immutability. If the customer’s name was Jane Smith when they made their purchase, it doesn’t matter that they’ve since changed their name to Jane Jones, the invoice should still show Jane Smith.
Often developers push these types of requirements on the data warehouse guys – that’s where history gets handled. The only thing is that if your ABC owns the customer’s name, then no other code base can deal with it. If it’s your data, you have to handle all historical representations of it.
On the one hand, this would seem to kill the data warehouse. On the other hand, it means that the principles of data warehouses are now core to every code-base.
This means you don’t ever delete data (see my previous blog post on the subject), and you definitely don’t overwrite it with an update – even if you think you’re in a simple CRUD domain. The only case where you can get away with traditional CRUD is if we’re talking about private data – data that is only ever acted on by a single actor.
This sounds like the collaboration you talk about with CQRS
It’s similar in principle but different in practice.
In a collaborative domain, an inherent property of the domain is that multiple actors operate in parallel on the same set of data. A reservation system for concerts would be a good example of a collaborative domain – everyone wants the “good seats” (although it might be better call that competitive rather than collaborative, it is effectively the same principle).
A customer’s name would not fall under that category. It isn’t an inherent property of the domain for multiple actors to operate on that data. While there can be multiple readers, one can easily enforce a single writer without any adverse effects. Doing that with a reservation system would cause the online system to behave as if users were lining up in front of a box office – not a desirable outcome.
Private data would be something like a user’s shopping cart. Until they make a purchase, that data doesn’t need to be visible anywhere. Here you could theoretically do simple CRUD – that is, until the business realizes that there’s extremely valuable information to be extracted from the historical record of things people do with their carts.
I think you’re ready to make your point, so just make it already
OK – so we now realize that Update and Delete don’t exist in their traditional form. Delete is really just a kind of update, and update is effectively an “upsert” – a combination of update and insert to retain history. This can be done by having ValidFrom and ValidTo columns for our data.
In which case, Create is really just a special case of Upsert, which looks like this:
UPDATE Something SET ValidTo = NOW() WHERE Id=@Id AND ValidTo = NULL; INSERT INTO Something SET { regular values }, Id=@Id, ValidTo = NULL;
And then we’d have 2 forms of Read – reading the current state (ValidTo = NULL), and reading history (ValidFrom <= Instant AND (ValidTo >= Instant OR ValidTo = NULL))
Here we don’t need fancy N-Tier architectures, data transfer objects, service layers, or domain models. A simple 2-Tier approach could probably suffice. We don’t need a task-based UI, events, denormalized view models, or any of that CQRS stuff. This was at the crux of my previous anti-CQRS post.
The only thing is that this is exactly CQRS.
Say what?
Have we not effectively separated the responsibility of commands/upserts and queries/reads?
As Greg Young has said before, “the creation of 2 objects where there previously was one”.
Effectively 2 paths through our ABC.
CQRS.
Let me give you a second to gather your thoughts.
*
You see, CQRS is an approach, a mind-set – not a cookie cutter solution. Frameworks that guide you to applying CQRS exactly the same way everywhere are taking you in the wrong direction. The fact is that you couldn’t possibly know what your Aggregate Roots were before you figured out how to break your system down into ABCs. Attempting to create commands and events for everything will make you overcomplicate your solution.
So the built-in history of this model is event-sourcing?
Well, it’s not event-sourcing in the sense that we don’t necessarily have events. It achieves many of the benefits of event-sourcing by giving us the full history of what happened.
On the whole issue of replaying events to fix bugs – that’s a bit problematic, logically, unless we have a closed system. A closed system is one that doesn’t interact with anything else – no other systems, no users, nothing. As such, closed systems aren’t that common.
In an open system, one with users, let’s say there was a bug. This bug could have caused the wrong data to be written and/or shown to users. As such, users could have submitted subsequent commands based on that erroneous data that they would not have submitted otherwise. There’s no way for us to know.
The problem with replaying events when we fix the bug is that we’re in essence rewriting history – making it as if the user didn’t see the wrong data. The only problem is that we can’t know which events not to replay – we can’t automatically come up with the right events that should have come afterwards. We could try to sit together with our users and have them try to revise history manually, but our organization often isn’t in a bubble. Our users interacted with customers and suppliers. It isn’t feasible to try to undo the real-world impacts of this situation.
Why didn’t you just tell us this from the very beginning?
I did, you just weren’t listening.
You wanted a cookie cutter, and until you tried CQRS out as cookie cutter (and saw it create a bunch of complexity) you wouldn’t listen to anything else.
As developers, we’re trained to solve problems – the faster the better. Unfortunately, this causes us to be blind to things that don’t immediately present themselves as solutions.
When applying CQRS with ABCs, the solutions you end up with are very simple, but the process of getting there is quite hard and takes practice. Finding the boundaries of ABCs such that data isn’t duplicated between them and that data doesn’t travel between them either via RPC or publish/subscribe – it may feel impossible the first several times you try. Keep at it – it is almost always possible.
We haven’t touched on the whole saga/aggregate-root thing yet, but that isn’t as important until you can successfully apply the principles described here.
Also, this post has already gotten long enough, so it looks like now would be a good time to stop.
Until next time…
|
If you liked this article, you might also like articles in these categories: 
If you've got a minute, you might enjoy taking a look at some of my best articles.I've gone through the hundreds of articles I've written over the past 6 years and put together a list of the best ones as ranked by my 5000+ readers. You won't be disappointed.  If you'd like to get new articles sent to you when they're published, it's easy and free.Subscribe right here.  Follow me on Twitter @UdiDahan.
Something on your mind? Got a question? I'd be thrilled to hear it. Leave a comment below or email me, whatever works for you. 49 CommentsYour comment... |
October 2nd, 2011 at 7:33 pm
Cool to see you and your mate Greg stating everything and the opposite of everything.
October 2nd, 2011 at 7:39 pm
Great stuff. Hurry up with the next one
October 2nd, 2011 at 9:13 pm
An interesting read, as so often. I would expect all this to evolve with your experience as well, so the CQRS of 2 years ago may not be the very same CQRS as today’s one.
thanks for posting on Sunday btw, so little content on that day!
October 3rd, 2011 at 6:16 am
One question that we came up with was how would you handle the following situation…
ABC 1 is responsible for handling a persons name (could contain other info of course).
ABC 2 is responsible for handling your security pass. If your name changes you should automatically be sent a new security pass.
How does ABC 2 know to do its job if there is no pub/sub or communication?
October 3rd, 2011 at 8:12 am
Hi Udi,
What an exquisite picture you’ve used for this post:D
October 3rd, 2011 at 8:24 am
Shane,
I didn’t say that there wasn’t ANY pub/sub between ABCs, but rather that data (like the person’s name) wouldn’t be duplicated via that mechanism. You could have ABC1 publish an event saying PersonNameChanged(PersonId=123) and have ABC2 subscribed to that and do whatever it needs to do there.
October 3rd, 2011 at 10:10 am
Speaking about “N+1″ generated by the split ABCs you’re writing about. I can imagine that it can be generated only in case where you don’t delegate showing some list to the ABC responsible for it, but rather querying it one by one, from the one of one-to-many relationship. It’d be non-straightforward anyway. Can you give an example of this possibility?
Thank you for the counterexample for event sourcing. It’s vast. I still do find this technique useful, though. Better to have more info for resolving any kind of problematic situations. Even those, with wrongly chosen commands (by a user), than trying to examine sql tables.
Upserting, or more: versioning can now be easily handled with a Raven’s bundle, can’t it?
October 3rd, 2011 at 10:57 am
Hi Udi, wazzup ?
You kind of lost me here on the “everywhere/forever” issue
1. In the example Shane gave and your answer – if the new person name will not be published/transmitted – how would you expect the new name tag to carry the new name. If it isn’t cached (a.k.a. duplicated) by second service how would it be autonomous (i.e. not depend on service 1 being online for its operation)
2. Again regarding keeping data private from the DW perspective. I totally agree that services should retain the history and that data should be versioned (so that old invoice of version X would be to a name of version Y). I don’t agree that the data cannot be replicated into a datamart/OLAP cube/whatever for reporting purposes esp. (something I call “Aggregated Reporting” see http://www.infoq.com/articles/BI-and-SOA )
Arnon
October 3rd, 2011 at 3:27 pm
@Arnon Rotem-Gal-Oz
Here’s my take on it, hopefully I’m not far off.
1. You know the name change by the type of event, in that example the event is “PersonNameChanged” and all you need is the ID of the person in order to process the event.
Part of the process may be to look up various pieces of information in order to finish it but that can be retrieved with Queries against the ABC. In that example I see the need to get the person’s email address and possibly the previous and new names, the second ABC that is responsible for the security pass doesn’t need to store the old or new name as long as it can query the other one when necessary.
The process to generate and then send out a new security pass will end up becoming a saga because it depends on the availability of other services for querying data in order to finish. A saga works in this scenario because if one of the services is down it will wait for it to come back up then continue on with the next step.
I know from personal experience it gets a bit confusing when you start trying to combine all of the pieces into something larger. I was recently reading an old programming classic that helped me get my head around it though “The Pragmatic Programmer From Journeyman to Master”. I highly recommend it if you haven’t already read it and there’s a section that describes how Unix based systems are built with all of their single purpose console applications. I think it’s a very good analogy for anyone familiar with the theory behind Unix and Linux and does a much better job of explaining it than I can here.
October 4th, 2011 at 3:29 am
Hi Udi, interesting read. I am curious about the ValidTo business. The assumption there seems to be that we might regularly need the “historical” data, but what if we don’t? Could the same not be accomplished with an archive? Basically a “copy values to the historical database (or tables) and update the current values” type thing.
October 4th, 2011 at 8:41 am
@Kevin
The kind of integration where you send an event that something changed and then you need to query the other service whenever you need the data is not a very good option performance wise. It can work well when you are within the same memory space but not in a distributed environment (see the fallacies of distributed computing http://nighthacks.com/roller/jag/resource/Fallacies.html).
You create a temporal coupling between the two services which is why a lot of RPC based system fail miserably – Suggesting to add a saga (or orchestration which is another alternative here) to solve this adds a lot of complexity – Properly implementing Sagas requires coordination logic, compensations etc (see http://www.rgoarchitects.com/Files/SOAPatterns/Saga.pdf). Considering the alternative is to pass the data with the original event and caching the *immutable* versioned data it propagates demonstrates further why it is a faulty idea.
Not to mention what would happen in reporting and BI scenarios which I mentioned in my point 2.
Arnon
October 4th, 2011 at 10:07 am
Hi,
I really like the idea of every application and service being equipped with a little web server for rendering views of their data, but I don’t think it will work. We don’t own every application that uses our data so we can’t implement a html view that will satisfy everyone. There’s the same problem with exposing the data in any other way so I don’t see how the html view is better than a web service or RPC?
The next thing: ‘forever’ – any service that is responsible for some data is also responsible for making all previous versions of that data available. Why? Just because of the principle of not giving your data to anyone else – you own it and you’re the only source of that data therefore you have to keep all the previous versions because someone might ask for that. This is sick imho. Versioning is very hard to implement correctly (especially in a relational database) and could be easily avoided if you let other software keep a copy of the data – it’s immutable after all so I don’t see why not?
Third – the ‘evil integration’ and your thesis that it will never help when there are duplicate responsibilities in two systems. Why? I can easily see how integrating these systems can help avoid duplicate responsibilities – for example you can use integration to transfer the responsibility for duplicate actions to just one system and make the other system a read-only view for the first one’s data. Isn’t it a CQRS? Even if the data is duplicated, the modifications are performed only by the owner system.
I didn’t want to look like a nitpicker with all these questions, but this post as a whole is too abstract and too general for me to understand what’s it really about and so I concentrated only on more specific statements. I’d be very happy to see practical illustrations of these concepts in application architecture.
October 4th, 2011 at 5:53 pm
When an order is placed the customer name should be kept as the name the order was registered with ad-infinitum. This I agree with. However, it would seem strange to have to look up the customer name when, for instance, printing the order. The fact the the Customer Management BC (ABC) and the Order Management BC (ABC) are separate does *not*, IMHO, mean that the data is duplicated if the customer name is kept in the Order ABC. By *not* keeping the original data (customer name) on the order one would effectively complicate matters somewhat as we now need to request *historical* data from the customer ABC, and that is *totally* unnecessary.
If I have misinterpreted what you meant then let me know
Regards,
Eben
October 4th, 2011 at 10:53 pm
Well… Since Udi is not responding… We have no choice but turn this into discussion thread
@Kevin: I have a feeling that those ABC don’t rely on querying each other that much. They wouldn’t be autonomous in that case. They split coherent sets of data like User into chunks according to usage scenarios. ABCs then coordinate consistency of that coherent peace of data by an ID which serves as a correlationId.
@Sean: Probably it has to do with autonomy of services and not having to cascade changes to all interested parties. If one ABC issues an event about something related to concept like User even if other ABCs’ representation of User changed it still knows what first ABC is talking about. Kind of data versioning I guess.
@nightwatch: I guess standard replication of reference data and things like Master Data Management were too painful to solve. Maybe that’s the reason for Udi style SOA.
October 5th, 2011 at 5:55 am
@Sharas I re-read my original comment just now and I see that is does sound like I’m suggesting ABC 2 is responsible for calling ABC 1
I agree an ABC shouldn’t be querying another one, once you have all your ABC’s set up you will end up using them together in some sort of composite way. That’s the ultimate point I was trying to get across, the code running the saga to handle the “security pass” scenario would probably be a composite application with some high level business process that makes use of more than one ABC.
I suggested reading that book and thinking of it in terms of console apps, you can create a GUI that uses multiple console apps together to meet a business requirement but those individual apps wouldn’t make calls to each other (ideally).
October 6th, 2011 at 11:03 pm
@nightwatch – allow me to give you my personal take on some of this.
Taking an immutable copy of the data does help one ABC being more autonomous at runtime. Unfortunately it makes it a lot more structural coupled to the other ABC since it will have to know exactly how the other ABC’s data is structured.
Example:
1) ABC-1 stores customer data (fullname, address line 1, address line 2)
2) ABC-2 takes a copy of the customer data in order to show it on the screen.
3) Time goes by and some day ABC-1 decides to remove the fullname field and use two fields firstname + lastname instead.
4) Now ABC-2 has to update all of its customer name handling code. Had it only included a widget there would have been no problem.
By encapsulating not only data, but also displaying of the data, completely in an ABC, removes structural coupling and thereby improves maintaince. This is how you avoid ending up having a one-big-ball-of-mud code base. But I will agree that it seems to have a price at runtime.
If you don’t own an external system and cannot give it it’s own web-widgets, then its an integration issue and a composite UI won’t work. But … you could make the integration by building you own little integration widget in your own (new) ABC and use that one instead. In this way you prevent the integration “story” to creep into other ABCs.
/Jørn
October 7th, 2011 at 8:19 pm
Jørn, we have yet to see this concept work in a real application. I don’t believe it will. GUI is not everything, sometimes it could be built from independent widgets but it’s rarely the case.
October 7th, 2011 at 11:55 pm
Scooletz,
Let’s take a shopping cart for example. The ABC which owns which product IDs are in the shopping cart would emit them. Then, the ABC which owns the names of those products would get a callback (jQuery) that the product IDs were loaded and emit their names. Another ABC that owns the prices would do the same.
Finally, all of these would be composed into a table using Knockout.js as described here:
http://blog.hansenfreddy.com/2011/07/07/tabular-data-composition-in-the-browser-for-soa/
October 7th, 2011 at 11:59 pm
Arnon,
I’m good buddy – nice to see you chiming in here.
Since ABCs can have their components deployed to multiple physical places/systems, any history they maintain internally can be visualized everywhere.
While DW technology can be used *within* an ABC, that is an implementation detail.
Reporting is done by building composite UIs, or by exporting data to excel for ad-hoc / research-centric work.
Cheers.
October 8th, 2011 at 12:02 am
Kevin,
The ABC which receives the PersonNameChanged event can send a message to an integration endpoint to print a new badge. At that endpoint, all the necessary ABCs can be deployed where each emits its own information – just like a composite UI.
No need for sagas here.
October 8th, 2011 at 12:03 am
Sean,
Most organizations require the history – it could be that you’re only looking at the context of a single system/app. ABCs span system boundaries.
October 8th, 2011 at 1:09 am
nightwatch,
Applications aren’t services/ABCs – they are the result of deploying components from multiple services/ABCs to the same process. Therefore, an ABC doesn’t have its own webserver, but if hosted in a web server, can emit html/xml/json either as a part of a web page, or as a web-service.
You’re right that versioning is hard, so we want to avoid making it that much harder by having all sorts of copies of data without clear logical ownership. Note that within an ABC you can make as many physical copies of the data as you see fit, and structure them however you like. BTW, if relational databases are making matters more complicated – why use them?
It sounds like you’re looking for irrefutable proof as to why my statements are true (as they are quite unconventional). It is unlikely that this medium of blog posts and comments will have high enough bandwidth for that to happen. If you can make it to one of my courses, there you’ll have the proof that you’re looking for.
October 8th, 2011 at 1:11 am
Eben,
Since the printing of an order is a physical thing, we can have multiple ABCs take part in it. An ABC is not a system. The customer care ABC will be responsible for emitting the customer name on the printed order, as well as on the web site, as well as in the back-end CRM system.
October 10th, 2011 at 9:01 am
Udi, I’d love to take your course but currently have no venture capital for that. Hope this changes some day. BTW I don’t think I’m looking for any ‘irrefutable proof’ – I’m a software guy old enough to know that technologies are like religion – there are many of them but none can offer an irrefutable proof. However, some religions are better at solving your earthly problems than others and especially in enterprise software you need some gurus to lead you.
October 10th, 2011 at 8:45 pm
@Jørn,
“4) Now ABC-2 has to update all of its customer name handling code. Had it only included a widget there would have been no problem.”
I don’t understand this logic. If this was “standard” CQRS then ABC-2 would just have 1 (one) handler for that event. There is no “all of it’s customer name handling”!
The problem would be equally large (or small). It would just be located somewhere else. Instead of fixing a handler you would be fixing a widget. The only difference is that with a widget in ABC you *could* be using that same widget in multiple UI’s which *could* be an advantage. However it could also be a bad thing. For example, instead of that widget showing 1 line, it is now showing 2. I can’t imagine any UI that could accept that change.
–
Werner
October 11th, 2011 at 5:28 am
I understand how this works from a viewmodel perspective. What about how would one ABC interact with another to calculate a result?
Simple ex. We have an HR system. A payroll system needs information about the employee. pay rate, and address to calculate taxes. How would these work together if you aren’t sending data?
October 11th, 2011 at 8:13 am
@Udi: Regarding forever. The example you gave about invoices and user names. So if ABC1 has InvoiceId stored against UserId and user name is defined in ABC2. When user name changes it’s an upsert in ABC2 but it is still the same user with same id right?
Now when I want to show an invoice issued with old name in a composite app, how do I ask ABC2 to display previous name?
Or is the answer that ABC1 should store name if it’s concerned with history and immutability? It seems kinda complex in that case to decide which ABC needs or deserves to have the name most. Come to think of it, maybe it becomes arbitrary which one has it, since you can compose them in any way you like in composite apps. In that case it’s hard to reason about meaning and responsibilities of an ABC.
Good example would be much appreciated. Hope you’ll get a chance to answer.
October 11th, 2011 at 3:58 pm
Ah OK, I was thinking there was some offline process that needed to be done somewhere. I think I need to read this post a few more times along with all the comments
October 13th, 2011 at 12:09 pm
What do you think to use CQRS to separate writers from readers of my domain?
October 13th, 2011 at 7:15 pm
Hello Udi,
What are your thoughts on “Progressive Enhancement” in regards to the javascript and the browsers hat have it disabled (albeit a smaller number these days) ?
I really like the idea of composing the services (ABCs) on the client but some businesses stil wont accept some users not having a availability, which suggests composing the services (ABCs) on the server for some applications as the cost of maintaining both architectures would likely be to great. Would you still advocate this?
Also would a service encapsulate a collection of ABCs? or would an ABC represent a single service (the level at which the versioning and contract is agreed at)?
Many thanks, Lee
Do you think it is feasible to compose the
October 13th, 2011 at 11:38 pm
nightwatch,
I do offer discounts for people who don’t have their company paying for them.
October 14th, 2011 at 12:15 am
@Werner,
>> “4) Now ABC-2 has to update all of its customer name handling code. Had it only included a widget there would have been no problem.”
> I don’t understand this logic. If this was “standard” CQRS then ABC-2 would just have 1 (one) handler for that event. There is no “all of it’s customer name handling”!
Well, even though its only one handler, its still “all of its customer name handling”. Nothing wrong in that statement. The fact is that ABC-2 need to change its code because ABC-1 did. And maybe its only one handler – but it is also the UI code that displays the data from that handler.
> Instead of fixing a handler you would be fixing a widget.
> The problem would be equally large (or small).
> It would just be located somewhere else.
And that is in fact the whole point. It is only those who are responsible for the customer name that needs to modify their code – everybody else can happily go on with their job, totally ignorant of the fact that Customer Care just decided to change their data model.
> The only difference is that with a widget in ABC you *could* be using that same widget in multiple UI’s which *could* be an advantage.
Yes, that might, or might not, be an advantage, depending of how generic the widget is.
> However it could also be a bad thing. For example, instead of that widget showing 1 line, it is now showing 2. I can’t imagine any UI that could accept that change.
Yes, and if they suddenly decide that all addresses must include a photo of the person then it won’t work either. So ABC-1 is not allowed to break the layout without coordination – but it can at any time change its internal data model and code base without affecting others.
In short: a widget based approach reduces coupling but doesn’t eliminate coordination completely.
I do admit that the cost up front is bigger than with a more direct “copy-the-data” approach since you need to invent widget/plugin-mechanisms in various places where you would normally just hard code something. It’s a long term investment.
/Jørn
October 17th, 2011 at 12:30 am
@Steve Sheldon,
You wouldn’t use systems as ABCs – there would be one ABC responsible for the employee’s pay rate, and that would cut across both your HR and Payroll systems.
October 17th, 2011 at 2:35 am
Interesting post. This sounds quite similar to an approach we are experimenting with at work. I have an example of what I think would be two ABCs, where one would have to share data with the other, and I would like your take on it.
Take, for example, a service (X) for managing financial stocks and quote data; and a service (Y) for investment portfolio tracking and reporting.
In order to generate an investment performance report, data belonging to both X and Y would need to be combined and complex calculations performed. It would seem that either Y would need to query X for quote data and use it to do the calculations, or Y would supply X with its trade data and allow X to do the calculations (which seems weird, but for the sake of argument). I can’t seem to envision a design where Y would not be a client of X in some way, or some component (Z) compose the two and combine the data itself.
What do you think? Thanks!
October 17th, 2011 at 9:54 pm
Great blog post, great discussions. I’d like to point out that the discussion in the comments thus far has been heavily focused on the write side of things. I have a question on the read side. I’m having a hard time wrapping my mind around the concept of what seems to be in ABCs a grand unified set of ui composite widgets that work for every presentation scenario. The only way it makes sense to me right now is if I imagine an ABC for each field… Is that too granular? I don’t know, you use granular fields in your example, but was that just for the sake of example? Composing widgets together for any number of views suggests in my mind that I shift the complexity of “all that CQRS stuff” to some kind of universal presentation engine that glues together shape-shifting widgets. Explained in the abstract, it seems like I’d be normalizing many permutations in denormalized views of the data I’d be working with in each ABC. Yea, my head is spinning right now. Anyway, without going to one of your classes (which I’d sure like to do) could you tell me how close I am to the point?
October 17th, 2011 at 9:59 pm
(edit for above comment) ixnay the, “in each ABC” at the end of, “it seems like I’d be normalizing many permutations in denormalized views of the data I’d be working with (in each ABC)”
Guess I’m saying, it seems like I’d be normalizing all the little pieces of denormalized data… “Ok, so here’s a first name. I display it as a 12pt font here, it’s displayed in drop boxes on these views, and I let people edit it here…” etc…
October 21st, 2011 at 7:30 pm
Aaron,
Then maybe those aren’t the ABCs you need – maybe you need something that will cut across both those systems, that can then be composed back again.
October 21st, 2011 at 7:32 pm
Joshua,
How something gets presented (12pt font) is actually the responsibility of a different ABC.
October 31st, 2011 at 11:20 am
@Udi,
>”You wouldn’t use systems as ABCs – there would be one ABC responsible for the employee’s pay rate, and that would cut across both your HR and Payroll systems.”
Ok, I see where this is going. I like this concept, being modular, compositional and yet with some thought towards performance.
I have to think about this a bit in terms of how we’d manage things with development.
November 4th, 2011 at 12:57 am
Steve,
Yes, this really takes you way down the rabbit hole
November 20th, 2011 at 5:22 pm
I don’t see a lot of difference between coordinating widget changes with widget consumers versus coordinating public data representations such as service contracts or ETL packages with consumers that rely on them.
In either implementation the data owner is free to “at any time change its internal data model and code base without affecting others” as you suggest. It’s the public representation of that data that matters to consumers.
In my experience the internal data model of line of business systems do not change so often as to demand this sort of architecture. Rebranding, and UI changes are a far more common requirement and the widget approach dramatically breaks down here. It is similarly unsuited for corporate activity such as mergers or acquisitions as any new solution would require a massive data migration and system consolidation to align with this architecture.
Replication and data warehousing are more suitable solutions to these problems so it’s little wonder why they are ubiquitous and this approach isn’t.
November 21st, 2011 at 2:22 am
Sean,
In the “widget approach”, all the styling is kept separate from the widget itself so there isn’t a problem with rebranding. This architecture has similarly weathered mergers and acquisitions so I’m not exactly sure what your concerns are there.
December 7th, 2011 at 4:27 am
Maybe a silly question but is the VM in MVVM similar to an ABC?
December 7th, 2011 at 2:11 pm
Phil – no, the VM stands for View Model and wouldn’t be considered autonomous.
January 17th, 2012 at 6:28 am
Udi, in response to Araon you suggested that perhaps his ABCs are wrong. Could you expand on how he might deal with his example? Does the following sound sensible:
A Portfolio ABC that knows what investments a client has. A Stock ABC that knows the details of individual stocks (such as name and historical price). He might then have an Investment Tracking System that uses both of these ABCs to allow a client to maintain his portfolio and look at its current value. If this system should then provide some significant functionality such as the creation of a detailed performance report, where might that report be generated? Would a third ABC be introduced… Performance Report ABC?
If Performance Report ABC is required, would you expect the system to ask for a report based on a whole set of data that the system passes in (meaning the system needs to know what data is relevant to the Performance Report ABC) or would you expect the system to pass in a set of identifiers and expect the Performance Report ABC to retrieve the data it needs from the other ABCs (which would mean the Performance Report ABC is not so autonomous)…. OR does this linkage indicate you have not got the ABC divide correct?
January 17th, 2012 at 11:38 am
Piers,
There’s always a problem in dealing with a very shallow description of a problem domain. Details matter and should influence the breakdown.
More generally, reporting is usually handled in 2 ways. The first is via a composite UI (things usually not thought of as reports). The second is via a data dump to excel where users can do whatever they want with the data – ad-hoc reporting.
A third, and interesting alternative to reporting, is to think of what real world event users are trying to see by using reports and then make the system surface that explicitly.
January 18th, 2012 at 5:44 am
I agree Udi it is difficult to work with such shallow detail… so perhaps going shallower is an alternative
The question I have, which may not be Aaron’s, is:
Say you are analysing a business and you have identified two candidate ABCs that own distinct data and provide unique functionality that is reused by a number of systems (not necessarily the same set of systems). You then identify a third ABC, that would manage some data of its own, encapsulate significant business logic and could be used by another set of systems. However, you realise this new ABC consumes some of the data currently assigned to those other ABCs. So we appear to have one ABC that is reliant on the detailed data that is managed by other ABCs (not just the correlation ids exposed by the other ABCs). Is this situation symptomatic of a poor ABC split or quite normal?
If it is quite normal, have you any suggestions as to how the third ABC retains its autonomy? If it is only passed correlation Ids, then it will be reliant on the other ABCs in order to retrieve the detail it needs, or it could access their data stores directly or it could keep a read only copy of the data it needs. The alternative whereby, the systems using the new ABC have to pass in the extra detail when they want some function performed, means that those systems need to know what data to pass in.
In Araon’s case this could be a Portfolio ABC, Stocks ABC and a Financial Forecasting ABC where the Finacial Forecast ABC will consume the detail from a portfolio along with detailed stock market data, apply some complex business rules (such as tax treatments, growth assumptions etc. etc.) and generate a forecast.
August 23rd, 2012 at 3:52 pm
Udi, in #20, you said the ABC receiving the PersonNameChanged event sends to an integration endpoint. Why wouldn’t the PersonNameChanged event handler be deployed directly to the integration endpoint, which ultimately queries ABC1 (asks to emit, whatever) and commands ABC2? Why the extra middle-man? Either way, who owns the handler at the integration endpoint?
August 24th, 2012 at 4:52 am
Jeremy,
The reason to have an ABC subscribed to the PersonNameChanged event which then sends a PrintBadge command is that it is not the integration ABC’s responsibility to know that PersonNameChanged requires that a new badge be printed.