For instance, in an HR system, the process of interviewing candidates – wouldn’t you just insert, update, and delete these Appointment objects?

If I were to put on my domain-driven hat, I would describe those requirements differently – interview appointments have a lifecycle: proposed, accepted, cancelled, etc. It seems that only a user of the role HR Interviewer should be able to make appointments for themselves, so the service layer code would probably look something like this:

using (ISession session = SessionFactory.OpenSession())
using (ITransaction tx = session.BeginTransaction())
{
    ICandidateInterviewer interviewer = session.Get<ICandidateInterviewer>(message.InterviewerId);
    ICandidate candidate = session.Get<ICandidate>(message.CandidateId);

    interviewer.ScheduleInterviewWith(candidate).At(message.RequestedTime);
    tx.Commit();
}

The “ScheduleInterviewWith” method accepts an ICandidate and returns an IAppointment. IAppointment has a method “At” which accepts a DateTime parameter and returns void – just changes the data of the appointment. The state of the appointment at creation time would probably be proposed. The appointment object would probably be added to the list of appointments for that interviewer – that’s what will cause it to be persisted automatically.

Later, when the candidate accepts the meeting, we could have the following method on ICandidate – void Accept(IAppointment); that would obviously check that the candidate is the right person for that interview, the appointment’s current state (not cancelled), etc – finally updating its state. What part of this looks like create, update, delete? If that’s what your service layer to domain interaction looks like, do you now know what your messages will be looking like?CRUD seems to be what most of us are familiar with. Moving to domain-driven thinking takes time and practice, but is well worth it. Contrast this with a more traditional O/R mapping solution:

using (ISession session = SessionFactory.OpenSession())
using (ITransaction tx = session.BeginTransaction())
{
    ICandidateInterviewer interviewer = session.Get<ICandidateInterviewer>(message.InterviewerId);
    ICandidate candidate = session.Get<ICandidate>(message.CandidateId); 

    Appointment a = new Appointment(); 

    a.Interviewer = interviewer;
    interviewer.Appointments.Add(a); 

    a.Candidate = candidate;
    candidate.Appointments.Add(a); 

    a.Time = message.RequestedTime; 

    session.Save(a);  

    tx.Commit();
} 

As you can see, we’ve got simpler, more expressive, and more testable code when employing the domain model pattern, than using “just” O/R mapping. I’m not saying that the domain model pattern doesn’t need O/R mapping in the background for it to work. But that’s just it – the persistence gunk needs to be in the background and the business logic needs to be encapsulated.

So, while I’ll agree with Dave that the Domain Model is more lifestyle than pattern, I would argue against these conclusions:

If this post had a point, it’s only to share the idea that Domain Model is a big, big thing. It’s probably overkill in a lot of cases where you have simple applications that have very simple purposes.

As you just saw in the example above, there is no “overkill” to be seen. The domain model in the example wasn’t “a big, big thing”.

The domain model. Use it.

Why not have a better lifestyle?  

So, I’ve implemented the pattern in NHibernate, adding the following method to ISession:

As well as adding the following interface to the NHibernate package:

All this enables you to have a stronger separation between your service layer classes and your domain model class, as well as for you to express each service-level use case as a domain concept – an interface.

Once you have such an interface, you can create a fetching strategy for that use case and define exactly how deep of an object graph you want to load so that you only hit the DB once for that use case.

The nice thing is that its all configured with Spring. In other words, if you the entry for your fetching strategy class exists, you get the improved performance, if it doesn’t, you don’t. All without touching your service layer classes.

Just as an example, when I’m in the use case modeled by “ICustomer”, I want to get all the customer’s orders, and their orderlines. This would be done by having a class like this:

And the configuration would look like this (as a part of the regular spring template):

If you want to take a look at the full solution, you can find it here. For some reason, the combined file was too big for the upload on my blog so it’s split into two. Unzip both packages into the same directory. You’ll find a file called “db_scripts.sql” which contains the schema for the DB. Don’t forget to update your connection string in the “hibernate.cfg.xml”. If you’re looking for the changes I made to the NHibernate source, you can find it in the “Updated NHibernate Files” directory. The only real change is to the “SessionImpl.cs” file.

Relevant NHibernate and Spring binaries.

Source code of example.

BTW, there is some intelligent thread-safe caching going on in SessionImpl now so that you get a much smaller performance hit (in terms of code that uses reflection) on subsequent usages of the same interfaces.

Let me know what you think.

DevHawk (aka Harry Pierson) writes:

As you might guess then, I’m not a fan of Astoria. I believe the sweet spot for so called “data services” will be read only (because they don’t need transactions, natch). I’m sure there are some read/write scenarios Astoria will be useful for, but I think they will be limited – at least within the enterprise.

After catching up to Pablo Castro (the man behind Astoria) at DevTeach, and chatting with him and Tim Mall about Astoria, I heard something interesting. In order to provide business context for updates, you will be able to use something like a WS-Action header.

So, it’s based on REST, but for updates, it’s like WS-*. Hmm… Best of both worlds, or worst? What do you think?

I love you, Oren.

I know that the ADO.NET Entity Framework guys are open to this as well, but I’m pretty sure that the “Entity Model” thinking will hold them back. You just can’t divorce data and behavior – not when employing state machines or decorators.

I’m sold.

Hi Udi,

I’m enjoying the recent discussion on Entity Framework, disconnected clients, Unit of Work, and messaging. A few weeks ago I wrote a note to self to “Ask Udi” about the new ADO.NET Sync Services coming out of Microsoft, and this seems to be the perfect timing.

It really seems that Sync Services is meant to address the disconnected problem raised by Andres, but it appears to be primarily targeted at enabling DataSet-centric, grid-oriented UIs, not task-oriented UIs. They are spinning Sync Services as being an “SOA” way of doing data synchronization because “it uses WCF” and ” supports n-tier”. But under the hood it’s all about DataSet-style deltas. See Steve Lasker’s first Q&A for the SOA spin, and the rest of his blog for more info.

I’m curious what your thoughts are on the subject. Do you see Sync Services as a valid service-oriented solution for occasionally-connected clients, or is it just another attempt by the ADO.NET team to keep us all hooked on DataSet-driven development?

Oran

Get it via the Dr. Dobb’s site here.

Or download directly here.

Additional References

• Occasionally Connected Systems Architecture
• Entity Framework: Disconnected Problems & Solutions
• Tasks, Messages, & Transactions–the holy trinity

Want more? Go to the “Ask Udi” archives.

Andres: “Basically, there is no disconnected mode… if you plan to build a multi-tier application with ADO.NET Orcas in the middle tier, you will need to hack your own change tracking mechanism in the client, send the whole changeset, and apply it in the middle tier.”

Udi: “[In] the UI, … tasks often corresponded very well to the coarse-grained messages we employed in terms of SOA.”

Jesse: “I have to disagree with Andres and agree with Udi.”

(Sorry, I just couldn’t resist. Here’s the important part)

“Andres rightly points out that you don’t want 5 different methods for updating an order. Although you technically could do it that way, you probably don’t want to be ignoring the transactional context of the conversation or locking tables for significant time periods.”

And, “The backend will still have all the update methods you are used to using, so you’re not writing a bunch of extra code in the DB. However, this approach is far more powerful. One reason why is that now you have an operation defined and you can control everything about that operation. You know when the call comes in through that method that you have a customer service rep trying to update an order and can build in logic based off of that.”

Andres: “Now, we need to send those changes to the server, together, because I want them executed in a single transaction. I cannot have a service called ‘ModifyOrderCustomerInformation’, another ‘AddALineToOrder’ and ‘DeleteLineFromOrder’. I need an ‘UpdateOrder’ service. You can use a diffgram to do it, or you could build your own change-serialization mechanism.”

And, “The only way I see is to map the UI to the service interface, so the user can ‘Change Address’ or ‘Update Marital Status’ as different operations in the UI layer, but I can’t let the user ‘Modify the Customer’. It’s a lot of work, and I seriously doubt that the users will like it.”

OK, now we’re all set.

Just to get something small out of the way, every Human-Computer Interaction (HCI) professional I have worked with has been in favor of task-based UIs. Every user that I have met that has used both styles of UI, task based and “grid” based, has reported that they were more productive when using the task based UI for “interactive work”. Data entry is not interactive work, so grids might be more suitable there, although improvements in bulk-loading technology and OCR have decreased the amount of “plain” data entry that I’m encountering. It also may be that I’m working less on systems where data entry is of significant importance.

First of all let’s agree that writing business logic for something like “DeleteLineFromOrder” or “UpdateMaritalStatus” is easier than for “UpdateOrder”, given that it fit with the overall system design.

Second, let’s agree that the business logic for something generic like “UpdateOrder” is composed of the various specific cases like “DeleteLineFromOrder” and “AddLineToOrder”.

Third, let’s agree that if the UI was task based, it would be easy for the client side to activate the correct web service/method – seeing as there is a high correlation between the tasks and the service methods.

Therefore, the question becomes how do we do all the above specific work in a single transaction? One answer may be to make the service statefull and using something like WS-Atomic Transaction to tie the various calls together. The problems in this approach are already well known. Another solution uses a messaging paradigm, something I wrote about before.

If we represent each of the tasks not as “web methods”, but rather as messages we could send all these messages together in one envelope to the server, and have it process all of them in a single transaction, activating the specific business logic pieces one after the other.

I’ll be using a code-first approach to describe the solution instead of an XML-first one since it abstracts away the technology, but it is still a contract-first approach.

Each message type is just a class that implements the “IMessage” interface. So we’d have a class called “DeleteLineFromOrderMessage” which is a Data-Transfer Object (DTO) whose members/properties are the data needed for processing – in this case, the order Id and the order line number. The same would be true for the class “AddOrderLineMessage”, it would contain the order Id and some other member for the order line data quite probably a DTO itself (so that we can use it in other places as well).

The client would generate an instance of the appropriate message class as the user finishes each sub-task, saving them up. When the user would click “confirm”, the client would send all these message objects to the server in one go with this API,

void IBus.Send(params IMessage[] messages);

Like so:

myBus.Send(addOrderLineMsg, delOrderLineMsg);

The bus would take all these objects and wrap them in a single SOAP envelope, and send them to the server, probably on a transactional channel, but that’s a configuration issue. At the server side, the bus would activate the transaction (because of the transactional channel) and start dispatching each of the specific messages to its message handler, one at a time, in the same thread.

So as you can see, there is no “transactional conversation”, and therefore we’re not locking tables in between calls, because we’ve gotten rid of “in between”. It’s just like the generic update in terms of transaction time and network hops, just with specific, simple business logic.

Andres might retort to that, “Even if Udi is right, and that’s the way applications should be built, I doubt most mere mortals in Earth will want to do it.” In fact, he did J If the frameworks supporting this style of development were supplied by Microsoft, I’m sure that most developers wouldn’t have a problem with it. When provided with such a framework, every developer I’ve worked with said that they wouldn’t want to go back to the “old way” (would that make Orcas outdated?). I’ll be putting up my frameworks soon, as well as examples on how to use them, it’s just taking longer than I expected.

Before closing, I just wanted to address the points Andres raised about concurrency:

“You could also need to know the previous values for optimistic concurrency checkings.”

I’ve written about how to do this before in regards to Realistic Concurrency, and the best example I have in terms of code is Better Domain-Driven Design Implementation.

This has gone a little longer than I planned, but I still don’t think I’ve covered everything in enough depth. I’d most appreciate investigative questions to help me shed light on the murkier parts of this, either as comments, posts on your own blog, or even via email. Let’s continue the conversation.

Andres Aguiar points out an enormous change to the behavior of data access in tiered architectures that is coming with the ADO.NET Orcas Entity Framework. He sums it up nicely, “Basically, there is no disconnected mode.” He then goes on to talk about the ramifications of this:

“This basically means that if you plan to build a multi-tier application with ADO.NET Orcas in the middle tier, you will need to hack your own change tracking mechanism in the client, send the whole changeset, and apply it in the middle tier. From this point of view, it’s a huge step backwards, as that’s something we already have with DataSets today.”

I do quite a bit of work on large-scale distributed systems, some that could even be called “multi-tier”, but I really don’t like that term. I’ve got to tell you, I’m not that worried. I could say that it’s because I’m using an object/relational mapping tool, but then again, the Entity Framework is supposed to be that too. It also might be because I don’t use DataSets that much today anyway, but that begs Andres’ question – did I develop my own change-tracking mechanism on the client?

The answer lies in the second part of Andres’ question, I don’t “send the whole changeset, and apply it in the middle tier”. I have an entire podcast up on why I don’t think that that’s a good idea, specifically around Web Services and SOA, but it’s broadly applicable. However, the question remains, what do we do about the client side?

Just over a year ago, me and Arjen Poutsma (of Spring fame) had a discussion about this exact topic at the Software Architecture Workshop in beautiful Cortina, Italy. Here’s the important part:

I think it started by me saying that services should not expose CRUD style operations. Arjen countered by mentioning that most user interfaces in line-of-business applications exposed the same model to the user by having them fill out data in grids. My retort to that was that while humans can get used to almost anything as long as it’s consistent, that doesn’t mean that it is a good solution. In the systems that I work on there is usually an HCI (human-computer interaction) person on the project who designs the UI, mostly around the tasks they perform. These tasks often corresponded very well to the coarse-grained messages we employed in terms of SOA. We finally agreed that the successor of SOA would be TOA (Task-Oriented Architecture) in its aggregation of client-side aspects to the already server-centric principles of SOA.

As a concrete example, instead of updating customer data in a grid showing everything laid out flat, users would activate tasks like “Change address”, or “Update marital status”. The form shown would contain only the relevant data to be updated – no need to perform “change tracking”, it is perfectly clear what data needs to be sent. Furthermore, the intent behind the changed data would be sent as well – the server wouldn’t apply the data generically “in the middle tier”. Rather, the current customer data would be retrieved as a part of the customer object, and the appropriate method called upon it (ChangeAddress, or UpdateMaritalStatus) with the relevant data.

I actually view this change in the Entity Framework as a step forward, it’s like taking off the training wheels. I’m positive that most developers will be able to make the change and move to these more advanced architectures.

<WWF announcer voice>
Are you ready… to RUMBLE !?!?
</WWF announcer voice>

It was Datasets vs. O/R mapping, a slight twist on the infamous datasets vs. custom objects debate, all over again. They pulled me in, kicking and screaming, I swear, I really do. The lines were drawn, maintainability, performance, all the things that architects like to philosophize about in terms of other people’s work.

Anyway, I won’t give you the play-by-play ‘cause we were there almost all night. I’ll just cut to the chase.

First things first – any comparison of solutions without the context of a problem leads nowhere, fast, and stays there. So the first question I asked (when I got the chance to speak) was “are we talking about querying/reporting here?” and the answer was something like “well, yeah, but a lot of other things too”. So my suggestion was that we discuss the solutions in terms of two contexts – querying/reporting and OLTP.

What I mean by OLTP is the data-updating kind of work that you do on certain items. Examples of this include “insert order”, “change customer address”, and “discount product”. Querying/reporting doesn’t change data, and often involves dealing with large sets of data pulled from different kinds of entities (in ERD terms).

Luckily, my suggestion to deal with them separately was accepted. Secondly, I proposed that an object model (specifically implementing the Domain Model pattern) designed for OLTP would perform poorly when used for querying/reporting – simply because it wasn’t designed for it. The structure of a domain model is such that it makes it possible to define / implement business rules in one place. That’s possible, not easy.

Well, the dataset people weren’t going to just hand me the OLTP side of the equation without a fight, so they mentioned how easy it was to just “AcceptChanges”, and that my way was much more complex. My rebuttal came in the form of a question (are you seeing a pattern here?): Do you just swallow DbConcurrencyExceptions are do you throw all the user’s changes away when it happens? I didn’t quite make out the answer since there was a lot of mumbling going on, but I’m pretty sure they had one. I mean, you can’t develop multi-user systems using datasets without running into this situation.

The example that clinched OLTP was this. Two users perform a change to the same entity at the same time – one updates the customer’s marital status, the other changes their address. At the business level, there is no concurrency problem here. Both changes should go through. When using datasets, and those changes are bundled up with a bunch of other changes, and the whole snapshot is sent together from each user, you get a DbConcurrencyException. Like I said, I’m sure there’s a solution to it, I just haven’t heard it yet.

Now, here’s where things get interesting. I didn’t say that using a domain model automatically solves this problem. Rather, I described how each client could send a specific message, one a ChangeMaritalStatusMessage, the other a ChangeAddressMessage, to the server – in essence, giving the server the context in which each bit of data is relevant. The server could just open a transaction, get the customer object based on its Id, call a method on the customer (ChangeMaritalStatus or ChangeAddress), and commit the transaction. If two of these messages got to the server at the same time, the transactions would just cause them to be performed serially, and both transactions would succeed. The important part here is not losing the context of the changes.

When we talked about querying/reporting, things seemed quite a bit clearer. Datasets, or rather datatables seemed like a fine solution – most 3rd party controls support them out of the box. One guy mentioned that datasets performed poorly for large sets of data and that by designing custom entities for the result set, he could improve performance and memory utilization by, like, 70%. To tell you the truth, I think that if you need the performance, do it, if not, just use datasets. There isn’t much of an issue of correctness.

Just as an ending comment, in response to something someone said about scalability, I asked if they were reporting against the live OLTP data. The response was “yes”. Well, there’s a database scalability problem if I ever saw one. OLTP works most correctly when employing transactions that have an isolation level of serializable. The problem with them is that they lock up the whole table, or get blocked when a table scan is going on. Querying often results in a table scan. You can see the problem. Anyway, a common solution to this problem is to just reduce the isolation level, a quick fix that improves performance almost immediately. You take one hit in that your reports may be showing incorrect data, especially if they do aggregate type work. You might take another hit if your OLTP transactions need to do aggregate type work themselves. That second hit is pretty much unacceptable. A different solution is to accept the fact that the heaviest querying can usually show data that isn’t up to date up to the second.

In such a solution, you would have another database for reporting. It wouldn’t be just a replica of the OLTP database, but rather a lot more denormalized – which is a really not nice way of saying designed for reporting. You could then move the data from your OLTP database to the reporting database in some way (more to come on this topic) and you increase the scalability of your database. Just to define that a bit better – your OLTP database will be able to handle more transactions per unit of time, and reports will run faster, meaning that you will both improve their latency and the number of queries that can be handled per unit of time.

Anyway, I was pretty tired after all that, but if I had to sum it up I’d say something like this: before debating solutions, define the problem, you get a lot more insight into the solutions and you get it faster. That’s just win-win all around.

The overarching question is: what do you REALLY gain by O/R mapping?

(As an aside, among the comments of his post are those using the acronym ORM. Please stop – that acronym is already taken by Object Role Modeling.)

I’ve been using O/R mapping techniques on mission critical projects for some time now, and if I wanted to compare it to what I did before, it would not be to writing all the sql by hand. I don’t remember ever doing that – there was always code generation involved. Because, let’s face it – there’s a lot of drudgery involved for things that aren’t performance critical. No reason to do THAT by hand.

So, for me, the ONE THING that O/R mapping gives me better than what I did before is this:

O/R mapping gets me better object-oriented business logic.

That’s it.

Like Clemens said, if you “don’t know SQL and RDBMS technology in any reasonable depth”, don’t expect to get good performance. Obviously this is true for any technique. But I guess that empirically speaking, the percentage of people without said knowledge is larger in the group where you don’t HAVE to write sql.

So, I’ll bet you’re asking yourself, “if that’s all Udi gets from O/R mapping, why does he keep doing it?” Or maybe you’re asking yourself “should I get a beer? This is getting long…”

The fact of the matter is that I don’t know a better way to write business logic than by using OO techniques. I grant that data is important, but the reason that many applications are built is business logic – there’s something that this new system can/should do, that the old systems couldn’t (often using the same data).

If I could sum up my understanding of Clemens position, it would be this:

A lot of developers probably aren’t experienced, or knowledgeable enough the use O/R mapping well. Therefore writing the sql yourself is better.

While I agree with the first statement, and I think that the same could be said about communication, threading, .net and many other things, I don’t think that the conclusion logically follows.

So, I guess I would sum up my position like this:

If you would like to develop a persistent domain model, O/R mapping techniques will probably help you. If you would like your solution to perform well, you should probably learn how databases work, as well as what the O/R mapping tool does under the covers.

I just wanted to answer the question he asked in his post “BTW: one thing I’d really like in SQL server that I know Oracle has: Identities that are unique across multiple tables (I can’t remember the right word for that now ).” – why not just use GUIDs? There’s really great support for them in Sql 2k and in .net – and you get uniqueness not only across tables in a single DB, but globally (at least as close as you can do it today).

Just have as your primary key a column of type “uniqueidentifier” and give it a default value of NEWID() – this will take the column behaviour quite similar to IDENTITY – you don’t have to fill it in your code. When you need to take the data from the DB and into a data structure in your code, be it a dataset or custom class, all you need is System.Guid (a struct).

I’ve done some performance tests and it appears that guids are slower by a factor of 10-20% – but you do get global uniqueness, so it just might be worth the price, especially if you need replication capabilities. These numbers, of course, are by no means a valid benchmark. I suggest checking for yourself under loads that simulate your production environment.

One thing to note, for best performance on retrieval, it makes sense to add an identity column, not as a part of the primary key, but rather as your clustered index – keep the “uniqueidentifier” column as a non-clustered index. This makes a big difference when it comes to insertion times.

I’ve met a lot of people who haven’t thought that using GUIDs was so easy, and were left to suffer through replication with IDENTITY columns. I think it might be soon that the IDENTITY column will be replaced as the defacto standard for primary keys in tables. If you know better, well, don’t hesitate to prove me wrong

The post is a result of a yahoo search (people DO use Yahoo to search, apparently) for “how to serialize a datareader”. The searcher got here, but I’m pretty sure that they didn’t find the answer, seeing as I don’t usually delve to the technological depths.

But, because whenever I’m asked a question I can’t answer, I go find the answer, I thought I might as well have this blog do the same.

The answer to the question “how to serialize a datareader” is:

You can’t, although if you try hard enough I’m sure you’ll find a way, but you really shouldn’t.

The reason is quite simple.

1. The datareader is connected to your database. Unlike the dataset which holds a copy of the data, disconnected from your database, the datareader uses up database resources as long as you use it.

2. If you want to serialize the datareader you got back from calling ExecuteReader on a Command object, it probably means you need to pass it far, far away – at best, a different process, at worst, a different machine. Just think about how much longer it takes to make inter-process or inter-machine calls than normal in-process calls. It takes orders of magnitude longer.

3. (1+2) Do you really want to be taking up database resources for that long ? Probably not, it will so adversely affect the scalability of your system that no amount of tweaking will help.

So, only use datareaders for quickly getting data out of the database, in-process. If you want to pass the data to anyone else, make a copy of it. A dataset acts as a decent representation of the data. Personally, I prefer to loop over the rows in a datareader, making an entity object (like Customer) out of each row, and adding it to an entity collection (like CustomerCollection), and then passing that to whoever needs it.

To sum up, if .Net doesn’t provide you with something you think you need, AND, you don’t really understand .Net that well, it probably means you shouldn’t be trying to do it. Although that’s something of an over-generalizing-bordering-on-patronizing statement, I hope you won’t take it the wrong way and realize that the guys at Microsoft did put in a lot of thought on this platform, and if mistakes were made, then they’re trying to fix them as we speak. So, although it may feel a little awkward, try to adjust your way of work to the way that the platform guides you. (Yes, I realize that I don’t follow my own advice, but that’s only because I’ve found a better way )