Scaling NServiceBus Sagas using the Observer Pattern

At NSBCon2014 in Brooklyn, NY, Jimmy Bogard gave an excellent presentation called Scaling NServiceBus. In the presentation, he showed the common missteps made by developers when first working with sagas, mainly, the missteps made around saga contention (start at 12:32 to see his presentation on saga contention).

After showing many refactoring steps to alleviate contention around sagas, he arrived at a saga style based on the Observer Pattern (based off the well-known Observer GoF design pattern).

McDonaldsLogo

To illustrate the usage of this design pattern in the context of a saga, he used ordering at McDonalds as an example. To skip to the McDonalds part, start at 28:00. To paraphrase Jimmy:

“When you order at McDonald’s, you get a receipt with the menu items in your order, and that order has a unique id. Physically, that order is represented by a tray. On that tray is another copy of the order receipt with the menu item(s) on it and the order id.”

“There are food prep stations for each item on your order (such as shake, fries, hamburger, etc…). Each food prep station has a worker(s) responsible for preparing the menu item for that particular food station. When the menu item is ready, a worker walks over to the tray (correlated by the order id on the receipt on the tray) and puts the finished menu item onto the tray. When they do this, they look at the finished items on the tray, look at the menu items on the order receipt, and if all the items are on the tray that are on the receipt, the worker will call the customer’s name on the receipt to tell them the order is ready (actually, in SOME McDonald’s they do this, in others, it’s up to the customer to check the items on their tray against the menu items on the receipt to see if the order is completed). That worker is now free to return to their station and start working on the next item they need to prepare.

An important point to note here the worker(s) for each menu item station have full autonomy to do their job. Each menu item station can be viewed as its own “boundary” doing what it needs to do, in the most efficient way to do it, in order to get it’s menu item prepared for a given order. A hold-up at the fries station (“can someone grab more fries from the freezer in the back?”) doesn’t result in a hold-up at the shake station (but will impact how long it will take for the full order to be ready).

Based on Jimmy’s presentation, along with my technical interpretation of it, I uploaded a project to GitHub that illustrates the way this could work using NServiceBus Sagas.

You can download the code here.


To start with, given the above explanation and the code interpretation, let’s take one more look at the definition of the observer pattern:
“Define a one-to-many dependency between objects so that when one object changes state, all its dependents are notified and updated automatically.”

Let’s take a look at the solution structure:

McDonaldsObserverPatternSolutionStructure

  • ConsoleClient: this is where you hit enter to “place an order”
  • Conventions: messages conventions are used so we don’t have to mark message classes with with ICommand, IEvent or IMessage.
  • FriesEndpoint: an endpoint that handles preparing fries.
  • Messages: all messages are in this project.
  • SagaEndpoint: this is the coordinator of the entire order process.
  • ShakeEndpoint: an endpoint that handles preparing shakes.

Let’s examine some of the projects more in-depth to get an idea of how the saga works.


ConsoleClient

class Program  
{  
    static void Main(string[] args)  
    {  
        ServiceBus.Init();  
        Console.WriteLine("Press ‘Enter’ to place an Order. To exit, Ctrl + C");  
        while (Console.ReadLine() != null)  
        {  
            var placeOrder = new PlaceOrder { OrderId = Guid.NewGuid(), 
                MenuItems = new List<string> { "Fries", "Shake" } };  
            ServiceBus.Bus.Send(placeOrder);  
        }  
    }  
}  

When you click enter you’re placing an order by sending the PlaceOrder command. On this command is the OrderId which is the unique identifier for this order. Also included are a list of menu items on your order. You can think of this command as representing the data that will end up on the receipt.

NOTE: I used a List for the order items because it’s the “simplest possible thing that could work” approach. There could be additional properties associated with each menu item that would affect how that menu item’s station prepares the item such as size (Small, Medium, Large). There are also properties associated with one menu item, but not another. For example, there could be a “flavor” property, such as chocolate or vanilla for a shake, but “flavor” makes no sense when applying that to fries. Each menu item could easily be a class with it’s own set of properties for the menu station, but for the sake of the example, I’ve kept with a List for each menu item.

Back to our example. In the ConsoleClient, when PlaceOrder is sent, it’s handled by the OrderSaga

OrderSaga

The OrderSaga is the coordinator of the order. The PlaceOrder messages starts the saga. In the handling method for PlaceOrder, we set the OrderId and the MenuItems in saga data. You can think of the OrderSaga as the physical tray that has the receipt on it with the OrderId (your unique order number) and the list of menu items in your order:

public void Handle(PlaceOrder message)  
{  
    Log.Warn(" order placed.");  
    Data.OrderId = message.OrderId;  
    Data.OrderList.AddRange(message.MenuItems);  
}  

After we set the saga data, the Saga iterates through the menu items and for each item in the order, it dispatches the appropriate command to prepare that particular menu item. For brevity, I’ve only include the ability to make two items, fries and a shake (my two favorite menu items at McDonalds).

foreach (var item in message.MenuItems)  
{  
    if (item == "Fries")  
        Bus.Send(new MakeFries { OrderId = message.OrderId });  
    if (item == "Shake")  
        Bus.Send(new MakeShake { OrderId = message.OrderId });  
}  

Each of these “make” commands is routed to the appropriate handler in the appropriate endpoint.

FriesEndpoint and ShakeEndpoint

Each endpoint contains a handler that handles it’s respective command. The endpoint represent the menu item station for that given command. Looking at MakeShakeHandler:

public class MakeShakeHandler : IHandleMessages<MakeShake>  
{  
    private readonly IBus bus;

    public MakeShakeHandler(IBus bus)  
    {  
        this.bus = bus;  
    }

    public void Handle(MakeShake message)  
    {  
        Thread.Sleep(3000); //3 seconds  
        bus.Reply(new ShakeCompleted { OrderId = message.OrderId });  
    }  
}  

I use a Thread.Sleep call to simulate making the menu item (I’m sure it takes more than 3 seconds to make a shake, but again, for the example, I’ve tried to keep things simple and short). When the MakeShakeHandler is done making the shake, it uses Bus.Reply to send ShakeCompleted back to the OrderSaga.

Back to OrderSaga

When ShakeCompleted is handled by the OrderSaga, it removes that menu item from the list of menu items on the order:

public void Handle(ShakeCompleted message)  
{  
    Log.Warn(" ShakeCompleted");  
    RemoveMenuItemFromOrderList("Shake");  
}  

Here is the code in the RemoveMenuItemFromOrderList method:

private void RemoveMenuItemFromOrderList(string menuItem)  
{  
    Log.Warn(string.Format(" removing menu item {0} from order list.", menuItem));  
    Data.OrderList.Remove(menuItem);  
    if (TheOrderIsComplete())  
        PublishOrderFinishedAndMarkSagaAsComplete();  
}  

Every time a menu item is removed from the order list in the saga, we check if the saga is done via the TheOrderIsComplete method. This method simply checks if there are any menu items still left in the list the saga saved when it first started:

private bool TheOrderIsComplete()  
{  
    return !Data.OrderList.Any();  
}  

If there are no more items in the order, then we publish OrderCompleted and mark the saga as complete.

The MakeFriesHandler (and any other menu item handlers) would work in the same way as the MakeShakeHandler.


When running all the handlers and the ClientConsole in the solution, you can see your order get placed in the OrderSaga console, the ShakeEndpoint and the FriesEnpoint handling the commands for their respective menu item, and track the reply back to the OrderSaga when each menu item finishes. You can also watch the saga mark itself as complete

RunningAllThreeHandlersAndWatchignTheSagaCoordinateYourOrder


In Summary

Since each menu item station is running in its own endpoint (and not all in the same endpoint), each menu item can be worked on independently in it’s own “boundary”, and then bus.replied back to the saga so the saga can remove that menu item from it’s list of menu items for the given order.

The saga is acting in the role of the observer. Instead of the saga “polling” to see if a particular menu item has been finished, and doing that for each menu item in the order every n seconds (because we all know that polling is evil), it’s up to the menu item stations to report back to the saga that they’re done. This way, the saga isn’t constantly “bugging” every menu item station with “are you done yet, are you done yet, are you done yet”…

Mission accomplished! We now have a saga that handles orders at McDonalds. Now go get yourself a happy meal 😉

Michael McCarthy

Read more posts by this author.