public static IObservable<T> WithCachedSnapshot<T, TKey>(
    this IObservable<T> source,
    Func<T, TKey> keySelector)
{
    var cache = new Dictionary<TKey, T>();
    return Observable.Defer(() => {
            lock (cache)
                return cache.Values.ToList().ToObservable();
        })
        .Concat(source.Do(s => {
            lock (cache) cache[keySelector(s)] = s;
        }));
}

If you can’t take my word for it, here is the proof that it works:

[TestMethod()]
public void CacheByKeyTestHelper()
{
    // Arrange
    TestScheduler scheduler = new TestScheduler();
    var source = scheduler.CreateHotObservable(
        OnNext(100, new { Ticker = "MSFT", Price = 30.5 }),
        OnNext(101, new { Ticker = "IBM", Price = 130.2 }),
        OnNext(102, new { Ticker = "MSFT", Price = 31.5 }),
        OnNext(103, new { Ticker = "IBM", Price = 130.1 }),
        OnNext(104, new { Ticker = "AAPL", Price = 150.5 }));

    // Act
    var target = RxExtensions.WithCachedSnapshot(
        source,
        quote => quote.Ticker);
    var results = scheduler.Run(() => target, 0, 0, 105);
    var cachedResults = scheduler.Run(() => target, 0, 0, 200);

    // Assert
    results.AssertEqual(
        OnNext(100, new { Ticker = "MSFT", Price = 30.5 }),
        OnNext(101, new { Ticker = "IBM", Price = 130.2 }),
        OnNext(102, new { Ticker = "MSFT", Price = 31.5 }),
        OnNext(103, new { Ticker = "IBM", Price = 130.1 }),
        OnNext(104, new { Ticker = "AAPL", Price = 150.5 }));

    cachedResults.AssertEqual(
        OnNext(105, new { Ticker = "MSFT", Price = 31.5 }),
        OnNext(105, new { Ticker = "IBM", Price = 130.1 }),
        OnNext(105, new { Ticker = "AAPL", Price = 150.5 }));
}

private Recorded<Notification<T>> OnNext<T>(
    long ticks, T value)
{
    return new Recorded<Notification<T>>(
        ticks,
        new Notification<T>.OnNext(value));
}

// Daily low price feed
double min = double.MaxValue;
var feedLo = feed
    .Where(p => p < min)
    .Do(p => min = Math.Min(min, p))
    .Select(p => "New LO: " + p);

This implementation depends on the external (to the framework) state (current low value stored in “min” local variable), “Do” combinator to continuously update this state with every price tick, and a “Where” combinator to suppress the ticks that do not represent running low value. Turns out, there is a better, more elegant and framework friendly way to implement this. It relies on the built-in “Scan” and “HoldUntilChanged” combinators:

// Daily low price feed
var feedLo = feed
    .Scan(Math.Min)
    .HoldUntilChanged()
    .Select(p => "New LO: " + p);

The code above is functionally equivalent to the original one but is cleaner, more compact, and easier to read. This is how it works:

• “Scan” operator is used to accumulate the running min value. “Scan” is similar to “Aggregate” in the LINQ-to-Objects world but, instead of collapsing the sequence to a single accumulated value, it keeps outputting the current accumulated value with every tick. As a result “Math.Min” function is continuously applied with every new tick with accumulated min value and new price as arguments and its result is “ticked out”. This produces a stream where every successive number is either less than or equal to the previous one. Close to what we need except we want to suppress duplicates.
• “HoldUntilChanged” is another built-in combinator that wraps an underlying stream and only repeats a value from it if this value differs from the previous one. Perfect for removing successive duplicates (please, note, it will NOT remove ALL the duplicates, only successive ones, so it is not a proper substitute to Distinct, when such a combinator exists) . We use “HoldUntilChanged” instead of original “Where” clause.

Here is the full code:

// simulate market data
var rnd = new Random();
var feed = Observable.Defer(() =>
    Observable.Return(Math.Round(30.0 + rnd.NextDouble(), 2))
    .Delay(TimeSpan.FromSeconds(1 * rnd.NextDouble())))
    .Repeat();

// Daily low price feed
var feedLo = feed
    .Scan(Math.Min)
    .HoldUntilChanged()
    .Select(p => "New LO: " + p);

// Daily high price feed
var feedHi = feed
    .Scan(Math.Max)
    .HoldUntilChanged()
    .Select(p => "New HI: " + p);

// Combine hi and lo in one feed and subscribe to it
feedLo.Merge(feedHi).Subscribe(Console.WriteLine);

Console.ReadKey(true);

Happy holidays!

This article is one from the “Processing Financial Data with Microsoft Reactive Framework (Reactive Extensions for .NET)” series of articles that demonstrate some basic applications of Rx framework to financial data. This article shows how to calculate a running VWAP (volume weighted average price over a defined window of time). Other articles in the series are:

• Detecting daily Hi/Lo prices
• Throttling high-frequency data feed
• Data feed failover

Visual Studio 2008 solution containing full source code for all examples can be downloaded here.

Calculating VWAP/Sliding Window Implementation

We will use this simple “Trade” type to represent a trade event. To simplify things we will only consider Price and Volume:

class Trade
{
    public double Price { get; set; }
    public int Volume { get; set; }
}

The following “TradeWithVWAP” type wraps underlying trade and attaches running VWAP value to it. It is used for calculation output:

class TradeWithVWAP
{
    public Trade Trade { get; set; }
    public double VWAP { get; set; }
}

Our main module starts with a simple market data generator that produces infinite stream of random prices ($30 to $33 range) at random volumes (1,000 to 100,000 shares) at a random frequency (0 to 3 seconds between trades). BuildVwapFeed functions takes generated trade stream and converts it to another stream with running VWAP attached to every trade. And finally we simply subscribe to VWAP-wrapped feed and output the results.

static void Main(string[] args)
{
    // Market data simulator
    var rnd = new Random();
    IObservable<Trade> feed =
        Observable.Defer(() =>
            Observable.Return(new Trade
            {
                Price = Math.Round(30.0 + 3 * rnd.NextDouble(), 2),
                Volume = 1000 + rnd.Next(0, 100000) / 100 * 100
            })
            .Delay(TimeSpan.FromSeconds(3 * rnd.NextDouble())))
        .Repeat();

    // Build a wrapper feed to add running VWAP to each trade
    IObservable<TradeWithVWAP> vwap = BuildVwapFeed(
        feed,
        TimeSpan.FromSeconds(5));

    // Subsribe and print results
    vwap.Subscribe(f => Console.WriteLine(
        "Price={0}, Volume={1:#,##0}, VWAP={2}",
        f.Trade.Price,
        f.Trade.Volume,
        f.VWAP));
    Console.ReadKey(true);
}

“BuildVwapFeed” takes two parameters: underlying feed and desired sliding window for VWAP calculation. The first thing it does is time stamping the incoming feed. This is to keep track of an individual trade position within a sliding window. Time stamping is built into Reactive Framework and is as simple as applying a “Timestamp” operator. Timestamp<T> takes a stream of type T and converts it to a stream of Timestampted<T>. Then we allocate a list of Timestampted<T> – this is where we keep the trades currently inside our sliding window.

The “Do” operator is used to maintain the sliding window. The “Do” operator simply repeats the stream it wraps but can produce arbitrary side effects. We use the operator to add new trades to sliding window and to remove trades that have expired.

After the sliding window is updated we no longer need the timestamp. We get rid of it using another Rx built-in operator: RemoveTimestamp<T>. The last code block transforms incoming feed into “VWAP-stampted” by using the sliding window to calculate and attach running VWAP to each trade:

private static IObservable<TradeWithVWAP> BuildVwapFeed(
    IObservable<Trade> feed,
    TimeSpan slidingWndLen)
{
    // Timestamp original feed
    var timestampted = feed.Timestamp();

    // Maintain a sliding window of recent trades
    var wnd = new List<Timestamped<Trade>>();
    var slidingWindowFeed = timestampted.Do(f =>
    {
        // remove expired trades
        while (
            wnd.Count > 0 &&
            f.Timestamp - wnd.First().Timestamp > slidingWndLen)
        {
            wnd.RemoveAt(0);
        }
        // add newest trade
        wnd.Add(f);
    });

    // Calculate running VWAP from trades in the sliding window
    var vwap =
        from w in slidingWindowFeed.RemoveTimestamp()
        let windowVolume = wnd.Sum(l => l.Value.Volume)
        let windowTotal = wnd.Sum(
            l => l.Value.Volume * l.Value.Price)
        select new TradeWithVWAP
        {
            Trade = w,
            VWAP = Math.Round(windowTotal / windowVolume, 2)
        };
    return vwap;
}

There are many other ways to produce VWAP on-the-fly. The implementation shown does not claim to be very efficient. But it does demonstrate how easy, almost trivial, it is to implement non-trivial stream transformations with Rx. Building your own CEP engine may soon become as easy as LINQ, Rx, and .NET…

This article is one from the “Processing Financial Data with Microsoft Reactive Framework (Reactive Extensions for .NET)” series of articles that demonstrate some basic applications of Rx framework to financial data streams. This article demonstrates automatic and transparent failover from primary data feed to a backup one implemented in terms of Microsoft Reactive Framework. Other articles in the series are:

• Detecting daily Hi/Lo prices
• Throttling high-frequency data feed
• Running VWAP calculation

Visual Studio 2008 solution containing full source code for all examples in this article can be downloaded here.

Failover Implementation

The effect of automatic failover is implemented using the “Catch” operator built into the Rx framework.  The “Catch” operator simply repeats the first stream until it fails. When the first stream fails, exception is swallowed and the second stream is subscribed to and repeated. You can chain any number of streams this way. Here is a code sample that uses “Catch” to switch from some primary feed to a backup:

// simulate market data feed
var rnd = new Random();
var feedSimulator = Observable.Defer(() =>
    Observable.Return(Math.Round(30.0 + rnd.NextDouble(), 2))
    .Delay(TimeSpan.FromSeconds(rnd.NextDouble())))
    .Repeat();

// Simulate fatal problem with feed 5 seconds down the road
var feedProblem =
    Observable.Throw<double>(new Exception("Fatal Error!"))
    .Delay(TimeSpan.FromSeconds(5));

// Bloomberg feed: merging simulator with error generator
var bloombergFeed = from p in feedSimulator.Merge(feedProblem)
                    select new { Feed = "Bloomberg", Price = p };
// Reuters feed
var reutersFeed = from p in feedSimulator
                  select new { Feed = "Reuters", Price = p };

// Automatic, transparent failover feed
var failoverFeed = bloombergFeed.Catch(reutersFeed);

// Subscribe and print results to console
bloombergFeed.Subscribe(
    p => { },
    x => Console.WriteLine("Bloomberg feed: {0}", x.Message));
failoverFeed.Subscribe(
    price => Console.WriteLine(price));

Console.ReadKey(true);

The first code block is a simple market data simulator implemented using Reactive Framework.

The second code block simulates a feed problem. It never produces any values and, when subscribed to, throws an exception after 5 seconds. We will combine market data simulator with problem simulator to simulate a feed that breaks down the road.

The third and forth code blocks create two feeds: Bloomberg and Reuters. Bloomberg feed is constructed to “break” after 5 seconds and Reuters is simply based on our market simulator.

Next code block is “var failoverFeed = bloombergFeed.Catch(reutersFeed);”. This statement creates a wrapper feed that starts with Bloomberg and when it fails automatically and transparently switches to Reuters. A subscriber will not notice an interruption. Notice that Reuters feed is not going to be subscribed to until failover happens.

This article is one from the “Processing Financial Data with Microsoft Reactive Framework (Reactive Extensions for .NET)” series of articles that demonstrate some basic applications of Rx framework to financial data. This article shows how to throttle your incoming feed to a desired frequency. Other articles in the series are:

• Detecting daily Hi/Lo prices
• Data feed failover
• Running VWAP calculation

Visual Studio 2008 solution containing full source code for all examples can be downloaded here.

Throttling Implementation

The key operator here is “Sample”. Sampling at a desired rate effectively returns your a throttled stream. My intuition was that if underlying feed does not produce a value during the sampling interval, “Sample” will repeat the most recent available value when sampling is due. But it doesn’t and this makes it a perfect fit for throttling. Note that there is also an operator called “Throttle” but its name is misleading and its behavior most likely will not match your definition of throttling. The way implemented it will not emit any values at all if underlying feed produces faster than throttling interval, so it looks more like “Choke” to me.

Lets look at the following code sample:

// simulate very fast (no delays) market data feed
var rnd = new Random();
var unthrottledFeed = Observable.Defer(() =>
    Observable.Return(30.0)
    .Select(p => Math.Round(p + rnd.NextDouble(), 2))
    .Repeat());

// running unthottled feed as a demo
ManualResetEvent unthrottledDone = new ManualResetEvent(false);
int count = 0;
unthrottledFeed
    .Do(p => ++count)
    .Until(Observable.Return("Done unthrottled feed")
    .Delay(TimeSpan.FromSeconds(5)))
    .Subscribe(Console.WriteLine, () => unthrottledDone.Set());
unthrottledDone.WaitOne();
Console.WriteLine(
    "\nDone unthrottled feed. Avg rate: {0} values per second",
    count / 5);

// running same feed, now throttled
Console.WriteLine("Now throttlig at 1 value per second...\n");
var throttledFeed = unthrottledFeed.Sample(TimeSpan.FromSeconds(1));
throttledFeed.Subscribe(Console.WriteLine);

Console.ReadKey(true);

The first code block is a simple fast feed simulator. It produces the values as fast as you CPU allows, or about 3,000 per second (with printing to the console) on my laptop. Most UI data binding engines will not be happy to accommodate this rate.

Second code block runs non-throttled feed for about 5 seconds just so you can get a feel that frequency is pretty high.

Third code block wraps non-throttled feed with a “Sample” operator with one second sampling interval. As discussed above current implementation of “Sample” is effectively throttling. When you subscribe to the throttled feed values are emitted with frequency of one value per second, with any values in-between dropped.

This article is one from the “Processing Financial Data with Microsoft Reactive Framework (Reactive Extensions for .NET)” series of articles that demonstrate some basic applications of Rx framework to financial data. This article focuses on detecting running high/low values (like daily max and min) from a market data feed. Other articles in the series are:

• Throttling high-frequency data feed
• Data feed failover
• Running VWAP calculation

Visual Studio 2008 solution containing full source code for all examples can be downloaded here.

Detecting Running High/Low Prices

The key operators in this implementation are “Do” and “Where”, both built into the Reactive Framework. The first one is used to produce side effect of recording the running High or Low value, and the second is used to filter incoming values that do not produce newest High or Low:

// simulate market data
var rnd = new Random();
var feed = Observable.Defer(() =>
    Observable.Return(Math.Round(30.0 + rnd.NextDouble(), 2))
    .Delay(TimeSpan.FromSeconds(1 * rnd.NextDouble())))
    .Repeat();

// Daily low price feed
double min = double.MaxValue;
var feedLo = feed
    .Where(p => p < min)
    .Do(p => min = Math.Min(min, p))
    .Select(p => "New LO: " + p);

// Daily high price feed
double max = double.MinValue;
var feedHi = feed
    .Where(p => p > max)
    .Do(p => max = Math.Max(max, p))
    .Select(p => "New HI: " + p);

// Combine hi and lo in one feed and subscribe to it
feedLo.Merge(feedHi).Subscribe(Console.WriteLine);

Console.ReadKey(true);

The first block of code is a simple market data feed simulator. It generates random prices in the $30-$31 range at random intervals of zero to one second. Market simulator itself is written using Rx (all in one statement, isn’t this cool?).

The second block of code extracts ticks from the base feed only if they represent running lowest prices received from the feed. This is achieved by composition of the following:

• base stream of prices
• “Where” clause to filter away ticks that are greater or equal to the current running minimum price
• “Do” operator to record the current minimum value; “Do” operator simply wraps underlying stream and repeats the values it receives; its power comes from being able to execute arbitrary code per each received value and thus produce side effects (recording min value in our case)
• “Select” projection is simply used here to label the values with “New LO: “ prefix for presentational purposes

The third block of code does the same for capturing running High values.

At the end we merge both feeds into one and subscribe to it.

If you run the code you should see something similar to this run:

The feed slowly but surely converges to $30 for low and $31 for high (limits of our feed generator). Once extreme  values are reached, no additional ticks will come out.

I have recently discovered Microsoft Reactive Framework (thanks to spontaneous introduction at New York .Net Meetup Group) and have been playing with it a lot trying to apply it to many financial problems I’ve coded over the last years. I am truly amazed with the technology and the power it unleashes for processing and analyzing streaming financial data (like quotes, market data feeds, trade executions, etc.). Seamless integration with LINQ combined with very powerful support for composition makes self coded CEP in .Net a low hanging fruit. In the following series of articles I am going to demonstrate sample Rx-based implementations for some commonly occurring tasks in finance, including:

• Detecting daily Hi/Lo prices
• Throttling high-frequency data feed
• Data feed failover
• Running VWAP calculation

Visual Studio 2008 solution containing full source code for all examples can be downloaded here.

MVVM is a proven design pattern in WFP/Silverlight world. Its use helps achieve separation of concerns, frees UI from any logic to enhance testing, and generally makes the code easier to understand and maintain. In many LOB multi-tier applications MVVM is extended to include a presenter class. Presenter usually controls the view model, receives necessary services from dependency injection container and has some logic to populate the view model with business entities and/or reference data and send any changes back. The resulting code is usually pretty clean, easy to maintain and test. This article argues though that most of the code in such presenters to populate the view models is also very mechanical, routine, boring and can be avoided all together by introducing small attribute-based view model framework to declaratively populate view models from service calls and avoid extra coding, and, thus, avoid writing extra mechanical, routine, and boring tests.

If you prefer the “code first” approach VS 2008 solution can be downloaded here: MVVMAttributes.zip

The Base Line

Consider the following simplified application as our baseline: one of the application’s views is the “CustomerView” which gets customer information from the customer service, lets user edit the information (name and state of residency in our example) and allows saving the data back (omitted in our example). The following is the baseline Prism/MVVM implementation:

Business Entities

namespace MVVMAttributes.BusinessObjects
{
    public class CustomerData
    {
        public string Name { get; set; }
        public string State { get; set; }
    }
}

This class represents very simple transfer object to pass customer info between tiers.

View Model

public class CustomerViewModel : DependencyObject
{
    private readonly ObservableCollection<string> _states
        = new ObservableCollection<string>();

    // Customer model DP wrapper
    public CustomerData Customer
    {
        get { return (CustomerData)GetValue(CustomerProperty); }
        set { SetValue(CustomerProperty, value); }
    }
    public static readonly DependencyProperty CustomerProperty =
        DependencyProperty.Register("Customer",
        typeof(CustomerData), typeof(CustomerViewModel));

    // Observable list of states
    public ObservableCollection<string> States
    {
        get { return _states; }
    }
}

This is a typical view model that encapsulates business model (Customer property) and reference data (States observable collection to present the user with the list of all states to pick from).

View

<UserControl
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    x:Class="MVVMAttributes.Customer.CustomerView"
>
    <Grid>
        <Grid.RowDefinitions>
            <RowDefinition Height="Auto"/>
            <RowDefinition Height="Auto"/>
        </Grid.RowDefinitions>
        <Grid.ColumnDefinitions>
            <ColumnDefinition Width="Auto"/>
            <ColumnDefinition Width="200"/>
        </Grid.ColumnDefinitions>
        <TextBlock Text="First Name:"/>
        <TextBlock Text="State:" Grid.Row="1"/>
        <TextBox Text="{Binding Customer.Name}"
                 Grid.Column="1"/>
        <ComboBox Grid.Row="1" Grid.Column="1"
                  SelectedItem="{Binding Customer.State}"
                  ItemsSource="{Binding States}"/>
    </Grid>
</UserControl>

The view binds its view model to the UI elements. In particular, it binds customer name to a text box (Text property) and customer’s state to a ComboBox (SelectedItem property). It also binds ComboBox’s ItemsSource to the full list of available states exposed by the view model. View contains no additional code (except implementing ICustomerView) so code behind file is not shown. This is ICustomerView interface:

namespace MVVMAttributes.Customer
{
    public interface ICustomerView
    {
        object DataContext { get; set; }
    }
}

Presenter

Presenter code is below. Code walk-through follows.

namespace MVVMAttributes.Customer
{
    public class CustomerPresenterWithServices
    {
        private readonly ICustomerView _view;
        private readonly ICustomerService _service;
        private readonly IReferenceDataService _refDataService;
        private CustomerViewModel _viewModel;

        public CustomerPresenterWithServices(
            ICustomerView view,
            ICustomerService service,
            IReferenceDataService refDataService)
        {
            _view = view;
            _service = service;
            _refDataService = refDataService;
        }

        public void ShowCustomer(int customerId)
        {
            _viewModel = new CustomerViewModel();
            _viewModel.Customer = _service.GetCustomer(customerId);
            ReplaceList(
                _viewModel.States,
                _refDataService.GetStates());

            _view.DataContext = _viewModel;
        }

        private void ReplaceList(
            ObservableCollection<string> list,
            string[] values)
        {
            list.Clear();
            foreach (var item in values)
            {
                list.Add(item);
            }
        }
    }
}

Presenter implementation is typical for MVVM designs with dependency injection frameworks. Constructor injects and caches the view, customer service, and reference data service. The ShowCustomer public method creates the view model, calls services to get domain model (customer data) and reference data. When view model is built up, it is assigned to view’s DataContext for data binding to take over.

As you can see the wiring code to cache services, call them and update the view model is nothing fancy, highly repetitive code, with a risk of being “cut-and-paste” in high volume in a real application with dozens of views/presenters and maybe hundreds of service calls. Now add a requirement to populate the view model asynchronously and the wiring code to do that will likely be repeated (and thus need to be tested) in all those presenters. This is where the view model framework I introduced in the abstract comes to rescue.

The View Model Attribute-based Framework

Attributes

Lets introduce the following two attributes that constitute the core of the declarative part of the framework:

namespace MVVMAttributes.Infrastructure
{
    public abstract class ServiceCallAttribute : Attribute
    {
        public Type ServiceType { get; set; }
        public string MethodName { get; set; }
        public string MethodParameters { get; set; }
    }

    [AttributeUsage(
        AttributeTargets.Property,
        AllowMultiple = false)]
    public class ServiceEntityAttribute : ServiceCallAttribute
    {
    }

    [AttributeUsage(
        AttributeTargets.Property,
        AllowMultiple = false)]
    public class ServiceListAttribute : ServiceCallAttribute
    {
    }
}

Two attributes (ServiceEntityAttribute and ServiceListAttribute) share the same base class that defines the type of the service to use, the method to call, and, if method has any parameters, the names to use to resolve the parameters values.

Attributed View Model

Now when we have the attributes we can amend our original model to declaratively specify how to get data to populate it:

namespace MVVMAttributes.Customer
{
    public class CustomerViewModel : DependencyObject
    {
        private readonly ObservableCollection<string> _states
            = new ObservableCollection<string>();

        // Customer model DP wrapper
        [ServiceEntity(
            ServiceType = typeof(ICustomerService),
            MethodName = "GetCustomer",
            MethodParameters = "customerId")]
        public CustomerData Customer
        {
            get { return (CustomerData)GetValue(CustomerProperty); }
            set { SetValue(CustomerProperty, value); }
        }
        public static readonly DependencyProperty CustomerProperty =
            DependencyProperty.Register(
            "Customer",
            typeof(CustomerData), typeof(CustomerViewModel));

        // Observable list of states
        [ServiceList(
            ServiceType = typeof(IReferenceDataService),
            MethodName = "GetStates")]
        public ObservableCollection<string> States {
            get { return _states; }
        }
    }
}

The “ServiceEntity” attribute on the “Customer” property specifies that this property should be populated by call to “GetCustomer” method on the “ICustomerService” service. The method expects one parameter and the caller will resolve its value using “customerId” name.

The “ServiceList” attribute labels the “States” property and states that this collection should be filled in using the “GetStates” method of the “IReferenceDataService” service. The method expects no parameters.

Now that view model is tagged with “service call” attributes we can use some generic factory to create and populate it. Once the code to populate view model is moved to a generic factory, our original presenter simplifies to this:

public class CustomerPresenter
{
    private readonly ICustomerView _view;
    private CustomerViewModel _viewModel;
    private readonly IViewModelFactory _viewModelFactory;

    public ICustomerView View { get { return _view; } }

    public CustomerPresenter(
        ICustomerView view,
        IViewModelFactory viewModelFactory)
    {
        _view = view;
        _viewModelFactory = viewModelFactory;
    }

    public void ShowCustomer(int customerId)
    {
        _viewModel =
            _viewModelFactory.BuildViewModel<CustomerViewModel>(
            new Dictionary<string, object> {
            { "customerId", customerId } });

        _view.DataContext = _viewModel;
    }
}

The presenter still injects the view but is no longer concerned with services and populating of its view model. Instead, it injects a generic view model factory to create the view model and “build it up” with service calls.

The magic behind it all – View Model Factory

The last and most interesting part of the framework is view model factory. It creates a view model, inspects its properties and their attributes using reflection, dynamically resolves necessary services, calls required methods and updates the view model. Below is the code and walk-though:

namespace MVVMAttributes.Infrastructure
{
    public interface IViewModelFactory
    {
        TViewModel BuildViewModel<TViewModel>(
            IDictionary<string, object> parameters);
    }
}

The above declares the public API for the view model factory. The view model is requested by specifying BuildViewModel’s generic type, and the “parameters” dictionary is used to resolve values for any parameters for the service calls necessary. Here is the implementation itself (partitioned to facilitate the walk-through):

namespace MVVMAttributes.Infrastructure
{
    public class ViewModelFactory : IViewModelFactory
    {
        private readonly IUnityContainer _container;

        public ViewModelFactory(IUnityContainer container)
        {
            _container = container;
        }

        public TViewModel BuildViewModel<TViewModel>(
            IDictionary<string, object> parameters)
        {
            TViewModel viewModel = _container.Resolve<TViewModel>();

            BuildUp<ServiceEntityAttribute>(
                viewModel, parameters, SetEntity);
            BuildUp<ServiceListAttribute>(
                viewModel, parameters, ReplaceList); 

            return viewModel;
        }

…

ViewModelFactory’s constructor injects the container, so it later can resolve requested services. The “BuildViewModel” method constructs the view model (again using the container) and “builds it up” by scanning for “ServiceEntityAttribute” first to populate “entities” and then for “ServiceListAttribute” to populate collection type data. Here is the code for “BuildUp<TAttribute>” method:

private void BuildUp<TAttribute>(
    object viewModel,
    IDictionary<string, object> parameterLookup,
    Action<PropertyInfo, object, object> setProperty)
    where TAttribute : ServiceCallAttribute
{
    var q =
        from p in viewModel.GetType().GetProperties()
        from a in p.GetCustomAttributes(typeof(TAttribute), false)
        select new { Property = p, Attribute = (TAttribute)a };

    foreach (var item in q)
    {
        object result = CallService(
            item.Attribute, item.Property, parameterLookup);
        setProperty(item.Property, viewModel, result);
    }
}

A simple LINQ to reflection query is used to scan for properties marked for auto-population with service calls. For each found property specified service call is made by “CallService” and then view model is updated using one of the two “setProperty” strategies: property assignment or list replacement. Here is the “CallService” implementation:

private object CallService(
    ServiceCallAttribute attribute,
    PropertyInfo property,
    IDictionary<string, object> parameterLookup)
{
    Type serviceType = attribute.ServiceType;
    object service = _container.Resolve(serviceType);
    MethodInfo method = serviceType.GetMethod(attribute.MethodName);
    string[] parameterNames = attribute.MethodParameters != null ?
        attribute.MethodParameters.Split(',') :
        new string[] { };
    object[] parameters = parameterNames.Select(
        name => parameterLookup[name]).ToArray();

    return method.Invoke(service, parameters);
}

Here the service specified in the attribute is first resolved using the DI container. Then specified method from the attribute is called on this service using reflection. Any parameters are resolved against caller provided parameter lookup dictionary.

Conclusion

In the article we explored how custom attribute based frameworks can leverage composite application frameworks (like Prism) and MVVM pattern to remove any “wiring” code necessary to create and populate a view model in a typical n-tier application. We have created a simple declarative framework that takes care of instantiating required services, calling them, and automatically updating view model with service call results.

The framework we built is extremely simple to keep the article from bloating. There are several improvements and enhancements that can be applied to it, like:

• Additional attributes to specify different types of service call results processing; in addition to property assignment and list replacing that we implemented, we can also have an attribute that specifies an optional translator between service call result and view model property
• ViewModelFactory can take on responsibility to populate view model asynchronously; the benefit of this approach is that we only need to implement the asynchronous behavior once in the factory and none of the presenters need to change
• To complete interaction with middle tier the framework should also provide a declarative way to submit modified business entities back using service calls; this can be achieved, for example, by mapping view model’s properties to “Save” service calls and providing declarative way to translate these properties into “Save” call parameters.

The complete solution (VS 2008) demonstrating the framework can be downloaded here: MVVMAttributes.zip

Some times you need to read entities from a table along with some properties that are reference data or child entities. This relationship is usually represented by foreign keys in your database. LINQ to SQL makes this job extremely easy but, by default, also very inefficient. There are multiple ways this inefficiency can be addressed. Some of these alternatives are standard facilities provided by LINQ to SQL and some are custom coded patterns and tricks. This article walks your through these solutions, explaining their pros and cons and helping you decide which one is right for you.

Default Behavior

Consider the following simple subset from the AdventureWorksLT database:

Lets say your job is to select and display all the customer addresses in the database (we will limit results to first 3 to simplify output), with the following fields: First Name, Last Name, City, State, and Modified Date. You have your Data Access Layer (DAL) to get the data and your driver code to print the results. Something like this:

using(AdventureWorksDataContext ctx = new AdventureWorksDataContext())
{
IEnumerable<CustomerAddress> q = BuildQuery(ctx);

foreach (var ca in q)
{
Console.Out.WriteLine(“{0}, {1} lives in {2}, {3} as of {4}”,
ca.Customer.LastName,
ca.Customer.FirstName,
ca.Address.City,
ca.Address.StateProvince,
ca.ModifiedDate);
}
}

private static IEnumerable<CustomerAddress> BuildQuery(
    AdventureWorksDataContext ctx)
{
    var q = (from ca in ctx.CustomerAddresses
             select ca).Take(3);
    return q;
}

Very simple but also very inefficient. If you turn on the logger (ctx.Log = Console.Out;) you can see actual SQL statements generated and issued during program execution:

SELECT TOP (3) [t0].[CustomerID], [t0].[AddressID], [t0].[AddressType], [t0].[rowguid], [t0].[ModifiedDate]
FROM [SalesLT].[CustomerAddress] AS [t0]
-- Context: SqlProvider(Sql2008) Model: AttributedMetaModel Build: 4.0.21006.1

SELECT [t0].[CustomerID], [t0].[NameStyle], [t0].[Title], [t0].[FirstName], [t0].[MiddleName], [t0].[LastName], [t0].[Suffix], [t0].[CompanyName], [t0].[SalesPerson], [t0].[EmailAddress], [t0].[Phone], [t0].[PasswordHash], [t0].[PasswordSalt], [t0].[rowguid], [t0].[ModifiedDate]
FROM [SalesLT].[Customer] AS [t0]
WHERE [t0].[CustomerID] = @p0
-- @p0: Input Int (Size = -1; Prec = 0; Scale = 0) [29485]
-- Context: SqlProvider(Sql2008) Model: AttributedMetaModel Build: 4.0.21006.1

SELECT [t0].[AddressID], [t0].[AddressLine1], [t0].[AddressLine2], [t0].[City], [t0].[StateProvince], [t0].[CountryRegion], [t0].[PostalCode], [t0].[rowguid], [t0].[ModifiedDate]
FROM [SalesLT].[Address] AS [t0]
WHERE [t0].[AddressID] = @p0
-- @p0: Input Int (Size = -1; Prec = 0; Scale = 0) [1086]
-- Context: SqlProvider(Sql2008) Model: AttributedMetaModel Build: 4.0.21006.1

SELECT [t0].[CustomerID], [t0].[NameStyle], [t0].[Title], [t0].[FirstName], [t0].[MiddleName], [t0].[LastName], [t0].[Suffix], [t0].[CompanyName], [t0].[SalesPerson], [t0].[EmailAddress], [t0].[Phone], [t0].[PasswordHash], [t0].[PasswordSalt], [t0].[rowguid], [t0].[ModifiedDate]
FROM [SalesLT].[Customer] AS [t0]
WHERE [t0].[CustomerID] = @p0
-- @p0: Input Int (Size = -1; Prec = 0; Scale = 0) [29486]
-- Context: SqlProvider(Sql2008) Model: AttributedMetaModel Build: 4.0.21006.1

SELECT [t0].[AddressID], [t0].[AddressLine1], [t0].[AddressLine2], [t0].[City], [t0].[StateProvince], [t0].[CountryRegion], [t0].[PostalCode], [t0].[rowguid], [t0].[ModifiedDate]
FROM [SalesLT].[Address] AS [t0]
WHERE [t0].[AddressID] = @p0
-- @p0: Input Int (Size = -1; Prec = 0; Scale = 0) [621]
-- Context: SqlProvider(Sql2008) Model: AttributedMetaModel Build: 4.0.21006.1

SELECT [t0].[CustomerID], [t0].[NameStyle], [t0].[Title], [t0].[FirstName], [t0].[MiddleName], [t0].[LastName], [t0].[Suffix], [t0].[CompanyName], [t0].[SalesPerson], [t0].[EmailAddress], [t0].[Phone], [t0].[PasswordHash], [t0].[PasswordSalt], [t0].[rowguid], [t0].[ModifiedDate]
FROM [SalesLT].[Customer] AS [t0]
WHERE [t0].[CustomerID] = @p0
-- @p0: Input Int (Size = -1; Prec = 0; Scale = 0) [29489]
-- Context: SqlProvider(Sql2008) Model: AttributedMetaModel Build: 4.0.21006.1

SELECT [t0].[AddressID], [t0].[AddressLine1], [t0].[AddressLine2], [t0].[City], [t0].[StateProvince], [t0].[CountryRegion], [t0].[PostalCode], [t0].[rowguid], [t0].[ModifiedDate]
FROM [SalesLT].[Address] AS [t0]
WHERE [t0].[AddressID] = @p0
-- @p0: Input Int (Size = -1; Prec = 0; Scale = 0) [1069]
-- Context: SqlProvider(Sql2008) Model: AttributedMetaModel Build: 4.0.21006.1

Now you can see why it is inefficient. The first query reads top 3 rows from CustomerAddress table as we would expect. What is not expected is that it doesn’t try to make joins to Customer and Address tables, as you would, if you write SQL query yourself. This is because LINQ to SQL does not know about your intention to access child entities upfront. Instead, it issues two separate queries on-demand for EACH row in CustomerAddress table, one to fetch the Customer data, and one to fetch the Address. For a 3 row recordset we got 7(!) queries total. For a recordset of 1000 rows that would be 2001(!) queries. We clearly need to address this.

There are many ways to solve this and we are going to examine the following three solutions:

1. Using the LoadWith() option provided by LINQ to SQL
2. Hinting LINQ to SQL to use joins by explicitly including child data in the compound object returned from the LINQ query
3. Hinting LINQ to SQL to use joins by using intermediate anonymous compound objects while still returning “clean” LINQ to SQL entity (CustomerAddress class in our case)

The last solution is by far my favorite when it comes to DAL API, so, please, bear with me.

LoadWith() option

Using the LoadWith() data option we can hint LINQ to SQL that every time entity X is being requested its child entity Y should be requested as well. Since now LINQ to SQL “knows” upfront we are interested in child entity (entities) it can optimize the query using joins to retrieve all the data in one shot. In fact, this is how our modified code may look like:

private static IEnumerable<CustomerAddress> BuildQuery(
    AdventureWorksDataContext ctx)
{
    // NEW CODE - LoadWith options
    var dataOptions = new System.Data.Linq.DataLoadOptions();
    dataOptions.LoadWith<CustomerAddress>(ca => ca.Address);
    dataOptions.LoadWith<CustomerAddress>(ca => ca.Customer);
    ctx.LoadOptions = dataOptions;
    // END NEW CODE - LoadWith options

    var q = (from ca in ctx.CustomerAddresses
             select ca).Take(3);
    return q;
}

Now, if we run our driver code with logging we see the following SINGLE(!) SQL query is generated:

SELECT TOP (3) [t0].[CustomerID], [t0].[AddressID], [t0].[AddressType], [t0].[rowguid], [t0].[ModifiedDate], [t1].[AddressID] AS [AddressID2], [t1].[AddressLine1], [t1].[AddressLine2],
[t1].[City], [t1].[StateProvince], [t1].[CountryRegion], [t1].[PostalCode], [t1].[rowguid] AS [rowguid2], [t1].[ModifiedDate] AS [ModifiedDate2], [t2].[CustomerID] AS [CustomerID2], [t2].[NameStyle], [t2].[Title], [t2].[FirstName], [t2].[MiddleName], [t2].[LastName], [t2].[Suffix], [t2].[CompanyName], [t2].[SalesPerson], [t2].[EmailAddress], [t2].[Phone], [t2].[PasswordHash], [t2].[PasswordSalt], [t2].[rowguid] AS [rowguid3], [t2].[ModifiedDate] AS [ModifiedDate3]
FROM [SalesLT].[CustomerAddress] AS [t0]
INNER JOIN [SalesLT].[Address] AS [t1] ON [t1].[AddressID] = [t0].[AddressID]
INNER JOIN [SalesLT].[Customer] AS [t2] ON [t2].[CustomerID] = [t0].[CustomerID]
-- Context: SqlProvider(Sql2008) Model: AttributedMetaModel Build: 4.0.21006.1

We got the perfect query using the proper joins. Why do we need to look into alternative solutions? Well, the LoadWith() option has several limitations that may prevent us from using it in some scenarios:

• LoadWith() option has a global scope and can’t be applied selectively to specific queries. Once applied, ALL the queries requesting the entity will automatically include joins to pull registered child entities. This is not desired in many cases since we want to keep unrelated queries lightweight and free of “LoadWith()” joins. Also, once applied LoadWith() option can not be revoked.
• Since LoadWith() option applies to the data context it needs to be passed around to various classes that construct the queries. This is a “smell” since in most cases we would prefer to deal with data context at the higher levels only.
• Setting load options is not allowed after results have been returned from query involving base or child entity. For example the following [simplified] code throws InvalidOperationException:

using (AdventureWorksDataContext ctx = new AdventureWorksDataContext())
{
    var q2 = from ca in ctx.Customers
             select ca;
    q2.ToArray();   // this forces query execution

    // this will throw if using LoadWith()
    IEnumerable<CustomerAddress> q = BuildQuery(ctx);
}

Using compound object (projection) to hint LINQ to SQL into an efficient join

Take a look at the following changes to the code:

using (AdventureWorksDataContext ctx = new AdventureWorksDataContext())
{
    var q = (from ca in ctx.CustomerAddresses
             select new {
                 CustomerAddress = ca,
                 ca.Customer,
                 ca.Address
             }).Take(3);

    foreach (var item in q)
    {
        Console.Out.WriteLine("{0}, {1} lives in {2}, {3} as of {4}",
            item.CustomerAddress.Customer.LastName,
            item.CustomerAddress.Customer.FirstName,
            item.CustomerAddress.Address.City,
            item.CustomerAddress.Address.StateProvince,            item.CustomerAddress.ModifiedDate);
    }
}

The main thing that is different here from our original solution is the use of anonymous type in the select statement. In this type we explicitly mention child entities, so LINQ to SQL optimize the select query to include them upfront with JOIN. In fact, if we run this with logging turned on, we will see exact same single statement that was generated when using LoadWith() options.

This solution does not rely on any global settings like LoadWith(), which eliminates unwanted side effects and much easier to manage in a component/layered architecture.  The obvious drawback of this method is using anonymous type. For example, we aren’t able to simply return resulting IEnumerable from our Data Access Layer. We can replace the anonymous type with a composite class, but would it be nice if we can simply return IEnumerable<CustomerAddress> where access to child properties won’t cause additional database roundtrips?

The perfect solution – the best of both worlds

Here is solution I like the best and I use it a lot in my DAL code:

private static IEnumerable<CustomerAddress> BuildQuery(
    AdventureWorksDataContext ctx)
{
    var q = (from ca in ctx.CustomerAddresses
             select new
             {
                 CustomerAddress = ca,
                 ca.Customer,
                 ca.Address
             }).Take(3)
             .AsEnumerable()
             .Select(ca => ca.CustomerAddress);
    return q;
}

Here is how this works. The select part of it starts exactly the same as the one we used in the “anonymous type” solution. Using anonymous composite type forces the JOINs. Then there is “.AsEnumerable()” – very important part of the solution that I explain later. The last part (.Select(ca => ca.CustomerAddress)) simply “cleanses” anonymous type and “converts” it back to “clean” CustomerAddress class that we want to use/expose. If we don’t use AsEnumerable() before cleansing the whole expression will be evaluated as IQueryable by LINQ to SQL implementation and cleansing will “optimize away” the composite anonymous type benefits, and LINQ to SQL will no longer assume you need child properties, and will NOT generate JOINs. Injecting .AsEnumerable() takes control away from LINQ to SQL at the time when JOINs are assumed, and gives control to LINQ to Objects for cleansing.

The end result is the best of both world: single efficient query that uses JOINs and clean DAL API (IEnumerable<CustomerAddress>).

There are various reasons you may want to run several full-blown independent WPF applications side-by-side in the same process. In my particular case this was a client project where we were building a composite WPF trading desk. One of the requirements was to be able to run different versions of the same component (montage, P&L, blotter, etc.) side-by-side but within seemingly integrated environment. The best solution turned out to be running different versions in completely independent environment (application) and only communicate across domains using a thin protocol to provide “integrated” look and feel. Running components in separate domains let us avoid all the problems that come with signing, versioning, evolving dependencies, and many more.

This post is a step by step guide to demonstrate the techniques involved.

Step 1: Creating basic WPF application

The first step is to create a simple host application. Select “File/New/Project…” in Visual Studio and pick “Visual C#/Windows/WPF Application”. Give it a name of “WPFDomainLab” and click OK.
The default application with a single main window is generated. Now go to the properties for “App.xaml” and change its “Build Action” from “Application Definition” to “Page”. This allows us to add our own Main() method with explicit WPF application startup code. If you try to compile at this point you should be getting “Program ‘XXXX.exe’ does not contain a static ‘Main’ method suitable for an entry point”.

We now need to add Main method to handle WPF startup. The following class will do it:

using System;

namespace WPFDomainLab
{
    class Startup
    {
        [STAThread()]
        static void Main()
        {
            App app = new App();
            app.MainWindow = new Window1();
            app.MainWindow.Show();
            app.Run();
        }
    }
}

Save this as Startup.cs and include it in your project. Now the project should compile. Before running it open Window1.xaml and have it display something interesting:

<Window x:Class="WPFDomainLab.Window1"
    xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
    xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
    Title="Window1" Height="100" Width="300">
    <Grid>
        <ContentControl
            Content="{Binding DomainName}"/>
    </Grid>
</Window>

This will display the current domain’s friendly name. We now need to open the code-behind file (Window1.xaml.cs) and add the code to provide binding data context:

using System;
using System.Windows;

namespace WPFDomainLab
{
  /// <summary>
  /// Interaction logic for Window1.xaml
  /// </summary>
  public partial class Window1 : Window
  {
    public Window1()
    {
      InitializeComponent();
      this.DataContext = new {
        DomainName =
          "I live in " +
          AppDomain.CurrentDomain.FriendlyName
      };
    }
  }
}

If you compile and run the app at this point you should get the following window:

At this point we have a single WPF application running in the default application domain.

Step 2: Moving WPF app into a dedicated domain

Lets go ahead and move this application to its own dedicated domain. These are the changes needed to accomplish this in a simple way (Startup.cs):

1:  using System;
2:
3:  namespace WPFDomainLab
4:  {
5:    class Startup
6:    {
7:      [STAThread()]
8:      static void Main()
9:      {
10:        AppDomain domain =
11:          AppDomain.CreateDomain(
12:            "another domain");
13:        CrossAppDomainDelegate action = () =>
14:        {
15:            App app = new App();
16:            app.MainWindow = new Window1();
17:            app.MainWindow.Show();
18:            app.Run();
19:        };
20:        domain.DoCallBack(action);
21:      }
22:    }
23:  }

Lines 10-12 create another domain.

Lines 13-19 are used to package application startup logic in a delegate that will be remotely executed in another domain.

Line 20 actually runs the delegate in “another domain” and creates full-blown WPF application there. If you compile and run the app now you should get the following:

Step 3: Starting second WPF application in its own domain

Adding a second instance of our WPF application running in yet another domain looks simple now, but it has some caveats. Consider the following changes to create two separate domains each running a WPF app (Startup.cs):

1:  using System;
2:
3:  namespace WPFDomainLab
4:  {
5:    class Startup
6:    {
7:      [STAThread()]
8:      static void Main()
9:      {
10:        var domain1 = AppDomain.CreateDomain(
11:          "first dedicated domain");
12:        var domain2 = AppDomain.CreateDomain(
13:          "second dedicated domain");
14:
15:        CrossAppDomainDelegate action = () =>
16:        {
17:          App app = new App();
18:          app.MainWindow = new Window1();
19:          app.MainWindow.Show();
20:          app.Run();
21:        };
22:
23:        domain1.DoCallBack(action);
24:        domain2.DoCallBack(action);
25:      }
26:    }
27:  }

Lines 10-13 create two different domains.

Lines 23-24 actually create WPF applications in respective domains.

If you run host application at this point you will find that it doesn’t quite work as expected. First a single window appears stating “I live in ‘first dedicated domain’ domain”. Only when you close the first window, the second one appears (after a pause), now stating “I live in ‘second dedicated domain’ domain”.

The reason for this strange behavior is that all three domains (default one created by CLR and two domains we explicitly created) share the same execution thread. The first WPF app “hijacks” Window message pump and blocks the thread at “app.Run()”, so “domain2.DoCallBack(action);” is not even executed until the first app terminates.

Step 4: Starting a dedicated thread for each domain

To remedy this we need to create a dedicated thread for each of the domains that we create. The following changes accomplish the task (Startup.cs):

1:  using System;
2:  using System.Threading;
3:
4:  namespace WPFDomainLab
5:  {
6:    class Startup
7:    {
8:      [STAThread()]
9:      static void Main()
10:      {
11:        var domain1 = AppDomain.CreateDomain(
12:          "first dedicated domain");
13:        var domain2 = AppDomain.CreateDomain(
14:          "second dedicated domain");
15:
16:        CrossAppDomainDelegate action = () =>
17:        {
18:          Thread thread = new Thread(() =>
19:          {
20:            App app = new App();
21:            app.MainWindow = new Window1();
22:            app.MainWindow.Show();
23:            app.Run();
24:          });
25:          thread.SetApartmentState(
26:            ApartmentState.STA);
27:          thread.Start();
28:        };
29:
30:        domain1.DoCallBack(action);
31:        domain2.DoCallBack(action);
32:      }
33:    }
34:  }

Lines 18-24 create a dedicated thread object to run WPF app and package WPF app startup code inside this thread’s start method.

Lines 25-26 sets the thread’s apartment to STA, this is a WPF requirement, otherwise it will not run (it will throw).

Lines 27 actually starts a dedicated thread in the given domain.

If you run application host now you will see both windows running in the same process in two different domains happily coexisting side by side. Applications are fully isolated, each getting its own static variables, base directory and configuration file (if customized). The performance is sluggish though and we are going to address this next.

Step 5: Optimizing assembly loading

When you ran the final application from step 4 you probably noticed that its startup time is extremely bad. Depending on your system it may take up to 10 seconds for the second application UI to show up.

The reason is the default .Net loader behavior. By default every domain gets its own copy of assemblies loaded into domain’s context independently of other domains. It means that all .Net Framework and WPF assemblies will need to be loaded and JIT-ed twice (once in each domain). In addition to duplicating the work loading assemblies in non-default domain is extremely slow (I haven’t figured out why). Fortunately there is a simple solution for the problem and a simple change will do it:

1:  using System;
2:  using System.Threading;
3:
4:  namespace WPFDomainLab
5:  {
6:    class Startup
7:    {
8:      [STAThread()]
9:      [LoaderOptimization(
10:        LoaderOptimization.MultiDomainHost)]
11:      static void Main()
12:      {
13:        ...
14:      }
15:    }
16:  }

The attribute on line 9 does the trick. Basically it instructs .Net loader to load all GAC-installed assemblies in a domain-neutral way, thus sharing the assembly code between domains. This avoids loading and JIT-ing assemblies multiple times.

You can read more about domain-neutral assemblies here.

The final solution can be downloaded from here.