Category: RX

Reactive Drag in WPF

It has been way too long since I last updated my blog. Well I’ve been a bit lazy and busy at the same time, you guys get the excuse ūüėÄ

So without further ado, let’s get down to the fun stuff: doing a proper drag in WPF by using the Reactive Extensions.

I have covered some RX basics and usages 2 years ago for Silverlight/WP7. This is not only a revisit but more on using RX to solve a more complicated case as an example. Let’s start with the basics again:

var mouseDownEvents = Observable.FromEventPattern<MouseEventArgs>(this, "MouseLeftButtonDown");
var mouseMoveEvents = Observable.FromEventPattern<MouseEventArgs>(this, "MouseMove");
var mouseUpEvents = Observable.FromEventPattern<MouseEventArgs>(this, "MouseLeftButtonUp");

var dragEvents =
    from mouseDownPosition in mouseDownEvents
    from mouseMovePosition in mouseMoveEvents.StartWith(mouseDownPosition)
                                             .TakeUntil(mouseUpEvents)
    select mouseMovePosition;

dragEvents.Subscribe(eventPattern =>
{
    // Move things on UI.
});

This is a very simple drag example using RX, it is basically saying, react to all mouse move events, only after the mouse down event has happened and before any mouse up events have occurred. In the subscribe lambda expression, you can do things like get the current mouse position and use it to move things around on the UI.

However, in WPF¬†a more appropriate way¬†to do drag and drop,¬†is to use the System.Windows.DragDrop.DoDragDrop method, so that you can define the drag source, drag data, allowed effects, etc…

dragEvents.Subscribe(eventPattern =>
{
    DragDrop.DoDragDrop(this, this, DragDropEffects.All);
});

It all looks nice and easy, except that:

  1. The DoDragDrop method is a synchronous method. Your code won’t pass that line until you drop.
  2. The method captures the mouse so no mouse up event will fire.

If no mouse up event is fired, the subscription won’t know when to stop, that means even after you dropped the object, the subscription will still react to mouse movements on the current element – a memory leak.

So it seems it’s reasonable enough to change the MouseLeftButtonUp event query to a Drop event query:

// This is the non-working query:
// var mouseUpEvents = Observable.FromEventPattern<MouseEventArgs>(this, "MouseLeftButtonUp");

// This should work:
var dropEvents = Observable.FromEventPattern<MouseEventArgs>(this,"Drop");

var dragEvents =
    from mouseDownEvent in mouseDownEvents
    from mouseMoveEvent in mouseMoveEvents.StartWith(mouseDownEvent)
                                          .TakeUntil(dropEvents)
    select mouseMoveEvent;

// ...

If you play around with this code for a while, you may realize that there are 2 major catches:

  1. If the current element is part of a complicated UI, you have no idea which element’s “Drop” event you should listen to. The drop could happen at anywhere.
  2. Even if you use VisualTreeHelper or Window.GetWindow to get the root element, if the drop happens on somewhere that does not allow dropping (e.g. the AllowDrop property is not set to true), then no Drop event will be raised.

So it looks like we can’t rely on drop either.

If you look at the code carefully, the drag detection is not a drag detection, it is really a “drag start” detection. As soon as you detect the starting of a drag, you can call DoDragDrop method and the framework can handle it from there. So we can see the problem as: how can I detect the drag start moment and not worry about the rest of the mouse movements?

Well logically the drag start is the one immediate mouse move event right after a mouse down event, so accordingly, we can modify the query to this:

var dragStartEvents =
    from mouseDownEvent in mouseDownEvents
    from mouseMoveEvent in mouseMoveEvents.StartWith(mouseDownEvent)
                                          .TakeUntil(mouseUpEvents)
                                          .Take(1)
    select mouseMoveEvent;

dragStartEvents.Subscribe(eventPattern =&amp;gt;
{
    DragDrop.DoDragDrop(this, this, DragDropEffects.All);
});

Now we are specifying query to only get 1 mouse move event after a mouse down event before any mouse up events. Now you can drag the hell out to anywhere without worrying about detecting where it is dropped.

In fact, the StartWith method includes the mouse down event before the first move event, so the subscription is not really reacting to the first move event, but rather, the mouse down event before that. I know this seems awkward, but I can reproduce this behaviour in .NET 4.0/4.5.

So if this annoys you, we can make the query a little bit error proof:

var dragStartEvents =
    from mouseDownEvent in mouseDownEvents
    from mouseMoveEvent in mouseMoveEvents.StartWith(mouseDownEvent)
                                          .TakeUntil(mouseUpEvents)
                                          .Skip(1)
                                          .Take(1)
    select mouseMoveEvent;

Now, this looks a solid query. It stops at mouse up events, skips the first mouse down event then take the real first mouse move event.

Preventing Accidental Drag

If you are a perfectionist, you may want to consider accidental drag detection. Basically we allow the mouse movement within a very small area around the mouse down position without triggering the drag. To do this, we need to alter the event queries a bit so that instead of getting event sequences, we get mouse position sequences:

var mouseDownEvents = Observable.FromEventPattern<MouseEventArgs>(this, "MouseLeftButtonDown").Select(pattern => pattern.EventArgs.GetPosition(this));
var mouseMoveEvents = Observable.FromEventPattern<MouseEventArgs>(this, "MouseMove").Select(pattern => pattern.EventArgs.GetPosition(this));
var mouseUpEvents = Observable.FromEventPattern<MouseEventArgs>(this, "MouseLeftButtonUp").Select(pattern => pattern.EventArgs.GetPosition(this));

Then we need to incorporate one more “Where” filter in the drag start detection query:

var dragEvents = 
    from mouseDownPosition in mouseDownEvents
    from mouseMovePosition in mouseMoveEvents
                                  .Where(mouseMoveEvent => 
                                             Math.Abs(mouseMovePosition.X - mouseDownPosition.X) > SystemParameters.MinimumHorizontalDragDistance
                                             || Math.Abs(mouseMovePosition.Y - mouseDownPosition.Y) > SystemParameters.MinimumVerticalDragDistance
                                        )
                                  .StartWith(mouseDownPosition)
                                  .TakeUntil(mouseUpEvents)
                                  .Skip(1)
                                  .Take(1)
    select mouseMovePosition;

There you go folks, take the reactive looking functional monad code (or whatever you want to call it) and rock hard out there!

RX: A reactive approach to event handling

Microsoft released RX (Reactive Extensions) for .NET on November 2009, but I bet there are only 1% of the developers are really using it so far. Apart from that Microsoft is not really pushing it, the real reason, IMO, could be both that it is only suitable for some very specific development cases, and the idea it is using is quite hard to get around with from scratch. Because it is using a reversed logic to process objects.

Let’s put it simple:

  1. RX is the library to support Reactive Programming.
  2. Reactive Programming means your code “react” to something when it becomes available instead of “act” on it actively. So it’s more of a passive approach.
  3. RX aims to convert objects that you want to interact into a list, and invoke an action each time an item is added to this list. The idea is similar to listening to an ObservableCollection.

Why do I reckon RX is only suitable for some very specific cases? Let’s take a look at this very simple example:

Simple example: Processing a list

Let’s say we have a list, filled with objects, and we only want to print out the string objects in this list:

List list = new List<object>() { "string 1", 1, "string 2", 2 };

What we normally do is to use a loop to process this list:

foreach(object obj in list) if (obj is string) Console.Write(obj.ToString());

In the normal way, we actively go through each object in the list and process it. The RX model takes this in a passive way, asynchronously:

list.ToObservable().Where(obj => obj is string).Subscribe(obj => Console.Write(obj.ToString()));

In the RX model, you convert the normal list to a list that can be observed by calling list.ToObservable(). This will return you an IObservable<object> which is a RX supported list and able to use all of the magical RX functions and extensions. The Where functions takes a function delegate that determines what object in the list will be selected, and returns the filtered list. The Subscribe function can take an Action<object>, which will be called against each object in the list as soon as they become available. In our case, because the list is pre-filled, so the objects are already available. When you execute the Subscribe function, it performs similar to our for loop, processing each of the already-available objects in the list.The real difference here is that:

  1. The monitoring and processing happens asynchronously
  2. If you add more items to the list after the RX statement, the newly added items will also be processed as soon as they are added. Because you have “subscribed” to the list and are monitoring it.

This could be useful when you are not sure when items will be added to the list, if you don’t really want to use an ObservableCollection. However, RX runs slower than the traditional model and requires more memory, as it is doing a whole bunch of conversion and event handling behind the scene.So far, RX has been mainly used in Silverlight, where user interaction with the UI can become quite complicated and asynchronous processing is the only option in most cases.Let us walk through the following example to understand why RX could be critical to a rich user interface.

Real life example: Detecting mouse button press and hold event

Take a minute and think about how you would implement the detection of a mouse button press and hold event in Silverlight, let’s say, we want to do something when the mouse left button is down for 2 seconds without moving?

The triditional model

You definitely need a timer to time this 2 second interval:

DispatcherTimer myTimer = new DispatcherTimer() { Interval = TimeSpan.FromSseconds(2) };
myTimer.Tick += (s, e) =>{
    // Do something when the timer ticks.
};

To start the timer, we will need to listen to the MouseLeftButtonDown event:

this.MouseLeftButtonDown += (s, e) =>
{
    myTimer.Start();
}

We also need to detect mouse up and mouse movement in order to cancel the timer, so that the timer tick event doesn’t miss fire when the user doesn’t really mean to hold down the mouse button:

this.MouseLeftButtonUp += (s, e) =>
{
    myTimer.Stop();
}
this.MouseMove += (s, e) =>
{
    myTimer.Stop();
}

This could be all if you are working on a normal Silverlight app, but it gets messier for a WP7 app where multi-touch needs to be considered. You want to cancel the timer as well when a second finger is pressed on the screen.To do this, you need to modify the MouseLeftButtonDown event handler above:

this.MouseLeftButtonDown += (s, e) =>
{
    myTimer.Start();
    // Register a 2nd event handler to detect 2nd finger press.
    this.MouseLeftButtonDown += SecondMouseDown;
    // Because we are attaching a new handler each time the mouse left button is down,
    // it is important to detach this event handler when it is invoked,
    // so that the same handler doesn't get invoked multiple times.
}

public void SecondMouseDown(object sender, MouseButtonEventArgs e)
{
    // Detach this event handler to prevent been invoked multiple times.
    this.MouseLeftButtonDown -= SecondMouseDown;
    myTimer.Stop();
}

This traditional event handling model works fine, except that it requires creating a timer plus 4 event handlers, and code is scattered making reading and maintenance difficult.

The RX model

I’m gonna give you the complete working RX code to do this, then explain through it:

public partial class MainPage : PhoneApplicationPage{
    // Constructor
    public MainPage()
    {
        InitializeComponent();
        Observable.Throttle(Observable.FromEvent(this, "MouseLeftButtonDown"), TimeSpan.FromSeconds(2))
                      .TakeUntil(Observable.FromEvent(this, "MouseLeftButtonUp"))
                      .TakeUntil(Observable.FromEvent(this, "MouseMove"))
                      .ObserveOnDispatcher()
                      .Subscribe(e =>
                      {
                          // Do something here...
                      });
    }
}

This is it! 1 single RX statement with no event listener at all.First, we use the Throttle extension to specify that, whatever we are monitoring, we don’t care for an initial period of time. This extension takes 2 parameters, the first parameter tells the extension what observable list to monitor, and the 2nd parameter says how long the initial period is.Then we specify with the 2 TakeUntil, that we want to stop monitoring by MouseLeftButtonUp or MouseMove event.We also need to use the ObserveOnDispatcher extension to specify that when we perform any actions on the list, perform it on the current thread (i.e. the UI thread). This can prevent any invalid cross thread access exceptions been thrown.Next we use the Subscribe function to start monitoring this list. When the MouseLeftButtonDown event happens, and after the initial 2 seconds period, execute the Action we specified for the Subscribe extension.