What a bingle mess

I have to admit that the last 2 Bingle Maps updates were a total mess. Beginning with version 1.5 crashing all around when starting, sleeping, and exiting the app after updating from a previous version, then there’s this not-any-better version 1.6, which silences the crashes but made the app not saving any data at all!

I have to apologise to all of the users out there who’s experiencing these frustrations and am grateful for those who sent in the error reports and the helpful feedbacks.

1.6 was pulled from Market place as soon as I realised the situation, and 1.7 – which is suppose to end all this madness, was submitted to App Hub yesterday. While waiting for Microsoft to test and certify the new version which usually takes a week, I decided to put down some details on what has really happened to the last 2 updates, technically.

It all starts with auto complete

The first new feature I implemented for version 1.5 was auto complete for the search and direction finding boxes. This required some structure changes in one of my data model, which is saved and loaded when the app starts and exits. This is the very culprit for all of those crashes. But how can I not have caught this during my testing?

Turned out that the crashes would only happen if 1.5 tries to load the saved data from a previous version, and this only happens once. This is because I use Silverlight Serializer (SS) to serialise the data to isolated storage, and SS is a binary serialiser, which does not deal with data structure change very well. Therefore, when 1.5 starts for the first time after updating, SS tries to deserialise the old binary data into an updated 1.5 object, which then causes the first exception. This exception stops all other necessary initialisation of the app leaving the app in a half broken state. When you sleep or exit the app however, since there was no data loaded, the app will do a clean up in the isolated storage so that the next time the app starts, it starts anew. This is exactly why the crashes only happens once.

Then there was the new WP7.1 SDK

Half way through 1.5, the WP7.1 SDK beta 2 was released and as a developer I upgraded my phone to a Mango dev build as soon as I can. I was amazed by how far Mango has reached and excited about all of the new features in the SDK. So I quickly wrapped up 1.5 with some extra performance improvements and decided to update the whole project to 7.1.  While I was at it, I figured that it was a good chance to also update all third party libraries that the app uses to their latest version too, such as RestSharp and DropNet, and of course the Silverlight Serializer.

Not long after that, I started getting tons of error reports due to the 1.5 crashing bug. It was till then I realised that it was a huge mistake to rush updating everything to the latest version before making sure that 1.5 was all OK. Because once the project is updated to target the 7.1 runtime, you can’t downgrade back to 7.0. So I had no choice but to revert back in my version control history.

That was a real harsh lesson for me to completely understand how crucially important it is to keep your version control history nicely commented. All of my SVN check-ins were made without comments and there are thousands of them. So after hours of looking at the file diffs in WinMerge, I finally reverted the project back to where I pushed out 1.5, EXCEPT forgetting the updated Silverlight Serializer.

The new Silverlight Serializer is targeting Silverlight 4 framework, which is incompatible with the Silverlight 3.7 CLR running on all of the devices out there. But guess what, I never had a chance to catch this (again) because my phone was running Mango, which has Silverlight 4 CLR. So again, I foolishly thought that I had patched up the 1.5 bug and submitted 1.6 to App Hub. You probably know the rest of the story from here.

One thing surprises me is that the App Hub QA team didn’t run into this bug either. The whole testing and certification process was smooth for 1.6 and it was published to Marketplace within a week. Does it mean that they are using Mango phones to test the new submissions now?

So what’s now

Finding out that 1.6 became a bigger disaster was a nightmare to me. I realised that I really have to clean this up and do it properly from now. So I started re-organising my SVN repo into Git and established proper branches just like an enterprise development project. I then wiped the Mango rom on my phone after that, re-flashed the factory rom, went through the lengthy Zune update process and finally got the phone back to official NoDo environment. From here I could test the re-compiled Silverlight Serializer for 3.7 CLR and made sure every thing works fine on a normal NoDo device.

So 1.7 was submitted and it is running fine on my NoDo phone now. Hopefully this would put an end to the madness. I’d label this as the worst disaster in my development experience so far. But I guess sometimes the most valuable things can only be learnt from the toughest lessons.

Bingle Maps 1.2 with Google Global Business Search

Bingle Maps 1.2 has been released to the Windows Phone 7 Marketplace today. This version utilizes Google’s “Places” API and “Geocode” API to power search and address lookup feature. Bing’s doing a good job on routing but I’d compare routing data from both side to see if I should also switch routing data over. AFAIK, Google’s Directions V3 API only returns encoded way points and decoding them is not that simple.

Categorized business search would most likely come in next version (aka, the “what’s nearby” feature), and I’m also looking into transit information and compass support.

Also celebrating Bingle Maps passing 20,000 users 😀

Bingle Maps 1.0 released to Marketplace!

Luckily my second submission has been approved by MS and the app is available in Marketplace already!

Download It Now via Zune Desktop Software.

Or just search “bingle” in Marketplace.

WP7: Toolkit Fed 2011 Release

Head over to codeplex to grab the just released Fed 2011 Silverlight For Windows Phone Toolkit.

In this version the toolkit brings you default tilt effect on most common controls and also included the PerformanceProgressBar by Jeff Wilcox.

Definately not something huge but why not just grab it and recompile your project while it’s completely free?

Update: I’ve also noticed that page transition animations are now using better easing functions then the previous version.

RX: A reactive approach to event handling

Microsoft released RX (Reactive Extensions) for .NET on November 2009, but I bet there are only 1% of the developers are really using it so far. Apart from that Microsoft is not really pushing it, the real reason, IMO, could be both that it is only suitable for some very specific development cases, and the idea it is using is quite hard to get around with from scratch. Because it is using a reversed logic to process objects.

Let’s put it simple:

1. RX is the library to support Reactive Programming.
2. Reactive Programming means your code “react” to something when it becomes available instead of “act” on it actively. So it’s more of a passive approach.
3. RX aims to convert objects that you want to interact into a list, and invoke an action each time an item is added to this list. The idea is similar to listening to an ObservableCollection.

Why do I reckon RX is only suitable for some very specific cases? Let’s take a look at this very simple example:

Simple example: Processing a list

Let’s say we have a list, filled with objects, and we only want to print out the string objects in this list:

List list = new List<object>() { "string 1", 1, "string 2", 2 };

What we normally do is to use a loop to process this list:

foreach(object obj in list) if (obj is string) Console.Write(obj.ToString());

In the normal way, we actively go through each object in the list and process it. The RX model takes this in a passive way, asynchronously:

list.ToObservable().Where(obj => obj is string).Subscribe(obj => Console.Write(obj.ToString()));

In the RX model, you convert the normal list to a list that can be observed by calling list.ToObservable(). This will return you an IObservable<object> which is a RX supported list and able to use all of the magical RX functions and extensions. The Where functions takes a function delegate that determines what object in the list will be selected, and returns the filtered list. The Subscribe function can take an Action<object>, which will be called against each object in the list as soon as they become available. In our case, because the list is pre-filled, so the objects are already available. When you execute the Subscribe function, it performs similar to our for loop, processing each of the already-available objects in the list.The real difference here is that:

1. The monitoring and processing happens asynchronously
2. If you add more items to the list after the RX statement, the newly added items will also be processed as soon as they are added. Because you have “subscribed” to the list and are monitoring it.

This could be useful when you are not sure when items will be added to the list, if you don’t really want to use an ObservableCollection. However, RX runs slower than the traditional model and requires more memory, as it is doing a whole bunch of conversion and event handling behind the scene.So far, RX has been mainly used in Silverlight, where user interaction with the UI can become quite complicated and asynchronous processing is the only option in most cases.Let us walk through the following example to understand why RX could be critical to a rich user interface.

Real life example: Detecting mouse button press and hold event

Take a minute and think about how you would implement the detection of a mouse button press and hold event in Silverlight, let’s say, we want to do something when the mouse left button is down for 2 seconds without moving?

The triditional model

You definitely need a timer to time this 2 second interval:

DispatcherTimer myTimer = new DispatcherTimer() { Interval = TimeSpan.FromSseconds(2) };
myTimer.Tick += (s, e) =>{
    // Do something when the timer ticks.
};

To start the timer, we will need to listen to the MouseLeftButtonDown event:

this.MouseLeftButtonDown += (s, e) =>
{
    myTimer.Start();
}

We also need to detect mouse up and mouse movement in order to cancel the timer, so that the timer tick event doesn’t miss fire when the user doesn’t really mean to hold down the mouse button:

this.MouseLeftButtonUp += (s, e) =>
{
    myTimer.Stop();
}
this.MouseMove += (s, e) =>
{
    myTimer.Stop();
}

This could be all if you are working on a normal Silverlight app, but it gets messier for a WP7 app where multi-touch needs to be considered. You want to cancel the timer as well when a second finger is pressed on the screen.To do this, you need to modify the MouseLeftButtonDown event handler above:

this.MouseLeftButtonDown += (s, e) =>
{
    myTimer.Start();
    // Register a 2nd event handler to detect 2nd finger press.
    this.MouseLeftButtonDown += SecondMouseDown;
    // Because we are attaching a new handler each time the mouse left button is down,
    // it is important to detach this event handler when it is invoked,
    // so that the same handler doesn't get invoked multiple times.
}

public void SecondMouseDown(object sender, MouseButtonEventArgs e)
{
    // Detach this event handler to prevent been invoked multiple times.
    this.MouseLeftButtonDown -= SecondMouseDown;
    myTimer.Stop();
}

This traditional event handling model works fine, except that it requires creating a timer plus 4 event handlers, and code is scattered making reading and maintenance difficult.

The RX model

I’m gonna give you the complete working RX code to do this, then explain through it:

public partial class MainPage : PhoneApplicationPage{
    // Constructor
    public MainPage()
    {
        InitializeComponent();
        Observable.Throttle(Observable.FromEvent(this, "MouseLeftButtonDown"), TimeSpan.FromSeconds(2))
                      .TakeUntil(Observable.FromEvent(this, "MouseLeftButtonUp"))
                      .TakeUntil(Observable.FromEvent(this, "MouseMove"))
                      .ObserveOnDispatcher()
                      .Subscribe(e =>
                      {
                          // Do something here...
                      });
    }
}

This is it! 1 single RX statement with no event listener at all.First, we use the Throttle extension to specify that, whatever we are monitoring, we don’t care for an initial period of time. This extension takes 2 parameters, the first parameter tells the extension what observable list to monitor, and the 2nd parameter says how long the initial period is.Then we specify with the 2 TakeUntil, that we want to stop monitoring by MouseLeftButtonUp or MouseMove event.We also need to use the ObserveOnDispatcher extension to specify that when we perform any actions on the list, perform it on the current thread (i.e. the UI thread). This can prevent any invalid cross thread access exceptions been thrown.Next we use the Subscribe function to start monitoring this list. When the MouseLeftButtonDown event happens, and after the initial 2 seconds period, execute the Action we specified for the Subscribe extension.

WP7: Activating before Deactivated, a problem of Tombstoning

Tombstoning is the process that you save your app’s “state” to, usually Isolated Storage when the user leaves the app, so that your app can reload the state when it is been reactivated via the Back button.

Before going into the problem, you should have a good understanding on this concept. Shawn Wildermuth has a very good walk through of tombstoning and James Ashley also talked about why deactivating the application is not the same as tombstoning.

Assuming you save your app state using the Application_Deactivated event handler, there should be no problem at all, except this boundary case: what if the app is re-activated even before the Application_Deactivated handler finishes it’s execution?

It’s quite simple to produce this situation. You can:

• either put a break point in debug mode in the Application_Deactivated event handler, then press Home then Back buttons, then F5 your app to resume it;
• or simply put Thread.Sleep(5000) in the event handler and press Home then Back buttons within 5 seconds.

If you observe this process carefully in debug mode, the behavior is quite interesting here: the Application_Activated event handler will still be invoked after Application_Deactivated but your app instance is not destroyed and recreated. This means when the app is reactivated, none of the constructor of your objects will be called, as if there weren’t a deactivation at all. To put this in short: tombstoning is not needed in this case!

Having this in mind, we should carefully code our logic so that we don’t do unnecessary tombstoning operations that may cause unwanted side effects.

For example, if you were saving and loading your setting like this:

private ObservableCollection<string> _searchResults;
public ObservableCollection<string> SearchResults{
    get { return _searchResults; }
    set { _searchResults = value; RaisePropertyChanged("SearchResults"); }
}

private void Application_Activated(object sender, ActivatedEventArgs e)
{
    SearchResults = LoadSearchResultsFromIsoStore();
}

private void Application_Deactivated(object sender, DeactivatedEventArgs e)
{
    SaveSearchResultsToIsoStore(SearchResults);
}

The above code could trigger an unnecessary write and read of IsolatedStorage in our special case, and if SearchResults is large and bound to a UI list with complicated item template, there will be much more side effects then just performance hit for reloading it unnecessarily.

Unfortunately, it’s most likely that we won’t be able to avoid the potential redundant write to IsolatedStorage in Application_Deactivated, because there is no way that we can predict if the app will be reactivated too soon. But we can always be smart in Application_Activated. For example:

private void Application_Activated(object sender, ActivatedEventArgs e)
{
    if (SearchResults == null) SearchResults = LoadSearchResultsFromIsoStore();
}

In this example, we can easily know that if the app has been destroyed or not by checking whether SearchResults is null. If the app instance was never recreated, then SearchResults would still be pointing to something other than null. Depends on how you organise your tombstoning logic, there could be a thousand different ways to be smart here. Just keep this case in mind when you code.

WP7: Everything About ListBox Performance

If you found this blog post, I bet you’ve already had a hard time dealing with the unsatisfactory performance of the Silverlight framework of WP7. Let’s face it, managed code has no chance against native code (read: objective-c) when coming to execution performance. And one of the most frustrating part of WP7’s performance issue could be scrolling a ListBox-like control.

Boldly speaking, ListBox-like control is the most critical control for a WP7 Silverlight app. For any kind of average complex WP7 app, you’ll have to use a scrollable list in one or more of your screens.

Stack Panel

The golden rule here is that the simpler the UI is, the faster the performance can be. So if you list is small and static enough, like less than 30 items not using an ObservableCollection, you may try putting a StackPanel in a ScrollViewer and have the items in the StackPanel generated in code.

Items Control

If what you need is just a list that binds to a collection and executes some code or navigates to another screen when you pressed on one of its items, you could consider re-implementing your ListBox using the ItemsControl.

The standard ListBox extends the Selector class which further extends ItemsControl.

The ItemsControl class has all of the important ListBox properties such as ItemsSource and ItemTemplate. What it is lacking is the ability to “select” an item and it doesn’t scroll by itself. Therefore, you may implement your list in the following way:

<ScrollViewer VerticalScrollBarVisibility="Visible">
    <ItemsControl ItemsSource="{Binding MyDataCollection}">
        <ItemsControl.ItemTemplate>
            <DataTemplate>
                <HyperlinkButton NavigateUri="/TargetPage.xaml" Click="DoSomeThing">
                    <HyperlinkButton.Template>
                        <ControlTemplate x:Key="SimpleHyperlinkButton" TargetType="HyperlinkButton">
                            <ContentPresenter Content="{TemplateBinding Content}"/>
                        </ControlTemplate>
                    </HyperlinkButton.Template>
                    <!-- Your control template here -->
                </HyperlinkButton>
            </DataTemplate>
        </ItemsControl.ItemTemplate>
    </ItemsControl>
</ScrollViewer>

As you can see, you can bind collection to the ItemsControl‘s ItemsSource property just like a ListBox. The use of the HyperlinkButton lets you directly set the URI of the new page that you want to navigate to, saves from writing event handler in the code behind. Or you can also use the Click event to do extra processing in code. Note that overwriting the HyperlinkButton‘s Template is compulsory as the default template is only capable of displaying string value. You may also use a Button instead of a HyperlinkButton, and the template overwriting is still recommended as it simplifies the layout of the Button control, if you don’t need the extra border and pressed/disabled effect of the Button control.

The ItemsControl does not virtualise UI elements in the list. This can have a memory impact if your list is large. If you are not planning using the LongListSelector (see later) in this case, I would recommend, if applicable, loading a limited number of items at load time and give the users a button at the end of the list to load more items:

Deferred Load List Box

David Anson blogged about loading items in the ListBox with his DeferredLoadListBox. This is also a ListBox that uses StackPanel but delay creates the UI. The catch with DeferredLoadListBox is that it only defers the load. Once the UI elements are created, it does not virtualise them either. In fact, during my testing on a Samsung OMNIA 7, the over all scrolling performance of DeferredLoadListBox is even worse than the standard ListBox.

Long List Selector

The LongListSelector shipped in the SilverlightToolkit for WP7 is, in my opinion, the ideal control for loading large number of items when binding to a data collection. This control virtualise the UI items in the list to keep it’s memory footprint small. To put it simple, it destroys the UI elements when they are off the screen and recreates them when they are back in. You can set the BufferSize property to specify how many page’s items that you want to keep in memory. The default value is 1 page so UI items are (supposed to) destroyed immediately when they are out, you may test increasing this value to suit your app. Another interesting property is MaximumFlickVelocity. Decreasing this number can slow down the inertial scroll speed and buy some more time for the list to load it’s items. The default value is 4000. You may find 2500 a good number to start with if you need to tweak this.

It is also important to know that binding a collection that implements the IList interface (such as ObservableCollection) is much faster and resource friendly than those don’t, this is because you can also virtualise data using such a collection. Take a look at this page for more about UI and data virtualisation.

Pay very much attention to the Loaded event of items in your list especially when using the LongListSelector. This event is raised every time when an item is re-created. Which means handler of this event may get executed multiple times as you scroll the list. I would suggest keep away from the Loaded event of the items. If you really need to execute something on each load, consider doing it in a background thread. I have previously talked about this technique and limitation about it.

Lazy List Box

Last but not least, Peter Torr has implemented a very unique way to improve scrolling performance via his LazyListBox. His basic principle is “Don’t do work while the user is scrolling“. If you don’t want to read through his thoroughly explained blog, you can just download his sample code and copy out the LazyListBox class and start using it like a normal ListBox. The only change here would be, when you create the ItemTemplate for LazyListBox, you create a simpler (preferably no image) item for preview purpose. Because LazyListBox will show items using this template while it is scrolling. The real template is LoadedItemTemplate. LazyListBox will reload items in the current visible area with this template as soon as scrolling is stopped. Similar to LongListSelector, LazyListBox also virtualises off screen UI elements. However, you can optionally store some UI elements using the CachedItemTemplate to avoid heavy reloading activities. I suggest have a good read of his blog post if you really want to take LazyListBox as your solution.

What more to expect

After all, before Microsoft releases updates to improve the performance of the Silverlight run-time of WP7, there’s limited amount of optimisation that we can do.

When the WP7 developer tools first shipped Microsoft Silverlight Developer Jeff Wilcox tweeted invaluable performance tips for 3 straight hours. Tim Heuer collected Jeff’s tidbits and posted those on his blog.

While the list is fairly long, I’ve picked some obvious ones that are usually ignored by developers:

• Pivot starts up faster than Panorama.
• Always use JPEG over PNG if you can.
• Set the compile option of image and media files as “Content” instead of “Resource” (in the property window).
• Avoid using images larger than 2000 x 2000 pixels, not even the background of your Panorama.
• The simpler your layout is, the faster the performance.

WP7: Limitation of Multi-Threaded UI Operations

Loek Ouweland has recently blogged about more responsive UI operations utilising the BackgroundWorker class. To put it simple: if you execute your code on the same thread as the UI, then the UI won’t be updated/re-drawn until the execution is finished. This is actually not a new problem to WP7, it’s a common behavior of almost all platforms out there.

The example in Loek’s post is a for loop that generates many star patterns on the UI. During the generation process, Loek wanted to show the progress of the generation using a progress bar. The progress bar would never change during the for loop because the loop happens on the same UI thread, which prevents the UI from being updated until all of the stars are added to the UI. The solution was to use a BackgroundWorker to perform the loop and in each loop use the Dispatcher object of the UI thread to add the star to the UI.

While this seemed to be working just fine, there’s an obvious limitation in this approach, that all UI element related operations still need to happen on the UI thread.

Let’s take Loek’s example again. The reason why Dispatcher is used in the BackgroundWorker solution is that you can’t directly access objects on the UI thread from another thread.

Usually, where you really want to use this technique is when you have a large amount of complex UI elements that you need to generate using a loop in the code behind. The limitation of this approach is, that you are neither able to access the UI layer nor able to generate any UI elements in the background thread and pass them to the UI thread. As soon as you try to create any object that inherits the DependencyObject class in the background thread, you’ll get the same Invalid cross-thread access exception.

Which means, you may give your UI a chance to update itself between the moment of jumping to the next loop, but you may still experience UI freeze up while you use Dispatcher to generate that complex UI element inside each loop, depends on how complex the UI element is.

WP7: How easy to get your information synced!

This has nothing to do with development or anything like that. Just to note down that I’m surprised by WP7 today by how easy it is to get my contacts synced.

I was thinking about how I am going to transfer all of my contacts from my iPhone 4 to Omnia7. I never get this done since I got the Omnia7 because it sounds like a headache. The only way I could think of, is to sync my iPhone contacts to somewhere and sync them onto Omnia7, and I firstly thought about Outlook on my office computer. But there are tons of company contacts that I don’t want to get mixed up with my personal contacts. Creating another outlook data account sounds overkilling just for transferring my contacts. So I gave up the Outlook idea and synced my contacts to Google Contacts (thank god that at least iTunes supports this one and only cloud service). Just about when I wondered how I’m gonna get my Google Contacts to Omnia7, I realised that they are already in my Omnia7! Turned out that I added my Google account to the phone when I set it up, and the phone syncs everything automatically including contacts.

This is really a big plus to WP7. It really shows off how much better WP7 is integrated with multiple cloud services than iOS, and making a lot of headache tasks effortless now.

Well I don’t wanna repeat after all the other tech blogs about how much deeper that social networking is integrated in WP7, I am just surprised that, been the most successful mobile OS ever, Apple is doing much less in this aspect for iOS.

Silverlight: Unhandled Exception and MVVM Light

The MVVM Light Toolkit is one of the most popular MVVM framework that Silverlight developers are using. While it’s very handy for data binding and manipulation, you have to beware that there’s a little difference to handle unhandled exceptions.

Usually you would expect all unhandled exceptions would end up in the automatically generated Application_UnhandledException event handler in yoru App.xaml.cs file. A lot of developers would create their custom exception class and catch all exceptions in this event handler as a centralized place to process the exceptions and perform appropriate actions accordingly. But, when you try to raise an exception in, say, a property in one of your MVVM Light view model, this may not always be the case.

For example, in my main view model, I have a simple property:

private int _price;
public int Price
{
    get { return _price; }
    set
    {
        if (value < 0) throw new MyException("Price cannot be less than 0.");
        _price = value;
        RaisePropertyChanged("Price");
    }
}

Then in the Application_UnhandledException event handler, I wanted to show the exception message to the users:

private void Application_UnhandledException(object sender, ApplicationUnhandledExceptionEventArgs e)
{
    if (e.ExceptionObject is MyException)
    {
        // Show all MyException's message to the user.
        e.Handled = true;
        MessageBox.Show(e.ExceptionObject.Message);
    }
    else
    {
        // Handle other exceptions here.
    }
}

This looks good enough except one problem, the exception will never end up in the Application_UnhandledException event handler. This is because in the MVVM Light view model locator (by default the ViewModelLocator.cs file), uses a static reference of your main model to bind data to your UI components. static reference is probably running in a different system thread then your instantiated UI components. So when you raise your exception in your view model, the exception won’t reach any event handlers that are in a different thread.

The solution to this problem is rather simple: you raise the exception in the same UI thread.

private int _price;
public int Price
{
    get { return _price; }
    set
    {
        if (value < 0)
        {
            // Throw the exception in the UI thread.
            App.Current.RootVisual.Dispatcher.BeginInvoke(() => { throw new MyException("Price cannot be less than 0."); });
            // Prevent the property from being set to the incorrect value.
            return;
        }
        _price = value;
        RaisePropertyChanged("Price");
    }
}

The modified property uses the Dispatcher object of the current application’s RootVisual, which is usually your Main Page. The BegainInvoke function will throw the exception in the same thread of the RootVisual, which would be the same thread as your application. Thus making the exception catchable in the application’s Application_UnhandledException event handler.

Note that BeginInvoke executes the action in another thread. This is an asynchronous operation that does not interrupt your current execution. So you usually need to put in the extra “return;” statement to manually interrupt the execution if you need to. This is not necessary in a normal case since raising an unhandled exception only interrupts the execution process in it’s own thread.

Note: Catching exceptions in Application_UnhandledException may raise performance concern. If you are a Windows Phone 7 developer, you should test your exception catching performance on a real device. In most cases, this will cause perceivable slowness and is usually not a recommended approach.