Concurrency: Aggregating concurrent requests to a heavy function

I have been dealing with some interesting optimization problems these days, and one of them was a heavy method being called multiple times with the same input in a very short period of time.

The method DownloadCustomersInternal is used to download customer data from a server asynchronously and returns a Task. It resides in a class that is shared across multiple components, and it often gets called by these components almost at the same time. When there’s a lot of data to download, this method can take a while to finish, it makes sense for the callers of this method to wait on the same task if there’s already one. So a wrapper method DownloadCustomers is written:

private Task _downloadTask;
private readonly object _downloadLock = new object();

private Task DownloadCustomers()
{
    lock (_downloadLock)
    {
        // If there is already a download task, just return this one.
        if (_downloadTask != null)
            return _downloadTask;

        // Perform the real download and assign it's task to the download task reference.
        _downloadTask = DownloadCustomersInternal();

        // When the download process is done, clear the download task reference.
        return _downloadTask = _downloadTask.ContinueWith(task =>
        {
            lock (_downloadLock)
                _downloadTask = null;
        });
    }
}

The wrapper method simply checks if there is already a _downloadTask and returns it if there is. This ensures that all calls to this method get the same Task instance before _downloadTask is set to null, which only happens after DownloadCustomersInternal is finished asynchronously. Because the code sets _downloadTask to null uses the same lock, it guarantees thread safety of the variable.

This will resolve the problem by returning the same Task to callers who requests the data while the data is being downloaded, but what if the method is called right after a previous download is finished? Downloading the same data in very close succession can also be considered redundant, so it would be nice to introduce some sort of “cooling period” before the next download can be initiated. Therefore, an updated version of the above method is written:

private Task _downloadTask;
private readonly object _downloadLock = new object();
private DateTime _lastDownloadTime = DateTime.MinValue;
private static readonly TimeSpan MinimumDownloadInterval = TimeSpan.FromSeconds(5);

private Task DownloadCustomers()
{
    lock (_downloadLock)
    {
        // If there is already a download task, just return this one.
        if (_downloadTask != null)
            return _downloadTask;

        // Check if the time between this download request and the previous one is too close.
        var intervalFromLastDownload = DateTime.Now - _lastDownloadTime;
        if (intervalFromLastDownload < MinimumDownloadInterval)
        {
            // If this download request is too close to the previous one,
            // make this request wait until the minimum download interval is passed,
            // then perform the download.
            var waitTaskSource = new TaskCompletionSource<bool>();
            _downloadTask = waitTaskSource.Task.ContinueWith(_=>DownloadCustomersInternal()).Unwrap();

            // Force the wait in a background thread to ensure a non-blocking UI experience.
            ThreadPool.QueueUserWorkItem(_ =>
            {
                Thread.Sleep(MinimumDownloadInterval - intervalFromLastDownload);
                waitTaskSource.SetResult(true);
            });
        }
        else
            // If this download request is far enough from the last one,
            // just perform the normal download.
            _downloadTask = DownloadCustomersInternal();

        // When the download process is done, always clear the download task reference
        // and update the download timestamp.
        return _downloadTask = _downloadTask.ContinueWith(task =>
        {
            lock (_downloadLock)
            {
                _downloadTask = null;
                _lastDownloadTime = DateTime.Now;
            }
        });
    }
}

The key in this update is determining if a call is too close to a previous call when there is no task running. If the call is too close, simply wait in the background until MinimumDownloadInterval elapses. The waiting code in the background is also thread safe because waitTaskSource.Task.ContinueWith (line 22) is assigned to _downloadTask in the same scope of the first lock, and waitTaskSource is only set to finish when the background wait is elapsed.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s