Archives

by dkt

Better PowerShell Splatting

One of PowerShell’s more useful and differentiating features is its ability to splat command-line arguments. Normally, to invoke a command in PowerShell with arguments, you could do something like this:

Get-Process -Name *Tomcat* -ComputerName somebox.around.here

It may be useful to capture the arguments first, then invoke the command later:

$myArgs = @{ Name = "*Tomcat*"; ComputerName = "somebox.around.here" }
Get-Process @myArgs

It can also be incredibly useful if you’re writing wrapper functions:

Function Import-ModuleForce {
    param(
        [Parameter(Mandatory = $true)]
        [string[]]
        $Name
    )
 
    $PSBoundParameters.Force = $true
    Import-Module @PSBoundParameters
}
 
# unfortunately, aliases can't be set up to script blocks;
# only functions and cmdlets get that honor
Set-Alias imf Import-ModuleForce

It can also be useful when you’re passing in the same arguments to multiple functions consecutively:

$myArgs = @{
    Name = "*Tomcat*"
    ComputerName = "somebox.around.here"
}
 
Get-Service @myArgs
Get-Process @myArgs

But sometimes splatting doesn’t work, notably when you have extra values in the hashtable that the function isn’t expecting:

$myArgs = @{
    Name = "*Tomcat*"
    ComputerName = "somebox.around.here"
    DependentServices = $true
}
 
Get-Service @myArgs
Get-Process @myArgs

A nasty little error is PowerShell’s response:

Get-Process : A parameter cannot be found that matches parameter name 'DependentServices'.
+ Get-Process <<<<  @myArgs
    + CategoryInfo          : InvalidArgument: (:) [Get-Process], ParameterBindingException
    + FullyQualifiedErrorId : NamedParameterNotFound,Microsoft.PowerShell.Commands.GetProcessCommand

The Get-Service cmdlet has a DependentServices argument, but Get-Process does not. Although this can sometimes be a helpful error message, in this case, it would be nice if the command just ignored the arguments it couldn’t understand.

So that’s why I wrote Invoke-Splat:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

Function Invoke-Splat {
    <#
    .Synopsis
        Splats a hashtable on a function in a safer way than the built-in
        mechanism.
    .Example
        Invoke-Splat Get-XYZ $PSBoundParameters
    #>
    param(
        [Parameter(ValueFromPipeline=$true, Mandatory=$true, Position=0)]
        [string]
        $FunctionName,
        [Parameter(ValueFromPipeline=$true, Mandatory=$true, Position=1)]
        [System.Collections.Hashtable]
        $Parameters
    )
 
    $h = @{}
    ForEach ($key in (gcm $FunctionName).Parameters.Keys) {
        if ($Parameters.$key) {
            $h.$key = $Parameters.$key
        }
    }
    if ($h.Count -eq 0) {
        $FunctionName | Invoke-Expression
    }
    else {
        "$FunctionName @h" | Invoke-Expression
    }
}

Rewriting the example from above:

$myArgs = @{
    Name = "*Tomcat*"
    ComputerName = "somebox.around.here"
    DependentServices = $true
}
 
Invoke-Splat Get-Service $myArgs
Invoke-Splat Get-Process $myArgs

And mixing Invoke-Splat with the @ operator:

Function Import-ModuleForce {
    param(
        [Parameter(Mandatory = $true)]
        [string[]]
        $Name,
 
        [string]
        $SomethingElse
    )
 
    $PSBoundParameters.Force = $true
    Invoke-Splat Import-Module $PSBoundParameters
}
 
$myArgs = @{
    Name = "MyModule"
    SomethingElse = "AnotherString"
}
 
Import-ModuleForce @myArgs
by dkt

Twelve Ways to Improve WPF Performance

There is no shortage of information out there on how to speed up the performance of WPF applications, but too often the focus is on the weird stuff instead of the simpler issues. I’m not going to talk about things like writing to WritableBitmaps to optimize drawing—it’s a topic covered to death elsewhere. Instead, this is meant to be a slightly more practical guide to squeezing performance out of WPF in ways that are probably more likely affecting you.

Some general notes

ItemsControl and its subclasses ListBox and ListView exacerbate performance problems because these controls are highly dynamic (resolution happens “late”), involve WPF collections (which are slow), and have difficult and unpredictable lifetimes for their child controls. Scrollbar performance is often a big problem in larger WPF apps because of problems that seem trivial for small collections, but suddenly blow up with larger data sets.

Also, it can be difficult in WPF to know exactly when the system is finished with an object. For views, you get the FrameworkElement.Unloaded event, but it gets raised at times you might not expect (such as system theme changes) and not at times when you might (application shutdown). On viewmodels associated with views, you’ll never get a WPF notification that a viewmodel is about to go unused by a view. Blend-style behaviors also have their own set of lifetime problems.

Then there are some problems (like this and this) where WPF leaks for you too.

Finally, there are things (this, this, this, this, this, and this) that simply perform worse than you likely expect.

Finally, there are old UI/WinForms problems (this, this, and this) that never really went away—they’re just less likely to happen.

  1. Fix Binding Errors
  2. Hard-code widths and heights where possible
  3. Avoid CollectionView.Grouping
  4. Optimize bindings to collections that change
  5. Avoid DynamicResources
  6. Avoid ResourceDictionary
  7. Simplify your visual tree
  8. Be wary of System.Windows.Interactivity.Behavior<T>.OnDetaching
  9. Do not use DependencyPropertyDescriptor for any reason…ever
  10. Be careful of viewmodel events
  11. Batch up Dispatcher.BeginInvoke
  12. In general, beware of memory leaks

I. Fix Binding Errors and Exceptions

Every time a binding error occurs, your app hangs for just a split second as it writes out errors to the trace log. If you have a lot of binding errors, then those split seconds start to add up. Make sure to go through your bindings, especially those on ItemsControls (ListViews, custom grids, etc.) and verify that there are no binding errors.

Open up your app in the debugger and play around, especially where there is slowness. Make sure all bindings resolve without errors.

RelativeSource in DataTemplates may also result in bindings that break initially, but then later resolve properly. Be wary of them, and try to use inherited attached properties instead of relying on RelativeSource in DataTemplates.

Viewmodel bindings

  1. Make sure that your views and view models are in sync. Use ReSharper 6 to help you find broken bindings.
  2. If you’re binding to a collection of objects with mixed types, add different DataTemplates so that none of them refer to non-existent properties.
  3. Make sure that your converters aren’t throwing exceptions. These have a cost too.

View-based RelativeSource bindings

  1. When using ListBoxes and ListViews, it’s a common problem to have this problem. Avoid RelativeSource.FindAncestor expressions at all cost here, because the deferred behavior of templates cause the object and its bindings to be created (and resolved) before the ListBoxItem/ListViewItem is added to the visual tree.
  2. An alternative is to define an attached dependency property on the ListBoxItem/ListViewItem, and use property inheritance to give your child items the necessary property values. This essentially pushes property values down the visual tree instead of searching up.

II. Hard-code sizes where possible

This may not always be a practical or desirable solution, but layout passes perform faster when widths and heights do not have to be recalculated. They may also help stop a layout pass from rippling through an entire visual tree.

And always set specific widths on columns in a grid (be it a ListView + GridView or any third-party control), because these tend to be very expensive, especially with larger data sets.

III. Avoid CollectionView.Grouping

Grouping in WPF doesn’t perform terribly well, especially with ListViews and GridViews. Create a collection with mixed viewmodel types–your original collection, and one that represents the “group”. Use DataTemplates to change the appearance of your “group” objects.

For example, if you have a PersonViewModel class with a property that you want to group by (let’s say Region), it is faster to create a mixed collection of MyGroupViewModel and PersonViewModel objects, ordered correctly by group, with different DataTemplates, than it is to bind to a grouped collection. Unfortunately, it’s a lot more work.

IV. Optimize bindings to collections that change

Repeatedly calling ObservableCollection<T>.Add when the collection is data-bound can be a prohibitively expensive operation, especially with thousands of rows. Unfortunately, the framework provides no easy, satisfactory fix.

Fix 1: Use ObservableCollection as-is, but break bindings

  1. Break the binding to the collection.
  2. Update the collection while not data-bound.
  3. Re-bind.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37

// some methods removed for brevity
public partial class MyViewModel : INotifyPropertyChanged
{
    private ObservableCollection<T> _people;
 
    public IList People
    {
        get { return _people; }
        private set
        {
            if (_people != value)
            {
                _people = value;
                OnPropertyChanged("People");
            }
        }
    }
 
    void BatchAddPeople(IEnumerable<Person> newPeople)
    {
        var currentPeople = _people;
 
        // stop WPF from listening to the changes that we're about
        // to perform
        this.People = null;
 
        // change
        foreach (var person in newPeople)
        {
            currentPeople.Add(person);
        }
 
        // cause WPF to rebind--but only once instead of once for
        // each person
        this.People = currentPeople;
    }
}

Fix 2: Use the older .NET 2.0-era collections instead

  1. Use System.ComponentModel.BindingList<T> (from the old days) instead; it has an API for suppressing change notifications.

Fix 3: Reimplement ObservableCollection.

  1. Create your own collection that implements INotifyCollectionChanged.
    • Raise INotifyCollectionChanged as sparingly as you can.
    • Raise the event with a NotifyCollectionChangedAction.Reset event for anything more trivial than a simple single-item add, remove, change, or move. Do not take advantage of the NotifyCollectionChangedEventArgs constructors that take collections of items; you will find support for it spotty at best.
  2. Implement System.Collections.IList on your collection. WPF does not use the generic System.Collections.Generic.IList<T> interface; it is completely ignored. If you don’t implement the interface, WPF will perform almost all operations (including accessing rows by number!) strictly by the IEnumerable implementation, and it won’t be very optimal or fast about it.
  3. (Probably) implement System.Collections.Generic.IList<T> as well. WPF doesn’t use it, but you probably will (through LINQ, Rx, etc.)

V. Avoid DynamicResources

Even in .NET 4.0, DynamicResource access is still slower than StaticResource access. And worse, once you start nesting DynamicResources (for example, a ListView whose Style contains a ControlTemplate that references objects through DynamicResources), you’re likely to run into situations where you leak controls.

VI. Avoid ResourceDictionary

This advice is practically impossible to follow, but do your best. There is a huge cost in constructing ResourceDictionaries, and depending on where you place them, you are probably constructing many more objects than you realize.

A common, sensible, and logical pattern is to keep usages of elements as close to where you use them as possible. Many people place resources in UserControl.Resources, or break up their themes into multiple ResourceDictionaries for clarity and separation. Although this is arguably good programming practice, it also tends to be insanely slow. If your windows/controls or your ListBoxItem/ListViewItems in a ListBox/ListView are coming up more slowly than you would like, it’s probably a combination of too much ResourceDictionary construction and/or DynamicResources. (Yes, even in .NET 4.0.) Collapse ResourceDictionaries as much as you can, and try to ensure that the contents of these dictionaries is only loaded once. The easiest and surest way is to include it in the resources of your System.Windows.Application object, almost always difficult or infeasible for composite applications.

I have also frequently taken to creating static classes that contain nothing but highly reusable resources (think static framework classes like the Brushes class) because it’s easier to guarantee that objects are only being created once, and hopefully at app startup instead of triggered by the user interacting with the application and forcing a lazy load at an undesirable time. Not necessary the healthiest design, but the performance is quite a bit better.

Using implicit ControlTemplate/DataTemplate styles will also help keep your code and XAML organized without the need for either StaticResource or DynamicResource.

VII. Simplify your visual tree

Shallow visual trees are better than deeper visual trees. Again, ItemsControls will usually exacerbate performance problems with deep visual trees because if they’re not being virtualized, they’re being destroyed and recreated; if they are being virtualized, changing DataContext in a deeper tree takes more time than changing DataContext in a shallower one.

VIII. Be wary of System.Windows.Interactivity.Behavior<T>.OnDetaching

Sadly, System.Windows.Interactivity.Behavior<T>.OnDetaching will generally not get called. Put a breakpoint and see for yourself.

OnAttached signals the addition of a behavior to a control (generally at instantiation of your XAML); OnDetaching signals the removal of a behavior from a control (generally never, as behaviors don’t get removed from controls). Don’t put sensitive disposing behavior in OnDetaching. The Unloaded event is a better place for that, but be aware that it will get raised every time the control is removed from the visual tree.

IX. Do not use DependencyPropertyDescriptor for any reason…ever

DependencyPropertyDescriptor.AddValueChanged classes cause the WPF framework to take a strong reference to the source of the event that isn’t removed until you call DependencyPropertyDescriptor.RemoveValueChanged. This class is frequently used in conjunction with Behaviors, so if you have RemoveValueChanged in OnDetaching, you’re likely leaking memory. Because of WPF’s references to your objects, it is not just enough to drop references to your view and view model.

A better alternative is to rely on data binding where you can; create a DependencyProperty for the sole purpose of listening to changes on your target property, and use the change notifications in DependencyProperty in order to listen to changes on the target property. It keeps the observed object (generally, your view model) from accidentally holding a strong reference to your view.

X. Be careful of view model events

If your views or behaviors rely on events being raised from a viewmodel (as innocuous as INotifyPropertyChanged.PropertyChanged or INotifyCollectionChanged.CollectionChanged), subscribe to them weakly. Viewmodels tend to have a longer lifetime than views (consider a virtualized ItemsControl), so it’s possible that your view model will inadvertently gather references to views. Use classes like PropertyChangedEventManager or CollectionChangedEventManager, or (painfully) use the WeakEventManager to create your own event manager for your custom events. It’s painful, but usually necessary in order to prevent view models from taking references to views.

XI. Batch up Dispatcher.BeginInvoke

If your application displays data from a network, you’re probably using background threads to accomplish the task (which is good). However, you’re probably not consciously counting your Dispatcher.BeginInvoke calls (which is not as good). The more you’re able to coalesce multiple Dispatcher.BeginInvokes into a single call, the more likely WPF will be able to help you out by making single subsequent layout and render passes, and the lower your overall CPU usage.

To be fair, WPF is much better at trying to help you here than VB6/WinForms—you won’t often see apps that dance and flicker incessantly any more—but there is still a non-zero cost to updating the screen, and if you have a particularly large application, this could be a problem for you.

XII. In general, beware of memory leaks

This is a bit of a generalization of the last few points, but memory leaks make apps behave worse over time. Fixing memory leaks goes a long way in fixing the performance of an application. And since most developers are constantly restarting WPF apps as they work on them, they often go undetected until the software is delivered.

—DKT

by dkt

Doing Simple Things with ExpressionVisitor

LINQ is one of the more powerful technologies in .NET. Particularly, it introduced expression trees to the language, giving you the limited ability to inspect and rewrite code from code. Basically, code manipulating code–good stuff.

For example, here is a very simple expression tree that adds 73 to a given number:

1

Expression<Func<int, int>> someExpr = x => x + 73;

You can actually address the individual elements inside of the expression tree:

1
2
3
4
5
6

var nodeType = someExpr.Body.NodeType;
var otherValue = ((ConstantExpression)((BinaryExpression)someExpr.Body).Right).Value;
 
// writes "Add" and "73" to the console
Console.WriteLine(nodeType);
Console.WriteLine(otherValue);

(Let’s ignore those little ugly casts for now.)

Here is another expression tree that takes the average of three numbers:

1

Expression<Func<int, int, int, double>> someExpr = (x, y, z) => (x + y + z) / 3.0;

And here’s how we’d list the names of the parameters in the numerator:

1
2
3

Console.WriteLine(((ParameterExpression)((BinaryExpression)((BinaryExpression)((UnaryExpression)((BinaryExpression)someExpr.Body).Left).Operand).Left).Left).Name);
Console.WriteLine(((ParameterExpression)((BinaryExpression)((BinaryExpression)((UnaryExpression)((BinaryExpression)someExpr.Body).Left).Operand).Left).Right).Name);
Console.WriteLine(((ParameterExpression)((BinaryExpression)((UnaryExpression)((BinaryExpression)someExpr.Body).Left).Operand).Right).Name);

Of course, inspecting trees like this is a technique that doesn’t scale well.

Luckily, in .NET 4.0 (or in .NET 3.5, courtesy of The Wayward Weblog), the System.Linq.Expressions.ExpressionVisitor class makes examining expression trees a bit less painful:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

using System;
using System.Linq.Expressions;
 
class MyExpressionVisitor : ExpressionVisitor
{
    protected override Expression VisitParameter(ParameterExpression node)
    {
        Console.WriteLine(node.Name);
        return base.VisitParameter(node);
    }
}
 
class Program
{
    public static void Main(string[] args)
    {
        Expression<Func<int, int, int, double>> someExpr = (x, y, z) => (x + y + z) / 3.0;
        var myVisitor = new MyExpressionVisitor();
        myVisitor.Visit(someExpr);
    }
}

ExpressionVisitor contains a method for every possible expression tree node, so we could actually write the whole expression back out to the console by overriding the methods that we care about:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

class MyExpressionVisitor : ExpressionVisitor
{
    protected override Expression VisitBinary(BinaryExpression node)
    {
        Console.Write("(");
 
        this.Visit(node.Left);
 
        switch (node.NodeType)
        {
            case ExpressionType.Add:
                Console.Write(" + ");
                break;
 
            case ExpressionType.Divide:
                Console.Write(" / ");
                break;
        }
 
        this.Visit(node.Right);
 
        Console.Write(")");
 
        return node;
    }
 
    protected override Expression VisitConstant(ConstantExpression node)
    {
        Console.Write(node.Value);
        return node;
    }
 
    protected override Expression VisitParameter(ParameterExpression node)
    {
        Console.Write(node.Name);
        return node;
    }
}

It actually writes out a slightly different output:

(((x + y) + z) / 3)xyz

Removing the extra parentheses actually turns out to be quite tricky because more context is required to determine which ones can be removed and which ones can’t. But for most purposes, that won’t be a problem. The extra xyz, however…we’ll need to get rid of that:

1
2
3
4
5
6
7
8
9
10
11

class Program
{
    public static void Main(string[] args)
    {
        Expression<Func<int, int, int, double>> someExpr = (x, y, z) => (x + y + z) / 3.0;
        var myVisitor = new MyExpressionVisitor();
 
        // visit the expression's Body instead
        myVisitor.Visit(someExpr.Body);
    }
}

By walking just the Body of the lambda, we ignore the Parameters that we don’t need to have listed twice:

(((x + y) + z) / 3)

Much better.

You can use this technique to regenerate C# from LINQ, SQL from LINQ, or a lot of other different languages. Instances of ExpressionVisitor are at the heart of every interpretation of a LINQ tree, and doing anything fancy under the surface of LINQ requires a good understanding of this class. It’s also a nice illustration of the visitor pattern, which has applications even beyond LINQ.

by dkt

Minimal WCF and REST (or whatever else you might want)

The original versions of WCF were great if you wanted to expose SOAP-RPC endpoints with predefined methods driven off of reflection in your code, but as developers in the .NET space move to join the rest of the world with REST and general HTTP services that aren’t quite so static as SOAP-RPC, where does that leave WCF?

Here is a minimalist version of a standalone WCF service—no XML in app.config or web.config, no IIS, no generated classes—just a copy-pasteable example of getting a minimal arbitrary HTTP server up and running in WCF. Note that this code requires .NET 4.0 because it uses the new ByteStreamMessageEncodingBindingElement class to allow for tighter control over the serialized output.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60

using System;
using System.IO;
using System.ServiceModel;
using System.ServiceModel.Channels;
using System.Text;
 
[ServiceContract]
[ServiceBehavior(
    InstanceContextMode = InstanceContextMode.Single,
    AddressFilterMode = AddressFilterMode.Any)]
public class GenericService
{
    /// <summary>
    /// Starts up a generic HTTP service.
    /// </summary>
    /// <param name="args">
    /// Arguments to the event.
    /// </param>
    public static void Main(string[] args)
    {
        Uri uri = new Uri("http://localhost:9000/");
        var serviceImpl = new GenericService();
        using (var host = new ServiceHost(serviceImpl))
        {
            // this gives you complete control over the stream
            var binding = new CustomBinding(
                new ByteStreamMessageEncodingBindingElement(),
                new HttpTransportBindingElement());
 
            host.AddServiceEndpoint(typeof(GenericService), binding, uri);
 
            // begin listening for connections
            host.Open()
 
            // display a little message and wait for the user
            // to kill the console app
            Console.WriteLine("Listening on URI: {0}", uri);
            Console.WriteLine("Press ENTER to stop the service.");
            Console.ReadLine();
        }
    }
 
    [OperationContract(Action = "*", ReplyAction = "*")]
    public Message Process(Message message)
    {
        // this is the incoming URI; we're not doing anything with it
        // here, but you almost certainly will
        Uri uri = message.Headers.To;
 
        // build a stream of bytes that represents the text "Test" in UTF-8
        var bytes = Encoding.UTF8.GetBytes("Test");
        var memStream = new MemoryStream(bytes);
 
        // create and return the response message
        return Message.CreateMessage(
            MessageVersion.None,
            "OK",
            new BodyStreamProviderWriter(memStream));
    }
}

And the implementation of BodyStreamProviderWriter:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38

using System.IO;
using System.ServiceModel.Channels;
using System.Xml;
 
/// <summary>
/// Allows an arbitrary <see cref="Stream"/> to be serialized.
/// Use in conjunction with the overloads of the
/// <see cref="Message"/>.CreateMessage method.
/// </summary>
public sealed class BodyStreamProviderWriter : BodyWriter, IStreamProvider
{
    private readonly Stream innerStream;
 
    public BodyStreamProviderWriter(Stream stream) : base(false)
    {
        this.innerStream = stream;
    }
 
    protected override void OnWriteBodyContents(XmlDictionaryWriter writer)
    {
        writer.WriteStartElement("Binary");
        writer.WriteValue((IStreamProvider)this);
        writer.WriteEndElement();
    }
 
    Stream IStreamProvider.GetStream()
    {
        return this.innerStream;
    }
 
    void IStreamProvider.ReleaseStream(Stream stream)
    {
        if (stream != null)
        {
            stream.Dispose();
        }
    }
}

GenericService.cs
BodyStreamProviderWriter.cs

And presto–you’ve got a miminal WCF service for your RESTful purposes (or whatever else you might want). —DKT

by dkt

Collections, CollectionViews, and a WPF Binding Memory Leak

A colleague of mine recently came across some code that should work, but doesn’t:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

private ListBox SomeListBox;
private ObservableCollection<string> SomeStrings = new ObservableCollection<string>();
 
public void SomeMethod()
{
    SomeListBox.ItemsSource = null;
    ThreadPool.QueueUserWorkItem(o => AddToStrings());
}
 
private void AddToStrings()
{
    // this line will actually crash the app!
    SomeStrings.Add("blah");
 
    Dispatcher.BeginInvoke(new Action(() => SomeListBox.ItemsSource = SomeStrings);
}

So basically, detaching an ObservableCollection<T> from a view, adding an element on a background thread, and then reattaching it back on the UI thread again. Generally, view-bound objects can only be modified from the UI thread, but shouldn’t detaching the collection first should make that a non-issue?

When you actually run the code, you get one of those obnoxious “InvalidOperationException: The calling thread cannot access this object because a different thread owns it” messages, and then your app will probably die. These usually happen because you’re trying to modify a control or an object bound to a control, neither of which applies here. It also happens when you’re trying to do anything with a subclass of DispatcherObject or a DependencyObject on the wrong thread, but that doesn’t apply here either, right? ObservableCollection<T> does not have any inherent thread affinity, and it is definitely safe to pass between threads so long as access to it is synchronized.

So we took the control out of the picture by writing SomeListBox.ItemsSource = null, and ObservableCollection<T> can be modified from any thread. What’s the problem? Those are the only two objects in the picture, right?

ICollectionView

(If you already know what ICollectionView is, feel free to skip to the end of the story.)

Actually, when any IEnumerable is bound to an ItemsControl, an implementation of ICollectionView is created to wrap the collection, and it provides a means for sorting, grouping, and filtering without affecting the original collection. So when you bind a ListBox, or ListView, or the WPF Toolkit DataGrid, or some other third-party ItemsControl to a grid, a CollectionView is created specifically to maintain intermediate state on behalf of the control without having to touch the original collection.

Consider some List<string>:

Index Value
0 Cheese
1 Apple
2 Tomato
3 Salad

We bind it to a DataGrid, and then we tell the DataGrid to sort itself. The list should look like this:

Index Value
0 Apple
1 Cheese
2 Salad
3 Tomato

Note how the objects now have different indices. It’s an important detail—WPF manages to accomplish sorting, grouping, and filtering all without modifying the original collection because of the use of a CollectionView (or one of its subclasses ListCollectionView, BindingListCollectionView, or the internal CollectionViewProxy class). It also pulls off some shiny tricks occasionally, batching changes where appropriate and postponing actual CollectionView reordering (which can obviously get expensive) until subsequent pumps of the current Dispatcher. And that’s why CollectionView inherits from DispatcherObject.

And of course, if our collection changes, the CollectionView listens for those notifications and responds appropriately, causing hilarious consequences when the change happens on a background thread.

The Fix

The implementation of CollectionView is somewhat broken when collections implement INotifyCollectionChanged:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

internal CollectionView(IEnumerable collection, int moveToFirst)
{
    /* some code snipped for brevity */
    INotifyCollectionChanged changed = collection as INotifyCollectionChanged;
    if (changed != null)
    {
        // this is the problem
        changed.CollectionChanged += this.OnCollectionChanged;
 
        /* some more code */
    }
 
    /* still more code */
}

Nowhere anywhere is this event handler unwired. So collections essentially retain strong references of their corresponding CollectionViews. The data binding system that is responsible for creating the CollectionView maintains a weak reference, but as long as the original collection is alive, the data binding system will be hooked into the CollectionView, and therefore the original collection as well.

The fix actually isn’t so bad:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

using System.Collections;
using System.Collections.ObjectModel;
using System.Collections.Specialized;
using System.Windows;
using System.Windows.Data;
 
public class BetterObservableCollection<T> :
    ObservableCollection<T>, ICollectionViewFactory
{
    public ICollectionView CreateView()
    {
        return new BetterListCollectionView(this);
    }
}
 
public class BetterListCollectionView :
    ListCollectionView, IWeakEventListener
{
    public BetterListCollectionView(IList list) : base(list)
    {
        INotifyCollectionChanged changed = list as INotifyCollectionChanged;
        if (changed != null)
        {
            // this fixes the problem
            changed.CollectionChanged -= this.OnCollectionChanged;
            CollectionChangedEventManager.AddListener(list, this);
        }
    }
}

As soon as the collection is detached from the view, the view’s reference to the CollectionView is broken, and the CollectionView is allowed to disappear because nothing (neither the original collection nor the binding system) is holding on to a strong reference any more.

What “The Fix” Breaks

The CollectionView is responsible for managing sorting, grouping, and filtering. If you disconnect the collection from the view, the CollectionView will get garbage-collected, and the sorting, grouping, and/or filtering you had on the control will be lost. That may or may not matter for your particular use case, but it’s definitely something to think about. If that is important, it may make more sense to bind to a System.ComponentModel.BindingList<T>, which tracks its own grouping, sorting, and filtering inside the list itself—the tradeoff is then you lose the ability to have multiple views on the same collection. —DKT

by dkt

Attached Behaviors are the New OLE (a.k.a. Long Live Components)

Pruning through the old Google Reader starred items list was supposed to only take me an hour or so Saturday morning. But sandwiched between some other diversions, it ended up taking me the whole weekend. One of the last things I read was
Mac OS X 10.6 Snow Leopard: the Ars Technica review (that was an hour right there all by itself). Flipped through some of the old Mac OS X articles, particularly those right at the turn of the Classic Mac OS transition almost ten years ago. You remember Classic Mac OS, right? With Cyberdog and OpenDoc? Windows users will remember the ideas more as OLE and its foundation COM, and a generation of applications where the screen jumped to and fro whenever you clicked somewhere you weren’t supposed to as menubars popped up and disappeared. And of course, there was J2EE .

Maybe these technologies suffered from being too ambitious or too undefined in scope, but for whatever reason, they’ve been consigned to the dustbin of history. But the dream of having arbitrary components that could just be snapped together like building blocks never seems to die, and maybe we’re finally reaching the point in technology’s history where that could be a possibility. OLE, OpenDoc, and J2EE all assumed those building blocks were gigantic—plug six or seven them together and boom, you’ve got an app—whereas realistically, they’re probably much closer to LEGO®-sized. You’ll need a lot of them before you get something that makes sense—and consequently, you need a little bit of computer know-how.

Fellow Lab49er Charlie Robbins blogged a few weeks ago about a few
sample Silverlight projects that added physics-based behaviors to a canvas with a few items in it. It’s downright silly what you can do with behaviors now, and I wonder if we’re not too far off from seeing a scaled-down version of the component dream realized. It always seemed like overkill to have an IntegerTextBox, or a DateTextBox. But an IntegerBehavior/DateBehavior that can be applied to a TextBox, or a grid cell? Or a behavior that could be applied to a Label to make it an editable TextBox upon a double-click? Or better yet, two behaviors, one to make a label editable on a double-click, and another to restrict to date entry? Much more awesome…

If you haven’t read up on attached behaviors yet, you should. I would argue that a proper MVVM design necessitates attached behaviors—they’re everything that doesn’t belong in the viewmodel layer, but since your views should be stupid and codeless, your “enforce integerness” behavior has to end up somewhere. With viewmodels and attached behaviors, you might never need to write another line of code-behind ever again. —DKT

by dkt

Links O’Clock; 2009.09.25

Normally, throughout the week, I star a whole bunch of things in Google Reader that I want to go back to when I’m not busy jostling for space on the New York subways or when I’m back at my comfortable desk at home with my giant monitors. Unfortunately for me, I star articles about 500% faster than I can read them, so every once in a while, I have to give myself a weekend dedicated to purging that star list so that I don’t forget about things that I really really did want to read (Mr. Siracusa doesn’t write as often as most people, but he makes up for it).

So it’s time for the first-ever Links O’Clock here; most of these articles go back at least two months or more, but hey, maybe you missed it the first time the Internet collectively decided they were worth reading. I don’t think Links O’Clock will normally be this long, but we’ll see.

Also gonna try giving Google Reader’s “Shared items” feature a ride—I keep hitting it by accident in my main newsreader, but now I’ll hit it on purpose (link).

It seems fitting that my desire to purge my starred article backlog leads me to this article likening obesity to information overload:
http://www.core77.com/blog/featured_items/no_more_feeds_please_how_abundant_information_is_making_us_fat_14248.asp

I once tried using a 3D designing app once. I flung the mouse out the window in disgust after several hours of hair-pulling madness. This would have been a far more entertaining experience:
http://www.ilovesketch.com/

For those of you trying to skin your latest GUI, you might be tearing your hair out trying to pick perfect colors. Don’t go bald—just use this:
http://colorschemedesigner.com/

Matthias Shapiro talking about effective visualization with some very concrete and awesome examples—it’s an Ignite Show video, so it’s five minutes long, which is about the longest video I’ll watch before I start wishing I had my computer hooked up to my TV:
http://www.designerwpf.com/2009/09/16/now-playing-on-the-ignite-show/

Another interesting visualization by Ben Fry—living in this country is often defined by driving, so I shouldn’t be so surprised that roads alone do such a remarkable job punctuating the landscape:
http://benfry.com/allstreets/index.html

I’m always a sucker for a good how’d-they-done-it story: The making of the NPR News iPhone App, complete with mockups, requirements identification and gathering—all that good process-y stuff:
http://www.npr.org/blogs/inside/2009/08/the_making_of_the_npr_news_iph.html

If you’re working on building iPhone applications and you want to generate super-realistic pictures of what your app will look like before you even install the Developer Tools:
http://mockapp.com/

Another UI mocking tool for iPhone apps, but you’ll have to write a little code
http://giveabrief.com/

—DKT

by dkt

More 2D/3D Tricks: ItemsControl3D

Building on some of the cute tricks of using a Viewport3D as a two-dimensional surface, we can actually devise a fully-bindable System.Windows.Controls.ItemsControl that renders to a Viewport3D instead of a Panel. It turns out to all be quite remarkably simple:

  1. Define a class, ItemsControl3D, that will be our magic Viewport3D-holding ItemsControl.
  2. Build a ControlTemplate that contains a Viewport3D with a named ModelVisual3D whose children will be modified as the ItemsControl sees fit.
  3. Override GetContainerForItemOverride() to provide instances of ItemsControl3DItem as the “container” for the individual items in the ItemsControl:
    1. Make the “container” item a zero-size, completely non-visible FrameworkElement.
    2. Create an ItemsControl3DItem.Geometry property; this Geometry3D object will be used to populate the ModelVisual3D in our Viewport3D.
  4. I additionally chose to implement container recycling (in some early drafts of ItemsControl3D, it cut processor usage down 25%).

First, the static constructor:

1
2
3
4
5
6
7
8
9
10
11
12

static ItemsControl3D()
{
    DefaultStyleKeyProperty.OverrideMetadata(
        typeof(ItemsControl3D),
        new FrameworkPropertyMetadata(typeof(ItemsControl3D)));
 
    ItemsPanelProperty.OverrideMetadata(
        typeof(ItemsControl3D),
        new FrameworkPropertyMetadata(
            new ItemsPanelTemplate(
                new FrameworkElementFactory(typeof(ItemsControl3D.ItemsPanel)))));
}

The first line should be familiar to those in the audience who have authored custom controls before; it merely indicates that somewhere, WPF should expect to find:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

<<b>Style x:Key="{x:Type ThreeDee:ItemsControl3D}"</b>
        TargetType="{x:Type ThreeDee:ItemsControl3D}">
  <Setter Property="Template">
    <Setter.Value>
      <ControlTemplate TargetType="{x:Type ThreeDee:ItemsControl3D}">
        <Border>
          <Grid>
            <Viewport3D Name="PART_Viewport">
              <Viewport3D.Camera>
                <OrthographicCamera
                         Position="0,0,1" LookDirection="0,0,-1" UpDirection="0,1,0"
                         Width="{Binding Path=ActualWidth, ElementName=PART_Viewport}"/>
              </Viewport3D.Camera>
 
              <ModelVisual3D>
                <ModelVisual3D.Content>
                  <AmbientLight Color="White">
                </ModelVisual3D.Content>
              </ModelVisual3D>
 
              <ModelVisual3D x:Name="PART_SceneRoot"/>
            </Viewport3D>
 
            <ItemsPresenter/>
          </Grid>
        </Border>
      </ControlTemplate>
    </Setter.Value>
  </Setter>
</Style>

The line after that (the overriding of the ItemsPanel metadata) indicates that for all ItemsControl3D, the default Panel presenting children for the panel should be ItemsControl3D.ItemsPanel. This is an inner class that we’ve defined that is specially crafted to hold the child elements of the ItemsControl.

In the style, we’ve given one of the ModelVisual3D children a name (PART_SceneRoot); that’s because in OnApplyTemplate(), we’re going to look for it and use that as the place to hold the 3D objects that we generate.

We override a trio of methods in order to perform basic container housekeeping. GetContainerForItemOverride() either creates a new container or reuses an existing one; ClearContainerForItemOverride(…) adds an unused ItemsControl3DItem back to the pool; IsItemsItsOwnContainerOverride(…) is useful to override if you wanted to manually create and add ItemsControl3DItem objects to the ItemsControl3D.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

private readonly Stack<ItemsControl3DItem> _unusedContainers =
        new Stack<ItemsControl3DItem>();
 
protected override DependencyObject GetContainerForItemOverride()
{
    if (_unusedContainers.Count == 0)
    {
        return new ItemsControl3DItem();
    }
    else
    {
        return _unusedContainers.Pop();
    }
}
 
protected override void ClearContainerForItemOverride(
    DependencyObject element, object item)
{
    _unusedContainers.Push((ItemsControl3DItem)element);
}
 
protected override bool IsItemItsOwnContainerOverride(object item)
{
    return (item is ItemsControl3DItem);
}

Lastly, the actual “panel” that the ItemsControl thinks is doing the work:

1
2
3
4
5
6
7
8
9
10
11
12
13

private new sealed class ItemsPanel : Panel
{
    protected override Size MeasureOverride(Size availableSize)
    {
        var ic3d = (ItemsControl3D)GetItemsOwner(this);
        if (ic3d != null)
        {
            <b>ic3d.UpdateViewportChildren(InternalChildren);</b>
        }
 
        return Size.Empty;
    }
}

And that’s all the panel needs to do. The magic method call is actually the property accessor Panel.InternalChildren—internal code in Panel works together with ItemsControl in order to derive the appropriate children (this is ultimately what will cause GetContainerForItemOverride() and other methods to be called).

Lastly, the private method UpdateViewportChildren in ItemsControl3D:

private void UpdateViewportChildren(UIElementCollection children)
{
    if (_sceneRoot == null) return;
  
    _sceneRoot.Children.Clear();
    foreach (ItemsControl3DItem item in children)
    {
        var m = item.Model;
        if (m != null)
        {
            _sceneRoot.Children.Add(m);
        }
    }
}

And in case you were wondering, ItemsControl3DItem at a high level:

1
2
3
4
5
6
7
8

public class ItemsControl3DItem : FrameworkElement
{
    public double X { get; set; }
    public double Y { get; set; }
    public Brush Background { get; set; }
    public Geometry3D Geometry { get; set; }
    public ModelVisual3D Model { get; }
}

The properties of ItemsControl3DItem (X, Y, Background, Geometry) are all used to determine the Model property.

You can use an ItemsControl3D in XAML as easily as any other subclass of ItemsControl:

1
2
3
4
5
6
7
8
9
10
11
12
13
14

<ThreeDee:ItemsControl3D ItemsSource="{Binding ...}">
    <ItemsControl.ItemContainerStyle>
        <Style TargetType="{x:Type ThreeDee:ItemsControl3DItem}">
            <Setter Property="X" Value="{Binding Year}"/>
            <Setter Property="Y" Value="{Binding Profit}"/>
            <Setter Property="Background" Value="Blue"/>
        </Style>
    </ItemsControl.ItemContainerStyle>
 
    <!-- you could also hardcode children in the control just like 
         ListBoxItems in a ListBox or ListViewItems in a ListView -->
    <!-- <font color="#555555"><ThreeDee:ItemsControl3DItem X="5" Y="6" Background="Blue"/>
         <ThreeDee:ItemsControl3DItem X="2" Y="3" Background="Red"/></font> -->
</ThreeDee:ItemsControl3D>

It should be noted that this exact technique can be used to generate full-blown three-dimensional visualizations with nothing more than basic ItemsControl-style bindings. Coupled with an abstract data model, you’ve got yourself a pretty canvas to paint with, and it’s pretty responsive to updates as well and doesn’t blow a hole through your CPU either. The sample app updates all of the values in a collection of 1,000 points, ten times a second, while using less than 10% of my two-year-old MacBook’s CPU.

The Sample Code: ItemsControl3D.zip

—DKT

Sept 24: Fixed the problems with the download. Of course, since the AssemblyInfo.cs file was missing, the [assembly:ThemeInfo(ResourceDictionaryLocation.SourceAssembly)] tag was missing too—that caused the default template that I defined for ItemsControl3D to not be found. This is fixed now.

by dkt

2D in a WPF 3D World

It’s a fairly common and standard trick to take advantage of the hardware acceleration afforded by modern graphic cards and render two-dimensional figures in 3D. Here is a simple and fast way to take the 3D out of a Viewport3D:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

<Viewport3D>
  <Viewport3D.Camera>
    <OrthographicCamera Position="0,0,1" LookDirection="0,0,-1" UpDirection="0,1,0"
                        Width="{Binding ActualWidth}" Height="{Binding ActualHeight}"/>
  </Viewport3D.Camera>
 
  <ModelVisual3D>
    <ModelVisual3D.Children>
      <!-- render colors...relatively properly -->
      <ModelVisual3D>
        <ModelVisual3D.Content>
          <AmbientLight Color="White"/>
        </ModelVisual3D.Content>
      </ModelVisual3D>
 
      <ModelVisual3D x:Name="SceneRoot">
        <ModelVisual3D.Content>
          <!-- this is where stuff you want to render actually goes -->
        </ModelVisual3D.Content>
      </ModelVisual3D>
 
    </ModelVisual3D.Children>
  </ModelVisual3D>
</Viewport3D>

When the camera is set up this way, the center of the Viewport3D represents (0,0). The bottom-left is (−width / 2, −height / 2) and the top-right is (+width / 2, +height / 2).

Now what can you put in something like this? Same thing you’d normally put in a Viewport3D:

1
2
3
4
5
6
7

<Model3DGroup>
  <GeometryModel3D Geometry="...">
    <GeometryModel3D.Material>
      <DiffuseMaterial Brush="..."/>
    </GeometryModel3D.Material>
  </GeometryModel3D>
</Model3DGroup>

The Brush property can be any old System.Windows.Media.Brush. And as for that Geometry? Well, there are squares:

1
2
3
4

<MeshGeometry3D x:Key="Square">
                Positions="-1,-1,0  1,-1,0  1,1,0  -1,1,0"
                Normals="0,0,1  0,0,1  0,0,1  0,0,1"
                TriangleIndices="0 1 2 0 2 3"/>

isosceles triangles:

1
2
3
4

<MeshGeometry3D x:Key="IsoscelesTriangle">
                Positions="-1,-1,0  1,-1,0  0,1,0"
                Normals="0,0,1  0,0,1  0,0,1"
                TriangleIndices="0 1 2"/>

Coming up with these little nuggets of numbers is the topic of another post. But for now, some sample code:

ThreeDee.png

Featuring basic hit-detection and a bit of interactivity (you can drag a square to increase/decrease the space between boxes, and change the angle of rotation). It’s pure WPF (no unsafe code or interop), it’s 2D, and it’s fast.

ThreeDee.zip

—DKT

Sorry the code formatting is off…the iPhone WordPress client totally destroyed it. I’ll fix it soonish.

by dkt

Uniqueness of Simple Demographics in the U.S. Population [Link]

Sometimes, math and statistics makes me sad:

“It was found that 87% (216 million of 248 million) of the population in the United States had reported characteristics that likely made them unique based only on {5-digit ZIP, gender, date of birth}. About half of the U.S. population (132 million of 248 million or 53%) are likely to be uniquely identified by only {place, gender, date of birth}, where place is basically the city, town, or municipality in which the person resides. And even at the county level, {county, gender, date of birth} are likely to uniquely identify 18% of the U.S. population. In general, few characteristics are needed to uniquely identify a person.”

It’s quite terrifying to know that such a birthdate, a zip code, and a gender could potentially pass as a unique identifier for someone with the vast and overreaching “anonymized” databases that exist out there.


http://privacy.cs.cmu.edu/dataprivacy/papers/LIDAP-WP4abstract.html

http://arstechnica.com/tech-policy/news/2009/09/your-secrets-live-online-in-databases-of-ruin.ars

—DKT