Last week I wanted to start a new .NET project, and I needed a directory object picker for it. I searched the web and found some solutions, but they just didn’t look right and decided that I can do a lot better than those. So in the spirit of creating good and reusable code, I started implementing my own IDsObjectPicker wrapper in my framework. I thought it would be easy, and it turned out to be anything but, but the result was awesome: clean, easy to use, and it works. And just like last time, I learned a lot about COM interop, you can read all about it here, or just scroll through and grab it already.
Array marshaling
Everything a C programmer needs to know about the Windows built-in directory object picker is here on MSDN. The plan for .NET is straightforward: create the managed structures, then the interfaces, and finally a CommonDialog derived class which hides all the nasty details. The first problem is that the .NET marshaler is very limited when it comes to arrays, if you search for it on the web, you’re guaranteed to find a lot of despair and few good answers.
The first task was the array pointers in DSOP_INIT_INFO. In structures, the marshaler can only handle an in-place array with a constant size. My first try was implementing an ICustomMarshaler. However, you can’t use it from inside a structure definition, only in method parameters, and when you do that, you have to deal with all fields, not just the arrays.
So what to do now? Just leave it as an IntPtr and deal with it somewhere else. And here comes my invention: I created reusable, generic functions which implement array pointer marshaling, and you can use them to create one-line wrapper properties for IntPtr‘s instead of custom, dirty, and repeated code. The functions are named ArrayToPtr and PtrToArray, and can be found in MarshalUtils.cs. ArrayToPtr first allocates space for the array, then stores the elements, and finally, it stores the array pointer and the size somewhere. It also frees the previous contents, so in order to clean up, you just need to set your array to null. Currently it only supports Unicode strings and structures, but it’s easy to add new providers.
A bigger problem arises when you want to marshal an in-place array, not a pointer, as in DS_SELECTION_LIST. In native code, you can change the size of a structure any way you wish, but we can’t do that in managed code, a structure needs to have a fixed size. So let’s say we specify a constant array size of 1. After the marshaler reads the structure from unmanaged memory, it constructs a managed structure, and then what? We have no way to read the rest of the elements because we don’t know where they are, the managed structure is at a different memory address than the unmanaged one. No tricks this time, the only way is to read entire the structure manually, given the unmanaged memory pointer.
You can see my results in ObjSel.cs. I created two wrapper classes for the aforementioned structures, _DSOP_INIT_INFO and _DS_SELECTION_LIST. They are pretty simple actually, I think I perfectly managed to make them as clean as possible.
STGMEDIUM nightmares
After I was done with the initialization structures, I needed some more for data retrieval. To my surprise, I found that they were already implemented in the framework, in the System.Runtime.InteropServices.ComTypes namespace. Lucky, I thought. How wrong I was…
There was a mysterious error when I tried to read an STGMEDIUM, but only when I specified additional attributes to fetch. IDataObject.GetData kept returning an 0×80004001 (E_NOTIMPL) error. Took me days to hunt down (in my free time, of course). The problem was that STGMEDIUM.pUnkForRelease was declared as an object, marshaled as an IUnknown. When I didn’t fetch any attributes, it was null, but when I did, it was set to something, and it makes sense, because in this case, additional stuff was needed to be freed, and that’s exactly what this field is for. Except the marshaler tried to do something with it that it didn’t actually support. I guess that marshaling it as an object works most of the time, but not in this case. I reimplemented the whole thing in ObjIdl.cs, changed that single field to an IntPtr and it worked.
Next I tried to run my code on a 64 bit machine, and guess what, it didn’t work. STGMEDIUM contains a union, and the only way to do that in .NET is with explicit field offsets. This union is the second member of the structure, so I set the offsets to 4. The problem is that on a 64 bit machine, the second field is at offset 8. The solution was that I implemented the union in a separate structure, with all 0 offsets. And then it just worked! I have to admit, Microsoft did a good job with the 64 bit architecture: a 64 bit app will find the 64 bit implementation of the COM class which was compiled with 64 bit structures, and the same goes for 32 bits, because it will load the COM class from under the Wow6432Node in the registry.
And behold: DirectoryObjectDialog
All this work culminated in DirectoryObjectDialog.cs. It has a very easy to use interface. It hides the underlying nasty details, but not completely, you can still do with it whatever you can in native code. It’s also integrated with .NET’s directory stuff, so you can retrieve the results as Principal or DirectoryEntry objects, and also SID‘s. Here’s a sample code:
var dlg = new DirectoryObjectDialog { MultiSelect = true }; dlg.AddScope(DirectoryScope.Computer, users: true, groups: true); dlg.AddScope(DirectoryScope.Domain, users: true, groups: true); if (dlg.ShowDialog() == DialogResult.OK) { foreach (var sel in dlg.Selections) Console.WriteLine("{0}: {1}", sel.Principal.SamAccountName, sel.Principal.Sid); } |
Easy, huh? 
When you run it, it displays the standard dialog and prints the results to the console window. It’s all nicely documented, so there’s not much more I want to say about it. Feel free to use it!
Conclusions
First of all, the unmanaged API is ugly. It supports separate up-level and down-level scopes, and also up-level and down-level filters, but there are stupid inconsistencies between them, they don’t even share anything at the definition level. For example, in up-level scopes, you can select normal and built-in groups separately, but in down-level scopes, you can choose between all or all but built-in groups. (These differences pretty much decided what my interface looked like because I had to implement the common denominator in order to hide these levels.)
You can specify if the group, user or computer checkboxes are turned on by default, but not for the well-known principal checkbox. When you add multiple scopes, and don’t specify a starting scope, the documentation says that the first one will be the default, but during my tests, it always defaulted to the computer scope. The documentation also says you can’t choose multiple starting scopes, but it doesn’t throw an error either, I found that the last of them will be the default one. So in order to work in a predictable way, you should specify your scopes in ascending order of priority and say that all of them should be the starting one.
Second, the .NET marshaler is stupid. It works in most cases, but when it doesn’t, you’re facing a very hard road. Like I mentioned above, the biggest difficulties arise with arrays. I also hate that it uses the MarshalAsAttribute for parameter and structure marshaling, too, but there are certain differences, like some features can only be used in one context but not the other. It’s nothing but a bad design choice. So I put it on my to-do list to create a new, smart marshaler. The two wrapper classes in my framework code are only meant as temporary fixes.
And last but not least, I can see now that my framework is growing pretty nicely. Not only that, but it also looks just beautiful. Even the low-level Windows wrapping stuff, and that’s something. I think that very soon I’ll start providing binaries so that people can really start using my framework. Come on, people!