The Desktop Heap Guy

When I worked at Microsoft as an Escalation Engineer, I spent my days analyzing various failures in Windows. My team was called CPR, Critical Problem Resolution, and we were there to debug issues discovered in Windows by Microsoft’s customers. These problems came to my team because they were either “critical situations” for a business customer, or because the issue was particularly technically challenging.

In this role I worked on all kinds of weird failures, but one type of issue became a regular thing for me: desktop heap failures. I didn’t set out to be an expert on desktop heap, but when a more senior engineer on my team decided that someone else needed to understand desktop heap, I was asked to become the subject matter expert.

I started investigating every customer case that looked like the root cause might be a desktop heap failure. The symptoms were usually one of two things:

1. Processes would fail to start altogether.
2. An application would run but certain windows would fail to display.

This was perplexing for users and developers alike. The underlying cause wasn’t obvious, and the same software that had previously worked without issue would suddenly start failing. There was already some limited documentation on this problem, including a KB article, but I needed more information if I was going to really understand the problem and help customers.

I quickly learned that only a handful of people at Microsoft actually knew anything about desktop heap! So I studied the Windows source code, reached out to some other engineers on the Windows team, and began to pull together an understanding of the problem. I continued to get involved with every reported desktop heap failure, and eventually I became the unofficial Desktop Heap Guy. Within Microsoft support, people had learned that there was this person who actually wanted to help with desktop heap issues! This of course led to me getting even more cases to investigate.

Eventually I decided that we really needed some documentation on desktop heap. This led to me writing series of blog posts, and recording a Channel 9 video. Even better, changes were made to the memory manager in Windows Vista and Windows Server 2008 that helped avoid the problem altogether, and the problem mostly became of thing of the past. Even so, for years after I left the Escalation Engineer role at Microsoft I would still get questions about desktop heap. Once the Desktop Heap Guy, always the Desktop Heap Guy, I guess!

explaining_desktop_heap
That’s me, explaining desktop heap in 2007
Advertisements

MonoGame and shared projects

When targeting multiple platforms with Xamarin, a common approach is to use a Visual Studio shared project for the code or assets that are common. In the case of a MonoGame project, this means moving the .mgcb file to a shared project. However, a shared project doesn’t support a Build Action of “MonoGameContentReference”. A simple fix to this is to open the .projitems file from the shared project in a text editor, and change the ItemGroup entry for the .mgcb file from “None” to “MonoGameContentReference”. A disadvantage to this approach is that the Visual Studio IDE will no longer show the .mgcb file, but the build should work.

Proliferation of Dependencies

Ten years ago or so, I wrote lots of relatively small Windows applications. It was important to me to minimize dependencies on external libraries besides those available in Windows. I wanted to ship a single exe that anyone could drop on Windows 2000 or later and run it without installing anything else. To that end I wrote my code in C against the Win32 API directly. No MFC, no .NET, or anything else. If I could restrict my calls to functions in kernel32, user32, gdi32, and advapi32, all the better.

This weekend I’ve been working on a web application. While reflecting on my Win32 days, it struck me just how many dependencies I’ve taken to deliver this web application. Of course there’s the baseline trio of HTML, CSS, and JavaScript, but then I’m also leveraging Node.js, Express, Jade, Passport, Socket.io, MongoDB, jQuery, and Knockout, plus a bunch of other small npm packages. Wow, what a change from my old “minimize dependencies” philosophy!

Digest authentication with Express 4

While adding digest authentication to a Node.js / Express 4 site, I ran into an issue. I was using Passport middleware for authentication, along with the passport-http strategy. Authentication worked fine with an Express router for my root path, but when I added authentication to a sub path (e.g. /api/cats) every request failed with an HTTP 400.

After some debugging, I found that digest.js from passport-http checks that the request URL matches the URI that was supplied with the authentication credentials. This is a problem because a mounted route handler sets req.url to a path relative to the mounted path, so in the example above req.url is ‘/’ while creds.url is ‘/api/cats/’, and the string comparison fails.

It turns out I’m not the first to encounter this, and there’s already a pull request for a fix, but it hasn’t been merged yet. In the meantime, I didn’t want to modify the passport-http code, so I made a change in my code to work around the issue. I added a custom middleware function that was called immediately before calling passport.authenticate that updates req.url using the same approach as in the pull request fix. Of course a side effect of this is that now req.url is no longer relative to the mounted path, but that isn’t an issue for my code.

// Middleware to ensure req.url works with digest auth
function (req, res, next) {
    req.url = (req.baseUrl ? req.baseUrl : "") + req.url;
    next();
}