Jarrett House North: Microsoft

Getting Things Done with Outlook 2007, revisited

Tim Jarrett — Mon, 04 Feb 2008 20:34:10 GMT

A while ago I posted a few things that I found about implementing the GTD methodology with Outlook. Since I recently changed jobs, I’ve had an opportunity to carry some of the best practices forward as well as start from ground zero (a true Inbox Zero!) in some other areas. Here’s a quick roundup of what I did on my brand new inbox to facilitate maximum productivity.

The very first thing I did was to download and install Taglocity, which has saved my bacon so many times. I don’t know why people who design software to manage large volumes of information don’t get this, so I’ll just say why I find this so superior to the built-in Categories feature: it is much much faster to type in multiple tags for an inbound email than it is to make multiple mouse movements to pick multiple categories from a list. It’s fundamentally the same principle as why Keyword Assistant is absolutely necessary with iPhoto (at least, pre-2008). Email may be full text searchable, but from an actionability standpoint it’s just as opaque as photos until you give it context through tags. And the more tags, frankly, the better. All the UIs that assume that you’ll only be assigning one or two categories or tags are fundamentally broken because they don’t help solve the problem of how to find something later.

The second thing I did was to create exactly one sub-folder in my Inbox, called _archive. The underscore is a habit; it’s left over from when I had a billion subfolders and wanted to be sure my Archive folder bubbled to the top of the list.

The third piece was adopting the discipline that I’ve learned from practicing a little (a very little) GTD:

Scan each mail for actionability.
If it’s calendar related, triage it (right now that means “accept it” but a more complex triage process is required as my calendar actually gets full).
If it’s a task, do it quickly (< 2 min) or tag it and add it to the task list.
If it’s useful reference, tag it and add it into the archive.
If it’s none of those things, delete it.

Lastly, I set up a few smart folders: Tag folders (smart folders that look at categorized items across all my mailboxes, created through Taglocity) for all my projects; a smart mailbox for Unread Mail and for Unread or For Follow Up items. Today, I added one other smart mailbox—items in my inbox that weren’t flagged, meaning that they hadn’t been processed or moved to the task list. I also set up a custom Shortcut bar and added task age to my To Do list view. The last three items were based on the helpful advice from David Ornstein in this blog post.

Some stuff I might try to do in the future: custom button bars based on the posts by Simon Guest (and again) and Omar Shahine, and maybe tweak some of my task creation settings based on the advice by Melissa Macbeth.

And what has fallen by the wayside? The Hipster PDA was cool for about five minutes. I’ve graduated, on those occasions where I don’t have my laptop, to a little Moleskine notebook. But increasingly everything goes directly into Outlook. Likewise, I’m not bothering with the customized Project form hack mentioned in the same old post; it never worked well enough under Outlook XP for me to try bringing it forward into Office 2007.

And I’m on the edge about Google Desktop; while I was hooked on it before, I’m starting to think critically about the tradeoff between security and functionality that it provides, and I’m not sure I like the conclusions I’m drawing. More later.

Getting to Inbox Zero with Outlook and Taglocity

Tim Jarrett — Tue, 30 Oct 2007 18:06:45 GMT

A follow up to my earlier note about tags and Microsoft Outlook: I am happy to say that Taglocity has changed my life. I used to have folders in folders in folders and dealing with any received mail was torture. Now I’ve implemented tags and my workflow has totally changed.

I used to deal only with my unread mail, which was nice but it meant crud built up in my inbox. I used to flag mail messages as To Dos, but half the time I never got to reviewing the To Do list. Now I tag each mail message as it comes in (unless Taglocity can tag it for me), take whatever action is necessary on it, and move it to one archive folder. If I need to see a collection of messages about a particular subject, I use Taglocity’s filters or have it create a search folder for me.

My morning routine is a lot simpler too. I come in in the morning and the only things in my inbox are the ones that have come in since the night before. I delete most of the bacn, tag anything that I responded to the prior evening through Outlook Web Access (which doesn’t support tags), archive all tagged messages, and start processing all the new stuff.

The best part: that empty inbox. Now I work from my action list like I should have been doing all along. Inbox Zero is a good thing.

Some specific notes on Taglocity: using the Tag Cloud and other parts of the UI to assign tags and filters is a little challenging, since I tend to have a lot of tags. As in my tag collection in iPhoto, I find typing the tag name to be much, much easier. But having a Tag Cloud for my email is kind of cool anyway.

Outlook tags

Tim Jarrett — Fri, 05 Oct 2007 16:56:53 GMT

I am an email junkie. There, I said it. So the question is, what to do about it?

I have two problems with my work email (home is a story for a different day). First, I tend to save every message that isn’t outright spam or one-word answers—and it’s only recently that I started deleting the latter. Second, I have a file folder for everything, a habit that I started back when I first used Eudora in the mid-90s. It’s the second habit that is especially bad; it doesn’t scale worth a tinker’s when you are receiving over a hundred messages a day that are non-spam. (Yeah, I know. I threw up a little in my mouth when I wrote that.)

So what to do? First thing for me that really has helped is installing Google Desktop on my Windows machine. Much faster than the native Windows search engine, and with the double-control-key quick lookup, much easier to get into and use. But the next thing is to eliminate folders, and that is proving much harder. Because often the title line or even the content of an email doesn’t tell me which customer or software release it is in reference to, Google Desktop can’t find everything.

So I’m going to start exploring tagging. After all, it works well for me for Flickr/iPhoto. Here are some quick links about tagging hacks in Outlook:

Tagging Outlook tasks
Email tagging using categories and a custom edit-in-cell view
Taglocity, a commercial tagging solution with a limited free edition, including auto-tagging

Somewhere there is another tool that I really liked at Microsoft—it collapsed all the messages in a thread into a single mail message, deleted all the redundant text, and trashed the original messages. Now that’s efficient.

Great mysteries of life: WPF edition

Tim Jarrett — Fri, 28 Sep 2007 20:47:28 GMT

The Windows Presentation Foundation of Microsoft’s .NET Framework 3.0 gives you a lot of bang for the buck—for instance, it includes a free spell checker. Unfortunately, you sometimes get what you pay for. There is no ability to add a custom dictionary in the current version of the spellchecker. There also appears to be no documentation on which dictionary the information is being drawn from, where it is stored on disk—even where the ignore list for an individual user is stored.

So I tried some experiments: I created a Windows Search index over my AppData folder, opened a WPF application, and told it to ignore a misspelled word. I then searched for the misspelled word in my AppData folder and didn’t find it—meaning that the file containing the ignore list was not stored there. I even searched the registry and didn’t find the word. So where is it stored? It’s not in the base framework folders either...

Scripting data from SQL Server tables as DML

Tim Jarrett — Fri, 28 Sep 2007 15:31:11 GMT

(Warning: technical post ahead.) Ever since leaving the PowerBuilder/Sybase/ERWin world behind, something I’ve missed is the ability to easily generate portable SQL scripts for populating a table with test data. There are plenty of solutions in SQL Server for migrating data—DTS/Integration Services, BCP, and others. But DTS and Integration Services have to be maintained in the increasingly clumsy SQL utilities and cannot be easily inspected to see if things have changed, and BCP is opaque—you can’t really examine a BCP result file in any easy way to see what the data looks like within. No, give me DML—even if it’s bulky, a long list of INSERT/UPDATE statements has the advantage of being easily readable and even modifiable.

Unfortunately, there isn’t an easy way using the Microsoft tools to produce DML from existing data in a table; all the scripting support in the old SQL Enterprise Manager and the new SQL Server Management Studio are aimed at producing DDL scripts that create or modify the tables. Management Studio in SQL Server 2005 will create template scripts for insert or update scripts, but won’t actually put data into them—a curious omission.

SQL Scripter to the rescue. This nifty app offers the ability to script the data from any or all tables from a database as insert, update, or insert when new/update when existing statements. There’s even features for export of the data to CSV, Excel, and PDF. Pretty cool for a free utility. I’m now changing my process for creating a new demo database to use SQL Scripter to move my demo data from one environment to another.

Vista update: CSCService kills puppies

Tim Jarrett — Fri, 31 Aug 2007 20:56:25 GMT

Following up on my earlier post about built in system services sucking CPU: when we last left the story I had disabled the Offline Files service, better known as CSCService, as a likely candidate for my regular out-of-resources situation. Four days later, it looks clear that CSCService is the culprit. I have had no resource errors, no forced reboots, or anything like the pain I was experiencing.

This isn’t to say that life is roses now. Vista is still slow and seems to get slower (to the point of being almost unresponsive) under relatively light loads. But it recovers now and it never did.

So the next question is, what caused this process’s CPU and memory consumption to render the system unavailable, and why did it go haywire in the first place? I don’t know the answer to the second question, but I can only suspect that there’s something in my list of offline files that caused the service to start killing my system. I’ll try purging the list and reactivating the feature to see what happens.

But the other question: I’m pretty sure that the unresponsiveness has to do with the fact that CSCService was running in the same process space with half a dozen other services, including the window manager. Bad, bad, bad, bad, bad, as Juliette Lewis said in Natural Born Killers. I think I read something about changing the affinity setting for svchost processes in the registry to prevent this behavior; that might be the other thing worth trying to get the feature working again.

For now, I’m just happy that the perp has been fingered.

Heh: towel flap

Tim Jarrett — Fri, 31 Aug 2007 20:37:57 GMT

Who knew: the Microsoft gym towel flap turned into a real turnaround in Microsoft HR.

Microsoft screws up an Office Online feature launch by simulshipping it with an announcement of a delay in the Mac version of Office. How can Microsoft be surprised at the reaction that got?

A possible solution to Vista issues

Tim Jarrett — Tue, 28 Aug 2007 16:33:14 GMT

My previous exploration of Vista service packs and hotfixes led nowhere close to fixing my Vista issues. I was a little dejected for a while. But now I may have something to go on.

Excel 2007 just locked up on me today, as did Outlook. Recognizing the symptoms of an incipient total freeze-up of the system, I went in to take a look at the Task Manager. This once, I caught the conditions early enough that I was able to launch it and do some exploration. I quickly found a svchost process that was consuming a fair percentage of CPU (around 33%), and more troubling was also consuming memory—as I watched and investigated, it climbed from around 33 MB to over 60 MB.

I ran tasklist to see what that svchost process was running (svchost can run multiple services), but couldn’t figure out which process was the problem one. I found that if I right clicked on the process on the Process tab and chose Services, it would take me to the first service in the list that was running in that process. I then sorted the list of services by PID, opened a command prompt, and started net stopping the services owned by that PID systematically.

I found a few surprises; for instance, if you stop the uxsms process, which is responsible for the window manager, your screen goes totally black—but still accepts keyboard input. I was able to type in net start uxsms and bring back up the window manager. But none of the services I stopped fixed the climbing memory consumption, until I hit pcasvc, which is a service that is provided for compatibility with older versions of Windows. When I stopped the service, the memory usage stopped climbing and fell back, and I was able to do a clean reboot—though my Excel session never recovered.

A search indicates that other users have trouble with the same svchost process, though they indicate other culprits (ReadyBoost is one that gets mentioned). So there may be something going on here.

Update: Further testing indicates another possible culprit, which I disabled at the same time: CSCService, which supports Offline Files. It now appears pcasvc is OK. We’ll see if disabling CSCService does the trick.

Databound menu item names in XAML

Tim Jarrett — Thu, 23 Aug 2007 17:34:52 GMT

I keep telling the engineers who work with me that once we ship, we’ll have to write some articles with all the tips and tricks that we’ve discovered in Microsoft’s .NET Framework v.3, specifically WPF and WCF. The technology is easy enough to use, as I’ve written before, that even a product manager can do it.

My engineers challenged me to find a way to dynamically bind the name of our application into the menus, so that we would not have to update the menu names separately when we changed the code name of the product to the final released name. After some playing around, it turns out to be pretty trivial. As in my About Box example, you have to reference the AssemblyInfo class, which has the name of the assembly as a property that can be databound. Then it’s just a question of databinding the name to the menu.

With the way that menus are created by default in Expression Blend, this can look tricky because the text that is shown is in a parameter of the MenuItem tag called Header. What you have to remember is that WPF allows you to embed other controls into various contexts, but that you generally have to be explicit about how you’re doing it. So your code might have started out looking like this:

&lt;MenuItem Header="_Help">
  &lt;MenuItem Header="_About MyProgramName"/>
  &lt;MenuItem Header="MyProgramName Help"/>
&lt;/MenuItem>

After you break out the header element and embed the databound names, it ends up looking like this:

&lt;MenuItem Header="_Help">
  &lt;MenuItem>
    &lt;MenuItem.Header>
       &lt;TextBlock>
          &lt;TextBlock Text="About " />
          &lt;TextBlock Text="{Binding Path=Info.Title, Mode=Default, Source={StaticResource ApplicationBaseDS}}" />
       &lt;/TextBlock>
    &lt;/MenuItem.Header>
  &lt;/MenuItem>
  &lt;MenuItem>
     &lt;MenuItem.Header>
        &lt;TextBlock>
           &lt;TextBlock Text="{Binding Path=Info.Title, Mode=Default, Source={StaticResource ApplicationBaseDS}}" />
           &lt;TextBlock Text=" Help" />
        &lt;/TextBlock>
     &lt;/MenuItem.Header>
  &lt;/MenuItem>
&lt;/MenuItem>

It’s probably obvious enough to an experienced WPF programmer, but I found no ready references for making this happen so I figured I’d give it back to Google.

End in sight to reboot hell?

Tim Jarrett — Fri, 03 Aug 2007 20:27:05 GMT

I have been struggling with Windows Vista for a month or two. Regularly the system ran out of resources, regularly reboots were required to re-enable functionality. The symptoms were eerily reminiscent of classic GDI resource heap exhaustion: windows would refuse to open, pop-up menus stubbornly stayed closed, applications reported an inability to save to disk or access the registry.

I am now trying various patches to see if I can fix the problem. After a blue screen of death (yes, those still happen on Vista), it occurred to me that the problem must be in a device driver; after all, that code gets to play at a privileged OS level where it can do things like attempt to overwrite read-only memory. I suspected the video driver, and attempted to use an update from Intel’s web site to update the driver (for the record, it’s the Intel Mobile Chipset 945), but was told that the upgrade was not installable on my machine (an HP Compaq NC6320 laptop).

Now I am getting a message from the Windows problem reporting system that one of my issues may be fixed by a hotfix for KB 931671. We’ll see if this does fix the problem, or if I continue down the path of no return with this OS. Already not a good sign: I am being forwarded for the second time in a 20 minute call to another department because the reps I have been speaking to are not authorized to distribute the hotfix.

Vista: Very, very hungry

Tim Jarrett — Mon, 23 Jul 2007 17:01:20 GMT

I run Windows Vista at the office. Generally I get along with it just fine, and our company’s software plays pretty happily with it. But every now and then in my daily work I hit some kind of wall. Sometimes it manifests as a problem with Microsoft Outlook: when I try to launch Word to read an attachment, it starts up the Office Installer instead, then complains that it is suffering from "Windows Installer error 1450" and can’t proceed. Cancelling or clicking OK brings me to the same place: a copy of Word that complains that it hasn’t been installed for the current Windows user.

Other times, the problem manifests as a refusal to open other software applications, even Notepad, or to open new explorer windows. When I hit this point, even clicking on the funky little restart menu to try to get to the restart menu option won’t open the submenu. I have to hold the power key down to force the power to cycle.

It feels for all the world like the bad old days of Windows 3.1 or Windows 95 when one of the system resource heaps would be exhausted. But that shouldn’t be happening in Vista, or any post-NT OS, for that matter.

What’s weird is that applications that are already open, e.g. Firefox, appear to run just fine as long as you keep them running.

I can’t find anything on Microsoft.com or on the web at large about the issue, so I’m posting something to jog my own memory the next time I run into the problem.

Product managers writing code, and other scary things

Tim Jarrett — Mon, 02 Jul 2007 16:29:40 GMT

When your product manager checks code into your source control project, it means one of three things:

Your engineering team is short-handed.
Your project is really in trouble.
Your development environment is really, really easy to use.

While I’m not sure that #2 isn’t true, I found #3 to be absolutely true working on a recent software release with my team. We’re using the Windows Presentation Foundation, aka Avalon, and I used Microsoft’s Expression Blend to make the project’s About window.

WPF is designed to enforce separation of code and presentation, and tools like Expression Blend enable that. The presentation markup enabled me to automatically include the current assembly information (version number, product name, copyright string), plus free memory information, and even a little animation.

The trick was in learning how to do data binding. I specified various modules of the .NET Framework as data sources, then bound text objects to methods in the framework. Interestingly, the easiest way to get assembly information was in the Microsoft.VisualBasic.ApplicationServices namespace, in a class called AssemblyInfo; of course it doesn’t matter since the same code can be accessed by any .NET compatible language, including C#. The only challenge came in showing the available memory, since the methods in Microsoft.VisualBasic.Devices.ComputerInfo return memory in bytes. I wanted to show the information in megabytes, so I attached some simple C# code as a converter to represent the information in megabytes and format it appropriately for the user’s region.

Like I said: simple.

Holy crap: an honest to goodness Easter egg

Tim Jarrett — Thu, 03 May 2007 05:02:36 GMT

Thanks to Macintouch, I just spent a virtual eternity (okay, five minutes, but these days that feels like an eternity) playing Asteroids. The original Asteroids. Tucked away inside Microsoft Office 2004.

Word was a while back that Microsoft had put a moratorium on Easter eggs. I wonder how the team slipped this one in. I wonder whether Rick Schaut would have anything to say on the topic, if we asked him nicely.

Thought for the day: Platform lock-in, good and bad

Tim Jarrett — Wed, 02 May 2007 23:04:33 GMT

A review of a new PowerShell book on Slashdot features a great comment from an anonymous coward who gives the best argument against supporting multiple platforms that I’ve ever read:

I hate re-using code because it forces me to solve new problems every day. I'd rather create new value on Mondays only, and then spend the rest of the week re-doing the same work on my other platforms. It gives my mind a chance to rest, and I can drink heavily mid-week and still be able to do my job.

I sure hope they charge extra for it, make it a resource hog, lock out third-party extensions, and then discontinue it as soon as I'm dependent on it. I really liked the 1980s and look forward to reliving them.

The nice thing about the comment is that it contains the pro and con of supporting multiple system architectures back to back, and both perspectives are funny, and true.

Webex Outlook Addin and Outlook 2007

Tim Jarrett — Wed, 25 Apr 2007 19:43:23 GMT

If, like me, you live and die by Webex, you might have been as frustrated as I was to find that the Outlook add-in for Webex (which allows you to schedule online meetings right from your Outlook calendar) doesn’t install on Outlook 2007. I was quite surprised to find this, actually, since I had happily been running it on my old computer. But I realized I had actually migrated it—I had originally installed it against Outlook 2003, and it had happily continued to work when I upgraded to Outlook 2007.

So I hacked my way into making it work. Basically all you have to do is start from a working installation of the add-in (downloadable from your Webex account page), then copy the binary files from the Webex program files directory to the machine with Outlook 2007, register the DLLs, then add a registry entry that registers the add-in. On my machine, the registry entry for the last step looks like this:

Windows Registry Editor Version 5.00

[HKEY_CURRENT_USERSoftwareMicrosoftOfficeOutlookAddinsOutlookAddin.Addin] "FriendlyName"="ADOutlook2K Addin" "Description"="ATLCOM Outlook Addin" "LoadBehavior"=dword:00000003 "CommandLineSafe"=dword:00000000

Note that installing this WILL NOT work unless the DLLs from the add-in have properly been registered.

Once I did the above steps, I simply recorded my sign-in information and was able to start scheduling meetings again. Yee haw.

Update: Apparently WebEx has fixed this issue, though it doesn’t explain why I had to hack and post about it before anyone told us that it was an issue, much less that they had fixed it. Sigh.

Zune Phone to suck just as hard as the Zune...

Tim Jarrett — Wed, 21 Feb 2007 08:43:11 GMT

As someone who is counting the days until his current Cingular contract expires in June so that he can pick up an iPhone, I thought the rumor that Microsoft was planning a Zune branded phone was pretty funny. Because, of course, the Zune brand has shown such market power to date that it has completely destroyed the iPod’s hold on the market.

Oh, wait. It hasn’t? Um, never mind.

But the take of the Crazy Apple Rumors Site on this rumor is one of the funnier things I’ve read in a while:

Beyond just the name, however, sources indicate that the Zellular Phone Call will have certain limitations inherited from the Zune platform.
For example, all calls will be wrapped in Microsoft’s DRM and the end-user license agreement will state that the contents of each call will be the property of Microsoft in perpetuity. Also, the Zellular Phone Call will only allow you to call someone three times. After that, every time you try to call that person you will hear a recording of a representative of the Recording Industry Association of America calling you a thief and yelling obscenities and threats at you.

Heh.

I would need an iAntacid

Tim Jarrett — Wed, 29 Nov 2006 17:11:40 GMT

I am working at home this morning so that my hacking cough (getting better) does not disturb my coworkers. One benefit of doing this is being able to drink delicious, delicious home brewed coffee… so much better than the stuff from the single-serving machine at work. It is perhaps a good thing that our office does not have the Starbucks iCup machine (as seen at Microsoft). My already precarious digestive balance would be upset permanently.

Thanks to Jenni, whose blog I’m returning to after too long a hiatus, for the link.

Waiting for Vista...

Tim Jarrett — Wed, 22 Nov 2006 19:20:03 GMT

I have a new laptop on the way at work, and, yesterday’s post notwithstanding, I’m looking forward to getting it so that I can load Windows Vista on it.

Why, you may rightly ask, would I want to do that to myself? Well, Vista is the first Microsoft OS in seven years that I haven’t used as my primary OS while it was in prerelease status, so I’m feeling a little behind the curve. But also it just feels like time. I’ve been using XP since 2001 in prerelease form (starting during my summer internship), and all the novelty of the new issues in XP has worn off. I’m tired of:

Long delays in the UI when booting and waiting for services to start
Long delays in the UI when switching from one network to another or into disconnected status
Bad power management during sleep (see yesterday’s post)
Weird screen switching behavior
Needing to reboot all the time
Three or more keystrokes to create a folder

Will any of this change in Vista? At this point, I have no idea, and that’s what I’m looking forward to exploring.

Windows Live Writer and Manila blogs

Tim Jarrett — Mon, 14 Aug 2006 22:02:43 GMT

There's a little virtual ink today about Windows Live Writer, a blog writing tool that launched on Friday and apparently uses the MetaWeblogAPI to publish to MSN Spaces or other blog authoring platforms. Interestingly, former ColdFusion guy JJ Allaire appears to be involved with this project, which to me suggests there’s more than meets the eye here: no way JJ would get involved with something that is only a blog publishing tool.

As always with these tools, I can’t test the auto-configuration feature because the version of Manila on my blog server doesn't implement the getUsersBlogs method of the MetaWeblogAPI. Digging deeper it looks like the blog information is stored in the registry under HKEY_CURRENT_USERSoftwareWindows Live WriterWeblogs. So I created a profile for another site that used the MetaWeblogAPI and tried editing the values to point to my Manila site instead. I'll know in a second if that worked. —Oh well. Attempts to post are failing with a message Can't split the URL because it is not of the form 'http://www.server.com/hello.html'—I assume that this is referring to the RPC URL on Manila servers, which has no . extension or trailing slash but just appends /RPC2 to the top level domain.

Out of box functionality: no way to do HTML entity conversion is apparent (which is why this post doesn’t have curly quotes).

I think this tool would probably look and work better with a more modern blogging platform; the automatic WYSIWYG preview is pretty cool for blogging platforms that support it, for instance. But Flock still works better.

Why do DVRs suck: reliability or user experience?

Tim Jarrett — Mon, 31 Jul 2006 19:57:34 GMT

A Microsoft friend of mine was quoted in a CNET article this morning (Why my cable DVR stinks). Arvind (whose last name was misspelled in the article) said (and this is something I’ve heard from him before) that the rich-client, local processing model that Windows Media Center uses is a much better way to provide DVR features than a networked DVR model such as the one Cablevision is trying. Aside from being consistent with Microsoft’s general vision of keeping the end nodes smart, I think Arvind has a point about the need for a “much richer user experience.” Certainly a graphical UI is easier to sort through when deciding what to watch than a text listing, even a well designed one—especially in a 30 foot UI.

I think the bigger issue is the reliability one, and here Arvind may have overstated his point, which appears to be that Vista is a step on the way to reliability nirvana. I don’t care how reliable media center PCs are running Vista. They just have to be more reliable than the equivalent set-top hardware, including Tivos and Comcast’s proprietary DVR. (We had two total Comcast DVR meltdowns before we gave up and went Tivo.)

The problem with the cable guys is that they refuse to acknowledge the need for moving the content off the box. Hey guys—RIAA vs. Diamond Multimedia showed that space-shifting content was legal seven years ago. Why does Comcast still make it impossible to get my content off their DVR? At least my new Tivo lets me burn shows to DVD, though I would prefer it if I could back up the hard drive directly over the network.

Congrats to Mark Russinovich and Sysinternals

Tim Jarrett — Tue, 18 Jul 2006 21:55:18 GMT

Slashdot: Microsoft acquires Winternals and Sysinternals. Regardless of how you feel about Microsoft, this is great news for Winternals the company and Mark Russinovich the industry figure. (For those that don’t recognize the name, think Sony BMG: Russinovitch’s blog at Sysinternals blew the whistle on Sony BMG’s rootkit.)

It’s clear that this is a talent acquisition; Microsoft has said they are aware of some product overlap with Winternals’s product line, which generally means some sort of phased migration plan is in order.

I think the Slashdot advice to download the free Winternals utilities now is a very very good idea. I always forget that Regmon exists until I need it, and then I wonder how I lived without it.

I also find the statement that they’ll rationalize the Sysinternals community features with Microsoft.com offerings somewhat disturbing. If the value of Mark’s blog, for instance, is its refusal to spout the Microsoft party line and thus carrying a strong reputation for truthful investigation into technical issues, aren’t they destroying some value by bringing him into the fold? Or are they afraid of having another high-profile blogger get too much of an independent rep, as Scoble did?

A farewell to Gates

Tim Jarrett — Fri, 16 Jun 2006 21:40:14 GMT

I haven’t said anything about Bill Gates’ announcement that he’ll be stepping out of day-to-day work over the next two years, yielding the chief architect reins to Ray Ozzie. Primarily it’s because my life has been pretty busy, but partly it’s because Gates has seemed, from the outside at least, like a non-factor in recent years. With Vista coming to market without big features like WinFS and several years overdue, and with Microsoft continuing to struggle to get customers to re-up for the newest versions of Office, it feels like Bill hasn’t had anything really new and compelling to show the market in a really long time.

Scott Rosenberg does a good job of connecting the dots in his supposition about Bill’s role in the Vista slip: on the one hand Bill had to watch as the scope of Vista was pared down and many revolutionary features were put aside, and on the other the culture that Bill and the Windows team had fostered meant that slips in the schedule were never acknowledged until it was far too late (thanks to Microsoft blogger Philip Su for some incisive and honest observations about the management culture in Windows).

My perspective from a greater distance is this: Bill and Steve Ballmer centralized Microsoft’s strategic decisions to an enormous degree, despite the generally broad operational freedom that individual product units enjoyed. And it’s not clear that that organizational move paid off. Might Microsoft be further along the path in dominating the enterprise software market—a place where the company does well with SQL Server and Exchange but has yet to make a splash with other business and IT apps despite years of investment—if the enterprise business had been able to duck the Windows tax? And by that, I mean the internal Windows tax that every business at Microsoft faces: every business has to show how it’s relevant to the corporate cash cow, and heaven forbid that the busines s plan suggests that the new initiative should embrace a multi-platform, multivendor view of the world instead of the Windows party line.

The result is that Gates rides off into the sunset as the champion of the desktop, but with many goals for Microsoft in the enterprise, on mobile devices, and in the home left unrealized. And with many users frustrated by years of vulnerabilities and diminishing returns.

Can Ray Ozzie turn this around? Maybe. He certainly has more enterprise cred than Bill. I’m not convinced that his vision is as broad as Bill’s, but that just might be a good thing for the long-term health of the company.

Welcome to the alumni club, Scoble

Tim Jarrett — Mon, 12 Jun 2006 14:22:40 GMT

Boston Globe: Blogger who often rapped Microsoft will join a start-up. What a misleading headline. That’s like saying “Man who sent emails daily leaves company.” I saw Robert’s tweaking of Mr. Softie as an important part of his working to build credibility with tech influentials, the people whom Microsoft most needed to win over. If your employer does something stupid, and you are trying to model a behavioral pattern of honesty and transparency through public discourse, you don’t clam up, you call them on it. And Scoble did that, time and time again.

I appreciate Scoble’s honesty. A lesser man would have claimed credit for the groundswell of blogging that happened at the company during his tenure. Scoble wisely disclaims, “I’m not the only blogger at Microsoft. There are about 3,000 of them here. They are not having the plug pulled on them. They changed the world. I just was the cheerleader.” But by his very public risk taking, Scoble made the world safer for them against some old school Microsofties who wanted badly to take them down.

All of which is to say, it’s very odd that the Globe chose to print some random laptop-toting schmoe’s picture instead of Robert’s with the story. Too bad it’s not available on line. It gave me a good chuckle.

Feeds: Microsoft gets RSS

Tim Jarrett — Mon, 08 May 2006 03:35:12 GMT

Tip from Dave Winer: this appears to be a new central directory for Microsoft.com RSS feeds. Nice! Wonder who’s responsible for the work. The layout actually looks much better than the Microsoft Blogs page, much as it hurts me to say it.

(Mis)Use Case: Vista User Account Protection

Tim Jarrett — Fri, 21 Apr 2006 15:24:37 GMT

Paul Thurrott: Where Vista Fails. A long list of major and minor feature and UI issues in the latest (February) community preview of Vista, the next version of Windows. Some of these issues seem minor, but one in particular, the User Account Protection model, caught my eye. It’s good to see that Windows will be moving toward a model of requiring separate point authentications to perform certain actions, but it sounds like they’ve overdone it with the actions that require re-authenticating, not to mention the warning dialogs:

Once Firefox is installed, there are two icons on my Desktop I'd like to remove: The Setup application itself and a shortcut to Firefox. So I select both icons and drag them to the Recycle Bin. Simple, right?
Wrong. Here's what you have to go through to actually delete those files in Windows Vista. First, you get a File Access Denied dialog (Figure) explaining that you don't, in fact, have permission to delete a ... shortcut?? To an application you just installed??? Seriously?
OK, fine. You can click a Continue button to "complete this operation." But that doesn't complete anything. It just clears the desktop for the next dialog, which is a Windows Security window (Figure). Here, you need to give your permission to continue something opaquely called a "File Operation." Click Allow, and you're done. Hey, that's not too bad, right? Just two dialogs to read, understand, and then respond correctly to. What's the big deal?
What if you're doing something a bit more complicated? Well, lucky you, the dialogs stack right up, one after the other, in a seemingly never-ending display of stupidity. Indeed, sometimes you'll find yourself unable to do certain things for no good reason, and you click Allow buttons until you're blue in the face.

Hopefully this gets adjusted in future builds. Otherwise I think a lot of people won’t get the benefit of the feature—they’ll disable it based on the annoyance factor.

Vista slips, employee grumbles go public

Tim Jarrett — Wed, 22 Mar 2006 17:42:43 GMT

In the software industry, it’s predictable that major releases slip. The more features that get added in, the more ambitious the release, the higher the testing burden, the greater the risk of incompatibility with other products, the more complicated the interaction matrix between features, the bigger the regression risk. More risk = more uncertain schedule.

So the announcement that ~~Longhorn~~ Vista, which was at one time to have shipped last year, is slipping broad consumer availability into 2007 is unsurprising. (The discrepancy between the November delivery date for businesses and January availability for consumers has to do with delivery models. Businesses can get upgrade versions under Software Assurance; the assumption is that consumers will be waiting for new PCs to come pre-loaded with Vista through the channel, which involves a manufacturing delay. I assume that’s the difference, anyway, and not that there are actual code differences between the two versions.)

What is a little more interesting is the level of public griping that is coming from Microsoft bloggers like Mini-Microsoft, who is questioning the apparent lack of accountability in senior management (“Fire the leadership now!”). What is astounding is the level of bitterness and dissatisfaction expressed by various anonymous Microsoft employees in the comment thread. One says that accountability will be seen “this August when reviews are handed out to junior employees.” Another complains about problems on the Vista application compatibility test team: “Cut the number of testers (several times) from approx 50 to now much less than a dozen” and notes that application compatibility measures are hovering at “< 40 percent.”

This is the downside of blogging, for Microsoft the public agency: all the dirty laundry gets exposed, all the internal secrets get aired. Of course it is an upside, too. The anonymous comment thread is actually shedding some light on real management problems at Microsoft that otherwise would continue to be swept under the rug. They might still be swept under for all I know, but the possibility is there of having that discussion.

Of course I can’t resist a little schadenfreude over one gripe aired in the course of the thread: “What's the difference between OS X and Vista? Microsoft employees are excited about OS X.” That’s a little unfair, of course, but it’s also funny, and as Apple pushes forward toward its fifth major OS release since Windows XP while Microsoft struggles for the first one (and alas, in this game, major efforts like SP2 don’t count as more than point releases), it sounds like the truth.

Disclosure: I am a former Microsoft employee who did not work on Windows, though I have friends who do. While I am a Mac user at home, it is in my professional interest that Microsoft keeps the IT ecosystem healthy by shipping Vista on time.

GTD with Outlook Part III: A high level strategy

Tim Jarrett — Thu, 16 Mar 2006 18:41:13 GMT

So far in my ongoing review of implementing the GTD methodology using Outlook XP, I’ve talked about using improved search to make your archives more useful and managing your task views. Today I’m going to take a step back, now that I’ve implemented most of the GTD workflow in my daily routine, and give a higher level picture of how everything has been implemented for me so far and what challenges remain. I will give an outline of my project list implementation but the details will wait for next time.

First: my new strategy to manage stuff in Outlook is simple. The inbox stays clear; I have a list of tasks from which I work on an ongoing basis, and a list of projects that I review daily for next actions. If I’m ever in a place where I can’t make a task note directly, I use my brand spanking new Hipster PDA (a stack of 3"x5" index cards held together with a binder clip), and transfer any tasks to my task list when I get back to my desk. (The Hipster PDA is particularly useful at the breakfast table, on my bedstand, and other places where the computer should never be.)

That all sounds simple, but the devil is in the details. As I outlined last time, something as simple as how you view your tasks makes a big difference. And the really difficult part, as the GTD methodology attests, is keeping your task list clean and free of multistep projects, which are treated differently. The problem is that Outlook doesn’t provide a good form for project management. So I’ve been running with this recommendation from the Getting Things Done In Outlook page, which provides a modified contact form as a way to track projects together with a customized contact folder and view. Then I build out the project plan in the description field (or, if the project is to write something, I’ll brainstorm the outline right there), and click the “New Task for Contact” button to add the next action for the project to my task list. Doing that adds a link to the project form into the Contacts field at the bottom of the task list, and makes the task show up in the Activities tab of the project form.

So that’s all the major areas of GTD—except that there are a ton of additional details and neat features that I’ve glossed over. Next time I’ll talk about more task tips and tricks, including features in tasks that support the creation of deferred tasks.

GTD with Outlook Pt 2: Task views

Tim Jarrett — Thu, 09 Mar 2006 04:50:10 GMT

I’ve spent another week and change on GTDsince implementing good search in Outlook. I’ve spent it exploring the core concepts: capturing all (most) of the stuff that’s been floating around in my brain waiting to be done or dealt with. Discovery number 1: while it’s a relief to capture all (most) of the stuff into lists so I’m not spending all the time panicking about things I’ve forgotten, I’ve learned that I’ve made a lot of commitments that need to be fulfilled. Hence my relatively light blogging as I regain some balance.

My capture systems are just now starting to get in shape. I spent much of my airplane time between here and Vegas (and back) setting up some of the systems and have some progress to report.

First thing: my new favorite view in Outlook (XP version) is the Calendar view, with the small Task pane to the side. Something about having the task view shrunk to a manageable size is really helpful in preventing it from being terrifyingly unmanageable. But the out of the box view, which features only in progress tasks ordered by no particular mechanism and ungrouped, needs work. We can do this.

If you right click on the view title (“Tasks”), you can edit the view definition. I found it most helpful to group it by status. The out-of-box statuses (statii?) in Outlook are Not Started, In Progress, Completed, Waiting for Someone Else, and Deferred. That turns out to be just about perfect for GTD task organization. I have yet to use the Deferred status; I keep the Completed group collapsed and open it when I need motivation. That leaves Not Started, In Progress, and Waiting for Someone Else. Email “next actions” that need responses go to Waiting and everything else keeps getting worked.

Okay, but what about when you need better statuses? This is where Categories come into play. I did a quick listing of categories that made sense from a “next action” perspective: phone, email, mail, computer, research, writing, errands. Then I went to the main Tasks folder, went down the list adding categories, and grouped the whole shebang by Status and Category. I also set the view to show the long description column when present, so I could glance at the list and see information like, for instance, phone numbers. Advantage: the main Tasks view can become a printable view to bring with you for errands and phone calls.

Next up: project lists and how there’s not a perfect system for tracking them in Outlook… yet. (But how one kind of project list freed me from keeping project names in the Category field.)

GTD part 1: improving your archives

Tim Jarrett — Mon, 27 Feb 2006 17:09:35 GMT

As mentioned last week, I’m trying to improve my workflow by looking at the Getting Things Done (GTD) methodology and specifically thinking about how GTD applies to Outlook. One of the references I came across recommended some more general purpose solutions to improve Outlook that at first I couldn’t reconcile with GTD—what does a better search tool for Outlook have to do with GTD? Everything, it turns out.

I have long been a “filer” with my email. Both my home and work accounts have dozens of dedicated folders, some relating to projects, some to broad topics like “Company,” “Personal,” etc. While this isn’t necessarily a bad thing, the problem is that of course most emails don’t fit neatly into one category, and it can be challenging to find something after I’ve filed it—which of course defeats the purpose of an organizational system.

The built in search in Outlook (I’m using Outlook XP, but I seem to recall the same problem when I used Outlook 2003 at Microsoft) doesn’t help matters much either. Searching just the content in a single folder is dog-slow, and if you want to search across all the folders in a mailbox you might as well go brew a fresh pot of coffee.

Enter Lookout, a dedicated plugin to Outlook that quickly, efficiently, and quietly indexes the contents of your Outlook mailbox and makes retrieval lightning fast. The software is so good that Microsoft bought the company a while back and uses the technology as the core of the MSN Toolbar Suite to index your whole computer. But the MSN Toolbar Suite (and Google Desktop) have always given me the willies for some reason. I don’t like running system wide utilities and I don’t necessarily see the utility of indexing everything on my hard drive when (a) most of my work is on Outlook and (b) the rest is in relational databases or on network drives. Lookout has just about the right scope for my comfort zone, and it works extremely well.

From a GTD perspective, Lookout increases my comfort with saving items for reference and getting them out of my inbox. It also makes me think critically about what I’m saving and whether I ought to be throwing some of it away (horrors!).

Getting Things Done and Outlook

Tim Jarrett — Fri, 24 Feb 2006 00:29:51 GMT

Confession: I am a lapsed Franklin Covey user, a former Palm user, and otherwise the former user of more productivity methodologies than I can count. So I have read Merlin Mann’s 43 Folders, a blog on implementing productivity workflows on a Mac using the Getting Things Done methodology, with healthy skepticism for the last year or so. One big knock is that for some reason Merlin’s preferred tool, Quicksilver, has always run like a dog on my system. But I finally started reading the actual Getting Things Done book and am convinced that I ought at least to give it a whirl. The idea of ruthlessly keeping the mailbox and other sources of angst clean, immediately dealing with, deleting, incubating, or delegating incoming “stuff,” and totally outsourcing your worry center, all sounds really good to me.

Except, of course, the one really good source of tips I have for GTD, 43 Folders, is all about Mac based solutions. And in spite of my long standing Mac userdom, my work environment is still a Windows XP PC running Outlook.

So I’m going to give some Outlook based solutions a whirl and talk about how they work over the next few days. First off, a few pointers to existing resources, since I’d rather not reinvent the wheel:

The 43 Folders wiki has a page on GTD in Outlook
A great, if old, summary page on setting up GTD in Outlook
The official ($10) resource from David Allen Company on GTD and Outlook
Managing GTD projects in Outlook
Tips, tricks, and other hints in Outlook
An actual GTD add-in

Back on the road...

Tim Jarrett — Tue, 21 Feb 2006 07:52:45 GMT

For a day or so, anyway. And I don’t have a lot to say, except that untangling someone’s spaghetti SQL code is even less fun to do when it’s been developed in Crystal Reports.

For those of you who don’t know what I’m griping about, thank your lucky stars.

Just when you thought the browser wars were over...

Tim Jarrett — Tue, 31 Jan 2006 23:26:43 GMT

The public beta of Internet Explorer 7 hit today. Reaction: Dave Winer, CNET, PC Magazine/ABC News/Go, PC World, RealTechNews, Ars Technica.

The ironic thing is that the folks at A List Apart posted a new CSS based fluid three column layout today, claiming they had found “The Holy Grail,” a three column layout using CSS and a minimum of browser specific hacks. Unfortunately for them, as promised, the layout breaks on the IE7 beta.

Other notes: the Google toolbar works without complaint; my home page and default search provider settings were honored, even in the new dedicated search field; RSS (or “feeds”) autodetection works as promised; the built-in RSS reader is category aware and provides some nice search and sorting features; and lots of other stuff.

Friday crazy rumors day: Clinton replacing Ballmer?

Tim Jarrett — Fri, 06 Jan 2006 22:46:01 GMT

Boy, is it fun to speculate and what-if today. Namely, what if this rumor is true? The one that has Steve Ballmer stepping aside as Microsoft CEO ... in favor of Bill Clinton???

For one thing, you’d probably see a lot of red states joining Massachusetts’ ex-CIO in putting policies in place to get Microsoft out of their government. It would probably be the first time that open source advocates and most mouthbreathing conservatives would agree on anything.

Nonetheless, I will say that the only thing that could convince me this rumor is true would be if I looked out my window and saw pork on the wing.

(Via Scripting News.)

Allchin: moving out

Tim Jarrett — Wed, 21 Sep 2005 23:07:57 GMT

As long as I’m shooting my mouth off about the industry: will anyone miss Jim Allchin? The news that he’s retiring next year draws a major chapter in Microsoft’s history to a close. Allchin presided over both high and low points in Windows’s history, including Vista (f.k.a. Longhorn), which can’t decide if it wants to be the coolest thing since sliced bread or the most troubled Windows since version 3.0.

Allchin is known to be a ferocious competitor, and questions about his tactics, including the infamous Burst.com email deletion flap, have surfaced throughout his tenure. The insistence on tying Internet Explorer to Windows and “leveraging” the Windows monopoly into control of the Internet comes to mind as a less civilized moment, as does his admission in Congressional testimony that release of Windows source code would endanger national security due to flaws in the code.

Paradoxically enough, that’s one thing I will miss about Allchin: his willingness to speak up. In an industry where there are too many press release mouthers, his calling open source software an “intellectual property destroyer” was entertaining, if not as entertaining as Steve Ballmer calling it a cancer.

I only ever was in one meeting with Jim Allchin, and all I can say about him is that he was very intelligent and very hard on his people when their ideas weren’t crisply defined and clearly thought out.

Resolving permissions issues in SQL Server

Tim Jarrett — Tue, 30 Aug 2005 17:52:08 GMT

I thought it was simpler in SQL Server 2000 than it turns out to be to move a database from one server to another and restore permissions. At least Microsoft provides a stored procedure to map existing SQL Server logins to users in the database being moved.

New machine setup joys

Tim Jarrett — Fri, 22 Jul 2005 18:55:18 GMT

I seem to be averaging 0.8 laptops per month at this firm, not that I’m complaining. The new machine finally has enough RAM and processor power to run both our OEM’d applications and our core technology servers without breaking a sweat. Of course, there were more than a few minor things to be done in transferring everything over.

The one thing I always seem to forget is to install IIS on a new machine before upgrading to the new version of the .NET Framework. If IIS is installed when the new version of the framework is loaded, it can register ASP.NET automatically. If IIS is not installed, setting up ASP.NET is a manual process.

It’s easy enough to do, fortunately: just navigate to %system directory%Microsoft.NETFrameworkv.1.1.4322 (or whatever) and run aspnet_regiis.bat. And bounce IIS for good measure. Maybe now that I’ve written it down I won’t forget next time...

Outlook command lines

Tim Jarrett — Tue, 19 Jul 2005 16:58:57 GMT

An Outlook link that popped up while I was writing the previous post, but that didn’t fit the theme, is this list of command line switches for Outlook. Most of it is break/fix stuff—reset profiles, start without extensions, etc., but some of it is useful for integration with other applications.

The calendar culture

Tim Jarrett — Tue, 19 Jul 2005 16:49:18 GMT

I was forcibly reminded yesterday that many aspects of software that we consider intuitive and automatic are actually cultural, and have nothing whatever to do with GUI design. My mini-Waterloo? Calendar management and responding to meeting invitations.

I was baptized early in my career in the religion of managing a calendar. Early on it was the Franklin Planner, which really is a religion. Then it was the calendar system in Lotus Notes. At Microsoft, of course, it was Outlook. —Of course, Microsoft takes calendar management to dizzying heights. Invitations are sent to secure half-hour chat times with someone down the hall. People spend hours looking at shared calendars for multiple individuals to find ideal meeting times. Calendars are blocked off with “work time” appointments so that you don’t get pulled away by someone who has observed that you don’t have anything scheduled for the afternoon before something is due. When you can’t find a time for everyone to meet together, you start triaging meeting attendees and making calculated decisions about who can reschedule their conflicts and who cannot be moved. And finding spare time in a conference room becomes an art in itself. All of this is done through Outlook’s interface without ever speaking to anyone.

With that as background, it is perhaps a little more understandable that I forgot that the calendar culture is a culture, and not everyone understands its rules. So I shouldn’t have been surprised when no one RSVP’d for a functional spec review for which I sent out an invitation two weeks previously. I was a little irritated that day when no one showed up, however. I spoke to my counterpart in engineering about it, and he was sanguine: “I don’t use Outlook’s calendar,” he replied. “I view the meeting invitation as a reminder about the time, and then I delete it.”

Computing is a cultural artifact. Things like the “Accept,” “Decline” buttons on invitations only have meaning in a shared context where everyone agrees on how they are used. This is one of the reasons that specs are important, of course—they provide a formal definition of an agreed shared context for how something appears and is to be used. But we can never forget that specs and user interfaces and user scenarios are just the beginning. As William Gibson wrote (several times) and I am continually relearning, “The street finds its own uses for things.” It is how your users interact with the software that defines what it can do, not what is written in the spec.

Others on the Outlook culture of meeting management: Jeremy Gilby on meeting request filtering and Kirk Allen Evans on taking a Microsoft class on using Outlook more effectively. Allister Frost is practically a computer anthropologist’s dream. Read these posts and ask yourself why you would want to filter the Outlook 2003 calendar by labels, think about sending information as tasks vs. calendar items, integrate tasks and the calendar to block out free time, memorize a keyboard shortcut to transform emails into calendar appointments, or use Outlook as a backup brain to remind yourself what you have been doing.

Whither VB? The withering away of a category killer

Tim Jarrett — Mon, 20 Jun 2005 16:17:27 GMT

Rogers Cadenhead posts about something that has been on my mind recently, the ending of support for Visual Basic 6. A while ago Scoble and Dan Appleman blogged the opposing viewpoint: making customers move their apps off VB6 is the right thing to do because VB.NET is a far superior language.

Let me illustrate a real world perspective on the debate, rather than the strawman that Appleman provides. Say you are a small software company that makes a product that is written in a variety of languages, and a core part of the application suite having to do with automatically sending e-mails is written as a Visual Basic 6 application.

Now why would someone write an email notification process in VB6? I don’t know. Why would someone write the standard procurement system for the Department of Defense in PowerBuilder? This happens all the time: mission critical apps get written in less than mission critical languages, and then get maintained, more or less, forever.

So fast forward about eight years from the creation of this app. The mail landscape has changed, and the architecture of the mailing process, which used to leverage client apps for sending mails, needs to change too. Email viruses and spammers have made old approaches to writing mail functionality painful; organizations are abandoning POP/SMTP based mail and retreating to MAPI. Meanwhile your VB app, which relied on a once attractive piece of third party code to provide MAPI support, is stagnating, and has started to show issues with age (like memory leaks, and interop issues with the security measures added across the Microsoft mail stack). In other words, it’s time to freshen the app. So the natural thing to do is to look at Visual Basic.NET, the successor to VB6, right?

Um, wrong. VB.NET famously no longer supports the programming interfaces used by VB6. And the VB6 to VB.NET migration tools bite; in fact, the tool blows up prior to successfully migrating the VB6 project. And no one in your development team has expertise in the .NET framework. In the years since VB, the team has moved on to C++ and Java. So which language will be the natural choice for migrating the mail app? Not VB.NET. What might have been a no-brainer to move up the ladder to the next version of VB has turned into a major nightmare for our small software company, and the only clear loser is Microsoft.

SQL refactoring: replace insert cursor with table variable

Tim Jarrett — Tue, 14 Jun 2005 18:41:00 GMT

I lost my copy of the classic programmer’s cookbook Refactoring several years ago, alas, but its philosophy of careful replacement of smelly code with clean code to improve the performance and maintainability of a software program is one that has stayed with me long after I ceased being an active programmer. One regret I had about the book was that it primarily addressed refactoring for object oriented languages (all the examples were in Java). But one of the biggest opportunities for cleaning up code is in non-object oriented languages such as Transact SQL. Recently I had an opportunity to clean up some legacy stored procedure code that I was adapting for an integration project I was working on, and one particular refactoring struck me as especially useful.

A common mistake when writing stored procedures is to overuse cursors. The cursor provides a way to operate on one row of data at a time, which is logical to a programmer used to thinking about working with arrays by looping through them. But SQL is fundamentally a language that is about set operations, and you can realize tremendous gains if you can stop operating on one row at a time and instead operate on a bunch of them at once. Microsoft provided a way to do that in SQL Server 2000 by introducing the table datatype.

Table variables help us by giving us an entire rowset in memory that acts precisely like a table in the database. An arti cle by Alex Grinberg on SQL Server Central illustrates some of the applications of the table variable. I used one of the ideas to replace a cursor that was being used to import data into a table in our system using a counter table.

In fact, the counter table probably caused the use of the cursor in the first place, because a developer had helpfully written a stored procedure to get the next ID value from the table (a common construct for database structures that need to be cross-platform and therefore can’t use features like IDENTITY ). Such a function is a good idea in a client application because it enforces a consistent method for creating IDs for new records, but because it enforces getting only one ID at a time it leads inexorably to row-at-a-time inserts and other abuses of SQL.

So the first step in converting this insert cursor into a sensible insert was to create a new procedure that allowed me to get a bunch of IDs out at once. Where the original procedure had as inputs the table for which the counter was being incremented and the new ID as output, this one also took the number of keys, the starting key value, and the ending key value. Thus even for a very large number of rows, I was only performing a single transaction to get a block of keys to work with.

Now the cursor replacement. The original logic of the stored procedure declared a select cursor against the staging table containing the data to be imported, then opened the cursor, got a new id, and inserted the contents of the cursor plus the ID into the destination table. So for each row of data to be imported we performed two transactions, an update on the counter table and an insert to the destination table. For 500 rows, this was taking about 25 seconds—not huge, but definitely a place where there could be a big improvement. Here are the steps I used to replace the insert cursor:

Declare a table variable, @tbl, with the same columns as the cursor, plus an ID column defined with the IDENTITY property.
Populate @tbl with an INSERT INTO...SELECT FROM statement that draws data from the staging table. Now each row in @tbl has the values from the staging table plus a temporary ID, ranging from 1 to the number of rows. (We didn’t specify a seed for the ID field; more on that in a second.
Count the number of rows in @tbl (select count(*) from @tbl).
Call the updated counter procedure with the number of rows in @tbl, getting back the first ID in the range (call it @startid.
Finally, insert into the destination table, selecting from @tbl and adding @startid to the identity column in @tbl. Since the IDENTITY column started at 1, we want to subtract 1 from @startid for each row as well, or else we’ll exceed the range of keys reserved.

Here’s some sample code:


DECLARE @tbl TABLE (col1 int, col2 int, col3 int, temp_id IDENTITY (1,1))
INSERT INTO @tbl (col1, col2, col3)
SELECT stagecol1, stagecol2, stagecol3 FROM staging_table WHERE ...
DECLARE @startid int
DECLARE @total int
SELECT @total = COUNT (*) FROM @tbl
exec p_increment_counter_multi 'dest_table', @total, @startid OUTPUT
INSERT INTO dest_table ( dest_id, destcol1, destcol2, destcol3 )
SELECT temp_id + @startid - 1, col1, col2, col3 FROM @tbl

So instead of two transactions for each row in the staging table, we end up with four transactions total, one of which is a select to populate the table variable and one which just gets the total number of rows. The performance benefits can be substantial: In my particular procedure, for 500 source rows, the time to execute the procedure went from 25 seconds to 6 seconds.

Of course, you could have done the same thing using a temporary table, though it appears that table variables are slightly more efficient in memory. But the basic principle is the same: an application-level counter column is no reason to insert one row at a time.

The same principle, with a twist, can be applied to update cursors as well; more on that in a while.

MSN Toolbar Tabs - first reactions

Tim Jarrett — Thu, 09 Jun 2005 17:31:20 GMT

There are other tabbed browsing add-ons for IE, but when I saw that the MSN toolbar had added a tabbed browsing enhancement, I decided to check it out. After all, I still know people at MSN I can yell at if there’s something wrong. And, actually, yeah, there’s a few things I would change.

First things: I can’t stress how glad I am to have tabs rather than the damned taskbar group (multiple browsers collapsed into one toolbar button with a number on it). There’s no good way to do blogging and newsreading with toolbar groups. Tabs are a hell of a lot more usable. I also appreciate that the toolbar supports the standard CTRL-T keyboard shortcut for creating a new tab.

But there are quite a few missing features from the MSN implementation. For one thing, there’s no option to make new links open automatically as a tab in an existing browser window. So if you click a link from email or another application, it still spawns a new browser window. And links defined to open in new windows still do; there’s no way to override that behavior to make the new window open in a tab instead, as there is with Firefox or with Safari. Also, there is no “open in new tab” on the right-click context menu, which renders the tab feature a lot less useful.

Verdict: on a scale of 1 to 5, a 2. The new tab support is better than having no tabs at all, but to call it half baked is too generous. It feels like the team focused on tabs as a feature, rather than looking at the customer problem, which is window clutter and impaired productivity, and thinking about what is required to address that in a tabbed browser implementation. Microsoft is traditionally good at thinking through user scenarios; I look forward to the next version.

Schooled by Scoble, and my response

Tim Jarrett — Wed, 25 May 2005 16:12:34 GMT

Scoble commented on my piece yesterday on MSN Virtual Earth and gently points out, through a link to the Channel 9 interview with the team behind Virtual Earth, that there’s considerably more to the new offering than following what Google did with Google Maps. I agree; certainly the eagle-eye view is impressive (if not destined for the first release; it would be rude to call it vaporware, though), as are the hybrid view and the UI work. I probably misspoke in calling this a “me-too” release; several of the features are brand-new to the market. I’m not sure that changes the main point I made, though.

Launching a product isn’t just features, it’s time to market. Shimon commented that there’s no question that Microsoft will keep innovating in this space and lap the competition. My question, as in my first post, is what took so long? Certainly the first feature, combining satellite and map in the same interface, is something that Microsoft could have done years ago. But from all appearances it took the arrival of competition for the company to deliver that value to customers.

My point is that the competition is good—for customers, for the company, and for its shareholders. And that brings me dangerously close to a hobbyhorse that I’ve been on and off for a long time. Microsoft can’t be the only company in a space and still deliver maximum value, because it generally does its best work in response to competition. That’s not a reflection on the company’s technical skills but on its great organizational strength: the way it responds to a challenge.

Mapping: When being a smart follower isn't enough

Tim Jarrett — Tue, 24 May 2005 16:14:59 GMT

Microsoft announced that they will debut a new mapping service, MSN Virtual Earth, this summer (thanks to Slashdot for the link). The service combines satellite images with map data, provides Sims-like isometric views, and allows layering information about businesses and services atop the search results.

This isn’t a surprising move. After all, MSN Maps have been around for a while, and Microsoft has had Terraserver since 1998. What’s different is that Microsoft’s announcement has a feel of desperation and me-too-ness about it, coming several months after Google debuted satellite images in their slick Google Maps service.

Integration of maps and satellite images is a natural incremental feature that provides radical amounts of value to users. It’s just the sort of software that you used to expect Microsoft to release. Embarrassing, then, that they got beaten to their own punch by a company that had no prior competence in mapping or imaging.

The good news in this scenario is that customers are getting a choice, as Microsoft feels the sting of competition. The bad news—for customers and for its investors— is that the most highly capitalized software company in the world isn’t capable of turning all its resources into bringing products like this to the market faster.

Correction: InfoCard federates

Tim Jarrett — Tue, 24 May 2005 13:53:55 GMT

Johannes Ernst, whom I linked from my piece on InfoCard last week, wrote in to point out that I erred in my quick description of the service. He says that in InfoCard:

…the PC does not actually store the identity information, only pointers to it. The actual identity information is stored by identity providers, who are the "3rd party" in the system (the other ones being the relying party, such as a website, and the PC component).

This makes InfoCard much less like Apple’s Keychain (or for that matter the existing Windows saved password feature) and more like, well, a federated identity system. Interestingly, this is consistent with what I remember from the discussion of the future of Passport back in 2001 with MSN execs.

Reinstalling again

Tim Jarrett — Wed, 11 May 2005 16:53:52 GMT

Another morning lost to rebuilding the machine. I asked the IT guys at my office to allow me to upgrade to Windows XP from 2000, and they did—alas, something went awry and the install created an entirely new system directory. So I’m reinstalling services and trying to find installation disks, and basically just trying to get things back to normal. It is nice to be back on XP, though.

Transforming XML with XPath in SQL Server

Tim Jarrett — Tue, 10 May 2005 16:54:35 GMT

Since the next version of SQL Server (2005, aka Yukon) is at least nominally coming out this year, it seems a bit late to write a tip about using the XML features in SQL Server 2000. However, better late than never.

As part of a project I’m working on, I needed to bring XML data into our production database for subsequent processing. The XML document that our partner’s application provides is what I would consider “loosely typed”—all the useful information about the data contained within is defined outside the actual schema. For an example, say the canonical sample XML document is written like this:

<ROOT> <Customer CustomerID="VINET" ContactName="Paul Henriot"> <Order CustomerID="VINET" EmployeeID="5" OrderDate="1996-07-04T00:00:00"> <OrderDetail OrderID="10248" ProductID="11" Quantity="12"/> <OrderDetail OrderID="10248" ProductID="42" Quantity="10"/> </Order> </Customer> <Customer CustomerID="LILAS" ContactName="Carlos Gonzlez"> <Order CustomerID="LILAS" EmployeeID="3" OrderDate="1996-08-16T00:00:00"> <OrderDetail OrderID="10283" ProductID="72" Quantity="3"/> </Order> </Customer> </ROOT>

In the flavor of data description used by our partner, the same document would be written like this:

<ROOT> <Customer CustomerID="VINET" ContactName="Paul Henriot"> <Order> <OrderAttribute rawname="EmployeeID">"5"</OrderAttribute> <OrderAttribute rawname="OrderDate=">"1996-07-04T00:00:00"</OrderAttribute> <OrderDetail> <OrderDetailAttribute rawname="OrderID">"10248"</OrderDetailAttribute> <OrderDetailAttribute rawname="ProductID">"11"</OrderDetailAttribute> <OrderDetailAttribute rawname="Quantity">"12"</OrderDetailAttribute> </OrderDetail> ... <ROOT>

That’s only slightly unfortunate; after all, XML is meant to be transformed, and if you wanted to turn the sample into sensible row-and-column data for a relational database you could use the features provided in OPENXML to do so.

Ah, but it turns out the XPATH syntax needed to transform the data (using the ColPattern mapping in the OPENXML) is slightly nontrivial. To extract the text of the elements, you need the text() function, which must be called after the predicate specifier. That is, if your OPENXML statement looks at the /Root/Customer/Order/OrderDetail branch, rather than writing OrderDetailAttribute/text()[@rawname="Quantity"] to get the value of the quantity of the order, you need to write child::OrderDetailAttribute[@rawname="Quantity"]/text(). (Note: If you wanted to include details from both the order and orderdetail branch, you actually have to specify child:: rather than SQL Server’s shorthand, otherwise you get a crossproduct.) (see below)

Anyway, once you have the correct XPath sequence, the rest is relatively trivial. The stored procedure below accepts the XML document as a parameter (you need to do this, generally speaking, unless you know that the XML document will be smaller than 4000 characters in size, since SQL Server 2000 doesn’t let you use local variables of type TEXT) and transforms it into a rowset, ready to be inserted into the database or otherwise manipulated:


CREATE PROCEDURE dbo.import_xml
	@xmldoc text
as
	DECLARE @hdoc int
EXEC sp_xml_preparedocument @hdoc OUTPUT, @xmldoc
--let's peek at the rowset
SELECT * FROM 
OPENXML (@hdoc,'/Root/Customer/Order',1)
 WITH (CustomerID varchar(300) 
'../@CustomerID',
       EmployeeID varchar(30) 
'OrderAttribute[@rawname="EmployeeID"]',
       OrderID varchar(300) 
'OrderDetail/OrderDetailAttribute[@rawname="OrderID"]',
       ProductID varchar(30) 
'OrderDetail/OrderDetailAttribute[@rawname="ProductID"]',
       Quantity varchar(30) 
'OrderDetail/OrderDetailAttribute[@rawname="Quantity"]')
exec sp_xml_removedocument @hdoc
GO

which produces, for the sample file excerpted above,

CustomerID	EmployeeID	OrderID	ProductID	Quantity
VINET	5	10248	11	12

This is probably old hat to those of my readers who know what the hell I’m talking about, but it was eye opening and nonobvious to me.

Update: Corrected after some input from a reader who knew better than I what the hell I was talking about (thanks, Michael Rys). As it turns out, you don’t need the text() function; the XPath shown above will correctly pull the text in the node once you’ve written the predicate correctly. Nothing to see here; move along.

Solving ASP.NET application problems

Tim Jarrett — Thu, 05 May 2005 18:15:10 GMT

I have been working with a partner of ours to get an ASP.NET web application running on my Windows 2000 computer at the office. We had no joy for several hours last night trying to figure out why none of the .aspx pages in the application could be contacted. We were getting an interesting error message:

Server Error in '/BogusAppname' Application. ---------------------------------------------------- The resource cannot be found. Description: HTTP 404. The resource you are looking for (or one of its dependencies) could have been removed, had its name changed, or is temporarily unavailable. Please review the following URL and make sure that it is spelled correctly. Requested Url: /BogusAppname/login.aspx
---------------------------------------------------- Version Information: Microsoft .NET Framework Version:1.1.4322.2032; ASP.NET Version:1.1.4322.2032

I had to give up with the guy from our partner company at 8 PM my time last night. This morning, I did some Googling and found an all-too-simple-sounding solution: the application wasn’t correctly registered as a virtual directory. I went to the IIS manager, created a new virtual directory with the same name as the directory I was trying to hit, and lo and behold the application started working.

The difference was only apparent in the properties dialog on the application directory. The virtual directories on the server had “Virtual Directory” as the first tab on the properties dialog, but our application directory had “Directory”.

Perhaps this will save someone’s thinning hair.

Expansion begins at home, in Redmond

Tim Jarrett — Wed, 04 May 2005 18:03:39 GMT

CNet: After looking around, Microsoft decides to expand at home. That the travel across the 520 bridge to get to hypothetical Seattle offices should be viewed as an insurmountable obstacle to development should surprise no one. Microsoft doesn’t know how to build software (broad generalization) unless all the people are within two minutes’ walk of each other. For all its high-techness, the most surprising thing about the Microsoft culture is how meeting-centric it is.

The thing that caught my eye was this sentence in the third paragraph: “The company also plans to demolish and rebuild 600,000 square feet of older buildings that lack the power and cooling capacity needed for modern computer equipment.” I wonder if that means a final sayonara for buildings 1 through 6? These were the original Microsoft buildings on the Redmond campus, and while they’re earthquake damaged, mold infested, and confusing as hell (check out the satellite photo here and then imagine navigating around the corridors inside those Xs), they’s historic. They also fit sympathetically into the wooded ravine landscape in a way that none of the later buildings manage. Maybe one of my Microsoftie friends can comment on this?

Email productivity tips

Tim Jarrett — Wed, 27 Apr 2005 14:16:32 GMT

Now that I’m back in the real world (that is, not blogging all day long), I am definitely feeling the need to revisit some of the recommendations for time management at 43 Folders. Fortunately Merlin posted a roundup of email and task management recommendations today, including the following (drawn from the three individual posts):

Shut off auto-check, or set it to something reasonable like every 20 minutes.
Pick off the easy mails—if you can reply to something with a 1-2 line response, do it.
Write less.
Be honest—delete or archive the mails you’ll never do anything about.
Process each piece of incoming email as: delete, archive, defer for later response, generate an action, or respond immediately. Then go back to the response and action items and do them in batches.
Outlook and Entourage allow you to categorize task items. Use categories to provide the context around task items. Merlin suggests using functional categories (“chores,” “errand,” “write,” “calls”), computer-related categories, and categories like “agenda” to prevent items from falling off the plate.

Microsoft Blog Portal 2.0

Tim Jarrett — Fri, 25 Mar 2005 19:16:54 GMT

Jana's Joint: Blog OPML. The updated version of the Microsoft.com Blog Portal (which I worked on right before I left the company—I was there to ship the 1.0 version) brings OPML for collections of Microsoft blogs out of the realm of “easter egg” and into the user interface in an incredibly intuitive way:

The fun part is you can go create your own OPML feeds by using the search function on the page. Each product search for blogs will generate a feed.

So, cool stuff. Next: incorporate the blog search results into regular Microsoft.com search. Right, guys?