Rick Schaut on users providing reliable reproducible scenarios (or not)

Posted by Pierre Igot in: Macintosh
May 31st, 2004 • 12:33 am

Microsoft developer Rick Schaut has yet another lengthy piece on the Word debugging process. In this piece, he obviously (albeit without naming names) refers to the response that his previous piece on the topic generated, including the response in the blog you are currently reading.

In my response, I expressed my utter surprise when reading his statement that, as recently as last year, the MacBU team “still [didn]’t know that the [Disk Is Full] problem involve[d] the OS’ open file limit“. In response to this, Rick Schaut tries to argue that there are different meanings of the verb to know and that, just because he said that they didn’t know, didn’t mean that they didn’t know.

The verb certainly has a variety of meanings, but in this context, its use has to be analysed in relation to the use of the word involve in the same sentence. What Rick said is that they didn’t know that the bug involved the OS’s open file limit. He didn’t say “we didn’t know that the bug was directly caused by the open file limit”, which sounds more like what he meant to say (I think).

Anyway, his big issue now is that they didn’t “know”, because they didn’t have a way to consistently reproduce the bug. He then goes on to blame the users (what else is new) for complaining too much and not putting enough effort in providing them with such reproducible scenarios.

Maybe the problem has less to do with user attitude and more with some very simple facts. As Rick says, Office 2004 is now out, and he says that the MacBU has a “SWAT team” of people scouring the Internet for bug reports. But why on earth isn’t Microsoft providing a facility on the MacBU web site itself for people to send bug reports? As I’ve said in the past, they only have to look as far as Apple itself to see an example of a fairly well designed tool for reporting bugs that actually works and helps users provide more useful information and sample files if needed.

Then there is the little mentioned fact (on Rick Schaut’s web site anyway) that Word users have actually paid for the product, and deserve some kind of service in return. Instead of providing that service, Rick complains that the users are not putting enough effort in providing reproducible scenarios. Maybe it has to do with the fact that some bugs are not always easy to reproduce reliably, and that Word users might not have hours to spend on doing bug testing for Microsoft free of charge!

When it comes to the Disk Is Full bug, I simply find it impossible to believe that, once the bug had surfaced (and that was a long time ago), Microsoft was unable to create the required conditions in-house to reproduce the bug often enough for them to identify the source properly and fix the bug. It’s not the users’ job to do that kind of extensive testing for Microsoft and provide perfect, complete, reliable bug reports on a silver platter to Microsoft free of charge. Lest we forget, most of the time, when they are using Word, users are trying to get some work done — and not the kind of work that Microsoft seems unable to do itself.

I might be guilty of making a “judgement” on Microsoft’s ability to fix its own software here, but that judgement is based on mounds of evidence amassed over years of daily frustration while using the product and chronic inaction on Microsoft part.

If Rick Schaut is unable to see that and to initiate the measures that would actually help remedy the situation (such as providing a decent bug reporting facility, doing much more testing in-house, and stop relying on exploited volunteer MVPs to communicate with their users), then I am afraid there is not much to add, and Mac Word users can expect things to remain the same in the foreseeable future, Microsoft-blessed blogs notwithstanding.

6 Responses to “Rick Schaut on users providing reliable reproducible scenarios (or not)”

  1. Pierre Igot says:

    The idea that some of the code is over 20 years old is indeed scary… especially since it’s Microsoft code. If the code were excellent to begin with, it could be argued that it is still relevant and appropriate today. (After all, Mac OS X itself is based on a Unix architecture that’s not exactly brand new…) But it’s Microsoft code, and… let’s just say that Microsoft has done little to inspire confidence that it’s quality code that deserves to still be in use 20 years later.

    Will we still be using this code in ten years time? I don’t see any indication that we will not.

    Anyhow, it’s all a matter of “opportunity cost“, isn’t it?

  2. Paul Robertson says:

    I wholeheartedly agree with your comments Pierre.

    From Rick’s blog I have come to accept that Word will NEVER reach the level of reliability that I deem acceptable.

    It appears that in order to achieve significantly better levels of reliability, large chunks of code would need to be scrapped and rewritten, something that I believe MS will not entertain, even for the Windows version let alone that for the Mac!

    I really appreciate the efforts Rick has made to reveal significant details of the current stage of development of Word, however all this information has done is to convince me that Word is handicapped in so many ways by its ancient code base.

    I’m sure that Rick and Word’s development team are making herculean efforts to improve the situation, but they appear to be loosing that battle, a battle that surely will only get worse.

    My biggest worry is what will be the situation in ten years time, will we still be using a version of Word that contains code from the 1980s?

  3. Pierre Igot says:

    Yes, I did mention this article a couple of weeks ago.

  4. Paul Robertson says:

    Pierre, if you hadn’t already spotted it, Adam C. Engst at TidBITS has written an interesting article outlining a fantasy feature set for a writer’s word processor.


  5. Paul Robertson says:

    Should have picked up on that, it just goes to show how lost I am without NetNewsWire. (My ibook has gone back to Apple for its 4th Logic board, only if it fails again will they talk to me about a replacement machine)

  6. Pierre Igot says:

    Yeah, can’t do without NetNewsWire either :). Sorry to hear about your iBook… That’s awful (and absurd). Reminds me of my PowerBook adventures in 2001… (First they sent me a PB with a US English keyboard when I had ordered one with a French Canadian keyboard; then they claimed that the poor AirPort reception was not normal and did another machine swap; of course, the AirPort reception was just as bad; and believe it or not, all three machines had 0 dead pixels.)

Leave a Reply

Comments are closed.