OS X 10.10.3: Fixes major responsiveness issues on 2014 Mac Pro

Posted by Pierre Igot in: Macintosh
April 15th, 2015 • 6:03 pm

This has been a very traumatic few months for me as a Mac user. As indicated in several previous blog posts, I purchased a new Mac Pro along with a new 4K display back in July 2014. The machine shipped with Mavericks and worked pretty well until I upgraded to Yosemite.

I waited until OS X 10.10.1 came out before upgrading, but this was still not cautious enough. I soon realized that, on my Mac Pro, Yosemite suffered from significant responsiveness issues that were not tied to any specific piece of software or to a particularly high level of CPU activity.

When I say “significant”, I really mean it. These were not just slight delays that one can accept to live with in a 1.0 or 1.0.1 version of a new software product until the developers get around to further optimizing the software. These were massive, outrageous delays in seemingly simple tasks, such as switching apps, clicking on menu headings to pull them down, resizing windows, etc.

As an experienced troubleshooter myself, I went through all kinds of steps to try and isolate the problem and hopefully find a solution. Nothing worked. There was not a single application that I could identify as the culprit. It soon became quite clear to me, with my extensive experience as a troubleshooter, that this was a problem with the OS itself.

In desperation, I even went so far as to do a brand new “clean install” of my system and applications — something that I hadn’t done in years. When you use a fairly advanced setup such as mine, with a number of third-party applications and lots of customizations (none of which would qualify as “hacks”, although Apple probably does not do much in-house testing to ensure that utilities such as Default Folder X, Typinator and Keyboard Maestro always work well with their latest system software), doing a clean install is a very tedious, time-consuming process, because you not only have to migrate all your files back and reinstall all your applications, but also restore all the customizations one by one, without relying on existing application support files that might have somehow become “corrupted”.

But I did it, and even before I had completed the process of restoring all my customizations, I knew that the clean install had not solved the issue.

The problem with responsiveness issues is obviously that they are somewhat subjective. I am a fast typist and, generally speaking, a fast GUI user. I even use Keyboard Maestro to automate a number of repetitive interactions with the GUI using macros; those macros effectively mimic the behaviour of an extremely fast GUI user. So I am particularly sensitive to responsiveness issues. And let me tell you, on my machine at least, the situation was a nightmare.

I had all kinds of Keyboard Maestro macros that no longer worked properly because the system was not responsive enough, so I had to artificially add all kinds of built-in delays to the macros to try and make them work with the OS. (And even that only went so far, because the system’s own delays were not consistent and predictable.)

But even in my own “manual” interactions with the Yosemite GUI, I was hampered by all kinds of undesirable delays that would cause me to miss targets, accidentally trigger things I didn’t mean to trigger, etc. It was maddening!

Given that my Mac Pro was a recent purchase and that this was clearly an unacceptable level of responsiveness on a supposedly fast machine, I felt that I had no choice but to go through Apple Care. I won’t get into all the details, because telling them in full would be far too long, but suffice to say that I “burned through” four different “senior representatives”, who all agreed that what I was experiencing was unacceptable, but were at a loss to explain it, let alone reproduce it themselves or get their engineering counterparts to acknowledge the seriousness of the situation and do something about it.

Things were so bad that there was talk, at some point, of downgrading to OS X 10.9, or even of trying to get my Mac Pro’s video cards replaced, in case that might somehow miraculously fix the problem — as if it wasn’t obvious enough that this was a software problem and not a hardware problem.

I spent a lot of time executing troubleshooting tasks for the senior representatives, or for the engineering people that they were talking to about my issues, all the while knowing very well that none of them would do anything to fix the problem.

What was particularly maddening, of course, was that responsiveness issues are hard to capture, and also tend to be intermittent in nature, which means that you often cannot reproduce them on demand. However, I did end up purchasing a piece of software (ScreenFlow) that enabled me to create video captures of some of the most egregious problems, and I was also able to narrow down, to some extent, the circumstances under which the responsiveness problems would occur.

Unfortunately, I also wasted a lot of time on what can only be described, in retrospect, as wild goose chases, because I was seeing various behaviours that seemed to be connected but ultimately proved to be unrelated (or loosely related at best) to my responsiveness issues. For instance, I couldn’t help but notice that Yosemite was eating up the 32 GB of physical RAM on my Mac Pro like candy, and that responsiveness issues seemed to be getting worse the closer I got to 32 GB of memory used. The last senior representative that I talked to seemed to be unable to find out whether this kind of RAM usage was normal in Yosemite or not, and couldn’t explain, for instance, how a single web page left open in Safari could end up using 1.5+ GB of memory all by itself, on top of the memory used by the Safari process itself.

While Apple Care’s representatives proved to be unable to help me in any significant way, I was fortunate enough to get a variety of suggestions and tips from readers of this blog and my Twitter feed. This is how, for example, I found out that having turned the “Reduce transparency” option on in the Accessibility settings in System Preferences was actually causing a significant menu-flashing bug on top of my general responsiveness issues, and I ended up using the “Increase contrast” option in the same Accessibility settings for a while, because it helped reduce the severity of this bug and also, somehow, seemed to slightly alleviate the general responsiveness issues in Yosemite on my machine.

I, of course, had mentioned to the Apple Care representatives that I was also a member of AppleSeed and, as such, had access to early builds of the next system upgrade, OS X 10.10.3. (OS X 10.10.2 had come out early on with no improvements.) I asked whether it was a good idea, in my circumstances, to give those early builds of 10.10.3 a try. The representative that I was in touch with at the time was obviously at a loss himself and figured that I didn’t have much to lose. So I did proceed with the installation.

The very first 10.10.3 build available through AppleSeed included the new Photos application, but did nothing to alleviate my problems, so I was not particularly hopeful that things would get better in subsequent builds (worried as I was that the focus would be mostly on this new Photos application and not on fixing bugs).

But then two things happened. I got a tip from a reader about this post on Reddit, which suggested a link between responsiveness issues in Yosemite and a specific kernel extension (AppleGraphicsPowerManagement.kext). I was intrigued, because I had always suspected (with my limited knowledge of the engineering underpinnings of OS X) that, in the effort to minimize battery consumption for laptop computers, Apple had gone too far in implementing mechanisms that put all kinds of things to sleep automatically when idle. Certainly, several of my responsiveness issues looked like problems where, when I was clicking on something, the computer system appeared to be somehow “waking up”, and only responded to my clicks after a few seconds. And of course, my responsiveness issues had everything to do with the GUI, i.e. with “graphics”, and I had been suspecting problems with graphics drivers all along.

And then within a few hours of that discovery, AppleSeed released a new build of the OS X 10.10.3 update, which I installed, and which included a number of updated versions of kernel extensions, including this particular one (and several other graphics drivers)!

I soon realized that this update was a major step forward, that it clearly addressed several of the issues that I had been experiencing. For instance, the menu-flashing bug was completely gone. There would still be the occasional slight delay between a click on a menu heading or button and the appearance of the actual menu, but there was no longer a problem with the menu flashing and disappearing before I had a chance to make a selection.

Most important, however, was the fact that the general problem with the massive delays that I had been experiencing on my Mac Pro was gone.

I am not sure I can convey how relieved I felt at this point. I immediately communicated with the Apple Care senior representative that I was in touch with. He, of course, had no inkling that this particular build might feature any bug fix that would address the problem that we had been discussing for over two months. He was just as pleasantly surprised to hear about this.

While I was expressing my huge relief to him, I also made it quite clear that I didn’t feel that this outcome made everything alright and that everything that had come before then was forgotten (let alone forgiven). For one thing, I had no guarantee (nor did he) that whatever this particular build fixed would not be broken again in subsequent builds. Since no one on the engineering side had communicated with him about the fact that this build might contain a fix, we had no idea whether the fix was even intentional — and not simply an accidental by-product of some other improvement or optimization.

In fact, I was so worried about this that I refrained from installing any subsequent builds of the 10.10.3 update until the official update came out last week. I wanted to make sure that I had a system that worked, and given that I was otherwise very busy, the easiest solution was to do nothing. Then when the update finally came out, I made sure I had an up-to-date backup of my existing system that was updated right before I installed the update, and I made sure I immediately tested the final version of the update to confirm that the responsiveness problems had not come back. (They had not.)

I also made it quite clear to the senior representative that I felt that there had been massive communication issues between the engineering side and the customer service side at Apple, and that this had caused me and him to waste lots of time on wild goose chases and other useless (in retrospect) troubleshooting steps. While he himself was at least getting paid for dealing with me, I, on the other hand, had no way to get any compensation for the countless hours I had spent on troubleshooting this problem. If at least Apple’s engineers had said, early on, “Yes, we know that there is a major responsiveness issue for some users, and we’re working on a fix, which should be available around such and such a date”, then we would not have wasted so much time!

There is a part of me that still hasn’t forgiven Apple for this, and maybe never will. The whole experience has been very traumatizing, and I am really angry at Apple for having caused me to waste so much of my time on trying to fix a problem that I could do nothing about. As I said to the senior representative, “the impression of callousness on the part of Apple’s engineers will take a very long time to dissipate”. I realize that this particular problem probably only affected a small minority of users, but when someone goes through four different senior representatives, provides reliable scenarios for reproducing specific issues, provides graphic evidence in the form of screen captures (the engineers even asked me to record a video of my monitor with my iPhone at some point!), and clearly demonstrates that he is no fool and knows what he is talking about, the issue has to be treated more seriously and professionally, with a more acceptable level of… responsiveness in the communication itself.

I cannot help but feel, at times, that Apple’s engineers think that they are above all this and incapable of making such egregious mistakes that cause significant hardship for some of their users. While I agree that customer complaints might not always be entirely legitimate and that some users are fussy and moan about things that, in the big picture, are really rather small potatoes, this was not one of those cases. The issues I had were very real, and the very fact that they were suddenly fixed by a single system update proves that they were.

Because of the lack of communication, I still don’t know if Apple’s engineers even realize that they have fixed those significant responsiveness issues for me (and hopefully others). Part of me is still afraid that I might encounter a regression in a future system update. But at least now I had an official version of Yosemite that I can fall back on if things go awry again.

In closing, I should also stress that this is not the end of my complaints about responsiveness in Yosemite. There are still multiple aspects of the GUI in Yosemite that don’t feel right to me. The responsiveness issues are no longer egregious to the point that I find them unbearable, but there are still problems, including several bugs that I identified in the process of trying to further circumscribe the issues that were specific to my Mac Pro and mistakenly took as manifestations of these issues, when in fact they were just made more visible and noticeable by the massive delays that my responsiveness issues were causing.

I still often notice, for example, a slight delay between the time I click on a menu heading in the toolbar and the time the blue highlighting behind the heading comes on and the menu below gets drawn. It’s a small fraction of a second, but it’s noticeable. It doesn’t affect my ability to use the menu, but it makes the whole process feel slower.

The particular problem that I identified in this blog post about the blue highlighting for the “Help” menu is still there. It’s cosmetic, but it looks sloppy.

The bug in OS X’s Preview that causes it to fail to keep up with fast window resizing (especially when using a Keyboard Maestro macro) is still there.

And there are, more generally, multiple situations where you can almost witness the process of Yosemite “building” the user interface in real time. For example, yesterday I posted on Twitter a link to this video recording that I created and that shows what happens when I play back, in slow motion, a video recording of System Preferences drawing the “General” preference pane in real time in Yosemite. This video clip, which is effectively the recording of the playback of a recording, shows that there is a delay of 0.07 second between the time the contents of the “General” preference pane appear and the time some of the blue highlighting that should be part of the UI appears (the blue highlights on the radio buttons, the blue highlights on the arrows for the pop-up menus, etc.).

Why is there such a delay? It is not caused by some background activity that would happen to be slowing my system down at the time. The 0.07-second delay happens each and every time I access the “General” preference pane. It’s a small delay, but it’s noticeable. And it certainly contributes to the feeling that Yosemite is slow, even if it’s not really that slow.

I don’t know what Apple did to the GUI in Yosemite. They clearly rebuilt a lot of things (the font change to Helvetica playing an important part in this process), and in the process they created a lack of smoothness, fluidity, and zippiness that is really quite disturbing. It makes you feel like you using a system that is somehow more “fragile”, less reliable, less solid and could fall apart at any point — something that Apple’s engineers have decided is “good enough” for most people, but that more demanding, more attentive users will find rather disappointing and unworthy of Apple’s legendary attention to detail (which is becoming more and more of a legend and less and less of a reality with every passing year, I am afraid).

I cannot finish this post without thanking all the Betalogue readers and Twitter followers who have written to me to share their own stories and offer suggestions, tips, or simply commiseration. Unlike Apple’s customer service, they were actually able, collectively, to make a difference for me, and I can only hope that, with this blog post, I in turn might able to make a difference for other users.


OS X Activity Monitor: Yet more evidence of Apple’s sloppy work in Yosemite

Posted by Pierre Igot in: Macintosh, Mail
March 13th, 2015 • 5:00 pm

Here’s another example of Apple’s sloppy work in OS X 10.10 (Yosemite) that is very easy to reproduce (at least on my Mac Pro):

  1. Open the Activity Monitor application (in the Application/Utilities folder).
  2. Option-click on your desktop to hide the foreground application, i.e. Activity Monitor, and switch back to the Finder.
  3. Click on the Activity Monitor icon in the Dock or press command-Tab to switch back to Activity Monitor.

Notice anything? The toolbar now looks like this:

yosemite-activity-monitor1

Normally, when Activity Monitor is in the foreground, the toolbar should look like this:

yosemite-activity-monitor2

In other words, somehow Yosemite “forgets” to switch the background chrome of the window toolbar back to its foreground version. Yet the window is indeed in the foreground, as indicated by the three coloured buttons on the left, which are coloured (and not greyed-out), i.e. in the state they are supposed to be in when the window is in the foreground.

And even the other toolbar buttons that look like they are in the background and disabled are actually active and you can click on them. (This actually forces them to switch back to their normal foreground state visually, one by one.)

So the issue is purely cosmetic. But it’s still very sloppy on Apple’s part.

I haven’t been able to reproduce this problem with any other application. I also cannot reproduce the problem when step #2 in the process above involves choosing “Hide Activity Monitor” in the “Activity Monitor” application menu. But I can also reproduce it when I switch to another application and use “Hide Others” to hide all other applications, including Activity Monitor.

Fixing the problem involves switching to another application and then back to Activity Monitor. This forces a “refresh” of the window display and then Activity Monitor uses the appropriate chrome for the foreground window.

This is not the first bug I have encountered in OS X with the system’s commands for hiding applications. Long-time Betalogue readers might remember that I reported, back in 2012, on a selection-highlighting bug that was first introduced in Lion and has been with us ever since. Because of this bug, after I use “Hide Others” to hide all applications other than the foreground application, if the applications thus hidden include Mail, when I bring Mail back to the foreground, it continues to use the background (grey) colour to highlight messages in the message list, and it also fails to make the I-beam cursor visible in messages that are in the process of being edited.

The same bug also affects, for some reason, stand-alone Safari-based web browsers created with the Fluid application. I have several of those and they all suffer from the same problem as Mail. If they are hidden by “Hide Others” and then brought back to the foreground, the selection highlighting is all wrong.

Here again, fixing the problem involves switching to another application and then back to Mail or the affected stand-alone browser app. In fact, it’s such an annoyance that I have created Keyboard Maestro macros to execute this fix each time I switch to those applications, because I use “Hide Others” all the time and so my stand-alone browser apps often end up in this undesirable state.

This selection-highlighting bug has now been with us for three years. And now, thanks to Apple’s sloppy work in Yosemite, we can add a new one to the list, which only affects Activity Monitor, but is definitely easy to reproduce (at least on my system).

Sigh.


OS X 10.10 (Yosemite): Preview fails to keep up with fast window resizing (continued)

Posted by Pierre Igot in: Macintosh
March 9th, 2015 • 1:31 pm

Since my original post about a bug with window resizing in Preview in Yosemite, I have received feedback indicating that not everyone is able to reproduce this bug on their system.

This leads me to suspect that the problem might be limited to specific hardware configurations, like the other responsiveness issues that I have be experiencing since upgrading to Yosemite, which seem to all involve issues of drawing/refreshing the contents of windows, menus, etc. on the screen.

In my original post, I indicated that, in order to reproduce the problem, you really had to use very quick mouse movements in Preview, which will cause it to fail to keep up with your window resizing and only partly resize its contents to fit the frame.

I also indicated that the issue was particularly noticeable when using Keyboard Maestro macros, because these macros effectively mimic very fast mouse movements — in fact faster than what the human hand can achieve.

This means that, while reproducing the problem with a human hand can be challenging, with a Keyboard Maestro macro I am able to reproduce the bug reliably 100% of the time.

So in the interest of further identifying the circumstances under which this Yosemite bug occurs, here is a scenario involving Keyboard Maestro that makes it possible to reproduce the bug without fail:

  1. If you don’t have it yet, download and install Keyboard Maestro on your system.
  2. Create the following macro (or download it here and import it) or a similar one:
  3. km-resize

  4. Open any PDF document in Preview and switch the view mode to “Single Page” (in the “View” menu).
  5. Make the Preview window containing the PDF fairly small.
  6. Execute the macro.

If you have the same bug I have on my system (a 2014 Mac Pro), then the window will resize properly, but Preview will entirely fail to scale the contents to fit the resized window, as it normally should.

The only way to force Preview to scale the contents of the window is to adjust the window size a second time. This will force it to “refresh” the contents and redraw the page at the size it should normally be in a window of that size.

If you can reproduce this bug on your system, I would be very interested to know about it. Please use the link in the sidebar to contact me and send me your hardware configuration.

Thanks!


OS X 10.10 (Yosemite): Preview fails to keep up with fast window resizing

Posted by Pierre Igot in: Macintosh
March 6th, 2015 • 5:56 pm

Here’s a typical example of the many things that are wrong in Yosemite in terms of performance and responsiveness. When you first upgrade to Yosemite, you might notice all kinds of different responsiveness issues and it can be all quite confusing, giving you a general impression of unresponsiveness even though you cannot quite clearly pinpoint what is wrong.

Over time, however, it gradually becomes easier to identify specific behaviours in specific applications that help explain this general impression. Here is one such specific behaviour.

It has to do with the Preview application and with what happens when you view a PDF in “Single Page” view mode and you resize the window to make it bigger.

In order to reproduce the problem you need to do the following:

  1. Open a PDF file (any PDF file will do) in Preview.
  2. In Preview, go to the “View” menu and select the “Single Page” option (as opposed to the default “Continuous Scroll”).
  3. Then grab the bottom-right corner of the window and resize it to make it quite small.
  4. Then grab the bottom-right corner of the window again and drag it to the right and down as fast as you can with your mouse.

The speed is important here. If you drag it at a “normal” speed, you won’t encounter any problems.

If you drag it fast enough, on the other hand, sooner or later this will happen:

(Click here to get the MP4 video on DropBox.)

Note the situation in the video around the 02:00 mark: Even though the window’s bottom-right corner is near the edge of the video frame, the bottom-right corner of the PDF page is nowhere near that edge. It’s because I’ve resized the window too fast for Preview, and it has failed to keep up with my mouse movement.

But I’ve kept my mouse button down. And around the 03:00 mark, I nudge the mouse pointer a little bit more, which forces Preview to finally refresh its display of the PDF page so that it properly fills the space available inside the window’s edge (allowing for a tiny bit of padding for esthetic reasons, of course).

How pathetic is this? My hardware is not to blame: I have a very powerful 2014 Mac Pro with tons of RAM, and there’s nothing running in the background that would excuse such a shortcoming. (In actual fact, the ScreenFlow application is running and recording all this in real time, of course, but this behaviour is perfectly reproducible even when ScreenFlow is not running.)

Now, you might be tempted to say: “Yeah, but who resizes at that speed with the mouse anyway? If you resize at a more ‘normal’ speed, there is no problem here.” And that might be what Apple’s engineers think. (In other words, things are “good enough” for most users here.)

Well, for one thing, this is a pretty sad attitude to have. There is no question that my Mac is powerful enough and that Preview should not be struggling with this. Any self-respecting engineer who strives for excellence in his or her work should be dissatisfied with this and insist that the problem be fixed.

And then there’s the fact that it effectively penalizes users for being too fast! In what world is that considered OK? We live in an age of superfast hardware that should have no trouble keeping up with human interactions, especially when everything is local and no networking is involved. Yes, I am a fast user. So what? Does it really make me a bad user?

Finally, there’s one more thing: This shortcoming has a significant impact on people who, like me, might use Keyboard Maestro to automate various tasks, including the moving and resizing of windows in OS X. I have two large monitors and I have several macros that enable me to automatically move and resize my windows in any OS X application, including Preview, so that I can easily organize my document windows side by side on either screen.

I often have four different documents open in four different windows side by side and my large screens enable me to me to view these documents at a fairly low zoom level while maintaining legibility. But the two screens do not have the same size/resolution, and so when I switch a document window from one screen to the other I need to resize it. If that resizing involves a PDF document open in a window in Preview, then guess what happens?

Well, for window manipulation, Keyboard Maestro’s macros effectively behave like a superfast user, mimicking mouse movements at the fastest possible speed. So of course they are going to be affected by this bug in Yosemite’s Preview application. Sure enough, every time I use a macro that includes a step like this one:

km-move-resize-window

and the window involved happens to be on my second screen, where the window size has to be smaller, the macro correctly moves it to the main screen and correctly resizes the window to fill the bigger screen, but Preview fails to keep up with it and so the PDF page in the window is not resized properly.

In fact, Keyboard Maestro is faster than I’ll ever be with my mouse, and it’s so fast that Preview fails to resize the PDF page itself altogether! The window is bigger, but the PDF page keeps the same size it had on the secondary screen.

The only solution that I have found to work around this problem is to add an extra step to the macro, just for Preview:

km-move-resize-window-nudge

This extra step effectively mimicks the little nudge illustrated around the 03:00 mark in the video above, which forces Preview to refresh its display of the PDF page so that it finally fills the space available inside the window.

But it’s not an ideal solution, because any extra step like this increases the risk that something might go wrong during the execution of the macro. If Yosemite experiences any kind of “hiccup” during the execution (which it tends to do), then the nudge might not work as expected and I will still have to resize the window manually in order to force Preview to refresh its display of the PDF page.

This is a new problem in Yosemite. Preview in Mavericks never had such a problem, and I had been using my Keyboard Maestro macros with it for a long time without a hitch.

When people talk about Apple’s sloppy work in Yosemite, I believe that this is the kind of thing that they are thinking of. It took me a while to narrow it down because of all the other responsiveness problems I was having with Yosemite, but this, as far as I can tell, is an easily reproducible problem for anyone running Yosemite, even people who don’t have the egregious responsiveness problems that I have been experiencing.

When will Apple get around to fixing it (if ever)? Unfortunately, because the application works “well enough” for the majority of users, I am not too optimistic, which is why I’ll have to keep this extra “nudge” step in my macros and in my own dragging movements for quite a while, I fear.

rdar://20069961


Word 2011 for OS X: Easily confused

Posted by Pierre Igot in: Microsoft
March 5th, 2015 • 3:44 pm

If you’ve ever felt like confusing the heck out of Word 2011 for OS X just for fun (and who wouldn’t, really?), try this:

  1. In the Finder, create a folder called “Confused Word” on your desktop.
  2. In Word 2011, create a new document and save it as “Test1.docx” in the “Confused Word” folder. Leave the document open in a document window in Word.
  3. In the Finder, select the “Test1.docx” file in the “Confused Word” folder, make its name editable, and change it to “Test2.docx”.
  4. Switch back to Word 2011 and confirm that the document window you left open now has the document title “Test2.docx” instead of “Test1.docx”.
  5. Switch back to the Finder, select the newly renamed “Test2.docx” file and duplicate it to create a second file in the same folder called “Test2 copy.docx”.
  6. Make the file name “Test2 copy.docx” editable and change it to “Test1.docx”.
  7. Double-click on “Test1.docx” to open it in Word 2011.

The first strange thing that happens is that the “Test1.docx” document opens as “Read-Only”. Why? I have absolutely no idea.

But now hit command-S in Word 2011 to save the “Test1.docx” document that is labelled “Read-Only”.

Since it’s (allegedly) read-only, you’d think that Word 2011 would complain about your attempt to save it and ask you to save it under a new name or something, right?

Think again. Not only does Word 2011 not complain, but it actually changes the document title in the title bar to “Test2.docx (Read-Only)”!

As they say on the Interwebs, WTF?

It gets even funkier if, before hitting that command-S shortcut, you actually make some changes to the contents of the “Test1.docx” document. In that case, when you hit command-S, Word changes the document title in the title bar to “Test2.docx”, without the “Ready-Only” part. And guess what happens to the “Test2.docx” document that you’d left open in the other document window in Word 2011?

It gets renamed to something like “Word Work File D_xxxxxx.tmp”, where “xxxxxx” is a sequence of numbers. Admittedly, this is a very temporary situation, because as soon as you switch to another app and then back to Word 2011, the document title of that other document window changes back to “Test2.docx” as well.

So now you have two document windows with the same document title/file name and two different states for the content (since you typed something else in that other document window before hitting command-S, remember?).

And now guess what happens when you close those two document windows and return to the Finder?

Well, there are no longer two files inside that “Confused Word” folder on your desktop. There’s only one file left, called “Test1.docx”, and its contents do not include the changes you made to the document before hitting command-S.

Nice to know you’ve got our backs, Microsoft.


Numbers 3.5 (iWork): Hijacks cursor keys when editing cells

Posted by Pierre Igot in: Macintosh, Pages
February 27th, 2015 • 10:41 am

On February 25, 2015, I used Bug Reporter to submit the following bug report (#19951698) to Apple:

Summary:
When I select a cell and start typing, the cursor blinks inside the cell. Yet if I use the Left or Right cursor key, Numbers jumps to the previous/next cell, instead of navigating inside the current table cell.

Steps to Reproduce:
1. Open any Numbers document containing a table.
2. Select a table cell by clicking ONCE on it.
3. Start typing.
4. Press the Left or Right cursor key.

Expected Results:
I expect the Left or Right cursor key to move INSIDE the current table cell by one character to the left or to the right.

Actual Results:
The Left or Right cursor key JUMPS to the previous/next table cell.

Version:
Numbers 3.5.2.
OS X 10.10.2.

Notes:
To add insult to injury, the behaviour is not consistent across iWork apps. In tables in Pages, I get the expected behaviour.

I think this report is quite self-explanatory. For completeness’s sake, I probably should have added that when, instead of simply selecting a cell in Numbers before you start typing, you double-click on the cell to make its contents editable right away, then, for whatever reason, the cursor keys work as expected.

I can see absolutely no visual difference between a selected cell that becomes editable when you start typing and a cell that becomes editable when you double-click on it. And yet, in the first instance, the cursor keys do not work as expected — they jump from cell to cell, even though the I-beam cursor is blinking in the cell — whereas, in the second instance, they work as expected — they jump from character to character inside the cell whose contents you are currently editing.

I also should have pointed out that the impact of this hijacking of the cursor keys is that it also affects one’s ability to use the keyboard to navigate word-by-word inside cells, because the shortcuts for jumping from word to word inside a cell are the same standard shortcuts as elsewhere in OS X, i.e. option-Left and option-Right. In Numbers 3.5, after you select a cell and start typing in it, if you use option-Left and option-Right, instead of jumping from word to word, the application actually gives to those shortcuts the meaning that they have when you are not inside a cell, i.e. they insert a new column to the left or to the right of the current column. (I have already written here about the idiocy of that other hijacking of commonly used keyboard shortcuts in iWork.)

And of course, the additional impact of this hijacking of the cursor keys is that it also affects one’s ability to select text inside cells, because the shortcuts for extending the selection to the left or to the right are based on the cursor keys and are shift-Left and shift-Right. In Numbers 3.5, after you select a cell and start typing in it, if you use shift-Left and shift-Right, instead of extending the selection to the left or to the right inside the cell, the application gleefully selects you out of the cell and starts extending the selection of cells to the left or to the right.

(I should also mention the shift-option-Left and shift-option-Right shortcuts, which are also affected. In that case, Numbers ignores the Shift modifier and assumes that you meant to hit option-Left and option-Right, and so it inserts a new column to the left or to the right of the current column.)

Finally, it should be stressed that, when you enter a cell by double-clicking on it, all these other cursor-key-based shortcuts also work as expected, as do the cursor keys themselves. So at least the inconsistency is consistent (although I doubt very much that this is deliberate).

To Apple’s credit, I got an answer to my bug report within 48 hours:

Engineering has determined that this issue behaves as intended based on the following information:

Yes, this is behaving as designed. In tables in Pages we are expecting a user to type more text in a cell and so we favor text navigation with the arrow keys. For Numbers, we are expecting cells to contain short bits of data, like a number in each cell, so for that pattern we use the arrow keys to navigate to the next cell. That way a user could type 1, right arrow, 2, right arrow, 3, and quickly enter values into 3 cells.

If you have questions regarding this resolution, please update your bug report with that information.

We are now closing this bug report.

Please be sure to regularly check new Apple releases for any updates that might affect this issue.

I am afraid this makes very little sense. If indeed the idea behind the hijacking of the cursor keys is to enable the user to “quickly enter values” in a series of cells, then why is the behaviour different when the user double-clicks on a cell to make it editable?

In addition, we already have long-standing shortcuts for quickly entering values in a series of cells: the Tab and shift-Tab keys.

Also, just because the bits of data might be short, it does not mean that the user is less likely to make errors in his or her text input that require small corrections, which are one of the primary uses of cursor keys!

As well, the very reason for having universal, system-wide standard shortcuts is so that people can develop “muscle memory” that enables them to use these shortcuts in all contexts without even thinking and expect them to work all the time. Any application that hijacks universal keyboard shortcuts to give them a different meaning goes against such expectations, and there is simply no way that you can reasonably expect someone who has developed such muscle memory to remember to do the non-instinctive thing when she or he is in a specific application and refrain from using the shortcuts.

Finally, if indeed the expectation is that in tables in Pages, the user will “type more text in a cell”, and therefore have even more reason to use keyboard shortcuts for text navigation and selection, why are the keyboard shortcuts for word-by-word navigation and for extending text selection also hijacked in that application? (Granted, the hijacking there is limited to what happens when you are not inside a table cell, but it’s still far too easy to think that you are currently editing a cell and accidentally use these shortcuts when you shouldn’t.)

All this is to say that I cannot help but feel that the response I got from Apple is a lie and that nobody has really thought this through. If they had really thought this through and paid real attention to the issue, the behaviour would be the same whether you enter a cell by selecting it and starting to type or by double-clicking on it.

And even if the whole thing is deliberate, I think it’s the wrong decision. The theoretical advantage to some users (who might never have heard of the use of the Tab key to “quickly enter values” in consecutive cells) is simply far too insubstantial compared to the on-going disruption caused by this hijacking of universal, standard system-wide shortcuts.


Responsiveness problems in Yosemite: Menu-delay bug caused by ‘Reduce transparency’ option?

Posted by Pierre Igot in: Macintosh
December 19th, 2014 • 7:09 pm

When I was young and careless, I used to love being an early adopter of new technology, including computer technology. This meant that I often installed new system releases as soon as they became available, come what may. I was even quite excited when I was invited to join the AppleSeed program back in the early 2000s, because it meant that I would be able to use new versions of the system even before their official release.

Well, I am now older and wiser, and, more important, I have a lot of work that I need to do with my computer, which means that I cannot really afford to be so careless anymore. These days, I am increasingly cautious when it comes to experimenting with new builds of upcoming system releases, especially major system upgrades.

So when OS X 10.10 (Yosemite) came out, I didn’t install it right away. I read about it online, which is how I discovered, for instance, that there was an option to turn off the additional transparency “features” added by Apple to OS X in this new version. I saw absolutely no benefit to this additional transparency, and to me it was just the latest incarnation of Apple’s never-ending fascination with resource-wasting eye candy, which came to the fore (in a rather painful way) in the early days of OS X and has been plaguing every version of the system ever since. So I was relieved to see that at least Apple offered an option to reduce the transparency.

When the first incremental update of Yosemite (OS X 10.10.1) came out, I decided to finally take the plunge and upgrade my system. I figured that surely the worst bugs had been eliminated by then, and that the risk was fairly minimal.

As soon as I installed Yosemite, one of the first things I did was of course to activate the “Reduce transparency” option in the “Accessibility” preference pane. After all, if the option is there, it is meant to be used, and fully supported by Apple, right?

Right.

When it comes to Apple, it seems that things are never that simple. There has undeniably been a trend, in recent years, of letting quality standards slip and shipping stuff before it is actually ready for prime time. I am afraid that Yosemite has turned out to be a prime example of this.

As I have already explained in a couple of blog posts, in my experience, on a 2014 Mac Pro with 32 GB of RAM and a 1 TB SSD drive, driving two monitors via DisplayPort (an older 30″ Apple Cinema Display with a DisplayPort adapter, and a brand new 31.5″ Sharp PN-K321 display with a native DisplayPort connection), Yosemite feels distinctly sluggish.

I find this very disappointing. I specifically bought my rather expensive computer setup (in addition to the Mac Pro and the display, I also had to buy external Thunderbolt storage, of course) in anticipation of Yosemite, with what I thought was a guarantee that I would have, for years to come, a very fast machine that would be more than capable of handling my software requirements. (I don’t really do any hard-core audio or video editing, and so in some ways the Mac Pro is overkill, but I also want a very quiet machine and I don’t think that there’s anything else on the market that beats the new Mac Pro in that respect. Right now, all I can hear is my external Thunderbolt storage — and it is moving to another room in the basement very soon.)

Whereas things were quite zippy with Mavericks on the Mac Pro (with a few exceptions), as soon as I upgraded to Yosemite, I realized that this was not an OS optimized for speed and responsiveness. Even with the 10.10.1 update, basic interactions that involve the graphical user interface (GUI), such as switching apps, pulling down menus, resizing windows, etc., are sometimes unexpectedly and unexplainably sluggish.

I don’t know for sure if this is because Apple is so focused on battery-saving behaviours these days and feels that the majority of its users are willing to sacrifice a certain level of responsiveness in exchange for extended battery life on their laptops, but right now, my 2014 Mac Pro is only my “fastest Mac ever” on paper. In practice, I really feel, at times, as if we were back to the early days of OS X, when the hardware was desperately playing catch up with the software, especially because precious CPU cycles were wasted on “trendy junk”, a.k.a. “eye candy”. (I certainly remember how the pulsating Aqua buttons and animated progress bars felt on my Mac at the time…)

If that is the reality on my Mac Pro, I simply cannot imagine what it must be on other Mac computers, which are inevitably slower than the Mac Pro at a number of things.

A lot of my computing activities involve typing text and, overall, things aren’t too bad in Yosemite in that department. But I also rely heavily on keyboard-driven OS X automation tools such as Keyboard Maestro, LaunchBar and Typinator, and since the actions that are automated by these tools involve things that are normally done visually through the GUI, unfortunately, they are affected by responsiveness issues as well.

In addition, one particular area where Apple’s engineers appear to have really messed things up in Yosemite is accessibility, a feature on which these tools rely for executing their automatic processes. Because of this, I have had to rewrite or revise several of my Keyboard Maestro macros, and I have had to try and fine-tune LaunchBar’s settings, with mixed results so far. There are still things that do not work properly at all (using automated Tab keystrokes inside Open/Save dialog boxes in Keyboard Maestro macros, resizing windows in Preview with Keyboard Maestro macros, getting LaunchBar to hide its bar once the shortcut has been triggered, etc.). This means that, overall, my work environment has lost a significant amount of polish, smoothness and efficiency, and I am forced to do more things “manually” because Yosemite does not seem to be able to keep up with these automation tools. Yes, this, even on a 2014 Mac Pro with tons of RAM and a large SSD drive, which was working fine under Mavericks.

As regular readers of this blog and my Twitter feed know, one particular issue in Yosemite has been driving me up the wall in the past few weeks. It’s an issue that specifically involves the display of menus in the operating system: displaying menus under the menu headings in the menu bar, displaying menus under menu buttons or in front of pop-up controls in windows, etc.

Probably because of all the changes that Apple has made in Yosemite, with the switch to Helvetica Neue as the system font and new transparency “features”, there are now very visible and noticeable problems with displaying menus in OS X, at least on my machine (and on the machines of several people who have responded by email to my previous blog posts to confirm that they are experiencing similar issues).

When I click on a menu heading in the menu bar, instead of drawing both the blue highlighting behind the menu heading and the menu itself instantaneously, as it used to do, OS X seems to always need a fraction of a second to do so. It’s a tiny fraction, but it’s big enough to be noticeable, and to add to the feeling that the system is sluggish and unresponsive.

(I also highlighted, in a previous blog post, the specific behaviour of the “Help” menu, where the drawing of the blue highlighting is now so visibly slow that you can actually record the “filling down” of the rectangle with blue colour in real time with QuickTime Player.)

To be true, these problems do not seem to affect the usability of the menus themselves, and their impact is mostly of a subjective and “aesthetic” nature. Things just don’t feel right.

There is, however, one particular issue that is not just subjective and aesthetic in nature, and that is the one that I have been focusing on in the past couple of weeks, because it affects the usability of the system in a very real way. It took me a while to figure out exactly what was going on, but I eventually determined the following.

From time to time, when I click on a menu heading in the menu bar (in any application) or on a pop-up menu control in a window in Yosemite, instead of OS X showing the menu right away (even with the tiny delay mentioned above), nothing happens for several seconds. There is no change to the visual aspect of the control, and there is no Spinning Beach Ball of Death either. Then, after several seconds, all of a sudden the menu appears, but it just flashes for a fraction of a second and then immediately disappears again, without letting me select a menu item in it!

I then have to click on the same menu heading or control again, and this second time things work as expected.

There is absolutely no way to predict when this will happen. Sometimes, there is also an additional (and very noticeable) delay between the time the menu is displayed under the menu heading and the time the blue highlighting behind the menu heading itself is drawn.

It also does not matter whether I just execute a simple click action (i.e. a “click-and-release”) with the mouse button or a click-and-hold action (i.e. a click without releasing the mouse button). The latter, of course, used to be the only way to pull down menus a long time ago in Mac OS. But in both cases, the problem is the same. The menu-delay/flashing problem occurs with both types of clicking as far as I can tell.

As you can imagine, this problem, when it occurs on a regular basis, is downright infuriating, especially for someone who is used to working fast, like I am. My mouse-handling skills are of course not perfect (I’m only human), but they are pretty accurate, and, while I sometimes miss my target with the mouse, it’s relatively rare, and has nothing to do with this. (Indeed, the fact that the menu does eventually pull down or pop up, even if only fleetingly, proves that I did click on the target properly.)

It also does not matter whether I use the wireless Magic Mouse or the regular (wired) Apple Mouse. They are both affected.

Since I felt that this problem was clearly unacceptable on any machine, let alone on a fast machine like the 2014 Mac Pro, I decided to do something about it, even though I knew that it would be a time-consuming process, because the bug was intermittent, difficult to describe, and hard to reproduce reliably. But, assuming the eventual success of the process, I felt that the time spent on addressing the problem was worth it, simply because there was too much at stake. (I make my living with this machine, for crying out loud. And I have to try and retain my sanity.)

As I have said in my previous blog posts and on Twitter, as an experienced troubleshooter myself, I tried all kinds of things, to no avail. I won’t go into all the details here (this post is already long enough), but when I tell you that I even went as far as to do a so-called “clean install” of Yosemite after erasing the entire contents of my startup volume, if you’ve ever done this yourself, you’ll know that I really did try everything I could think of and could be reasonably expected of me as a Mac user.

I also spent quite a bit of time on the phone with Apple Care. I ended up talking to a couple of regular tech support people, and to two different senior advisors. They were mostly kind and understanding, and got me to do things that I couldn’t have done on my own (such as letting them remote-watch my system over the Internet using Apple Support’s proprietary tools, and also running data capture tools resulting in large log files destined to be uploaded to Apple’s servers).

But ultimately the whole process was very disappointing. The key thing was to try and get the senior advisors to talk to Apple’s engineers about this, and I did achieve this, but the initial response from the engineers was more than pathetic. Believe it or not, the first thing that they mentioned about the data captured on my system was that my system was being “spammed” (their word, not mine) with error messages such as these:

Dec 11 15:27:03 Mac-Pro-2014.local mdworker[22216]: MySpotlightImporter:persistentStoreCoordinator unable to add persistent store coordinator - Error Domain=NSCocoaErrorDomain Code=256 "The file couldn’t be opened." UserInfo=0x7f81226240b0 {NSSQLiteErrorDomain=14, NSUnderlyingException=I/O error for database at /Users/original/Library/Application Support/DVDpedia/Database.dvdpd.  SQLite error code:14, 'unable to open database file'}
Dec 11 15:27:03 Mac-Pro-2014.local mdworker[22216]: MySpotlightImporter:importFileAtPath:attributes:type: to find object id from path /Users/original/Library/Caches/Metadata/CoreData/DVDpedia/CCCF1415-9FB0-4846-B06E-9F265CFFD005/DVD/_records/6/1/0/p1634.pedia_md_d
Dec 11 15:27:03 Mac-Pro-2014.local mdworker[22216]: CoreData: error: (14) I/O error for database at /Users/original/Library/Application Support/DVDpedia/Database.dvdpd.  SQLite error code:14, 'unable to open database file'
Dec 11 15:27:03 Mac-Pro-2014.local mdworker[22216]: CoreData: error: -addPersistentStoreWithType:SQLite configuration:(null) URL:file:///Users/original/Library/Application%20Support/DVDpedia/Database.dvdpd options:{
	    NSReadOnlyPersistentStoreOption = 1;
	} ... returned error Error Domain=NSCocoaErrorDomain Code=256 "The file couldn’t be opened." UserInfo=0x7f8122624210 {NSSQLiteErrorDomain=14, NSUnderlyingException=I/O error for database at /Users/original/Library/Application Support/DVDpedia/Database.dvdpd.  SQLite error code:14, 'unable to open database file'} with userInfo dictionary {
	    NSSQLiteErrorDomain = 14;
	    NSUnderlyingException = "I/O error for database at /Users/original/Library/Application Support/DVDpedia/Database.dvdpd.  SQLite error code:14, 'unable to open database file’";

To them, it was somehow an indication that there might be something uniquely wrong with my system, and that it had to do with something called “DVDpedia”, which apparently they had never heard of.

DVDpedia is part of a family of applications made by a developer named Bruji, which are an excellent alternative to the increasingly bloated and underwhelming Delicious Library (a tool that I gave up on after the last major upgrade).

DVDpedia is even sold through the Mac App Store!

It boggled my mind that Apple’s engineers apparently could not be bothered to look it up and figure out what DVDpedia was themselves. And it boggles my mind that they would even think that this was the cause of my problem.

DVDpedia is a stand-alone application that manages a database containing records of one’s collection of DVD and Blu-Ray disks (using data pulled from Amazon and other sources). In my mind, there was just no way that this app could cause the responsiveness problem with menus that I was experiencing. As I told the senior advisor, I was even able to reproduce the problem on my clean install of Yosemite before I reinstalled DVDpedia (which I don’t need for my work and is only for my personal use).

In addition, while I am not a developer, I see clearly from the log quoted above that the problem involves not the DVDpedia application itself, but the mdworker process, which is an OS X process used for Spotlight indexing. How could DVDpedia even be blamed for this?

Still, to be thorough, I did contact Bruji via Twitter and their prompt answer was unequivocal:

A warning from a bug in the Apple Spotlight framework, harmless. You can delete Caches/Metadata/CoreData/DVDpedia to fix.

So there you go. Instead of taking my complaints seriously, Apple’s engineers clearly did their utmost to try and find a third-party tool that might be responsible for my problems, even after all the troubleshooting work that I had done.

Now, of course, I couldn’t rule out the involvement of third-party software in this bug entirely, especially because the bug was hard to reproduce on a clean system before reinstalling all kinds of ordinary apps (iWork, Microsoft Office, Adobe Creative Suite, etc.). But, having eliminated any software that involved kernel extensions and the like, I was finding it difficult, if not impossible, to imagine that a regular, stand-alone application with no nasty behind-the-scenes process running in the background could explain, all by itself, a system-wide bug such as the one that I was experiencing.

I don’t know if the initial response from the engineers was due to communication issues with the senior advisor or simply to a lack of goodwill on their part (I suspect the latter), but I still find it shocking that they couldn’t even look into the DVDpedia thing themselves before getting back to me with that as a response.

Nonetheless, I did turn the Spotlight indexing of DVDpedia stuff off by following instructions provided by Bruji, and produced a new set of data for the engineers to examine, presumably without the “spamming” identified above. (I don’t know where they got these logged error messages from. I couldn’t find them myself via the Console.)

But that’s all I’ve heard from engineers so far, except for a separate response from the bug report that I submitted independently, via Bug Reporter a while ago, which eventually elicited a generic “Bug report is a duplicatemessage. This might be viewed as encouraging, except that, in reality, it only confirms that I am not the only one suffering from the problem. It gives no indication of whether the engineers are able to reproduce the problem themselves or working on a fix.

Meanwhile, the beauty of the Internet is, of course, that, by talking about this on my blog and on Twitter, I managed to elicit various responses from readers. Some shared Yosemite problems that had nothing to do with mine. But other shared that they too were suffering from the exact same menu-delay/flashing problem as the one I was experiencing.

Eventually, one of those readers suggested that I try turning on the “Increase contrast” option in the “Accessibility” preference pane in System Preferences, under Vision › Display. Apparently, this was helping him alleviate the problem somewhat.

Even though the increased contrast results in a fairly ugly visual interface, I was able to confirm that, on my machine at least, it did seem to not just alleviate the problem, but actually eliminate the menu-delay/flashing problem altogether!

Since the “Increase contrast” feature actually (and logically) automatically disables the “Reduce transparency” feature at the same time, I figured that the next step would be to try and reproduce the menu-delay/flashing problem without the “Increase contrast” setting turned on, but also without the “Reduce transparency” setting turned on.

(As mentioned above, turning on the “Reduce transparency” setting was one of the first things that I did after installing Yosemite.)

That was a couple of days ago, so maybe it’s still too early to tell. But so far, with both “Reduce transparency” and “Increase contrast” off — i.e. with the default values for these settings in Yosemite, I haven’t been able to reproduce the menu-delay/flashing problem.

Then this morning I turned the “Reduce transparency” setting back on, just to see. And within an hour, I was able to reproduce the menu-delay/flashing problem once. Needless to say, I have turned it back off.

The most reasonable conclusion for me at this point is that there is something wrong with Yosemite, and that the “Reduce transparency” feature actually makes it worse, much worse. It triggers a serious responsiveness issue that has a major impact on usability, whereas with the feature off, the responsiveness issue appears to remain dormant or at least be limited to “only” a general feeling of sluggishness.

Still, I am sure that there are tons of other users who are also using this “Reduce transparency” feature, which is well documented. (I am far from being alone in finding this extra transparency totally useless and distracting, to say the least.) So, if the menu-delay/flashing problem is indeed caused by this feature, how come there aren’t more people complaining about it and bringing it to Apple’s attention?

My theory at this point, based on the multiple troubleshooting steps that I have taken, is that the severity of the menu-delay/flashing problem itself is not identical for everyone, and depends on a variety of factors, including the size and number of displays used. I have two large screens and I have noticed that the problem is less prevalent if I only use the big Sharp screen, and even less prevalent if I only use the (smaller) Cinema Display. (But it’s still there, even with only one smaller display. It’s just far more intermittent, and the delays don’t seem to be as long.) It is also possible that this problem only affects certain Mac models, including the 2013/2014 Mac Pro and also (at the very least) some MacBook Pros, possibly because they use a certain kind of video card.

I wouldn’t be surprised, at this stage, if the severity of the bug was actually proportional to the number of pixels that the video cards in those machines have to handle, and also if it was affected by some other general factors, such as the amount of RAM currently in use (because the problem is definitely more noticeable when there are more apps running, even if they are not doing anything or using any CPU power).

This whole mess also highlights a couple of significant ironies:

  • This menu-delay/flashing problem when “Reduce transparency” is on effectively means that, far from improving system responsiveness by eliminating the waste of resources for useless eye candy, this accessibility feature actually make Yosemite less responsive. (This is also why I never thought that this feature was a possible suspect in all this, and I am eternally grateful to the Betalogue reader who brought it to my attention. I don’t know where he got the idea from.)
  • It is possible that this menu-delay/flashing problem only affects a specific category of machines, which happens to be a range of pro-level machines that people buy precisely because they seek power and high performance levels. So they, like me, pay a premium for quality hardware, and now they end up being somehow punished for it!
  • Spending hours on the problem with Apple Care yielded nothing, but sharing my problems with others online enabled me to find an apparent solution by myself.

I honestly find the response I have received from Apple Care (and from Apple engineers via Apple Care) so far severely insufficient. The very fact that, aside from this particular bug, Yosemite feels sluggish, even on a fast machine, is to me an indication that Apple’s priorities lie elsewhere and that they do not take performance issues seriously enough.

I realize that these issues are hard to describe, reproduce, and observe. But with the resources at its disposal, Apple has an obligation to engage in a serious effort of improving quality control and quality assurance processes. It is simply unacceptable that such a bug is allowed to slip through.

Allowing people to use beta software is all well and good, but it’s not an excuse for not doing your own testing in-house, recreating real-world configurations (with displays of various sizes and combinations, and third-party software from all kinds of sources and in all kinds of combinations).

Also, when I see how Apple tends to responds to the bug reports that I submit either via Bug Reporter or as part of my involvement with AppleSeed, I have serious reservations about the effectiveness of their communication system.

Right now, I am beta-testing the next build of OS X 10.10.2 on a separate partition. I am not allowed to talk about features or anything like that, but there is one serious new, 100% reproducible bug (on my machine at least) that causes a kernel panic every time I trigger it. I have sent a detailed bug report about it, but because it involves my specific system configuration and a couple of third-party hardware devices (external Thunderbolt storage, which I have no choice but to use with the storage-deprived Mac Pro!), I am not optimistic that the bug will be fixed by the time 10.10.2 comes out. Maybe it will. But I won’t be surprised if it isn’t. And if it isn’t, I’ll be deprived of any further improvements made to Yosemite until Apple fixes it, which could take months.

This whole experience has been quite traumatic for me as a Mac user. The menu-delay/flashing problem was driving me insane in my daily use of my Mac, and I sure hope that switching the transparency back on, regardless of how much I loathe it, keeps it at bay permanently.

But more generally, I still find the overall sluggishness of Yosemite unacceptable. I certainly expect better performance from a 2014 Mac Pro with tons of RAM and a fast SSD. I simply should not have to wait a couple of seconds when switching apps, for instance, especially when these apps are not doing anything significant at the time.

Is there anyone at Apple who cares about all this? Anyone who has the power to do anything about it? I do not know, but this general sloppiness and the lack of transparent (!) communication when real problems occur are highly problematic. I really feel that Apple has enough money in the bank to do a much better job of supporting its Mac users and providing them with a top-quality computing experience. I really feel that, as Mac users, we have yet to see the real benefits of the great financial boom that Apple has experienced in the past ten years or so.


OS X 10.10 (Yosemite): Responsiveness problems not solved by clean install

Posted by Pierre Igot in: Macintosh
December 8th, 2014 • 10:29 am

I just don’t understand.

Last week I reported about noticeable responsiveness issues that I was experiencing with OS X 10.10.1 (Yosemite) on my 2014 Mac Pro:

  1. visibly slow drawing of the blue highlighting behind the “Help” menu heading in the menu bar;
  2. interruptions when scrolling down a Finder window with a long list of files (namely, the /System/Library/Extensions/ folder, which contains over 200 .kext files) with the Magic Mouse;
  3. recurring addition of new log files in /private/var/log/asl/ involving the sandboxd process;
  4. and, last but not least, recurring lack of responsiveness of various UI controls, including menu headings and buttons for pop-up menus.

I also mentioned that I had talked to someone (named Justin) at AppleCare, who had promised to escalate the issue if I couldn’t solve it with the usual troubleshooting steps.

On November 28, I called AppleCare again and talked to a man who seemed much less interested in my problem than Justin had been a few days before. However, to his credit, he promptly put me in touch with a senior advisor, whose name was Ben.

Ben was very nice to me on the phone and he too agreed that the problem sounded real and was not normal for a powerful machine such as my 2014 Mac Pro. However, before referring the issue to engineering, he wanted me to try one other thing, which was to reinstall Mavericks (the OS that the Mac Pro came with) on a separate partition and to check if I could reproduce my issues there.

This took me a while, because of course I had to redownload the Mavericks installer from the App Store and create a bootable USB volume with it. (Ben wanted me to use Internet Restore for this, which would have neutralized my Mac for the duration of the download, i.e. at least two hours.)

With my Mavericks partition, I was above to confirm the following:

  1. There was no problem with drawing the blue highlighting behind the “Help” menu heading in the menu bar in Mavericks.
  2. The interruptions when scrolling down a Finder window with the contents of the /System/Library/Extensions/ folder with the Magic Mouse did also occur in Mavericks, so it wasn’t a new problem in Yosemite, just one that I had not noticed before.
  3. The recurring addition of new log files in /private/var/log/asl/ involving the sandboxd process didn’t seem to occur in Mavericks.
  4. The lack of responsiveness of various UI controls, including menu headings and buttons for pop-up menus, didn’t exist in Mavericks.

In addition, through additional tests of my own, I was able to establish the following:

  1. The interruptions when scrolling down a Finder window with the contents of the /System/Library/Extensions/ folder only occurred with the Magic Mouse, and not with the old wired Apple Mouse, so it was a Magic Mouse-specific issue, probably unrelated to other responsiveness issues in Yosemite.
  2. The recurring addition of new log files in /private/var/log/asl/ involving the sandboxd process was much worse in the early build of OS X 10.10.2 that I had been testing as part of my involvement in the AppleSeed program than in the official 10.10.1 release. I still saw the problem on occasion in 10.10.1, but much less frequently. There was also no clear indication whether this had any link to responsiveness issues.
  3. The only situation where I did not see the problem with the drawing of the blue highlighting behind the “Help” menu heading in the menu bar in Yosemite was when I used the Yosemite-based Internet Restore service (while on the phone with Ben). But with all other versions of Yosemite that I tried, including on a clean partition with nothing else installed on it, I could see the problem.

Ben had given me his contact info, so I sent him an email explaining all this. At the same time, however, I stressed that the most important issue for me was that recurring, undesirable delay when clicking on various UI controls to bring down menus. Such a lack of responsiveness is unacceptable on a machine as powerful as a 2014 Mac Pro.

I am convinced that the problem is caused by Yosemite itself, and not by any incompatibility with third-party software. Since Ben didn’t get back to me right away, or even after I left a message on his voicemail a few days later, I took it upon myself to rebuild my entire startup volume from scratch.

You need to realize that this is a massive undertaking. Wiping the system volume and rebuilding everything involves hundreds of manual steps, not just to reinstall the software, but also either having to redo all the configuration of the software manually in the software itself (which involves trying to remember where you have to do to set this and that, given that settings are never always where you expect them to be) or manually copying all kinds of preference/configuration files from an backup of the old volume to the new.

But that’s exactly what I did. It took me hours, and I made sure I didn’t install anything that involved any kind of kernel extension or any other “questionable” (at least in the eyes of Apple’s engineers as I imagine them) software practices on the part of third-party developers. (This means that, even now, several days later, I am still without my beloved Default Folder X, for instance.)

It was materially impossible for me to the following:

  1. Install only one new piece of software, configure it and start using it.
  2. Continue using the entire system for several hours (if not days) with only that one new piece of software, and see if the responsiveness issue with the UI controls resurfaces.

I have far too many pieces of software that I use and need for my work to be able to do this. It would take me weeks. So, yes, I reinstalled things in batches. But I did try and do the above for the most “suspicious” pieces of software, i.e. for system-wide utilities such as Keyboard Maestro, LaunchBar, and Typinator. These particular pieces of software might be deemed “suspicious” by Apple’s engineers not because they involve kernel extensions or anything like that, but because they effectively act as an additional layer between the user and the machine, which intercepts and processes some interactions in order to automate specific actions. They are extremely important for my productivity and I cannot really imagine doing any substantial work on my machine without them.

Still, for about 48 hours I was reasonably optimistic that, through the rebuilding on my entire system from scratch, I had been able to eliminate that one responsiveness issue when clicking on UI controls that bring up menus.

And then yesterday afternoon I noticed it again. And again.

It might be deemed subtle and minor by some, but to me it’s obvious and extremely annoying. It is very simple to describe: I do a simple click (without holding the mouse button down) on a menu heading or a button in a window, and then nothing happens for two or three seconds, and then all of a sudden the menu that the click was supposed to cause to appear right away finally appears, but then immediately disappears again, before I have even had a chance to select a menu item in it.

And then of course I click on the same UI control again, and everything works fine.

I just do not understand how I can be the only one noticing this and finding it intolerable. I realize that we live in the age of mobile devices, where people “tap” instead of clicking, and are much more tolerant of unresponsiveness, because, with a finger tap, there is always a reasonable possibility that the tap was not accurate enough and that the system didn’t detect it.

But this is not the same. Clearly the system detects my click, since it does display the menu (albeit very briefly) after a couple of seconds. So the problem cannot be blamed on imperfect clicking skills on my part.

To me, it looks as if the UI control in question has been somehow “put to sleep” by the OS and take a couple of seconds to “wake up”.

Is that what is really going on here? If it is, there is no excuse for such a behaviour. On a desktop machine with an unlimited supply of power (unlike a battery-powered laptop), power-saving “features” that come at the expense of basic responsiveness are simply not acceptable, or should at the very least be optional.

Then again, my interpretation of the situation might be wrong. This particular problem might have nothing to do with power-saving features in OS X, and might be simply an unintentional bug of some kind. But if so, how come I seem to be the only one noticing it, and making a fuss about it? All I am doing is using readily available software on my system to do real-world work. If the problem was caused by a specific piece of software, I should be able to eliminate it by quitting that piece of software. But that does not seem to be the case. It seems that it is the cumulative effect of having multiple pieces of software running, even if they are not using significant amounts of CPU power, that is causing the responsiveness problem to surface.

After doing a clean install and completely rebuilding my OS X environment from scratch, how can any kind of specific software incompatibility be blamed as the source of the problem?

I also do not see how this could be a hardware issue. My Mac Pro works fine otherwise, and there are no responsiveness issues when typing text or doing other stuff that does not involve clicking on UI controls to bring up menus.

I just do not understand. And I don’t know what to do. Ben won’t respond to my emails and phone calls. Maybe he feels that, if he ignores me long enough, I will stop complaining and learn to accept the inevitability of this issue.

But I won’t. I will never find such a lack of responsiveness on my expensive 2014 Mac Pro acceptable. What recourse do I have, though? I don’t see how giving up on Apple products or boycotting them will help. I know enough about the alternatives to know that they would drive me even crazier on a daily basis. I already have to suffer through the obligation of having to use Microsoft Word for my work, for goodness’s sake. This complete piece of shit has an enormous number of responsiveness problems, even on a superfast 2014 Mac Pro, and here again nobody seems to care but me. What can I do? I have to use Word for my work. I do as much as I can to avoid it, but there is a limit.

So now I am supposed to accept that Apple is going down that same route and stuffing unresponsive software down our throats and that there’s nothing that I can do about it?

I will never, ever accept this. It is simply not right. But I do not know what else to do.

Now, if you’ll excuse me, I have work to do, and my rebuiding of my system is still far from complete and will still take me several more hours. I don’t know if you can tell, but I am pissed.


Numbers 3.5: Keeps displaying time in date cells with no time

Posted by Pierre Igot in: Macintosh
November 27th, 2014 • 4:56 pm

At the same time I upgraded my Mac Pro to Yosemite last week, I decided to upgrade to the latest versions of Pages and Numbers. Because of the disastrous changes made by Apple in Pages 5 compared to Pages ’09, I have permanently switched to Nisus Writer Pro as my default word processor, and I have no intention of going back.

However, I still have a fairly large number of existing documents in Pages format and I still have a couple of custom templates that I developed myself over the years and will need more time to rebuild using another word processor.

Also, I still positively loathe anything Microsoft, including Excel 2011, and so I need an alternative for crunching numbers. I have been using Numbers ’09 for years and do all my manual bookkeeping in Numbers ’09 spreadsheets. However, my needs remain fairly basic and I figure that support for Numbers ’09 will not continue forever. In addition, Numbers ’09 has some outstanding issues on my machine that will clearly never get fixed, including the inability to resume properly (i.e. to reopen all spreadsheets that were still open when the application was quit) after a machine restart.

(In light of its history, particularly with AppleWorks, I also don’t trust Apple to maintain backward file format compatibility forever, so I am fairly convinced that one day, it’ll be impossible to read iWork ’09 files on any current computer.)

All this to say that I have now switched to Pages 5 and Numbers 3.5. I still don’t like the fundamental changes that Apple has made to the interface. (I much preferred the flexible stand-alone inspectors to this extra column on the side.) But I guess I’ll have to live with them.

There are several new annoyances in those applications, however, and I feel that I should document the most egregious ones here.

One particular annoyance in Numbers 3.5 involves table cells that are formatted using the data format called “Date & Time”:

numbers35-datetime

I often use the data format for dates illustrated in the picture above. It clearly indicates that I want to display the date in the YYYY/MM/DD format and with no time indication at all.

And yet, every time I enter a table cell in Numbers 3.5 that contains a date and is formatted using this particular data format, here’s what I get:

numbers35-datetime2

Numbers 3.5 correctly makes the contents of the cell editable, but, in addition to the date, it displays a value of “00:00:00” for the time, even though my data format clearly indicates that “None” for the time. (Numbers also displays the date with “00:00:00” appended to it in the “Actual” area in the status bar at the bottom of the window.)

This only affects the table cell when it’s editable, and it’s a new problem that didn’t exist in Numbers ’09. When the cell is not editable, Numbers 3.5 correctly displays the date only, with no indication of time.

But it’s a very big annoyance when editing table cells containing dates!

There are two ways to make cells editable: one is to double-click on the cell with the mouse and one is to press option-Return. Also, depending on whether the “Wrap text in cell” option (under the “Text” tab) is checked for the cell in question and depending on how wide the cell is, the “00:00:00” might be displayed next to the date on the same line or on a separate line below.

This makes it very hard to predict where the insertion point is going to end up after double-clicking on the cell. More often than not, it will end up somewhere in the “00:00:00”, and therefore force me to use extra keystrokes with the cursor keys to go to the date part in order to edit it.

When making the cell editable with the keyboard (with option-Return), things are even worse, since this shortcut automatically put the insertion point at the very end of the cell, i.e. after the “00:00:00” part. If I want to edit the date, I have to track all the way back with cursor keys or switch to the mouse.

Again, I find myself wondering: What on earth were Apple’s software engineers thinking? Or are they become so sloppy that annoyances like these are now going to be regular occurrences and take years to fix each time, like they do in Microsoft products? What on earth happened to quality control? What on earth happened to proper software testing? What on earth happened to listening to user feedback?

Like I said in a recent tweet, I am really starting to feel as if we are returning to some kind of Dark Ages of Computing. No attention to detail. Innumberable bugs. Poor performance. Etc.

The bad news for Apple is that we Mac users are acutely aware of how useful and efficient computers could and should be. The general masses of iOS users might be more tolerant and more blasé about the whole thing, but I sure hope that there still is a strong core of OS X users who care about those things and will continue to put pressure on Apple to rediscover the demanding attitude that once made it a great computing company.


Mail 8 (Yosemite): Now includes attribution at same quote level as quotation itself

Posted by Pierre Igot in: Macintosh
November 27th, 2014 • 3:37 pm

Every so often (albeit with worryingly increasing frequency these days), Apple’s software engineers make changes that make you shake your head in disbelief.

Now that I have upgraded to Yosemite (a not-too-pleasant experience), I have a fresh new batch of such changes to describe.

Here’s one that has to do with the new version of Mail included in Yosemite. As far as I can tell, when Mail is set to quote the text of the original message in replies (and also to increase the quote level of the original text that is quoted), the attribution of the quote, i.e. the top line that reads “On [date] [time], XXX wrote:” is now no longer at the root level of your reply, but at the same quote level as the quoted text that follows:

Mail8-QuoteLevel

This makes no sense whatsoever. Why would the attribution look like it is part of the quoted text? It is not part of the quoted text! It is something that Mail adds automatically as a way to introduce the quoted text, and the insertion of this attribution is just a way to save you time by not requiring you to type out this attribution yourself in your reply. It is meant to read like something that you wrote, not like something that the sender of the original message wrote!

I just don’t understand how this could happen. Are Apple’s software engineers becoming blind? Don’t they care about such details at all? Do they consider that the majority of OS X users don’t care about such details either?

I do realize that the lack of universal, flexible standards for quoting text in replies has created pervasive problems in email correspondence over the years. Even in my own correspondence, I still get replies from people where there is no quote level whatsoever and the respondent has manually used a different text colour (in rich text, of course) to insert his/her reply manually next to the original text, without so much as a line break between the original text and his/her reply.

But that does not mean that I don’t care about the appropriate use of quote levels, and that does not mean that Apple should add to the confusion by misusing quote levels itself in its Mail application!


OS X 10.10 (Yosemite): Unacceptable levels of responsiveness

Posted by Pierre Igot in: Macintosh
November 27th, 2014 • 10:32 am

In the summer, I bought a new Mac Pro. My previous machine (a 2009 Mac Pro) was getting long in the tooth. It would have lasted a while longer, but then one of my screens broke and I had to replace that, so I figured that, the 2013 Mac Pro having been out for a while, it was as good a time as any to proceed with the upgrade that I had been planning to do eventually.

When I purchased the expensive new Mac Pro (along with a new screen, and Thunderbolt storage), I figured that I was making an investment for the next few years and paying for the guarantee of a high-performance machine that would be able to take full advantage of the upcoming software releases, including OS X 10.10 (Yosemite). I certainly did not expect to have to deal with a sluggish operating system any time soon.

Then along came Yosemite. I didn’t upgrade until the official 10.10.1 version came out. Last Thursday, I downloaded the OS X 10.10.1 package from the App Store and installed it. As system upgrades go, it was relatively painless. I had made sure that all my applications were up to date, and that known issues were properly dealt with. (I use Snapz Pro X for screen captures, but I can live without audio support for a little while, although the slow release of software updates by Ambrosia is getting irritating.)

After installing Yosemite, however, I immediately started noticing significant responsiveness issues. Most notably, I would regularly encounter a situation where clicking on a menu heading in the menu bar would fail to highlight the menu heading and bring down the menu itself right away. It would take a second or two, by which time my muscle memory had already taken me elsewhere, and so I would lose the focus and have to start all over again.

This was just one particular, obvious manifestation of the responsiveness problem. But it certainly wasn’t the only one. There were pervasive issues throughout the user interface and in various applications.

I did what an experienced troubleshooter would do, i.e. I tried to determine whether there were obvious culprits. I stopped using a couple of things that I didn’t absolutely need, such as Flux (which seemed to be making the problem even worse in the evenings). I cleared caches. I deleted preference files. I went through extensions, launch agents and launch daemons. I checked the system logs in the Console. I noticed a number of suspicious-looking error messages in the Console, with no indication of what I could do to stop them.

In particular, I noticed that I would occasionally get this message:

14-11-27 09:50:39,746 sandboxd[283]: ([241]) deleted(241) deny mach-lookup com.apple.desktoppicture

along with the creation of a new log file in /private/var/log/asl/, inside a folder called “AUX” followed by the date. The log files thus created all had similar contents, always starting with:

Process:         deleted [265]
Path:            /System/Library/PrivateFrameworks/CacheDelete.framework/deleted
Load Address:    0x102062000
Identifier:      deleted
Version:         ??? (???)
Code Type:       x86_64 (Native)
Parent Process:  launchd [1]

This was still going on a week later. After some house cleaning, I thought I had eliminated the problem. And then this morning, another similar one started. This time, I still get something from sandboxd, but it involves a different “mach-lookup”:

14-11-29 14:41:10,278 sandboxd[295]: ([378]) softwareupdated(378) deny mach-lookup com.apple.webinspector

Here again, each time the message appears,a new log file is created inside a folder called “AUX” followed by the date, in /private/var/log/asl/. The full logs all have similar contents, always starting with:

Process:         softwareupdated [378]
Path:            /System/Library/CoreServices/Software Update.app/Contents/Resources/softwareupdated
Load Address:    0x107289000
Identifier:      softwareupdated
Version:         ??? (???)
Code Type:       x86_64 (Native)
Parent Process:  launchd [1]

I have no idea what it means. I was able to reproduce this problem on a clean install of OS X 10.10.1 on a separate partition, although the actual error message in the Console was, again, different in that system, and the number of logs thus created was much smaller. So there’s something going on here. However, I have no idea whether it has anything to do with the responsiveness issue.

Responsiveness issues are hard to describe. But over the past 24 hours, I have been able to identify one obvious symptom, which is 100% reproducible on my 2014 Mac Pro with a clean install of OS X 10.10.1 on a separate partition. Whenever I click on the “Help” menu, in any OS X app, the drawing of the blue highlighting behind the blue heading is so slow that I can actually see it happen in real time.

I have even been able to record the problem with QuickTime Player’s screen recording feature. Here’s the movie that I created:

And here’s a frame of the movie that illustrates the very problem I am describing:

SlowYosemite

This is a picture of Yosemite caught in flagrante delicto drawing the blue highlighting for the “Help” menu heading as a snail’s pace.

Now, I don’t know about you. But I have spent thousands of dollars on this new Mac Pro, and I certainly didn’t expect to find myself, in late 2014, repeatedly staring at such sluggishness.

Again, I can reproduce this particular problem on a clean install of OS X 10.10.1 on a separate partition. This is not something that is due to some third-party software conflict.

How can Apple’s engineers justify this? Do they really think we won’t notice? Do they really think we won’t care?

Well, I am sorry, but after having paid that kind of money for this machine, I do notice, and I do care.

Granted, this particular behaviour with the “Help” menu heading appears to be unique. I cannot reproduce the same extremely slow (relatively speaking) “filling down” of the blue background with other menu headings. But it’s just one of the most obvious symptoms of a pervasive lack of responsiveness in Yosemite, even on a brand new 2014 Mac Pro with tons of RAM and a large SSD startup disk.

Other menu headings behave somewhat better, but they still don’t feel normal to me. I quite often notice that the menu itself is drawn by OS X before the blue highlighting of the menu heading comes on. And, as I said, sometimes there is even a delay of a second or two before anything happens after I click on the menu heading.

How on earth did we get to this? How on earth can such sluggishness be deemed acceptable on such a powerful machine? I really feel that I have been had.

I might be in a minority that actually notices such things and cares about them. I know that other people are reporting even more egregious responsiveness issues. These are positively awful. My issues are not that bad. But they are still bad enough to be deemed unacceptable after having spent so much money on new hardware.

Yesterday I decided that I had had enough and picked up the phone to call Apple Care. After all, my new Mac Pro is under warranty. I was fortunate enough to get through to a nice fellow named Justin, who immediately grasped that I was an experienced troubleshooter myself and didn’t make me go through the most obvious steps, which I had already done.

He readily agreed that the type of performance issues I was describing was not acceptable for a user of a fully-decked 2014 Mac Pro. And he also fully acknowledged that having to spend hours trying to troubleshoot such issues was a waste of my time and that it was important that we came to some kind of resolution.

We tried a few things, and then we came to the conclusion that I definitely would have to try and reproduce my issues on a clean install of OS X on a separate partition. He asked me to proceed with that and set up an appointment for Apple to call me back Friday morning.

So that is what I did last night. I cursed OS X’s default behaviour, which erases the OS X installer after it’s done installing the software, and proceeded to re-download Yosemite from the App Store, which of course took a couple of hours with my 7 Mbps connection. I created a separate partition on my SSD and installed the software, which was reasonably fast. (Still, I don’t understand why the OS X installer cannot install OS X on another volume in the background while letting the user continue to use his current system on his current startup volume.)

I didn’t spend enough time on that clean partition with nothing installed to try and reproduce all kinds of responsiveness issues. But the slow drawing of the “Help” menu heading highlighting is definitely there, and that in itself is unacceptable.

I fully intend to use this particular example (and refer to this blog post) when Apple (Justin or someone else) calls me back tomorrow morning to find out what happened with my clean install of OS X 10.10.1 on a separate partition.

Will they acknowledge that there is a responsiveness issue with Yosemite? Will they try to argue that it’s “normal” for a .1 release and will be addressed eventually? I guess we’ll see. Justin promised me that he would escalate the issue if we couldn’t resolve it. I’d be really curious to have the point of view of an engineer on this situation.

That said, I want to stress that the problem with the “Help” menu heading is just one of the tips of the iceberg as far as I am concerned. I have chosen to single it out because, in my view, it’s so egregious and outrageous, and also fairly easy to reproduce (at least on my machine).

Since Mavericks was running fine on this Mac Pro (except for some more minor issues), it is really hard to imagine that there is any basis for blaming the hardware for any of this. It is so obviously a problem with Yosemite that I sure hope to be able to convince someone that something is seriously wrong and that something really need to be done about it really soon. Otherwise, I am going to start talking about asking for my money back!


OS X Tip: Better than ‘Paste and Match Style’

Posted by Pierre Igot in: Macintosh, Microsoft, Pages
November 15th, 2014 • 11:48 am

Yesterday, Jason Snell’s Six Colors web site posted a tip about using OS X’s built-in keyboard customization features to create a universal shortcut for a command for pasting as plain text.

While the tip is not useless, it neglected to mention the fact that the menu command labelled “Paste and Match Style” is not nearly as universal as it should be. For instance, Adobe’s InDesign has a similar command, but it’s called “Paste without Formatting”. In Nisus Writer Pro, the command is a subitem in the submenu “Paste” in the “Edit” menu, and it is called “Paste Text Only”.

Secondly, even when in applications where the command exists and is called “Paste and Match Style”, it does not always work as expected.

For instance, for the particular purpose that the Six Colors article mentions (converting rich-text hyperlinks to plain-text URLs), the command only works if you initially used the “Copy Link” command to copy the hyperlink in the first place. If you used the regular “Copy” command, “Paste and Match Style” is unable to convert the link into its URL. In several applications, including Microsoft Word 2011 and OS X’s own TextEdit, when my Clipboard contains a rich-text hyperlink copied using the regular “Copy” command, the “Paste and Match Style” command does not convert the hyperlink into a plain-text URL and inserts it. It simply inserts the rich-text hyperlink with the font face, size, etc. of the underlying paragraph where it is pasted.

Nisus Writer Pro’s “Paste Text Only” strips the hyperlink and simply pastes the text of the hyperlink label (not the URL) as plain text.

In fact, I am not aware of a single OS X application where using the “Paste and Match Style” command (or its equivalent) when the Clipboard contains a rich-text hyperlink copied using the regular “Copy” command manages to paste the URL of the hyperlink as plain text. The only time it works is when the hyperlink label (i.e. the text of the link, as opposed to its underlying URL) matches the URL itself.

And, as anyone who has ever received phishing spam knows, the text label of a hyperlink not always matches its underlying URL. It can actually even be a different URL!

So, outside of the situation where the hyperlink is originally copied using the “Copy Link” command, I am not quite sure how useful Six Colors’s tip actually is.

It does highlight, however, the need for some kind of universal “Paste without Formatting” command that has a consistent behaviour across all OS X applications.

After years of using various options, I now have a solution that meets most of my needs. But it relies on a couple of third-party tools: Keyboard Maestro and BBEdit.

Here it is:

kmmacro-pasteplain

As you can see, it does not rely on any application’s built-in command for pasting without formatting. Instead, it uses a couple of Clipboard filters included in Keyboard Maestro, a few find/replace operations with regular expressions, and a text factory included in BBEdit.

The first step is based on Keyboard Maestro’s built-in “Remove Styles” Clipboard filter. This removes bold, italics, font size, font face, and a variety of other rich-text attributes.

Then the macro uses Keyboard Maestro’s built-in “Trim Styles” Clipboard filter. This is primarily because of Microsoft Word’s idiotic word-by-word selection behaviour, which automatically selects the trailing space after the last selected word (unless there is a punctuation sign). Since I use word-by-word selection all the time when copying text from within Word documents, I constantly end up with an undesirable trailing space at the end of my Clipboard text. The Keyboard Maestro filter takes care of that. (It also trims spaces at the beginning of the Clipboard text if there are any, and also tabs and other types of “whitespace”.)

Then the next two steps are designed to work around another problem in which Microsoft Word’s idiotic design plays a significant role. If the text you are copying happens to be inside a paragraph formatted with bullets or automatic numbering, and if your selection includes the very first word in the paragraph, then Word also includes the bullet or automatic number formatting with the copied text.

Of course, because Microsoft’s engineers are narrow-minded and like to pretend (subconsciously at least) that Word users only ever use Word and have no need for compatibility and user-friendliness across applications, when you paste the copied text that includes the bullet or automatic number formatting in the middle of another paragraph without bullets or automatic numbering or even in an empty paragraph in a Word document, Word does not include the bullet or automatic numbering with the pasted text.

And if you use Word’s own “Paste and Match Formatting” command with such a Clipboard (which includes both the text and bullet or automatic number formatting), Word does not include the bullet or automatic numbering with the pasted plain text.

If, however, you try and paste this particular Clipboard into another application as plain text (for example, in BBEdit, or in a search field on a web page), then the bullet or automatic number will be included in its plain text form (i.e. as actual bullet or number characters, plus whatever separates them from the following text) at the beginning of the pasted text. This is extremely irritating, and I have talked about it before.

The macro above takes care of this. The find/replace operations with regular expressions remove various kinds of bullets and automatic numbering (with actual numbers or letters), as well as the extra tab character (or period plus tab character, or closing parenthesis plus tab character) between the bullet/number and the beginning of the text. (I should note that I am far from being an expert at writing regular expressions, and these might not be the most efficient way to achieve this. But they seem to be working for me.)

I have another step in the macro for removing tab characters by themselves, which I find useful in situations where the previous steps didn’t remove all the tab characters. You might consider it optional or undesirable.

And finally I have a step that uses one of BBEdit’s built-in text factories to change straight apostrophes and quotation marks to curly ones. This is because text copied from web pages and other sources often contains straight quotes and apostrophes, and the contexts where I want to use the pasted plain text usually require curly ones.

I could have used Keyboard Maestro’s own Clipboard filter for smart quotes, but it does not deal with apostrophes at all. Peter Lewis once told me, “Keyboard Maestro does not attempt to figure out apostrophes because it is almost impossible to do right.” But that’s because he’s trying to deal with the use of single quotes, which is an English-specific problem. (We don’t use single quotes in French at all, so all our apostrophes are curled to the left.)

I find that BBEdit’s text factory handles the situation quite well for me, both in English and in French.

And that’s all, folks!

Because this is a Keyboard Maestro and it’s inside my “Global” group of macros that are enabled everywhere, in all applications, I can use it in all the applications that I work with, including the ones that have their own built-in “Paste without Formatting” or “Paste and Match Styles” command. And I have now become completely dependent on it.

In fact, I even use this macro inside other Keyboard Maestro macros that I have for looking up stuff in various on-line databases. This way, I can just select whatever I want to look up in my current document, without having to worry about the cruft that might be included with the selection when it’s copied to the Clipboard, and then invoke the macro that I have to look up stuff in a particular database, which in turn uses my “Paste plain and trimmed” macro when it needs to paste the selection into the database’s search field.

Of course, my macro is not perfect. There are a few situations where it strips stuff that I didn’t really want it to strip. If I really cared, I could create another macro, with fewer steps, that only strips some of the stuff, and try and remember to use that macro in such cases. But I cannot really be bothered. I’d rather have one universal macro, with an easy one-hand shortcut, that works well most of the time, and then fix whatever needs to be fixed manually in those few cases when I need to do so.

My macro also does not address the issue raised by Six Colors, which is the extraction of the URL as plain text from a rich-text hyperlink. For that, I am afraid I don’t have an easy solution (unless the hyperlink was originally copied using the “Copy Link” command, which is not universal). I tend to do it “manually” whenever I need to. Unfortunately, in Microsoft Word, that involves taking repeated trips to the “Edit Hyperlink” modal dialog box. If I really needed to do this all the time, I would probably try to find an easier way to do it. And it would probably involve Keyboard Maestro again.

But that might be for another post, when I finally get around to it.


OS X 10.9 (Mavericks): Bug with ‘Secure Keyboard Mode’

Posted by Pierre Igot in: Macintosh
November 12th, 2014 • 10:26 am

OS X has a special keyboard mode that it switches to whenever the cursor is in a password field (in an OS X dialog, in a web page in Safari, Firefox, in apps that require password input, etc.). This mode, called ‘Secure Keyboard Mode’, is intended to protect your password input from any kind of snooping by other apps, so that your password remains secure.

In a normal configuration, you don’t really notice when OS X switches from regular keyboard input mode to secure keyboard mode or vice versa, because it’s something that takes place behind the scenes.

However, there are a number of OS X utilities that rely on the ability to monitor your keyboard typing and are therefore affected when OS X switches keyboard input modes like this. I use at least two of these apps myself: Keyboard Maestro and Typinator. The usability of these fantastic apps is totally dependent on their ability to monitor keyboard input, so secure keyboard mode definitely affects them.

Recently, I have noticed a new problem on my Mac Pro running OS X 10.9.5. Sometimes — usually in the morning, after I wake my machine from sleep and start working — I notice that my Keyboard Maestro macros and Typinator features do not work properly. Upon careful examination of my user interface, I notice this on the right-hand side of my menu bar:

typinator-secure

The Typinator icon in the menu bar normally looks like this:

typinator-normal

What does this alternate icon with the dots mean? Well, a click on the Typinator menu provides the answer:

typinator-explanation

The dots indicate that OS X is stuck in Secure Keyboard Mode (SKM). Normally, the switching in and out of SKM is handled automatically by OS X. As soon as you enter your password and press Return to exit the password field, OS X switches out of SKM. But sometimes it does not. As the Typinator dialog box explains, the normal trick to fix this situation is to quit the parent app of the password field that switched SKM on in the first place.

Typinator tries to help by telling you who the culprit likely is. In this particular case, it tells me it’s Safari.

The problem is that, after I quit Safari, the problem does not go away. So I try quitting other likely culprits, including Firefox, 1Password, etc. Still nothing. I try quitting and relaunching Typinator itself, to no avail. Nothing helps but a complete restart of the machine, which clears the problem.

After this happened to me a few times over the course of a couple of weeks, I decided that I had to try and do something about it. So I contacted the tech support service at Ergonis, the software company that produces Typinator. They were quite helpful, but still insisted that there was some other app causing this problem. I explained that I had not been able to identify a culprit.

Typically, I would wake my machine in the morning, start working, and then notice that something was wrong when I tried to use one of my Keyboard Maestro macros or Typinator features. The different icon that Typinator uses in the menu bar is not that noticeable, and I was not in the habit of keeping an eye on it constantly to determine exactly when it would change.

Since Ergonis thought that 1Password was a prime candidate, they even contacted the Agile Software developers themselves in order to discuss the situation. They also ended up adding a bit of code to Typinator for me that would cause it to play a special sound whenever SKM was turned on, and a different sound whenever it was turned off.

This worked great, in that it clearly indicated to me whenever SKM was turned on and then back off. But it also made me realize that there was one other context where SKM was turned on that I had not thought of: the login window when I woke my computer from sleep. (It’s configured to ask for my password after it’s been put to sleep.)

When I woke my machine in the morning, the special sound would play immediately. And then as soon as I entered my password, the sound indicating that SKM was off would play as well.

Except that sometimes it did not. Sometimes, on my machine at least, after I enter the password requested by OS X upon waking the machine, and OS X logs me in, it fails to switch back out of SKM. And sure enough, if I go to the Typinator alert about SKM right after this happens, the dialog actually correctly identifies the background process called loginwindow as the culprit. For some reason, if I don’t check the dialog right away, Typinator ends up losing track of who the culprit app is, and becomes unhelpful by blaming another app that has nothing to do with it.

Of course, I cannot remember exactly when the problem started, but it’s definitely been in the past couple of months, so I suspect one of the more recent OS X updates.

Now, the problem is that, unlike standalone apps like Safari, 1Password, etc., you cannot quit the background process called loginwindow. It’s constantly running once you log in, and the only way to force it to restart is to restart the entire computer (or possibly log out completely and then back in, which is almost as time-consuming).

I thought about it for a moment and then wondered if maybe, since it was obviously a glitch in loginwindow, I could force it to snap out of it by going through the login window again. I tried using the Fast User Switching menu to switch to another user environment and back to my regular user environment in OS X, but that didn’t help.

Finally, I simply put the computer back to sleep and then woke it up again. It asked me for my password, and voilà: as soon as I entered my password, it logged me back in and this time it switched out of SKM.

So I had, if not a fix, at least a workaround! I gleefully fired an email to Ergonis outlining my findings and thanking them for helping me identify the source of the problem with the addition of the special sounds (which I will definitely keep on). And I confirmed that 1Password was not involved at all in the problem, that it was clearly a problem with OS X itself.

Since then (that was several weeks ago), I have only experienced the problem a couple of times, and the workaround worked each time.

This morning, however, after waking my computer, I experienced the glitch again. I put my display back to sleep (using a hot corner) and woke it back up, but this time it didn’t play the special sound that indicates that SKM is on. I entered my password and was able to log back in, but SKM stayed on. (I also noticed a couple of other glitches in the user interface, but I don’t know if they were related at all.)

I tried again, but this time putting the entire system to sleep (using the Apple menu).

But when I tried to wake the system up, the screen stayed dark, and I saw a few tiny streaks of dancing pixels across it, as well as the ominous Spinning Beach Ball of Death. This time, I had obviously hit a more severe manifestation of the same bug, and I had no choice but to do a hard reboot of the machine, cursing Apple for their inability to produce glitch-free software and wasting my time with length reboots. (Even with an SSD startup volume, it takes time to relaunch everything and especially to reload every open web page off the Internet.)

I am afraid that this is where I am at now. The bug with SKM is still there, unaddressed. There’s little point in submitting a bug report to Apple, as the problem is very intermittent, and there is no 100% reliable way of reproducing it. It just happens from time to time.

Hopefully, if it happens again in the future, the workaround described above will continue to work and will enable me to avoid having to reboot. But, as I have just experienced this morning, there might be something more sinister at work here, a more serious bug of which the SKM glitch is just the milder manifestation.

I will probably upgrade to Yosemite in the coming months, and we’ll see if the problem still exists in the new OS. The best that I can hope for at this point is that Apple has somehow accidentally, unwittingly fixed it in Yosemite by updating the loginwindow process code. Who knows?


Fixing ‘Find Previous’ in Adobe InDesign CC 2014

Posted by Pierre Igot in: Macintosh
July 22nd, 2014 • 8:51 am

Following my blog post from last week (‘Find Previous’ in Adobe InDesign), I actually got some feedback from Vipul-Bansal, an InDesign engineer, on the Adobe Forums:

We had tried really hard to just provide two buttons “Find Previous” and “Find Next” but there were quiet [sic] a few design limitations because of which we can not implement it.

1) We have introduced a new search scope “To Beginning of Story” and we wanted the search scope drop down to populate on the basis of “Search Direction” i.e. we wanted to have either “To End of Story” or “To Beginning of Story” depending upon the whether the user wants to search in “Forward” or “Backward” direction. So it becomes mandatory for the user to select one particular direction and then perform the search operation.

2) Similar to the above reason it also becomes mandatory for the user to “select a direction” before performing “Change/Find” operation. And it would have been impossible if we would have provided two diff. buttons for “Find”

It has been mentioned in the shared blog post that there is no shortcut to change the Search Direction, but we have implemented it. You can use the default shortcut : Ctrl+Alt+Enter(for Win) or Cmd+Opt+return(for MAC) or you can also edit the same by going to Keyboard Shortcut>Edit Menu(Product Area in the KBSC dialog)>Toggle search Direction.

I won’t discuss the validity of the justifications given here. But it is true that InDesign CC 2014 also includes a command called “Toggle search direction” in the “Edit” menu section of the “Keyboard Shortcuts” dialog box:

indesign-cc2014-shortcuts-togglesearchdirection

I’ve assigned the command-shift-option-G shortcut to it. (It’s already assigned to a command, but it’s a baseline grid-related command that I rarely use.)

With command-G assigned to the “Find Next” command, I now use the following Keyboard Maestro macro:

km-adobe-indesign-find-previous

Et voilà! This macro effectively switches the search direction, jumps to the previous occurrence, and then switches the search direction again. It’s simple, but it pretty much works as a way to make InDesign behave like a normal OS X application when it comes to “Find Next” and “Find Previous” keyboard shortcuts.


‘Find Previous’ in Adobe InDesign

Posted by Pierre Igot in: Macintosh
July 16th, 2014 • 8:56 am

Here’s a real-world scenario. (It’s very real in my world anyway.)

I have a 200-page document in InDesign with lots of text in various frames. I need to review all occurrences of the word “department” in this document and change some of them — but only some — to “Department”.

I cannot use a batch “Change All” command for this, even with a fancy grep pattern, because the reasons for capitalizing some of the occurrences (but not all) are too complex. I really need to review each and every occurrence individually to decide whether to capitalize the word or not.

Before InDesign CC 2014, the “Find/Change” window in InDesign looked like this:

indesign-replace-before2014

I would start at the beginning of the document and start clicking on the “Find” button. I would examine the occurrence and, if it needed to be capitalized, I would simply click on the “Change/Find” button. This would change that particular occurrence and jump to the next one. If the occurrence didn’t need to be capitalized, I would just click on the “Find” button again (which now actually read “Find Next”).

As you can imagine, when a document contains dozens of occurrences of the same word and you have to review them all and only change some of them, the task soon becomes tedious. If the occurrences that actually need to be changed are few and far between, you naturally start clicking on “Find Next” semi-automatically, and inevitably you end up clicking too soon at times, and only realize that the found occurrence did need to be changed after you’ve clicked on the “Find Next” button and already jumped to the next one.

So what do you do then? Well, prior to InDesign CC 2014, there was no option to reverse course and jump back to the previous occurrence. The only solution was as follows:

  1. leaving the “Find/Change” window open, click back on the document window;
  2. make a generous assumption about how many pages behind the previous occurrence of the found text was, and browse back by that number of pages;
  3. click somewhere in a text frame on the page to move the cursor back on that page;
  4. switch back to the “Find/Change” window;
  5. start clicking on “Find” or “Find Next” again, going through a whole bunch of occurrences that you have already reviewed, simply because your generous assumption was too generous and you went back too far;
  6. try to recognize the found occurrence that you actually skipped over so that you won’t skip over it again;
  7. change it properly this time by clicking on “Change/Find” rather than “Find Next”;
  8. resume your review of the next occurrences.

I don’t need to tell you that this technique was completely ridiculous and highly unreliable. And yet, this was the reality in InDesign up until about a month ago, when Adobe finally released a version of InDesign (the so-called InDesign CC 2014) that featured some kind of “Find Previous” command:

indesign-replace-after2014

Now, as you can see, there still isn’t a “Find Previous” button in there. Instead, there is a new section labelled “Direction” with two radio buttons, for changing the direction of the search. If you click on the “Backward” radio button, the “Find Next” button changes to “Find Previous”:

indesign-replace-after2014rev.png

In other words, in the scenario outlined above, if you accidentally click one time too many and need to reverse course to jump to the previous occurrence, you can now do this:

  1. click on the “Backward” radio button to change searching direction;
  2. click on the button now labelled “Find Previous” to jump to the previous occurrence;
  3. change the occurrence properly by clicking on “Change” — Don’t click on “Change/Find”, as this will change the found occurrence, but it will also continue the search backwards and will take you to the next previous occurrence, which you have already reviewed!
  4. click on the “Forward” radio button to change searching direction again;
  5. click on the button now labelled “Find Next” again to resume your review of the next occurrences.

I don’t know about you, but I only find this marginally less ridiculous than the situation prior to InDesign CC 2014. It still involves far too many steps. It is still far too clumsy for the user to be able to develop any kind of rhythm. Yes, it is better than the situation before, but it is still far from ideal.

Why on earth did Adobe not follow the example of all the other software applications out there that have settled on the most obvious and most intuitive solution, which is simply to also include a “Find Previous” button in their dialogs that is available at all times? Who the hell gave Adobe’s engineers the idea that asking the user to manually change search direction every time he or she needs to track back was the best option?

It gets worse.

Now imagine that, instead of having to change some occurrences of “department” to “Department”, you actually have to review all occurrences of a specific text string and make edits in some of the occurrences that cannot be done through a simple “Change” operation. This is, again, a perfectly realistic scenario in my world. Natural language is such that, sometimes, edits can only be done manually on a case-by-case basis. Even a fancy grep pattern (assuming you’ve mastered the art of composing grep patterns) will not do.

In such a scenario, in order to work as efficiently as possible, you want to be able to keep your hands on the keyboard, instead of having to constantly switch back and forth between the mouse (to control the buttons in the “Find/Change” window) and the keyboard (to make the actual edits).

So what’s the natural solution here? Well, you bring up the “Find/Change” window, you enter your search string, and you start your search with the mouse. But then you close (or relegate to the background) the “Find/Change” window and you start jumping from occurrence to occurrence in the document itself with a keyboard shortcut, namely command-G, the standard keyboard shortcut for “Find Next”.

This does work in InDesign. But what if, once again, after a while the task of reviewing all the found occurrences starts becoming tedious and you accidentally press command-G one time too many, skipping an occurrence that you actually needed to edit?

Well, in versions of InDesign prior to CC 2014, there was no “Find Previous” option whatsoever, so you were stuck with having to follow the steps I described above, i.e. make a generous assumption about how many pages behind the previous occurrence of the found text was and browse back by that number of pages, and so on and so forth.

But now that we do have a “Find Previous” command of sorts in InDesign CC 2014, surely there is an easier way? I am afraid not. The only way to access this “Find Previous” command is through the graphic interface, with the “Direction” section in the new and improved “Find/Change” window. There is simply no option to do this with the keyboard.There is no “Find Previous” command in the list of commands under “Edit › Keyboard Shortcuts…” that you might be able to assign a keyboard shortcut to.

Well, let me correct this. There is a “Find Previous” command in the “Keyboard Shorcuts” dialog box:

indesign-cc2014-shortcuts

But it’s actually the same entry as the “Find Next” command! Yes, if you leave the “Find/Change” window in InDesign in “Backward” mode, then in the “Keyboard Shorcuts” dialog box, the “Find Next” command is listed as “Find Previous”.

Needless to say, there’s no point in trying to assign a different keyboard shortcut to it, like, say, shift-command-G, which is the standard shortcut for “Find Previous” in most other OS X apps.

First of all, the shift-command-G shortcut is already assigned to the “Ungroup” command, and given Adobe’s long history of enforcing non-standard keyboard shortcuts in OS X (try typing a non-breaking space in InDesign!), I doubt very much that Adobe will ever reconsider and assign a different shortcut to the “Ungroup” command.

Most important, however, whatever shortcut you might assign to “Find Previous” in the “Keyboard Shortcuts” dialog will also be assigned to “Find Next”, since it is, in effect, the same command. It won’t help you avoid having to switch from the keyboard back to the mouse in order to change the direction of your search when you want to backtrack.

At this point, as far as I can tell, there is no option to assign a keyboard shortcut to the command for changing the search direction itself.

The way things are going, I wouldn’t be surprised if Adobe, in response to feedback about this new feature like mine (should they choose to respond), decided to simply add the option to assign a keyboard shortcut to the command for changing the search direction itself. It would still be ridiculously complicated, but at this point, I have pretty much given up on Adobe ever understanding what users in the real world face with real-world scenarios such as the ones described here actually need from InDesign — which is what most other text editors offer, i.e. a simple “Find Previous” command accessible at all times using a simple, direct keyboard shortcut, preferrably the standard shift-command-G.

Apparently, Adobe’s engineers suffer from the “Not Invented Here” syndrome and have decided that they have invented a better solution. Ahem.