I think what happens here is that the BOM doesn’t seem to bother PHP for root level files, but it bothers PHP if it’s included in the “header” (beginning) of an “include” file, because obviously an include file is actually inserted inside an existing file when building the dynamic pages. So we end up with a page that contains a BOM somewhere in the middle of it… and that doesn’t work.

However, based on what you guys are saying, I should probably get rid of the BOM everywhere. Intuitively, it doesn’t make much sense to have a BOM for UTF-8. I wonder why it even is an option.

The BOM (Byte Order Mark) is composed of two invisible characters at the very beginning of the file. They’re used to distinguish between big endian and little endian byte order in UTF-16. There is also a BOM for UTF-8, even though there is no byte order issue with it.

When a PHP script uses the header function to send special HTTP headers (like a graphical counter that sends a Content-Type: image/png header), it must happen before any content is produced. Therefore, the php opening tag must be the very first characters in the file. When there’s a BOM, the two invisible characters composing it are treated as content and the header function doesn’t work anymore.

I remember reading that using a BOM with UTF-8 is not recommended, for this reason among others. I agree it’s not clear in BBEdit that UTF-8 no BOM is actually the “standard” encoding and UTF-8 tout court is the special one.

I’ve had BOM problems when coding up PHP with Dreamweaver and utf-8, and found that Topstyle works well for hunting them down and getting rid of them.

Maybe there’s a better way in DW… but this works..

Being a traditional BBEdit ‘disliker’, I’ll take it as another hint at that application’s particular ‘quality’. ;)

Or take it from the horse’s mouth: http://www.unicode.org/faq/utf_bom.html

Pierre: I made a quick test with BOMs and PHP. The problem is indeed with the header() function. If you put anything before the php block in the file (a blank line, some text), PHP fails with the error you mentioned. If the file is a BOMmed UTF-8, PHP treats the BOM as text and fails the same way. It does not seem to like any flavour of UTF-16, though.

<?php

Since none of those octets represent white space, PHP assumes it is the beginning of the content, and sends out the all the default HTTP headers, and begins the content with those bytes. So, when you try to send out additional HTTP headers, the content has already begun, so it?s too late ? you can?t bring back what you?ve already sent.

When I found this out, I read in a forum somewhere, when I was looking for a solution, that this problem either has been, or would be fixed in PHP5. But I can?t find that forum now, so I can?t be certain. I just know that my ISP has an old version, and I was forced to locate an editor for windows that allowed by to choose not to output the BOM.

Here’s an excerpt from BBEdit’s manual which may help to further clarify the issue:

no BOM: When saving Unicode files, you should always include a byte-order mark (BOM) so that the reading application knows what byte order the file?s data is in. For maximum compatibility, the BOM should be used whenever possible. Use one of the ?no BOM? options only if there is a specific reason to do so, such as providing compatibility with software that malfunctions when a BOM is present. (For purposes of recognition when you use this option, the UTF-16 BOM is FEFF, and the UTF-8 BOM is EFBBBF.)

After reading this, I thought, “Okay, sure thing, UTF-8 with a BOM for me, I sure am probably not one of those users with a ‘specific reason’ to use the ‘no BOM’ option.” So far so good, although from the sounds of the comments here I have probably come within a hair’s breadth of running afoul of the BOM/PHP conflict that y’all have been discussing. Maybe I am one of those users with specific reasons to use the no BOM option. :-)

I’ve also discovered that PHP doesn’t like UTF-16. Lord knows why, I naively experimented with encoding some files as UTF-16, which broke rendering of PHP includes in those pages. Not really surprising in retrospect.

As for a PHP script encoded in UTF-16, I see no reason why PHP fails to parse it. It has a BOM, PHP should recognise it and act accordingly. What’s inside a PHP block should be converted to whatever encoding PHP likes to parse and the rest should be output as is.

? Unicode Ribbon Campaign ? No ASCII, anywhere
? <http://ithink.ch/unicode>

I’ve also discovered that PHP doesn’t like UTF-16. Lord knows why, I naively experimented with encoding some files as UTF-16, which broke rendering of PHP includes in those pages. Not really surprising in retrospect.

Simply: because PHP isn’t a Unicode application. To PHP, a script encoded in UTF-16 looks something like this:

(NULL)H(NULL)i(NULL)!

You can just imagine how confusing that would be. PHP will work okay with UTF-8 because ASCII characters have the same octet values in UTF-8 as they do in ISO-89859-1. If you try a sorting function, though, PHP will not see your é character as an accented letter, but as a garbled string of three nonsense characters (much like the UTF-8 BOM). UTF-8 is fine for most purposes in PHP (4), but it’s certainly not fully supported, or even a good idea, to be honest. PHP 5′s default package contains some Unicode-compatible functions, but they don’t cover everything, unfortunately.