Skip to content

Document iconv implementation quirks #5646

Description

@orlitzky

tl;dr document some if the problems I ran into in php/php-src#22413

PHP's iconv() function mostly defers to the system's iconv_open() and iconv() routines. POSIX 2024 covers these:

As you can see, there are three magic suffixes that can be appended to the "to" charset:

  • //IGNORE
  • //TRANSLIT
  • //NON_IDENTICAL_DISCARD

In practice, support for these varies widely, and when they are supported they may not conform to POSIX. For example, musl does not support //IGNORE, and (man 3 iconv) BSD/Solaris replace unknown translations with ? rather than raise an error that would require //TRANSLIT... so they do not have //TRANSLIT.

The details are sordid, and we should not try to capture them in the PHP documentation. Instead, we should simply state that support for any iconv() feature can vary widely depending on your implementation. Right now the PHP iconv docs mention //TRANSLIT and //IGNORE, but say only that //TRANSLIT is unreliable.

  1. We should also mention that //IGNORE is implementation-dependent. PHP has a workaround for iconv implementations (glibc) that return an error upon ignoring an invalid sequence, which helps standardize PHP's behavior, but //IGNORE may not be supported at all (musl), and there is no workaround for that.
  2. //NON_IDENTICAL_DISCARD needs a mention.
  3. The "to" and "from" charsets are also implementation-defined. Musl for example only supports a few, and will fall back to UTF-8 if you pass in something it doesn't understand.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions