Localizing PHP web sites using gettext

Developing multi language web sites using PHP is actually very easy. A common approach is having an include file for every supported language which contains an array that maps string ids to localized text (for example “WelcomeText” => “Welcome to our homepage.” would be included using something like <?= $strings["WelcomeText"] >). However there are several problems with this approach. First of all, when the application is updated and additional strings are added, there is no way to determine which new strings were added and if they are present in every language (unless you write a script for it). What happens if a newly added string is not yet translated into a specific language?

Using gettext with PHP

A widely used framework for internationalization is gettext. It can be used with a variety of programming languages, including PHP. There are basically two ways to use gettext in your PHP applications. You can use the native gettext PHP extension or you can use a library implemented in PHP that does not need any extension, such as php-gettext. I will use the native PHP extension, but once you have read the post you should be able to use the php-gettext-library, too (have a look at the included example).

Using gettext in your application

Using gettext to get translated strings couldn’t be easier. Just call gettext(“Text to be translated”) and you get a localized version of “Text to be translated” if available, or “Text to be translated” otherwise. If you’re lazy, you can use _() instead of gettext().
Let’s try this out. Create a new PHP file (we’ll call it test.php), and insert the following code:

[php]< ?php
echo _("Hello World!");
?>[/php]

When you open that page in your browser, you will see “Hello World!”.

Localizing your application

Now that you have created a first version of your script, you may want to create localized versions for different languages. In order to do that, you either need the gettext utilities (windows version) or the graphical editor poEdit, which we will be using. So install and launch poEdit. Create a new catalog (File -> new catalog). Choose the language you want to translate the application to. I’m going to use German, GERMANY and iso-8859-1 for both character sets. The last option (plural forms) is an advanced feature of gettext which I’m going to explain later, so for now, just leave it blank. On the paths tab, set the base path to the directory containing your test.php file, and add “.” as a path. On the keywords tab you can add names of additional functions that call gettext. You may want to use this if you’re using php-gettext. Now click OK to open the save dialog. Now create a sub folder called “locale” in your script directory. Create a subdirectory inside locale for every language you support. We’ll create one for “de_DE” (the first part is the language, the second part is the country). Inside that folder create another one called “LC_MESSAGES”. You should now have a directory structure like locale/de_DE/LC_MESSAGES/. Save the file inside that directory as “messages.po”. poEdit will now automatically scan all source files inside the path you specified earlier and extract all strings that are passed to gettext() or _() (or any other methods you may have added in the keywords tab). Click OK. Now that’s cool! poEdit just extracted all strings you want to be localized.

In the upper half of the poEdit window, you have a list of strings (the original string on the left and your translation on the right). Select the first string in the list. In the lower left hand corner, you have 2 text boxes. The first one contains the original string, and the second one is still empty. Type your translation into that box. I’ll enter “Hallo Welt!”. If you had more entries in your file, you could navigate between them by pressing ctrl-up and ctrl-down. Save the file. poEdit automatically created “messages.mo” in the same directory as “messages.po”. This is the compiled version that will later be used by PHP.

Initializing the gettext library

Let’s see if our script can now display the localized string. The first thing you have to do is telling the gettext library which locale you want to use and where the language files are stored. Let’s create a new file called “localization.php” that will handle all the gettext initialization. Copy the following code into localization.php (Note: this code is for the php gettext extension, php libraries such as php-gettext may require different initialization):

[php]< ?php
$locale = "de_DE";
if (isSet($_GET["locale"])) $locale = $_GET["locale"];
putenv("LC_ALL=$locale");
setlocale(LC_ALL, $locale);
bindtextdomain("messages", "./locale");
textdomain("messages");
?>[/php]

In the first line, we set the default locale to “de_DE”. The second line allows you to override this by appending ?locale=… to the URL. You should replace this by real code to select the locale (perhaps by looking at the Accept-Language header), but it works for our tutorial. The next two lines actually set the current locale to that value and the bindtextdomain call creates a text domain which will use our messages.mo file in the locale directory. The textdomain function selects that domain as the default domain.

Include that file in your test.php by adding a require_once(“localization.php”); line to the top of the file. Now reload test.php in your browser and it should now say “Hallo Welt!”. Now try test.php?locale=en_US and it should say “Hello World!”. You have just created a multi language PHP script!

Updating your application

Let’s add another string to the script. I’ll add echo _("Welcome to my test page");. When you reload the page with the German locale, you will see that Hello World is localized, while the second string is not. Let’s change that. In poEdit (reopen the .po file if you have closed it), click the update button (that’s the one with the wheel). A new dialog should show one new string. Click OK, translate it (in German it would be “Willkommen auf meiner Testseite”) and save the file. Reload test.php in your browser. Nothing has changed? When using the PHP gettext extension, .mo files are cached by the gettext library. The only way I have found to clear the cache is to restart the web server. Once you have done that, both strings should be translated.

Using plurals

One last thing I want to discuss is gettext’s plural support. This may seem a bit complicated at first, but is actually very powerful. Sometimes you need a string that supports plural forms. A typical example would be “0 comments”, “1 comment”, “2 comments”, “3 comments”… In English you can just add an ‘s’ if the number n != 1. However, in other languages it’s not that easy. gettext supports all these cases. In your scripts you use something like this:

[php]< ?php
$n = 3;
printf(ngettext("%d comment", "%d comments", $n), $n);
?>[/php]

If you have never used (s)printf before, have a look at it’s documentation. The interesting part is of course the ngettext call. ngettext takes three arguments. The first one is the singular string, the second one the plural string and the last one is the number. While that was quite easy, localizing these strings is a bit harder. Add the two lines to your test.php script.

Before you update your language file in poEdit, open the catalog options window (catalog -> options). Now let’s fill in that last field. For our German locale, set the plural forms field to “nplurals=2; plural=(n != 1);”. Essentially this specifies that there are two plural forms and the plural form is determined by the expression (n != 1). This evaluates to 0 if n == 1 and 1 if n != 1. For more information about this field, including samples for various languages, please see the gettext documentation. Now, let’s press the update button again and select the new string. You will notice that the lower left hand corner of the window has changed. It now displays the original singular and plural forms and the translation box has been replaced by two (depends on the number of plural forms the language has). Enter “%d Kommentar” in the first tab and “%d Kommentare” in the second tab. Save the file, reload the web server if needed and your page should show the correct translation (you may want to change the value of $n to see the effect of the ngettext function).

Improving localization in your scripts

This concludes my introduction to localizing PHP web sites using gettext. Here are a few ideas to improve localization in your applications:

  • During development, you may want to use a php library instead of the PHP gettext extension because they do not require you to restart the web server every time you modify the language files. You can create a wrapper function (e.g. __()) that calls either the library function or the extension, depending on which is available or selected). You can add the name of the wrapper function as a keyword in poEdit, so that it is recognized.
  • You may want to select a default locale according to the browser’s Accept-Language header, so that the user does not have to select his language first.

The performance of gettext

See my follow-up post “Benchmarking PHP Localization – Is gettext fast enough?” for Benchmarks. In general, the gettext Extension is faster than using a String-Array. The pure PHP implementation of gettext is slower and not recommended if you can use the PHP Extension.

18 Responses to “Localizing PHP web sites using gettext”

  1. » Про мульти?зычно?ть « Заметки веб-разработчика « ZOOB Says:

    [...] Localizing PHP web sites using gettext [...]

  2. skausl Says:

    How is the performance of gettext compared to using string ids?

  3. Pablo Hoch Says:

    skausl: That’s an interesting question. I’ll benchmark it and post the results here.

  4. Benchmarking PHP Localization - Is gettext fast enough? » Pablo’s Development Blog Says:

    [...] Last year, I wrote a post about using gettext to localize PHP web pages. Gettext makes it easy to maintain the translations and always provides a fallback locale. But is it fast? [...]

  5. paperogiallo Says:

    Your step-by-step tutorial is the best on the Net: plain, easy to understand (also for my poor english!) and very authoritative. So, very thanks! :-)

  6. Matt Says:

    Hi,

    I’m making my first localized site and having some problems. You seem to have a firm understanding of gettext with PHP so I thought maybe you could help.

    I developed this site locally on my Mac with PHP 4.4.4 and Apache 1.3.3 installed and after a lot of researching and experimenting, I got the translations to work on the local copy of my site. Not launch time has come and I uploaded the files (minus the compiled .mo file) to the server which is running Apache 2.x on a RedHat box and PHP 4.4.4 that was compiled with gettext, I compiled the .mo file via ssh directly on the server and asked the server’s owner to restart the server, but the gettext isn’t working. He tried graceful and hard restarting of the server. Any thoughts?

    Matt

  7. Les Says:

    The only problem I have with your (informitive) article is that I’m confused by trying to tell the difference between the PHP extension, and the other library you mentioned? Which one you refer to through out your article isn’t clear.

    I am interested in looking at this approach but before I can tell which approach is best, you would have to make the article [a call for an update maybe?] is more clearer in regards to either using the extension, or the library you refer to in your article.

    Otherwise, it’s a good article nonetheless.

  8. Jeroen Says:

    I cannot reproduce the caching problem on our development machine or our (shared) web host running PHP 5 (mod-php on the dev box, CGI at our host). So the webserver does NOT have to be restarted when .mo files change, which means gettext is a valid alternative for shared hosting environments.

  9. B. Peter Says:

    Thank you for your great article.

    I use Debian linux, apache 2 and php 5. I had to change only one line in your script:

    putenv(“LANGUAGE=$locale”);

    Peter

  10. Jaime G. Says:

    Great article. I’m not a programmer, just a writer, and belong to a multilingual community. I got as far as editing my strings in poedit, however after that I wasn’t able to do anymore. As I mentioned before, I’m not a programmer, and don’t know what is going on in the Php code, I only know simple Html. Is there anyway I can make my site multilingual, without having to know Php? I think I didn’t “include” the “localization.php” file correctly. If you know any resource for a non programmer such as myself, I’d appreciate it a lot thank you.

    Jaime

  11. Damian Says:

    The correct URL for poEdit’s website is http://www.poedit.net.

  12. Scott Taylor Says:

    (second attempt at posting this, first one got tags stripped out)
    Yes, Jaime, you can do multilingual web sites without being a programmer. Think of PHP as tags you can embed in regular html that get special processing on the server before they are sent to the user.

    First, you need to make sure your web sever supports PHP and gettext. Odds are pretty good that it does, but you need to check. Make a new file and paste this into it:

    <?php phpinfo();?>

    Save the file as “phpinfo.php”. Transfer the file to your web server, and then load it in a browser. If you see a long page of settings, including one that says “GetText Support – enabled” you are set up for localization! If not, you are stuck, and you’ll need to talk to a programmer.

    Second, you need a localized web page.
    Pablo’s excellent tutorial above covers how to tag the English phrases in your page. Make sure you save your web pages as .php instead of .html. Here’s an example web page, save it as “test.php”.

    <?php require_once “localization.php”;?>
    <html><head></head>
    <body>
    <a href=”?locale=en_US”>English</a> |
    <a href=”?locale=es_ES”>Spanish</a> |
    <a href=”?locale=de_DE”>German</a>
    <br>
    <?php echo _(“Hello World!”);?><br>
    <p><?php echo _(“My name is”);?> Bob.</p>
    </body>
    </html>

    Third, you need to use Poedit to scan your web pages, enter the translations, and save the “messages.mo” file. In his tutorial above, Pablo shows you how to create a German translation, but you can substitute English (en_US), Spanish (es_ES), Chinese (zh_CH), French (fr_FR), Japanese (ja_JP), etc. In Poedit I’d suggest choosing “utf-8″ as your charset, rather than the “iso-8859-1″ used in the tutorial, utf-8 supports more languages.

    Fourth, you need a “localization.php” file. Create a new file called “localization.php”. Copy this text into it:

    <?php
    $locale = false;
    if (isSet($_GET["locale"])){
    $locale = $_GET["locale"];
    setcookie(“locale”, $locale, time()+60*60*24*30, “/”);// save a cookie
    }
    if (!$locale && isSet($_COOKIE["locale"])){
    $locale = $_COOKIE["locale"];
    }
    putenv(“LC_ALL=$locale”);//needed on some systems
    putenv(“LANGUAGE=$locale”);//needed on some systems
    setlocale(LC_ALL, $locale);
    bindtextdomain(“messages”, “./locale”);
    bind_textdomain_codeset(“messages”, ‘UTF-8′);
    textdomain(“messages”);
    ?>

    Pablo gives a similar example above, but this one is a slight improvement, it saves a cookie so you only have to have to set your locale once. You can also detect the user’s preferred language automatically, but that’s a lot more complex.

    Finally, transfer your test web pages, the localization.php file, and your entire “locales” folder to your web server. Bring up the “test.php” page in your browser. Everything should show up in English. Now click on the language link. If you created a translation for that language, the page should show up translated!

    If this does not work, it’s time to go get a programmer.

  13. Scott Taylor Says:

    (third attempt to post this, second one got smart-quoted, which messes up the code sections)
    Yes, Jaime, you can do multilingual web sites without being a programmer. Think of PHP as tags you can embed in regular html that get special processing on the server before they are sent to the user.

    First, you need to make sure your web sever supports PHP and gettext. Odds are pretty good that it does, but you need to check. Make a new file and paste this into it:

    <?php phpinfo();?>

    Save the file as "phpinfo.php". Transfer the file to your web server, and then load it in a browser. If you see a long page of settings, including one that says "GetText Support – enabled" you are set up for localization! If not, you are stuck, and you’ll need to talk to a programmer.

    Second, you need a localized web page.
    Pablo’s excellent tutorial above covers how to tag the English phrases in your page. Make sure you save your web pages as .php instead of .html. Here’s an example web page, save it as "test.php".

    <?php require_once "localization.php";?>
    <html><head></head>
    <body>
    <a href="?locale=en_US">English</a> |
    <a href="?locale=es_ES">Spanish</a> |
    <a href="?locale=de_DE">German</a>
    <br>
    <?php echo _("Hello World!");?><br>
    <p><?php echo _("My name is");?> Bob.</p>
    </body>
    </html>

    Third, you need to use Poedit to scan your web pages, enter the translations, and save the "messages.mo" file. In his tutorial above, Pablo shows you how to create a German translation, but you can substitute English (en_US), Spanish (es_ES), Chinese (zh_CH), French (fr_FR), Japanese (ja_JP), etc. In Poedit I’d suggest choosing "utf-8" as your charset, rather than the "iso-8859-1" used in the tutorial, utf-8 supports more languages.

    Fourth, you need a "localization.php" file. Create a new file called "localization.php". Copy this text into it:

    <?php
    $locale = false;
    if (isSet($_GET["locale"])){
    $locale = $_GET["locale"];
    setcookie("locale", $locale, time()+60*60*24*30, "/");// save a cookie
    }
    if (!$locale && isSet($_COOKIE["locale"])){
    $locale = $_COOKIE["locale"];
    }
    putenv("LC_ALL=$locale");//needed on some systems
    putenv("LANGUAGE=$locale");//needed on some systems
    setlocale(LC_ALL, $locale);
    bindtextdomain("messages", "./locale");
    bind_textdomain_codeset("messages", "UTF-8");
    textdomain("messages");
    ?>

    Pablo gives a similar example above, but this one is a slight improvement, it saves a cookie so you only have to have to set your locale once. You can also detect the user’s preferred language automatically, but that’s a lot more complex.

    Finally, transfer your test web pages, the localization.php file, and your entire "locales" folder to your web server. Bring up the "test.php" page in your browser. Everything should show up in English. Now click on the language link. If you created a translation for that language, the page should show up translated!

    If this does not work, it’s time to go get a programmer.

  14. Alex Hempton-Smith Says:

    I’m working on a project at the moment that will have modules which ‘plug into’ the core admin panel.
    How would I provide more than one translation *.mo file? One for the core and one for each of the modules.

    Thanks for a great article by the way :)

  15. Till Says:

    Must agree to Alex.
    Has anyone a solution to that? (… which ‘plug into’ the core …)

  16. rado Says:

    seeking for same thing :/

  17. Lars Says:

    I will have this “plugin problem” too – and very soon.

    Thinking about it, I guess, one could translate the plugin via the usual way with e.g. poEdit – but use another textdomain for it.
    If your plugin is called “plugin1″, you should be able to do this:

    bindtextdomain(“messages”, “./locale”);
    bind_textdomain_codeset(“messages”, “UTF-8″);
    bindtextdomain(“plugin1″, “./locale”);
    bind_textdomain_codeset(“plugin1″, “UTF-8″);
    textdomain(“messages”);

    This way you need the .po and .mo files calles plugin1.mo and plugin1.po in your locale directory.

    To actually use these translations you would use dgettext which overrides your textdomain for just its call. Example:

    echo _(“Normal output”);
    echo dgettext(“plugin1″, “Plugin1 output”);

    Hope this helps…

  18. links for 2008-04-25 | svenkubiak.de Says:

    [...] Localizing PHP web sites using gettext A widely used framework for internationalization is gettext. It can be used with a variety of programming languages, including PHP. There are basically two ways to use gettext in your PHP applications. You can use the native gettext PHP extension or you c (tags: php gettext localization tutorial web) [...]