Flat tireMany open source projects follow the “do not reinvent the wheel” philosophy. It described the idea of using something that is existing, something proven, like the wheel we use for centuries. In open source, where work is not paid this free artifacts are appreciated, but also in commercial software people like to get something for free. I do as well.
But what if your wheel has a flat tire?

symfony uses Prado framwork for its i18n support. Prado itself uses standardized localization (CLDR) data. But the original author at Prado chose to convert the CLDR data into a custom format. Here the trouble starts.
The Prado proprietary data format was not updated since 3 years. I tried to contact the Prado folks three times, but never got an reply. In meanwhile the tickets in the symfony trac regarding I18N data being outdated started to pile up.
So in order to update symfony I18N data I had to reverse engineer the format used in Prado. It is a serialized PHP array, but I could not find how the CLDR data was converted. After searching for a while I found that Prado claimed to have imported the data from ICU.
Because the import script originally used is unknown for me I had a look at the ICU data.
Hmm yet another proprietary format… Hey cool, it is very close to YAML!
So I wrote a little script to convert proprietary ICU format into proprietary Prado format by first making a valid YAML out of it using symfony sfYaml and then postprocessing and serializing the array. During the process I found quite some mess in both, the ICU and Prado data. But because I do not want to rewrite everything around the data files, I just imported the mess and keep it messy compatible.

The morale of this tale is twofold:

  • Stick to Open Data Formats – Less Hassle for everybody
  • Open Source can have a short life – If you release something open source, be ready to maintain it. If you use something open source, be ready to maintain it yourself.

Now I pass the ball to all symfony users and developers: Please test this data files. Due to the vast changes from CLDR (2Megabytes of new data) and the conversion process I cannot assure or prove that they are compatible. So they most likely will appear earliest in symfony 1.3. And perhaps in Prado if they decide to take them as well (and hopefully give credit).

There are two packages:

  1. The full package, including the ICU original files from this week, the intermediate yml results and the finalized symfony/Prado dat files.
  2. Just the symfony/prado data dir files. You should be able to find the lib/i18n/data directory inside you symfony library path.

Please report issues preferrably to my symfony-project.com email account, the symfony mailing list or here. I cannot support Prado users, but as the i18n hasn’t changed at Prado for 3 years it most likely will work there as well.

Update:

I just commited the new I18N data for symfony 1.3. I was able to optimize the resulting files a bit towards symfony, so I could remove some extra postprocessing. This required some changes in symfony, but I assume that they are backwards compatible for 95% of the users. These changes also made the files incompatible with older symfony versions, but I anyway did not want to risk breaking stable applications.

The latest conversion script and data files are available in symfony svn.