How can you be sure that speakers of different languages will find the information they need once you translate the labels that make up your website navigation?

Will French speakers be able to find the same information as easily as English speakers? What if the ‘standard’ translation of a label in English has a different connotation in Hungarian? Will that have an impact on how easy it is to find a piece of information?

These are the kinds of questions we've been grappling with in coming up with the navigation labels which signpost users to information on the new Europa website.

The best way to find the answers to these questions? Testing with real users.

Tree testing

In 2014, we ran several rounds of tests on the proposed labels in English to make sure at least 80% of users would click on the labels we expected, a process called 'tree testing'.

The principle of tree testing is quite simple. Each user is instructed to carry out a task, for example: 'Find out how the EU helps small companies get loans'.

We then present the user with a list of proposed navigation labels, and ask them to click the label they think will take them to the information.

When preparing a tree test, you need to keep some rules in mind:

  • Avoid leading instructions. If an instruction contains a word that appears in any of the navigation labels, users will be more likely to click on that label (whether that’s the correct destination or not).

  • Labels can be changed in a subsequent iteration. Instructions cannot. Instructions must be exactly the same in all iterations, so you can compare results.

We carried out this exercise for 31 instructions, each representing important tasks people might want to perform on our website.

For each instruction, we then analysed where people clicked and how frequently. The results served as our baseline for many other tests.


In this example, out of 155 people who responded, 142 clicked on one of the expected labels: 'Business, Economy'; 'Funding, Tenders'; 'EU regional and urban investment'. That’s a 91.6% success rate!

From English to 23 languages

The English labels that form the top-level classification for the new Commission site were developed over several months, involving many meetings with the DGs and 3 rounds of user-testing with hundreds of participants. After each test, the labels were reformulated until we achieved success rates of 80% or greater for all instructions, across all types of users.

The next step was to translate the instructions and labels into the other 23 official EU languages, the goal being to achieve that same success rate of at least 80% in all languages.

This was a challenging exercise:

  • a 1 or 2-word label has many implicit words and concepts behind it: therefore a ‘good’ translation might be a standard bilingual-dictionary-type translation, a free interpretation or something in between.

  • the labels have been tested in all 24 EU languages, but we won’t be able to do multiple rounds of testing for all languages: therefore our best chance of reaching 80% success was to stick as closely as possible to the English originals, which are proven to work.

  • we have a general, but not yet exact, idea of what type of content will go behind each label: therefore if translations are more specific than the original English label, there is a risk they might no longer be ‘correct’ once content is added.

Problem labels

Most translations were very close to their English baseline. Where there were significant changes, we tried to find a compromise – keeping the meaning but avoiding extra words wherever possible.  

Some labels posed consistent problems across many languages; for example, the word 'Strategy'. Many translators found using the equivalent of 'Strategy' as a standalone label difficult in their language, because its meaning is broader than in English or because it sounds like a planning document.

Some translators added adjectives to 'Strategy' in order to make it clearer: 'Political Strategy', 'Strategic Priorities', 'Internal and External Strategies', etc. All of which have slightly different connotations that the English original does not.

'Strategy' is just one example. Each language is unique, and other labels created the same kinds of translation issues. In some cases, the final version was a compromise, as you can see below.

Multilingual tree test

After agreeing on the labels and instructions for each language, we put the translations to the test to see how they performed with users – to know if people could successfully find the information.

We created a tree-testing survey, containing all these navigation translations and sent the test to participants across the world.

Stay tuned for our next blog post about our analysis and results.

Related disciplines: 


Comments

Nikki Labrum's picture

It's really useful that you're documenting some of the challenges faced when developing websites for a multi-lingual audience. I look forward to the next blog post!

Beverley Pold's picture

Thank you for sharing your research and introducing me to "tree testing". It has also given me an insight into how the EU endeavours to be fully inclusive across so many languages

Beverley Pold's picture

Thank you for sharing your research and introducing me to "tree testing". It has also given me an insight into how the EU endeavours to be fully inclusive across so many languages

P. Vandenberghe's picture

Hello, a question to the expert panel : how would you evaluate https://blogs.ec.europa.eu/eupolicylab/#projects where the titles of the projects (in one language) are hidden under pictures of flowers (a universal language of course)? Is this a good approach, is it inclusive and in accordance to AnySurfer principles? Or in compliance with demands for transparency?

Add new comment