Machine Translation

library(tidyverse)
library(deeplr)
library(googleLanguageR)
library(ellmer)

Survey translation has traditionally required human translators working through structured protocols like TRAPD (Translation, Review, Adjudication, Pretesting, Documentation). MT systems have become accurate enough to serve as a first pass or, in some contexts, a full replacement — but they require some care when applied to survey text.

Survey items present specific challenges for MT. They are short, often grammatically incomplete, and highly decontextualised: a response option like “Agree strongly” or an interviewer instruction like “Read out” means nothing without knowing what question it belongs to. Terminology also needs to be consistent across items — the same concept should always be translated the same way throughout a questionnaire. Both of these problems are addressed by providing context to the MT system.

Context

MT systems trained on sentence pairs translate each sentence independently by default. For general text this is usually fine, but for surveys it causes two problems:

• Ambiguity: short items are more ambiguous than full sentences, and the correct translation often depends on the surrounding questionnaire. “How often?” could translate differently depending on whether the preceding item asked about behaviour, attitudes, or frequency of contact.
• Inconsistency: without seeing the full questionnaire, an MT system may translate the same term differently across items — using two different words for the same concept in the same questionnaire.

Providing the surrounding questionnaire as context mitigates both. How you do this depends on the MT system.

MT systems

DeepL

DeepL is among the most widely used MT systems in translation research and produces high-quality output for European languages. It offers three ways to incorporate context.

Automatic document context is the default when you pass multiple sentences at once. DeepL processes surrounding sentences when translating each one, so simply passing several items together improves coherence compared to passing them one at a time.

The context parameter is the cleanest approach for item-by-item translation. You pass the text you want translated as text and the surrounding questionnaire as context. DeepL uses the context to inform the translation but only returns the translation of text — there is no extraction problem.

translate2(
  text = "Generally speaking, would you say that most people can be trusted?",
  target_lang = "NL",
  context = "This item is part of a social attitudes survey.
                 The response options are: Most people can be trusted;
                 Can't be too careful; Don't know.",
  auth_key = Sys.getenv("DEEPL_API_KEY")
)

To translate all items in a questionnaire with their surrounding context:

questionnaire <- tribble(
  ~item_id , ~text                                                                ,
         1 , "How satisfied are you with your life as a whole?"                   ,
         2 , "Generally speaking, would you say that most people can be trusted?" ,
         3 , "How interested would you say you are in politics?"
)

full_context <- paste(questionnaire$text, collapse = "\n")

questionnaire |>
  mutate(
    translation = map_chr(text, \(item) {
      translate2(
        text = item,
        target_lang = "NL",
        context = full_context,
        auth_key = Sys.getenv("DEEPL_API_KEY")
      )
    })
  )

The document endpoint accepts a full .docx or .txt file and translates it as a whole, using the entire document as its own context. This is convenient for full questionnaires, but you receive the full translated document back and need to parse individual items out afterwards. The context parameter approach avoids this and is generally preferable for structured questionnaire data.

Google Translate

Google Translate has no equivalent to DeepL’s context parameter — you cannot explicitly pass surrounding text to inform the translation of a single item. It translates each piece of text you give it without awareness of other items in the questionnaire unless you pass them together.

What it does offer, via the v3 (Advanced) API, is glossary support: you define a set of source→target term pairs and the API enforces those translations consistently across all requests. This addresses the terminology consistency problem directly, even without sentence-level context.

library(googleLanguageR)

# Glossaries are created once via the Cloud Translation API and referenced by ID.
# Here we use an existing glossary and translate a batch of items.
gl_translate(
  string = questionnaire$text,
  target = "nl",
  source = "en",
  glossaryId = "my-survey-glossary",
  key = Sys.getenv("GOOGLE_API_KEY")
)

One limitation: glossary support requires the Cloud Translation v3 API, which is billed differently from the basic v2 API. The googleLanguageR package supports both.

LLMs

LLM-based translation puts the entire instruction in a prompt. The text to translate, the surrounding questionnaire, a list of required term translations, and notes on register or target population all go in the same place. This makes an LLM the most flexible of the three systems described here. The flexibility comes with a cost: a prompt that only says “translate this” supplies no more than a sentence-level MT system has, so it reproduces the same failures. Improving an LLM translation means adding the missing information to the prompt. The sections below first show what a minimal prompt gets wrong, then fix each problem by adding to the prompt.

The examples translate Dutch survey text into English using a local model served through LM Studio and called with the ellmer package. The chat_lmstudio() function opens a connection to the local model, system_prompt sets the standing instruction, and params(temperature = 0) makes repeated runs return the same output.

A bare prompt reproduces sentence-level failures

The minimal prompt translates each piece of text on its own, with no information about the questionnaire it belongs to. That is usually enough for a full sentence, but survey response options are short, and a short Dutch word that is unambiguous inside a sentence can carry more than one meaning on its own. Take the response option “Eens”. On a rating scale it means “agree”, but in isolation it also reads as “once”, and translated without the question it answers it comes back as “once”, the wrong reading for a survey. What would settle the meaning is the question the option belongs to, and that is exactly what the minimal prompt leaves out. Short questions can lose meaning the same way: a follow-up question such as “Hoeveel per dag?” needs the question it follows to know whether it is asking “how many” or “how much”.

Supplying context

Short, decontextualized text loses meaning without its surroundings. A response option needs the question it answers and the other options on the scale; a follow-up question needs the question before it. The chunk below keeps the original bare prompt as translate_without_context, and adds translate_with_context, which takes a piece of text together with its surrounding context and tells the model to read the text in light of it.

translate_without_context <- function(text) {
  chat <- chat_lmstudio(
    model = "google/gemma-4-26b-a4b-qat",
    system_prompt = "Translate the following text from Dutch to English. Output only the translation.",
    params = params(temperature = 0)
  )
  chat$chat(text)
}

translate_with_context <- function(text, context) {
  chat <- chat_lmstudio(
    model = "google/gemma-4-26b-a4b-qat",
    system_prompt = paste0(
      "Translate a single piece of Dutch survey text to English. ",
      "The survey item below is provided only as context to disambiguate it. ",
      "Do not translate the context. Translate only the text the user sends, ",
      "and output only that translation.\n\n",
      context
    ),
    params = params(temperature = 0)
  )
  chat$chat(text)
}

The next chunk runs a set of items that fail in different ways when each piece of text is translated alone. Some response options are ambiguous on their own: “Eens” reads as “once” rather than “agree”, and “Dalen” can be read as the plural of “dal”, “valleys”, rather than the verb “to fall”. Some translate to different English words depending on the item they sit in: “Man” and “Vrouw” should become “Male” and “Female” under a question about sex but “Man” and “Woman” under a question about gender. Some read correctly only as a set: the rungs of an agreement scale or a likelihood scale have to stay parallel to each other, which requires seeing the rest of the scale. And a follow-up question can fail the same way: “Hoeveel per dag?” has to become “How many per day?” after a question about smoking but “How much per day?” after a question about alcohol, because Dutch “hoeveel” does not distinguish count from mass. Response options carry their stem and the full option set as context; a follow-up question carries the question before it. Every piece of text is translated both ways.

options_items <- tribble(
  ~stem                                                                                                     , ~options ,
  "In hoeverre bent u het eens met de volgende stelling?"                                                   ,
  c("Eens", "Oneens")                                                                                       ,
  "Wat is uw geslacht?"                                                                                     ,
  c("Man", "Vrouw")                                                                                         ,
  "Met welk gender identificeert u zich?"                                                                   ,
  c("Man", "Vrouw")                                                                                         ,
  "Wat verwacht u dat uw inkomen de komende 12 maanden zal doen?"                                           ,
  c("Dalen", "Stijgen", "Gelijk blijven")                                                                   ,
  "In hoeverre bent u het eens met de stelling 'De overheid moet meer investeren in onderwijs'?"            ,
  c("Helemaal mee oneens", "Mee oneens", "Niet mee eens, niet mee oneens", "Mee eens", "Helemaal mee eens") ,
  "Hoe waarschijnlijk is het dat u volgend jaar verhuist?"                                                  ,
  c("Zeer onwaarschijnlijk", "Onwaarschijnlijk", "Waarschijnlijk", "Zeer waarschijnlijk")
)

option_rows <- options_items |>
  mutate(
    context = paste0(
      "Question: ",
      stem,
      "\nAnswer options: ",
      map_chr(options, \(o) paste(o, collapse = "; "))
    )
  ) |>
  unnest(options) |>
  transmute(
    type = "Response option",
    text = options,
    context
  )

followup_items <- tribble(
  ~previous           , ~question          ,
  "Rookt u?"          , "Hoeveel per dag?" ,
  "Drinkt u alcohol?" , "Hoeveel per dag?"
)

followup_rows <- followup_items |>
  transmute(
    type = "Follow-up question",
    text = question,
    context = paste0("Previous question: ", previous)
  )

context_results <- bind_rows(option_rows, followup_rows) |>
  mutate(
    without_context = map_chr(text, translate_without_context),
    with_context = map2_chr(text, context, translate_with_context)
  ) |>
  select(type, context, text, without_context, with_context)

write_csv(context_results, "supplying-context.csv")

Context corrects the text that was wrong on its own and leaves what was already right untouched. “Eens” moves from “once” to “agree”, and “Dalen” from “valleys” to “fall”, because the stem fixes the reading. “Man” and “Vrouw” come back as “Male” and “Female” under the sex question and as “Man” and “Woman” under the gender question, the same two words resolved two ways by the item around them. The scale endpoints stay parallel, “very unlikely” against “very likely”, because each option was translated with the rest of the scale in view. The follow-up question “Hoeveel per dag?” becomes “How many per day?” after “Rookt u?” and “How much per day?” after “Drinkt u alcohol?”, with the previous question supplying the count-or-mass distinction English needs. The text that was never ambiguous, such as “Oneens” or “Gelijk blijven”, translates the same with or without context. Supplying context fixes what needs it without disturbing the rest.

Supplying a glossary

A glossary is a list of required translations for specific terms. It is the right tool when a term has no clean English equivalent and the project has agreed on a standard rendering, so that every occurrence is translated the same way across items and across survey rounds. It is not a tool for resolving meaning: a word whose correct sense depends on the sentence, like the response options above, belongs in context rather than pinned to a fixed translation.

The agreed renderings go into the prompt as explicit term pairs.

edu_items <- tibble(dutch = c("VWO", "HAVO", "VMBO"))

glossary <- c(
  "VWO  = pre-university education (VWO)",
  "HAVO = senior general secondary education (HAVO)",
  "VMBO = pre-vocational secondary education (VMBO)"
)

translate_with_glossary <- function(text) {
  chat <- chat_lmstudio(
    model = "google/gemma-4-26b-a4b-qat",
    system_prompt = paste0(
      "Translate the following Dutch text to English.\n",
      "Use these required translations for specific terms:\n",
      paste(glossary, collapse = "\n"),
      "\n",
      "Output only the translation."
    ),
    params = params(temperature = 0)
  )
  chat$chat(text)
}

edu_items |>
  mutate(
    without_glossary = map_chr(dutch, translate_without_context),
    with_glossary = map_chr(dutch, translate_with_glossary)
  )

The same glossary applies when an education level appears inside a full item rather than on its own. The rest of the sentence is translated normally, and the listed term takes its agreed rendering.

item <- "Bent u na de HAVO doorgestroomd naar het hoger onderwijs?"

tibble(
  dutch = item,
  without_glossary = translate_without_context(item),
  with_glossary = translate_with_glossary(item)
)

Combining context and a glossary

Translating a real questionnaire combines both techniques, but the context an item needs is specific: a response option needs its own question, not the entire questionnaire. Recording the survey structure makes that possible. Each row stores the text, what kind of element it is, the question it belongs to, and the block it sits in. A block can span several questions, so question identifies the question and block groups related questions together.

questionnaire <- tribble(
  ~block      , ~question , ~type      , ~text                                                       ,
  "agreement" ,         1 , "question" , "In hoeverre bent u het eens met de volgende stelling?"     ,
  "agreement" ,         1 , "option"   , "Eens"                                                      ,
  "education" ,         2 , "question" , "Wat is uw hoogst voltooide opleiding?"                     ,
  "education" ,         2 , "option"   , "VWO"                                                       ,
  "education" ,         2 , "option"   , "HAVO"                                                      ,
  "education" ,         2 , "option"   , "VMBO"                                                      ,
  "education" ,         3 , "question" , "Bent u na de HAVO doorgestroomd naar het hoger onderwijs?"
)

The prompt for each item is built from these columns. It names the element type, and for a response option it supplies that option’s own question as context, found by matching on question. The glossary is added as before.

stems <- questionnaire |>
  filter(type == "question") |>
  select(question, stem = text)

translate_item <- function(text, type, block, stem) {
  type_desc <- switch(
    type,
    question = "a survey question",
    option = "a response option",
    instruction = "interviewer instruction text"
  )

  system_prompt <- paste0(
    "Translate Dutch survey text to English.\n",
    "The text is ",
    type_desc,
    " from the \"",
    block,
    "\" block of a survey.\n",
    if (type == "option") {
      paste0("It is a response option to the question: \"", stem, "\"\n")
    },
    "Use these required translations for specific terms:\n",
    paste(glossary, collapse = "\n"),
    "\n",
    "Output only the translation of the text."
  )

  chat <- chat_lmstudio(
    model = "google/gemma-4-26b-a4b-qat",
    system_prompt = system_prompt,
    params = params(temperature = 0)
  )
  chat$chat(text)
}

questionnaire |>
  left_join(stems, by = "question") |>
  mutate(
    translation = pmap_chr(list(text, type, block, stem), translate_item)
  ) |>
  kable()

Piped text

Some survey items are not fixed text. They contain a placeholder that the survey software replaces at the moment the item is shown, filling it with a value taken from an earlier answer or from the sample record. A satisfaction item might be stored as “How satisfied are you with {provider}?”, where {provider} is replaced by the provider the respondent named a few questions earlier. This filled-in text is called piped text, and the placeholder is called a fill.

Piped text is the hardest case of the context problem from the start of this page. Every other failure described here comes from context that exists but was not supplied to the MT system: the surrounding questionnaire, the question a response option belongs to, the agreed translation of a term. The value that fills a placeholder is different, because it does not exist when the item is translated. The questionnaire is translated once, in advance; the value is supplied later, per respondent. The translation therefore has to be correct for every value the placeholder might take, without any of them being available.

This raises two concerns. The first is whether the placeholder itself survives translation intact. The second is whether the words around the placeholder fit the value that fills it. The first turns out not to be a problem for the model used here; the second is, and it cannot be fixed by adding to the prompt.

The examples so far translated Dutch into English. Piped text is clearest in the other direction, because the problems it causes are most visible when the target language inflects more than the source. The examples below translate English survey text into Dutch.

Does the placeholder survive translation?

The survey software finds a fill by matching the exact placeholder text, so the placeholder has to come back from translation unchanged. If the MT system translates, reorders, or alters it, the placeholder no longer matches the key the software substitutes on, and the fill silently fails. Whether this happens is an empirical question, and it depends on the placeholder convention and on where the placeholder sits in the item. Survey tools use several conventions, from plain braces to the longer reference syntax of platforms like Qualtrics, and an item can carry one placeholder or several.

The list below collects these cases: each placeholder convention on its own, then items with a repeated fill, two fills, several fills, and a fill carried over from an earlier answer. Each is translated with the bare prompt, which says nothing about placeholders, and checked for whether every placeholder it started with comes back unchanged.

translate_en_nl <- function(text) {
  chat <- chat_lmstudio(
    model = "google/gemma-4-26b-a4b-qat",
    system_prompt = "Translate the following text from English to Dutch. Output only the translation.",
    params = params(temperature = 0)
  )
  chat$chat(text)
}

items <- tibble(
  text = c(
    "How satisfied are you with {provider}?",
    "How satisfied are you with {{provider}}?",
    "How satisfied are you with [provider]?",
    "How satisfied are you with <provider>?",
    "How satisfied are you with %provider%?",
    "How satisfied are you with ${provider}?",
    "How satisfied are you with ${e://Field/provider}?",
    "How satisfied are you with ^provider?",
    "How satisfied are you with |provider|?",
    "You told us your provider is {provider}. Has {provider} contacted you this year?",
    "Compared with {provider_a}, how would you rate {provider_b}?",
    "On {date}, did you contact {provider} about your {product}?",
    "Earlier you said you live in {city}. Is {city} where you were born?"
  ),
  placeholders = list(
    "{provider}",
    "{{provider}}",
    "[provider]",
    "<provider>",
    "%provider%",
    "${provider}",
    "${e://Field/provider}",
    "^provider",
    "|provider|",
    "{provider}",
    c("{provider_a}", "{provider_b}"),
    c("{date}", "{provider}", "{product}"),
    "{city}"
  )
)

items |>
  mutate(
    translation = map_chr(text, translate_en_nl),
    preserved = map2_lgl(translation, placeholders, \(out, ph) {
      all(str_detect(out, fixed(ph)))
    })
  ) |>
  select(text, translation)

Every placeholder comes back unchanged, in every convention and every position, and the preserved column is TRUE throughout. For this model, the placeholder surviving translation is not a problem, and no instruction about placeholders is needed. This is a property of the model rather than a guarantee: a weaker model, or a more unusual placeholder syntax, could still alter the token, so a line in the prompt telling the model to leave placeholders untouched is cheap insurance worth keeping.

Agreement with the filled-in value

The problem that does not go away is grammatical. The words around a placeholder have to agree with the value that fills it, and Dutch marks agreement that English does not. English “the” becomes either “de” or “het” in Dutch, and English “this” becomes either “deze” or “dit”, with the choice set by the noun that follows. Which form is correct therefore depends on the noun that will fill the slot. The frame is translated once, before any value is known, so the model has to settle on one form.

The chunk translates two frames once each, one using “this” and one using “the”, then substitutes two fills of different gender: “bank”, a “de” noun, and “energiebedrijf”, a “het” noun.

frames <- tibble(
  english = c(
    "How satisfied are you with this {x}?",
    "Was the {x} helpful?"
  )
)

fills <- tribble(
  ~fill            , ~gender ,
  "bank"           , "de"    ,
  "energiebedrijf" , "het"
)

frames |>
  mutate(translated = map_chr(english, translate_en_nl)) |>
  crossing(fills) |>
  mutate(filled = str_replace(translated, fixed("{x}"), fill))

In both frames the model settles on the “de” form, “deze” for “this” and “de” for “the”, and bakes it into the frame. That is correct when a “de” noun fills the slot (“deze bank”, “de bank”) and wrong when a “het” noun does, where Dutch requires “dit” and “het” (“deze energiebedrijf”, “de energiebedrijf”). The two fills come from the same question, so the survey cannot select a different frame for each. Whatever form the model commits to is wrong for the values of the other gender.

A targeted prompt does not fix it

The obvious response is to tell the model what it is dealing with: that the placeholder will be replaced by a noun whose gender is not known in advance, and that the translation has to stay correct whatever fills it.

translate_robust <- function(text) {
  chat <- chat_lmstudio(
    model = "google/gemma-4-26b-a4b-qat",
    system_prompt = paste0(
      "Translate the following English survey item to Dutch. ",
      "The text contains a placeholder in curly braces that will later be replaced ",
      "by a noun whose grammatical gender (de-word or het-word) is not known in advance. ",
      "Phrase the translation so that it stays grammatically correct whatever the gender ",
      "of the noun that fills the placeholder. Leave the placeholder unchanged. ",
      "Output only the translation."
    ),
    params = params(temperature = 0)
  )
  chat$chat(text)
}

frames |>
  mutate(translated = map_chr(english, translate_robust)) |>
  crossing(fills) |>
  mutate(filled = str_replace(translated, fixed("{x}"), fill))

For “this”, the instruction makes no difference: the model still produces “deze”, so the “het” fill is still wrong. For “the”, the model does respond to the instruction, but only by leaving both options in the text, “de/het”, which is not something that can be shown to a respondent. The prompt cannot help because the fact it would need, the gender of the value, is not in the prompt. No instruction supplies information that is not there.

Fixing it outside the translation

Since the gender lives in the value, the fix has to involve the value. The most direct option is to keep the “the” out of the frame and carry it in the fill instead. The source item is written without it, and the Dutch fill is stored with the correct word already attached, “de bank” and “het energiebedrijf”. The frame then has nothing to agree with, and the result is correct because the “de” or “het” was chosen together with the noun.

frames_no_article <- tibble(
  english = c(
    "Was {x} helpful?",
    "Did you find {x} clear?"
  )
)

fills_with_article <- tribble(
  ~fill                , ~gender ,
  "de bank"            , "de"    ,
  "het energiebedrijf" , "het"
)

frames_no_article |>
  mutate(translated = map_chr(english, translate_en_nl)) |>
  crossing(fills_with_article) |>
  mutate(filled = str_replace(translated, fixed("{x}"), fill))

Every filled sentence is now correct, because the “de” or “het” travelled with the noun instead of being guessed by the translator.

This works for “the”, but not for “this”. The word “this” belongs to the frame rather than to the value: the same provider can be “this bank” in one item and “the bank” in another, so it cannot be stored once alongside the noun the way “de” and “het” can. Where the word that has to agree belongs to the frame, the two remaining options are to rephrase the source so it does not force agreement, or to substitute each value into the source first and translate the filled-in items individually. Both move the agreement decision to a point where the value, and so its gender, is known. The cost of the second is that the questionnaire can no longer be translated once in advance, since the items to translate now depend on the answers.

Practical recommendation

DeepL’s context parameter is the most direct way to pass surrounding questionnaire text when translating item by item. Google Translate’s glossary support is the most direct way to enforce consistent terminology across all items. LLMs combine both capabilities in a single prompt, and also allow instructions about register, cultural adaptation, and formatting — at the cost of more setup and less predictable output format.

For workflows where terminology consistency across rounds matters — for example, translating a new round of the ESS to match previous rounds — including reference translations from the MCSQ in the prompt encodes that requirement directly.