Multinational Trials—Recommendations on the Translations Required, Approaches to Using the Same Language in Different Countries, and the Approaches to Support Pooling the Data – Comments

Overall comments are first, followed by SPECIFIC comments for each section of the MS.  It is recommended to read all of the comments, i.e. every section’s comments as some comments may be relevant for more than one section, particularly for sections 1 and 2.

Overall comments

  • The document takes on three important issues that build upon the Principles of Good Practice for the Translation and Cultural Adaption Process for PRO Measures.  It would be useful to review this document closely to make sure that the terminology used in the original document is used here as well. This is particularly an issue in Section 1 where the text does not identify individuals who should be responsible for the steps in the same terminology used in the previous report. e.g., Key Actors.
  • Thanks for all your hard work in preparing this report. As well as my section-specific comments, I'd recommend that you do a better job highlighting the limitations of your research.
  • Very well thought out and provides good guidelines.
  • This is well covered report.
  • Keep up the great work; I know researchers conducting international work will find this document useful.
  • I read your draft report with interest.  From time to time these issues arise in the work that I do, so it was great to see these issues clearly articulated and thoughtful advice given.  I have no specific comments.
  • I found this draft both comprehensive and useful.
  • It's a very thoughtful and comprehensive document that will be used and cited quite a bit, I'm sure.
  • I agree with the non-prescriptive tone of most of the document, lacking in specific recommendations of one or another set of rules for selecting languages needed (section 1), procedures for translation of one versus multiple language version, or minimum requirements for pooling. This reads like a best practices guideline and not a cook book with a right and wrong way, and again I agree with that.
  • Each section has a conclusion. But the brief conclusion in sections 1 and 2 are followed by a longer "summary" in section 3. Changing the terminology and length leaves the impression it is a summary of the whole document.  To fix that I suggest concluding each section with a header like: "Section 1 Conclusion," "Section 2 Conclusion," Section 3 Conclusion," and making them about equal length and tone, and THEN write an Executive Summary at the end covering all three areas with the bottom line positions.

Section 1

  • In each section there are references to a literature search, but the first section does not provide any references to previous work. The addition of this information would lead to greater consistency across sections.
  • The report suggests a 3% threshold but there is no explanation of where this number came from.  There is an * after the first mention of this on page 1 but this leads nowhere.
  • I found the description of the grid very difficult to follow. The categories were quite vague and confusing.  For example, in the Analysis of the Population what is meant by "Origin of Immigration"? Also are you to provide specific origins for each ethnic group? In the literacy rate section, are you looking for literacy rate in the predominant language, the national language, each of the languages spoken, etc. etc.? Also it is not clear how this information is used in the decision tree. In the Epidemiology section, what are you asking for - incidence rates? Prevalence rate? Something else?  Also just putting down the rates by ethnic group, age and socio-economic group ignores the possibility of important interactions between the three categories. And again, it is not clear how this information feeds the flow chart.  Similar problems exist in the Analysis of the Languages section. Perhaps I am not as familiar with this as those who are writing this section, but I would have difficulty completing this chart and would make decisions based on my own interpretation and this would not promote the consistency you are aiming to promote.
  • This section does not provide any guidance on possible sources for this information.  My own experience on a disease management project in a multi-ethnic neighborhood in London was that the information was difficult if not impossible to obtain in the detail that these charts require.
  • One aspect which is important to consider is the number of patients from a country specific language. What is considered substantial to include that group of patients do an official translations? For example if South Africa is enrolling 10 patients in a trial, is it okay to use UK English and not do country specific adaptation? Also at what n does it make sense to have a country specific adaptation?

            NB this last bullet point could be relevant for section 2 as well

  • In selecting the language, one might also need to take the consideration of the level of language ability. In Table 1 under Main Languages, suggest to add the grade level for the language
  • Under epidemiology of the disease in the country, one may need to identify the incidence of the disease contributed by the immigration from certain countries. The language version for these countries may be needed. Such as HBV in US, a high percentage of them were due to people who immigrated from Asia, and they may not able to speak English.
  • I don't buy into your 3% figure. You should provide more clarification on rationale and/or better caveats.
  • In the analysis of the languages, it's unclear to me how it would be addressed if an individual spoke more than 1 language. Therefore, their main language might not be available, yet someone is also proficient in another language.

Section 2

  • The document handles differences in the same language between countries e.g. English for UK and USA but does not deal with regional differences that can occur within a country e.g. Spain.
  • The document assumes that the source document for translation will always be English but this is not always the case.
  • The background section is a bit confusing. I'm not sure how the reference to the US Hispanic population was to be interpreted. Was the author saying that the US Spanish speaking population included individuals from different cultural and linguistic backgrounds (e.g., Central American countries, South American countries, Caribbean countries, etc.) and could not be dealt with monolithically? 
  • Sometimes the reference is to languages spoken in more than one country and sometimes to languages spoken in more than one region of a country. It would be good to reference both.
  • The examples are excellent and really help to understand the benefits and limitations of each approach. You might consider adding comments on how the group arrived at relative cost and relative time assessments.
  • In Scenario No. 1 I would just use the country names under "Target Countries" so that it is the same as the Description section.  Latin America includes Central America and also Brazil - so you would be better off just using “Argentina".
  • On page 13, I think you need more detail on the sections that cannot be translated universally - perhaps an example of each would help the reader.
  • On the decision tree, I was not sure what you meant by "geographic similarities" and how those impacted the approach to be selected.  Also, would you always do as the developer or client wants in selecting the methodology? What if it was inappropriate to the needs of the study?
  • Possible Approaches-I'm not clear why there are more options and scenarios described in the tables (Page 7-11) then what is presented on Page 6. Also it's confusing that you state there are 3 approaches, yet there are 4 bullet points on Page 6.
  • First sentence of Introduction is not a sentence; it's missing some words.
  • Regarding section 2 conclusion: "The results of our work in developing the scenarios and decision-tree confirmed that no single approach would suffice in every situation."  Isn't it more accurate to say that no single approach could be regarded as superior to another, with each having advantages and disadvantages." The way it's phrased now it implies one can evaluate one's prospective clinical trial need and determine which approach would be best for that trial, and yet the experts agreed and stated elsewhere that there are no data to guide the decision of number of language versions needed or procedures to translate.
  • A call for more research with specific suggestions would be nice.

Section 3

  • The issue of whether to go for the ability to pool data or to go for accuracy of language is discussed. If the selection factors include how the data are to be subsequently used then this would be helpful e.g. if for local market access then validity of local sample will be important.
  • This section is very well written.  The content is clear. The references to the literature are helpful and the various options are well documented.  This section could be enhanced by a few more examples such as those in the section on classical test theory.
  • Need to clarify somewhere that even if there is rationale to pool data for efficacy purposes, we need to justify it further for PROs.
  • Since you explicitly state that there is no existing gold standard to define the level of accepted similarity or variance, no one is going to buy the additional level of scrutiny needed prior to pooling data. Somehow the Challenges Around interpretation section needs to be re-worked to be a little more definitive.
  • A call for more research with specific suggestions would be nice for section 3 (call for more research on pooling).
  • There must be a conclusion section to end the section.

NB this last comment was written in “section 3” but it may have been intended as relating to the entire paper

PRO Task Force: Changing Culture or Language of an Application | Task Forces Index