How were our language tests developed?

General Overview

  • Language tests include: English, French, Spanish, Italian, German, Mandarin, Portuguese, and Dutch.
  • Questions follow a multiple-choice format with one correct answer and two-to-four distractors each.
  • Questions target language proficiency equivalent to B2 (upper-intermediate) level on the Common European Framework. 
  • Each test consists of 15 questions.
  • 10-minute time limit per test.

Test Development 

These tests were developed by linguistics experts with proven track records, supported by further subject matter experts and our team of organizational psychologists and psychometricians. The test writers used our purpose-built platform to share input regarding the level of difficulty and cognitive burden of each question along with the full scope of the test, and other details. The gathered data, together with the content of the US Department of Labor O*NET, was used to inform content and construct validity. 

The tests were piloted by making them temporarily accessible via Practice Aptitude Tests, a testing platform where job seekers can practice psychometric tests similar to those they might face when applying for a job. Using pilot data and feedback from practice test-takers, the tests and questions were evaluated empirically and revised to improve reliability, validity, and fairness, the three most important aspects of evaluating a test. As the database of responses grows, the tests are periodically analyzed and updated to improve them in an iterative fashion over time. 


Scoring

Test scores are calculated as a simple percentage of questions correct out of the total number of questions (ranging from 0-100%). Results are provided together with a global average and distribution of scores for each test from a representative sample of job candidates that are kept up to date. This provides objective data for candidate comparison across the talent pool.


Implementation Guidance

Language tests are most applicable when hiring for expatriate or global roles but they can be useful for various selection and development processes. Relevant roles might be linguistics professors, translators, or travel agents. We highly recommend that a language test is used as one of several in a battery of tests selected based on information gathered by analyzing the skills required for the job in question.


Test Security

Access to the test and its questions is tightly controlled by our password-protected test bank. Subject matter experts involved in its development and review sign non-disclosure agreements. Additionally, questions are randomly assigned to candidates, controlling for the overall difficulty of the test; thus, no two candidates see the same test but their results remain comparable. 


Key Pilot Statistics

  • Samples consisted of a minimum of 100 potential job seekers per test.
  • Sample demographics across tests were as follows:
    • Average age was 33.7 years;
    • Genders included female (52.7%), male (45.5%) and other (1.8%);
    • Ethnicities included Asian (32.7%), Black or African American (30.8%), White (23.1%), Other Ethnicity (9.6%) and Mixed or Multiple Ethnicities (3.8%);
    • Countries included India (25.5%), Uganda (12.7%), United States (10.9%), Canada (9.1%), Saudi Arabia (3.6%) and 19 others.
  • Average score was 64.78 (SD = 24.98)
  • Average time to complete each test was 406.6 seconds (6 minutes 46 seconds)  
  • Average completion rate of 97.1% 
  • Items range in difficulty such that roughly 20% of test-takers answered the most difficult questions correctly and roughly 80% answered the easiest questions correctly.
  • Approximately 85% of test-takers believed each test to be at a reasonable level of difficulty and 90% felt the questions were relevant to measuring knowledge of the language in question.

*Most recent data on the test’s performance is available through the Neuroworx platform which is updated regularly.