How were our soft skills tests developed?

General Overview

Soft skills tests include: Accountability, Adaptability, Communication, Decision-Making, Interpersonal Skills, Leadership, Problem Solving, Project Management, Teamwork, Time Management and more.
Questions follow a situational judgment format with two correct answers (most and least effective) chosen from four available options.
Tests target a moderate difficulty level.
Total number of questions ranges from 8 to 10.
10-minute time limit per test.

Test Development

Our team of organizational psychologists and psychometricians developed these tests, strictly adhering to industry best practices to maximize reliability, validity, and fairness. By combining theoretical knowledge of relevant constructs and decades of research, the team has developed situational judgment questions that effectively measure the defined skills in a sample of job applicants who have varying professional experience.

A panel of subject matter experts weighed in on the most and least effective actions to take in response to each scenario and these results were used to further refine the tests and answer keys. A draft version of each test was piloted with a panel of individuals representing diverse backgrounds, after which their input and the questions were rigorously analyzed to maximize reliability and validity while minimizing bias. As the database of responses grows, the tests are periodically updated to ensure that each test-taker has a fair chance of demonstrating the true level of their ability.

Scoring

Test scores are calculated as a simple percentage of questions correct out of the total number of questions (ranging from 0-100%). Each scenario has two questions - which is the most and which is the least effective action to take; this means the total number of questions is twice the number of scenarios for scoring purposes. Results are provided together with a global average and distribution of scores for each test from a representative sample of job candidates that are kept up to date. This provides objective data for candidate comparison across the talent pool.

Implementation Guidance

Soft skills are applicable and indeed highly valued for most jobs. Some are more important in certain contexts. For example, accountability is crucial in remote positions and interpersonal skills are critical for customer-facing jobs. We highly recommend that a soft skill test is used as one of several in a battery of tests selected based on information gathered by analyzing the job in question.

Test Security

Access to the test and its questions is tightly controlled by our password-protected test bank. Subject matter experts involved in its development and review sign non-disclosure agreements. These questions can only be found within the secure environment of our platform to prevent candidates from being able to practice them in advance.

Key Pilot Statistics

Samples consisted of a minimum of 100 potential job seekers per test.
Average score was 57.2% (SD = 19.65).
Average time to complete each test was 515.0 seconds (8 minutes 35 seconds).
Average completion rate of 94.8%.
Difficulty of the items ranged from p = .14 to p = .95 (in other words, 14% of test-takers got the most difficult questions correct and 95% got the easiest questions correct).
Cronbach’s α ranged from .56 to .67, which is considered moderate reliability given the brevity of the tests.

*Most recent data on the test’s performance is available through the Neuroworx platform which is updated regularly.