Imagine this challenging scenario: You’re a medical student talking with a recent immigrant to the United States who is about to undergo fairly minor surgery. The person shares customs that are observed when someone from their culture dies, and they ask you to ensure that these rituals are honored if they don’t survive.
What do you do? Tell the patient not to worry about the customs since the surgery is minor? Say you’ll respect the customs but can’t ensure that other hospital staff will do so? How about contact your supervisor for suggestions on how best to proceed?
These are the kinds of questions thousands of medical school applicants will face on the AAMC’s situational judgment test (SJT) — now known as the Professional Readiness Exam (PREview®) — a 75-minute test designed to assess professional and interpersonal skills central to the practice of medicine.
After seven years of research that included a 2-year pilot with 19,000 applicants, the AAMC is now launching the PREview exam more broadly. Eighteen schools have signed on to either recommend or require the online test.
The PREview exam, which opens for registration on April 5, was designed to measure knowledge of effective behavior in eight professional competencies. Among them are such essentials as adaptability, teamwork, ethics, and cultural competence.
“Future physicians need to know how to care for the whole patient and collaborate with multidisciplinary teams,” says Dana Dunleavy, PhD, director of the AAMC’s admissions and selection research and development program. “They also need to navigate complex cultural, emotional, and ethical challenges.”
Participating medical schools are hoping the exam will help them wade through sometimes massive applicant pools. “We receive around 10,000 applications for roughly 150 spots,” says Joel Maurer, MD, assistant dean for admissions at the College of Human Medicine at Michigan State University (MSU) in Lansing.
“Situational judgment testing helps us assess maturity, and it’s my experience that people with maturity have a better chance of succeeding when challenges arise in medicine,” he adds.
The PREview exam will offer medical school admissions officers a validated, standardized measure of such skills, say those involved.
“You’ve got to be more than super-smart to be a good physician, and we need a way to put a quantifiable value on preprofessional competencies,” says Charlene Green, PsyD, director of admissions at the University of California, Davis School of Medicine in Sacramento.
“Future physicians need to know how to care for the whole patient and collaborate with multidisciplinary teams. They also need to navigate complex cultural, emotional, and ethical challenges.”
Dana Dunleavy, PhD
Director, AAMC admissions and selection research and development program
While some applicants are a bit unnerved by the prospect of another admissions hurdle, many appreciate the opportunity to showcase their capabilities.
“I’m glad to have the chance to be viewed more holistically,” says Samsudeen Iyamah, 24, of Arlington, Texas, a future medical student at Morehouse School of Medicine in Atlanta, who took PREview in August. “I want schools to know about the person I am.”
A PREview sneak peek
It takes much more than anatomy and biochemistry to succeed in medical school, admissions directors say. But measuring characteristics like resilience is no simple matter.
That’s why the AAMC collaborated with more than 150 admissions officers, faculty, staff, and student and diversity affairs officers to create PREview. The result is 30 scenarios on topics ranging from study partner conflicts to problematic social media posts.
Here’s how the PREview exam works: Each provided scenario is followed by several possible reactions. Students need to rate each of those reactions by choosing one of four options: “very effective,” “effective,” “ineffective,” and “very ineffective.”
The test is taken online and monitored in real time, with a remote proctor ensuring the test’s security and helping examinees handle any technology issues. The exam’s $100 fee covers sitting for the test and distributing the score to an unlimited number of medical schools. (Applicants who qualify for the AAMC’s Fee Assistance Program test free.) Test-takers can take any one of the offered exam dates from June through October.
Test scoring has its own sophisticated system. Medical school experts previously rated the effectiveness of the possible reactions, and examinees get points if their choices align with those ratings. But they get partial credit for ratings that come close to those of the experts.
How — and when – will schools use the test?
Admissions teams will receive an applicant’s score — that’s a number from 1 to 9, with 9 being the highest score. They’ll also receive a percentile rank to facilitate comparison to other examinees as well as a confidence band to show the score’s accuracy.
But the question on test-takers’ minds is how much those numbers matter.
Given the exam’s relative newness, Dunleavy advises schools to use PREview scores as one of many data points in the admissions process. She recommends that they view the scores as an opportunity to strengthen an application rather than to screen it out. And research from the pilot project suggests that schools did just that.
“In the early stages of using the PREview exam, we’re using the score only when it contributes positively to the applicant’s record, as a plus factor. A similar example might be having research experience: It’s not essential for all applicants but it’s a plus,” says Christina Grabowski, PhD, associate dean for admissions and enrollment management at the University of Alabama at Birmingham (UAB) Marnix E. Heersink School of Medicine.
For MSU’s Maurer, a low score may highlight the need for a deeper dive. “If somebody doesn’t do well, for example, we could go back and look at their essays or letters of recommendation for other evidence that raises questions about an applicant’s fit for us,” he says.
As for when schools will use PREview, Dunleavy expects they’ll do so at multiple points, including when selecting students to interview and again when making acceptance decisions.
Grabowski hopes eventually to use it early in the process so as to trim her huge pool of potential interviewees. That’s down the road, though. “First we need to train our admissions team,” she says. “It’s a lot easier to train the 18 who make the final decision than the 80 to 100 who do our initial screening.”
What students need to know
Three hours or less: that’s the amount of time most applicants spent preparing for PREview, according to an AAMC survey of pilot participants.
To prepare, Dunleavy advises using the AAMC’s free materials — including a six-page examinee guide — and focusing on getting comfortable with the questions’ format. She also emphasizes taking the provided tutorial on using the testing platform.
“It’s important to understand what your computer and room set-up should be before sitting down to take the test,” she says. “We don’t want examinees to be surprised or distracted by logistical issues on test day.” In addition, she notes that test-takers concerned about a quiet, private space might want to reserve a room at their school or local library.
As for boning up in the hopes of acing answers, that might be tough, says Adam Sturdivant, MPH, MS, who took the test in August. “This is about how you approach the world, so you can’t just pivot and get those skills,” says Sturdivant, 29, who will be matriculating into medical school at UAB.
Still, he offers some strategies. Among them are working through the practice test’s answer key, which includes explanations for why some replies are deemed effective and others aren’t. And during the test, he urges staying focused. “Don’t get distracted thinking about details that aren’t included in the scenario,” he says. “Don’t answer the question that you want to answer. Answer the question that’s being asked.”
Grabowski offers advice for students who don’t fare well on the practice exam. “That’s useful feedback,” she says. “The PREview exam measures significant competencies for professionals, and your initial performance can be important information on where you may need to learn and grow.”
How well does it work?
PREview faced a lot of tinkering, testing, and discussion before its rollout.
For example, in 2017, the AAMC administered a prototype of the test to students who were about to start medical school. Then, they circled back at the end of the school year. The results were positive: PREview scores correlated with major markers including grades in courses like Doctoring and Society and others that cover such crucial topics as patient-provider communication.
Dunleavy also reports that test scores mean the same thing over time since a score in any one year can be compared to those from other years.
In addition, pilot schools have been monitoring the PREview exam and report encouraging results. “When we compare the PREview score to our assessment of candidates’ performance on interviews, for example, we see a solid correlation,” says Green. “We have a strong sense that it works."
Green also is confident that it works well for students from backgrounds historically underrepresented in medicine.
“As someone who was part of the process early on, we were there specifically to pull out questions that might have biases or any scenarios that could be misinterpreted based on one’s cultural background,” she says.
In addition, data from pilot schools show no score differences based on gender or for test-takers from rural, lower socioeconomic, or limited English-proficiency backgrounds. There are, however, small group differences in mean scores for African American and Hispanic test-takers compared to White and Asian examinees.
“We are pleased that the differences are small and much smaller than what we see on other standardized tests and GPAs,” says Dunleavy. “But that’s not good enough. We are committed to improving our test content and process with the goal of reducing group differences as much as possible, hopefully to zero differences.”
One way the AAMC is working toward that goal is engaging experts in diversity, equity, and inclusion (DEI) to assess the test and related processes. “We will do this every year,” she says. “We don’t view this as ‘one and done.’”
Overall, those involved believe the PREview exam will ultimately serve the goal of greater inclusivity in medicine.
“In discussing the exam with our DEI constituents, the sentiment has always been that it would help level the playing field for applicants,” says David Acosta, MD, AAMC chief diversity and inclusion officer.
What’s more, he adds, “the AAMC encourages medical schools to enroll applicants who have meaningful service, radical empathy, resilience, and relationship-centeredness. I believe the PREview exam will assist us in getting a step closer to identifying such applicants.”
