The percentage of test-takers who passed the nephrology certification exam on their first try increased by 6% in 2020, after dropping to an all-time low in 2019. However, the 2020 pass rate of 80% is still below the 83% rate of 2018 and is the second-lowest rate ever.
The large drop in 2019 set off a dismayed discussion on Twitter and other venues about what was going wrong, and the reaction to 2020’s improvement was much more restrained.
“I am relieved we are not seeing a continued drop in the pass rate, but nephrology still has the lowest pass rate of any internal medicine specialty,” Scott Gilbert, MD, of Tufts Medical Center and chair of the ASN Workforce and Training Committee, said in an email to Kidney News. “We need to continue to make nephrology an attractive field for the best candidates, and then improve our training experience to allow candidates to succeed on the boards.”
“Interpreting this year-to-year is less useful than monitoring the trends over time,” Gilbert said, and the long-term trends point toward a challenging time for the field of nephrology. The 80% pass rate is a 10-point drop from 2016, and there have been instances in the past when scores have bumped up before continuing to decline.
Long-term decline
In 2014, the pass rate dropped 7 percentage points to 80%, only to bounce back to 89% in 2015 and 90% in 2016. But that recovery was followed by another decline to 83% in both 2017 and 2018, then 74% in 2019. The trend lines point to a long-term decline: in the five-year period from 2006 to 2010, the pass rates averaged 94%; from 2011 to 2015, they averaged 86%; and from 2016 to 2020, they averaged 82% (and for the past three years, 79%).
For the 16 subspecialties included in the American Board of Internal Medicine (ABIM) certification tests in 2020, the average pass rate was 92%. Nephrology’s pass rate is a full 5 percentage points below the rate of the next-lowest specialty (hospice and palliative medicine), and all the other subspecialties had pass rates of 89% or higher.
Matthew A. Sparks, MD, assistant professor of medicine at Duke University, associate director of its fellowship program, and a member of the ABIM nephrology board, said he doesn’t see a big difference between 2019’s 74% rate and 2020’s 80% pass rate: “I am happy it is not lower, but I am not really happy with 80%. Every program director has to look at this and figure out what should they do to improve and start taking steps to figure this out. This needs to be tackled on multiple fronts, and that [extends to] fellowship programs, to educational leaders, to the ABIM board, [and] to the exam-writing committee. All need to really come together to tackle this issue. I think any fellow in the United States should be capable of passing this test on the first attempt.”
Why the drop?
Various explanations for the drop in test performance have been put forward, with the most commonly cited contributors being a drop in the quality of the fellows who form the test-taking pool, a mismatch between the test material and the clinical experience of fellows, and a failure of fellowship programs to prepare trainees for the exam.
The concerns about the quality of the candidates coming into nephrology stem from the difficulty fellowship programs have had in recent years in attracting candidates. In 2010, the initial pass rate was 98%, and in that period, nephrology programs had their pick of candidates, with 1.5 candidates applying for each available position. But the subspecialty dropped in popularity, and in recent years there have been fewer applicants than positions available. Nephrology is not the first choice of a significant number of fellows matched into it through the National Resident Matching Program, even as the number of programs has grown.
Evidence that nephrology may not be attracting the strongest candidates comes from the internal medicine certification exam. A decade ago, candidates who completed the nephrology certification exam had among the highest scores on the internal medicine certification exam compared with other subspecialties. Candidates who took the 2019 nephrology exam had the lowest scores on the internal medicine exam compared to other subspecialties, according to Bradley Brossman, PhD, vice president of psychometrics at ABIM. ABIM declined to give updated figures for the most recent test.
Relevance of the test
Another area of concern relates to the question of how accurately the test reflects the needs and realities of the fellows who take it, as the test creators try to balance the inevitable tension of including routine and rare conditions.
“I thought that they tested both the common diseases that we see in everyday practice, as well as rare diseases and things that we do not see commonly. It is important to test both,” according to Manasi Bapat, MD, who passed her exam in 2018 and then joined a large private practice, the East Bay Nephrology Medical Group in California. “Even though these rare diseases are not something we see every day, we need to know them so that we are prepared to diagnose them when we finally encounter them in practice. So I think it was pretty fair.”
In contrast, Yusra R. Cheema, MD, who passed the test in 2014 and is now director of the fellowship program at Northwestern University, says the exam overemphasizes esoteric subjects in a way that often makes it necessary to teach to the test: “There are things they like to test because they make good test questions. When doing review sessions with the fellows, I find myself saying, ‘I know in real life we would do A, B, and C, but in this test we are going to have to choose one best answer.’ That is kind of a non-real-world scenario” that fellows need to be aware of to pass the test.
Bapat and Cheema both stressed that they personally benefited from fellowship programs that had the resources to provide time for study sessions devoted to test preparation.
Anyone who fails the exam can take it again, and the ultimate pass rate for certification remained at 97% across all disciplines, including nephrology, according to the most recent numbers ABIM released. Bapat said that this ultimate pass rate supports her conjecture that some candidates may fail due to inadequate preparation. “The second time around, these folks may be putting in more time and succeeding. I think the burden of adequate preparation falls more on the individual physician appearing for the exam, but programs may be able to bolster this with dedicated time and board review sessions. These efforts from training programs are of paramount importance from my personal experience,” she said.
Fellowship programs on the edge?
Whether or not prep sessions are a key to success, the pass rates have led some to wonder whether some fellowship programs are falling short in providing adequate preparation. As far back as 2014, in an editorial in the American Journal of Kidney Diseases, Christina Yuan, MD, of Walter Reed National Military Medical Center and two co-authors calculated that the general pass rates indicated that many nephrology training programs may not be meeting the minimum pass-rate threshold required by the Accreditation Council for Graduate Medical Education (ACGME) to remain accredited. Since that editorial, the pass rates have declined, making this worry “even more” likely, Yuan said in an email to Kidney News.
If some programs have near-to-perfect pass rates—as two of the programs contacted for this article confirmed—then other programs must be below average. If the five-year nephrology average pass rate is 82% overall, and ACGME requires an 80% pass rate to accredit a fellowship program, there must be programs that are skating on the edge. But neither ABIM nor ACGME will make this information public.
“The American Board of Internal Medicine publicizes pass rates for internal medicine residency programs,” said ASN Executive Vice President Tod Ibrahim. “To date, however, ABIM has refused to do the same with any fellowship training programs, including nephrology. As such, the community and, more important, the applicants have no way to compare programs related to how their graduates perform on initial certification. I recognize that this issue is challenging, but the fact that we cannot even have the discussion is disappointing.”
Responding to the challenge
The nephrology community has recognized and responded to these challenges in recent years, raising hopes that 2020’s uptick is not an aberration but a harbinger of the trend reversing. For example, the ABIM Nephrology Committee, partly in reaction to criticism that it was dominated by older academic nephrologists, has become younger and more diverse in recent years.
For its part, ASN has increased its educational support, Ibrahim said: “ASN provides the in-training examination for nephrology fellows, the Kidney Self-Assessment Program (KSAP), the Nephrology Self-Assessment Program (nephSAP), and the board review course and update to help support everyone, including nephrology fellows, who is preparing for the ABIM exam.”
Only time will tell whether the increase represents a trend or was the result of a statistical tick or other confounder. Nephrology was part of an overall trend—pass rates improved in nine of the 16 subspecialties.
Could the greater success have been an effect of the most consequential factor of the year, the pandemic? Did being homebound lead to more study time? Bapat said that was unlikely because fellows “were probably busier because the hospitals were packed, and we had so many dialysis patients.” Sparks noted that “you have to be in the right frame of mind to retain information,” and the many stresses from many directions would make study more difficult.
“I’ve oftentimes been accused of being an optimist,” Cheema said. But at least the scores have moved back to a positive direction, and the recent nephrology match was “more successful than we have had in many years. So I am hopeful about the direction in which nephrology is going.”
