How One Algorithm Determines the Future for Thousands of People

How One Algorithm Determines the Future for Thousands of People

On March 17 of this year, thousands of medical students gathered in auditoriums, conference rooms, and classrooms all over the country to find out where they’d be spending the next three to six years of their professional careers. In a sense, it all came down to one advanced algorithm.

The Competitive Nature of Residency

Each year, thousands of students graduate from medicals school and launch their careers in residency programs throughout the country. Residency programs are essentially postgraduate training programs in which the resident is allowed to perform as a licensed practitioner under carefully scrutinized supervision. These programs last for a few years and once the resident completes the required duties, he is able to “graduate” into a full-time position.

Residency programs are competitive both on the side of the applicant and the sponsoring institution. A large percentage of residents will stick around and continue their careers at the hospital or healthcare facility where they go through residency; thus, programs want to get the best possible talent they can. Medical students know that certain residency programs are more esteemed than others and therefore have preferences themselves.

It could be said that residency begins with “Match Day,” something that’s a big event on medical school campuses all over the country.

“Match Day is the culmination of the process of applying and interviewing for residency programs at health care institutions across the country,” explains Rush University, which placed 115 students in 21 specialties at 69 different institutions this spring. “Once they’re done interviewing, students rank their choices in a computerized system, and the programs in turn rank their top student picks.”

The National Resident Matching Program

Because thousands of medical students graduate each year and apply for positions in thousands of different residency programs, there has to be some sort of system for bringing order to what would otherwise be a chaotic process. The system, previously alluded to by Rush University, is known as the National Resident Matching Program, or NRMP. It uses an advanced mathematical algorithm to place applicants into residency and fellowship positions around the country.

The algorithm – awarded a 2012 Nobel Prize in Economic Sciences – is classified as “applicant-proposing,” which means the preferences of the applicant are expressed in a rank order list (ROL). Programs do not get to initiate placement, which means no applicant can obtain a better outcome than the one produced by the NRMP algorithm.

“The process begins with an attempt to match an applicant to the program most preferred on that applicant’s rank order list,” NRMP.org explains. “If the applicant cannot be matched to that first choice program, an attempt is made to place the applicant into the second choice program, and so on, until the applicant obtains a tentative match or all the applicant’s choices on the ROL have been exhausted.”

Matches are known as tentative because programs may or may not have unfilled positions available. And, if they do, it’s possible that another (less preferred) applicant is tentatively matched to the program and will be removed to make room for the preferred applicant.

“When an applicant is removed from a tentative match, an attempt is made to re-match that applicant, starting from the top of the applicant’s ROL,” NRMP.org continues. “This process is carried out for all applicants until each applicant has either been tentatively matched to the most preferred choice possible or all choices submitted by the applicant have been exhausted.”

It all sounds a bit complicated, right? That’s because it is. The algorithm is based on the work of David Gale and Lloyd Shapley, American economists and mathematicians who have also contributed heavily to game theory. The same algorithm is actually used in a number of other applications as well, including finding recipients for organ donations and assigning schools in Boston and New York to high school students. But the NRMP is where this algorithm truly shines.

Each year, roughly 94.4 percent of U.S. fourth-year medical students participating in the program are assigned a residency. Roughly 80 percent of these students are matched with a residency program in their top three.

The system isn’t without its flaws, though. While it’s better than any system that’s previously existed, things could get more difficult in the coming years. Because conventional wisdom says that students have a better chance at a match if they include more schools on their list, students are making longer lists. The average number of programs listed on a students ROIL has grown from 7.4 to 10.3 over the past decade.

While the NRMP algorithm can handle these increases, it does make for a more complicated process. Programs can’t possibly interview all of their applicants, meaning there isn’t always great knowledge of who applicants are (from a program’s perspective).

A Taste of the Future

It’s a bit crazy to think that an algorithm developed by two mathematicians in a computer lab is determining the future of thousands of medical students each year. While applicants do have a say in where they end up, it’s ultimately the algorithm that makes the final choice. Intimidating, scary, exciting? Nobody is sure what to think.

In the coming years, as artificial intelligence and big data grow in prominence, it’s possible that we’ll see even more societal processes improved and determined by advanced algorithms. These algorithms won’t lead to perfect outcomes, but – as the National Resident Match Program can attest to – it’ll be pretty close.

This post is part of our contributor series. It is written and published independently of TNW.

Read next: Why go responsive: the secrets of success for entrepreneurs online