Reference letters for Resident Selection: Frustrations and Future Direction
Rachel Bentley, MSc Candidate, Translational Medicine
At the Medical Grand Rounds on October 3, the Department of Medicine was fortunate to have Dr. Lawrence Hookey, the current Chair of the Queen’s Gastroenterology Division and Medical Director of the Endoscopy Units at Kingston General and Hotel Dieu hospitals. Presenting alongside his collaborator Dr. Mala Joneja, Dr. Hookey’s presentation focused on the problems with the current system in which letters of recommendation are written for resident selection and some potential improvements to this process.
It is widely accepted that letters of recommendation are among the most important aspects of a resident’s application; many program directors say they would consider a weaker candidate with a well-crafted reference letter more favourably, and would consider a stronger candidate with a poorly-crafted reference letter less favourably.
As Dr. Joneja highlighted, the concept of vouching for someone has been around a long time and holds a lot of power. It is unlikely that reference letters will become a thing of the past as they provide a unique milieu for personalized comments and an opportunity to make comparisons among applicants. The difficulty that program directors are currently facing is that these letters are poor at predicting future performance, which is the role they are meant to play in the selection process.
As Dr. Hookey’s research has elucidated, the majority of letter writers see their role as being an advocate for the applicant, with a fear that any negative comment could damage the resident’s career. Dr. Hookey outlined how this reluctance to be negative in letters leads to a code of sorts, where letter writers will intentionally omit a certain element or hide a criticism in somewhat positive but not glowing terms, requiring program directors to read between the lines. Additionally, with a lack of standardization, the stock a program director puts in a certain letter of recommendation can depend on their knowledge of the letter writer more than the content of the letter itself. For example, if a certain letter writer gives high praise to even the most mediocre of applicants, the program director would likely not weigh this letter heavily when making a decision.
Dr. Hookey’s research has striven to address the evident disconnect that exists between what program directors want in a letter and what referees are writing. After surveying a number of program directors to assess what they want in a letter, Dr. Hookey’s group reviewed letters to determine how many letters qualified as high-quality letters based on the criteria from program director responses. Interestingly, despite a fair bit of consensus among program directors as to what constitutes a high-quality letter, few of the letters written by program directors met all the criteria of a high-quality letter.
One of the most important characteristics that a letter of recommendation can speak to is the applicant’s communication skills. Communication skills are integral in the medical field and the letter writers, who typically interact with the applicant regularly for an extended period of time, are in a much better position to evaluate the applicant’s typical behaviour than the program director, who likely interacts with the applicant only during the interview. Interestingly, in Dr. Hookey’s study, despite being explicitly asked to comment on the applicant’s communication skills, not all letters included such statements, with less than half commenting on the applicant’s interactions with patients.
Another important aspect of these letters that Dr. Hookey pointed out was that, without qualifiers or specific examples about the applicant’s qualities and skills, the letters can be too subjective and so do not provide much value to the program directors. Moving forward, these shortcomings need to be communicated to letter writers; program directors should perhaps agree on a more standardized format for reference letters so that all necessary aspects are included in a descriptive and useful way. The fact that there is no mechanism for feedback to the letter writers is detrimental to both sides, as the letter writers may not know that their current format is not helpful and the program directors are clearly not getting an output that is predictive of performance. Additionally, improving this process could optimize the selection process, and ultimately improve patient care.
It was a pleasure to have Dr. Hookey at our latest Medical Grand Rounds discussion, and the TMED graduate students appreciated the opportunity to share Dr. Hookey’s insights into the difficulties and future directions for the writing of reference letters.