Eva Vivalt

Sometimes I’ve seen people be encouraged to pursue a risky career strategy. And I think people should pay more attention to who is giving the advice, how it benefits them, whether they are in a position to know the odds, and what are the downside risks to the encourager if their advice goes wrong, etc. Because I’ve seen many cases where someone was encouraged, and encouraged, and it was almost irresponsible for them to be encouraged so much against high odds of failure. (I am referring here to the type of encouragement that is optimistic, rather than the type of encouragement that indicates something is unlikely but makes an argument that it is worth it anyway.)

From a societal standpoint, it often makes sense to prod people into taking risks, for example if people are too personally risk-averse. But the incentives are often not aligned. I will give a few examples, though I’ll avoid some of the particularly devastating ones I’ve observed to preserve the anonymity of the affected parties.

For one, consider an academic advisor to a PhD student. The advisor’s incentives are likely to try to push the student into the top academic job they can get. A lot of people are sensitive to the needs of the student, but not everyone is. Suppose an advisor thinks the student could get a non-academic job if they went on the job market early but they have an outside chance of an academic job if they went on the job market later. The student might be encouraged to stay in the program longer in the hopes of getting an academic job, but they could easily end up without an academic job even after staying longer. If the student knows the risks and can make the decision, that’s fine. But the advisor can only benefit, in a self-interested sense, from playing up the odds of success. If the student fails, it’s no skin off the advisor’s nose (again, apart from altruistic preferences).

And consider the flip side: if you’re asked to give advice, and you give a more negative assessment than the recipient has in mind, not only might your advice be rejected, but now the recipient might also dislike you. Even without any other outside incentive to be overoptimistic, this gives an advice-giver the incentive to shade their assessments up. If the person succeeds, they will thank you for it, and if they don’t succeed, they probably won’t hold your advice against you.

Further, being overoptimistic can sometimes be helpful in getting the necessary work done, because it can be motivating. Being realistic can be discouraging. So it is not only individuals who might be overly optimistic in their advice. Sometimes there can be institutional pressures towards optimism (e.g., competitions that want to encourage many high-quality applications though the odds of success are low).

I’m not saying we should always tell people our unfiltered assessment. People sometimes don’t want to hear that. And I’m sympathetic to the view that optimism can be helpful at the societal level. But I haven’t described the most egregious cases I’ve observed. And I do think some people are overly trusting of encouragement and don’t think through the incentives of those giving the encouragement. For example, one should be careful about accepting bad jobs in the hopes they might, with dubious odds, lead to a better job someday. And people are naturally attracted to hearing good news about themselves. It’s easy for people to update on the good more than the bad (e.g., Eil and Rao 2011). Be careful out there and always consider who is giving the encouragement and their (perhaps subconscious) incentives.

This advice post is geared at undergrads seeking reference lettes for grad school, particularly in economics. Graduate students have different considerations, and I think the advice would likely be less applicable to them or already known.

The main thrust of my advice is: consider the other side of the market.

When you apply to a program and you list a reference, that reference will typically be asked to rate you relative to your peers on a variety of dimensions. This is true even when applications say they will solicit reference “letters” – yes, they ask for letters, but they also generally ask some additional questions that are used to weed out candidates.

For example, a typical program might ask recommenders to rate you compared to your peers on “work ethic” or “integrity” or “teamwork”. The options might be something like “top 1%”, “top 10%”, “top 20%”, “top 50%”, “below average” (the precise options vary from program to program). Almost all programs require answering these kinds of questions.

Now suppose you are a student who ended up in the bottom half of the class. You skipped most lectures, so as a prof I would be hard-pressed to know who you are or have anything distinctive to say about you. If all I know about you is how you did on your assignments, and you did poorly, I’ll be forced to tick the “below average” box for most of the questions. That doesn’t help you. Since it doesn’t help you, my policy is to not write reference letters for students who did below the average in my class unless there are extremely unusual mitigating circumstances. Providing reference letters is a repeated game: I don’t want to say things that are false, because then my future references would carry less weight. If reference letter readers can clearly see from your transcript that you did poorly in my class, but I am to rate you highly, I had better have a very good justification to provide for why I am doing so.

Now suppose you are a student who did well overall on all exams and assignments but didn’t actively participate in class. Because you did well, you are compared favourably to your peers, and your application does not get immediately screened out. Depending on the program you are aiming for, that might be enough. But your letter will be pretty uninformative and boring compared to what it could be. All I would be able to say is something like “Student X was in my class. Here is a description of the class. Student X’s highest marks were on the exams/final paper/etc.” It is almost a “mad libs” style, fill-in-the-blank reference. Admissions committee memebers could get most of the information in it from reading your transcript – the letter does not add much value. It’s good enough for many programs (depending on what kinds of programs you are targeting) but maybe not enough for the very best.

A great letter gives readers information they don’t otherwise know about you. If you want such a letter, I need to know more about you. How can I know more about you? Maybe you actively participate in class or office hours. Then I can reference your curiosity or your engagement and say that you always had good questions. Maybe you write a really great term paper. Then I can describe it and why it was great. Maybe you are not a student in my class but someone who provided research assistance. Then I have far more opportunity to observe and comment on your abilities.

I hope this gives you some idea. It’s fun to write letters for great students, but sometimes I need you to help me help you.

Finally, a good reference takes time, and we are all very busy. Please contact your references no later than 3 weeks before a deadline. Ideally, after they have agreed, send them invitations to submit references for your programs all at once so they can do all your references in one batch.

When requesting a reference, it helps to also say something about which types of programs you are planning to apply to. Ideally, attach a list of programs and their deadlines. If you just ask me for a reference without saying where you are planning to apply, in most cases my first question will be to ask you to provide the list of programs and I won’t give you a definite yes or no response until then. (It makes sense: you recommend people for particular things, not to ANY thing in the universe. I don’t want to commit to recommending you for something unrealistic.) So you may as well provide the list or some sense of where you are thinking of applying upfront, and this will also help ensure it gets in by the deadline.

Summing up, my advice is:

• Knowing what you now know about how letters are evaluated, ask for references from professors whose classes you did well in. My personal policy is to not provide reference letters for undergrads who do below-average in my class unless there are truly extenuating circumstances such that I can still give them a high ranking, but other people may set different bars.
• Great letters need something more. You don’t need all your letters to go on and on at length, but the more competitive the program, the more you want your letters to contain information beyond what the admissions committee can tell from the rest of your application materials.
• If you want to make things easy for your profs, request reference letters at least 3 weeks in advance. Specify which programs you are interested in applying to and what their deadlines are. Doing this can also improve your applications: you might get good advice about other places or programs you might consider applying to!

I am looking to hire 1-2 pre-doctoral fellows at the University of Toronto in applied microeconomics. The deadline is March 31, 2025.

Here are descriptions of the different types of work available:

1. Work on the Social Science Prediction Platform

Prof. Stefano DellaVigna (UC Berkeley) and I are seeking a pre-doctoral research fellow to assist with the Social Science Prediction Platform (SSPP) and be an integral part of the team. The Social Science Prediction Platform is an online platform that enables researchers to collect ex ante forecasts of what their studies will find. These forecasts can be useful in a number of applications, some of which are summarized in this Science Policy Forum piece.

Since the SSPP started, we have seen an enormous growth of projects posted on the platform, and we are beginning to analyze data from the first 100 projects.

2. Work on guaranteed income experiments

I have been evaluating several different guaranteed income experiments with Alex Bartik (UIUC), David Broockman (UC Berkeley), Patrick Krause (OpenResearch), Sarah Miller (Michigan), and Elizabeth Rhodes (OpenResearch). Some results have been written up, but there is still more to do, including analysis of children’s outcomes, crime, intrahousehold outcomes, and potential analysis of heterogeneity across sites. Some of this work may include building and estimating models.

This role would also include the chance to work closely with OpenResearch, a nonprofit research organization with a startup mentality, and would provide the opportunity to learn about cutting-edge best practices in research.

3. More open-ended work

There is the opportunity for more open-ended work in evidence-based decision-making and topics in AI. Past pre-docs have worked on existing projects while developing their own projects through active discussion and mentorship.

Pre-docs may work on one or more of the three topics above, depending on fit, availability, and interests.

Eligibility

Applicants must, at minimum, have:

• A bachelor’s degree (or be graduating this year);
• Experience in R or Stata;
• Work authorization in Canada, whether by citizenship or an open work permit. This is strictly required. In practice, citizens of several countries can often obtain an open work permit. In the pre-interview application screener, you will see some questions designed to help you figure out if you might be eligible (though you may need to do further investigation on your own).

The ideal candidate would have:

• A strong quantitative background and potentially a master’s degree;
• Proficiency with more than one programming language;
• Familiarity with ML;
• Previous research experience, such as through past research assistantships or an independent research project;
• An interest in pursuing a PhD in Economics or a related field.

To apply:

To apply, please fill out a pre-interview screener here, including uploading a transcript and CV. Only shortlisted applicants will be contacted for an interview.