Dr. Bernoulli gets a job: Mathematics of the Job Search – Faculty Version

I recently found out that the Duke Computer Science department had 404 applicants for the open position in their department. I mentioned that to a CS professor from a different university, and he didn't seem surprised by that number. Moreover, when you think about how many "faculty candidate" lectures there usually are within a CS-like department each hiring season, and you consider that those interviewees are likely a small selection of the total applicants, then 404 starts sounding reasonable.

When there are 404 applicants who each have PhD degrees, publications, and possible post-doctoral or existing faculty appointments, let's also assume that the objective function that each department is maximizing is pretty flat. If you don't like that assumption, then assume we have no prior information, and so we will maximize entropy and assume that each applicant has a 1/404 chance of being picked for the job (in reality, this probability is itself conditioned on whether the state steps in and has a hiring freeze... so the real probability might be closer to 1/1000). So that is a very low number. Can we fight low probability with high volume of applications?

Assume we apply to N schools where the probability of getting an offer is

p = 1/404

at each of them. Then the probability of not getting an offer from each of them is

1 - p = 403/404,

and so the probability of not getting an offer from all of them is

(1 - p)^N = (403/404)^N.

So finally we arrive at the probability of getting an offer from at least one of them, which is

1 - (1 - p)^N = 1 - (403/404)^N.

Hypothetically speaking, let's say you apply to N = 50 such positions. Then you have a

1 - (403/404)^N = 1 - (403/404)⁵⁰ ≈ 11.65%

probability of getting the offer. Of course, if you were paying attention, you remember that p (1/404) is very small in this example. Consequently, the (1 - (1 - p)^N) curve looks linear for a wide region around the origin. So even though you remember your fourth-grade math teacher teaching you that you cannot additively accumulate probabilities (i.e., your probability of getting a job is not (N × p)), in this small-p case, it is a pretty decent approximation. In particular, even with our ostensibly large N, it is the case that

N × p = (50)(1/404) ≈ 12.38%,

which is pretty close to our slightly more dismal 11.65%.

In December, I ran into a woman who just got finished submitting all of her faculty positions. She said she applying to just 10 of them because she was exhausted and figured she was just practicing this round. Setting N = 10 reduces your chances to 2.45%. Having said that, the distribution across the applicant pool is certainly not flat. Her home institution, research, adviser, and other factors make her a very attractive candidate who will likely do well with such a low N... In fact, she was recently interviewed at a university near me (that, again, may have to deal with hiring freezes, etc., in the near future).

Now, in my case... Maybe I should burn my CV and dust off my résumé... I hope I'm not too old and outdated.

Is an SPSS monster like a SAS bunny rabbit?

A friend of mine had a Google Talk status of "Now I'm the SPSS monster" today. Lately, I have picked up the contagious habit of making fun of people who use gooey (GUI) SPSS, and so I responded by e-mail, "Is an SPSS monster like a SAS bunny rabbit?" She responded, "Could be. Or an R-invader." I couldn't resist letting this snowball turn into the avalanche it really could be, and so...

Kick S. Way to JMP on that one and even Z-score. Such a rejoinder makes me want to click away to one of the Minitabs of my browser. Phew, all of this stat talk makes me want to regress back into MATLAB; even if I am still centrally limited there, at least I can feel normal again.

Anyway, I wasn't trying to be mean. If I was, I hope you won't log this transformation and hold it against me later. I'm certain I can transcend and function better in the future; a higher power law need not intervene. Hopefully this hypothesis is correct and you will see some significant change. That should help you restore your confidence.

On a different note, I saw some Monte Carlo tulips at the zoo last weekend; it seems risky to have planted those at this time of the season, but hopefully they will Excel. If they do die, I'm afraid this story will have a heavy tail indeed.

By the way, yesterday for graduate appreciation day, Jessie got a coupon for $1 coffee at the expensive campus Starbucks. With the discount, prices are about normal. I guess there is no such thing as a scale free lunch. Shoot, I'm afraid my coffee has gone cold and is starting to taste a little bit like Poisson.

Well, enough of this. I'm sure if you remove the outlier that is e-mail thread, you'll find that the remaining e-mails are far less skewed and better fit the distribution you have come to expect.

I hope all of your days are better than average! --
Ted

There are parts of that that I'm not that excited about, but overall I'm pretty proud of myself.