Stop the interviews!

A friend of mine recently got this agenda for an onsite interview:

 

This is after... 

• a 30 min call with the recruiter
• a 45 min coding interview with a lead dev
• a 45 min coding interview with a Backend Sr. SDET
• a 45 min technical with Frontend Sr. SDET

What are we even doing anymore? 7 straight hours?!  12 interviews?! (don't tell me lunch with the CTO isn't an interview 😄)  Plus prep & travel time.  What an incredible waste - of the candidate's time, of the company's time.  Oh, and they didn't offer him the job.

This level of indecision is a red flag!   If a team can't make a decision after 3 - 4 conversations, the candidate should seriously question whether over-analysis permeates the organization.

This is egregious and not far outside the norm. 1  Everyone knows software engineering interviews are broken.  In an industry with some of the best minds our generation, we can't figure this out?

Form over Function

It isn't just length.  I would argue the structure is equally broken.  Interviews don't test the candidate's skill as an engineer; they test their skill as an interviewee:

• Have you practiced Leetcode toy problems, especially binary trees, string splicing, & recursion - all of which you actually use maybe once a year? 2
• Oh, you practiced Leetcode?  That's too bad.  We're going to ask you trivia questions about this specific technology we like.
• Please design Netflix...in an 45 minutes. ...  Candidate spent more than 4.5 minutes on non-functional requirements and didn't say "load balancing"; -10 points.
• Or, maybe we'll send an unpaid take home project that looks suspiciously like the exact problem our company is trying to solve.  Please spend only 4 hours (though everyone is actually spending 8).
• Have you rehearsed your STAR stories and pull out the right one for every occasion?
• Sure you've been job hunting for months because the market is a dumpster fire, but you're up on the latest AI tools, right?
• Oh, and when you've gone through our 7+ interview pipeline, we're going to hire someone we already know because that's the way humans work.

The skills to successfully job hunt have almost nothing to do with the job.  They are marketing, sales, storytelling, & performance art. 

How Did We Get Here?

In discussing this with a friend who is an surgeon, he said he has never had to prove his technical skills in an interview.  Why?  Medical candidates have licensure, continuing education requirements, case history, and (potentially) litigation history.  The interviews are generally just for culture fit.  Similarly, Computer Networking & System Admin roles have long relied on certifications as a signal of candidate technical skill. 

In contrast, Software Engineering has largely ignored certifications, has no uniform system for gauging continuous learning, and the vast majority of candidates' professional work is closed source. 3  Interviewers are flying blind, and candidates must prove themselves anew for each company.  So, we move cautiously, with gauntlet interviews, because bad hires are extremely expensive. 

But, I think the problem goes deeper.... 

We're Not Wired For This 

It is incredibly difficult not to take a job rejection personally 4, but it borders on ridiculous that interviewers & interviewees alike are essentially gauging the person on the other side of the table, in all their complexity, in a one hour performative interaction.  There is precious little information on which to make a decision.  It is taking hints & subtleties from conversation and using inference to make a decision. Pair that with people's biases, dissimilar interests & backgrounds, mood, time of day, flawed questions 5, etc., and it seems closer to dumb luck than science.  Picking the wrong STAR story in the heat of the moment might be the difference between an offer & nothing.

But, yes, there is science.  We try to compensate for biases by structuring interviews carefully: take-home assignments, asking for STAR stories, & posing Leetcode problems and system design prompts, but overall interviews are still terrible predictors of job performance.  (And, AI may have already beaten them).

I posit that perhaps it is not the interview technique that is flawed, but the very idea of interviews - especially of strangers.  We evolved in small tribal groups in which each person had an informed social network & a long history with others in the group. We are terrible at quickly assessing the compatibility of a person with whom we have no mutual connections.

But if you think about it, we already inherently know this, don't we?  Internal candidates & employee referrals are far more likely to receive an offer.  In analyzing Ashby's Talent Trend's Report, Gemini gave the following probabilities for candidates:

Metric	Inbound (Cold)	Employee Referral	Internal Candidate
App-to-Interview	~3%	~40%	~42%
Interview-to-Offer	~27%	~16%	~32%
App-to-Offer (Total)	~0.81%	~6.4%	~13.44%
Hiring Probability	1 in 123	1 in 16	1 in 7.4

Anecdotally, I think candidate priority shakes out something like this:

1. Internal candidates who have worked directly with hiring manager
2. Internal candidates who have worked with hiring team
3. Other internal candidates / Referrals from hiring manager
4. Referrals from within hiring team
5. Other employee referrals
6. Friend-of-a-friend referrals
7. Non-referral external candidates found by internal recruiter
8. Cold applicants 

We know this, but positions are often opened to multiple (or likely all) levels of that hierarchy with countless hours being spent on & by candidates with marginal chances.  Often roles are opened to the public, in the name of "fairness", even when the hiring team already has a candidate in mind.  I find this deeply unethical.

It makes sense that we would hire those we know.  Interviews are an attempt to predict job performance, and direct working experience with a candidate is a goldmine of information in an otherwise information-sparse process.  On the other side of the table, candidates higher in the hierarchy also have an information asymmetry advantage in that they are more likely to be familiar with the problems the team is encountering, the scale at which they work, the personalities of the interviewers, etc.

Millions of years of evolution have wired us to make interpersonal judgements based on existing social connections.  Why pretend that is not the way we hire?

Predicting Criticism

"WAIT!!" I hear you saying.  "What about discrimination, 'good ol' boys' clubs", exclusion of underrepresented groups?  Yes, valid points.  You could even level the criticism that I am advocating for nepotism.

I am not dismissing those concerns; we must account for them.  But, the core idea here is that conducting "fair" interviews across a group of people ranging from friends to complete strangers is inherently flawed.  Perhaps instead of conducting interviews which give the facade of treating everyone as strangers, we make more friends.  That is, we look for ways to connect & work together that are lower stakes, which account for some of our biases, and when the time comes to hire, we are honest about the pool we are actually drawing from.

For many people a job provides their sole income, insurance, and significant social connection.  Modern job offers are a bit like extending a marriage proposal after a handful of dates. 4  Sure, we could plan 4 hour-long dates in a structured way that allows us to better analyze the potential suitors, but that would be as ridiculous as it sounds.  Why do we do it when interviewing?

Where Do We Go From Here?

Some readers may also say something along the lines of "You shouldn't criticize, unless you have a better idea."  This variant of Tu quoque is so common, I'm surprised it doesn't have it's own name (as far as I can tell) - I propose "mandatory solution fallacy".

Solve the problem that has tripped up an entire industry for decades in a single blog post?  😆. Probably not.  I have some ideas, but don't expect a brilliant part 2.  My hope is that we can start having an honest conversation about interviewing and find a better way. 

 

1 In my experience, it takes about 20 hours to properly prep and interview for a role. 

2 Leetcode problems strike me as the silliest of the exercises, because we already have an objective measure for this, at least for college attendees....  Did the candidate take a data structures class from an accredited college or university?  Did they earn a grade you find acceptable?  Done.

3 Yes, some candidates have an extensive GitHub presence, but how many people have you seen get an abbreviated interview pipeline based on that?

4 For the job seekers, a rejection likely has relatively little to do with you as a person, and more to do with the stars aligning with a particular group of people. When you are looking for a job, you want meaningful work (perhaps in a specific industry), interesting work, specific technologies, growth (meaning working on something you don't know well), great company culture, good pay & benefits, etc. It is incredibly hard to tick all of those boxes. Companies have their own set of boxes, so don't be surprised if you get turned down for not ticking all of them. 

5 Many interview questions boil down to "does this person know what I know?".  The most obvious of these are technical trivia questions, but perhaps more problematic are those that ask the candidate to address technical or people problems that the team is facing internally.  Either the team has already figured out the issue (and the candidate will be graded on whether they arrived at the same conclusions the team did), or the candidate is being asked to solve a problem the team cannot, on the spot.  Internal candidates obviously have a massive advantage here.  For everyone else, it's close to dumb luck.

6 Except they aren't marriage proposals are they?  Loyalty is dead.  Neither companies nor employees expect things to last long, especially in tech, so it is "who can make a buck and move on before the other side calls it quits?"  The employee just has far more to lose if it ends unexpectedly.  Long interview pipelines don't fit with the current state of things.