There are only 3 interview questions – that is an interesting proposal by Forbes. Brandt would shorten the interview to 3 questions –
1. Can you do the job?
2. Will you love the job?
3. Can we tolerate working with you?
2. Will you love the job?
3. Can we tolerate working with you?
That was easy. I’m going to pass over commenting on more interview questions, and focus on mistakes which we can (all) make during interviews, if we aren’t careful. If we miss a great candidate, then we are missing out on huge opportunity. If we mistakenly hire a bad candidate, we’re digging a hole in our boat. In interviewing 200+ engineers for Microsoft, I cannot go as far as the Codist does, but I do understand Andrew Wulf is saying:
“give me an hour one on one and I can tell more about how a (senior level) programmer will be in the long run than all the coding tests in the world. Why? Good programmers talk like they work when you engage them in details of past projects they spent months on. You can’t fake something you invested a good chunk of your life on. A good programmer …remember(s) how they dealt with some nasty design problem or fixed a complicated bug. Good programmers …have good programming in their DNA, and love to talk about them with a peer. Crappy programmers evade and toss out bullshit or forgot what they did… Programming is not about writing lines of code, it’s about completing projects or shipping applications “
On to the science:
Let’s look at what scientists think about our thinking when we hire. It may be politically correct to say we are blank slates, but in fact science can quickly prove that you are deluding yourself if you don’t think you make cognitive errors in evaluating candidates…you have to focus on that fact, to avoid being trapped in fantasy land….(hello! real world is over here! and it’s messy sometimes.)
First piece of science – Be careful, sometimes we humans can make uninformed decisions or interpret small or irrelevant data. There’s a lot of research cited here that reminds me about this in our hiring decisions, daily interactions, etc. This is a summary of the reading I’ve done recently involving how we make decisions, and how to ensure we make *better* decisions in light of what we know about our own minds. Blink, Sway, How We Decide, and How Doctors Think all summarize some very painful, general truths about the mind. Every interviewer should be conscious of these traits in order to get the best candidates. At the worst: You have a first impression, and it is based on something irrelevant. You only notice things which confirm your mistaken first impressions. Lastly, the questions you ask will make all of this more likely, unless you pay attention. So let’s look at each of these cognative issues as they apply to interviewing:
- First Impressions. Everyone has them. If you think you don’t, then you are wrong. Go back and find your impression of the candidate. The first impression has many components, including the nearly 200 biases enumerated in human thought. As I discussed here, micro-expressions on people’s faces may also influence your opinions, even if they are irrelevant or misguided. People have irrational tendencies while hiring, so ensure a diversity of opinions and experiences in hiring. If not, you will end up with org-wide weaknesses.
- Predictably irrational inputs influence your judgment, Minor changes in words or initial impressions radically change perception of our subsequent experience. A few words or stray expression by people will change your views drastically, or at least pre-dispose you to have a specific feeling later on that day. As Bargh and Williams pointed out of one study: “We’re finding that we have these unconscious behavioral guidance systems that are continually furnishing suggestions through the day about what to do next, and the brain is considering and often acting on those, all before conscious awareness.”
- Here’s another experiment showing how little it takes to prime our minds for positive or negative: Two nearly-identical bios were created in an experiment involving students with one substitude professor. The bio was handed out before class, half to each group of students. The bio of the substitute professor contained either cold or warm, in decribing the professor. Half of the students were given 1 bio, and half given the other. The students all attended the exact same lecture. And, depending on which bio they read, it radically changed students’ rating of the same lecture! Worse, the students recalled specific experiences to validate the positive or negative impressions formed depending on whether they got the ‘warm’ or ‘cold’ bio. Remember it is 2 words. Shading people’s views using words is also know as Priming. Priming is accomplished by using associated words to evoke a specific response.
- In another incredible example, priming groups of students caused groups to walk faster or slower, depending on whether they heard words associated with old age. Naturally, the people hearing ‘old’ words walked much slower.
- In a different example: People were asked to recall their last 2 social security digits – the group with higher social security digits bid MUCH higher for items than people who have low social security digits.
- Finally, our surroundings, impressions, or preconceived notions can have so much influence that almost no one even noticed virtuoso violinist Joshua Bell playing in NYC subway, because he ‘looked like every other musician.’ No one expected high-priced musicians, so no one noticed him. He had sold out a concert hall at 100/seat the night before. No one cared or noticed the quality because no one expected good music in the subway. 6 people stopped in 45 minutes. Watch a summary or read the original or see if you could hear the quality of the performance?
- So the point to remember here is that a lot of very small and irrelevant inputs will influence your views, so pay attention to the specifics of why you are thinking what you are thinking…And challenge other’s opinion for specifics! That helps reduce the priming, and focuses you on the facts of whether the candidate will do a good job.
- Confirmation/Diagnostic Bias – You have a first impression, then you spend the interview grasping information to prove your initial opinion. Change is hard, but it is even harder when you aren’t aware. So pay attention.
- Barbara Walter’s Questions – the past is changing, the future is fuzzy, and “We” have usually done everything.
- Actual hypotheticals? they sound impossible, but they help reduce our failings!
- So the candidate is finished; now how should you decide? You need to keep in mind what we have learned about how groups decide things. Just like biases, if you think YOU aren’t influenced, you are most likely wrong. Group-thinkeffects us all. Here’s the easiest way to see it.
- The Asch conformity experiments prove conformity really sets in when 3 or more people state the same opinion. Hystrically, even when given lines drawn on paper, if 3 actors immediately state the wrong answer, the subject will conform. Hiring is much more subtle than deciding which line is longer. Here’s what’s important about Asch: just 1 person saying any other answer was enough to have the subject ignore the crowd. So, how do we use that fact to our advantage.
- Encourage everyone to state their opinion:Leaders communicate. Reward dissent! Seek diverse attitude and opinion. Force someone into the devil’s advocate role; if someone gets religious about something, put them on the other side. Tell your leaders you expect them to disagree, not seek uniformity. Accept that this takes more time than dictating group-think.
- The US Supreme Court uses a simple rule on case reviews: in the first discussion, no one speaks two times, before everyone speaks once. Thanks to Sway again, for this delicious summary.
You can also hear all of these biases in the review (caka, calibration, annual review, salary review, performance treatment, whatever your company calls it). Here’s some guidance I’ve found useful:
Here’s a great write up about (at least with MSFT) what it means to be going to the “Senior” level of developer – which is level 63 at present, and where top folks usually start to struggle. Up to 63, it’s usually that those folks are so smart that almost anything succeeds, after Senior level, the competition to get the 20% of A ratings, or avoid the 10-20% worst ratings, are hard.
Here’s what I’ve found useful to keep in mind for both annual and mid-year reviews
*Managers should be put in with IC (individual contributors, aka-people who produce work. Otherwise managers stop producing work and start watching work. All managers should be directly involved in producing one or more of the areas they manage. Only exception may be newly promoted managers, who need first 6 months to learn to guide people.
*Who has accomplished the most results for your team – regardless of level, write that person down. Then write down the person regardless of level who has produced the least results all year. Now add one name to the top, but specifically put the lowest level, salary, position, category, into the top performer list and the highest level, salary, etc into the underperforming list. For example, top person’s results OVERALL better be from your highest ranking, or your ranking is screwed up! Next find the lowest paid category and level and put the top person there (sometimes the junior person gets passed over unless you do this!). Also, if you don’t find your highest level/ranked person who produces the least, you’re letting the top folks coast.
*If you have more than 75 people, you will have to meet the curve of your company (or so says the handbooks). But I’ve found some managers apply this to their teams! Like 8 people or 5 people being forced into a curve. NUTS!!! Anyway, I won’t address that further. But the point of the curve is that it breaks out into buckets (some companies use a 9 grid system, some use Welch’s 20-70-10 rule, stickynote stackrank, and some use?? So the point of htis story is this – whatever buckets you end up with, remember this: a person who is on the bubble, or at the margin is going to get pushed in 1 direction or another – this happens, your GM or whatever will do it, or HR will do it, or Diversity scrub will do it. Live with the bubbles moving. It hurts a lot. But in the end, the bubble is a close call. The thing to avoid is making a huge mistake on someone (new, different, tranferring, or just lucky-great year) and MAKE SURE that person is rewarded exactly in the right bucket. the close-calls are OK to miss (you can’t control it all!) but you can make sure every rock star year is rewarded with Rockstar awards.
^If you find yourself tempted to “push up” everyone in hopes of negotiating, you are attending your first review. Or your immediate manager above you SUX. It’s one or the other. Because any manager who cannot link high and low awards to specific results is going to get killed by any decent senior manager. Avoid this temptation, it results in the lowest rewards for the team overall, as once people start picking, it means everyone is suspect!
read: Get Rid of the Performance Review!: How Companies Can Stop Intimidating, Start Managing–and Focus on What Really Matters by Culbert. It is an excellent book about speaking FRANKLY all year long (not at review time only) and the Performance Preview, which is simply genius, and forces us as managers, as senior managers, and employees to all admit what GOOD, BAD, and GREAT look like…in advance!