As a manager who has usually taken over teams that have been struggling in some way, there is something that I’ve consistently had to do: fix the hiring process. Hiring well is a huge lever for team transformation. Getting it right doesn’t just mean hiring good people (although that part is crucial), but doing so in a way that is time effective for you, your team, and the candidates.
For a small-scale hiring process, you can often take on the process yourself and spend some time fixing it. However, as a leader (especially a new one), you want to be careful of the optics, and creating a perception that you’re bringing in “your people” as replacements. It can help to bring the current team along and make them part of this from the beginning. For a scaled process, you’ll have no choice because you can’t do it alone, so it’s important to determine what your levers are—and how to use them effectively.
When given the opportunity to establish a process, we’re all biased to advocate for one in which we would be successful ourselves. In hiring, this plays out in two main ways: “A” players build monocultures, hiring people just like them, and “B” players hire “C” players, hiring people who won’t threaten them.
In other words, top performers too narrowly define what top performance is, and okay performers hire mediocre people; in both cases, they’re making selections that bolster their own position.
The hiring hot take is a regular feature of every tech conversation. Newsletters. Conferences. Blogs. Twitter. We talk about hiring a lot, because the market is competitive, and hiring well makes a big difference. Hiring effectively goes well beyond the “quality” of people you hire – it sets them up for an experience inline with their expectations, and contributes to – or is detrimental to – your company brand.
Within that, we talk about diversity in hiring. Companies set goals, often publicly, commit to targets that are way in excess of their current demographics, without similar transparency on how they approach inclusion. Meanwhile, under-indexed people receive a volume of inbound interest, often for roles completely different to, or way less senior than the ones they have.
Message from a recruiter I received last week. Note – my LinkedIn profile is set to Colombia – 4,500KM away from this event.
With the volume of information out there, it’s hard to pick through what’s good and what’s not, what’s correlation and what’s cause. Much of the information is situational – what works in one context may not work in another – and conflicting.
Meanwhile, hiring processes have an information imbalance that makes learning from them hard. You know how the people you hired performed – after a long lag time – you don’t know how the people you didn’t hire would have done. You know how some of the people who applied found you, but you don’t know anything about the people who looked at your job posting and decided “nope”, or the ones who never found you at all.
In the product development process, we have a process for learning about the people we are trying to reach. User research. It’s something we do a lot of at Automattic, as we aim to understand the people we hope to serve with our products. It informs our roadmaps and prioritization, the way we present things, and how we talk about them.
To that end, as we reviewed our hiring process, we realized that the demographics of people we attract to apply are not inline with the demographics of the people we hope to hire. Whilst we have implemented a strong focus on metrics, and made certain adjustments, we’ve not seen the improvements we want. If this was a product, we would go to our users and ask them – so why not do the same here?
To that end, we are kicking off a user research project, to better understand the ways that people think about the process of finding a new job. Like all our user research, it’s compensated. Like everything we do, we share it openly – so whilst we will use the results to inform our process, we will also be sharing a public write up of the things we learned.
For our initial research, we’re looking for women and non-binary people (trans/cis/gnc) who may experience similar gender discrimination in the workplace, who have multiple years of experience in a software development role. If you’re open to participating, please fill in some information in our pre-screening form.
Towards the end of 2017, we reopened hiring on the mobile team at Automattic which had been shut since the start of the year as we got things in order. I think of this as phase 1 of hiring on the team – nailing the basics of the process, and making some progress with diversity numbers.
An up front note about diversity: Diversity is more than gender, and gender is not binary. However whilst acknowledging the limitations and flaws, we can use gender as metric as to how our hiring process is doing with respect to inclusion. We can use it as an indicator for diversity at every phase in the process, and use that to identify where we can improve the process for everyone.
For reference, I was the first woman on the team, which was ~24 people when I joined at the end of 2016. When we opened hiring, there were two women, although both in leadership positions – this helps a lot.
The TL;DR of how our funnel looked at the end of phase 1 is this. You can see that using our raw (and flawed) metric, the diversity improves at every step in the process. This makes sense given that data shows women hold themselves to higher standards when applying for jobs.
Q4 2017
3 people
Q1 2018
5 people
Q2 2018
2 people
Q3 2018
5 people
Q4 2018
1 person
Revamping the Process
Whilst we didn’t change any of the steps in the process, we did revisit each one and improve it.
We revisited our job postings to appeal to a broader spectrum of people, emphasising impact and collaboration. The job postings now score 95 and 96 in Textio.
We diligently posted a hiring stats update each month (the way we review all our projects on the team regularly!), breaking down progress at each step of the process. Monthly was a good cadence for this – it allowed us enough data to make good decisions, but short enough timeframes to meaningfully iterate.
We standardized code tests with checklists of what we were looking for, what was necessary, what was nice to have. This also helps us give better feedback.
At Automattic, everyone who is hired does a (paid) trial project where we work together to determine mutual fit. We treated our trial projects as a pre-onboarding. Every new hire has completed a shippable feature and has meaningful interactions with at least three people on the team.
Previously, onboarding had been inconsistent and hit-or-miss. We got serious about onboarding, identifying good first projects and teams, and working to make sure every new hire felt welcome and set up to succeed.
Successes
31% of our new hires are women.
We added teammates in six new countries: Chile, Turkey, Germany, Serbia, Hong Kong, and Ireland.
63% of new hires speak English as a second language.
We added three languages to the ten already spoken on the team (helpful for internationalization!): Hebrew, Serbian, and Polish.
Outreach:
WordCamps – we presented at WordCamps in Montevideo, Montreal, and Taipei, as well as WCEU and WCUS.
People from the team also presented at various local meet-ups and ThatConference.
Whilst we’re pleased with the progress we made in phase 1, it’s clear we have more work to do. 2019 marks the start of phase 2, where we will be focusing on:
Racial diversity.
Under-represented geographies based on our user base: APAC and Brazil in particular.
LGBT.
Gender diversity.
We’re also keen to take advantage of the openness that working in Open Source allows – so we’re supporting our teammates in sharing more about their work online and IRL at events.
If you liked this post, you might also like being part of our team: we’re hiring.
I did an interview with GitPrime about hiring and onboarding. I really like how it came out, and I hope you like it too!
“We talk about how engineers hate process,” says Cate Huston, Mobile Lead at Automattic, “but here’s my theory: engineers often love process, but when they love it, they call it culture.”
It’s a classic example of what you do being more important than what you say. Of course, put all the important stuff in your onboarding materials—but also conduct your interviews and onboarding in a way that’s honest to the culture you foster every day.
“This is foundational,” Huston says. “Think about the kind of culture you want to build on the team, then put in processes that back that up — and talk about it openly.”
In this interview, Huston shares how she uses her hiring and onboarding process to develop the culture she wants to see at Automattic — and create momentum even before individuals have joined the team.
The receptionist test: how does someone treat the receptionist?
Alternative: a person with power or authority is someone’s first interaction with a company, e.g. by staffing the front desk. How are they treated (i.e. do people assume they are the receptionist).
In the modern era there are three ways to pass the receptionist test.
Look the person up, discover they are not actually the receptionist.
Ask that person anything about themselves or what they do, discover they are not actually the receptionist.
Treat everyone you encounter with respect.
Personally, I prefer 3 or 2 to 1. But it’s amazing how many people don’t pick any of them.
Caveat: Diversity is more than gender. I’ve used gender in some of these examples because I have enough anecdotal data to support these theories wrt to gender but I don’t want to extrapolate beyond that. In general my policy is to test and measure women because we can actually have data for that, but then follow the same strategies for all under-indexed groups.
We talk about “Diversity and Inclusion” but perhaps it should be “Inclusion and Diversity” because inclusion needs to come first. Don’t hire people into an environment they can’t be successful in. On a practical level, it’s a waste of everyone’s time. On a human level, it’s harmful.
Inclusion
Inclusion is about how welcome people will feel if they are there. Consider if you have to be a certain “type” of person (outgoing? heavy drinking? hyper-competitive?) to be successful on the team. Are these characteristics really necessary? Would your team benefit from people who are quieter and more thoughtful, more collaborative? Would those people feel welcome – and able to be successful – if you hired them?
If you realise your team is hyper-competitive, for example, you might want to think about how that is encouraged by the hiring process and the environment you have on the team. How much of that do you have control over? If it’s created by your promotion process, can you influence it?
A good rule for inclusion pre-work to diversity is to stop doing things you would have to change if the demographics of your team better reflected the demographics of the world. If you find yourself watching interactions or jokes and thinking they wouldn’t be okay if there were women / people of color / lgbt / … people on the team… maybe shut that stuff down now if you ever want to have women / people of color / lgbt / … people on the team.
On-boarding
What is your on-boarding process like? How long does it take for someone to be ramped up and productive? What kind of help and support do you give them?
Some things to consider:
It’s much easier to onboard someone well than to fix it later.
Instead of considering where you most need people, ask where on the team can we most support a new person.
Juniors
It’s tempting to improve diversity by hiring juniors. I think the ability to onboard a junior engineer is a measure of the health of a team. Be honest – is your team really healthy enough for one?
Some things to consider:
For women at least anecdotally, the first job seems to be a huge predictor of whether they will stay in tech.
If you care about D&I, you will be mindful of the compound effects to the individual of screwing up here.
A lot of Diversity as Performance Art is this PR exercise of women who are known for working at companies whilst female. This works to a certain extent, I think particularly for hiring more junior folk. For a more meaningful and sustained impact, look for ways to give under-indexed folk more recognition for their actual work – what you actually pay them for – being an awesome engineer, or product manager, or designer, or whatever it is they do.
This works more generally, too. This is one of my favourite comments on a recent launch post from my team.
Mentoring
These may not be people you will hire now, but you might hire them later. Either way, people talk. As my friend Julia put it:
In some ways, what Julia is talking about is mentoring – proactively building relationships and supporting people makes you someone that people want to work with when it’s the right time. It’s also something that makes them suggest to their friends to look at, too.
Personally I have somewhat mixed feelings about mentoring which I’m not going to get into here. However I’ve found that making myself visible in the community and offering some amount of mentoring definitely 1) makes people willing to circulate job postings for my team in their network and 2) generates connections that may result in working together in the longer term.
Job Postings
Use Textio to refine your job posting and ensure you’re not using male-coded language. Be honest, but aspirational (but not delusional). E.g. if you want more collaboration on the team then emphasise collaboration (but first make sure you have some signs of healthy collaboration).
Targeted Outreach
Consider where you are placing job ads. If you are looking to recruit under-indexed folk, skip Hacker News and look for places where under-indexed people are more likely to read.
With Technically Speaking we estimate our audience is at least half women. We also work really hard to be inclusive of other under-indexed groups. There are job posting sites with 10x or more reach total – but nowhere close to the level of reach we have with under-indexed folk.
If you’re looking at events as a way to recruit consider things like, whether they have a code of conduct, what the representation is like on stage, and off stage (whether they offer diversity scholarships is a good question to ask).
If you start evaluating inclusion and representation as you evaluate how to spend your recruitment budget, you’ll likely make different choices on how to spend it.
Specific Outreach
The more senior you go, the more you can expect to have to reach out directly to people. I don’t want the to be taken as a sign that you should hire people you know. More that for senior women my observation is that they are likely to go and work for someone they know.
Work on your own network and make yourself available. Follow more under-indexed folk on Twitter even after you discover they are not just offering an education but being normal human beings with varied interests. Make an effort to be more involved in communities where there is better diversity and more effort for inclusion. E.g. choose “welcoming Javascript evening with soft drinks and childcare” over the “brogramming with beer” event. Choose the Slack community with a strong – and enforced – Code of Conduct over the one which is fine, except for that channel, and that one, oh wait is there really a channel for…
Hiring
Now you’re hiring! Yay!
The good news is that whatever your applicant pool looks like by gender, your next step should have a better ratio – because women are much more likely to self-select out of roles they are “not qualified” for and men are more likely to have a go.
Next, consider context. One thing I care about a lot in hiring for my current team is that people have experience with complex applications over longer time-frames. This can be a hard thing to have if you’ve mainly worked in consultancy – which is often the case for people in developing countries. Aim to be equitable rather than equal. If someone comes from a place with less opportunity, factor that in as you evaluate them.
Prioritise in your queue. Some hiring processes are designed to make people “prove they want it” but anything that selects for that will select for people for whom failure is safer – so, white men. Be prepared to be a bit more pro-active and a bit more on top of the process for under-indexed folk – they’re much less likely to chase you.
Showcase existing diversity. Volunteer – naturally – information that will make it clear that people who aren’t white men can be successful in the environment (do this regardless of the perceived gender and race of the interviewee). As they ask you questions about projects or team organisation choose your examples.
Consider how inclusive they would be to others. One of the things I find interesting as I interview is the people (well: men) who are rude to me as part of the process. Whilst at the Conglomerate I would often see men who had “one bad interview” with a woman getting let through anyway, in general I now try to work with people who consider that a deal breaker. Pay attention to their language, how they interact with under-indexed folk on the team, and how they react to inclusive examples.
Make your process welcoming. There’s a lot of discussion about good hiring processes. The bad news is that we are constrained to design systems we will be successful at, so it’s impossible to discuss these things except from a place of bias. This is why most discussions on these topics are not that helpful (this also applies to promotion systems). The good news is that you get most of the benefit by making a conscious effort throughout not to be an asshole. Be kind to people, clear, and respectful of their time. Insist on this from everyone involved in the process. It really goes a long way.
Factor out anxiety. In general, anxiety in the candidate is just noise in the system – hopefully the environment they end up working in is not one that causes them to live in a state of panic. Make an effort to be understanding of anxiety and to reduce it where possible, and you’ll get a much more useful idea of how they operate and how they would fit into your team.
This Seems Like a Lot of Work
It is. But it doesn’t get easier with time. You may as well start today.
I spend more time than I thought I would thinking about technical interviews lately, because something I’ve been doing is conducting technical interviews for a few startups including Glowforge (they’re hiring!). Since March I’ve done ~36 interviews.
Designing hiring systems is hard because you only have partial data – specifically you don’t know who your false negatives were. I read the articles about how we need to completely redesign hiring systems and how the technical interview is a terrible thing and think there’s merit to a lot of what is said. But ultimately I think there are two main problems with technical interviews (by which I mean – you write code together): 1) a lot of interviewers are bad, and 2) people think you learn more than you do.
We should start with the distinction between good questions and bad questions. Bad questions tell you essentially nothing. If you ask someone to code a solved computer science problem on a white board all you learn about is whether they have memorised the solution to that particular problem and can do it in a state of stress.
A good technical question is one with multiple answers, and tradeoffs. It is a problem large enough that it cannot be solved by a single call to a library function. It requires some thought and discussion.
Once you are asking a good question you can start to get a sense of:
How does someone approach a problem? Do they dive in and start writing code or do they think about things first?
How do they break it down?
Can they discuss tradeoffs? Consider how different inputs would change?
Do they understand what the benefits of different data-structures are? (Different from coding their own version – data structures are foundational and different ones are suitable for different problems).
But I think the most important question I aim to answer in each interview is: do they listen?
I don’t think you learn that much from someone who knows the answer, writes it down, and that is pretty much the extent of your communication. Maybe you’ve learned they are pretty smart. That they wrote code when they knew exactly what to do quickly.
But you don’t know how they will respond to a problem they don’t immediately know the answer to. And unless it’s a job they are going to be incredibly bored in, that’s a pretty important thing to learn about them.
When they can’t immediately solve something you can start to see:
Can they look at their own work objectively and see problems?
Do they listen to feedback?
The other question I aim to get a sense of is: How do they respond to new information? One approach I use for this is “unreasonable product manager”, which means I change the requirements fundamentally. I want to see how they incorporate this information into what they have.
Those are some things I think you can learn, and to be clear – I don’t always get a sense of all of these in each interview, and I’ll note that accordingly in my feedback. However, there are some glaring omissions that are clearly relevant in the day to day work of software engineering.
How do they function in the last 20% of shipping a project (which can be pretty boring)?
How do they work on a codebase of significant scope?
Can they work with legacy code or will they insist on rewriting it all?
Can they prioritise?
Do they give thoughtful code reviews?
Will they be a good mentor to other developers?
Will they take direction? (This is partially covered above, but more in a “may show up if they really won’t” way).
Will they adapt to the processes you have? (E.g. code coverage, testing standards).
It’s important to recognise the limitations of these interviews – as an individual, not just as an organisation – because part of being a good interviewer, of trying to be a good interviewer, is not to give feedback on aspects you really don’t know anything about, and not to delude yourself that you do.
Cookie Consent
We use cookies to improve your experience on our site. By using our site, you consent to cookies.
.jpg)
