Thomas Stokes
Over the past year, we’ve been collecting data on the current state of UX Research hiring and interview practices — people feel we have room to improve these experiences. Folks use the word “horrible” a lot:
“It’s horrible…
complain to my wife constantly about this…
the UX of job applications is not friendly…” –job seeking participant
This sentiment is shared not only by job seekers but also by hiring managers:
“It was horrible… we got all of the wrong people.” –hiring manager participant
The processes we use to evaluate and select job candidates have significant consequences. For example, we estimate hiring a poor-fitting UXR candidate may cost a team around $36,600 on average. Beyond hiring expenses and productivity losses, interviews can have profound emotional impacts. As one of our participants put it, we can either empower job-seekers or shatter their hopes:
“Perhaps designing hiring process with empathy as a north star might shift UX interview narratives into more empowering experiences, not where dreams and confidence are dashed” –job seeking participant
We wanted to learn how to enhance our hiring and interview practices to benefit both our hiring team and job seekers. Based on our findings, we’ve identified three areas where UX research hiring teams can improve interview experiences and outcomes:
Reducing friction during applications and screening
Using more effective interview techniques
Providing training and resources for interviewers
Early pain with applications and screening
Friction begins in the first phases of job interviews. First, let’s look at common issues faced by job seekers:
Time-consuming: Many expressed frustration with the excessive time required to determine application requirements (e.g., resumes, portfolios), and then produce, assemble, and submit them. Further, they were unsure which materials (e.g., cover letters) would be reviewed by the hiring team.
Ambiguous job descriptions: Vague job descriptions make it difficult to discern meaningful differences between job openings.
Frustrating applications systems: Many expressed frustration with the user experience of systems used to collect applications (e.g., Workday, among others).
Opaque interview loops: Many felt unsure about how many interviews they may have to go through and what they will entail, even after screening calls. This can make it difficult for them to decide whether to apply for a job or continue with the hiring process.
“Ghosting” and lack of feedback: A pervasive frustration was that timely and valuable feedback is rare. They described instances of being “ghosted” after screening calls, or receiving unduly late rejections many months later.
As one interviewee summarized:
“The UX of applying to UX jobs sucks” –job seeking participant
Unfortunately, some of these issues — such as the application system used by your company — are beyond hiring managers’ control. But they can still take steps to improve the applicant experience:
Don’t require any unneeded application materials. Consider removing unnecessary fields (e.g., cover letters) from the application form or indicating they’re not required.
Tailor job descriptions, where possible, to reflect the duties and role the candidate will have on your team. This will help candidates determine if the role is a good fit for them and will reduce the number of applications you need to sort through.
Nudge towards timely feedback within the constraints of your organizational policies. Even a simple message like “you are no longer under consideration” can help provide a sense of closure.
Likewise, during this phase, hiring managers face difficulties screening candidates. We observed that the relationship with a recruiter was directly related to difficulty screening candidates.
If the manager and their recruiter had a well-defined working relationship and shared understanding of the qualities of a strong candidate, the screening process was streamlined and efficient. If the recruiter lacked this understanding, they often presented unsuitable matches, wasting the time and effort of everyone involved. But without a recruiter, the burden of the entire candidate search process fell on the manager.
In short, hiring managers can make their lives easier by:
Helping recruiters to understand the role and ideal candidate profile. Take time to communicate your team’s needs and capabilities.
Clearly defining expectations is key to a mutually beneficial relationship and streamlined recruitment. Outline the role each of you will play in screening candidates (i.e., who does what and when). Include specific expectations for feedback timelines and the number of candidates to be presented at once.
Providing examples of “must-have” and “nice-to-have” candidate qualifications. Include descriptions, examples, and sample questions that the recruiter can use in their interviews.
By following these steps, hiring managers can streamline the recruitment process and make it easier for everyone involved.
Lack of structured techniques
We found six activities common to most UXR interviews: initial phone screens, 1-to-1 interviews with a hiring manager, technical interviews, behavioral interviews, interviews with research team members, and portfolio presentations.
But regardless of their popularity, are they effective in identifying qualified candidates?
Predictive validity measures how accurately an interview technique can predict an applicant’s future job performance. From the large body of research available on the topic, we can find measures for the predictive validity of a technique and compare these measurements to determine which are most effective.
Further, we find that validity increases when we use multiple activities. The benefit of additional activities is nevertheless limited, as we also found that most hiring managers and job seekers prefer 2–3 total phases of interviews. Instead of adding many interview activities, the focus should be on maximizing the effectiveness of each round in the process.
Structure the interview
We found one key opportunity in that many teams are not using structured techniques, which include standardized questions, evaluation criteria, grading rubrics, and scoring techniques. Structured interviews have significantly higher predictive validity than unstructured interviews.
Unfortunately, only 50% of the UX hiring managers we surveyed used rubrics. Other surveys have found similarly low usage of structured techniques, with about 50% using standardized questions, 36% using rubrics, and 22% having scoring systems.
Many teams would benefit from adding structure to their interview practices.
Structure the work sample test
Work sample tests assess a candidate’s ability to perform a job through tasks or projects that mimic in-role responsibilities — and have the highest potential predictive validity of all common interview methods. In UX research interviews, they come in a few different formats.
Portfolio reviews (69% of responses) are most common. They provide a window into a candidate’s past work and communication skills, and are reusable across applications — a major benefit for candidates. Crucially, however, they often lack structure, which can increase bias and decrease validity.
Two less common work sample techniques are take-home tests (28% of responses) and whiteboard exercises (33%). Take-home tests provide a research project prompt and instructions for the candidate to conduct on their own time. Whiteboard exercises assess how a candidate thinks through research problems through on-the-spot scenarios.
Both may be structured and relevant to key job activities — but neither are reusable. Worse, take-home tests require a significant time investment, and whiteboard exercises are more limited in scope and can be stressful.
We suggest structuring portfolio reviews by providing candidates with precise guidance on the duration, format, and expectations for the session. Inform them of the evaluation criteria you will use and the target audience of the presentation portion. Additionally, create standardized questions, evaluation criteria, and scoring rubrics for the hiring team.
Interviewers lack training & resources
The Peter principle is an observation that people get promoted for having skills in their current role rather than the necessary skills for the new position. Without additional training, this can cause problems.
Indeed, most don’t get interview training before assessing job candidates. We found that 64% of UX research hiring managers have not had any form of interview training. Guidelines and job rubrics were among the only internal resources that a majority of hiring managers could access.
This isn’t a unique problem in our discipline; across industries, 61% of hiring teams receive no training. Nevertheless, it is the responsibility of UX Research leaders to seek enablement.
We encourage hiring managers to take ownership of their personal development and find training on effective interviewing techniques. By contacting your organization’s human resources division, you may find that there are untapped resources for your team.
Beyond that, there’s tons of useful information online. Here are resources we recommend:
Nielsen Norman Group on structured interviews: The best UX-specific content we found is Evan Sunwall’s video and article, which provides specific steps for implementing structured interviews.
Free interview guides & question banks: There are plenty of resources & templates for interviews on government websites: Performance
Based Interviewing (PBI) & sample Questions, assessment strategy, structured interviews, job analyses, work samples
Research on interview validity: Want to dive into the research on validity? Start with Schmidt & Hunter (1999) and Salgado & Moscoso (2002).
Work sample tests from other fields: Jacob Kaplan-Moss detailed his experience with using different formats of work sample tests for software engineers in a series of articles.
Pew’s AI hiring report: The Pew Research Center recently released a report on public perceptions of AI in hiring processes. Results show a mix of fears and hopes for AI’s potential impact.
Other hiring reports: Reports detail broad trends across industries and UX-specific changes in hiring and headcounts.
Book recommendation: Work Rules! by Laszlo Bock (former SVP of People Operations) describes his experiences using data-driven techniques to improve Google’s HR practices.
The bottom line
Done poorly, job interviews can have measurable consequences for UX Research teams and powerful emotional impacts on applicants. So it’s crucial to create an effective process that efficiently, effectively, and empathically assesses a candidate’s skills, experience, and fit for the role.
Our research found several pain points during the application and interview stages of UXR job interviews and four key areas where UXR leaders can focus to improve outcomes and experiences:
Help job seekers: Job seekers encounter burdensome application processes. UX leaders can improve candidate experiences by asking for only essential application materials, tailoring their job descriptions, and providing timely feedback.
Make recruiters your partner: UX leaders can lighten their workload and streamline recruitment by communicating their team’s needs and capabilities, clearly defining expectations, and providing examples of candidate qualifications to recruiters.
Introduce structure to interviews: UX research teams underutilize structured techniques in their interviews and portfolio reviews. UX leaders can increase the validity of their interview processes and decrease bias by introducing structure.
Seek enablement and resources: The majority of interviewers have had zero training. UX leaders should make use of existing resources and seek enablement to improve interview skills.
Source: Medium
Original content : https://shorturl.at/cowJO
