Mixing
proxies for false negatives in hiring [closed]
Clash Royale CLAN TAG#URR8PPP
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty margin-bottom:0;
up vote
3
down vote
favorite
I'm at a midsize company that is trying to grow moderately aggressively, so we have a constant, high interview load.
There are many tools that are built around telling you your interviewing funnel, or the percentage of people who you reject after each round of interviewing. This number is useful for quantifying how much time you spend interviewing at each stage, however it doesn't give you much insight into how to fix your hiring process. For instance, if the first phone screen cuts 50 out of 100 candidates, is that a good thing or a bad thing?
When you think about it for a second, you'll see that there's no way to know. Let's call the percentage you cut simply the "negatives" -- you don't know how many of those were "true negatives" (people who truly were a bad match) versus "false negatives" (people you rejected but who would have succeeded in the role you're recruiting for). If 40 of the 50 you cut are false negatives, then that's really bad because you could have hired all those people and you didn't! False positives are bad too, but you can at least approximate that number for the last round of your interviews by looking at how many people quit / are fired.
It's basically impossible to know for certain what your exact false negative rate is given your limited exposure to candidates you reject, but in order to improve the quality and speed of our hiring I would like to approximate it. What information could I ask coworkers, recruiters, managers, candidates, former candidates, etc. for in order to make a good guess? (My only thought so far is to somehow use referrals or references, or maybe look at the places candidates actually went instead of my company.)
interviewing references
asked Mar 8 '16 at 9:00
Dan
1675
closed as too broad by gnat, paparazzo, Jim G., The Wandering Dev Manager, AndreiROM Mar 8 '16 at 20:34
Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.
suggest improvements |Â
up vote
3
down vote
favorite
I'm at a midsize company that is trying to grow moderately aggressively, so we have a constant, high interview load.
There are many tools that are built around telling you your interviewing funnel, or the percentage of people who you reject after each round of interviewing. This number is useful for quantifying how much time you spend interviewing at each stage, however it doesn't give you much insight into how to fix your hiring process. For instance, if the first phone screen cuts 50 out of 100 candidates, is that a good thing or a bad thing?
When you think about it for a second, you'll see that there's no way to know. Let's call the percentage you cut simply the "negatives" -- you don't know how many of those were "true negatives" (people who truly were a bad match) versus "false negatives" (people you rejected but who would have succeeded in the role you're recruiting for). If 40 of the 50 you cut are false negatives, then that's really bad because you could have hired all those people and you didn't! False positives are bad too, but you can at least approximate that number for the last round of your interviews by looking at how many people quit / are fired.
It's basically impossible to know for certain what your exact false negative rate is given your limited exposure to candidates you reject, but in order to improve the quality and speed of our hiring I would like to approximate it. What information could I ask coworkers, recruiters, managers, candidates, former candidates, etc. for in order to make a good guess? (My only thought so far is to somehow use referrals or references, or maybe look at the places candidates actually went instead of my company.)
interviewing references
asked Mar 8 '16 at 9:00
Dan
1675
closed as too broad by gnat, paparazzo, Jim G., The Wandering Dev Manager, AndreiROM Mar 8 '16 at 20:34
Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.
1
woa turned philosophical fast... I'd say, depending on what you are trying to achieve false negatives are probably not that bad. As long as the one candidate you end up with is within the 90% of the ones who apply. You are never going to get THE best (whatever that may even mean). In the end you are trying to fill a single position and want to filter as many as possible. I may be overthinking this.
– Underdetermined
Mar 8 '16 at 9:11
another way to look at this is that the phone screen is showing you flaws in your ad - it attracted people who would never qualify. Therefore next time, if you change your ad, you can expect a different reject rate from your unchanged phone screen.
– Kate Gregory
Mar 8 '16 at 14:19
@Underdetermined: Good point, however because of our current growth we don't think of it as hiring the best person for a single position (because we can almost always hire more people if we find more good people to hire). If that weren't the case, I would be less concerned with false negatives.
– Dan
Mar 8 '16 at 21:19
suggest improvements |Â
up vote
3
down vote
favorite
up vote
3
down vote
favorite
I'm at a midsize company that is trying to grow moderately aggressively, so we have a constant, high interview load.
There are many tools that are built around telling you your interviewing funnel, or the percentage of people who you reject after each round of interviewing. This number is useful for quantifying how much time you spend interviewing at each stage, however it doesn't give you much insight into how to fix your hiring process. For instance, if the first phone screen cuts 50 out of 100 candidates, is that a good thing or a bad thing?
When you think about it for a second, you'll see that there's no way to know. Let's call the percentage you cut simply the "negatives" -- you don't know how many of those were "true negatives" (people who truly were a bad match) versus "false negatives" (people you rejected but who would have succeeded in the role you're recruiting for). If 40 of the 50 you cut are false negatives, then that's really bad because you could have hired all those people and you didn't! False positives are bad too, but you can at least approximate that number for the last round of your interviews by looking at how many people quit / are fired.
It's basically impossible to know for certain what your exact false negative rate is given your limited exposure to candidates you reject, but in order to improve the quality and speed of our hiring I would like to approximate it. What information could I ask coworkers, recruiters, managers, candidates, former candidates, etc. for in order to make a good guess? (My only thought so far is to somehow use referrals or references, or maybe look at the places candidates actually went instead of my company.)
interviewing references
asked Mar 8 '16 at 9:00
Dan
1675
I'm at a midsize company that is trying to grow moderately aggressively, so we have a constant, high interview load.
There are many tools that are built around telling you your interviewing funnel, or the percentage of people who you reject after each round of interviewing. This number is useful for quantifying how much time you spend interviewing at each stage, however it doesn't give you much insight into how to fix your hiring process. For instance, if the first phone screen cuts 50 out of 100 candidates, is that a good thing or a bad thing?
When you think about it for a second, you'll see that there's no way to know. Let's call the percentage you cut simply the "negatives" -- you don't know how many of those were "true negatives" (people who truly were a bad match) versus "false negatives" (people you rejected but who would have succeeded in the role you're recruiting for). If 40 of the 50 you cut are false negatives, then that's really bad because you could have hired all those people and you didn't! False positives are bad too, but you can at least approximate that number for the last round of your interviews by looking at how many people quit / are fired.
It's basically impossible to know for certain what your exact false negative rate is given your limited exposure to candidates you reject, but in order to improve the quality and speed of our hiring I would like to approximate it. What information could I ask coworkers, recruiters, managers, candidates, former candidates, etc. for in order to make a good guess? (My only thought so far is to somehow use referrals or references, or maybe look at the places candidates actually went instead of my company.)
interviewing references
asked Mar 8 '16 at 9:00
Dan
1675
asked Mar 8 '16 at 9:00
Dan
1675
asked Mar 8 '16 at 9:00
Dan
1675
asked Mar 8 '16 at 9:00
Dan
1675
1675
closed as too broad by gnat, paparazzo, Jim G., The Wandering Dev Manager, AndreiROM Mar 8 '16 at 20:34
Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.
closed as too broad by gnat, paparazzo, Jim G., The Wandering Dev Manager, AndreiROM Mar 8 '16 at 20:34
Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.
1
woa turned philosophical fast... I'd say, depending on what you are trying to achieve false negatives are probably not that bad. As long as the one candidate you end up with is within the 90% of the ones who apply. You are never going to get THE best (whatever that may even mean). In the end you are trying to fill a single position and want to filter as many as possible. I may be overthinking this.
– Underdetermined
Mar 8 '16 at 9:11
another way to look at this is that the phone screen is showing you flaws in your ad - it attracted people who would never qualify. Therefore next time, if you change your ad, you can expect a different reject rate from your unchanged phone screen.
– Kate Gregory
Mar 8 '16 at 14:19
@Underdetermined: Good point, however because of our current growth we don't think of it as hiring the best person for a single position (because we can almost always hire more people if we find more good people to hire). If that weren't the case, I would be less concerned with false negatives.
– Dan
Mar 8 '16 at 21:19
suggest improvements |Â
1
woa turned philosophical fast... I'd say, depending on what you are trying to achieve false negatives are probably not that bad. As long as the one candidate you end up with is within the 90% of the ones who apply. You are never going to get THE best (whatever that may even mean). In the end you are trying to fill a single position and want to filter as many as possible. I may be overthinking this.
– Underdetermined
Mar 8 '16 at 9:11
another way to look at this is that the phone screen is showing you flaws in your ad - it attracted people who would never qualify. Therefore next time, if you change your ad, you can expect a different reject rate from your unchanged phone screen.
– Kate Gregory
Mar 8 '16 at 14:19
@Underdetermined: Good point, however because of our current growth we don't think of it as hiring the best person for a single position (because we can almost always hire more people if we find more good people to hire). If that weren't the case, I would be less concerned with false negatives.
– Dan
Mar 8 '16 at 21:19
1
1
woa turned philosophical fast... I'd say, depending on what you are trying to achieve false negatives are probably not that bad. As long as the one candidate you end up with is within the 90% of the ones who apply. You are never going to get THE best (whatever that may even mean). In the end you are trying to fill a single position and want to filter as many as possible. I may be overthinking this.
– Underdetermined
Mar 8 '16 at 9:11
woa turned philosophical fast... I'd say, depending on what you are trying to achieve false negatives are probably not that bad. As long as the one candidate you end up with is within the 90% of the ones who apply. You are never going to get THE best (whatever that may even mean). In the end you are trying to fill a single position and want to filter as many as possible. I may be overthinking this.
– Underdetermined
Mar 8 '16 at 9:11
another way to look at this is that the phone screen is showing you flaws in your ad - it attracted people who would never qualify. Therefore next time, if you change your ad, you can expect a different reject rate from your unchanged phone screen.
– Kate Gregory
Mar 8 '16 at 14:19
another way to look at this is that the phone screen is showing you flaws in your ad - it attracted people who would never qualify. Therefore next time, if you change your ad, you can expect a different reject rate from your unchanged phone screen.
– Kate Gregory
Mar 8 '16 at 14:19
@Underdetermined: Good point, however because of our current growth we don't think of it as hiring the best person for a single position (because we can almost always hire more people if we find more good people to hire). If that weren't the case, I would be less concerned with false negatives.
– Dan
Mar 8 '16 at 21:19
@Underdetermined: Good point, however because of our current growth we don't think of it as hiring the best person for a single position (because we can almost always hire more people if we find more good people to hire). If that weren't the case, I would be less concerned with false negatives.
– Dan
Mar 8 '16 at 21:19
suggest improvements |Â
1 Answer
1
active
oldest
votes
up vote
12
down vote
To estimate the efficiency of your hiring procedure, you could do some blind testing.
In the next hiring wave, take those people who got rated the highest by your interviewers (group A), and invite them for the next phase. But also pick the same amount of people from those which did not pass (group B), which you pick completely at random.
Do not tell the people who do the next interviewing phase which candidates are from which group. Do not tell the candidates either.
Then see how many people pass the next phase(s) of the hiring process. When your interviewing practice is good, then you will have more people from group A passing than people from group B. When you notice no difference between these groups, then your telephone interview practices are obviously flawed.
Another thing you could do (which will only work when your organization is large enough that people don't know each other) is have some of your best people who are already working at you apply to your company "under cover" and see how they perform during the interview process. When they are rejected even though you already know they are a good fit, then something must be wrong.
answered Mar 8 '16 at 9:28
Philipp
20.3k34884
Great idea, I think this would at least tell me when the interviews are inconsistent (which is another important metric). Unfortunately it doesn't tell me as clearly whether a candidate would have succeeded if we had hired them (because my average interview could still be a poor measure of that). I guess if interviews make the right choice, say, 80% of the time then this would still tell me something valuable.
– Dan
Mar 8 '16 at 10:26
@Dan The only way to figure out your true number of false negatives would be to hire every single applicant and try them out for a year. But this might be slightly above your budget.
– Philipp
Mar 8 '16 at 10:48
1
@Philipp Well, it wouldn't be necessary to hire EVERY applicant, just a number that would give you statistically significant results. For a small company, this would still be wildly impractical. If you normally hire one new person per year, to hire 50 as an experiment would be ridiculous. But if you normally hire 1000 people a year, hiring 50 who the interviewers rejected and comparing their performance to those interviewers approved would be a fascinating experiment.
– Jay
Mar 8 '16 at 14:14
suggest improvements |Â
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
12
down vote
To estimate the efficiency of your hiring procedure, you could do some blind testing.
In the next hiring wave, take those people who got rated the highest by your interviewers (group A), and invite them for the next phase. But also pick the same amount of people from those which did not pass (group B), which you pick completely at random.
Do not tell the people who do the next interviewing phase which candidates are from which group. Do not tell the candidates either.
Then see how many people pass the next phase(s) of the hiring process. When your interviewing practice is good, then you will have more people from group A passing than people from group B. When you notice no difference between these groups, then your telephone interview practices are obviously flawed.
Another thing you could do (which will only work when your organization is large enough that people don't know each other) is have some of your best people who are already working at you apply to your company "under cover" and see how they perform during the interview process. When they are rejected even though you already know they are a good fit, then something must be wrong.
answered Mar 8 '16 at 9:28
Philipp
20.3k34884
Great idea, I think this would at least tell me when the interviews are inconsistent (which is another important metric). Unfortunately it doesn't tell me as clearly whether a candidate would have succeeded if we had hired them (because my average interview could still be a poor measure of that). I guess if interviews make the right choice, say, 80% of the time then this would still tell me something valuable.
– Dan
Mar 8 '16 at 10:26
@Dan The only way to figure out your true number of false negatives would be to hire every single applicant and try them out for a year. But this might be slightly above your budget.
– Philipp
Mar 8 '16 at 10:48
1
@Philipp Well, it wouldn't be necessary to hire EVERY applicant, just a number that would give you statistically significant results. For a small company, this would still be wildly impractical. If you normally hire one new person per year, to hire 50 as an experiment would be ridiculous. But if you normally hire 1000 people a year, hiring 50 who the interviewers rejected and comparing their performance to those interviewers approved would be a fascinating experiment.
– Jay
Mar 8 '16 at 14:14
suggest improvements |Â
up vote
12
down vote
To estimate the efficiency of your hiring procedure, you could do some blind testing.
In the next hiring wave, take those people who got rated the highest by your interviewers (group A), and invite them for the next phase. But also pick the same amount of people from those which did not pass (group B), which you pick completely at random.
Do not tell the people who do the next interviewing phase which candidates are from which group. Do not tell the candidates either.
Then see how many people pass the next phase(s) of the hiring process. When your interviewing practice is good, then you will have more people from group A passing than people from group B. When you notice no difference between these groups, then your telephone interview practices are obviously flawed.
Another thing you could do (which will only work when your organization is large enough that people don't know each other) is have some of your best people who are already working at you apply to your company "under cover" and see how they perform during the interview process. When they are rejected even though you already know they are a good fit, then something must be wrong.
answered Mar 8 '16 at 9:28
Philipp
20.3k34884
Great idea, I think this would at least tell me when the interviews are inconsistent (which is another important metric). Unfortunately it doesn't tell me as clearly whether a candidate would have succeeded if we had hired them (because my average interview could still be a poor measure of that). I guess if interviews make the right choice, say, 80% of the time then this would still tell me something valuable.
– Dan
Mar 8 '16 at 10:26
@Dan The only way to figure out your true number of false negatives would be to hire every single applicant and try them out for a year. But this might be slightly above your budget.
– Philipp
Mar 8 '16 at 10:48
1
@Philipp Well, it wouldn't be necessary to hire EVERY applicant, just a number that would give you statistically significant results. For a small company, this would still be wildly impractical. If you normally hire one new person per year, to hire 50 as an experiment would be ridiculous. But if you normally hire 1000 people a year, hiring 50 who the interviewers rejected and comparing their performance to those interviewers approved would be a fascinating experiment.
– Jay
Mar 8 '16 at 14:14
suggest improvements |Â
up vote
12
down vote
up vote
12
down vote
To estimate the efficiency of your hiring procedure, you could do some blind testing.
In the next hiring wave, take those people who got rated the highest by your interviewers (group A), and invite them for the next phase. But also pick the same amount of people from those which did not pass (group B), which you pick completely at random.
Do not tell the people who do the next interviewing phase which candidates are from which group. Do not tell the candidates either.
Then see how many people pass the next phase(s) of the hiring process. When your interviewing practice is good, then you will have more people from group A passing than people from group B. When you notice no difference between these groups, then your telephone interview practices are obviously flawed.
Another thing you could do (which will only work when your organization is large enough that people don't know each other) is have some of your best people who are already working at you apply to your company "under cover" and see how they perform during the interview process. When they are rejected even though you already know they are a good fit, then something must be wrong.
answered Mar 8 '16 at 9:28
Philipp
20.3k34884
To estimate the efficiency of your hiring procedure, you could do some blind testing.
In the next hiring wave, take those people who got rated the highest by your interviewers (group A), and invite them for the next phase. But also pick the same amount of people from those which did not pass (group B), which you pick completely at random.
Do not tell the people who do the next interviewing phase which candidates are from which group. Do not tell the candidates either.
Then see how many people pass the next phase(s) of the hiring process. When your interviewing practice is good, then you will have more people from group A passing than people from group B. When you notice no difference between these groups, then your telephone interview practices are obviously flawed.
Another thing you could do (which will only work when your organization is large enough that people don't know each other) is have some of your best people who are already working at you apply to your company "under cover" and see how they perform during the interview process. When they are rejected even though you already know they are a good fit, then something must be wrong.
answered Mar 8 '16 at 9:28
Philipp
20.3k34884
edited Mar 8 '16 at 9:33
answered Mar 8 '16 at 9:28
Philipp
20.3k34884
answered Mar 8 '16 at 9:28
Philipp
20.3k34884
answered Mar 8 '16 at 9:28
Philipp
20.3k34884
20.3k34884
Great idea, I think this would at least tell me when the interviews are inconsistent (which is another important metric). Unfortunately it doesn't tell me as clearly whether a candidate would have succeeded if we had hired them (because my average interview could still be a poor measure of that). I guess if interviews make the right choice, say, 80% of the time then this would still tell me something valuable.
– Dan
Mar 8 '16 at 10:26
@Dan The only way to figure out your true number of false negatives would be to hire every single applicant and try them out for a year. But this might be slightly above your budget.
– Philipp
Mar 8 '16 at 10:48
1
@Philipp Well, it wouldn't be necessary to hire EVERY applicant, just a number that would give you statistically significant results. For a small company, this would still be wildly impractical. If you normally hire one new person per year, to hire 50 as an experiment would be ridiculous. But if you normally hire 1000 people a year, hiring 50 who the interviewers rejected and comparing their performance to those interviewers approved would be a fascinating experiment.
– Jay
Mar 8 '16 at 14:14
suggest improvements |Â
Great idea, I think this would at least tell me when the interviews are inconsistent (which is another important metric). Unfortunately it doesn't tell me as clearly whether a candidate would have succeeded if we had hired them (because my average interview could still be a poor measure of that). I guess if interviews make the right choice, say, 80% of the time then this would still tell me something valuable.
– Dan
Mar 8 '16 at 10:26
@Dan The only way to figure out your true number of false negatives would be to hire every single applicant and try them out for a year. But this might be slightly above your budget.
– Philipp
Mar 8 '16 at 10:48
1
@Philipp Well, it wouldn't be necessary to hire EVERY applicant, just a number that would give you statistically significant results. For a small company, this would still be wildly impractical. If you normally hire one new person per year, to hire 50 as an experiment would be ridiculous. But if you normally hire 1000 people a year, hiring 50 who the interviewers rejected and comparing their performance to those interviewers approved would be a fascinating experiment.
– Jay
Mar 8 '16 at 14:14
Great idea, I think this would at least tell me when the interviews are inconsistent (which is another important metric). Unfortunately it doesn't tell me as clearly whether a candidate would have succeeded if we had hired them (because my average interview could still be a poor measure of that). I guess if interviews make the right choice, say, 80% of the time then this would still tell me something valuable.
– Dan
Mar 8 '16 at 10:26
Great idea, I think this would at least tell me when the interviews are inconsistent (which is another important metric). Unfortunately it doesn't tell me as clearly whether a candidate would have succeeded if we had hired them (because my average interview could still be a poor measure of that). I guess if interviews make the right choice, say, 80% of the time then this would still tell me something valuable.
– Dan
Mar 8 '16 at 10:26
@Dan The only way to figure out your true number of false negatives would be to hire every single applicant and try them out for a year. But this might be slightly above your budget.
– Philipp
Mar 8 '16 at 10:48
@Dan The only way to figure out your true number of false negatives would be to hire every single applicant and try them out for a year. But this might be slightly above your budget.
– Philipp
Mar 8 '16 at 10:48
1
1
@Philipp Well, it wouldn't be necessary to hire EVERY applicant, just a number that would give you statistically significant results. For a small company, this would still be wildly impractical. If you normally hire one new person per year, to hire 50 as an experiment would be ridiculous. But if you normally hire 1000 people a year, hiring 50 who the interviewers rejected and comparing their performance to those interviewers approved would be a fascinating experiment.
– Jay
Mar 8 '16 at 14:14
@Philipp Well, it wouldn't be necessary to hire EVERY applicant, just a number that would give you statistically significant results. For a small company, this would still be wildly impractical. If you normally hire one new person per year, to hire 50 as an experiment would be ridiculous. But if you normally hire 1000 people a year, hiring 50 who the interviewers rejected and comparing their performance to those interviewers approved would be a fascinating experiment.
– Jay
Mar 8 '16 at 14:14
suggest improvements |Â
1
woa turned philosophical fast... I'd say, depending on what you are trying to achieve false negatives are probably not that bad. As long as the one candidate you end up with is within the 90% of the ones who apply. You are never going to get THE best (whatever that may even mean). In the end you are trying to fill a single position and want to filter as many as possible. I may be overthinking this.
– Underdetermined
Mar 8 '16 at 9:11
another way to look at this is that the phone screen is showing you flaws in your ad - it attracted people who would never qualify. Therefore next time, if you change your ad, you can expect a different reject rate from your unchanged phone screen.
– Kate Gregory
Mar 8 '16 at 14:19
@Underdetermined: Good point, however because of our current growth we don't think of it as hiring the best person for a single position (because we can almost always hire more people if we find more good people to hire). If that weren't the case, I would be less concerned with false negatives.
– Dan
Mar 8 '16 at 21:19