How can one mitigate cheating on a pre-interview technical test?

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP





.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty margin-bottom:0;







up vote
14
down vote

favorite
1












One strategy to screen out inappropriate candidates in a technical field is to give them a programming test, and only invite candidates who perform well.



For example, if I wanted to employ a programmer, I might send them a test from codility.com.



My question is:



Without supervising the entire test by webcam, how can I know that the candidate who completed the test is the same person applying for the vacancy?



Similarly, are there other ways of cheating this test, and can they be mitigated?



Obviously, we can weed out the non-programmers at the next stage - if they can't support the answers on the test, obviously it wasn't them - but I'd like not to have to waste the time phone-interviewing a possible cheater in the first place



This question is somewhat related to: Preventing cheating in a phone interview, but the notable difference is that I'm definitely not asking about cheating on trivia-type questions here, I'm referring to in-depth programming knowledge. Given the nature of a remote programming test, Googling
for answer strategies is acceptable and encouraged.







share|improve this question


















  • 15




    Weeding them out with follow-up questions in the next stage is the best approach I've found.
    – Iain Galloway
    Jan 30 '14 at 14:46






  • 31




    Don't subject candidates to pre-interview technical tests.
    – DA.
    Jan 30 '14 at 16:11






  • 7




    @yochannah Just to offer some of my own experience: I hate it when companies make me do the technical stuff before interviewing. At least give a phone screen first. It gives them a chance to list any concerns about the position (is it the work they really want to do? is the employee way too expensive for you even if they excel at the technical assessments?) You're pretty much always going to save yourself money and time by just doing a quick 20 minute cursory "get to know you" phone chat before the tech stuff, and that small filter will help weed out at least some potential cheaters too.
    – ely
    Jan 30 '14 at 19:11







  • 3




    Maybe you can help us -- and your job candidates -- if you explain even more carefully what constitutes "cheating" and what doesn't. You did mention other people doing the test in place of the candidate (cheating) and searching for help on google (encouraged). I've done such tests and i've spent time wondering about what is acceptable or not. BTW, if you do choose to clarify this, please do it in the question; all these comments can get deleted -- especially when folks start having debates :-)
    – Martin F
    Feb 1 '14 at 5:55






  • 4




    Is this really so much of a problem that you need to change your interview process? I mean, are there really so many candidates "cheating"? What percentage of your candidates do you suspect of cheating, or have you discovered were falsely representing themselves?
    – user70848
    Mar 3 '16 at 19:40
















up vote
14
down vote

favorite
1












One strategy to screen out inappropriate candidates in a technical field is to give them a programming test, and only invite candidates who perform well.



For example, if I wanted to employ a programmer, I might send them a test from codility.com.



My question is:



Without supervising the entire test by webcam, how can I know that the candidate who completed the test is the same person applying for the vacancy?



Similarly, are there other ways of cheating this test, and can they be mitigated?



Obviously, we can weed out the non-programmers at the next stage - if they can't support the answers on the test, obviously it wasn't them - but I'd like not to have to waste the time phone-interviewing a possible cheater in the first place



This question is somewhat related to: Preventing cheating in a phone interview, but the notable difference is that I'm definitely not asking about cheating on trivia-type questions here, I'm referring to in-depth programming knowledge. Given the nature of a remote programming test, Googling
for answer strategies is acceptable and encouraged.







share|improve this question


















  • 15




    Weeding them out with follow-up questions in the next stage is the best approach I've found.
    – Iain Galloway
    Jan 30 '14 at 14:46






  • 31




    Don't subject candidates to pre-interview technical tests.
    – DA.
    Jan 30 '14 at 16:11






  • 7




    @yochannah Just to offer some of my own experience: I hate it when companies make me do the technical stuff before interviewing. At least give a phone screen first. It gives them a chance to list any concerns about the position (is it the work they really want to do? is the employee way too expensive for you even if they excel at the technical assessments?) You're pretty much always going to save yourself money and time by just doing a quick 20 minute cursory "get to know you" phone chat before the tech stuff, and that small filter will help weed out at least some potential cheaters too.
    – ely
    Jan 30 '14 at 19:11







  • 3




    Maybe you can help us -- and your job candidates -- if you explain even more carefully what constitutes "cheating" and what doesn't. You did mention other people doing the test in place of the candidate (cheating) and searching for help on google (encouraged). I've done such tests and i've spent time wondering about what is acceptable or not. BTW, if you do choose to clarify this, please do it in the question; all these comments can get deleted -- especially when folks start having debates :-)
    – Martin F
    Feb 1 '14 at 5:55






  • 4




    Is this really so much of a problem that you need to change your interview process? I mean, are there really so many candidates "cheating"? What percentage of your candidates do you suspect of cheating, or have you discovered were falsely representing themselves?
    – user70848
    Mar 3 '16 at 19:40












up vote
14
down vote

favorite
1









up vote
14
down vote

favorite
1






1





One strategy to screen out inappropriate candidates in a technical field is to give them a programming test, and only invite candidates who perform well.



For example, if I wanted to employ a programmer, I might send them a test from codility.com.



My question is:



Without supervising the entire test by webcam, how can I know that the candidate who completed the test is the same person applying for the vacancy?



Similarly, are there other ways of cheating this test, and can they be mitigated?



Obviously, we can weed out the non-programmers at the next stage - if they can't support the answers on the test, obviously it wasn't them - but I'd like not to have to waste the time phone-interviewing a possible cheater in the first place



This question is somewhat related to: Preventing cheating in a phone interview, but the notable difference is that I'm definitely not asking about cheating on trivia-type questions here, I'm referring to in-depth programming knowledge. Given the nature of a remote programming test, Googling
for answer strategies is acceptable and encouraged.







share|improve this question














One strategy to screen out inappropriate candidates in a technical field is to give them a programming test, and only invite candidates who perform well.



For example, if I wanted to employ a programmer, I might send them a test from codility.com.



My question is:



Without supervising the entire test by webcam, how can I know that the candidate who completed the test is the same person applying for the vacancy?



Similarly, are there other ways of cheating this test, and can they be mitigated?



Obviously, we can weed out the non-programmers at the next stage - if they can't support the answers on the test, obviously it wasn't them - but I'd like not to have to waste the time phone-interviewing a possible cheater in the first place



This question is somewhat related to: Preventing cheating in a phone interview, but the notable difference is that I'm definitely not asking about cheating on trivia-type questions here, I'm referring to in-depth programming knowledge. Given the nature of a remote programming test, Googling
for answer strategies is acceptable and encouraged.









share|improve this question













share|improve this question




share|improve this question








edited Dec 22 '17 at 17:02









Volker Siegel

445514




445514










asked Jan 30 '14 at 11:47









yochannah

4,21462747




4,21462747







  • 15




    Weeding them out with follow-up questions in the next stage is the best approach I've found.
    – Iain Galloway
    Jan 30 '14 at 14:46






  • 31




    Don't subject candidates to pre-interview technical tests.
    – DA.
    Jan 30 '14 at 16:11






  • 7




    @yochannah Just to offer some of my own experience: I hate it when companies make me do the technical stuff before interviewing. At least give a phone screen first. It gives them a chance to list any concerns about the position (is it the work they really want to do? is the employee way too expensive for you even if they excel at the technical assessments?) You're pretty much always going to save yourself money and time by just doing a quick 20 minute cursory "get to know you" phone chat before the tech stuff, and that small filter will help weed out at least some potential cheaters too.
    – ely
    Jan 30 '14 at 19:11







  • 3




    Maybe you can help us -- and your job candidates -- if you explain even more carefully what constitutes "cheating" and what doesn't. You did mention other people doing the test in place of the candidate (cheating) and searching for help on google (encouraged). I've done such tests and i've spent time wondering about what is acceptable or not. BTW, if you do choose to clarify this, please do it in the question; all these comments can get deleted -- especially when folks start having debates :-)
    – Martin F
    Feb 1 '14 at 5:55






  • 4




    Is this really so much of a problem that you need to change your interview process? I mean, are there really so many candidates "cheating"? What percentage of your candidates do you suspect of cheating, or have you discovered were falsely representing themselves?
    – user70848
    Mar 3 '16 at 19:40












  • 15




    Weeding them out with follow-up questions in the next stage is the best approach I've found.
    – Iain Galloway
    Jan 30 '14 at 14:46






  • 31




    Don't subject candidates to pre-interview technical tests.
    – DA.
    Jan 30 '14 at 16:11






  • 7




    @yochannah Just to offer some of my own experience: I hate it when companies make me do the technical stuff before interviewing. At least give a phone screen first. It gives them a chance to list any concerns about the position (is it the work they really want to do? is the employee way too expensive for you even if they excel at the technical assessments?) You're pretty much always going to save yourself money and time by just doing a quick 20 minute cursory "get to know you" phone chat before the tech stuff, and that small filter will help weed out at least some potential cheaters too.
    – ely
    Jan 30 '14 at 19:11







  • 3




    Maybe you can help us -- and your job candidates -- if you explain even more carefully what constitutes "cheating" and what doesn't. You did mention other people doing the test in place of the candidate (cheating) and searching for help on google (encouraged). I've done such tests and i've spent time wondering about what is acceptable or not. BTW, if you do choose to clarify this, please do it in the question; all these comments can get deleted -- especially when folks start having debates :-)
    – Martin F
    Feb 1 '14 at 5:55






  • 4




    Is this really so much of a problem that you need to change your interview process? I mean, are there really so many candidates "cheating"? What percentage of your candidates do you suspect of cheating, or have you discovered were falsely representing themselves?
    – user70848
    Mar 3 '16 at 19:40







15




15




Weeding them out with follow-up questions in the next stage is the best approach I've found.
– Iain Galloway
Jan 30 '14 at 14:46




Weeding them out with follow-up questions in the next stage is the best approach I've found.
– Iain Galloway
Jan 30 '14 at 14:46




31




31




Don't subject candidates to pre-interview technical tests.
– DA.
Jan 30 '14 at 16:11




Don't subject candidates to pre-interview technical tests.
– DA.
Jan 30 '14 at 16:11




7




7




@yochannah Just to offer some of my own experience: I hate it when companies make me do the technical stuff before interviewing. At least give a phone screen first. It gives them a chance to list any concerns about the position (is it the work they really want to do? is the employee way too expensive for you even if they excel at the technical assessments?) You're pretty much always going to save yourself money and time by just doing a quick 20 minute cursory "get to know you" phone chat before the tech stuff, and that small filter will help weed out at least some potential cheaters too.
– ely
Jan 30 '14 at 19:11





@yochannah Just to offer some of my own experience: I hate it when companies make me do the technical stuff before interviewing. At least give a phone screen first. It gives them a chance to list any concerns about the position (is it the work they really want to do? is the employee way too expensive for you even if they excel at the technical assessments?) You're pretty much always going to save yourself money and time by just doing a quick 20 minute cursory "get to know you" phone chat before the tech stuff, and that small filter will help weed out at least some potential cheaters too.
– ely
Jan 30 '14 at 19:11





3




3




Maybe you can help us -- and your job candidates -- if you explain even more carefully what constitutes "cheating" and what doesn't. You did mention other people doing the test in place of the candidate (cheating) and searching for help on google (encouraged). I've done such tests and i've spent time wondering about what is acceptable or not. BTW, if you do choose to clarify this, please do it in the question; all these comments can get deleted -- especially when folks start having debates :-)
– Martin F
Feb 1 '14 at 5:55




Maybe you can help us -- and your job candidates -- if you explain even more carefully what constitutes "cheating" and what doesn't. You did mention other people doing the test in place of the candidate (cheating) and searching for help on google (encouraged). I've done such tests and i've spent time wondering about what is acceptable or not. BTW, if you do choose to clarify this, please do it in the question; all these comments can get deleted -- especially when folks start having debates :-)
– Martin F
Feb 1 '14 at 5:55




4




4




Is this really so much of a problem that you need to change your interview process? I mean, are there really so many candidates "cheating"? What percentage of your candidates do you suspect of cheating, or have you discovered were falsely representing themselves?
– user70848
Mar 3 '16 at 19:40




Is this really so much of a problem that you need to change your interview process? I mean, are there really so many candidates "cheating"? What percentage of your candidates do you suspect of cheating, or have you discovered were falsely representing themselves?
– user70848
Mar 3 '16 at 19:40










9 Answers
9






active

oldest

votes

















up vote
13
down vote



accepted










I don't believe you can come up with a control that is both reasonably effective and less time consuming than bringing the candidate in and verifying that they can support answers.



My approach would be to make the stakes high enough that cheating is also shooting yourself in the foot. Have the candidates sign off on an honor code that clearly states that if they are found cheating, both the person who showed up for the interview and the name & contact details of the resume provided will be blacklisted from company searches for 5 years. It's reasonably easy to keep a listing of cheaters - their pictures, their aliases, and their phone numbers and email addresses. Then use that as criteria #1 in screening future candidates.



Yes - it's still possible to cheat your way around that - new phone number, new alias, etc. But at some point, your ability to weed out people with no skills will be clear enough that it doesn't seem worth the work.



In all honesty, if this is happening for any significant percentage of the time - you may have to give up on the pretest. And if it isn't a significantly frequent occurrence, figure the time wasted on verifying the candidate in person is going to be more minimal than all the tech work of doing more.






share|improve this answer
















  • 7




    Note that a list like that is illegal in Europe.
    – Juha Untinen
    Sep 6 '15 at 21:28






  • 3




    Hmm, can you provide information on the law that makes it illegal? I'm not sure why "cheating a company" and then keeping a record of the cheaters is somehow illegal.
    – Nelson
    Mar 4 '16 at 7:28






  • 3




    In France, at least, any computerized information you keep about individuals shall be accessible to the individual. So the list would not be forbidden, but the candidate is legally allowed to know wether a computer file blacklists him. Don't know elsewhere.
    – gazzz0x2z
    Mar 4 '16 at 11:00






  • 1




    In the US there is some level of precedent - I worked in a company where if you declined a job offer, we wouldn't make another for X number of years. The goal was to avoid wasting company resources with someone who was fishing for opportunities. Given that we didn't have the information handling laws of Europe, there was no concern there.. but I don't think anyone have been concerned at disclosing to the individual "sorry you are ineligible because you turned down a job offer on Y date"
    – bethlakshmi
    Mar 4 '16 at 15:57






  • 1




    @Nelson Adding some further information concerning France: the blacklist cannot be shared beyond the scope of recruiting. It would be illegal for example to disclose such information to anyone not involved in the recruitment process at the company. Everything would have to be informed to the CNIL and respect the measures to secure the access to such information (cryptography, etc).
    – Adam Smith
    Dec 27 '17 at 0:37


















up vote
24
down vote













Just as the important thing about a whiteboard programming test is the conversation it facilitates, the important thing about a take home test can be the conversation it facilitates.



Tell them to keep the code handy because there will be a conversation about it afterwards. Then, for those whose submissions are good enough for a phone or in-person interview, you can ask stuff:



  • did you write these tests before or after the UpdateTotals() method? why?

  • you're using [tool whose artifacts are in their submission - eg .sln means Visual Studio] - what do you like about it?

  • have you run [static analysis, code prettying, etc] on this code?

  • what warning level did you build this at? why?

  • You chose a loop rather than recursion, why?

  • Is NewPolicy() thread-safe? Should it be? What would need to change? Is that worth it?

Someone who didn't write the code will bomb these questions totally. Someone who did might still do poorly. You will have a framework and a context for the sorts of questions people love to ask in interviews, while also proving they did the assignment themselves.



(A really suspicious or cruel interviewer could ask trick questions, like asking about something that isn't even there, but I wouldn't.)






share|improve this answer


















  • 1




    If there is some time between the pre-test and interactive interview, ask them one more. What improvements do you think you can make to the code? or Where are the predicted inefficiencies you see in this code? Anyone who didn't write it will have NO idea how to tackle either of these questions. Those that are smart enough to do this for other people's code would just write their own, because it is easier.
    – Nelson
    Mar 4 '16 at 7:31

















up vote
10
down vote














Without webcam-invigilating the entire test, how can I know that the
candidate who completed the test is the same person applying for the
vacancy?




If you simply send off a take-home exam and later receive the results, then the answer is you cannot know for sure that the test taker and the candidate who shows up in person are actually the same individual.



There are organizations that will proctor a test for you, and check identification before the exam begins. If having someone else "take the test" is that much of a repeated problem for you, you might have to pay to have it solved or have folks take the test in your office, rather than remotely.



Although I could imagine it happening, I haven't actually seen this as a real problem in my companies. But perhaps your context is different, perhaps you are hiring in volumes that make even a 15 minute phone screen prohibited, or perhaps other issues will get in the way.






share|improve this answer



























    up vote
    2
    down vote













    In a sense, you are looking for a way to verify that the interviewee is passing some kind of Turing Test. For example, even if the interviewee visits your office in person, she or he could have a subcutaneous radio transmitter embedded in their ear canal that is hooked up to Watson over at IBM. Are you planning to check for subcutaneous ear transmitters for all the candidates who visit you? No, because the likelihood of that kind of cheating is low, and the cost to the cheater is high enough that few cheaters would be willing to bother.



    So the goal is to orient the test so that cheating is expensive, and so that the signal you receive from the candidate is hard-to-fake. Nothing is ever perfectly cheat-proof, so you need to decide on how expensive you want to make the evaluation. A test that's ridiculously elaborate will drive away potentially good candidates who don't have the time to putz around with complicated tests in their already busy lives. A test that's too easy will get many good submissions from many folks, making it hard to separate the sheep and goats.



    In the domain of programming, some tips that I have used when building and conducting programming tests:



    1. Make the test less about trivia and more about creativity / ingenuity. For example, I never explicitly ask people to submit unit tests with their code samples, but I penalize them heavily if they don't simply do that all on their own. If they take the initiative to make something good, rather than something that minimally meets the requirements of the test, it's less likely they are cheating (since cheating costs someone else to use time, they are less likely to go the extra mile).

    2. Make the question very open-ended, allow the candidate to use Google or other people, but ask for an explanation for their approach. By the time the "cheater" has explained all about the cheating approach taken, the candidate ought to actually know about it, well enough to write a write-up. Here again, unless the cheater has access to a magic oracle person with no personal time constraints, it's going to be expensive to get someone else to take a lot of time to both solve a problem creatively and write the write-up.

    3. Use tools like collabedit or Google Docs to have the candidate solve a problem in real time with one of your programmers. During the interview, make sure to ask questions that are very specific, and look for quick answers. If the programmer is doing well at solving the problem, they should be able to de-focus on the programming, quickly answer related side questions, and then re-focus on the programming. If they need extended pauses every single time to answer routine side questions, they are probably consulting something else for help.

    4. Make the candidate pass through several filters. Ask a wide range of questions that have a wide range of difficulty. After all, a big part of technical interviewing is to figure out the technical skill boundaries of a candidate. No one knows everything and you need to know where the limits, cracks, and low aptitude areas are. If you give a phone screen with 5 questions, ranging from easy to hard, document both the candidate's answers and also the speed with which they answer. Then give another phone screen that is similar, and possibly a third. It will be difficult for a candidate who is cheating to arrange for the same assistant to be there all the time, and if you see glaring inconsistencies in aptitude on certain kinds of questions, it can be a red flag.

    5. Ask for a certified sample instead of the answer to a test. Ask the candidate to submit a sample of code from a college course, an open source project, etc. and ask for the name of a reference (not a student, but a faculty member, a teacher, a teaching assistant, a project leader) who can vouch that the candidate wrote the code. Then ask the candidate to explain their coding choices for that problem / project.

    The basic idea here is that (1) make cheating expensive and (2) find a trade-off for the expensiveness of (1) such that good non-cheaters are not put off by the barrier to entry at your company.



    I would also add that in my years of experience designing and conducting programming tests at my current company, cheating is the least of my worries. Few candidates can even follow basic instructions. For example, throw in a simple instruction such as: "You are not allowed to use built-in string manipulation functions to solve this problem." In my experience, greater than 50% of the candidates still use built-in functions, unapologetically, and have no answer when asked why they did it.



    If you just use a few of these screens, it will beat out a large percentage of cheating, laziness, and low aptitude noise. Then, those few extremely diligent cheaters who make it through will be less of a time-waste for your company.



    You might also try telling the candidates that they may receive help from a friend, but only if they tell you the names of all the people who helped them. If the candidate doesn't seem like the genuine article in the in-person interview, then hey, maybe you can call up their friend who helped them cheat and gave the better answer in the coding test?






    share|improve this answer



























      up vote
      2
      down vote













      I wouldn't worry about cheating at this point at all.



      You will have lots of potential candidates, and you want to filter out those that are useless. The test filters out those that are useless, except the cheaters. So don't take this test as evidence of any ability, just as the key to coming to the in person interview.



      You could as the first question in the interview ask about the test. My current company used codility.com (I think) and a cheater would have been completely, absolutely, stuck when asked about one of the problems posed (which was a very cleverly hidden logic bug).



      The idea is that the test is for filtering out people where an interview is a waste of your time. Cheaters will be able to waste your time, but they won't be able to get a job. If you are really worried about cheaters, take a photo of each candidate during the interview because I know of cases where the person starting the job was not the person doing the interview.






      share|improve this answer



























        up vote
        1
        down vote













        Your goal with a programming test, like an interview or any other hiring process, is to help determine if someone is a good fit for the company. If you need verification that they actually took the test, (and assuming you can control the contents of the test) you can accept the test as-is and bounce off it in future interviews.



        For instance, you could ask in a test to develop a simple program that requires a couple different data structures in a language where there are several classes to fit that bill, then ask in an interview why they chose that one over another. If they gave you "perfect" code that implies strong competency with the language, then can't tell you why they chose an StringBuilder over a String or some other type, then you probably have your red flag.






        share|improve this answer



























          up vote
          1
          down vote













          Two other options not listed above are:



          1. Have the requirements vague. Mention you need to compare strings, but don't mention if the comparison needs to be case or language sensitive. Use dates and times, and see if the programmer asks about time zones and leap years. This gives you an idea that the person knows enough to ask the right questions -- if they go straight to a Google search, you know that they might not be the right person.


          2. Ask them to tweak the program during the interview. Tell them, "the requirements changed -- the end user asked if you can give them this extra information. How would you alter your code to provide this?" You could tell that some people didn't feel comfortable maintaining the code they submitted to you (which strongly suggests that someone did the work for them or they found it online and didn't totally understand the code).






          share|improve this answer






















          • Updated answer based on comments
            – bryanjonker
            Mar 3 '16 at 18:02

















          up vote
          0
          down vote













          This may not be exactly the answer that you're looking for, but I handle this by giving developer candidates a simple 10-question quiz at the beginning of an in-person interview. The questions are geared toward the expected skill level for the position and cover concepts, not code samples. For example, I may ask a junior .NET developer candidate to describe the difference between a method and property, or to explain what LINQ does and how it's an improvement over its predecessors. I may ask a .NET architect candidate to explain boxing, its impact, and how it can be mitigated.



          This is obviously not as good as a programming assignment, but it does give me a reasonably good view of the candidate's knowledge of the concepts and technologies that he/she will be using. It also gives me good insight into their written communications skills as well.



          If the candidate survives this test, I expect any deeper questions about technique, etc to shake out in the second interview with the development team.






          share|improve this answer




















          • Boxing seems like a very very poor example of a cognitive technical question.
            – RandomUs1r
            Dec 22 '17 at 21:35

















          up vote
          0
          down vote













          Ask questions that don't have canned answers, let me give you an example:



          1. What is the difference between a class and an interface?
            Google has everything you need to answer this, so you're testing memorization here, not skill.


          2. When would you use a class vs an interface?
            Much more interesting answers that demonstrate skill, experience, and understanding.


          A technical test full of 2. type questions would be much harder to cheat on and give a better assessment of the candidate's skills.






          share|improve this answer



















            protected by Community♦ Dec 3 '17 at 4:58



            Thank you for your interest in this question.
            Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).



            Would you like to answer one of these unanswered questions instead?














            9 Answers
            9






            active

            oldest

            votes








            9 Answers
            9






            active

            oldest

            votes









            active

            oldest

            votes






            active

            oldest

            votes








            up vote
            13
            down vote



            accepted










            I don't believe you can come up with a control that is both reasonably effective and less time consuming than bringing the candidate in and verifying that they can support answers.



            My approach would be to make the stakes high enough that cheating is also shooting yourself in the foot. Have the candidates sign off on an honor code that clearly states that if they are found cheating, both the person who showed up for the interview and the name & contact details of the resume provided will be blacklisted from company searches for 5 years. It's reasonably easy to keep a listing of cheaters - their pictures, their aliases, and their phone numbers and email addresses. Then use that as criteria #1 in screening future candidates.



            Yes - it's still possible to cheat your way around that - new phone number, new alias, etc. But at some point, your ability to weed out people with no skills will be clear enough that it doesn't seem worth the work.



            In all honesty, if this is happening for any significant percentage of the time - you may have to give up on the pretest. And if it isn't a significantly frequent occurrence, figure the time wasted on verifying the candidate in person is going to be more minimal than all the tech work of doing more.






            share|improve this answer
















            • 7




              Note that a list like that is illegal in Europe.
              – Juha Untinen
              Sep 6 '15 at 21:28






            • 3




              Hmm, can you provide information on the law that makes it illegal? I'm not sure why "cheating a company" and then keeping a record of the cheaters is somehow illegal.
              – Nelson
              Mar 4 '16 at 7:28






            • 3




              In France, at least, any computerized information you keep about individuals shall be accessible to the individual. So the list would not be forbidden, but the candidate is legally allowed to know wether a computer file blacklists him. Don't know elsewhere.
              – gazzz0x2z
              Mar 4 '16 at 11:00






            • 1




              In the US there is some level of precedent - I worked in a company where if you declined a job offer, we wouldn't make another for X number of years. The goal was to avoid wasting company resources with someone who was fishing for opportunities. Given that we didn't have the information handling laws of Europe, there was no concern there.. but I don't think anyone have been concerned at disclosing to the individual "sorry you are ineligible because you turned down a job offer on Y date"
              – bethlakshmi
              Mar 4 '16 at 15:57






            • 1




              @Nelson Adding some further information concerning France: the blacklist cannot be shared beyond the scope of recruiting. It would be illegal for example to disclose such information to anyone not involved in the recruitment process at the company. Everything would have to be informed to the CNIL and respect the measures to secure the access to such information (cryptography, etc).
              – Adam Smith
              Dec 27 '17 at 0:37















            up vote
            13
            down vote



            accepted










            I don't believe you can come up with a control that is both reasonably effective and less time consuming than bringing the candidate in and verifying that they can support answers.



            My approach would be to make the stakes high enough that cheating is also shooting yourself in the foot. Have the candidates sign off on an honor code that clearly states that if they are found cheating, both the person who showed up for the interview and the name & contact details of the resume provided will be blacklisted from company searches for 5 years. It's reasonably easy to keep a listing of cheaters - their pictures, their aliases, and their phone numbers and email addresses. Then use that as criteria #1 in screening future candidates.



            Yes - it's still possible to cheat your way around that - new phone number, new alias, etc. But at some point, your ability to weed out people with no skills will be clear enough that it doesn't seem worth the work.



            In all honesty, if this is happening for any significant percentage of the time - you may have to give up on the pretest. And if it isn't a significantly frequent occurrence, figure the time wasted on verifying the candidate in person is going to be more minimal than all the tech work of doing more.






            share|improve this answer
















            • 7




              Note that a list like that is illegal in Europe.
              – Juha Untinen
              Sep 6 '15 at 21:28






            • 3




              Hmm, can you provide information on the law that makes it illegal? I'm not sure why "cheating a company" and then keeping a record of the cheaters is somehow illegal.
              – Nelson
              Mar 4 '16 at 7:28






            • 3




              In France, at least, any computerized information you keep about individuals shall be accessible to the individual. So the list would not be forbidden, but the candidate is legally allowed to know wether a computer file blacklists him. Don't know elsewhere.
              – gazzz0x2z
              Mar 4 '16 at 11:00






            • 1




              In the US there is some level of precedent - I worked in a company where if you declined a job offer, we wouldn't make another for X number of years. The goal was to avoid wasting company resources with someone who was fishing for opportunities. Given that we didn't have the information handling laws of Europe, there was no concern there.. but I don't think anyone have been concerned at disclosing to the individual "sorry you are ineligible because you turned down a job offer on Y date"
              – bethlakshmi
              Mar 4 '16 at 15:57






            • 1




              @Nelson Adding some further information concerning France: the blacklist cannot be shared beyond the scope of recruiting. It would be illegal for example to disclose such information to anyone not involved in the recruitment process at the company. Everything would have to be informed to the CNIL and respect the measures to secure the access to such information (cryptography, etc).
              – Adam Smith
              Dec 27 '17 at 0:37













            up vote
            13
            down vote



            accepted







            up vote
            13
            down vote



            accepted






            I don't believe you can come up with a control that is both reasonably effective and less time consuming than bringing the candidate in and verifying that they can support answers.



            My approach would be to make the stakes high enough that cheating is also shooting yourself in the foot. Have the candidates sign off on an honor code that clearly states that if they are found cheating, both the person who showed up for the interview and the name & contact details of the resume provided will be blacklisted from company searches for 5 years. It's reasonably easy to keep a listing of cheaters - their pictures, their aliases, and their phone numbers and email addresses. Then use that as criteria #1 in screening future candidates.



            Yes - it's still possible to cheat your way around that - new phone number, new alias, etc. But at some point, your ability to weed out people with no skills will be clear enough that it doesn't seem worth the work.



            In all honesty, if this is happening for any significant percentage of the time - you may have to give up on the pretest. And if it isn't a significantly frequent occurrence, figure the time wasted on verifying the candidate in person is going to be more minimal than all the tech work of doing more.






            share|improve this answer












            I don't believe you can come up with a control that is both reasonably effective and less time consuming than bringing the candidate in and verifying that they can support answers.



            My approach would be to make the stakes high enough that cheating is also shooting yourself in the foot. Have the candidates sign off on an honor code that clearly states that if they are found cheating, both the person who showed up for the interview and the name & contact details of the resume provided will be blacklisted from company searches for 5 years. It's reasonably easy to keep a listing of cheaters - their pictures, their aliases, and their phone numbers and email addresses. Then use that as criteria #1 in screening future candidates.



            Yes - it's still possible to cheat your way around that - new phone number, new alias, etc. But at some point, your ability to weed out people with no skills will be clear enough that it doesn't seem worth the work.



            In all honesty, if this is happening for any significant percentage of the time - you may have to give up on the pretest. And if it isn't a significantly frequent occurrence, figure the time wasted on verifying the candidate in person is going to be more minimal than all the tech work of doing more.







            share|improve this answer












            share|improve this answer



            share|improve this answer










            answered Jan 30 '14 at 14:51









            bethlakshmi

            70.4k4136277




            70.4k4136277







            • 7




              Note that a list like that is illegal in Europe.
              – Juha Untinen
              Sep 6 '15 at 21:28






            • 3




              Hmm, can you provide information on the law that makes it illegal? I'm not sure why "cheating a company" and then keeping a record of the cheaters is somehow illegal.
              – Nelson
              Mar 4 '16 at 7:28






            • 3




              In France, at least, any computerized information you keep about individuals shall be accessible to the individual. So the list would not be forbidden, but the candidate is legally allowed to know wether a computer file blacklists him. Don't know elsewhere.
              – gazzz0x2z
              Mar 4 '16 at 11:00






            • 1




              In the US there is some level of precedent - I worked in a company where if you declined a job offer, we wouldn't make another for X number of years. The goal was to avoid wasting company resources with someone who was fishing for opportunities. Given that we didn't have the information handling laws of Europe, there was no concern there.. but I don't think anyone have been concerned at disclosing to the individual "sorry you are ineligible because you turned down a job offer on Y date"
              – bethlakshmi
              Mar 4 '16 at 15:57






            • 1




              @Nelson Adding some further information concerning France: the blacklist cannot be shared beyond the scope of recruiting. It would be illegal for example to disclose such information to anyone not involved in the recruitment process at the company. Everything would have to be informed to the CNIL and respect the measures to secure the access to such information (cryptography, etc).
              – Adam Smith
              Dec 27 '17 at 0:37













            • 7




              Note that a list like that is illegal in Europe.
              – Juha Untinen
              Sep 6 '15 at 21:28






            • 3




              Hmm, can you provide information on the law that makes it illegal? I'm not sure why "cheating a company" and then keeping a record of the cheaters is somehow illegal.
              – Nelson
              Mar 4 '16 at 7:28






            • 3




              In France, at least, any computerized information you keep about individuals shall be accessible to the individual. So the list would not be forbidden, but the candidate is legally allowed to know wether a computer file blacklists him. Don't know elsewhere.
              – gazzz0x2z
              Mar 4 '16 at 11:00






            • 1




              In the US there is some level of precedent - I worked in a company where if you declined a job offer, we wouldn't make another for X number of years. The goal was to avoid wasting company resources with someone who was fishing for opportunities. Given that we didn't have the information handling laws of Europe, there was no concern there.. but I don't think anyone have been concerned at disclosing to the individual "sorry you are ineligible because you turned down a job offer on Y date"
              – bethlakshmi
              Mar 4 '16 at 15:57






            • 1




              @Nelson Adding some further information concerning France: the blacklist cannot be shared beyond the scope of recruiting. It would be illegal for example to disclose such information to anyone not involved in the recruitment process at the company. Everything would have to be informed to the CNIL and respect the measures to secure the access to such information (cryptography, etc).
              – Adam Smith
              Dec 27 '17 at 0:37








            7




            7




            Note that a list like that is illegal in Europe.
            – Juha Untinen
            Sep 6 '15 at 21:28




            Note that a list like that is illegal in Europe.
            – Juha Untinen
            Sep 6 '15 at 21:28




            3




            3




            Hmm, can you provide information on the law that makes it illegal? I'm not sure why "cheating a company" and then keeping a record of the cheaters is somehow illegal.
            – Nelson
            Mar 4 '16 at 7:28




            Hmm, can you provide information on the law that makes it illegal? I'm not sure why "cheating a company" and then keeping a record of the cheaters is somehow illegal.
            – Nelson
            Mar 4 '16 at 7:28




            3




            3




            In France, at least, any computerized information you keep about individuals shall be accessible to the individual. So the list would not be forbidden, but the candidate is legally allowed to know wether a computer file blacklists him. Don't know elsewhere.
            – gazzz0x2z
            Mar 4 '16 at 11:00




            In France, at least, any computerized information you keep about individuals shall be accessible to the individual. So the list would not be forbidden, but the candidate is legally allowed to know wether a computer file blacklists him. Don't know elsewhere.
            – gazzz0x2z
            Mar 4 '16 at 11:00




            1




            1




            In the US there is some level of precedent - I worked in a company where if you declined a job offer, we wouldn't make another for X number of years. The goal was to avoid wasting company resources with someone who was fishing for opportunities. Given that we didn't have the information handling laws of Europe, there was no concern there.. but I don't think anyone have been concerned at disclosing to the individual "sorry you are ineligible because you turned down a job offer on Y date"
            – bethlakshmi
            Mar 4 '16 at 15:57




            In the US there is some level of precedent - I worked in a company where if you declined a job offer, we wouldn't make another for X number of years. The goal was to avoid wasting company resources with someone who was fishing for opportunities. Given that we didn't have the information handling laws of Europe, there was no concern there.. but I don't think anyone have been concerned at disclosing to the individual "sorry you are ineligible because you turned down a job offer on Y date"
            – bethlakshmi
            Mar 4 '16 at 15:57




            1




            1




            @Nelson Adding some further information concerning France: the blacklist cannot be shared beyond the scope of recruiting. It would be illegal for example to disclose such information to anyone not involved in the recruitment process at the company. Everything would have to be informed to the CNIL and respect the measures to secure the access to such information (cryptography, etc).
            – Adam Smith
            Dec 27 '17 at 0:37





            @Nelson Adding some further information concerning France: the blacklist cannot be shared beyond the scope of recruiting. It would be illegal for example to disclose such information to anyone not involved in the recruitment process at the company. Everything would have to be informed to the CNIL and respect the measures to secure the access to such information (cryptography, etc).
            – Adam Smith
            Dec 27 '17 at 0:37













            up vote
            24
            down vote













            Just as the important thing about a whiteboard programming test is the conversation it facilitates, the important thing about a take home test can be the conversation it facilitates.



            Tell them to keep the code handy because there will be a conversation about it afterwards. Then, for those whose submissions are good enough for a phone or in-person interview, you can ask stuff:



            • did you write these tests before or after the UpdateTotals() method? why?

            • you're using [tool whose artifacts are in their submission - eg .sln means Visual Studio] - what do you like about it?

            • have you run [static analysis, code prettying, etc] on this code?

            • what warning level did you build this at? why?

            • You chose a loop rather than recursion, why?

            • Is NewPolicy() thread-safe? Should it be? What would need to change? Is that worth it?

            Someone who didn't write the code will bomb these questions totally. Someone who did might still do poorly. You will have a framework and a context for the sorts of questions people love to ask in interviews, while also proving they did the assignment themselves.



            (A really suspicious or cruel interviewer could ask trick questions, like asking about something that isn't even there, but I wouldn't.)






            share|improve this answer


















            • 1




              If there is some time between the pre-test and interactive interview, ask them one more. What improvements do you think you can make to the code? or Where are the predicted inefficiencies you see in this code? Anyone who didn't write it will have NO idea how to tackle either of these questions. Those that are smart enough to do this for other people's code would just write their own, because it is easier.
              – Nelson
              Mar 4 '16 at 7:31














            up vote
            24
            down vote













            Just as the important thing about a whiteboard programming test is the conversation it facilitates, the important thing about a take home test can be the conversation it facilitates.



            Tell them to keep the code handy because there will be a conversation about it afterwards. Then, for those whose submissions are good enough for a phone or in-person interview, you can ask stuff:



            • did you write these tests before or after the UpdateTotals() method? why?

            • you're using [tool whose artifacts are in their submission - eg .sln means Visual Studio] - what do you like about it?

            • have you run [static analysis, code prettying, etc] on this code?

            • what warning level did you build this at? why?

            • You chose a loop rather than recursion, why?

            • Is NewPolicy() thread-safe? Should it be? What would need to change? Is that worth it?

            Someone who didn't write the code will bomb these questions totally. Someone who did might still do poorly. You will have a framework and a context for the sorts of questions people love to ask in interviews, while also proving they did the assignment themselves.



            (A really suspicious or cruel interviewer could ask trick questions, like asking about something that isn't even there, but I wouldn't.)






            share|improve this answer


















            • 1




              If there is some time between the pre-test and interactive interview, ask them one more. What improvements do you think you can make to the code? or Where are the predicted inefficiencies you see in this code? Anyone who didn't write it will have NO idea how to tackle either of these questions. Those that are smart enough to do this for other people's code would just write their own, because it is easier.
              – Nelson
              Mar 4 '16 at 7:31












            up vote
            24
            down vote










            up vote
            24
            down vote









            Just as the important thing about a whiteboard programming test is the conversation it facilitates, the important thing about a take home test can be the conversation it facilitates.



            Tell them to keep the code handy because there will be a conversation about it afterwards. Then, for those whose submissions are good enough for a phone or in-person interview, you can ask stuff:



            • did you write these tests before or after the UpdateTotals() method? why?

            • you're using [tool whose artifacts are in their submission - eg .sln means Visual Studio] - what do you like about it?

            • have you run [static analysis, code prettying, etc] on this code?

            • what warning level did you build this at? why?

            • You chose a loop rather than recursion, why?

            • Is NewPolicy() thread-safe? Should it be? What would need to change? Is that worth it?

            Someone who didn't write the code will bomb these questions totally. Someone who did might still do poorly. You will have a framework and a context for the sorts of questions people love to ask in interviews, while also proving they did the assignment themselves.



            (A really suspicious or cruel interviewer could ask trick questions, like asking about something that isn't even there, but I wouldn't.)






            share|improve this answer














            Just as the important thing about a whiteboard programming test is the conversation it facilitates, the important thing about a take home test can be the conversation it facilitates.



            Tell them to keep the code handy because there will be a conversation about it afterwards. Then, for those whose submissions are good enough for a phone or in-person interview, you can ask stuff:



            • did you write these tests before or after the UpdateTotals() method? why?

            • you're using [tool whose artifacts are in their submission - eg .sln means Visual Studio] - what do you like about it?

            • have you run [static analysis, code prettying, etc] on this code?

            • what warning level did you build this at? why?

            • You chose a loop rather than recursion, why?

            • Is NewPolicy() thread-safe? Should it be? What would need to change? Is that worth it?

            Someone who didn't write the code will bomb these questions totally. Someone who did might still do poorly. You will have a framework and a context for the sorts of questions people love to ask in interviews, while also proving they did the assignment themselves.



            (A really suspicious or cruel interviewer could ask trick questions, like asking about something that isn't even there, but I wouldn't.)







            share|improve this answer














            share|improve this answer



            share|improve this answer








            edited Jan 30 '14 at 19:42

























            answered Jan 30 '14 at 14:58









            Kate Gregory

            105k40232334




            105k40232334







            • 1




              If there is some time between the pre-test and interactive interview, ask them one more. What improvements do you think you can make to the code? or Where are the predicted inefficiencies you see in this code? Anyone who didn't write it will have NO idea how to tackle either of these questions. Those that are smart enough to do this for other people's code would just write their own, because it is easier.
              – Nelson
              Mar 4 '16 at 7:31












            • 1




              If there is some time between the pre-test and interactive interview, ask them one more. What improvements do you think you can make to the code? or Where are the predicted inefficiencies you see in this code? Anyone who didn't write it will have NO idea how to tackle either of these questions. Those that are smart enough to do this for other people's code would just write their own, because it is easier.
              – Nelson
              Mar 4 '16 at 7:31







            1




            1




            If there is some time between the pre-test and interactive interview, ask them one more. What improvements do you think you can make to the code? or Where are the predicted inefficiencies you see in this code? Anyone who didn't write it will have NO idea how to tackle either of these questions. Those that are smart enough to do this for other people's code would just write their own, because it is easier.
            – Nelson
            Mar 4 '16 at 7:31




            If there is some time between the pre-test and interactive interview, ask them one more. What improvements do you think you can make to the code? or Where are the predicted inefficiencies you see in this code? Anyone who didn't write it will have NO idea how to tackle either of these questions. Those that are smart enough to do this for other people's code would just write their own, because it is easier.
            – Nelson
            Mar 4 '16 at 7:31










            up vote
            10
            down vote














            Without webcam-invigilating the entire test, how can I know that the
            candidate who completed the test is the same person applying for the
            vacancy?




            If you simply send off a take-home exam and later receive the results, then the answer is you cannot know for sure that the test taker and the candidate who shows up in person are actually the same individual.



            There are organizations that will proctor a test for you, and check identification before the exam begins. If having someone else "take the test" is that much of a repeated problem for you, you might have to pay to have it solved or have folks take the test in your office, rather than remotely.



            Although I could imagine it happening, I haven't actually seen this as a real problem in my companies. But perhaps your context is different, perhaps you are hiring in volumes that make even a 15 minute phone screen prohibited, or perhaps other issues will get in the way.






            share|improve this answer
























              up vote
              10
              down vote














              Without webcam-invigilating the entire test, how can I know that the
              candidate who completed the test is the same person applying for the
              vacancy?




              If you simply send off a take-home exam and later receive the results, then the answer is you cannot know for sure that the test taker and the candidate who shows up in person are actually the same individual.



              There are organizations that will proctor a test for you, and check identification before the exam begins. If having someone else "take the test" is that much of a repeated problem for you, you might have to pay to have it solved or have folks take the test in your office, rather than remotely.



              Although I could imagine it happening, I haven't actually seen this as a real problem in my companies. But perhaps your context is different, perhaps you are hiring in volumes that make even a 15 minute phone screen prohibited, or perhaps other issues will get in the way.






              share|improve this answer






















                up vote
                10
                down vote










                up vote
                10
                down vote










                Without webcam-invigilating the entire test, how can I know that the
                candidate who completed the test is the same person applying for the
                vacancy?




                If you simply send off a take-home exam and later receive the results, then the answer is you cannot know for sure that the test taker and the candidate who shows up in person are actually the same individual.



                There are organizations that will proctor a test for you, and check identification before the exam begins. If having someone else "take the test" is that much of a repeated problem for you, you might have to pay to have it solved or have folks take the test in your office, rather than remotely.



                Although I could imagine it happening, I haven't actually seen this as a real problem in my companies. But perhaps your context is different, perhaps you are hiring in volumes that make even a 15 minute phone screen prohibited, or perhaps other issues will get in the way.






                share|improve this answer













                Without webcam-invigilating the entire test, how can I know that the
                candidate who completed the test is the same person applying for the
                vacancy?




                If you simply send off a take-home exam and later receive the results, then the answer is you cannot know for sure that the test taker and the candidate who shows up in person are actually the same individual.



                There are organizations that will proctor a test for you, and check identification before the exam begins. If having someone else "take the test" is that much of a repeated problem for you, you might have to pay to have it solved or have folks take the test in your office, rather than remotely.



                Although I could imagine it happening, I haven't actually seen this as a real problem in my companies. But perhaps your context is different, perhaps you are hiring in volumes that make even a 15 minute phone screen prohibited, or perhaps other issues will get in the way.







                share|improve this answer












                share|improve this answer



                share|improve this answer










                answered Jan 30 '14 at 12:35









                Joe Strazzere

                224k107661930




                224k107661930




















                    up vote
                    2
                    down vote













                    In a sense, you are looking for a way to verify that the interviewee is passing some kind of Turing Test. For example, even if the interviewee visits your office in person, she or he could have a subcutaneous radio transmitter embedded in their ear canal that is hooked up to Watson over at IBM. Are you planning to check for subcutaneous ear transmitters for all the candidates who visit you? No, because the likelihood of that kind of cheating is low, and the cost to the cheater is high enough that few cheaters would be willing to bother.



                    So the goal is to orient the test so that cheating is expensive, and so that the signal you receive from the candidate is hard-to-fake. Nothing is ever perfectly cheat-proof, so you need to decide on how expensive you want to make the evaluation. A test that's ridiculously elaborate will drive away potentially good candidates who don't have the time to putz around with complicated tests in their already busy lives. A test that's too easy will get many good submissions from many folks, making it hard to separate the sheep and goats.



                    In the domain of programming, some tips that I have used when building and conducting programming tests:



                    1. Make the test less about trivia and more about creativity / ingenuity. For example, I never explicitly ask people to submit unit tests with their code samples, but I penalize them heavily if they don't simply do that all on their own. If they take the initiative to make something good, rather than something that minimally meets the requirements of the test, it's less likely they are cheating (since cheating costs someone else to use time, they are less likely to go the extra mile).

                    2. Make the question very open-ended, allow the candidate to use Google or other people, but ask for an explanation for their approach. By the time the "cheater" has explained all about the cheating approach taken, the candidate ought to actually know about it, well enough to write a write-up. Here again, unless the cheater has access to a magic oracle person with no personal time constraints, it's going to be expensive to get someone else to take a lot of time to both solve a problem creatively and write the write-up.

                    3. Use tools like collabedit or Google Docs to have the candidate solve a problem in real time with one of your programmers. During the interview, make sure to ask questions that are very specific, and look for quick answers. If the programmer is doing well at solving the problem, they should be able to de-focus on the programming, quickly answer related side questions, and then re-focus on the programming. If they need extended pauses every single time to answer routine side questions, they are probably consulting something else for help.

                    4. Make the candidate pass through several filters. Ask a wide range of questions that have a wide range of difficulty. After all, a big part of technical interviewing is to figure out the technical skill boundaries of a candidate. No one knows everything and you need to know where the limits, cracks, and low aptitude areas are. If you give a phone screen with 5 questions, ranging from easy to hard, document both the candidate's answers and also the speed with which they answer. Then give another phone screen that is similar, and possibly a third. It will be difficult for a candidate who is cheating to arrange for the same assistant to be there all the time, and if you see glaring inconsistencies in aptitude on certain kinds of questions, it can be a red flag.

                    5. Ask for a certified sample instead of the answer to a test. Ask the candidate to submit a sample of code from a college course, an open source project, etc. and ask for the name of a reference (not a student, but a faculty member, a teacher, a teaching assistant, a project leader) who can vouch that the candidate wrote the code. Then ask the candidate to explain their coding choices for that problem / project.

                    The basic idea here is that (1) make cheating expensive and (2) find a trade-off for the expensiveness of (1) such that good non-cheaters are not put off by the barrier to entry at your company.



                    I would also add that in my years of experience designing and conducting programming tests at my current company, cheating is the least of my worries. Few candidates can even follow basic instructions. For example, throw in a simple instruction such as: "You are not allowed to use built-in string manipulation functions to solve this problem." In my experience, greater than 50% of the candidates still use built-in functions, unapologetically, and have no answer when asked why they did it.



                    If you just use a few of these screens, it will beat out a large percentage of cheating, laziness, and low aptitude noise. Then, those few extremely diligent cheaters who make it through will be less of a time-waste for your company.



                    You might also try telling the candidates that they may receive help from a friend, but only if they tell you the names of all the people who helped them. If the candidate doesn't seem like the genuine article in the in-person interview, then hey, maybe you can call up their friend who helped them cheat and gave the better answer in the coding test?






                    share|improve this answer
























                      up vote
                      2
                      down vote













                      In a sense, you are looking for a way to verify that the interviewee is passing some kind of Turing Test. For example, even if the interviewee visits your office in person, she or he could have a subcutaneous radio transmitter embedded in their ear canal that is hooked up to Watson over at IBM. Are you planning to check for subcutaneous ear transmitters for all the candidates who visit you? No, because the likelihood of that kind of cheating is low, and the cost to the cheater is high enough that few cheaters would be willing to bother.



                      So the goal is to orient the test so that cheating is expensive, and so that the signal you receive from the candidate is hard-to-fake. Nothing is ever perfectly cheat-proof, so you need to decide on how expensive you want to make the evaluation. A test that's ridiculously elaborate will drive away potentially good candidates who don't have the time to putz around with complicated tests in their already busy lives. A test that's too easy will get many good submissions from many folks, making it hard to separate the sheep and goats.



                      In the domain of programming, some tips that I have used when building and conducting programming tests:



                      1. Make the test less about trivia and more about creativity / ingenuity. For example, I never explicitly ask people to submit unit tests with their code samples, but I penalize them heavily if they don't simply do that all on their own. If they take the initiative to make something good, rather than something that minimally meets the requirements of the test, it's less likely they are cheating (since cheating costs someone else to use time, they are less likely to go the extra mile).

                      2. Make the question very open-ended, allow the candidate to use Google or other people, but ask for an explanation for their approach. By the time the "cheater" has explained all about the cheating approach taken, the candidate ought to actually know about it, well enough to write a write-up. Here again, unless the cheater has access to a magic oracle person with no personal time constraints, it's going to be expensive to get someone else to take a lot of time to both solve a problem creatively and write the write-up.

                      3. Use tools like collabedit or Google Docs to have the candidate solve a problem in real time with one of your programmers. During the interview, make sure to ask questions that are very specific, and look for quick answers. If the programmer is doing well at solving the problem, they should be able to de-focus on the programming, quickly answer related side questions, and then re-focus on the programming. If they need extended pauses every single time to answer routine side questions, they are probably consulting something else for help.

                      4. Make the candidate pass through several filters. Ask a wide range of questions that have a wide range of difficulty. After all, a big part of technical interviewing is to figure out the technical skill boundaries of a candidate. No one knows everything and you need to know where the limits, cracks, and low aptitude areas are. If you give a phone screen with 5 questions, ranging from easy to hard, document both the candidate's answers and also the speed with which they answer. Then give another phone screen that is similar, and possibly a third. It will be difficult for a candidate who is cheating to arrange for the same assistant to be there all the time, and if you see glaring inconsistencies in aptitude on certain kinds of questions, it can be a red flag.

                      5. Ask for a certified sample instead of the answer to a test. Ask the candidate to submit a sample of code from a college course, an open source project, etc. and ask for the name of a reference (not a student, but a faculty member, a teacher, a teaching assistant, a project leader) who can vouch that the candidate wrote the code. Then ask the candidate to explain their coding choices for that problem / project.

                      The basic idea here is that (1) make cheating expensive and (2) find a trade-off for the expensiveness of (1) such that good non-cheaters are not put off by the barrier to entry at your company.



                      I would also add that in my years of experience designing and conducting programming tests at my current company, cheating is the least of my worries. Few candidates can even follow basic instructions. For example, throw in a simple instruction such as: "You are not allowed to use built-in string manipulation functions to solve this problem." In my experience, greater than 50% of the candidates still use built-in functions, unapologetically, and have no answer when asked why they did it.



                      If you just use a few of these screens, it will beat out a large percentage of cheating, laziness, and low aptitude noise. Then, those few extremely diligent cheaters who make it through will be less of a time-waste for your company.



                      You might also try telling the candidates that they may receive help from a friend, but only if they tell you the names of all the people who helped them. If the candidate doesn't seem like the genuine article in the in-person interview, then hey, maybe you can call up their friend who helped them cheat and gave the better answer in the coding test?






                      share|improve this answer






















                        up vote
                        2
                        down vote










                        up vote
                        2
                        down vote









                        In a sense, you are looking for a way to verify that the interviewee is passing some kind of Turing Test. For example, even if the interviewee visits your office in person, she or he could have a subcutaneous radio transmitter embedded in their ear canal that is hooked up to Watson over at IBM. Are you planning to check for subcutaneous ear transmitters for all the candidates who visit you? No, because the likelihood of that kind of cheating is low, and the cost to the cheater is high enough that few cheaters would be willing to bother.



                        So the goal is to orient the test so that cheating is expensive, and so that the signal you receive from the candidate is hard-to-fake. Nothing is ever perfectly cheat-proof, so you need to decide on how expensive you want to make the evaluation. A test that's ridiculously elaborate will drive away potentially good candidates who don't have the time to putz around with complicated tests in their already busy lives. A test that's too easy will get many good submissions from many folks, making it hard to separate the sheep and goats.



                        In the domain of programming, some tips that I have used when building and conducting programming tests:



                        1. Make the test less about trivia and more about creativity / ingenuity. For example, I never explicitly ask people to submit unit tests with their code samples, but I penalize them heavily if they don't simply do that all on their own. If they take the initiative to make something good, rather than something that minimally meets the requirements of the test, it's less likely they are cheating (since cheating costs someone else to use time, they are less likely to go the extra mile).

                        2. Make the question very open-ended, allow the candidate to use Google or other people, but ask for an explanation for their approach. By the time the "cheater" has explained all about the cheating approach taken, the candidate ought to actually know about it, well enough to write a write-up. Here again, unless the cheater has access to a magic oracle person with no personal time constraints, it's going to be expensive to get someone else to take a lot of time to both solve a problem creatively and write the write-up.

                        3. Use tools like collabedit or Google Docs to have the candidate solve a problem in real time with one of your programmers. During the interview, make sure to ask questions that are very specific, and look for quick answers. If the programmer is doing well at solving the problem, they should be able to de-focus on the programming, quickly answer related side questions, and then re-focus on the programming. If they need extended pauses every single time to answer routine side questions, they are probably consulting something else for help.

                        4. Make the candidate pass through several filters. Ask a wide range of questions that have a wide range of difficulty. After all, a big part of technical interviewing is to figure out the technical skill boundaries of a candidate. No one knows everything and you need to know where the limits, cracks, and low aptitude areas are. If you give a phone screen with 5 questions, ranging from easy to hard, document both the candidate's answers and also the speed with which they answer. Then give another phone screen that is similar, and possibly a third. It will be difficult for a candidate who is cheating to arrange for the same assistant to be there all the time, and if you see glaring inconsistencies in aptitude on certain kinds of questions, it can be a red flag.

                        5. Ask for a certified sample instead of the answer to a test. Ask the candidate to submit a sample of code from a college course, an open source project, etc. and ask for the name of a reference (not a student, but a faculty member, a teacher, a teaching assistant, a project leader) who can vouch that the candidate wrote the code. Then ask the candidate to explain their coding choices for that problem / project.

                        The basic idea here is that (1) make cheating expensive and (2) find a trade-off for the expensiveness of (1) such that good non-cheaters are not put off by the barrier to entry at your company.



                        I would also add that in my years of experience designing and conducting programming tests at my current company, cheating is the least of my worries. Few candidates can even follow basic instructions. For example, throw in a simple instruction such as: "You are not allowed to use built-in string manipulation functions to solve this problem." In my experience, greater than 50% of the candidates still use built-in functions, unapologetically, and have no answer when asked why they did it.



                        If you just use a few of these screens, it will beat out a large percentage of cheating, laziness, and low aptitude noise. Then, those few extremely diligent cheaters who make it through will be less of a time-waste for your company.



                        You might also try telling the candidates that they may receive help from a friend, but only if they tell you the names of all the people who helped them. If the candidate doesn't seem like the genuine article in the in-person interview, then hey, maybe you can call up their friend who helped them cheat and gave the better answer in the coding test?






                        share|improve this answer












                        In a sense, you are looking for a way to verify that the interviewee is passing some kind of Turing Test. For example, even if the interviewee visits your office in person, she or he could have a subcutaneous radio transmitter embedded in their ear canal that is hooked up to Watson over at IBM. Are you planning to check for subcutaneous ear transmitters for all the candidates who visit you? No, because the likelihood of that kind of cheating is low, and the cost to the cheater is high enough that few cheaters would be willing to bother.



                        So the goal is to orient the test so that cheating is expensive, and so that the signal you receive from the candidate is hard-to-fake. Nothing is ever perfectly cheat-proof, so you need to decide on how expensive you want to make the evaluation. A test that's ridiculously elaborate will drive away potentially good candidates who don't have the time to putz around with complicated tests in their already busy lives. A test that's too easy will get many good submissions from many folks, making it hard to separate the sheep and goats.



                        In the domain of programming, some tips that I have used when building and conducting programming tests:



                        1. Make the test less about trivia and more about creativity / ingenuity. For example, I never explicitly ask people to submit unit tests with their code samples, but I penalize them heavily if they don't simply do that all on their own. If they take the initiative to make something good, rather than something that minimally meets the requirements of the test, it's less likely they are cheating (since cheating costs someone else to use time, they are less likely to go the extra mile).

                        2. Make the question very open-ended, allow the candidate to use Google or other people, but ask for an explanation for their approach. By the time the "cheater" has explained all about the cheating approach taken, the candidate ought to actually know about it, well enough to write a write-up. Here again, unless the cheater has access to a magic oracle person with no personal time constraints, it's going to be expensive to get someone else to take a lot of time to both solve a problem creatively and write the write-up.

                        3. Use tools like collabedit or Google Docs to have the candidate solve a problem in real time with one of your programmers. During the interview, make sure to ask questions that are very specific, and look for quick answers. If the programmer is doing well at solving the problem, they should be able to de-focus on the programming, quickly answer related side questions, and then re-focus on the programming. If they need extended pauses every single time to answer routine side questions, they are probably consulting something else for help.

                        4. Make the candidate pass through several filters. Ask a wide range of questions that have a wide range of difficulty. After all, a big part of technical interviewing is to figure out the technical skill boundaries of a candidate. No one knows everything and you need to know where the limits, cracks, and low aptitude areas are. If you give a phone screen with 5 questions, ranging from easy to hard, document both the candidate's answers and also the speed with which they answer. Then give another phone screen that is similar, and possibly a third. It will be difficult for a candidate who is cheating to arrange for the same assistant to be there all the time, and if you see glaring inconsistencies in aptitude on certain kinds of questions, it can be a red flag.

                        5. Ask for a certified sample instead of the answer to a test. Ask the candidate to submit a sample of code from a college course, an open source project, etc. and ask for the name of a reference (not a student, but a faculty member, a teacher, a teaching assistant, a project leader) who can vouch that the candidate wrote the code. Then ask the candidate to explain their coding choices for that problem / project.

                        The basic idea here is that (1) make cheating expensive and (2) find a trade-off for the expensiveness of (1) such that good non-cheaters are not put off by the barrier to entry at your company.



                        I would also add that in my years of experience designing and conducting programming tests at my current company, cheating is the least of my worries. Few candidates can even follow basic instructions. For example, throw in a simple instruction such as: "You are not allowed to use built-in string manipulation functions to solve this problem." In my experience, greater than 50% of the candidates still use built-in functions, unapologetically, and have no answer when asked why they did it.



                        If you just use a few of these screens, it will beat out a large percentage of cheating, laziness, and low aptitude noise. Then, those few extremely diligent cheaters who make it through will be less of a time-waste for your company.



                        You might also try telling the candidates that they may receive help from a friend, but only if they tell you the names of all the people who helped them. If the candidate doesn't seem like the genuine article in the in-person interview, then hey, maybe you can call up their friend who helped them cheat and gave the better answer in the coding test?







                        share|improve this answer












                        share|improve this answer



                        share|improve this answer










                        answered Jan 30 '14 at 16:38









                        ely

                        2,57111731




                        2,57111731




















                            up vote
                            2
                            down vote













                            I wouldn't worry about cheating at this point at all.



                            You will have lots of potential candidates, and you want to filter out those that are useless. The test filters out those that are useless, except the cheaters. So don't take this test as evidence of any ability, just as the key to coming to the in person interview.



                            You could as the first question in the interview ask about the test. My current company used codility.com (I think) and a cheater would have been completely, absolutely, stuck when asked about one of the problems posed (which was a very cleverly hidden logic bug).



                            The idea is that the test is for filtering out people where an interview is a waste of your time. Cheaters will be able to waste your time, but they won't be able to get a job. If you are really worried about cheaters, take a photo of each candidate during the interview because I know of cases where the person starting the job was not the person doing the interview.






                            share|improve this answer
























                              up vote
                              2
                              down vote













                              I wouldn't worry about cheating at this point at all.



                              You will have lots of potential candidates, and you want to filter out those that are useless. The test filters out those that are useless, except the cheaters. So don't take this test as evidence of any ability, just as the key to coming to the in person interview.



                              You could as the first question in the interview ask about the test. My current company used codility.com (I think) and a cheater would have been completely, absolutely, stuck when asked about one of the problems posed (which was a very cleverly hidden logic bug).



                              The idea is that the test is for filtering out people where an interview is a waste of your time. Cheaters will be able to waste your time, but they won't be able to get a job. If you are really worried about cheaters, take a photo of each candidate during the interview because I know of cases where the person starting the job was not the person doing the interview.






                              share|improve this answer






















                                up vote
                                2
                                down vote










                                up vote
                                2
                                down vote









                                I wouldn't worry about cheating at this point at all.



                                You will have lots of potential candidates, and you want to filter out those that are useless. The test filters out those that are useless, except the cheaters. So don't take this test as evidence of any ability, just as the key to coming to the in person interview.



                                You could as the first question in the interview ask about the test. My current company used codility.com (I think) and a cheater would have been completely, absolutely, stuck when asked about one of the problems posed (which was a very cleverly hidden logic bug).



                                The idea is that the test is for filtering out people where an interview is a waste of your time. Cheaters will be able to waste your time, but they won't be able to get a job. If you are really worried about cheaters, take a photo of each candidate during the interview because I know of cases where the person starting the job was not the person doing the interview.






                                share|improve this answer












                                I wouldn't worry about cheating at this point at all.



                                You will have lots of potential candidates, and you want to filter out those that are useless. The test filters out those that are useless, except the cheaters. So don't take this test as evidence of any ability, just as the key to coming to the in person interview.



                                You could as the first question in the interview ask about the test. My current company used codility.com (I think) and a cheater would have been completely, absolutely, stuck when asked about one of the problems posed (which was a very cleverly hidden logic bug).



                                The idea is that the test is for filtering out people where an interview is a waste of your time. Cheaters will be able to waste your time, but they won't be able to get a job. If you are really worried about cheaters, take a photo of each candidate during the interview because I know of cases where the person starting the job was not the person doing the interview.







                                share|improve this answer












                                share|improve this answer



                                share|improve this answer










                                answered Jan 15 '17 at 10:42









                                gnasher729

                                71.7k31133225




                                71.7k31133225




















                                    up vote
                                    1
                                    down vote













                                    Your goal with a programming test, like an interview or any other hiring process, is to help determine if someone is a good fit for the company. If you need verification that they actually took the test, (and assuming you can control the contents of the test) you can accept the test as-is and bounce off it in future interviews.



                                    For instance, you could ask in a test to develop a simple program that requires a couple different data structures in a language where there are several classes to fit that bill, then ask in an interview why they chose that one over another. If they gave you "perfect" code that implies strong competency with the language, then can't tell you why they chose an StringBuilder over a String or some other type, then you probably have your red flag.






                                    share|improve this answer
























                                      up vote
                                      1
                                      down vote













                                      Your goal with a programming test, like an interview or any other hiring process, is to help determine if someone is a good fit for the company. If you need verification that they actually took the test, (and assuming you can control the contents of the test) you can accept the test as-is and bounce off it in future interviews.



                                      For instance, you could ask in a test to develop a simple program that requires a couple different data structures in a language where there are several classes to fit that bill, then ask in an interview why they chose that one over another. If they gave you "perfect" code that implies strong competency with the language, then can't tell you why they chose an StringBuilder over a String or some other type, then you probably have your red flag.






                                      share|improve this answer






















                                        up vote
                                        1
                                        down vote










                                        up vote
                                        1
                                        down vote









                                        Your goal with a programming test, like an interview or any other hiring process, is to help determine if someone is a good fit for the company. If you need verification that they actually took the test, (and assuming you can control the contents of the test) you can accept the test as-is and bounce off it in future interviews.



                                        For instance, you could ask in a test to develop a simple program that requires a couple different data structures in a language where there are several classes to fit that bill, then ask in an interview why they chose that one over another. If they gave you "perfect" code that implies strong competency with the language, then can't tell you why they chose an StringBuilder over a String or some other type, then you probably have your red flag.






                                        share|improve this answer












                                        Your goal with a programming test, like an interview or any other hiring process, is to help determine if someone is a good fit for the company. If you need verification that they actually took the test, (and assuming you can control the contents of the test) you can accept the test as-is and bounce off it in future interviews.



                                        For instance, you could ask in a test to develop a simple program that requires a couple different data structures in a language where there are several classes to fit that bill, then ask in an interview why they chose that one over another. If they gave you "perfect" code that implies strong competency with the language, then can't tell you why they chose an StringBuilder over a String or some other type, then you probably have your red flag.







                                        share|improve this answer












                                        share|improve this answer



                                        share|improve this answer










                                        answered Jan 30 '14 at 15:00









                                        bradleyjx

                                        91




                                        91




















                                            up vote
                                            1
                                            down vote













                                            Two other options not listed above are:



                                            1. Have the requirements vague. Mention you need to compare strings, but don't mention if the comparison needs to be case or language sensitive. Use dates and times, and see if the programmer asks about time zones and leap years. This gives you an idea that the person knows enough to ask the right questions -- if they go straight to a Google search, you know that they might not be the right person.


                                            2. Ask them to tweak the program during the interview. Tell them, "the requirements changed -- the end user asked if you can give them this extra information. How would you alter your code to provide this?" You could tell that some people didn't feel comfortable maintaining the code they submitted to you (which strongly suggests that someone did the work for them or they found it online and didn't totally understand the code).






                                            share|improve this answer






















                                            • Updated answer based on comments
                                              – bryanjonker
                                              Mar 3 '16 at 18:02














                                            up vote
                                            1
                                            down vote













                                            Two other options not listed above are:



                                            1. Have the requirements vague. Mention you need to compare strings, but don't mention if the comparison needs to be case or language sensitive. Use dates and times, and see if the programmer asks about time zones and leap years. This gives you an idea that the person knows enough to ask the right questions -- if they go straight to a Google search, you know that they might not be the right person.


                                            2. Ask them to tweak the program during the interview. Tell them, "the requirements changed -- the end user asked if you can give them this extra information. How would you alter your code to provide this?" You could tell that some people didn't feel comfortable maintaining the code they submitted to you (which strongly suggests that someone did the work for them or they found it online and didn't totally understand the code).






                                            share|improve this answer






















                                            • Updated answer based on comments
                                              – bryanjonker
                                              Mar 3 '16 at 18:02












                                            up vote
                                            1
                                            down vote










                                            up vote
                                            1
                                            down vote









                                            Two other options not listed above are:



                                            1. Have the requirements vague. Mention you need to compare strings, but don't mention if the comparison needs to be case or language sensitive. Use dates and times, and see if the programmer asks about time zones and leap years. This gives you an idea that the person knows enough to ask the right questions -- if they go straight to a Google search, you know that they might not be the right person.


                                            2. Ask them to tweak the program during the interview. Tell them, "the requirements changed -- the end user asked if you can give them this extra information. How would you alter your code to provide this?" You could tell that some people didn't feel comfortable maintaining the code they submitted to you (which strongly suggests that someone did the work for them or they found it online and didn't totally understand the code).






                                            share|improve this answer














                                            Two other options not listed above are:



                                            1. Have the requirements vague. Mention you need to compare strings, but don't mention if the comparison needs to be case or language sensitive. Use dates and times, and see if the programmer asks about time zones and leap years. This gives you an idea that the person knows enough to ask the right questions -- if they go straight to a Google search, you know that they might not be the right person.


                                            2. Ask them to tweak the program during the interview. Tell them, "the requirements changed -- the end user asked if you can give them this extra information. How would you alter your code to provide this?" You could tell that some people didn't feel comfortable maintaining the code they submitted to you (which strongly suggests that someone did the work for them or they found it online and didn't totally understand the code).







                                            share|improve this answer














                                            share|improve this answer



                                            share|improve this answer








                                            edited Mar 3 '16 at 17:48

























                                            answered Mar 3 '16 at 17:09









                                            bryanjonker

                                            1114




                                            1114











                                            • Updated answer based on comments
                                              – bryanjonker
                                              Mar 3 '16 at 18:02
















                                            • Updated answer based on comments
                                              – bryanjonker
                                              Mar 3 '16 at 18:02















                                            Updated answer based on comments
                                            – bryanjonker
                                            Mar 3 '16 at 18:02




                                            Updated answer based on comments
                                            – bryanjonker
                                            Mar 3 '16 at 18:02










                                            up vote
                                            0
                                            down vote













                                            This may not be exactly the answer that you're looking for, but I handle this by giving developer candidates a simple 10-question quiz at the beginning of an in-person interview. The questions are geared toward the expected skill level for the position and cover concepts, not code samples. For example, I may ask a junior .NET developer candidate to describe the difference between a method and property, or to explain what LINQ does and how it's an improvement over its predecessors. I may ask a .NET architect candidate to explain boxing, its impact, and how it can be mitigated.



                                            This is obviously not as good as a programming assignment, but it does give me a reasonably good view of the candidate's knowledge of the concepts and technologies that he/she will be using. It also gives me good insight into their written communications skills as well.



                                            If the candidate survives this test, I expect any deeper questions about technique, etc to shake out in the second interview with the development team.






                                            share|improve this answer




















                                            • Boxing seems like a very very poor example of a cognitive technical question.
                                              – RandomUs1r
                                              Dec 22 '17 at 21:35














                                            up vote
                                            0
                                            down vote













                                            This may not be exactly the answer that you're looking for, but I handle this by giving developer candidates a simple 10-question quiz at the beginning of an in-person interview. The questions are geared toward the expected skill level for the position and cover concepts, not code samples. For example, I may ask a junior .NET developer candidate to describe the difference between a method and property, or to explain what LINQ does and how it's an improvement over its predecessors. I may ask a .NET architect candidate to explain boxing, its impact, and how it can be mitigated.



                                            This is obviously not as good as a programming assignment, but it does give me a reasonably good view of the candidate's knowledge of the concepts and technologies that he/she will be using. It also gives me good insight into their written communications skills as well.



                                            If the candidate survives this test, I expect any deeper questions about technique, etc to shake out in the second interview with the development team.






                                            share|improve this answer




















                                            • Boxing seems like a very very poor example of a cognitive technical question.
                                              – RandomUs1r
                                              Dec 22 '17 at 21:35












                                            up vote
                                            0
                                            down vote










                                            up vote
                                            0
                                            down vote









                                            This may not be exactly the answer that you're looking for, but I handle this by giving developer candidates a simple 10-question quiz at the beginning of an in-person interview. The questions are geared toward the expected skill level for the position and cover concepts, not code samples. For example, I may ask a junior .NET developer candidate to describe the difference between a method and property, or to explain what LINQ does and how it's an improvement over its predecessors. I may ask a .NET architect candidate to explain boxing, its impact, and how it can be mitigated.



                                            This is obviously not as good as a programming assignment, but it does give me a reasonably good view of the candidate's knowledge of the concepts and technologies that he/she will be using. It also gives me good insight into their written communications skills as well.



                                            If the candidate survives this test, I expect any deeper questions about technique, etc to shake out in the second interview with the development team.






                                            share|improve this answer












                                            This may not be exactly the answer that you're looking for, but I handle this by giving developer candidates a simple 10-question quiz at the beginning of an in-person interview. The questions are geared toward the expected skill level for the position and cover concepts, not code samples. For example, I may ask a junior .NET developer candidate to describe the difference between a method and property, or to explain what LINQ does and how it's an improvement over its predecessors. I may ask a .NET architect candidate to explain boxing, its impact, and how it can be mitigated.



                                            This is obviously not as good as a programming assignment, but it does give me a reasonably good view of the candidate's knowledge of the concepts and technologies that he/she will be using. It also gives me good insight into their written communications skills as well.



                                            If the candidate survives this test, I expect any deeper questions about technique, etc to shake out in the second interview with the development team.







                                            share|improve this answer












                                            share|improve this answer



                                            share|improve this answer










                                            answered Jan 30 '14 at 15:04









                                            Roger

                                            7,17132644




                                            7,17132644











                                            • Boxing seems like a very very poor example of a cognitive technical question.
                                              – RandomUs1r
                                              Dec 22 '17 at 21:35
















                                            • Boxing seems like a very very poor example of a cognitive technical question.
                                              – RandomUs1r
                                              Dec 22 '17 at 21:35















                                            Boxing seems like a very very poor example of a cognitive technical question.
                                            – RandomUs1r
                                            Dec 22 '17 at 21:35




                                            Boxing seems like a very very poor example of a cognitive technical question.
                                            – RandomUs1r
                                            Dec 22 '17 at 21:35










                                            up vote
                                            0
                                            down vote













                                            Ask questions that don't have canned answers, let me give you an example:



                                            1. What is the difference between a class and an interface?
                                              Google has everything you need to answer this, so you're testing memorization here, not skill.


                                            2. When would you use a class vs an interface?
                                              Much more interesting answers that demonstrate skill, experience, and understanding.


                                            A technical test full of 2. type questions would be much harder to cheat on and give a better assessment of the candidate's skills.






                                            share|improve this answer
























                                              up vote
                                              0
                                              down vote













                                              Ask questions that don't have canned answers, let me give you an example:



                                              1. What is the difference between a class and an interface?
                                                Google has everything you need to answer this, so you're testing memorization here, not skill.


                                              2. When would you use a class vs an interface?
                                                Much more interesting answers that demonstrate skill, experience, and understanding.


                                              A technical test full of 2. type questions would be much harder to cheat on and give a better assessment of the candidate's skills.






                                              share|improve this answer






















                                                up vote
                                                0
                                                down vote










                                                up vote
                                                0
                                                down vote









                                                Ask questions that don't have canned answers, let me give you an example:



                                                1. What is the difference between a class and an interface?
                                                  Google has everything you need to answer this, so you're testing memorization here, not skill.


                                                2. When would you use a class vs an interface?
                                                  Much more interesting answers that demonstrate skill, experience, and understanding.


                                                A technical test full of 2. type questions would be much harder to cheat on and give a better assessment of the candidate's skills.






                                                share|improve this answer












                                                Ask questions that don't have canned answers, let me give you an example:



                                                1. What is the difference between a class and an interface?
                                                  Google has everything you need to answer this, so you're testing memorization here, not skill.


                                                2. When would you use a class vs an interface?
                                                  Much more interesting answers that demonstrate skill, experience, and understanding.


                                                A technical test full of 2. type questions would be much harder to cheat on and give a better assessment of the candidate's skills.







                                                share|improve this answer












                                                share|improve this answer



                                                share|improve this answer










                                                answered Dec 22 '17 at 21:42









                                                RandomUs1r

                                                68929




                                                68929















                                                    protected by Community♦ Dec 3 '17 at 4:58



                                                    Thank you for your interest in this question.
                                                    Because it has attracted low-quality or spam answers that had to be removed, posting an answer now requires 10 reputation on this site (the association bonus does not count).



                                                    Would you like to answer one of these unanswered questions instead?


                                                    Comments

                                                    Popular posts from this blog

                                                    What does second last employer means? [closed]

                                                    List of Gilmore Girls characters

                                                    Confectionery