Here is the flaw in this line of reasoning:
As a career Software Engineer with a BS in Comp Sci and 3 decades of getting paid to develop software, I can tell you why coding tests are not a measure of one’s ability. It only measures up-to-the-moment memory of a specific algorithm (in the case of silly coding challenge tests) or a specific language.
When I got my degree in Computer Science, we were taught that there was a meaningful difference between the future career professionals who spent the required 4 years or more getting a degree like ours and the “ITT Tech” grads or the “script kiddies” with 3 or 6 months of total programming experience to their name (or nowadays the churn-and-burn major contracting houses of some overpopulated countries that recruit non-programmers off the street, give them a few video courses, then send them overseas slave-labor with an H1-B to generate income for them). We were told that taking all those numerous Comp. Sci programming labs, computer courses, Analysis of Algorithms, courses in databases and file structures, compilers theory and building, and a seeming endless parade of advanced math courses (4 quarters of Calculus, Discrete Math, Linear Algebra, Group Theory, Turing analysis, etc, etc) – that all of this would set us apart. It would make us career computer science software engineers.
So you graduate and go on to spend the next few years, then a decade, then several decades doing just that - using all manner programming languages, algorithms, frameworks, APIs, platforms, across multiple various computer architectures and systems, etc. A true journeyman that you can give a task to and end up with a complete solution. Then comes the problem with these tests:
- Did I write elaborate graphical C-based X-Windows applications on Unix back in the day? Yes. Spent many months doing that, read and coded my ass off for 70-90 hours weeks and was very good at it. Can I write a 10 or 20 line X-Windows app today? No. Don’t remember.
- Did I work extensively in Visual Basic and Visual C++ with MFC using Visual Studio for years? Yes. Did I do some really great work, using best practices after lots of extensive research, generating some pretty great software? Yes. Can I code a 10 line VB program right now? No. Don’t remember.
- Did I work VAX 11/780 macro assembler and generate some of the cleanest most elegant assembly programs around? Sure. Remember it? Nope.
- COBOL? Yes. Remember? Nope.
- JYACC/JAM? Yes & Nope.
- JavaScript? Yes, about 5 years. Remember most of it? Nope.
And on and on ad infinitum.
However, if I take a week or two to deep dive back into any one of these areas and start to flex that muscle memory, I will be back doing great work in no time at all. The point is that I CAN do the work, and do it very well. But having instant recall of everything I’ve ever done throughout my entire career is not something I or anyone else is capable of.
So this means that companies that focus on “can you write 10 lines of code in Objective-C” and using that as their “filter” are woefully ignorant of what makes a great and successful software engineer/developer. Another example is that I spent over 3 long years (70-90 hr work weeks) writing sophisticated iOS apps for a company and would frequently optimize other developer’s code, etc. Before I worked on iOS, I had no direct experience in it. Then I became very experienced and was very successful at it. However, I haven’t used it in a couple of years, and I can already feel the effects of the “Immediacy of recall” fading. Sure, ask me some general questions about various categories of APIs, or how the Xcode IDE works, or what is the process for getting apps uploaded and approved and I can speak to that. But ask me what the parameters of a given API function are and I will fail your test. Do you see the pattern? Its NOT about what you are retaining in you head at this very moment, its about how SUCCESSFUL you’ve been when given any and all tasks in the past. Heck, I’ve done IVR scriping, programmed CSU/DSU units, wrote firewall code, uploaded config tables into network switches, written sophisticated fuzzy logic algorithms for commercial software packages, etc - but I can’t replicate any of that right here as I sit without rolling up my sleeves and “climbing back down into it”.
And THAT is the problem with these silly logic puzzles and 10-line coding tests in a specific language. You may get roughly correct answers from a person who has never programmed in their life, but just spend the past weekend watching Lynda/Coursera/YouTube videos in a particular language and being able to write a loop construct. You may also get a complete failure when asking a senior career developer that same question because they haven’t worked in the environment for a period of time (but were crazy mad scientists in it at some point previously).
Experienced software devs just smile and chuckle at all the constant rehashing and re-labeling of the same old concepts they understood 10, 20 , and 30 years ago, that are merely re-packaged with a shiny new name (or cloned git repo). Today’s JQuery, Node.JS, Ruby on Rails, etc, is yesterday’s PHP, which is the day before’s Perl, which is the day before’s shell scripts, etc. Today’s GO is yesterday’s Lisp, and on and on.
So which do you really want? An army of one-trick script kiddies with hardly any varied experience chasing that chase the very latest webdev or language “flavor of the day” and act like its the “greatest thing since sliced bread”? Or someone more seasoned who has seen and done so much more that treats all these things as just tools in their toolbox. Its like asking a well-seasoned journeyman carpenter if they’ve used the very latest “X25-Titanium Acmetron” hammer and failing them if they didn’t. They would just look at you and think: “It’s just a f*king hammer buddy, and I’ve used that think for 30 years.” But the naive hiring person would say, "but no… the “X25-Titanium Acmetron” is the “greatest thing since sliced bread” and if you haven’t used this particular model, then you can’t work here. Similarly, asking a seasoned career software engineer if they’ve used a specific new version of Oracle or if they have know the very latest API enhancements in Swift of Obj-C that Apple churns out every year is silly. If you task them with writing an iOS app, the good ones will deep dive and learn everything they need to learn to do excellent work. They will scan all the available API calls to do a certain task, look at 3rd party APIs if they needs something more, and generally knock it out of the park. Give that same task to a “3-monther” and they will robotically spew information they learned yesterday about design patterns and APIs. A seasoned developer will smile about the fact that they were doing MVC before it has a name, or working on object-oriented code in C even before C++ came into use.
TL;DR - You can’t really judge a candidate’s true ability, skill, or breadth of experience based on whether they can write you a 10 line program in your favorite language of the day. Present-moment knowledge has nothing to do with pass success in a given field. You wan’t the best Perl developer? All you’ll ever get is Perl code. A JQuery master? Same. Like a carpet installer, all you’ll get from them is carpet installation. But get yourself a true software developer with a long and successful track record and it vastly opens up how successful this person will be at your organization. They can do it all - just like a master craftsman is able to design and build any manner of furniture they may be asked, as opposed to someone who can only “assemble” boxed parts from IKEA.
and I saw yours.