14th September 2018 1 By Jonny

If there’s one thing the IT industry has a lot of, it’s certifications. There is a bewildering array of certifications available for people who like collect the spaghetti soup of letters on offer. Some of these are undoubtedly more prestigious than others. Obviously, any certifications I personally hold are incredibly prestigious and well viewed within the industry as a whole, and those I don’t hold are somewhat lightweight and irrelevant (until I hold them at which point they become incredibly valuable).

Stepping into my DeLorean

Probably about the millionth blog post to include a ‘Back to the Future’ tag. Thinking back to when I was younger than I am now – approximately 22 years ago as of writing – and I was finishing university and (naviely) believing that exams* were pretty much behind me. Like most graduates I expect, I had no idea what exactly I was going to do, and if I could have a good chat to my younger self I’d certainly have a lot to say. I’d probably also mention in passing that a career in IT is for the most part fun, reasonably well paid, and offers plenty of opportunities for travel and experience. I’d also note at that point that travelling with work really isn’t all it’s cracked up to be … a hotel here is very much like a hotel there. Oh yeah, and IT companies tend to be cheap, so get used to Travel Taverns

I’d also note that my thought that studying for exams as a thing of the past is really not true at all. Listing out the various qualifications that I have held (or hold) over time looks as follows:

  • MCP
  • MCSE
  • CNA
  • CNE
  • MCNE
  • CCA
  • CDE
  • LPIC-1
  • LPIC-2
  • CLA
  • CLE
  • RHCE
  • RHCA
  • MCSA
  • CKA

I’ve probably missed some off there, and I haven’t put the dates of those certifications, as for the most part I can’t remember them. Also, some of those certifications were updated to be kept current. It used to be the case that once you were certified that you didn’t need to keep your certification current. For example, my MCSE was for Windows NT4 (and various other bits and bobs). Even when Windows 2000 was released your MCSE in NT4 remained valid, although you could re-certify on Windows 2000. It’s now the case that certifications are retired and you have to keep them up to date to be allowed to continue to use the certification. Not necessarily a bad thing in itself, but certifications now seem to have a shelf-life of about 2 years, which does mean the treadmill feels like it’s getting faster.

Exams or tests?

As alluded to above, the method of being certified is usually described as an exam, e.g. Red Hat have their exams coded such as EX310. There is a variation in testing formats, but overall I wouldn’t describe them as exams as I was used to them. Some of this is undoubtedly pendantry on my part – however the test is scheduled by the candidate – exams for me were end of term events that a whole class/year would then have their knowledge from that term tested. Exams would be testing not only your ability to memorise the information presented (way too much of that in my opinion), but also how to apply that information and use the methods that should have been taught through that term to provide answers to previously unseen problems. It wasn’t always about getting the right answer, but demonstrating that knowledge had been learned and applied.

All of the IT certifications I have gathered have been tested on computers and, I strongly believe, marked by a computer too. For the most part, the test questions have been multiple choice, or multi-answer, and it’s very clear you’re either right or wrong. Or a good/lucky guesser. Certainly back when I achieved my MCSE – originally this stood for Microsoft Certified Solutions Engineer, although the industry (I think) has been backing away from ‘engineer’ tags lately, partly because ‘real’ engineers get a bit upset at their title being devalued. Anyway … back when I achieved my MCSE, it comprised 6 exams (the MCP was a single exam from memory) and covered topics such as networking, Windows NT4, and various other Microsoft software. The only problem with these tests was that there seemed to be a lack of depth of question pool, and the same questions would frequently appear leading to claims of there being many ‘paper MCSEs‘. Obviously, I was never a paper MCSE myself, being incredibly experienced in IT at the age of 22 or 23. This was also not helped by the proliferation of ‘brain dumps’ which were people simply reproducing the questions from the tests they sat.

It’s unfair to pick on Microsoft alone here, Novell had similar qualifications (the CNA and CNE in my list), and they too used a similar testing format. Mutliple choice and multiple answer questions. I believe most other vendors at the time employed similar testing techniques. I’m also suspicious that the vendors didn’t care too much about the charges of paper MCSEs etc. as they probably wanted to have as many qualified people as possible. If you have a qualification you’re more likely to recommend the technology to friends/customers, and from the vendor’s perspective, the more qualified engineers/professionals they have can be an indication as to popularity of their products.

It is worth noting that nowadays the vendors appear to be much stricter on their NDAs they make candidates agree to in terms of not divulging the contents of the exam. This is a good thing and to be encouraged.

Practice makes perfect

The other main type of test that is common is the ‘practical’ based test. These are, I expect, much more expensive to deliver than traditional multiple choice based tests, but they do feel as though they actually test you on what you know and, importantly, what you can do. The first practical based test I sat was the Novell CDE practicum – and maybe because it was the first and therefore oldest in memory, it is probably still the ‘best’ test I have sat. The novelty of the experience probably helped too, however it felt more like a real scenario. From memory, you were presented with 3 NetWare servers running NDS (maybe it was eDirectory by then), described as ‘having a problem’. I don’t think the actual problem was fully explained, so you needed to do some troubleshooting before actually fixing things. You were then told that the expectation was that you’d resolve the issue and have the 3 NetWare servers functioning and replicating the NDS tree and partitions by the end of the test.

Given the immaturity of virtualisation at that time and Internet connectivity, looking back, it was impressive to be able to deliver an exam in this fashion. The fact that it felt like a true test of knowledge and ability made it even more rewarding to pass. Sadly, the CDE didn’t really attain the relevancy it probably merited, due in part to Novell’s star waning at the time, and that the test was both difficult to pass and limited in availability.

One of the main problems with the existing testing (as I see it anyway) is that there isn’t very much in the way of jeopardy (and no, I don’t mean you get the answers, and have to give a question). What is the penalty for failure? At university, the penalty for failure was that you’d be kicked off the course. That was always a last resort though, and universities will try as hard as possible to keep people on their course, probably due to money to be honest. But, if you keep failing your exams, you’ll eventually be told that maybe you’re better off doing something else (presumably another course at university to keep the money coming in). It is so very tempting to suggest media studies as the ‘something else’ here …

Anyway, to try and get back to my point here – should I fail a test (and I’m happy to admit that there has been the occasional failure along the way), it’s no problem, I’ll just resit the test, in some cases less than a week later. There is a cost to this, of course, the tests typically cost about US$100+ per session, however, frequently employers will bear this cost, or if you work for a vendor, the cost is absorbed into the cost of running the business.

Unfortunately, the practical based tests fare worse here, as they frequently have few, if any, different scenarios, so you’re potentially facing the exact same set of tasks/questions in any resit. The practical based tests are, in my opinion, the best way of determining whether a candidate can actually do what the certificate suggests they can do, but there is an inherent weakness given that there is a lack of scenarios.

Having recently passed the Google Associate Cloud Engineer test (there’s that engineer word again …), their testing policy is that if you fail, you must wait 2 weeks to resit, and if you fail again, a longer period of waiting is required. This seems sensible to be honest.

Closing thoughts

Unsurprisingly, there is no magical method to assess candidates ability with your products via testing. If it’s multiple choice/multi-answer style tests, then you’re basically testing the ability to remember specific items on a page. Likewise, if it’s a practical based test, there is no guarantee that the candidate hasn’t had multiple attempts, and their only knowledge is the tasks on the exam.

It may be a cliché, but there really is no substitute for experience … although my younger self 22 years ago would tell present day me that’s not very fair, and that he’s younger and more motivated than present day me. Which may well be true.

I also missed out one certification … LF CSA.