I was watching some program on the television once, and it was talking about how people train to be professional counterfeit identifiers by studying real money first, in great depth, becoming so familiar with the real thing, that by the time they are introduced with all manner of fakes, the fakes are then easier for them to spot. I can’t even tell you how long ago that was because I was a kid, but this simple concept had a real impact on me and changed how I study and learn forever.
Here is how I always try to learn: First, have a good foundation of true and correct information to start with. Then, build upon that information with good structure for introducing larger concepts. Introduce traps, tricks, incorrect information, or what not to do, after seeing the example of how it should happen first. Think about it. You don’t introduce someone to the concept of driving a car by showing them videos of crashes. First, you show the goal: Vehicles help you get from one place to another, safely. Then you teach by showing people how to keep their vehicle inside the lanes, not out of them. You only introduce crash videos as a cautionary and educational device for what not to do. As a result, when you take your final driving test, hopefully the experience is fun and focused on staying on the roadway and following all safety procedures and laws.
I feel that test taking actually works in conjunction with the learning process. Taking a test is not just about filling in the blanks or spouting out information to get a certificate or license. You can learn a lot about yourself with the act of testing and that’s why test taking can be fun too. Ideally the test taking experience is a confidence builder. This is especially true of practice tests. I love practice tests! Seriously. There are three reasons for this:
- I evaluate the test process itself. – The way the practice questions are laid out gives me some idea of what to expect, so I have a peek at evaluating the style, format, and general feel of the actual test and how to make sense of it. For computerized tests, I also appreciate the opportunity to familiarize myself with software features on the platforms to optimize my own success. I always like to familiarize myself with how to flag questions for review. If I have the option of skipping questions so that I can focus on what I can do, I can still be successful without running out of time. Then I go back to work on those questions I was less sure of, knowing most of my answers are already filled in.
- I gain confidence. – Putting myself in the mindset and being able to take a practice test helps my confidence and reduces anxiety for when I have to do the real thing, because it feels more like I have been there before. You may also find that when you feel prepared, you also perform better.
- I learn my weak spots. – When I miss practice test questions, I actually get really excited, because that reveals to me what my weak spots are that I need to study. The more feedback I get that helps me know where to focus my attention, the easier it is for me to see what I need to study and from what direction mistakes may be coming.
Notice how I showed you first what I like about tests? (I did that on purpose.) From the title, you might have been thinking this was just a complainer piece, but I am not simply complaining. I do like to point out when something isn’t working for a purpose. Think of it as free feedback when considering how to make things better. And we all know that before telling people what you don’t like, it’s always nice to acknowledge the positive aspects of what did work. I do appreciate actual practice and tests, especially when they have been presented well and constructively monitored – the same way a driving instructor rides along before you’re ever put into your final test alone.
First, to the test makers: I want to acknowledge that I know if you are the one in the room tasked with creating a test for others, you probably really are very smart and know your stuff, and that’s great! But speaking on the side of the test taker, being knowledgeable and qualified doesn’t mean you know how to communicate, teach, or present that knowledge for others. Don’t worry, it’s not the end of the world. You can learn. Especially if you do it on purpose. Intelligent people never truly stop learning and looking for ways to improve upon things. Since you’ve taken the time to read this far, I would guess that chances are you fall into the category of people who are curious and want to make improvements. Or maybe you were just bored! But if it’s the former, good for you. Growth should never stop.
Now, please allow to me to vent my frustrations – 90’s Teen Magazine Style! Here are – (fanfare please) The Ten Things I Hate About Your Test!
- Your grammar, punctuation, and spelling were either downright wrong, or they were inconsistent throughout your test. If you don’t care enough to proofread your test, why should I trust you to know what the question is, let alone the answers?
- Your formatting was all messed up. Don’t split questions across pages, separate questions from answer spaces, or make me question fonts and flips just to understand what you’re asking. It’s irritating and it makes it really hard for me to review my answers.
- Your matching questions were a puzzle instead of a test question. When I finally figured it out, there was no space for me to fill in my answer! Am I supposed to draw lines? Draw pictures? Communicate by emoji? You should have said so. If you want lines, ask for lines. If you want letters, give me a space to write them. Don’t make me guess the directions.
- Your questions were vague. Either that, or your answers were. If multiple answers can reasonably be correct that is not a problem with me – it’s the question.
- You tested me using language we didn’t review. Acronyms, jargon, and unfamiliar terminology should be kept in their appropriate categories and not be surprising on a test, unless you’re testing my ability to accurately interpret them. You said “DOA.” Did that mean “dead on arrival” or “date of arrest”?
- You tested trivia instead of competence. I don’t need to memorize the font size on page 47 that tells me to “call John first.” It’s more important that I know who John is and how it relates to me and what I do.
- You use double negatives! “Which statement is not false?!” Geez, you could have at least made NOT in all caps. It’s better to just ask me plainly: “Which statement is true?” Confusing people doesn’t test their competence, it makes me question yours!
- You gave me no room to show my work or write a complete answer. If you ask an essay question, you should provide me enough lines and space to get an appropriate answer. And you should be specific about which major points I need to include, otherwise how do I know you’re even grading this based on facts versus just following your mood? Did you even create an answer key for your shitty test?
- You told me to “choose all that apply” and then included “all of the above” in the same question. By the way, did you know that sometimes the correct answer also included statements that were true, but were placed below the “all the above” answer? Hell no. Please get it straight.
- You didn’t test your own test. You never proofread, reviewed, or checked to see if your test questions were comprehensive, or if they were actually in any presented material – and I can tell.
The cost of bad test development and training.
Yes, that list was meant be kind of a funny rant, but it’s also very real.
If you’re a supervisor, training coordinator, teacher, or someone in charge of giving a test on some material and you have created a bad test with basic mistakes, it makes you look unprofessional, and it causes doubt of your training and knowledge. You might be the smartest and most capable person in the room on that subject, so do you really want people doubting you because you presented an ill prepared test that you could have easily fixed? People will quickly lose respect for you if they can tell you didn’t even use spell check, which is quick, free, and easy. (If you’re using Microsoft Word, all you have to do is hit F7.)
Bad tests cause confusion for the test takers and actually shakes the confidence of trainees because they also want to know they are being trained on competence and real comprehension, not just jumping over hurdles to get released to work without a trainer or supervisor present. If the test is full of loopholes, so that basically anyone can pass, what does that really do? This doesn’t give anyone proof of learning. Worse, it can create more problems down the road. When this person who was supposed to be tested and passed as fully trained is now expected to be able to handle their job solo, there is more pressure on them. Are your people asking themselves if they were truly trained well enough for this? Are they constantly asking questions and acting like it’s their first day, or like they never learned anything?
Then there are your customers and/or callers. If you work in public safety, the stakes are pretty high that if there are significant knowledge gaps that weren’t caught, the person answering the call might inadvertently make things worse. If you’re a caller on the line with emergency services, you want to know the person on the other end of the line is doing the best they can based on real training, not just a strong desire to help. And if you’re a field responder like a police officer or fire/med services, don’t you also want to know that the person you rely on at the other end of the radio or telephone has met some standards of knowledge and training so they can meet your basic needs?
If an operator’s performance is ever questioned in an incident for review or lawsuit, how employees have been trained and tested for competence can also become a question. If you’re the agency tasked with meeting a standard, are you prepared to say under oath that your employees are trained with consistency, fair scoring, and fair evaluation standards? Have you considered how sloppy training and testing may color the way people think about your agency or profession as a whole?
Do something about it.
Testing matters. A good test shows a trainer and a trainee exactly what’s understood and what still needs work. A bad test just creates confusion — and then blames the learner for being confused. That’s backwards, and honestly, it’s a shame, because learning is supposed to be a little fun.
Here’s my suggestion: next time you’re taking a test, or even just a survey, pay attention to what worked and what didn’t. What confused you? What made you want to throw your pencil? Hang onto that. It’ll make you better the next time you’re the one building something for someone else to sit through.
Want to Go Deeper? Here’s Where I’d Start
If you want the real research behind everything I just griped about, here’s the good stuff:
Standards for Educational and Psychological Testing — the actual professional standard, built jointly by the American Educational Research Association, the American Psychological Association, and the National Council on Measurement in Education. This is as official as it gets on testing quality and fairness. You can find it through NCME (ncme.org/resources/books/testing-standards) or, if you want something a little easier to scan, testingstandards.net.
Constructing Written Test Questions for the Basic and Clinical Sciences by Susan M. Case and David B. Swanson. Yes, it’s written for medical schools, not dispatch centers. Doesn’t matter. Clear questions, no trick wording, testing what someone can actually do instead of what they memorized — that stuff works everywhere.
Thanks for reading and good luck!