Playtest Surveys - Tips & Tricks
We’re about to go a lot broader with Nut Hunt – I made a PNP version of the game, and we’re moding a TTS version. This means a lot more not in person playtests, and a lot more surveys.
I’ve been talking through what should go in a survey with some other designers and publishers, and combined with my experience running playtests for Nut Hunt – these are my best practices, tips, and tricks for putting together a great playtest survey.
Playtesting at Gen Con 2021
Optional Surveys Even at Live Playtests
You should be ready with both verbal & written questions because people communicate & feel comfortable to express themselves differently. - Seppy Fight in a Box Games
Historically I haven’t offered surveys at live playtesting events. I like interacting with our players, asking them questions and engaging. Seppy over at Fight in a Box (check them out) opened my eyes to the other side of that coin: not everyone is comfortable communicating in my preferred method.
I’ll be thinking hard about incorporating written surveys into our live playtesting, and how to effectively communicate their optionality – so that they don’t discourage verbal communicators from talking things through and ignoring the written aspect.
Keep it Short
This might be personal preference – but as a playtester I very much dislike long multi-page surveys. As a publisher, our playtesters are doing us a favor in playing our game – we should be respectful of their time.
If people want to spend more time giving feedback, they will – and there are plenty of ways to give them the opportunity to interact more. But they should not feel obligated to fill out a long form.
Prioritize the Focus
The first key to keeping it short is to prioritize what you are focusing on in each playtest. Early on you might want feedback on each of your mechanics. Maybe you want to understand how much agency players felt like they have and perceived variance.
Later you might focus on UI/UX issues, and rules clarity. And finally, in your later playtests it might be about polishing the game, understanding your target demographic, and consumer pricing expectations.
As your game evolves so will your playtests, and so will the questions you should be asking.
Have Multiple Surveys
The second key I’ve found to keeping it short, is to not be afraid of having multiple surveys that ask different questions. The goal is to do enough playtesting to get good data – even if not everyone is being asked every question.
3 Questions We Always Ask
What was your least favorite part of the game? There is nothing worse than a playtester not liking something, and not telling you. Our goal is to make our games as amazing as possible, and so this negative feedback is exactly what we are looking for.
What is your favorite game? I’m still trying to find the right wording to get what I want out of this question – but the goal is to get some sense of player preferences in order to contextualize other survey answers. If someone hates area control games, and they hate your area control game that isn’t nearly as instructive as someone who loves trick taking games hating your trick taking game.
Anything else you want us to know? I always provide a space for open ended feedback.
Goldilocks Scale
Board Game Design Lab had a good podcast on an approach for scoring that they use in the example I sent you, the Goldilocks approach (1-2-3-2-1). Rather than try to figure out what score is good on a scale of 1-5, this system invites players to say if each element was too much/not enough/just right. Plus, I never rate anything a 5 since I have a high standard for that, but a "just right" 3 is welcoming to me. - Pax Unplugged Playtester
In earlier playtesting I often used multiple choice questions, or a 1-5 rating grid. One of our playtesters recommended using a goldilocks scale – and now I’m a big proponent of the idea.
A goldilocks scale asks survey takers to rate aspects from 1) Too Little, 3) Just Right, 5) Too Much.
I think the scale is much more intuitive than other rating systems, and allows us to more efficiently ask questions. It can be a little tricky to find the right wording for our questions – but I think it is worth the added effort.
What are your top trips and tricks for effective playtesting?
