I feel like all the ethics training I’ve done as a grad school has not set me up well for the ‘real world.’ Or at least, has me looking at everything really different and not understanding how user studies work outside of academia.
In academia, it feels like if I even want to ask for some feedback, I need ethics approval. Which is not true – I can have friends help me with experiment design and doing a run-through – as long as I do not use any of the data they may give. But, the rules still make you paranoid enough (or at least me paranoid enough) to be very careful about what you do.
However, with our start-up, we need to some (a lot) of user testing. Because, while we have what we think is a great idea, without this data, we don’t know for sure. And it’s a lot of work to put in – so we’d rather have some data to back us up. But, getting data is tricky. Tricky, because I don’t really understand how you go about running user studies outside of academia. And when I’ve tried to ask some of the mentors and people we talk to, it seems like no one really knows, and just sorts of does them.
I’ve heard of people testing their product, by taking their app to places where people are lined up waiting for something (like to get into a library) and asking them to take a look at it while they wait. Which seems simple enough, and I understand how that is unlikely to cross a line.
But what if you need to get test data from children? Because your app isn’t for the 18+ crowd. And you wanted teachers to try it in schools. What then? Having teachers try simple versions (without any data collection) and instead reporting qualitative results (“the kids were really engaged”, “they seemed to have a lot of fun”) seems okay. And easy enough to do (once you have the connection to a teacher or two). But, what about the next step, and actually storing data during the sessions?
It turns out it’s really really hard to find any answers for these questions. I’ve looked. And asked. While ethics applications and the whole procedure can often feel really annoying, overwhelming, and useless, it does have it’s purpose. And at least, at the end of the day, when you have your approval, you know you’re okay to proceed.