User studies and Murphy’s law

Depending on what time you read this (assuming it’s the day I post), I will either be getting ready for, running, or recovering from running a user study. I have ~50 participants all lined up to finally test my experiment tomorrow. Fingers crossed everything works.

I firmly believe that running a user study is tempting Murphy’s Law – “If anything can go wrong, it will.” And this week, preparing for it, has been one of those weeks where that has been ringing true to often. So hopefully everything will run smoothly today, and things will start to fall into place.

I had another grad student ask me recently for advice with running user studies. The key thing I told him, is to expect things to go wrong. And to realize, that as much as you test things, someone is going to try to do something you never expected/wanted them to do, and it will break something. That because of this, expect to throw out some user data, for a variety of reasons, and therefore try and get way more participants then you think you need. Always better to have extra, than not enough and have to try and recruit more afterwards.

This week, I’ve had random errors pop in my experiment (I think/hope/believe they’re all sorted out now). I’ve had problems with logging in and testing in the right computer lab, but that’s all been dealt with. I have a time crunch when it comes to setting up the experiment. There’s a lab in the room right up until the experiment starts, which means I don’t get any prep time to prepare. Instead, I will have to set things up as participants are showing up/already there. Fun times.

On the other hand, by late afternoon, I’ll have a bunch of data to analyze, which will mean I can finally figure out where my research is heading. I’m hoping that the results fall in our favor (the preliminary results we’ve gathered suggest they will) but this will be the ultimate test. And assuming everything goes okay tomorrow (and with the data), we have a couple more pools of participants we want to tap afterwards, which will likely double or triple (or even more) our total number of participants.

It’s been a few crazy weeks here, between getting things ready for a workshop, doing final prep on my experiment and having to finalize accepting papers and collecting reviews for a conference. I’m so looking forward to this weekend. Of course, I’ve got stuff scheduled all weekend, my parents are visiting, and I’m suppose to analyze my data for Tuesday! Let’s see if I’m any more sane and relaxed by Monday.


Join the discussion

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s