Well, at the moment I don’t really know what to say about my experiment. At least not without having a potential meltdown on the way.
First up, I swear the lab I’m using is cursed. I have had problems with the lab every single time I go to run experiments. And some of these problems have led me to lose all my data from a single session. So, even though I’ve run over 90 students through the sessions, I have only around 70 pieces of data at the moment. This also means I’m not done collecting, even though I should be/want to be. I’m running more sessions today, to try to get a bit more data – if the cursed lab will allow me.
Second, from my initial analysis of the results, things aren’t so promising. In the first part, we were hoping to do less than a specific number. And while, yes, we’re less than that number, it does not appear to be significant. And when you take into account what participants said they were doing, then we fail even more – because they were just acting randomly, not actually trying to succeed.
Third, on another metric we have, while we’re doing much better than chance (we are averaging around 60% vs 20%), it’s still no where near where we were when we did the isolated experiments and were getting results in the 80s/90s. Of course, then you look and realize participants spent an average of 4 minutes on this part. At 12 items they needed to watch and label, that only works out to 20 seconds each. Even less if you factor in that they had to spend time swtiching between tasks. Sigh.
Of course, everyone who tried out my study for me, when I was getting it ready for testing, took about half an hour. These students, averaged only 20 minutes. It’s too bad there’s not an easy way to select students who are actually motivated and willing to help you and spend them time doing a good job. And, you can tell some didn’t try hard at all, they spent less than a minute completing some sections. I have a feeling I’m going to potentially have to try and isolate and remove some data.
I wish I knew what the magic word was, that got students to actually try their hardest during these studies. At this point, I just feel completely beaten down by my research. Hopefully the weekend will provide enough distance that I can approach with a better frame of mind on Monday.