Over-analyzing data?

I’m (still) analyzing data from the experiment I ran last month. Why? Because every time I meet with my supervisor, he ends up deciding that we should also try x. And usually y and z as well.

Personally, I feel like we’re getting close to (or more likely, way past) that point where you’re no longer analyzing your data, but massaging it to get results. Which seems kind of crazy, because, at the end of the day, our few “big” results, we found pretty much immediately. And most of the data analysis I’ve done since has resulted in finding either nothing new or just no result.

So, my question is when does data analysis cross this line? Does this line even exist?

We haven’t started removing data (or making up new stuff). We’re just re-organizing (as in sorting) by different variables. And then running another anova or t-test. And I know we’ll have definitely (in my opinion) crossed the lined should it feel like we’re removing or adding data.

I’ve read papers before, and seen lots of newspaper articles referring to studies, that have looked at and recombined data from previous experiments. Or papers that are finding a “new use” for old data. (Although, is it really a new use?) Some very interesting research has come out of people re-purposing old data. And often, the grouping seems to be the only way to actually conduct studies that have much larger sample sizes.

I guess, I’m partly annoyed by this because I feel like I can never move on from the data analysis. I want to focus on writing up what we’ve found. But, I also feel like we’re now trying to come up with stuff that we never thought about in the design of the study. Which isn’t to say it’s not there – maybe we will find something amazing (although I doubt it). However, at some point, I feel like we should be saying enough is enough, let’s write up what we have and go from there. And for me, that point should’ve been about two weeks ago.

But, since it wasn’t. I need to go do some more data analysis. Oh, hello R


Join the discussion

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s