I've been using MTurk in affiliation with my lab. We design our surveys using Qualtrics. We permitted 60 hits for our last batch, but somehow on Qualtrics we have over 80 responses. Has anyone experienced this problem, or have any suggestion on what could have caused this? This is my first time using mturk forum, so please feel free to give any advice! Thanks
I am not sure how the program actually works, but I think part of the problem is the definition of "HIT". Do you count each response as a "HIT" which would seem to limit you to 60HITS/60responses, or does each "HIT" allow more than one response, which is how the 60/80 split could occur. Like I said, I don't know how it is worded when you set up a survey, but that could be it.
Did you allocate enough time on the HIT? I suspect that the extra participants are ones who completed the survey, but may have taken longer than you allowed; and the HIT expired on them. Or, workers began the survey before accepting the HIT. And, when they returned to accept the HIT and submit the completion code, there were no more HITs left in the batch. You may want to search for your requester ID on http://turkopticon.differenceengines.com/ to see if any workers have mentioned problems with the HIT expiring prematurely.