Want to earn a quick $5? Send me a PM - Cowfin
Page 1 of 2 1 2 LastLast
Results 1 to 10 of 18

Thread: Returning HITs and the workerId

  1. #1
    User
    Join Date
    Jul 2010
    Posts
    6
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default Returning HITs and the workerId

    I run surveys on Mturk and I have two questions:
    1. I use some standard attention filter questions to catch people who randomly respond to questions. The point is that if someone is not actually answering questions they shouldn't be paid. Since some filters are harder than others, and mistakes do happen, instead of rejecting work based on attention filters, I ask people to return the HIT when a filter question is failed. I received a nasty-gram recently saying that returning HITs is almost as bad as a rejection, and that it negatively affects their ability to do HITs in the future. I am not aware of any qualification that limits based on return rates, or of any way for a requester to access the return rate of a worker. Amazon told me that return rates are displayed to workers for informational purposes only, but I wanted to double check with the community. Does returning HITs negatively impact workers besides the unpaid time spent on the HIT?

    2. I maintain an outside database of workers so that I can do custom qualifications without using the API. However, to see if a worker qualifies for my HIT, I need to look them up using their workerId. To my knowledge this is only accessable after a HIT is accepted. Is this true, or is there a way to access the workerId programmatically before the HIT is accepted. Second, if not, and if I wanted to avoid having people accept the HIT, find out they do not qualify and ask to return the HIT, is there a better way to get their workerId than asking a worker to type it in?

  2. #2
    User
    Join Date
    Jul 2012
    Posts
    44
    Thanks
    1
    Thanked 0 Times in 0 Posts

    Default

    Hello. Maybe I am not the best person to answer this but since there is no replay...

    Returning the HIT is definitely nothing compared to rejecting a HIT. I never saw any HIT's asking for "abandonment rate" as a qualification. Or I did and it was something ridiculously low (almost impossible to have). You can return HIT's all day as I noticed without any consequence which I cannot say for rejection. Rejection ruins worker's account completely and it can get it banned.
    <-- removed by admin -->

  3. #3
    User Whimsy's Avatar
    Join Date
    Apr 2012
    Location
    FL, USA
    Posts
    805
    Thanks
    18
    Thanked 31 Times in 19 Posts

    Default

    " Does returning HITs negatively impact workers besides the unpaid time spent on the HIT?"

    Nope. I took surveys for a while and it was nothing more than an extreme annoyance to find out I don't qualify for a HIT and be asked to return it. I was never told to return it if I failed quality checks - that would probably burn more (since I pay attention), but to have taken the time to do the survey and then be told to return it -- that's where you're going to get nasty-grams. People won't believe that you're not using their data. They will think this is your way of getting answers for free. Doesn't mean it's true, but perception is reality.

    There isn't a way to find someone's ID before they've accepted a HIT.

    You can set a qualifier for your HIT. You can have people complete a test, a questionnaire, a task, an essay - probably anything you'd like. I'm not a requester but I do understand your dilemma with wanting to keep the quality control high for your surveys without pissing people off in the process.

  4. #4
    User
    Join Date
    Jul 2010
    Posts
    6
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    The alternative to asking them to return the HIT is to reject it. I figure they like returning better than rejecting. I clearly state that the survey contains filters in the opening, and most filters are dreadfully easy (e.g. Select answer 3.).

    The qualification tests/essays etc. can only be created through the programming API, and I don't have the time to learn some .NET programming language to figure out how to use it. Though I do see the annoyance of accepting a HIT to find out you don't qualify. That's what Im trying to improve right now. I think Im going to start having people enter their ID before accepting to check if they already qualify. If they don't they can take a test to qualify themselves. If they qualify they can accept the HIT. It will then double check if they qualify (i.e. check to make sure they entered the correct id number etc.) and if they need to take the test again cause they copy and pasted wrong they can waste their time if they wish since it was their own fault.

  5. #5
    User Whimsy's Avatar
    Join Date
    Apr 2012
    Location
    FL, USA
    Posts
    805
    Thanks
    18
    Thanked 31 Times in 19 Posts

    Default

    Quote Originally Posted by andrewlong View Post
    The qualification tests/essays etc. can only be created through the programming API, and I don't have the time to learn some .NET programming language to figure out how to use it.
    Or you could have a qualification "test" where there's a link to a survey off-site, just like the ones you do now. I'm sure there are options.

    Just suggesting - not recommending anything. I wouldn't want to deal with the masses when it comes to making money, that's for sure.

  6. #6
    User
    Join Date
    Jul 2012
    Location
    Coimbatore
    Posts
    101
    Thanks
    2
    Thanked 0 Times in 0 Posts

    Default

    I would like to share my views.... not sure they will work.

    The attention question... The survey must not proceed until the worker correctly does what is intended. A message should flash on the screen that he needs proper attention to proceed further. If that is repeated for more than three times, you can ask the worker to return the hit mentioning the reason.

    But one thing you people must take care, not all..... though:
    Mention a justifiable time in which it can be completed or better donot mention. Some hits mention quick / 5 mins survey, but in no way that could be completed in 5 mins. In a case I took a survey mentioned 10mins, duration 60 mins, I could complete the same only after 45mins. I did that with mu usual speed but.....for 15cents. too bad.

    Please don't forget to mention the page numbering (1 of n pages).. Some requester mention the number of questions that will be asked.... that is also okay.

    Make your survey simple and avoid to0 much of repetitive questions which makes the worker to feel bored and to miss the attention question.

  7. #7
    User nobody's Avatar
    Join Date
    May 2012
    Location
    Margaritaville
    Posts
    317
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    1. I have never seen a HIT that uses the return rate as a qualifier. Since rejections do count against us, I would rather have a return than a rejection. In fact, I don't know of any turkers who would choose a rejection over a return.

    2. I agree with Whimsy's suggestion. You could put something like "Qualified workers only" somewhere in the title. When the HIT is viewed, at the top you could put the link to your database/check outside of mturk. That way, no one can blame you if they accept the HIT without checking to see if they are qualified first.
    Last edited by nobody; 07-19-2012 at 06:48 PM.

  8. #8
    User 2muchTurkin's Avatar
    Join Date
    Jul 2012
    Location
    Michigan 4 the win
    Posts
    1,281
    Thanks
    126
    Thanked 150 Times in 80 Posts

    Default

    I agree here. I would much rather have a return than a rejection. Me personally I do not have a problem with accepting a hit to see if I qualify. No big deal I just return it if I do not meet survey requirements. The only time I have seen any hits requesting certain return rate was 20%. I can not imagine being able to work my return rate up that high. Take care and good luck with your project.

  9. #9
    User Crayonshaver's Avatar
    Join Date
    Mar 2012
    Location
    Birmingham, AL
    Posts
    313
    Thanks
    15
    Thanked 34 Times in 22 Posts

    Default

    I have seen some surveys where the list of Amazon Turk IDs was actually listed in the hit description and you could check yourself. I've seen others where there is an external link to a file that has all the IDs in them and you can check yourself.

    The alternative is to make sure that you tell us in the HIT description that there will be a qualifying question and you may be asked to RETURN the HIT. Full knowledge and they shouldn't moan. I appreciate you not wrecking our work chances by even asking. Thank you!

  10. #10
    User Whimsy's Avatar
    Join Date
    Apr 2012
    Location
    FL, USA
    Posts
    805
    Thanks
    18
    Thanked 31 Times in 19 Posts

    Default

    Another idea:
    Post your HIT for a somwhat nominal amount. Pay everyone but give bonuses to the people who pass all the quality checks.

Page 1 of 2 1 2 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •