Newbie Requestor Question

Discussion in 'Requesters' started by tanstaafl1963, Jan 16, 2014.

  1. tanstaafl1963

    tanstaafl1963 Member

    Joined:
    Jan 16, 2014
    Messages:
    6
    Likes Received:
    0
    I'm just starting out on mTurk as a requestor and would like a little guidance.

    I understand folks don't want to work for unknown requestors, or are hesitant to, but in the groups experience, what is the best way to gain the confidence of workers?

    Have others here gone through an initial period where there was little/no activity?
    Is it a good idea, when starting out, to pay a higher rate and pay immediately, to establish the credentials?

    Any help/guidance would be appreciated!
     
  2. BloodRayen

    BloodRayen User

    Joined:
    Oct 23, 2013
    Messages:
    2,187
    Likes Received:
    0
    there are several things you should do.
    Things like fast approvals and payments are important.
    Another thing should be to implement a custom qulification. So you can grade workers and make more difficult/ more important tasks only available for top notch workers. This saves you lots of time and money since less HITs have to be redone. An additional benefit for this is that you can avoid mass rejections which can kill your reputation.
    If you are starting you should avoid posting massive batches till you are sure that your interface and your instructions are clear. Several requesters came up with great amount of HITs only to see that they made minor mistakes in their instructions. This was followed by big drama since many people caught mass-rejections and were not exactly sure why.
    Last but not least you should try to communicate with your workers as much as possible. Only if you tell workers what you need and listen to their critics, you can build a stable relation.
     
  3. tanstaafl1963

    tanstaafl1963 Member

    Joined:
    Jan 16, 2014
    Messages:
    6
    Likes Received:
    0
    Thank you

    Thanks! I'm assuming that there are times, especially at the beginning, when you might not get responses?

    Do you see sometimes that people will 'reject' because the price offered is too low? Or do they tend to ignore and just move one? :confused:
    I guess that's hard to say since people who move on won't mark it as rejected, huh? :p
     
  4. Kris

    Kris User

    Joined:
    Sep 21, 2012
    Messages:
    801
    Likes Received:
    0
    By reject, do you mean why workers won't do a particular HIT? I can only speak for myself, but if I see something where the price strikes me as too low, I'll just ignore it in order to work on something else. Workers don't have a way of rejecting a HIT in the same sense that a requester can reject a worker's submission. And many workers stay away from new requesters simply because they're new.

    BloodRayen's advice is good. Additional suggestions might depend on what kind of HIT you're going to be putting up - survey, writing task, transcription, information search, etc. In general, workers won't look at a HIT that's likely to pay less than 0.10/minute. I don't do much writing or transcription, so I'm not sure what the going rates are for writing word counts or transcription minutes.
     
  5. Tjololo

    Tjololo Pony
    Staff Member

    Joined:
    Jun 14, 2013
    Messages:
    13,720
    Likes Received:
    105
    I'm not sure if you've checked this, but you might want to make sure that you're removing the restriction for "Masters only". The Masters crowd, while having proven themselves to amazon in some way (nobody really knows how), is very small and picky. If you want fast responses, remove the masters qualification.

    This qualification is on by default, if you need help figuring out how to remove it let me know, it's not too difficult, but the requester interface is insanity XD

    The qualification is a fantastic idea. I'd suggest making a qualification hit for each batch (or at least a test), that way you 1: screen out the bots, and 2: ensure people know how to do your hits before they start, that should cut down on your bad data. That being said, if you DO start getting bad data, I suggest contacting the worker to let them know, then removing their qual if their quality doesn't increase.

    I suggest trying as hard as you can NOT to reject people. Remove quals, attempt communication, whatever you can do, try as hard as you can without rejection. I'm not saying allow bad workers to work on your batches, rejections are a part of life, but seemingly "frivolous" rejections are the quickest requester death sentence.

    Finally, just as a note, don't use time limits as a rejection threshold, unless it's a 10 minute task and you're screening out <1m. That will get rid of bots, but I wouldn't go too much more than that; it's very difficult to estimate how quickly a worker will complete a batch.

    Just my thoughts! Take it with a grain of salt, just my 2 cents
     
  6. tanstaafl1963

    tanstaafl1963 Member

    Joined:
    Jan 16, 2014
    Messages:
    6
    Likes Received:
    0
    Yep.. my first go-round had that on, and someone responded indicating the Masters don't touch stuff that inexpensive.

    I'm not rejecting anyone yet.. I haven't even gotten a nibble, but it's only been a day or 2. So I guess I'm impatient.

    I've read (twice now) the mention of price per minute. (.10/min). Again, newbie here.. so forgive me :)

    If I put a HIT out for say $0.10 per HIT and leave it as a 1hr task, is that where they would consider it 60 minutes of work for $0.10? I mean, I'm paying per HIT, not how long it actually takes them to complete the writing task. So if actual work time is 1 minute, then it's $0.10/minute, but if it takes them 5 minutes, then it's $0.02 min. Or am I really out of my mind... ? (sorry, that's probably a separate topic) :)
     
  7. BloodRayen

    BloodRayen User

    Joined:
    Oct 23, 2013
    Messages:
    2,187
    Likes Received:
    0
    the $0.10/ min is actual work time. timers are not important for that.
    in addition it is hard to judge the actual work time by looking at the time people needed to complete the task. sometimes worker queue up work. If the pay is to low people will complain to you, i think
     
  8. Tjololo

    Tjololo Pony
    Staff Member

    Joined:
    Jun 14, 2013
    Messages:
    13,720
    Likes Received:
    105
    It is very hard to judge the actual work time, especially for someone who's writing the hits.

    I would say try posting your hits as a small batch, 5-10 hits or something, get the average time, then post the real batch with the average time in the description or something.

    You're right about the .10/min thing, but the pay/min is a touchy issue. Some people refuse to work for less than .10/min ($6/hr, about minimum wage). Others don't care as much. As for me, if a task is fun, I don't mind working for less than minimum wage, because I'm enjoying myself lol.

    I'd say your main focus right now would be getting well known. Post in the greatest hits thread, let people know who you are and do your hits and learn about you. Answer their questions, get a good TO built up. After that, people will have you on their "watch list", and your name will spread.
     
  9. tanstaafl1963

    tanstaafl1963 Member

    Joined:
    Jan 16, 2014
    Messages:
    6
    Likes Received:
    0
    Thank you both for your help on this..
     
  10. TurkingBear

    Joined:
    May 8, 2013
    Messages:
    16,225
    Likes Received:
    5
    I want to see this actual hit. If you can PM me what your hit is I can give you real detailed instrusctions why no one is nibbing.
     
  11. swiftkick

    swiftkick User

    Joined:
    Jan 29, 2012
    Messages:
    355
    Likes Received:
    0
    HITs that pay well get completed fast. If no one has touched your HIT at all, it could be horribly underpaid, broken or you somehow have a bad rep already.

    If you post a link, or the title of you HIT, we can take a look and give you some feedback.
     
  12. tanstaafl1963

    tanstaafl1963 Member

    Joined:
    Jan 16, 2014
    Messages:
    6
    Likes Received:
    0
  13. swiftkick

    swiftkick User

    Joined:
    Jan 29, 2012
    Messages:
    355
    Likes Received:
    0
    The issue is definitely your compensation. Respectfully, your pay rate is less than we get paid for doing a one sentence rewrite. I don't think you are going to get any takers until you get these over $1.

    I don't do a lot of writing personally, but I think you should aim for 3-4 cents per word when research is involved. While that may be outside your budget, that is what you will have to compete with to attract workers to your task.

    Good luck with your project.
     
    #13 swiftkick, Jan 17, 2014
    Last edited by a moderator: Jan 17, 2014
  14. tanstaafl1963

    tanstaafl1963 Member

    Joined:
    Jan 16, 2014
    Messages:
    6
    Likes Received:
    0
    I appreciate the swift kick, swiftkick :)

    Turk was presented as an easy, cheap option for some ebooks. I'm just glad I didn't pay for this advice :)
     
  15. TurkingBear

    Joined:
    May 8, 2013
    Messages:
    16,225
    Likes Received:
    5
    Yeah the hit is really underpaid. Some of the writing requesters pay about a dollar for 100 words. Thats if it's a free write. No specific instructions but a random article in a sense.

    The more specific you get the more it will eventually cost you because you are looking for expertise. Hope that helps.
     
  16. Tjololo

    Tjololo Pony
    Staff Member

    Joined:
    Jun 14, 2013
    Messages:
    13,720
    Likes Received:
    105
    Another hint: If you haven't heard about it, there's something called the Turkopticon (TO). Workers use this as a method to rate and comment on requesters, so that other workers know how the person is...Are they responsive, do they pay well, how long is a particular survey, etc.

    You have one rating on the TO, visible here: http://turkopticon.ucsd.edu/ATU5N4Q8VMJ4I the person rating you states the same thing, that the hit is underpaid, even though they saw the payment wrong (they said a penny but it's .10).

    When looking at your TO in the future, take what workers do with a grain of salt; some are lazy and will blast a TO for almost no reason, or if they failed to follow instructions, etc. Earlier today I found a TO response complaining about failing an English test, and it looked like it was written by a third grader.

    Over time your TO will normalize, and you can use it to converse with your raters to get more information, etc. It's a good thing to know about and keep checking though.
     
  17. studystack

    studystack New Member

    Joined:
    Dec 4, 2014
    Messages:
    4
    Likes Received:
    0
    Creating a User Testing task for a website

    I'd like to create a HIT for users to test my website. And while they are testing my website, I'd like to be able to record their screen while they use the site and also the audio so that users can tell me what they think about my site. If you are familiar with UserTesting.com's PEEK tasks, that's what I'd like to achieve.

    Has anyone ever seen a HIT that does this? I've created HITs before that ask users to check out my website and then answer some questions about what they thought. But I'd rather see exactly what they do and let them say what they think about the site.

    Technically I think I might be able to accomplish this by requiring users to use Chrome and to have installed the Screencastify Chrome extension before accepting my HIT. The HIT would instruct the user to start the Screencastify (Screen Video Recorder) and then go to my website and perform some tasks. When they are done they would end the record process and save the screencast to their computer. Then they'd have to go to a page on my site where they can upload the screencast to my server.

    Are there any mturk policies that prohibit this?
     
  18. BloodRayen

    BloodRayen User

    Joined:
    Oct 23, 2013
    Messages:
    2,187
    Likes Received:
    0
    technically yes, HITs that require a download violate ToS. If you need a reqular extension like unity or inquist, which are installed on many PC`s anyway, it is debatable in my mind; even flash and java would violate the ToS if you are strict
     
  19. studystack

    studystack New Member

    Joined:
    Dec 4, 2014
    Messages:
    4
    Likes Received:
    0
    But what if I say this HIT is only available to users who already have Chrome and the ScreenCastify extension installed. So technically I'm not asking anyone to install anything. Can you point me to the specific policy?
     
  20. BloodRayen

    BloodRayen User

    Joined:
    Oct 23, 2013
    Messages:
    2,187
    Likes Received:
    0
    the policies: https://www.mturk.com/mturk/help?helpPage=policies

    personal opinion!: to say that users must use a specific browser is totally ok, just note that or turkers will rate you negatively. for the extension it is ok if you clearly!!!! say that it is required to the HIT (Caps in the HIT title, maybe). this way users know what to expect from your HIT
     

Share This Page