Hi, I have submitted quite a few HITs on labeling body poses of people and we will add more HITs with this task in the future. Please take a look: The groupId is 2F7PLSP75KLO22CULB8QMQVJRD9ZTA Its a rather simple task that everyone can do. The default reward is 2 cents. Those people that are able to accurately complete the task will receive a qualification an an upgrade in payment. I am also very interested in feedback regarding the labeling tool used in the task. all the best, M.
You can also find the HITs by searching for "Label body parts of the person in the image" or "Max Planck" Best, M.
Ninety whole cents? Pretty darn close to irresistible...let's see; If I work on these for 3 hours I can buy a 4-pack of toilet paper. Oh no, wait..I've got enough for that in my couch cushions. Sigh...smh
Fair enogh The key issue right now is the quality of labeling, the HITs have to be cheap because we have to average the results from multiple workers to get a decent quality. This is of course not fair to workers who try to do their best, but have to compensate for others that are not as careful. We can definitely pay more for HITs, but can only do that for people that are careful and reliable. I did about one hundred of our HITs to test the interface, and the average time per HIT is about a minute. I guess this requires a bit of routine, which you get after some time. In any case, if you decide to invest more time in our HITs I am open to suggestions.
How about we offer him some constructive ideas rather than just offering rude and snippy comments about his pay rate? If people do nothing but complain like this when requesters post here, requesters will eventually stop posting here. Generally, TurkPoseEstimation, work is only considered worth doing when you average about 10 cents a minute. Offering a task that takes two minutes and offers ten percent of that may not net you many quality workers. You may want to consider increasing the rate for these HITs if you want to attract good workers.
You could always set up a qualification test for the workers who are willing and able to do good work for you. You can weed out "bad" workers and make it so only those who meet your requirements can do the task. Then up the pay for those qualified workers and everyone's happy.
At that pay rate, the only way you are going to get quality work is with a custom qual. No way is almost anyone, unless they are really poor/bored/bad etc gonna do this work they way it is!
Thanks for the feedback, I was originally very exited about the possibility to have qualification tests for the workers, but it turned out that the quialification tests do not support the "external question" type that I use to implement interface in this HIT. Take a look here: AWSMechTurk/2007-06-21/AWSMechanicalTurkRequester/ApiReference_ExternalQuestionArticle.html It says I might be mistaken at this point, and will need to dig more into this issue. For now I am thinking of granting qualification based on the current performance of the workers, and then paying more to workers with qualification.
I get this thinking to a degree, but it is an insult that a requester knew they were paying at max $1.20 an hour and expecting quality. You have to pay for quality. Even if there was an incentive to get a qualification, I wouldn't touch these for less than .15, and that is if I am very bored (this hasn't happened in awhile). That is the reality. Most of us here give quality work, we deserve more for our time than this. Qualification test is the way to go. Some requesters do a number of 0.00 hits and give out qualifications based on those results. This may be a better option for you possibly.
Alright, so following the discussion here we have a new policy. The tasks for qualified users will be paid 10 cents per HIT. We also have a set of qualification tasks that are paid 3 cents per HIT. In order to obtain a qualification one needs to perform at least 25 qualification HITs correctly. All the best and hope you enjoy working on our HITs. We are a research institution working on algorithms for computers to understand contents of images e.g. to make cars automatically recognize pedestrians on the street or cameras to focus on faces of people. We do appreciate your contribution.
Here are the links to the updated HITs, Qualification task: https://www.mturk.com/mturk/preview?groupId=2CC8EBDPGFL8RIINWAINLWIOVAIH2D Tasks for qualified users: https://www.mturk.com/mturk/preview?groupId=2M4MJTUHYW2IPV2I78J54M1697IJOQ
I think you deserve a lot of credit for taking another look at the pay rate on your HITs and setting up a qualification test.
Looks interesting. I am doing them now. I am finding that the blue font is hard to read against the black box and in the pictures too. I did 25 of them. I enjoyed them once I got the hang of the instructions.
Thank you for coming here and communicating with us on a personal level. I appreciate the chance to help you in your research. Would it be a possibility to get some sort of mid-review? I have read the directions thoroughly but I'm still not sure if I am doing the work to your satisfaction. Could you look at some today and approve or reject? I see the auto-approve time on the qualification is 15 days. I would hate to have put forth all this effort for $0.75 and a chance at qualification only to get 25 rejections because I didn't quite understand.
Ok, gave 30 of the quals a try. I know I did the first few wrong because I missed one of the items in the tutorial/examples. Now we wait and see if they approve in a speedy manor. At that time, I will decide if its worth it.
Well I hope these quals will be up for a while. I'm far too chicken to do 25 hits with the risk of rejection and only 1 TO. Did two to see if they approve
Thanks for being gracious and listening to the suggestions! I have done a few also but I want to make sure I am doing them correctly before I do any more.