hi everyone! it's my first time using Mturk,so I'm trying to figure out some questions I'm going to re-direct my participants to other survey,where I'm going to ask them to write down their worker's id in order to correspond their answers to their numbers. How many symbols are in a worker's id?as far as I understood,those are combinations or letters and numbers,right? and another question,just to make sure that I don't require too much from my participants.I read some reviews,where people recommend choosing master workers or people who have HIT Approval rate is greater than or equal to 95% AND Number of HITs Approved is greater than 1000. Aren't those qualifications too strict? I just have a feeling that by choosing that options I will significanly limit a subset of my potential workers.... thank you very much in advance!
Hi, There are 14 digits in my worker ID with combination of alphabets and numbers. regards, V. Karthikeyan
If you want to loosen up the requirements, you will have to add traps in your survey, questions that ask the worker to check a certain option to make sure they are reading the questions, also, the same question more than once in different stages of the survey to see if the reply is the same. But ultimately, I had days when I did over 1000 HITs which ware all approved, so I don't think 95% approval rate and >1000 HITs is an exaggeration but master qualification is as there are no real instructions on how one could obtain that status.
Wouldn't recommend. If you have hundred questions, ask the first 5 questions again, there is a probability that I might answer 4 questions exactly as earlier and the last diff. That doesn't mean I am not paying attention. A trap question is the best bet. If you feel the qualifications are too tight, loosen up. Keep in mind person new to turk might not be familiar with the interface. There is a chance he might not go by the instructions and get it wrong. End result- you waste your time on useless data, you reject him, he goes and bitches about you in the forum and turkopticon. People will perceive you as a scammer. My recommendation. Approval not less than 70% Approved hit not less than 200. Good enough a qualification. Not too liberal, not too strict.
thanks everyone for your kind feedback!!! another little question: what means time alotted? is it a maximum time I let a participant answer 1 specific question,or here we talk about a whole period of time required for answering ALL the questions? I'm a bit confused... I think that my participants will need 20 min to finish a survey. If I put 20,does that mean that a person who fills in the questionnaire for more than 20 min is rejected? WOuld you recommend setting 20 min,or it's always better to allow more time? thanks!
Time allotted is the time given by you (the requester) to me (the worker) to complete the task. If you post ten questions and give 10 minutes, I will have ten minutes to answer the ten Q. If the timer runs out, the hit gets abandoned. It will affect our quality score. IF you expect the work to take about 20 minutes. Give 60 minutes to the task. It is a good practice to let the worker know how long it will take. He will be able to decide if he has that much time on hand at one stretch. Also, it will help you decide if he is working honestly. A 20 min task might be completed by some in 15 min. But definitely not 5 min. So, if someone is submitting the hit in 5 min, you know that he is playing around.
What he said. Many Turkers are turking from home and have to keep an eye on the kids, answer the door, etc. Allot twice the amount of time you think it will take. There is nothing to be gained by setting the expiration time to a tight fit. It would also be considerate to state up front if streaming video is involved (so the bandwidth-impaired amongst us can skip your HIT).
Hi, Always allow extra time to complete a task because some people will do a task quickly even much faster the time specified by you but some may require more time to complete and also the internet connections which may be slow and will take time to load. With regards, V. Karthikeyan