I did quite a few of the ten cent HITs of their's entitled "Evaluate the Pronunciation and the Sound of a Word" 12 of the 36 I did got rejected and it shot my approval rate from 99% to 96%. The instructions weren't clear enough to the point where subjectivity couldn't play a role in how workers did their HITs. Here's how so: Each HIT, you would play a series of clips of a bot pronouncing a word. First you were to choose whether the pronunciation was acceptable, perfect, or unacceptable. This part was not a problem. However, next you were to determine if the bot's voice sounded of good, poor, or a garbled quality. This was very subjective and a fellow worker who was doing the HIT also at the time often disagreed with me, yet he still got a similar amount rejected: 8 rejected of the 30 he did. I will be wary of this requester in this future and I'd like others to be aware that the regulations are rough with this requester.
I did 40, and got a single rejection, with the mystifying message: "It seems you answered all of our gold standard HITs completely incorrect." LOL, all of them? That's pretty consistent.
I had that for all of them. I believe they add 2 or 3 words with answers they expect you to have. Here's an important question that could have played a factor, how often did you answer "No Sure" for the pronunciation bit? There were quite a lot of words in there that were not actual words, or just ridiculous, but I still knew how they were pronounced and answered accordingly.
I answered not sure for stuff that was foreign in origin that I wasn't familiar with. Using gold standards on judgment calls really isn't the best of ideas. If they really wanted better results, they should have been far more clear as to what each category actually meant and provided examples. Higher pay to get people to take it seriously would have helped too.
The HIT is back and I did it again today, this time I took a more careful approach. I'll check the results soon and if all were accepted I'll do this HIT again in the future. Tip: most commonly, the quality of the voice is good and the pronunciation is correct.
1 of the 10 test HITs were approved. Ridiculous. Never working for this requester again. I congratulate anyone who can do this task efficiently by his standards with such unclear instructions.
This makes me glad I didn't do any of these HITs. There was no 'play' button for me so I couldn't listen to the clips, I attempted to but now I'm glad I couldn't do them. This sucks.
In the future, there probably is a way to get these kinds of pages working that many can help you with. But indeed it does suck. My approval rate is down to 94% because of this (I previously was put from 98 to 96 from this requester, now another 2% was knocked off) and I can't do any of the HITs I'd regularly do other than CrowdSorce's. 122 pending HITs, as a relatively new worker, these getting approved can shoot me back up to over 95 percent (I did the math) which I need right now.
Same with me, did over 100 HITs, went from 99% approval to 94.5%. I commented on one rejection and actually got an answer back.. On Sat, Jul 21, 2012 at 1:45 PM, CMUDict MTurk <cmudict.mturk@gmail.com> wrote: Hi, I regret that your HITs were rejected. But, I would like to let you know that we have tried to be as fair as possible in our assessment and have taken care to ensure that our turkers are compensated well and on time. At the same time, we are striving hard to ensure high quality of our data collection process. However, we have had some instances where people have just run automated scripts and bots On Sat, Jul 21, 2012 at 2:28 PM, Me <-------@gmail.com> wrote: Message from Me(------@gmail.com) --------------------------------- Customer ID: ---------------- I've done your HITs in the past with no problems! I was just about to post good feedback for you on Turkoptikon. There is no reason why you should be rejecting these. They were all perfect. Your rating is horrible on Turkoptikon, I thought I would give you a chance. Good luck trying to get other serious turkers to do work for you in the future...
I did around 80, and only had 7 rejected. I queried them about it, as 7 rejections isn't my normal. They said I was a little too lenient, so I judged more harshly. Luckily I only did like 30 because 7 were approved... I don't know what's with these people. They told me they'd keep my number in mind when evaluating again, and indeed I had all 20 approved, then however... They started rejecting me again. Very annoying, and the few rejections i've ever had in bulk.
I tend to do these in very short spurts.. I just did a few tonight, but am leery to do a large amount.. they are easy and quick and pay decent, but I can't risk the rejections
Hi guys, just to let you know I have had extensive communications with this requester. I've helped them see that a workers approval rate is their LIFE on mturk. The requester is currently looking into reversing a lot of the past rejections, and looking into making a running tally of workers approval rates to eventually give bonuses. I've done their last two batches with ZERO rejections. The batch before that I got 3/120 rejected. So they are doing much better, and have taken to not only judging by their 'gold standard' HITs but majority rules, AND giving those who have high approval % with them the benefit of the doubt. So feel free to try a few more, you'll most likely get them accepted. Just pay attention, it's a very easy task to do. And they are trying. If you have an issue, just message them. If you try them again, and have a good experience post it to TO. They are a private research campaign and due to Mturks "reject any work you don't like!!!" they didn't know that approvals meant so highly to workers. I can post my emails, and my accepted HITs if anyone doesn't believe me.