--cut and paste from my response on Turker Nation---- Hi, I mentioned that I looked over the process of including control questions. Here is what I I did. * Looked at the process of for using control questions and the process and technology to accept and reject hits. * The next was the actual control questions itself. The process was okay and the software to accept or reject did what it was supposed to do. Next I looked at the data and the labeling of control questions. If there was a problem with the way we labeled data for the control questions and rejected hits if they did not match our data, it can explain the mass rejections. So, I found some issues with a small % of our labeled data. But due to the nature of the data, this small % manifested a lot more in several hits. So long story short, we identified the problem. We are going to remove them for the hits we submit going forward. We will also make changes to the process of labeling data for other categories as well. Now the question is how to we make good for the hits that have been rejected. Not all hit rejections were a problem. But there was a large chuck where this data inconsistency can explain the rejections. We will make this right. I have a meeting with the Amazon folks on Tuesday and will ensure that this can be reversed. We are going to be a major requester on Amazon for the months and years ahead with very large hits. Again to set expectations, NOT ALL of them are going to be reversed. But I was able to see the problems that people hit. I am trying to make this right. Thank you for your patience and sorry for the trouble. Thanks Arjun
Amazon is silly then, because I have seen several HITs that say "This is for invited workers only, all others will be rejected." Many of these HITs have a value of $0.00, so I am assuming that they are paid via bonuses only, or that they are merely to raise people's ratings due to one reason or another. My suggestion is that you gather all of our names (those of us who were mass-rejected) and send us emails inviting us to do enough blank HITs to get our ratings back up. It is simple enough to go through all the emails from those of us that complained, asked why we were rejected, etc., pull the emails from those, invite us to do work for you and work on a bunch of blank HITs. Then make sure to keep a record of those turkers that you invited, and anyone who was not invited that does said HITs will be rejected with an explanation that they were not invited so they need to bugger off. Just make sure to put "Invited worker only" on the description, instructions, and anywhere else that it fits so there's no confusion whatsoever.
mm....... every body cursing at mr. jayaram. but I worked in his Hits (82) but 3 rejected. even these 3 Hits only from my end mistakes. as per my knowledge his Hits based on camera sales market analyzes through twitter and or some messenger chatting.I hope I will do in his Hits future.
Don't know that any of us were actually "cursing at" him. I did 44 of his HITs and 31 were rejected, and I didn't make any mistakes [if one were doing the work honestly, it would be damn near impossible to make mistakes on such a simple HIT]. Perhaps those of us that have a better grasp of the English language are the ones with a higher rejection rate.
Right, you worked the hits from the second large batch, apparently after the problems with rejections that everyone else is complaining about were resolved.
I'm another turker who got majorly burned by this requester. I completed 1383 HITS for him - 898 were rejected. 898!!!!!! My approval percentage dropped from almost 99% to barely 96. I've read this thread, arun, and I'm not satisfied with your answers. I will be contacting mturk about you, hopefully if enough of us do, you won't be able to use this service anymore and rip us off. Also, when something went wrong with Dolores Labs' HITS once, they set up blank HITS for individual workers who got rejected. We were able to bring our percentage up that way. Dolores Labs emailed each worker with a specific link to their batch of HITS. It CAN be done.
I would strongly suggest everybody here to read the updates left by Arjun on proboards: http://turkers.proboards.com/index.cgi?board=shame&action=display&thread=5800&page=9 He really is taking unprecedented steps to get this resolved.
Compass Labs Raises $5 Million To Pinpoint Purchase Intent On Twitter Well, that was fast. Only two months after launching at TechCrunch Disrupt, startup Compass Labs has already raised a round of funding. Compass Labs, which aims to provide targeted advertising on social networks like Twitter and Facebook around what users intend to purchase, has raised $5 million from NEA, Triple Point Capital, Jim Clark, Mike Ramsay and others. This brings the startup’s total funding to $6 million Compass Labs looks at Twitter streams and tries to determine when someone has an intent to purchase a product, then it serves up related ads either through direct messages or through banner ads on third-party Twitter clients. So if you Tweet, “I’m looking for a Canon camera” it will reply in stream or on a banner with an ad from a camera retailer for that camera. Compass Labs uses natural language processing to parse out the Tweets that have serious intent versus just talking about a product generally. Campaigns can be set to target people at different parts of the purchasing cycle, from exploratory to ready to buy right now. View full story: http://techcrunch.com/2010/07/19/compass-labs-raises-5-million-to-pinpoint-purchase-intent-on-twitter/