WOW. Have I lost a lot of respect for Turkopticon! How did the CrowdSource good reviews get mixed into the CrowdFlower bad reviews? CrowdFlower has had consistently bad reviews and low point ratings until a day or so ago. Now it looks like CrowdFlower is a good source for HITS and they have good rankings. How is that fair? I'll definitely be passing the word that Turkopticon has sold out and is no longer a trusted source of information for MTurkers! Does anyone know why Turkopticon combined the reviews for CrowdFlower with the reviews for CrowdSource? It's very misleading for new Turkers and makes me question how Turkopticon maintains and edits their database of reviews. I hope Turkopticon reverses this decision to more accurately reflect the true nature of CrowdFlower. They sure ain't no CrowdSource!
I'm still trying to figure out how ANY requester who pays a whopping .01 per hit can get a high rating for generosity. Are reviewers that stupid or is there something sinister going on behind the scenes? I don't trust any of the reviews for any of the requesters now. One thing is clear; most of the people using mturk have no head for business and no concept of ethics.
I don't think Turkopticon was the one that merged the accounts. Crowdsource changed their workerID sometime last week and that is what caused the TO for them to change. I may be wrong, but I think you are putting the blame in the wrong place.
Regardless of what or who casued CrowdFlower and CrowdSource reviews to merge, Turkopticon is still responsible for their browser add-on. I've seen several comments about CrowdSource changing their "ID," but that still doesn't explain how a crappy requester like CrowdFlower suddenly acquired all of CrowdSources positive reviews. CrowdFlower and CrowdSource haven't merged have they? They are still separate entities and their reviews should be separate. Will it be acceptable if Turkopticon merges the reviews for Behavioral Science Lab and Behavioral Philosophy Lab because the names are similar? I notice that CrowdSource's reviews reset to zero and now show just 8 reviews from the last week. Maybe Turkopticon should reset CrowdFlower back to zero or remove the CrowdSource reviews, otherwise they lose all credibility with me.
I don't write a lot of reviews, but I usually give all fives except for leaving communication blank. If I have a real problem with the pay, that's what I would use the pay scale for, not the generosity scale. I don't know what other people use it for, but if I were going to bother with it, I would use generosity to indicate the frequency and amount of bonuses. If a requester is giving a lot of them it will probably be indicated in the actual reviews. When I contribute to TO, it's usually for a requester who I feel has given me a really raw deal with a rejection. But that doesn't happen very often. I don't know what makes you think that most of the people using MTurk would have a "head for business". I think you are placing ethics in the wrong place regarding TO reviews. Unless a reviewer has a particular interest in a requester getting a particular score, ethics don't enter into it at all.
TO is not very reliable overall anyways. Its full of reviews from people complaining because their HITs arent approved 5 minutes after they submit them. Shego, are you talking about requesters like lang_labs and their glowing reviews? Yeah, pay is crap but people are happy because they can increase their numbers without a risk for rejection. Sounds like Crowdflower, since there are some decent batches that they put up once in a while and they dont reject. Crowdsource on the other hand, I remember a guy around here got close to 200 rejections some weeks ago. So as you can see, everyone has their different experiences and opinions. I am not one of those that bashes CF because I have worked for them and will work them again. If I feel like the HITs are well-paying and can be done quickly, I will do them, I dont care if Satan puts them up.
Response from Turkopticon Hi all - This is clearly some sort of bug and we're looking into it. Our mod taint brought this thread to our attention. Thanks and we'll update you soon! Lilly Team Turkopticon
Thanks Lilly for the update! I know I rely on Turkopticon every time I do HITS. It really bothers me that a parasite like CrowdFlower suddenly had great ratings and caused me to question the efficacy of your add-on. Without Turkopticon, I don't believe I'd do HITS on MTurk, there are so many scammers and con artists. I hope you get the issue resolved quickly
I just signed up for Turkopticon a couple days ago and haven't really had a chance to explore it yet. I was just puzzled by the "generosity" rating being high for low paying requesters. However, I can see the point about bonuses and how often they post hits etc. being considered generous. These days it could be considered generous if the requester doesn't reject work when the pay is low. As far as where ethics belong: Ideally, in any situation in life people should be honest, ethical and fair. I realize this isn't going to be the case; but since I behave that way, I assume (or hope) that others will too. The words "mturk" and "ethical" clearly don't belong in the same sentence I understand. It's just unfortunate that the only ones losing out are the workers because we have no real recourse when we are abused. If requesters don't like our submissions, they can reject them. If a requester rejects our work and uses it anyway to get out of paying for it, or if they reject by mistake etc. we can only hope they will make it right because if they don't; we're screwed. At any rate, I will do my research next time before I pop off at the mouth-so to speak.
Who even completes the $.01 HITs? I wouldn't even consider doing them unless they took less than 10 seconds a piece, and most of the ones I have seen don't.
Thanks for the update BuddysGirl!! Thanks to Turkopticon for fixing this so quickly! All is right with the world again.
There are actually a decent amount of penny categorization HITs that take 5 seconds or less, which easily passes the $.10/minute benchmark I set for myself. I did a master's batch of penny HITs last week that I averaged 18-20 per minute. Just have to keep your eyes open.