I have to stop going on Turkopticon, every time I do I end up angry and stressed out. So many reviews deeply offend the integrity of the rating system, based on emotional responses or biased reports. I saw a review that gave a great and well-known requester all 1's because they have not paid and it's been 5 days. Come on, really? Requesters have 30 days to pay. Most jobs only pay every two weeks. I saw a review that gave netmsi (our beloved porn kings) all 1's because the HIT was broken and wouldn't load the images. Really? A 1/5 for pay? 1/5 for communication? I saw a review that gave a well-known but very poor paying requester all 5's. I saw a review that was flagged for TOS violation because the worker got rejected. These drive me nuts. If a HIT hasn't even approved or rejected yet, how can they be rated a 1 for fairness? If no message was sent to the requester, how can they get a 1 for communication. If they pay 2 cents per minute, they are not a 5 for pay!! AARRG. It drives me nuts because people are giving false reviews for the sole purpose of trying to cause damage to the requester instead of accurately rating the HIT. If they can't even rate a HIT accurately, why would anyone think they could complete a HIT accurately? Thank you for reading my rant. Now I can go back to work.
That's typical for practically any rating system on the internet and I've found that unfair negative reviews are generally outnumbered by others both on TO and other places..
Using Turk Opticon is curtain-ally useful if you use it properly. Never forget, it's ONLINE. If I see a negative review surrounded by positive reviews. I investigate the person who left it. Look at their other reviews and determine if their credible. A lot of people spam the hits, then get a rejection and want to leave a review for their rejection, but fail to mention they submitted shitty work or completed the job in two seconds. Just ignore these comments, and unfair reviews. If a person summits shitty work, then leaves an unfair review. It usually shows something about that person. And they won't be around too long.
I agree. This is why I take Turkopticon with a grain of salt. I've seen many reviews where the person has rated across the board 1's when they really only had an issue in one of the categories. Rating systems are useful but they shouldn't be the big picture
My experience in posting hits and from what I have seen on TO from requesters I know are bad and some just had a rough start(or so it looks) is that people are in the 80%-95% going to post a bad review if they had a bad experience. However it is 5%-10% chance (Might even be lower)of someone posting a good review after having a good experience. Due to this It is almost impossible to go by the stats. It is better to just read the reviews and see what you think for yourself keeping in mind that there are most likely a LOT of good reviews that are simply not there. EDIT=Just my 2 pennies
Very good idea. Somewhere a requester can go to find good workers is what that would be =] Have to say though.. it would be a tough site to get started.
Positive experiences are less posted by most of the turkers. Negatives one spread faster. It damages the reputation of requester
Absolutely. The best way is trial by error enough said. On the internet, you have honest workers, reliable workers, pathetic workers, scamming workers. My point, you don't know which reviews to believe. When I look at products online at Sears, I think to myself is this a legit review or a review I typed through Mturk? Today, I tried a requestor, that had no TO reviews what so ever. But the hits were set to 60 minute auto approval, so I gave a few a try. Went and took a shower, got ready, cleaned the house for the day. Came back to the desk, they were approving. I sit down and did hits all day for the requestor, made $75.00 for them, all are approved today, the same day. Requestor had no T.O. Once you turk long enough, you know how to use T.O. and filter out reliable and non reliable reviews, but you also know how to rely on your keen senses.
Garbage in, garbage out. Take what you can out of it. I rule out the ones with poor grammer, one or two word comments, grading that makes no sense, etc. I've taking on plenty of hits with no TO and really had no problems. Some of us actually contribute to it in a useful way. Others like to use it without contributing in any way but quote it religiously, requester retaliation, or starting verbal wars with posters.
Wow I see what you mean. I just looked at one and they were all negatives and the complaints were just that they took three weeks to pay. Well, don't they have that right? If they can take 30 days to pay, then they have 30 days to pay. People are jerks.
A good point, though after some time this should pan out to paint a more accurate picture. Here is another quote "complaints were just that they took three weeks to pay". While the second quote is true as well, it has served it's purpose. The only reason I signed up on here is because I'm about to get scammed (doing work for free). So the negative reviews do outnumber the good. This is true. Let me make sure I got this right. These requesters can basically tell anyone they want that their answers are wrong (to get the work for free) and there is nothing else we can do about it but give them a bad review? This is leaving the door wide open for scams. It won't be long, and we will have a even bigger problem on our hands. Just be thankful the big issue at the present is biased reviews.
I've really been enjoying this thread. Great observations! Seems like this highlights how TurkOpticon is most useful for Turkers with less experience, and how it becomes less and less useful as you get more. I think we were all there once, and if you think back...finding something like TO (if you found it early in your Turking career) felt like an "aha" moment, right? When you're new to Turking, as james57's post demonstrates, the flaws mentioned in TO's setup are much less important. Its value is in its ability to warn you that people are not necessarily happy with a requester, so that you don't risk working for free, or getting rejections you can't afford to get. But an experienced Turker is looking for more---they've developed good instincts about requesters and are most interested in new leads about *good* work. That's become of equal or more importance than ways to find out about *bad* work. Which is probably why having good requesters labeled as bad ones becomes more and more annoying as you gain more and more Turking experience. (That process is probably hurried along if you naturally tend to be someone who notices and thinks about systems that aren't working or don't seem fair.) Just my 2c.
I think you hit the nail right on the head. I understand your perspective much clearer now. What I will do is give good honest reviews for requesters I have worked for, 99.9% of my experience has been positive thus far. There are a lot of good reviews to give. I think we will just have to be patient and it will pan out in time. Thanks.
You're quite welcome. Really, I was glad you posted what you did. Your post, in comparison to the others in the thread, was what made me think about why there would be such differences in how TO is viewed. My post happened because you made me think about it. So thanks, right back!
