Idk why anything would be wrong with it. If thats how you can make more money and give accurate results. Its smart working. All of the Hits will still be completed eventually
I did not realize that my use of "cherry picking" would be ascribed such a pejorative meaning by the community. I was merely trying to describe as accurately as possible the reasons for Dustin getting the results that he did from the HITs he posted. I did not intend any offense to any one or any group of posters in this thread. I do not see how failing to describe what was happening in detail would be helpful to anyone.
Well this thread took a new turn. I will start off saying that I neither condone or promote "cherry-picking". As a requester all I can say is that I don't care how the work gets done as long as the work that is submitted is correct and submitted in a timely manner. I think the Ctrl-F "Find" function is a legitimate way to get a worker to the right information, just don't assume that because something is returned using the "Find" function that it is immediately correct. I trust that "cherry-picking" or not, everyone is submitting good work and putting two eyes on each HIT. Again I appreciate everyone that is submitting work, and will be spending the next evening and morning checking quality. These HITs have now run out, but expect some to be resubmitted tomorrow and the others to be approved. I'll also be coming back and re-turking some of these, as well as creating a new search to find Twitter handles for the executives (if they exist). I'll be away from the forum for the next two hours or so, ask any questions if you have them and I'll get to them when I get back.
Well I tried my best. I submitted every one that I worked on. There were quite a bit that there was just no information to verify the person's employment even if the information was present in the search results; it just wasn't present after clicking the results or that the information was old. There were also some where the Name and Title matched, but only part of the Company Name matched. When I search to verify the information, a completely different person with the same Name appeared. It either matched the Company Name or it was partial match as well. Example: The current information is John Smith, Manager, Microsoft Europe. The LinkedIn just says Microsoft for the company name. The search results pull up another John Smith, Manager, Microsoft Europe or John Smith, Manager, Microsoft. They are different John Smith's. One is white and the other black. I suggest for future hits to list different examples of situations a turker may come across and how to answer them.
I did this too for a good amount of them, then I started thinking about the time it was taking and whether it would come up that I wasn't taking long enough on them so I started clicking the link prior. I was very hesitant to use the button for "no information found", so if I didn't find information, many times I would move on to the next HIT. It worked easier to check the links prior to accepting rather than accepting and having to return, thus highering my return rate. So I agree with you, I don't think checking the time spent on a hit is a valid way of telling if someone was spamming or trying to cheat. I was legitimately working through the hits, but I am quite certain some of them only took me a few seconds because I already had the information found prior to accepting.
There are a lot of people who don't have 10,000 hits on this site though. I think a better way would be to review the submissions and people who submitted good work could get some sort of x8 qualification. Some sort of qualification test could be possible too.
Personally, if the linkedin page was good, they took about 10, maybe 15 seconds to complete from acceptance to searching to submitting. With a fast connection and a good PC setup, and knowing your keyboard shortcuts, they took no time at all. I did every hit that I got and didn't check before accepting, I just checked to auto accept. For .08, even if they took an entire minute to do that is fine. This was a really great opportunity. The ones I did mark as no info found, almost all of them were very generic exec and company names, which returned enough results that it was impossible to tell the correct exec, or the person was only peripherally involved with the company and I could not find a title or business for the exec in the first 2 or 3 pages of google results. I will say that if the company name was, say, Communications England Company, and the linkedin page has CEC as the company name, with the same title, and the business was obviously a communication company in England, then I marked this as valid. I ran across this several times, or perhaps the LLC or LTD was left off of the company name I still marked it as valid. With all that said, I would probably reverse the pay scale. The ones that were .06 should have been the .08 ones, and the ones that paid .08 should have gotten the .06. The more work, the more pay, and I did a few of the .07 and .06 ones and they definitely took longer. More of an incentive for them to be done correctly. And with qualifications, if you find that a lot of data is not accurate, I would either go with 5,000 approved and 99% approval, or develop some sort of qualification test where you give a few exec's that you already know if the pages are valid and if they are not, you know the current title etc. Compare the test takers answers, and if its something like 85-90% similarity or better grant qualification, and if they pass the test give .15 bonus or so, if they fail nada. Of course, we would all be grandfathered in for the great work we did last night But thanks for the great opportunity! My worker ID is ANVP****UGLY, fyi.
I was checking the links for the name and company title before accepting the HITS myself, so mine were done fast as far as the timer goes. Doesn't make the results for the HIT spam by definition does it.....? The instructions for the invalid links were kind of too involved for me to feel good about and spend too much time on till I have chance to go back and practice... Guess we'll see how it goes...hope it works out for everyone.
That makes good sense. I don't see how having 10,000 HITS makes you more able to do these. A qualification test or previous work results seems much more logical.
Hey you guys that just put your worker id out in public. That is NOT a good idea and you may want to remove it.
I did the same for some. If I knew a certain company had many divisions such as Microsoft, I did not do that. There can be a couple of John Smith's working for Microsoft and I need to know which division the John Smith works for.
Since you are a new requester for many of us, people may have checked the links first because they were concerned about moving on down the page with a greater chance of rejection. I believe that you should not write these workers off yet unless they checked off the webpage and the executive's company was not located there. I hope you do not mind me giving you my opinion.
Dustin I agree with this post very much, we are sort of in a race with all of the other Turks to get in as much work as possible so we can make a decent wage for ourselves while we work on HITs. Even more so when there is a good batch like the one you posted last night. I know I was doing most of them very quickly but as you said I was also making sure to put " two eyes " on each HIT, I'm guessing most fellow mturkforum members were doing the same. Not to say you didn't have workers just checking the box and hitting submit I think you'll always have some people trying to scam the system but there are probably less of those types here from the forum. As for your HITs I noticed you only had a 90% percent approval rate to do the work to me that seemed low to me, mine as an example is at 99.9%. I would think you could push that requirement up to 97 or 98% and weed out some of the people that are trying to scam you.
My work may appear as though I did them fast, but I did not accept the HIT until I actually found the person and the information needed. So I put in time and my work is honest even if the time it shows I spent on it doesn't show.
Well, I have just over 4000 with 99.9% and I'd hate to be booted out of doing these hits because some people like to scam. I agree that there needs to be a qualification though.
I am surprised that nobody has said this yet. We see HITs all of the time that say.. if you return too many marked as "I can't find the information" you will be rejected. It doesn't matter if we actually do the work and we are 100% sure that we can't get the information. Some requestors just see this as a sign that we just checked the boxes without doing the work and will randomly reject based on that one fact. Mass rejections can ruin our stats and prevent us from qualifying for HITS or even worse, get our accounts suspended. We have absolutely no recourse if this happens. It is unfortunate because it creates what you have seen. A bunch of HITS returned, people checking the HITS before accepting them and average times that are a fraction of what was actually spent on the HIT. As everyone has said the only way to avoid scammers is to set qualifications higher or to make a qualification test.
