Hello Everyone, I am new to mturk forum. I am running a study which involves collecting human judgements on tweets (more generally social media data). There are two sets of tasks: 1. IDENTIFY WHETHER TWEET IS AN EVENT OR NOT 2. EVALUATE THE CREDIBILITY LEVEL OF TWEETS I have launched these hits on Mturk in the past and so far I have received pretty positive feedback. I will be launching more of these every day for the next month and a half. It will be great if you can work on them and also give me feedback. You can search for these hits by the keywords (in ALL CAPS) mentioned above. For both tasks you will have to pass a one time qualification test. The purpose of the qualification test is to explain you how to do the task and better assist you in finishing the hit. Also, for the 2nd set of task, I will be assigning workers randomly to three conditions (Case1, Case2, Case3). Once you are assigned to one particular conditions you will be qualified to work only on that condition. That said, I will post the same number of hits in all three conditions, so that each case is treated equally My Mturk requester name is - Comp.Social Group. You can grab some hits now at - https://www.mturk.com/mturk/searchbar?selectedSearchType=hitgroups&searchWords=comp.social Looking forward to working with you all
hey there, your HITs look promising, but i have two questions. 1. How will you rate a workers accuracy? Some of the tweets are easy to check with google and general media. So I do not really see why you need all 5 options. in my mind it would be enough to judge them as true, false, need mor information. 2. What is your estimated time of approving/rejecting? AA is set to 30 days, but since you post here I assume you know turkers thoughts on fast requesters. Also I think that you will be one of the faster requesters as you directly ask us to join on your project
The current research study requires us to keep all 5 options. If you have done one of my hits, you will see that my AA is pretty quick, within a day (mostly a few hours). However, grading for bonus only occurs after all assignments on a hit is completed, which is why there can be a delay. On an average each hit has 50 assignments and the more Turkers come and do it, the quicker it will be finished, speeding the process of bonus payment. So spread the word if you can
I see, maybe something to improve in the future. yeah, I saw that you are incredible fast. think you were approving while I was still working on my groups HITs
Hello Turkers, Some more new hits on your way We have posted a small batch of the credibility assessment HITs. You can find them by the hit title - EVALUATE THE CREDIBILITY LEVEL OF TWEETS. Would love to have you work on them Thanks, Comp.Social Group
Hi, I just did 3 of your HITs. I think the ones I did were Case 3. They were the only ones that show I was eligible to do them. Does that mean that I have the qualification or was that a test batch? Thanks!
Thanks bobbie01 for doing the hits Yes you should have the qualification. Did you pass the qualification test? Turkers are assigned randomly to one of the three conditions (Case 1, 2, or 3). You were assigned to Case 3. Once you are assigned to one particular condition you can work only on that condition. That said, I will post the same number of hits in all three conditions, so that each case is treated equally. So you are not losing out on anything when you are assigned to a condition.
I must have passed it. I recognized the HITs. I did some the other day so I guess they were the qualification tests. Will we only be able to do 3 each time you post them?
For this week, I will be posting smaller batches of these credibility hits, titled - EVALUATE THE CREDIBILITY LEVEL OF TWEETS. But I will post these more frequently starting the 3rd week of September and extending the entire October and most of November. Also, I have been posting another set of hits called IDENTIFY WHETHER TWEET IS AN EVENT OR NOT which are larger batches. I usually post those earlier during the day.
The EVALUATE THE CREDIBILITY LEVEL OF TWEETS are still up https://www.mturk.com/mturk/searchbar?selectedSearchType=hitgroups&searchWords=comp.social Catch them while they last
New hits posted again on credibility assessment of tweets - https://www.mturk.com/mturk/searchbar?selectedSearchType=hitgroups&searchWords=Comp.Social+Group
Can you tell us what you are looking for in a reason? I'm always worried that I'm not putting the correct thing. Sometimes it's as simple as it seems credible.
Hi bobbie01, when you choose a particular credibility label for the tweet, you reach that conclusion based on some reasoning. Different people use different criteria to judge the credibility of an item. I would want to know what reasoning and what criteria you have used in judging that tweet and choosing the particular credibility label.
There are some more hits.. Please take them quick https://www.mturk.com/mturk/searchbar?selectedSearchType=hitgroups&searchWords=Comp.Social+Group