Page 1 of 2 1 2 LastLast
Results 1 to 10 of 12

Thread: Top 1000 Requesters

  1. #1
    User ergo's Avatar
    Join Date
    Jan 2009
    Location
    Maryland
    Posts
    358
    Thanks
    0
    Thanked 1 Time in 1 Post

    Default Top 1000 Requesters

    Panos Ipeirotis has another interesting post up on his blog, along with a new tool. He's put together a list of MTurk's top 1000 requesters in terms of how much money they've spent on rewards so far in 2009. Check it out:

    http://behind-the-enemy-lines.blogsp...ical-turk.html

  2. #2
    User lightdark's Avatar
    Join Date
    Feb 2009
    Posts
    136
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    The quick hits are not represented well. I can't even find AEXLR15TK0KXH - Content Review in the top 300. I know they should be.

  3. #3
    Furca Media Andy's Avatar
    Join Date
    Jan 2009
    Location
    Santa Monica, CA
    Posts
    2,025
    Thanks
    1,170
    Thanked 3,450 Times in 825 Posts

    Default

    I'm ranked #298 currently :P

  4. #4
    User lightdark's Avatar
    Join Date
    Feb 2009
    Posts
    136
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    Quote Originally Posted by Andy View Post
    I'm ranked #298 currently :P
    Was that how much you actually put in, minus Amazon fees, or is it undervalued?

  5. #5
    User ergo's Avatar
    Join Date
    Jan 2009
    Location
    Maryland
    Posts
    358
    Thanks
    0
    Thanked 1 Time in 1 Post

    Default

    Could have sworn I posted something above explaining that Panos only scrapes MTurk a few times an hour, which is why some requesters will be undervalued or not represented, but appears I was imagining things. Would be nice if Amazon themselves would share their data as it would be complete, but I think it's a pretty safe bet that such will never happen.
    Last edited by ergo; 08-06-2009 at 06:03 PM.

  6. #6
    User
    Join Date
    Aug 2009
    Posts
    6
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    Hi guys,

    Some comments on the limitations of the tool: I crawl the website every 15 minutes, logging in using my own account.

    So, if there is a qualification test, then I cannot really capture the HIT. Same thing if my Turker account does not meet the requirements for the HIT.

    Also, HITs that are posted and completed in less than 15 minutes may not be captured as the crawler captures only tasks that appear at NN:00, NN:15, NN:30, and NN:45. My analysis indicates that there are not that many tasks that disappear within 15 minutes, compared to the total amount of tasks posted. (Initially, I was crawling every 1 hour and increased the frequency to every 15 minutes but I do not seem to capture many more HITs than before.)

    The major limitation of the tool is the fact that it cannot capture the value of tasks that get multiple Turkers to complete the same HIT. So, there is definitely some undervaluing factor there, depending on the requester. For some requesters the reward amount is close to accurate, for some others it may be a 2x to 5x underestimate. And for survey-like HITs it may be even a 10000x underestimate.

    Cheers,
    Panos

  7. #7
    User justfight's Avatar
    Join Date
    Feb 2009
    Location
    India
    Posts
    246
    Thanks
    1
    Thanked 13 Times in 4 Posts

    Default

    Interesting stuff. Actually, I've bookmarked this blog's page with posts labeled "mechanical turk" [http://behind-the-enemy-lines.blogsp...hanical%20turk] since when I first came to know of this blog via ChatBox here, thanx to ergo. Though I'd like to admit that I haven't had the opportunity to read more of the posts due to certain limitations of my own.

    BTW, the page at http://hyperion.stern.nyu.edu/mturk/requesters.php did take time in loading & did load successfully on my Firefox browser [version 3.0.11], however the page loaded was absolutely blank. Tried to reload a few times in vain. Fortunately, it loaded well on my Opera browser (due to which I didn't have to try it on my Google Chrome, IE or Flock browser).
    Confession is good for the soul, but bad for your career.

  8. #8
    User
    Join Date
    Aug 2009
    Posts
    6
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    I tried loading the page on FF 3.5.2 (Windows Vista) and it seems to work fine.

    I am using some Google Libraries to display the table, and it is not truly compatible across all-browsers. Not sure what I can do other than wait and see

    Can you upgrade FF to the latest version and see if the problem persists?

  9. #9
    User lightdark's Avatar
    Join Date
    Feb 2009
    Posts
    136
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default

    It works on FF 3.0.7 (I'm waiting for certain addons to update so I can)

    Just don't be too fast hitting the page forward button, once about a second has no issues.

  10. #10
    User justfight's Avatar
    Join Date
    Feb 2009
    Location
    India
    Posts
    246
    Thanks
    1
    Thanked 13 Times in 4 Posts

    Default

    Quote Originally Posted by ipeirotis View Post
    Can you upgrade FF to the latest version and see if the problem persists?
    Can't do that for the time being for specific reasons. But this isn't a big deal as it seems I'm the only one having problem so far. So most probably the fault lies in my system.

    And I didn't close the tab the very moment the page was completely loaded, but was empty. I waited for at least 5 minutes [sometimes it takes about half minute for an image to appear even after the page (happens only in cases where the page consists of an image, the ones with .jpg extension & alike) is completely loaded with this current Firefox version of mine] to see if the content loads, but it didn't. Anyways, as stated earlier, the problem lies in my system.
    Last edited by justfight; 08-07-2009 at 11:50 AM.
    Confession is good for the soul, but bad for your career.

Page 1 of 2 1 2 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •