Lots of new HITs on labeling human pose in the images

Discussion in 'Do My HIT!' started by TurkPoseEstimation, Mar 29, 2013.

  1. TurkPoseEstimation

    TurkPoseEstimation Verified Requester

    Joined:
    Mar 29, 2013
    Messages:
    65
    Likes Received:
    0
  2. Frankyboy07

    Joined:
    Aug 20, 2013
    Messages:
    215
    Likes Received:
    0
    edit: wrong place
     
    #82 Frankyboy07, Aug 24, 2013
    Last edited by a moderator: Aug 24, 2013
  3. TurkPoseEstimation

    TurkPoseEstimation Verified Requester

    Joined:
    Mar 29, 2013
    Messages:
    65
    Likes Received:
    0
    Hi,

    I posted a new set of tasks:

    https://www.mturk.com/mturk/preview?groupId=2EFB3Q39Z0B8QZHXQ55V0TVRKPFVK3

    I also noticed that by now some people are submitting quite a number of annotations rather quickly. Although in most of the cases the quality is really good, on several occasions I spotted a major glitches such as missed joints or annotation points not at the correct position. Please be careful and do not compromise on quality as I might have to withdraw qualification if quality of annotations is not good. I also encourage everyone to contact me in case there are questions with respect to annotation process. I know that pose annotations is a tedious task, but we need good quality of annotations in order to use these data.

    Best,
     
  4. TurkPoseEstimation

    TurkPoseEstimation Verified Requester

    Joined:
    Mar 29, 2013
    Messages:
    65
    Likes Received:
    0
    Hi everyone,

    I had to take a break to collect more images and now I can finally post more HITS. Please take a look at the tutorial again:

    http://datasets.d2.mpi-inf.mpg.de/amt-human-pose/tutorial/person/index.html

    Specifically, I made a visualization that hopefully clarifies annotation of shoulders and hips:

    http://datasets.d2.mpi-inf.mpg.de/amt-human-pose/tutorial/person/images/skeleton_joint_positions.jpg

    Here is a new batch:

    https://www.mturk.com/mturk/preview?groupId=2EFB3Q39Z0B8QZHXQ55V0TVRKPFVK3

    Best,
     
  5. TurkPoseEstimation

    TurkPoseEstimation Verified Requester

    Joined:
    Mar 29, 2013
    Messages:
    65
    Likes Received:
    0
    Hi everyone,

    what is the best way to fight these spam messages? Can one signal then to moderators somehow?

    Best,
     
  6. TurkPoseEstimation

    TurkPoseEstimation Verified Requester

    Joined:
    Mar 29, 2013
    Messages:
    65
    Likes Received:
    0
    Hi,

    sorry for a slight delay with reviewing submitted tasks, I should be done by tomorrow evening.

    Best,
     
  7. TurkPoseEstimation

    TurkPoseEstimation Verified Requester

    Joined:
    Mar 29, 2013
    Messages:
    65
    Likes Received:
    0
    Hi everyone,

    I processed the submitted HITs, and also uploaded the new ones. This time the task is slightly different. We would like to ask you to estimate the 3D orientation of the torso and head. This task should be a bit quicker than pose estimation. Please take a look:

    https://www.mturk.com/mturk/preview?groupId=2ZM3HVWAVKI8YTCHZ2Y2799QHHWOGQ

    The qualifications for pose estimation apply to this task as well. We have more data for pose estimation task, but the new task is more urgent and I would like to do it first.

    In case you would like to work on the task, but don't have qualification please let me know via a private message in this forum.

    Best,
     
  8. TurkPoseEstimation

    TurkPoseEstimation Verified Requester

    Joined:
    Mar 29, 2013
    Messages:
    65
    Likes Received:
    0
    oh, one more thing: it the interface does not load for you it might be because the browser (eg. Chrome) blocks content within the page that is loaded from a different website. In this case the AMT website loads our tool that i hosted on our website. The way to resolve this is to allow external content by clicking on the "shield" icon in the web address bar and then allowing external code in the context menu.
     
  9. TurkPoseEstimation

    TurkPoseEstimation Verified Requester

    Joined:
    Mar 29, 2013
    Messages:
    65
    Likes Received:
    0
  10. TurkPoseEstimation

    TurkPoseEstimation Verified Requester

    Joined:
    Mar 29, 2013
    Messages:
    65
    Likes Received:
    0
  11. TurkPoseEstimation

    TurkPoseEstimation Verified Requester

    Joined:
    Mar 29, 2013
    Messages:
    65
    Likes Received:
    0
  12. TurkPoseEstimation

    TurkPoseEstimation Verified Requester

    Joined:
    Mar 29, 2013
    Messages:
    65
    Likes Received:
    0
  13. TurkPoseEstimation

    TurkPoseEstimation Verified Requester

    Joined:
    Mar 29, 2013
    Messages:
    65
    Likes Received:
    0
  14. TurkPoseEstimation

    TurkPoseEstimation Verified Requester

    Joined:
    Mar 29, 2013
    Messages:
    65
    Likes Received:
    0
  15. TurkPoseEstimation

    TurkPoseEstimation Verified Requester

    Joined:
    Mar 29, 2013
    Messages:
    65
    Likes Received:
    0
    Hi,

    sorry i takes me a bit longer to review submitted HIT's this time, I should be done with this by the end of the weekend.

    Best,
     
  16. TurkPoseEstimation

    TurkPoseEstimation Verified Requester

    Joined:
    Mar 29, 2013
    Messages:
    65
    Likes Received:
    0
  17. TurkPoseEstimation

    TurkPoseEstimation Verified Requester

    Joined:
    Mar 29, 2013
    Messages:
    65
    Likes Received:
    0
    Hi everyone,

    I added a new type of task that requires to label cars in images of highway scenes. Please take a look:

    https://www.mturk.com/mturk/preview?groupId=246WIOV1S7SQWXUWO2UYNZ8N8VHZK5

    Since this is a new task I am very interested in any kind of feedback about the task and labeling tool. Also if you are interested in qualification and have previously worked on similar tasks let me know. When you write asking for qualification please include a short description of the tasks you did before (just a few lines).

    Best,
     
  18. BloodRayen

    BloodRayen User

    Joined:
    Oct 23, 2013
    Messages:
    2,187
    Likes Received:
    0
    Hey, your task look interesting. I cannot work on them right now since I`m at work. But what I got from your instructions is a clear border, which vehicles to mark and which can be ignored. On some you say ignore them cause they are too far away, but the last marked one is on almost the same distance.
    Will you reject people who judge it different or review every task if it is acceptable?
     
  19. TurkPoseEstimation

    TurkPoseEstimation Verified Requester

    Joined:
    Mar 29, 2013
    Messages:
    65
    Likes Received:
    0
    Hi,

    in general I won't reject any HITs, the worst thing that can happen is that I might withdraw the qualification if the results are consitently missing vehicles that should be labeled or labeling is consistently imprecise. In any case I will first try to give feedback over MTurk email. This is what I have been doing in the past with the human pose estimation task.

    Best,
     
  20. TurkPoseEstimation

    TurkPoseEstimation Verified Requester

    Joined:
    Mar 29, 2013
    Messages:
    65
    Likes Received:
    0

Share This Page