Grading Hubpages

Discussion in 'Requesters' started by bitler2000, Jan 10, 2013.

  1. bitler2000

    bitler2000 User

    Joined:
    Mar 5, 2012
    Messages:
    468
    Likes Received:
    0
    When Hubpages first appeared on Mturk a few months ago, I received their qual. I rated their articles, and I THOROUGHLY ENJOYED IT. Since, my rating accuracy dropped to 68 and I was no longer able to rate articles. I MISSED IT. Now Hubpages has granted me, like a lot of people, a chance to bring their accuracy up again. I am having an issue understanding exactly how they want their articles graded. I read their examples, and I feel that if they had feedback on every article I rated, like, "You rated substance at a 5, but Hubpages feel it should have been a 6" it would be more helpful for me to grasp what they want. Now, it could be that I am not cut out to rate these articles, but I wish I could get my accuracy up, I dearly love reading, and I think this is the perfect hit for me to do. Does anyone have any input on how they rate their articles? Any hints or tricks, or websites to help them gain proficiency on rating articles? I am going to email Hubpages and invite them into this thread, maybe if you are having the same issues I am, they can offer some advice.
     
  2. Inconceivable!

    Joined:
    Oct 25, 2012
    Messages:
    193
    Likes Received:
    0
    I was having this problem. I started accepting one, then doing something else to run up the clock, then going back and finishing. I also started grading lower than I normally would because I always got messages that I was being to lenient, even though I almost never rated anything a 9 or 10 to begin with. My accuracy was down around 77 and after I started doing this it is back up to around 82 - 83. Not great, but something to do when there's nothing else.
     
  3. Cathalain

    Cathalain User

    Joined:
    Nov 3, 2012
    Messages:
    321
    Likes Received:
    0
    I got my qual back, but honestly, I'm so soured on doing these now that I really don't want to do their work anymore. What amuses me is what they consider a "grade 9" article, I rate it MAYBE a 6 or 7; they don't have William Shakespeare working for them, after all. I consider things like page design (and I've seen some that are designed VERY nicely!), media and accuracy of language/grammar (and most of the articles I've seen of theirs are an utter FAIL in the grammar and spelling department).

    Apparently my editing skills are "too tough" for them, but I used to work in a lot of journalism fields, and the crap they get away with today wouldn't pass a freshman-level English course "way back when". Just saying.
     
  4. bitler2000

    bitler2000 User

    Joined:
    Mar 5, 2012
    Messages:
    468
    Likes Received:
    0
    Ok I invited Hubpages to take a look and this is the response I received.:

    I took a look at the advice given on the thread, and the advice given by Inconveivable! is bad advice. There's no point in "letting the clock run down." We know that time spent per HIT can be gamed, and we don't consider it. I told some people that they were rating too fast, and they took that to mean that they should make it appear that they are spending more time per HIT ... but that by itself won't help.

    It looks to me like the main mistake that you have made in the past is not noticing that some articles had gibberish writing in them. Take a look at the last example given in the directions to see how to correctly rate articles that contain nonsense words.

    I can give you a few specific examples. This one you rated a 7/6/7 but we don't think that it deserves such good marks. You gave much lower marks to these two articles, both of which we think are decent and didn't deserve the low marks that you gave them. (You gave both of those articles all 4s, which is much too harsh.)

    It's easier to give good ratings if you actually read the full article ...

    Good luck to you.




    I don't think I am cut out to do their hits.
     
  5. ewd76

    ewd76 User

    Joined:
    Oct 8, 2012
    Messages:
    1,133
    Likes Received:
    0
    Really? Who'd have though that?
     
  6. ewd76

    ewd76 User

    Joined:
    Oct 8, 2012
    Messages:
    1,133
    Likes Received:
    0
    Not all crap is equal.
     
  7. Whimsy

    Whimsy User

    Joined:
    Apr 22, 2012
    Messages:
    805
    Likes Received:
    0
    IMO, if they want people to take the time to read every word - they should pay for that up front. Dangling an elusive bonus is turning into a joke. 5 cents is worth 30 seconds. Pay up front for the time you want people to spend. 30-50 cents an article and the quality of the assessments will rise, I'm sure of it. They'd still moderate and have a qual - but people would stop stressing so much over hitting an hourly rate that isn't abysmal.

    As for tips, bah. I have gone back to fill some time in with them but I only use turking for $50 a week, the rest of the time I'm writing other places. I'd go for HubPages, but I'm an affiliate and I make my money through sales.

    Their rubric for rating is too detailed, with details sometimes conflicting. It's possible to have bad spelling but good grammar or mechanics. It is possible to have lots of GREAT media, but so-so text. Or an AWESOME review but poorly laid out media. If they went to a [check] box system, more of their questions would be concretely answered. Of course, it would also require more time - which means spending more money. They may find their review-first system isn't the way to go, too. As the site and the web change, so will their HITs.
     
  8. bitler2000

    bitler2000 User

    Joined:
    Mar 5, 2012
    Messages:
    468
    Likes Received:
    0
    Thank you for that, I do feel better about the whole thing. I was accepting like 5 at a time, then taking my time to read and evaluate them all, then submit them at once. I feel this Hubpage rep was unfair in her assessment of my work, all I did was email her, asking for help, to provide some tips here, and all they saw was the second comment, and assumed I did not put that much effort into my work, and told me to read the articles. I felt like I had been knocked down and kicked in the gut. Maybe I am being dramatic, oh well, I'll get over it, but I will never work for them again.
     
  9. Whimsy

    Whimsy User

    Joined:
    Apr 22, 2012
    Messages:
    805
    Likes Received:
    0
    Over 3,000 HITs later and I'm not sure wtf they want. I started off with a Qual of 97. It is now below 80. I'm not cut out for their HITs either, but I do them in the hopes one day they'll have a better way of relaying what they really want. That rating scale isn't cutting it.

    And if they've "given" the qual back - the one that's 1 <<< there's only a couple hundred people with the qual. They need more than that. There are so many Hubs published or republished every day, plus the qual checks they throw in.

    Or they should round up their "stellar" graders and just pay them a higher rate and be damned with the rest of us. I'm not EVER planning on my qual going back up to where there isn't a push to just get them done quickly since my interpretation of their rating system is pretty set now. One day I'll lose the qual and I'll just wait for them to bring in a different type of HIT.

    Writing them is almost helpful in the fact I think Ms. F wants to help, but doesn't see the problems from the viewpoint of the workers. She's been with HP for years and it's second nature for her.
     
  10. Cathalain

    Cathalain User

    Joined:
    Nov 3, 2012
    Messages:
    321
    Likes Received:
    0
    MTurk....the only place where you can "fail" for having an opinion.
     
  11. Cathalain

    Cathalain User

    Joined:
    Nov 3, 2012
    Messages:
    321
    Likes Received:
    0
    This is EXACTLY my problem with HubPages: what they want seems to be a matter of conflicting statements. For example: an article I might read could be rated a 6 on grammar/mechanics (and I'm sorry, but if I see even ONE spelling error - to me, that's bad, and if it's too tough a standard for everyone else, oh well, don't write articles if you can't spell and use English properly), but a 9 for substance (that is, the article has LOTS of good information that I might want to research or things that I didn't know). Yet, no matter what, if I actually do the work (which is being honest about what I see), I'm in danger of having my qual revoked or being told "you're not doing it right, fail".

    Uh, maybe if HubPages paid for better writers (and I've seen what it takes to qualify to write for them - not too damned much, I'll say that), perhaps they wouldn't need us to "grade" these articles, would they.

    I only did some of them to hit my 5k mark on submitted; at this point, I'm pretty much done with them, too. I read and grade VERY carefully - hell, you can't NOT when you proofread for a living - and if my editing skills aren't up to their standards, oh well - not much I can do about that.
     
  12. bitler2000

    bitler2000 User

    Joined:
    Mar 5, 2012
    Messages:
    468
    Likes Received:
    0
    I have noticed that many have had the same experience that I have had, our qual was granted a few months ago, then it reduced down slowly to where some have lost it. Why did they offer a chance for us to get it back? Did they lose too many people? Did they realize their grading system is flawed? Why do they need us to grade articles if they already know what grade they are?
     
  13. prakashkl

    prakashkl User

    Joined:
    Aug 3, 2012
    Messages:
    194
    Likes Received:
    0
    Their experts are rock bottom in standards. I used to be a member of hubpages When ever I publish an article which compete with an article published by their experts from India, these Indian experts will downgrade it. I had published around 30 articles n were having around 1000 visitors to ny pages. All in all hubpages is a great one to make money but their experts are at rock bottom
     
  14. Whimsy

    Whimsy User

    Joined:
    Apr 22, 2012
    Messages:
    805
    Likes Received:
    0
    I understand them wanting an unbiased base for rating the articles. There are cliques everywhere and ways to game whatever systems are in place, especially when asking the community to patrol "their own". I've had good luck with my accounts over at Squidoo and am glad they don't have a rating system like HP is implementing. I Truly Believe a writer can get their point across without having perfect grammar or spelling. Do KW/SEO research, use headings, write for people. Get views, get paid. Bing, bang, boom. Simple, right?

    But no. There's still a lot of IM crap floating around about link wheels and throwing articles up in directories - sometimes the same content repeatedly. Spun content, scraped content, poorly translated content. I believe they want to weed stuff like that out - along with the people who put up a Hub that's like a diary post. Without relevance or insight, that's not what HP is for.

    HP indexes every page after it goes live. Other sites have a cut off point. This has mostly been about things away from grading, but in the end it's like they're trying to sort the sand of the desert by color grade.

    On a side note, sites like Wizzley, Zujuva, Squidoo and HubPages are my saving grace. My grammar and mechanics are awful and I can't get a job as a writer. I'd hate to be my proofreader XD I'm good enough of a story teller/seller for the masses though.
     
  15. Inconceivable!

    Joined:
    Oct 25, 2012
    Messages:
    193
    Likes Received:
    0
    Who knows then. I just know my rating got a little better when I accepted one then came back to it, but maybe it was just coincidence. I don't really do them anymore anyway. 5 cents for the amount of reading they want doesn't really seem worth it, especially when the bonus they give is usually below .45.
     
  16. ewd76

    ewd76 User

    Joined:
    Oct 8, 2012
    Messages:
    1,133
    Likes Received:
    0
    I was wondering the same thing. To know this, they must be reading them, and if they are reading them and coming up with their own conclusions they don't need any help from mTurk.
     
  17. bitler2000

    bitler2000 User

    Joined:
    Mar 5, 2012
    Messages:
    468
    Likes Received:
    0
    This is one of their sample articles:

    Review and Summary of On The Waterfront
    EXPLORE»ENTERTAINMENT AND MEDIA»MOVIES AND MOVIE REVIEWS»CLASSIC MOVIES
    Terry Malloy is full of contradictions. A former boxer who cares for pigeons, a mob henchman who dislikes not being apprised of what is going on around him, and a quiet fighter who allows himself to be brutally beaten in order to rally the longshoremen of the docks together against the mob. The transformation of Terry takes place throughout the film, and cannot easily be associated with any one character’s introduction or any plot twist.


    The culmination of all those elements result in a reformed Terry who realizes the greater good is more important than his immediate interests. While he blames his brother Charley for forcing him to take the fall in a boxing match, he had placed himself in such positions by his own free-will. It isn’t until he takes responsibility for his life that he can begin to change its trajectory.


    Redemption through self-sacrifice is a common theme in American film. Heroes are often called upon to make amends for their own mistakes, and to act in such a way as to prevent the suffering of others, no matter the cost to themselves. Terry Malloy’s efforts in the film are not in vain. His actions lead to the protest of his coworkers, and they refuse to return to work until he can enter the building with them. Terry forcing himself across the dock into the building is one of the most stirring scenes in the film, providing a uniquely peaceful ending to a violent story.

    They graded it a 4 in substance, "Publisher's message is burdened by inconsistent tone and tedious diction or media techniques
    Supporting elements, if present, at least do not distract the visitor
    Content relies on generalities rather than specifics and may contain clearly promotional material"

    a 4 in organization, "The content does not contain a logical structure, reducing the impact of the message the publisher may have had
    Title is ok, but broad or not fully supported by the content
    Subheadings, if present, are not supported
    Visually the content may be below average
    links, media, and supporting elements are not directly related to the topic, and they may distract from the content visually"

    an 8 in grammar and mechanics, "Deviations from standard usage are infrequent and minor, and do not significantly detract from the page"

    Now if I was to grade this article, I would give it higher marks in substance but really I don't know what they want when asking for substance.
     
  18. ewd76

    ewd76 User

    Joined:
    Oct 8, 2012
    Messages:
    1,133
    Likes Received:
    0
    I'm inclined to agree with them and you as far as the basic grading goes, but I also think it's hard to know why they only graded the content as a 4.
     
  19. Cathalain

    Cathalain User

    Joined:
    Nov 3, 2012
    Messages:
    321
    Likes Received:
    0
    As difficult as it may seem, I can see why the rating on this is a 4; instead of talking about the movie itself, or the plot/theme/etc., a lot of this article seems to be focused strictly on one character, that of Terry Malloy, which I guess one could see as "promotion". It's also not really telling the reader ABOUT the film so much - it's a basic skim-over that I could read in any of Leonard Maltin's "Movie Encyclopedia" review books. WHAT makes the film a classic? What is it about the story that grabs the attention of the viewer?

    This is just how I'm reading it, but I can see why the 4 would be justified. Also, it's way too short in length - a good movie review should have more information than this.

    This is very difficult for me to judge because I can't see the original article. Organization, to me, means a nice page layout, with attractive headers and good paragraphs. It could be that the media links are not relevant or broken (or that there aren't any). Without seeing the original page, I can't tell you if a 4 is justified here.

    For the most part, I agree with their rating for this category. I do see some minor grammatical errors, but it's not anything that most would notice.
     
  20. bitler2000

    bitler2000 User

    Joined:
    Mar 5, 2012
    Messages:
    468
    Likes Received:
    0
    That makes sense. See i wish hubpages would explain their grading like this.
     

Share This Page