"You have exceeded the maximum allowed page request rate for this website."

Discussion in 'General' started by ohsostrange, Aug 27, 2012.

  1. ohsostrange

    Joined:
    Aug 13, 2012
    Messages:
    2,891
    Likes Received:
    779
    If the hits on a page all look like garbage, or I come across a blank page because there's nothing there but requesters I've blocked from my sight, I can flip through a stack of pages pretty quickly... Or at least I could before today. Now I keep seeing "You have exceeded the maximum allowed page request rate for this website." over and over again.

    WTF? Has this been happening to any of you?
     
  2. mailtosudhas

    mailtosudhas Member

    Joined:
    Jul 30, 2012
    Messages:
    19
    Likes Received:
    0
    No this is new to me
     
  3. Khalinov

    Khalinov User

    Joined:
    Dec 12, 2011
    Messages:
    873
    Likes Received:
    0
    Did you get that on mturk.com or on some other site?

    Sounds like the server in question is throttling you (which is different from an ISP throttling you). Sometimes a webmaster will do that to thwart site rippers / scrapers.
     
  4. ohsostrange

    Joined:
    Aug 13, 2012
    Messages:
    2,891
    Likes Received:
    779
    Yeah, it's mturk. If it's the server doing it, then it's mturk doing it, because surely they own their own server.

    I don't understand the ripper/scraper part. What are they afraid of, exactly? All I'm trying to scrape off that website is pennies, and that only by finding something worth doing as quickly as possible, so I can earn them.
     
  5. kalhana

    kalhana User

    Joined:
    Aug 22, 2012
    Messages:
    112
    Likes Received:
    0
    Sometimes that happens to me if I am running to many scripts on that page. Or maybe because you have so many blocked requesters? I don't know.
     
  6. ohsostrange

    Joined:
    Aug 13, 2012
    Messages:
    2,891
    Likes Received:
    779
    Oh... I did add a script to the page. I'll bet that's it.
    Thanks.
     
  7. lilkolo91

    lilkolo91 User

    Joined:
    Jul 25, 2012
    Messages:
    322
    Likes Received:
    0
    yeah this happens to me all the time. Just refreshing the page remedies it.
     
  8. ds_36

    ds_36 User

    Joined:
    May 25, 2012
    Messages:
    150
    Likes Received:
    0
    Same with me. It seems to have to do with running too many scripts and going between pages too fast. I've tried to slow down now.
     

Share This Page