Oriel — AI & technology worth noticing, every day.

Breaking the open web won't stop the bad bots

Web page · https://techpolicy.press/breaking-the-open-web-wont-stop-the-bad-bots

First seen by Oriel on .

AI summary

The article argues that proposed federal and New York laws targeting anonymous web crawlers would chill legitimate journalism, research, archiving, and security work without addressing the harms of abusive scraping.

  • The Stealth Bot Prohibition Act could impose FTC penalties of up to $53,000 for conduct deemed reasonably likely to burden a website, but the bill does not define that standard.
  • The New York bill would let covered news publishers subpoena crawler identities without showing harm, even though disclosure does not limit request volume or compensate publishers.
  • Examples of beneficial scraping include the Wall Street Journal’s analysis of 31,000 financial disclosures, the Wayback Machine’s archive of more than one trillion webpages, and accessibility audits of over one million websites.
  • The authors warn that compelled identification and vague liability could deter anonymous public-interest research and burden the First Amendment right to receive information.
  • They recommend interoperable technical standards for managing machine access and, if legislation is adopted, liability based on identifiable harms such as server overload, fraud, or prohibited data collection.

#web scraping #ai policy #open web #internet archive

Generated by gpt-5.6-sol. Check the original for details.

Metric trends

Last 30 days · Up to 120 recorded samples per source and metric, with reposts kept separate · First and latest retained

Points

Score reported by the source.

  • Hacker News: 323, latest observation ; +316 over observed period since

Comments

Comment count reported by the source.

  • Hacker News: 147, latest observation ; +147 over observed period since

Aliases

SourceExternal IDURL
Hacker News50263837https://www.techpolicy.press/breaking-the-open-web-wont-stop-the-bad-bots/ Read discussion