robots

This article is a stub. You can help the IndieWeb wiki by expanding it.

robots are automated scripts that crawl, search or perform multiple requests for information.

See Also

• robots txt
• meta robots
• bandwidth: Dealing with bot user agents
• 2018-04-03 CJR: The bots beat: How not to get punked by automation