Top Qs
Timeline
Chat
Perspective
Anubis (software)
Software designed to prevent scraping From Wikipedia, the free encyclopedia
Remove ads
Anubis is a software program that adds a JavaScript-powered proof of work challenge to websites before users can access them, as a form of deterring the impacts of web scraping from bots. It has been adopted mainly by Git forges and free and open-source software projects.[4][5]
It was created by Xe Iaso in response to Amazon's web crawler overloading their Git server, as it did not respect the robots.txt exclusion protocol and would work around restrictions.[4][6]
Remove ads
Adoption
It is used by a number of projects, including:[7]
- The GNOME Project's GitLab instance[4]
- The Linux kernel mailing list archives[1] and Git server[8]
- sourceware.org[7]
- FFmpeg[1][6]
- FreeBSD
- FreeDesktop
- Wine[9]
- UNESCO[6]
- The Science Olympiad Student Center[9]
- Enlightenment[9]
- FreeCAD[10]
- Krantenbank Zeeland
- ScummVM[11]
Criticism
Tavis Ormandy criticized the project, citing the low financial cost for scraping companies to calculate the hashes.[12]
References
External links
Wikiwand - on
Seamless Wikipedia browsing. On steroids.
Remove ads