Skip to main content

One post tagged with "headless"

View All Tags

· 6 min read
Mark

Web scraping can be difficult because many websites attempt to block developers from scraping their sites. They do this by detecting IP addresses, inspecting HTTP request headers, using CAPTCHA, inspecting Javascript code and more. In response to these blocks, web scrapers can be made extremely hard to detect. This is because many of the same techniques that developers use to avoid these blocks are also used by them on the site they want to scrape. Here are some tips for making scraping a website easy: