Ultimate

X_SCRAPE_ROBOTS_DISALLOWED

Means

robots.txt disallows this path

Typical cause

terminal. The site's own file refuses this path for the declared user agent

Fix

declare robots: { ignore: '<the written reason this run is permitted>' } on the scrape() definition, or scrape a path robots.txt allows

Listed under Scraping

This page as markdown