2022-11-20 05:14:07 [scrapy.downloadermiddlewares.retry] ERROR: Gave up retrying (failed 3 times): User timeout caused connection failure: Getting https://www.rei.com/robots.txt took longer than 180.0 seconds.. 2022-11-20 05:14:07 [scrapy.downloadermiddlewares.robotstxt] ERROR: Error downloading : User timeout caused connection failure: Getting https://www.rei.com/robots.txt took longer than 180.0 seconds.. Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/twisted/internet/defer.py", line 1693, in _inlineCallbacks result = context.run( File "/usr/local/lib/python3.9/site-packages/twisted/python/failure.py", line 518, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb) File "/usr/local/lib/python3.9/site-packages/scrapy/core/downloader/middleware.py", line 49, in process_request return (yield download_func(request=request, spider=spider)) File "/usr/local/lib/python3.9/site-packages/twisted/internet/defer.py", line 892, in _runCallbacks current.result = callback( # type: ignore[misc] File "/usr/local/lib/python3.9/site-packages/scrapy/core/downloader/handlers/http11.py", line 360, in _cb_timeout raise TimeoutError(f"Getting {url} took longer than {timeout} seconds.") twisted.internet.error.TimeoutError: User timeout caused connection failure: Getting https://www.rei.com/robots.txt took longer than 180.0 seconds.. 2022-11-20 05:23:07 [scrapy.downloadermiddlewares.retry] ERROR: Gave up retrying (failed 3 times): User timeout caused connection failure: Getting https://www.rei.com/sitemap-stores.xml took longer than 180.0 seconds.. 2022-11-20 05:23:07 [scrapy.core.scraper] ERROR: Error downloading Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/twisted/internet/defer.py", line 1693, in _inlineCallbacks result = context.run( File "/usr/local/lib/python3.9/site-packages/twisted/python/failure.py", line 518, in throwExceptionIntoGenerator return g.throw(self.type, self.value, self.tb) File "/usr/local/lib/python3.9/site-packages/scrapy/core/downloader/middleware.py", line 49, in process_request return (yield download_func(request=request, spider=spider)) File "/usr/local/lib/python3.9/site-packages/twisted/internet/defer.py", line 892, in _runCallbacks current.result = callback( # type: ignore[misc] File "/usr/local/lib/python3.9/site-packages/scrapy/core/downloader/handlers/http11.py", line 360, in _cb_timeout raise TimeoutError(f"Getting {url} took longer than {timeout} seconds.") twisted.internet.error.TimeoutError: User timeout caused connection failure: Getting https://www.rei.com/sitemap-stores.xml took longer than 180.0 seconds..