2022-11-20 02:31:25 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'Romanel' 2022-11-20 02:31:26 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'Genève' 2022-11-20 02:31:27 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'Kriens' 2022-11-20 02:31:28 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'Egerkingen' 2022-11-20 02:31:30 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'Biel' 2022-11-20 02:31:31 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'Villeneuve' 2022-11-20 02:31:32 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'Jona' 2022-11-20 02:31:33 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'Matran' 2022-11-20 02:31:34 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'St.' 2022-11-20 02:31:35 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'Dübendorf' 2022-11-20 02:31:37 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'Conthey' 2022-11-20 02:31:38 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'Oberentfelden' 2022-11-20 02:31:39 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'Bern' 2022-11-20 02:31:40 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'Brig' 2022-11-20 02:31:42 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'Lyssach' 2022-11-20 02:31:43 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'Pratteln' 2022-11-20 02:31:44 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'Biel' 2022-11-20 02:31:45 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'Villmergen' 2022-11-20 02:31:46 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'Heimberg' 2022-11-20 02:31:47 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'Etoy' 2022-11-20 02:31:48 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'Spreitenbach' 2022-11-20 02:31:50 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'Meyrin' 2022-11-20 02:31:51 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'Pratteln' 2022-11-20 02:31:52 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'Cham' 2022-11-20 02:31:53 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'Vz' 2022-11-20 02:31:55 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'Dietlikon' 2022-11-20 02:31:56 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'Winterthur' 2022-11-20 02:31:57 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'Neuchâtel' 2022-11-20 02:31:58 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'Seewen' 2022-11-20 02:32:00 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'St.' 2022-11-20 02:32:00 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'Altendorf' 2022-11-20 02:32:02 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'Hinwil' 2022-11-20 02:32:03 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'Sierre' 2022-11-20 02:32:04 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'Vz' 2022-11-20 02:32:06 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'Frauenfeld' 2022-11-20 02:32:07 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'Emmen' 2022-11-20 02:32:08 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'Chur' 2022-11-20 02:32:10 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'Spreitenbach' 2022-11-20 02:32:11 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'Dübendorf' 2022-11-20 02:32:12 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'Montagny' 2022-11-20 02:32:13 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'Ittigen' 2022-11-20 02:32:14 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'Heimberg' 2022-11-20 02:32:16 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'Steinhausen' 2022-11-20 02:32:17 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'Niederwangen' 2022-11-20 02:32:18 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'Signy' 2022-11-20 02:32:19 [scrapy.core.scraper] ERROR: Spider error processing (referer: https://www.livique.ch/medias/STORE-de-CHF-3945545063279172348.xml?context=bWFzdGVyfHJvb3R8ODc5OXx0ZXh0L3htbHxoY2EvaGFiLzExMjE2MzUxMzYzMTAyLnhtbHw2ZGQwOWQ3ZTgzZTFkYjE1MTQ1MDQ4ODgxNzEwZGQ4ZDViNTY2NjE5NDg0NjVhOWFmOWU4MzNlZjVlOWU1ZGQw) Traceback (most recent call last): File "/usr/local/lib/python3.9/site-packages/scrapy/utils/defer.py", line 240, in iter_errback yield next(it) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/utils/python.py", line 338, in __next__ return next(self.data) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/referer.py", line 336, in return (self._set_referer(r, response) for r in result or ()) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/urllength.py", line 28, in return (r for r in result or () if self._filter(r, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/usr/local/lib/python3.9/site-packages/scrapy/spidermiddlewares/depth.py", line 32, in return (r for r in result or () if self._filter(r, response, spider)) File "/usr/local/lib/python3.9/site-packages/scrapy/core/spidermw.py", line 79, in process_sync for r in iterable: File "/opt/app/locations/structured_data_spider.py", line 120, in parse_sd yield from self.post_process_item(item, response, ld_item) File "/opt/app/locations/spiders/livique_ch.py", line 24, in post_process_item brand_wikidata = self.brands[brand] KeyError: 'Rickenbach'