Desktop version Jobs
  • Analysis
  • Categories
  • Logs
  • Crawler.stats
  • projectrental_dk
    spiderrsbolig.dk
    job1f4e589e469a11f0a347f6e3f816cb0c
    first_log_time2025-06-11 08:08:46
    latest_log_time2025-06-11 08:08:54
    runtime0:00:08
    crawled_pages 1
    scraped_items 30
    shutdown_reasonN/A
    finish_reasonfinished
    log_critical_count0
    log_error_count0
    log_warning_count11
    log_redirect_count0
    log_retry_count0
    log_ignore_count0
    latest_crawl
    latest_scrape
    latest_log
    current_time
    latest_item{'area': 91.0, 'available': '2025-07-01', 'deposit_numb': 18100.0, 'external_property_attributes': {'external_property_id': '07b590d17bc4b4e2f71bc87916f270f4', 'external_provider_id': 433, 'property_url': 'https://www.rsbolig.dk/ledige-boliger/'}, 'fixed_expanses': 1250.0, 'house_number': 6, 'images': ['https://rsbolig.dk/Files/Boligdata/Afdelinger/1-094/Billeder/1.JPG', 'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-094/Billeder/4.JPG', 'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-094/Billeder/12.JPG', 'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-094/Billeder/6.JPG', 'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-094/Billeder/11.JPG', 'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-094/Billeder/2.JPG', 'https://rsbolig.dk/Files/Boligdata/PlantegningerEseeBase/01-94-1254.png', 'https://rsbolig.dk/Files/Boligdata/PlantegningerEseeBase/3D/01-94-1254.png', 'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-094/Billeder/9.JPG', 'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-094/Billeder/10.JPG', 'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-094/Billeder/5.JPG', 'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-094/Billeder/13.JPG', 'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-094/Billeder/8.JPG', 'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-094/Billeder/7.JPG', 'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-094/Billeder/3.JPG'], 'pets_allowed': False, 'property_type': 'Apartment', 'rent': 7685.0, 'rooms': 3, 'street_name': 'Toften', 'zip_code_code': 6880}

    • WARNING+

    • warning_logs
      11 in total 

      2025-06-11 08:08:46 [py.warnings] WARNING: /usr/local/lib/python3.10/site-packages/scrapy/core/spidermw.py:433: ScrapyDeprecationWarning: houses.spiders.rsbolig.RsboligDKSpider defines the deprecated start_requests() method. start_requests() has been deprecated in favor of a new method, start(), to support asynchronous code execution. start_requests() will stop being called in a future version of Scrapy. If you use Scrapy 2.13 or higher only, replace start_requests() with start(); note that start() is a coroutine (async def). If you need to maintain compatibility with lower Scrapy versions, when overriding start_requests() in a spider class, override start() as well; you can use super() to reuse the inherited start() implementation without copy-pasting. See the release notes of Scrapy 2.13 for details: https://docs.scrapy.org/en/2.13/news.html
  warn(

      2025-06-11 08:08:46 [py.warnings] WARNING: /usr/local/lib/python3.10/site-packages/scrapy/downloadermiddlewares/offsite.py:83: UserWarning: allowed_domains accepts only domains, not URLs. Ignoring URL entry http://www.rsbolig.dk in allowed_domains.
  warnings.warn(message)

      2025-06-11 08:08:52 [scrapy.core.scraper] WARNING: Dropped: Duplicate item found: {'area': 78.7,
 'available': '2025-01-15',
 'deposit_numb': 17310.0,
 'external_property_attributes': {'external_property_id': 'bb8408880a96651104cce8535ee71d05',
                                  'external_provider_id': 433,
                                  'property_url': 'https://www.rsbolig.dk/ledige-boliger/'},
 'house_number': 29,
 'images': ['https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14_1 N P Jeppesens vej.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14_2 Adelvej.JPG',
            'https://rsbolig.dk/Files/Boligdata/PlantegningerEseeBase/01-14-0680.png',
            'https://rsbolig.dk/Files/Boligdata/PlantegningerEseeBase/3D/01-14-0680.png',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14_1 Lybækvej.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14 Lybækvej.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '21_1 Skelmos14ej.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14_1 Fresiavej.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14 Regnspovevej.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14_1 Byvejen.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14 Vesterhavsvej.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14_1 Regnspovevej.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14_1 Vesterhavsvej.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14_1 Bjørnevej.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14 Skelmosevej.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14 Mads Degnbolsvej.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14 Bjørnevej.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14 Adelvej.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14_2 Fresiavej.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14 Trine Rahbeksvej (2).jpg',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14 Hovervej.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14_1 Mads Degnbolsvej.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14 Byvejen.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14_2 Bjørnevej.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14 N P Jeppesens vej.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14_2 Vesterhavsvej.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14 Fresiavej.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14_1 Adelvej.JPG'],
 'pets_allowed': True,
 'property_type': 'Apartment',
 'rent': 6311.0,
 'rooms': 3,
 'street_name': 'Byvejen',
 'zip_code_code': 6950}
{'area': 78.7,
 'available': '2025-01-15',
 'deposit_numb': 17310.0,
 'external_property_attributes': {'external_property_id': 'bb8408880a96651104cce8535ee71d05',
                                  'external_provider_id': 433,
                                  'property_url': 'https://www.rsbolig.dk/ledige-boliger/'},
 'house_number': 29,
 'images': ['https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14_1 N P Jeppesens vej.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14_2 Adelvej.JPG',
            'https://rsbolig.dk/Files/Boligdata/PlantegningerEseeBase/01-14-0680.png',
            'https://rsbolig.dk/Files/Boligdata/PlantegningerEseeBase/3D/01-14-0680.png',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14_1 Lybækvej.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14 Lybækvej.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '21_1 Skelmos14ej.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14_1 Fresiavej.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14 Regnspovevej.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14_1 Byvejen.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14 Vesterhavsvej.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14_1 Regnspovevej.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14_1 Vesterhavsvej.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14_1 Bjørnevej.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14 Skelmosevej.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14 Mads Degnbolsvej.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14 Bjørnevej.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14 Adelvej.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14_2 Fresiavej.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14 Trine Rahbeksvej (2).jpg',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14 Hovervej.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14_1 Mads Degnbolsvej.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14 Byvejen.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14_2 Bjørnevej.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14 N P Jeppesens vej.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14_2 Vesterhavsvej.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14 Fresiavej.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-014/Billeder/Afd. '
            '14_1 Adelvej.JPG'],
 'pets_allowed': True,
 'property_type': 'Apartment',
 'rent': 6311.0,
 'rooms': 3,
 'street_name': 'Byvejen',
 'zip_code_code': 6950}
      2025-06-11 08:08:52 [scrapy.core.scraper] WARNING: Dropped: Duplicate item found: {'area': 25.0,
 'available': '2025-04-15',
 'deposit_numb': 3000.0,
 'external_property_attributes': {'external_property_id': '874b00cfe858a66e48d098bf68bd93e1',
                                  'external_provider_id': 433,
                                  'property_url': 'https://www.rsbolig.dk/ledige-boliger/'},
 'fixed_expanses': 660.0,
 'house_number': 1,
 'images': ['https://rsbolig.dk/Files/Boligdata/Afdelinger/1-035/Billeder/Afd. '
            '35 Brogaardsvej 3.JPG',
            'https://rsbolig.dk/Files/Boligdata/PlantegningerEseeBase/01-35-0935.png',
            'https://rsbolig.dk/Files/Boligdata/PlantegningerEseeBase/3D/01-35-0935.png',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-035/Billeder/Afd. '
            '35 Brogaardsvej 1.JPG'],
 'pets_allowed': True,
 'property_type': 'Apartment',
 'rent': 3459.0,
 'rooms': 1,
 'street_name': 'Brogaardsvej',
 'zip_code_code': 6950}
{'area': 25.0,
 'available': '2025-04-15',
 'deposit_numb': 3000.0,
 'external_property_attributes': {'external_property_id': '874b00cfe858a66e48d098bf68bd93e1',
                                  'external_provider_id': 433,
                                  'property_url': 'https://www.rsbolig.dk/ledige-boliger/'},
 'fixed_expanses': 660.0,
 'house_number': 1,
 'images': ['https://rsbolig.dk/Files/Boligdata/Afdelinger/1-035/Billeder/Afd. '
            '35 Brogaardsvej 3.JPG',
            'https://rsbolig.dk/Files/Boligdata/PlantegningerEseeBase/01-35-0935.png',
            'https://rsbolig.dk/Files/Boligdata/PlantegningerEseeBase/3D/01-35-0935.png',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-035/Billeder/Afd. '
            '35 Brogaardsvej 1.JPG'],
 'pets_allowed': True,
 'property_type': 'Apartment',
 'rent': 3459.0,
 'rooms': 1,
 'street_name': 'Brogaardsvej',
 'zip_code_code': 6950}
      2025-06-11 08:08:52 [scrapy.core.scraper] WARNING: Dropped: Duplicate item found: {'area': 84.0,
 'available': '2024-12-15',
 'deposit_numb': 35400.0,
 'external_property_attributes': {'external_property_id': '4e767ac6c065a2288caceb846d06e1cc',
                                  'external_provider_id': 433,
                                  'property_url': 'https://www.rsbolig.dk/ledige-boliger/'},
 'fixed_expanses': 1000.0,
 'house_number': 2,
 'images': ['https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/02.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/03.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/07.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/01.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/04.JPG',
            'https://rsbolig.dk/Files/Boligdata/PlantegningerEseeBase/3D/01-51-1190.png',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/05.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '3.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '6.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '6_1.JPG',
            'https://rsbolig.dk/Files/Boligdata/PlantegningerEseeBase/01-51-1190.png',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '3_1.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/08.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/06.JPG'],
 'pets_allowed': False,
 'property_type': 'Apartment',
 'rent': 7475.0,
 'rooms': 2,
 'street_name': 'Fjordudsigten',
 'zip_code_code': 6950}
{'area': 84.0,
 'available': '2024-12-15',
 'deposit_numb': 35400.0,
 'external_property_attributes': {'external_property_id': '4e767ac6c065a2288caceb846d06e1cc',
                                  'external_provider_id': 433,
                                  'property_url': 'https://www.rsbolig.dk/ledige-boliger/'},
 'fixed_expanses': 1000.0,
 'house_number': 2,
 'images': ['https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/02.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/03.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/07.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/01.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/04.JPG',
            'https://rsbolig.dk/Files/Boligdata/PlantegningerEseeBase/3D/01-51-1190.png',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/05.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '3.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '6.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '6_1.JPG',
            'https://rsbolig.dk/Files/Boligdata/PlantegningerEseeBase/01-51-1190.png',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '3_1.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/08.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/06.JPG'],
 'pets_allowed': False,
 'property_type': 'Apartment',
 'rent': 7475.0,
 'rooms': 2,
 'street_name': 'Fjordudsigten',
 'zip_code_code': 6950}
      2025-06-11 08:08:52 [scrapy.core.scraper] WARNING: Dropped: Duplicate item found: {'area': 84.0,
 'available': '2024-12-15',
 'deposit_numb': 35400.0,
 'external_property_attributes': {'external_property_id': '4e767ac6c065a2288caceb846d06e1cc',
                                  'external_provider_id': 433,
                                  'property_url': 'https://www.rsbolig.dk/ledige-boliger/'},
 'fixed_expanses': 1000.0,
 'house_number': 5,
 'images': ['https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/02.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/03.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/07.JPG',
            'https://rsbolig.dk/Files/Boligdata/PlantegningerEseeBase/01-51-1199.png',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/01.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/04.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/05.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '3.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '6.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '6_1.JPG',
            'https://rsbolig.dk/Files/Boligdata/PlantegningerEseeBase/3D/01-51-1199.png',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '3_1.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/08.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/06.JPG'],
 'pets_allowed': False,
 'property_type': 'Apartment',
 'rent': 7475.0,
 'rooms': 2,
 'street_name': 'Fjordudsigten',
 'zip_code_code': 6950}
{'area': 84.0,
 'available': '2024-12-15',
 'deposit_numb': 35400.0,
 'external_property_attributes': {'external_property_id': '4e767ac6c065a2288caceb846d06e1cc',
                                  'external_provider_id': 433,
                                  'property_url': 'https://www.rsbolig.dk/ledige-boliger/'},
 'fixed_expanses': 1000.0,
 'house_number': 5,
 'images': ['https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/02.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/03.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/07.JPG',
            'https://rsbolig.dk/Files/Boligdata/PlantegningerEseeBase/01-51-1199.png',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/01.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/04.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/05.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '3.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '6.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '6_1.JPG',
            'https://rsbolig.dk/Files/Boligdata/PlantegningerEseeBase/3D/01-51-1199.png',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '3_1.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/08.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/06.JPG'],
 'pets_allowed': False,
 'property_type': 'Apartment',
 'rent': 7475.0,
 'rooms': 2,
 'street_name': 'Fjordudsigten',
 'zip_code_code': 6950}
      2025-06-11 08:08:52 [scrapy.core.scraper] WARNING: Dropped: Duplicate item found: {'area': 115.0,
 'available': '2024-12-15',
 'deposit_numb': 46700.0,
 'external_property_attributes': {'external_property_id': 'a86f8abe3e0ad05174acf7ddffa9c5db',
                                  'external_provider_id': 433,
                                  'property_url': 'https://www.rsbolig.dk/ledige-boliger/'},
 'fixed_expanses': 1300.0,
 'house_number': 8,
 'images': ['https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/02.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/03.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/07.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/01.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/04.JPG',
            'https://rsbolig.dk/Files/Boligdata/PlantegningerEseeBase/3D/01-51-1212.png',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/05.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '3.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '6.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '6_1.JPG',
            'https://rsbolig.dk/Files/Boligdata/PlantegningerEseeBase/01-51-1212.png',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '3_1.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/08.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/06.JPG'],
 'pets_allowed': False,
 'property_type': 'Apartment',
 'rent': 9000.0,
 'rooms': 3,
 'street_name': 'Fjordudsigten',
 'zip_code_code': 6950}
{'area': 115.0,
 'available': '2024-12-15',
 'deposit_numb': 46700.0,
 'external_property_attributes': {'external_property_id': 'a86f8abe3e0ad05174acf7ddffa9c5db',
                                  'external_provider_id': 433,
                                  'property_url': 'https://www.rsbolig.dk/ledige-boliger/'},
 'fixed_expanses': 1300.0,
 'house_number': 8,
 'images': ['https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/02.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/03.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/07.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/01.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/04.JPG',
            'https://rsbolig.dk/Files/Boligdata/PlantegningerEseeBase/3D/01-51-1212.png',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/05.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '3.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '6.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '6_1.JPG',
            'https://rsbolig.dk/Files/Boligdata/PlantegningerEseeBase/01-51-1212.png',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '3_1.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/08.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/06.JPG'],
 'pets_allowed': False,
 'property_type': 'Apartment',
 'rent': 9000.0,
 'rooms': 3,
 'street_name': 'Fjordudsigten',
 'zip_code_code': 6950}
      2025-06-11 08:08:52 [scrapy.core.scraper] WARNING: Dropped: Duplicate item found: {'area': 114.0,
 'available': '2024-12-15',
 'deposit_numb': 46700.0,
 'external_property_attributes': {'external_property_id': '1c20dc5c83eeb112a0992908a68d5020',
                                  'external_provider_id': 433,
                                  'property_url': 'https://www.rsbolig.dk/ledige-boliger/'},
 'fixed_expanses': 1500.0,
 'house_number': 10,
 'images': ['https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/02.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/03.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/07.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/01.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/04.JPG',
            'https://rsbolig.dk/Files/Boligdata/PlantegningerEseeBase/01-51-1216.png',
            'https://rsbolig.dk/Files/Boligdata/PlantegningerEseeBase/3D/01-51-1216.png',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/05.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '3.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '6.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '6_1.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '3_1.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/08.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/06.JPG'],
 'pets_allowed': False,
 'property_type': 'Apartment',
 'rent': 9000.0,
 'rooms': 3,
 'street_name': 'Fjordudsigten',
 'zip_code_code': 6950}
{'area': 114.0,
 'available': '2024-12-15',
 'deposit_numb': 46700.0,
 'external_property_attributes': {'external_property_id': '1c20dc5c83eeb112a0992908a68d5020',
                                  'external_provider_id': 433,
                                  'property_url': 'https://www.rsbolig.dk/ledige-boliger/'},
 'fixed_expanses': 1500.0,
 'house_number': 10,
 'images': ['https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/02.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/03.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/07.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/01.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/04.JPG',
            'https://rsbolig.dk/Files/Boligdata/PlantegningerEseeBase/01-51-1216.png',
            'https://rsbolig.dk/Files/Boligdata/PlantegningerEseeBase/3D/01-51-1216.png',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/05.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '3.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '6.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '6_1.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '3_1.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/08.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/06.JPG'],
 'pets_allowed': False,
 'property_type': 'Apartment',
 'rent': 9000.0,
 'rooms': 3,
 'street_name': 'Fjordudsigten',
 'zip_code_code': 6950}
      2025-06-11 08:08:52 [scrapy.core.scraper] WARNING: Dropped: Duplicate item found: {'area': 115.0,
 'available': '2024-12-15',
 'deposit_numb': 46700.0,
 'external_property_attributes': {'external_property_id': 'a86f8abe3e0ad05174acf7ddffa9c5db',
                                  'external_provider_id': 433,
                                  'property_url': 'https://www.rsbolig.dk/ledige-boliger/'},
 'fixed_expanses': 1500.0,
 'house_number': 12,
 'images': ['https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/02.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/03.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/07.JPG',
            'https://rsbolig.dk/Files/Boligdata/PlantegningerEseeBase/3D/01-51-1218.png',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/01.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/04.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/05.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '3.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '6.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '6_1.JPG',
            'https://rsbolig.dk/Files/Boligdata/PlantegningerEseeBase/01-51-1218.png',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '3_1.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/08.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/06.JPG'],
 'pets_allowed': False,
 'property_type': 'Apartment',
 'rent': 9000.0,
 'rooms': 3,
 'street_name': 'Fjordudsigten',
 'zip_code_code': 6950}
{'area': 115.0,
 'available': '2024-12-15',
 'deposit_numb': 46700.0,
 'external_property_attributes': {'external_property_id': 'a86f8abe3e0ad05174acf7ddffa9c5db',
                                  'external_provider_id': 433,
                                  'property_url': 'https://www.rsbolig.dk/ledige-boliger/'},
 'fixed_expanses': 1500.0,
 'house_number': 12,
 'images': ['https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/02.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/03.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/07.JPG',
            'https://rsbolig.dk/Files/Boligdata/PlantegningerEseeBase/3D/01-51-1218.png',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/01.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/04.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/05.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '3.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '6.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '6_1.JPG',
            'https://rsbolig.dk/Files/Boligdata/PlantegningerEseeBase/01-51-1218.png',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '3_1.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/08.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/06.JPG'],
 'pets_allowed': False,
 'property_type': 'Apartment',
 'rent': 9000.0,
 'rooms': 3,
 'street_name': 'Fjordudsigten',
 'zip_code_code': 6950}
      2025-06-11 08:08:52 [scrapy.core.scraper] WARNING: Dropped: Duplicate item found: {'area': 109.0,
 'available': '2024-12-15',
 'deposit_numb': 44600.0,
 'external_property_attributes': {'external_property_id': '415e90f71cb140a39aa5cbbc42755075',
                                  'external_provider_id': 433,
                                  'property_url': 'https://www.rsbolig.dk/ledige-boliger/'},
 'fixed_expanses': 1500.0,
 'house_number': 16,
 'images': ['https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/02.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/03.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/07.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/01.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/04.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/05.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '3.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '6.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '6_1.JPG',
            'https://rsbolig.dk/Files/Boligdata/PlantegningerEseeBase/01-51-1224.png',
            'https://rsbolig.dk/Files/Boligdata/PlantegningerEseeBase/3D/01-51-1224.png',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '3_1.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/08.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/06.JPG'],
 'pets_allowed': False,
 'property_type': 'Apartment',
 'rent': 8718.0,
 'rooms': 3,
 'street_name': 'Fjordudsigten',
 'zip_code_code': 6950}
{'area': 109.0,
 'available': '2024-12-15',
 'deposit_numb': 44600.0,
 'external_property_attributes': {'external_property_id': '415e90f71cb140a39aa5cbbc42755075',
                                  'external_provider_id': 433,
                                  'property_url': 'https://www.rsbolig.dk/ledige-boliger/'},
 'fixed_expanses': 1500.0,
 'house_number': 16,
 'images': ['https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/02.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/03.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/07.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/01.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/04.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/05.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '3.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '6.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '6_1.JPG',
            'https://rsbolig.dk/Files/Boligdata/PlantegningerEseeBase/01-51-1224.png',
            'https://rsbolig.dk/Files/Boligdata/PlantegningerEseeBase/3D/01-51-1224.png',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '3_1.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/08.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/06.JPG'],
 'pets_allowed': False,
 'property_type': 'Apartment',
 'rent': 8718.0,
 'rooms': 3,
 'street_name': 'Fjordudsigten',
 'zip_code_code': 6950}
      2025-06-11 08:08:52 [scrapy.core.scraper] WARNING: Dropped: Duplicate item found: {'area': 114.0,
 'available': '2024-12-15',
 'deposit_numb': 46700.0,
 'external_property_attributes': {'external_property_id': '1c20dc5c83eeb112a0992908a68d5020',
                                  'external_provider_id': 433,
                                  'property_url': 'https://www.rsbolig.dk/ledige-boliger/'},
 'fixed_expanses': 1500.0,
 'house_number': 16,
 'images': ['https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/02.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/03.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/07.JPG',
            'https://rsbolig.dk/Files/Boligdata/PlantegningerEseeBase/01-51-1225.png',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/01.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/04.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/05.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '3.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '6.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '6_1.JPG',
            'https://rsbolig.dk/Files/Boligdata/PlantegningerEseeBase/3D/01-51-1225.png',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '3_1.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/08.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/06.JPG'],
 'pets_allowed': False,
 'property_type': 'Apartment',
 'rent': 9000.0,
 'rooms': 3,
 'street_name': 'Fjordudsigten',
 'zip_code_code': 6950}
{'area': 114.0,
 'available': '2024-12-15',
 'deposit_numb': 46700.0,
 'external_property_attributes': {'external_property_id': '1c20dc5c83eeb112a0992908a68d5020',
                                  'external_provider_id': 433,
                                  'property_url': 'https://www.rsbolig.dk/ledige-boliger/'},
 'fixed_expanses': 1500.0,
 'house_number': 16,
 'images': ['https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/02.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/03.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/07.JPG',
            'https://rsbolig.dk/Files/Boligdata/PlantegningerEseeBase/01-51-1225.png',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/01.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/04.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/05.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '3.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '6.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '6_1.JPG',
            'https://rsbolig.dk/Files/Boligdata/PlantegningerEseeBase/3D/01-51-1225.png',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/Type '
            '3_1.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/08.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-051/Billeder/06.JPG'],
 'pets_allowed': False,
 'property_type': 'Apartment',
 'rent': 9000.0,
 'rooms': 3,
 'street_name': 'Fjordudsigten',
 'zip_code_code': 6950}

      INFO

      DEBUG

    • latest_crawl

      2025-06-11 08:08:51 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.rsbolig.dk/directproviders/searchtenancies.ashx?availonly=true> (referer: None)

    • latest_item

      {'area': 91.0,
 'available': '2025-07-01',
 'deposit_numb': 18100.0,
 'external_property_attributes': {'external_property_id': '07b590d17bc4b4e2f71bc87916f270f4',
                                  'external_provider_id': 433,
                                  'property_url': 'https://www.rsbolig.dk/ledige-boliger/'},
 'fixed_expanses': 1250.0,
 'house_number': 6,
 'images': ['https://rsbolig.dk/Files/Boligdata/Afdelinger/1-094/Billeder/1.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-094/Billeder/4.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-094/Billeder/12.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-094/Billeder/6.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-094/Billeder/11.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-094/Billeder/2.JPG',
            'https://rsbolig.dk/Files/Boligdata/PlantegningerEseeBase/01-94-1254.png',
            'https://rsbolig.dk/Files/Boligdata/PlantegningerEseeBase/3D/01-94-1254.png',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-094/Billeder/9.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-094/Billeder/10.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-094/Billeder/5.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-094/Billeder/13.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-094/Billeder/8.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-094/Billeder/7.JPG',
            'https://rsbolig.dk/Files/Boligdata/Afdelinger/1-094/Billeder/3.JPG'],
 'pets_allowed': False,
 'property_type': 'Apartment',
 'rent': 7685.0,
 'rooms': 3,
 'street_name': 'Toften',
 'zip_code_code': 6880}

    • latest_scrape

      2025-06-11 08:08:52 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.rsbolig.dk/directproviders/searchtenancies.ashx?availonly=true>

    • latest_stat

      2025-06-11 08:08:46 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)

    • scrapy_version

      2.13.2

    • Head

      2025-06-11 08:08:46 [scrapy.utils.log] INFO: Scrapy 2.13.2 started (bot: scrapybot)
2025-06-11 08:08:46 [scrapy.utils.log] INFO: Versions:
{'lxml': '5.4.0',
 'libxml2': '2.13.8',
 'cssselect': '1.3.0',
 'parsel': '1.10.0',
 'w3lib': '2.3.1',
 'Twisted': '25.5.0',
 'Python': '3.10.18 (main, Jun 10 2025, 23:52:59) [GCC 12.2.0]',
 'pyOpenSSL': '25.1.0 (OpenSSL 3.5.0 8 Apr 2025)',
 'cryptography': '45.0.4',
 'Platform': 'Linux-6.8.0-58-generic-x86_64-with-glibc2.36'}
2025-06-11 08:08:46 [scrapy.addons] INFO: Enabled addons:
[]
2025-06-11 08:08:46 [scrapy.utils.log] DEBUG: Using reactor: twisted.internet.epollreactor.EPollReactor
2025-06-11 08:08:46 [scrapy.middleware] INFO: Enabled extensions:
['scrapy.extensions.memusage.MemoryUsage',
 'scrapy.extensions.logstats.LogStats',
 'scrapy.extensions.throttle.AutoThrottle',
 'scrapy.extensions.corestats.CoreStats',
 'crawler_utils.ftp_log_uploader.FTPLogUploader']
2025-06-11 08:08:46 [scrapy.crawler] INFO: Overridden settings:
{'AUTOTHROTTLE_DEBUG': True,
 'AUTOTHROTTLE_ENABLED': True,
 'AUTOTHROTTLE_TARGET_CONCURRENCY': 8.0,
 'COMMANDS_MODULE': 'houses.commands',
 'CONCURRENT_REQUESTS': 12,
 'CONCURRENT_REQUESTS_PER_DOMAIN': 16,
 'DEFAULT_ITEM_CLASS': 'houses.items.HousesItem',
 'DOWNLOAD_DELAY': 0.3,
 'FTP_PASSWORD': 'H4KgYJ4zNyTTVokQGa_4',
 'FTP_USER': 'ftpuser',
 'HTTPCACHE_DIR': '/etc/scrapyd_cache',
 'HTTPCACHE_ENABLED': True,
 'HTTPCACHE_POLICY': 'crawler_utils.httpcache.GradualExpiresPolicy',
 'HTTPCACHE_STORAGE': 'crawler_utils.httpcache.FilesystemCacheStorage',
 'LOG_FILE': '/app/logs/rental_dk/rsbolig.dk/1f4e589e469a11f0a347f6e3f816cb0c.log',
 'NEWSPIDER_MODULE': 'houses.spiders',
 'RANDOMIZE_DOWNLOAD_DELAY': False,
 'SPIDER_MODULES': ['houses.spiders'],
 'TELNETCONSOLE_ENABLED': False,
 'TWISTED_REACTOR': 'twisted.internet.epollreactor.EPollReactor',
 'URLLENGTH_LIMIT': 5000,
 'USER_AGENT': 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, '
               'like Gecko) Chrome/29.0.1547.65 Safari/537.36'}
2025-06-11 08:08:46 [scrapy.middleware] INFO: Enabled downloader middlewares:
['scrapy.downloadermiddlewares.offsite.OffsiteMiddleware',
 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware',
 'houses.middlewares.Fake404Middleware',
 'crawler_utils.middlewares.MyUserAgentMiddleware',
 'scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware',
 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware',
 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware',
 'scrapy.downloadermiddlewares.retry.RetryMiddleware',
 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware',
 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware',
 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware',
 'rotating_proxies.middlewares.RotatingProxyMiddleware',
 'rotating_proxies.middlewares.BanDetectionMiddleware',
 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware',
 'scrapy.downloadermiddlewares.stats.DownloaderStats',
 'scrapy.downloadermiddlewares.httpcache.HttpCacheMiddleware']
2025-06-11 08:08:46 [scrapy.middleware] INFO: Enabled spider middlewares:
['scrapy.spidermiddlewares.start.StartSpiderMiddleware',
 'scrapy.spidermiddlewares.httperror.HttpErrorMiddleware',
 'scrapy.spidermiddlewares.referer.RefererMiddleware',
 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware',
 'scrapy.spidermiddlewares.depth.DepthMiddleware']
2025-06-11 08:08:46 [crawler_utils.yaml_export_pipeline] INFO: YamlExportPipeline initialized with FTP host: crawler-data.boligzonen.dk, user: ftpuser
2025-06-11 08:08:46 [crawler_utils.yaml_export_pipeline] INFO: Output directories: ['upload/rental_dk', 'upload/boligsurf_rental_dk']
2025-06-11 08:08:46 [crawler_utils.yaml_export_pipeline] INFO: Output tempdir: /tmp/rental_dk
2025-06-11 08:08:46 [crawler_utils.yaml_export_pipeline] INFO: Using PostCodeValidator for project: rental_dk
2025-06-11 08:08:46 [scrapy.middleware] INFO: Enabled item pipelines:
['houses.pipelines.DuplicatesPipeline',
 'crawler_utils.yaml_export_pipeline.YamlExportPipeline']
2025-06-11 08:08:46 [scrapy.core.engine] INFO: Spider opened
2025-06-11 08:08:46 [py.warnings] WARNING: /usr/local/lib/python3.10/site-packages/scrapy/core/spidermw.py:433: ScrapyDeprecationWarning: houses.spiders.rsbolig.RsboligDKSpider defines the deprecated start_requests() method. start_requests() has been deprecated in favor of a new method, start(), to support asynchronous code execution. start_requests() will stop being called in a future version of Scrapy. If you use Scrapy 2.13 or higher only, replace start_requests() with start(); note that start() is a coroutine (async def). If you need to maintain compatibility with lower Scrapy versions, when overriding start_requests() in a spider class, override start() as well; you can use super() to reuse the inherited start() implementation without copy-pasting. See the release notes of Scrapy 2.13 for details: https://docs.scrapy.org/en/2.13/news.html
  warn(

2025-06-11 08:08:46 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)
2025-06-11 08:08:46 [py.warnings] WARNING: /usr/local/lib/python3.10/site-packages/scrapy/downloadermiddlewares/offsite.py:83: UserWarning: allowed_domains accepts only domains, not URLs. Ignoring URL entry http://www.rsbolig.dk in allowed_domains.
  warnings.warn(message)

2025-06-11 08:08:46 [rotating_proxies.middlewares] INFO: Proxies(good: 0, dead: 0, unchecked: 7, reanimated: 0, mean backoff time: 0s)
2025-06-11 08:08:51 [scrapy.extensions.throttle] INFO: slot: world.proxymesh.com | conc: 1 | delay: 2838 ms (-2161) | latency: 5416 ms | size: 21439 bytes
2025-06-11 08:08:51 [crawler_utils.httpcache] INFO: ***Dont cache
2025-06-11 08:08:51 [rotating_proxies.expire] DEBUG: Proxy <http://woningpanda:MTHPlFvNKIPJZnI9YbSA@world.proxymesh.com:31280> is GOOD
2025-06-11 08:08:51 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.rsbolig.dk/directproviders/searchtenancies.ashx?availonly=true> (referer: None)
2025-06-11 08:08:51 [scrapy.core.scraper] DEBUG: Scraped from <200 https://www.rsbolig.dk/directproviders/searchtenancies.ashx?availonly=true>
{'area': 92.0,
 'available': '2025-08-01',
 'deposit_numb': 17000.0,
 'external_property_attributes': {'external_property_id': '9b34b429725344a2b1a8586308b340c1',
                                  'external_provider_id': 433,
                                  'property_url': 'https://www.rsbolig.dk/ledige-boliger/'},
 'fixed_expanses': 1792.0,
 'house_number': 10,
 'images': ['https://rsbolig.dk/Files/Boligdata/PlantegningerEseeBase/01-05-0188.png',
            'https://rsbolig.dk/Files/Boligdata/PlantegningerEseeBase/3D/01-05-0188.png',
            'ENCODED:https://rsbolig.dk/Files/Boligdata/Afdelinger/1-005/Billeder/Afd.%205_4.JPG',

    • Tail

      2025-06-11 08:08:52 [paramiko.transport] DEBUG: kex algos: sntrup761x25519-sha512@openssh.com, curve25519-sha256, curve25519-sha256@libssh.org, ecdh-sha2-nistp256, ecdh-sha2-nistp384, ecdh-sha2-nistp521, diffie-hellman-group-exchange-sha256, diffie-hellman-group16-sha512, diffie-hellman-group18-sha512, diffie-hellman-group14-sha256, ext-info-s, kex-strict-s-v00@openssh.com
2025-06-11 08:08:52 [paramiko.transport] DEBUG: server key: rsa-sha2-512, rsa-sha2-256, ecdsa-sha2-nistp256, ssh-ed25519
2025-06-11 08:08:52 [paramiko.transport] DEBUG: client encrypt: chacha20-poly1305@openssh.com, aes128-ctr, aes192-ctr, aes256-ctr, aes128-gcm@openssh.com, aes256-gcm@openssh.com
2025-06-11 08:08:52 [paramiko.transport] DEBUG: server encrypt: chacha20-poly1305@openssh.com, aes128-ctr, aes192-ctr, aes256-ctr, aes128-gcm@openssh.com, aes256-gcm@openssh.com
2025-06-11 08:08:52 [paramiko.transport] DEBUG: client mac: umac-64-etm@openssh.com, umac-128-etm@openssh.com, hmac-sha2-256-etm@openssh.com, hmac-sha2-512-etm@openssh.com, hmac-sha1-etm@openssh.com, umac-64@openssh.com, umac-128@openssh.com, hmac-sha2-256, hmac-sha2-512, hmac-sha1
2025-06-11 08:08:52 [paramiko.transport] DEBUG: server mac: umac-64-etm@openssh.com, umac-128-etm@openssh.com, hmac-sha2-256-etm@openssh.com, hmac-sha2-512-etm@openssh.com, hmac-sha1-etm@openssh.com, umac-64@openssh.com, umac-128@openssh.com, hmac-sha2-256, hmac-sha2-512, hmac-sha1
2025-06-11 08:08:52 [paramiko.transport] DEBUG: client compress: none, zlib@openssh.com
2025-06-11 08:08:52 [paramiko.transport] DEBUG: server compress: none, zlib@openssh.com
2025-06-11 08:08:52 [paramiko.transport] DEBUG: client lang: <none>
2025-06-11 08:08:52 [paramiko.transport] DEBUG: server lang: <none>
2025-06-11 08:08:52 [paramiko.transport] DEBUG: kex follows: False
2025-06-11 08:08:52 [paramiko.transport] DEBUG: === Key exchange agreements ===
2025-06-11 08:08:52 [paramiko.transport] DEBUG: Strict kex mode: True
2025-06-11 08:08:52 [paramiko.transport] DEBUG: Kex: curve25519-sha256@libssh.org
2025-06-11 08:08:52 [paramiko.transport] DEBUG: HostKey: ssh-ed25519
2025-06-11 08:08:52 [paramiko.transport] DEBUG: Cipher: aes128-ctr
2025-06-11 08:08:52 [paramiko.transport] DEBUG: MAC: hmac-sha2-256
2025-06-11 08:08:52 [paramiko.transport] DEBUG: Compression: none
2025-06-11 08:08:52 [paramiko.transport] DEBUG: === End of kex handshake ===
2025-06-11 08:08:52 [paramiko.transport] DEBUG: Resetting outbound seqno after NEWKEYS due to strict mode
2025-06-11 08:08:52 [paramiko.transport] DEBUG: kex engine KexCurve25519 specified hash_algo <built-in function openssl_sha256>
2025-06-11 08:08:52 [paramiko.transport] DEBUG: Switch to new keys ...
2025-06-11 08:08:52 [paramiko.transport] DEBUG: Resetting inbound seqno after NEWKEYS due to strict mode
2025-06-11 08:08:52 [paramiko.transport] DEBUG: Got EXT_INFO: {'server-sig-algs': b'ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com,rsa-sha2-512,rsa-sha2-256', 'publickey-hostbound@openssh.com': b'0', 'ping@openssh.com': b'0'}
2025-06-11 08:08:52 [paramiko.transport] DEBUG: Attempting password auth...
2025-06-11 08:08:52 [paramiko.transport] DEBUG: userauth is OK
2025-06-11 08:08:52 [paramiko.transport] INFO: Authentication (password) successful!
2025-06-11 08:08:52 [paramiko.transport] DEBUG: [chan 0] Max packet in: 32768 bytes
2025-06-11 08:08:52 [paramiko.transport] DEBUG: Received global request "hostkeys-00@openssh.com"
2025-06-11 08:08:52 [paramiko.transport] DEBUG: Rejecting "hostkeys-00@openssh.com" global request from server.
2025-06-11 08:08:52 [paramiko.transport] DEBUG: [chan 0] Max packet out: 32768 bytes
2025-06-11 08:08:52 [paramiko.transport] DEBUG: Secsh channel 0 opened.
2025-06-11 08:08:52 [paramiko.transport] DEBUG: [chan 0] Sesch channel 0 request ok
2025-06-11 08:08:52 [paramiko.transport.sftp] INFO: [chan 0] Opened sftp connection (server version 3)
2025-06-11 08:08:52 [paramiko.transport.sftp] DEBUG: [chan 0] stat(b'/upload')
2025-06-11 08:08:52 [paramiko.transport.sftp] DEBUG: [chan 0] stat(b'/upload/logs')
2025-06-11 08:08:52 [paramiko.transport.sftp] DEBUG: [chan 0] stat(b'/upload/logs/rental_dk')
2025-06-11 08:08:52 [paramiko.transport.sftp] DEBUG: [chan 0] stat(b'/upload/logs/rental_dk/rsbolig.dk')
2025-06-11 08:08:52 [paramiko.transport.sftp] DEBUG: [chan 0] open(b'/upload/logs/rental_dk/rsbolig.dk/36a21294467011f0b0aa56c86ebbb4c8.log', 'wb')
2025-06-11 08:08:52 [paramiko.transport.sftp] DEBUG: [chan 0] open(b'/upload/logs/rental_dk/rsbolig.dk/36a21294467011f0b0aa56c86ebbb4c8.log', 'wb') -> 00000000
2025-06-11 08:08:52 [paramiko.transport.sftp] DEBUG: [chan 0] close(00000000)
2025-06-11 08:08:52 [paramiko.transport.sftp] DEBUG: [chan 0] stat(b'/upload/logs/rental_dk/rsbolig.dk/36a21294467011f0b0aa56c86ebbb4c8.log')
2025-06-11 08:08:52 [paramiko.transport.sftp] DEBUG: [chan 0] open(b'/upload/logs/rental_dk/rsbolig.dk/1f4e589e469a11f0a347f6e3f816cb0c.log', 'wb')
2025-06-11 08:08:52 [paramiko.transport.sftp] DEBUG: [chan 0] open(b'/upload/logs/rental_dk/rsbolig.dk/1f4e589e469a11f0a347f6e3f816cb0c.log', 'wb') -> 00000000
2025-06-11 08:08:52 [paramiko.transport.sftp] DEBUG: [chan 0] close(00000000)
2025-06-11 08:08:52 [paramiko.transport.sftp] DEBUG: [chan 0] stat(b'/upload/logs/rental_dk/rsbolig.dk/1f4e589e469a11f0a347f6e3f816cb0c.log')
2025-06-11 08:08:52 [paramiko.transport.sftp] DEBUG: [chan 0] open(b'/upload/logs/rental_dk/rsbolig.dk/be788e34469111f0a347f6e3f816cb0c.log', 'wb')
2025-06-11 08:08:52 [paramiko.transport.sftp] DEBUG: [chan 0] open(b'/upload/logs/rental_dk/rsbolig.dk/be788e34469111f0a347f6e3f816cb0c.log', 'wb') -> 00000000
2025-06-11 08:08:52 [paramiko.transport.sftp] DEBUG: [chan 0] close(00000000)
2025-06-11 08:08:52 [paramiko.transport.sftp] DEBUG: [chan 0] stat(b'/upload/logs/rental_dk/rsbolig.dk/be788e34469111f0a347f6e3f816cb0c.log')
2025-06-11 08:08:52 [paramiko.transport.sftp] DEBUG: [chan 0] open(b'/upload/logs/rental_dk/rsbolig.dk/9878369e467811f0b0aa56c86ebbb4c8.log', 'wb')
2025-06-11 08:08:52 [paramiko.transport.sftp] DEBUG: [chan 0] open(b'/upload/logs/rental_dk/rsbolig.dk/9878369e467811f0b0aa56c86ebbb4c8.log', 'wb') -> 00000000
2025-06-11 08:08:52 [paramiko.transport.sftp] DEBUG: [chan 0] close(00000000)
2025-06-11 08:08:52 [paramiko.transport.sftp] DEBUG: [chan 0] stat(b'/upload/logs/rental_dk/rsbolig.dk/9878369e467811f0b0aa56c86ebbb4c8.log')
2025-06-11 08:08:52 [paramiko.transport.sftp] DEBUG: [chan 0] open(b'/upload/logs/rental_dk/rsbolig.dk/fa07a270468011f0b0aa56c86ebbb4c8.log', 'wb')
2025-06-11 08:08:52 [paramiko.transport.sftp] DEBUG: [chan 0] open(b'/upload/logs/rental_dk/rsbolig.dk/fa07a270468011f0b0aa56c86ebbb4c8.log', 'wb') -> 00000000
2025-06-11 08:08:52 [paramiko.transport.sftp] DEBUG: [chan 0] close(00000000)
2025-06-11 08:08:52 [paramiko.transport.sftp] DEBUG: [chan 0] stat(b'/upload/logs/rental_dk/rsbolig.dk/fa07a270468011f0b0aa56c86ebbb4c8.log')
2025-06-11 08:08:52 [paramiko.transport.sftp] DEBUG: [chan 0] open(b'/upload/logs/rental_dk/rsbolig.dk/5bed40fa468911f0b0aa56c86ebbb4c8.log', 'wb')
2025-06-11 08:08:52 [paramiko.transport.sftp] DEBUG: [chan 0] open(b'/upload/logs/rental_dk/rsbolig.dk/5bed40fa468911f0b0aa56c86ebbb4c8.log', 'wb') -> 00000000
2025-06-11 08:08:52 [paramiko.transport.sftp] DEBUG: [chan 0] close(00000000)
2025-06-11 08:08:52 [paramiko.transport.sftp] DEBUG: [chan 0] stat(b'/upload/logs/rental_dk/rsbolig.dk/5bed40fa468911f0b0aa56c86ebbb4c8.log')
2025-06-11 08:08:52 [paramiko.transport.sftp] DEBUG: [chan 0] listdir(b'/upload/logs/rental_dk/rsbolig.dk')
2025-06-11 08:08:52 [paramiko.transport.sftp] INFO: [chan 0] sftp session closed.
2025-06-11 08:08:52 [paramiko.transport] DEBUG: [chan 0] EOF sent (0)
2025-06-11 08:08:52 [crawler_utils.yaml_export_pipeline] INFO: Uploading /tmp/rental_dk/rsbolig.dk_1749629331.yaml to /upload/rental_dk
2025-06-11 08:08:52 [crawler_utils.yaml_export_pipeline] INFO: FTP host: crawler-data.boligzonen.dk, user: ftpuser
2025-06-11 08:08:52 [paramiko.transport] DEBUG: EOF in transport thread
2025-06-11 08:08:52 [paramiko.transport] DEBUG: starting thread (client mode): 0x9c27e80
2025-06-11 08:08:52 [paramiko.transport] DEBUG: Local version/idstring: SSH-2.0-paramiko_3.5.1
2025-06-11 08:08:52 [paramiko.transport] DEBUG: Remote version/idstring: SSH-2.0-OpenSSH_9.6p1 Ubuntu-3ubuntu13.11
2025-06-11 08:08:52 [paramiko.transport] INFO: Connected (version 2.0, client OpenSSH_9.6p1)
2025-06-11 08:08:52 [paramiko.transport] DEBUG: === Key exchange possibilities ===
2025-06-11 08:08:52 [paramiko.transport] DEBUG: kex algos: sntrup761x25519-sha512@openssh.com, curve25519-sha256, curve25519-sha256@libssh.org, ecdh-sha2-nistp256, ecdh-sha2-nistp384, ecdh-sha2-nistp521, diffie-hellman-group-exchange-sha256, diffie-hellman-group16-sha512, diffie-hellman-group18-sha512, diffie-hellman-group14-sha256, ext-info-s, kex-strict-s-v00@openssh.com
2025-06-11 08:08:52 [paramiko.transport] DEBUG: server key: rsa-sha2-512, rsa-sha2-256, ecdsa-sha2-nistp256, ssh-ed25519
2025-06-11 08:08:52 [paramiko.transport] DEBUG: client encrypt: chacha20-poly1305@openssh.com, aes128-ctr, aes192-ctr, aes256-ctr, aes128-gcm@openssh.com, aes256-gcm@openssh.com
2025-06-11 08:08:52 [paramiko.transport] DEBUG: server encrypt: chacha20-poly1305@openssh.com, aes128-ctr, aes192-ctr, aes256-ctr, aes128-gcm@openssh.com, aes256-gcm@openssh.com
2025-06-11 08:08:52 [paramiko.transport] DEBUG: client mac: umac-64-etm@openssh.com, umac-128-etm@openssh.com, hmac-sha2-256-etm@openssh.com, hmac-sha2-512-etm@openssh.com, hmac-sha1-etm@openssh.com, umac-64@openssh.com, umac-128@openssh.com, hmac-sha2-256, hmac-sha2-512, hmac-sha1
2025-06-11 08:08:52 [paramiko.transport] DEBUG: server mac: umac-64-etm@openssh.com, umac-128-etm@openssh.com, hmac-sha2-256-etm@openssh.com, hmac-sha2-512-etm@openssh.com, hmac-sha1-etm@openssh.com, umac-64@openssh.com, umac-128@openssh.com, hmac-sha2-256, hmac-sha2-512, hmac-sha1
2025-06-11 08:08:52 [paramiko.transport] DEBUG: client compress: none, zlib@openssh.com
2025-06-11 08:08:52 [paramiko.transport] DEBUG: server compress: none, zlib@openssh.com
2025-06-11 08:08:52 [paramiko.transport] DEBUG: client lang: <none>
2025-06-11 08:08:52 [paramiko.transport] DEBUG: server lang: <none>
2025-06-11 08:08:52 [paramiko.transport] DEBUG: kex follows: False
2025-06-11 08:08:52 [paramiko.transport] DEBUG: === Key exchange agreements ===
2025-06-11 08:08:52 [paramiko.transport] DEBUG: Strict kex mode: True
2025-06-11 08:08:52 [paramiko.transport] DEBUG: Kex: curve25519-sha256@libssh.org
2025-06-11 08:08:52 [paramiko.transport] DEBUG: HostKey: ssh-ed25519
2025-06-11 08:08:52 [paramiko.transport] DEBUG: Cipher: aes128-ctr
2025-06-11 08:08:52 [paramiko.transport] DEBUG: MAC: hmac-sha2-256
2025-06-11 08:08:52 [paramiko.transport] DEBUG: Compression: none
2025-06-11 08:08:52 [paramiko.transport] DEBUG: === End of kex handshake ===
2025-06-11 08:08:52 [paramiko.transport] DEBUG: Resetting outbound seqno after NEWKEYS due to strict mode
2025-06-11 08:08:52 [paramiko.transport] DEBUG: kex engine KexCurve25519 specified hash_algo <built-in function openssl_sha256>
2025-06-11 08:08:52 [paramiko.transport] DEBUG: Switch to new keys ...
2025-06-11 08:08:52 [paramiko.transport] DEBUG: Resetting inbound seqno after NEWKEYS due to strict mode
2025-06-11 08:08:52 [paramiko.transport] DEBUG: Got EXT_INFO: {'server-sig-algs': b'ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com,rsa-sha2-512,rsa-sha2-256', 'publickey-hostbound@openssh.com': b'0', 'ping@openssh.com': b'0'}
2025-06-11 08:08:52 [paramiko.transport] DEBUG: Attempting password auth...
2025-06-11 08:08:53 [paramiko.transport] DEBUG: userauth is OK
2025-06-11 08:08:53 [paramiko.transport] INFO: Authentication (password) successful!
2025-06-11 08:08:53 [paramiko.transport] DEBUG: [chan 0] Max packet in: 32768 bytes
2025-06-11 08:08:53 [paramiko.transport] DEBUG: Received global request "hostkeys-00@openssh.com"
2025-06-11 08:08:53 [paramiko.transport] DEBUG: Rejecting "hostkeys-00@openssh.com" global request from server.
2025-06-11 08:08:53 [paramiko.transport] DEBUG: [chan 0] Max packet out: 32768 bytes
2025-06-11 08:08:53 [paramiko.transport] DEBUG: Secsh channel 0 opened.
2025-06-11 08:08:53 [paramiko.transport] DEBUG: [chan 0] Sesch channel 0 request ok
2025-06-11 08:08:53 [paramiko.transport.sftp] INFO: [chan 0] Opened sftp connection (server version 3)
2025-06-11 08:08:53 [paramiko.transport.sftp] DEBUG: [chan 0] open(b'/upload/rental_dk/rsbolig.dk_1749629331.yaml', 'wb')
2025-06-11 08:08:53 [paramiko.transport.sftp] DEBUG: [chan 0] open(b'/upload/rental_dk/rsbolig.dk_1749629331.yaml', 'wb') -> 00000000
2025-06-11 08:08:53 [paramiko.transport.sftp] DEBUG: [chan 0] close(00000000)
2025-06-11 08:08:53 [paramiko.transport.sftp] DEBUG: [chan 0] stat(b'/upload/rental_dk/rsbolig.dk_1749629331.yaml')
2025-06-11 08:08:53 [paramiko.transport.sftp] INFO: [chan 0] sftp session closed.
2025-06-11 08:08:53 [paramiko.transport] DEBUG: [chan 0] EOF sent (0)
2025-06-11 08:08:53 [crawler_utils.yaml_export_pipeline] INFO: Successfully uploaded /tmp/rental_dk/rsbolig.dk_1749629331.yaml to upload/rental_dk
2025-06-11 08:08:53 [crawler_utils.yaml_export_pipeline] INFO: Uploading /tmp/rental_dk/rsbolig.dk_1749629331.yaml to /upload/boligsurf_rental_dk
2025-06-11 08:08:53 [crawler_utils.yaml_export_pipeline] INFO: FTP host: crawler-data.boligzonen.dk, user: ftpuser
2025-06-11 08:08:53 [paramiko.transport] DEBUG: starting thread (client mode): 0x8e0a4a0
2025-06-11 08:08:53 [paramiko.transport] DEBUG: Local version/idstring: SSH-2.0-paramiko_3.5.1
2025-06-11 08:08:53 [paramiko.transport] DEBUG: Remote version/idstring: SSH-2.0-OpenSSH_9.6p1 Ubuntu-3ubuntu13.11
2025-06-11 08:08:53 [paramiko.transport] INFO: Connected (version 2.0, client OpenSSH_9.6p1)
2025-06-11 08:08:53 [paramiko.transport] DEBUG: === Key exchange possibilities ===
2025-06-11 08:08:53 [paramiko.transport] DEBUG: kex algos: sntrup761x25519-sha512@openssh.com, curve25519-sha256, curve25519-sha256@libssh.org, ecdh-sha2-nistp256, ecdh-sha2-nistp384, ecdh-sha2-nistp521, diffie-hellman-group-exchange-sha256, diffie-hellman-group16-sha512, diffie-hellman-group18-sha512, diffie-hellman-group14-sha256, ext-info-s, kex-strict-s-v00@openssh.com
2025-06-11 08:08:53 [paramiko.transport] DEBUG: server key: rsa-sha2-512, rsa-sha2-256, ecdsa-sha2-nistp256, ssh-ed25519
2025-06-11 08:08:53 [paramiko.transport] DEBUG: client encrypt: chacha20-poly1305@openssh.com, aes128-ctr, aes192-ctr, aes256-ctr, aes128-gcm@openssh.com, aes256-gcm@openssh.com
2025-06-11 08:08:53 [paramiko.transport] DEBUG: server encrypt: chacha20-poly1305@openssh.com, aes128-ctr, aes192-ctr, aes256-ctr, aes128-gcm@openssh.com, aes256-gcm@openssh.com
2025-06-11 08:08:53 [paramiko.transport] DEBUG: client mac: umac-64-etm@openssh.com, umac-128-etm@openssh.com, hmac-sha2-256-etm@openssh.com, hmac-sha2-512-etm@openssh.com, hmac-sha1-etm@openssh.com, umac-64@openssh.com, umac-128@openssh.com, hmac-sha2-256, hmac-sha2-512, hmac-sha1
2025-06-11 08:08:53 [paramiko.transport] DEBUG: server mac: umac-64-etm@openssh.com, umac-128-etm@openssh.com, hmac-sha2-256-etm@openssh.com, hmac-sha2-512-etm@openssh.com, hmac-sha1-etm@openssh.com, umac-64@openssh.com, umac-128@openssh.com, hmac-sha2-256, hmac-sha2-512, hmac-sha1
2025-06-11 08:08:53 [paramiko.transport] DEBUG: client compress: none, zlib@openssh.com
2025-06-11 08:08:53 [paramiko.transport] DEBUG: server compress: none, zlib@openssh.com
2025-06-11 08:08:53 [paramiko.transport] DEBUG: client lang: <none>
2025-06-11 08:08:53 [paramiko.transport] DEBUG: server lang: <none>
2025-06-11 08:08:53 [paramiko.transport] DEBUG: kex follows: False
2025-06-11 08:08:53 [paramiko.transport] DEBUG: === Key exchange agreements ===
2025-06-11 08:08:53 [paramiko.transport] DEBUG: Strict kex mode: True
2025-06-11 08:08:53 [paramiko.transport] DEBUG: Kex: curve25519-sha256@libssh.org
2025-06-11 08:08:53 [paramiko.transport] DEBUG: HostKey: ssh-ed25519
2025-06-11 08:08:53 [paramiko.transport] DEBUG: Cipher: aes128-ctr
2025-06-11 08:08:53 [paramiko.transport] DEBUG: MAC: hmac-sha2-256
2025-06-11 08:08:53 [paramiko.transport] DEBUG: Compression: none
2025-06-11 08:08:53 [paramiko.transport] DEBUG: === End of kex handshake ===
2025-06-11 08:08:53 [paramiko.transport] DEBUG: Resetting outbound seqno after NEWKEYS due to strict mode
2025-06-11 08:08:53 [paramiko.transport] DEBUG: kex engine KexCurve25519 specified hash_algo <built-in function openssl_sha256>
2025-06-11 08:08:53 [paramiko.transport] DEBUG: Switch to new keys ...
2025-06-11 08:08:53 [paramiko.transport] DEBUG: Resetting inbound seqno after NEWKEYS due to strict mode
2025-06-11 08:08:53 [paramiko.transport] DEBUG: Attempting password auth...
2025-06-11 08:08:53 [paramiko.transport] DEBUG: Got EXT_INFO: {'server-sig-algs': b'ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ssh-ed25519@openssh.com,sk-ecdsa-sha2-nistp256@openssh.com,rsa-sha2-512,rsa-sha2-256', 'publickey-hostbound@openssh.com': b'0', 'ping@openssh.com': b'0'}
2025-06-11 08:08:53 [paramiko.transport] DEBUG: EOF in transport thread
2025-06-11 08:08:53 [paramiko.transport] DEBUG: userauth is OK
2025-06-11 08:08:53 [paramiko.transport] INFO: Authentication (password) successful!
2025-06-11 08:08:53 [paramiko.transport] DEBUG: [chan 0] Max packet in: 32768 bytes
2025-06-11 08:08:54 [paramiko.transport] DEBUG: Received global request "hostkeys-00@openssh.com"
2025-06-11 08:08:54 [paramiko.transport] DEBUG: Rejecting "hostkeys-00@openssh.com" global request from server.
2025-06-11 08:08:54 [paramiko.transport] DEBUG: [chan 0] Max packet out: 32768 bytes
2025-06-11 08:08:54 [paramiko.transport] DEBUG: Secsh channel 0 opened.
2025-06-11 08:08:54 [paramiko.transport] DEBUG: [chan 0] Sesch channel 0 request ok
2025-06-11 08:08:54 [paramiko.transport.sftp] INFO: [chan 0] Opened sftp connection (server version 3)
2025-06-11 08:08:54 [paramiko.transport.sftp] DEBUG: [chan 0] open(b'/upload/boligsurf_rental_dk/rsbolig.dk_1749629331.yaml', 'wb')
2025-06-11 08:08:54 [paramiko.transport.sftp] DEBUG: [chan 0] open(b'/upload/boligsurf_rental_dk/rsbolig.dk_1749629331.yaml', 'wb') -> 00000000
2025-06-11 08:08:54 [paramiko.transport.sftp] DEBUG: [chan 0] close(00000000)
2025-06-11 08:08:54 [paramiko.transport.sftp] DEBUG: [chan 0] stat(b'/upload/boligsurf_rental_dk/rsbolig.dk_1749629331.yaml')
2025-06-11 08:08:54 [paramiko.transport.sftp] INFO: [chan 0] sftp session closed.
2025-06-11 08:08:54 [paramiko.transport] DEBUG: [chan 0] EOF sent (0)
2025-06-11 08:08:54 [crawler_utils.yaml_export_pipeline] INFO: Successfully uploaded /tmp/rental_dk/rsbolig.dk_1749629331.yaml to upload/boligsurf_rental_dk
2025-06-11 08:08:54 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'downloader/request_bytes': 293,
 'downloader/request_count': 1,
 'downloader/request_method_count/GET': 1,
 'downloader/response_bytes': 22128,
 'downloader/response_count': 1,
 'downloader/response_status_count/200': 1,
 'elapsed_time_seconds': 5.93261,
 'finish_reason': 'finished',
 'finish_time': datetime.datetime(2025, 6, 11, 8, 8, 52, 197386, tzinfo=datetime.timezone.utc),
 'httpcache/firsthand': 1,
 'httpcache/miss': 1,
 'httpcache/uncacheable': 1,
 'httpcompression/response_bytes': 252455,
 'httpcompression/response_count': 1,
 'item_dropped_count': 9,
 'item_dropped_reasons_count/DropItem': 9,
 'item_scraped_count': 30,
 'items_per_minute': None,
 'log_count/DEBUG': 183,
 'log_count/INFO': 33,
 'log_count/WARNING': 11,
 'memusage/max': 109711360,
 'memusage/startup': 109711360,
 'proxies/good': 1,
 'proxies/mean_backoff': 0.0,
 'proxies/reanimated': 0,
 'proxies/unchecked': 7,
 'response_received_count': 1,
 'responses_per_minute': None,
 'scheduler/dequeued': 1,
 'scheduler/dequeued/memory': 1,
 'scheduler/enqueued': 1,
 'scheduler/enqueued/memory': 1,
 'start_time': datetime.datetime(2025, 6, 11, 8, 8, 46, 264776, tzinfo=datetime.timezone.utc)}
2025-06-11 08:08:54 [scrapy.core.engine] INFO: Spider closed (finished)
2025-06-11 08:08:54 [paramiko.transport] DEBUG: EOF in transport thread

    • Log

      /2/log/utf8/rental_dk/rsbolig.dk/1f4e589e469a11f0a347f6e3f816cb0c/?job_finished=True&ui=mobile

    • Source

      http://c2.boligzonen.dk:80/logs/rental_dk/rsbolig.dk/1f4e589e469a11f0a347f6e3f816cb0c.log

  • sourcelog
    last update time2025-06-11 08:08:54
    last update timestamp1749629334
    downloader / request bytes293
    downloader / request count1
    downloader / request method count / GET1
    downloader / response bytes22128
    downloader / response count1
    downloader / response status count / 2001
    elapsed time seconds5.93261
    finish reasonfinished
    finish timedatetime.datetime(2025, 6, 11, 8, 8, 52, 197386, tzinfo=datetime.timezone.utc)
    httpcache / firsthand1
    httpcache / miss1
    httpcache / uncacheable1
    httpcompression / response bytes252455
    httpcompression / response count1
    item dropped count9
    item dropped reasons count / DropItem9
    item scraped count30
    items per minuteNone
    log count / DEBUG183
    log count / INFO33
    log count / WARNING11
    memusage / max109711360
    memusage / startup109711360
    proxies / good1
    proxies / mean backoff0.0
    proxies / reanimated0
    proxies / unchecked7
    response received count1
    responses per minuteNone
    scheduler / dequeued1
    scheduler / dequeued / memory1
    scheduler / enqueued1
    scheduler / enqueued / memory1
    start timedatetime.datetime(2025, 6, 11, 8, 8, 46, 264776, tzinfo=datetime.timezone.utc)