Automated VDAB Scraper - Advanced Tool for Extracting Belgian Job Listings with Email Addresses

Project Overview

I created an advanced system that automatically retrieves job listings daily from the Belgian VDAB portal (Vlaamse Dienst voor Arbeidsbemiddeling en Beroepsopleiding). My tool analyzes and filters job postings for the presence of email addresses, providing key value to the recruitment processes of my client, Mesoworks, operating in the Belgian market.

Initially, data was stored in Google Sheets, but as part of the optimization, I performed a complete migration to a PostgreSQL database. Currently, the system is fully integrated with the JobPortal platform, enabling efficient management of acquired job listings from Belgium.

Key Features and Technologies

VDAB Scraping Automation

• Daily Belgian listing retrieval - I used Python with Selenium and BeautifulSoup libraries to create a reliable VDAB scraper handling both Flemish and French versions of the portal
• Advanced offer filtering - I implemented algorithms for detecting email addresses in Belgian job listing content, taking into account specific local formats
• Multilingual support - I adapted the system to handle listings in Flemish, French, and English, common in the Belgian job market

Data Processing and Management

• Migration from Google Sheets to PostgreSQL - I increased system efficiency and scalability by implementing an optimized database
• Advanced listing categorization - I created a system that classifies listings by Belgian regions (Flanders/Wallonia/Brussels) and economic sectors
• Duplicate prevention mechanisms - I implemented algorithms detecting and merging duplicate job listings from different portal sections

Infrastructure and JobPortal Integration

• Microservice architecture - I designed a scalable Docker-based system enabling independent component scaling
• API for JobPortal integration - I created a FastAPI interface allowing seamless data exchange between scrapers and the client's system
• Automatic updates and monitoring - I implemented a system of cyclic data updates with error notifications and performance monitoring

Belgian Context and Challenges

The VDAB portal is the main source of job listings in the Flemish region of Belgium, which presented specific challenges:

Challenge: Multilingualism and Belgian Regionalization

Belgium has three official languages and a strong regional division (Flanders, Wallonia, Brussels), which complicates data scraping.

My solution: I created a system that recognizes the language of the listing and its regional affiliation, enabling precise categorization and analysis of data specific to particular regions of Belgium.

Challenge: Complex VDAB Portal Structure

The VDAB portal has an advanced, dynamic structure with multiple filtering options and search parameters.

My solution: I implemented an intelligent page navigator that simulates user interactions with VDAB filtering systems and adapts to changes in the page structure.

Challenge: Email Address Identification in Local Context

Belgian email addresses often contain specific national and regional domains.

My solution: I adapted email detection algorithms to accommodate Belgian specifics, including .be domains, domains specific to regions, and Flemish institutions.

Measurable Project Results

• Complete database of Belgian job listings - the system retrieves and analyzes over 3000 listings daily from the VDAB portal
• High filtering efficiency - I identified and acquired over 28% more listings containing email addresses than with a manual process
• Time and resource savings - I reduced the time needed for listing acquisition by 94%, from 22 hours weekly to full automation
• Increased data accuracy - I achieved 97% accuracy in identifying and extracting email addresses from Belgian job listings

Belgian Job Market Specifics

The system has been specially adapted to the characteristics of the Belgian job market:

• Handling regional differences - consideration of job market specifics in Flanders, Wallonia, and the Brussels Capital Region
• Linguistic diversity - automatic recognition and processing of listings in Dutch (Flemish), French, and English
• Compliance with Belgian standards - adaptation of the system to local address formats, phone numbers, and contact conventions

Conclusions and Perspectives

My advanced VDAB scraper with email address detection capability has significantly streamlined Mesoworks' HR processes in the Belgian market. The system I created enables automatic acquisition of valuable recruitment contacts, which translates into concrete business benefits.

The use of modern technologies such as Python, Selenium, PostgreSQL, FastAPI, and Docker allowed me to create an efficient, scalable, and reliable solution, fully adapted to the specific requirements of the Belgian job market.

The system is regularly updated to adapt to changes in the VDAB portal structure and evolving recruitment needs in the Belgian market.