Automated LeForem Scraper - Advanced Tool for Extracting Job Listings with Email Addresses

Project Overview

I created an advanced system that automatically retrieves job listings from the Belgian LeForem portal daily. My tool analyzes and filters postings for the presence of email addresses, providing key value to the recruitment processes of my client, Mesoworks.

Initially, data was stored in Google Sheets, but as part of my optimization, I performed a complete migration to a PostgreSQL database. Currently, the system is fully integrated with the JobPortal platform, enabling efficient management of acquired job listings.

Key Features and Technologies

LeForem Scraping Automation

• Daily listing retrieval - I used Python with Selenium and BeautifulSoup libraries to create a reliable LeForem scraper
• Advanced offer filtering - I implemented precise algorithms for detecting email addresses in job posting content
• Block avoidance mechanisms - I applied proxy rotation and session management to increase scraper reliability

Data Integration and Storage

• Migration from Google Sheets to PostgreSQL - I increased system efficiency and scalability
• Full synchronization with JobPortal - I integrated my solution with the client's existing platform
• Data management API - I created an API using FastAPI for easy access to collected data

Infrastructure and Performance

• Microservice-based architecture - I ensured independent scaling of individual components
• Asynchronous task handling - I used Celery with Redis for efficient task queue management
• Containerization with Docker - I enabled easy deployment and environment management

Measurable Project Results

• HR process automation - elimination of over 25 hours of manual work weekly
• Increased recruitment efficiency - 250% increase in candidates acquired from Belgian listings with direct email contact
• Solution scalability - the system currently handles over 5,000 job listings daily from the Belgian market
• Integration with client ecosystem - seamless cooperation with the existing JobPortal platform

Technical Challenges and Solutions

Challenge: Dynamic LeForem Page Structure

The Belgian LeForem portal uses dynamically generated JavaScript content, which made standard scraping difficult.

My solution: I implemented Selenium in headless mode with advanced element wait handling. I developed an algorithm that detects changes in page structure and automatically adjusts selectors.

Challenge: Detecting Email Addresses in Various Formats

Email addresses were often hidden or presented in different formats to prevent automated collection.

My solution: I created an advanced pattern recognition system using regular expressions and NLP techniques to detect even masked email addresses.

Conclusions

My advanced LeForem scraper with email address detection capability has significantly streamlined Mesoworks' HR processes in the Belgian market. Through automation of job listing scraping, filtering for contact information, and integration with JobPortal, the client can acquire job candidates much more efficiently.

The use of modern technologies such as Python, Selenium, PostgreSQL, FastAPI, and Docker allowed me to create an efficient, scalable, and reliable solution that meets all of the client's business requirements operating in the Belgian market.