Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper presents an adaptive browserless price extraction system that enhances robustness against structural variations in websites, addressing the limitations of existing methods. By integrating a Bayesian approach for dynamic rule weight updates and a genetic algorithm for optimizing global parameters, the system achieves a significant precision improvement from 77.2% to 87.3% while reducing processing time by approximately 14%. The findings highlight the competitive edge of this hybrid approach over traditional browser-based and LLM methods, making it a viable solution for efficient price extraction in e-commerce.
Achieving an 87.3% precision in price extraction while cutting processing time by 14% positions this adaptive browserless system as a game-changer for e-commerce analytics.
Price extraction from websites is a key task for market monitoring, price comparison, and business analytics in e-commerce. Existing approaches can be broadly divided into four groups, and understanding their trade-offs in accuracy and scalability is essential for selecting suitable extraction strategies. Classical methods rely on manually written wrappers and rule induction from labeled pages, offering high accuracy but adapting poorly to structural changes and requiring considerable maintenance effort. Browser-based methods, using tools such as Selenium and Puppeteer, handle dynamic JavaScript content but consume large computational resources and scale poorly. Browserless approaches retrieve HTML directly via HTTP requests, offering significant gains in speed and cost, but rely on rules calibrated for specific sites. Methods based on machine learning and large language models offer adaptability but require training data and substantial computation. Our main contribution is an adaptive browserless price extraction system that improves robustness to structural differences between websites. We implemented a baseline architecture combining HTML page fragmentation with syntactic, semantic, and frequency rules, and extended it in two ways: a Bayesian approach that dynamically updates rule weights, and a genetic algorithm that optimizes the system's global parameters. This hybrid scheme increased precision from 77.2% to 87.3% and reduced average per-page processing time by approximately 14% relative to the baseline, confirming it as a competitive alternative to manually tuned browserless solutions and to more resource-intensive browser- or LLM-based methods, offering high extraction accuracy at low computational cost.