Search papers, labs, and topics across Lattice.
This paper details an end-to-end pipeline for extracting structured data from heterogeneous real estate questionnaire documents, which are often challenging due to their varied formats. By classifying documents into categories and employing the DeepSeek R1 Large Language Model to extract 35 predefined property attributes, the authors successfully processed 2,766 unique property records from a dataset of 3,965 documents. The methodology demonstrated high data quality with a Jaccard consistency score of 0.82 and effective market segmentation through K-Means clustering, highlighting the feasibility of automated data extraction at scale.
Extracting structured data from complex real estate documents is now feasible at scale, achieving a remarkable consistency score of 0.82.
Real estate property listings expose structured metadata through the API. Still, the richest property-level information (i.e., legal status, structural condition, utility supplies, heating systems) sits in attached questionnaire documents that no automated system currently processes at scale. These documents are heterogeneous. Some are digitally generated with selectable text, others are scanned physical forms. There are even more complex layouts that contain checkbox annotations that defeat conventional text extraction. In this paper, we present an end-to-end pipeline for acquiring, classifying, and extracting structured data from selectable text documents. The pipeline was applied to 3965 questionnaire documents collected from a live property platform via reverse-engineered REST APIs. First, we classified each document into one of three structural categories (text_only, scanned, and special_char), then extracted 35 predefined property attributes from eligible documents using DeepSeek R1 as the Large Language Model, prompted to return a structured JSON object. All 2781 submitted documents were processed successfully, producing a final dataset of 2766 unique property records. Downstream validation confirmed the data quality. Cosine similarity matching achieves a Jaccard consistency score of 0.82, and K-Means clustering produces interpretable market segments with a silhouette score of 0.2088. Results show that the proposed extraction from each property document is both feasible and reliable at this scale.
