Search papers, labs, and topics across Lattice.
This paper introduces a novel approach to encoding building semantics for AI model training by leveraging large language model (LLM) embeddings to capture nuanced relationships between building object subtypes. They trained GraphSAGE models to classify 42 building object subtypes using LLM embeddings (GPT, LLaMA) and compared performance against one-hot encoding. Results showed that LLM embeddings, particularly compacted LLaMA embeddings, significantly improved classification accuracy, achieving a weighted average F1-score of 0.8766 compared to 0.8475 for one-hot encoding, demonstrating the efficacy of LLM-based encodings for enhancing AI's understanding of complex building semantics.
Ditch one-hot encoding: LLM embeddings give your AI a deeper understanding of building semantics, boosting F1-scores by nearly 3% when classifying building object subtypes.
Accurate representation of building semantics, encompassing both generic object types and specific subtypes, is essential for effective AI model training in the architecture, engineering, construction, and operation (AECO) industry. Conventional encoding methods (e.g., one-hot) often fail to convey the nuanced relationships among closely related subtypes, limiting AI's semantic comprehension. To address this limitation, this study proposes a novel training approach that employs large language model (LLM) embeddings (e.g., OpenAI GPT and Meta LLaMA) as encodings to preserve finer distinctions in building semantics. We evaluated the proposed method by training GraphSAGE models to classify 42 building object subtypes across five high-rise residential building information models (BIMs). Various embedding dimensions were tested, including original high-dimensional LLM embeddings (1,536, 3,072, or 4,096) and 1,024-dimensional compacted embeddings generated via the Matryoshka representation model. Experimental results demonstrated that LLM encodings outperformed the conventional one-hot baseline, with the llama-3 (compacted) embedding achieving a weighted average F1-score of 0.8766, compared to 0.8475 for one-hot encoding. The results underscore the promise of leveraging LLM-based encodings to enhance AI's ability to interpret complex, domain-specific building semantics. As the capabilities of LLMs and dimensionality reduction techniques continue to evolve, this approach holds considerable potential for broad application in semantic elaboration tasks throughout the AECO industry.
