Search papers, labs, and topics across Lattice.
This study evaluates the effectiveness of Microsoft GraphRAG, a knowledge-graph retrieval system, compared to a standard vector-similarity retrieval system (Naive RAG) in generating cyber threat detection plans from security reports. The findings reveal that GraphRAG consistently produces more resilient detection strategies that target higher-level indicators of compromise, maintaining a 100% detection rate even after the rotation of easily changed clues, while Naive RAG's effectiveness plummets to 29%. This research underscores the importance of leveraging knowledge graphs in threat intelligence to enhance the durability of detection mechanisms against evolving cyber threats.
GraphRAG's detection plans remain effective even after adversaries change their tactics, while traditional methods falter dramatically.
When a security researcher publishes a report on a cyberattack, detection engineers are supposed to turn it into working detection rules. In practice, most automated attempts at this only extract the simplest clues from the report --- bad IP addresses, domain names, and file hashes --- and turn them into block lists. This is a weak strategy, because attackers can change these simple clues within hours or days, so the resulting detections stop working almost as soon as they are deployed. Security teams describe this idea with the Pyramid of Pain. This project asks whether feeding a report into a knowledge-graph retrieval system, Microsoft GraphRAG, rather than a standard vector-similarity retrieval system (Naive RAG), produces detection plans that rely more on these durable, top-of-pyramid clues. Both systems are given the same report, the same generation instructions, and the same language model to write the final plan; only the retrieval step differs. In a detailed case study of one APT28 report, the GraphRAG plan kept firing at 100\% of its detections after every IP address, domain, and file hash in the report was rotated, while the Naive RAG plan kept firing at only 29\%. Repeating the comparison across nine real CTI reports from four vendors confirms the same pattern: GraphRAG plans consistently reach higher, harder-to-evade levels of the pyramid, even when the two systems end up close on total score. The results support treating knowledge-graph-aware retrieval as the architecturally correct foundation for automatically generating SOC-deployable hunting plans, while showing that the wording of the generation prompt matters almost as much as the retrieval back-end itself.
