Web Scale Graph Mining for Cyber Threat Intelligence

Overview of the TITAN architecture: an industry-scale graph mining framework that generates real-time TI by propagating reputation scores across millions of interconnected entities, incidents, and organizations. Built on a time evolving 5-partite graph, the system operates through four key components: (1) dynamic graph construction and updates, (2) integration of known TI and security domain knowledge to bootstrap reputation scores for unknown entities; (3) reputation propagation to iteratively update risk scores; and (4) model calibration to probabilistically align scores for use by security analysts

Abstract

Defending against today’s increasingly sophisticated and large-scale cyberattacks demands accurate, real-time threat intelligence. Traditional approaches struggle to scale, integrate diverse telemetry, and adapt to a constantly evolving security landscape. We introduce Threat Intelligence Tracking via Adaptive Networks (TITAN), an industry-scale graph mining framework that generates cyber threat intelligence at unprecedented speed and scale. TITAN introduces a suite of innovations specifically designed to address the complexities of the modern security landscape, including: (1) a dynamic threat intelligence graph that maps the intricate relationships between millions of entities, incidents, and organizations; (2) real-time update mechanisms that automatically decay and prune outdated intel; (3) integration of security domain knowledge to bootstrap initial reputation scores; and (4) reputation propagation algorithms that uncover hidden threat actor infrastructure. Integrated into Microsoft Unified Security Operations Platform (USOP), which is deployed across hundreds of thousands of organizations worldwide, TITAN’s threat intelligence powers key detection and disruption capabilities. With an impressive average macro-F1 score of 0.89 and a precision-recall AUC of 0.94, TITAN identifies millions of high-risk entities each week, enabling a 6x increase in non-file threat intelligence. Since its deployment, TITAN has increased the product’s incident disruption rate by a remarkable 21%, while reducing the time to disrupt by a factor of 1.9x, and maintaining 99% precision, as confirmed by customer feedback and thorough manual evaluation by security experts–ultimately saving customers from costly security breaches.

Citation

Web Scale Graph Mining for Cyber Threat Intelligence
Scott Freitas, Amir Gharib
arXiv (arXiv). 2024.
Project PDF BibTeX Deployed in Microsoft Unified Security Operations Platform Presented at Microsoft Ignite 2024