Research on Network Security Situational Awareness Based on Crawler Algorithm

TLDR

Network security situational awareness provides a critical foundation for security solutions by depicting a target system’s security state through assessment of actual or potential cyber‑attacks. The study aims to perceive and measure the overall network security situation to support global information flow security and stability, and to develop a text‑based event analysis tool. Using the Scrapy web crawler, data were collected from Zhiming security event sites and integrated with the China Computer Network Intrusion Prevention Center vulnerability database to build a comprehensive event database, after which a text‑processing tool was designed to clean and process security event data. The crawler‑based database contains 43,848 records, representing a 12.79%–29.33% increase over traditional algorithms, while reducing reading time by 63.5%–87.2%.

Abstract

Network security situation awareness is a critical basis for security solutions because it displays the target system’s security state by assessing actual or possible cyber-attacks in the target system. Aiming at the security and stability of global information flow, this paper studies the perception and measurement of the overall situation of network security. Through the Scrappy web crawler framework, data were collected from several Zhiming network security event websites, and based on the vulnerability database of China Computer Network Intrusion Prevention Center, the network security event database was designed and established, which enriched the data of situational awareness research. This study investigates the analysis and processing of network security events, a crucial parameter in the stage of security insight and perception, and builds and implements a text-based network security event analysis tool. By designing a network security event analysis tool based on text processing, the data cleaning of network security time text information is completed, and a set of network security event processing solutions with high applicability and comprehensiveness are formed. Statistical experimental results show that the network security event database built based on the crawler algorithm contains 43,848 pieces of data, which increases the capacity by 12.79% and 29.33% compared with the traditional algorithm, and reduces the reading time by 63.5% and 87.2%.