technical research paper on network security

• Create Account

Main navigation dropdown

Publications, recent advances in network security management, publication date, manuscript submission deadline, 16 november 2021, call for papers.

As the backbone of communications amongst objects, humans, companies, and administrations, the Internet has become a great integration platform capable of efficiently interconnecting billions of entities, from RFID chips to data centers. This platform provides access to multiple hardware and virtualized resources (servers, networking, storage, applications, connected objects) ranging from cloud computing to Internet-of-Things infrastructures. From these resources that may be hosted and distributed amongst different providers and tenants, the building and operation of complex and value-added networked systems is enabled.

These networked systems are, however, subject to a large variety of security attacks, such as distributed denial-of-service, man-in-the-middle, web-injection and malicious software attacks, orchestrated in a more or less stealthy manner through the Internet. While they are gaining in sophistication and coordination (i.e. advanced persistent threats), these attacks may affect the fundamental security goals of confidentiality, integrity, availability and non-repudiation of resources. The accessibility, distribution, and increased complexity of networked systems make them particularly vulnerable targets. In that context, cybersecurity techniques offer new perspectives for protecting these networked systems, through the elaboration of intelligent and efficient management methods for detecting, analyzing and mitigating such attacks.

IEEE Transactions on Network and Service Management (IEEE TNSM) is a premier journal for timely publication of archival research on the management of networks, systems, services and applications. Following the success of the recent TNSM Special Issues on cybersecurity techniques for managing networked systems in 2020 and 2021, this Special Issue will focus on recent advances in network security management. We welcome submissions addressing the important challenges (see the non-exhaustive list of topics below) and presenting novel research or experimentation results. Survey papers that offer an insightful perspective on related work and identify key challenges for future research will be considered as well. We look forward to your submissions!

About the Special Issue

Topics of interest for this Special Issue, include, but are not limited to the following:

• Network and service management for security
• Security of network and service management
• Security management architecture, protocols and APIs
• Secure and resilient design and deployment of networked systems
• Monitoring and detection of threats and attacks
• Artificial intelligence, machine learning for cyber-security
• Analytics and big data for security management
• Modeling for security management
• Configuration and orchestration of security mechanisms
• Algorithms for security management
• Security automation, policy-based management
• NFV-based security functions and services
• Security of programmable components
• Chaining and orchestration of security functions
• Distributed security management
• Intrusion detection, tolerance, prevention, and response
• Resilience against large-scale distributed attacks
• Trust and identity management
• Verification and enforcement of security properties
• Vulnerability prevention and remediation
• Performance of security management
• Security of cloud applications and services
• Security of data-center infrastructures
• Security of 5G networks and services
• Security of smart environments
• Security of Internet of Things
• Security of SCADA, industrial and health networks
• Security of SDN- and NFV-based systems
• Network forensics, auditing and responses to incidents
• Privacy-preserving solutions for cybersecurity
• Detailed experience reports from experimental testbeds
• Security-related business, regulation, and legal aspects

Submission Format

Papers will be evaluated based on their originality, presentation, relevance and contribution to the field of security of software-defined virtualized systems, as well as their overall quality and suitability for the special issue. The submitted papers must be written in good English and describe original research which has not been published nor currently under review by other journals or conferences. Previously published conference papers should be clearly identified by the authors at the initial submission stage and an explanation should be provided of how such papers have been extended in order to be considered for this Special Issue.

Author guidelines for the preparation of manuscript (including number of pages and potential extra page costs) can be found on the IEEE TNSM   Submit a Manuscript  page.

For more information, please contact Remi Badonnel .

Submission Guidelines

All manuscripts and any supplementary material should be submitted through the IEEE Manuscript Central service . Authors must indicate in the submission cover letter that their manuscript is intended for the “Recent Advances in Network Security Management ” Special Issue.

Important Dates

Paper Submission Date: 30 October 2021 16 November 2021 (Extended Deadline) Notification of Acceptance: 15 February 2022 Publication Date*: 1 June 2022

(* online published version will be available in IEEE Xplore after the camera ready version has been submitted with final DOI)

Guest Editors

Rémi Badonnel (Lead) Telecom Nancy – LORIA / INRIA, France

Sandra Scott-Hayward Queen's University Belfast, UK

Carol Fung Virginia Commonwealth University, United States

Qi Li Tsinghua University, China

Jie Zhang Nanyang Technological University (NTU), Singapore

Cristian Hesselman SIDN, The Netherlands

Fulvio Valenza Politecnico di Torino, Italy

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

The current research status of ai-based network security situational awareness.

1. Introduction

2. related concepts of network security situational awareness, 3. the classic models of network security situational awareness, 3.1. endsley’s three-level model, 3.2. jdl model, 3.3. tim bass model, 3.4. ooda model, 4. the current research status of ai-based network security situational awareness, 4.1. overview of artificial intelligence, 4.2. current state of research on neural-network-based network security situational awareness, 4.2.1. bp neural network, 4.2.2. wavelet neural network, 4.2.3. rbf neural network, 4.2.4. long short-term memory network, 5. implementation and application, 5.1. vulnerability scanning and management, 5.2. behavioral analysis and anomaly detection, 5.3. security incident response and management, 6. summary and future directions, author contributions, data availability statement, conflicts of interest, abbreviations.

• Denning, D.E. An Intrusion-Detection Model. IEEE Trans. Softw. Eng. 1987 , SE-13 , 222–232. [ Google Scholar ] [ CrossRef ]
• Vigna, G.; Kemmerer, R.A. NetSTAT: A network-based intrusion detection system. J. Comput. Secur. 1999 , 7 , 37–71. [ Google Scholar ] [ CrossRef ]
• Mukherjee, B.; Heberlein, L.T.; Levitt, K.N. Network intrusion detection. IEEE Netw. 1994 , 8 , 26–41. [ Google Scholar ] [ CrossRef ]
• Lenders, V.; Tanner, A.; Blarer, A. Gaining an edge in cyber space with advanced situational awareness. IEEE Secur. Priv. 2015 , 13 , 65–74. [ Google Scholar ] [ CrossRef ]
• Friedrich, M.; Biermann, M.; Gontar, P.; Biella, M.; Bengler, K. The influence of task load on situation awareness and control strategy in the ATC tower environment. Cogn. Technol. Work. 2018 , 20 , 205–217. [ Google Scholar ] [ CrossRef ]
• Green, B.; Parry, D.; Oeppen, R.S.; Plint, S.; Dale, T.; Brennan, P.A. Situational awareness—What it means for clinicians, its recognition and importance in patient safety. Oral Dis. 2017 , 23 , 721–725. [ Google Scholar ] [ CrossRef ]
• Eggemeier, F.T.; Crabtree, M.S.; LaPointe, P.A. The effect of delayed report on subjective ratings of mental workload. In Proceedings of the Human Factors Society Annual Meeting (27th) on the Effect of Delayed Report on Subjective Ratings of Mental Workloads, Norkfolk, VA, USA, 10–14 October 1983. [ Google Scholar ]
• Wellens, A.R. Group Situation Awareness and Distributed Decision Making: From Military to Civilian Applications. In Individual and Group Decision Making: Current Issues ; Lawrence Erlbaum Associates, Inc.: Hillsdale, NJ, USA, 1993; pp. 267–291. [ Google Scholar ]
• Bass, T.; Gruber, D. A glimpse into the future of ID. Mag. USENIX SAGE 1999 , 24 , 40–49. [ Google Scholar ]
• Gong, J.; Zang, X.-D.; Su, Q.; Hu, X.-Y.; Xu, J. Survey of network security situation awareness. J. Softw. 2016 , 28 , 1010–1026. [ Google Scholar ]
• Shi, L.; Liu, J.; Liu, Y.; Zhu, H.; Duan, P. Review of network security situational awareness. Comput. Eng. Appl. 2019 , 55 , 1–9. [ Google Scholar ]
• Endsley, M.R. Situation awareness global assessment technique (SAGAT). In Proceedings of the IEEE 1988 National Aerospace and Electronics Conference, Dayton, OH, USA, 23–27 May 1988; pp. 789–795. [ Google Scholar ]
• Hall, D.L.; Llinas, J. An introduction to multisensor data fusion. Proc. IEEE 1997 , 85 , 6–23. [ Google Scholar ] [ CrossRef ]
• Bass, T.; Robichaux, R. Defense-in-depth revisited: Qualitative risk analysis methodology for complex network-centric operations. In Proceedings of the MILCOM Proceedings Communications for Network-Centric Operations: Creating the Information Force (Cat. No. 01CH37277), McLean, VA, USA, 28–31 October 2001; Volume 1, pp. 64–70. [ Google Scholar ]
• Boyd, J. A Discourse on Winning and Losing ; Air University Press: Maxwell Air Force Base, AL, USA, 2018; p. 347. [ Google Scholar ]
• Shen, D.; Chen, G.; Cruz, J.B., Jr.; Haynes, L.; Kruger, M.; Blasch, E. A markov game theoretic data fusion approach for cyber situational awareness. In Multisensor, Multisource Information Fusion: Architectures, Algorithms, and Applications ; SPIE: Bellingham, WA, USA, 2007; Volume 6571, pp. 143–154. [ Google Scholar ]
• Jia, X.F.; Liu, Y.; Yan, Y.; Wu, D. Network security situational awareness method based on capability-opportunity-intent model. Appl. Res. Comput. 2016 , 6 , 1775–1779. [ Google Scholar ]
• Nikoloudakis, Y.; Kefaloukos, I.; Klados, S.; Panagiotakis, S.; Pallis, E.; Skianis, C.; Markakis, E.K. Towards a machine learning based situational awareness framework for cybersecurity: An SDN implementation. Sensors 2021 , 21 , 4939. [ Google Scholar ] [ CrossRef ]
• Aggarwal, K.; Mijwil, M.M.; Al-Mistarehi, A.H.; Alomari, S.; Gök, M.; Alaabdin, A.M.Z.; Abdulrhman, S.H. Has the future started? The current growth of artificial intelligence, machine learning, and deep learning. Iraqi J. Comput. Sci. Math. 2022 , 3 , 115–123. [ Google Scholar ]
• Jain, A.; Tiwari, S. Prediction and Visualisation of Viral Genome Antigen Using Deep Learning & Artificial Intelligence. In Proceedings of the 2021 5th International Conference on Computing Methodologies and Communication (ICCMC), Erode, India, 8–10 April 2021; pp. 1430–1437. [ Google Scholar ]
• Kothari, D.; Patel, M.; Sharma, A.K. Implementation of Grey Scale Normalization in Machine Learning & Artificial Intelligence for Bioinformatics using Convolutional Neural Networks. In Proceedings of the 2021 6th International Conference on Inventive Computation Technologies (ICICT), Coimbatore, India, 20–22 January 2021; pp. 1071–1074. [ Google Scholar ]
• Visvikis, D.; Cheze Le Rest, C.; Jaouen, V.; Hatt, M. Artificial intelligence, machine (deep) learning and radio (geno) mics: Definitions and nuclear medicine imaging applications. Eur. J. Nucl. Med. Mol. Imaging 2019 , 46 , 2630–2637. [ Google Scholar ] [ CrossRef ]
• Aggour, K.S.; Gupta, V.K.; Ruscitto, D.; Ajdelsztajn, L.; Bian, X.; Brosnan, K.H.; Kumar, N.C.; Dheeradhada, V.; Hanlon, T.; Iyer, N.; et al. Artificial intelligence/machine learning in manufacturing and inspection: A GE perspective. MRS Bull. 2019 , 44 , 545–558. [ Google Scholar ] [ CrossRef ]
• Ali, D.; Frimpong, S. Artificial intelligence, machine learning and process automation: Existing knowledge frontier and way forward for mining sector. Artif. Intell. Rev. 2020 , 53 , 6025–6042. [ Google Scholar ] [ CrossRef ]
• Cioffi, R.; Travaglioni, M.; Piscitelli, G.; Petrillo, A.; De Felice, F. Artificial intelligence and machine learning applications in smart production: Progress, trends, and directions. Sustainability 2020 , 12 , 492. [ Google Scholar ] [ CrossRef ]
• Haenlein, M.; Kaplan, A. A Brief History of Artificial Intelligence: On the Past, Present, and Future of Artificial Intelligence. Calif. Manag. Rev. 2019 , 61 , 5–14. [ Google Scholar ] [ CrossRef ]
• Ongsulee, P. Artificial intelligence, machine learning and deep learning. In Proceedings of the 2017 15th International Conference on ICT and Knowledge Engineering (ICT&KE), Bangkok, Thailand, 22–24 November 2017; pp. 1–6. [ Google Scholar ]
• Xin, Y.; Kong, L.; Liu, Z.; Chen, Y.; Li, Y.; Zhu, H.; Gao, M.; Hou, H.; Wang, C. Machine learning and deep learning methods for cybersecurity. IEEE Access 2018 , 6 , 35365–35381. [ Google Scholar ] [ CrossRef ]
• Shinde, P.P.; Shah, S. A review of machine learning and deep learning applications. In Proceedings of the 2018 Fourth International Conference on Computing Communication Control and Automation (ICCUBEA), Pune, India, 16–18 August 2018; pp. 1–6. [ Google Scholar ]
• Yang, H.; Zeng, R.; Xu, G.; Zhang, L. A network security situation assessment method based on adversarial deep learning. Appl. Soft Comput. 2021 , 102 , 107096. [ Google Scholar ] [ CrossRef ]
• Geluvaraj, B.; Satwik, P.M.; Ashok Kumar, T.A. The future of cybersecurity: Major role of artificial intelligence, machine learning, and deep learning in cyberspace. In Proceedings of the International Conference on Computer Networks and Communication Technologies: ICCNCT 2018, Bengaluru, India, 10–12 July 2018; Springer: Singapore, 2019; pp. 739–747. [ Google Scholar ]
• Rumelhart, D.E.; Hinton, G.E.; Williams, R.J. Learning representations by back-propagating errors. Nature 1986 , 323 , 533–536. [ Google Scholar ] [ CrossRef ]
• Zhang, R.; Pan, Z.; Yin, Y. Research on assessment algorithm for network security situation based on SSA-BP neural network. In Proceedings of the 2021 7th International Symposium on System and Software Reliability (ISSSR), Chongqing, China, 23–24 September 2021; pp. 140–145. [ Google Scholar ]
• Kou, G.; Wang, S.; Zhang, D. Recognition of network security situation elements based on depth stack encoder and back propagation algorithm. J. Electron. Inf. Technol. 2019 , 41 , 2187–2193. [ Google Scholar ]
• Fu, T.; Lu, Y.; Zhen, W. APT attack situation assessment model based on optimized BP neural network. In Proceedings of the 2019 IEEE 3rd Information Technology, Networking, Electronic and Automation Control Conference (IT-NEC), IEEE, Chengdu, China, 15–17 March 2017; pp. 2108–2111. [ Google Scholar ]
• Yin, K.; Yang, Y.; Yang, J.; Yao, C. A network security situation assessment model based on BP neural network optimized by DS evidence theory. J. Phys. Conf. Ser. 2022 , 2258 , 012039. [ Google Scholar ] [ CrossRef ]
• Du, Z.; Yao, H.; Fu, Y.; Cao, Z.; Liang, H.; Ren, J. Network Situation Assessment Method Based on Improved BP Neural Network. Electronics 2023 , 12 , 483. [ Google Scholar ] [ CrossRef ]
• Alexandridis, A.K.; Zapranis, A.D. Wavelet neural networks: A practical guide. Neural Netw. 2013 , 42 , 1–27. [ Google Scholar ] [ CrossRef ]
• Ong, P.; Zainuddin, Z. Optimizing wavelet neural networks using modified cuckoo search for multi-step ahead chaotic time series prediction. Appl. Soft Comput. 2019 , 80 , 374–386. [ Google Scholar ] [ CrossRef ]
• Huang, C.; Wang, C. Network Security Situation Awareness Based on the Optimized Dynamic Wavelet Neural Network. Int. J. Netw. Secur. 2018 , 20 , 593–600. [ Google Scholar ]
• Zhao, J.; Li, X.; Cao, Y.; Liu, J.; Yan, J.; Li, C. Analysis and Application of intelligent Power Control System Cyber Security Situation Awareness Based on Wavelet Neural Network. J. Phys. Conf. Ser. 2021 , 2078 , 012067. [ Google Scholar ] [ CrossRef ]
• Hwang, Y.S.; Bang, S.Y. An efficient method to construct a radial basis function neural network classifier. Neural Netw. 1997 , 10 , 1495–1503. [ Google Scholar ] [ CrossRef ]
• Xie, T.; Yu, H.; Wilamowski, B. Comparison between traditional neural networks and radial basis function networks. In Proceedings of the 2011 IEEE International Symposium on Industrial Electronics, Gdansk, Poland, 27–30 June 2011; pp. 1194–1199. [ Google Scholar ]
• Li, Y. Prediction of network security situation awareness based on an improved model combined with neural network. IEEE Secur. Priv. 2021 , 4 , e181. [ Google Scholar ]
• Chen, Z. Research on Internet Security Situation Awareness Prediction Technology Based on Improved RBF Neural Network Algorithm. J. Comput. Cogn. Eng. 2022 , 1 , 103–108. [ Google Scholar ]
• Hochreiter, S.; Schmidhuber, J. Long Short-Term Memory. Neural Comput. 1997 , 9 , 1735–1780. [ Google Scholar ] [ CrossRef ]
• Van Houdt, G.; Mosquera, C.; Nápoles, G. A review on the long short-term memory model. Artif. Intell. Rev. 2020 , 53 , 5929–5955. [ Google Scholar ] [ CrossRef ]
• Zhang, H.; Kang, C.; Xiao, Y. Research on Network Security Situation Awareness Based on the LSTM-DT Model. Sensors 2021 , 21 , 4788. [ Google Scholar ] [ CrossRef ]
• Ding, C.; Chen, Y.; Algarni, A.M.; Zhang, G.; Peng, H. Application of fractal neural network in network security situation awareness. World Sci. 2022 , 2 , 2240090. [ Google Scholar ] [ CrossRef ]
• Wang, Q.; Bu, S.; He, Z.; Yangdong, Z. Toward the Prediction Level of Situation Awareness for Electric Power Systems Using CNN-LSTM Network. IEEE Trans. Ind. Inform. 2021 , 17 , 6951–6961. [ Google Scholar ] [ CrossRef ]
• Shu, X.; Tian, K.; Ciambrone, A.; Yao, D. Breaking the target: An analysis of target data breach and lessons learned. arXiv 2017 , arXiv:1701.04940. [ Google Scholar ]
• Alexopoulos, N.; Habib, S.M.; Schulz, S.; Mühlhäuser, M. The tip of the iceberg: On the merits of finding security bugs. ACM Trans. Priv. Secur. 2020 , 24 , 3. [ Google Scholar ] [ CrossRef ]
• Farris, K.A.; Shah, A.; Cybenko, G.; Ganesan, R.; Jajodia, S. Vulcon: A system for vulnerability prioritization, mitigation, and management. ACM Trans. Priv. Secur. 2018 , 21 , 16. [ Google Scholar ] [ CrossRef ]
• Dissanayake, N.; Jayatilaka, A.; Zahedi, M.; Babar, M.A. Software security patch management—A systematic literature review of challenges, approaches, tools and practices. Inf. Softw. Technol. 2022 , 144 , 106771. [ Google Scholar ] [ CrossRef ]
• Nunes, P.; Medeiros, I.; Fonseca, J.; Neves, N.; Correia, M.; Vieira, M. On combining diverse static analysis tools for web security: An empirical study. In Proceedings of the 2017 13th European Dependable Computing Conference (EDCC), Geneva, Switzerland, 4–8 September 2017; pp. 121–128. [ Google Scholar ]
• Nunes, P.J.C.; Fonseca, J.; Vieira, M. phpSAFE: A security analysis tool for OOP web application plugins. In Proceedings of the 2015 45th Annual IEEE/IFIP International Conference on Dependable Systems and Networks, Rio de Janeiro, Brazil, 22–25 June 2015; pp. 299–306. [ Google Scholar ]
• Li, L.; Ding, S.H.; Tian, Y.; Fung, B.C.; Charland, P.; Ou, W.; Song, L.; Chen, C. VulANalyzeR: Explainable Binary Vulnerability Detection with Multi-task Learning and Attentional Graph Convolution. ACM Trans. Priv. Secur. 2023 , 26 , 3. [ Google Scholar ] [ CrossRef ]
• Olswang, A.; Gonda, T.; Puzis, R.; Shani, G.; Shapira, B.; Tractinsky, N. Prioritizing vulnerability patches in large networks. Expert Syst. Appl. 2022 , 193 , 116467. [ Google Scholar ] [ CrossRef ]
• Zhang, X.; Wang, T. Elastic and reliable bandwidth reservation based on distributed traffic monitoring and control. IEEE Trans. Parallel Distrib. Syst. 2022 , 33 , 4563–4580. [ Google Scholar ] [ CrossRef ]
• Zhang, X.; Wang, Y.; Yang, M.; Geng, G. Toward concurrent video multicast orchestration for caching-assisted mobile networks. IEEE Trans. Veh. Technol. 2021 , 70 , 13205–13220. [ Google Scholar ] [ CrossRef ]
• Finsterbusch, M.; Richter, C.; Rocha, E.; Muller, J.A.; Hanssgen, K. A survey of payload-based traffic classification approaches. IEEE Commun. Surv. Tutor. 2013 , 16 , 1135–1156. [ Google Scholar ] [ CrossRef ]
• Roughan, M.; Sen, S.; Spatscheck, O.; Duffield, N. Class-of-service mapping for QoS: A statistical signature-based approach to IP traffic classification. In Proceedings of the 4th ACM SIGCOMM Conference on Internet Measurement, Sicily, Italy, 25–27 October 2004; pp. 135–148. [ Google Scholar ]
• Rezaei, S.; Liu, X. Deep learning for encrypted traffic classification: An overview. IEEE Commun. Mag. 2019 , 57 , 76–81. [ Google Scholar ] [ CrossRef ]
• Aceto, G.; Ciuonzo, D.; Montieri, A.; Pescapé, A. DISTILLER: Encrypted traffic classification via multimodal multitask deep learning. J. Netw. Comput. Appl. 2021 , 183 , 102985. [ Google Scholar ] [ CrossRef ]
• Lin, P.; Ye, K.; Hu, Y.; Lin, Y.; Xu, C.Z. A Novel Multimodal Deep Learning Framework for Encrypted Traffic Classification. IEEE/ACM Trans. Netw. 2022 . early access. [ Google Scholar ] [ CrossRef ]
• Pacheco, F.; Exposito, E.; Gineste, M.; Baudoin, C.; Aguilar, J. Towards the deployment of machine learning solutions in network traffic classification: A systematic survey. IEEE Commun. Surv. Tutor. 2018 , 21 , 1988–2014. [ Google Scholar ] [ CrossRef ]
• Montieri, A.; Ciuonzo, D.; Aceto, G.; Pescapé, A. Anonymity services tor, i2p, jondonym: Classifying in the dark (web). IEEE Trans. Dependable Secur. Comput. 2018 , 17 , 662–675. [ Google Scholar ] [ CrossRef ]
• Wang, L.; Mei, H.; Sheng, V.S. Multilevel identification and classification analysis of Tor on mobile and PC platforms. IEEE Trans. Ind. Inform. 2020 , 17 , 1079–1088. [ Google Scholar ] [ CrossRef ]
• Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A.N.; Kaiser, Ł.; Polosukhin, I. Attention is all you need. Advances in neural information processing systems. arXiv 2017 , arXiv:1706.03762. [ Google Scholar ]
• Devlin, J.; Chang, M.W.; Lee, K.; Toutanova, K. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv 2018 , arXiv:1810.04805. [ Google Scholar ]
• Radford, A.; Narasimhan, K.; Salimans, T.; Sutskever, I. Improving Language Understanding by Generative Pre-Training. 2018. Available online: https://www.cs.ubc.ca/~amuham01/LING530/papers/radford2018improving.pdf (accessed on 7 March 2023).
• Zhao, R.; Deng, X.; Yan, Z.; Ma, J.; Xue, Z.; Wang, Y. MT-FlowFormer: A Semi-Supervised Flow Transformer for Encrypted Traffic Classification. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Washington, DC, USA, 14–18 August 2022; pp. 2576–2584. [ Google Scholar ]
• Deshmukh, P.; Satyanarayana, G.S.R.; Majhi, S.; Sahoo, U.K.; Das, S.K. Swin transformer based vehicle detection in undisciplined traffic environment. Expert Syst. Appl. 2023 , 213 , 118992. [ Google Scholar ] [ CrossRef ]
• Zhao, R.; Huang, Y.; Deng, X.; Xue, Z.; Li, J.; Huang, Z.; Wang, Y. Flow Transformer: A Novel Anonymity Network Traffic Classifier with Attention Mechanism. In Proceedings of the 2021 17th International Conference on Mobility, Sensing and Networking (MSN), Exeter, UK, 13–15 December 2021; pp. 223–230. [ Google Scholar ]
• Lin, X.; Xiong, G.; Gou, G.; Li, Z.; Shi, J.; Yu, J. Et-bert: A contextualized datagram representation with pre-training transformers for encrypted traffic classification. In Proceedings of the ACM Web Conference 2022 Virtual Event, Lyon, France, 25–29 April 2022; pp. 633–642. [ Google Scholar ]
• Schlette, D.; Caselli, M.; Pernul, G. A comparative study on cyber threat intelligence: The security incident response perspective. IEEE Commun. Surv. Tutor. 2021 , 23 , 2525–2556. [ Google Scholar ] [ CrossRef ]
• Killcrece, G.; Kossakowski, K.P.; Ruefle, R.; Zajicek, M. State of the Practice of Computer Security Incident Response Teams (CSIRTs) ; Carnegie Mellon University, Software Engineering Institute: Pittsburgh, PA, USA, 2003. [ Google Scholar ]
• Zhang, X.; Wang, Y.; Geng, G.; Yu, J. Delay-Optimized Multicast Tree Packing in Software-Defined Networks. IEEE Trans. Serv. Comput. 2023 , 16 , 261–275. [ Google Scholar ] [ CrossRef ]
• Tøndel, I.A.; Line, M.B.; Jaatun, M.G. Information security incident management: Current practice as reported in the literature. Comput. Secur. 2014 , 45 , 42–57. [ Google Scholar ] [ CrossRef ]

Click here to enlarge figure

Share and Cite

Wang, M.; Song, G.; Yu, Y.; Zhang, B. The Current Research Status of AI-Based Network Security Situational Awareness. Electronics 2023 , 12 , 2309. https://doi.org/10.3390/electronics12102309

Wang M, Song G, Yu Y, Zhang B. The Current Research Status of AI-Based Network Security Situational Awareness. Electronics . 2023; 12(10):2309. https://doi.org/10.3390/electronics12102309

Wang, Maoli, Guangxue Song, Yang Yu, and Bowen Zhang. 2023. "The Current Research Status of AI-Based Network Security Situational Awareness" Electronics 12, no. 10: 2309. https://doi.org/10.3390/electronics12102309

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

Research Progress and Future Trend Analysis of Network Security Situational Awareness

• Conference paper
• First Online: 02 November 2021
• Cite this conference paper

• Junwei Zhang 19 ,
• Huamin Feng 20 ,
• Biao Liu 20 ,
• Ge Ge 21 &
• Jing Liu 22  

Part of the book series: Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering ((LNICST,volume 394))

Included in the following conference series:

• International Conference on Mobile Multimedia Communications

1214 Accesses

1 Citations

With the continuous expansion of the network scale in recent years, network security problems have become increasingly prominent, and network security incidents have emerged one after another. Network security situation awareness is an essential part of network security defense that allows cybersecurity operators to cope with the complexity of today’s networks and threat landscape. In this paper, we thoroughly review and systematize the origin and the models of network security situational awareness and the evolution of its definition, and then we give its definition. Additionally, we introduced the key technologies in this field from the three functional modules of network security situation extraction, network security situation assessment, and network security situation prediction, and analyzed their advantages and disadvantages. Last but not least, we explicitly propose four possible research directions that the researchers in network security can work on in the future.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save.

• Get 10 units per month
• Download Article/Chapter or eBook
• 1 Unit = 1 Article or 1 Chapter
• Cancel anytime
• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Available as EPUB and PDF
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Survey of Network Security Situational Awareness

A Survey of Network Security Situational Awareness Technology

Overview of Network Security Situation Awareness Based on Big Data

Gutzwiller, R., Dykstra, J., Payne, B.: Gaps and opportunities in situational awareness for cybersecurity. Digital Threats: Res. Pract. 1 (3), 1–6 (2020)

Article   Google Scholar  

Endsley, M.R.: Design and evaluation for situation awareness enhancement. Hum. Fact. Soc. Annu. Meet. 32 , 97–101 (1988)

Endsley, M.R.: Situation awareness global assessment technique (SAGAT). In: Proceedings of the IEEE 1988 National Aerospace and Electronics Conference, Dayton, OH, USA, pp. 789–795. IEEE (1988)

Google Scholar  

Husák, M., Jirsík, T., Yang, S.J.: SoK: contemporary issues and challenges to enable cyber situational awareness for network security. In: Proceedings of the 15th International Conference on Availability, Reliability and Security, Virtual Event, Ireland, pp. 2. Association for Computing Machinery (2020)

Giacobe, N.: Application of the JDL data fusion process model for cyber security. Proc. SPIE Int. Soc. Opt. Eng. 7710 , 5 (2010)

Ao, Z.G.: Cyberspace operations situational awareness. In: Cyberspace operations: mechanism and planning, pp. 691–699. Publishing House of Electronics Industry, Beijing (2018)

Bass, T.: A glimpse into the future of ID. Login:: the magazine of USENIX & SAGE, vol. 24, pp. 40–45 (1999)

Bass, T.: Intrusion detection systems and multisensor data fusion. Commun. ACM 43 (4), 99–105 (2000)

Wang, H.: Survey of network situation awareness system. Comp. Sci. 33 , 5–10 (2006)

Lai, J., Wang, H., Jin, S.: Research on network security situational awareness system based on NetFlow. Comp. Appl. Res. 24 (08), 173–175 (2007)

Wei, Y., Lian, Y., Feng, D.: Network security situation assessment model based on information fusion. Comp. Res. Develop. 46 (3), 353–362 (2009)

Jia, Y., Wang, X., Han, W., Li, A., Cheng, W.: YHSSAS: security situational awareness system for large-scale networks. Comp. Sci. 38 (002), 4–8 (2011)

Franke, U., Brynielsson, J.: Cyber situational awareness – a systematic review of the literature - ScienceDirect. Comp. Secur. 46 , 18–31 (2014)

Gong, J., Zang, X., Su, Q., Hu, X., Xu, J.: Overview of network security situational awareness. J. Softw. 28 (4), 1010–1026 (2017)

Jia, Y., Han, W., Yang, H.: Research status and development trend of network security situational awareness. J. Guangzhou Univ. 3 , 1–10 (2019)

Jajodia, S.: Topological Analysis of Network Attack Vulnerability. ACM (2006)

Wang, L., Singhal, A., Jajodia, S.: Toward Measuring Network Security Using Attack Graphs, vol. 49. ACM (2007)

Pan, N.: Techniques and tools for analyzing intrusion alerts. ACM Trans. Inf. Syst. Secur. 7 (2), 274–318 (2004)

Pan, N., Xu, D.: Alert correlation through triggering events and common resources. In: Proceedings of the Computer Security Applications Conference, 2004, 20th Annual, pp. 360–369. IEEE Computer Society (2004)

Barford, P., Yan, C., Goyal, A., Li, Z., Paxson, V., Yegneswaran, V.: Employing honeynets for network situational awareness. Adv. Inf. Secur. 46 (1), 71–102 (2010)

Wang, J., Zhang, F., Fu, C., Chen, L.: Research on index system in network situational awareness. Comput. Appl. 27 (008), 1907–1909, 1912 (2007)

Hailong, W., Gong, Z.: Heterogeneous multi-sensor information fusion model for botnet detection. In: Proceedings of the 2010 International Conference on Intelligent Computation Technology and Automation, pp. 428–431 (2010)

Liu, X., Wang, H., Cao, B.: Network security situation awareness model based on multi-source fusion. J. PLA Univ. Sci. Technol. (2012)

Wu, H., Hu, A., Song, Y., Bu, N., Jia, X.: A new intrusion detection feature extraction method based on complex network theory. In: Proceedings of the 2012 Fourth International Conference on Multimedia Information Networking and Security, pp. 852–856 (2012)

Tsang, C., Kwong, S.: Multi-agent intrusion detection system in industrial network using ant colony clustering approach and unsupervised feature extraction. In: Proceedings of the 2005 IEEE International Conference on Industrial Technology, pp. 51–56 (2005)

Lai, J., Wang, H., Zheng, F., Feng, G.: Network security situation element extraction method based on DSIMC and EWDS. Comput. Sci. 37 (011), 64–69 (2010)

Chang, Y., Ma, Z., Li, X., Gong, D.: Security situation element extraction based on probabilistic neural network. Cybersp. Secur. 11 (128(10)), 60–65 (2020)

Duan, Y., Li, X., Yang, X., Yang, L.: Network Security Situation Factor Extraction Based on Random Forest of Information Gain (2019)

Zhang, J.: Research on some key technologies of network security situation assessment. Doctor’s degree, National University of Defense Technology (2013)

Chen, X., Zheng, Q., Guan, X., Lin, C.: Hierarchical network security threat situation quantitative assessment method. J. Softw. 17 (004), 885–897 (2006)

Lai, J.: Research on several key technologies of network security situational awareness based on heterogeneous sensors. Doctor’s degree, Harbin Engineering University (2009)

Zhang, Y.: Research and system implementation of network security situational awareness model. Doctor’s degree, University of Science and Technology of China (2010)

Meng, J.: Research on key technologies of network security situation assessment and forecast. Doctor’s degree, Nanjing University of Science and Technology (2012)

Jia, Y., Wu, H., Jiang, D.: A hierarchical framework of security situation assessment for information system. In: Proceedings of the 2015 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery, pp. 23–28 (2015)

Peng, X., Li, J.H., Ou, X., Peng, L., Levy, R.: Using Bayesian networks for cyber security analysis. In: Proceedings of the 2010 IEEE/IFIP International Conference on Dependable Systems and Networks, DSN 2010, Chicago, USA. IEEE (2010)

Szwed, P., Skrzynski, P.: A new lightweight method for security risk assessment based on fuzzy cognitive maps. Int. J. Appl. Math. Comp. Sci. 24 (1), 213–225 (2014)

Boyer, S., Dain, O., Cunningham, R.: Stellar: a fusion system for scenario construction and security risk assessment. In: Proceedings of the Third IEEE International Workshop on Information Assurance (IWIA 2005), pp. 105–116 (2005)

Li, X., Deng, X., Jiang, W.: A novel method of network security situation assessment based on evidential network. In: Chen, X., Yan, H., Yan, Q., Zhang, X. (eds.) ML4CS 2020. LNCS, vol. 12486, pp. 530–539. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-62223-7_46

Chapter   Google Scholar  

Yang, H., Xie, X., Li, Z., Zhang, L.: Simulation of network security situation estimation model under multiple intrusion environment. Comp. Simulat. 033 (006), 270–273 (2016)

Zhao, G., Wang, H., Wang, J.: Research on survivability situation assessment of network based on grey relational analysis. Small Microcomp. Syst. 10 , 1861–1864 (2006)

Wang, C.: Assessment of network security situation based on grey relational analysis and support vector machine. Appl. Res. Comp. (2013)

Zhuo, Y., He, M., Gong, Z.: Rough set analysis model for network situation assessment. Comp. Eng. Sci. 34 (3), 1–5 (2012)

Li, X., Li, X., Zhao, Z.: Combining deep learning with rough set analysis: a model of cyberspace situational awareness. In: Proceedings of the 2016 6th International Conference on Electronics Information and Emergency Communication (ICEIEC), pp. 182–185 (2016)

Xiao, C., Qiao, Y., He, H., Li, J.: Multi-level fuzzy situation assessment based on optimal clustering criteria. Comp. Appl. Res. 30 (4), 1011–1014 (2013)

Wen, Z., Chen, Z., Tang, J.: Network security situation assessment method based on cluster analysis. J. Shanghai Jiaotong Univ. (Chin. Ed.) 50 (9), 1407–1414 (2016)

Leau, Y.-B., Manickam, S.: Network security situation prediction: a review and discussion. In: Intan, R., Chi, C.-H., Palit, H.N., Santoso, L.W. (eds.) ICSIIT 2015. CCIS, vol. 516, pp. 424–435. Springer, Heidelberg (2015). https://doi.org/10.1007/978-3-662-46742-8_39

Ioannou, G., Louvieris, P., Clewley, N.: A Markov multi-phase transferable belief model for cyber situational awareness. IEEE Access 7 , 39305–39320 (2019)

Lin, Z., Chen, G., Guo, W., Liu, Y.: PSO-BPNN-based prediction of network security situation. In: Proceedings of the 2008 3rd International Conference on Innovative Computing Information and Control, p. 37 (2008)

Tang, C., Yi, X., Qiang, B., Xin, W., Zhang, R.: Security situation prediction based on dynamic BP neural with covariance. Procedia Eng. 15 , 3313–3317 (2011)

Zhang, H., Huang, Q., Li, F., Zhu, J.: A network security situation prediction model based on wavelet neural network with optimized parameters. Digital Commun. Netw. 139–144 (2016)

Feng, W., Fan, Y., Wu, Y.: A new method for the prediction of network security situations based on recurrent neural network with gated recurrent unit. Int. J. Intell. Comput. Cybernet. 13 (1), 25–39 (2018)

Ren, W., Jiang, W., Jiang, X., Sun, Y.: Network security situation prediction method based on RBF neural network. Comp. Eng. Appl. 42 (31), 136–138, 144 (2006)

Jiang, Y., Li, C., Yu, L., Bao, B.: On network security situation prediction based on RBF neural network. In: Proceedings of the 36th China Control Conference (2017)

Hu, J., Ma, D., Liu, C., Shi, Z., Yan, H., Hu, C.: Network security situation prediction based on MR-SVM. IEEE Access 7 , 130937–130945 (2019)

Lu, H., Zhang, G., Shen, Y.: Cyber security situation prediction model based on GWO-SVM. In: Barolli, L., Xhafa, F., Hussain, O.K. (eds.) IMIS 2019. AISC, vol. 994, pp. 162–171. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-22263-5_16

Wang, Y., Li, W., Liu, Y.: A forecast method for network security situation based on fuzzy Markov chain. In: Huang, Y.-M., Chao, H.-C., Deng, D.-J., Park, J.J.H. (eds.) Advanced Technologies, Embedded and Multimedia for Human-centric Computing. LNEE, vol. 260, pp. 953–962. Springer, Dordrecht (2014). https://doi.org/10.1007/978-94-007-7262-5_108

Wen, Z., Chen, Z.: Network security situation prediction method based on hidden Markov model. J. Cent. South Univ. 46 (10), 137–143 (2015)

MathSciNet   Google Scholar  

Liang, W., Long, J., Chen, Z.: A security situation prediction algorithm based on HMM in mobile network. Wirel. Commun. Mob. Comput. 2018 , 5380481 (2018)

Lai, J., Wang, H., Liang, W., Zhu, L.: Study of network security situation awareness model based on simple additive weight and grey theory. In: Proceedings of the 2006 International Conference on Computational Intelligence and Security, pp. 1545–1548 (2006)

Zhang, F., Wang, J., Qin, Z.: Using gray model for the evaluation index and forecast of network security situation. In: Proceedings of the 2009 International Conference on Communications, Circuits and Systems, pp. 309–313 (2009)

Deng, Y., Wen, Z., Jiang, X.: Network security situation prediction method based on grey. Theory 2 , 69–73 (2015)

Download references

Acknowledgments

This work was supported by the “High-precision” Discipline Construction Project of Beijing Universities (No. 20210071Z0403).

Author information

Authors and affiliations.

School of Cyber Engineering, Xidian University, Xian, 710126, China

Junwei Zhang

Beijing Electronic Science and Technology Institute, Beijing, 100070, China

Huamin Feng & Biao Liu

National Administration of State Secrets Protection, Beijing, 100031, China

School of Cyberspace Security, Beijing University of Posts and Telecommunications, Beijing, 100876, China

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Junwei Zhang .

Editor information

Editors and affiliations.

Fujian Normal University, Fuzhou, China

Jinbo Xiong

Illinois State University, Normal, IL, USA

Guizhou University, Guiyang, China

Changgen Peng

Youliang Tian

Rights and permissions

Reprints and permissions

Copyright information

© 2021 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering

About this paper

Cite this paper.

Zhang, J., Feng, H., Liu, B., Ge, G., Liu, J. (2021). Research Progress and Future Trend Analysis of Network Security Situational Awareness. In: Xiong, J., Wu, S., Peng, C., Tian, Y. (eds) Mobile Multimedia Communications. MobiMedia 2021. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 394. Springer, Cham. https://doi.org/10.1007/978-3-030-89814-4_39

Download citation

DOI : https://doi.org/10.1007/978-3-030-89814-4_39

Published : 02 November 2021

Publisher Name : Springer, Cham

Print ISBN : 978-3-030-89813-7

Online ISBN : 978-3-030-89814-4

eBook Packages : Computer Science Computer Science (R0)

Share this paper

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

• Open access
• Published: 13 August 2019

Analysis framework of network security situational awareness and comparison of implementation methods

• Yan Li   ORCID: orcid.org/0000-0001-5326-9327 1 ,
• Guang-qiu Huang 2 ,
• Chun-zi Wang 1 &
• Ying-chao Li 1  

EURASIP Journal on Wireless Communications and Networking volume  2019 , Article number:  205 ( 2019 ) Cite this article

19k Accesses

35 Citations

Metrics details

Information technology has penetrated into all aspects of politics, economy, and culture of the whole society. The information revolution has changed the way of communication all over the world, promoted the giant development of human society, and also drawn unprecedented attention to network security issues. Studies, focusing on network security, have experienced four main stages: idealized design for ensuring security, auxiliary examination and passive defense, active analysis and strategy formulation, and overall perception and trend prediction. Under the background of the new strategic command for the digital control that all countries are scrambled for, the discussion of network security situational awareness presents new characteristics both in the academic study and industrialization. In this regard, a thorough investigation has been made in the present paper into the literature of network security situational awareness. Firstly, the research status both at home and abroad is introduced, and then, the logical analysis framework is put forward concerning the network security situational awareness from the perspective of the data value chain. The whole process is composed of five successive stages: factor acquisition, model representation, measurement establishment, solution analysis, and situation prediction. Subsequently, the role of each stage and the mainstream methods are elaborated, and the application results on the experimental objects and the horizontal comparison between the methods are explained. In an attempt to provide a panoramic recognition of network security situational awareness, and auxiliary ideas for the industrialization of network security, this paper aims to provide some references for the scientific research and engineering personnel in this field.

1 Introduction

The information technology revolution has made great changes in the way of human communication in the world today. Especially in recent years, in-depth studies of the industrialization concepts of cloud computing, large data, Internet of Things, and mobile terminals have made the control of digital information become a new strategic commanding point, and the problem of network security has also received more attention in a wider range. The exposure of “prism plan” in June 2013 brought information security from economic interest to the level of national security. In February 2014, the establishment of the “central network security and information group” marked the awakening of the national consciousness of the Internet in China and highlighted the importance of the national information security strategy. However, the ability of network overall defense at the national level to attack risk is still relatively weak [ 1 ]. How to prevent organized malicious network attack has become a hot topic in the field of security.

Studies on network security have started since the birth of information networks. The exponential growth of network size and application, especially the random dynamic access relationship built on the static Internet physical connection network based on OSI model, makes the study of network security more complicated. Before the 1960s, the focus on the network security research is how to build an absolute security system and reduce design vulnerabilities to ensure the confidentiality, integrity, and availability of the system, which can be regarded as the first stage of network security research. However, people soon realized the impossibility in practical operation [ 2 ]. The existence of malicious intrusion provokes the thought to build a security assistant system with an aim to detect the intrusion in time and take corresponding measures. The most typical application is the intrusion detection system (IDS) [ 3 ]. The intrusion detection is originated from Anderson’s Technical Research Report [ 4 ], and the subsequent researches can be divided into two categories: anomaly detection and misuse detection. At present, the IDS of most research institutions and commercial organizations is based on these two categories. Intrusion detection technology provides predictive warning information to ensure network security when network attacks occur, but it is too weak to do anything about the wall-around stealth attack and multi-step compound attack. Such a passive defense technology is unsatisfactory in the real-time detection. On this basis, the focus of the third stage research after the 1990s shifted from passive defense to active analysis [ 5 , 6 ], which is originated from the development of hacker technology. The intent is to carry out an integrated safety assessment before the occurrence of network attacks, formulate a defense strategy, or still provide predetermined service function given the damaged network. In 1990, Bass first proposed the concept of Cyber Situation Awareness CSA [ 7 , 8 ], which intends to perceive elements in the time and space environment, so that people can better grasp the overall network security situation and predict future trends, which to a certain extent promotes the integration of network security research and other disciplines. The development, especially the combination with some advanced stochastic models, has made theoretical progress (such as stochastic algebra [ 9 ], game theory [ 10 ], Bayesian network [ 11 ]). However, most of them are based on CSA conceptual model to optimize the evaluation algorithm with few breakthroughs in the practical application and systematic expositions (Table  1 gives a brief summary of the four main stages of the development of network security studies).

This paper gives a systematic introduction to the field of network security situational awareness, with an aim to provide insightful guidance for understanding the related concepts, promoting their application in practice and carrying out large-scale network expansion. In addition, a general analysis framework of network security situational awareness is proposed from the perspective of value chain. The framework divides the process of network security situational awareness into five stages: factor acquisition, model representation, measurement establishment, solution analysis, and situation prediction, which summarizes the current research progress in each stage and discusses the practical application results of typical methods. Moreover, this paper also elaborates the visualization of perception analysis results and situational awareness in the large data environment and prospects the key issues and research trend of this topic.

2 Research status at home and abroad

Situational awareness is first seen in the study of military academia. The human factor analysis of Theureau [ 12 ] in aviation has greatly promoted the application of this field in human-machine interaction, medical emergency scheduling, and real-time battlefield command. In 1988, Endsley [ 13 ] defined situation awareness as the three-level model of situation factor acquisition, situation understanding, and situation prediction, and the application framework of situational awareness in dynamic decision making was proposed in 1995 [ 14 ]. On this basis, the case study of the practical application of situational awareness is started, for example, Boyd control cycle model [ 15 ], Tadda JDL data fusion model [ 16 ] based on Endsley’s three-level model, cognitive fusion control model [ 17 ], and so on.

Inspired by the air traffic control (ATC) situational awareness, Bass [ 7 ] of the US Air Force Communications and Information Center first proposed the concept of network situational awareness, in an attempt to apply the ATC data fusion to network situational awareness. Since then, the attention of most studies is paid to the data fusion analysis with the ignorance of the essential definition of cybersecurity situational awareness. At present, there is no clear and unified expression of network security situational awareness. However, confirmation is made that network security situational awareness and situational awareness belong to the relationship between instance and type instead of that of subset, which means the relevant theory of situational awareness and the method can be applied in the field of network security situational awareness after the specific processing. The literature [ 19 ] has a systematic explanation for the definition of network security situational awareness and the understanding of the basic concept. Based on the explanation above, this paper offers the basic operation mechanism of network security situational awareness and illustrates the role of each link in the cognitive process of network security status in the mechanism.

2.1 Network security situation awareness and intrusion detection

The general model of the intrusion detection system (IDS) is first proposed by Denning [ 20 ]. Its core idea is to set up a regular set of rules that can be updated and modified under the condition of a unified clock. Thereafter, information is collected by an agent from the network process records and compared with the defined rules, and then, determination is made whether the activity set exists, which is trying to break the integrity, confidentiality, and availability of resources. The structure of IDS can be mainly divided into three types: host-based detection [ 21 ], network-based detection [ 22 , 23 ], and agent-based detection [ 24 ]. The host-based detection mainly matches the process record information on a single host. This obviously does not meet the security requirements under the network environment; thus, the network-based detection is built after adding some elements on the host-based detection, such as network traffic and protocol information; however, with the gradual use of distributed systems, IDS on distributed hosts also needs information interaction, which contributes to the formation of agent-based detection. Technically, IDS is mainly divided into two types [ 25 ], abnormal intrusion detection and misuse intrusion detection. Abnormal behavior is the opposite of normal or harmless behavior, so the rule set in abnormal behavior detection is the mode of the normal operation of the system. When detecting the deviation from the normal model, the alarm signal is generated. The advantage of this method is that any exploratory behavior will be recorded in addition to the prescribed “normal” action. But there will be a higher “false alarm rate” because the normal mode of the system is dynamic and cannot be completely normalized at the beginning of the establishment of the detection model; misuse behavior is abnormal or harmful behavior, so the rule set of misuse behavior detection is a model of system harmful behavior. When it detects the behavior that matches the harmful pattern, it produces an alarm. In the case of clear matching, this method has high accuracy, especially for the typical known attack model. But there is a big “rate of missing report” because it is almost impossible to passively carry out the whole sample summarization of harmful behavior under the background of diverse aggressive behaviors.

Through the brief summary of IDS, there are two main bottlenecks: passive response and false alarm rate / missing report rate , and the researchers have done a great deal of improvement on these two points. The main improvement of the passive response mode is on the automatic or semi-automatic response mode [ 26 ]. The main reason for false alarm rate or missing report rate is that there is a gray area between normal and abnormal, for which the IDS system and administrators cannot be analyzed in a unified perspective. Therefore, the improvement of this aspect is mainly the multi-level fusion analysis of more information [ 27 , 28 , 29 ], which is consistent with the summary of the four main stages of network security research in Table  1 . In fact, the initial research on network situational awareness is also based on IDS. Bass [ 7 , 30 ] proposed a multi-sensor integration intrusion detection framework after the concept of situation awareness, and literature [ 31 , 32 ] also put forward a similar framework. On this basis, lots of influential security situational awareness applications appeared, such as NVisionIP [ 33 ], VisFlowConnect-IP [ 34 ], and UCLog+ [ 35 ].

It can be seen that the network security situational awareness is a more advanced research stage and development direction to make up the defects of IDS. On the one hand, the existing results of IDS are the basis of the in-depth study of the network security situational awareness, and the latest methods and results of the network security situational awareness can relieve the contradictions of IDS. As shown in Table  2 , there are differences and strong connections between network security situational awareness and IDS. First of all, the focus of IDS is the presence or occurrence of attacks (or exceptions) in the network, and network security situational awareness is concerned with the security trend of a whole network. The analysis of attack behavior in network security situational awareness plays a fairly important part, and attack behavior is carried out step by step in normal behavior steps. Furthermore, the results of fusion analysis in network security situational awareness will also make IDS better explain and describe the rules of abnormal behavior or misuse behavior; secondly, before rule comparison, the core information acquisition results of IDS is the attack precursor and post which is in the network management audit category. However, the fusion analysis of network security situational awareness is definitely the element information abstraction of the whole network. With the elaborate study, the input information of IDS also has a great expansion, but the input of IDS must be a subset of the input of the network security situational awareness, and the output of the IDS can also be used as the input element of the network security situational awareness. In turn, the result of the network security situational awareness will make IDS’s information collection more precise and effective. Thirdly, at the functional level, the core function of IDS is to intercept suspected attack behavior through abnormal/misuse detection comparison and guide network administrators to take measures to defend the next attack. The core purpose of network security situational awareness is to carry out the security situation prediction, which is intended to guide the administrator to take configuration measures before the attack, which will certainly improve the detection efficiency of IDS. The pre/post-rule detection method based on standard IDS is also the most effective and reliable prediction method of network security situational awareness; fourthly, the analysis of IDS mainly focuses on attack behavior, but it is not capable of multi-step attack or attack around the wall. Most fusion analysis of network security situational awareness also deals with the analysis of aggressive behavior or abnormal behavior, because such behavior produces more benefit than normal access behavior. However, the overall analysis results including other behaviors will give IDS guidance both in particle size and in the accuracy of description; fifthly, in the early warning period, IDS carries out the acquisition analysis and warning based on audit information after attack, and the passive response mode is difficult to guarantee the network security in real time. Network security situational awareness does the active security situation perception before the attack, and it does not aim to eliminate the attack but to ensure that the network system is still safe or can still provide a predetermined function under the conditions of a certain attack. At last, in the detection efficiency, the core breakthroughs of IDS are high rate on false alarm / missing report and weak real time. If the configuration is too strict, the assertion of “suspect is wrong” will affect the effectiveness of the system. Loose configuration means “only heavy person should be judged” will miss the report. The compromise state between the two extremes requires the system to have the human gray perception ability, rather than the computer cognitive logic which means one or the other. The fusion process of network security situational awareness (NSSA) is easier to cross boundaries with artificial intelligence and other multidisciplinary research results for further improving the flexibility of detection, and the fusion analysis of flow data in large data environment will greatly promote the real-time performance of detection.

2.2 Status of foreign research

The study of situational awareness comes from a series of studies and elaborations of more than 15 articles by Endsley [ 13 , 14 , 36 ]. Bass [ 7 ] proposed the concept of network situational awareness for the first time and combined it with cyberspace. Driven by the new technologies such as the Internet of Goods, big data, and mobile applications, the innovation and promotion of the Internet application level have expanded rapidly, and the topology has become increasingly complex. As the public information shows (Fig.  1 ), all countries have raised their network security awareness to the national strategic level. From the summary of the cybersecurity strategies, publicized in various countries in recent years, it can be seen that although countries have different understandings of cybersecurity and strategy implementation, countries are aware of the need to take action to protect the key information and related infrastructure, as well as to achieve the prediction of intelligent network security situation with new methods and technologies.

Departments and public security strategies for network security in time series of countries

The great emphasis from governments can bring more financial support in terms of the fund. Besides, the spontaneous and continuous attention of many researchers to this field has made the researches on cybersecurity the top hot issue. In order to fully understand the research status of network security situational awareness, this paper firstly searched and reviewed articles on this topic in the past 10 years in the core database in September 2017, and sorted out a total of 10 large citations of review literature [ 16 , 37 , 38 , 39 , 40 , 41 , 42 , 43 , 44 , 45 ]. Based on the actor-network theory, Kopylec et al. [ 37 ] explored the critical relationship between physical and network infrastructure, and demonstrated the results of situational awareness through visual cascading. From the viewpoint of network’s key equipment administrators, he managed to maximize the understanding of the process of the risk propagation, thus providing systematic guidance in related planning and emergency response. Based on the combination of computer automation technology with human irregularity (abnormal or new mode) processing capabilities, literature [ 38 ] describes the research ideas and tools provided by the VizSec R&D community, which enables network managers to better identify the potential cyber threats. With aspect to the multidisciplinary integration, Jajodia et al. [ 39 ] conducted the research in relation to the questions and methods of network situational awareness in 2010 with an excellent conclusion and analyzed the key problem of the network situational awareness, as well as summarizes the main reasons for the lack of network situational awareness. Tadda and Salerno [ 16 ], Giacobe [ 40 ], and Schreiber-Ehle and Koch [ 42 ] inquired into the application process of JDL model in the field of situational awareness, especially in literature [ 40 ] for the favorable induction and summary of the data source information at level 0/1 in JDL model. In addition, Klein et al. [ 41 ] and Vincent [ 45 ] et al. applied the OODA loop model [ 15 ] to the network situational awareness and some stages in the model are prerequisites for others. Through such a class decision paradigm, the various activities in network defense are integrated. Much emphasis in literature [ 43 ] is attached to the information security of industrial networks. The difference between industrial networks and general computer networks makes the commonly used “detection/repair” methods in general computer networks not fully applicable. In light of this, the current state of distributed computing systems has been evaluated in the present paper, and the key elements in defensive countermeasures can help to reduce the risks to an acceptable threshold. In 2014, Franke and Brynielsson [ 44 ] conducted an effective summary of 102 articles in the four major scientific databases, being regarded as one of the best researches in the past 3 years, where 11 sub-categories were compared and the current status of the research was discussed according to the research field or content. The literature [ 46 ] provides an overview of the problems, challenges, threats, and solutions in social network security. In a strict sense, computer network security is an integral part of social network security. Therefore, some of the methods mentioned provide a meaningful reference, and the logic of their induction and comparison has greatly inspired this current paper.

By summarizing the literature review, it can be found that the main thread of foreign research is to instantiate the situational awareness model and method in the field of network security situational awareness, and continuously test and optimize the process in practice. In order to effectively analyze the research details of network security situational awareness, this paper concludes 75 papers in the core database in recent years and the research points of these articles are mainly concentrated on 9 aspects (the key points in these 9 aspects are shown in Table  3 ). The research content is mapped with the traditional Endsley model [ 36 ], the JDL model [ 40 ], and the logical phase of the OODA model [ 45 ]:

The concept of the model (integration with other disciplines) [ 16 , 18 , 39 , 45 , 50 , 51 , 52 , 53 , 54 , 55 , 56 , 57 , 63 , 67 , 68 ]

The completeness and regularization of data acquisition variables [ 40 , 42 , 45 , 47 , 73 , 87 ]

The optimization of related algorithms [ 58 , 59 , 60 , 61 , 62 , 63 , 64 , 65 , 66 , 67 ]

The information fusion analysis [ 40 , 42 , 53 , 69 , 70 , 71 , 72 , 73 , 74 ]

The automation of process tools [ 33 , 34 , 35 , 73 , 75 , 84 , 85 , 87 ]

The visualization of work at each stage [ 5 , 11 , 55 , 61 , 76 , 77 , 78 , 79 , 86 ]

Practice testing and efficiency gains in large-scale real-world networks [ 80 , 81 , 82 ]

The software engineering implementation of sensing methods [ 42 , 83 , 84 , 85 , 88 ]

The practical application of analysis and prediction results in specific fields [ 42 , 47 , 73 , 79 , 87 , 89 , 90 ]

In the research for the concept of model, some papers are aimed at explaining interpretations of traditional situational awareness models in network security situational awareness (such as literature [ 16 , 39 , 45 ]). Some papers focus on the combination of situational awareness with security issues in specific fields. For example, Ralston et al. [ 47 ] summarize the safety perception problem of distributed control system and data acquisition control system. Barford et al. [ 48 ] defines and explains the scope, background, and research objectives of network-aware defense. Alexandros et al. [ 49 ] summarizes the security threats and detection technologies in the field of wireless networks. Literature [ 50 ] has incorporated sensitive devices into the priority perception area and showed that how the DPI is installed at the boundary of the network perceives the health of the system; some literatures try to integrate the concepts of other disciplines into network security situational awareness, such as the combination with game theory in literature [ 51 , 52 , 53 , 63 ], the combination with Petri network [ 54 ], and the combination with the Bayesian network [ 55 ]; also, some other articles try to provide a more general operational model (such as the literature [ 56 , 57 , 67 ]).

Data acquisition is the basis of network security situational awareness. The attention now is paid to how to ensure that the collected information is a complete set for the fusion analysis in next step (completeness) and to standardize the collected information to promote the mutual call between different systems (regularization). Giacobe [ 40 ] has effectively combed the scope of source data and entities. In literature [ 45 ], the categories of sensors are divided into three categories: activity, configuration, and topology. In addition, in the specific field, the scope or type of collected information may be different [ 42 , 47 , 73 , 87 ].

The research on perception algorithm or architecture accounts for the largest part in all literature, with a proportion of more than 70%. Most of the articles give the logic of the algorithm and the demonstration effect in the experimental application. Literature [ 58 ] divides the common methods in situation awareness into five categories: Bayesian approach, knowledge-based approach, artificial neural systems approach, fuzzy logic approach, and genetic algorithm approach. In the algorithm for network security situational awareness, there are algorithms for data sources (such as the algorithm for the attacker [ 59 ], the algorithm for intrusion detection data [ 60 ], and the algorithm for the vulnerability logic association analysis [ 61 ]). Some algorithms are targeted at the behavior analysis of attackers or defenders (for example, hidden Markov chains are used to predict internal attack threats in document [ 62 ], combined with game theory [ 63 ], machine learning method [ 64 ], and honeypot technology [ 65 ], etc.); also, there are many algorithms for improving efficiency and enabling them to be extended in large scale networks (such as real-time decision analysis method [ 66 ], and fast calculation method for static statistical data [ 67 ]).

The fusion analysis ability on the related information is the advantage of network security situational awareness. The core method is to derive the hidden knowledge from the data from different sources. The related literatures are divided into three parts: one is the instantiation of data fusion model in traditional situational awareness in the network security situational awareness (such as [ 40 , 42 ]); one is to propose a specific fusion technology or idea based on the characteristics of network security data. For example, Paffenroth et al. [ 70 ] and Mathews et al. [ 71 ] have designed data models or coordinate working systems to integrate data from different network sensors. Literature [ 69 , 72 ] discuss the uncertainty in the network security situation. Sanfilippo [ 73 ] design a multi-sensor fusion framework to improve the perception ability; other literatures attempt to promote the efficiency of information fusion (e.g., [ 53 , 74 ]).

Automation based on the full use of the computing power of the computer is one of the effective ways to improve efficiency. In the IDS phase (the second stage of Table  1 ), the working mechanism of IDS is automated, but it also becomes the bottleneck of the system in turn, since the rule of the computer is not consistent with the perspective of human fuzzy evaluation. At present, the research on automation is mainly focused on information collection (such as literature [ 33 , 34 , 35 , 75 ]). In addition, systematic implementation of the overall application effect has realized automatic processing to a certain extent (such as [ 84 , 85 ]); the automation ability is also a prerequisite for the practical application of large-scale networks (e.g., [ 73 , 87 ]).

Visualization is undoubtedly an important part of network security situational awareness [ 86 ]. Tamassia et al. [ 76 ] give a clear statistical result on this aspect. Most of the current literature focuses on the friendly interaction between human and machine. Beaver et al. [ 77 ] effectively filter the analysis process and data in IDS and present them to administrators in a visual way. In literature [ 78 ], with the help of the unique professional knowledge of the participants, a real-time evaluation visual framework is designed to allow network managers to participate in the analysis loop manually; some articles focus on machine learning methods for visual rendering (such as artificial neural network [ 79 ] and cluster analysis [ 77 ]). In addition, most active analysis models such as attack graphs are combined with visualization technology [ 5 , 11 , 55 , 61 ].

Effect test constitutes the core of the model construction. In most of the articles, there is a chapter for the simulation experiment, but most of these experiments are analyzed with a brief abstract topology, for the verification of the correctness of the model. There are two aspects of research in this segment. One is the construction of basic data sets that can be used for horizontal comparison among multiple models (for example, the data set produced by the security contest held in literature [ 80 ] in 2010. Fink [ 81 ] collates the data set by each team in the competition). The other is the practice of wide area environment (at present, the attention to this aspect is little; literature [ 82 ] has made a preliminary attempt on this).

Consideration for the overall logic rather than a certain segment is the consensus view of the scholars [ 83 ], in view of the fact that the overall logic means that it should be designed from the perspective of software engineering. Only on this basis, the process and result of perceptual analysis can become effective tools. D’Amico and Whitley [ 84 ] design the overall analysis process based on the different roles and present it visually; literature [ 85 ] gives a task flow chart according to processes, goals, and concerns. There is a long way to go now, and the design and realization of network security situational awareness can be done from the perspective of instrumentalism software, which integrates the characteristics of all kinds of users in the network, and give a friendly target understanding method when human-machine interaction with necessary attention [ 42 ].

There are some articles concerning the analysis method of network security situational awareness and the practical application of prediction results in specific fields. The present statistical results mainly concentrate on three parts: one is the application of industrial control networks [ 47 ], especially in the field of power grid control [ 79 , 87 ]; one is for the emergency management of the key equipment, such as the shared situational awareness metamodeling proposed in Literature [ 89 ] and the operational architecture proposed by Adams [ 90 ]; and another is in the military field [ 42 ], such as the practice application of nautical training [ 73 ].

2.3 Status of domestic research

When it comes to the dominance of policies China, great importance is attached to the network security from top to bottom. As a consequence, China has established the emergency response mechanism related to network security at all levels, which is similar to European and American countries, such as CCERT(China education and scientific research network computer emergency team), set up in May 1999, and CNCERT/CC (National Computer Network Emergency Technology Processing Coordination Center, referred to as the “National Internet Emergency Center”), established in September 2002, as well as the central network security and information leading group, formed on February 27, 2014. On April 19, 2016, General Secretary Xi Jinping emphasized the importance, task, and goal of network security in his speech at the Symposium on Network Security and Informatization [ 91 ], and clearly put forward that perceiving network security situation is the most basic and basic work. Due to the limited space, this paper does not make too much interpretation of China’s network security policies and industrial development.

Domestic scholars have devoted great interest and enthusiasm to academic research. Almost every relevant core journal has dealt with the topics related to “network security.” In order to summarize the current research situation in China and keep in line with the research ideas of foreign literature, this paper firstly sorted out the review literature based on the author’s accumulation and effective search in this field. A total of 9 [ 17 , 19 , 92 , 93 , 94 , 95 , 96 , 97 , 98 ] comprehensive literature has a large number of citations or strong reference significance. In literature [ 92 ], the research and development of cryptography, trusted computing, network security, and information hiding in information security theory and technology are introduced. Especially in Section 4, Professor Feng Dengguo summarizes the research status and development trend of network information security and points out that the network-based security technology is the future trend of the development of the information security technology. Almost all network attacks are implemented by using the security flaws in system software or application software. Based on this premise, Liu and other scholars [ 93 ] conclude the research status at home and abroad from three aspects: malicious software, software vulnerabilities, and software security mechanisms from the perspective of software design for ensuring safety (study of the first stage in Table  1 ). Literature [ 94 ] provides an interpretation from the concept, necessity, structure of system, and basic model of intrusion detection and points out the development direction of intrusion detection system. In recent years, the research on the intrusion detection system probes further into the existing problems. Yingxu et al. [ 95 ] analyzes the characteristics and detection difficulties of industrial control system attacks. The performance and characteristics of different detection techniques are compared in order to provide theoretical support for researchers in the field of industrial control security. In 2005, Professor Lin Chuang of Tsinghua University [ 96 ] discusses the research methods and evaluation techniques used in the stochastic network security model which can be employed for the active evaluation and improves the network survivability. The analysis shows that most of the active evaluation models in the last 10 years (the third stage in Table  1 ) are extended on the basis of the models described in this article. In the study of situational awareness, literature [ 97 ] introduces the basic concepts of network situational awareness and expounds the relationship between situational awareness and IDS. Gong et al. [ 98 ] put forward a logical research framework on the basis of full understanding of situational awareness and attached emphasis on the method of network assessment. Based on the fusion algorithm of cross-layer swarm optimization, Liu et al. [ 17 ] puts forward a cognitive sensing and control model. Under the background of the transition of network development from perceptual network to perceptual network, the related algorithms of quantitative perception are given. Gong et al. [ 19 ] discuss the relationship between network security situational awareness and situational awareness at the conceptual level and further proposes the definition and explanation of network security situational awareness. Based on Endsley’s three-stage model [ 14 ], the stages of network security situational awareness are divided, and the specific analysis methods of each stage are compared.

In light of the comparison between the domestic and foreign literature, it is found that the time Chinese scholars pay attention to network security situational awareness is close to that of foreign scholars, but most of them are in the state of “following,” with few original and innovative articles. Most of the high-cited articles in ESI are aimed at the breakthrough of the model algorithm optimization and application level [ 96 , 99 ], especially in the aspect of situation quantitative computing perception [ 115 , 117 , 124 , 129 ], which can be regarded as the main line of domestic research in this field. At the same time, after a careful screening of domestic research literature, it can be found that a considerable number of articles on the topic of “information fusion, situational awareness” only stay at the micro-cognitive level (which is generally different from foreign literature based on the improvement of Endsley’s model [ 36 ], JDL model [ 40 ], and OODA model [ 45 ]), that is, more data sources are integrated from the bottom up instead of the top down. However, these first partial then overall studies have also made remarkable progress and have played an obvious role in promoting the whole field. By summarizing about 100 articles among core journals in the CNKI, the research focus of these articles is mainly concentrated on five aspects (the summary of the key research contents in these five aspects and the typical article representatives are listed in Table  4 ):

The definition or explanation of concept [ 17 , 19 , 97 , 98 , 100 , 101 , 102 ]

The intrusion detection data fusion [ 103 , 104 , 105 , 106 , 107 ]

The active evaluation model attempt [ 96 , 101 , 108 , 109 , 110 , 111 , 112 , 113 , 114 , 124 , 125 , 126 , 128 , 129 , 132 , 143 , 153 , 154 , 155 , 159 , 160 , 161 , 162 , 177 ]

The systematic evaluation after quantification [ 102 , 109 , 115 , 116 , 117 , 121 , 122 , 123 , 124 , 173 ]

The implementation of design and application in special fields [ 92 , 118 , 119 , 120 ]

(1) The research on the definition or interpretation of the concept mainly focuses on two aspects: one is the basic conceptual explanation, and the other is the practical significance of network security situational awareness in special field after merging with other subjects. The basic conceptual explanation is mostly found in the summary literature, such as the definition of the basic content and research category in literature [ 100 ], the description of the concept of intrusion detection in literature [ 19 ], and the definition of the network security situation perception by the literature [ 17 , 19 , 97 , 98 ]. Prior to achieving multisensory integration with other disciplines, it is necessary to do the abstract definition, which can explain whether the integration is effective, and the effect after the combination, such as the definition of color Petri net (CPN) in literature [ 101 ] and the definition of risk propagation model in [ 102 ].

(2) The fusion and utilization analysis on IDS includes two aspects: the collection of more complete data sources and the integration and utilization of multiple types of data. In the collection of multi-data sources, there is a good display in the evaluation framework of literature [ 103 ]. Li and Lan [ 104 ] combine data attributes with time attribute and space attribute, which is beneficial to the evidence fusion of subsequent data; there are lots of articles for multi-type data fusion; literature [ 105 ] combines multiple IDS and manual survey techniques, and studies its optimal allocation and strategy based on game theory. Ren et al. [ 106 ] puts forward an intrusion detection model based on data mining and ontology, which can cluster and classify the underlying alerts, discover and filter attacks, and then based on the established ontology attack knowledge model, correlate these attacks to identify, track, and predict the effect of multi-step attacks, such as the fuzzy clustering anomaly intrusion detection method in literature [ 107 ].

(3) The attempt of the active evaluation model mainly revolves around the attack model, and each article usually contains three components: model definition, model solving algorithm, and solution result. The definition of the model is generally combined with other disciplines, such as Petri network [ 96 , 153 , 154 , 155 ], game theory [ 108 , 124 , 159 , 160 , 161 , 162 ], and Bayesian network [ 114 , 132 ], and some articles also focus on the improvement of model description ability [ 125 , 126 ]; the solution algorithm depends on the definition of the model, and it is generally shown together with the solution result. There are lots of literature [ 109 , 110 , 111 , 112 , 113 , 114 ] trying to improve on this point, such as the reachable path analysis based on attack graph [ 101 , 128 , 129 , 143 , 177 ], defense strategy analysis [ 111 , 124 , 161 ], and survivability analysis [ 126 ].

(4) There are three main parts in the systematic evaluation after quantifying: systematization of evaluation index, index quantification, and quantified results and its application. The research on the systematization of evaluation index and the quantification of corresponding indicators mainly proceed from two angles: security attribute and attack behavior. From the perspective of security attributes, it is more focused on the definition and interpretation of network security. For example, Wang et al. [ 121 ] propose an attack technology classification method to meet the Amoroso classification standard; from the perspective of attack behavior, most of the researches take the attack as the center to quantify the important factors in the attack process. According to the statistics and analysis of the existing literature, the quantification of the 3 elements (attack severity, attack occurrence/success probability, and attack income) has basically formed a certain standard [ 102 , 122 , 123 , 124 ]. On the basis of index system and index quantification, risk assessment algorithm can be developed to get the perception or evaluation result [ 109 , 115 , 116 , 117 ].

(5) The active participation of all parties will definitely promote the production of relevant research results and deepen the application in the industry. The emergency response of China’s network security follows the PDCERF methodology (the preparation, detection, eradication, suppression, recovery, and tracking of 6 stages). A large number of practical products and systems have been put into use, such as information sharing and analysis center, large network security events coordination early warning positioning and rapid isolation control, security event planning system, large-scale network security state simulation platform, linkage system, and backup and recovery system [ 92 ]; on the combination of industry applications, similar to foreign countries, it mainly focuses on two aspects: ICS [ 118 , 119 ] and ECPS [ 120 ].

2.4 Summary of the present research

This section summarizes the research history, development stage, and present situation at home and abroad of network security situational awareness. In general, in the background of winning the commanding heights of network security strategy for all countries, the research on this aspect is of great significance and has made considerable progress, but the result of the study is still on the path of exploration, and the main problems are concentrated in three aspects.

Firstly, there is no comprehensive analytical perspective in terms of concept and ideology. Foreign researches mainly focus on the instantiation of situational awareness in this field, and domestic researches concentrate more on the integration of more information and efficiency improvement. However, according to the summary of Table  1 in this paper, network security situational awareness is a more advanced stage of network security research. It is not a model or a method. It should be a more valuable framework from all the existing network security concepts or means.

Secondly, there is no practical deep integration at the level of model and algorithm. Both foreign and domestic articles on models and algorithms are over 70%. Although multidisciplinary integration is an important breakthrough in this field, after the groundbreaking formulation, most of the articles begin to model and algorithm optimizations blindly. This is incorrect since these improvements should be carried out on the basis of integration practice. In addition, fusion perception must be a process of multiple cycles between information and decision-making. Most of the existing models are unidirectional, and the level of feedback effect should be effectively embodied in the model after perception decision.

Thirdly, there is no meaningful horizontal comparison in terms of effectiveness and application level. Every article or model will be verified by experiments, but few articles are compared as a whole. The existing and previous literature are more compared in the complexity of the algorithm, and the result of perception is a comprehensive synthesis of intelligence. It is different to judge directly for so many constraint factors, and the current application value comparison should focus on the horizontal comparison within a certain stage based on a standard data set.

The following chapters are arranged as follows: The second section abstracts the experiment object from the actual network topology and configuration of a medium scale software company to ensure the accuracy verification and relative comparison in the following chapters under the same standard. In the third section, from the perspective of system engineering, the network security situational awareness analysis is divided logically and gives out a new reasonable frame. From the fourth to the eighth, each segment of the whole framework is expounded, focusing on the role of this segment, the mainstream method, the application results on the experimental network, and the horizontal comparison between the methods within a certain segment. The ninth section briefly introduces the research dimension and direction of network security in a big data environment. The tenth section is the summary of the full text.

3 Experimental basis

In order to effectively compare and summarize the different methods in different stages of the proposed framework, this section first briefly introduces the experimental environment used in this paper as the basis for subsequent chapters. A medium-sized software development company is chosen as an experimental object. Figure  2 is the network topology graph of the enterprise. The network God is used as the monitoring device between the internal and external networks through the dedicated telecommunication lines and the external network links. 10.10.0.10 is a web server which provides the function of publicity website and product demonstration. 10.10.0.140 is a log server that can be accessed from the external network (because company personnel are often on business trips, both internal and external network access are required to go through the external network). 10.10.0.15 is the company’s database server, running SQL Server, Oracle, the two relational databases, and a non-relational database MongoDB. 10.10.0.16 is the test server, and the products the company has delivered and is developing have the latest version of the deployment on the test server. 10.10.0.11 is the internal development server. All the company’s source code and important project solutions, process information, etc. are all on this server. The company has a development team of about 100 people, which is mainly divided into two categories due to the different development technologies. 10.10.0.58 represents the technical team developed by.net, and 10.10.0.59 denotes the technical team developed by Java.

The graph of experimental network topology

4 Logical analysis framework

Network security situational awareness usually involves multiple different phases, and the systematic approach is preferred to process the data related to cybersecurity. There are two main methods for logical division: the first method is the engineering hierarchical method (such as Figure 2 in literature [ 45 ], Figure 3 in literature [ 97 ], Figure 1 in literature [ 103 ], and Figure 4 in literature [ 126 ]) and the second is the conceptual hierarchy (such as Figure 3 in literature [ 45 ] and Figure 1 in literature [ 14 ]), but neither of these methods can provide an easy-to-understand architecture from the perspective of the data processing stage. From the perspective of data value chain, the present paper adopts the systematic engineering method which is widely accepted by industry to decompose the typical cybersecurity situational awareness process into five continuous processing stages, including element acquisition, model representation, metric establishment, solution analysis, and situation prediction, as is shown in Fig.  3 below.

The element acquisition phase is concerned with how to effectively obtain the security-related data as much as possible, which is mainly divided into two tasks: data acquisition and data preprocessing. Data acquisition refers to the effective storage process including collecting configuration information in the network, behavior information in the log, and vulnerability information which can be achieved by using a scanner, a sensor, or a specially written tool. Data preprocessing is a process of regularizing original data before data modeling or analysis and utilization.

The model representation stage is focused on the correlative expression of the effective elements, which is mainly divided into two tasks: element reduction and formal representation. According to the purpose of the analysis, it is necessary to reduce the acquired objects effectively during the element acquisition process in order to achieve the efficient analysis. The formal expression refers to the process of precision abstraction including the attributes of the reduced elements, the relationship between the elements, and the order relationship.

The metric establishment stage is the process of refining the value of each element object before the solution analysis, mainly including the quantitative classification and evaluation index system to determine two tasks. The quantitative process is a process of numerically assigning the attribute values of each element (in this present paper, the qualitative classification is treated as a special quantitative classification without special explanation), and the confirmation of the evaluation index system is the process to regularize the logical relationship between the attribute values of the elements.

Solution analysis is the algorithmic process based on the first three stages mentioned above, which mainly includes three tasks: the determination of the solution algorithm, the verification of the correctness of the algorithm, and the improvement of the algorithm. The solution algorithm is the process of effectively combining the target with the model and the metric to ascertain the analysis step. The correctness verification of the algorithm is to validly correspond to the input and output of the algorithm. On this basis, the efficiency of the algorithm should be considered to improve in order to expand in the true scale network environment.

Situation prediction is a process of comprehensive evaluation and decision-making based on the analysis results, which mainly includes two tasks: result visualization and decision-making after knowledge application. The result visualization is the process of presenting and constructing the solution results in an easy-to-understand way. After the analysis and decision-making, the feedback loop will be applied to the current network for cybersecurity reinforcement (such as vulnerability repair and configuration upgrade) to complete a perceptual loop.

Network security situational awareness operation mechanism

5 Phase I: Element acquisition

The function of the element acquisition phase is to effectively capture the key data used in each phase of the cybersecurity situational awareness. In general, element acquisition refers to the collection of all the elements related to cybersecurity. In the narrow sense, element acquisition refers to the collection of the elements involved in a certain perception process. The purpose of this present paper is to sort out the basic framework of cybersecurity situational awareness, and the core implementation methods of each stage are compared horizontally, so the element acquisition in this current paper refers to the generalized element acquisition.

Undoubtedly, element acquisition is the premise of cybersecurity situational awareness. Other subsequent stages are unable to work without basic data collection. Most of the documents collected so far have clearly defined the functions and important impacts of this stage in the logical description of the framework. However, as for the implementation, most of them only mention the data acquisition through automated scanning tools or sensors, and according to the following-up model to directly stipulate or preprocess, there are also some literature introducing the way to obtain data or tools [ 33 , 34 , 35 ] and so on. Strictly speaking, element acquisition is divided into three parts: data generation, data acquisition, and data preprocessing. In light of the division of logic analysis framework in Section 3, data preprocessing is generally carried out after the model definition or measurement establishment phase. Data acquisition is generally completed by combining manual and automatic methods. The focus is generally on the development of automated tools. This section focuses on the classification of data from the perspective of data generation.

In the existing cybersecurity situational awareness literature, the basic data collection part is mostly according to the needs of model analysis to reverse the data used (narrow element acquisition), which is not conducive to data standard unification and model-to-model comparison verification. According to the logic of engineering, this present paper briefly summarizes and classifies the data in cybersecurity analysis from the perspective of data generation.

Here, the data is divided into two categories: static data and dynamic data. Static data refers to data that does not change substantially in a cybersecurity situational awareness analysis cycle shown in Fig.  3 . Dynamic data refers to changes in the cybersecurity situational awareness analysis cycle shown in Fig.  3 as the analysis process going on. As shown in Table  5 , the static data mainly includes host information (such as host IP address or MAC address unique identifier, running service or program, file, data and other confidential assets, operating system, hardware composition, system configuration, and permission configuration), network information (such as network device information, network topology information, protocol information, firewall information, and network configuration information), and IDS information (such as basic information of intrusion detection system, expert knowledge base, and alarm information), and the dynamic information mainly includes activity information (such as source address, destination address, and activity description), behavior information (such as source address, destination address, protocol in use, transmission data size, and compression algorithm), vulnerability information (such as vulnerability name, logo, basic information such as release time, vulnerability host information, attack methods, attack effects, and repair methods), attack information (such as attack source address and attack method), and perceived result information (e.g., perceptual result information of the last perceived loop and the action information after perception).

6 Phase II: Model representation

Formal modeling is the key link in the cybersecurity situational awareness operation mechanism. The description ability in the modeling stage of reduced state and formalization will directly affect the subsequent perceptual analysis results. Through the summary of the existing literature, the cybersecurity situational awareness model is mainly divided into three categories: mathematical model, stochastic model, and biological heuristic model. The core concepts and typical representatives of each classification are shown in Table  6 below.

6.1 Mathematical model

The mathematical model is used to analyze the cybersecurity situational awareness. The main idea is to use mathematical language or mathematical symbols to summarize or approximate the security-related features or quantity dependencies of computer network systems. The mathematical model here refers to the mathematical model in the narrow sense, that is, the mathematical expression of the relationship between variables in the cybersecurity system. Therefore, the perceptual analysis method based on a mathematical model is more biased towards the form of quantitative analysis. It mainly includes analytic hierarchy model, Bayesian model, fuzzy set/rough set model, reliability/survability model, etc.

The Analytic Hierarchy Process (AHP) was proposed by Professor T.L. Saaty and is now widely used in decision-making. Chen et al. [ 99 ] proposed a hierarchical security threat assessment model (Fig. 4 is the model results obtained by the experimental network according to the method in literature [ 100 ]), and Fig. 5 is Tomcat service, FTP service, and the overall security situation of each host and local area network are security situation quantification results, based on the subjective quantization method in literature [ 99 ]; Tomcat service, FTP service, and the overall security situation of each host and local area network are security situation quantification results. The hierarchical model is consistent with the decision-maker’s thinking process in both the analysis and the calculation process, which ensures the results are intuitively understandable (for example, the security situation index is relatively high in Fig.  5 at around 17:30; because most people fill in the logs around this moment, the frequent external network mapping will lead to higher security risks). The construction of an effective hierarchical structure is the key to the application of this model, and some literature has studied the instantiation of the hierarchy [ 127 ], but the current element quantization process basically adopts the subjective experience value method, which cannot be compared and quantified between every two factors in the classical analytic hierarchy process, thus leading to the lack of objectivity, and the current hierarchical structure is only suitable for the local area network which contributes to the difficulty in carrying out large-scale promotion, as well as no effective prediction of the future situation.

Hierarchical structure partition diagram of experimental network

Hierarchical security situational awareness results of experimental network. a Threat situation of Tomcat on server. b Threat situation of FTP on server. c Threat situation of host level. d Threat situation of system level

In order to effectively reflect the uncertainty and subjective elements in the cybersecurity situational awareness analysis, the probabilistic method is usually used for quantitative description [ 128 , 129 ], in which Bayesian logic is the most commonly used model. The relationship rules and mathematical reliability of Bayes are very similar to those of human thinking reasoning. Bayesian calculation can synthesize the latest evidence information and prior information to ensure that the calculation results maintain two important characteristics: continuity and accumulation. There are literatures adopting Bayesian mathematical methods for cybersecurity situational assessment [ 131 ], but most of them are used as quantitative computing tools in combination with other models, especially the combination of Bayesian and attack graphs [ 114 , 130 , 132 ], combining graph theory and probability theory to complete a Bayesian network, using graph theory to show the structure and interdependence at the qualitative level, and using probability theory to carry out quantitative expression and reasoning at the quantitative level. Some progress has been made in this perspective, but the Bayesian network is a decomposition form of the joint probability distribution at the theoretical level. The variables in the actual solution are not independent from each other, and the joint probability is too complex to suit the large-scale networks.

The fuzzy set contraposes the traditional set. In the traditional set, the relationship between the object and the set is clear (either one or the other), but in reality, some objects do not have a clear affiliation of the set, There exists an interval of degree of membership (membership function). Some literatures apply fuzzy similarity and fuzzy comprehensive evaluation in cybersecurity situational awareness analysis [ 133 , 134 ]; the rough set extends the classical set theory, which uses the upper and lower approximations to approximate any set, and it can analyze incomplete information such as inaccuracy, inconsistency, and incompleteness without prior knowledge, discover hidden knowledge, and reveal potential laws. Zhao and Xue [ 135 ] and Kong et al. [ 136 ] utilized the idea of rough concentration mode classification in the cybersecurity situational assessment, using each security evaluation index as the condition attribute set C, and determining the decision attribute D of the load situation assessment result according to C and then according to the D synthesis comprehensive security situation network. However, the current research in this area is limited to describe the uncertainty in the process of fuzzy sets or rough sets, and it is impossible to combine the target or core problem of cybersecurity situational awareness with the fuzzy set or rough set method. The practicability and the continuity of research are limited. In combination with other models or methods, it is generally carried out at a certain point in the analysis process and adopted more as a quantitative tool for uncertainty.

Feng et al. [ 137 ] combined the reliability theory with the vulnerability analysis process to quantify the security of the distributed system. It is intended to ascertain the system maintenance probability of the security state under the specified conditions and the specified cost c through the reliability function Rs( c ). Figure  6 below is the vulnerability state modeling result of the attack on the Ftp service on 10.10.0.11 (internal development server) according to the literature [ 137 ], and the average attack cost for this service is E ( C ) = 1/ λ 1  +1/ λ 2  +1/ λ 3  +1/ λ 4  +1/ λ 5 . In literature [ 138 ], the mathematical conditions are used to obtain the criteria of complete probability control or partial probability control of complex attack networks. It is theoretically proved that if there are effective defense nodes in the network, the complex network can still provide normal service when it is attacked and destroyed and suggests ways to defend against node selection and control networks. The advantage of adopting the reliability or survivability model for cybersecurity situational awareness analysis is that there is a mathematical derivation process to ensure the rigor of the analysis, but the preconditions of these formulas also greatly limit its large-scale network conditions of actual perceptual analysis, the diversity of influencing elements in the real network often makes the calculation result unsatisfied, and the model generally cannot provide the repair method after confirming the network insecurity state, so that the system has the ability of active defense.

Modeling results of reliability quantitative model of experimental network

6.2 Stochastic model

The stochastic analysis model is a non-deterministic model. Its main feature is that the exogenous variables in the model will change with specific conditions, which has a high degree of fit with the occurrence of cybersecurity-related behaviors. During the attack, the choice of the attacker’s assault means the choice of the defender’s resist strategy and the normal user’s operation are random. Using a stochastic model for cybersecurity situational awareness, it is possible to describe the logical relationship between the random behaviors and behaviors of various elements of the system more clearly, and thus, it is easier to fully describe the network status, and it can also include the influence of unknown behavior, based on Stochastic model cybersecurity situational awareness is the focus of current academic circles, including attack tree/graph model, Petri net, game theory, and Markov’s model.

The attack tree model was proposed by Scheier [ 139 ] in 1999. It can be seen as an extension of the fault tree, which is intuitive and easy to understand, but the description capabilities are limited. The attack graph model was first proposed by Swiler and Phillips [ 5 ] in 1998. It is currently the most widely used method. Sheyner et al. [ 140 ] adopt the model detection method to generate the attack graph, and Ammann et al. [ 61 ] generate an attack graph through the idea of graph theory which starts from the initial state and searches forward. The literature [ 141 ] focuses on the attack, and a tool for generating an attack graph is given. There are also literature focuses on large-scale construction and visualization of attack graphs [ 142 , 143 ]. Early attack graphs tend to construct state attack graphs [ 5 , 61 , 140 , 141 , 142 , 143 ], but it is easy to cause the explosion of state space. As the research progresses, it tends to construct the causality diagram [ 144 ], and its edges represent the connection relationship between nodes or the logical relationship of atomic attacks, which is more scalable and easier to use for large-scale networks. Figure  7 is the result of the attack graph of attacker Eve attacking the FTP service located on the development server (10.10.0.11) in the experimental network in Section 2. Figure  7 a is a graphical description, and Fig.  7 b is a formal description of the attack step. The advantages of attack graph model is directness and descriptive and is easy to combine with other methods which are the currently basic model of cybersecurity situational awareness analysis; the current research focuses on the refinement of the original [ 125 ] or improved model [ 145 ] to enhance the description ability and fusion with other disciplines [ 11 , 146 ] and thus to enhance the analytical ability.

An attack map for FTP on 11 servers in the experimental network. a A graphical description of an attack graph. b Formal description of Apache attacks in attack steps

Models similar to the attack graph also include privilege graphs and state transition graphs. Dacier [ 147 ] abstracted the nodes in the graph into the permission state and proposed the privilege graph model. Ortalo et al. [ 148 ] established the Markov model based on the concept of privilege graph and presented the security evolution process. Dr. Wang Lidong [ 149 ] refined this process, but the privilege graph model is difficult to describe the dependencies between states or random events, so subsequent research on the extension of this model has little influential results; Porras and Kemmerer [ 150 ] proposed the intrusion detection method based on state transition graph for the first time. Each node in the graph represents a temporary state of the system, and the edge represents the state transition and transfer process. The probabilistic model in literature [ 151 ], the semi-Markov process model in literature [ 152 ], and so on are all the extensions based on it. The advantage of the state diagram is that it is more descriptive, but there are problems of state space explosion under large-scale networks, and the solutions to this problem [ 128 , 143 ] are still not satisfied.

Petri Net (PN) was first proposed by Karl A. Petrie in 1962 to perform effective mathematical simulations of discrete parallel systems. It consists of three elements: place, transition, and the directed arc (Arc); N  = ( P , T ; F ) can have any number of tokens in the place to represent the resource (Token), and the initial application scenario is through the flow of Token in the place to detect the protocol Error (deadlock state). In the combination of Petri net and cybersecurity situational awareness, the place P usually represents the descriptive local state of the system. The transition T represents an attack event or normal activity that can change the state of the system. The directed arc F effectively associate the local state and the event. On the one hand, it refers to the local state that can cause the change to occur, and on the other hand, it points to the change of the local state caused by the change. The following Fig.  8 shows the experimental network in the second section which is the Petri net model modeling result of the FTP service attack for 10.10.0.11. Compared with the classic Petri net, the place is not a Token, but the probability of a transition occurring in a local state. The number attached to the transition represents an attack or success probability, on this basis, qualitative reachable identity analysis or quantitative analysis by correlation matrix, state equation, etc., for example, using the “or” principle of maximum risk estimation (maximum probability between different paths) and the probability of the intermediate place P7 is max (0.4 × 0.4, 0.7 × 0.5, 0.8 × 0.1) = 0.35. It can be seen that Petri net not only has the characteristics of intuitive and vivid of graphical modeling, but also is more suitable for asynchronous and parallel attack process. The research progress in this direction includes coloring Petri nets with increased model description ability [ 153 ], a stochastic Petri net with increasing random occurrence time for transitions [ 154 ], a fuzzy Petri net described for uncertainty in the modeling process [ 155 ], etc.

Petri net modeling results for FTP attacks on 11 servers in the experimental network

With the deepening of cybersecurity situational awareness research, researchers have realized two problems: First, the cybersecurity confrontation process is not simply a technical matter, and different people who apply in different scenarios will produce the opposite result with the same technology implementation means; Second, the analysis of cybersecurity must not be the behavior of one party. In an environment with active defense, the security situation will variate on the choice of two or more parties, which has a very high degree of agreement with the strategic dependence of game theory. Once proposed, it has become a hot topic of research [ 156 ]. Traditional research on intrusion detection or aggression behavior is based on a game analysis [ 157 ]. Considering the application in the real environment, it is certainly a repetitive multistage incomplete information dynamic game [ 158 ], and there is a refined Bayesian Nash equilibrium. Each cybersecurity situational awareness model based on game theory contains at least five parts: N  = {1,2,…, n } is a collection of people in the game (generally combines multiple similar objects and divides them into attackers, defender, and normal user | N | = 3). S  = { S 1 , S 2 ,…, S k } is the set of game states in the offensive and defensive process. θ  = { θ A , θ D }is the set of action strategies of both offense and defense. P is the transition probability between game states S . R n  =  S i  ×  θ  ×  S j ∈ (−∞,+∞)), which represents the income function of the person n in the state S i transitioning to the state S j ; GM  = { N , S , θ , P , R }, according to this basic definition, after a finite-step ( k -step) game process, the system transforms between different states to form a tree structure, the goal of the player is to make their function maximized, and the model’s Nash equilibrium strategy f * can be obtained by means of Shapley algorithm or problem transformation solution [ 162 ]. The combination of game theory makes the focus of cybersecurity situational awareness rise from technology to management strategy and can portray the psychological activities of each participant, which greatly improves the description ability of the model and the scientific nature of the analysis results. The improvement direction focuses on static game turning to dynamic game [ 159 ], model-related element quantification [ 10 ], or combining with other methods [ 160 ] and presents practical application effects [ 108 , 124 , 161 ], etc.

The basic idea of the Markov model is that the transition of the next state is only related to the current state but not the historical state. The Markov model consists of three elements: S is the set of non-empty states composed of all possible states of the system, P is the system state transition probability matrix, and Q is the initial probability distribution of the system, M  = { S , P , Q }. The intention of applying the Markov model to the cybersecurity situational awareness is to predict the attack and defense evolution effectively when the initial conditions are met, but there will be a large number of camouflage attacks or covert attacks during the attack. Forcing the application with inefficiency will lead to the extreme result of statistics (overexaggerating the impact of a certain accident or neglecting the impact of a key step), so Markov is generally combined with other models [ 53 , 109 , 162 ]. To obtain causal knowledge through Markov’s method, and to simplify the operation process by one-step transition probability matrix, the model can be performed efficiently under large-scale networks.

The risk communication model is proposed by Zhang et al. [ 102 ], whose core idea is that the risk of a network subject will spread to the object with non-vulnerability or even the whole network because of the high relevance of the network system, so it needs effective means to effectively evaluate the risk state of the whole network information system. The risk communication model (vulnerability diffusion model) is generally composed of two parts: network abstraction and propagation algorithm. The network abstraction describes the logical access relationship structure of the system, and the propagation algorithm describes the rules of risk diffusion. Figure  9 below is the result of abstract modeling of the risk diffusion logic access to the development server (10.10.0.11), the database server (10.10.0.15), and the test server (10.10.0.16) in the second section of the experimental network, after the attacker’s attack on the web server (10.10.0.10), in which the weight of the directed edge represents the attack revenue. If we use the cumulative effect algorithm that ensures the optimal result of the final risk diffusion to determine the diffusion value between the nodes λ uv , that is \( {\lambda}_{uv}=\frac{w\left(u,v\right)}{\sum \limits_{m\in N(v)}w\left(m,v\right)} \) where w ( u , v ) represents the weight between nodes u and v ; we can get the results shown in Table  7 below. From the result, we can see that the risk state of the network is not only related to the object with vulnerabilities, but also related to the logical access structure and the distribution state of the vulnerabilities, and the risk propagation model can be used to identify the most security threats or risk propagation paths.

Logic access modeling results of risk diffusion for experimental network

6.3 Biological heuristic model

The intelligent computing method, which is inspired by the natural phenomena or processes of nature, is called the biological heuristic calculation method. The basic principle is to explore the solution of a problem combined with the known information and to effectively record and accumulate related information during the exploration process and guide the next move and correct the previous steps, and then get better overall results. The attacker’s attack process and the defender’s defense process are also the same. They are all based on the current knowledge state to seek the maximum benefit at the least cost. This promising approach can be regarded as the specific application of artificial intelligence in the field of cybersecurity situational awareness. At present, the research is in its infancy, the high-dimensional and non-linear data in the offensive and defensive process are abstracted, and the results of the solution through heuristic calculation are tested and improved in terms of feasibility and optimality. Models that have made some progress include neural network models and artificial immune models.

The general method based on neural network is to use the collected real-time security status indicators (such as vulnerability information, attack methods, and defense methods) as the input vector X , and regard the indicators of situation awareness results (such as confidentiality and integrity) as the output vector Y . In this regard, a non-linear mapping from X to Y is constructed by effective training [ 163 , 164 ]. Literature [ 165 ] introduced the neural network learning method in IDS research, which greatly improved the accuracy of the alarm effectively. The literature [ 166 ] integrated the self-encoding network and deep belief network structure technology into the risk identification model and proposed a lightweight intrusion detection model which can reduce training time and test time to a certain extent and reduce the false alarm rate.

Computer immunology, which imitates the biological immune system [ 167 ], has been widely used in cybersecurity situational awareness analysis. The literature [ 168 ] proposed an immune model that applies the dynamic clonal selection algorithm to the network intrusion detection system. Based on the correspondence between the changes of antibody concentration in the human immune system and the invasion intensity of pathogens, Li Tao proposed an immune-based cybersecurity risk detection model [ 169 ], and an immune-based network monitoring model was established by the dynamic model of immune memory and the recursive equation of response [ 170 ]. In literature [ 171 ], the artificial immune algorithm is used as a multi-objective solution method for risk assessment, which shows the change of cybersecurity status under different attack strategies to some extent. However, as a new approach to cybersecurity situational awareness analysis, the immune model must fully mimic the mechanism of immunology to function. The complexity and agnostic of immunology will make the modeling and solving process more complicated. Whether it can effectively reflect the evolution of the security situation remains to be tested.

6.4 Combination and comparison between models

Table  8 shows the classification results of each model in 9 dimensions. It can be seen that there is no model that can meet the high standard requirements of more than 5 dimensions at the same time, which also indicates that the research on network security situational awareness is still in the exploration stage. For the formal modeling phase of model representation, there are two main improvement aspects: one is to improve or enhance the research for a certain model, such as in-depth analysis based on attack graph [ 101 , 143 , 146 ] and the application of fuzzy set ideas in the field of perception [ 107 ]. Most of them belong to the second category, that is, through the combination of models, the purpose of analysis can be achieved by means of the advantages of multiple models, such as Bayesian attack map [ 114 , 128 , 129 ], fuzzy Petri net [ 155 ], and Markov game [ 162 ].

7 Phase III: Establishment of metrics

The core purpose of metric establishment is to refine or quantify the value of each element object involved in cybersecurity situational awareness before solving the solution. According to the cybersecurity situational awareness operation mechanism in Fig.  3 , the metric establishment phase may occur after the formal representation of the model, or directly on the basis of element acquisition, so this phase is mainly divided into two cases: one is model element quantification and the other is the evaluation system and index.

7.1 Model element quantification

In the process of formal modeling in Section 5, the relevant elements have been defined in detail. To conduct the solution analysis needed for cybersecurity situational awareness, it is also necessary to quantify each element in the model (from the perspective of model description ability, the process of quantifying the value of elements is also the process of describing the refinement of capabilities). Therefore, this stage has a strong correlation with the idea of model construction. Through the existing literature statistics and analysis, it is found that the models are focused on different points, but each model contains a description of the attack behavior. The quantification of the three elements of attack severity, attack occurrence, success probability, and attack revenue has basically formed certain standards or norms.

The metrization premise of serious attacking is the qualitative classification of attack types. The variety of cyber attacks leads to different types of attack. At present, the six-member representation method, proposed by Christy [ 122 ], has strong practicality and has been accepted by most people. Based on the qualitative classification method, it is divided into several levels to quantify the severity of the threat [ 102 , 124 ]. This method is generally associated with the alarm mechanism of IDS and is widely used in intrusion detection. The widely used method in the attack model is CVSS vulnerability evaluation mechanism [ 10 , 123 ], which is divided into three aspects: basic evaluation criteria, life cycle assessment, and environmental assessment. The final result is 0~1. The higher score indicates the greater threat to the vulnerability.

The purpose of quantifying the occurrence of attack/successive probability is to measure the authenticity of the attack or the possibility of successful attack. The network attack process is filled with a large amount of false and useless information. The information provided by each host and security device is often inaccurate; this brings great difficulty to the comprehensive estimation of the information fusion model. Currently, the subjective probability estimation method of experts is mainly used in each experimental model [ 10 , 128 , 162 ] (Tables  9 and 10 are the quantitative criteria used in the follow-up analysis of this article [ 124 ]), and the Bayesian network can effectively express the probabilistic reasoning of uncertainty knowledge, and thus in this research, Bayesian-based estimation methods [ 55 ] have also made some progress.

The quantification of the attack revenue is an important part of the attack effectiveness evaluation. Generally, the destructive size of the attack is qualitatively measured (for example, the attack acquires the root permission of a service [ 5 , 6 ], etc.), and then the quantitative value of the damage degree is given according to the qualitative classification. The quantitative research can be carried out from the perspective of the attacker and the defender. From the view of the attacker, the quantitative research refers to the return obtained by the attack under a certain attack cost, while the defender refers to the loss of the system at a certain defense cost. In general, the attack revenue is less than the network system loss. For the sake of simplicity, the defense loss is used as the attack benefit in most models [ 124 ]. This method is also adopted in the subsequent analysis of this paper.

7.2 Indicator system and index

The indicator system is used to evaluate and reflect a certain situation in a certain field and is widely used at all levels. Different from the point-based quantification of each element in the model, the cybersecurity situational assessment index system should proceed from the whole, intending to exhaustively classify the attributes related to the cybersecurity situational evaluation, giving the clear meaning of each class; the quantitative operation is carried out based on mutual related and complementary systematic indicators, and through the mathematical calculation method to obtain the cybersecurity situational index value to be evaluated, through the change of the index value to reflect the change of cybersecurity status.

The cybersecurity situational indicator system and index distract the network administrator’s concerns free from the scattered or massive log data monitoring; facilitate the intuitive response to the cybersecurity state, especially the relative number of changes help to find abnormalities better; and then confirm the main influencing elements and achieve effective protection. It mainly includes two aspects of work: one is to comprehensively and systematically ascertain the elements related to cybersecurity situational awareness (the evaluation system in Fig.  3 and the quantified parts of each metric element) and the second is to establish a mapping model between systemic elements and result index (mathematical analysis method and solution analysis part are confirmed in Fig.  3 ).

Based on the effective synthesis of the explanation of the specific meaning of network security and the study of reliability, Lin etal. [ 96 ], divides the attributes that are generally concerned about in security into five parts: reliability, availability, insurance, confidentiality and integrity, and gives the concrete content of each index in the field of security. Meaning and the way of quantification are discussed. Survivability goes beyond the concept of security. It quantifies the ability to correctly perform predetermined functions. It is the ability to provide normal services when the system is facing threats based on the security evaluation. Feasibility quantifies the operational performance of the network system in the event of possible failures, providing a comprehensive quantitative evaluation standard between security and system performance. Figure  10 provides a brief summary of the cybersecurity assessment indicators in literature [ 96 ].

Security attributes system and index calculation method

Based on the hierarchical index system [ 99 ], the literature [ 172 ] proposed a cybersecurity situational assessment method based on the configuration index system. In this method, the indicators are divided into three levels: comprehensive index, evaluation dimension, and situation element (as shown in Fig.  11 below). The cybersecurity situational comprehensive index is divided into five levels. The evaluation dimensions are mainly based on three dimensions: basic operation index (reflecting the safe operation of network equipment and services), vulnerability index (reflecting the vulnerability of the network itself in the absence of attacks) and risk index (reflecting the impact of network attacks on the network). Each dimension can choose different situation assessment factors. The proposed quantification methods for each factor are also given (e.g. the factor in the basic operation index is quantified by overload rate, etc.).

The index evaluation system based on configuration

8 Phase IV: Solution analysis

After the formal description of the model in stage II and the element refinement measurement in stage III, the fine-grained abstraction of the related perceived objects in the network is basically completed. The next step is the solution analysis, the core of the cybersecurity situational awareness, whose main aim is to analyze and calculate the corresponding models and data effectively, so as to obtain the qualitative or quantitative results which can reflect the network security status and express the mapping process from elements and their quantitative features to the judgment results of network security status. In some research papers, this part is generally touched upon in the form of “model solution algorithm.” In addition, some researches extend traditional methods in the field of cybersecurity situational awareness, and some introduce new theories and methods into this area. At present, more than 60% of the literature on cybersecurity situational awareness searched in China and abroad is targeted at the improvement of solution methods, trying to improve the accuracy and efficiency of the analysis results.

8.1 Classification of solution analysis method

Although there are various methods in the process of cybersecurity situational awareness, theoretically, they can be divided into three categories: formula analysis method, logical reasoning method, and information fusion analysis method, as is shown in Fig.  12 below.

Classification of network security situation aware solution analysis method

Formula analysis method is also called mathematical calculation method, the earliest one applied to cybersecurity situational awareness, including statistical description analysis and decision evaluation analysis. Statistical description analysis uses the basic mathematical statistics to reflect the network security status, such as the statistics of the number of real-time network security events [ 33 , 34 , 35 ], network congestion [ 35 , 92 ], and vulnerability top- k sorting [ 10 , 123 , 172 ], which has been widely used in network security monitoring systems at all levels. This method has high objectivity and strong maneuverability. However, it can only present the results, but cannot effectively retrospect the causes of the state. Decision-making evaluation analysis method is elicited from the multi-objective decision theory, relies on the first three stages of element abstraction and index system to construct the evaluation function, and obtains situation awareness results through the evaluation function. Dapoigny's fast calculation method of static statistical data [ 68 ], the formula of analytic hierarchy process (1) - (12) in reference [ 100 ], the formula (1) - (4) of fuzzy evaluation method [ 135 ], and the formula of average attack cost in reference [ 138 ] are all the application fields of this method in network security situational awareness. Formula analysis is generally used in conjunction with the mathematical model in stage II and is also the basis for the quantitative analysis of other solving methods in this section. The advantage of this method is that it can intuitively and visually perceive the results of the reaction and the computational complexity of polynomial content can be easily generalized in large-scale networks. However, owing to no unified criterion for the function evaluation and related parameters selection, the high subjectivity easily leads to a large deviation between the mapping Y  =  F ( X ) from the set of factor indicators X to the set of perceptual results Y and the actual situation.

In view of the shortcomings of formula analysis method, logical reasoning method has gradually become a breakthrough in problem-solving. It can gather uncertain information from multi-sources and with multi-attributes, simulate human thinking modes, and obtain intelligent evaluation results, including rule reasoning method, graph model reasoning method, and pattern recognition method. Rule-based reasoning method is developed from the rule-based expert system. It can be tentatively solved by imitating the association reasoning ability of experts. In the field of network security situational awareness, it is mainly combined with intrusion detection system to improve the efficiency or accuracy of intrusion detection such as the model for compound attack mode detection proposed by Bao et al. [ 28 ], the effective parameter selection method put forward by Ilgun et al. [ 29 ] based on the category principal component analysis, the multifunctional simulation platform proposed in literature [ 32 ], an ontology-based attack knowledge model established in literature [ 106 ] by clustering and classifying the underlying alerts, and the hierarchical intrusion scene reconstruction methods brought up by Fu et al. [ 127 ]. Graph model reasoning analysis is one of the most effective methods to explore the correlation of related elements in network security situational awareness. The knowledge of logic relation, reasoning method, and probability calculation is included in the state transition of directed graph. Attack graph model [ 5 , 101 , 112 , 128 , 129 ], Bayesian model [ 11 , 62 , 114 , 132 ], Markov model [ 53 , 109 , 162 ], and so on all adopt this method. The solution method mainly includes two steps: reachability analysis and quantitative calculation analysis. The reachability analysis mainly explains whether the current network system or a service component has the possibility of being attacked, including the analysis results such as attack reachability and attack path. Figure  13 below is the result of the reachability analysis of the internal development server (10.10.0.11) in the experimental network using the analysis method in reference [ 5 ]. It can be seen that (a) the file on the server 11 is likely to be attacked and (b) there are nine attack paths (left 3, middle 2, right 4) in Fig.  13 . These attack paths can be attacked. They fall into 3 categories (Fig.  13 , left, middle right). On the basis of reachability analysis, the quantitative computational analysis provides comparative criteria such as maximum attack probability [ 6 , 129 , 132 ], maximum attack revenue [ 102 , 146 ], and minimum cut set analysis [ 115 , 116 ]. The number in each node in Fig.  11 below is the result of using the method of quantifying model elements in phase III to evaluate the attack benefit [ 124 ]. Using the algorithm of maximum reachability in literature [ 129 ], the maximum probability path in each attack path can be known (represented by the dotted lines in Fig.  13 ). The analysis process of graph model reasoning is clear, which conforms to human logical thinking and is easy to understand, but it also increases the complexity of reasoning (such as large storage cost of graph and reasonableness of uncertain representation). Therefore, the promotion of graph model reasoning in large-scale networks is the most important breakthrough of this method. With the development of machine learning, the pattern recognition method is used to solve the perceptual process in which the relationship between the factor index set X and the perceptual result set Y cannot be established by function or logic reasoning. It uses the historical monitoring data (including both the factor data and the result data) as the training sample to determine the situation template and evaluates the situation by the implicit pattern matching. The combination of intrusion detection and unknown attack detection has made some progress [ 7 , 27 , 168 ]. However, this method cannot provide scientific evidences for the results of perception because of the large amount of calculation, and it is still far from the actual use.

Inference analysis results of experimental network diagram model

Formula analysis method and logic reasoning method have their advantages and disadvantages. There is no general solution method to solve all the problems encountered at present. Therefore, the original intention of information fusion analysis method is to combine the advantages of various solution methods and try to use the solution method in a complementary way. One is to provide more data sources and obtain more accurate perception results through data diversity and association degree on the premise of the basically unchanged solution method. For example, Bass [ 7 , 8 ] integrates the heterogeneous distributed network sensor data into intrusion detection system, and Yong et al. [ 103 ] bring the vulnerability information and service information together for the theory of multi-source fusion through D-S evidence. Moreover, in literature [ 32 ], the real-time perceptual slicing and its fusion methods are introduced. The other is to take the mutual complement of solution algorithms on the premise that the input elements and measurement values are basically unchanged. For example, Poolsappasit et al. [ 11 ] combine the Bayesian network with the qualitative causal analysis of attack tree/graph to form a multi-objective optimization platform. Furthermore, the concept of fuzzy centralized credibility is introduced into the Petri net model in literature [ 155 ] and is evaluated by the hierarchical method. Zhang et al. [ 162 ] combines Markov’s inefficiency analysis with the attack-defense game and proposes a security situation evaluation algorithm with three sub-algorithms.

Table  11 compares the results of the three categories of the six methods of solution analysis in this section. The comparison is made from seven dimensions: time complexity, space complexity, generality, scalability, number of articles, the visual property of the analysis results, and the degree of difficulty in understanding of the analysis results.

8.2 Verification and optimization

After stage IV, the first four stages are usually validated in the way of experiments. The verification work is mainly divided into two parts: one is to verify the validity of the model abstraction and the other is to verify the rationality of the analysis results. The validity verification of model abstraction is to judge whether the formal expression of network elements and their associations tally with the actual situation of the experiment and to verify whether the initial results of the solution analysis are in line with the current network security status. The rationality of the analysis results includes not only the correctness verification of the solution method in Section 7.1, but also the verification concerning the conformity of the initial results of the analysis to the real security state of the current network.

Validation is the comparison between the experimental results and the expected objectives in the model, and optimization is the comparison of the descriptive ability, solution efficiency, and analysis results between models. Some researchers have improved the formal abstraction in order to describe the key elements of network security situational awareness more concisely. For example, Ammann et al. [ 59 ] proposed a more concise and extensible model based on the core concept of attack graph. Hamid et al. [ 125 ] combined the take-grant protection model with attack graph and refined the node granularity to component level. Besides, Luo et al. [ 110 ] constructed the hierarchical attack graph based on the underlying data to improve the accuracy of intrusion intention detection, and characterized the random strategy selection of attack and defense parties by game theory, which is targeted at making the analysis results more accurate or reduce the complexity of the algorithm to adapt to large-scale networks. Poolsappasit et al. [ 11 ], based on the risk management framework of Bayesian network, can ensure to obtain more decision information under resource constraints. Wu et al. [ 101 ] proposed an attack-based framework. In literature [ 112 ], the problem of the optimal compensation set is transformed into a single-weighted collision set to solve the problem. It is proved that the method based on such transformation has better performance. The attack graph simplification algorithm and the maximum reachable probability algorithm in literature [ 129 ] can be better adapted to the large-scale complex network. In addition, Yun et al. [ 143 ] raised an automatic attack algorithm for large-scale networks. There are also researches that aim to improve both the formal abstraction and the algorithm to obtain better analysis results. This aspect is more a combination of the formal method in Section 5 and the solution method in Section 7.1. For example, in literature [ 11 , 114 ], the combination of Bayesian operation and attack graph is used for dynamic security risk assessment. Moreover, Dietterich et al. [ 64 ] applied the theory of machine learning in the process of network security situational awareness, the combination of Petri nets and fuzzy sets [ 155 ], and the combination of game theory and Markov [ 74 , 162 ], as well as the comprehensive application of information fusion methods in network security situation [ 7 , 67 , 74 , 103 ].

9 Phase V: Situation prediction

According to the stage division of the operation mechanism of network security situation awareness in Section 3 of this paper, the last stage is situation prediction, whose core role is through knowledge application to enhance network security and form feedback loop process on the basis of the analysis results obtained in the first four stages. However, most of the literature on this stage is missing, and in a simple experimental network or some special scenarios, the results of solution analysis can directly reflect the current situation and correspond to the defense decision-making measures. In the real network environment, there is a certain distance from the solution results to the situation judgment and then to the application of the decision-making measures, requiring the effective methodological support. The failure to validate the decision-making knowledge and form feedback loop is one of the main reasons why most of the cybersecurity situation awareness methods cannot be popularized.

9.1 Result visualization

As is shown in Fig.  3 , the first four stages of the network security situation awareness mechanism fully utilize rational thinking and the computing advantages of machines, but cannot make full use of human perception ability to turn abstract model or language representation graphical more easily to express the intrinsic meaning and enhance cognitive effect. To present the hidden information and rules in data through visual graphics is the main function of information visualization, also the research emphasis [ 174 ]. Visualization analysis is a new direction of multidisciplinary research, which undoubtedly shares great similarities with the status quo of multidisciplinary integration of cybersecurity situation awareness research. At present, the combination is mainly carried out at two points after the model representation in stage II and the solution analysis in stage IV.

After stage II, the visualization of the elements and their relations is mainly carried out. Simple graphics, such as Figs.  8 a and 10 in this paper, reveal the visual graphic expression of the abstraction of the experimental network model. The visualization of the physical and logical connections of the network is the basis of all the analysis methods. Phan et al. [ 175 ] propose time visualization system of the self-building structure, and the graphical descriptions of various attack graphs [ 128 , 129 , 140 , 141 , 142 ] belong to this category; the visualization of the analysis results is carried out after stage 4 solution analysis, and the focus can be more easily understood by graphical analysis. Tamassia et al. [ 76 ] conduct a basic investigation on the visualization of security perception. Figure  13 is a concise example of visualization of analysis results, especially in large-scale network analysis, and visualization can greatly improve the efficiency of analysis. Figure  14 shows the results of the attack graph analysis results visualization reduction effect.

Attack graphical analysis [ 140 ] and visualize simplified [ 176 ]

Graphical representation is an important part of information visualization, but it is also the primary stage of visualization. Visualization is not only the process of passive information mining, but also the process of human subjective consciousness participation. The framework proposed by Erbacher [ 78 ] allows network managers to participate artificially in the analysis loop, to make immediate assessments with the help of the unique expertise of the participants and to combine artificial intelligence with visualization [ 77 , 79 ], but most of these articles remain within the technical perspective [ 44 ]. There is still a long way to go for the flexible analysis of network security situational awareness in general scenarios.

9.2 Knowledge application

The effective analysis of the above stages brings the perception results of network security status. If there are potential threats or attacks in the results, network security administrators are required to take corresponding defensive measures to strengthen the security of the target network, which is called the application feedback loop process of perception knowledge in the cybersecurity situation awareness mechanism, as shown in Fig.  3 . Obviously, it is unrealistic to completely eliminate the loopholes or threats in the perception results. The feedback process of knowledge application based on perceptual analysis results is transformed into an optimal reinforcement decision-making problem. At present, relative research mainly involves three categories: the minimum cost reinforcement of key objectives, the maximum benefit reinforcement of the whole network, and the multi-objective security reinforcement.

The concept of minimum cost reinforcement based on key objectives is to take the key assets in the network as the starting point of reinforcement and to seek a method to ensure security at the minimum cost. Most of the literature will give the defense measures to ensure that the key objectives do not suffer losses [ 11 , 101 , 140 , 141 , 142 ] after the case study. Based on the analysis results of the experimental network in this paper (Figs.  9 and 13 ), assuming that the data on the 10.10.0.11 internal development server is the key objective, the reinforcement objective g can be expressed as g  = (10-Apache˄10-Windows˄10-Linux)˅ (10-Tomact˄10-Ftp˄11-Ftp)˅ (58-Windows˄16-Vmare˄16-Windows˄11-Linux). Thirty-six kinds of reinforcement solutions can be obtained as { D i , D j , D k }, and the minimum reinforcement cost is \( \underset{i,j,k}{\min}\left[\mathrm{Cost}\left({D}_i\right)+\mathrm{Cost}\left({D}_j\right)+\mathrm{Cost}\left({D}_k\right)\right] \) , among which D i ∈ {10-Apache,10-Windows,10-Linux}, D j ∈ {10-Tomact,10-Ftp,11-Ftp}, and D k ∈ {58-Windows,16-Vmare,16-Windows,11-Linux}. On this basis, in literature [ 177 ], the important assets in the network are represented by the combination of the initial condition logic expressions of the network, and the reinforcement scheme is obtained from the attack source. Wang et al. [ 178 ] quantify the probability relationship of the state transition caused by vulnerability through Markov model, analyze the possible attack means and the corresponding defense cost, and put forward the scheme of minimum cost of reinforcement. Starting from the view of network administrator’s concern, this method can guarantee the core assets against loss at a relatively minimum cost. However, it neglects the correlation between defense measures and other normal access and easily leads to the failure to normally respond for some other assets or services not listed as key objectives.

The focus of the maximum benefit reinforcement of the whole network is how to ensure its maximum security with the current perceptual analysis results. Noel et al. [ 179 ] set out from the initial conditions of the network, calculating the true value of the logical expression to find the security measures to maximize the security of key assets. Jajodia and Noel [ 180 , 181 ] start from the perspective of network administrators as defenders, focusing on maximizing the security protection of enterprise networks, to seek the most effective defense measures to ensure the maximum return. This method can maximize the network security efficiency to a certain extent, but taking security as the starting point will lead to excessive time complexity or loss of normal service function for security in practical applications.

Multi-objective security reinforcement attempts to combine the advantages of minimum cost reinforcement of key objectives and maximum benefit reinforcement of the whole network to achieve maximum security of the whole network under the premise of the normal operation of key objectives and basic functions. Frigault et al. [ 130 ] and Bayesian attack graph are combined with and calculate the probabilistic relationship between the attack behavior and the defense alarm index in the course of attack. Several sets of reinforcement measures are established under the guidance of the safety index and compared with each other by quantitative analysis. Dewri et al. [ 182 ] take the idea of game and adopt the theory of multi-objective analysis and co-evolution of competition to construct an optimal security reinforcement model, ensuring the maximum security return under the premise of certain security costs and normal functions in the co-evolution of attack decision and defense decision. This method can consider the application effect of decision-making from different angles. But the subjectivity of the expense or reward in the objective matrix of this method is so large that it will lead to the lack of objectivity of knowledge application feedback, and also have great limitations in storage and calculation during the large-scale network promotion.

10 Network securities under large data

With the development of the information society, the age of big data has come quietly, the speed of data production is getting faster and faster, and the value implied in the data will bring about a revolutionary development to the society. As the carrier of digital resources, the computer network has penetrated into all aspects of social life, and the network structure is becoming more and more complex. With the rapid growth of interaction, new technologies are needed to ensure network security. Information security is becoming a big data analysis problem, and large security data need to be effectively associated, analyzed, and excavated [ 183 ]. The discussion of data classification and storage in the fourth section in this article also indicates that the data of cybersecurity situational awareness conforms to the 4V characteristics of big data [ 184 ]. The combination of big data analysis and cybersecurity situational awareness naturally produces new network security solutions: network securities analysis under large data. Big data is a mixture of new resources, new technologies, and new concepts [ 185 ]; the research of network security analysis under large data also naturally revolves around these three dimensions.

From the dimension of new resources, large data is more resources, which can be collected, preprocessed, and stored on the basis of more large-scale data throughput. The combination of mass data and traditional models or analysis methods will achieve better perception accuracy. For example, the collection of relevant data in the security competition in literature [ 80 , 81 ], 35 billion network intrusion detection system alarm data sets collected worldwide from the HP laboratory, used to identify malicious attacks. The BotCloud project analyzed 720 million Netflow data involving 16 million hosts to establish the correlation between hosts. Cerullo et al. [ 186 ] embody the advantages of mass data association analysis in network security and form a multi-type security event intelligent association analysis model in a wide time period. Behavioral association analysis based on large data volume can greatly improve the detection rate of network anomaly [ 103 , 187 ].

From the dimension of the new technology, large data is a new generation of data management and analysis technology. It can apply large data technology in the field of cybersecurity situation awareness and mine more data value. Based on the flow data processing method in large data analysis, OpenSOC [ 188 ] constructs a large data security analysis framework for network packets and streams to realize real-time detection of network anomalies. Using large data batch processing architecture Apache Spark, Fischer and Keim [ 189 ] designed the network security situation visualization tool NStreamAware, which can monitor and visualize the network data flow. Marchal et al. [ 190 ] also proposed a security monitoring framework for mass data analysis based on Spark. Based on Hadoop and Map Reduce technology, WINE project [ 191 ] can efficiently handle large-scale security datasets, including 5,500,000 malware samples, 30 TB data set based on reputation, 100,000 spam samples, and 75 million security threats and telemetry data sets of sensors from the whole world. Giura and Wang [ 192 ] proposed a conceptual attack pyramid model, which grouped all possible security-related events in the organization into multiple scenarios; used the MapReduce method to do parallel progress in each scene or between scenes; and used different algorithms to detect possible attacks.

From the perspective of new concept, big data is a new way of thinking. The way that from the traditional analysis centered on computing to data centric brings new connotation of data-driven decision. In the traditional analysis and decision-making method, we first analyze the possible causality, and then establish the model which is restricted by the factors, and get the results through the algorithm analysis to predict and take measures. The core concern is the rationality of the model abstraction and the effectiveness of the algorithm. In the mechanism shown in Fig.  3 , model abstraction and solution analysis play a key role. However, in the model of large data analysis, the first is to collect relevant data, carry out time series analysis, determine the implicit intrinsic relationship, then carry on the evolution prediction, and determine the key parameters to control effectively. The core concern is data association and the way of evolution. A typical application of big data analysis concepts in cybersecurity situational awareness is deep learning. Literature [ 193 ] applies deep learning to network traffic protocol classification and unknown protocol detection, which greatly improves the accuracy of protocol recognition, especially when the protocol is not encrypted, annd the recognition rate can reach 54.94%. The results of Deep Instinct [ 194 ] also show that the security solution using deep learning technology can resist unknown attacks.

Through the summary of this paper, we can see that there are still some difficulties in information collection, model representation, measurement establishment, and solution analysis and situation prediction. The combination of technology and concept of big data and network security situational awareness can greatly expand the research space in the field of network security, and to a certain extent, it has improved the technical level of APT attack detection, network anomaly detection, network intelligence analysis, advanced threat discovery, threat information acquisition and sharing, and so on [ 190 , 192 ]. The Ali Co’s cloud shield platform, the 360 company’s NGSOC platform [ 195 ], and a series of academic research [ 183 , 184 , 185 , 186 , 187 , 188 , 189 , 190 , 191 , 192 , 193 , 194 , 195 ] all show that the massive storage, parallel processing, and fusion analysis of large data can provide effective support for the research difficulties of cybersecurity situation awareness. The introduction of large data technology provides an opportunity for the ladder breakthroughs in this field.

11 Conclusion

This paper introduces the basic concept and core methods of network security situation awareness and highlights the system engineering perception framework from the perspective of data value chain which consists of five stages: element acquisition, model representation, measurement establishment, solution analysis, and situation prediction. It gives a detailed introduction of the basic function, main methods, and application effects of different stages. In the element acquisition stage, the perceptual data are classified and summarized, and the standardized design and implementation of the database are briefly described. In the model presentation stage, the core concepts, representative technologies, and modeling results of each model are discussed. In the measurement establishment stage, the model elements are quantified and the index volume is evaluated according to the model elements. In the solution analysis stage, the application premise and analysis of typical algorithms are discussed, and the horizontal comparison between algorithms is made. In the situation prediction stage, the importance of knowledge application feedback loop is emphasized, and the basic methods of visualization of analysis results and selection of defense measures are discussed.

Availability of data and materials

Data sharing is not applicable to this article as no datasets were generated or analyzed during the current study.

China Information Yearbook 2014[M], Editorial board of China Information Yearbook (Publication of the electronics industry, 2015)

Miller B P. Fuzz-revisited: a re-examination of the reliability of UNIX utilities and services. ftp://grilled.cs.wisc.edu/technical_papers/fuzz-revisited.ps.Z , 2001.

Google Scholar  

S.E. Smaha, Haystack: an intrusion detection system[A]. Aerospace Computer Security Applications Conference[C] (IEEE, 2002), pp. 37–44

J.P. Anderson, Computer security threat monitoring and surveillance[A] (James P Anderson Co Fort [C], Washington, 1980), pp. 26–32

C. Phillips, L.P. Swiler, A graph-based system for network-vulnerability analysis[A] (The Workshop on New Security Paradigms[C]. IEEE, 1998), pp. 71–79

R.W. Ritchey, P. Ammann, Using model checking to analyze network vulnerabilities[A] (Proceedings of IEEE Symposium on Security and Privacy[C]. IEEE, 2000), pp. 156–165

T. Bass, Multisensor data fusion for next generation distributed intrusion detection systems[A] (Proceedings of the Iris National Symposium on Sensor & Data Fusion[C]. Hopkins University Applied Physics Laboratory, 1999), pp. 24–27

T. Bass, Intrusion systems and multisensor data fusion: creating cyberspace situation awareness. Commun. ACM 43 (4), 99–105 (2000). https://doi.org/10.1145/332051.332079]

Article   Google Scholar  

J. Mcdermott, Attack-potential-based survivability modeling for high-consequence systems[A] (IEEE International Workshop on Information Assurance[C]. IEEE Comp. Soc, 2005), pp. 119–130

W. Yuanzhuo, L. Chuang, C. Xueqi, et al., Analysis for network attack-defense based on stochastic game model[J]. Chin. J. Comput. Phys. 33 (33), 1748–1762 (2010)

N. Poolsappasit, R. Dewri, I. Ray, Dynamic security risk management using Bayesian attack graphs[J]. Dependable Secure Comput. 9 (1), 61–74 (2012)

J. Theureau, Nuclear reactor control room simulators: human factors research and development[J]. Cogn. Tech. Work 2 (2), 97–105 (2000)

M.R. Endsley, Design and evaluation for situation awareness enhancement[J]. Proceed. Hum. Factors Ergon. Soc. Ann. Meet. 32 (1), 97–101 (1988)

M.R. Endsley, Toward a theory of situation awareness in dynamic systems[J]. Hum. Factors 37 (1), 32–64 (1995)

Boyd J R. A Discourse on Winning and Losing[C]// Strategic game of 1987.

G.P. Tadda, J.S. Salerno, Overview of cyber situation awareness. Cyber Situational Awareness[M] (Springer US, 2010), pp. 15–35

X.W. Liu, H.Q. Wang, H.W. Lü, J.G. Yu, S.W. Zhang, Fusion-based cognitive awareness-control model for network security situation[J]. J. Soft. 27 (8), 2099–2114 (2016)

MathSciNet   Google Scholar  

U. Franke, J. Brynielsson, Cyber situational awareness a systematic review of the literature. Comput. Secur. 46 , 18–31 (2014). https://doi.org/10.1016/j.cose.2014.06.008

J. Gong, X.D. Zang, Q. Su, X.Y. Hu, J. Xu, Survey of network security situation awareness[J]. J. Softw 28 (4), 1010–1026 (2017)

D.E. Denning, An intrusion-detection model. IEEE Trans. Softw. Eng 13 (2), 222–232 (1987)

H. Debar, M. Dacicr, Andreas wespi towards taxonomy of intrusion-detection systems. Comput. Netw 31 (8), 805–822 (1999)

http://www.cs.ucsb.edu/~kemm/NetSTAT/documents.html .

G. Vigna, R.A. Kemmerer, NetSTAT: a network-based intrusion detection system. Journal of Computer Security 7 (1), 37–71 (1999)

http://www.cs.purdue.edu/coast/projects/aafid.html .

B. Mukherjee, L.T. Heberlein, Network Intrusion Detection[M]. IEEE Netw., 26–41 (1994)

J. Shi, S.Q. Guo, Y. Lu, L. Xie, An intrusion response method based on attack graph. J. Softw. 19 (10), 2746–2753 (2008)

Z.H. Tian, X.Z. Yu, H.L. Zhang, B.X. Fang, A real time network intrusion forensics method based on evidence reasoning network. Chin. J. Comput. Phys. 5 (37), 1184–1193 (2014)

X.H. Bao, Y.X. Dai, P.H. Feng, P.F. Zhu, J. Wei, A detection and forecast algorithm for multi-step attack based on intrusion intention. J. Softw. 16 (12), 2132–2138 (2005)

Article   MATH   Google Scholar  

K. Ilgun, R.A. Kemmerer, P.A. Porras, State transition analysis: a rule-based intrusion detection approach. IEEE Trans. Softw. Eng. 21 (3), 181–199 (1995)

T. Bass, R. Robichaux, in Proc. of the Communications for Network-Centric Operations: Creating the Information Force (MILCOM) . Defense-in-depth revisited: qualitative risk analysis methodology for complex network-centric operations (IEEE, 2001), pp. 64–70

Batsell S G, Rao N S, Shankar M . Distributed intrusion detection and attack containment for organizational cyber security. http://www.ioc.ornl.gov/projects/documents/containment.pdf , 2005

J. Shifflet, A technique independent fusion model for network intrusion detection. Proceedings of the Midstates Conference on Undergraduate Research in Computer Science and Mathematics 3 (1), 13–19 (2005)

R. Bearavolu, K. Lakkaraju, W. Yurcik, NVisionIP: an animated state analysis tool for visualizing NetFlows (FLOCON Network Flow Analysis Work shop (Network Flow Analysis for Security Situational Awareness), 2005)

X. Yin, W. Yurcik, A. Slagell, The design of VisFlowConnect-IP: a link analysis system for IP security situational awareness[A] (IEEE International Workshop on Information Assurance[C]. IEEE, 2005), pp. 141–153

Z. Li, J. Taylor, E. Partridge, et al., UCLog: A unified, correlated logging architecture for intrusion detection[J] (Telecommunication Systems – TELSYS, 2004), pp. 12–27

Endsley, M. R. and Garland D.J(Eds.)(2000) Situation awareness analysis and measurement. Mahwah: Lawrence Erlbaum Associates.

J. Kopylec, A. D'Amico, J. Goodall, in Critical Infrastructure Protection[M] . Visualizing cascading failures in critical cyber infrastructures (Springer US, 2007), pp. 351–364

Goodall J R. Introduction to visualization for computer security[A]. The Workshop on Vizsec[C]. DBLP, 2008.1-17.

Jajodia S, Liu P, Swarup V, et al. Cyber situational awareness[M]. Springer US, 2010.132(2):1-4.

N.A. Giacobe, Application of the JDL data fusion process model for cyber security[J]. Proc Spie 7710 (5), 1–10 (2010)

Klein G, Tolle J, Martini P. From detection to reaction - a holistic approach to cyber defense[A]. Defense Science Research Conference and Expo[C]. IEEE, 2011.1-4.

S. Schreiber-Ehle, W. Koch, The JDL model of data fusion applied to cyber defense - a review paper[A] (Sensor Data Fusion: Trends, Solutions, Applications[C]. IEEE, 2012), pp. 116–119

M. Cheminod, L. Durante, A. Valenzano, Review of security issues in industrial networks[J]. IEEE Trans. Ind. Inf. 9 (1), 277–293 (2013)

U. Franke, J. Brynielsson, Cyber situational awareness – a systematic review of the literature[J]. Comput. Sec. 46 , 18–31 (2014)

V. Lenders, A. Tanner, A. Blarer, Gaining an edge in cyberspace with advanced situational awareness[J]. IEEE Secur. Priv. 13 (2), 65–74 (2015)

S. Rathore, P.K. Sharma, V. Loia, Y.-S. Jeong, J.H. Park, Social network security: issues, challenges, threats, and solutions. Inf. Sci 421 , 43–69 (2017)

P.A. Ralston, J.H. Graham, J.L. Hieb, Cyber security risk assessment for SCADA and DCS networks[J]. ISA Trans. 46 (4), 583–594 (2007)

P. Barford, M. Dacier, T.G. Dietterich, M. Fredrikson, J. Giffin, S. Jajodia, et al., in Cyber Situational Awareness . Cyber SA: situational awareness for cyber defense (Springer, 2010), pp. 3–13

A.G. Fragkiadakis, E.Z. Tragos, I.G. Askoxylakis, A survey on security threats and detection techniques in cognitive radio networks. IEEE Comm. Surveys Tutorials 15 , 1 (2013)

D. King, G. Orlando, J. Kohler, in Proceedings – IEEE Military Communications Conference MILCOM . A case for trusted sensors: encryptors with deep packet inspection capabilities (2012)

X. Liang, Y. Xiao, Game theory for network security[J]. IEEE Commun. Surv. Tutorials 15 (1), 472–486 (2013)

M.H. Manshaei, Q.Y. Zhu, T. Alpcan, et al., Game theory meets network security and privacy. ACM Comput. Surv. 45 (3), 25 (2013)

H. He, Y. Shuping, P. Wu, in Proceedings e2009 International Conference on Information Engineering and Computer Science . Security decision making based on domain partitional Markov decision process (ICIECS, 2009), p. 2009

S. Stevens-Adams, A. Carbajal, A. Silva, et al., in Foundations of Augmented Cognition[M] . Enhanced training for cyber situational awareness (Springer, Berlin Heidelberg, 2013), pp. 90–99

Chapter   Google Scholar  

S. Roschke, F. Cheng, C. Meinel, High-quality attack graph-based IDS correlation[J]. Log. J. IGPL 21 (4), 571–591 (2013)

Article   MathSciNet   Google Scholar  

J. Preden, L. Motus, M. Meriste, A. Riid, in 2011 IEEE International Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support, CogSIMA 2011 . Situation awareness for networked systems (2011), pp. 123–130

Y. Cheng, Y. Sagduyu, J. Deng, J. Li, P. Liu, in Proceedings of SPIE - The International Society for Optical Engineering . Integrated situational awareness for cyber attack detection, analysis, and mitigation (2012), p. 8385

M.L. Hinman, Some computational approaches for situation assessment and impact assessment[A] (International Conference on Information Fusion[C]. IEEE, 2002), pp. 687–693

B. D’ Amhrosio, Security situation assessment and response evaluation (SSARE)//DISCEX'O1. Proceedings: DARPA Information Survivability Conference & Exposition II (IPPP Computer Society, Los Alamitos, 2001), pp. 387–394

H. Hu, X. Wang, X. Yang, in 1st International Conference on Multimedia Information Networking and Security, MINES 2009 . A decision-support model for information systems based on situational awareness, vol 2 (2009), pp. 405–408

P. Ammann, D. Wijesekera, S. Kaushik, in ACM Conference on Computer and Communications Security 2002[C] . Scalable, graph-based network vulnerability analysis[A] (DBLP, Washington DC, 2002), pp. 217–224

T. Ke, M.-T. Zhou, W.-Y. Wang, in Proceedings of 2009 4th International Conference on Computer Science and Education, ICCSE 2009 . Insider cyber threat situational awareness framwork using dynamic bayesian networks (2009), pp. 1146–1150

J.-Y. Cai, V. Yegneswaran, C. Alfeld, P. Barford, Honeynet games: a game theoretic approach to defending network monitors. J Comb Optim 22 (3), 305–324 (2011)

Article   MathSciNet   MATH   Google Scholar  

T.G. Dietterich, X. Bao, V. Keiser, et al., in Cyber Situational Awareness[M] . Machine learning methods for high level cyber situation awareness (Springer US, 2010), pp. 227–247

P. Barford, Y. Chen, A. Goyal, Z. Li, V. Paxson, V. Yegneswaran, in Cyber Situational Awareness . Employing honeynets for network situational awareness (Springer, 2010), pp. 71–102

A. Stotz, M. Sudit, in FUSION 2007-2007 10th International Conference on Information Fusion . Information fusion engine for real-time decision-making (inferd): a perceptual system for cyber attack tracking (2007)

R. Dapoigny, P. Barlatier, et al., Formal foundations for situation awareness based on dependent type theory[J]. Information Fusion 14 (1), 87–107 (2013)

W. Streilein, J. Truelove, C. Meiners, G. Eakman, in Proceedings e IEEE Military Communications Conference MILCOM . Cyber situational awareness through operational streaming analysis (2011), p. 1152e7

J. Li, X. Ou, R. Rajagopalan, in Cyber Situational Awareness . Uncertainty and risk management in cyber situational awareness (Springer, 2010), pp. 51–68

R. Paffenroth, P.D. Toit, R. Nong, et al., Space-time signal processing for distributed pattern detection in sensor networks[J]. IEEE J. Sel. Top. Sign. Proces. 7 (1), 38–49 (2013)

M.L. Mathews, P. Halvorsen, A. Joshi, et al., in International Conference on Collaborative Computing: Networking, Applications and Worksharing[C] . A collaborative approach to situational awareness for cybersecurity[A] (IEEE, 2012), pp. 216–222

L. Wang, Sushil Jajodia. k-zero day safety: a network security metric for measuring the risk of unknown vulnerabilities. IEEE Transac. Depend Secure Comput 11 , 1 (2014)

F. Sanfilippo, A multi-sensor fusion framework for improving situational awareness in demanding maritime training[J]. Reliab. Eng. Syst. Saf. 161 , 12–24 (2017)

D. Shen, G. Chen, J. Cruz Jr., L. Haynes, M. Kruger, E. Blasch, in Proceedings of SPIE e The International Society for Optical Engineering . A markov game theoretic data fusion approach for cyber situational awareness, vol 6571 (2007)

Renaud Deraison. Nessus Scanner. http://www.nessus.org . [EB/OL].2004.

R. amassia, B. Palazzi, C. Papamanthou, in Graph Drawing[C] . Graph drawing for security visualization[A] (Springer, 2009), pp. 2–13

J. Beaver, C. Steed, R. Patton, X. Cui, M. Schultz, in Proceedings of SPIE e The International Society for Optical Engineering . Visualization techniques for computer network defense, vol 8019 (2011)

R. Erbacher, in ACM International Conference Proceeding Series . Visualization design for immediate high-level situational assessment (2012), pp. 17–24

K.J. Ross, K.M. Hopkinson, M. Pachter, Using a distributed agent-based communication enabled special protection system to enhance smart grid security[J]. IEEE Transactions on Smart Grid 4 (2), 1216–1224 (2013)

A. Doupé, M. Egele, B. Caillat, et al., in Twenty-Seventh Computer Security Applications Conference[C] . Hit ‘em where it hurts: a live security exercise on cyber situational awareness[A] (DBLP, Orlando, 2011), pp. 51–61

G. Fink, D. Best, D. Manz, et al., in Foundations of Augmented Cognition [M] . Gamification for measuring cyber security situational awareness (Springer, Berlin Heidelberg, 2013), pp. 656–665

S. Lee, D.H. Lee, K.J. Kim, in Frontiers of High Performance Computing and NetworkingeISPA 2006 Workshops . A conceptual design of knowledge-based real-time cyber-threat early warning system (Springer, 2006), pp. 1006–1017

G. Klein, H. Günther, S. Träber, Modularizing cyber defense situational awareness – technical integration before human understanding[J]. Commu. Comp. Inform. Sci 318 , 307–310 (2012)

A. D’Amico, K. Whitley, The real work of computer network defense analysts[A] (The Workshop on Vizsec[C]. DBLP, 2008), pp. 19–37

R.F. Erbacher, D.A. Frincke, P.C. Wong, et al., A multi-phase network situational awareness cognitive task analysis[J]. Inform. Visual. 9 (3), 204–219 (2010)

K. Giles, W. Hagestad, Divided by a common language: cyber definitions in Chinese, Russian and English[A] (International Conference on Cyber Conflict[C]. IEEE, 2013), pp. 1–17

U. Adhikari, T.H. Morris, N. Dahal, et al., Development of power system test bed for data mining of synchrophasors data, cyber-attack and relay testing in RTDS[A] (Power and Energy Society General Meeting[C]. IEEE, 2012), pp. 1–7

U. Franke, Optimal IT service availability: shorter outages, or fewer? Netw. Serv. Manag. IEEE. Transactions. 9 (1), 22e33 (2012)

I.A. Kirillov, S.A. Metcherin, S.V. Klimenko, Metamodel of shared situation awareness for resilience management of built environment[A] (International Conference on Cyberworlds[C]. IEEE, 2012), pp. 137–143

K. Adams, A. Wassell, M.G. Ceruti, et al., Emergency-management situational-awareness prototype (EMSAP)[A] (IEEE First International Multi-disciplinary Conference on Cognitive Methods in Situation Awareness & Decision Support[C]. IEEE, 2011), pp. 110–114

X. Jinping, Speech at the Symposium on Internet Security and information technology (April 19, 2016) [N] (people’s daily, 2016), p. 2

S. Changxiang, Z. Huanguo, F. Dengguo, C. Zhenfu, H. Jiwu, Overview of information security[J]. Sci. Chin. Ser. E. Inform. Sci. 37 (2), 129–150 (2007)

J. Liu, P. Su, M. Yang, L. He, Y. Zhang, X.Y. Zhu, H. Lin, Software and cyber security - a survey[J]. J. Software 29 (1), 42–68 (2018)

J.-c. Jiang, H.-t. Ma, D.-e. Ren, S.-h. Qing, A survey of intrusion detection research on network security[J]. J. Software 11 (11), 1460–1466 (2000)

Y.-x. Lai, Z.-H. Liu, X.-t. Cai, K.-x. Yang, Research on intrusion detection of industrial control system[J]. J. Commun. 38 (2), 143–156 (2017)

L. Chuang, W. Yang, L. Quanlin, Stochastic modeling and evaluation for network security. Chin. J. Comput. 28 (12), 1943–1956 (2005)

H.Q. Wang, J.B. Lai, L. Zhu, Y. Liang, Survey of network situation awareness system. J. Comput. Sci. 33 (10), 5–10 (2006)

Z.H. Gong, Y. Zhuo, Research on cyberspace situational awareness. J. Software 21 (7), 1605–1619 (2010)

X.Z. Chen, Q.H. Zheng, X.H. Guan, C.G. Lin, Quantitative hierarchical threat evaluation model for network security. J. Software 17 (4), 885–897 (2006)

Us: progress and trend of network situational awareness research [J]. Anonymous. China information security, 2011 (2).

D. Wu, Y.-f. Lian, K. Chen, Y.-l. Liu, A security threats identification and analysis method based on attack graph. Chin. J. Comput. 35 (9), 1938–1950 (2012)

Y.Z. Zhang, B.X. Fang, Y. Chi, X.C. Yun, Risk propagation model for assessing network information systems. J. Software 18 (1), 137–145 (2007)

W. Yong, L. Yifeng, F. Dengguo, A network security situational awareness model based on information fusion. J. Comput. Res. Dev. 46 (3), 353–362 (2009)

M.-z. Li, J.-p. Lan, Smart home intrusion detection algorithm based on spatial-temporal field information fusion. J. Beijing Univ. Posts Tel. 40 (3), 76–84 (2017)

F. Ling, Z. Weijun, M. Shue, Security technology management strategy of multi-intrusion detection systems and manual investigation portfolio[J]. J. Southeast Univ. (Natural Science Edition) 45 (4), 811–816 (2015). https://doi.org/10.3969/j.issn.1001-0505.2015.04.034]

W.-w. Ren, L. Hu, K. Zhao, Intrusion alert correlation model based on data mining and ontology. J Jilin Univ. (Eng. Sci.) 45 (3), 899–906 (2015)

T. Chenghua, L. Pengcheng, T. Shensheng, X. Yi, Anomaly intrusion behavior detection based on fuzzy clustering and features selection. J. Comput. Res. Dev. 52 (3), 718–728 (2015)

W. Yichuan, M. Jianfeng, L. Di, Z. Liumei, M. Xianjia, Game optimization for internal DDoS attack detection in cloud computing. J. Comput. Res. Dev. 52 (8), 1873–1882 (2015)

F. Xuewei, W. Dongxia, L.J. Huang Minhuan, A mining approach for causal knowledge in alert correlating based on the markov property. J. Comput. Res. Dev. 51 (11), 2493–2504 (2014)

Z.-y. Luo, B. You, J.-z. Xu, Y. Liang, Automatic recognition model of intrusive intention based on three layers attack graph. J Jilin Univ. (Eng. Sci.) 44 (5), 1392–1397 (2014)

Y. Yu, C.-h. Xia, X.-y. Hu, Defense scheme generation method using mixed path attack graph. J. Zhejiang Univ. (Eng. Sci) 51 (9), 1745–1759 (2017)

F. Yan, S.-f. Liu, H. Leng, Study on analysis of attack graphs based on conversion. Chin. J. Electronics 42 (12), 2477–2480 (2014)

M. Chunguang, W. Chenghong, Z. Donghong, L. Yingtao, A dynamic network risk assessment model based on attacker’s inclination. Journal of Computer Research and Development 52 (9), 2056–2068 (2015)

N. Gao, L. Gao, Y.-y. He, Dynamic security risk assessment model based on bayesian attack graph[J]. Journal of Sichuan University(Engineering Science Edition) 48 (1), 111–118 (2016)

H.U. Hao, Y.E. Run-guo, Z.H.A.N.G. Hong-qi, Y.A.N.G. Ying-jie, L.I.U. Yu-ling, Quantitative method for network security situation based on attack prediction[J]. Journal on Communications 38 (10), 122–134 (2017)

G. Hai-Hui, X. Da, C. Tian-Ping, Yang Yi-Xian. Quantitative evaluation approach for real-time risk based on attack event correlating. 35 (11), 2630–2636 (2013)

L. Kenan, Z. Yuqing, W. Chensi, M. Hua, A system for scoring the exploitability of vulnerability based types. Journal of Computer Research and Development 54 (10), 2296–2309 (2017)

H.U.A.N.G. Jia-Hui, F.E.N.G. Dong-Qin, W.A.N.G. Hong-Jian, A method for quantifying vulnerability of industrial control system based on attack graph. Acta Automatica Sinica 42 (5), 792–798 (2016)

G. Meng-Zhou, F. Dong-Qin, L. Cong-Li, C. Jian, Vulnerability analysis of industrial control system based on attack graph. Journal Of Zhejiang University (Engineering Science) 48 (12), 2123–2131 (2014)

W. Yufei, G. Kunlun, Z. Ting, Q. Jian, Assessing the harmfulness of cascading failures across space in electric cyber-physical system based on improved attack graph. Proceedings of the CSEE 36 (6), 1490–1499 (2016)

W. Jinrong, F. Dingyi, C. Xiaojiang, W. Huaijun, H. Lu, Taxonomy of software attack technique oriented to automated modeling[J]. Journal of SiChuan University: Engineer Science Edition. 47 (Z1), 91~98 (2015)

J. Christy, Cyber threat & legal issues[A] (Shadowcon Conference[C], USA, 1999), pp. 29–50

CVSS. Common Vulnerability Scoring System[EB/OL]. http://nvd.nist.gov/cvss.cfm , 2008.

J. Wei, F. Bin-Xing, Z. Hong-Li, Evaluating network security and optimal active defense based on attack-defense game model[J]. Chinese Journal of Computers. (4, 1), 817–827 (2009)

H.R. Shahriari, R. Jalili, Vulnerability take grant (VTG): an efficient approach to analyze network vulnerabilities[J]. Computers & Security 26 (5), 349–360 (2007)

H. Tianfield, in IEEE International Conference on Internet of Things[C] . Cyber security situational awareness[A] (IEEE, 2017), pp. 782–787

X. Fu, J. Shi, L. Xie, Layered intrusion scenario reconstruction method for automated evidence analysis. Journal of Software 22 (5), 996–1008 (2011)

C.X. Jun, F.B. Xing, T.Q.F.Z.H. Liang, Inferring attack intent of malicious insider based on probabilistic attack graph model. Chinese Journal of Computers. 37 (1), 62–72 (2014)

Y. Yun, X. Xi-shan, J. Yan, An Attack graph based probabilistic computing approach of network security. Chinese Journal of Computers. 33 (10), 001987–001996 (2010)

M. Frigault, L.Y. Wang, A. Singhal, S. Jajodia, Measuring network security using dynamic Bayesian network[A]. Proceedings of the 4th ACM Workshop on Quality of Protection[C]. IEEE, 23–30 (2008)

L. Wang, B. Wang, Y. Peng, Research the information security risk assessment technique based on Bayesian network[A]. International Conference on Advanced Computer Theory and Engineering[C] (IEEE, 2010), pp. 600–604

S.J. Zhang, J.H. Li, S.S. Song, L. Li, X.Z. Chen, Using Bayesian inference for computing attack graph node beliefs. Journal of Software 21 (9), 2376–2386 (2010)

Y.T. Liao, C.B. Ma, C. Zhang, A new fuzzy risk assessment method for the network security based on fuzzy similarity measure. The 6th World Congress on. Intelligent Control and Automation 2 , 8486–8490 (2006)

T.P. Chen, X.Y. Zhang, L.Q. Zheng, Network security risk assessment based on fuzzy integrated judgment[J]. Journal of Naval University of Engineering, 38–41 (2009)

L. Zhao, Z. Xue, Synthetic security assessment based on variable consistency dominance-based rough set approach. High Technology Letters. 16 (4), 413–421 (2010)

L.S. Kong, X.F. Ren, Y.J. Fan, in IEEE International Conference on Intelligent Computing and Intelligent Systems[C] . Study on assessment method for computer network security based on rough set[A] (IEEE, 2009), pp. 617–621

Feng PH, Lian YF, Dai YX, Bao XH. A vulnerability model of distributed systems based on reliability theory. Journal of Software, 2006,17(7):1633 − 1640.

L. Yan, H. Guangqiu, C. Lixia, The probability controllability of complex network via attack[J]. Journal of Frontiers of Computer Science & Technology 10 (10), 1407–1419 (2016)

B. Scheier, Attack trees: modeling security threats[J]. Dr Dobb’s Journal 12 (24), 21–29 (1999)

O. Sheyner, J. Haines, S. Jha, in Proceedings of the IEEE Symposium on Security and Privacy . Automated generation and analysis of attack graphs[C] (IEEE Computer Society Press, Oakland, 2002), pp. 273–284

L.P. Swiler, C. Phillips, D. Ellis, S. Chakerian, in Proceedings of the DARPA Information Survivability Conference and Exposition II, Anaheim, CA . Computer attack graph generation tool (2001), pp. 307–321

J. Homer, A. Varikuti, X.M. Ou, M.Q. MA, Improving attack graph visualization through data reduction and attack grouping //Proceedings of the 5th International Workshop on Visualization for Computer Security(VizSec2008) Cambridge, MA, USA, 2008 (Springer Verlag, Belin Heidelberg, Germany, 2008), pp. 68–79

Y. Yun, X. Xishan, Q. Zhichang, et al., Attack graph generation algorithm for large-scale network system[J]. Journal of Computer Research and Development 10 , 2033–2139 (2013)

K. Ingols, M. Chu, R. Lippmann, S. Webster, S. Boyer, Modeling modern network attacks and counter measures using attack graphs//Proceedings of the 25th Annual Computer Security Applications Conference (Honolulu, Hawaii, USA, 2009), pp. 117–126

L. Weixin, Z. Kangfeng, W. Bin, Alert processing based on attack graph and multi-source analyzing [J]. journal of communications 2015 (9), 135–144

L.I.U. Wei-xin, Z.H.E.N.G. Kang-feng, H.U. Ying, et al., Approach of goal-oriented attack graph-based threat evaluation for network security[J]. JOURNAL OF BEIJING UNIVERSITY OF POSTS AND TELECOM 38 (1), 82–86 (2015)

M. Dacier, Towards quantitative evaluation of computer security[D] (Institut National Polytechnique de Toulouse, France, 1994)

R. Ortalo, Y. Deswarte, M. Kaaniche, Experimenting with quantitative evaluation tools for monitoring operational security. IEEE Transactions on Software Engineering 25 (5), 633–650 (1999)

L. Wang, A quantitative computer system and network security risk assessment method[D] (Harbin Institute of Technology, 2002)

P.A. Porras, R. Kemmerer, in Proceedings of the Eighth Annual Computer Security Applications Conference[C] . A penetration state transition analysis: a rule-based intrusion detection approach[A] (IEEE, 1992), pp. 220–229

F. Stevens, T. Courtney, S. Singh, A. Agbaria, J.F. Meyer, W.H. Sanders, P. Pal, Model-based validation of an intrusion-tolerant information system[A] (Proceedings of 23rd Symposium on Reliable Distributed Systems ( SRDS 2004)[C]. Florianópolis, Brazil, 2004), pp. 184–194

B. Madan, Go eva-Popstojanova K, Vaidyanathan K,Trivedi KS. A method for modeling and quantifying the security attributes of intrusion tolerant systems[J]. Performance Evaluation 56 (1-4), 167–186 (2004)

G. Xiang, Zhu Yue-fei,Liu Sheng-li. Attack composition model based on generalized stochastic colored Petri nets[J]. Journal of Electronics & Information Technology 35 (11), 2608–2614 (2013)

L.I.N. Chuang, W.A.N.G. Yuan-zhuo, Y.A.N.G. Yang, Q.U. Yang, Research on network dependability analysis methods based on stochastic Petri net[J]. ACTA ELECTRONICA SINICA 34 (2), 322–332 (2006)

G.A.O. Xiang, Z.H.U. Yue-fei, L.I.U. Sheng-li, F.E.I. Jin-long, L.I.U. Long, Risk assessment model based on fuzzy Petri nets[J]. Journal on Communications 2013 (s1), 126–132

R. ANDERSON, in Proceedings of 17th Annual Computer Security Application Conference[C] . Why information security is hard-an economic perspective[A] (IEEE Computer Society, Washington, DC, USA, 2001), pp. 39–40

Y.B. REDDY, A game theory approach to detect malicious nodes in wireless sensor networks[A]. Procof the 3rd International Conference on Sensor Technologies and Application[C] (IEEE Computer Society, Washington, DC, 2009), pp. 462–468

S.G. SHEN, Y.J. LI, H.Y. XU, Signaling game based strategy of intrusion detection in wireless sensor networks[J]. Computers & Mathematics with Applications 62 (6), 2404–2416 (2011)

J. Chunful, Z. Anming, Z. Wei, M. Yong, Incomplete informational and dynamic game model in network security[J]. J. Comp. Res. Dev 43 (s2), 530–533 (2006)

J.-M. Zhu, B. Song, Q.-F. Huang, Evolution game model of offense-defense for network security based on system dynamics[J]. J. Comm. 1 , 54–61 (2014)

W. Lin, H. Wang, J. Liu, L. Deng, A. Li, Q. Wu, Y. Jia, Research on cooperative active defense technology in network security based on non-dynamic game theory[J]. J. Comp. Res. Dev. 48 (2), 306–316 (2011)

Y. Zhang, X.B. Tan, X.L. Cui, H.S. Xi, Network security situation awareness approach based on Markov game model. J. Software 22 (3), 495–508 (2011)

J.X. Ran, B. Xiao, Risk evaluation of network security based on NLPCA−RBF neural network[A]. International Conference on Multimedia Information Networking and Security[C] (IEEE, 2010), pp. 398–402

Y. Liang, H.Q. Wang, J.B. Lai, Quantification of network security situational awareness based on evolutionary neural network. The 6th International Conference on Machine Learning and Cybernetics , vol 6 (2007), pp. 3267–3272

G. Wang, J. Hao, J. Ma, et al., A new approach to intrusion detection using artificial neural networks and fuzzy clustering[J]. Expert Syst. Appl. 37 (9), 6225–6232 (2010)

N. Gao, L. Gao, Y.Y. He, A lightweight intrusion detection model based on autoencoder network with feature reduction[J]. Acta Electron. Sinica 45 (3), 730–739 (2017)

S.A. Hofmeyr, S. Forrest, Architecture for an artificial immune system. Evolutionary Computation 7 (1), 45–68 (2000)

J. Kim, J.B. Peter, in Proceedings of the World Congress on Computational Intelligence[C] . Towards network intrusion detection: artificial immune system for investigation of dynamic clone selection[A] (IEEE Press, Piscataway, 2002), pp. 1015–1020

L. Tao, Network security risk detection based on immune[J]. Sci. Chin. Ser. E. Inform. Sci. 35 (8), 798–816 (2005)

L. Tao, An immune based model for network monitoring [J]. Chin J Comp 29 (9), 1515–1522 (2006)

F. Dai, K. Zheng, S. Luo, B. Wu, in Proc of 2015 IEEE International Conference on Communications[C] . Towards a multi objective framework for evaluating network security under exploit attacks [A] (IEEE Press, New York, 2015), pp. 8814–8819

J. Zhang, F. Liu, W. Han, et al., Research and implement of configurable network security index system[A] (International Conference on Applied Robotics for the Power Industry[C]. IEEE, 2012), pp. 645–648

Y.Z. Zhang, X.C. Yun, Network operation security index classification model with multidimensional attributes. Chin. J. Comp. 35 (8), 1666–1674 (2012)

D. Keim, J. Konlhammer, G. Ellis, F. Mansmann, Mastering the information age: solving problems with visual analytics (Eruographics Association, Goslar, 2010), pp. 1–168

D. Phan, J. Gerth, M. Lee, A. Paepcke, T. Winograd, in Viz SEC 2007[C] . Visual analysis of network flow data with timelines and event plots[A] (Springer, 2008), pp. 85–99

Y. Ye, X.-S. Xu, Y. Jia, Z.-C. Qi, W.-C. Cheng, Research on the risk adjacency matrix based on attack graphs[J]. J. Comm. 32 (5), 112–120 (2011)

L. Wang, S. Noel, S. Jajodia, Minimum cost network hardening using attack graphs [J]. Computer Communications 29 (18), 3812–3824 (2006)

S. Wang, Z. Zhang, Y. Kadobayashi, Exploring attack graph for cost-benefit security hardening [J]. Comp. Security 32 , 158–169 (2013)

S. Noel, S. Jajodia, B. O'Berry, et al., Efficient minimum-cost network hardening via exploit dependency graphs [A].// Proc of the 2003 Annual Computer Security Applications Conference [C] (IEEE Press, New Jersey, 2003), pp. 86–95

S. Jajodia, S. Noel, Topological vulnerability analysis: a powerful new approach for network attack prevention, detection, and response [J] (Algorithms, architectures and information systems security, Indian institute platium jubilee series, 2009), pp. 285–305

K. Ingols, M. Chu, R. Lippmann, et al., in Proc of the 2009 Annual Computer Security Applications Conference [C] . Modeling modern network attacks and countermeasures using attack graphs [A] (IEEE Press, New Jersey, 2009), pp. 117–126

R. Dewri, I. Ray, N. Poolsappasit, et al., Optimal security hardening on attack tree models of networks: a cost-benefit analysis. Int. J. Info. Security 11 (3), 167–188 (2012)

Gartner. Information security is becoming a big data analytics problem[EB/OL].[2012]. https://www.gartner.com/doc/1960615/information-security-big-data-analytics .

V. Mayer-Schnberger, K. Cukier, Big data: a revolution that will transform how we live, work, and think (John Munay Publishers, USA, 2013)

Big data white paper (2016). Beijing: China information and Communication Research Institute (Institute of telecommunications, Ministry of industry and information technology), 2016.

G. Cerullo, L. Coppolino, S. D’Antonio, et al., Enabling convergence of physical and logical security through intelligent event correlation[M]//Intelligent Distributed Computing IX (Springer, Berlin, 2016), pp. 427–437

M.H. Bhuyan, D.K. Bhattacharyya, J.K. Kalita, Network anomaly detection: methods, systems and tools. IEEE Commun. Surveys Tutorials 16 (1), 303–336 (2014)

Cisco. OpenSOC: Big data security analytics framework [EB/OL]. http://opensoc.github.io/ , 2017.

F. Fischer, D.A. Keim, NStreamAware: real-time visual analytics for data streams to enhance situational awareness[C]// Proceedings of the Eleventh Workshop on Visualization for Cyber Security (ACM, New York, 2014), pp. 65–72

S. Marchal, X. Jiang, R. State, et al., A big data architecture for large scale security monitoring[C]//Proceedings of the 2014 IEEE International Conference on Big Data. Anchorage: IEEE, 56–63 (2014)

T. Dumitras, D. Shou, Toward a standard benchmark for computer security research: the worldwide intelligence network environment (WINE)[C]//Proceedings of the First Workshop on Building Analysis Datasets and Gathering Experience Returns for Security (ACM, New York, 2011), pp. 89–96

P. Giura, W. Wang, Using large scale distributed computing to unveil advanced persistent threats. Science 1 (3), 93–105 (2012)

Wang Z. The applications of deep learning on traffic identication [EB/OL]. [2017]. https://www.blackhat.com/docs/us-15/materials/us-15-Wang-The-Applications-Of-Deep-Learning-On-Traffic-Identification-wp.pdf .

Musthaler L. How to use deep learning AI to detect and prevent malware and APTs in real-time[EB/OL]. [2017-03-20]. http://www.networkworld.com/article/3043202/security/how-to-use-deep-learning-ai-to-detect-and-prevent-malwareand-apts-in-real-time.html .

X. Chen, Z. Xuemei, W. Wang, et al., Big data analytics for network security and intelligence. Adv. Eng. Sci. 49 (3), 1–12 (2017)

Download references

Acknowledgements

Not applicable

About the Authors

Yan Li was born in Chengde City, Hebei Province in 1984. He received the B.S., M.S., and the Ph.D. degree from Xi’an University of Architecture & Technology, Xi’an, China, all in information management and information system. He is currently working in the School of Management of Xi’an Polytechnic University. His main research directions include system engineering, big data application analysis, and network security. He worked in software companies from 2009 to 2017. He has been engaged in software development for 4 years in active network, and later served as general manager in medium-sized software enterprises. He has rich theoretical and practical experience. At present, he focuses on theoretical research and system development in the field of block chain security and certification. (corresponding author; email: [email protected] )

Guang-qiu Huang received the B.S. and the M.S. degree from Xi’an University of Architecture & Technology, Xi’an, China, and the Ph.D. degree from Northeast University, Shenyang, China, all in mining engineering. He has worked in education for 25 years at Xi’an University of Architecture & Technology, where he is now a professor and doctoral supervisor in the School of Management. His teaching and research involves systems engineering, information management and information systems, computer intelligence, and optimization design of mining engineering. He is the consultant expert of the Government of Xi’an City and the assessment expert of National Natural Science Foundation. He has completed 78 research projects including national key scientific research projects, projects of National Natural Science Foundation, and provincial and ministerial level research projects. He won the Henry Fok Prize, the Baosteel Education Award, the First Prize of the Government of Shaanxi Province, and has published over 300 refereed conference and journal papers, 8 books, 43 software copyrights, and 9 patents. (email: [email protected] )

Chun-zi Wang received the B.S., M.S., and Ph.D. degree from Xi’an University of Architecture & Technology, Xi’an, China, all in Management Science and Engineering. She has worked in education for 8 years at Xi’an Polytechnic University, where she is now an associate professor and master supervisor in the School of Management. She has taught 4 courses, such as Java language programming, network information security, object-oriented technology, and statistics. Her teaching and research involves network security, risk management, and optimal decision. She has published over 20 refereed conference and journal papers and presided over 10 research projects, including Natural Science basic Research Project of Provincial Science and Technology Department and Provincial Education Department project. (email: [email protected] )

Ying-chao Li received a bachelor’s degree from Xi'an Technological University in 2009, specializing in software engineering. He has 10 years of experience in the industry, mainly engaged in software project system architecture design and research and development management. He is good at distributed and big data technology. He had in-depth study of design patterns and database optimization. His main work experience is as follows: in 2017, he is the project leader of Shaanxi Province’s key industrial project “Research on Complex Heterogeneous Data Fusion and Management Model of Provincial Food and Drug Regulation,” “Shaanxi food safety supervision comprehensive business system" project leader,” and “Emaplink Smart Distributed Service Platform” project leader. The technical leader of “Cisco Smart Business Configurator for Collaboration (SBCC)” project. “Shaanxi Telecom Electronic operation and maintenance system” takes charge of database design, performance optimization, and so on. At present, he holds the position of technical director of Legend Software Co., Ltd. and is responsible for the construction of information projects in the field of food and drug supervision. Many software project copyright and invention patents were created during the period. (email: [email protected])

This study was funded by The Fund Project for Science and Technology Research and Development Plan of Shaanxi Province (grant number: 2013 K1117) and Xi’an Polytechnic University doctoral research start-up fund(grant number: 20170914)

Author information

Authors and affiliations.

Xi’an Polytechnic University, Xi’an, 710048, Shaanxi, China

Yan Li, Chun-zi Wang & Ying-chao Li

Xi’an University of Architecture & Technology, Xi’an, 710075, Shaanxi, China

Guang-qiu Huang

You can also search for this author in PubMed   Google Scholar

Contributions

LY conceived of the whole article and has completed two to seven sections of the article. HG completed the first section and participated in the overall discussion and proofreading. WC completed the content of the eighth section and conclusion and participated in the overall discussion and proofreading. LY-C participated in the discussion and proofreading work. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Yan Li .

Ethics declarations

Competing interests.

The authors declare that they have no competing financial interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/ ), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and permissions

About this article

Cite this article.

Li, Y., Huang, Gq., Wang, Cz. et al. Analysis framework of network security situational awareness and comparison of implementation methods. J Wireless Com Network 2019 , 205 (2019). https://doi.org/10.1186/s13638-019-1506-1

Download citation

Received : 10 April 2019

Accepted : 03 July 2019

Published : 13 August 2019

DOI : https://doi.org/10.1186/s13638-019-1506-1

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Network security
• Network situational awareness
• Big data network security
• Intrusion detection
• Data fusion analysis

This is a potential security issue, you are being redirected to https://csrc.nist.gov .

You have JavaScript disabled. This site requires JavaScript to be enabled for complete site functionality.

An official website of the United States government

Here’s how you know

Official websites use .gov A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS A lock ( Lock Locked padlock icon ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

NIST CSWP 36 (Initial Public Draft)

Applying 5g cybersecurity and privacy capabilities: introduction to the white paper series.

    Documentation     Topics

Date Published: August 15, 2024 Comments Due: September 16, 2024 Email Comments to: [email protected]

Michael Bartock (NIST) , Jeffrey Cichonski (NIST) , Murugiah Souppaya (NIST) , Karen Scarfone (NIST) , Parisa Grayeli (MITRE) , Sanjeev Sharma (MITRE) , Thomas McCarthy (AMI) , Muthukkumaran Ramalingam (AMI) , Presanna Raman (AMI) , Stefano Righi (AMI) , Jitendra Patel (AT&T) , Bogdan Ungureanu (AT&T) , Tao Wan (CableLabs) , Matthew Hyatt (Cisco) , Steve Vetter (Cisco) , Dan Carroll (Dell Technologies) , Steve Orrin (Intel) , Corey Piggott (Keysight Technologies) , Michael Yeh (MiTAC Computing) , Gary Atkinson (Nokia Bell Labs) , Rajasekhar Bodanki (Nokia Bell Labs) , Don McBride (Nokia Bell Labs) , Aarin Buskirk (Palo Alto Networks) , Bryan Wenger (Palo Alto Networks) , Todd Gibson (T-Mobile)

Announcement

5G technology for broadband cellular networks will significantly improve how humans and machines communicate, operate, and interact in the physical and virtual world. 5G provides increased bandwidth and capacity, and low latency. However, professionals in fields like technology, cybersecurity, and privacy are faced with safeguarding this technology while its development, deployment, and usage are still evolving.

To help, the NIST National Cybersecurity Center of Excellence (NCCoE) has launched the Applying 5G Cybersecurity and Privacy Capabilities white paper series. The series targets technology, cybersecurity, and privacy program managers within commercial mobile network operators, potential private 5G network operators, and organizations using and managing 5G-enabled technology who are concerned with how to identify, understand, assess, and mitigate risk for 5G networks. In the series we provide recommended practices and illustrate how to implement them. All of the capabilities featured in the white papers have been implemented in the NCCoE testbed on commercial-grade 5G equipment.

We are pleased to announce the following white paper which introduces the series: Applying 5G Cybersecurity and Privacy Capabilities: Introduction to the White Paper Series . This publication explains what you can expect from each part of the series: information, guidance, recommended practices, and research findings for a specific technical cybersecurity or privacy-supporting capability available in 5G systems or their supporting infrastructures.

This document introduces the white paper series titled Applying 5G Cybersecurity and Privacy Capabilities. This series is being published by the National Cybersecurity Center of Excellence (NCCoE) 5G Cybersecurity project. Each paper in the series will include information, guidance, and research findings for an individual technical cybersecurity- or privacy-supporting capability available in 5G systems or their supporting infrastructures. Each of the capabilities has been implemented in a testbed as part of the NCCoE project, and each white paper reflects the results of that implementation and its testing.

Control Families

None selected

Documentation

Publication: https://doi.org/10.6028/NIST.CSWP.36.ipd Download URL

Supplemental Material: 5G Cybersecurity Project

Publication Parts: CSWP 36A

Document History: 08/15/24: CSWP 36 (Draft)

general security & privacy

communications & wireless

telecommunications

• Skip to main content
• Skip to search
• Skip to footer

Products and Services

Cisco Security

Master your goals. innovate. we'll tackle threats..

Get powerful security across all your networks, cloud, endpoints, and email to protect everything that matters, from anywhere.

If it's connected, you're protected

Cisco Security “The Hacker”

More connected users and devices creates more complexity. Cisco Security Cloud makes security easier for IT and safer for everyone anywhere security meets the network.

Deliver smarter, stronger security

Protect your organization across a multicloud environment, while simplifying security operations, improving scalability, and driving data-informed outcomes, powered by Cisco Talos.

Unlock better user experiences

Create a seamless experience that frustrates attackers, not users, by granting access from any device, anywhere, and adding more proactive security controls.

Deliver cost-effective defenses

Improve ROI by consolidating vendors, reducing complexity and integrating your security.

Strengthen security resilience

Unified, end-to-end protection maximizes value, minimizes risk, and closes security gaps everywhere to defend against evolving threats. Protect access, apps, and innovation across your network to secure your future.

Cisco Secure Firewall

Better visibility and actionable insights across networks, clouds, endpoints, and email allows users to respond confidently to the most sophisticated threats at machine scale.

Featured security products

Cisco hypershield.

A new groundbreaking security architecture that makes hyperscaler technology accessible to enterprises of all sizes and delivers AI-native security for modern data centers and cloud.

Cisco Secure Access (SSE)

A converged cybersecurity solution, grounded in zero trust, that radically reduces risk and delights both end users and IT staff by safely connecting anything to anywhere.

Detect the most sophisticated threats sooner across all vectors and prioritize by impact for faster responses.

Cisco Multicloud Defense

Gain multidirectional protection across clouds to stop inbound attacks, data exfiltration, and lateral movement.

Secure applications and enable frictionless access with strong MFA and more. Establish user and device trust, gain visibility into devices, and enable secure access to all apps.

Cisco Identity Services Engine (ISE)

Simplify highly secure network access control with software-defined access and automation.

Security Suites delivered by Cisco Security Cloud

Cisco User Protection Suite

Get secure access to any application, on any device, from anywhere. Defend against threats targeting users and deliver seamless access for hybrid work.

Cisco Cloud Protection Suite

Secure your apps and data with a powerful, flexible framework for a hybrid and multicloud world.

Cisco Breach Protection Suite

Secure your business by investigating, prioritizing, and resolving incidents through unified defense and contextual insights from data-backed, AI-powered security.

Customer stories and insights

Global partnerships fight to end child exploitation together.

"Marriott has long championed human rights and human trafficking awareness. Combating CSAM is an important extension of that work. The IWF provided the level of rigor we needed in a URL list, and Cisco's security technology provided the means to easily apply it."

Abbe Horswill, Director, Human Rights and Social Impact

Company: Marriott International

The NFL relies on Cisco

"From securing stadiums, broadcasts, and fans to protecting the largest live sporting event in America, the right tools and the right team are key in making sure things run smoothly, avoiding disruptions to the game, and safeguarding the data and devices that make mission-critical gameday operations possible."

Add value to security solutions

Cisco Security Enterprise Agreement

Instant savings

Experience security software buying flexibility with one easy-to-manage agreement.

Services for security

Let the experts secure your business

Get more from your investments and enable constant vigilance to protect your organization.

Sharpen your security insights

Cisco Cybersecurity Viewpoints

Set your vision to a more secure future with Cisco Cybersecurity Viewpoints. With specialized content from podcasts to industry news, you'll walk away with a deeper understanding of the trends, research, and topics in our rapidly changing world.

More From Forbes

Vital primer about technical ai governance (taig) goes the extra mile.

• Share to Facebook
• Share to Twitter
• Share to Linkedin

Nailing Technical AI Governance (TAIG) for best success when developing and fielding modern-day AI.

In today’s column, I am continuing my ongoing coverage regarding the governance of AI, see my prior dozens upon dozens of discussions and analyses at the link here and the link here , just to name a few.

Readers are aware of my longstanding persistence and frank remarks on this weighty topic, along with my many presentations at conferences and summits. The overall governance of AI is still being worked out, and if we don’t get things established in the right way, we will in a sense reap what we sow and end up in a regretful morass. All hands are needed on deck. AI governance must be kept at the front and center of our minds and actions.

The good news is this. A newly released paper on Technical AI Governance (TAIG) will be my focus here in today’s column and provides a prized primer of a technical nature on what is happening and where we need to go on the vital and rapidly evolving matter of how to best govern AI. I applaud the researchers who put the paper together. Of course, laudable too are the many referenced works that underlie the useful compilation and analysis by the authors.

I will go ahead and identify key highlights from the paper and add commentary to showcase the crucial basis for the examined topics. Readers are urged to dive into the extensive paper for additional details and the nitty-gritty. It’s worthwhile reading, for sure.

Let’s get underway.

Governance Of AI Is A Key Priority

Best High-Yield Savings Accounts Of 2024

Best 5% interest savings accounts of 2024.

Just in case you aren’t already generally up-to-speed, the need for sensible and suitable governance of AI is a big consideration and a top-of-mind concern. There are plenty of day-to-day issues and even potential existential challenges that arise with the expanding use and advancement of AI.

I’ll do a brief overview tour to make sure we are all on the same page.

One notable consideration is the realization that AI is a dual-use proposition, see my discussion at the link here .

This means that on the one hand, AI can be used for the good of humankind, such as aiding in curing cancer or assisting the attainment of notable goals such as the United Nations Sustainability Development Goals (SDGs), as I depicted in the link here . Meanwhile, lamentedly, the same or similar AI can oftentimes be readily recast into adverse uses that harm or endanger humanity. Envision AI that is upliftingly devised to detect toxic chemicals and prevent humans from being harmed that with a few simple changes can be aimed at crafting new toxins that could be used for mass destruction. That’s a dual-use proposition.

One moment, AI is used to benefit humanity, the next moment it is the keystone of so-called Dr. Evil projects.

Many gotchas about contemporary AI are less obvious and not necessarily headline-grabbing.

One such everyday qualm is that AI might contain undue biases as a result of the data training or algorithms being utilized, see my coverage at the link here and the link here . Consider this. You go to get a mortgage for your home and are turned down by an AI loan approval app. Why didn’t you get approved? Firms will at times just shrug their shoulders and insist that the AI said you aren’t qualified. Period, end of story.

They use the AI app as a wink-wink protective shield and bluster you into assuming that the AI “must be right” and dare not question how or why it made a life-altering decision about you. It could be that hidden within the AI internals there are computational paths that are based on discriminatory factors. You are none the wiser and get nixed unfairly, perhaps illegally.

All of this can happen on a nearly unimaginable scale. If an employed loan rep or agent were making biased decisions about loans, they presumably would not likely have a far reach. In the case of AI, ramping up the scale is relatively trivial. An AI loan approval app can be run on a multitude of servers in the cloud and perform its actions on a massive scale. Thousands, hundreds of thousands, and even many millions of people might be impacted by AI apps that are doing the wrong thing and getting away with it.

A sneaky angle is to proclaim that the AI did it, as though the AI was able to be cognizant and act on its own accord. We do not yet have legal personhood for AI, see my discussion at the link here , nor is any of today’s AI sentient, see my explanation at the link here , and thus it is false to suggest that the AI was responsible for the actions undertaken. People develop and field AI. People are supposed to be responsible for what they do.

People need to be held accountable.

Governance Of AI Dovetails Into Human Responsibility

This has given rise to the significance of AI Ethics and AI Law, namely, respectively, AI Ethics is the ethical considerations underlying AI, see my discussion at the link here , and AI Law is the legal ramifications associated with AI, see the link here .

We refer to Responsible AI or Accountable AI as a means of asserting that people making and fielding AI ought to abide by various ethical principles and legal precepts, see my elucidation at the link here . The idea is that those devising and implementing AI cannot just wave their arms and say they had no awareness of what their AI might do. They are obligated in ethical and legal ways to think things through, seek to include double-checks and precautionary mechanisms, and ultimately have solemn and formal responsibility for what their AI undertakes.

As with most things in life, there is controversy and gray areas that can arise.

A continual societal and cultural battle is underway between wanting to stretch the boundaries of AI and at the same time seeking to keep AI within suitable bounds, see the link here . You’ve undoubtedly heard of this heated debate or read about it. The logic goes like this. If we put in place new laws governing AI, this will stifle innovation. The AI that you hoped might solve world hunger is going to be onerously delayed or maybe never be developed. Allow AI makers to roam free else innovation will be extinguished.

The other side of the innovation-at-all-cost coin is that you are handing the keys to devising and fielding unbridled AI without any semblance of control. In the techno-sprinting rush to come out with brand-new whiz-bang AI, checks and balances are invariably left by the wayside. Be the first, that’s the mantra, and clean up on aisle seven later on. The deal is that though you might get innovation, maybe, it can come at a hefty cost to people’s safety and security. The next thing you know, in a sense, people are getting hurt (financially, mentally, physically) because you handed out the keys without dutiful restrictions and controls in place.

Without AI governance the free trajectory might land anywhere.

So, which is it, do we allow the horse wildly out of the barn, do we restrict the horse but maybe stifle what the horse can accomplish, and/or can we find some reasonable balance amongst two otherwise seemingly polarized ends of the spectrum?

There is more, a lot more.

Another viewpoint is the larger scoped global sphere associated with AI.

Countries are concerned that if they don’t avidly pursue AI, they will fall behind other countries that are doing so, see my coverage at the link here . This might mean that the countries that are lagging in AI will become economically and politically disadvantaged. Those countries at the leading edge of AI will possibly rise to be geopolitical powerhouses. They might wield their advanced AI in untoward ways, threatening other nations, strongarming other nations, and so on, see my discussion at the link here .

All of this boils down to something of grand significance, consisting of, yes, you guessed it, the governance of AI.

I hope you can see from my quick overview that there are indubitably nuances, twists and turns, and the whole kit and kaboodle is mired in tradeoffs. There are no easy answers to be had. If you are looking for something interesting, important, and challenging to work on, please consider the governance of AI as a topic for your devout attention. We definitely need more eyes and ears on these vital matters.

Governance Of AI Has Lots Of Hands Afoot

I’ve said repeatedly and vociferously that it takes a village to appropriately figure out the governance of AI.

There are all sorts of specialties and avenues for those interested in the governance of AI. By that, I am asserting that we need a mixture of all kinds of stakeholders to enter the dialogue and deliberations. No singular subset of stakeholders will do. The problem afoot is multi-faceted and requires experts from many walks of life. Governance of AI is a decidedly team sport when done right.

The governance of AI is best tackled via a myriad of angles:

• Overall policymaking as per leaders, regulators, lawmakers, etc.
• National, state, and local considerations.
• Multinational perspectives for global considerations.
• Business and economic determinations.
• AI Ethics perspectives.
• AI Law per legal implications.
• AI technological facets.

That’s a lot of hands and lots of opportunity for greatness, while at the same time lots of potential for confusion, miscommunication, missed handoffs, and similar difficulties.

I witness this daily.

In my role serving on several national and international AI standards bodies, along with my advisement to congressional leaders and other officeholders, a crucial element that I have seen often become an especially problematic issue is the gap between the AI tech side of things and those that are tasked with policymaking and the like.

Here’s what that signifies.

You can end up with non-technical policymakers that only tangentially or vaguely grasp the technical AI facets of whatever AI governance subtopic is at hand. Due to their distance from the technical underpinnings, they are unable to discern what’s what. As a result, sadly, they at times compose AI governance language that is off target. They genuinely think or believe they are on target, but their lack of technical AI expertise prevents them from realizing they are amiss.

Confounding the matter is the circumstance of AI technical experts who then try doggedly to explain or articulate the AI advances to such policymakers. This at times is nearly comical, were it not so serious a matter, in that the AI experts will assume that all they need to do is pour out more and more technical facts and figures to get the policymakers into the needed frame of mind. Often, this doesn’t work out.

Things can get even more tousled.

There are situations whereby policymakers ask AI technical experts to write what the AI governance stipulations should be. The odds are that the language used will be technically accurate but legally or ethically have gaping holes. Those AI experts are versed in the language of technology, not the language of policymaking.

Policymakers might seek to scrutinize the language and sometimes, even if not able to understand it, figure they will simply push it forward since the techies say it is golden. Later, once enacted, all manner of legal interpretations arise that turn the depictions upside down. It becomes a legal entanglement of epic proportions.

Something that equally is disturbing consists of policymakers that aren’t versed in AI technical language that opt to change initial draft language as based on their policymaking expertise. The assumption is that edits here or there will turn the AI technical indications into silver-tongued policies. Unfortunately, this tends to change the meaning of the AI technical indications and render the seemingly policy-strong rendition into a confusion of what aspects of AI are being encompassed.

Think of all this as two clouds passing in the night. There is the AI technical side. There is the policymaking side. At times, they drift past each other. In other cases, they get mixed together in the worst of ways, ultimately creating blinding snowstorms, ferocious thunder and lightning, but not providing the clear-as-day language needed for governance of AI purposes.

Another way I often describe this is by invoking the Goldilocks principle.

It goes like this. If policies for the governance of AI are overly one-sided in terms of polished policy language but discombobulated AI-technical language, the porridge is said to be too cold. The other direction is the governance of AI language that is AI-technically polished but discombobulated as to the policy language at play, which is a porridge that is too hot.

The right way to go is the Goldilocks principle. Get the AI technical side correct and apt. Get the policy side correct and apt. Dovetail them together correctly and aptly. Do not fall or fail on either side. The most successful approach entails devising the two hand-in-hand. Any attempt to simply toss the language of one to the other, doing so over the transom, is likely doomed to be a flop.

I realize that seems blazingly obvious and you might assume that everyone would do things the right way. It seems as apparent as apple pie. Well, I dare to suggest that the real world doesn’t come out that way, certainly not all the time, indeed, not even most of the time.

The real world is a tough place to be, especially when seeking to do right by the governance of AI.

Concentrating On Technical AI Governance (TAIG)

I trust that I have whetted your appetite for what will next be the main course of this meal.

As noted earlier, there is a recently posted paper on Technical AI Governance (TAIG) that has done a superb job of pulling together the otherwise widely disparate breakthroughs and advances involved in the governance of AI from a technology perspective. I am eager to walk you through the essence of the paper.

Here we go.

The paper is entitled “Open Problems in Technical AI Governance” by Anka Reuel, Ben Bucknall, Stephen Casper, Tim Fist, Lisa Soder, Onni Aarne, Lewis Hammond, Lujain Ibrahim, Alan Chan, Peter Wills, Markus Anderljung, Ben Garfinkel, Lennart Heim, Andrew Trask, Gabriel Mukobi, Rylan Schaeffer, Mauricio Baker, Sara Hooker, Irene Solaiman, Alexandra Sasha Luccioni, Nitarshan Rajkumar, Nicolas Moës, Neel Guha, Jessica Newman, Yoshua Bengio, Tobin South, Alex Pentland, Jeffrey Ladish, Sanmi Koyejo, Mykel J. Kochenderfer, and Robert Trager, arXiv , July 20, 2024.

At a high level, these are key ingredients of the TAIG compilation and analysis:

• “The rapid development and adoption of artificial intelligence (AI) systems has prompted a great deal of governance action from the public sector, academia, and civil society.”
• “However, key decision-makers seeking to govern AI often have insufficient information for identifying the need for intervention and assessing the efficacy of different governance options.”
• “We define AI governance as the processes and structures through which decisions related to AI are made, implemented, and enforced. It encompasses the rules, norms, and institutions that shape the behavior of actors in the AI ecosystem, as well as the means by which they are held accountable for their actions.”
• “Furthermore, the technical tools necessary for successfully implementing governance proposals are often lacking, leaving uncertainty regarding how policies are to be implemented.”
• “As such, in this paper, we aim to provide an overview of technical AI governance (TAIG), defined as technical analysis and tools for supporting the effective governance of AI.”
• “By this definition, TAIG can contribute to AI governance in a number of ways, such as by identifying opportunities for governance intervention, informing key decisions, and enhancing options for implementation.”

The attention to TAIG is sorely needed and the paper provides nearly fifty pages of insightful curation, summary, and analysis, plus nearly fifty additional pages of cited works.

For those of you who are doing or considering doing research in TAIG, you ought to use this paper as an essential starting point. Besides reading the paper, you can glean a lot from the cited works portion. Take a look at the listed references that are cited. This can aid in revealing both what and who has been making inroads on TAIG. Proceed to access and assimilate the content of those cited works.

Naturally, this one paper doesn’t cover all prior work, so make sure to look beyond the references given. Another consideration is that this paper is a point-in-time endeavor. The field of TAIG is rapidly evolving. You can’t just read the paper and think you are done with your homework. You have only begun. Get plugged into the TAIG realm and ensure you are reading the latest posted research on an ongoing basis.

Moving on, I next want to explore the framework that the paper proposes for seeing the big picture of TAIG.

Their framework or taxonomy is essentially a matrix consisting of rows that list what they refer to as capacities and the columns are what they define as targets. They describe the matrix this way (excerpts):

• “We present a taxonomy of TAIG arranged along two dimensions: capacities , which refer to actions such as access and verification that are useful for governance, and targets , which refer to key elements in the AI value chain, such as data and models, to which capacities can be applied.” (ibid).
• “We outline open problems within each category of our taxonomy, along with concrete example questions for future research.” (ibid).
• “At the same time, we are conscious of the potential pitfalls of techno-solutionism – that is, relying solely on proposed technical fixes to complex and often normative social problems – including a lack of democratic oversight and introducing further problems to be fixed.” (ibid).
• “Furthermore, some of the TAIG measures highlighted are dual-use. For example, while hardware-enabled mechanisms for monitoring advanced compute hardware could provide increased visibility into the private development of the largest models, they could also potentially be applied to unreasonably surveil individuals using such hardware for legitimate purposes.” (ibid).

I relished that they emphasized the dangers of techno-solutionism.

Allow me to elaborate.

Suppose that a concern is raised that an AI system seems to contain undue bias. Again, this is not sentience, it is due to data training or algorithms that steer the AI system in a discriminatory direction.

Someone with an AI techie bent might instantly proclaim that this bias can be solved via a programming fix. They tweak the algorithm so that the specifically noted bias is now shall we say corrected and will no longer be applied. Whew, problem solved, everyone can go back to relaxing and stand down from an all-hands alert.

Imagine though that the bias was only one of many that were lingering in the AI. It could be that the data used for training contained a wide variety of undue biases. Perhaps the data was based on discriminatory practices across the board, having been done for many years. All in all, the AI mathematically and computationally pattern-matched on the data and now has a rat’s nest of these hidden biases.

The one-time one-focus fix was like plugging the hole in the dam with your little finger. There wasn’t any effort expended toward discerning what else might be amiss. It was a rush to judgment and make a quick fix for an issue or problem of a much larger nature associated with the AI in total.

That’s what can happen when techno-solutionist blinders are being worn. The chances are that a technological fix will be the only idea that comes to mind. It is the veritable adage that if all you know is a hammer, the entire world seems to be a nail, fixable exclusively via hammering, even when say a screwdriver or other tool might be a wiser choice.

The gist is that though TAIG is vital, we need to bring into the huddle all the other dimensions and facets when holistically considering how to resolve or solve various AI governance considerations. Notably, the paper acknowledges that those other views are crucial. I’ve seen some papers that do not mention that point, possibly leading the reader down a primrose path that all they need to do is be totally proficient at TAIG and nothing else matters.

Nope, don’t fall into that mental trap, thanks.

Another point they make that is worthy of noting consists of identifying the dual-use properties of AI. I already discussed that earlier. The crux is that whatever governance of AI is devised, it must be able to handle not just the goodness pursuits of AI, but also recognize and cope with how to govern the evildoer pursuits of AI too.

Sorry to report that there are indeed bad people out there.

On top of that, we must also consider those who are not bad but who by happenstance trip over their own feet into badness. How so? Here is what I am saying. Let’s envision an AI maker who has purist intentions and develops AI that can defuse bombs. No more human intervention or human risk involved. Good for the world. Happy face.

Turns out that someone else comes along and readily tweaks the AI to devise bombs that are extraordinarily hard to defuse. The tactics that are in the AI to defuse bombs are handily all in one place. It would have been arduous to otherwise figure out what ways bombs are defused. Now, via a few quick changes to the AI, the AI serves up all kinds of deplorable means of making bombs that are incredibly hard to defuse.

The AI maker didn’t think about that. They were enamored of their heroic pursuit to defuse bombs. In their erstwhile development of AI, it never dawned on them that this could happen. The casual passerby didn’t need to lift a finger per se and had the AI maker do all the heavy lifting for them.

Again, that’s why the governance of AI across all dimensions is so crucial.

It can stir those who are making AI to consider and reconsider what they are doing. This does not need to be an on/off-only stipulation. It could be that by the use of various technical precautions, we can reduce the risks of these switchable dual-use AI dilemmas. Make the effort to switch the core of the AI high enough that the hurdle to doing so becomes much tougher to overcome.

And, before I seem to have suggested an option that is techno-solutionism, meaning that I alluded to the idea that a technical fix by itself might help, we can also consider for example the legal considerations too. Perhaps AI laws might state that when dual use is a possibility, AI makers are obligated to undertake precautionary measures. They will be stirred toward thinking about what the AI can and might do, what ways to devise the AI, and whether they ought to be devising the AI at all.

This might not be on their minds otherwise and they can oftentimes become fixated on stretching AI without a sense of asking whether and how they are doing so has sobering risks or downsides.

Entering Into The Matrix On TAIG Is Quite Helpful

I noted that the paper identifies essentially a set of rows and columns for proffering a framework or taxonomy of TAIG. Establishing or even floating a taxonomy is a useful means of organizing a field of inquiry into a structured approach. You can then put together the puzzle pieces into a holistic whole. From this, you can identify what is being missed, and what is being well-covered, and generally understand the lay of the landscape.

They identify various capacities, consisting of six rows, and then various targets, consisting of four columns.

Here is what those are:

• Capacities ( six rows ): (1) Assessment, (2) Access, (3) Verification, (4) Security, (5) Operationalization, (6) Ecosystem Monitoring.
• Targets ( four columns ): (a) Data, (b) Compute, (c) Models and Algorithms, (d) Deployment.

You ought to view this as the conceptual infrastructure or scaffolding that you can then take say a particular capacity, such as “Assessment”, and proceed to examine Assessment via the four distinct viewpoints of “Data”, “Compute”, “Models and Algorithms”, and “Deployment”. Do the same for “Access”, such as examining Access via the four distinct viewpoints of Data, Compute, Models and Algorithms, and Deployment. And so on for the remaining list of capacities.

Do you have that snugly tucked away in your noggin?

Good, kudos.

On a brief aside, regarding the “rows” of capacities and “columns” of targets, I do want to mention that you can flip this orientation around if that’s your preference. There is nothing wrong with flipping the matrix and thinking of this as rows of targets and columns of capacities, especially if you are a researcher who concentrates on the “targets” aspects. You might find the switcheroo more appealing. Do you.

Next, let’s see how the paper defines the notion of capacities and targets (excerpts):

• “Capacities encompass a comprehensive suite of abilities and mechanisms that enable stakeholders to understand and shape the development, deployment, and use of AI, such as by assessing or verifying system properties.” (ibid).
• “These capacities are neither mutually exclusive nor collectively exhaustive, but they do capture what we believe are the most important clusters of technical AI governance.” (ibid).
• “The second axis of our taxonomy pertains to the targets that encapsulate the essential building blocks and operational elements of AI systems that governance efforts may aim to influence or manage.” (ibid).
• “Each capacity given above can be applied to each target.” (ibid).
• “We structure our paper around the resulting pairs of capacities and targets, with the exception of operationalization and ecosystem.” (ibid).

The paper mentions that they are drawing upon a wide range of research and literature, including from varied domains such as Machine Learning (ML) theory, Applied ML, cybersecurity, cryptography, hardware engineering, software engineering, and mathematics and statistics. You would be more likely to appreciate the primer if perchance you have some knowledge of those underpinnings. Just giving you a friendly heads-up.

Each of the capacities is carefully delineated and defined, likewise for the targets.

Here are the short-version definitions for capacities (excerpts):

• (1) “ Assessment : The ability to evaluate AI systems, involving both technical analyses and consideration of broader societal impacts.” (ibid).
• (2) “ Access: The ability to interact with AI systems, including model internals, as well as obtain relevant data and information while avoiding unacceptable privacy costs.” (ibid).
• (3) “ Verification : The ability of developers or third parties to verify claims made about AI systems’ development, behaviors, capabilities, and safety.” (ibid).
• (4) “ Security : The development and implementation of measures to protect AI system components from unauthorized access, use, or tampering.” (ibid).
• (5) “ Operationalization : The translation of ethical principles, legal requirements, and governance objectives into concrete technical strategies, procedures, or standards.” (ibid).
• (6) “ Ecosystem Monitoring : Understanding and studying the evolving landscape of AI development and application, and associated impacts.” (ibid).

Here are the short-version definitions for targets (excerpts):

• (a) “ Data: The pretraining, fine-tuning, retrieval, and evaluation datasets on which AI systems are trained and benchmarked.” (ibid).
• (b) “ Compute: Computational and hardware resources required to develop and deploy AI systems.” (ibid).
• (c) “ Models and Algorithms : Core components of AI systems, consisting of software for training and inference, their theoretical underpinnings, model architectures, and learned parameters.” (ibid).
• (d) “ Deployment: The use of AI systems in real-world settings, including user interactions, and the resulting outputs, actions, and impacts.” (ibid).

I had just moments ago told you that you can think of this as rows of capacities and columns of targets (or the other way round if you prefer). In the rows as capacities and columns of targets, you can construe this as follows:

• Targets : (a) Data, (b) Compute, (c) Models and Algorithms, (d) Deployment.
• Targets: (a) Data, (b) Compute, (c) Models and Algorithms, (d) Deployment.

That above will hopefully instill in you the overall sense of the framework or taxonomy they are employing. The paper is demonstrably shaped around that design.

Here are some thoughts on approaching the paper.

If you are mainly interested in say security, you could presumably skim the rest of the material and go straight to the Security section. Within the Security section, you might decide you are only interested in Deployment. Voila, that’s the only portion you might deeply read, namely Security (as a capacity) and it's considered Deployment (as a target).

Suppose instead that you are mainly passionate about Data. You could look at the Data elements within each of the six capacities, exploring Data as it relates to (1) Assessment, (2) Access, (3) Verification, (4) Security, (5) Operationalization, and (6) Ecosystem Monitoring. There might be a particular instance that catches your eye. At that juncture, zone in and make that your best buddy.

My overarching recommendation is to read the entire paper and not just cherry-pick one specific spot. You are welcome to home in on a specific area of interest, but at least skim the rest of the paper too. I am urging that having a holistic mindset is going to do you the most overall good. If you opt to myopically only look at one subsection or sub-sub-section, I dare say you might not be seeing the forest for the trees.

Just a suggestion.

Sampler To Get You Further Into The Zone

There isn’t available space here for me to go into the details underlying each of the capacities and their respective targets. That’s why you ought to consider reading the paper. Boom, drop the mic.

I would like to provide a glimpse of what you will find, doing so by doing a whirlwind tour of the capacity labeled as Assessment. Buckle up for a fast ride.

Recall that a moment ago I indicated that they defined Assessment this way:

• (1) “Assessment: The ability to evaluate AI systems, involving both technical analyses and consideration of broader societal impacts.” (ibid).

They go into a much deeper depiction and provide cited references that have done a great deal of work on the topic.

As a further sampler about Assessment, here is a snippet I’d like you to see (excerpt):

• “Evaluations and assessments of the capabilities and risks of AI systems have been proposed as a key component in AI governance regimes. For example, model evaluations and red-teaming comprised a key part of the voluntary commitments agreed between labs and the UK government at the Bletchley Summit. Furthermore, the White House Executive Order on Artificial Intelligence requires developers of the most compute-intensive models to share the results of all red-team tests of their model with the federal government.” (ibid).

I’ve in my column covered the importance of red-team testing for AI, see the link here , and given repeated attention to the numerous White House executive orders concerning AI, see the link here . The research paper does a yeoman’s job of digging into the details.

One of the especially fascinating aspects is their listing of open questions that are still being explored in the given domain and sub-domains. There is an old saying that the way to really know about a subject is by knowing what questions remain unanswered. It tells you volumes about what is known and what is still being pondered.

When I was a professor, I often advised my graduate students and undergraduate students to examine prevailing open questions and pick one that suits their interests. The nice thing is that they would then be somewhat assured that the topic at hand isn’t already packed up and put away. This is crucial for their academic pursuits. If you pick a topic that seems to be completely resolved, and unless you get lucky and find some hidden treasure, you are beating a dead horse, as it were. You will only be treading the same terrain that has already been trodden upon (this can be useful on a confirmational basis, but usually won’t earn you many gold stars).

To help yourself and help the advancement of knowledge, choose a topic that still has open questions. You might make a contribution that resolves the said matters. Even if that doesn’t seem in the cards, the odds are that you’ll make some progress, and others following in your footsteps will be able to leverage whatever steps you’ve made.

As a furtherance of sampling, I will share with you just one selected open question under Assessment for each of the four targets of Data, Compute, Models and Algorithms, and Deployment.

Here are the ones I opted to pluck out of the respective lists:

• Capacity: Assessment; Target: (a) Data – “How can methods for identifying problematic data be scaled to large (on the magnitude of trillions of tokens/samples) datasets?”
• Capacity: Assessment; Target: (b) Compute – “How efficiently can AI models be trained using a large number of small compute clusters?”
• Capacity: Assessment; Target: (c) Models and Algorithms – “How can potential blind spots of evaluations be identified?”
• Capacity: Assessment; Target: (d) Deployment – “How can dynamic simulation environments be designed to better reflect real-world environments?”

Each of those is a gem.

I shall pick one, though it is tempting to want to expand upon each of them. Life offers tough choices.

The first open question above on Assessment and the target of Data asks what kind of technological means can be devised to discover problematic data in extremely large datasets. You would want to do this when performing the Assessment of a budding AI system, or possibly do so with an existing AI system, after the fact but wanting to see what maybe was missed at the get-go.

Let’s contemplate this.

I’ll tie this back to my remarks about potential bias hidden in data used for data training of AI.

Look before you leap is a handy watchword in these matters. Before you leap into data training for a budding AI system, you ought to think and look carefully at the data that is being used. Don’t merely scan, ingest, or digest data without any preparatory analysis. That is AI Development 101 in my classes.

Okay, so you decide that you will do things right by examining whatever data is being used for the data training. This can be a bigger piece of pie than you can chew. The quantity of computational resources to analyze the voluminous data might be humongous. There is a cost involved. There is time involved in terms of wanting to proceed ahead on the AI but possibly sitting around twiddling thumbs while the data analysis is occurring.

What techniques and technologies can do this effectively and efficiently?

The aim is to use the least amount of computation to get the most bang for the buck out of finding problematic data. Your newly discovered or invented methods might enable faster advancement for AI systems. It might reduce the cost of devising AI systems and make it less costly to develop them. Furthermore, assuming the capability does a buffo job of finding problematic data, you are helping to avert downstream issues.

When I refer to downstream issues, this goes back to my example about the discovery of a bias once an AI is already in production and being used. Trying to deal with data issues at that stage is way late. Perhaps customers or clients have already suffered harm. There might be penalties assessed for what the AI maker did. All of this might have been avoided had the right tool in the right place at the right time been able to identify problematic data upstream, before all the other subsequent steps of developing and fielding the AI. For more about the significance of thinking about AI upstream and downstream, see my analysis at the link here .

I challenge you as follows.

If TAIG is something you profoundly care about, and you want to try and make a mark in this realm, mindfully explore the open questions listed in the research paper. Find one or more that speak to you. If you can’t find any that do so, feel free to divine additional questions that aren’t perchance listed in the paper. You can readily devise additional questions by reviewing the content and scouring the research in whichever sub-domain has piqued your interest.

I assure you that there is an ample supply of open questions.

What is your motivation to dive in?

Easy-peasy, fame, fortune, being a contributor, advancing knowledge, solving challenging puzzles, and otherwise putting your mind to work. Maybe in fact improving AI so that we can truly garner the benefits and better mitigate the gotchas and troubling hazards. If you like, saving the world (perhaps that’s a slight overstretch, but you get the drift).

Hopefully, that’s enough to inspire you.

Congratulations, you are now familiar with AI governance, especially the dimension having to do with the technical or technological elements. I bestow upon you an honor badge for your interest and courage. Score one for humankind.

What’s next for you?

If Technical AI Governance (TAIG) is your bailiwick or might become so, reading the research paper as a primer would seem prudent. Here’s a link to the paper for your ease of access, see the link here.

I’ll select one more quote for now from the paper, allowing me to make a final point: “We note that technical AI governance is merely one component of a comprehensive AI governance portfolio, and should be seen in service of sociotechnical and political solutions. A technosolutionist approach to AI governance and policy is unlikely to succeed.” (ibid).

Notice that the expressed viewpoint is that TAIG Is just one of many domains and stakeholder roles that are crucial to all-around robust AI governance. I pointed this out at the outset of this discussion and am glad to bring it back into focus, here at the conclusion of this discussion.

Suppose that the technical side of AI isn’t your forte. That’s fine. No worries. You can become an active participant and contributor in many other ways. This is a village of many.

Vince Lombardi famously said this: “Individual commitment to a group effort — that is what makes a team work, a company work, a society work, a civilization work.”

Join the team, you are appreciated, and needed, and can shape the future of AI and possibly humanity. Enough said.

• Editorial Standards
• Reprints & Permissions

Join The Conversation

One Community. Many Voices. Create a free account to share your thoughts. 

Forbes Community Guidelines

Our community is about connecting people through open and thoughtful conversations. We want our readers to share their views and exchange ideas and facts in a safe space.

In order to do so, please follow the posting rules in our site's  Terms of Service.   We've summarized some of those key rules below. Simply put, keep it civil.

Your post will be rejected if we notice that it seems to contain:

• False or intentionally out-of-context or misleading information
• Insults, profanity, incoherent, obscene or inflammatory language or threats of any kind
• Attacks on the identity of other commenters or the article's author
• Content that otherwise violates our site's  terms.

User accounts will be blocked if we notice or believe that users are engaged in:

• Continuous attempts to re-post comments that have been previously moderated/rejected
• Racist, sexist, homophobic or other discriminatory comments
• Attempts or tactics that put the site security at risk
• Actions that otherwise violate our site's  terms.

So, how can you be a power user?

• Stay on topic and share your insights
• Feel free to be clear and thoughtful to get your point across
• ‘Like’ or ‘Dislike’ to show your point of view.
• Protect your community.
• Use the report tool to alert us when someone breaks the rules.

Thanks for reading our community guidelines. Please read the full list of posting rules found in our site's  Terms of Service.

Rambling Trump, Musk interview marred by tech issues

• Medium Text
• Musk blames cyberattack for technical problems
• Interview attracts more than 1.3 million listeners
• Musk opens with questions about assassination attempt
• Trump says Harris "looks like the most beautiful actress"

HARRIS MOMENTUM

Trump back on x.

Sign up here.

Reporting by Richard Cowan and Andy Sullivan; Additional reporting by Alexandra Ulmer, Kanishka Singh, Abhirup Roy, Eric Beech and Nathan Layne; Writing by Joseph Ax; Editing by Caitlin Webber, Alistair Bell, Heather Timmons and Stephen Coates

Our Standards: The Thomson Reuters Trust Principles. , opens new tab

Thomson Reuters

Andy covers politics and policy in Washington. His work has been cited in Supreme Court briefs, political attack ads and at least one Saturday Night Live skit.

Russia destroys Ukrainian unit armed with NATO weapons in Kursk region, RIA says

Russian forces destroyed a Ukrainian reconnaissance and sabotage unit that was armed with weapons from NATO countries in Russia's western Kursk region, the RIA state-run media agency reported on Friday, citing unidentified security sources.

IEEE Account

• Change Username/Password
• Update Address

Purchase Details

• Payment Options
• Order History
• View Purchased Documents

Profile Information

• Communications Preferences
• Profession and Education
• Technical Interests
• US & Canada: +1 800 678 4333
• Worldwide: +1 732 981 0060
• Contact & Support
• About IEEE Xplore
• Accessibility
• Terms of Use
• Nondiscrimination Policy
• Privacy & Opting Out of Cookies

A not-for-profit organization, IEEE is the world's largest technical professional organization dedicated to advancing technology for the benefit of humanity. © Copyright 2024 IEEE - All rights reserved. Use of this web site signifies your agreement to the terms and conditions.