Abstract
As artificial intelligence (AI) systems become embedded in high-stakes environments, security assessment must account for both the vulnerabilities of individual models and the dependencies through which their effects may influence a larger system. A central challenge is to connect these analyses with mitigation decisions while preserving the conditions and evidence that support each conclusion. This paper proposes a formal, lifecycle-aware security-testing framework that brings model-centric and system-centric analysis into a common assessment process. Within a finite scope, it relates threats to their data and deployment contexts, assesses mitigation outcomes using metric-specific criteria, and separates individual-defense evaluation from defense selection. To preserve the basis of these decisions, linked records associate findings with their source conditions and with any assumptions used in subsequent interpretation. A supporting reporting specification organizes this evidence for engineering and organizational review. Selected elements of the method are illustrated through reported digital-patch outcomes and dependency calculations in a perception example, with empirical observations distinguished from analytical assumptions.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data Availability
This study uses the publicly available COCO 2017 dataset. The experimental data generated in this study, including evaluation results, are not publicly deposited but are available from the corresponding author upon reasonable request.
References
Apruzzese, G., Anderson, H., Dambra, S., Freeman, D., Pierazzi, F.: “Real attackers don’t compute gradients”: Bridging the gap between adversarial ml research and practice. In: 2023 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), pp. 339–364 (2023). https://doi.org/10.1109/SaTML54575.2023.00031
Kiribuchi, N., Zenitani, K., Semitsu, T.: Securing AI systems: A guide to known attacks and impacts. arxiv:2506.23296 (2025)
Vassilev, A., Oprea, A., Fordyce, A., Anderson, H., Davies, X., Hamin, M.: Adversarial machine learning: A taxonomy and terminology of attacks and mitigations. Technical Report NIST AI 100-2 E2025, National Institute of Standards and Technology (2025). https://doi.org/10.6028/NIST.AI.100-2e2025. Accessed 04 Jun 2026.
Bloomfield, R., Rushby, J.: Assurance of AI systems from a dependability perspective. arxiv:2407.13948 (2025)
Chaitou, H., Bou-Chaaya, K., Njaka, H.A., Rais, H.: A holistic dynamic security testing framework for ai systems. In: Management of Digital EcoSystems, pp. 495–509. Springer, Cham (2027)
National Institute of Standards and Technology: Artificial intelligence risk management framework (ai rmf 1.0). Technical Report NIST AI 100-1, NIST (2023). https://doi.org/10.6028/NIST.AI.100-1
International Organization for Standardization (ISO), International Electrotechnical Commission (IEC): ISO/IEC 42001:2023 — Artificial intelligence — Management system (2023). https://www.iso.org/standard/42001. Accessed 26 Mar 2026
Databricks: A Practical Guide to Securing and Deploying AI: The Databricks AI Security Framework (2024). https://www.databricks.com/resources/whitepaper/databricks-ai-security-framework-dasf. Accessed: 23 Jun 2025
Google: Introducing Google’s Secure AI Framework (SAIF) (2023). https://blog.google/innovation-and-ai/technology/safety-security/introducing-googles-secure-ai-framework/. Accessed 26 Mar 2026
National Cyber Security Centre: Machine learning principles. Web page (2024). https://www.ncsc.gov.uk/collection/machine-learning-principles. Accessed 27 Mar 2026
Yang, G.-Y., Wang, F.: Taming silent failures: A framework for verifiable AI reliability. IEEE Reliab. Mag. 2(4), 46–55 (2025). https://doi.org/10.1109/MRL.2025.3626482
Shapira, A., Shigol, S., Shabtai, A.: Frame: Comprehensive risk assessment framework for adversarial machine learning threats. Appl. Soft Comput. 194, 114930 (2026). https://doi.org/10.1016/j.asoc.2026.114930
Goodfellow, I.J., Shlens, J., Szegedy, C.: Explaining and harnessing adversarial examples. In: International Conference on Learning Representations (ICLR), (2015)
Madry, A., Makelov, A., Schmidt, L., Tsipras, D., Vladu, A.: Towards deep learning models resistant to adversarial attacks. In: International Conference on Learning Representations (ICLR) (2018). arxiv:1706.06083
Tramèr, F., Zhang, F., Juels, A., Reiter, M.K., Ristenpart, T.: Stealing machine learning models via prediction apis. In: USENIX Security Symposium (2016). arxiv:1609.02943
IBM: Adversarial Robustness Toolbox (ART). Trusted-AI (2023). https://github.com/Trusted-AI/adversarial-robustness-toolbox. Accessed on 11 Jun 2025
Lee, M., Kolter, Z.: On physical adversarial patches for object detection, (2019). arXiv:1906.11897 arXiv preprint
Xu, W., Evans, D., Qi, Y.: Feature squeezing: Detecting adversarial examples in deep neural networks, (2017). arXiv:1704.01155 arXiv preprint
Dziugaite, G.K., Ghahramani, Z., Roy, D.M.: A study of the effect of jpg compression on adversarial images, (2016). arXiv:1608.00853 arXiv preprint
Guo, C., Rana, M., Cisse, M., Van Der Maaten, L.: Countering adversarial images using input transformations, (2017). arXiv:1711.00117 arXiv preprint
Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., Zitnick, C.L.: Microsoft COCO: Common Objects in Context. In: Computer Vision – ECCV 2014. Lecture Notes in Computer Science, vol. 8693, pp. 740–755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48
Ultralytics: YOLOv5. Software repository (n.d.). https://github.com/ultralytics/yolov5
Kumar, V., Mayo, J., Bahiss, K.: ADMIn: Attacks on Dataset, Model and Input. A Threat Model for AI Based Software. arxiv:2401.07960 (2024)
MITRE: ATLAS: Adversarial Threat Landscape for Artificial-Intelligence Systems. https://atlas.mitre.org/. Accessed 26 Mar 2026
OWASP: OWASP Top 10 for Large Language Model Applications. https://owasp.org/www-project-top-10-for-large-language-model-applications/. Accessed 26 Mar 2026
OWASP Foundation: OWASP AI Testing Guide v1. Web page (2025). https://owasp.org/www-project-ai-testing-guide/. Accessed 27 Mar 2026
Majumdar, S., Pendleton, B., Gupta, A.: Red teaming ai red teaming, (2025). ArXiv:2507.05538
Llorca, D.F., Hamon, R., Junklewitz, H., Grosse, K., Kunze, L., Seiniger, P., Swaim, R., Reed, N., Alahi, A., G’omez, E., Sánchez, I., Kriston, Á.: Testing autonomous vehicles and ai: perspectives and challenges from cybersecurity, transparency, robustness and fairness. European Transport Research Review 17 (2024)
Funding
No funding was received for this study.
Author information
Authors and Affiliations
Contributions
Hassan Chaitou contributed primarily to Sections 3 and 4. Njaka-Harilala Andriamandratomanana contributed primarily to Section 5. All authors contributed to the other sections of the manuscript, reviewed the full paper, and approved the final version.
Corresponding authors
Ethics declarations
Competing interests
The authors declare no competing interests.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Chaitou, H., Andriamandratomanana, NH. & Rais, H. A dual-layer security framework for AI systems: a lifecycle-aware security testing approach. World Wide Web 29, 50 (2026). https://doi.org/10.1007/s11280-026-01437-4
Received:
Revised:
Accepted:
Published:
Version of record:
DOI: https://doi.org/10.1007/s11280-026-01437-4