Ing. Pablo Lorenzatto, Lic. Carlos Giudice, Lic. Matías Grinberg

Introduction

As AI systems keep evolving and being integrated in day to day life we are beginning to find new kinds of risks that were not as prevalent with previous Machine Learning systems. For the purpose of this work we identify two broad categories of risks:

• External: Risks mediated by an intentioned attacker who wants to influence the AI in some way to further their goals.
• Internal: Risks inherent to the model behavior without the influence of an external attacker.

The potential harm of these risks is explored in the NIST AI RFM [1], highlighting:

• Spreading false information

• Supporting harmful discourse

• Enabling dangerous knowledge

The first step in addressing these dangers is understanding the attack surface in each lifecycle phase. The OWASP AI guidelines [2] highlight:

• Development-time: data collection, preparation, and model training/acquisition.
• Usage: input/output interaction.
• Runtime attacks: in production.

AI systems are hardware, software, processes, and artifacts. OWASP recommends traditional cybersecurity and AI-specific controls of which we highlight these two:

• AI Governance ensures central visibility over AI, like data and other assets.
• Controlled model behavior ensures safety, reliability, and ethics, even without external intervention.

Threat Modeling

Inner Threats: AI as a Novel Threat Actor