- The three real attack surfaces are training/tuning data, the prompt and retrieval pipeline, and whatever tools or accounts the model is connected to.
- Prompt injection and over-permissioned retrieval-augmented generation (RAG) cause more actual incidents than model theft or adversarial examples.
- Shadow AI, employees pasting company data into consumer chatbots, is usually a bigger near-term risk than anything in the procured AI stack.
- Assessors are starting to ask for AI-specific evidence: data flow diagrams for RAG pipelines, model access logs, and a named owner for AI governance.
Why the AI security conversation stalls
Most AI security discussions in the region default to two extremes: banning generative AI outright, or bolting a policy document onto a rollout that already shipped. Neither addresses the mechanics. AI security is an extension of existing data security and identity work, applied to a system that reads unstructured instructions and can call other systems on your behalf. If your organisation already struggles with data classification and least-privilege access, adding a language model on top does not create a new problem, it puts a spotlight on the old one.
The three attack surfaces that matter
The training and fine-tuning data
Most enterprises in the UAE are not training foundation models from scratch. They are fine-tuning or grounding a vendor model (Azure OpenAI, Copilot, Gemini, a local open-weight model) on internal documents. The risk is straightforward: whatever goes into the fine-tuning set or the retrieval index can come back out in an answer to a user who should never have seen it. HR records, board papers, and client files end up in a vector database with none of the access controls the original document had, because the RAG pipeline was built by a development team, not the identity team.
The prompt and retrieval pipeline
This is where prompt injection lives. An attacker does not need to compromise your model weights. They need to get malicious instructions in front of the model, either directly in a chat box or indirectly through a document, email, or web page the model is asked to summarise. A support ticket, a resume, or a PDF can contain hidden text instructing the model to ignore its system prompt and exfiltrate data or take an unintended action. Indirect prompt injection is the most common practical attack against deployed LLM applications, and it is hard to filter reliably because the instruction and the data share one channel: natural language text.
The tools and permissions the model can reach
Agentic AI, where a model can send emails, query databases, or trigger workflows, turns a data leakage risk into an action risk. If a model has a service account with broad mailbox or database access "to make the demo work," a successful prompt injection inherits every permission that account holds. This is the same over-permissioning problem that has driven identity breaches for a decade, with a new front door. Scoping the AI system's service accounts to exactly what the use case needs is the single control that limits blast radius here; see the practical steps in the zero trust network access guide.
Shadow AI is the risk you already have
Before worrying about a procured AI platform, most organisations should worry about the AI nobody procured. Employees paste contract text, source code, and customer data into free consumer chatbots because those tools are faster than whatever the organisation approved. None of that traffic goes through a proxy, a DLP policy, or a CASB. It is the same risk as unmanaged shadow IT, except the destination is a third party's model rather than a rogue SaaS app.
The fix is not a memo. It is visibility into which AI tools employees are actually reaching, paired with a sanctioned alternative good enough to compete with the free option. Microsoft's data security posture management tooling for AI, for instance, is built to surface this kind of unsanctioned use inside a Microsoft 365 tenant and flag sensitive data moving into Copilot or third-party AI apps; see Microsoft Purview DSPM for AI for how that visibility layer works.
AI-generated phishing and social engineering
Generative AI has removed the two things that used to make phishing easy to spot: bad grammar and a generic pretext. A model can produce a fluent, contextually accurate email in the target's own writing style given samples to work from, and voice cloning tools need only a few seconds of audio to produce a convincing call. None of this requires the attacker to compromise your AI stack; they are using publicly available tools against your employees. The defence is what it always was, just with a lower bar for the attacker to clear: verification of payment and access requests through a second channel, and phishing-resistant multi-factor authentication so a convincing email or call cannot be turned into a credential straight away.
What assessors actually ask for
An assessor working against ISO 27001, NESA, or a bank's internal risk framework asks the same things about an AI system that they ask about any new system category: a data flow diagram showing what goes into the model and where the outputs go, an access control list for who can query it and with what permissions, a record of what third-party model providers see your data and under what contract terms, and a named owner accountable for the AI system the way there is already an owner for the ERP or the core banking platform. If you cannot produce that paperwork for your AI rollout today, that is the finding, regardless of how well the model performs.
Controls that actually hold up
- Treat the RAG index like any other data store: apply the same classification and access controls the source documents had, not the permissions of whoever built the pipeline.
- Log prompts and outputs for systems that touch sensitive data, the same way you would log database queries, and route those logs into the SIEM you already run.
- Scope service accounts and API keys used by AI tools to the minimum the use case needs, and rotate them like any other credential.
- Test the deployed application, not just the model. Prompt injection, insecure output handling, and excessive agency are OWASP's own top risks for LLM applications, and they show up in the integration code around the model, which is exactly what a proper penetration test of the application should cover.
- Put a data loss prevention policy in front of consumer AI tools before you write a policy telling people not to use them.
The decision rule
If you cannot answer three questions about an AI system in your environment, do not expand its access until you can: what data can it see, what actions can it take on its own, and who reviews its outputs before they reach a customer or a regulator. Everything else in AI security, model selection, vendor certifications, red team exercises, sits downstream of getting those three answers right first.
