AI and LLM security means treating the model itself, not just the infrastructure around it, as something that can be attacked: through manipulated prompts, poisoned training or fine tuning data, extracted system prompts, or outputs that leak information the model was never meant to repeat. For a UAE bank running fraud scoring, credit decisioning, or a customer-facing chatbot on a large language model, most of that attack surface sits outside what a firewall, SIEM, or endpoint agent was built to see.
- Prompt injection and system prompt extraction are practical, low-skill attacks against most customer-facing LLM deployments in production today.
- Standard network and endpoint tooling does not inspect token streams, so model-specific abuse goes unlogged by default.
- The highest-value fix is usually organisational: deciding who owns AI model risk before evaluating a single tool.
- Data flowing into third-party LLM APIs needs the same classification and loss-prevention treatment as data leaving the organisation by any other route.
The attack surface is the model, not the server around it
A large language model deployed for fraud detection, underwriting, or customer support has three places an attacker can work: the training or fine tuning data, the prompt going in, and the completion coming out. Poison the training data and a fraud model quietly stops flagging a pattern of transactions. Craft the right prompt and a support bot can be talked into repeating its own system instructions, or into describing account information it was told never to disclose. Neither of these looks like a conventional exploit. There is no payload to sandbox, no exploit chain to trace in an EDR timeline. The "vulnerability" is the model behaving exactly as designed, on an input nobody anticipated.
This matters more in banking than almost anywhere else because these models sit directly on sensitive data pipelines and make decisions that used to require a human sign-off. A chatbot that hallucinates a wrong answer is embarrassing. A chatbot that leaks a customer's balance to an unauthenticated prompt, or a scoring model that silently misclassifies fraud after a handful of adversarial transactions, is a different category of failure entirely.
What a compromised model actually looks like
Traditional breaches leave a fairly recognisable trail: unusual outbound traffic, a dumped credential, a ransom note. AI failures are quieter. A fraud model degraded by manipulated inputs does not throw an alert, it just stops catching what it used to catch, and nobody notices until the loss numbers move. A support model that leaks personally identifiable information does so one conversation at a time, which rarely trips a DLP rule written for file transfers and email attachments. By the time the pattern is visible in the data, weeks of decisions may already be affected, and reconstructing which ones is a genuine forensic problem because most teams are not logging prompts and completions with the same discipline they log authentication events.
Regional data localisation and breach notification rules do not pause for that ambiguity. If customer data left the organisation through a model's output rather than through a database export, it is still a disclosure, and the clock on notifying regulators starts the same way.
Why legacy security controls do not catch this
Firewalls inspect network traffic. SIEM correlation rules are tuned for host and network events. Endpoint agents watch process behaviour. None of them were built to parse a prompt, notice that a completion contains an account number, or flag a sudden spike in requests shaped like extraction attempts. Feeding an AI powered SIEM raw application logs does not fix this: without tuning for model-specific signals, the tool just adds another noisy dashboard.
The most common gap found in reviews of live LLM deployments is embarrassingly simple: a static API token sitting in frontend JavaScript, giving anyone who opens the browser's developer tools a working credential to the model endpoint, while the admin console two clicks away is behind proper multi-factor authentication. Model endpoints get treated as an application feature rather than a privileged interface, and privileged-access thinking never gets applied to them.
Controls that actually reduce the risk
A handful of controls do meaningful work here, in roughly the order they pay off:
- Input sanitisation and a prompt firewall in front of any customer-facing model, to strip or flag known injection patterns before they reach the model.
- Output filtering for account numbers, national ID formats, and other regulated data classes, so a leak has to get past a second control even if the model is fooled.
- Short-lived, scoped credentials for model endpoints instead of static tokens, with the same access review cadence as any other privileged system.
- Logging and retention of prompts and completions, not just API status codes, so an incident can actually be reconstructed after the fact.
- Data security posture management that maps where sensitive data actually flows into AI systems, rather than assuming it stays inside the model's own environment. Microsoft's DSPM for AI and the underlying Purview classification layer are built specifically for this, discovering and labelling sensitive data before it reaches a prompt rather than after it leaves in a completion.
- Adversarial testing of the model itself before launch and on a recurring schedule after, using the same red team mindset applied to network penetration testing, aimed at prompt injection, extraction, and inversion rather than open ports.
Model risk has no clear owner, and that is the actual problem
Ask who owns AI model risk at most financial institutions and the honest answer is nobody, or everybody in theory and nobody in practice. IT owns the infrastructure the model runs on. Compliance owns the regulatory obligations around the data it touches. The data science or innovation team owns the model's accuracy and performance. Security is usually invited to review the deployment after it is already live, if at all. That gap, not a missing product, is why static tokens end up in frontend code and why nobody is watching for prompt injection: there is no single team whose job it is to notice.
Fixing this is a governance decision before it is a technology purchase. It means putting AI systems on the same asset register as any other production system, assigning an owner who is accountable for both its accuracy and its security, and requiring a security review before a model with access to customer data goes live, not after.
What assessors and auditors are starting to ask for
Formal, LLM-specific regulation is still catching up across the region, but the underlying expectations are not new. Assessors reviewing automated decisioning systems, whether the label is machine learning, statistical scoring, or generative AI, expect evidence that a decision can be explained and reconstructed, that the data feeding the model is classified and access-controlled, and that changes to the model or its prompts are logged like any other change to a production system. A model that cannot produce a defensible answer to "why was this transaction flagged" or "why was this loan declined" is an audit finding waiting to happen, independent of whether an AI-specific clause exists in the relevant framework yet.
Where to start
Do not start by evaluating vendors. Start by answering four questions: which AI systems are actually live in production, who owns the risk on each one, what data goes into them, and what comes out. Map every LLM integration including the ones procured by a business unit outside IT's visibility, because those are usually the ones with a static key in the frontend. Then run a red team exercise against the highest-risk system, aimed specifically at prompt injection and data extraction, before spending a single dirham on a monitoring platform. The organisations that get hurt are not the ones without budget. They are the ones that bought tooling before they knew what they were protecting.
