The Adversarial Frontier
Poisoning, prompt injection, manipulated inputs
This module deals with security. Earlier modules dealt with models that are misused, or that fail on their own; the concern here is models that are deliberately attacked by someone who wants a particular result. It covers three attack classes: adversarial evasion, training-data poisoning, and prompt injection against AI-integrated applications. Each is connected to the financial threat of manipulated inputs submitted to fraud and underwriting models. The module returns to Module 5 to show how a manipulated application can defeat an automated credit decision, and it sets out a realistic defensive posture: what can be hardened, what cannot, and where a person must remain in the decision path.
- Explain evasion, poisoning and prompt-injection attacks in plain terms, and identify the point in a financial workflow at which each of them takes effect.
- Recognise manipulated or adversarial inputs aimed at fraud, credit and monitoring models.
- Describe a realistic defensive posture, state what defence cannot achieve, and identify the decisions on which a person must remain in the decision path.
- Adversarial-threat literacy
- Input-integrity checking
- Recognising indirect prompt injection in AI-assisted tools
- Setting realistic expectations of model robustness
Lessons in this module
How it lands across the four desks
The models you rely on are themselves targets: evasion attacks tuned to pass below a detection threshold, poisoning that corrupts what the model learns from its own case outcomes, and manipulated inputs built to score clean. You learn to recognise the patterns each attack leaves in ordinary monitoring data, and what your defences can and cannot do about them.
This module returns to the question Module 5 left open: how a manipulated application, built from adversarially tuned inputs and gamed features, defeats an automated credit decision, and why checking the integrity of the inputs submitted with an application is now part of underwriting.
You govern the controls and set the institution's expectations of model robustness: what a vendor can honestly promise, what the phrase 'adversarially robust' does and does not mean, and the circumstances in which keeping a person in the decision path is a security control rather than a matter of preference.
The AI assistants you adopt form part of the attack surface as well, because indirect prompt injection can be carried in the documents and web content you ask a tool to process. You learn where that risk sits before you delegate a task to a tool that reads material from outside the institution.
Key literature · 7 sources
Every module rests on a verified scholarly and institutional evidence base. The full core and further reading lists open with the module.
- Szegedy, C. et al. (2014) 'Intriguing Properties of Neural Networks.' ICLR 2014: the discovery of adversarial examples.
- Goodfellow, I. J., Shlens, J. & Szegedy, C. (2015) 'Explaining and Harnessing Adversarial Examples.' ICLR 2015: the mechanism and the reliability/transferability of evasion.
- Biggio, B. & Roli, F. (2018) 'Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning.' Pattern Recognition 84: the field's consolidated, sober consensus.
- NIST (2025) 'Adversarial Machine Learning: A Taxonomy and Terminology' (AI 100-2e2025): the shared vocabulary for evasion, poisoning and their variants.
- Greshake, K. et al. (2023) 'Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection.' AISec '23: the naming and demonstration of indirect injection.
- OWASP GenAI Security Project (2025) 'OWASP Top 10 for LLM Applications': prompt injection and related integration risks as leading concerns.