A machined metal part in close-up
← All workshops

Course title

Adversarial Threats and Defence in AI Systems

Jailbreaking, prompt injection and layered defence

Overview

Language models and AI agents introduce attack vectors that conventional security controls were not designed to address. This workshop examines how AI systems are compromised, how to test them as an adversary would, and how to defend them through layered controls.

A model can be persuaded to disregard its instructions. An agent processing an email can be redirected by a concealed instruction within it. A knowledge base can be seeded with false material. These threats are documented in practice, and they fall outside the scope of firewalls and endpoint protection.

The session maps the AI threat landscape against recognised references, including the OWASP Top 10 for LLM Applications and MITRE ATLAS, followed by practical red teaming of a sample assistant. It concludes with defensive architecture, monitoring, incident response, and Singapore's guidance on securing AI systems.

Learning outcomes

By the end of the workshop, you will be able to

  1. 01

    Analyse how jailbreaking, prompt injection, and data poisoning attacks operate against AI systems.

  2. 02

    Map AI-specific risks using the OWASP Top 10 for LLM Applications and MITRE ATLAS.

  3. 03

    Conduct a structured red team exercise against an AI assistant or agent and document the findings.

  4. 04

    Design layered defences, monitoring, and an incident response plan for AI deployments.

Programme

What you will learn

Part 1 · 3 modules

The AI threat landscape

1.1How AI systems are attacked
  1. (a)Where AI changes the attack surface
  2. (b)The OWASP Top 10 for LLM Applications
  3. (c)Adversary tactics catalogued in MITRE ATLAS
1.2Jailbreaking and prompt injection
  1. (a)Jailbreak techniques and why they work
  2. (b)Direct and indirect prompt injection
  3. (c)Hidden instructions in documents, web pages and email
1.3Attacks on data and models
  1. (a)Data and knowledge base poisoning
  2. (b)Sensitive data leakage through outputs
  3. (c)Model theft and supply chain risk

Part 2 · 2 modules

Testing AI systems

2.1Red teaming
  1. (a)Planning an AI red team exercise
  2. (b)Hands-on attacks on a sample assistant
  3. (c)Recording and rating findings
2.2Securing agents
  1. (a)Excessive agency and over-permissioned tools
  2. (b)Tool impersonation and MCP risks
  3. (c)Human approval for high-impact actions

Part 3 · 3 modules

Defence and response

3.1Layered defences
  1. (a)Input and output filtering and guardrails
  2. (b)Least privilege, isolation and sandboxing
  3. (c)Separating instructions from untrusted content
3.2Monitoring and response
  1. (a)Logging prompts, outputs and tool calls
  2. (b)Detecting abuse and anomalies
  3. (c)An incident response plan for AI systems
3.3Guidance and standards
  1. (a)Singapore guidance on securing AI systems
  2. (b)Security across the AI life cycle
  3. (c)Assurance for clients and regulators

Who should attend

  • Cybersecurity and IT security teams
  • Developers building AI assistants and agents
  • Risk and compliance officers
  • IT managers responsible for AI adoption

In-house delivery

Tailored for your organisation.

Delivered at your offices or conducted virtually, each run is customised around your active course portfolio and internal operating procedures.

Book a consultation All workshops