Jailbreaking and Adversarial Attacks

Planned topics include jailbreaks, adversarial prompting, prompt injection and attacks on LLM agents, red-teaming, and the robustness of safety fine-tuning and its defenses.

Add lecture notes, readings, and transcripts here as the course progresses.

Further reading

• Chapter 3.3: Robustness — textbook chapter on adversarial examples, Trojans, and robustness challenges.
• AI Safety (CS 2881), Fall 2026 — includes material on model policies and safety.
• AI Safety for Mathematicians — mathematical and research context for technical safety work.