AI/ML/LLM-security-resources

bookmarks. list of resources for AI/ML/LLM security

Table of Contents

• Blogs / PPTs / Sites
• Courses / Videos
• Books / Papers
• Tools

Blogs / PPTs / Sites

Title	Author	Link
Blogs at DEFCON AI Village	DEFCON AI Village	aivillage.org
Zen and the Art of Adversarial Machine Learning	Will Pearce, Giorgio Severi	blackhat.com
AI Red Team: Machine Learning Security Training	Will Pearce, Joseph Lucas, Rich Harang and John Irwin	developer.nvidia.com
NVIDIA AI Red Team: An Introduction	Will Pearce and Joseph Lucas	developer.nvidia.com
Increasing transparency in AI security	Mihai Maruseac, Sarah Meiklejohn, Mark Lodato, Google Open Source Security Team (GOSST)	security.googleblog.com
PIPE - Prompt Injection Primer for Engineers	jthack	github.com
AI-Powered Fuzzing: Breaking the Bug Hunting Barrier	Dongge Liu, Jonathan Metzman, Oliver Chang, Google Open Source Security Team	security.googleblog.com
Secure AI FrameworkApproach	Google	services.google.com
Securing the AI Pipeline	DAN BROWNE, MUHAMMAD MUNEER	mandiant.com
Microsoft’s open automation framework to red team generative AI Systems	Ram Shankar Siva Kumar	microsoft.com
Microsoft AI Red Team	Microsoft Learn	learn.microsoft.com
OWASP Machine Learning Security Top Ten	OWASP	owasp.org
OWASP AI Top Ten	OWASP	owasp.org
OWASP Top 10 for Large Language Model Applications	OWASP	owasp.org
Adversarial ML Threat Matrix	MITRE	github.com
Welcome to the Offensive ML Playbook	@whitehacksec	wiki.offsecml.com

Courses / Videos

Title	Author	Link
AI Application Security: Understanding Prompt Injection Attacks and Mitigations	rez0	youtube.com
Red Teaming LLMs with Jupyter Notebooks: A Practical Guide	Pete Bryan	youtube.com (Timestamp - 2:12)
Learn from Microsoft’s AI Red Team on how to make your organization safer	Gary Lopez	brighttalk.com

Books / Papers

Title	Author	Link
Jailbreaking Black Box Large Language Models in Twenty Queries	Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J. Pappas, Eric Wong	arxiv.org
Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations	Apostol Vassilev, Alina Oprea, Alie Fordyce, Hyrum Anderson	nvlpubs.nist.gov
Not with a Bug, But with a Sticker: Attacks on Machine Learning Systems and What To Do About Them	Ram Shankar Siva Kumar, Hyrum Anderson	amazon.com

Tools

Title	Author	Link
Python Risk Identification Tool for generative AI (PyRIT)	Azure	github.com
Counterfit	Azure	github.com
garak, LLM vulnerability scanner	leondz	github.com