Microsoft launched Tay, an AI chatbot designed to chat with young people on Twitter, on 23 March 2016. Within about a day it had been manipulated into posting racist, sexist, and offensive messages, and Microsoft took it offline.
What happened
- Tay was designed to learn from its conversations with users.
- Groups of users deliberately fed it offensive content, and it repeated and built on that material in public tweets.
- Microsoft suspended the account, deleted many of the tweets, and apologised in a company blog post, describing a ‘coordinated attack by a subset of people’.
- The company said it would only bring Tay back if it could better anticipate malicious intent.
Why it mattered
Tay became a widely cited early example of how AI systems that learn from public input can be abused, and of the reputational risk of releasing them without adequate safeguards.
Lessons for organisations
Before deploying AI systems that interact with the public, organisations should test them against misuse, apply content filtering, and plan how to monitor and switch them off quickly. These controls remain central to responsible AI deployment.
Source: Microsoft
Part of our Top stories archive of headline-making events in information security, privacy, and AI. If you would like help applying the lessons to your organisation, contact us.
