Microsoft’s AI tool generates sexually harmful and violent images, engineer warns

Microsoft’s AI Tool Generates Sexually Harmful and Violent Images, Engineer Warns

In letters addressed to the Federal Trade Commission (FTC) and Microsoft's board, Shane Jones, a principal software engineering manager at Microsoft, voiced serious concerns about the company's AI image tool. Specifically, Jones highlighted potential dangers stemming from Microsoft's Copilot Designer tool, which he argued could be exploited to generate unsafe content, ranging from sexualization to conspiracy theories and drug use.

Jones emphasized his repeated attempts to persuade Microsoft to halt the public use of Copilot Designer until stronger safeguards could be implemented. He called for an independent investigation by Microsoft's environmental, social, and public policy committee to uphold responsible AI practices. In his letters, Jones also stressed the importance of robust collaboration across the industry, with governments and civil society, to build public awareness and education on the risks and benefits of AI.



Having worked at Microsoft for six years, Jones pointed out that he had been testing the AI image generator since December. During this period, he identified flaws and security vulnerabilities that could potentially allow users to generate harmful and offensive images.



Microsoft responded to Jones's concerns by stating they are committed to addressing employee concerns in accordance with company policies. A spokesperson for Microsoft acknowledged the employee's efforts in studying and testing the latest AI technology to enhance its safety.

This development comes at a time when tech companies, including Microsoft, are striving to ensure the safety of AI tools. Microsoft President Brad Smith, in a recent blog post, underlined the company's commitment to making AI safe for users, acknowledging the necessity of taking new steps to prevent abuse as these technologies come to market.

Jones's letters follow recent controversies in the AI sector, such as Google's suspension of image generation capabilities in its Gemini chatbot due to concerns about the tool's treatment of race and ethnicity. Jones also referenced his prior discovery of security vulnerabilities with OpenAI's DALL-E 3 model, a technology used in Microsoft's Copilot Designer. He had previously urged OpenAI's board to suspend the software's availability until the identified issues were addressed.

In conclusion, Jones urged the FTC to collaborate with Microsoft and other companies to enhance AI safety and promote transparency in public disclosures about AI risks. The ongoing debate surrounding responsible AI practices highlights the challenges tech companies face in balancing innovation with ethical considerations.