Missing the Mark

AI‐generated images have become so good in recent years that individuals often cannot distinguish them from ‘real’ images. This development, combined with the rapid spread of AI‐generated content online, creates a series of societal risks. Watermarking, a technique that involves embedding information within the content to indicate their AI‐generated nature, has emerged as a primary mechanism to address these risks. Indeed, watermarking and AI labelling measures are now becoming a legal requirement in many jurisdictions, including under the 2024 EU AI Act. Yet, despite the widespread use of AI image generation systems, the practical implications and current status of implementation of these measures remain largely unexamined. The present paper therefore provides both an empirical and a legal analysis of these measures. In our legal analysis, we identify four categories of generative AI deployment scenarios and outline how the legal obligations could apply in each. In our empirical analysis, we find that only a minority number of AI image generators currently implement adequate watermarking (38%) and deep fake labelling (18%) practices. In response, we suggest a range of avenues of how the implementation of these legally mandated techniques can be improved, and publicly share our tooling for detecting watermarks

Digital watermarking · Embedding · Empirical research · Generative grammar · Labelling · Software deployment · Adversarial Robustness in Machine Learning · Ethics and Social Impacts of AI · Explainable Artificial Intelligence (XAI

• AI models collapse when trained on recursively generated data

  Open Access•Ilia Shumailov, Zakhar Shumaylov et al.•Nature•2024

  Add to list

• Seeing without knowing

  Open Access•Mike Ananny, K Crawford•New Media & Society•2016

  Add to list