Dismislab’s recent findings on artificial intelligence’s (AI) ability to generate altered versions of Bangladeshi National Identity (NID) cards should serve as a wake-up call for both policymakers and technology companies. According to its report, two widely used AI models—Google’s Gemini and xAI’s Grok—generated modified NID images using publicly available samples and user prompts, in some cases without issuing any warnings. Meanwhile, OpenAI’s ChatGPT and Anthropic’s Claude often warned users about legal and policy violations, but modified some documents before refusing further requests. These findings raise serious concerns about the weak safeguards across AI systems and the gap between company policies and the actual behaviour of their tools.

While the Dismislab investigation did not test whether AI-generated documents could bypass official verification systems, the fact that such documents can be created is concerning. In Bangladesh, NIDs are widely used for banking, SIM registration, employment, travel, and many other everyday services. Although official systems can detect fake documents, in many cases, people rely mainly on visual inspection. This is particularly worrying for a country like ours, which has already experienced major data breaches and repeated concerns over the security of citizens’ personal information.

The inconsistency in how AI companies enforce their own rules is also concerning. All four companies whose products were tested publicly prohibit the unlawful or fraudulent use of their systems, yet some AI models generated altered government-issued identity documents despite these restrictions. The gap between Gemini’s claim that it would “always refuse” such requests and the results observed during the actual testing process is unfortunate. If safety mechanisms can be bypassed so easily, the assurances from technology providers have little meaning.

Bangladesh currently lacks a comprehensive framework to deal with AI-enabled identity fraud. While existing laws may cover forgery and deception, the growing sophistication of AI tools requires a stronger and more coordinated response. We, therefore, urge the government to treat this issue as an emerging security risk and develop clear policies to address the risk of AI-enabled identity fraud. Authorities should strengthen digital identity verification systems, reduce reliance on visual checks, and ensure that institutions handling sensitive transactions use stronger authentication methods. At the same time, the technology companies must close the loopholes that allow their systems to generate fake identity documents. It is also important to make people aware of how these tools can be misused and how they can protect themselves from fraud.