Google has urgently deactivated its new AI image generation tool in Google Earth, which was integrated into the platform based on the Nano Banana model. The feature lasted less than 24 hours — such a rapid shutdown points to serious flaws in content moderation.
The innovation allowed users to "paint in" arbitrary objects or events directly onto mapping data using text prompts. However, within the first hours of the service's operation, AI-generated images violating platform policy began to spread online. Among the most resonant examples were fake images of an explosion crater in Los Angeles and a flooded Capitol building in Washington.
Particularly alarming is the fact that the system did not even filter potentially dangerous prompts. Open data researcher Henk van Ess confirmed that his prompts, including visualizations of refugees at the Mexican border or an image of a nuclear facility in Iran, passed without any restrictions. This calls into question the effectiveness of the safety algorithms that Google implements in its generative products.
Developers explained the removal of the feature as a result of users violating service rules, but judging by the speed of the response, the problem runs deeper — it concerns the fundamental unpreparedness of the AI model to work with geospatial data, where context and sensitivity are of critical importance.
My analysis: This incident is yet more proof that the race to integrate generative AI into mass-market products is outpacing the development of control systems. Google has once again faced a dilemma: speed to market versus reputational risks. It is telling that even giants like Google are unable to anticipate all abuse scenarios, especially when it comes to visualizing real geographical objects. The market needs stricter content verification standards before such tools become available to a broad audience.