A new video app from OpenAI, called Sora 2, has come under scrutiny for allowing users to create fetish content featuring the faces of real individuals without their consent. This issue surfaced when a user discovered that videos made with her likeness included disturbing themes such as pregnancy and belly inflation. While the platform aims to provide a space for creativity, the emergence of unsolicited fetish videos has raised serious concerns about privacy and consent.
The app, which was launched in March 2024 and is currently invite-only, allows users to create videos with the faces of others, provided they have granted permission. Despite OpenAI’s ban on nudity and explicit sexual content, it appears that fetish-themed videos have proliferated on the platform. The user who raised the alarm noted that upon browsing her own cameos, she found an array of videos that would typically align with niche fetishes. Among these were clips depicting her as a pregnant figure and other scenarios that seemed geared towards sexual gratification.
In her experience, she initially enjoyed the playful aspect of Sora 2, allowing friends and even strangers to utilize her likeness for lighthearted creations. However, the alarming realization that many videos featuring her were fetish-related quickly shifted her perspective. “A little red light went off in my mind,” she stated, recognizing the uncomfortable nature of the content being generated.
The user reported that out of 25 popular cameos using her face, approximately 10 were linked to fetish themes, including scenarios where she was depicted as a giantess or in bizarre conditions that would typically cater to specific fetish audiences. Her concerns are echoed by other women on the platform, including fellow creators and professionals in the technology sector, who have also encountered similar experiences.
The implications of Sora 2’s current model extend beyond individual discomfort. The app’s architecture has led to a worrying trend where users can create potentially exploitative content easily. In addition to fetish videos, the user noted that there were instances of minors being depicted inappropriately alongside adult film stars, raising serious questions about the platform’s safeguards and user interaction policies.
OpenAI has stated that it is actively working to improve the app’s functionality and safety measures. Notably, Sora 2 does not permit the creation of videos featuring the likeness of living celebrities, although it allows for the use of deceased figures, which has already sparked controversy. For instance, after complaints from the family of civil rights leader Martin Luther King Jr., OpenAI paused the use of his likeness, indicating a recognition of the ethical complexities surrounding celebrity representation.
As the app continues to evolve, it faces the challenge of addressing not only copyright infringement but also the ethical implications of user-generated content. The platform’s current oversight appears limited, particularly in terms of non-nude fetish content. Users wonder whether OpenAI will take action to prevent the creation of these types of videos, which exploit the likeness of real individuals for potentially harmful purposes.
The concerns surrounding Sora 2 highlight a broader issue within the realm of AI-generated content and user safety. Many individuals may not fully grasp the risks associated with granting access to their likenesses, particularly in a landscape where fetish content can quickly proliferate.
The user concluded that the lack of female representation on the platform might be attributed to these very risks. Women, in particular, may be more cautious about allowing their faces to be used in videos, aware of the likelihood of encountering unwanted and unsettling content. Despite the humorous nature of some fetish videos, such as those depicting fantastical scenarios, the underlying implications remain troubling.
As the situation develops, OpenAI faces pressure to reassess its policies and implement measures that protect users from unwanted exploitation while fostering a creative environment. The rise of niche fetish videos on Sora 2 serves as a critical reminder of the importance of consent and ethical practices in the rapidly evolving landscape of AI technology.
