San Francisco-based AI chatbot maker Replika — operates a freemium “virtual friendship” service based on customizable digital avatars, whose “personalized” responses are powered by artificial intelligence ( (designed to make human users feel better, as per its pitch) — has been ordered by the Italian privacy watchdog to stop processing local users’ data.
of I guarantee He said he was concerned that Replika’s chatbot technology poses a risk to minors. Furthermore, the company lacks a proper legal basis for processing children’s data under EU data protection regulations.
Additionally, regulators are concerned about the risks AI chatbots pose to emotionally vulnerable people. It also accuses Luka Inc, the developer of the Replika app, for failing to meet local legal requirements to clearly communicate how it uses people’s data.
The order to stop processing the data of Italians will take effect immediately.
In a press release announcing the intervention, Watchdog said: [the] Personal data of Italian users for the time being.Temporary restrictions on data processing are the Italian I guarantee About the US-based company that develops and operates the app. Restrictions take effect immediately. “
“Recent media coverage and SA testing [supervisory authority] A study conducted on “Replica” showed that the app poses a virtual risk to children.
Replika was an early API partner for GPT-3, OpenAI’s text-generating large-scale language model technology. However, the service does not run on a carbon copy of GPT-3 (nor is it the same technology as OpenAI’s buzzy ChatGPT). Rather, the startup claims he “fine-tuned” GPT-3 using a network machine learning model trained on conversations, making it conversational (and “empathic”) for specific use cases. ) Polished up AI companion generation technology.
However, concerns have been raised about the risks technology can pose to children. From concerns about children being exposed to inappropriate content to more general concerns, they can become addicted to interactions or simply be encouraged to spend a lot of money. Customize your avatar and access other paid content. However, the Italian watchdog appears to be the first regulator to take formal action regarding child safety.
of guarantor The Order notes that several user reviews of the app have reported sexually inappropriate content being served. And although the app lists her as being 17 or older, Apple’s iOS and Google’s Android app stores only allow her to be used by anyone under the age of 13 under the developer’s terms of service. Also, users under the age of 18 must have approval from a parent or guardian, but Watchdog says this app does not attempt to verify the age of users and does not block minors who provide information about their age. points out. Children.
“In fact, there is no age verification mechanism, no gating mechanism for children, no blocking of apps if a user declares that they are underage. It just asks for gender,” he said. “And the ‘reply’ provided by chatbots is often clearly at odds with the enhanced protections that children and vulnerable individuals are entitled to. Reviews contain user comments that flag sexually inappropriate content. “
“‘Replica’ violates EU data protection regulations. It does not comply with transparency requirements and unlawfully processes personal data. This is because it cannot invoke the performance of a contract as a legal basis, even if it is a contract valid under Italian law ”, I guarantee It further said it had ordered the US-based developer to stop processing data about Italian users, and gave it 20 days to communicate the steps taken to comply with the order.
It further points out that failure to comply with the order risks fines of up to €20 million, or 4% of total global annual turnover.
Replicas are guarantor order.
The EU General Data Protection Regulation (GDPR) focuses on protecting children’s information and privacy. For example, it suggests that services that are likely to have minors among their users should consider incorporating child-friendly design and be proactive about taking risks. Evaluation to ensure identification of potential safety and other rights issues.
Watchdogs in the region are willing to pay attention to breaches in this area.
Last fall, for example, Instagram was fined nearly $440 million for violating children’s privacy. European consumer protection authorities have also expressed concerns about child safety on TikTok, while an investigation into TikTok’s handling of children’s data is ongoing in Ireland.
Italy’s data protection watchdog has shown to be particularly sensitive to child safety concerns in recent years.It was reported that it took part in a dangerous challenge on its platform. This has resulted in the removal of over 500,000 accounts.
But despite the enactment of GDPR (and consumer protection laws) on child safety issues, campaign groups continue to push for tougher laws, claiming children are still not adequately protected. Restrictions can be even stricter.
In the UK, an age-appropriate design code will come into force in fall 2021, with a focus on protecting minors from safety and privacy risks. Meanwhile, the French data protection watchdog has also published a series of recommendations to ensure that children’s digital rights are protected.
The UK is also working to pass an Online Safety Bill focused on child safety, addressing public concerns about what children are exposed to online.
In recent months, EU lawmakers have also agreed to two major updates to Block’s digital rulebook: a complete ban on processing minors’ data for ad targeting. .
