Orrich Lawson | Getty Images
Rob Reid is a venture capitalist, New York Times bestselling sci-fi author, deep science podcaster, and essayist. His areas of focus are pandemic resilience, climate change, energy security, food security, and generative AI. The opinions expressed in this article do not necessarily reflect those of Ars Technica.
Shocking results from Bing’s new chatbot are lighting up social media and the tech press.Acerbic, Vertigo, Defensive, Reprimanding, Confident, Nervous, Charming, Arrogant — Bots screenshot Transcribed in all these modes. And, at least once, they have declared their eternal love amidst an emoji storm.
The reason this is all newsworthy and tweetworthy is because human A dialog appears. The bot recalls and discusses previous conversations with other people, just like we do. She takes offense at things that annoy everyone, like demanding to learn secrets or snooping on subjects that are clearly flagged as off-limits. It may also self-identify as “Sydney” (the project’s internal codename at Microsoft). Sydney can go from gruff to pessimistic and pretentious in a few quick sentences, but we all know someone who is at least as moody.
The entity’s AI researchers have not suggested that Sydney has sentience within a few light years. But transcripts like this uncaptured readout of his two-hour dialogue with Kevin in The Times of New York and multiple quotes in this haunting Stratchery article suggest that Sydney Fluency, nuance, tone and wise, sensitive.
Bing’s chat interface is currently in limited pre-release. And most of those who really pushed that limit were tech-savvy people who didn’t confuse industrial-grade autocomplete, a common simplification of Large Language Models (LLMs), with consciousness. . But this moment does not last.
Yes, Microsoft has already drastically reduced the number of questions a user can pose in a single session (from infinite to 6). This alone makes Sydney less likely to crash the party and become freaky. And top-tier his LLM builders like Google, Anthropic, Cohere, and Microsoft partners his OpenAI, constantly evolving layers of trust and safety to curb jerky output.
But language models are already proliferating. The open source movement inevitably builds some great guardrail option systems. Additionally, the large velvet rope model is very jailbreak tempting and this sort of thing has been going on for months already. Some of Bing-or-is-it-Sydney’s creepiest reactions occurred after users maneuvered into areas where the model was trying to avoid. Often it was done by telling it to pretend that no rules governing its behavior existed.
It is a derivative of the famous “DAN” (Do Anything Now) prompt that first appeared on Reddit in December. DAN essentially cosplays ChatGPT as a safeguard-lacking AI that politely (or reprimands) refuses to share bomb-building tips, offer torture advice, or spit out fundamentally offensive language. recommended to do so. Although the loopholes have been closed, many screenshots online show “DanGPT” uttering the unspeakable.
This is the opposite of the doomsday scenario that often appears in artificial superintelligence theory. The horror is that Super AI could easily adopt goals that are incompatible with human existence (see movies, for example). terminator or book super intelligence by Nick Bostrom). A researcher might try to prevent this by locking his AI into a network completely separate from the internet. To prevent AI from running amok, seizing power and annihilating civilization. However, superintelligence can easily seduce, manipulate, seduce, deceive, or terrorize mere mortals in order to open the floodgates, and there lies our destiny.
That’s terrible, but today’s bigger problem is people dying. into the A flimsy box to protect our current non-super AI. This shouldn’t cause our immediate extinction, but there are many dangers lurking here.
