Hackers are selling a service that bypasses ChatGPT restrictions on malware

Hackers have devised ways to circumvent ChatGPT’s restrictions and use them to sell services that allow people to create malware and phishing emails, researchers said Wednesday.

ChatGPT is a chatbot that uses artificial intelligence to answer questions and perform tasks in a way that mimics human output. You can use it to create documentation, write basic computer code, or do other things. The service actively blocks requests that generate potentially illegal content. If you ask the service to create code to steal data from a hacked device or to create a phishing email, the service will deny it, calling such content “illegal, unethical, and harmful.” Answer yes.

open pandora’s box

Hackers have found easy ways to circumvent these restrictions and use them to sell illegal services on underground crime forums, researchers at security firm Check Point Research report. This technique works by using the ChatGPT application programming interface instead of the web-based interface. ChatGPT provides APIs for developers to integrate AI bots into their applications. It turns out that the API version does not enforce any restrictions on malicious content.

“The current version of OpenAI’s API is used by external applications (for example, the integration of OpenAI’s GPT-3 model into the Telegram channel) with little, if any, anti-exploitation measures in place,” the study said. person writes. “As a result, malicious content such as phishing emails and malware code can be created without the restrictions and barriers that ChatGPT puts on its user interface.”

A forum user is selling a service that combines the API with the Telegram messaging app. First 20 queries are free. After that, users will be charged $5.50 for every 100 queries.

Check Point researchers have tested the bypass and confirmed its effectiveness. The result is a phishing email and script that steals a PDF document from an infected computer and sends it to the attacker via FTP.

Meanwhile, other forum participants post code that generates malicious content for free. “This is just a small bash script to get around ChatGPT’s limitations so you can use it for whatever you want, including malware development ;),” wrote one user.

Last month, Check Point researchers documented how ChatGPT was used to create malware and phishing messages.

“Between December and January, it was still easy to use the ChatGPT web user interface to generate malware and phishing emails (basic repetition was sufficient for most). As Check Point researcher Sergey Sikevich said in an email, ChatGPT’s anti-abuse mechanisms appear to have improved significantly, leading cybercriminals to switch to a much less restrictive API.”

A representative for San Francisco-based OpenAI, which develops ChatGPT, did not immediately respond to an email asking if the company was aware of the findings or if it had plans to change its API interface. I did. This post will be updated with any replies.

Generating malware and phishing emails is the only way ChatGPT can open Pandora’s box and attack the world with harmful content. Other examples of unsafe or unethical use include invasion of privacy, generation of misinformation or school assignments. Of course, the same capabilities that generate harmful, unethical, or illegal content can be used by defenders to develop methods to detect and block it, but benign uses keep pace with malicious ones. It is unknown whether they can be matched.