ChatGPT is a large language model developed by OpenAI. It is designed to understand and respond to natural language input from users, and I can provide information, answer questions, and engage in conversation on a wide range of topics .
CNC staff tested and probed with questions to the AI engine, for Open AI to explain, the recent phenomenon of hackers using the ChatGPT by cyber criminal and scammer.The results, was “eyebrow raising experience” The AI’s response was a clear recognition, that Malicious intent can bypass well designed computer language that had intentions to create benefits to society.
Furthermore, it provided alternative hacking ideas,
“..there are some instances where hackers may attempt to use AI language models like myself to aid in their attacks. One such technique is called "GPT-3 phishing," which involves using an AI language model like myself to generate convincing phishing emails that are more likely to trick users into giving up sensitive information. Hackers may also use AI language models to generate automated responses to social engineering attacks, making it easier to scam unsuspecting victims.”
Leading Security Vendor - Test The Platform & Scan The Dark Web
There have been many discussions and research on how cybercriminals are leveraging the OpenAI platform, specifically ChatGPT, to generate malicious content such as phishing emails and malware. Proof of such a threat already exists: ChatGPT successfully conducted a full infection flow, creating a convincing spear-phishing email and running a reverse shell, which even accepts commands in English.
Check Point Researchers Examine The Hackers Activities On ChatGPT
CPR researchers recently found an instance of cybercriminals using ChatGPT to “improve” the code of a basic Infostealer malware from 2019. Although the code is not complicated or difficult to create, ChatGPT improve
Hackers have found a simple way to bypass those restrictions and are using it to sell illicit services in an underground crime forum, researchers from security firm Check Point Research(CPR), reported. The technique works by using the application programming interface for one of OpenAI's GPT-3 models known as text-davinci-003, instead of ChatGPT, which is a variant of the GPT-3 models that's specifically designed for chatbot applications.
OpenAI makes the text-davinci-003 API and other model APIs available to developers so they can integrate the AI bot into their applications. It turns out the API versions don’t enforce restrictions on malicious content.
Barriers To Malicious Content Creation
As part of its content policy, OpenAI created barriers and restrictions to stop malicious content creation on its platform.
Several restrictions have been set within ChatGPT’s user interface to prevent the abuse of the models. For example, if you ask ChatGPT to write a phishing email impersonating a bank or create malware, it will not generate it.
Bypassing Limitations To Create Malicious Content
However, CPR is reporting that cyber criminals are working their way around ChatGPT’s restrictions and there is an active chatter in the underground forums disclosing how to use OpenAI API to bypass ChatGPT 's barriers and limitations. This is done mostly by creating Telegram bots that use the API. These bots are advertised in hacking forums to increase their exposure.
“The current version of OpenAI's API is used by external applications (for example, the integration of OpenAI’s GPT-3 model to Telegram channels) and has very few if any anti-abuse measures in place,” the researchers wrote. “As a result, it allows malicious content creation, such as phishing emails and malware code, without the limitations or barriers that ChatGPT has set on their user interface.”
Check Point researchers tested text-davinci-003 API how well it worked. The result: a phishing email and a script that steals PDF documents from an infected computer and sends them to an attacker through FTP.
The generation of malware and phishing emails is only one way that ChatGPT and its other GPT-variants are opening a Pandora’s box that could bombard the world with harmful content. The increasing concern is the unsafe or unethical uses are the invasion of privacy and the generation of misinformation or school assignments.
Conversely, the same ability to generate damaging, unethical, or illicit content can be used by defenders to develop ways to detect and block it, but the jury is still out whether the benign uses will be able to keep pace with criminal activity.
