Malicious LLM Applications

It’s a lovely sunny morning here in south Wales, a perfect time take a look at a pre-print paper covering some of the darker uses of LLMs:

“Malla: Demystifying Real-world Large Language Model Integrated Malicious Services“

So what’s a Malla? I don’t think it’s caught on yet, but ‘Malicious LLM Applications’.

They’ve got names like ‘Evil-GPT’, ‘Fraud-GPT’ and ‘BadGPT’ (nice marketing!) and are available on underground marketplaces on the dark web. They are focussed on tasks like writing phishing emails, writing code for things like computer viruses and spreading misinformation. Typically they either use jailbreaking techniques or fine tune models to get them to do tasks that mainstream LLMs like ChatGPT will refuse to do.

Why am I sharing this? It’s not to scare people, but also we mustn’t get complacent. We need to be aware that just because the tools we use everyday have lots of safety features, these aren’t the same tools that bad actors are using, and we need to be alert to this.