I’ve tried giving screenshots of phishing emails to a local Qwen instance and so far it always correctly detected it as scam, even points out the exact elements that it based its judgement on. Sending screenshots to it ad-hoc isn’t too scalable for family and friends. I’d like to be able to either forward emails for screening, or perhaps have it screen everything from a mailbox.
Has anyone done anything like this? Is there anything self-hostable that does this?
I’d be pretty concerned about prompt injection risks with feeding a LLM unsanitized data. You definitely need a good harness around it…
Good point. It’ll have to have no access to the internet or anything local outside of its container. Just text in, text out.
You could (and probably should) use a system-one style inference system for spam classification. Much cheaper and the structured output means it’s impossible to go rogue and curl some malware or whatever. It can absolutely misclassify but its output is programmatically structured and just ranks a pre-selected set of output tokens.
In your case that’s
Spam
Not_spam
Yeah if it’s just a basic input with a function call for spam or not spam the risk is low. What’s the worst case outcome, it tricks it into saying no it isn’t a scam and you have to delete it manually? Hahaha