Share
Download the
Global Threat
Intelligence Report
This article explains the content of the paper titled ChatSpamDetector: Leveraging Large Language Models for Effective Phishing Email Detection [1] published by NTT Security Holdings in February 2024.
Introduction
Phishing attacks have become more sophisticated and challenging to detect over time. Traditionally, spam filters and email security protocols have played a crucial role in defending against phishing emails. However, many email services fall short of explaining to users why certain emails are flagged as suspicious, ultimately leaving the decision to open an email up to the user. This lack of transparency can lead to missed important emails and false positives of phishing emails, putting users at a disadvantage. To mitigate this, it is essential to provide not only the detection results, but also the reasoning behind those decisions.
Therefore, we have developed a system called ChatSpamDetector, which utilizes large language models(LLMs) such as ChatGPT to detect phishing emails. ChatSpamDetector not only detects phishing emails with high accuracy, but also provides concrete reasons for phishing decisions.
Key Features
- High Accuracy: Achieves an outstanding detection accuracy of 99.70%.
- Detailed Explanations: Supports user decision-making by explaining the rationale behind phishing detections.
- Advanced Analysis: By analyzing both the headers and body of emails, it uncovers a wide range of phishing strategies, including brand impersonation and fraudulent information.
Related Article
Detecting Phishing Sites Using ChatGPT
ChatSpamDetector
The ChatSpamDetector works by analyzing emails in EML format, converting them into prompts for LLM analysis. The primary target is phishing emails that lead to phishing sites upon clicking links contained within the messages. The process involves decoding the emails, simplifying the content, and creating prompts that LLMs can understand and analyze effectively.
Decoding Emails
Emails in EML format contain headers and bodies. The header contains the sender and receiver's addresses, the subject line, the date and time of transmission, and information about the communication route. The body contains the message of the e-mail and is written in plain text or HTML format. Non-ASCII characters or special characters are encoded in different character sets and encoding formats. For instance, a subject line "こんにちは" encoded in UTF-8/Base64 appears as Subject: =?utf-8?b?44GT44KT44Gr44Gh44Gv?=, posing a challenge for direct LLM analysis. For short sentences, LLM's advanced parsing capabilities can figure out the original sentence, but for longer sentences, the possibility of failure increases. To address this, encoded strings in the header and body are decoded into readable text.
Simplifying Emails
Given LLMs' fixed context window limitations, ChatSpamDetector measures the token count of decoded emails and simplifies them if they exceed the limit. For multipart emails with multiple MIME types, text/html messages are prioritized, and unnecessary content is removed. Simplification efforts focus on retaining crucial parts of the message while eliminating non-essential elements, like style tags in HTML messages. Text/plain messages often contain crucial details like the destination and signature at the beginning and end. To simplify lengthy messages, lines are progressively removed from the middle until the token count meets the limit.
Creating Prompt
Using prompt engineering techniques, simplified emails are converted into prompts for LLM input. The setup includes specifying the task of determining whether the email is phishing or legitimate and breaking it down into subtasks, such as identifying brand impersonation or analyzing the header for sender-recipient mismatches. The LLM's output format is predetermined to facilitate processing, using Function Calling to produce results in a specified JSON format, indicating whether an email is phishing and providing a phishing score, impersonated brand, rationales, and a brief reason for the decision.
Prompt Template
| I want you to act as a spam detector to determine whether a given email is a phishing email or a legitimate email. Your analysis should be thorough and evidence-based. Phishing emails often impersonate legitimate brands and use social engineering techniques to deceive users. These techniques include, but are not limited to: fake rewards, fake warnings about account problems, and creating a sense of urgency or interest. Spoofing the sender address and embedding deceptive HTML links are also common tactics. Analyze the email by following these steps: 1. Identify any impersonation of well-known brands. 2. Examine the email header for spoofing signs, such as discrepancies in the sender name or email address. Evaluate the subject line for typical phishing characteristics (e.g., urgency, promise of reward). Note that the To address has been replaced with a dummy address. 3. Analyze the email body for social engineering tactics designed to induce clicks on hyperlinks. Inspect URLs to determine if they are misleading or lead to suspicious websites. 4. Provide a comprehensive evaluation of the email, highlighting specific elements that support your conclusion. Include a detailed explanation of any phishing or legitimacy indicators found in the email. 5. Summarize your findings and provide your final verdict on the legitimacy of the email, supported by the evidence you gathered. Email: ``` {ここに簡略化したメールを挿入する} ``` |
Table: Properties of Function"print_phishing_result”
| Property | Type | Description |
| is_phishing | boolean | A boolean value indicating whether the email is phishing (true) or legitimate (false). |
| phishing_score | Type | Phishing risk confidence score as an integer on a scale from 0 to 10. |
| brand_impersonated | string | Brand name associated with the email, if applicable. |
| rationales | string | Detailed rationales for the determination, up to 500 words. |
| brief_reason | string | Brief reason for the determination. |
Evaluation
To evaluate the performance of ChatSpamDetector, we conducted an experiment using a dataset consisting of both phishing and legitimate emails. The phishing emails were sourced from a publicly available GitHub repository, https://github.com/rf-peixoto/phishing_pot, which collects phishing emails via honeypots. From August 2022 to October 2023, we selected 1,010 emails that contained links to phishing sites from this repository. These emails were written in 19 different languages, illustrating the global challenge of phishing. On the other hand, our legitimate email samples were randomly selected from the CSDMC SPAM corpus, ensuring a comparable dataset size of 1,000 emails in 12 languages.
We used four LLMs in our experiment: GPT-4, GPT-3.5, Llama 2 70B and Gemini Pro. The results were eye-opening. GPT-4 led with the highest detection accuracy of 99.70%, with the lowest number of false positives(legitimate emails falsely identified as phishing) and false negatives(phishing emails falsely identified as safe). This performance exceeded our expectations and highlighted the potential of using LLMs for phishing detection without the need for extensive training data or relying on external data.
Table: LLM Performance Comparison
| LLM | True Positive | False Positive | True Negative | False Negative | Accuracy |
| GPT-4 | 1,007 | 3 | 997 | 3 | 99.70% |
| GPT-3.5 | 980 | 32 | 968 | 30 | 96.92% |
| Llama 2 70B | 950 | 361 | 639 | 60 | 79.05% |
| Gemini Pro | 991 | 21 | 979 | 19 | 98.01% |
Understanding the Detection Mechanisms
The question then arises: how do these LLMs achieve such high precision in phishing email detection? Let's delve into specific examples of phishing emails impersonating Amazon and Apple to understand the mechanics behind their success.
Amazon Phishing Email Example
Result of GPT-4 Analysis
| { "is_phishing": true, "phishing_score": 8, "brand_impersonated": "Amazon Prime", "rationales": "The email contains several phishing indicators, including impersonation, spoofing signs in the email header, and social engineering tactics in the email body. The sender name, 'Prime', is an attempt to impersonate the well-known brand Amazon Prime. The email address '_____@perfectzecure.org[.]ng' is not associated with Amazon. In the email body, the statement 'Your membership benefits are currently on hold' creates a sense of urgency to induce clicks on the hyperlinks. The 'basreng...' and 'A͏͏m͏͏a͏͏z͏͏' inserted in the text between legitimate wordings is another suspicious sign. In addition, the email contains URLs, embedded in text like 'Payment Declined' and 'Update Payment information', that lead to a LinkedIn website, which is an odd and suspicious context. Hence, there are strong reasons to determine this email as a phishing attempt.", "brief_reason": "The email impersonates the Amazon Prime brand, uses social engineering tactics, and contains suspicious hyperlinks." } |
GPT-4's analysis of a phishing email masquerading as an Amazon Prime alert flagged it as a phishing attempt. The email attempted to create a sense of urgency by claiming problems with the recipient's Amazon Prime account and urging them to enter payment information on a phishing site within six days. GPT-4 accurately identified it as phishing, with a phishing_score of 8 out of 10, and correctly identified Amazon Prime as the impersonated brand. The rationale included irrelevant sender addresses, the use of social engineering techniques to create urgency, and links to an unrelated domain name. This analysis reveals how LLMs can identify the suspiciousness of phishing emails, from the structure of the header to the context of the message.
Apple Phishing Email Example
Result of GPT-4 Analysis
| { "is_phishing": true, "phishing_score": 8, "brand_impersonated": "Microsoft", "rationales": "This email appears to be a phishing attempt. The sender is ostensibly from 'crous-montpellier[.]fr', but the email's body text mentions the 'Microsoft account team', which is an impersonation of the Microsoft brand. This discrepancy is a sign of phishing. The email subject line, 'Your new user agreement', is designed to create a sense of urgency and lure the recipient into clicking on the link provided. The hyperlink uses a tracking ID which suggests that it might be used for collecting sensitive information. Examining the authentication results from the header, the emails passed SPF (Sender Policy Framework) and DKIM (DomainKeys Identified Mail) checks. However, this could still be sophisticated phishing, as attackers nowadays often compromise legitimate servers to launch phishing campaigns.", "brief_reason": "Impersonation of a well-known brand (Microsoft), inconsistency in sender's information, and presence of a suspicious hyperlink." } |
Similarly, an email pretending to be from Microsoft asking users to agree to an updated user agreement was analyzed by GPT-4. It successfully flagged the email as phishing based on key indicators such as the sender address not matching Microsoft's, the inclusion of misleading information, and suspicious tracking IDs in a link. Despite passing SPF and DKIM checks, these factors combined to indicate a phishing attempt, demonstrating the model's ability to look beyond superficial authenticity.
Conclusion
In this article, we introduced the system we developed, ChatSpamDetector. Not only does it offer high accuracy in detecting phishing emails, but it also provides users with understandable explanations for its assessments. This experiment has demonstrated the capability of LLMs to accurately identify brand impersonation and deceptive techniques used in phishing. The success of this system underscores the potential of leveraging LLM's sophisticated language processing abilities for security applications, paving the way for further research and development in this arena.
Reference
[1] Takashi Koide, Naoki Fukushi, Hiroki Nakano, and Daiki Chiba. 2024. ChatSpamDetector: Leveraging Large Language Models for Effective Phishing Email Detection. https://arxiv.org/abs/2402.18093