AI Phishing: How Artificial Intelligence Is Changing Cyber Fraud

AI phishing is a form of cyber fraud in which criminals use artificial intelligence to develop, personalise or deliver deceptive communications.

Like traditional phishing, its purpose is usually to persuade someone to take an action that benefits the attacker. This might involve disclosing a password, opening a malicious attachment, approving a fraudulent payment or sharing confidential information.

The difference lies in how artificial intelligence supports the attack.

Generative AI tools can produce natural-sounding messages, adapt their wording to different audiences and help criminals create convincing impersonations.

For example, a fraudulent email might appear to come from a company’s finance director. It could refer to a genuine supplier, use familiar business terminology and request an urgent payment.

The message may look professional and contain no obvious language errors. However, the apparent sender and payment instructions could be fraudulent.

According to the National Cyber Security Centre (NCSC, 2024), generative AI can make phishing communications more convincing by removing some of the spelling, grammar and translation errors previously associated with scams.

AI phishing therefore makes traditional indicators of suspicious communication less dependable.

Artificial intelligence is changing phishing attacks by making it easier for criminals to create persuasive content, personalise messages and imitate trusted individuals.

The NCSC’s assessment of cyber threats to 2027 identifies AI as a technology that is likely to improve the efficiency and effectiveness of several existing criminal techniques, including social engineering (NCSC, 2025).

Social engineering means manipulating people into taking actions they would not otherwise authorise. Several developments are particularly relevant to workplace phishing.

AI writing tools can produce emails that use professional language, consistent formatting and an appropriate tone. Previously, some fraudulent emails contained obvious spelling mistakes or unusual sentence structures. AI-generated communications may lack those warning signs.

A message requesting an employee to update login details, for instance, could closely resemble a routine communication from an IT department. Employees therefore need to consider the request itself rather than relying on how professionally it is written.

AI can help criminals analyse publicly available information and develop messages relevant to a particular target.

Information about an employee’s job title, employer, responsibilities or professional contacts may help make a fraudulent request appear legitimate.

A facilities manager could receive a message referring to a building maintenance contract, while an accounts employee might receive payment instructions that appear to involve an existing supplier. Targeted phishing of this kind is commonly called spear phishing.

Although personalised phishing existed before generative AI, AI tools can reduce the effort involved in preparing convincing messages.

Generative AI can assist criminals in producing multiple versions of a message without drafting each one manually.

Messages can be adapted for different recipients, languages or business situations. This creates opportunities for attackers to test and refine their communications while targeting more people.

The NCSC (2025) assesses that AI is likely to increase the volume and impact of cyber intrusions by improving existing attack techniques rather than necessarily introducing entirely new forms of attack.

AI-generated audio and video create additional opportunities for criminals to impersonate trusted people. Voice-cloning technology can imitate aspects of someone’s speech using audio samples, while deepfake technology can generate or manipulate realistic-looking video and audio.

A fraudulent request might therefore appear to come from a senior manager through a telephone call, voice message or video communication. The important distinction is that recognising a familiar voice or appearance is no longer sufficient evidence that a request is genuine.

AI phishing can appear through emails, text messages, telephone calls and other digital communication channels.

The technology used may differ, but these attacks generally rely on impersonation, deception or pressure to obtain information or influence a decision.

Criminals use AI tools to draft fraudulent emails that imitate legitimate workplace communications.

These emails might request password resets, encourage users to open documents or direct employees to fake login pages. An attacker could impersonate a software provider and claim that an employee’s account requires urgent verification.

The message may be carefully written, but the linked website could be designed to steal login credentials.

Spear phishing targets a specific person or group rather than sending the same message to a broad audience. AI may help criminals prepare emails that refer to an employee’s role, workplace activities or known professional relationships.

For example, a project manager might receive a fraudulent message concerning a genuine upcoming project. Familiar details can make the communication appear credible, even when the sender has no legitimate connection to the organisation.

Voice phishing, also known as vishing, involves fraudulent telephone or voice communications.

AI-generated voices may allow criminals to imitate someone the recipient knows. An employee might receive a call that appears to come from a senior manager requesting confidential information or an urgent payment.

Even a convincing voice should not replace established identity checks or financial approval procedures.

Deepfake phishing uses AI-generated or manipulated video to support impersonation. A criminal might attempt to appear as a colleague or senior executive during a video interaction.

Although these attacks can be technically more demanding than ordinary email phishing, they create an additional challenge for organisations that rely on visual familiarity when confirming identity.

Sensitive requests should still be verified through an independently established communication channel.

Text message phishing is commonly called smishing. AI can help attackers draft convincing messages that appear to come from delivery companies, service providers or workplace contacts.

A message might claim that an employee must urgently confirm an account, arrange a delivery or review a document. The link may instead lead to a fraudulent website intended to collect information.

AI phishing can be harder to detect because fraudulent communications may closely resemble legitimate messages in language, context and presentation.

Traditional phishing awareness often emphasised spelling mistakes, poor formatting and unusual wording. These remain useful warning signs when present, but their absence does not establish that a message is safe.

The NCSC’s guidance on recognising scams explains that modern fraudulent communications can be convincing enough to deceive experienced users (NCSC, 2021).

Three factors are particularly important.

First, apparent familiarity can create misplaced confidence. A message that mentions a known supplier or colleague may appear legitimate even when the information was obtained from public sources.

Second, urgency can influence decision-making. Criminals may claim that immediate action is needed to prevent account suspension, resolve a payment problem or address an operational emergency.

Third, impersonation can weaken familiar identity checks. A recognisable name, email display name or AI-generated voice does not independently confirm who is communicating.

For employers, the practical implication is that security arrangements should not depend entirely on employees identifying every fraudulent message.

Employees can recognise possible AI phishing attempts by examining what a message asks them to do, checking the sender through a trusted route and identifying unusual pressure or changes to established procedures.

There is no reliable visual test that distinguishes every AI-generated phishing message from genuine communication.

However, several warning signs remain relevant.

• Unexpected requests for sensitive information: A message asks for passwords, verification codes, confidential documents or account details without a legitimate explanation.
• Urgent payment instructions: Someone requests an immediate transfer or unexpected change to a supplier’s bank details.
• Pressure to bypass procedures: The sender claims normal approval arrangements should be ignored because the matter is confidential or urgent.
• Unusual links or attachments: A communication contains an unexpected document, unfamiliar website address or request to download software.
• Changes in normal communication patterns: A colleague or supplier makes a request through an unfamiliar channel or asks for something outside their usual responsibilities.
• Requests for secrecy: The sender discourages the employee from checking the request with colleagues or managers.

These signs do not prove that a message is fraudulent. Equally, a message without these characteristics is not necessarily genuine. The most useful response is to verify requests involving money, access or sensitive information independently.

For example, an employee receiving unexpected payment instructions from a supplier should contact the supplier using a telephone number already held in the organisation’s verified records, rather than one included in the suspicious message.

Organisations can reduce AI phishing risks by combining technical security measures, clear working procedures and employee awareness. The NCSC recommends a layered approach to phishing prevention rather than relying solely on employees to identify suspicious messages (NCSC, 2018).

Organisations should use appropriate email filtering, malicious link detection and anti-spoofing controls to reduce the number of fraudulent messages reaching employees.

Technical controls such as SPF, DKIM and DMARC can help receiving systems assess whether messages are authorised to use a particular email domain.

These measures do not stop every phishing attempt, particularly when attackers use legitimate but compromised accounts or deceptive lookalike domains.

Multi-factor authentication (MFA) adds another identity check when someone accesses an account. Even if an attacker obtains a password through phishing, an additional authentication requirement may make unauthorised access more difficult.

Organisations should use phishing-resistant authentication methods where appropriate, particularly for accounts with access to sensitive information or important systems. MFA is valuable but does not remove the need for other security measures.

Verification procedures are particularly important for requests involving financial transactions, confidential information or changes to access permissions.

Organisations should establish clear processes for confirming:

• New or amended supplier payment details
• Unusual financial transfers
• Requests for passwords or authentication codes
• Changes to employee access permissions
• Unexpected instructions from senior management

Employees should understand that a familiar email address, voice or video appearance is not sufficient proof of identity.

Employees need to understand how modern phishing attacks work and how to respond to suspicious communications.

Training should cover AI-generated messages, targeted impersonation, fraudulent links, voice phishing and the limitations of familiar warning signs. Employees should also know how to report a suspicious message without feeling that they must first prove it is fraudulent.

Awareness training works best when supported by effective technical controls and clear organisational procedures.

A straightforward reporting procedure can help organisations investigate suspicious communications before further employees are affected.

Employees should know where to report potential phishing attempts, what information to provide and whom to contact if they have already followed a fraudulent instruction.

Managers should encourage prompt reporting, including when an employee has clicked a suspicious link or disclosed information.

The NCSC’s guidance recommends helping users report suspected phishing messages as part of a broader organisational defence strategy (NCSC, 2018).

Organisations should have arrangements for responding when phishing is suspected or confirmed.

These may include isolating affected devices, restricting compromised accounts, investigating suspicious activity and assessing whether information has been exposed. Responsibilities should be clear before an incident occurs. An established response procedure can help limit disruption and support timely decisions.

Employees who suspect they have interacted with a phishing attack should report the incident promptly and follow their organisation’s security procedures.

The appropriate response depends on whether they opened a message, clicked a link, disclosed information, downloaded a file or authorised a payment.

A practical response involves the following steps.

1. Stop interacting with the suspicious communication. Avoid clicking additional links, opening attachments or replying to the apparent sender.
2. Notify the relevant internal contact. Report the incident to the IT, security or designated management team using the organisation’s established reporting process.
3. Explain what happened. Provide the message and details of any information shared, links opened or payments authorised. Employees should not delete potentially useful evidence unless instructed by the response team.
4. Secure affected accounts or devices. Follow IT instructions, which may include changing passwords through a trusted website, revoking active sessions or isolating a device.
5. Escalate financial concerns immediately. If a payment has been made or financial details disclosed, contact the organisation’s finance team and relevant financial provider promptly.
6. Support the investigation. The responsible team should assess the incident, establish its impact and identify any further action needed.

Suspicious emails can also be forwarded to the NCSC’s Suspicious Email Reporting Service at report@phishing.gov.uk (NCSC, 2021).

If an incident involves a personal data breach, the organisation should assess its notification responsibilities. Under the UK GDPR, certain personal data breaches must be reported to the Information Commissioner’s Office (ICO) without undue delay and, where feasible, within 72 hours of becoming aware of the breach. Notification depends on the risk to individuals, rather than phishing alone automatically triggering a reporting requirement (ICO, 2025).

AI phishing awareness helps employees understand why apparently genuine communications may still require verification.

Artificial intelligence is making familiar phishing techniques easier to adapt and more convincing in some circumstances. Organisations therefore need to look beyond obvious warning signs and focus on the actions attackers are trying to encourage.

Three principles are particularly useful:

• Professional wording and familiar identities do not prove a communication is genuine.
• Independent checks help protect sensitive information, financial transactions and system access.
• Employee awareness is most effective when supported by technical safeguards and clear reporting procedures.

Cyber criminals may change the way they prepare fraudulent messages, but organisations can strengthen their resilience by making secure verification and reporting part of everyday working practices.

Employees who understand how AI phishing, voice impersonation and social engineering work are better placed to recognise suspicious requests and follow secure verification procedures.

Human Focus Cyber Security Awareness Training is a CPD-certified online course covering phishing, vishing, social engineering, password security and practical cyber security habits. The course includes guidance on recognising fraudulent emails, text messages, QR-code scams and deepfake voice impersonation, alongside an online assessment and completion certificate.

Explore Human Focus Cyber Security Awareness Training to support your organisation’s employee awareness programme and reinforce safer responses to cyber fraud.