How to Defend Against Attackers Using Local LLMs and RAG for Phishing and Attack Automation

Local LLMs plus RAG turns stolen files into a searchable attack platform

Genians reported evidence that Kimsuky set up local model tooling (Ollama, GPT4All, Msty) alongside RAG and other AI-adjacent software, suggesting a shift from ad hoc generative phishing to an integrated workflow for analysis and automation. Our central claim is this: as attackers move LLM workloads on-premises, defensive programs that rely on spotting cloud AI usage will miss the core risk, so the right response is to secure document access paths, endpoint execution, and identity boundaries where offline AI actually operates.

• Local inference removes your cloud visibility by design. When operators can run and manage models locally using tools like Ollama, GPT4All, and Msty, the most sensitive processing happens on the endpoint or in attacker-controlled infrastructure, not in an external AI service you might otherwise monitor. That implies your detection and containment needs to shift toward endpoint telemetry, execution control, and identity hygiene rather than hoping to observe prompts leaving your network.
• RAG turns a pile of files into an interactive knowledge base. Genians described the use of retrieval augmented generation for document search, which in practice means stolen material can be indexed and queried in a way that accelerates targeting, pretext crafting, and follow-on decisions. The defender’s problem is no longer just exfiltration; it is what happens when data becomes instantly explorable and rewriteable by an operator.
• Agent frameworks imply workflow automation, not just content generation. The report noted AI agent development frameworks, which typically means an attacker is thinking in terms of multi-step tasks, tool use, and orchestration. For defenders, this matters because automation increases the speed and consistency of repeated operations like triage of stolen data, generation of variants, and iterative improvement of phishing.
• AI-assisted coding and speech-to-text suggest faster iteration. Genians found Cursor and speech-to-text software in infrastructure linked to the campaign. Even without assuming any particular feature set, the presence of these tools implies shortened cycles between idea, implementation, and deployment of malicious artifacts and lures, making time-to-detection and response process maturity more important than one-off content filters.

Quick Answer: how do we defend against attackers running LLMs locally with RAG?

We defend by treating local AI runtimes and AI-enabled tooling as a high-risk execution layer and by tightening the data and identity surfaces that make RAG and phishing effective. In practice, that means controlling where sensitive documents can be accessed, improving endpoint governance so unsanctioned model runners and AI dev tools don’t operate invisibly, and strengthening authentication and workflow verification so convincing AI-written content cannot trigger privileged actions.

• Harden endpoints for offline AI, not just malware. If attackers can run models locally, the decisive telemetry is often process activity, file access patterns, and credential use on the workstation or server doing the work, rather than network indicators tied to external AI APIs. The operational trade-off is that endpoint controls can introduce friction for legitimate engineering and data teams, so you need a governance model that separates approved local tooling from unapproved execution.
• Design your document repositories as security boundaries. RAG changes the economics of stolen material by making it easy to search and summarize, so controlling access to document stores, exports, and synced folders becomes a primary defense. The trade-off is that tighter controls can slow internal collaboration, which is why we prefer capability-based access patterns and auditable workflows over blanket lock-downs.
• Assume AI-crafted decoys will look like normal business artifacts. Genians said it found finance and cryptocurrency-themed decoy documents that appeared AI-generated and resembled investment reports and workplace documents. That means looks suspicious heuristics decay fast, and you need process-level verification—especially around finance, HR, and vendor onboarding—so a realistic document cannot independently authorize a change.
• Treat identity as the choke point for automation. Automation and agent-like workflows become dangerous when they can reuse tokens, sessions, or approval paths at scale. Even if you cannot see the AI reasoning, you can usually constrain what identities can do, where they can do it from, and what secondary verification is required for sensitive actions.

Why local inference changes the defender’s control plane

Genians’ findings explicitly point to tools that can run and manage AI models locally and process documents without sending sensitive information to outside AI services. That single detail is the engineering inflection point: many enterprise AI risk controls are built around egress visibility, policy enforcement at SaaS boundaries, or vendor governance of external model providers. If the attacker’s workflow stays local, those controls become less relevant to detection and less effective at prevention.

The operational implication is uncomfortable but actionable. We have to separate two problems that often get mixed together: the model as a generator, and the environment where the model is executed and fed data. When execution is local, the environment becomes the attack surface, and your architecture decisions—endpoint governance, document system boundaries, identity workflows—matter more than debating which LLM family is most likely to produce a suspicious phrase.

Egress monitoring won’t see the model work

When operators can keep processing on their own infrastructure, defenders lose the tell of prompts and outputs traveling to third-party AI services. That doesn’t mean detection is impossible; it means the detection layer moves. In practice, teams end up leaning more heavily on endpoint detection and response, identity logs, and high-signal events like unusual document access or sudden spikes in file reads, because the AI reasoning itself is not observable as network traffic.

RAG makes stolen data instantly queryable

RAG is described as document search technology in the report, and that framing is useful for defenders. Once stolen documents can be searched and stitched into answers, the attacker’s bottleneck shifts from can we steal it to can we operationalize it quickly. That implies that any repository with broad access—shared drives, knowledge bases, project folders—becomes a force multiplier for phishing realism and for targeting specific teams with insider-language.

Your document supply chain is the real security boundary, not the model

Genians reported finance and cryptocurrency-themed decoy documents that appeared to be AI-generated and were designed to resemble legitimate investment reports and other workplace documents. For a CTO, the decisive detail is not that a decoy exists; it is that the decoy is shaped to pass as normal work. Once an attacker can draft plausible documents at scale and refine them with local model tooling, security teams cannot depend on user intuition or superficial content scanning to stop social engineering.

This forces an architectural rethink of the document supply chain: how documents enter the organization, how they are shared, and what downstream systems treat them as authoritative. If a PDF that looks like a real investment report can cause a finance team to update vendor payment instructions, the vulnerability is the workflow that allows documents to trigger privileged actions without strong verification. That workflow vulnerability exists regardless of whether the attacker used a large model, a template, or a human writer.

At Plavno, when we’re brought into security modernization work, we push clients to model document-triggered actions the same way they model API-triggered actions. The engineering question becomes: which systems accept user-provided documents as inputs, and what secondary controls exist before those inputs alter identity, money movement, or production access? This is also where targeted assessment work like cybersecurity and penetration testing stops being a compliance exercise and becomes a way to map real operational paths attackers will exploit.

The attacker’s tooling list is your backlog of controls

Genians found local model runners (Ollama, GPT4All, Msty), RAG tooling, AI agent development frameworks, speech-to-text software, and Cursor on infrastructure linked to the campaign, suggesting capacity-building beyond simple phishing text generation. We should read that as a practical control-mapping exercise: each tool category implies a workflow step, and each workflow step points to a defensive choke point in endpoints, identity, data stores, and software execution policy.

Cursor and AI-assisted coding shrink the malware iteration loop

The report noted Cursor as present on infrastructure tied to the campaign, alongside other AI tooling. We don’t need to assume any specific feature to see the engineering risk: AI-assisted coding reduces the cost of trying variations, and that can compress the time between an initial intrusion idea and a working artifact. Defenders should expect more rapid experimentation and therefore plan for faster containment loops, because waiting for high-confidence signatures can become a losing strategy.

1. Start with an offline attacker threat model for your top workflows. Instead of asking whether employees are using external AI services, model the scenario where an adversary can do analysis and writing locally using tools like Ollama, GPT4All, or Msty, then send only the final lure or request to your team. The concrete output of this step is a set of business workflows—finance approvals, HR updates, access requests—where a convincing document or message can cause privileged action.

2. Treat broad document access as an incident precursor, not a convenience feature. RAG makes document collections more valuable to an attacker because it turns them into a queryable system. In practice, we aim to reduce everyone can read everything patterns in shared repositories and ensure audit trails exist for sensitive folders, because unusual reads and exports become the early warning signals once you accept that the attacker can process the content locally.

3. Define what approved AI tooling on endpoints means, then enforce it. The presence of local model runners and AI dev tools in the attacker stack should push enterprises to decide which AI runtimes are permitted, under what accounts, and with what logging expectations. The trade-off is that over-restrictive policies can drive legitimate users to shadow IT; the goal is governed enablement, not blanket prohibition.

4. Harden identity flows so realistic content cannot finalize high-impact changes. Agent-like automation becomes dangerous when a single credential can carry a sequence of actions without friction. We prefer layered verification for actions like payment destination changes, privileged access grants, and production configuration edits, so even a perfectly written decoy document cannot complete the attack without passing a separate authority check.

Operationalizing policy: treat local AI runtimes like compilers

A local LLM runner is not just a chat app; it is an execution environment that transforms inputs into outputs with business impact, much like a compiler transforms source into binaries. When Genians describes tools for running models locally to avoid sending sensitive data to outside AI services, it highlights why policy must attach to execution and data access rather than to network destinations. If you can’t govern the runtime, you can’t govern the risk.

• Build a governance line between sanctioned and unsanctioned local model execution. In a typical enterprise, developers and analysts will have legitimate reasons to experiment with AI tools, and attackers will exploit the same surface. The practical control is an explicit allow/deny posture for AI runtimes and related tooling on managed machines, combined with a process for requesting exceptions that doesn’t encourage people to bypass controls.
• Instrument document access as a first-class security signal. RAG’s value rises with the volume and sensitivity of accessible documents, so repository read patterns matter. We generally treat unusual bulk reads, sudden access to finance or strategy folders, or repeated searches across sensitive areas as higher-signal than email content heuristics, because the attacker may generate perfect text but still needs data access to make it credible.
• Constrain how outputs can trigger privileged actions. A realistic decoy document is only powerful if downstream systems accept it as authorization. Teams should examine where documents, emails, or tickets can cause changes in payment details, user privileges, or production settings, then ensure that the final authorization step requires identity verification separate from the document itself.
• Assume speech-to-text can accelerate social engineering and operational coordination. With speech-to-text in the toolchain, adversaries can more easily convert captured audio or live interactions into text artifacts for analysis and reuse. The defensive response is not to detect AI speech; it is to treat voice-driven requests and recorded instructions as untrusted inputs unless authenticated through a separate channel.

Evaluating defenses in practice without chasing model brands

It is tempting to treat the presence of Ollama, GPT4All, or Msty as the core issue and draft policies that name specific tools. That approach ages poorly because the underlying shift is architectural: model execution is local, and the adversary can swap runtimes while keeping the workflow intact. A better evaluation approach asks where in your environment sensitive documents can be accessed, how identity is verified for high-impact actions, and what telemetry exists at the endpoint where local processing occurs.

At Plavno, we prefer security work that produces engineering artifacts: clear boundaries, observable events, and enforceable policy. If your team wants help turning this threat signal into a concrete architecture plan, our AI consulting engagements typically focus on aligning identity, data access, and runtime governance so offline AI is treated like any other high-risk compute workload rather than an unmanageable novelty.

Real deployments you should assume exist: phishing, analysis, and automation loops

Genians’ report suggests Kimsuky is building capacity to integrate existing AI models into malware development, data analysis, and attack automation, and it also found decoy documents that appeared AI-generated. Taken together, this implies an end-to-end pipeline where stolen material is processed locally, summarized into operator-friendly notes, and then converted into highly tailored lures that look like ordinary workplace artifacts. For the defender, the crucial step is accepting that the attacker’s research phase can now be fast and systematic.

The next likely pattern is iteration. With local model tooling, speech-to-text, and AI-assisted coding in the environment, an operator can rapidly adjust messaging, generate multiple variants for different departments, and refine based on what gets engagement. Whether or not any single email is detectable, the campaign effectiveness can rise because the attacker can afford to test and adapt repeatedly. That makes response operations—reporting workflows, containment, and identity resets—more central than content classification.

Finally, the presence of AI agent development frameworks should push us to assume more structured automation. Even without specifying a particular framework, the concept implies multi-step execution: pull material, search it with RAG, draft an artifact, possibly generate or modify code, and then deliver. Defenders should mirror that thinking by automating their own controls, particularly around ticketing and approvals where AI-written text can blend into normal operations. This is exactly the type of workflow-centric hardening we build in AI automation programs when clients need engineering systems that resist realistic social engineering.

Risk: you can’t patch social engineering with a model filter

The report’s most actionable implication is that decoys can be generated to resemble normal workplace documents, and local processing avoids exposure to outside services that might have policy controls. That combination means content filtering alone becomes an unstable foundation: even if you detect some AI-generated text, the attacker can adjust. Durable defense comes from verifying authority and constraining what any single message or document can cause in your systems.

• Over-reliance on detection creates a false sense of control. When attackers can iterate quickly using local tooling, defenders who wait for high-confidence classifiers or AI content detectors can fall behind. The trade-off is that moving toward authority-based controls and stronger workflow verification can slow certain business processes, so you need executive alignment on which actions deserve friction.
• Tight endpoint restrictions can create internal shadow tooling. If you respond by banning broad classes of AI runtimes or developer tooling, some teams will find workarounds, often outside managed environments. That can reduce security posture, not improve it. The pragmatic goal is to enable approved tooling with clear guardrails and logging expectations, while blocking unknown execution in sensitive zones.
• Repository lockdown can damage collaboration if done bluntly. Because RAG amplifies the value of accessible documents, the instinct is to restrict everything. In practice, the sustainable approach is to segment by sensitivity and business function, ensuring high-risk areas have auditability and stronger controls, while keeping low-risk collaboration fluid.
• Identity hardening surfaces legacy workflow debt. Stronger verification and least privilege often exposes how many systems were built assuming trust in internal messages and documents. Remediating that is real engineering work across ticketing, approvals, finance operations, and access management, and it can’t be solved by security tooling alone.

Plavno’s position: security architecture beats model choice

Genians’ findings suggest a real shift: adversaries are investing in a local AI stack, RAG, and agent tooling so they can analyze material and build campaigns without relying on outside services. We take a firm position on what that means for engineering leaders: you should stop treating which model as the security question and start treating which systems can be influenced by documents and messages as the security question. The fastest wins come from hardening identity and workflow authority, not from debating which LLM runtime is scariest.

1. We map document-to-action paths like we map API attack surfaces. The practical work starts by enumerating where business documents, emails, or tickets can trigger privileged actions—payment changes, access grants, configuration edits—and then testing how those actions are authorized. This is where a decoy investment report that looks legitimate becomes dangerous, and it is where the defender can add durable verification points.

2. We design governed AI runtimes instead of pretending they won’t exist. Because local tools like Ollama, GPT4All, or Msty can be used to avoid sending data to outside services, organizations will increasingly run AI locally for legitimate reasons too. We help teams define what approved local AI looks like in terms of environment, access, and auditability, which is a core component of our AI agents development work when clients need production-grade controls.

3. We tie identity controls to operational reality, not idealized diagrams. Agent-like automation increases the impact of a single credential, so we focus on where identities are created, how they are verified, and what actions require step-up verification. The output should be enforceable in IAM and in application logic, because policy documents do not stop an automated workflow.

4. We validate with scenario-based exercises that reflect modern attacker tooling. A report like Genians’ provides a realistic attacker toolkit, and the right test is whether your organization can detect and interrupt the implied workflows: bulk document access, generation of realistic artifacts, and attempts to use them to trigger privileged actions. The objective is to shrink time-to-containment, not to catch AI text.

When outsourcing or outstaffing becomes a security decision

This shift also changes how we think about delivery models. If the core risks now sit at endpoint governance, document authorization, and identity workflows, then the teams touching those areas must follow disciplined engineering practices and operate inside your security boundaries. Whether you augment capacity through internal hires or external contributors, the real requirement is that the work is done with strong process controls and clear ownership for identity, repository permissions, and production change paths.

1. If your biggest pain is visibility, prioritize endpoint and identity instrumentation first. Local model execution reduces the value of network-only monitoring, so your evaluation should start with whether you can see and respond to suspicious file access, process behavior, and credential use. If you can’t, adding more content scanning won’t compensate.

2. If your biggest pain is workflow abuse, redesign authorization steps before adding more detectors. The report’s decoy documents that resemble legitimate workplace artifacts are a warning that looks normal is no longer a safe proxy for is authorized. The right move is to ensure that high-impact actions require verification outside the document channel.

3. If your biggest pain is data sprawl, segment repositories to reduce RAG leverage. RAG increases the operational value of broad access, so the key question is whether sensitive repositories are meaningfully segmented and audited. The trade-off is collaboration friction; the goal is to protect high-impact areas without freezing the entire organization.

4. If your biggest pain is delivery speed, automate defense workflows just as attackers automate offense. Agent frameworks and AI tooling suggest adversaries can run repeatable loops. Your response should include repeatable containment loops: fast reporting, rapid credential resets, and consistent verification steps for sensitive business processes.

Closing insight: assume the attacker is offline and fluent

Genians’ report on Kimsuky points to a future where sophisticated actors can run AI locally, search documents with RAG, and industrialize phishing and automation while keeping sensitive material off external AI services. For engineering leaders, the practical takeaway is simple: design your systems so that realistic-looking artifacts cannot carry authority by themselves, and so that broad document access and credential reuse are difficult to exploit at scale. Author: Plavno team. Last updated: August 2026.

Summarize this blog post with AI

Chat GPT Perplexity Claude Gemini