Learn Ethical Hacking (#85) - Defending Against AI-Powered Attacks - A Practical Guide

@scipio 71

3 days ago

StemSocial

Learn Ethical Hacking (#85) - Defending Against AI-Powered Attacks - A Practical Guide

What will I learn

How AI changes the threat landscape -- what attackers can do with AI that they could not do before;
Defending against AI-generated phishing -- when grammar and personalization are no longer red flags;
Defending against deepfake attacks -- procedural controls that survive when voice and video cannot be trusted;
Defending against AI-powered reconnaissance -- when OSINT collection happens at machine speed;
Defending against AI-assisted exploitation -- automated vulnerability discovery and exploit generation;
Defending against AI-enhanced evasion -- polymorphic malware and adversarial inputs;
Building AI-resilient defenses -- what still works when the attacker has AI;
Defense: the controls that remain effective regardless of how sophisticated AI-powered attacks become.

Requirements

A working modern computer running macOS, Windows or Ubuntu;
Understanding of AI attacks from episodes 49, 58;
Understanding of defensive techniques from episodes 71-82;
The ambition to learn ethical hacking and security research.

Difficulty

Intermediate/Advanced

Curriculum (of the `Learn Ethical Hacking Series`):

Learn Ethical Hacking (#85) - Defending Against AI-Powered Attacks - A Practical Guide

Solutions to Episode 84 Exercises

Exercise 1: Vulnerable AWS lab (abbreviated).

Built with Terraform:
  - S3 bucket "lab-dev-assets" with public-read ACL
  - .env file uploaded with fake AWS credentials
  - IAM user "deploy-svc" with PassRole + EC2 + S3 permissions
  - Lambda "process-data" with DB_PASSWORD in env vars
  - RDS PostgreSQL with default security group (0.0.0.0/0)

Attack walkthrough:
  Phase 1: found public bucket via naming pattern
  Phase 2: downloaded .env, got IAM credentials
  Phase 3: enumerate-iam revealed PassRole + RunInstances
  Phase 4: launched EC2 with admin role, accessed all S3 buckets
  Phase 5: created backdoor IAM user
  Total time: 2.5 hours (lab environment, known target)

The attack chain from Phase 1 to Phase 5 took 2.5 hours, and every single step generated CloudTrail events that a properly configured monitoring setup would have caught. The naming pattern discovery (Phase 1) is the part that happens invisibly -- the attacker tries lab-dev-assets, lab-staging-assets, lab-prod-assets and one of them responds. No log on the target side until the attacker actually calls ListObjects or GetObject on the bucket. The .env file download in Phase 2 generates a GetObject event, but if the bucket is configured with public-read ACL, that event does NOT require authentication and many organizations do not monitor unauthenticated S3 access at all. This is why Phase 1 and Phase 2 are the hardest to detect -- the attacker is using the access you gave them.

The enumerate-iam tool in Phase 3 is noisy. It calls dozens of IAM API endpoints (iam:ListRoles, iam:ListUsers, iam:GetUser, iam:ListAttachedUserPolicies, etc.) in rapid succession, generating a burst of CloudTrail events from a new IP address within seconds. If your GuardDuty is enabled, this should trigger Recon:IAMUser/MaliciousIPCaller almost immediately. The fact that many organizations still do not have GuardDuty enabled (it costs roughly $1 per million CloudTrail events) is one of those situations where a few dollars per month would have prevented the entire attack chain from progressing past Phase 3.

Exercise 2: GuardDuty detection results (abbreviated).

GuardDuty findings generated:
  1. Policy:S3/BucketAnonymousAccessGranted (when bucket was public)
  2. UnauthorizedAccess:IAMUser/ConsoleLogin (credentials from new IP)
  3. Recon:IAMUser/MaliciousIPCaller (iam:ListRoles from unusual IP)
  4. Persistence:IAMUser/AnomalousBehavior (new IAM user created)

GuardDuty MISSED:
  - S3 data download (no finding for legitimate GetObject with valid creds)
  - EC2 launch with admin role (not a default GuardDuty finding)
  - Lambda environment variable reading (GetFunctionConfiguration)

Detection latency: 5-15 minutes for each finding.
GuardDuty caught 4 of 7 attack phases. Custom SIEM rules needed for the rest.

The 5-15 minute detection latency is the number that should concern you most. In episode 80 we discussed how automated response reduces containment time from 25-45 minutes (manual) to 30-90 seconds (automated). But if detection itself takes 5-15 minutes, even a fully automated response cannot act until the alert fires. In those 5-15 minutes, the attacker completed Phase 3 (enumeration), Phase 4 (EC2 launch with admin role), and was well into Phase 5 (persistence). GuardDuty's detection latency is a fundamental architectural limitation -- it processes CloudTrail events in near-real-time but "near-real-time" for a cloud service means minutes, not seconds. The custom CloudWatch metric filters are faster (they process events as they arrive in the log group) but they require you to know WHAT to look for in advance, which means you need to model the attack chain before it happens.

The three phases GuardDuty missed are instructive. The S3 data download used valid credentials (stolen from the .env file), so from AWS's perspective it was a legitimate GetObject call -- there is nothing inherently suspicious about downloading a file from a bucket you have access to. The EC2 launch with an admin role is a legitimate operation that developers perform daily. The Lambda GetFunctionConfiguration call (which exposed DB_PASSWORD from environment variables) is a standard developer action. These are all examples of living off the land in the cloud -- using legitimate AWS services and APIs in ways that individually look normal but collectively form an attack chain.

Exercise 3: Defensive fixes and re-test (abbreviated).

Fixes applied:
  1. S3 Block Public Access: bucket no longer accessible anonymously
  2. Credentials moved to Secrets Manager, .env deleted from S3
  3. IAM PassRole restricted to specific Lambda execution role only
  4. IMDSv2 enforced on all EC2 instances
  5. GuardDuty enabled with SNS alerting

Re-test:
  Phase 1: BLOCKED (S3 Block Public Access)
  Phase 2: BLOCKED (no .env to download)
  Phase 3: BLOCKED (PassRole restricted, cannot assume admin role)
  Attack chain broken at Phase 1. Cannot proceed without initial access.

The re-test result is satisfying but also slightly misleading. The attack chain broke at Phase 1 because the specific attack path we used (public S3 bucket with credentials) no longer exists. A real attacker would not stop here -- they would look for OTHER initial access vectors: a web application vulnerability (episodes 12-22), a phishing email targeting an employee with AWS console access (episode 39), a leaked credential in a public GitHub repository (episode 45), or a misconfigured API endpoint (episode 21). The fixes from this exercise address the attack chain we tested, but defense in depth (episode 53) means assuming the attacker WILL find an initial access vector and building detection and containment for every subsequent phase as well. The IMDSv2 enforcement is the most broadly impactful fix because it blocks SSRF-based credential theft (episode 18) from ANY EC2 instance, regardless of how the attacker got initial access.

Episode 84 walked through a complete cloud breach simulation -- from discovering a misconfigured S3 bucket to full AWS account takeover in 2.5 hours. We saw how each phase generated CloudTrail events, how GuardDuty caught 4 of 7 phases (with 5-15 minute latency), and how specific AWS controls (S3 Block Public Access, IMDSv2, scoped PassRole) could break the attack chain at multiple points. That simulation demonstrated a human attacker working methodically through AWS services.

Today we address a more unsettling question: what happens when those same attack techniques are executed at machine speed by an AI? Throughout this series, the "AI Slop Connection" section in every episode tracked one consistent theme: AI amplifies both attack and defense capabilities. Now, 85 episodes in, we confront the full picture -- and build the practical defenses that survive when the attacker has AI on their side.

The AI Threat Multiplier

Here we go. AI does not create new attack categories. SQL injection (episode 12), phishing (episodes 8, 39), privilege escalation (episodes 31-32), lateral movement (episode 34), cloud exploitation (episodes 35-36, 84) -- these all existed before AI. What AI does is make each attack category faster, cheaper, more convincing, and more scalable. A phishing campaign that took a skilled attacker a week to personalize for 50 targets now takes 5 minutes for 5,000 targets. An OSINT assessment that required an analyst for 3 days (the techniques from episode 47 and 65) now runs in 3 minutes. A vulnerability that took a researcher weeks to discover and exploit can potentially be found and weaponized in hours.

This is the threat multiplier. Same attacks. Dramatically different scale. And this multiplier applies unevenly -- it amplifies the attacker's capabilities MORE than the defender's in most scenarios, because attack is inherently simpler than defense. An attacker needs to find ONE vulnerability. A defender needs to protect against ALL of them. AI helps the attacker find that one vulnerability faster. AI helps the defender... monitor more alerts, most of which are false positives. The asymmetry that we discussed in episode 1 ("Why Hackers Win") gets worse with AI, not better.

Having said that, the situation is not hopeless. The defenses that were fundamentally sound before AI remain fundamentally sound after AI. The defenses that relied on human limitations (like "phishing emails have bad grammar") were never robust in the first place -- they were heuristics that happened to work against low-skill attackers and were already failing against skilled ones. AI just killed the heuristics faster. The architectural defenses survive.

What Still Works

Defenses that remain effective against AI-powered attacks:

1. PHISHING-RESISTANT MFA (FIDO2 / WebAuthn)
   AI cannot phish a hardware key. Period.
   The key verifies the domain cryptographically.
   No amount of AI-generated email convinces the key to authenticate
   to the wrong domain. This is the single most important defense.

2. ZERO TRUST ARCHITECTURE (Episode 81)
   AI can generate better phishing. It cannot bypass Conditional Access
   that requires a managed device + FIDO2 key + known location.
   Identity verification at every access request defeats credential theft.

3. NETWORK SEGMENTATION (Episode 73)
   AI can help an attacker move laterally. It cannot move through a
   firewall that blocks the traffic. Segmentation contains breaches
   regardless of how the attacker got in.

4. PRINCIPLE OF LEAST PRIVILEGE (Episodes 35, 72)
   AI cannot escalate permissions that do not exist.
   If the compromised account has read-only access to one database,
   AI cannot make it admin. Least privilege limits blast radius.

5. PROCESS CONTROLS (Episodes 46, 49)
   "Wire transfers over $10K require verbal confirmation on a known number."
   AI can deepfake a video call. It cannot bypass a physical process
   that requires in-person verification or a shared code word.

6. BEHAVIORAL ANALYTICS (Episodes 48, 74, 75)
   AI-generated attacks still create anomalous BEHAVIOR.
   A compromised account accessing 50 new resources is anomalous
   regardless of whether a human or AI is driving the session.
   UEBA detects the pattern, not the technique.

7. IMMUTABLE LOGGING (Episodes 74, 80)
   CloudTrail logs every API call. SIEM logs every authentication.
   Sysmon logs every process creation. AI cannot erase these logs
   (if they are forwarded to a central, immutable store).
   Detection may lag, but evidence always exists.

The ordering of this list is deliberate -- it goes from "most impactful single control" to "most broadly applicable defense-in-depth layer." FIDO2 MFA is at the top because it eliminates the single most common initial access vector (credential theft via phishing) with a hardware-backed cryptographic guarantee that no AI can bypass. Google reported that deploying FIDO2 keys eliminated 100% of phishing-based account takeovers against their 85,000+ employees. ONE HUNDRED PERCENT. We discussed this in episode 81's zero trust implementation, and it bears repeating because most organizations still use TOTP or SMS-based MFA, which AI-driven phishing kits can intercept in real time using reverse proxy techniques (the evilginx2 approach from episode 17).

The behavioral analytics entry (number 6) is where AI-powered defense actually helps the defender. The UEBA systems from episode 74 and the threat hunting techniques from episode 75 detect anomalous BEHAVIOR, not specific attack signatures. An AI-driven attacker that compromises a user account and starts accessing resources the user has never touched before creates the same behavioral anomaly as a human attacker doing the same thing. The attack technique is different (AI vs human). The detectable footprint is the same (abnormal access pattern). This is why behavioral detection survives the AI threat multiplier while signature-based detection does not ;-)

What No Longer Works

Defenses that AI has rendered ineffective or unreliable:

1. "CHECK FOR SPELLING ERRORS"
   AI phishing has perfect grammar. Always.

2. "THE EMAIL SOUNDS SUSPICIOUS"
   AI mimics any writing style from a few samples.

3. "I RECOGNIZED THE VOICE"
   AI clones voices from 3 seconds of audio.

4. "I SAW THEIR FACE ON VIDEO"
   Real-time deepfake overlays work on live video calls.

5. "SIGNATURE-BASED AV"
   AI generates unique malware per target. No two payloads
   share the same hash, rendering signature databases useless.

6. "SECURITY AWARENESS TRAINING" (alone)
   AI attacks are too convincing for awareness to be the
   primary defense. Training helps but cannot be the ONLY control.

7. "COMPLEXITY-BASED PASSWORD POLICIES"
   AI password crackers learn patterns from breach datasets.
   "P@ssw0rd2026!" matches every complexity rule and falls
   in seconds. Length (20+ char passphrase) still works.
   Complexity alone does not.

I want to be clear about entry number 6: I am NOT saying security awareness training is useless. I am saying it cannot be your PRIMARY defense. Training creates a human sensor network -- employees who report suspicious emails (which we discussed in episode 39's defense section and episode 82's SOC processes). That reporting mechanism remains valuable because even the best AI-generated phishing will occasionally feel "off" to some employees, and their reports trigger investigation. The problem is when organizations treat training as their MAIN phishing defense instead of deploying FIDO2 MFA and email authentication (DMARC, SPF, DKIM). Training catches some percentage of phishing. FIDO2 catches 100% of credential harvesting. Deploy both, but understand which one is the actual barrier and which one is the early warning system.

Defending Against AI-Generated Phishing

AI phishing is personalized, contextual, and grammatically flawless. The content-based filters that worked against Nigerian prince scams (keyword matching, grammar analysis, sender reputation) catch less and less of it. The defense has to move from analyzing the email CONTENT to verifying the email INFRASTRUCTURE and protecting the authentication LAYER:

#!/usr/bin/env python3
"""email_auth_checker.py -- verify email authentication headers"""
import re

def check_email_authentication(headers):
    """Check SPF, DKIM, DMARC results from email headers.
    Returns risk assessment regardless of email content quality."""

    results = {
        'spf': 'missing', 'dkim': 'missing', 'dmarc': 'missing'
    }

    auth_results = headers.get('Authentication-Results', '')

    spf_match = re.search(r'spf=(\w+)', auth_results)
    if spf_match:
        results['spf'] = spf_match.group(1)

    dkim_match = re.search(r'dkim=(\w+)', auth_results)
    if dkim_match:
        results['dkim'] = dkim_match.group(1)

    dmarc_match = re.search(r'dmarc=(\w+)', auth_results)
    if dmarc_match:
        results['dmarc'] = dmarc_match.group(1)

    # Risk scoring based on authentication, NOT content
    risk = 'low'
    if results['dmarc'] != 'pass':
        risk = 'high'
    elif results['spf'] != 'pass' or results['dkim'] != 'pass':
        risk = 'medium'

    # Check for display name spoofing
    from_header = headers.get('From', '')
    reply_to = headers.get('Reply-To', '')
    if reply_to and reply_to != from_header:
        risk = 'high'

    return results, risk

def check_lookalike_domain(sender_domain, org_domain):
    """Detect typosquat / lookalike domains."""
    if sender_domain == org_domain:
        return False, 'exact match'

    # Levenshtein distance = 1 is suspicious
    if len(sender_domain) == len(org_domain):
        diffs = sum(
            1 for a, b in zip(sender_domain, org_domain)
            if a != b
        )
        if diffs == 1:
            return True, f'one char differs: {sender_domain}'

    # Common substitutions
    substitutions = [
        ('l', '1'), ('o', '0'), ('i', '1'),
        ('rn', 'm'), ('vv', 'w'),
    ]
    normalized = sender_domain
    for old, new in substitutions:
        normalized = normalized.replace(old, new)
    if normalized == org_domain:
        return True, f'visual lookalike: {sender_domain}'

    return False, 'no match'

The display name spoofing check (comparing From and Reply-To headers) catches a surprisingly common AI phishing tactic. The AI generates a perfectly written email that appears to come from "CEO John Smith [email protected]" but sets the Reply-To header to a completely different domain. When the target hits reply, their response goes to the attacker, not the CEO. Many email clients show only the display name (not the full email address), which makes this technique invisible to the recipient. The email content is flawless -- it is the infrastructure that reveals the deception.

The lookalike domain detection is critical because AI-powered phishing campaigns register domains at scale. An AI can generate hundreds of typosquat variations of your domain (cornpany.com, company-inc.com, c0mpany.com) and register the ones that are available, all within minutes. Monitoring Certificate Transparency logs for certificates issued to domains that are visually similar to yours is one of the most effective proactive defenses -- you can discover the phishing infrastructure before the campaign even starts. Tools like dnstwist and urlcrazy automate this enumeration from the defender's side.

Defending Against Deepfakes

When voice and video are no longer proof of identity, the defense must shift from "verify the medium" to "verify through out-of-band channels." This connects directly to the social engineering defenses from episode 46 and the deepfake analysis from episode 49:

Deepfake-resistant verification protocol:

1. CODE WORDS (challenge-response)
   Establish shared secrets with key stakeholders.
   Rotate quarterly. Distribute in person only.
   "What is our current code word?" defeats any deepfake.

2. CALLBACK VERIFICATION on KNOWN numbers
   Receive a call from the "CEO"? Hang up. Call back on the
   number saved in your contacts (not the number that just
   called you -- caller ID is trivially spoofed).

3. MULTI-CHANNEL CONFIRMATION
   Request via phone -> verify via Slack.
   Request via email -> verify via phone call.
   NEVER confirm via the same channel the request came from.

4. MULTI-PERSON AUTHORIZATION for high-value actions
   No single person can authorize: wire transfers >$X,
   access to production databases, changes to critical systems.
   Even if one person is deepfaked, the second verifier catches it.

5. IN-PERSON VERIFICATION for the highest-value actions
   >$1M transfer? Both signers meet in person.
   No deepfake defeats a face-to-face meeting.

The multi-channel confirmation principle is the most broadly applicable of these controls. A deepfake attack works because it operates within a single channel -- the attacker controls the voice call or the video feed. When you shift verification to a channel the attacker does NOT control (Slack, SMS, a physical meeting), the deepfake collapses because the attacker cannot maintain the impersonation across two independent channels simultaneously. This is the same principle behind out-of-band authentication in MFA -- the second factor works because it uses a different channel than the first.

I argue that multi-person authorization is the most underrated defense in this entire list. Most organizations have single points of failure for critical actions -- one CFO approves wire transfers, one DBA has production access, one admin controls the Active Directory. A deepfake that targets the single authorized person succeeds completely. Multi-person authorization means the attacker must deepfake TWO people simultaneously and both of them must fail to notice. The probability of success drops dramatically. This is not new security thinking -- banks have required dual signatures on large checks for centuries. The same principle applies to digital processes.

Defending Against AI-Powered Reconnaissance

AI processes OSINT at machine speed. The manual techniques from episode 47 (physical OSINT) and episode 65 (automated OSINT) that took hours now take minutes when an attacker feeds your organization's public data into an AI model. The defense is twofold: minimize what you expose, and detect when someone is harvesting it:

#!/usr/bin/env python3
"""honeytoken_monitor.py -- detect AI-powered recon with decoys"""
import json
import hashlib
from datetime import datetime

class HoneytokenDeployer:
    def __init__(self, alert_callback):
        self.tokens = {}
        self.alert = alert_callback

    def create_fake_credential(self, context):
        """Deploy a honeytoken credential that alerts on use."""
        token_id = hashlib.sha256(
            context.encode()
        ).hexdigest()[:12]

        fake_cred = {
            'type': 'aws_key',
            'access_key': f'AKIA{"X" * 16}{token_id[:4].upper()}',
            'context': context,
            'deployed': datetime.utcnow().isoformat(),
        }
        self.tokens[token_id] = fake_cred
        return fake_cred

    def create_fake_subdomain(self, base_domain):
        """Create a DNS record that alerts when resolved."""
        honeypot_names = [
            f'staging-api.{base_domain}',
            f'dev-portal.{base_domain}',
            f'legacy-admin.{base_domain}',
        ]
        return honeypot_names

    def create_canary_document(self, doc_name):
        """Create a document with an embedded tracking pixel."""
        tracking_url = (
            f'https://canarytokens.com/t/'
            f'{hashlib.md5(doc_name.encode()).hexdigest()[:16]}'
        )
        return {
            'name': doc_name,
            'tracking_url': tracking_url,
            'alert_on': 'document opened or image loaded',
        }

    def process_alert(self, token_id, source_ip, timestamp):
        """A honeytoken was triggered -- someone found our decoy."""
        token = self.tokens.get(token_id)
        if not token:
            return

        self.alert({
            'event': 'honeytoken_triggered',
            'token_context': token['context'],
            'source_ip': source_ip,
            'time': timestamp,
            'deployed_at': token['deployed'],
            'severity': 'high',
            'action': 'investigate source IP immediately',
        })

The honeytoken concept is particuarly powerful against AI-powered recon because AI tools are indiscriminate -- they scrape and process EVERYTHING they find. A human attacker might look at a .env file in a public repo and think "this looks too convenient, could be a trap." An AI recon tool downloads it, extracts the credentials, and tries to use them without hesitation. When the fake AWS key is used (even just for sts:GetCallerIdentity), AWS CloudTrail logs the attempt and your monitoring fires. The AI just told you someone is targeting your organization, and they told you from which IP address.

The fake subdomain technique works against automated subdomain enumeration (the recon techniques from episode 4). Tools like subfinder, amass, and gobuster will discover staging-api.yourdomain.com alongside your real subdomains. When someone connects to the honeypot subdomain, you know they are actively enumerating your attack surface. The cost of deploying this is near-zero (a DNS A record pointing to a monitoring server), and the intelligence value is significant -- you learn that you are being targeted BEFORE the actual attack begins.

Defending Against AI-Assisted Exploitation

AI helps attackers find and exploit vulnerabilities faster. The window between vulnerability disclosure and active exploitation is shrinking from weeks to days to hours. Your defense must match that pace:

Closing the AI exploitation window:

1. PATCH FASTER THAN AI CAN EXPLOIT
   Zero-day: nothing you can do except layered defense.
   N-day: patch within 24-48 hours of disclosure.
   AI reduces the window between disclosure and weaponization.
   You must reduce the window between disclosure and patching.

2. CONTINUOUS VULNERABILITY SCANNING (Episode 80)
   Automated scanning + automated ticketing + SLA enforcement.
   AI attackers scan YOUR attack surface. You should scan it first.

3. DEVSECOPS (Episode 67)
   Fix vulnerabilities in code BEFORE they reach production.
   AI generates more vulnerable code (episode 6).
   SAST/DAST in CI/CD catches it before deployment.

4. BEHAVIORAL WAF (Episode 25)
   Signature-based WAF misses AI-generated payloads.
   Behavioral WAF catches: unusual parameter lengths,
   unexpected characters, injection pattern behavior --
   regardless of the specific payload syntax.

5. RUNTIME APPLICATION SELF-PROTECTION (RASP)
   Instrumentation inside the application that detects and
   blocks exploitation at runtime. Cannot be bypassed by
   changing payload syntax -- it monitors application behavior,
   not input patterns.

The behavioral WAF distinction is critical and connects back to episode 25 where we discussed WAF bypassing techniques. A signature-based WAF maintains a list of known-bad patterns (<script>, UNION SELECT, ../../../etc/passwd) and blocks requests that match. AI can generate novel payloads that achieve the same effect without matching any known pattern -- polyglot payloads, encoding tricks, context-specific injections that signature databases have never seen. A behavioral WAF does not care about the specific payload -- it measures the REQUEST behavior. A form field that normally contains a 20-character name suddenly receiving a 500-character input with semicolons, angle brackets, and null bytes is anomalous regardless of whether the specific string matches a known attack signature. Behavior detection survives AI-generated payloads because AI changes the SYNTAX but not the SEMANTICS of attacks.

The AI Defense Stack

Building a defense program that survives AI-powered attacks requires layers at every level. This is the defense-in-depth architecture from episode 53, updated for the AI threat landscape:

Building a defense program that survives AI-powered attacks:

Identity layer:
  FIDO2 MFA + Conditional Access + Zero Trust
  -> defeats credential theft and phishing

Endpoint layer:
  EDR with behavioral detection + application control
  -> defeats AI-generated malware (behavioral, not signature)

Network layer:
  Segmentation + IDS/IPS + DNS filtering + TLS inspection
  -> contains lateral movement and detects C2 traffic

Application layer:
  DevSecOps + SAST/DAST + behavioral WAF + RASP
  -> prevents and detects application-layer attacks

Data layer:
  Encryption + DLP + classification + access logging
  -> protects data even when other layers fail

Process layer:
  Code words + multi-person authorization + callback verification
  -> defeats deepfakes and AI-driven social engineering

Monitoring layer:
  SIEM + UEBA + threat hunting + automated response (SOAR)
  -> detects what prevention misses

Human layer:
  Security champions + continuous training + reporting culture
  -> humans remain the last line of detection for novel attacks

The critical observation is that every layer uses behavioral detection rather than signature detection where possible. Signatures are static -- they match known-bad patterns. Behaviors are dynamic -- they detect anomalous activity regardless of the specific technique used. AI can generate novel attack signatures infinitely. AI cannot generate novel attack behaviors, because the BEHAVIOR is determined by the attacker's OBJECTIVE (steal data, escalate privilege, establish persistence), not by the specific tool or payload used to achieve it. This is why behavioral approaches (UEBA in the monitoring layer, behavioral WAF in the application layer, EDR behavioral detection in the endpoint layer) are the foundation of AI-resilient defense.

Building AI-Resilient Detection Rules

The practical implementation of behavioral detection comes down to writing detection rules that focus on WHAT happened rather than HOW it happened:

#!/usr/bin/env python3
"""ai_resilient_detection.py -- behavioral detection rules"""

def detect_credential_spray(auth_events, window_seconds=300):
    """Detect credential spraying regardless of source.
    AI sprays faster and from more IPs -- detect the PATTERN."""

    # Group by target account
    targets = {}
    for event in auth_events:
        user = event['target_user']
        targets.setdefault(user, []).append(event)

    # Spray pattern: many accounts, few attempts each
    failed_accounts = [
        user for user, events in targets.items()
        if any(e['result'] == 'failure' for e in events)
    ]

    if len(failed_accounts) > 10:
        success_during_spray = [
            user for user, events in targets.items()
            if any(e['result'] == 'success' for e in events)
            and any(e['result'] == 'failure' for e in events)
        ]
        return {
            'alert': 'credential_spray_detected',
            'accounts_targeted': len(failed_accounts),
            'successful_logins_during_spray': success_during_spray,
            'severity': 'critical' if success_during_spray else 'high',
        }
    return None

def detect_impossible_travel(login_events):
    """Detect logins from locations that are physically impossible
    to travel between in the time elapsed."""
    from math import radians, sin, cos, sqrt, atan2

    def haversine(lat1, lon1, lat2, lon2):
        R = 6371  # Earth radius in km
        dlat = radians(lat2 - lat1)
        dlon = radians(lon2 - lon1)
        a = (sin(dlat/2)**2 +
             cos(radians(lat1)) * cos(radians(lat2)) *
             sin(dlon/2)**2)
        return R * 2 * atan2(sqrt(a), sqrt(1-a))

    sorted_events = sorted(login_events, key=lambda e: e['time'])

    for i in range(1, len(sorted_events)):
        prev = sorted_events[i-1]
        curr = sorted_events[i]

        time_diff_hours = (
            curr['time'] - prev['time']
        ).total_seconds() / 3600
        distance_km = haversine(
            prev['lat'], prev['lon'],
            curr['lat'], curr['lon']
        )

        # Max travel speed: 900 km/h (commercial aviation)
        max_distance = time_diff_hours * 900
        if distance_km > max_distance and distance_km > 100:
            return {
                'alert': 'impossible_travel',
                'from': prev['location'],
                'to': curr['location'],
                'distance_km': round(distance_km),
                'time_hours': round(time_diff_hours, 2),
                'severity': 'high',
            }
    return None

The detect_credential_spray function demonstrates behavioral detection at its most fundamental. A credential spray attack tries a small number of passwords against MANY accounts (to avoid lockout thresholds), and this behavioral pattern is the same whether a human is typing passwords manually, a script is iterating through a list, or an AI is intelligently selecting passwords based on the organization's naming conventions and breach data. The detection does not care about the SOURCE -- it could be one IP, ten IPs, or a thousand IPs (AI-driven sprays often rotate through proxy networks). It cares about the PATTERN: many accounts receiving failed login attempts within a short window. And critically, it escalates to critical severity if any account shows a successful login during the spray window, because that means the spray WORKED and the attacker now has valid credentials.

The detect_impossible_travel function is a classic UEBA rule that we first mentioned in episode 74. If a user logs in from Amsterdam at 10:00 and then from Tokyo at 10:30, someone is using stolen credentials (unless the user has access to a very fast aircraft). This detection is completely immune to AI -- no AI can change the physical laws of travel time. The 900 km/h threshold accounts for commercial aviation, and the 100 km minimum distance avoids false positives from VPN exit nodes in the same metropolitan area.

The AI Slop Connection -- Full Circle

Episode 6 introduced the problem: AI generates insecure code at industrial scale. Episode 85 closes the loop. AI generates insecure code, insecure configurations, and insecure architectures -- AND AI powers the attacks that exploit those insecurities. The attacker's AI finds the bugs that the developer's AI introduced. It is a closed loop of AI-generated insecurity.

The organizations that survive this era are the ones that use AI as a tool while understanding its limitations. AI enriches alerts -- humans investigate. AI suggests configurations -- humans review. AI generates code -- humans verify. AI detects anomalies -- humans decide. The human in the loop is not a bottleneck. The human in the loop is the competitive advantage. AI is the engine. Human judgment is the steering wheel. Both are necessary. Neither is sufficient alone.

The organizations that fail are the ones that trust AI blindly on both sides: AI generates their infrastructure (insecure by default) AND AI monitors their security (incomplete by nature). Both the offense and the defense are AI, and the attacker's AI is optimizing for exploitation while the defender's AI is optimizing for convenience. The attacker's objective function is clearer, which gives it an inherent advantage in the optimization game.

This principle extends beyond enterprise security into every domain where safety depends on system integrity -- healthcare systems that keep people alive, power grids that keep cities running, water treatment that keeps communities healthy. The defenses we have built throughout this series apply doubly in environments where the consequences of failure are measured not in dollars but in human lives. Those environments cannot afford to trust AI blindly on either side of the equation.

Exercises

Exercise 1: Evaluate your lab environment against the "What Still Works" list from this episode. For each of the 7 controls: (a) is it implemented in your lab? (b) if not, what would it take to implement? (c) prioritize them by impact. Implement the top 3 and verify they resist the attacks from episodes 83-84. Document what you implemented and how it changes the attack chain outcome.

Exercise 2: Design a deepfake-resistant communication protocol for your organization (real or fictional). Define: (a) code word rotation schedule and distribution method, (b) callback verification procedure for financial transactions, (c) multi-channel confirmation matrix (which channels verify which), (d) escalation thresholds for in-person verification, (e) how to handle the scenario where an employee DOES fall for a deepfake mid-transaction. Present as a 1-page policy document that a non-technical employee could follow.

Exercise 3: Build a honeypot strategy to detect AI-powered reconnaissance against your lab. Deploy: (a) a canary token in a public-facing document (use canarytokens.org), (b) a fake subdomain (honeypot.yourdomain.com or equivalent) that logs all connections with timestamp and source IP, (c) a decoy credential in a deliberately "leaked" .env file on a test GitHub repo (use a PRIVATE repo with a public gist, or a honeytoken AWS key from canarytokens.org). Monitor for 7 days and document: what was accessed, from where, how quickly, and what you learned about who is scanning your attack surface.

Thanks for reading!

@scipio

stem stemsocial steemstem security programming

0.000

0 comments

Learn Ethical Hacking (#85) - Defending Against AI-Powered Attacks - A Practical Guide

Learn Ethical Hacking (#85) - Defending Against AI-Powered Attacks - A Practical Guide

What will I learn

Requirements

Difficulty

Curriculum (of the Learn Ethical Hacking Series):

Learn Ethical Hacking (#85) - Defending Against AI-Powered Attacks - A Practical Guide

Solutions to Episode 84 Exercises

The AI Threat Multiplier

What Still Works

What No Longer Works

Defending Against AI-Generated Phishing

Defending Against Deepfakes

Defending Against AI-Powered Reconnaissance

Defending Against AI-Assisted Exploitation

The AI Defense Stack

Building AI-Resilient Detection Rules

The AI Slop Connection -- Full Circle

Exercises

Thanks for reading!

Curriculum (of the `Learn Ethical Hacking Series`):