PAYLOAD OBFUSCATION TECHNIQUES IN 2026
Modern AV evasion strategies - from base64 encoding to polymorphic shellcode. What works, what doesn't, and why.
Obfuscation is a defender's problem as much as an attacker's. Understanding how payloads hide is what lets a blue team write the detections that catch them. This post looks at the landscape from a research angle: what modern evasion actually relies on, and why most of it fails against behavioural analysis.
A taxonomy of evasion
It helps to separate the layers, because each is defeated differently:
- Encoding (base64, hex) is not obfuscation - it is transport. It stops a naive string match and nothing else.
- Encryption at rest hides the payload until runtime, but the decryption stub itself becomes the signature.
- Polymorphism rewrites the stub every build so no static byte pattern is stable.
- Behavioural evasion (sleep, sandbox checks, indirect syscalls) targets the analysis environment, not the scanner.
The industry lesson of the last few years is blunt: static signatures lose, behaviour wins.
The classic example, and why it is detectable
The canonical trick is XOR-decoding a blob at runtime. It is worth showing precisely because it is so easy to catch - a tight decode loop over a high-entropy buffer is itself a signal.
/* Illustrative only: a textbook XOR decode stub.
Shown to explain what detections look for, not to ship. */
void decode(unsigned char *buf, size_t len, unsigned char key) {
for (size_t i = 0; i < len; i++)
buf[i] ^= key; /* single-byte key = trivial entropy signature */
}
A single-byte key leaves the plaintext's byte-frequency shape intact, so entropy and chi-square tests flag it immediately. Rolling or multi-byte keys raise the bar for static tools, but the runtime behaviour - allocate RWX memory, decode, jump into it - is unchanged, and that is what EDR hooks.
Where this is going
The takeaway for defenders is to stop chasing byte patterns and instrument behaviour: RWX allocations, unbacked executable memory, and syscall provenance.
Detailed write-up coming soon.