SkillAgentSearch skills...

vm-and-bytecode-reverse

Custom VM and bytecode reverse engineering playbook

Install / Use

npx skills add yaklang/hack-skills --skill vm-and-bytecode-reverse

Installs into whichever agent you are using.

About this skill
📄

SKILL.md

Installable skill definition

Quality Score

88/100

Supported Platforms

Universal

Tags

Our assessment of vm-and-bytecode-reverse

vm-and-bytecode-reverse scores 88/100 on our quality scale, 582nd of 1,155 Content & Media skills we index.

Its SKILL.md is 14 KB long, well organised into 35 sections with 11 code examples: a thorough specification that gives an agent plenty to work with.

With 2,288 GitHub stars, it is one of the more widely adopted skills in the catalogue.

Substance
30/30
Structure
20/20
Description
8/15
Adoption
14/20
Freshness
15/15

Maintenance, license and trust

  • The repository was last updated about 2 months ago, so vm-and-bytecode-reverse is actively maintained.
  • Our last check on 2026-11-01 found the source still online.
  • It is released under the MIT license, a permissive license that allows use, modification and commercial use with attribution.
  • Its trust signals score 100/100, with no cautions. These come from repository metadata, not a code audit — read the skill file before letting an agent act on it.

Safety scan

No issues found

Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful.

AI review by kimi-k2.7-code on 2026-10-02. Automated pattern scan on 2026-10-02. It catches known dangerous patterns, not every risk — read a skill before letting an agent act on it.

vm-and-bytecode-reverse compared with similar skills

All 4 of these similar skills score higher than vm-and-bytecode-reverse; compare them before choosing.

SkillScoreStarsUpdatedFormat
vm-and-bytecode-reverse (this skill)by yaklang882.3k2mo agoSKILL.md
siyuanby siyuan-note10047.3ktodayMCP Server
x402by browser-use100121.0k3d agoSKILL.md
algorithmic-artby anthropics100177.9k55d agoSKILL.md
pptxby anthropics100177.9k55d agoSKILL.md

Frequently asked questions

How do I install vm-and-bytecode-reverse?
Run npx skills add yaklang/hack-skills --skill vm-and-bytecode-reverse. The install tabs above show the steps for each supported agent.
Which AI agents does vm-and-bytecode-reverse work with?
It is written for Universal, as a SKILL.md file. Other agents that read the same format can often use it too.
Is vm-and-bytecode-reverse safe to use?
Our scan of the whole file found no instruction hijacking, hidden characters, credential access, data exfiltration or destructive commands. An AI review of the same text found nothing harmful. It is MIT-licensed and scores 100/100 on trust signals. Skills are instructions an agent will follow, so read the file before installing it and do not approve commands you do not understand.
Is vm-and-bytecode-reverse still maintained?
The repository was last updated about 2 months ago, so vm-and-bytecode-reverse is actively maintained.

name: vm-and-bytecode-reverse description: >- Custom VM and bytecode reverse engineering playbook. Use when CTF challenges or protected software implement custom virtual machines with proprietary bytecode, dispatcher loops, or maze-style challenges.

SKILL: VM & Bytecode Reverse Engineering — Expert Analysis Playbook

AI LOAD INSTRUCTION: Expert techniques for reversing custom virtual machines and bytecode interpreters. Covers dispatcher identification, opcode mapping, custom ISA reconstruction, disassembler/decompiler writing, maze challenges, and real-world VM protector analysis. Base models often fail to recognize the fetch-decode-execute pattern or attempt to analyze VM bytecode as native code.

0. RELATED ROUTING

Quick identification

| Binary Pattern | Likely VM Type | Start With | |---|---|---| | while(1) { switch(bytecode[pc]) } | Switch-based dispatcher | Map each case to an operation | | Indirect jump via table jmp [table + opcode*8] | Table-based dispatcher | Dump jump table, analyze handlers | | Nested if-else chain on byte value | If-chain dispatcher | Same as switch, just different syntax | | Stack push/pop dominant operations | Stack-based VM | Identify push, pop, arithmetic ops | | reg[X] = ... array operations | Register-based VM | Map register indices to operations | | 2D grid + direction input | Maze challenge | Extract grid, apply BFS/DFS |


1. CUSTOM VM IDENTIFICATION

1.1 Structural Indicators

VM Architecture Components:
┌─────────────────────────────────┐
│  Bytecode Program (data section)│
├─────────────────────────────────┤
│  Program Counter (pc/ip)        │
│  Register File / Stack          │
│  Memory / Data Area             │
├─────────────────────────────────┤
│  Dispatcher Loop                │
│  ├─ Fetch: opcode = code[pc]    │
│  ├─ Decode: lookup handler      │
│  └─ Execute: run handler        │
└─────────────────────────────────┘

1.2 IDA/Ghidra Signatures

Switch dispatcher (most common in CTF):

while (running) {
    unsigned char op = bytecode[pc++];
    switch (op) {
        case 0x00: /* nop */       break;
        case 0x01: /* push imm */  stack[sp++] = bytecode[pc++]; break;
        case 0x02: /* add */       stack[sp-2] += stack[sp-1]; sp--; break;
        // ...
        case 0xFF: /* halt */      running = 0; break;
    }
}

Table dispatcher (more optimized):

typedef void (*handler_t)(vm_ctx_t*);
handler_t handlers[256] = { handle_nop, handle_push, handle_add, ... };

while (running) {
    handlers[bytecode[pc++]](&ctx);
}

2. ANALYSIS METHODOLOGY

Step 1: Find the Dispatcher

Look for:

  • Large switch statement (many cases) in a loop
  • Array of function pointers indexed by a byte from a data buffer
  • Single function with high cyclomatic complexity
  • Cross-references to a data buffer read byte-by-byte

Step 2: Map Opcodes to Operations

For each case/handler, determine:

| Property | How to Identify | |---|---| | Opcode value | Case number or table index | | Operation type | Register/stack modifications | | Operand count | How many bytes consumed after opcode | | Operand type | Immediate value, register index, or memory address | | Side effects | Output, memory write, flag modification |

Step 3: Extract Bytecode Program

# Typical extraction from binary
import struct

with open('challenge', 'rb') as f:
    f.seek(bytecode_offset)
    bytecode = f.read(bytecode_length)

# Or from IDA:
# bytecode = idc.get_bytes(bytecode_addr, bytecode_len)

Step 4: Write Custom Disassembler

OPCODES = {
    0x00: ("nop",  0),    # (mnemonic, operand_bytes)
    0x01: ("push", 1),    # push immediate byte
    0x02: ("pop",  0),
    0x03: ("add",  0),
    0x04: ("sub",  0),
    0x05: ("xor",  0),
    0x06: ("cmp",  0),
    0x07: ("jmp",  2),    # jump to 16-bit address
    0x08: ("je",   2),
    0x09: ("jne",  2),
    0x0A: ("mov",  2),    # mov reg, imm
    0x0B: ("load", 1),    # load from memory[operand]
    0x0C: ("store",1),    # store to memory[operand]
    0x0D: ("print",0),
    0x0E: ("read", 0),    # read input
    0xFF: ("halt", 0),
}

def disassemble(bytecode):
    pc = 0
    while pc < len(bytecode):
        op = bytecode[pc]
        if op not in OPCODES:
            print(f"  {pc:04x}: UNKNOWN {op:#04x}")
            pc += 1
            continue

        mnemonic, operand_size = OPCODES[op]
        operands = bytecode[pc+1:pc+1+operand_size]
        operand_str = ' '.join(f'{b:#04x}' for b in operands)
        print(f"  {pc:04x}: {mnemonic:8s} {operand_str}")
        pc += 1 + operand_size

disassemble(bytecode)

Step 5: Analyze Disassembled Program

With the custom disassembly, apply standard reverse engineering:

  • Identify input reading (read opcode)
  • Trace data flow from input to comparison
  • Determine success/failure conditions
  • Extract the check logic (often XOR/ADD transformations of input compared against constants)

3. COMMON VM PATTERNS IN CTF

3.1 Stack-Based VM

Operations work on a stack (like JVM or Python bytecode).

| Opcode | Operation | Stack Effect | |---|---|---| | PUSH imm | Push immediate value | [...] → [..., imm] | | POP | Discard top | [..., a] → [...] | | ADD | Add top two | [..., a, b] → [..., a+b] | | SUB | Subtract | [..., a, b] → [..., a-b] | | MUL | Multiply | [..., a, b] → [..., a*b] | | XOR | Bitwise XOR | [..., a, b] → [..., a^b] | | CMP | Compare | [..., a, b] → [..., (a==b)] | | JMP addr | Unconditional jump | no change | | JZ addr | Jump if top is zero | [..., a] → [...] | | PRINT | Output top as char | [..., a] → [...] | | READ | Read char to stack | [...] → [..., input] | | HALT | Stop execution | - |

3.2 Register-Based VM

Operations use register indices (like x86, ARM).

| Opcode | Format | Operation | |---|---|---| | MOV r, imm | 0x01 RR II II | reg[R] = imm16 | | MOV r1, r2 | 0x02 R1 R2 | reg[R1] = reg[R2] | | ADD r1, r2 | 0x03 R1 R2 | reg[R1] += reg[R2] | | SUB r1, r2 | 0x04 R1 R2 | reg[R1] -= reg[R2] | | XOR r1, r2 | 0x05 R1 R2 | reg[R1] ^= reg[R2] | | CMP r1, r2 | 0x06 R1 R2 | flags = compare(r1, r2) | | JMP addr | 0x07 AA AA | pc = addr | | JE addr | 0x08 AA AA | if equal: pc = addr | | LOAD r, [addr] | 0x09 RR AA | reg[R] = mem[addr] | | STORE [addr], r | 0x0A AA RR | mem[addr] = reg[R] | | SYSCALL | 0x0B | I/O operation based on reg[0] | | HALT | 0xFF | stop |

3.3 Brainfuck-like / Esoteric VMs

| BF Command | VM Equivalent | Description | |---|---|---| | > | INC ptr | Move data pointer right | | < | DEC ptr | Move data pointer left | | + | INC [ptr] | Increment byte at pointer | | - | DEC [ptr] | Decrement byte at pointer | | . | OUTPUT [ptr] | Output byte at pointer | | , | INPUT [ptr] | Input byte to pointer | | [ | JZ forward | Jump past ] if byte is zero | | ] | JNZ back | Jump back to [ if byte is nonzero |


4. MAZE CHALLENGES

4.1 Identification

  • Binary reads directional input (WASD, arrow keys, UDLR)
  • 2D array in data section (walls, paths, start, end)
  • Position tracking with x,y coordinates
  • Win condition at specific coordinates

4.2 Map Extraction

# Extract maze grid from binary data section
MAZE_ADDR = 0x601060
WIDTH = 20
HEIGHT = 15

# From binary dump:
maze = []
for row in range(HEIGHT):
    line = ""
    for col in range(WIDTH):
        cell = bytecode[MAZE_ADDR + row * WIDTH + col - base_addr]
        if cell == 0: line += "."    # path
        elif cell == 1: line += "#"  # wall
        elif cell == 2: line += "S"  # start
        elif cell == 3: line += "E"  # end
        else: line += "?"
    maze.append(line)
    print(line)

4.3 Automated Solving

from collections import deque

def solve_maze(maze, start, end):
    """BFS solver returns direction string."""
    rows, cols = len(maze), len(maze[0])
    directions = {'U': (-1, 0), 'D': (1, 0), 'L': (0, -1), 'R': (0, 1)}
    queue = deque([(start, "")])
    visited = {start}

    while queue:
        (r, c), path = queue.popleft()
        if (r, c) == end:
            return path

        for name, (dr, dc) in directions.items():
            nr, nc = r + dr, c + dc
            if (0 <= nr < rows and 0 <= nc < cols and
                maze[nr][nc] != '#' and (nr, nc) not in visited):
                visited.add((nr, nc))
                queue.append(((nr, nc), path + name))

    return None

# Find start and end positions
for r, row in enumerate(maze):
    for c, cell in enumerate(row):
        if cell == 'S': start = (r, c)
        if cell == 'E': end = (r, c)

solution = solve_maze(maze, start, end)
print(f"Path: {solution}")

4.4 Direction Encoding

Different challenges encode directions differently:

| Encoding | Up | Down | Left | Right | |---|---|---|---|---| | WASD | W | S | A | D | | UDLR | U | D | L | R | | Arrow keys | ↑ (0x48) | ↓ (0x50) | ← (0x4B) | → (0x4D) | | Numbers | 1 | 2 | 3 | 4 | | Hex opcodes | 0x01 | 0x02 | 0x03 | 0x04 |


5. REAL-WORLD VM PROTECTORS

5.1 VMProtect Analysis Approach

1. Find VM entry: search for pushad/pushfd sequence
2. Identify VM context structure (registers, flags, bytecode pointer)
3. Locate handler table (often obfuscated with opaque predicates)
4. For each handler:
   a. Remove junk code / opaque predicates
   b. Identify the core operation
   c. Document handler semantics
5. Trace bytecode execution (instruction-level trace)
6. Reconstruct original code from trace

5.2 Tigress Obfuscator

Academic VM obfuscator with configurable protection layers.

| Feature | Approach | |---|---| | Single-dispatch VM | Standard handler extraction | | Split handlers | Handlers spread across multiple functions | | Nested VMs | Outer VM handler invokes inner VM | | Encrypted bytecode | Dynamic decryption before each fetch | | Polymorphic handlers | Different code for same operation on each build |

5.3 Common VM Protector Patterns

| Protector | Dispatcher Style | Difficulty | |---|---|---| | VMProtect | Table + opaque predicates | High | | Themida (Code Virtualizer) | CISC-like, large handler set | High | | Tigress | Configurable, academic | Medium-High | | Custom CTF VM | Simple switch | Low-Medium | | Movfuscator | All-mov computation | Medium |


6. TOOLS

| Tool | Purpose | Usage | |---|---|---| | IDA Pro | Identify dispatcher, reverse handlers | F5 decompile, xref analysis | | Ghidra | Free alternative with Sleigh processor modules | Write custom processor for VM ISA | | angr | Symbolic execution through VM | Treat entire VM as constraint system | | Pin / DynamoRIO | Dynamic instrumentation for tracing | Record opcode handler execution sequence | | REVEN | Full-system trace recording | Replay and analyze VM execution | | Unicorn | Emulate VM execution | Fast handler emulation | | Miasm | IR-based analysis | Lift VM handlers to IR for analysis | | Custom Python | Write disassembler/decompiler | Per-challenge custom tooling |

Ghidra Sleigh Processor Module

For recurring VM architectures, write a Sleigh processor specification:

define space ram      type=ram_space      size=2  default;
define space register type=register_space  size=1;

define register offset=0 size=1 [ R0 R1 R2 R3 FLAGS PC SP ];

define token opcode(8)
    op = (0,7)
;

:NOP    is op=0x00 { }
:PUSH   imm is op=0x01; imm { SP = SP - 1; *[ram]:1 SP = imm; }
:POP    is op=0x02 { SP = SP + 1; }
:ADD    is op=0x03 { local a = *[ram]:1 (SP+1); *[ram]:1 (SP+1) = a + *[ram]:1 SP; SP = SP + 1; }

7. DECISION TREE

Binary contains custom bytecode interpreter

Truncated for display — read the full file on GitHub.

Related Skills

View on GitHub
GitHub Stars2.3k
CategoryContent
Updated2mo ago
Forks303

Languages

CSS

Trust signals

100/100

From repository metadata: license, adoption, age and documentation. Not a code audit — see the Safety scan above for what the skill file itself contains.

No cautions