EpochZero Learn
EpochZero LearnMulti-Domain Tech Learning Hub
Courses
LeaderboardAbout
Dashboard
EpochZero
EpochZero Learn
Multi-Domain Tech Learning Hub

Structured learning for Reverse Engineering, Cloud Security, Cryptography, and Web Development. Articles, videos, tests, and peer discussion.

Learn

  • Learning Path
  • All Articles
  • Video Lessons
  • Podcast
  • eBooks & PDFs
  • Question Banks
  • Cheatsheets
  • MCQ Banks

Tests & Forum

  • All Tests
  • REMA Tests
  • Cloud Tests
  • Forum
  • REMA Forum
  • Cloud Forum
  • Crypto Forum
  • Web Dev Forum

Campus

  • REMA Club
  • Full Stack Dev Club
  • Extension Activity
  • Events
  • CTF Competitions
  • Workshops
  • Industrial Visits

Platform

  • Dashboard
  • Leaderboard
  • About
  • Verify Certificate

© 2026 EpochZero Learn. Educational content for learning purposes.

Course Instructor: Ashish Revar

All articles
Educationx64calling-conventionrip-relative

Extending to x64 Analysis

The key differences between x86 and x64 that matter for reverse engineering — sixteen registers, the Microsoft x64 ABI calling convention, shadow space, RIP-relative addressing, and reading API calls in x64dbg.

Ashish Revar12 May 202616 min read2 views

Sign in to mark this article as read and track your progress.

Related to this topic

Continue learning

Listen

The Malicious Process Lifecycle: From Execution to Evasion

How malware executes, establishes persistence, injects into legitimate processes, and evades detection inside modern operating systems.

Reference material

eBook
REMA eBook 2026
v1.0
Open resource
Cheatsheet
REMA Cheatsheet 2026
v1.0
Open resource
MCQ Bank
REMA MCQ Bank 2026
v1.0
Open resource
Question Bank
REMA Question Bank 2026
v1.0
Open resource

External references

Microsoft x64 ABI Calling Convention — Official Docs

Official Microsoft documentation for the x64 ABI including register usage, shadow space, and stack alignment requirements.

reference
Compiler Explorer — x64 Output

Select x86-64 gcc or MSVC compiler. Write a function with 5+ arguments and observe the x64 calling convention — register args vs stack args.

tool
REMA eBook 2026 — Chapter 3, x64 Analysis

Full x64 register layout diagrams and calling convention tables in the REMA eBook Chapter 3.

ebook
More articlesTest your knowledge

Why x64 Now?

The 64-bit x86-64 (AMD64) architecture extends the 32-bit x86 instruction set. As more malware targets 64-bit Windows exclusively, analysts must read x64 disassembly as fluently as x86.

Learning x86 first gives you the foundation. x64 differences are incremental — the core concepts (registers, flags, stack frames, calling conventions) are identical. Only the specifics change.

Key Differences at a Glance

Featurex86 (32-bit)x64 (64-bit)
General-purpose registers8 (EAX–EDI)16 (RAX–RDI extended + R8–R15)
Register size32 bits (4 bytes)64 bits (8 bytes)
Instruction pointerEIPRIP
Calling convention (Windows)Arguments pushed on stack right-to-leftFirst four args in RCX, RDX, R8, R9
Return valueEAXRAX
Address space4 GB (2³²)256 TB (2⁴⁸, current implementations)
Stack alignment4-byte before CALL16-byte before CALL
Data referenceAbsolute addressesRIP-relative offsets

The Sixteen Registers

x64 extends all eight x86 registers to 64 bits and adds eight new registers:

Extended:   RAX  RBX  RCX  RDX  RSP  RBP  RSI  RDI
New:        R8   R9   R10  R11  R12  R13  R14  R15

Each 64-bit register still provides access to its lower portions:

RAX (64 bits)
 └── EAX  (lower 32 bits)
      └── AX   (lower 16 bits)
           ├── AH  (high byte of AX)
           └── AL  (low byte of AX)

R8  (64 bits)
 └── R8D  (lower 32 bits)
      └── R8W  (lower 16 bits)
           └── R8B  (lower byte)

Writing to a 32-bit register (e.g., MOV EAX, 1) zero-extends to 64 bits in x64 — the upper 32 bits of RAX are cleared. This is a common source of confusion when reading x64 disassembly.

The Microsoft x64 Calling Convention

This is the most important difference for reverse engineers. In x86, all arguments go on the stack. In x64 Windows, the first four arguments go into registers.

Argument 1  →  RCX
Argument 2  →  RDX
Argument 3  →  R8
Argument 4  →  R9
Argument 5+ →  Stack (RSP+0x28, RSP+0x30, ...)
Return value →  RAX

Floating-point arguments use XMM0–XMM3 in the corresponding positions instead.

Shadow space (home space)

The caller must reserve 32 bytes of stack space before every CALL — even if the function takes fewer than four arguments. This is the "shadow space" or "home space" where the callee can spill its register arguments.

; x64 call to CreateFileW
lea   rcx, [filename_buffer]    ; Arg 1: lpFileName
mov   edx, GENERIC_READ         ; Arg 2: dwDesiredAccess (32-bit OK — zero-extends)
xor   r8d, r8d                  ; Arg 3: dwShareMode = 0
xor   r9d, r9d                  ; Arg 4: lpSecurityAttributes = NULL
mov   dword ptr [rsp+0x20], OPEN_EXISTING   ; Arg 5: dwCreationDisposition
mov   dword ptr [rsp+0x28], 0   ; Arg 6: dwFlagsAndAttributes
mov   qword ptr [rsp+0x30], 0   ; Arg 7: hTemplateFile
sub   rsp, 0x28                 ; reserve shadow space (32 bytes) + align
call  CreateFileW
add   rsp, 0x28                 ; restore stack

Reading API calls in x64dbg: always check RCX, RDX, R8, R9 for arguments 1–4. Arguments 5+ are at [RSP+0x28], [RSP+0x30], etc. after the shadow space.

RIP-Relative Addressing

In x86, global data is referenced by absolute address: MOV EAX, [0x00403000]

In x64, global data is referenced relative to the current instruction pointer: MOV RAX, [RIP+0x2A7F]

IDA Pro and Ghidra resolve RIP-relative references automatically, displaying the computed address and symbol name. This difference is largely transparent during analysis — the tools handle it for you.

Quick Reference: What to Check in x64dbg

Looking forCheck here
Argument 1RCX
Argument 2RDX
Argument 3R8
Argument 4R9
Arguments 5+Stack: RSP+0x28, RSP+0x30, ...
Return valueRAX
String pointer (arg 1)RCX — follow in dump
C++ this pointerRCX (implicit first argument)
Loop counterRCX or any R-register
Global data referenceLook for [RIP+offset] pattern

Heaven's Gate

Heaven's Gate is a technique that allows a 32-bit process running on 64-bit Windows (via WOW64) to switch the CPU into 64-bit mode using a far JMP to segment selector 0x33.

; 32-bit code transitioning to 64-bit
jmp  0x33:Heaven_Gate_64bit_code
Heaven_Gate_64bit_code:
    ; CPU is now in 64-bit mode
    ; can make 64-bit syscalls directly
    ; bypasses 32-bit API hooks installed by security tools

Malware uses Heaven's Gate to bypass 32-bit API hooks installed by AV and EDR products — those hooks only monitor 32-bit code paths.

Bypass for analysts: use a 64-bit debugger (x64dbg in 64-bit mode) or a kernel-level monitor that observes both 32-bit and 64-bit execution paths.

MITRE ATT&CK: T1055.014 — Process Injection: VDSO Hijacking (closest mapping)