[infosecinstitute] Hooking System Calls Through MSRs

In this article we presented the details of using sysenter instruction to call from user-mode to kernel-mode. In older versions of Windows operating systems, the “int 0x2e” interrupt was used instead, but on newer systems sysenter is used. When the “int 0x2e” interrupt is being used, it uses the 0x2e interrupt descriptor from the Interrupt Descriptor Table (IDT), while the system call number is passed in the eax register. On the other hand, the sysenter instruction can be used to transition from user to kernel-mode faster than by using the “int 0x2e” instruction. The instruction uses the Model Specific Registers (MSRs) specified below to do its thing. The MSR registers are control registers in the x86 machine used for debugging, program execution tracing, computer performance monitoring and toggling certain CPU features [1].