Mastering Assembly Programming

System call hardware interface

On the hardware level, the processor provides us with several ways to invoke a kernel procedure to handle a system call:

• Through an interrupt (INT instruction on 32-bit systems): The operating system allocates a descriptor for an interrupt with a specific number, which points to a procedure in the kernel space that handles the interrupt in accordance with its parameters (parameters are passed via registers). One of the parameters is the index into system call table (roughly speaking, the table of pointers to specific system call handlers).
• Using the SYSENTER instruction (32-bit systems excluding WOW64 processes): Beginning with Pentium II, we are able to use the SYSENTER instruction to perform fast calls to ...