Exceptions in Windows hooks

June 12. 2022 0 Comments

I think many of you a familiar with function SetWindowsHookEx. It is quite useful function, and we use it in a lot of places in our software. But recently I found interesting issue: if exception is raised in hook function, then Windows will terminate your application with status code 0x C000041D STATUS_FATAL_USER_CALLBACK_EXCEPTION. But there is one caveat: if you run application from debugger, Windows will not terminate application and will just suppress exception. So, if you have encounter that exception as developer, you may think that everything is correct, but when end user will run your application – application will crash.

I work primarily with languages that have native support for exceptions and it feels kind of wrong. There is exception handler in call stack and exception should be handled by it, but it does not.

All above will raise 2 questions:

Why is exception not handled properly?
Why is application terminated by Windows?

Why is exception not handled?

Here is call stack of crashed application:

kernel32!WerpReportFault+0xbe
KERNELBASE!UnhandledExceptionFilter+0x3d9
ntdll!LdrpLogFatalUserCallbackException+0x98
ntdll!KiUserCallbackDispatcherHandler+0x20
ntdll!RtlpExecuteHandlerForException+0xf
ntdll!RtlDispatchException+0x244
ntdll!RtlRaiseException+0x185
KERNELBASE!RaiseException+0x69
...
user32!DispatchHookW+0xb2
user32!CallHookWithSEH+0x29
user32!__fnHkINLPMSG+0x54
ntdll!KiUserCallbackDispatcherContinue
win32u!NtUserPeekMessage+0x14
user32!_PeekMessage+0x43
user32!PeekMessageW+0x143

As you can see there is function named CallHookWithSEH. SEH stands for “Structured Exception Handling”. It gives a clue that this function is supposed to handle exceptions which should not be needed if exceptions would be handled normally. Moreover, that fact is reflected in the function name.

Next clue is function KiUserCallbackDispatcherContinue. If you disassemble its code, then you will see following code:

ntdll!KiUserCallbackDispatch:
	mov     rcx,qword ptr [rsp+20h]
	mov     edx,dword ptr [rsp+28h]
	mov     r8d,dword ptr [rsp+2Ch]
	mov   rax,qword ptr gs:[60h]
	mov     r9,qword ptr [rax+58h]
	mov     rax,qword ptr [r9+r8*8]
	call    ntdll!KiUserCallForwarder
ntdll!KiUserCallbackDispatcherContinue:
	xor     ecx,ecx
	xor     edx,edx
	mov     r8d,eax

As you all know, when CPU executes call instruction, it will put return address on stack, which is address of the next instruction. So effectively, CPU is executing KiUserCallForwarder function and that is part of KiUserCallbackDispatch function. If you try to search internet, you will find this article that states that KiUserCallbackDispatch function is “a trampoline that is used to make full-fledged calls to user mode, from kernel mode”. Here is first confirmation that kernel mode is involved.

But where is switch from user mode to kernel? If you disassemble code from NtUserPeekMessage then you will see following code:

win32u!NtUserPeekMessage:
	mov     r10,rcx
	mov     eax,1001h
	test    byte ptr [SharedUserData+0x308 (00000000`7ffe0308)],1
	jne     win32u!NtUserPeekMessage+0x15 (00007ffc`8efa1065)
	syscall
	ret

Syscall instruction used to switch to kernel mode from user code. This is second confirmation that there is kernel mode.

At this moment whole picture looks like this: my code calls NtUserPeekMessage that transfer execution to kernel and then kernel calls KiUserCallbackDispatch function to execute code of hook in user mode.

If you are familiar with exception handling code, then you know that to process exception and “unwind”, exception handling code needs access to list of all exception handlers in x86 or to stack and code in x64 architecture. But user mode code is never able to access kernel side by design. As result, to do it properly, Microsoft must implement additional, quite clever techniques. I think Microsoft decided that it is not worth effort, and it will create security risk.

Why is application terminated?

But why Windows will terminate such application? To my surprise, Windows 7 will not terminate application. But Windows 10 does. In Windows 7 application may have serious issue but nobody sees it because all exceptions will be suppressed. And that issue could corrupt user’s data. And I think it is main reason that Microsoft decides that it is better to terminate bad application instead of user data corruption.

I hope it will help someone.