Case of frozen PeekMessage
Recently I was investigating why our application freezes during certain operations. After collecting few dump files, I found that every single time application freezes in quite old code that was written about 20 years ago. Loading documents could be quite long, and some person decided that it would be good idea if user can press Escape and cancel loading. To detect key press, that code calls something like this:
PeekMessage(Msg, CurrentFormHandle, WM_KeyFirst, WM_KeyLast, PM_Remove Or PM_NoYield)
Callstack looks like this:
win32u!NtUserPeekMessage+0x14 user32!_PeekMessage+0x43 user32!PeekMessageW+0x143
It looks quite innocent and I do not see any reason why it should hang. After some thinking I came with two theories:
- CurrentFormHandle returns invalid window handle that got reused by some thread that is sleeping
- Code that calls PeekMessage has infinite loop and PeekMessage is just took more time and appears in crash dumps.
Item #2 was quickly dismissed as analyzing code reveals that only one place has for-loop and I was able to find index value and max value on stack and they were 0 and 2. Clearly not infinite loop.
Then I tried to reproduce problem on my PC and after some time I was able to freeze our application. After that #1 was dismissed as I found that handle is correct value and it is windows that suppose to be. Then I start thinking that something could be wrong with that window, but there are too many possibilities. As result I decided to dive deeper and check what happens in kernel side.
To create kernel dump you need https://docs.microsoft.com/en-us/sysinternals/downloads/livekd and a lot of free space on disk as dump file will have the same size as amount of RAM you have. I went to directory where WinDbg is installed and ran following command:
livekd64.exe -o filename
After that it created 32 Gb file and you can open it in WinDbg the same way as any dump file.
First command you need is to find address of your process:
!process 0 0 MyApp.exe
You will see something like this: PROCESS ffff988f5ae03080
Then you set current process using this command:
It will dump all threads for that process. Typically, thread looks like this:
THREAD ffff988f4c60d080 Cid 8a50.69f0 Teb: 000000000028b000 Win32Thread: ffff988f6c4b2400 WAIT: (WrUserRequest) UserMode Non-Alertable ffff988f616130c0 QueueObject
Then you need to set current thread using this command:
.thread /p/r ffff988f4c60d080
After you can use typical call stack commands like k or kv. In my case callstack looked like this:
nt!KiSwapContext+0x76 nt!KiSwapThread+0x500 nt!KiCommitThreadWait+0x14f nt!KeWaitForSingleObject+0x233 nt!KeWaitForMultipleObjects+0x45b win32kfull!xxxRealSleepThread+0x362 win32kfull!xxxInterSendMsgEx+0xdd9 win32kfull!xxxSendTransformableMessageTimeout+0x3ea win32kfull!xxxCalcValidRects+0x32d win32kfull!xxxEndDeferWindowPosEx+0x1ac win32kfull!xxxSetWindowPosAndBand+0xc3 win32kfull!xxxSetWindowPos+0x79 win32kfull!xxxSetForegroundWindow2+0x772 win32kfull!xxxProcessActivationEvent+0xa6770 win32kfull!xxxProcessEventMessage+0x343 win32kfull!xxxScanSysQueue+0x83c win32kfull!xxxRealInternalGetMessage+0xee6 win32kfull!NtUserPeekMessage+0x158 win32k!NtUserPeekMessage+0x2a nt!KiSystemServiceCopyEnd+0x25 win32u!NtUserPeekMessage+0x14 user32!_PeekMessage+0x43 user32!PeekMessageW+0x143
It looks like PeekMessage is attempting send message to different thread and that thread does not respond. After checking special place, I confirmed that xxxInterSendMsgEx is for sending inter-thread messages. But how can I find which thread is it? Likely to me, special places told me that first parameter of xxxInterSendMsgEx is PWND and just pointer to regular HWND. As I ran kv command:
ffff8808`47efca80 00000000`00000000 00000000`00000000 ffff8808`49ff2310 : win32kfull!xxxInterSendMsgEx+0xdd9
And found window handle that I was looking for. Then using Spy++ from Visual Studio I found thread and found why application hangs.
It turns out that we have progress window that supposed to show that something is happens during long operation. That window is created in background thread. As result main window has pending deactivation and new foreground window should be progress window. And PeekMessage decided to send message to that window because it is in different thread. But activity in main thread and background thread reacts on the same WPF event and activity in main thread took lock in System.Windows.FrameworkTemplate.LoadContent and thread in background attempting to take the same lock and as result we have a deadlock.
Mystery solved. I hope it helps someone.