Main rule about overlapped I/O

February 29. 2020 0 Comments

Posted in:
Windows

Few days ago, I was investigating why one of our servers in cloud stops responding. Basically, it was so bad that we have to force restart server. After some investigation I found that there is IIS server that processes requests and some requests starts external application. And in certain situations, this process starts to allocate a lot of non-paged pool memory. About 100 megabytes per second.

Non-paged pool is special type of memory that is never paged out to disk. It is must be non-paged because at certain times of processing requests paging is unavailable. For example, when interrupt is processing. Also, for example disk driver structures should be non-paged because obviously they cannot be paged out. And in general, any I/O request leads to creating IRP structure. And very often it is allocated in non-paged pool. And usually only drivers able to allocate memory from non-paged pool.

Non-paged pool memory has fixed size and obviously cannot be bigger than amount of physical memory on that computer. As result in our case after all non-paged pool was exhausted, Windows not able to create pretty much any I/O request and crashed and then it should be restarted.

There are not many ways to allocate from non-paged pool for regular applications. Usually handles are one way. But quick check reveals that there were pretty much no handles or threads. I tried to search how else application can consume non-paged pool, but I couldn’t find anything.

Anyway, next step was trying to find who allocates non-paged pool. And for this case it is good idea to use poolmon.exe that is part of Windows Driver Kit. In Windows all allocations tagged with 4 characters. And by looking who allocates it could give me idea how to fix. It could also some 3^rd party driver etc.

So, I did run poolmon.exe pressed P until I can see only non-paged pool and then I pressed B to sort by total allocated bytes and reproduced scenario. To my surprise top 3 was Fmic, Irp and Even allocations. After checking file C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\triage\pooltag.txt I found that it is:
FMic - fltmgr.sys   -       IRP_CTRL structure
Irp - <unknown>    - Io, IRP packets
Even - <unknown>    - Event objects

Well it wasn’t really helpful. Then I took dump of that application when it was in bad state. This is our application and I have all source code. Anyway, call stack looks like this:

00 000000d7`631fd118 00007ff9`f56ab22d ntdll!NtFsControlFile+0x14
01 000000d7`631fd120 00000000`15086b11 KERNELBASE!ConnectNamedPipe+0x6d

It looks like our application communicates via named pipe. After checking source code, I found that our application starts secondary application and they communicating via named pipes. But by some reason developer decided to open that named pipe in overlapped mode. You can read more here: https://docs.microsoft.com/en-us/windows/win32/sync/synchronization-and-overlapped-input-and-output.

It looked quite strange because whole idea of using overlapped mode is to avoid waiting for establishing connection. Initially I thought it just coincidence and I captured dump in wrong time, but I took few more dumps and all of them have exactly the same call stack.

I did check code and it looks ok. I extracted that code to two different applications: primary and secondary and they did run fine. I kept them running for some time and I didn’t see any problem. I tried restarting secondary on every single request and they did run fine. I tried to run all requests in single secondary instance and I also did mixed mode. Every time it works fine.

Then I started looking for code and I got it. Here is code:

    HANDLE pipeHandle;
    OVERLAPPED overlapped = { 0 };

    overlapped.hEvent = CreateEvent(...);
    BOOL result = ConnectNamedPipe(pipeHandle, &overlapped);
    if (!result)
    {
        switch (GetLastError())
        {
            case ERROR_IO_PENDING:
            {
                HandleIoPending(pipeHandle, overlapped);
                break;
            }
            ...
        }
    }
    ...

And after some time, I got what was problem. In certain conditions, HandleIoPending would return while I/O request is still pending. As you can see that overlapped is allocated on stack and later when I/O request would complete Windows will try to update overlapped structure, but it is long gone or used by stack data of some other function. I tried to simulate those conditions and my small application started to consume a lot of non-paged pool. Victory!

So, if you are using overlapped I/O you should obey one rule: never free or reuse overlapped structure while requests are pending. Even if you cancel I/O you still have to wait for completion. One example is to use GetOverlappedResult with wait parameter TRUE. In my case I fixed code in HandleIoPending to always wait for request to complete. I have no idea why this leads to non-paged pool consumption instead of crash but perhaps this bug corrupting some critical structures somewhere in Windows.

Somebody can ask: why do we use overlapped I/O while anyway we are waiting for request to complete? Developer did introduce timeout on waiting and as result if secondary application stuck, we could detect this situation and kill it. And know that there are more more safer ways to do the same but by some reason develop did choose quite unsafe way.

I hope it helps someone.