Disclaimer. This is perhaps not a best way to do, but I couldn’t find any better way.Let’s imagine that you have some Task to run in the background. And that task has to update UI for example progress bar. Or that task has to call some callbacks in context of main thread. For firs case you would use Invoke, for second case you would use SynchronizationContext.Send. Implementation is very simple. WinForms will add request to queue of requests and post message to main thread and then it wait on request handle to be signaled. Main thread receives message, processes request and signal on request event. Very simple and quite basic. Post is the same as Send but [...Read More]
In this post I will explain my investigation about handle leaks and how it resulted in interesting discovery in MemoryCache class.
I received few reports stating that our application consuming handles even when it supposed to do nothing. And then I started my investigation. I explained how to find where is leaking handles here.
After application leaked about 50 handles, I paused application and checked call stack. And to my surprise I found a lot of callstack that looks like this:
From this callstack it wasn’t really obvious what part of [...Read More]
ntdll!NtCreateThreadEx+0x0000000000000014 KERNELBASE!CreateRemoteThreadEx+0x0000000000000294 KERNELBASE!CreateThread+0x000000000000003b verifier!AVrfpCreateThread+0x00000000000000bf clr!Thread::CreateNewOSThread+0x00000000000000b2 clr!Thread::CreateNewThread+0x0000000000000091 clr!ThreadpoolMgr::CreateUnimpersonatedThread+0x00000000000000cb clr!ThreadpoolMgr::MaybeAddWorkingWorker+0x000000000000011c clr!ManagedPerAppDomainTPCount::SetAppDomainRequestsActive+0x0000000000000024 clr!ThreadpoolMgr::SetAppDomainRequestsActive+0x000000000000003f clr!ThreadPoolNative::RequestWorkerThread+0x000000000000002b -------------------------------------- ntdll!NtCreateEvent+0x0000000000000014 verifier!AVrfpNtCreateEvent+0x0000000000000080 KERNELBASE!CreateEventW+0x000000000000006b verifier!AVrfpCreateEventW+0x00000000000000a5 clr!CLREventBase::CreateManualEvent+0x000000000000003a clr!Thread::AllocHandles+0x000000000000007b clr!Thread::CreateNewOSThread+0x000000000000007f clr!Thread::CreateNewThread+0x0000000000000091 clr!ThreadpoolMgr::CreateUnimpersonatedThread+0x00000000000000cb clr!ThreadpoolMgr::MaybeAddWorkingWorker+0x000000000000011c clr!ManagedPerAppDomainTPCount::SetAppDomainRequestsActive+0x0000000000000024 clr!ThreadpoolMgr::SetAppDomainRequestsActive+0x000000000000003f clr!ThreadPoolNative::RequestWorkerThread+0x000000000000002b mscorlib_ni!DomainNeutralILStubClass.IL_STUB_PInvoke()$##6000000+0x0000000000000069
I received few reports stating that our application consuming handles even when it supposed to do nothing. And then I started my investigation. At the beginning I would like to find if it is possible to find handle leaks with some tools. And I immediately found video by Jeff Dailey I believe: https://channel9.msdn.com/Blogs/jeff_dailey/Understanding-handle-leaks-and-how-to-use-htrace-to-find-them who explained how to do it. This video is more than 13 years old. I will explain steps here, because I really hate videos. It is much faster to read than watch.
You need to run C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\gflags.exe and it is usually installed with Debugging Tools and part of Windows SDK. You will see something like this:
If you work with COM objects you may accidently attempt to use or create COM object in one thread and accidently use it in another thread. And then unless original thread is free threaded .NET will attempt to switch to original thread.
And from there you will see one of the following outcomes.
- If that thread pump messages, then .NET will attempt to send message to that thread to switch. Sometimes it can lead to deadlock or just long delay if that thread is busy. Main thread usually pumps messages, so usually .NET able to switch to main thread.
- You will get error that Interface is not supported. Sometimes you will get that cryptic message even that interface is clearly supported. You will get this message because .NET don’t know how to marshal to original thread, specially if this is some background thread. In this case .NET is not able to marshal, so it will ask object itself to marshal by requesting few interfaces and one of them is IMarshal. Most objects do not support these interfaces and you will see that error message
- When original thread was terminated, or if that was thread pool thread and it will clean its state on next request. In this case you will get this one from MDA: Managed Debugging Assistant 'DisconnectedContext' : 'Transition into COM context 0xed058 for this RuntimeCallableWrapper failed with the following error: The object invoked has disconnected from its clients. (Exception from HRESULT: 0x80010108 (RPC_E_DISCONNECTED)). This is typically because the COM context 0xed058 where this RuntimeCallableWrapper was created has been disconnected or it is busy doing something else. Releasing the interfaces from the current COM context (COM context 0xec968). This may cause corruption or data loss. To avoid this problem, please ensure that all COM contexts/apartments/threads stay alive and are available for context transition, until the application is completely done with the RuntimeCallableWrappers that represents COM components that live inside them.'
Recently I was investigating deadlock related to COM in .NET. In we have some engine that executes asynchronous requests. It is written in native code and use COM like interfaces. Nothing like true COM where you have some registration etc. We just use COM like interfaces for interop between .NET and native code. And I have following code:
00 00000001`4eeed7d8 00007ff9`184b8027 ntdll!NtWaitForMultipleObjects+0x14 01 00000001`4eeed7e0 00007ff9`1a583905 KERNELBASE!WaitForMultipleObjectsEx+0x107 02 00000001`4eeedae0 00007ff9`1a583665 combase!MTAThreadWaitForCall+0x115 03 00000001`4eeedbb0 00007ff9`1a5329c7 combase!MTAThreadDispatchCrossApartmentCall+0xc5 04 (Inline Function) --------`-------- combase!CSyncClientCall::SwitchAptAndDispatchCall+0x325 05 00000001`4eeedc00 00007ff9`1a581e94 combase!CSyncClientCall::SendReceive2+0x407 06 (Inline Function) --------`-------- combase!SyncClientCallRetryContext::SendReceiveWithRetry+0x25 07 (Inline Function) --------`-------- combase!CSyncClientCall::SendReceiveInRetryContext+0x25 08 00000001`4eeede00 00007ff9`1a530900 combase!DefaultSendReceive+0x64 09 00000001`4eeede60 00007ff9`1a581bc4 combase!CSyncClientCall::SendReceive+0x330 0a 00000001`4eeee090 00007ff9`1a5a2e4e combase!CClientChannel::SendReceive+0x84 0b 00000001`4eeee100 00007ff9`1b1d8b95 combase!NdrExtpProxySendReceive+0x4e 0c 00000001`4eeee130 00007ff9`1a5a0cbb rpcrt4!NdrpClientCall3+0x395 0d 00000001`4eeee490 00007ff9`1a61c5f2 combase!ObjectStublessClient+0x13b 0e 00000001`4eeee820 00007ff9`1a5268a9 combase!ObjectStubless+0x42 0f 00000001`4eeee870 00007ff9`1a59cd6f combase!CObjectContext::InternalContextCallback+0x259 10 00000001`4eeee990 00007ff8`c79fed60 combase!CObjectContext::ContextCallback+0x7f 11 00000001`4eeeea30 00007ff8`c79ffb42 clr!CtxEntry::EnterContext+0x295 12 00000001`4eeeec10 00007ff8`c789de97 clr!IUnkEntry::UnmarshalIUnknownForCurrContext+0xbd 13 00000001`4eeeecc0 00007ff8`c789dde2 clr!IUnkEntry::GetIUnknownForCurrContext+0x20cd8f 14 00000001`4eeeecf0 00007ff8`c76924c1 clr!RCW::SafeQueryInterfaceRemoteAware+0x20bab2 15 00000001`4eeeed50 00007ff8`c75f1f79 clr!RCW::CallQueryInterface+0x8d 16 00000001`4eeeedc0 00007ff8`c75f226f clr!RCW::GetComIPForMethodTableFromCache+0xb5 17 00000001`4eeeee80 00007ff8`c75f2716 clr!ComObject::SupportsInterface+0xfe 18 00000001`4eeeeff0 00007ff8`c75f2642 clr!Object::SupportsInterface+0x9e 19 00000001`4eeef060 00007ff8`c75f255a clr!UnmarshalObjectFromInterface+0x7a 1a 00000001`4eeef0a0 00007ff8`69a6a493 clr!StubHelpers::InterfaceMarshaler__ConvertToManaged+0xca 1b 00000001`4eeef240 00007ff8`c74b2e89 0x00007ff8`69a6a493 1c 00000001`4eeef290 00007ff8`c75f13b3 clr!COMToCLRDispatchHelper+0x39 1d 00000001`4eeef2c0 00007ff8`c74b2de7 clr!COMToCLRWorker+0x1b4 1e 00000001`4eeef380 00000001`3aeb49fb clr!GenericComCallStub+0x57 1f 00000001`4eeef410 00000001`3aeb4721 AsyncEngine.AsyncResult.PerformCallback+0x4b
Many .NET developers sooner or later will have to do interop with unmanaged world. Let’s say for example that you need to use some Dll written in C++. Passing integers or floats relatively easy, but things are much more complicated with strings. Many C++ Dll still uses ANSI strings, so let’s talk about them. Normally when you read Microsoft documentation you will be slightly confused. Some pages states that you have to use UnmanagedType.LPStr like it stated here: https://docs.microsoft.com/en-us/dotnet/api/system.runtime.interopservices.unmanagedtype?view=netframework-4.8#System_Runtime_InteropServices_UnmanagedType_LPStr. And that looks like correct for ANSI strings. Moreover, you can try it and it works just fine.Some pages stated that you have to use IntPtr: https://docs.microsoft.com/en-us/previous-versions/ms172514(v=vs.100)?redirectedfrom=MSDN and then manually prepare strings to pass [...Read More]
One of complications for STA COM interfaces that they have thread affinity and any function from that interface should be called only from that thread. For example, imagine that main thread actively using some COM interfaces. And in some moment GC thread started to release unused interfaces.
But GC cannot call Release function of that interface in GC finalizer thread. GC thread should switch to main thread to call Release. This is quite slow and main thread could be busy. In case when there are a lot of interfaces went on .NET side it can create huge queue of interfaces waiting for their release and there could be some other objects waiting for finalizer to be called.As mitigate that, [...Read More]
Another thing that developer should be aware when consuming COM interface is apartment. It is quite long topic and I will not explain it in detail. For today discussion let’s talk about STA. If you never use COM interface in .NET from different threads you can ignore this post.
STA means single thread apartment. Normally main thread has STA. COM interface that arrives to .NET runtime usually will be marked as STA. It means that that COM interface will be used only from thread it originally passed to. It called thread affinity. There are ways to change that behavior, but it will be in later posts.
As result when you call any function on STA COM interface and current thread is not one that originally received that interface then .NET runtime will attempt to switch to that thread.[...Read More]
COM in .NET is very tricky to do properly, and it is source of a lot of confusions. It is quite extensive area and I will try to explain stuff I learnt from it. I would like to mention that all that I got from practice and from exploring source code of CLR and call stacks in WinDbg. But some conclusions could be wrong so use it at own risk.
Firstly, we have to start with basics. Let’s talk about case when .NET code consumes some external COM object..NET cannot consume COM interfaces directly, so when COM interface crosses .NET boundary runtime will create special structure called Runtime Callable Wrapper (RCW for short) and .NET native objects (it called [...Read More]