WinInet and SSL connection caching with INTERNET_FLAG_IGNORE_CERT_CN_INVALID
- Posted in:
Long-long time ago, our company created internal test tool that test web API for our servers. That tool simulates requests from our application, and it is using WinInet API. Tool always connection to test servers that are recreated every few days, it contains only test data. As this is testing server for our real production server, test tool using HTTPS protocol. But because it is using only for tests, it does have proper SSL certificate. Deploy script just install some self-signed certificate. Obviously testing tool cannot work by with that type of certificate and instead it just passes INTERNET_FLAG_IGNORE_CERT_CN_INVALID to HttpOpenRequest. Everything worked just fine for many years.
But several years ago, we discovered that our test tool became very slow in China. Main reason for this because China using country-level firewall that censor content and it takes time. Later we discovered that test tool is also slow when it is really far away from server. We started investigation. Very soon we discovered that our protocol became much more complex that before. Certain operations require multiple requests and as it turns out, test tool spend lot of on establishing SSL connection. Person who investigated that problem found that each request will establish new connection. As result that person decided to create request cache. Idea is really simple. After test tool finished with request, it will save it in cache and then it can be used by next time. Only one piece of code can use request at any give time to avoid multithreading issues. And that looks like fixed that issue and test tool speed increased by quite big margin.
But after some time, we were starting to get some strange reports. Test tool was working just fine on one machine but does not work on another machine. Some tests were working, and some were not. Some tests did not receive last few bytes of response. We blamed our servers, IT, Amazon, reptiloids etc. and finally I decided to disable cache and all problems immediately disappeared. Everything started to work everywhere but at slow speed. Speed became even slower because each test using even more requests than before.
Then I was started thinking. My thought was something like that: If WinInet always establishes connection, surely somebody will complain about that in internet and sure somebody else will provide solution for this. And I could not find anything. I was really surprised. And then I decided to write my own test that test different caching scenarios including no caching to all. And then I was surprise second time. There was no difference at all. All requests executed at about the same time and much faster than in test tool.
And I started digging. I spent quite some time on this and I found that even I copy-pasted code form test tool, due to difference in environment my version never passed INTERNET_FLAG_IGNORE_CERT_CN_INVALID to HttpOpenRequest. Also, I used production server, because nobody complained about any issues when using our application and honestly it was just faster to use production server because everyone knows its name.
And then it hit me. Many thanks to Stephen Sulzer in this thread: https://microsoft.public.inetsdk.programming.wininet.narkive.com/gOBWsOLA/help-bit-by-wininet-https-again. “Setting the IGNORE flags will cause WinInet to not reuse the socket connection”. As result if you are using flag INTERNET_FLAG_IGNORE_CERT_CN_INVALID not only it is bad for security, it also hurts performance by a lot if you are using multiple requests!
After that our IT team fixed test servers’ certificates and there is no need to use INTERNET_FLAG_IGNORE_CERT_CN_INVALID anymore and everything became fast again.
I hope it will help someone.