Load Balancer and System.Net Connection Management

When you make first HttpWebRequest from you application to access a web site or web service, you will use this code:

WebRequest request = WebRequest.Create(url);
WebResponse response = request.GetResponse();

The process of creating a connection with remote application is complex. In order to improve connection speed for later web requests, .NET framework creates a connection pool for each URL. When a web response comes back, the connection is still available for other web requests with the same URL.

By default, the connection’s KeepAlive attribute is true. But the connection can not stay forever, of course. In .NET 2.0, you can define MaxIdleTime -- when the connection has been idle for more than MaxIdleTime, the connection will be closed by framework.

The logic looks good so far. But when a load balancer is involved, a problem may arise. Load balancer directs a request to a Web service and creates connection for the request. If the connection is idle for a period, the load balancer will break the connection without sending notification out.

Let’s suppose the load balancer breaks idle connection after 90 seconds. But on application server side, .NET connection pool keeps idle connection for 100 seconds (by default). When a new web request is sent at this point, an HTTP transport exception will be thrown.

So it is important to make sure the connection timeout settings. After contacting system administrator for load balancer setting, developer can set MaxIdleTime for the connection:

request.ServicePoint.MaxIdleTime = maxIdleTimeValue;

Please make sure the .NET connection MaxIdleTime must be smaller than the load balancer's max connection idle setting. In this way, .NET framework will create a new connection when the connection has been broken by load balancer.


Rob Garrett said...

It's important to use good error trapping code here. In some cases it is impossible to obtain the timeout value from a load balanced server, especially if the server is accessed over the Internet and belongs to a 3rd party.

Jun Meng said...

Right, but for some cases, error trapping is not good option :)

In many cases, a client sends requests to a web service in several steps (e.g. loan application where several forms have to be filled out). The web service keeps state for the conversation with the client. To improve performance, the client should send requests to the same web service instance for those steps.

If the web service instance is destroyed for timeout, the client has to call some methods to create a new web service instance and to restore state.

If we check connection status while sending each web service request, you can imagine how big burden on client side code.

So for some cases, we had better make sure load balancer settings. In enterprise environment, this is still doable because even 3rd party should cooperate in order to make process smooth.