4

Closed

Memory Leak in UdpNetworkSender

description

I am debugging a memory leak in an application. I have noticed a constant raise in instances of SocketAsyncEventArgs, AsyncContinuation, LogEventInfo, Layout, NetworkTarget delegates etc. All retained on the managed heap by references from System.Threading.OverlappedData instances.
The application uses a Chainsaw target for logging.
 
I've looked at the implementation of UdpNetworkSender and I think the leak originates from the Socket.SendToAsync callback:
 
private void SocketOperationCompleted(object sender, SocketAsyncEventArgs e)
{
AsyncContinuation userToken = e.UserToken as AsyncContinuation;
Exception exception = null;
if (e.SocketError != SocketError.Success)
{
    exception = new IOException("Error: " + e.SocketError);
}
e.Dispose();
if (userToken != null)
{
    userToken(exception);
}
}
 
The thread pool has a cache of IO Completion objects that eventually hold references to the SocketAsyncEventArgs.
Since the SocketAsyncEventArgs.UserToken is not set to null, the AsyncContinuation user token is not garbage collected. The AsyncContinuation in turn holds a ref to NLog LogEventInfo instance with the message string and layout.

file attachments

Closed Oct 7, 2012 at 6:49 PM by Xharze
Moved to https://github.com/NLog/NLog/issues/131

comments

ketan1985 wrote Dec 29, 2011 at 1:35 PM

This memory leak issue is not just with the UdpNetworkSender, but also with the normal file i/o operation. The objects of AsyncContinuation and LogEventInfo are never released. I created a sample multi-threaded application and executed NLOG-Trace calls for logging in log.txt. This application continues to run for 5-10 mins and then "System.OutOfMemoryException". I also checked the gc-heap using immediate window commands and found the following information.

count memory Objects
4527529 144880928 NLog.Common.AsyncContinuation
2263769 153936292 NLog.LogEventInfo

jkowalski wrote Dec 29, 2011 at 4:55 PM

Can you provide a simple program/config file that reproduces the issue? I am unable to reproduce it on my machine. What OS/.NET version is this?

zvikagart wrote Jan 4, 2012 at 5:01 PM

Attached is a simple program that uses the Chainsaw target
I'm running on a windows 7 64 bit machine, .NET 4.0

please note that when the garbage collector decides to collect the memory of thread pool IO completion objects then the NLog objects are collected as well, so depending on the GC strategy the amount of NLog objects retained on the managed heap may vary. For example, I haven't noticed memory growth when running as a 32 bit application...
Anyway, the fix I proposed (setting the SocketAsyncEventArgs.UserToken to null) will allow the objects to be collected earlier.

Hesse wrote Jan 25, 2012 at 12:31 PM

I think a similar problem exists in the TCP logger. I originally posted a message over in the NLog forum some time ago, but didn't get a response. The issue is all of the messages are held in memory if there is not a logger attached. Eventually you will get an OutOfMemoryException. Here's the post:

I have an application setup to use NLog in .NET 4.0 with a TCP logger. In debug mode, I have the application configured to send lots of messages using the Log.Trace(message) syntax. Everything works fine when I'm viewing the messages using Log2Console.

However, when I don't have a logger attached (or maybe even with), the log messages all seem to be kept in memory by NLog.Internal.NetworkSenders.TcpNetworkSender+MySocketAsyncEventArgs. In my memory profiler, it claims this object is disposable but is not yet disposed. It looks like all of my messages are being kept in memory and not being cleared. Is there a setting somewhere that I'm missing to make these objects get cleaned up?

devdigital wrote May 4, 2012 at 2:38 PM

Is there an update on this? The issue also occurs with the WCF LogReceiverService.

masmazar wrote Jun 15, 2012 at 8:36 PM

I know it has been months since last post, but I thought I should post here anyways because the leak still exists in the latest code I got from GIT hub.
I think I made good progress eliminating 90% of the leak, so I will try to send my changes back to GIT hub.
There is still a bit of leak which I would be glad to explain to someone who is more familiar with this code than I am.