[Issues] [apr_memcache 0000068]: memory leaks in apr_memcache_getp

[issues at outoforder.cc]

A NOTE has been added to this issue. 
====================================================================== 
http://issues.outoforder.cc/view.php?id=68 
====================================================================== 
Reported By:                yangluo
Assigned To:                
====================================================================== 
Project:                    apr_memcache
Issue ID:                   68
Category:                   Error Handling
Reproducibility:            always
Severity:                   major
Priority:                   normal
Status:                     new
====================================================================== 
Date Submitted:             12-30-2006 03:12 EST
Last Modified:              05-02-2007 21:55 EDT
====================================================================== 
Summary:                    memory leaks in apr_memcache_getp
Description: 
When I was testing apr_memcache with my program I found it was using more
and more memory.
My code is just like the official example with a big loop calling
apr_memcache_getp().

The source code is attached.
====================================================================== 

---------------------------------------------------------------------- 
 michelnok - 04-06-07 11:10  
---------------------------------------------------------------------- 
apr_memcache_getp() MUST allocate memory for data received from Memcached.
It allows to specify in second parameter a SEPARATE pool to allocate
memory for data received. You've specified the same pool as for
apr_memcache_create(), so apr_memcache_getp() allocates memory in the pool
that lives all time the program is run. 

---------------------------------------------------------------------- 
 michelnok - 04-06-07 11:30  
---------------------------------------------------------------------- 
BUT even if you specify separate pool for apr_memcache_getp() then
apr_memcache still consume memory from the pool apr_memcache_create() was
called with.
It IS a bug. apr_memcache_getp() isn't supposed to allocate anything in
the pool apr_memcache_create() was called with. 

---------------------------------------------------------------------- 
 Codebender - 05-02-07 21:55  
---------------------------------------------------------------------- 
I'm running into this same issue.  Various operations, including
apr_memcache_getp(), are allocating memory from a pool other than the one
passed to it.  This means that one must periodically destroy and recreate
the memcache and memcache_server objects (clearing or recreating the pool)
to avoid unbounded growth of the memory usage.

This is because of the brigade_split_line() call in get_server_line(). 
This eventually calls apr_bucket_alloc(), which allocates directly from
the allocator specified when the brigade was created, rather than
allocating from a pool.  The brigades and brigade allocators are created
within mc_conn_construct(), based on an allocator created for each
connection object.

There's a commented-out call to apr_pool_destroy() in mc_conn_destruct(). 
Uncommenting this causes the memory allocated by the brigade operations to
be returned to the global pool when the connection object is destroyed.  I
don't know why this line was commented out, but I have to assume it was to
fix some other bug, so I'm wary of leaving it in that state.  It's
commented out in 0.6.0, the earliest version available in SVN.

Even with this call uncommented, you will need to ensure that the
connection objects are destroyed on a regular basis.  I tested this by
closing the connection handle externally, simulating a dropped TCP
connection, so something would need to be added to allow for a maximum
lifetime of connection objects.

If the lifetimes of these brigades could be separated from the lifetime of
the connection objects it would also solve the problem, I believe.  I'm not
sure what the performance penalty would be for creating and destroying
brigades for each get/set/add/etc., operation.  I plan to investigate
this, but I do currently have it working and stable through regular
recreation of the main memcache object and pool. 

Issue History 
Date Modified   Username       Field                    Change               
====================================================================== 
12-30-06 03:12  yangluo        New Issue                                    
12-30-06 03:12  yangluo        File Added: memoryleak.c                     
04-06-07 11:10  michelnok      Note Added: 0000089                          
04-06-07 11:30  michelnok      Note Added: 0000090                          
04-06-07 12:17  michelnok      Issue Monitored: michelnok                    
04-22-07 19:52  proforg        Issue Monitored: proforg                     
05-02-07 21:55  Codebender     Note Added: 0000091                          
======================================================================