by benthepoet » Thu Aug 25, 2011 4:54
The problem appears to be with XRender as the compositing type under Desktop Effects. Memory leak persisted on both kernel 3.0 and 2.6.39 as well as both ATI and Nvidia proprietary drivers. Notably every time you trigger the Present Windows effect, X's memory usage will increase quite a bit. When I opened up xrestop, KWin appeared to be the one hogging all the memory.
What I ended up doing was setting the compositing type to OpenGL and setting the QT_GRAPHICSSYSTEM variable to 'raster'. With these settings it doesn't seem like the leak is present, X's memory usage is now consistently around 30MB.