Watch, Follow, &
Connect with Us

For forums, blogs and more please visit our
Developer Tools Community.


ID: 16380, VCL leak fix for dynamic DLLs

by Dejan Maksimovic Email: Anonymous


A small fix for a leak in Forms.pas (Classes.pas in D6) procedure
MakeObjectInstance. Useful for DLLs.
Download Details
FTP  download also available
CDN Login Required to Download. (You will be redirected to the login page if you click on the Download Link)
To download this, you must have registered:
A free membership

For Delphi, Version 1.0  to 6.0 942 downloads
Copyright: All rights reserved


Size: 3,440 bytes
Updated on Tue, 17 Jul 2001 13:36:46 GMT
Originally uploaded on Tue, 17 Jul 2001 13:57:44 GMT
SHA1 Hash: DB57B6E9F111408DEA902F486D09942E414096A3
MD5 Hash: E848CCB5941B42AE346DFFCA2C12B492

    Explore the files in this upload

Description
Applies to: MakeObjectInstance and FreeObjectInstance (located in Forms.pas until Delphi 5. In Delphi 6, they are located in Classes.pas).

Reason: Memory allocated in these procedures is not freed until the host program exits. For common applications, this is not a problem, since the application closes when the last form is closed, and the memory is used until that moment. However, a DLL that uses Forms VCL and is loaded and unloaded many times, this represents a serious leak, since the host program does not exit, in the meantime. An example of this is a Shell Extension that uses Forms. Since it is loaded and unloaded by Explorer from time to time (usually 2/5 minutes, depending on the system), over time this DLL will represent a noticeable memory leak. Also, services that run 24x7 are even more vulnerable by this - the leak will really show, and can make a problem to the system.

Terminology: The VCL uses MakeObjectInstance to allocate instance objects which are later used to process messages for classes that encapsulate controls. When the instance object is no longer used, it is returned to the list of free instance objects, to be reused later. However, these instance objects (actually the blocks that contain them) are not freed until the program closes, and even then they are not freed by the program, but by the system (for more information, you may want to lookup memory management in the Win32 Help Documentation).

Background: This "leak" was present for a long time (since the days of Turbo Pascal, and after in Delphi 1, 2 etc.), and is also present in MFC applications. A probable reason that this was never removed or changed is that it is not a problem except in extreme cases, as mentioned above, and a fix isn't easy, as you will see from the code in the article. Borland is currently investigating a long-term solution for Delphi 7. In the meantime, you can use the code in this article, in case that you are making DLLs that use Forms, or just want to see MemProof show no leaks in your program:-)

Disclaimer: This code is provided "as is"! The author takes no responsibility for use or misuse of this code. Use the code at your own risk. The code and text in this article is not associated with Borland.

License: Everyone is free to use the code in this article. The code may not be distributed, as it is bound by the terms and conditions of Borland product license.

Limitations: This fix will have no effect if the application or the DLL are linked with run-time packages, since the code is not linked into the DLL statically then.

Synchronization of memory access: Synchronization of memory access has become a very known issue. The VCL is not intended to be thread safe, as any manipulation of VCL objects in the multi threaded application should be done by using Synchronize method of TThread objects. This is the reason why the code inside MakeObjectInstance is not synchronized. The code included in this article shows how the synchronization can be done. Although the synchronization method may not seem to be secure, it is. It is recommended that you use the sample only to see how synchronization works, and how it can be done, but that you do not use it, as it will slow down the code that is synchronized.

How to use: If you are using Delphi 6, open Classes.pas, otherwise open Forms.pas (the code was moved for portability in D6 from Forms to Classes.pas). The file(s) can be found in Delphi\Source\VCL directory (except for Delphi 6) or Delphi\Source\Rtl\Common (for Delphi 6). Make a backup of the file, before making any changes! Find the lines that refer to the types and procedures listed below, and replace them. For example:
- Find declaration of TObjectInstance and replace it with the one in this code article.
- Find declaration of TInstanceBlock (it should be right after the declaration of TObjectInstance), and replace with the declaration in this code article.
- If you wish to use synchronization, copy and paste the two variables in the synchronization section (ObjInstLock ObjInit) from this article into the code of Classes.pas/Forms.pas (you can place it anywhere in the Implementation section, but it must be before the MakeObjectInstance procedure).
- Find declaration of MakeObjectInstance and replace it with the code in this article.
- Find declaration of FreeObjectInstance and replace it with the code in this article.
Save the changed Classes/Forms.pas. Now you should recompile the Classes/Forms.pas, and copy the newly compiled file to Delphi\Lib directory. You should make a backup copy of Classes/Forms.dcu before replacing it. The backups are saved, in case you want to revert to the standard code later. From now on, your applications or DLLs will use the new code.

Idea behind the code: The original VCL allocates 4 Kb memory chunks to satisfy the need of object instances for the controls. It then organizes these object instances in a reusable way. The memory is not freed, however, when it is no longer needed (for non console applications this happens only after the applications terminates, so once again, I mention, that this is not a problem). The code in this article, on the other hand keeps a count of how many object instances are used in each instance block, and can therefore, safely remove the items from the free list, when no items are used from a single instance block, and free that block.

NOTES: The code uses some methods which certainly seem complex in the eyes of a class writer (person who programs with classes:-). I have tried to comment the code as much as possible, but linguistics is not my better side, so I'll stick to mathematics and programming, and hope that most readers will understand it. Probably the part that most will take time to understand is the pointer arithmetic, why and how it can be used, and linked lists. For linked lists, you will find many books that cover the subject (most carry the name "Data Structures"). As for pointer arithmetic, one must understand the Windows Memory Management, to understand why it can be used, and also that Inc increments pointer by the size of the structure they point to, not just by one.

How was the code verified: I made a test that created 2048 forms, and then destroyed them randomly.

Who did this: I am a low level programmer, doing software only drivers. This has been an interesting refreshment, after the low level code:-)

And finally, the code:
// Replacement for MakeObjectInstance and FreeObjectInstance

// This is the maximum number of object instances that can be
// put in one memory page, with other reference data.
Const InstanceCount = 312;

// The following 4 types are used for double linked lists of instance
// objects and blocks.
Type PObjectInstance = ^TObjectInstance;
TObjectInstance = Packed Record
Code :Byte;
Offset :Integer;
Case Integer Of
0 :(Next, Prev :PObjectInstance);
1 :(Method :TWndMethod);
End; // 13 bytes in size. Note that it's packed!
// The size matters because we have to know how many of these records
// can be put inside one Instance Block (see below), for the block to
// be as close as possible to one memory page size (but less than it!).

Type PInstanceBlock = ^TInstanceBlock;
TInstanceBlock = Packed Record
Next :PInstanceBlock; // Points to the previous Instance block
Code :Array[1..2] Of Byte; // Instruction bytes.
WndProcPtr :Pointer; // Window Procedure Pointer
InstCount :Integer; // How many object instances are used in the block
Prev :PInstanceBlock; // Points to the previous Instance block
// Used for faster list management.
Instances :Array[0..InstanceCount] Of TObjectInstance;
// ^ Instance objects.
End; // Size of the block is:
// 4 bytes for Next
// 2 bytes for Code
// 4 bytes for WndProcPtr
// 4 bytes for InstCount
// 4 bytes for Prev
// 312 * 13 bytes for Instances
// -----------------------------
// 18 + 313 * 13 = 18 + 4069 = 4807 bytes. Just 9 bytes less than
// a memory page:-)

Var InstBlockList :PInstanceBlock = Nil; // The list of blocks currently allocated
InstFreeList :PObjectInstance = Nil; // The list of free Object instances
// The below variables are only used if you need synchronization of the objects
// I suggest that you do not use these! It will only make a small
// overhead in the code, and due to the way VCL is made, it is not needed
{$IFDEF VCL_Synchronize}
ObjInstLock :TRTLCriticalSection; // Critical section used to guard the memory
ObjInstInit :Boolean = False; // Has the critical section been initialized?
{$ENDIF}

// CAUTION: There are three possible problems in the below procedure:
// 1 :) Synchronization is not enabled by default (according to Borland
// the data is not intended to be thread safe, and I think that's OK, more
// or less, if you are using VCL thread objects and Synchronize method,
// when using VCL objects from the threads).
// 2 :) VirtualAlloc may fail! If this happens, the return value is undefined
// Regardless of how unlikely this is to happen, it can happen. The thing
// is that, if it does happen, the procedure that called MakeObjectInstance
// will raise an exception, when it wants to access the block, anyway.
// 3 :) I am not sure how portable is the way to get the block, based on the
// instance address (Address And Not 4095, which will essentially
// align the Address to a 4 kb value). This works under Win9x/Me,
// NT/2000 and XP. That's enough for now:-)

Function MakeObjectInstance(Method :TWndMethod) :Pointer;
Const
BlockCode :Array[1..2] Of Byte = (
$59, { POP ECX }
$E9); { JMP StdWndProc }
PageSize = 4096;
Var
Block :PInstanceBlock;
Instance :PObjectInstance;
Begin
{$IFDEF VCL_Synchronize}
// Check if the synchronization object has been initialized
If Not ObjInstInit Then
Begin
// If not, initialize it, and tell that it's initialized
InitializeCriticalSection(ObjInstLock);
ObjInstInit := True;
End;
// The code after this line is synchronized between threads.
EnterCriticalSection(ObjInstLock);
// Since there might be a memory allocation failure, the try block
// makes sure that the critical section is released.
Try
{$ENDIF}
// Is there some object instance we can already use?
If InstFreeList = Nil Then
Begin
// No, so allocate a new block of object instances
Block := VirtualAlloc(Nil,
PageSize,
MEM_COMMIT,
PAGE_EXECUTE_READWRITE);
// No instances from the block are used at the moment
Block^.InstCount := 0;
// Fill in the instruction bytes
Move(BlockCode,
Block^.Code,
SizeOf(BlockCode));
// Tell it where the procedure is
Block^.WndProcPtr := Pointer(CalcJmpOffset(@Block^.Code[2],
@StdWndProc));
// Now, loop, and initialize all object instances
Instance := @Block^.Instances;
Repeat
// The below 3 lines are initializing the instructions
Instance^.Code := $E8; { CALL NEAR PTR Offset }
Instance^.Offset := CalcJmpOffset(Instance,
@Block^.Code);
// This is linked list manipulation
Instance^.Prev := Nil;
Instance^.Next := InstFreeList;
If InstFreeList <> Nil Then
InstFreeList^.Prev := Instance;
InstFreeList := Instance;
Inc(Longint(Instance),
SizeOf(TObjectInstance));
Until Longint(Instance) - Longint(Block) >= SizeOf
(TInstanceBlock);
// Linked list of Instance blocks
Block^.Prev := Nil;
Block^.Next := InstBlockList;
If InstBlockList <> Nil Then
InstBlockList.Prev := Block;
InstBlockList := Block;
End;
// There's certainly some free block - let's get it, and use it!
Instance := InstFreeList;
InstFreeList := Instance^.Next;
Result := Instance;
Instance^.Method := Method;
// VirtualAlloc allocates memory on 4 KB boundary, so we are
// certain that this line will get the right block.
Block := PInstanceBlock(DWORD(Result) And Not 4095);
// Now, tell the program that we used another instance from the
// Instance block.
Inc(Block.InstCount);
{$IFDEF VCL_Synchronize}
Finally
// No need to synchronize any more.
LeaveCriticalSection(ObjInstLock);
End;
{$ENDIF}
End;

Procedure FreeObjectInstance(ObjectInstance :Pointer);
Var Block :PInstanceBlock;
Inst, Last :PObjectInstance;
Begin
If ObjectInstance <> Nil Then
Begin
{$IFDEF VCL_Synchronize}
// Synchronize if needed
EnterCriticalSection(ObjInstLock);
Try
{$ENDIF}
// Linked list manipulation
PObjectInstance(ObjectInstance)^.Prev := Nil;
PObjectInstance(ObjectInstance)^.Next := InstFreeList;
InstFreeList^.Prev := ObjectInstance;
InstFreeList := ObjectInstance;
// Get the parent block
Block := PInstanceBlock(DWORD(ObjectInstance) And Not 4095);
// Decrement the number of used instances
Dec(Block^.InstCount);
// Is any instance used?
If Block^.InstCount = 0 Then
Begin
// If not, we can free this block
// The next while loop will remove all instances
// from the Free List, and only then can the block be freed
// Since the InstCount is 0, ALL of the instances ARE
// in the free list, so we don't need to worry about
// the validity of their fields.
Inst := @Block^.Instances[0];
Last := @Block^.Instances[InstanceCount];
While Inst <> Last Do
Begin
If Inst = InstFreeList Then
Begin
InstFreeList := InstFreeList^.Next;
If InstFreeList <> Nil Then
InstFreeList^.Prev := Nil;
End
Else
Begin
Inst^.Prev^.Next := Inst^.Next;
If Inst^.Next <> Nil Then
Inst^.Next^.Prev := Inst^.Prev;
End;
Inc(Inst); // Essentially, pointers are increased by
// the size of the structure they point to:-)
End;
If Inst = InstFreeList Then
Begin
InstFreeList := InstFreeList^.Next;
If InstFreeList <> Nil Then
InstFreeList^.Prev := Nil;
End
Else
Begin
Inst^.Prev^.Next := Inst^.Next;
If Inst^.Next <> Nil Then
Inst^.Next^.Prev := Inst^.Prev;
End;
// It's now time to remove the block from the list of
// Instance blocks
If Block = InstBlockList Then
Begin
InstBlockList := InstBlockList^.Next;
If InstBlockList <> Nil Then
InstBlockList^.Prev := Nil;
End
Else
Begin
Block^.Prev^.Next := Block^.Next;
If Block^.Next <> Nil Then
Block^.Next^.Prev := Block^.Prev;
End;
// The block is no longer references in the instance
// list nor Instance block list, and therefore can
// safely be removed
VirtualFree(Block, 0, MEM_RELEASE);
End;
{$IFDEF VCL_Synchronize}
Finally
// Release the lock.
LeaveCriticalSection(ObjInstLock);
// If no more blocks are left, we can delete the critical section
If InstBlockList = Nil Then
Begin
DeleteCriticalSection(ObjInstLock);
ObjInstInit := False;
End;
// NOTE: The section and ObjInstInit are not synchronized.
// However, we don't need to worry about them, since:
// The first form to be created, is certain to be the application
// form (and it's the first time the MOI will be called)
// and at the time, no threads are going to be created.
// Thus, the critical section will be initialized, in
// thread safe environment, and can be safely used after that.
// The last instance to be freed, is certainly the
// application instance. At that time, no threads are likely to
// exist, and after that the application terminates, and so
// the critical section can be safely deleted.
End;
{$ENDIF}
End;
End;

For more information, see http://www.alfasp.com

Server Response from: ETNACDC03