While working on Twist Down Lists, a recordable MacApp
implementation of hierarchical lists, I developed several useful
debugging aids for detecting memory leaks and access faults and
managing memory usage problems. Here I describe how to use these
debugging aids for more trouble-free MacApp programming.
In the article "Displaying Hierarchical Lists" in develop Issue 18, Martin Minow
suggests that MacApp offers "flexible libraries for displaying and managing structured
data." I accepted his challenge and decided to create a Twist Down Lists application with
MacApp version 3.3.1. As complete as MacApp is, you still have to test your
application to make sure it works. Among the problems I encountered, perhaps none
were more frustrating than the insidious memory leak and the dreaded access fault.
After discovering the nth memory leak and the mth access fault in my Twist Down Lists
application, I decided that the situation was unacceptable -- there had to be a better
way!
To solve these problems, I developed several debugging techniques. These techniques
were useful to me, so I decided to share them with you in this article. Here are some of
them:
Accompanying this article on this issue's CD and develop's Web siteis the complete
Twist Down Lists application, which you can look at to see the implementation of all
the debugging aids described in this article. Also provided are two engineering notes,
"EN1 - Object Counting and Display" and "EN2 - Object Heap Discipline," which go
into the gory details of implementing these debugging aids, and copies of the four
MacApp files UObject.h, UObject.cp, PlatformMemory.h, and PlatformMemory.cp,
which I modified to incorporate the debugging aids and which you can substitute for the
original files (or similarly modify them yourself).
Most of these debugging techniques are specifically for MacApp version
3.3.1. Later versions may already incorporate similar debugging features.*
I've found object counting to be the fastest way to discover memory leaks. To maintain
a running count of the number of objects in existence, I use a global variable named
gObjectCount. Whenever a TObject is created or cloned, gObjectCount is incremented;
when a TObject is destroyed, gObjectCount is decremented. The variable is incremented
in the TObject constructor or TObject::ShallowClone and is decremented in the TObject
destructor. To print the current value of gObjectCount, I use a global function named
PrintObjectCount. You can call this function at any point in the application where you
think it's useful. In my experience, one of the best places to test the value of
gObjectCount is at the beginning of the function TYourApplication::DoSetupMenus. That
point represents a set of stable application states that you should always be able to
return to. By monitoring the value of gObjectCount as the application runs, you can
obtain a set of characteristic values for gObjectCount. Any variation in these values
should be investigated as a possible memory leak.
For example, for Twist Down Lists, the object count just after startup is 49. After a
twistDownDocument is opened and closed, this count increases to 52. This increase is a
consequence of adding a print handler to the view; a TPrintInfo and two TDependencies
objects are created but never freed. Then, if you change the font size by choosing the
Other menu item, the object count increases to 55. In this case, the TDialogTEView,
TAdornerList, and TScroller objects are created when a new font size is entered in the
TNumberText; they're never freed. Thereafter, the quiescent value of gObjectCount
remains unchanged.
By using object counting, I've discovered TObject-based memory leaks in just
minutes. To implement it, you need to make changes to UObject.h and UObject.cp, as
described in "EN1 - Object Counting and Display." Or you can include the substitute
UObject.h and UObject.cp files that I've provided.
My global function DisplayMemoryInfo displays the amount of free memory, the size of
the temporary reserve, the size of the permanent reserve, the object heap size, the
amount of memory available in the object heap, and the amount of object heap space
used. If you have a memory leak, this function can give you information about the size
of the leak. As with object counting, you can get a set of characteristic values as you
run the application. The most useful of these indicators is the amount of object heap
space used. In my experience, it makes the most sense to call this function at the
beginning of the function TYourApplication::DoSetupMenus when you also display the
object count. Realize that each time the object heap is expanded, an overhead of 20
bytes is incurred. As a result, the amount of object heap space slowly increases until
the object heap reaches its maximum size. So if you see the amount of space used in the
object heap increasing by some multiple of 20, it might just be attributable to object
heap overhead.
To implement memory display, you need to make changes to UObject.h and UObject.cp,
as described in "EN1 - Object Counting and Display." Or you can include the substitute
UObject.h and UObject.cp files that I've provided.
While object counting and memory display let you quickly discover a memory leak, it's
object display that helps you to identify the cause of the memory leak or an access
fault. Turning on object display means that when a TObject-based object is
constructed, a message -- including "who, what, and where" -- appears in the
debugging window. Likewise, when the object is destroyed, a similar message appears.
You can use a Simple Input-Output Window (SIOW) instead of your
debugger's log window to display this information if you prefer.*
When an object is created, if object display is on, the debugger log window displays a
message similar to the following:
Construct TSomeMacAppObject@ 0x2D6ACA4 Id=74 Size=108 ObjCnt = 73 #Construct TMyObject@ 0x2D6ACA4 Id=74 Size=108 ObjCnt = 73
When the object is destroyed, the log window displays a message like this:
#Destruct TMyObject@ 0x2D6ACA4 Id=74 Size=108 ObjCnt = 73 Destruct TSomeMacAppObject@ 0x2D6ACA4 Id=74 Size=108 ObjCnt = 73
Each line gives the class name of the object, its location in the object heap, its class
ID, its size in bytes, and the current value of gObjectCount. In addition, the message
tells you whether the object was created or destroyed.
So why are there two lines for construction and destruction? When an object like
TMyObject is created, its TObject constructor is executed first, followed by the
constructors for any MacApp objects in the descendant chain, ending with the
constructor for TMyObject. In other words, objects are built from the bottom up. As
each constructor does its thing, it's given the chance to display a message identifying
itself. So when a new object is created, a series of messages is displayed that identify
each stage of the construction process.
When the object TMyObject is destroyed, the process is reversed, with the destructor
for TMyObject first displaying a message identifying itself, followed by the
destructors for any MacApp objects in the descendant chain and ending with the
destructor for TObject. Objects are destroyed from the top down.
Running an application with object display on provides a wealth of information about
what an application is doing -- information that you can't get any other way. It's also a
great way to find out what MacApp is doing. As described later in the section
"Implementing Object Display," you can use flags to specify how much information to
display.
DETECTING MEMORY LEAKS
Of course, when tracking down a memory leak, you're interested in finding an object
that was created but never destroyed. To find this object, it's necessary to match object
destructions with constructions. The leftover construction is the offending object that
wasn't destroyed. You match constructions and destructions by using the addresses
provided in the object display.
Be careful when matching object destructions and constructions, because
MacApp will reuse space in the object heap. I've often seen MacApp make a
TAppleEvent, shortly thereafter free it, and then go on to make another
TAppleEvent and store it at exactly the same address.*
If your debugger allows you to save the contents of the log window, sorting it on the
address field would bunch all items with the same address together. That would make it
much easier to match destructions with constructions. If you assign each object a
serial number in its constructor, it would be even easier to do the matching.
Consider a real example. The MacApp example application IconEdit has a memory leak.
(I found the leak because I used the application as a template.) Listing 1 shows the
offending code.
______________________________
Listing 1. An example of a memory leak
void TIconDocument::DoMenuCommand (CommandNumber aCommandNumber)
{
switch (aCommandNumber) {
case cSetColor:
{
CRGBColor newColor;
CStr255 thePrompt = "Pick a new color";
if (GetColor(kBestSystemLocation, thePrompt, fColor,
newColor)) {
if (TOSADispatcher::fgDispatcher->GetDefaultTarget()
->IsRecordingOn()) {
TSetPropertyEvent *appleEvent =
new TSetPropertyEvent;
appleEvent->ISetPropertyEvent(gServerAddress,
kAENoReply, this, pColor);
CTempDesc theNewColor;
theNewColor.PutRGBColor(newColor);
appleEvent->WriteParameter(keyAEData, theNewColor);
appleEvent->Send(); // <-- the problem
}
else {
TSetColorCommand *aSetColorCommand =
new TSetColorCommand();
aSetColorCommand->ISetColorCommand(this, newColor);
PostCommand(aSetColorCommand);
}
}
}
break;
default:
Inherited::DoMenuCommand(aCommandNumber);
break;
}
}
______________________________
With object counting and display, it took only minutes to discover the leak and identify
the offending objects. Deciding how to eliminate the leak took a little longer. The leak
arises because TAppleEvent::Send returns a reply TAppleEvent and neither it nor the
TAppleEvent that was sent is freed. This leak is fixed by using the code snippet
TAppleEvent * theReply = theEvent->Send(); FreeIfObject(theEvent); FreeIfObject(theReply);
in place of
appleEvent->Send();
Listing 1 is an example of a small memory leak, only 64 bytes. Because of its small
size, it's virtually undetectable by means other than object display. These small
memory leaks are a very serious problem because they fragment the object heap.
Suppose that every time a command is executed, a 64-byte memory leak is created and
they're uniformly distributed across the object heap. Now suppose the application
needs to create an object that's too big to fit in any of the available gaps in the object
heap. Under these circumstances, the application would come to a grinding halt and the
only thing the user could do is quit and restart the application (if the computer doesn't
crash).
DETECTING ACCESS FAULTS
Discovering access faults is easy. The Power Mac Debugger loudly, almost proudly,
proclaims, "Access Fault." If luck is with you, your machine doesn't crash or lock up.
Identifying the cause of an access fault is another matter. If the access fault involves a
TObject-based object, that means the application attempted to access an object that
doesn't exist. There are two ways that can happen: Perhaps the object was created, then
destroyed, and now the application attempts to access it. Or maybe it was never created
in the first place.
Object display can help you identify the offending object by providing an ordered
record of what was created and what was destroyed. If you've been testing with object
display on, you will have become familiar with what your application is doing. Then
the trick is to single step up to the point of the access fault without failing. At that
point, you should know which object the application is attempting to access. You can
carefully examine the results of the object display to determine the source of the
problem.
Access fault of the first kind. One type of access fault, which I'll call an access
fault of the first kind, arises from creating a TObject-based object, freeing it, and
then attempting to access it. Because it was freed, it no longer exists, so attempting to
access it causes an access fault.
When I was first teaching myself about MacApp's scripting capability, I made the
mistake of taking some MacApp code out of context and using it as a template in Twist
Down Lists. It was clearly the wrong thing to do because it resulted in an access fault of
the first kind. My mistake is illustrated by the following code, which I wrote in
TTwistDownApp::GetContainedObject. I show it here so that you can try it and see for
yourself how object display helps you find the first type of access fault.
TTwistDownDocument* theTwistDownDocument = NULL; theTwistDownDocument = (TTwistDownDocument*)aDocument; theTwistDownView = theTwistDownDocument->fTwistDownView; TOSADispatcher::fgDispatcher->AddTemporaryToken(theTwistDownView); result = theTwistDownView; return result;
Of course, in due time, MacApp freed the temporary token and, later on when the
application attempted to access fTwistDownView, an access fault was generated.
Running the application with object display on clearly showed twistDownView being
destroyed: you can't miss it and you know it's wrong. Then, a little bit of single
stepping led me to the culprit. I recognized that I shouldn't have told fgDispatcher to
add fTwistDownView as a temporary token. I fixed this by deleting the statement that
tells fgDispatcher to add it, and then carried on.
Access fault of the second kind. Another kind of access fault, which I'll call an
access fault of the second kind, arises from attempting to access a TObject-based object
that was never created. The only access fault of this kind that I've encountered arose
when I ran a script that asked the application to access a document when there were no
documents. In this case, TApplication::GetContainedObject doesn't verify that the
document exists before attempting to use it.
For this situation, the problem was immediately obvious. It was easily fixed by
inserting into TTwistDownApp::GetContainedObject the code shown in Listing 2, which
makes sure the document exists before attempting to access it.
______________________________
Listing 2. A solution to the GetContainedObject problem
...
else if (desiredType == cDocument && selectionForm == formName) {
CStr255 theName;
selectionData.GetString(theName);
CNoGhostDocsIterator iter(this);
for (TDocument* aDocument = iter.FirstDocument(); iter.More();
aDocument = iter.NextDocument()) {
if (aDocument != NULL) {
CStr255 name = gEmptyString;
aDocument->GetTitle(name);
if (name == theName) {
theTwistDownDocument = (TTwistDownDocument*)aDocument;
result = theTwistDownDocument;
return result;
}
}
}
}
______________________________
This case demonstrates the wisdom of trying to break your application by attempting to
get it to do outlandish things that no sane person would try. That's precisely how I
stumbled on this one.
Access fault of the third kind. All other access faults I've defined as access faults
of the third kind: they are, strictly speaking, outside the scope of MacApp. They arise
from mistakes you made when working with the system software -- for example,
failing to clear a parameter block before using it. As a result, object display isn't quite
as helpful at tracking down these access faults as it is with finding access faults
involving TObject-based objects. If you're lucky, object display will point you in the
general area of the problem.
The MacApp application IconEdit gives us an example. Along with the other files that
accompany this article, I've provided a test script, a modified version of the IconEdit
source code, and many IconEdit documents to help you conduct the following
experiment:
When IconEdit runs out of memory while being driven by the script, it will generate
an access fault of the third kind and drop into MacsBug with a bus error. This should
occur after the 22nd document has been opened and the script is telling IconEdit to
open the 24th saved document. The 23rd document has failed to open for lack of
memory, and the application is attempting to recover, yet the script has gone beyond
that point and is telling the application to open the 24th document. (Note that you'd
need to use a lot more documents to duplicate this condition if you didn't compile with
the substitute PlatformMemory files, which implement object heap discipline, as
described later.)
The problem occurs when the application attempts to return an out-of-memory
message to the script. Specifically, TAppleEvent::WriteLong generates another error
when it attempts to complete the reply Apple event that's supposed to tell the script
about the out-of-memory condition.
This is a case where the techniques described in this article don't help you very much
and may actually hinder you. I didn't do anything wrong, but I did spend time proving
that I didn't do anything wrong. Once I got assurance that all the TObject-descended
objects in my application appeared to be OK, I decided to see if IconEdit had the same
problem. It did.
In due time, I noticed that the TServerCommand constructor sets fSuspendTheEvent to
FALSE. TServerCommand is an ancestor of TODocCommand, which is responsible for
opening existing documents. At that point, I had nothing to lose by setting its value to
TRUE. That fixed the problem.
Be warned that the experts will tell you that fSuspendTheEvent should
never be set to TRUE because doing so can be dangerous. I've disregarded their
advice with no better rationale than that it appears to allow IconEdit and Twist
Down Lists to survive without crashing when I run a script that sends open
document commands until the application fails for lack of memory. As of this
writing I haven't found a more acceptable workaround.*
IMPLEMENTING OBJECT DISPLAY
To implement object display, you can plug in the substitute UObject.h and UObject.cp
files. (Implementation details are provided in "EN1 - Object Counting and Display.")
In addition, four new methods need to be added to TObject:
#if qDebug void TObject::PrintConstructorClassInfo(); void TObject::PrintDestructorClassInfo(); void TObject::PrintAppConstructorClassInfo(); void TObject::PrintAppDestructorClassInfo(); #endif
The constructors and destructors of all objects for which you want to be able to display
object information must be modified to call the appropriate method. For MacApp
objects, use the following code in the constructor:
#if qDebug this->PrintConstructorClassInfo(); #endif
and this code in the destructor:
#if qDebug this->PrintDestructorClassInfo(); #endif
You may not want to call these methods in TEvent and TToolboxEvent. The Macintosh
specializes in generating events, so displaying object information for them can be
overwhelming. In your application objects, use the following code in the constructor:
#if qDebug this->PrintAppConstructorClassInfo(); #endif
and this code in the destructor:
#if qDebug this->PrintAppDestructorClassInfo(); #endif
These calls should always be placed in the same relative position in constructors and
destructors. The very beginning or the very end are the two most obvious choices. Keep
in mind that although constructors don't generally make other objects, destructors
frequently free other objects. If these methods are invoked at random places in the
constructors and destructors, the resulting object information displayed in the log
window will be very hard to interpret.
There are three flags that you can use to control the amount of object information
that's displayed: gPrintBaseClassInfo, gPrintMacAppClassInfo, and gPrintAppClassInfo.
These flags determine whether object information is displayed at the TObject level, for
MacApp objects, or for your application's objects, respectively. All three flags can be
set with scriptable menu commands. However, it's probably best to set
gPrintBaseClassInfo programmatically to avoid being inundated with object
information for every TToolboxEvent that's generated. Simply surround the suspect
code as follows:
gPrintBaseClassInfo = TRUE; ... // suspect code here gPrintBaseClassInfo = FALSE;
In my experience, it's usually enough to display object information at the application
level and the MacApp level. However, some MacApp objects don't have constructors and
some don't have destructors. If you suspect that those objects are the source of a
problem, it may be useful to display object information at the TObject level.
MacApp uses its own object heap to store TObject-derived objects. The heap grows as
new objects are created -- by taking memory from free memory. Once memory has
been allocated to the object heap, it's never returned to free memory. As things stand,
the developer has little control over this situation. According to conventional wisdom,
the point of greatest memory usage occurs during printing. With Twist Down Lists,
memory usage problems occur when the application runs out of memory while loading
a hierarchical list, especially with 680x0 versions. That meant I had two problems to
deal with while testing recovery from an out-of-memory condition when loading a list:
the recovery itself and the lack of available memory in which to load required code
segments. I made the segment loading problem go away by implementing object heap
discipline, which let me concentrate on testing failure recovery. Object heap
discipline allows you to erect a barrier to further expansion of the object heap right
where you want it. At the same time, it allows you to leave as much memory as is
required to load code segments without having to fuss with 'seg!' and 'res!' resources.
When the object heap runs out of space, a request for a new block of memory is made
with a call to the global function PlatformAllocateBlock(size_t size). The trick is to
force PlatformAllocateBlock to reject the request when you want it to.
To do that, I created the global down-counter gOHRemainingIncrements to maintain a
count of the number of times the object heap will be allowed to expand. Each time
PlatformAllocateBlock allocates memory to the object heap, it decrements
gOHRemainingIncrements. When gOHRemainingIncrements reaches 0,
PlatformAllocateBlock will no longer honor requests for additional memory. The
revised version of PlatformAllocateBlock is shown in Listing 3.
______________________________
Listing 3. Revised PlatformAllocateBlock
void *PlatformAllocateBlock(size_t size)
{
Boolean heapPerm;
if (gUMemoryInitialized)
heapPerm = PermAllocation(TRUE);
void *ptr = NULL; // added
// void *ptr = NewPtr(size); // commented out
if (gOHRemainingIncrements > 0) { // added
ptr = NewPtr(size); // added
gOHRemainingIncrements--; // added
} // added
if (gUMemoryInitialized)
PermAllocation(heapPerm); // Reset perm flag before
// possible Failure
FailNIL(ptr);
return ptr;
}
______________________________
The initial value of gOHRemainingIncrements is set to 3 just to be safe. During
initialization, MacApp makes two allocations to the object heap; if
gOHRemainingIncrements is 0, the application doesn't start up because of lack of
memory. The second of those allocations sets up the initial size of the object heap. If
your 'mem!' resource specifies a small value for the initial size of the object heap, the
initial value of gOHRemainingIncrements might have to be larger than 3.
Immediately following the call to InitUMacApp in main, the value of
gOHRemainingIncrements is set with a call to the InitMaxObjectHeapSize global
function, which is shown in Listing 4.
gOHRemainingIncrements = InitMaxObjectHeapSize();
______________________________
Listing 4. Determining the number of times the object heap will be allowed to expand
short InitMaxObjectHeapSize()
{
long freeMem = FreeMem();
Size heapSizeIncrement = gSizeHeapIncrement;
short theNumber = 0;
if (freeMem > kFreeMemReserve)
theNumber = (freeMem - kFreeMemReserve)/heapSizeIncrement;
if (theNumber >= 1)
theNumber = theNumber -1;
else
theNumber = 0;
return theNumber; // The number of times we'll let the object
// heap be expanded
}
______________________________
As you can see, I use a very simple algorithm to determine the number of times the
object heap will be allowed to expand and still leave in free memory at least the
number of bytes specified by kFreeMemReserve.
The changes you need to make to MacApp to implement object heap discipline are
described in detail in "EN2 - Object Heap Discipline." The substitute
PlatformMemory.h and PlatformMemory.cp files are also provided.
One very good reason to use MacApp is its integrated failure handling scheme. Of
course, all failure recovery paths must be tested. In a well-crafted application,
failures should occur only while the application is attempting to create a new object
when there's insufficient space in the object heap for it. To test these situations, the
application must be forced to fail at selected points. It's not enough to adjust the
partition size and hope for a failure.
Ideally, you would be able to set a failure point with a debugger in a similar manner to
setting a breakpoint. That's not presently possible. Instead, in Twist Down Lists, I
added a global flag gFailHere, which is set and cleared with a scriptable menu
command. There are several ways to use this flag:
if (gFailHere)
Failure(errFailHere, 0);
TSomeObject* someObject = new TSomeObject;
FailNIL(someObject);
someObject->ISomeObject();
if (gFailHere)
Failure(errFailHere, 0);
TSomeObject::ISomeObject()
{
this->IObject();
if (gFailHere)
Failure(errFailHere, 0);
}
Other ways of using this technique to force a failure require application-specific
knowledge. In the case of Twist Down Lists, it's often convenient to give the name
FailHere to a file or folder on the volume you're going to open. With the following code,
when a twistDownElement named FailHere is encountered, the failure will be
triggered:
if (gFailHere) {
CStr63 failHereText = "FailHere";
CStr63 displayedText = gEmptyString;
twistDownElement->GetDisplayedText(displayedText);
if (failHereText == displayedText)
Failure(errFailHere, 0);
}
It must be possible to set and clear the gFailHere flag from a script. An application can
encounter the same failure conditions whether driven from a script or from the user
interface. The failure recovery path is, however, a little different. When the
application is being driven by a script, an Apple event must be sent to the script
telling it that a failure was encountered and what the failure was. MacApp will handle
the overhead, but you must do your part: you must test it to make sure it works and
returns appropriate error information to the script.
Were it not for the fact that all the list processing methods in TTwistDownDocument
are recursive, I probably wouldn't have felt the need to implement gFailHere. Failure
can occur if the application attempts to make a twistDownElement or a
twistDownControl when there isn't enough memory in the object heap to do it and the
object heap can't be further expanded. The failure might occur several levels into the
recursion. You can't call ReSignal to handle the failure because you'll jump all the way
up to the method that started the recursion. Instead, you must save the failure
information, work your way back up the recursion, and then signal the failure.
Using gFailHere turned out to be the best way to test the failure handling. Object
counting, memory display, and object display were very useful in testing recovery
from these kinds of failures. Object counting and memory display verified that
everything that needed to be freed was freed. Using object display to match
constructions with destructions gave further confirmation that the recovery was
successful.
The debugging aids I developed illustrate the power of MacApp. By modifying the
central organizing object, TObject, you can make many new capabilities, such as
object counting, memory display, and object display, extend to the objects that inherit
from it. In addition, you can easily modify the memory management scheme of MacApp,
so implementing object heap discipline isn't hard at all. Now that I have these
debugging aids, I no longer fear the dreaded memory leak and access fault. Object
counting, memory display, and object display don't exactly sound an alarm when
there's a TObject-based memory leak, but they come pretty close. And without object
display, finding an access fault was like looking for a needle in a haystack.
The faster you can fix your mistakes, the faster you can finish your applications. I
hope my debugging aids will help you get those applications out even quicker.
RELATED READING
CONRAD KOPALA (ckopala@aol.com) believes you should never trust a computer
you can't program. He's been a student of MacApp for the last six years and just
recently thinks he might know a smidgen about it. In the past, Conrad was an electrical
engineering professor and held positions with IBM and MCI. Now he does whatever he
wants.*
Thanks to our technical reviewers Tom Becker, Geoff Clapp, Mike Rossetti, Merwyn
Welcome, and Jason Yeo.*