Global Subclassing

Global subclassing revolves around the same principles as instance subclassing. With global subclassing, though, we are going to go one level deeper before performing the subclassing. By this I mean that the modifications to the window procedure function pointer will occur in the class itself, not the individual window created from the class. Only those windows created from the class after the subclassing has occurred will use the new subclassed window procedure. When I mention subclassing in this section, I am referring to global subclassing.

Tip

Instance subclassing affects a specific window and its window procedure. Global subclassing affects every window created from a window class that has had its class window procedure modified.

Unlike instance subclassing, global subclassing can capture the window creation messages -- and, more specifically, WM_CREATE and WM_NCCREATE. When a window is created, the WM_NCCREATE message is sent first to the window to finish creating its nonclient area in memory. The WM_CREATE message is sent next to finish creating the window’s client area in memory. Note that the window is still not displayed on the screen at this point. Other messages still need to be sent to the window to position, size, and paint it on the desktop.

The reason that instance subclassing cannot capture these messages is that when instance subclassing occurs, the window has already been created. Therefore, the window creation messages have already been processed. Global subclassing, on the other hand, occurs before a window is created. The subclassed window procedure is in place at the point when the window is created. Therefore, all window creation messages are captured.

Tip

Global subclassing allows the window creation messages to be processed. Instance subclassing does not have this ability.

To see how global subclassing differs from instance subclassing, we’ll examine the modifications needed to change the previous instance subclassing example (rolling up the form using the window procedure shown in Example 4-10) to perform global subclassing.

Changes to the Class

The only real change to the class module is that you must swap SetWindowLongPtr for the SetClassLongPtr API function. This change will provide the lpfnWndProc member of the class structure with a pointer to our new window procedure. Along with using this new API function, there is also a new constant, GCLP_CLSPROC. This constant tells SetClassLongPtr to modify the lpfnWndProc function pointer in the class structure to point to a new window procedure that we define.

To call SetClassLongPtr, we need to replace the following declarations and constants:

Private Declare Function SetWindowLongPtr Lib "user32" Alias "SetWindowLongA" _
		(ByVal hwnd As Long, _
		ByVal nIndex As Long, _
		ByVal dwNewLong As Long) As Long

Private Const GWLP_WNDPROC = -4

with these:

Private Declare Function SetClassLongPtr Lib "user32" Alias "SetClassLongA" _
	(ByVal hwnd As Long, ByVal nIndex As Long, ByVal dwNewLong As Long) As Long

Private Const GCLP_CLSPROC = (-24)

In addition, you must change the EnableSubclass function in our class module to use:

m_lOrigClassProc = SetClassLongPtr(m_hwnd, GCLP_CLSPROC, _
		AddressOf Module1.NewClassProc)

instead of:

m_lOrigClassProc = SetwindowlongPtr(m_hwnd, GWLP_WNDPROC, _
		AddressOf Module1.NewWndProc)

The same must be done for the DisableSubclass function. It should use:

SetClassLongPtr m_hwnd, GCLP_CLSPROC, m_lOrigClassProc

instead of:

SetWindowlongPtr m_hwnd, GWLP_WNDPROC, m_lOrigClassProc

Additionally, all references to the m_lOrigWndProc member variable should be changed to m_lOrigClassProc.

The SetClassLongPtr API function allows us to modify the structure of a class as long as we have a handle to the window that was created from this class. This means that before we can globally subclass a window class, we must first create a new window from that class. The window handle from this window is used in the first argument of the SetClassLongPtr function.

This example will create the window before globally subclassing the class. Usually, this window is hidden from the user so that the user does not inadvertently close the window and destroy its handle. This handle is also what we need to remove the global subclassing and restore the application to its initial state. Otherwise, the application would crash.

Tip

Global subclassing requires that a window, usually hidden, is created and its hWnd used for global subclassing. This window must remain in memory for the life of the application.

A minor modification is needed in the DisableSubclass function. The modification might seem minor, but it will have a big impact on whether the example will work. Look at the following line in the DisableSubclass function:

m_lOrigClassProc = 0

This line must be removed. If the reason for this is not apparent at first, let me explain. The object created from the CSubclass class holds the window handle (m_hwnd) of the original window we created before modifying the class. We need to keep this handle for the lifetime of the application. We also have the pointer to the original class procedure (m_lOrigClassProc), which we need to keep for the lifetime of the application. This is because when we modify a class through global subclassing and then create new windows from the modified class, those new windows can exist for the lifetime of the application.

When a message is sent to the globally subclassed window, that window will look up its own modified window procedure and call it. In this example, that window procedure would be the NewClassProc function in our BAS module. The problem doesn’t lie here, though. It lies at the end of the window procedure, where we try to call the original class procedure. Remember, the class was subclassed, not the window.

When a window is created from the modified class, the window procedure it receives from the class is considered to be that window’s original window procedure. If we had intentionally removed the global subclass and the code had set the m_lOrigClassProc variable to (as it would in instance subclassing), CallWindowProc would try to dereference a NULL pointer to get the address of the original window procedure. You might have thought that just because the global subclassing had been removed, all windows would suddenly revert to calling their original window procedure. This is not the case because the lpfnWndProc member of the window structure of preexisting windows is not modified by the call to SetClassLongPtr. Whenever a message is sent to a globally subclassed window, it still tries to call our NewClassProc function and, in turn, the original window procedure (indicated by m_lOrigClassProc). This is illustrated in Figure 4-5. This diagram will be discussed in more detail later in this chapter.

The lifetime of the globally subclassed example application where WndProc1 is the window procedure of the unmodified class and WndProc2 is the window procedure of the modified class

Figure 4-5. The lifetime of the globally subclassed example application where WndProc1 is the window procedure of the unmodified class and WndProc2 is the window procedure of the modified class

These are the only modifications that we will make to this class.

Tip

Do not lose the hWnd and original window procedure contained in the CSubclass class when implementing global subclassing.

The final code for the class is shown in Example 4-11, with revised lines in boldface.

Example 4-11. Modified CSubclass.cls Module for the Global Subclassing Example

#If Win32 Then
    Private Declare Function SetWindowLongPtr Lib "user32" Alias "SetWindowLongA" _
		(ByVal hwnd As Long, ByVal nIndex As Long, ByVal dwNewLong As Long) As Long
    Private Declare Function SetClassLongPtr Lib "user32" Alias "SetClassLongA" _
		(ByVal hwnd As Long, ByVal nIndex As Long, ByVal dwNewLong As Long) As Long
    m_lOrigClassProc = SetClassLongPtr(m_hwnd, GCLP_WNDPROC, _
                                       AddressOf Module1.NewClassProc)
    Private Const GWLP_USERDATA = (-21)
    Private Const GWLP_HWNDPARENT = (-8)
    Private Const GCLP_MENUNAME = (-8)
    Private Const GCLP_HBRBACKGROUND = (-10)
    Private Const GCLP_HCURSOR = (-12)
    Private Const GCLP_HICONSM = (-34)
    Private Const GCLP_HMODULE = (-16)
    Private Const GCLP_WNDPROC = (-24)
    Private Const DWLP_MSGRESULT = 0
    Private Const DWLP_DLGPROC = 4
    Private Const DWLP_USER = 8
#ElseIf Not Win16 And Not Win32 Then
    Private Declare Function SetWindowLongPtr Lib "user32" Alias "SetWindowLongPtrA" _
         (ByVal hwnd As Long, ByVal nIndex As Long, ByVal dwNewLong As Long) As Long
    Private Declare Function SetClassLongPtr Lib "user32" Alias "SetClassLongPtrA" _
         (ByVal hwnd As Long, ByVal nIndex As Long, ByVal dwNewLong As Long) As Long
    Private Const GWLP_HINSTANCE = (-6)
    Private Const GWLP_WNDPROC = (-4)
    Private Const GWLP_USERDATA = (-21)
    Private Const GWLP_HWNDPARENT = (-8)
    Private Const GWLP_ID = (-12)
    Private Const GCLP_MENUNAME = (-8)
    Private Const GCLP_HBRBACKGROUND = (-10)
    Private Const GCLP_HCURSOR = (-12)
    Private Const GCLP_HICONSM = (-34)
    Private Const GCLP_HMODULE = (-16)
    Private Const GCLP_WNDPROC = (-24)
    Private Const DWLP_MSGRESULT = 0
    Private Const DWLP_DLGPROC = 8
    Private Const DWLP_USER = 16
#End If

Private m_lOrigClassProc As Long
Private m_hwnd As Long

Public Property Get OrigWndProc(  ) As Long
    OrigWndProc = m_lOrigClassProc
End Property

Public Property Let hwnd(Handle As Long)
    m_hwnd = Handle
End Property

Private Sub Class_Initialize(  )
    m_lOrigClassProc = 0
    m_hwnd = 0
End Sub

Public Function EnableSubclass(  ) As Boolean
    If m_lOrigClassProc <> 0 Then
        'Already subclassed
        '  Do not allow to subclass a 2nd time
    Else
        m_lOrigClassProc = SetClassLongPtr(m_hwnd, GCLP_WNDPROC, AddressOf Module1.
NewClassProc)
    End If
    
    If m_lOrigClassProc <> 0 Then
        EnableSubclass = True
    Else
        EnableSubclass = False
    End If
End Function

Public Function DisableSubclass(  ) As Boolean
    'Remove global subclass
    If m_lOrigClassProc = 0 Then
        'Do not remove subclass - none exist
        DisableSubclass = False
    Else
        SetClassLongPtr m_hwnd, GCLP_WNDPROC, m_lOrigClassProc

'       DO NOT SET THIS TO ZERO (AS WITH INSTANCE SUBCLASSING)
'         OR WE WILL LOSE OUR NEW WINPROC
'       m_lOrigClassProc = 0

        DisableSubclass = True
    End If
End Function

Private Sub Class_Terminate(  )
    Call DisableSubclass
End Sub

Changes to the BAS Module

The subclassed window procedure in this module will force each globally subclassed window to roll up instead of minimize; this is similar to the last example in Section 4.3 of this chapter.

Some code needs to be added to this subclassed window procedure for it to work on the correct window. This code will iterate through every window belonging to this application and try to find the window handle that matches the one passed in to this function. This will be the window that the subclassed window procedure will operate on. The code modification is as follows:

Dim I As Integer
Dim FocusWindow As Long

FocusWindow = 0
For I = 0 To Forms.Count - 1
    If Forms(I).hwnd = hwnd Then
        FocusWindow = I
        Exit For
    End If
Next I

Global subclassing will allow more than one window to use the same subclassed window procedure. This can be very powerful because it allows us to add functionality to a number of similar windows without having to write subclassing code for each separate window, as we would with instance subclassing. The problem is that all the windows that have been globally subclassed will use the same window procedure (NewClassProc). So the trick is to determine which window the message is bound for. This is easier than it looks. Remember, we are always passed a hWnd window handle in our subclassed window procedure. We can use the hwnd argument to set up a loop iterating through each window of the project to determine which window we need to work with; of course, this is not the optimal solution. Whenever we need to set the form height property of this window, for example, we could simply use the following code:

Forms(FocusWindow).Height = 30

The window procedure then needs to be modified so that we perform our operations on the window in the FocusWindow ordinal position in the Forms collection. Example 4-12 shows the complete window procedure, with revised lines indicated in boldface.

Example 4-12. The Window Procedure for the Global Subclassing Example

                     Public Function NewClassProc(ByVal hwnd As Long, ByVal uMsg As Long, _ 
                ByVal wParam As Long, ByVal lParam As Long) As Long
    
    Dim I As Integer
    Dim FocusWindow As Long

    NewClassProc = CallWindowProc(CSubClsApp.OrigWndProc, hwnd, uMsg, _
                                  wParam, lParam)
    For I = 0 To Forms.Count - 1
        If Forms(I).hwnd = hwnd Then
            FocusWindow = I
            Exit For
        End If
    Next I
    
    'Modify the windows default processing if necessary
    If uMsg = WM_SYSCOMMAND And FocusWindow <> 0 Then
        If wParam = SC_MINIMIZE Then
            If Forms(FocusWindow).WindowState <> vbMaximized Then
                'Do not process message - instead do our own work
                Forms(FocusWindow).Height = 30
            End If
            NewClassProc = 0
        Else
            NewClassProc = CallWindowProc(CSubClsApp.OrigWndProc, hwnd, uMsg, _
                                          wParam, lParam)
        End If
    Else
        'Pass message to default handler
        NewClassProc = CallWindowProc(CSubClsApp.OrigWndProc, hwnd, uMsg, _
                                      wParam, lParam)
    End If
End 
Function

Tip

All globally subclassed windows will call the same subclassed window procedure in your BAS module. This means that your window procedure must be able to determine which window the message is intended for.

Changes to the Form

The number of code changes that must be made to the form are substantial. Most notably, the order of events needed to install and uninstall the global subclass must be changed. Also, there is a new button on the form called Create Wnd. This button will create a form; we use this form’s handle to initiate global subclassing. The main form looks the same as the form used in the instance subclassing examples, except for the addition of this new button. In addition to these changes, a brand-new form named frmGlobalSub is added to the project. This new form has no code associated with it, and only its Caption property is changed to read “Globally Subclassed Form”. This new form is used to provide the hWnd that is used in the call to SetClassLongPtr.

Example 4-13 shows the code behind the new button, which is named cmdCreateWnd. This button-click event first creates a new form from the frmGlobalSub form that was added to the project. In a real-life application, this window would be hidden from the user. Next, an instance of the CSubclass class is created and initialized with the handle of the window just created. Finally, this button is disabled. This is to remove the user’s ability to create another window and a new CSubclass object.

Example 4-13. The Create Wnd Button’s Click Event Procedure

Private Sub cmdCreateWnd_Click(  )
	'Create new form
    Dim CGSubForm As New frmGlobalSub
	CGSubForm.Caption = "Original Window"
    CGSubForm.Visible = True

    'Create subclassing object
    Set CSubClsApp = New CSubclass
	CSubClsApp.hwnd = CGSubForm.hwnd

    'Do not create a second instance, we will lose the original hwnd
    cmdCreateWnd.Enabled = False
End Sub

The Click event procedure for the Subclass button is entirely new; it is shown in Example 4-14. The code checks to make sure that a window has been created that can be used to initiate global subclassing. This is done by making sure that the cmdCreateWnd button has been disabled. If it has been disabled, the EnableSubclass public method in the CSubclass object is called. This method will use SetClassLongPtr to modify the lpfnWndProc member of the class structure. The last thing this method does is create a new form using the modified class.

Example 4-14. The cmdSubclass_Click Event Procedure

Private Sub cmdSubclass_Click(  )

   Dim NewForm As frmGlobalSub

	If cmdCreateWnd.Enabled = False Then
        Call CSubClsApp.EnableSubclass
        
        Set NewForm = New frmGlobalSub
        NewForm.Show
	Else
        MsgBox "Click on the 'Create Wnd' button to create a window to subclass."
    End If
End Sub

This button can be clicked multiple times to create multiple globally subclassed windows without crashing the application. This is because the EnableSubclass method will call the SetClassLongPtr API function only once. After this method has successfully been called, it knows not to call SetClassLongPtr a second time. Calling SetClassLongPtr a second time would cause us to lose the function pointer (m_lOrigClassProc) to the original window procedure in the class.

The Un-Subclass button still calls the DisableSubclass public method of the CSubclass class. The only difference is that it now checks to see if the cmdCreateWnd button has been disabled before calling the DisableSubclass method. Its single line of code then appears as follows:

If Not Me.cmdCreateWnd.Enabled Then Call CSubClsApp.DisableSubclass

All the code in the Form_Load event should be removed.

Next, add the following code before the existing code in the Form_QueryUnload event procedure:

'Remove all child windows to prevent a GPF
Dim I As Integer
For I = Forms.Count - 1 To 0 Step -1
	If Forms(I).Caption <> "Chapter 4 - Subclassing Example" And _
			Forms(I).Caption <> "Original Window" Then
	   	Unload Forms(I)
    End If
Next I

This event still destroys the CSubClsApp object, but certain processing must now happen before and after the CSubClsApp object destruction. Before the class is destroyed, all globally subclassed windows must be unloaded. This code destroys only the globally subclassed windows; destroying any other window would cause our application to crash. Because we know what the window captions will contain, it is a simple matter of looping through all the current windows in the application and only unloading the ones that have been globally subclassed. We do not want to unload the main form because it is already in the process of being unloaded. Neither do we want to unload the original window because, as I mentioned earlier, we would lose our window handle (m_hwnd) that is needed to release the global subclassing.

Instead of using the form’s caption to determine if the window has been globally subclassed, you might want to store an array of window handles. You can then loop through this array and unload each window in the array. When using this method, make absolutely certain that you unload every globally subclassed window or your application could crash.

After the class has been destroyed, the original window can be destroyed. The following code, which should be added to the end of the Form_QueryUnload procedure, does just that:

'Remove the original window to clean up
For I = Forms.Count - 1 To 0 Step -1
	If Forms(I).Caption = "Original Window" Then
    	Unload Forms(I)
    End If
Next I

Tip

Unload all globally subclassed forms before destroying the CSubclass object. Unload the original window that is used to get the hWnd to initiate global subclassing only after destroying the CSubclass object.

Example 4-15 shows the complete window procedure, with revised lines indicated in boldface.

Example 4-15. The Modified frmCh4.frm Module for the Global SubclassingExample

                     Private Sub cmdCreateWnd_Click(  )
    'Create new form
    Dim CGSubForm As New frmGlobalSub
    CGSubForm.Caption = "Original Window"
    CGSubForm.Visible = True
    
    'Create subclassing object
    Set CSubClsApp = New CSubclass
    CSubClsApp.hwnd = CGSubForm.hwnd

    'Do not create a second instance, we will lose the original hwnd
    cmdCreateWnd.Enabled = False
End Sub

Private Sub cmdSubclass_Click(  )
    If cmdCreateWnd.Enabled = False Then
        Call CSubClsApp.EnableSubclass
        
        Set NewForm = New frmGlobalSub
        NewForm.Show
    Else
        MsgBox "Click on the 'Create Wnd' button to create a window to subclass."
    End If
End Sub

Private Sub cmdUnSubclass_Click(  )
    If cmdCreateWnd.Enabled = False Then
        Call CSubClsApp.DisableSubclass
    Else
        MsgBox "Click on the 'Create Wnd' button to create a window to subclass."
    End If
End Sub

Private Sub Form_QueryUnload(Cancel As Integer, UnloadMode As Integer)
    'Remove all child windows to prevent a GPF
    Dim I As Integer
    For I = Forms.Count - 1 To 0 Step -1
        If Forms(I).Caption <> "Chapter 4 - Subclassing Example" _
	      And Forms(I).Caption <> "Original Window" Then
            Unload Forms(I)
        End If
    Next I
    
    'Make sure class is destroyed here
    Set CSubClsApp = Nothing

    'Remove the original window to clean up
    For I = Forms.Count - 1 To 0 Step -1
        If Forms(I).Caption = "Original Window" Then
            Unload Forms(I)
        End If
    Next I
End Sub

Quite a few code changes and additions were made to the form module, but global subclassing does operate a bit differently than instance subclassing.

How It All Works

When you run this code, the global subclassing application is easy to use; here are the steps you follow to use this application:

  1. Click the Create Wnd button.

  2. Click the Subclass button. (This button can be clicked multiple times to create globally subclassed windows.)

  3. Closing the application can be done in one of two ways:

    1. Click the Un-Subclass button and then close the application.

    2. Close the application without clicking the Un-Subclass button.

When you click the Create Wnd button, a new window is created with the caption “Original Window”. This window’s handle is needed by SetClassLongPtr to initiate global subclassing. This is also the window that is usually hidden from the user. Now the application has all the information it needs for global subclassing.

When the user clicks the Subclass button, a new window is created and displayed. This window has the caption “Globally Subclassed Form”. Here’s the interesting part. When you click the Minimize button on the title bar of the window with the caption “Original Window”, the window is minimized as it normally would be. When you click the Minimize button for the window with the caption “Globally Subclassed Form”, the form rolls up instead of being minimized. That’s not all. If you click the Subclass button repeatedly, new windows will be displayed with the caption “Globally Subclassed Form”. Each new window will also roll up when the Minimize button is clicked.

Instead of performing instance subclassing on each window separately, which would be a lot of work, we have altered the base functionality of every window to suit our needs. Clicking the Un-Subclass button and closing the application will destroy all the created windows and restore the original lpfnWndProc function pointer in the form window class.

Now that you know how to use this example application, let’s take a look at what is going on behind the scenes. Nothing of real interest happens when the application is first started. Clicking the Create Wnd button only sets up the application for global subclassing. At this point, no subclassing of any kind has occurred. The application will be displayed on the screen similar to Figure 4-6.

Screenshot of the example application after clicking the Create Wnd button

Figure 4-6. Screenshot of the example application after clicking the Create Wnd button

Nothing much has happened so far. Neither window has been modified, so they each will use their own default window procedure. The state of the application can be seen in Figure 4-7.

State of the application after clicking the Create Wnd button

Figure 4-7. State of the application after clicking the Create Wnd button

The next step is to click the Subclass button. This creates a new window, and the application now appears as in Figure 4-8.

Screenshot of the example application after clicking the Subclass button

Figure 4-8. Screenshot of the example application after clicking the Subclass button

The code for the Subclass button calls the EnableSubclass public method of the CSubclass class. The EnableSubclass method will use SetClassLongPtr to modify the class that was used to create the original window (from here on out, when I use the term “original window” I am referring to the window with the caption “Original Window”). The class that this function modifies is ThunderRT6FormDC. The SetClassLongPtr API function requires a window handle (m_hwnd) to modify the function pointer to the class window procedure. This window handle must be taken from a window created from the class that we want to globally subclass.

After calling SetClassLongPtr, the class’ lpfnWndProc function pointer to its window procedure is now pointing to our new subclassed window procedure, NewClassProc. As with instance subclassing, we retain the original window procedure in the m_lOrigClassProc member variable so that we can restore the class to its original state.

Because the class was globally subclassed after the original window was created, the original window will continue using the pointer to the class’s original window procedure (this is the same pointer that is contained in the m_lOrigClassProc member variable). When the original window was created, the class information used to create this window was copied into the window structure. The pointer to this window’s window procedure is no longer tied to the class’s window procedure. Therefore, it remains unchanged.

The second thing that the Subclass button does is create a new window from the same form object from which the original window was created. This new window uses the function pointer to the window procedure that is now contained in the modified class. Therefore, when you click the Minimize button for the original window, the window is actually minimized the way Windows intended, and when you click the Minimize button of the new window, it rolls up instead of minimizes. Figure 4-9 shows the state of the application at this point.

State of the application after clicking the Subclass button

Figure 4-9. State of the application after clicking the Subclass button

Each time the Subclass button is clicked, a new window is created. This new window will use the same class that we previously modified. Therefore, each new window will roll up as well.

The next step is to click the Un-Subclass button. The screen should look something similar to Figure 4-10. The code behind this button calls the DisableSubclass public member function of the CSubclass class. The only action DisableSubclass performs is to call SetClassLongPtr, which removes the global subclass. If you observe the functionality of each created window, you will notice that the way they function has not changed, even though the global subclass has been removed. This is because only the class structure has changed; not the window structure of each existing window. If you minimize the original window, it still works the way the Windows operating system intended it to work. Each new window created after the Subclass button was clicked retains the roll-up functionality.

Tip

Global subclassing affects the class structure, not the window structure of existing windows.

Screenshot of the application after clicking the Un-Subclass button

Figure 4-10. Screenshot of the application after clicking the Un-Subclass button

There is no reason for this application to crash at this point. The subclassed window procedure NewClassProc is still available at the same location. Therefore, no function pointers are pointing to invalid memory locations. We still have the member variable that contains the original class’s window procedure (m_lOrigClassProc). That way, when our subclassed window procedure, NewClassProc, calls CallWindowProc to pass on the message, a valid window procedure is found. The modified window class has been changed back to use its original class window procedure, which is also still valid.

The interesting thing to do now is click the Subclass button again. Because the m_lOrigClassProc member variable still contains the class’s original window procedure, the code will not allow SetClassLongPtr to be called a second time. This way we are assured of not losing the original class’s window procedure. After bypassing the call to set another global subclass, the code creates a new window. This window inherits its properties from the restored window class, which now has its original class window procedure. The newly created window uses this window procedure as its default. Clicking the Minimize button of this new window will minimize it the way Windows intended it to be minimized. This is shown in Figure 4-11.

State of the application after clicking the Un-Subclass button (the Globally Subclassed Window #2 is created after removing the global subclass with the SetClassLongPtr API function; notice that it uses the unmodified class’s window procedure as its own)

Figure 4-11. State of the application after clicking the Un-Subclass button (the Globally Subclassed Window #2 is created after removing the global subclass with the SetClassLongPtr API function; notice that it uses the unmodified class’s window procedure as its own)

It is also important to note that if the application is closed before the global subclass is removed, the CSubclass class will automatically remove the global subclass when the class is destroyed. This prevents our application from crashing.

Behind the Scenes with Spy++

To see how this application works from the inside, let’s use the Spy++ utility to spy on the global subclassing example. For this purpose, you should use the application’s compiled EXE, rather than running it from the development environment. The information that you see in Spy++ for an application that is running in the development environment most likely will not be correct because of the way VB handles the AddressOf operator while running an application in the IDE. Note, though, that when running this example on your computer you will not see the memory addresses and handles that I use here. I am using these values to show you the relationships between classes and their windows.

First, let’s start up the global subclassing example and then run Spy++. View the application’s windows by selecting the Spy Windows menu item or by pressing Ctrl-W. Find the line which has the window caption followed by the word ThunderRT6Main (i.e., “Chapter 4 -- Subclassing Example” ThunderRT6Main). I will refer to this window as the main form. At this point, the main window is displayed, and the information we gather from the General and Class tabs of the Windows Properties dialog box is as follows:

Window Caption = "Chapter 4 - Subclassing Example"
hWnd = 00190444
Wnd Proc = 6601FFCB
Class Name = ThunderRT6FormDC
Class Wnd Proc = 6601FFCB

A window has been created and displayed; that is about it. Next, click the Create Wnd button on the main form, then switch to Spy++ and press F5 to refresh the Spy++ display. A new window is displayed with the following attributes:

Window Caption = "Original Window"
hWnd = 00460536
Wnd Proc = 6601FFCB
Class Name = ThunderRT6FormDC
Class Wnd Proc = 6601FFCB

The main window’s attributes remain the same as before you clicked the Create Wnd button. Now everything is in place for us to successfully perform global subclassing. Here comes the interesting part. Click the Subclass button on the main form. Now let’s take a look at the attributes of all three windows:

Window Caption = "Chapter 4 - Subclassing Example"
hWnd = 00190444
Wnd Proc = 6601FFCB
Class Name = ThunderRT6FormDC
Class Wnd Proc = 004050C5

Window Caption = "Original Window"
hWnd = 00460536
Wnd Proc = 6601FFCB
Class Name = ThunderRT6FormDC
Class Wnd Proc = 004050C5

Window Caption = "Globally Subclassed Form"
hWnd = 00D5053E
Wnd Proc = 004050C5
Class Name = ThunderRT6FormDC
Class Wnd Proc = 004050C5

As you might notice, the Class Wnd Proc field changed from 6601FFCB to 004050C5. This field was changed through the SetClassLongPtr function. This Class Wnd Proc field is the same for all windows. This is correct because the class information is stored separately from the window information. By looking at this information, you can see that the first two windows that were created remained unchanged -- they continue to use their old window procedures. The third window is using the new window procedure (Wnd Proc) from the ThunderRT6FormDC class. This is because the first two windows were created using the original class information, while the third window was created after we modified the class’s window procedure. If we create more global subclassed windows, their information is the same as the window with the caption “Globally Subclassed Form” (except, of course, for the window handle, which must be unique for all windows).

Now click the Un-Subclass button on the main form. Switch to Spy++, press F5 to refresh its display, and now view each window’s information. The new window information is as follows:

Window Caption = "Chapter 4 - Subclassing Example"
hWnd = 00190444
Wnd Proc = 6601FFCB
Class Name = ThunderRT6FormDC
Class Wnd Proc = 6601FFCB

Window Caption = "Original Window"
hWnd = 00460536
Wnd Proc = 6601FFCB
Class Name = ThunderRT6FormDC
Class Wnd Proc = 6601FFCB

Window Caption = "Globally Subclassed Form"
hWnd = 00D5053E
Wnd Proc = 004050C5
Class Name = ThunderRT6FormDC
Class Wnd Proc = 6601FFCB

The class’s window procedure has been changed back to its original value. This value was kept in the m_lOrigClassProc member variable in the CSubclass class. Even though the class’s window procedure is changed back, each window’s window procedure remains as it was before clicking the Un-Subclass button. Every window whose Wnd Proc field points to the original window procedure will act like a normal window. Every window whose Wnd Proc field points to the new window procedure that we created will use our modified functionality.

Here is the tricky part, which can bring your application crashing down around you if you are not careful: because each window’s window procedure remains the same, subclassed windows will still call the globally subclassed window procedure, NewClassProc. The NewClassProc window procedure still needs to know the original window procedure to call whenever it receives a message. If we get rid of the value in the m_lOrigClassProc member variable, the application will crash because m_lOrigClassProc would then point to an invalid location in memory. So, we need to protect the m_lOrigClassProc member variable from being modified in any way.

What happens if the Subclass button is clicked at this point? If we click it, a new window appears with the caption “Globally Subclassed Form”. This form has the following attributes:

Window Caption = "Globally Subclassed Form"
hWnd = 3B5E042C
Wnd Proc = 6601FFCB
Class Name = ThunderRT6FormDC
Class Wnd Proc = 6601FFCB

The example code bypasses the call to SetClassLongPtr and instead creates a window from the current ThunderRT6FormDC class. This window will not have the modified window behavior (notice the value for the Wnd Proc field).

Get Subclassing and Hooking with Visual Basic now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.