VB Shell Programming | O'Reilly Media

Content preview·Buy reprint rights for this chapter

Before I start discussing the shell, it might be a good idea to define what, exactly, the shell is in terms of Windows. Simply put, the shell is a graphical user interface provided by Windows that allows you to access the various components of the operating system. Sounds good, huh? When you think about it, almost everything you do within Windows begins with the shell (unless you do everything from a DOS window or from console mode in Windows NT and Windows 2000). This includes running software, accessing files, configuring your system, and so on.

This shell provided by Windows is contained within the program Explorer.exe. For those of you who have been using Windows since the 16-bit days, you might think of Explorer as a glorified version of File Manager, Windows 3.1's utility for accessing the filesystem. This could not be farther from the truth. Explorer is really much more than a file manager. It provides a view of your entire system and the means to interact with it. Not only can you access files and create directories, you can configure your printer, schedule tasks, and even surf the Internet. Throughout the course of this book, we will use the terms shell and Explorer interchangeably. They really are one and the same.

You should also know that Explorer is always running. What you think of as Explorer—the browser program that allows you to navigate directories and access your files—is actually a secondary thread in the Explorer process. The primary instance of Explorer is the Desktop. You really are using the shell more than you might think.

On the surface, Explorer seems to be the Swiss Army knife of applications—the one application that lets you do everything. But this is not really the case. In actuality, Explorer is comprised of many different components working together to create the illusion of uniformity.

These components are built using the Component Object Model, or COM. And using this same technology, you can use Visual Basic to write components that fit seamlessly into the heart of Windows using documented interfaces. You actually have the power to extend the functionality of Windows itself.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy reprint rights for this chapter

On the surface, Explorer seems to be the Swiss Army knife of applications—the one application that lets you do everything. But this is not really the case. In actuality, Explorer is comprised of many different components working together to create the illusion of uniformity.

These components are built using the Component Object Model, or COM. And using this same technology, you can use Visual Basic to write components that fit seamlessly into the heart of Windows using documented interfaces. You actually have the power to extend the functionality of Windows itself.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy reprint rights for this chapter

Suppose that every time you copied a bitmap file, its image was made available on the clipboard. Currently, Windows does not support this functionality. But with a data handler (see Chapter 8) you could easily add this feature yourself. Maybe you would like to navigate into an Access database as if it were just another directory in the filesystem. You could do it with the proper namespace extension (see Chapter 12). Or you might like to automatically process information on a web page (say, your online brokerage account) every time you navigated to the URL. A browser helper object is the answer (see Chapter 13).

These are just a few examples of the many things you can accomplish by programming the shell. But all of these examples, and all shell components in general, share one common attribute: they integrate fully with Explorer. This gives the impression that they are actually a part of the shell itself, and technically, they are. Why is this important? Chances are, the application that is used the most by Windows users world-wide is Explorer.exe. It is probably familiar to more people than any other application. This means that, by integrating your application with the shell, you automatically make at least a portion of your application's functionality conveniently and easily available to anyone who is accustomed to working with the shell. An excellent example is the popular WinZip program developed by Niko Mak Computing, Inc: the two most common processes of archive management—adding and extracting files from a .zip file—can be accomplished from the shell without directly opening the WinZip program itself.

This shell integration in turn offers a number of advantages:

Greater ease of use: Because users of your application can work with an interface that's consistent with that of Windows as a whole, they will find your application easier to learn and use. As a result, users will be happy with, rather than frustrated by, your application.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy reprint rights for this chapter

Since our topic is the combination of shell programming and COM, we'll focus on building the following shell extensions, all of which rely on COM interfaces to be loaded and invoked by the shell:

Context menu handlers: A context menu is the pop-up menu that appears when you right-click on an object in Explorer. A context menu handler allows you to customize that menu by adding your own items to it. For instance, if you develop an application that stores thumbnails of graphics files, you might add an "Add to Thumbnail" option to the context menu of any supported graphics file. When the user selects the file, it is automatically added to the application's current thumbnail.
Drag-and-drop handlers: These are specialized context menu handlers; they control the pop-up menu that appears when a shell object is dragged and dropped using the right mouse button.
Icon handlers: Ordinarily, each file type has its own icon. For instance, every Word document is represented by a single icon that serves to identify it. An icon handler, though, allows you to define an icon for an individual instance of that file based on some attribute or condition of the file (its contents, its size, the date it was created, etc.). A classic example is the icon for the Recycle Bin: when it is empty, the Recycle Bin is represented by an empty trash can; when it is not empty, the Recycle Bin is represented by an overflowing trash can.
Property page extension: Every shell object has a Properties dialog that displays one or more property pages when the user selects the Properties menu option for that object. Like many of the features of the shell, the Properties dialog is extensible: you can write property page extensions that add pages to the dialog. An excellent example of this—and itself a powerful feature that can give the user greater control over his documents—is the Custom tab of a Word document's Properties dialog, which allows the user to define custom properties and modify their values.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy reprint rights for this chapter

This book doesn't cover everything; it's really only a starting point. After all, Windows is a big world, especially when you're talking about the shell. And even more so when you bring COM into the picture. But hopefully, by the time you have finished reading this book, you will have learned a little more about both and had some fun in the process. So, now that you know where we are going, it's time to start going. Enjoy.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy reprint rights for this chapter

The components that will be developed in this book are in-process COM servers that run in the address space of Explorer, Internet Explorer, or both. Therefore, a discussion of COM, as it relates to the task ahead, is in order. Because the components are in-process, every aspect of COM will not be discussed (e.g., marshalling). The focus is the fundamental principles of COM in Visual Basic terms. And the goal is to present these concepts in a simple and straightforward manner, with the hope that you will understand the components you create in Visual Basic a little better.

COM is an architecture. It is a standard for developing components that can interact with each other, regardless of the language in which they were written. This means that components that are written in C++, Java, and VB can all work together unaware of the language in which the other was written. This happens because COM is a binary standard. Simply put, when a COM component is loaded into memory, it looks a certain way. It's that simple. COM defines the rules that components use to interact with each other and the outside world. It is not a language. But any language that can call a function through a pointer can be used to write COM components.

A language like C++ offers a source code standard. This allows C++ programmers to reuse code at the source level. In other words, it provides the means for source code reusability. COM, on the other hand, has a much loftier goal. It promises code reuse at the binary level. Unlike C++ source code, a COM component does not need to be recompiled when it is used with a new C++ project. It does not have to be written in C++ either, for that matter. Once that finely tuned sorting algorithm has been placed in a COM server, it is available to any language that supports COM. There is also no need to worry about compiler specifics.

To avoid any confusion, the terms object, component, and COM component are all used interchangeably. One or more objects can exist in a server, whether that server is a DLL or an EXE. COM servers will also be referred to as ActiveX DLLs.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy reprint rights for this chapter

COM is an architecture. It is a standard for developing components that can interact with each other, regardless of the language in which they were written. This means that components that are written in C++, Java, and VB can all work together unaware of the language in which the other was written. This happens because COM is a binary standard. Simply put, when a COM component is loaded into memory, it looks a certain way. It's that simple. COM defines the rules that components use to interact with each other and the outside world. It is not a language. But any language that can call a function through a pointer can be used to write COM components.

A language like C++ offers a source code standard. This allows C++ programmers to reuse code at the source level. In other words, it provides the means for source code reusability. COM, on the other hand, has a much loftier goal. It promises code reuse at the binary level. Unlike C++ source code, a COM component does not need to be recompiled when it is used with a new C++ project. It does not have to be written in C++ either, for that matter. Once that finely tuned sorting algorithm has been placed in a COM server, it is available to any language that supports COM. There is also no need to worry about compiler specifics.

To avoid any confusion, the terms object, component, and COM component are all used interchangeably. One or more objects can exist in a server, whether that server is a DLL or an EXE. COM servers will also be referred to as ActiveX DLLs.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy reprint rights for this chapter

An interface is the basic mechanism by which a COM component exposes functionality. You can think of an interface as a contract. It describes what a component is supposed to do. How it does it is left up to you as the creator of the interface, to a certain degree. One of the fundamentals of COM is the idea of separating the interface from the implementation.

For instance, Example 2.1 shows an interface created in Visual Basic called Animal.

Example 2.1. The Animal Interface

'Animal.cls

Private Enum Kingdoms
    Mammal = 1
    Reptile = 2
    Insect = 3
    Bird = 4
    Fish = 5
End Enum

'Returns animal kingdom.
Private Function Kingdom(  ) As Kingdoms
End Function

'Returns the name of an animal in a string.
Private Function Name(  ) As String
End Function

'Returns the noise the animal makes in a string.
Private Function Noise(  ) As String
End Function

Notice that the functions in the class module animal.cls are just empty stubs. C++ programmers would recognize this as an abstract base class. This serves only to describe what the Animal interface looks like. By itself this does nothing. The interface must be implemented before it becomes useful. This is done in Example 2.2, which creates a Cow class. The Cow class implements and serves as a specific instance of the Animal class.

Example 2.2. Animal Implementation

'Cow.cls

Implements Animal

Private Function Animal_Kingdom(  ) As Kingdoms
    Animal_Kingdom = Mammal
End Function

Private Function Animal_Name(  ) As String
    Animal_Name = "Cow"
End Function

Private Function Animal_Noise(  ) As String
    Animal_Noise = "Moo!"
End Function

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy reprint rights for this chapter

One of the details VB hides from the programmer is the fact that an interface and a class are two distinct entities. This is easier to visualize once you know what a class really looks like; Figure 2.1 attempts to depict the Cow class graphically. A class is nothing more than a pointer to an array of function pointers (member functions), followed by public and private data. This array of function pointers is called a virtual function table, or vtable. This arrangement allows multiple instances of a COM component to share the same vtable, which is very efficient in terms of memory. Of course, member variables are not shared. Every instance of a component has its own copy of any public or private member variables. Also, if a class such as Cow had any methods of its own or implemented any additional interfaces, these would be added to the vtable. The order of a vtable is very important, because for all practical purposes the vtable is the physical representation of the interfaces an object has implemented.

Figure 2.1: The Cow class

The Animal and Cow interfaces are unique. Behind the scenes, VB has assigned a globally unique identifier (GUID) to both of these interfaces. A GUID that names an interface is called an IID. The IID for Animal is {101E95AB-018E-11D3-BB7C-444553540000}. Well, actually this is a string representation of an IID. An IID is a unique 128-bit number, and it is this value that is the true name for the Animal interface. After all, there's nothing really unique about the name Animal. What if another developer on the other side of the world wanted to create an interface named Animal with different attributes? GUIDs are how COM guarantees that an interface is unique, and this allows two interfaces with the same name to coexist peacefully on the same machine.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy reprint rights for this chapter

The interface and co-class definitions are stored in a special file called a type library. In the case of Animals.dll, VB creates the type library automatically and stores it as a resource inside the component. But you can also create your own type libraries. The type libraries that you create can then be referenced from VB. In fact, since VB does not allow us to create type information that is suitable for our needs in developing shell extensions, we're going to have to create our own type library. But before we do that, we will talk about what a type library is and what goes inside of one.

A type library is a language-independent binary file that contains all the information needed to use the component. This includes interface definitions, co-class definitions, structures (UDTs), enumerations, and constants. It is because of type libraries that Visual Basic can implement such great features as Auto List Members (shown in Figure 2.4), Auto Quick Info, and the Object Browser.

Figure 2.4: Auto List Members from type library

Object Browser is really just a simple type library browser. Although it can provide some very useful information, it does hide a number of things that Microsoft does not want VB programmers to know about. Fortunately, there is a utility called the OLE/COM Object Viewer (usually referred to as OLE View) that will allow us to view the type library generated for Animals.dllwithout hiding a thing. This utility ships with Visual Studio but is also freely available in the downloads section at http://www.microsoft.com/com.

Example 2.3 shows the type library listing for Animals.dll, which has been generated by OLE View. To view the type library yourself using OLE View, select the View TypeLib option from OLE View's File menu and use the Open dialog to navigate to the Animals.dll

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy reprint rights for this chapter

From looking at the type library, you can see that _Animal is derived from another interface called IDispatch. What is not apparent is that IDispatch is derived from yet another interface called IUnknown. Actually, all interfaces are ultimately derived from IUnknown. This means that all COM components share a dependable commonality.

The IUnknown interface contains three methods:

QueryInterface
AddRef
Release

The purpose of QueryInterface is to allow clients to discover whether a component supports a given interface. It is also used to navigate between interfaces on a given component. Before returning the requested interface (if it exists), AddRef is called to give the object a reference count.

AddRef and Release are used for reference counting. All objects in memory have an associated reference count. Every time an object is created or copied, this count is incremented by one. Every time an object is released, the reference count is decremented by one. When the reference count is zero, the object can safely unload itself. As a VB programmer, you have seen this entire process many times in code fragments like the following, probably without ever realizing precisely what was happening behind the scenes:

Dim Cow1 As Animal
	
'QueryInterface Animal for Cow interface and call AddRef.
'Cow1 now has a reference count of one.
Set Cow1 = New Cow

Dim Cow2 As Cow

'AddRef is called. Reference count is two.
Set Cow2 = Cow1

'Release Cow1. Reference count is one
Set Cow1 = Nothing

'Release Cow2. Reference count is 0 so component is unloaded.
Set Cow2 = Nothing

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy reprint rights for this chapter

As mentioned previously, _Animal is directly derived from an interface called IDispatch . Interfaces derived from IDispatch often have the [dual] attribute and appropriately are called dual interfaces. This is because the interface supports vtable binding (binding at compile time) and late binding (binding at run-time). The methods that comprise IDispatch facilitate the process known as late binding, which results from code like that shown in Example 2.6.

Example 2.6. Late Binding

'Late binding Cow

Dim cow1 As Object
Set cow1 = CreateObject("Animals.Cow") 

MsgBox cow1.Noise

CreateObject uses the ProgID for the component and maps it to a CLSID. This allows an instance of the component to be created. Internally, this is done by calling the CoCreateInstanceEx API. Once the component is loaded, a call to QueryInterface is made and a pointer to an IDispatch interface is returned. The generic Object datatype really means IDispatch. Then late binding is used to make the call to the Noise method.

Late binding is generally avoided whenever possible for reasons of efficiency (it's extremely slow). But in scripting languages like VBScript and JavaScript, late binding is the only choice available. This is because type information is used at compile time to bind method calls to the object. Code run in a scripting environment is not compiled. It is interpreted at runtime, line by line. Therefore, there needs to be a mechanism for calling the methods of an object in environments such as these. This is where IDispatch comes in. The four methods of IDispatch are:

GetTypeInfoCount
GetTypeInfo

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy reprint rights for this chapter

You should have a basic understanding of the components you are creating with Visual Basic. This chapter, far from being a comprehensive treatise on COM, is merely meant to introduce some of the fundamental concepts that we will be dealing with throughout the remainder of this book. It does not represent the extent of the book. Many new concepts will be discussed as the need arises. For now, you should have a basic understanding of COM, the architecture used by Visual Basic for creating component-based software. These components expose their functionality through interfaces. Interface definitions are stored in a type library, which provides information to clients that wish to use the object. Interface definitions are written in IDL, which is the standard language for defining interfaces, and compiled with the MIDL compiler or MKTYPLIB. The fundamental interface that all objects have in common is IUnknown. IUnknown contains methods that allow for interface discovery and reference counting. The interfaces of objects created with Visual Basic are derived from IDispatch, making the process of runtime binding possible.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy reprint rights for this chapter

Everything that can be viewed within the tree view of Windows Explorer represents what is called a namespace. This namespace not only represents files and directories, but also entities such as drives, printers, and network resources. The shell presents these items in a singular hierarchy with the desktop at the root. Objects in the namespace fall into two categories: folders and file objects.

Folders represent collections within the namespace. Many folders represent actual directories in the filesystem, but some folders are virtual. These virtual folders include Desktop, My Computer, Recycle Bin, Control Panel, Dial-up Networking, and Fonts. Virtual folders are not part of the native filesystem and often are referred to as system folders. Many of these virtual folders can only contain a specific type of file or object. For instance, the Control Panel can only contain Control Panel applications, and the Printer folder can only contain printers.

All folders, whether virtual or not, share the same fundamental properties. Folders are file objects that can contain other file objects. What are file objects? For the most part, file objects represent actual files, but they can include other resources like printers and drives, as well as other folders. The use of the word "file" is somewhat of a misnomer here, because a file object is really any object that is part of the shell namespace. And if an item is part of the shell namespace, a shell extension can be written for it.

Every file object visible in the shell is a member of a file class and can be programmatically extended by a shell extension. There are certain predefined actions that determine which shell extensions will be invoked. These actions include things like right-clicking, copying, moving, dragging, cutting, pasting, and even displaying an icon for a file. Shell extension handlers are in-process COM servers that implement a variety of interfaces, depending on the type of handler being implemented. There are five handlers that perform actions based on a specific file type:

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy reprint rights for this chapter

Folders represent collections within the namespace. Many folders represent actual directories in the filesystem, but some folders are virtual. These virtual folders include Desktop, My Computer, Recycle Bin, Control Panel, Dial-up Networking, and Fonts. Virtual folders are not part of the native filesystem and often are referred to as system folders. Many of these virtual folders can only contain a specific type of file or object. For instance, the Control Panel can only contain Control Panel applications, and the Printer folder can only contain printers.

All folders, whether virtual or not, share the same fundamental properties. Folders are file objects that can contain other file objects. What are file objects? For the most part, file objects represent actual files, but they can include other resources like printers and drives, as well as other folders. The use of the word "file" is somewhat of a misnomer here, because a file object is really any object that is part of the shell namespace. And if an item is part of the shell namespace, a shell extension can be written for it.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy reprint rights for this chapter

Every file object visible in the shell is a member of a file class and can be programmatically extended by a shell extension. There are certain predefined actions that determine which shell extensions will be invoked. These actions include things like right-clicking, copying, moving, dragging, cutting, pasting, and even displaying an icon for a file. Shell extension handlers are in-process COM servers that implement a variety of interfaces, depending on the type of handler being implemented. There are five handlers that perform actions based on a specific file type:

Context menu handlers
Icon handlers
Property sheet handlers
Drop handlers
Data handlers

There are also two types of handlers that are associated with file operations like copying, moving, renaming, and deleting:

Copy hook handlers
Drag-and-drop handlers

A context menu is the menu that appears when a file is right-clicked. A sample context menu is shown in Figure 3.1. Every item in the shell has an associated context menu. This menu provides the means to perform generic operations such as copying, moving, deleting, and renaming file objects.

A context menu handler allows items to be added to this menu for a specific file object. This allows custom processing to be performed on the file object via the menu selection.

Figure 3.1: Context menus provide a means for additional file processing from within the shell

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy reprint rights for this chapter

The registry plays a critical role in defining the shell extensions available for particular filesystem objects. In this section, we'll look at how the registry is used to define the shell extensions for particular file types, as well as how it determines the scope of a particular shell extension.

There are two entries in the registry that are associated with files of a specific type: the file association key and the application identifier key. For example, in Figure 3.6, HKEY_CLASSES_ROOT\.rad is a file association key. The file association key merely points to the application identifier key; that is, its default value contains the name of the application identifier key, which in the case of Figure 3.6 is HKEY_CLASSES_ROOT\radfile. The application identifier key contains the shellex subkey (shellex stands for "shell extension"), which defines the specific handler types and the CLSIDs of the objects designated to handle them. Some handlers, like context menu handlers and property page handlers, require a named value that points to the proper CLSID. This can be any name, but it must be unique at the level in which it resides .

Figure 3.6: Registry settings for shell extensions of a specific file type

Once Explorer has the CLSID for the component that is implementing a particular shell extension, it can find the physical location of the component by going to the HKEY_CLASSES_ROOT\CLSID key and finding the matching CLSID. A subkey of the CLSID key called InProcServer32 contains the physical location of the component. Explorer can then load the component and call methods on the appropriate interfaces. Figure 3.7 shows the mapping from a CLSID to a physical location.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy reprint rights for this chapter

The shell extensions discussed throughout this book will be developed for the mythical .rad file, which is just an imaginary file type I've used to demonstrate the concepts presented in the book. An existing file type could have been used instead, but by using a made-up type, we get to build everything from the ground up. If I had used an existing file type, many of the needed registry entries would already be in place, diminishing the "hands on" approach of the book. Changing registry settings for existing file types also has a tendency to change the way that Windows handles your applications, something you're not likely to appreciate. Also, chances are that you will be writing these extensions for your own file types, not for someone else's.

The format of a .rad file is exactly the same as an .ini file and looks like this:

[Animal]
Type = (dog, cat, fish, snake, cow, or armadillo)
Gender = (M or F)
Color = (Black, White, Gray, Brown, or Green)
Age = (positive integer)
Weight = (positive integer)

You can think of a .rad file as an .ini file with a specific format. The animal types and file format have been purposefully simplified to keep the focus away from the file itself and on the shell extensions. It's not because of laziness . . . Anyway, if you suddenly find yourself with large amounts of free time that just can't be used productively anywhere else, by all means extend the file in any way you wish.

The file association key and the application identifier key must be added to the registry for the .rad file. You can use rad.reg, which is included with the book's code (downloadable from http://vb.oreilly.com), or add it by hand. The keys are as follows:

HKEY_CLASSES_ROOT\.rad = radfile: This notation signifies that the default value for .rad is radfile. This is the file association key; it only serves as a pointer to

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy reprint rights for this chapter

All of the handlers that we will implement will be contained in a single ActiveX DLL project called RadEx. A stub project with the appropriate settings is included with the book's downloadable code for this chapter. The project is an ActiveX DLL project set for Apartment threading (see Appendix A ), which is a requirement for shell extensions. The component will not load if this is not set. That's all there is to it; there is nothing more to the project.

The easiest way to register components is to associate DLL files with regsvr32.exe. This simplifies component registration to a double click.

If the .dll file type is currently not associated with any program, you can make this association yourself by simply double-clicking on any .dll file in the shell. This action displays the Open With dialog box, which will allow you to select a program for association. Just navigate to regsvr32.exe, which is located in your system directory.

If you want to be really hard-core about it, you can add the keys for the association yourself into the registry. This information is placed under the application identifier for .dll files, which happens to be called dllfile, as illustrated in Figure 3.8.

Of course, the path to regsvr.exe might be different on your machine. Notice the %1 on the command line. This will be replaced by the name of the DLL that is being registered.

The source code that is provided with this chapter in the book's sample code serves as a template for future chapters. As the book progresses, the downloadable code for each subsequent chapter will contain source for the extensions created up to that point and an accompanying DLL. This is only to provide Binary Compatibility and keep the GUIDs referred to in this book the same as the GUIDs on your system. Remember, since each chapter contains a new component, it will have to be registered.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy reprint rights for this chapter

Before you test any of the components that we will build throughout the course of this book, it will be necessary to restart the shell. Restarting the shell does not mean shutting down every instance of Explorer you have running because, even after you have done that, there is still one more instance running: the Desktop. That's right, the Desktop is the first instance of Explorer. You will need to restart everything. The obvious method is to just reboot. While this works, it is a little time consuming. There are other ways to restart the shell without shutting down.

If you are developing under Windows NT, you can simply bring up the Task Manager and kill the Explorer process. You can then start a fresh instance by running Explorer from the Run menu.

If you are running Windows 98, restarting the shell is not as straightforward. First, you need to bring up the Task Manager by pressing Ctrl-Alt-Del. Select the Explorer process and then click the End Task button. The Shut Down Windows dialog will appear. This is the same dialog that appears when you select Shut Down from the Start menu. Now this is important: do not press OK. Press Cancel instead. Wait a few moments. Windows will then display a dialog box prompting you to end the task.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy reprint rights for this chapter

Under NT, shell crashes are not really a problem. You can access the Run command from the Task Manager and start up another instance of Explorer.exe.

Under Windows 98, there is no Run command that is available from the Task Manager. When developing for the shell, you can save yourself countless headaches by keeping a simple program running at all times during your development. This program can have a form with one command button that executes the following line of code:

Shell "Explorer.exe"

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy reprint rights for this chapter

The shell displays a context menu for a file object when it is clicked with the right mouse button. This context menu allows various operations to be performed on the file object from within the shell, like printing it or opening it with another program. For example, Figure 4.1 shows the context menu that's displayed when the user clicks on a file in Windows Explorer.

Figure 4.1: A context menu

The items on context menus fall into two categories: static and dynamic. Static context menu items are always the same for every file object of a given type. They can be associated with a file object with just a few registry entries and require no shell extension handlers. The "handler" in this circumstance—that is, the object that performs some action on the file object when that particular context menu item is selected—is usually a normal executable that is passed the name of the file as a command-line parameter. Dynamic context menus, on the other hand, are created with the help of a shell extension handler, which, as we discussed earlier, is a COM component that runs in-process to Explorer. This handler provides the means to display different context menu items for file objects of the same type. The exact appearance of the context menu typically is determined by some state internal to the file itself. Static menus warrant a brief discussion, but the main focus of this chapter will be on dynamic context menus.

Static context menu items are listed under the application identifier key under a subkey called shell (as opposed to the shellex key). These entries remain constant for every instance of the file object and require no implementation code.

Figure 4.2 illustrates how to add an Open context menu item to the .rad file. The subkey of

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy reprint rights for this chapter

Static context menu items are listed under the application identifier key under a subkey called shell (as opposed to the shellex key). These entries remain constant for every instance of the file object and require no implementation code.

Figure 4.2 illustrates how to add an Open context menu item to the .rad file. The subkey of shell (in this case open) is the verb value for the command. There are seven verbs, called canonical verbs, whose meaning is automatically recognized by the shell: open, find, explore, print, printto, openas, and properties. (The printto key is never shown in a context menu, but allows a file to be dragged to a printer object for printing.)

Figure 4.2: Registry entry for static "Open" context menu

The default value of the verb key contains the text for the context menu; in the case of Figure 4.2, the open verb is described in the context menu as "Open." The verb key's subkey is the command key, whose default value contains the path of the file that will be used to carry out the command. The %1 portion of this string in Figure 4.2 denotes the file that was selected within the shell. Whatever file is selected will be passed to notepad.exe on the command line. Of course, this only works because notepad.exe accepts command-line arguments.

However, don't believe for a second that you are limited to these seven canonical verbs. You can actually add you own commands to the context menu and call them anything you want. For example, let's add Register and Unregister commands to the context menu for DLLs. This will provide us with a convenient way to register and unregister components.

To accomplish this, we need to locate the application identifier key for a DLL, which happens to be dllfile. Then, under the shell subkey, we add two other keys:

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy reprint rights for this chapter

With the release of Internet Explorer 5.0, Microsoft has made it possible for you to define your own static context menu items. It's as simple as adding a new registry key at the following location:

HKEY_CURRENT_USER
    SOFTWARE
        Microsoft
            Internet Explorer
                MenuExt

The default value for the key can be either a URL or a program. An additional key called contexts must also be present. This key contains a binary value that determines to which context menu (Internet Explorer provides several, depending on the circumstances) you want to add the new menu item. The values are:

Context Menu	Value
Default	0x01
Image	0x02
ActiveX Control	0x04
Table	0x08
Selected Text	0x10
Hyperlink	0x20

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy reprint rights for this chapter

Static context menus are limited because they are the same for every file object of a given type. Also, the number of files that can be processed through a static menu is limited by the program that is used to carry out the command. What if you need to process 20 files? What if you need different processing options based on the state of the file itself? There are also situations where you might need one context menu for a group of files and another for a single file. This is where dynamic context menus come into play.

A context menu handler is an ActiveX DLL that implements two interfaces: IShellExtInit and IContextMenu. A third interface, IDataObject, is required to implement IShellExtInit. It is not implemented by the object itself but exists as a method parameter in IShellExtInit. We'll explore these interfaces in greater depth after we examine how the shell uses a context menu handler to assemble a context menu.

The process begins when one or more files is right-clicked in Explorer. When this occurs, the shell checks the shellex key under the application identifier key to see if a context menu handler has been defined for the selected file type. In the case of the .rad file, the shell would look under the following key:

HKEY_CLASSES_ROOT/
    radfile/
        shellex/
            ContextMenuHandlers/

If you select 15 files that are of all different types, there is still only one file with active focus: the last file selected in the group. It is this file for which the shell attempts to find an associated context menu handler.

If a context menu handler exists, the shell loads the handler and calls IShellExtInit::Initialize. One of the parameters of Initialize is a reference to IDataObject. The shell uses IDataObject to tell us how many files are selected and what their names happen to be. This gives us the opportunity (as the implementors of

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy reprint rights for this chapter

The components we will write in this book will all implement any given number of system interfaces. "System" in this context (no pun intended) means that these interfaces have already been defined by Microsoft. They are documented, and you can read all about them in the Platform SDK (though the details may be a little murky sometimes).

You can think of an interface as a defined functionality. When a component implements an interface, it is really saying, "I support this functionality!" Consider a Triangle component. It implements the interface Shape. Shape defines two methods: Draw and Color. Therefore, you could expect to access the following functionality through Triangle:

Triangle.Draw
Triangle.Color

Because the Circle, Square, and Trapezoid components also implement Shape, you would expect these objects to have the same functionality as well. This is what it means to implement an interface.

The components in this book all implement some functionality that is required by the shell. This means that when the shell loads our components, it will be able to gain access to our component through a defined mechanism: an interface.

With that said, let's talk about the interfaces a context menu handler component needs to implement before it can be loaded by the shell.

IShellExtInit contains one method (besides the IUnknown portion of the interface), Initialize, as shown in Table 4.1.

Table 4.1: IShellExtInit
Method	Description

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy reprint rights for this chapter

Let's put all of this into action and actually implement a context menu handler for the .rad file. We'll add a context menu item that displays the noise an animal makes in a message box. The menu item itself will be displayed in the format (Animal Name) Noise . Animal Name will be determined from the .rad file in question. Let's begin.

The first step to creating the .rad file context menu handler is to compile the type library containing the interface definitions and constants that will be needed from VB. Constants and UDTs will also be put into the type library with their associated interfaces. But only the groups of constants that are needed will be put in the library. For instance, we need the menu constants MF_BYPOSITION, MF_STRING, and MF_SEPARATOR. Therefore, the library will contain all of the MF_ constants. We don't need any of the menu state constants (MFS_ ), so they will not be included with the library.

The complete listing for the type library that will be used throughout the course of this book can be found in Appendix A. To compile the library, you need to have MKTYPLIB in your path. MKTYPLIB takes one argument on the command line, the name of the ODL file containing the type library definition. To compile, simply type:

mktyplib vbshell.odl

from the command line. If everything is in order, this should produce a file named vbshell.tlb. This is the type library.

To use this library from Visual Basic, you should select Project → References . . . from the main menu. You should then browse to the location of the .tlb file and select it. This will do two things. First, it will register the type library at that location; second, it will make it available to the References dialog for all future projects.

The context menu handler begins life as an ActiveX DLL project called RadEx. Our first step is to register the type library so that interface definitions are available for us to implement. That is done by selecting Project

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy reprint rights for this chapter

Icons are defined for a particular file class by adding a DefaultIcon key under the files association key in the registry. DefaultIcon is then set to a path containing the .exe or .dll that contains the icon to be displayed and the zero-based index of that icon within the file, if multiple icons are present. Figure 5.1, for example, shows the relevant registry keys for our example .rad file type, which is configured to use the second icon in notepad.exe. Every icon for the file class in question will have the same icon when this key is registered.

Figure 5.1: Registering the default icon for a file class

Icon handlers allow file objects of the same type to display different icons on a per-instance basis. This means that such things as the value of particular file attributes or state information internal to the file can be conveyed to the user through the shell. Take the .rad file, for instance. Using an icon handler, we could display a picture of the animal represented by the file (for a limited number of animals, of course), the age of the animal, the gender, or whatever.

Icon handlers are required to implement two interfaces: IPersistFile and IExtractIcon. These interfaces are interesting from a programmatic standpoint for several reasons. IPersistFile is not directly derived from IUnknown; it's derived from IPersist. And as luck would have it, VB doesn't like to implement interfaces that are not directly derived from IUnknown or IDispatch.

IExtractIcon is the general name given to one of two interfaces: IExtractIconA or IExtractIconW. These interfaces contain the ANSI and Unicode versions, respectively, of IExtractIcon. As fate would have it, we will have to implement both of them. The interfaces are defined almost the same (typedefs aside). The only difference is how the methods will be implemented. Also, one of the methods,

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Content preview·Buy reprint rights for this chapter

Icon handlers are required to implement two interfaces: IPersistFile and IExtractIcon. These interfaces are interesting from a programmatic standpoint for several reasons. IPersistFile is not directly derived from IUnknown; it's derived from IPersist. And as luck would have it, VB doesn't like to implement interfaces that are not directly derived from IUnknown or IDispatch.

IExtractIcon is the general name given to one of two interfaces: IExtractIconA or IExtractIconW. These interfaces contain the ANSI and Unicode versions, respectively, of IExtractIcon. As fate would have it, we will have to implement both of them. The interfaces are defined almost the same (typedefs aside). The only difference is how the methods will be implemented. Also, one of the methods, IExtractIcon::Extract, has to return the value S_FALSE (1). In other words, a vtable swap is in order.

Here's how custom icon handlers work. When the shell is about to display an icon for a file object (for the first time), it checks for the following registry key to determine if there is an icon handler for that particular type:

HKEY_CLASSES_ROOT\
    radfile\
        shellex\
            IconHandler

If an icon handler is

Table of Contents