By David Stutz, Ted Neward, Geoff Shilling
Price: $34.95 USD
£24.95 GBP

Cover | Table of Contents | Colophon

Table of Contents

Chapter 1: Introducing the CLI Component Model

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

The programmer of the 21^st century has a lot to worry about.

For one thing, useful software is far more complex than ever before. No longer is it acceptable to simply, present a simple terminal-based command prompt or a character-based user interface; users now demand rich, graphical user interfaces with all sorts of visual goodies. Data can seldom be structured to fit in flat files in a local filesystem; instead, the use of a relational database is often required to support the query and reporting requirements that computer users have come to depend on, as well as the ongoing transformations that shape and reshape long-lived data. A single computer once sufficed for application deployment, on which data sharing was accomplished using files or the clipboard; now most computers on the planet are wired for networking, and the software deployed on them must not only be network-aware, but must also be ready to adapt to changing network conditions. In short, building software has moved beyond being a craft that can be practiced by skilled individuals in isolation; it has become a group activity, based on ever more sophisticated underlying infrastructure.

Programmers no longer have the luxury of being able to complete an entire project from scratch, using tools that are close to the processor, such as assemblers or C compilers. Few have the time or the patience to write intermediate infrastructure, even for things as simple as an HTTP implementation or an XML parser, much less the skills to tune this infrastructure to acceptable levels of performance and quality. As a result, great emphasis is now placed on reusable code and on reusable components. The operating system plus a few libraries no longer suffices as a toolkit. Today's programmer, like it or not, relies on code from many different sources that works together correctly and reliably, in support of his applications.

Component software, a development methodology in which independent pieces of code are combined to create application programs, has arisen in response to this trend. By combining components from many sources, programs can be built more quickly and efficiently. However, this technique places new demands on programming tools and the software development process. Reliance on components that were created by untrusted or unknown developers, for example, makes it essential to have stringent control over the execution and verification of code at runtime. In our era of ubiquitous network connectivity, complex component-based software is often updated on-the-fly without local intervention and sometimes maliciously. Ask any virus victim about the necessity of preserving the sanctity of her computers and data, or talk to an unsophisticated computer user about the baffling loss of stability that comes from installing and uninstalling applications on his system, and you will discover that component-based software often contributes as much to the problem as to the solution.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

The CLI Virtual Execution Environment

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

The ECMA Common Language Infrastructure (CLI) is a standardized specification for a virtual execution environment. It describes a data-driven architecture , in which language-agnostic blobs of data are brought to life as self-assembling, typesafe software systems. The data that drives this process, called metadata , is used by developer tools to describe both the behavior of the software as well as its in-memory characteristics. The CLI execution engine uses this metadata to enable managed components from many sources to be loaded together safely. CLI components coexist under strict control and surveillance, yet they can interact and have direct access to resources that need sharing. It is a model that balances control and flexibility.

ECMA, the European Computer Manufacturers Association, is a standards body that has existed for many years. Besides issuing standards on its own, ECMA also has a strong relationship with ISO, the International Standards Organization, and based on this relationship, the CLI specification has been approved as ISO/IEC 23271:2003, with an accompanying technical report designated as ISO:IEC 23272:2003. The C# standard has also been approved, and has become ISO/IEC 23270:2003.

The CLI specification is available on the web sites mentioned in the Preface, and is also included on the CD that accompanies this book. It consists of five large "partitions" plus documentation for its programming libraries. At the time that the CLI was standardized, a programming language named C# was also standardized as a companion effort. C# exploits most of the features of the CLI, and it is the easy-to-learn, object-oriented language in which we have chosen to implement most of the small examples in this book. Formally, the C# and CLI specifications are independent (although the C# specification does refer to the CLI specification), but practically, both are intertwined, and many people consider C# to be the canonical language for programming CLI components.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

A CLI Implementation in Shared Source: Rotor

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

In the summer of 2001, a small team of developers in Redmond announced plans for a Microsoft rarity: a freely-available software distribution containing modifiable, redistributable, source code. This distribution, named the Shared Source CLI (SSCLI, also known affectionately by its code name, "Rotor"), was to contain a fully-functional CLI execution engine, a C# compiler, essential programming libraries, and a number of relevant developer tools. It had been quietly under development alongside the commercial .NET framework and represented an important facet of Microsoft's developer tool strategy. In particular, the SSCLI had three goals to meet: to validate the portability of the CLI standard, to help people learn about and understand Microsoft's commercial CLR offering, and to stimulate long-term academic interest in the CLI. Above all else, the SSCLI was to match the ECMA standard so that anyone who wished to understand or implement this standard would have a guide.

Although the SSCLI is nominally the subject of this book, the CLI standard is its heart. The SSCLI helps us illustrate how and why the CLI is such an interesting piece of work. The distribution itself is a large body of code, and as such, it can provide a significant leg up for researchers and experimenters working in the area of developer tools or systems design, as well as those teaching computer science. This book attempts to act as a top-level guide to the code for such people, giving information beyond the theory of the CLI to facilitate hacking and to explain the conventions of the code base. The CLI standard will be important for years to come, and there is no better way for you to understand it fully than by browsing, building, observing, and tweaking a running implementation.

While Rotor demonstrates one way to build a portable, programming language-independent version of the CLI standard, it is certainly not the only way. Alternate implementations exist at the time of writing, including two from Microsoft (the commercial .NET Framework and a version for the small devices that is called the "Compact Framework "), and two third-party, open source implementations, one from Ximian (called Mono) and one from the DotGNU project (called Portable.NET). Rotor itself, to provide additional developer tools and facilities, implements more than just the standard. To clarify what is contained in the distribution, Figure 1-3 contains a pictorial representation of the differences between Microsoft's commercial offering (.NET CLR), the CLI and C# specifications, and Rotor.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Chapter 2: Getting Started with Rotor

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

The expertise needed to build a virtual machine spans disciplines as diverse as systems design, compiler theory, and hardware architecture. Understanding how and why this is true is important, both for those using virtual machines to solve day-to-day problems and for those extending or implementing them. The purpose of this book is to explain the CLI specification in these terms, drawing on Rotor's source code for examples and clarification.

Before getting to these details, we'll take a detailed look at building, running, debugging, and modifying managed code with Rotor. A simple example will demonstrate these concepts: a managed component that echoes its input back to the console. This example will form a recurring basis for continuing discussions of Rotor's implementation in the chapters that follow.

Consider the simple CLI component in Example 2-1, which consists of a single type named Echo. The Echo type has a single property named EchoString, and a single method, DoEcho.

Example 2-1. A simple CLI component expressed in C# code

public class Echo
{
  private string toEcho = null;

    public string EchoString {
    get { return toEcho; }
    set { toEcho = value; }
  }

  public string DoEcho(  )
  {
    if (toEcho == null)
      throw new Exception("Alas, there is nothing to echo!");
    return toEcho;
  }
}

This component is written using the C# programming language and can be compiled into a CLI component using any C# compiler. C# was chosen for examples in this book, because it was developed as a companion language for the CLI standard and has direct syntax for many of the features found in the CLI.

The SSCLI source code distribution includes several compilers in addition to the C# compiler that will be used in this book. Most notably, there is a full JScript compiler that is itself written in C#. Although there are no JScript samples in this book, the source code for this compiler (found in the jscript directory) is worth browsing, since the typeless dynamic semantics of the language differ greatly than from those of C#. The implementation techniques used to support features such as runtime expression evaluation demonstrate alternative design approaches.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

A Simple Component Assembly

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Consider the simple CLI component in Example 2-1, which consists of a single type named Echo. The Echo type has a single property named EchoString, and a single method, DoEcho.

Example 2-1. A simple CLI component expressed in C# code

public class Echo
{
  private string toEcho = null;

    public string EchoString {
    get { return toEcho; }
    set { toEcho = value; }
  }

  public string DoEcho(  )
  {
    if (toEcho == null)
      throw new Exception("Alas, there is nothing to echo!");
    return toEcho;
  }
}

This component is written using the C# programming language and can be compiled into a CLI component using any C# compiler. C# was chosen for examples in this book, because it was developed as a companion language for the CLI standard and has direct syntax for many of the features found in the CLI.

The SSCLI source code distribution includes several compilers in addition to the C# compiler that will be used in this book. Most notably, there is a full JScript compiler that is itself written in C#. Although there are no JScript samples in this book, the source code for this compiler (found in the jscript directory) is worth browsing, since the typeless dynamic semantics of the language differ greatly than from those of C#. The implementation techniques used to support features such as runtime expression evaluation demonstrate alternative design approaches.

If you are unfamiliar with C#, don't worry. Readers familiar with any high-level, component oriented programming languages such as Java should have no problem reading and understanding these very simple examples. Many good online tutorials and books are available for those who would like to learn C#; the O'Reilly web site http://www.ondotnet.com is one good place to start.

Before we can compile and run the code for the Echo component, we need to prepare Rotor for first use.

Rotor is packaged as a compressed file archive, which can be expanded using your archiving utility of choice. On FreeBSD, tar comes with the system, while on Windows or Mac OS X, there are a number of options:

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Observing Managed Execution

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Because so much of what's happening in the execution engine is low-level, self-modifying code, trying to keep track of what's going on can be awkward. Rather than constantly walk through code in a debugger, readers can take advantage of a number of tracing and diagnostic facilities that exist in Rotor.

To demonstrate the use of tracing, we will use it to observe the JIT compiler in action. First, modify main.exe to contain a try block, as follows:

    public class MainApp {
      public static void Main(  ) {
        try {
          Echo e = new Echo(  );
          e.EchoString = "Echo THIS!";
          System.Console.WriteLine("First echo is: {0}", e.DoEcho(  ));
          e.EchoString = null;
          System.Console.WriteLine("Second echo is: {0}", e.DoEcho(  ));
        } catch {
          System.Console.WriteLine("Caught and recovered from bad Echo.");
        }
      }
    }

When you run this program, you will see:

    % csc -t:exe -r:echo.dll -debug main2.cs
    Microsoft (R) Visual C# Shared Source CLI Compiler version 1.0.0003
    for Microsoft (R) Shared Source CLI version 1.0.0
    Copyright (C) Microsoft Corporation 2002. All rights reserved.

    % clix main2.exe
    First echo is: Echo THIS!
    Caught and recovered from bad Echo.

Scattered throughout the code that implements the CLI execution engine are thousands of calls to chunks of code such as the following that are conditionally compiled for logging and debugging:

    #if defined(_DEBUG) || defined(LOGGING)
      const char *szDebugMethodName;
      const char *szDebugClassName;
      szDebugMethodName = compHnd->getMethodName(info->ftn, &szDebugClassName );
    #endif
    #ifdef _DEBUG
     static ConfigMethodSet fJitBreak;
     fJitBreak.ensureInit(L"JitBreak");
     if (fJitBreak.contains(szDebugMethodName, szDebugClassName,
                         PCCOR_SIGNATURE(info->args.sig)))
        _ASSERTE(!"JITBreak");

     // Check if need to print the trace
     static ConfigDWORD fJitTrace;
     if ( fJitTrace.val(L"JitTrace") )
       printf( "Method %s Class %s \n",szDebugMethodName, szDebugClassName );
    #endif

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Looking Ahead

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Within the rest of the book, we will focus in detail on each of the elements we have already touched on: types, assemblies and metadata, JIT compilation, managed execution, automatic memory management, and the platform adaptation layer. In the next chapter, we begin by examining the notion of type within the CLI and the execution engine, and how the CLI guarantees typesafety within the managed environment.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Chapter 3: Using Types to Describe Components

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Types are the universal abstraction that enables CLI-based programs to interact with the operating system, with foreign code, and with the world of the microprocessor. Below the CLI lurks a world of address spaces, threads, instructions, interrupts, and registers, defined by the operating system and microprocessor being used. Above the CLI, high-level programming languages project component-based abstractions that help to ease programmer interactions with those painfully concrete low-level constructs. Types are the organizational principle that bridges these two worlds safely, efficiently, and consistently. To understand how the CLI creates native code and maintains control over its execution, it is first important to understand its type system.

The notion of a type system can be difficult to define. For most programmers, the old adage, "I can't tell you what it is, but I know it when I see it" describes their definition of a type system. Intuitively, we know that primitive types, classes, structs, and such are part of a type system, and that languages will enforce certain rules regarding the use of these types. But to actually say, in formal terms, what a type system is and entails is difficult. Nonetheless, most programmers, regardless of their background, will be able to infer some interesting details about the CLI type system from Example 3-1, even if they're not familiar or comfortable with C#.

Example 3-1. The Echo component revisited

using System;
namespace SampleEcho {
  public enum EchoVariation { Louder, Softer, Indistinct }
  public struct EchoValue {
    public string theEcho;
    public EchoVariation itsFlavor;
  }
  public interface IEchoer {
    void DoEcho(out EchoValue[] resultingEcho);
  }

  public class Echo : IEchoer {
    private string toEcho = null;
    private static int echoCount = 0;
    private const System.Int16 echoRepetitions = 3;

    public delegate void EchoEventHandler(string echoInfo);
    public event EchoEventHandler OnEcho;

    public Echo(string initialEcho) {
      toEcho = initialEcho;
    }
    public string EchoString {
      get { return toEcho; }
      set { toEcho = value; }
    }
    public void DoEcho(out EchoValue[] resultingEcho) {
      if (toEcho == null) {
        throw(new Exception("Alas, there is nothing to echo!"));
      }
      resultingEcho = new EchoValue[echoRepetitions];
      for (sbyte i = 0; i < echoRepetitions; i++) {
        resultingEcho[i].theEcho = toEcho;
        switch (i) {
          case 0:
            resultingEcho[i].itsFlavor = EchoVariation.Louder;
            break;
          case 1:
            resultingEcho[i].itsFlavor = EchoVariation.Softer;
            break;
          default:
            resultingEcho[i].itsFlavor = EchoVariation.Indistinct;
            break;
        }
      }
      if (OnEcho != null) {
        OnEcho(System.String.Format("Echo number {0}", echoCount));
      }
      echoCount++;
      return;
    }
  }
}

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Types and Type Systems

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

The notion of a type system can be difficult to define. For most programmers, the old adage, "I can't tell you what it is, but I know it when I see it" describes their definition of a type system. Intuitively, we know that primitive types, classes, structs, and such are part of a type system, and that languages will enforce certain rules regarding the use of these types. But to actually say, in formal terms, what a type system is and entails is difficult. Nonetheless, most programmers, regardless of their background, will be able to infer some interesting details about the CLI type system from Example 3-1, even if they're not familiar or comfortable with C#.

Example 3-1. The Echo component revisited

using System;
namespace SampleEcho {
  public enum EchoVariation { Louder, Softer, Indistinct }
  public struct EchoValue {
    public string theEcho;
    public EchoVariation itsFlavor;
  }
  public interface IEchoer {
    void DoEcho(out EchoValue[] resultingEcho);
  }

  public class Echo : IEchoer {
    private string toEcho = null;
    private static int echoCount = 0;
    private const System.Int16 echoRepetitions = 3;

    public delegate void EchoEventHandler(string echoInfo);
    public event EchoEventHandler OnEcho;

    public Echo(string initialEcho) {
      toEcho = initialEcho;
    }
    public string EchoString {
      get { return toEcho; }
      set { toEcho = value; }
    }
    public void DoEcho(out EchoValue[] resultingEcho) {
      if (toEcho == null) {
        throw(new Exception("Alas, there is nothing to echo!"));
      }
      resultingEcho = new EchoValue[echoRepetitions];
      for (sbyte i = 0; i < echoRepetitions; i++) {
        resultingEcho[i].theEcho = toEcho;
        switch (i) {
          case 0:
            resultingEcho[i].itsFlavor = EchoVariation.Louder;
            break;
          case 1:
            resultingEcho[i].itsFlavor = EchoVariation.Softer;
            break;
          default:
            resultingEcho[i].itsFlavor = EchoVariation.Indistinct;
            break;
        }
      }
      if (OnEcho != null) {
        OnEcho(System.String.Format("Echo number {0}", echoCount));
      }
      echoCount++;
      return;
    }
  }
}

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

More on Value Types

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

As has been pointed out, not everything can be a reference. Within an individual component, for example, there must be real data—the numbers, strings, and so on that our programs manipulate to achieve some useful result. Value types are the abstraction that the CLI component model uses to represent the real data of a program to programmers and tools. Without value types , components would be nothing but empty shells—without values, not much can be done. All useful computational work eventually boils down to working with values.

Bytes, characters, integers (of all sizes), floating-point numbers, decimal numbers, enumerated values, and booleans are all value types. A value type, by ECMA Specification definition (Partition I, 7.2.1), is "represented as a sequence of bits"—in other words, values are actual data rather than an address to a location that contains data.

An instance of a value type can be used as a field of a type, as a parameter, as a method return value, or as a variable. When allocated as part of an object or within an array, the value lives within the object on the heap. When declared as a variable or used as a parameter, value types live on the stack. When passed as a parameter to a method, by default, a copy, rather than the address, of the value type is created and sent to the recipient of the method; in short, value types are passed by value. Example 3-10 shows a C# declaration from the Echo component that uses two different kinds of value types.

Example 3-10. A compound value type from the Echo component

public struct EchoValue {
    public string theEcho;
    public EchoVariation itsFlavor;
}

As this sample shows, value types can be grouped together into compound values—in C#, this is done using the struct keyword. Since we are dealing with "real data," value types have features that can be used for interop with data structures that already exist—it is possible to designate with great precision how to lay out a value type in memory, both in terms of ordering and alignment. In general, developers will not want or need to do this—layout is something best left to the JIT compiler unless interop with unmanaged code is needed, but it is definitely possible to take fine-grained control over this. (To be complete, it should be mentioned that it is possible to do explicit layout for nonvalue types, but value types are by far and away the most common use for this feature.)

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

More on Reference Types

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Reference types tie computational behavior directly to their heap-allocated state. There are three important classifications of reference types within the CLI: objects, interfaces, and encapsulated pointers, each of which can be found within the Echo component of Example 3-1. Enumerating these elements, the Echo class itself is an object type that implements an interface, contains a delegate, and uses a managed pointer to pass an out parameter.

Recall that the definition of a value type is tied to its data, which are types that are "represented as a sequence of bits." The location of the value's data is directly embedded into a value type instance. Conversely, a reference type "describes values that are represented in the location of a sequence of bits," according to the ECMA specification. A reference type's value data is never manipulated directly by clients but is always accessed indirectly.

A reference is essentially a small piece of memory that points to the actual location of the reference type—in many ways, it's fair to think of the reference as a pointer. However, references have several advantages over pointers in the classic C/C++ sense:

References are strongly-typed

An object instance cannot be assigned to a reference unless it is assignment-compatible; this means a programmer cannot assign a Person object to a Department reference unless the type Person inherits from Department (an unlikely scenario).

References cannot be incorrectly assigned

A reference cannot point to a memory location that is not occupied by an object of that specific (or compatible) type; similarly, a reference cannot be "manufactured" to point to an arbitrary location in memory.

References cannot dangle

As long as a reference points to an object, that object cannot be deallocated. Therefore, a reference will always either be good or null, which is a reference literal value that points nowhere.

These tie into another aspect that separates reference types from value types. With a value type, because the instance of the value type is the data in question (remember, a value type is "represented as a sequence of bits"), allocation of a value type occurs as soon as the value type is declared within the code:

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Type Interoperability

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Because the CLI type system regards interoperability with native code as an important goal, CLI consumers can expose their own component frameworks or unique features of an underlying operating system without compromise. Unlike execution environments that claim to provide "write once, run anywhere" facilities, the CLI was designed to augment existing system abstractions with its type system rather than fully duplicate such facilities in a new layer. To implement this approach, it follows that CLI types must not only be consistent among themselves, but must also be capable of representing the complete set of native constructs provided by the underlying system and microprocessor, and of using these constructs within its component model.

Built-in types are perhaps the simplest form of type interoperability to understand: they are directly understood by the CLI execution engine, and have obvious value type equivalents. For example, the built-in type System.Int32 represents a 4-byte signed integer. These types are commonly mapped directly to types that the microprocessor implements in hardware by a given CLI implementation. In the ECMA specification, these mappings and the semantics associated with them are termed the "virtual execution system."

The actual constants used to represent built-in types within the JIT compiler are shown in Example 3-21.

Example 3-21. The map used to convert abstract CLI types into processor-specific types (defined in clr/src/vm/jitinterface.cpp)

static const BYTE map[] = {
  CORINFO_TYPE_UNDEF,
  CORINFO_TYPE_VOID,
  CORINFO_TYPE_BOOL,
  CORINFO_TYPE_CHAR,
  CORINFO_TYPE_BYTE,
  CORINFO_TYPE_UBYTE,
  CORINFO_TYPE_SHORT,
  CORINFO_TYPE_USHORT,
  CORINFO_TYPE_INT,
  CORINFO_TYPE_UINT,
  CORINFO_TYPE_LONG,
  CORINFO_TYPE_ULONG,
  CORINFO_TYPE_FLOAT,
  CORINFO_TYPE_DOUBLE,
  CORINFO_TYPE_STRING,
  CORINFO_TYPE_PTR,            // PTR
  CORINFO_TYPE_BYREF,
  CORINFO_TYPE_VALUECLASS,
  CORINFO_TYPE_CLASS,
  CORINFO_TYPE_CLASS,          // VAR (type variable)
  CORINFO_TYPE_CLASS,          // MDARRAY
  CORINFO_TYPE_BYREF,          // COPYCTOR
  CORINFO_TYPE_REFANY,
  CORINFO_TYPE_VALUECLASS,     // VALUEARRAY
  CORINFO_TYPE_INT,            // I
  CORINFO_TYPE_UINT,           // U
  CORINFO_TYPE_DOUBLE,         // R

  // put the correct type when we know our implementation
  CORINFO_TYPE_PTR,            // FNPTR
  CORINFO_TYPE_CLASS,          // OBJECT
  CORINFO_TYPE_CLASS,          // SZARRAY
  CORINFO_TYPE_CLASS,          // GENERICARRAY
  CORINFO_TYPE_UNDEF,          // CMOD_REQD
  CORINFO_TYPE_UNDEF,          // CMOD_OPT
  CORINFO_TYPE_UNDEF,          // INTERNAL
};

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Using Types in Data-Driven Code

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Earlier, in the section entitled "Type, Object, and Component," we saw how the loading and compilation process of the CLI is data-driven, with many decisions made by examining embedded metadata tokens at the last possible moment. This technique is not limited to the JIT compiler—it can be used by custom programs as well. The use of type information to drive program decisions is called introspection or reflection —the component's code is reflecting on its own structure and making decisions based on this information.

Programs with sufficient permissions can create, manipulate, and examine type metadata, either from managed code (using the System.Reflection family of types) or from unmanaged code (using the unmanaged APIs described in clr/src/inc/metadata.h that are outside the CLI specification). Type descriptions can be used to defer decisions until runtime, enabling looser linkages between components and more robust load-time adaptations.

This last point deserves a bit more in the way of explanation—specifically, the idea of using component metadata to promote looser coupling between components may be a new concept for many. Consider, for a moment, a desire to take an existing in-memory object instance and save its current state to some secondary storage stream (e.g., the filesystem, or sent as part of an HTTP request, or even to a binary field in a database.) Under formal, object-oriented approaches, this is common behavior across types and therefore should be represented as a base type from which derived types inherit this functionality.

On closer examination, however, serious problems begin to creep in. To begin with, this base type knows absolutely nothing of the derived type's data, yet it's the derived type's data that needs to be stored (along with any further derived types that in turn derive from the derived type). In addition, because we also look to use inheritance as a mechanism for unifying commonality among domain types (Employee is a Person, whereas Department is not), this in turn begs the argument for multiple inheritance within the system, a road the C++ community already went down and discovered significant issues with.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Summary

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

We will have much more to say about the data-driven approach to execution used in the CLI in later chapters. For now, it is important to note that metadata-rich types are the abstraction that makes this approach possible.

The type system of the CLI is designed to promote maximal flexibility in a language-agnostic approach to component integration. By creating completely self-descriptive components and preserving their metadata as the executable representation, no intrinsic binding to the underlying platform is created until the JIT compiler is run. Using this approach, a single executable can adapt to a variety of platforms, environments, and system versions over time. Armed with more intimate knowledge about how this is possible in the type system of the CLI, we can now turn our attention to how types are packaged and distributed as stored component assemblies.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Chapter 4: Extracting Types from Assemblies

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Types attain their full power as an integration mechanism when they are packaged in a form that can be easily transported from machine to machine and reconstituted safely. The CLI devotes a great deal of its design to enabling exactly this scenario, using a packaging approach based on assemblies . Assemblies are central to understanding components, since as we saw in the discussion of metadata, the component architecture of the CLI is data-driven: the data found in assemblies is a blueprint for all of the types that will populate the execution engine at runtime. Although such metadata can be synthesized directly at runtime, it is far more common to find it in the form of a file on disk, in which form it can propagate from machine to machine and from microprocessor to microprocessor, via traditional disk-to-disk copy or via network download.

Assemblies are the basic unit of packaging and code security for the CLI runtime. The requirement that most influenced their design was the need for packaging that would allow self-contained components to be moved easily from location to location and yet still interoperate with high fidelity. To accommodate this, assemblies took on the following characteristics , which will serve to guide us further in our examination of the CLI:

Assemblies are self-describing

Assemblies, to enable data-driven execution, are completely self-descriptive and preserve full-fidelity metadata.

Assemblies are platform-independent

The CLI achieves a good measure of platform independence by ensuring a well-known, standard format for assemblies.

Assemblies are bound by name

Clients locate assemblies by querying for a four-part tuple that consists of a human-friendly name, an international culture, a multipart version number, and a public key token.

Assembly loading is sensitive to version and policy

Assemblies are loaded using tunable binding rules, which allow programmers and administrators to contribute policy to assembly-loading behavior.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Type Packaging

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Assemblies are the basic unit of packaging and code security for the CLI runtime. The requirement that most influenced their design was the need for packaging that would allow self-contained components to be moved easily from location to location and yet still interoperate with high fidelity. To accommodate this, assemblies took on the following characteristics , which will serve to guide us further in our examination of the CLI:

Assemblies are self-describing

Assemblies, to enable data-driven execution, are completely self-descriptive and preserve full-fidelity metadata.

Assemblies are platform-independent

The CLI achieves a good measure of platform independence by ensuring a well-known, standard format for assemblies.

Assemblies are bound by name

Clients locate assemblies by querying for a four-part tuple that consists of a human-friendly name, an international culture, a multipart version number, and a public key token.

Assembly loading is sensitive to version and policy

Assemblies are loaded using tunable binding rules, which allow programmers and administrators to contribute policy to assembly-loading behavior.

Assemblies are validated

Each time an assembly is loaded, it is subjected to a series of checks to ensure the assembly's integrity.

We'll examine each of these concepts in turn.

Assemblies contain blueprints for types in the form of metadata and CIL, which are referred to as modules . A module is a single file containing the structure and behavior for some or all of the types and/or resources found in the assembly. An assembly always contains at least one module but has the capacity to include multiple modules if desired, usually to gain packaging and performance flexibility.

The types exposed by an assembly are actually represented in the metadata as redirections to the modules that contain the types; it is not possible to expose types without modules. Allowing multiple modules in a single assembly makes it easier to isolate changes as requirements evolve. In particular, resources or types that are either infrequently accessed or are frequently changed can be contained in separate files.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Application Domains

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Application domains (also frequently called "app domains") are critical to understanding assembly loading within the execution engine. They tend to be a bit mysterious and are often described in terms of their similarity to process address spaces, since they scope the visibility of components and resource handles, as well as provide a security and fault isolation barrier. But from our component model implementation point of view, they are not mysterious at all; application domains are the architectural elements that are responsible for loading and unloading assemblies into the execution engine. In addition, while assemblies are resident in memory, application domains provide for isolation on their behalf.

Although the isolation provided by application domains may bear some passing similarities to an operating system address space, they actually coexist within a single address space for a process. Because of this, all domains in a process share execution engine services such as the garbage collector. Application domains provide the means for externalizing references to their components, which means that their components can set up channels of communication between one another under a programmer's control. Because component instances can pass such externalized references among themselves, threads of execution can traverse app domain boundaries; the execution engine carefully monitors these transitions to maintain isolation.

Assemblies are always loaded within the context of an app domain. All communication to and from external processes or components in other domains is mediated by the presence of a component's domain; the execution engine has remoting and marshaling machinery that enforces isolation under the control of the app domain. When the cost of using this machinery is too high or when it is unnecessary, managed processes have the alternative of caching their assemblies in a domain that is reserved for the purpose of sharing assemblies. This is a special case, and it should be used only when necessary, since it compromises the protection afforded by domain isolation.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Bootstrapping the Assembly Load Process

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Executing the code stored within an assembly is a chicken-and-egg scenario. The assembly cannot execute until it has been resolved, loaded into the CLI, verified, and JIT-compiled. The CLI itself is simply a body of code, contained in assemblies that must be loaded into the process space and run. Fortunately, this is a classic bootstrapping problem, and implementation solutions abound. For the SSCLI implementation, a special entry point into the primary assembly is all that is needed, along with some initial security conditions, which are attached to the assembly as data.

The bootstrap API makes hosting the CLI a simple thing to do, as evinced by Rotor's program launcher, clix.exe, whose code can be found in sscli/clr/src/tools/clix, and whose main function, Launch, appears without error handling in Example 4-5.

Example 4-5. The Launch function of clix.exe

DWORD Launch(WCHAR* pRunTime, WCHAR* pFileName, WCHAR* pCmdLine)
{
  HANDLE hFile = NULL;
  HANDLE hMapFile = NULL;
  PVOID pModule = NULL;
  HINSTANCE hRuntime = NULL;
  DWORD nExitCode = 1;
  DWORD dwSize;
  DWORD dwSizeHigh;
  IMAGE_DOS_HEADER* pdosHeader;
  IMAGE_NT_HEADERS32* pNtHeaders;
  IMAGE_SECTION_HEADER*   pSectionHeader;
  WCHAR exeFileName[MAX_PATH + 1];

  // open the file & map it
  hFile = ::CreateFile(pFileName, GENERIC_READ, FILE_SHARE_READ,
                       0, OPEN_EXISTING, 0, 0);
  hMapFile = ::CreateFileMapping(hFile, NULL, PAGE_WRITECOPY, 0, 0, NULL);
  pModule = ::MapViewOfFile(hMapFile, FILE_MAP_COPY, 0, 0, 0);
  dwSize = GetFileSize(hFile, &dwSizeHigh);

  // check the DOS headers
  pdosHeader = (IMAGE_DOS_HEADER*) pModule;
  if (pdosHeader->e_magic != IMAGE_DOS_SIGNATURE ||
      pdosHeader->e_lfanew <= 0 ||
      dwSize <= pdosHeader->e_lfanew + sizeof(IMAGE_NT_HEADERS32)) {
    // Error logic here
  }

  // check the NT headers
  pNtHeaders = (IMAGE_NT_HEADERS32*) ((BYTE*)pModule + pdosHeader->e_lfanew);
  if ((pNtHeaders->Signature != IMAGE_NT_SIGNATURE) ||
      (pNtHeaders->FileHeader.SizeOfOptionalHeader !=
          IMAGE_SIZEOF_NT_OPTIONAL32_HEADER) ||
      (pNtHeaders->OptionalHeader.Magic != IMAGE_NT_OPTIONAL_HDR32_MAGIC)) {
    // Error logic here
  }

  // check the COR headers
  pSectionHeader = (PIMAGE_SECTION_HEADER)
      Cor_RtlImageRvaToVa(pNtHeaders, (PBYTE)pModule,
                          pNtHeaders->OptionalHeader
                               .DataDirectory[IMAGE_DIRECTORY_ENTRY_COMHEADER]
                               .VirtualAddress,
                          dwSize);
  if (pSectionHeader == NULL) {
    // Error logic here
  }

  // load the runtime and go
  hRuntime = ::LoadLibrary(pRunTime);

  _  _int32 (STDMETHODCALLTYPE * pCorExeMain2)(
           PBYTE   pUnmappedPE,                // -> memory mapped code
           DWORD   cUnmappedPE,                // Size of memory mapped code
           LPWSTR  pImageNameIn,               // -> Executable Name
           LPWSTR  pLoadersFileName,           // -> Loaders Name
           LPWSTR  pCmdLine);                  // -> Command Line

  *((VOID**)&pCorExeMain2) = ::GetProcAddress(hRuntime, "_CorExeMain2");
  nExitCode = (int)pCorExeMain2((PBYTE)pModule, dwSize,
                           pFileName,                  // -> Executable Name
                           NULL,                       // -> Loaders Name
                           pCmdLine);                  // -> Command Line

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Securing Against Harmful Assemblies

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

The SSCLI supports Code Access Security (CAS) , which is a component-aware approach to security that extends traditional OS security concepts. The goal for the SSCLI is to provide a level playing field for the components themselves, to enable code from many sources to be combined into applications. Since programs run under the control of the execution engine, and since component code is verified when it is JIT compiled, it is possible for the CLI execution engine to intervene when components misbehave. Because this is possible, the runtime enforcement mechanisms of code access security have real teeth. They would not be possible without managed execution as their foundation.

Code access security combines permissions with evidence and policy. There are two parts to CAS: the assembly load phase and the runtime enforcement phase. We will talk briefly about the load phase at this point and defer the discussion of how runtime enforcement is achieved until Chapter 6.

Permissions represent specific capabilities, such as the ability to read a file. Permissions are used in permission grants and permission demands , which are runtime actions that are tracked and enforced by the CAS service within the execution engine. A permission grant (henceforth referred to as just a "grant") is an authorization based on some combination of policy and evidence; a demand is a check for the corresponding grant.

Within an assembly, permissions may be associated with resources, code identity, or user identity, and are granted to code on a per-assembly basis rather than on a per-user or per-process basis. Permissions are applied to code either declaratively, in which case custom attributes specify behavior in conjunction with policy, or imperatively, in which case code is written to manipulate the CAS service directly to specify behavior. There are numerous resource permissions built into the SSCLI, such as the FileIOPermission, the EnvironmentPermission, and the UIPermission. There is also support for code identity permissions based on strongname. Finally, there is very basic skeletal support for generic user identities and authorization, as well as role-based identities. To see how these are implemented and to learn about others, look in the

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Summary

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

In many ways, assemblies are what programmers think of as components. In their on-disk form, they are durable atoms that can move, as needed, from CLI to CLI and from application version to application version. As a key element of the CLI component model, they are the packages within which types are named and implemented, and from which types are extracted. Assemblies also define the unit of isolation for the code access security model, which facilitates safe interactions between independently developed components by enforcing isolation (in conjunction with the execution engine).

Binding to disk-based assemblies is usually name-based, and the namespace used to bind to assemblies provides scoping flexibility as systems evolve over time. While the common path is to load from disk, it is also important for compilers and tools to have the ability to create assemblies on the fly, and dynamic assemblies are supported for this purpose. Dynamic assemblies can be used to create new on-disk assemblies programmatically or create new in-memory assemblies that can be run immediately.

Once an assembly has been loaded into the CLI either dynamically or by using an application domain, its types and security data are ready to be converted from their passive PE format into the runtime structures that drive the CLI. Each type will be loaded and compiled in turn from the assembly on demand, which is the subject of the next two chapters.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Chapter 5: Synthesizing Components

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

In CLI component-based applications, references between types are represented symbolically using names, as we saw in Chapter 4. This chapter investigates how a set of running components can be synthesized just-in-time by following these symbolic names. Just-in-time synthesis customizes component structure and behavior to a local environment. Using this technique, the execution engine can create optimizations and adaptation wrappers for the benefit of the component.

A gap exists between the CLI's logical representation of a component, expressed as assembly metadata and CIL, and the physical structure and machine instructions needed to execute on an actual microprocessor. To create components and run the behaviors associated with them, the execution engine must bridge this gap and convert the logical representation into data types and instructions that the underlying CPU can understand. CIL must be transformed into opcodes and operands; component metadata must be realized as in-memory data structures that fit both the microprocessor's conventions and any constraints imposed by the host operating system. In short, the execution engine must play by the rules imposed by the hardware and operating system at runtime.

In a traditional approach to compilation, a compiler frontend parses high-level type descriptions and converts them into an intermediate representation, performing data layout at the same time. The back-end then converts the CIL to a flow graph, optimizes it, and produces relocatable native code along with two sets of symbols: imports , which will be used to locate foreign addresses during linkage, and exports , whose addresses will likewise be provided for the use of other modules. At link time , multiple modules are combined into a single executable image, addresses and offsets contained within their code are recalculated as necessary, and symbolic names are resolved by patching these recalculated addresses into the compiled code. Once the linker has produced a complete executable image, a loader

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

The Anatomy of a Component

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Given an instance of a managed component, how is it concretely represented within the CLI's execution engine? We know that we can get a detailed look at the logical structure of a component by running ildasm and viewing the metadata for its type members; however, this tool shows logical structure only. This is not enough; the decision process used by the execution engine to turn these elements into actual memory locations that contain processor instructions or data cannot be predicted by examining metadata alone (except for rare cases in which explicit layout information has been provided by the programmer).

The physical way in which Rotor maintains an object instance and its related type information is quite complex, and the elements that compose its parts are split across many different regions of memory. Figure 5-1 shows the anatomical detail, in gruesome detail. We will spend most of this and the next two chapters dissecting the parts contained in this diagram.

Figure 5-1: The structure of an object and its type is complex

To understand how the execution engine augments CLI metadata through the application of environment-specific layout rules, we will examine the elements of an object instance, and work our way backwards through the data structures that represent its type and their creation. Although this might seem like putting the cart before the horse, it gives us a chance to appreciate the large differences between the abstract world of the CLI and its concrete realization within a specific operating system/processor pair.

Object instances, although they appear to be tightly consolidated units in high-level programming languages, are actually not represented as monolithic chunks of contiguous memory within the SSCLI. Of course, an object can be represented with a single, pointer-sized reference in memory, as anyone who has looked at the parameters associated with CIL's opcodes can attest. Given that component references of this form are the only tangible manifestation for managed objects, it stands to reason that it is possible to find and navigate the important data structures associated with such instances by starting from references to them.

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Verifying and Compiling CIL

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

After the EEClass and its related MethodTable have been laid out, all of the type information necessary for compilation and most of the runtime structures necessary for execution are finished. At this point, the execution engine is ready to compile and execute the code for the type. But what is it that triggers JIT compilation?

In traditional toolchains (such as that of C++), compilation often occurs as far forward as the language can make it—the C++ compiler wants to eliminate as much information as possible from being needed at runtime, so as to minimize the amount of processing required. Frequently, this approach results in situations in which the assumptions used while compiling no longer apply—methods are compiled that are never called in a normal run of the program, for example, or precomputed layouts cannot be used against newer libraries.

The CLI adheres to a principle of maximal deferral: compilation (along with many other activities) does not occur until the last possible moment. In the case of method compilation, the "last possible moment" is the moment that a method is required to run. We need some kind of tripwire to inform us of this event, something that will fire just before method execution, giving the CLI a chance to invoke the JIT compiler on the CIL for that method. It would be possible to track all method invocations and force JIT compilation when necessary, but this would be a naïve implementation and would perform poorly, since only a small number of method invocations actually need to trigger compilation in a typical application.

The CLI chooses an approach that uses an indirect call to a helper function called the prestub helper. Although a type's MethodTable will eventually contain pointers to the native functions that implement its method bodies, every SLOT is initially loaded with a thunk that will trigger both JIT compilation and backpatching of the MethodTable when it is called. This tiny, method-specific piece of code is called a stubcall. With each SLOT holding a pointer to a stubcall, any call via the

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Calling Conventions in Managed Code

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

Once the CIL has been verified and compiled, the native code for the method can be safely executed. Since the CLI, like every modern execution environment, supports programming languages that use recursion, the stack is used to track execution state. Every method call has an activation record on this stack containing its arguments, return value, local variables, and other bookkeeping information such as a security object (which is used by the code access security engine). The structure of Rotor's activation records is shown for both Intel x86 processors and Motorola PowerPC processors in Figure 5-7.

Figure 5-7: Elements of an SSCLI stack frame, for the X86 and PPC architectures

As methods call other methods, the stack is maintained cooperatively using a variety of calling conventions. All calls begin with the setup of the callsite (the stack context associated with a method call by the caller). Parameters are always a part of the callsite, since they can clearly be pushed only by the caller because the method being called knows nothing of them. Past this, however, different calling conventions use different mechanisms; why they differ is often a matter of history, of small performance gains, or of codified personal tastes, and their differences can seem quite arbitrary. Nonetheless, they exist, and how they interoperate in the SSCLI is described in the following sections.

The standard calling convention used in code produced by the JIT compiler is referred to as the JIT calling convention . From the perspective of CIL, there are four possible ways to call code: the jump instruction (which is not verifiable, and so we won't cover it here) and three flavors of the call opcode: call, calli, and callvirt. Each of these has slightly different semantics, and each can additionally be modified to be a tailcall (which reuses the same activation record during recursive calls rather than profligately generating new records). The call instruction is nonvirtual, executing precisely the method targeted by the instruction, versus

Additional content appearing in this section has been removed.
Purchase this book now or read it online at Safari to get the whole thing!

Emitting Components Dynamically

Content preview·Buy PDF of this chapter|Buy reprint rights for this chapter

In an interesting twist, not only does the CLI provide the facilities to examine all this structure and metadata at runtime via the System.Reflection namespace, it also provides the ab