Designing Evolvable Web APIs with ASP.NET

Chapter 11. Hosting

Web API meets its downstairs neighbors.

Chapter 4 divided the ASP.NET Web API processing architecture into three layers: hosting, message handler pipeline, and controller handling. This chapter addresses in greater detail the first of these layers.

The hosting layer is really a host adaptation layer, establishing the bridge between the Web API processing architecture and one of the supported external hosting infrastructures. In fact, Web API does not come with its own hosting mechanism. Instead, it aims to be host independent and usable in multiple hosting scenarios.

In summary, the host adapter layer is responsible for the following tasks:

• Creating and initializing the message handler pipeline, encapsulated in an HttpServer instance.
• Receiving HTTP requests from the underlying hosting infrastructure, typically by registering a callback function.
• Transforming HTTP requests from their native representation (e.g., ASP.NET’s HttpRequest) into HttpRequestMessage instances.
• Pushing these instances into the message handler pipeline, effectively initiating the Web API request processing.
• When a response is produced and returned, the hosting adapter transforms the returned HttpResponseMessage instance into a response representation native to the underlying infrastructure (e.g., ASP.NET’s HttpResponse) and delivers it to the underlying hosting infrastructure.

In version 1.0, two hosting adapters were available: web hosting and self-hosting. The former hosting option allows Web API to be used on top of the classic ASP.NET hosting infrastructure, supported by the IIS (Internet Information Services) server. The latter hosting option—self-hosting—enables the use of Web API on any Windows process, namely console applications and Windows services. These two hosting adapters are available as independent NuGet packages: Microsoft.AspNet.WebApi.WebHost and Microsoft.AspNet.WebApi.SelfHost. Version 2.0 of ASP.NET Web API introduces the OWIN host adapter, available via the Microsoft.AspNet.WebApi.Owin package. This new alternative allows the usage of any OWIN-compliant host.

Our aim in this chapter is to provide the knowledge required for fully using Web API in these hosting scenarios. So, with that in mind, we will now take a deeper look into these host adapters, with a focus on their internal behavior. We’ll also detail how new hosting scenarios can be supported, by presenting a Azure Service Bus adapter, enabling the secure exposure of a privately hosted Web API application into the public web, via the Service Bus relay.

We end with the description of a special hosting option targeted at testing scenarios and usually designated by in-memory hosting. By directly connecting an HttpClient instance to a Web API HttpServer instance, this hosting option allows for direct in-memory HTTP communication between client and server.

By presenting the internal implementation structure, we provide the knowledge required for the correct configuration and optimization of Web API hosting aspects, such as message buffering. This chapter is also relevant for anyone trying to write a custom host or extend an existing one.

Addressing the Web API hosting mechanisms implies dealing with several external technologies, such as the classical ASP.NET pipeline, the WCF channel stack layer, or the OWIN specification. Aiming to be self-contained, this chapter also contains short introductions to these external technologies, focusing on the topics related to hosting.

Web Hosting

So-called web hosting uses the classic ASP.NET pipeline. In the following section, we start by reviewing the relevant aspects of this pipeline. Then, we briefly describe the ASP.NET routing infrastructure, used by both Web API and ASP.NET MVC. Finally, we describe how Web API integrates these two elements.

The ASP.NET Infrastructure

As shown in Figure 11-1, the ASP.NET infrastructure is composed of three main elements: applications, modules, and handlers.

Figure 11-1. ASP.NET pipeline

Applications

In ASP.NET, the unit of deployment is the application, represented by the class HttpApplication. You can create a specific derived class for each application by defining a custom global.asax file.

When a request is mapped to an application, the runtime creates or selects an HttpApplication instance to handle it. It also creates a context representing both the HTTP request and response:

public sealed class HttpContext : IServiceProvider
{
    public HttpRequest Request { get {...} }
    public HttpResponse Response { get {...} }
    ...
}

The application then flows this context through a set of pipeline stages, represented by HttpApplication member events. For instance, the HttpApplication.BeginRequest event is triggered when a request begins its processing.

Modules

An application contains a set of registered module classes, implementing the IHttpModule interface:

public interface IHttpModule
{
    void Dispose();
    void Init(HttpApplication context);
}

When a new application object is constructed, it creates an instance for each one of these module classes and calls the IHttpModule.Init method on those instances. Each module uses this call as an opportunity to attach itself to the pipeline events that it wants to process. A module can be attached to more than one event and an event can have more than one module attached.

These modules can then act as filters, processing both the HTTP request and response as they flow through the event pipeline. These modules also have the ability to short-circuit the request processing, immediately producing the response.

Handlers

After all the request events are triggered, the application selects a handler, represented by the IHttpHandler or the IHttpAsyncHandler interfaces, and delegates the processing of the request to it:

public interface IHttpHandler
{
    void ProcessRequest(HttpContext context);
    bool IsReusable { get; }
}
public interface IHttpAsyncHandler : IHttpHandler
{
    IAsyncResult BeginProcessRequest(HttpContext context,
        AsyncCallback cb, object extraData);
    void EndProcessRequest(IAsyncResult result);
}

When the handler processing ends, the context is then flowed back through the application pipeline, triggering the response events.

Handlers are endpoints on which the application ultimately delegates the request processing. They constitute the main integration point used by the multiple frameworks based on the ASP.NET infrastructure, such as Web Forms, ASP.NET MVC, or Web API.

For instance, in the ASP.NET Web Forms Framework, the System.Web.UI.Page class implements the IHttpHandler interface. This means that the class associated with an .aspx file constitutes a handler, called by the application if the request URI matches the .aspx filepath.

We make the handler selection by mapping the request URI to a file in the application’s directory (e.g., an .aspx file) or by using the ASP.NET routing feature. The former technique is used by ASP.NET Web Forms,^[4] while the latter is used by ASP.NET MVC. Web API also uses the ASP.NET routing functionality, as described in the next section.

ASP.NET Routing

In the ASP.NET infrastructure, we commonly configure routing by adding routes to the RouteTable.Routes static property, which holds a RouteCollection.

For instance, Example 11-1 shows the default mapping defined by the ASP.NET MVC project template, typically present in the global.asax file.

Example 11-1. ASP.NET MVC default route configuration

    protected void Application_Start()
    {
        ...
        RegisterRoutes(RouteTable.Routes);
        ...
    }

    public static void RegisterRoutes(RouteCollection routes)
    {
        routes.IgnoreRoute("{resource}.axd/{*pathInfo}");

        routes.MapRoute(
            "Default", // Route name
            "{controller}/{action}/{id}", // URL with parameters
            new { controller = "Home", action = "Index",
                                        id = UrlParameter.Optional }
        );
    }

The RouteTable.Routes static property defines a route collection, global to the application, where specific routes are added. The MapRoute method, used in Example 11-1 to add a route, isn’t a route collection instance method. Instead, it is an extension method, introduced by ASP.NET MVC, that adds MVC-specific routes. As we will see, Web API uses a similar approach.

Figure 11-2. ASP.NET routing classes

Figure 11-2 shows some of the classes participating in the ASP.NET routing process. The general route concept is defined by the abstract RouteBase class, containing the GetRouteData instance method. This method checks if a request context, namely the request URI, matches the route. If so, it returns a RouteData instance containing a IRouteHandler, which is simply a handler factory. In addition, the RouteData also contains a set of extra values, produced by the matching process. For instance, an HTTP request matched by the Default route of Example 11-1 will result in a RouteData containing the controller and action route values.

The RouteBase class also contains the GetVirtualPath method, performing the inverse lookup process: given a set of values, it returns a URI that would match the route and produce those values.

The abstract RouteBase class is not associated with a specific route matching process, leaving this characterization open for the concrete derived classes. One of those is the Route class, which defines a concrete matching procedure, based on:

• A URI template defining the URI’s structure (e.g., "{controller}/{action}/{id}")
• A set of default values (e.g., new { controller = "Home", action = "Index", id = UrlParameter.Optional })
• A set of additional constraints

The URI template defines both the structure that the URIs must have to be matched by the route, and the placeholders used to extract the route data values from the URI’s path segments.

On a request, the routing selection logic is performed by the UrlRoutingModule attached to the PostResolveRequestCache pipeline event. With each request, this module matches the current request against the routes in the global RouteTable.Routes collection. If there is a match, the associated HTTP handler is mapped to the current request. As a consequence, at the end of the pipeline, the application delegates the request processing to this handler. For instance, all the routes added by the MVC’s MapRoute extension method map to the special MvcHandler.

Web API Routing

The ASP.NET routing model and infrastructure is tied to the legacy ASP.NET model, namely the representation of requests and responses via HttpContext instances. Although it uses similar routing concepts, Web API uses the new HTTP class model and thus defines a new set of routing-related classes and interfaces, presented in Figure 11-3.

Figure 11-3. Web API routing classes

IHttpRoute represents a Web API route, and has characteristics similar to the classic ASP.NET Route class, including:

• A GetRouteData method that receives an HTTP request and the virtual path root and returns an IHttpRouteData containing a value dictionary
• A GetVirtualPath method that receives a value dictionary and a request message, and returns an IHttpVirtualPath with a URI
• A set of properties with the route template, the route defaults, and the constraints

An important distinction in Web API is the use of the new HTTP class model—specifically the HttpRequestMessage and HttpMessageHandler classes—instead of the old ASP.NET classes, such as HttpRequest and IHttpHandler.

Web API also defines a way of using the new routing classes on the classic ASP.NET routing infrastructure, as described in Figure 11-4. This internal adaptation layer is used when Web API is hosted on top of ASP.NET, and allows the simultaneous use of new and old routes in the same HTTP application.

Figure 11-4. Web API routing adaptation classes

The HostedHttpRouteCollection class is an adapter, providing an ICollection<IHttpRoute> interface on top of the classic ASP.NET RouteCollection. When a new IHttpRoute is added to this collection, it wraps it into a special adapter Route (HttpWebRoute) and adds it to the ASP.NET route collection.

This way, the global ASP.NET route collection can have both classic routes and adapters for the new Web API routes.

Global Configuration

When hosting on ASP.NET, the Web API–specific configuration is defined on a singleton HttpConfiguration object, accessible via the static GlobalConfiguration.Configuration property. This singleton object is used as the parameter to the default route configuration illustrated in Example 11-2.

Example 11-2. ASP.NET Web API default route configuration

// config equals GlobalConfiguration.Configuration
config.MapHttpAttributeRoutes();
config.Routes.MapHttpRoute(
    name: "DefaultApi",
    routeTemplate: "api/{controller}/{id}",
    defaults: new { id = RouteParameter.Optional }
);

The Routes property on this singleton configuration references a HostedHttpRouteCollection that wraps the global RouteTable.Routes collection, as shown in Figure 11-5. This means that all the Web API routes added to GlobalConfiguration.Configuration.Routes will end up being added as classical ASP.NET routes into the global RouteTable.Routes collection. As a consequence, when the UrlRoutingModule tries to find a route match, these Web API routes will also be taken into consideration.

Figure 11-5. Web API global configuration

The routes added by an ASP.NET MVC configuration are associated with the MvcHandler class. This means that all the requests that match one of these routes will be delegated to this MvcHandler at the end of the pipeline. Then, this special handler performs the MVC-specific request processing—that is, selecting the controller and calling the mapped action.

The scenario with Web API is rather similar: the routes added via GlobalConfiguration.Configuration.Routes are associated with the HttpControllerHandler that will ultimately handle all the requests matched by one of these Web API routes.

Figure 11-6 illustrates this characteristic, showing the RouteTable.Routes collection holding both MVC and Web API routes. However, note that the MVC routes are associated with MvcHandler, while the Web API routes are associated with HttpControllerHandler.

Figure 11-6. RouteTable.Routes containing both MVC and Web API routes

The Web API ASP.NET Handler

All ASP.NET requests that match a Web API route are handled by the new HttpControllerHandler, as we’ve been detailing. When called on its BeginProcessRequest method this handler performs the following actions:

• First, a singleton HttpServer instance is lazily created on the first handled request, via GlobalConfiguration.Configuration. This server instance contains the message handler pipeline, including the controller dispatching handler.
• Then, the ASP.NET HttpRequest message, present in the current HttpContext, is translated into a new HttpRequestMessage instance.
• Finally, this HttpRequestMessage is pushed into the singleton HttpServer instance, effectively starting the host-independent phase of the Web API processing, composed of the message handler pipeline and the controller layer.

The translation between the native ASP.NET message representation and the Web API message representation is configured by a service object that implements the IHostBufferPolicySelector interface. This interface has two methods, UseBufferedInputStream and UseBufferedOutputStream, that define whether the message body should be buffered. The HttpControllerHandler requests this service object from the global configuration and uses it to decide:

• If the HttpRequestMessage content uses the ASP.NET buffered or streamed request input stream
• If the HttpResponseMessage content is written to a ASP.NET buffered output stream

The web host policy registered by default always buffers the input stream. For the output stream, it uses the following rules based on properties of the returned HttpResponseMessage:

• If the content length is known, then the Content-Length is explicitly set and no chunking is used. The content is transmitted without buffering since its length was already determined.
• If the content class is StreamContent, then chunked transfer encoding will be used only if the underneath stream does not provide length information.
• If the content class is PushStreamContent, then chunked transfer encoding is used.
• Otherwise, the content is buffered before being transmitted to determine its length, and no chunking is used.

The translation of the ASP.NET HttpRequest message into a new HttpRequestMessage instance does more than just including the request message information. A set of contextual hosting information is also captured and inserted into the HttpRequestMessage.Properties dictionary. This information includes:

• The certificate used by the client, if the request was done on a SSL/TLS connection with client authentication
• A Boolean property stating if the request was originated in the same machine
• An indication if the custom errors are enabled

Note that this information does not originate from the request message. Instead, it is provided by the hosting infrastructure and reflects contextual aspects, such as the connection characteristics. For intance, the client certificate information, added as a message property, can be publicly accessed via a GetClientCertificate extension method. The remaining information is used privately by the Web API runtime.

Version 2.0 of the ASP.NET Web API introduces the concept of a request context as a way to group all this contextual information. Instead of being dispersed by different untyped request properties, the new HttpRequestContext class contains the following set of properties to represent this information:

• The client certificate
• The virtual path root
• The request’s principal

The Web API ASP.NET handler creates a WebHostHttpRequestContext instance (WebHostHttpRequestContext derives from HttpRequestContext) and fills it with the request’s contextual information. Upper layers can then access this information via the request’s GetRequestContext extension method.

Figure 11-7 visually summarizes the web hosting architecture, presenting the route resolution process and the dispatch into the HttpServer instance. In conclusion:

• Web API can be hosted on top of the ASP.NET infrastructure and share the same application with other frameworks, such as ASP.NET MVC or Web Forms.
• The ASP.NET routing infrastructure is used to identify the requests that are bound to the Web API runtime. These requests are routed to a special handler that converts the native HTTP representations into the new System.Net.Http model.
• We configure Web API at the start of the application, typically by adding code to the Application_Start method within the global.asax.cs file, and use the GlobalConfiguration.HttpConfiguration singleton object.
• When using web hosting, we must define some configuration aspects on the underlying host, not on the common Web API configuration. An example is the configuration of secure connections, using SSL or TLS, which we can do using the IIS Manager.

Figure 11-7. ASP.NET hosting architecture

Self-Hosting

Web API also contains an adapter for self-hosting (i.e., hosting on any Windows process, such as a console application or a Windows service). Example 11-3 shows the typical code required for this type of hosting.

Example 11-3. Self-hosting

    var config = new HttpSelfHostConfiguration("http://localhost:8080");
    config.Routes.MapHttpRoute("default", "{controller}/{id}",
                                    new { id = RouteParameter.Optional });
    var server = new HttpSelfHostServer(config);
    server.OpenAsync().Wait();
    Console.WriteLine("Server is opened");
    Console.ReadLine();
    server.CloseAsync().Wait();

Note that in this case, a server instance must be explicitly created, configured, and opened. This contrasts with web hosting, where the supporting HttpServer instance is implicitly and lazily created by the ASP.NET handler. Note also that Example 11-3 uses specific classes for the self-hosting scenario. The HttpSelfHostServer class derives from the general HttpServer class and is configured by an HttpSelfHostConfiguration, which itself derives from the general HttpConfiguration class. The hosting base address is explicitly defined in the self-hosting configuration. Figure 11-8 shows the relationship between these classes.

Figure 11-8. Self-host server and configuration classes

In version 1.0 of Web API, the HttpSelfHostServer internally uses the WCF (Windows Communication Foundation) channel stack layer to obtain request messages from the underlying HTTP infrastructure. The following section briefly presents the WCF high-level architecture, setting the groundwork for the description of Web API self-hosting characteristics.

WCF Architecture

The WCF architecture is divided into two layers: the channel stack layer and the service model layer, as depicted in Figure 11-9.

Figure 11-9. WCF architecture

The bottom channel stack layer is composed of a stack of channels and behaves similarly to a classical network protocol stack. The channels are divided into two types: transport channels and protocol channels. Protocol channels process the messages that flow up and down through the stack. A typical use case for a protocol channel is the addition of digital signatures at the sending side and the verification of those signatures at the receiving side. Transport channels handle interfacing with the transport medium (e.g., TCP, MSMQ, HTTP), namely by receiving and sending messages. They use encoders to convert between the transport medium byte streams and message instances.

The upper service model layer performs the interfacing between messages and method calls, dealing with tasks such as:

• Transforming a received message into a parameter sequence
• Obtaining the service instance to use
• Selecting the method to call
• Obtaining the thread at which to call the method

Figure 11-10. Binding, binding elements, and channels

The concrete channel stack layer organization is described by bindings, as shown in Figure 11-10. A binding is an ordered collection of binding elements, where each element roughly describes one channel or encoder. The first binding element describes the upper channel and the last element describes the lower channel, which is always a transport channel.

The HttpSelfHostServer Class

The HttpSelfHostServer class implements a self-hosted Web API server. As presented in Example 11-3, this server is configured by an instance of the HttpSelfHostConfiguration class, which derives from the more general HttpConfiguration and adds specific configuration properties relevant for the self-host scenario.

Internally, the HttpSelfHostServer creates a WCF channel stack and uses it to listen for HTTP requests. This channel stack is described by an instance of the new HttpBinding class, introduced by the Web API self-hosting support.

When starting the server, the HttpSelfHostserver.OpenAsync method creates an HttpBinding instance and asks the HttpSelfHostConfiguration instance to configure it. Then it uses this binding to asynchronously create the WCF channel stack. It also creates a pump that repeatedly pulls messages from this channel stack, converts them into HttpRequestMessage instances, and pushes these new requests into the message handler pipeline.

Similar to what happens in the web hosting scenario, the created HttpRequestMessage is enriched with an HttpRequestContext instance, containing the set of properties obtained from the hosting context. One of them is the client certificate, when TLS/SSL is used with client-side authentication.

This pump is also responsible for taking the returned HttpResponseMessage and writing it into the channel stack. In terms of response streaming, the self-host behaves quite differently from the web host. It statically uses either an explicit Content-Length header or a chunked transfer encoding, based on the following HttpSelfHostConfiguration option:

public TransferMode TransferMode {get; set;}

If TransferMode.Buffered is chosen, then the Content-Length is always explicitly set, independently of what is returned by TryComputeLength or by the ContentLength header property. Namely, if the length information is not provided by the HttpContent instance, the host will buffer in memory all the content to determine the length, and send it only afterward. On the other hand, if TransferMode.Streamed is chosen, then the chunked transfer is always used, even if the content length is known.

The HttpSelfHostConfiguration Class

As we’ve stated, the HttpSelfHostConfiguration defined in the HttpSelfHostServer has the task of configuring the internally used HttpBinding, which in turn configures the WCF message channel. As a consequence, the HttpSelfHostConfiguration class contains a set of public properties, as in Example 11-4, that reflect this internal implementation detail (i.e., are based on the WCF programming model). For instance, the MaxReceivedMessageSize, also available in the popular WCF BasicHttpBinding class, defines the maximum size of the received message. Another example is the X509CertificateValidator property, based on a type from the System.IdentityModel assembly and used to configure the validation of the client certificates received on SSL/TLS connections.

Example 11-4. HttpSelfHostConfiguration properties

public class HttpSelfHostConfiguration : HttpConfiguration
{
    public Uri BaseAddress {get;}
    public int MaxConcurrentRequests {get;set;}
    public TransferMode TransferMode {get;set;}
    public HostNameComparisonMode HostNameComparisonMode {get;set;}
    public int MaxBufferSize {get;set;}
    public long MaxReceivedMessageSize {get;set;}
    public TimeSpan ReceiveTimeout {get;set;}
    public TimeSpan SendTimeout {get;set}
    public UserNamePasswordValidator UserNamePasswordValidator {get;set;}
    public X509CertificateValidator X509CertificateValidator {get;set;}
    public HttpClientCredentialType ClientCredentialType {get;set;}

    // other members elided for clarity
}

Another way to configure the internal self-host behavior is to create an HttpSelfHostConfiguration derived class and override the OnConfigureBinding method. This receives the HttpBinding instance created internally by the HttpSelfHostServer and can change the binding settings before they are used to configure the WCF channel stack. Figure 11-11 shows the self-hosting architecture, specifically the use of the WCF channel stack and the relation between the configuration and WCF binding.

Figure 11-11. Self-hosting architecture

The Web API self-host’s reliance on WCF has both advantages and disadvantages. The main advantage is the availability of most of the WCF HTTP binding capabilities, such as message limiting, throttling, and timeouts. The major disadvantage is that this WCF dependency is exposed on the HttpSelfhostConfiguration public interface, namely on some of its properties.

Note the message pump that retrieves messages from the underlying channel stack and converts them into HttpRequestMessage instances before pushing them into the HttpServer.

URL Reservation and Access Control

When starting a self-hosted web server from a nonadministrator account, you’ll commonly encounter the following error:

HTTP could not register URL http://+:8080/.
Your process does not have access rights to this namespace

Why does this happen and how can we solve it? The answer to these questions requires a short introduction to the low-level HTTP handling architecture.

On Windows, a kernel-mode device driver, called HTTP.sys, listens for HTTP requests. Both IIS and the WCF self-hosting transport channel use this kernel-mode driver, via the user mode HTTP Server API. Server applications use this API to register their interest in handling requests for a given URL namespace. For instance, running Example 11-3 results in the registration of the http://+:8080 namespace by the self-host application. The + in the hostname represents a strong wildcard, instructing HTTP.SYS to consider requests originating from all network adapters. However, this registration is subject to access control, and by default only a process with administrative privileges is authorized to perform it. The aforementioned error occurs for this reason.

One solution would be to start the self-host application using an account with those privileges. However, running a server with administrative rights is seldom a good idea. A better solution is to grant the required permissions to the account under which the application will run. We do this by reserving the URL namespace for that account, allowing the associated applications to register URLs in the reserved namespace. We can make this reservation using the netsh command-line tool (which requires administrative privileges):

netsh http add urlacl url=http://+:8080/ user=domain\user

The domain\user value should be replaced by the identity under which the self-host application will run. This identity can also be one of the Windows special accounts, such as network service, which is typically used when HTTP servers are hosted inside Windows services. In this case, domain\user can be replaced by network service or local service.

Hosting Web API with OWIN and Katana

At the beginning of the chapter, we learned that ASP.NET Web API is built in a manner that is host-agnostic, and that characteristic makes it possible to run a Web API in either a traditional ASP.NET and IIS host or a custom process (self-hosting). This enables new opportunities for building and running Web APIs; it also surfaces some new challenges. For example, in many cases, developers do not want to have to write a custom console application or Windows service in order to self-host their Web API. Many developers who have some experience with frameworks such as Node.js or Sinatra expect to be able to host their application in a presupplied executable. Additionally, a Web API is generally only one component of several in a modern web application. Other components include server-side markup generation frameworks like ASP.NET MVC, static file servers, and real-time messaging frameworks like SignalR. Additionally, an application can be made up of many smaller components that focus on specific tasks such as authentication or logging. While Web API currently provides for different hosting options, a Web API host is not able to simultaneously host any of these other components, meaning that each of the different technologies in a modern web application would require its own host. In the web-hosted scenario, IIS and the ASP.NET request pipeline mask this constraint, but it becomes much more apparent when self-hosting.

What we really need is an abstraction that enables many different types of components to form a single web application and then allows the entire application to run on top of a variety of servers and hosts, based on the unique requirements of the application and deployment environment.

OWIN

The Open Web Interface for .NET (OWIN) is a standard created by the open source community that defines how servers and application components interact. The goal of this effort is to change how .NET web applications are built—from applications as extensions of large, monolithic frameworks to loosely coupled compositions of small modules.

To accomplish this goal, OWIN reduces interactions between server and application components to the following simple interface, known as the application delegate or app func.:

Func<IDictionary<string, object>, Task>

This interface is the only requirement for an OWIN-compatible server or module (also known as middleware). Additionally, because the application delegate consists of a small number of .NET types, OWIN applications are inherently more likely to be portable to different framework versions and even different platforms, such as the Mono project.

The application delegate defines an interaction whereby a component receives all state—including server, application, and request state—via a dictionary object known as the environment or environment dictionary. As an OWIN-based application is assumed to be asynchronous, the application delegate returns a Task instance after performing work and modifying the environment dictionary. OWIN itself defines several of the keys and values that may or must exist in the environment dictionary, as shown in Table 11-1. Additionally, any OWIN server or host can supply its own entries in the environment dictionary, and these can be used by any other middleware.

Table 11-1. Environment entries for request data

Required	Key name	Value description
Yes	"owin.RequestBody"	A Stream with the request body, if any. Stream.Null may be used as a placeholder if there is no request body.
Yes	"owin.RequestHeaders"	An IDictionary<string, string[]> of request headers.
Yes	"owin.RequestMethod"	A string containing the HTTP request method of the request (e.g., "GET", "POST").
Yes	"owin.RequestPath"	A string containing the request path. The path must be relative to the “root” of the application delegate.
Yes	"owin.RequestPathBase"	A string containing the portion of the request path corresponding to the “root” of the application delegate.
Yes	"owin.RequestProtocol"	A string containing the protocol name and version (e.g., "HTTP/1.0" or "HTTP/1.1").
Yes	"owin.RequestQueryString"	A string containing the query string component of the HTTP request URI, without the leading “?” (e.g., "foo=bar&baz=quux"). The value may be an empty string.
Yes	"owin.RequestScheme"	A string containing the URI scheme used for the request (e.g., "http", "https"); see URI Scheme.

In addition to prescribing server and application interactions to the application delegate and environment dictionary, OWIN also provides guidance for host and server implementors related to subjects such as processing URIs and HTTP headers, application startup, and error handling. The simplicity of the application delegate combined with the flexibility of the loosely typed environment dictionary makes it easy for smaller, more focused components to be developed and assembled by a developer into a single application pipeline. As will be covered more in Chapter 15, several examples of these more focused components are already being incorporated in the next release of ASP.NET.

The complete OWIN specification is listed online.

The Katana Project

Whereas OWIN is the specification for defining how servers and application components interact to process web requests, the Katana project is a collection of OWIN-compatible components that are created by Microsoft and distributed as open source software.^[5] Katana project components are organized by architecture layer, as illustrated in Figure 11-12. The HTTP data flow through the different layers and components is illustrated in Figure 11-13.

Figure 11-12. Katana component architecture with example components

Katana components are divided into one of three layers: hosts, servers, and middleware. The responsibility for each type of component is as follows:

Host

Hosts start and manage processes. A host is responsible for launching a process and initiating the startup sequence put forward in section 4 of the OWIN specification.

Server

Servers listen for HTTP requests, ensure that values are correctly placed in the environment dictionary, and call the application delegate for the first middleware in a pipeline of middleware components.

Middleware

Middleware are components that perform any number of different tasks on a request or response. They can be scoped to small tasks, such as implementing compression or enforcing HTTPS, or can function as adapters to an entire framework, such as ASP.NET Web API. Components are arranged in a pipeline structure, where each component has a reference to the next component in the pipeline. The host has the responsibility of constructing this pipeline during its startup sequence.

Figure 11-13. HTTP data flow through an ASP.NET Web API application running on an OWIN pipeline and Katana host

In a conventional, framework-based approach to running web applications, the host, server, and framework start independently of the application and then call into the application at designated points. In this model, a developer’s code is effectively extending the underlying framework, and as a result, the level of control over the request processing that application code will be determined by the framework. Additionally, it means that the application will pay a performance penalty for features of the framework that it does not use, but are run as a part of the framework itself.

In an OWIN-based web application, the startup sequence is reversed. After the host has initialized an environment dictionary and selected a server, it immediately discovers and calls into the developer’s application code to determine what components should be composed together in the OWIN pipeline. By default, Katana hosts discover a developer’s startup code based on the following rules:

• Look up or find a startup class (in order of precedence).
• If present, use the appSettings value for key owin:AppStartup.
• If present, use the type defined in the assembly-level attribute OwinStartupAttribute.
• Scan all assemblies looking for a type named Startup.
• If a startup class is found, find and call a configuration method matching the signature void Configuration(IAppBuilder app).

Following this default discovery logic, we simply need to add the following startup class definition to our project in order for the Katana host’s loader to find and run it:

public class Startup
{
    public void Configuration(IAppBuilder app)
    {
        app.Use(typeof(MyMiddleware));
    }
}

Within the startup class’s configuration method, we can construct the OWIN pipeline by calling the Use method of the supplied IAppBuilder object. The Use method is intended to be a generic means for allowing any component that implements the application delegate to be configured in the pipeline. Additionally, many middleware components and frameworks provide their own extension methods for simplifying pipeline configuration. For example, ASP.NET Web API provides the UseWebApi extension method, which enables the configuration code as follows:

var config = new HttpConfiguration();

// configure Web API
// ...

app.UseWebApi(config);

But what actually happens when you use Web API’s configuration extension method? Going deeper into the Web API configuration method and the Web API middleware component will help you to better understand both the OWIN pipeline and Katana implementation, as well as the decoupled nature of Web API’s host adapter design.

Web API Configuration

When the UseWebApi method is called from within a user’s startup class, the method, which is found in the System.Web.Http.Owin assembly’s WebApiAppBuilderExtensions class, constructs an instance of the HttpMessageHandlerAdapter class—the OWIN middleware component for Web API—and adds it to the IAppBuilder instance using the generic Use method. Looking at the UseWebApi method reveals more about how the Katana infrastructure binds middleware together to form the complete pipeline:

public static IAppBuilder UseWebApi(
    this IAppBuilder builder,
    HttpConfiguration configuration)
{
    IHostBufferPolicySelector bufferPolicySelector =
        configuration.Services.GetHostBufferPolicySelector()
            ?? _defaultBufferPolicySelector;

    return builder.Use(typeof(HttpMessageHandlerAdapter),
        new HttpServer(configuration), bufferPolicySelector);
}

The generic Use method takes the type of the Web API middleware as its first parameter, followed by an arbitrary array of additional parameters. In the case of the Web API middleware, we can see that there are two additional parameters: an HttpServer instance, which is configured with the supplied HttpConfiguration object, and an object that instructs the middleware on how to handle request and response streaming. The middleware itself is passed to Use as a type rather than an instance so that the infrastructure can, as a part of creating the middleware instance, configure it (via the middleware’s constructor) with a reference to the next middleware object in the pipeline. We can see this in action by examining the HttpMessageHandlerAdapter constructor: the next reference is supplied as the first parameter and is then followed by the additional parameters that were passed to the generic Use method:^[6]

public HttpMessageHandlerAdapter(OwinMiddleware next,
    HttpMessageHandler messageHandler,
    IHostBufferPolicySelector bufferPolicySelector) : base(next)

The output of the generic Use method is the modified IAppBuilder object, and therefore the extension method simply returns that object. Returning the IAppBuilder in this way enables us to use a fluid syntax when composing the OWIN pipeline in our startup class.

Web API Middleware

Once the Web API middleware has been added to the OWIN pipeline, an OWIN server can call the middleware’s application delegate for HTTP requests. Recall the signature for the OWIN application delegate:

Func<IDictionary<string, object>, Task>

Web API’s HttpMessageHandlerAdapter class exposes this function indirectly via its base class, OwinMiddleware, which is provided by the Microsoft.Owin NuGet package. This base class supplies the server with the application delegate function and then exposes a simpler API to its descendants:

public async override Task Invoke(IOwinContext context)

The context object provides a more strongly typed object model for accessing members of the environment dictionary like the HTTP request and response objects. The current list of accessors provided by IOwinContext is summarized in Table 11-2.

Table 11-2. Property accessors for IOwinContext

Request	A wrapper around the current request
Response	A wrapper around the current response
Environment	The wrapped OWIN environment dictionary
Authentication (.NET 4.5 and higher)	Accesses the authentication middleware functionality available for the current request

Each property in the context object provides strongly typed access to different members of the environment dictionary. To inspect each of the different wrapper types, see the Microsoft.Owin source.

As a request flows through the OWIN pipeline, when it reaches the HttpMessageHandlerAdapter Invoke method, it is processed according to the data flow illustrated in Figure 11-14.

Figure 11-14. Web API middleware data flow

Because the HttpMessageHandlerAdapter’s primary responsibility is to serve as a bridge between the OWIN pipeline and the Web API programming model, the first action that it performs is to convert the objects found in the OWIN environment dictionary into the fundamental types used by Web API. Not surprisingly, these are HttpRequestMessage and HttpResponseMessage. Prior to sending the HTTP request to Web API for processing, the middleware also extracts the user object, if it exists, from the environment dictionary (via IOwinContext.Request.User) and assigns it to the active thread’s CurrentPrincipal property.

Once the middleware has an HttpRequestMessage representation of the request, it can invoke Web API in a manner similar to the previously described Web API hosting infrastructure components. As is discussed in Chapter 12, the HttpServer type is derived from HttpMessageHandler and acts as the entry point into the Web API message handler pipeline (there is also an extension method overload that enables the developer to specify an additional HttpMessageHandler object known as the dispatcher, which is the last node in the message handler pipeline). Because an HttpMessageHandler cannot be invoked directly, the middleware wraps it in an HttpMessageInvoker object and then calls it with the following:

response = await _messageInvoker.SendAsync(request, owinRequest.CallCancelled);

This initiates processing of the HttpRequestMessage through Web API’s message handler pipeline and controller pipeline and sets a reference to the resultant HttpResponseMessage on a local variable. The message handler and controller pipelines are discussed at length in Chapter 12.

One additional responsibility of the Web API middleware component is determining what to do with an HTTP 404 Not Found status code on the HttpResponseMessage. This is important because in the context of the larger OWIN pipeline, this status code can mean one of two things:

• The request did not match any of the HttpRoutes that were specified in the HttpConfiguration object. In this case, the middleware should invoke the application delegate on its next middleware component.
• The application developer explicitly returned this status code as a part of the application’s protocol implementation (e.g., for request GET /api/widgets/123, item 123 cannot be found in the widgets data store). In this case, the middleware should not invoke the next middleware component in the chain, but instead return the 404 response to the client.

In Web API middleware terms, a 404 response code that is set by Web API’s route matching logic is called a “soft not found,” and it is identified by the presence of an additional setting—HttpPropertyKeys.NoRouteMatched—in the response message’s properties collection. A 404 response code without this setting will be assumed to be a “hard not found” and will result in an immediate 404 HTTP response to the client.

The OWIN Ecosystem

The full set of Katana components is broader than what has been discussed in this chapter. The most recent release of the Katana components includes components for authentication, including middleware for both social and enterprise providers, diagnostics middleware, the HttpListener server, and the OwinHost.exe host. OWIN-based authentication components will be covered in greater detail in Chapter 15. Over time, the list of OWIN-compatible components from Microsoft will continue to grow to include many of the common features currently in System.Web.dll. Additionally, the ecosystem of third-party components created by the community continues to grow, and at present includes many different HTTP frameworks and middleware components. We should expect to see the OWIN component space grow significantly over the next several years.

In-Memory Hosting

An additional Web API hosting option, mainly aimed at testing scenarios, is based on the direct connection between an HttpClient instance and an HttpServer instance. It is commonly designated by in-memory hosting.

As described in Chapter 14, an HttpClient instance can be configured by an HttpMessageHandler passed in the constructor. The client then uses this handler to asynchronously obtain the HTTP request from the HTTP request. Typically, this handler is either an HttpClientHandler that uses an underlying network infrastructure to send and receive HTTP messages, or a DelegatingHandler that performs pre- and post-processing on the request and response, respectively.

However, the HttpServer class also extends from the HttpMessageHandler, meaning that you can use it when constructing an HttpClient. This results in the direct in-memory communication between the client and the server, without any network stack overhead, which is useful in testing scenarios. Example 11-5 shows how to use this capability.

Example 11-5. In-memory hosting

    var config = new HttpConfiguration();
    config.Routes.MapHttpRoute("default", "{controller}/{id}",
                                    new { id = RouteParameter.Optional });
    var server = new HttpServer(config);
    var client = new HttpClient(server);
    var c = client.GetAsync("http://can.be.anything/resource").Result
        .Content.ReadAsStringAsync().Result;

The can.be.anything hostname in Example 11-5 means exactly that: since no network layer is used, the URI’s hostname part is ignored and therefore can be anything.

It is the symmetry between HttpClient and HttpServer—one is a message handler and the other receives a message handler—that allows the direct connection of the client to the server, as shown in Figure 11-15.

Figure 11-15. In-memory hosting diagram

Azure Service Bus Host

Finally, before we end this chapter, we are going to exemplify the development of a custom hosting adapter. As a motivator, we will use the Windows Azure Service Bus, which is a cloud-hosted infrastructure providing both brokered and relayed messaging capabilities. Brokered messaging includes mechanisms such as queues and topics, providing both temporal decoupling and message multicast between senders and receivers. Relayed messaging, which is the main topic in this section, allows the public exposure of APIs hosted on private networks.

Consider Figure 11-16, where an API (i.e., a set of resources) is hosted on a machine with the following characteristics:

• Located in a private network, without owning any public IP or a public DNS name
• Separated from the Internet by both NAT (network address translation) and firewall systems.

Figure 11-16. Service Bus usage scenario

A concrete example of such a setting is a home automation system providing a Web API. In a typical residential scenario, the Internet access (e.g., via DSL) has the characteristics depicted in Figure 11-16—that is, no public IP address or DNS name, and NAT and firewalls blocking inbound connections. However, it would be useful if this API could be consumed by external clients, located on the Internet. Consider, for instance, a scenario where a smartphone is used to remotely control the room temperature or view surveillance images.

As shown in Figure 11-16, the Service Bus relay feature solves these connectivity problems by acting as an intermediary between the client and the API host:

• First, the host establishes an outbound connection to the Service Bus relay. Since it is an outbound connection, not inbound, no public IP is required internally; the translation is performed by the NAT.
• As a consequence of this connection, the Service Bus relay creates and exposes a public endpoint using a domain name in its namespace (e.g., webapibook.servicebus.windows.net).
• Every request sent to this public endpoint is then relayed to the API host via the opened outbound connection. The responses produced by the API host are also returned via this outbound connection and delivered to the client by the Service Bus relay.

The Azure Service Bus is multitenant, and each tenant owns a DNS name with the structure {tenant-namespace}.servicebus.windows.net. For instance, the example in this section uses the name webapibook.servicebus.windows.net. When a host establishes the connection with the service bus, instructing the relay to start listening for requests, it must authenticate itself—that is, prove that it is allowed to use the tenant’s name. Also, the host must define a prefix path, which is combined with the tenant’s DNS name to form the base address. Only requests with this prefix are forwarded to the host by the relay.

The Azure Service Bus provides a SDK (software development kit) that integrates into the WCF programming model and provides special bindings for hosting services via the Service Bus relay. Unfortunately, at the time of this writing, it does not contain any support for the ASP.NET Web API. However, based on the hosting independence capabilities of Web API and inspired by the WCF-based self-host, we can build the custom HttpServiceBusServer class that uses the Service Bus relay to host ASP.NET Web API.

Figure 11-17 shows the HttpServiceBusServer host server and associated classes.

Figure 11-17. The HttpServiceBusServer and related classes

This new server is configured by an instance of the HttpServiceBusConfiguration class, which derives from the base HttpConfiguration, and adds the following properties specific to this hosting scenario:

• The public Service Bus relay address (e.g., https://tenant-namespace.servicebus.windows.net/some/path)
• The credentials required to establish the outbound connection to the Service Bus relay

This design, on which a specific configuration class derives from the base HttpConfiguration, is similar to the one used by the self-host and presented in Figure 11-8. Internally, the HttpServiceBusServer creates a WCF WebServiceHost, and adds an endpoint configured by the WebHttpRelayBinding, which is one of the new bindings in the Service Bus SDK. This new binding is similar to the WCF native WebHttpBinding, with the major difference that the service is exposed remotely on the Service Bus relay, instead of on the local hosting machine. All the requests received through this endpoint are handled by an instance of the DispatcherService class:

[ServiceContract]
[ServiceBehavior(InstanceContextMode = InstanceContextMode.Single,
                        ConcurrencyMode = ConcurrencyMode.Multiple)]
internal class DispatcherService
    [WebGet(UriTemplate = "*")]
    [OperationContract(AsyncPattern = true)]
    public async Task<Message> GetAsync()
    {...}

    [WebInvoke(UriTemplate = "*", Method = "*")]
    [OperationContract(AsyncPattern = true)]
    public async Task<Message> InvokeAsync(Message msg)
    {...}
}

This generic service implements two operations asynchronously: Get and Invoke. The Get operation handles all the HTTP requests with the GET method. The Invoke operation handles all the other request methods (Method = "*"). Notice that both operations have UriTemplate = "*", meaning that they both handle requests for any path.

When a request is received by any of these two methods, the native message representation is transformed into a new HttpRequestMessage instance. This new instance is then pushed into an inner HttpServer, created in the HttpServiceBusServer constructor and configured by the passed HttpServiceBusConfiguration. Unfortunately, the HttpServer.SendAsync method cannot be called directly, since it is protected. However, the HttpMessageInvoker can wrap any message handler, namely the HttpServer, and expose a public SendAsync method:

public DispatcherService(HttpServer server, HttpServiceBusConfiguration config)
{
    _serverInvoker = new HttpMessageInvoker(server, false);
    _config = config;
}

When the HttpServer produces the HttpResponseMessage, the DispatcherService converts it back to the WCF message representation and returns it.

The overall design is inspired by the WCF-based self-host adapter. There are, however, two differences. The first and most important is that the Service Bus host sits on top of the WCF Service Model, while the self-host uses the WCF channel stack directly. This choice, which introduces additional overhead, was adopted because it results in a simpler implementation.

The second difference is that HttpServiceBusServer does not derive from HttpServer. Instead of using an inheritance-based design, like the one chosen by the HttpSelfHostServer, the HttpServiceBusServer uses a compositional approach: an HttpServer instance is created and used internally.

The HttpServiceBusServer is available in the source code. The source code repository also includes an example showing the simplicity of using this new Web API host. The ServiceBusRelayHost.Demo.Screen project defines a Service Bus hosted service, containing only one resource:

public class ScreenController : ApiController
{
    public HttpResponseMessage Get()
    {
        var content = new StreamContent(ScreenCapturer.GetEncodedByteStream());
        content.Headers.ContentType = new MediaTypeHeaderValue("image/jpeg");
        return new HttpResponseMessage()
        {
            Content = content
        };
    }
}

where ScreenCapturer is an auxiliary class for capturing the desktop screen. The hosting of this resource controller is also straightforward:

var config = new HttpServiceBusConfiguration(
    ServiceBusCredentials.ServiceBusAddress)
{
    IssuerName = "owner",
    IssuerSecret = ServiceBusCredentials.Secret
};
config.Routes.MapHttpRoute(
    "default",
    "{controller}/{id}",
    new { id = RouteParameter.Optional });
var server = new HttpServiceBusServer(config);
server.OpenAsync().Wait();
...

First, an HttpServiceBusConfiguration instance is initialized with the Service Bus address, access credentials (IssuerSecret), and access username ("owner"). Then, the routes are added to the Routes property, just as in any other hosting scenario. Finally, an HttpServiceBusServer is configured with this configuration instance and then explicitly opened.

Figure 11-18 shows the result of accessing the screen resource, hosted via Azure Service Bus, through a plain old browser. Notice the use of a public DNS name in the browser’s address bar.

Figure 11-18. Accessing the screen resource hosted via Service Bus

Conclusion

This chapter focused on the way Web API interfaces with external hosting infrastructures. It described not only the originally available host adapters, web host and self-host, but also the new hosting options based on the OWIN specification and the Katana project. Finally, it also presented in-memory hosting and an example of a custom hosting adapter. In the following chapters, our focus will change to the upper layers of ASP.NET Web API, in particular routing and controllers.

---

^[4] Web Forms can also use routing via the RouteCollection.MapPageRoute method.

^[5] In addition to Microsoft’s Katana components, many popular open source web frameworks, such as NancyFX, FUBU, ServiceStack, and others, can also be run in an OWIN pipeline.

^[6] Details surrounding Web API’s dispatching logic, including HttpServer and HttpMessageHandler, will be discussed at length in Chapter 12.