Designing Evolvable Web APIs with ASP.NET

Chapter 10. The HTTP Programming Model

The messages, the whole messages, and nothing but the messages.

This chapter presents the new .NET Framework HTTP programming model, which is at the core of both ASP.NET Web API and the new client-side HTTP support, specifically the HttpClient class. This model was introduced with .NET 4.5 but is also available for .NET 4.0 via NuGet packages. It defines a new assembly—System.Net.Http.dll—with typed programming abstractions for the main HTTP concepts (namely, request and response messages, headers, and body content).

This model is complemented by the System.Net.Http.Formatting.dll assembly, which introduces the media type formatter concept, described in Chapter 13, as well as some utility extension methods and custom HTTP content types. This assembly is available via the “Microsoft ASP.NET Web API Client Libraries” NuGet package, and its source code is part of the ASP.NET project. Despite its name, this package is usable on both the client and server sides. In this chapter we will be describing features from both assemblies, without making any distinction between them.

The .NET Framework already contains more than one programming model for dealing with HTTP concepts. On the client side, the System.Net.HttpWebRequest class can be used to initiate HTTP requests and process the associated responses. On the server side, the System.Web.HttpContext and related classes (e.g., HttpRequest and HttpResponse) are used in the ASP.NET context to represent individual requests and responses. Also on the server side, the System.Net.HttpListenerContext is used by the self-hosted System.Net.HttpListener to provide access to the HTTP request and response objects.

Unfortunately, all these programming models have several problems that the new one aims to solve. Namely, the new System.Net.Http programming model:

Uses the same classes on the client and server sides
Is based on the new Task Asynchronous Pattern (TAP), not on the old Asynchronous Programming Model (APM), meaning that it can take advantage of the async and await language constructs introduced with .NET 4.5
Is easier to use in test scenarios
Has a more strongly typed representation of HTTP messages—namely, by representing HTTP header values as types, not as loose string dictionaries
Is more faithful to the HTTP specification, namely by not layering different abstractions on top of it
Packages the more recent versions as a portable class library, allowing its use on a wide range of platforms

In the next sections, all these properties will become clearer as we present this new model in more detail. We begin by introducing the types for representing the fundamental HTTP concepts, namely request and response messages. Afterward, we show how both message and content headers are represented and processed via a set of specific classes. Finally, we end by showing how to produce and consume the message payload content.

Before we start, note that the old HTTP programming models are still used and supported; for instance, the ASP.NET pipeline is still based on the old System.Net.HttpWebRequest.

Messages

As presented in Chapter 1, the HTTP protocol operates by exchanging request and response messages between clients and servers. Naturally, the message abstraction is at the center of the HTTP programming model, and is represented by two concrete classes, HttpRequestMessage and HttpResponseMessage, that belong to the new System.Net.Http namespace and are represented in Figure 10-1. Both messages comprise:

A start line
A sequence of header fields
An optional payload body

Figure 10-1. The HttpRequestMessage and HttpResponseMessage classes

For requests, the start line is represented by the following HttpRequestMessage properties:

The request’s Method (e.g., GET or POST), defining the request purpose
The RequestUri, identifying the targeted resource
The protocol Version (e.g., 1.1)

For responses, the start line is represented by the following HttpResponseMessage properties:

The protocol Version (e.g. 1.1)
The request StatusCode (a three-digit integer) and the informational ReasonPhrase string

The response message also contains a reference to the associated request message, via the RequestMessage property.

Both the request and response messages can contain an optional message body, represented by the Content property. In the section Message Content, we will address in greater detail how the message content is represented, created, and consumed, including a description of the HttpContent-based class hierarchy.

These two message types as well as the content can be enriched with metadata, in the form of associated headers. The programming model for dealing with these headers will be addressed in the section Headers.

The HttpRequestMessage and HttpResponseMessage classes are nonabstract and can be easily instantiated in user code, as shown in the following examples:

[Fact]
public void HttpRequestMessage_is_easy_to_instantiate()
{
    var request = new HttpRequestMessage(
        HttpMethod.Get,
        new Uri("http://www.ietf.org/rfc/rfc2616.txt"));

    Assert.Equal(HttpMethod.Get, request.Method);
    Assert.Equal(
        "http://www.ietf.org/rfc/rfc2616.txt",
        request.RequestUri.ToString());
    Assert.Equal(new Version(1,1), request.Version);
}

[Fact]
public void HttpResponseMessage_is_easy_to_instantiate()
{
    var response = new HttpResponseMessage(HttpStatusCode.OK);
    Assert.Equal(HttpStatusCode.OK, response.StatusCode);
    Assert.Equal(new Version(1,1), response.Version);
}

This makes these message classes very easy to use in testing scenarios, which contrasts with other .NET Framework classes used to represent the same concepts:

The System.Web.HttpRequest class, used in the ASP.NET System.Web.HttpContext to represent a request, has a public constructor but is reserved for infrastructure only.
The System.Web.HttpRequestBase class, used in ASP.NET MVC, is abstract and cannot be directly instantiated.
The System.Net.HttpWebRequest class, used to represent HTTP requests on the client side, has public constructors, but they are obsolete. Instead, this class should be instantiated via the WebRequest.Create factory method.

The HttpRequestMessage and HttpResponseMessage classes are also usable both on the client side and on the server side, because they represent only the HTTP messages and not other contextual properties. This contrasts with other HTTP classes, such as the ASP.NET HttpRequest class that contains a property with the virtual application root path on the server, which obviously doesn’t make sense on the client side.

The request method is represented by HttpMethod instances, containing the method string (e.g., GET or POST). This class also contains a set of static public properties with the methods defined in RFC 2616:

public class HttpMethod : IEquatable<HttpMethod>
{
        public string Method {get;}
        public HttpMethod(string method);

        public static HttpMethod Get {get;}
        public static HttpMethod Put {get;}
        public static HttpMethod Post {get;}
        public static HttpMethod Delete {get;}
        public static HttpMethod Head {get;}
        public static HttpMethod Options {get;}
        public static HttpMethod Trace {get;}
}

To use a new method, such as the PATCH method defined in RFC 5789, we must explicitly instantiate an HttpMethod with the method’s string, as shown in the following example:

[Fact]
public async Task New_HTTP_methods_can_be_used()
{
    var request = new HttpRequestMessage(
        new HttpMethod("PATCH"),
        new Uri("http://www.ietf.org/rfc/rfc2616.txt"));
    using(var client = new HttpClient())
    {
        var resp = await client.SendAsync(request);
        Assert.Equal(HttpStatusCode.MethodNotAllowed, resp.StatusCode);
    }
}

The response’s status code is represented by the HttpStatusCode enumeration, containing all the status codes defined by the HTTP specification:

public enum HttpStatusCode
  {
    Continue = 100,
    SwitchingProtocols = 101,
    OK = 200,
    Created = 201,
    Accepted = 202,
    ...
    MovedPermanently = 301,
    Found = 302,
    SeeOther = 303,
    NotModified = 304,
    ...
    BadRequest = 400,
    Unauthorized = 401,
    ...
    InternalServerError = 500,
    ...
  }

We can also use new status codes by casting integers to HttpStatusCode:

[Fact]
public void New_status_codes_can_also_be_used()
{
    var response = new HttpResponseMessage((HttpStatusCode) 418)
                       {
                           ReasonPhrase = "I'm a teapot"
                       };
    Assert.Equal(418, (int)response.StatusCode);
}

The HttpRequestMessage also contains a Properties property:

public IDictionary<string, Object> Properties { get; }

This is used to hold additional message information, while it is being processed locally on the server or client side. For instance, it can hold information that is produced at the bottom layers of the processing stack (e.g., message handlers) and is consumed at the upper layers (e.g., controllers).

The Properties property doesn’t reflect any standard HTTP message part and is not retained when the message is serialized for transfer. Instead, it is just a generic container for local message properties, such as:

The client certificate associated with the connection on which the message was received
The route data resulting from matching the message with the set of configured routes

These properties are stored in a dictionary and associated with string keys. The HttpPropertyKeys class defines a set of commonly used keys. Typically, these message properties are accessed via extension methods, such as the ones defined in the System.Net.Http.HttpRequestMessageExtensions class as follows:

public static IHttpRouteData GetRouteData(this HttpRequestMessage request)
{
  if (request == null)
    throw System.Web.Http.Error.ArgumentNull("request");
  else
    return HttpRequestMessageExtensions.GetProperty<IHttpRouteData>(
                                request, HttpPropertyKeys.HttpRouteDataKey);
}

The HttpRequestContext class, introduced with Web API v2, is another example of information that is attached to the request’s properties by the lower hosting layer and then consumed by the upper layers:

public static HttpRequestContext
    GetRequestContext(this HttpRequestMessage request)
{
    ...
    return request.GetProperty<HttpRequestContext>(
        HttpPropertyKeys.RequestContextKey);
}

public static void
    SetRequestContext(this HttpRequestMessage request,
                      HttpRequestContext context)
{
    ...
    request.Properties[HttpPropertyKeys.RequestContextKey] = context;
}

Namely, this class aggregates a set of properties, such as the client certificate or the requestor’s identity, into one typed model:

public class HttpRequestContext
{
    public virtual X509Certificate2 ClientCertificate { get; set; }
    public virtual IPrincipal Principal { get; set; }
    // ...
}

Headers

In HTTP, both the request and response messages, and the message content itself, can be augmented with information in the form of extra fields called headers. For instance:

The User-Agent header field extends a request with information describing the application that produced it.
The Server header field extends a response with information about the origin-server software.
The Content-Type header field defines the media type used by the representation in the request or response payload body.

Each header is characterized by a name and a value, which can be a list. The HTTP specification allows for multiple headers with the same name on a message. However, this specification also states that this is equivalent to only one header occurrence with both values combined. The set of registered HTTP headers is maintained by IANA.

As demonstrated in Figure 10-1, both the request and the response message classes have a Headers property referencing a typed header container class. However, the content headers (e.g., Content-Type) are not in the request or response header collection. Instead, they are in a content header collection, accessible via the HttpContent.Headers property:

[Fact]
public async void Message_and_content_headers_are_not_in_same_coll()
{
    using(var client = new HttpClient())
    {
        var response = await client
            .GetAsync("http://tools.ietf.org/html/rfc2616");
        var request = response.RequestMessage;
        Assert.Equal("tools.ietf.org",request.Headers.Host);
        Assert.NotNull(response.Headers.Server);
        Assert.Equal("text/html",
            response.Content.Headers.ContentType.MediaType);
    }
}

Notice how the Server header is in the response.Headers container, but the ContentType header is in the response.Content.Headers container.

The HTTP programming model defines three header container classes, one for each of the header contexts:

The HttpRequestHeaders class contains the request headers
The HttpResponseHeaders class contains the response headers
The HttpContentHeaders class contains the content headers

These three classes have a set of properties exposing the standard headers in a strongly typed way. For instance, the HttpRequestHeaders class contains an Accept property, declared as a MediaTypeWithQualityHeaderValue collection, where each item contains:

The MediaType string property with the media type identifier (e.g., application/xml)
The Quality property (e.g., 0.9)
The CharSet string property
The Parameters collection property

The following excerpt shows how easy it is to consume the Accept header, since the class model provides access to all the constituent parts (e.g., quality parameter, char set):

[Fact]
public void Classes_expose_headers_in_a_strongly_typed_way()
{
    var request = new HttpRequestMessage();
    request.Headers.Add(
        "Accept",
        "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8");

    HttpHeaderValueCollection<MediaTypeWithQualityHeaderValue> accept =
        request.Headers.Accept;
    Assert.Equal(4,accept.Count);

    MediaTypeWithQualityHeaderValue third = accept.Skip(2).First();
    Assert.Equal("application/xml", third.MediaType);
    Assert.Equal(0.9, third.Quality);
    Assert.Null(third.CharSet);
    Assert.Equal(1,third.Parameters.Count);
    Assert.Equal("q",third.Parameters.First().Name);
    Assert.Equal("0.9", third.Parameters.First().Value);
}

This feature greatly simplifies both the production and consumption of headers, abstracting away the sometimes cumbersome HTTP syntactical rules. These properties can also be used to easily construct header values:

[Fact]
public void Properties_simplify_header_construction()
{
    var response = new HttpResponseMessage();
    response.Headers.Date =
        new DateTimeOffset(2013,1,1,0,0,0, TimeSpan.FromHours(0));
    response.Headers.CacheControl = new CacheControlHeaderValue
    {
        MaxAge = TimeSpan.FromMinutes(1),
        Private = true
    };

    var dateValue = response.Headers.First(h => h.Key == "Date")
        .Value.First();
    Assert.Equal("Tue, 01 Jan 2013 00:00:00 GMT", dateValue);

    var cacheControlValue = response.Headers
        .First(h => h.Key == "Cache-Control").Value.First();
    Assert.Equal("max-age=60, private", cacheControlValue);
}

Notice how the CacheControlHeaderValue class contains a property for each HTTP cache directive (e.g., MaxAge and Private). Notice also how the Date header is constructed from a DateTimeOffset and not from a string, simplifying the construction of correctly formated header values.

Some header values are scalar (e.g., Date) and can be assigned directly, while others are collections represented by the HttpHeaderValueCollection<T> generic class, allowing for value addition and removal:

request.Headers.Date = DateTimeOffset.UtcNow;
request.Headers.Accept.Add(new MediaTypeWithQualityHeaderValue("text/html",1.0));

Figure 10-2 shows the three header container classes, one for each header context. These classes don’t have public constructors and can’t be easily instantiated in isolation. Instead, they are created when a message or content instance is created.

The properties exposed by each one of these classes are restricted to the headers defined by the HTTP RFC. For instance, HttpRequestHeaders contains only properties corresponding to the headers that can be used on an HTTP request. Specifically, it does not provide a way to add nonstandard headers. However, all three classes derive from an HttpHeaders abstract class, shown in Figure 10-3, which provides a set of methods for more low-level access to headers.

Figure 10-2. The three header container classes

Figure 10-3. The HttpHeaders base class provides untyped access to a header collection

First, the HttpHeaders class implements the following interface:

IEnumerable<KeyValuePair<string,IEnumerable<string>>>

This provides access to all the headers as a sequence of pairs, where the header name is a string and the header value is a string sequence. This interface preserves the header ordering and takes into account that header values can be lists. The HttpHeaders class also contains a set of methods for adding and removing headers.

The Add method allows for the addition of headers to the container. If the header has a standard name, its value is validated prior to addition. The Add method also validates if the header can have multiple values:

[Fact]
public void Add_validates_value_domain_for_std_headers()
{
    var request = new HttpRequestMessage();
    Assert.Throws<FormatException>(() =>
        request.Headers.Add("Date", "invalid-date"));
    request.Headers.Add("Strict-Transport-Security", "invalid ;; value");
}

On the other hand, the TryAddWithoutValidation method does not perform header value validation. However, if the value is not valid, it will not be accessible via the typed properties:

[Fact]
public async void
    TryAddWithoutValidation_doesnt_validates_the_value_but_preserves_it()
{
    var request = new HttpRequestMessage();
    Assert.True(request.Headers
        .TryAddWithoutValidation("Date", "invalid-date"));
    Assert.Equal(null, request.Headers.Date);
    Assert.Equal("invalid-date", request.Headers.GetValues("Date").First());

    var content = new HttpMessageContent(request);
    var s = await content.ReadAsStringAsync();
    Assert.True(s.Contains("Date: invalid-date"));
}

After seeing how message and content can be enriched with headers, in the next section we focus our attention on the content itself.

Message Content

In the new HTTP programming model, the HTTP message body is represented by the abstract HttpContent base class, shown in Figure 10-4. Both the HttpRequestMessage and HttpResponseMessage have a Content property of this type, as previously depicted in Figure 10-1.

In this section, we will show how:

Message content can be consumed via the HttpContent methods.
Message content can be produced via one of the existing HttpContent-derived concrete classes or through the creation of a new class.

Figure 10-4. The HttpContent base class and associated hierarchy

Consuming Message Content

When producing message content, we can choose one of the available concrete HttpContent-derived classes. However, when consuming message content, we are limited to the HttpContent methods or extension methods.

In addition to the Headers property described in the previous section, the HttpContent contains the following public, nonvirtual methods:

Task CopyToAsync(Stream, TransportContext)
Task<Stream> ReadAsStreamAsync()
Task<string> ReadAsStringAsync()
Task<byte[]> ReadAsByteArrayAsync()

The first one allows for the consumption of the raw message content in a push style: we pass a stream to the CopyToAsync method that then writes (pushes) the message content into that stream. The returned Task can be used to synchronize with the copy termination:

[Fact]
public async Task HttpContent_can_be_consumed_in_push_style()
{
    using (var client = new HttpClient())
    {
        var response =
            await client.GetAsync("http://www.ietf.org/rfc/rfc2616.txt",
            HttpCompletionOption.ResponseHeadersRead
            );
        response.EnsureSuccessStatusCode();
        var ms = new MemoryStream();
        await response.Content.CopyToAsync(ms);
        Assert.True(ms.Length > 0);
    }
}

The previous example uses the HttpCompletionOptions.ResponseHeadersRead option to allow GetAsync to terminate immediately after the response headers are read. This allows the response content to be consumed without buffering, using the CopyToAsync method.

Alternatively, the ReadAsStreamAsync method allows for the consumption of the raw message content in a pull style: it asynchronously returns a stream, from where the content can then be pulled:

[Fact]
public async Task HttpContent_can_be_consumed_in_pull_style()
{
    using (var client = new HttpClient())
    {
        var response = await
            client.GetAsync("http://www.ietf.org/rfc/rfc2616.txt");
        response.EnsureSuccessStatusCode();
        var stream = await response.Content.ReadAsStreamAsync();
        var buffer = new byte[2*1024];
        var len = await stream.ReadAsync(buffer, 0, buffer.Length);
        var s = Encoding.ASCII.GetString(buffer, 0, len);
        Assert.True(s.Contains("Hypertext Transfer Protocol -- HTTP/1.1"));
    }
}

The last two methods, ReadAsStringAsync and ReadAsByteArrayAsync, asynchronously provide a buffered copy of the message contents: the latter returns the raw byte content while the former decodes that content into a string:

[Fact]
public async Task HttpContent_can_be_consumed_as_a_string()
{
    using (var client = new HttpClient())
    {
        var response = await
            client.GetAsync("http://www.ietf.org/rfc/rfc2616.txt");
        response.EnsureSuccessStatusCode();
        var s = await response.Content.ReadAsStringAsync();
        Assert.True(s.Contains("Hypertext Transfer Protocol -- HTTP/1.1"));
    }
}

In addition to the HttpContent instance methods, there are also extension methods defined in the HttpContentExtensions static class. All these methods are variations of the following:

public static Task<T> ReadAsAsync<T>(
    this HttpContent content,
    IEnumerable<MediaTypeFormatter> formatters,
    IFormatterLogger formatterLogger)

This method receives a sequence of media type formatters and tries to use one of them to read the message content as a T instance:

class GitHubUser
{
    public string login { get; set; }
    public int id { get; set; }
    public string url { get; set; }
    public string type { get; set; }
}

[Fact]
public async Task HttpContent_can_be_consumed_using_formatters()
{
    using (var client = new HttpClient())
    {
        var response = await
            client.GetAsync("https://api.github.com/users/webapibook");
        response.EnsureSuccessStatusCode();
        var user = await response.Content
            .ReadAsAsync<GitHubUser>(new MediaTypeFormatter[]
            {
                new JsonMediaTypeFormatter()
            });
        Assert.Equal("webapibook", user.login);
        Assert.Equal("Organization", user.type);
    }
}

Recall that media type formatters, presented in greater detail in Chapter 13, are classes that extend the abstract MediaTypeFormatter class and perform bidirectional conversions between objects and byte stream representations, as defined by Internet media types.

There is also an overload that doesn’t receive the media type formatter sequence. Instead, it uses a set of default formatters, which currently are JsonMediaTypeFormatter, XmlMediaTypeFormatter, and FormUrlEncodedMediaTypeFormatter.

Creating Message Content

When creating messages with nonempty payload, we assign the Content property with an instance of an HttpContent derived class, chosen in accordance with the content type. Figure 10-4 shows some of the available classes. For instance, if the message content is plain text, then the StringContent class can be used to represent it:

[Fact]
public void StringContent_can_be_used_to_represent_plain_text()
{
    var response = new HttpResponseMessage()
        {
            Content = new StringContent("this is a plain text representation")
        };
    Assert.Equal("text/plain", response.Content.Headers.ContentType.MediaType);
}

By default, the Content-Type header is set to text/plain, but this value can be overriden.

The FormUrlEncodedContent class is used to produce name/value pair content, encoded according to the application/x-www-form-urlencoded rules—the same encoding rules used by HTML forms. The name/value pairs are defined via an IEnumerable<KeyValuePair<string,string>> passed in the FormUrlEncondedContent constructor:

[Fact]
public async Task FormUrlEncodedContent_can_represent_name_value_pairs()
{
    var request = new HttpRequestMessage
        {
            Content = new FormUrlEncodedContent(
                new Dictionary<string, string>()
                    {
                        {"name1", "value1"},
                        {"name2", "value2"}
                    })
        };
    Assert.Equal("application/x-www-form-urlencoded",
        request.Content.Headers.ContentType.MediaType);
    var stringContent = await request.Content.ReadAsStringAsync();
    Assert.Equal("name1=value1&name2=value2", stringContent);
}

The programming model also provides three additional classes for when the content is already available as a byte sequence. The ByteArrayContent class is used when the content is already contained in a byte array:

[Fact]
public async Task ByteArrayContent_can_represent_byte_sequences()
{
    var alreadyExistantArray = new byte[] { 0x48, 0x65, 0x6c, 0x6c, 0x6f};
    var content = new ByteArrayContent(alreadyExistantArray);
    content.Headers.ContentType = new MediaTypeHeaderValue("text/plain")
        { CharSet = "utf-8" };
    var readText = await content.ReadAsStringAsync();
    Assert.Equal("Hello", readText);
}

The StreamContent and PushStreamContent classes are both used for dealing with streams: the StreamContent is adequate for when the content is already available as a stream (e.g., reading from a file), while the PushStreamContent class is used when the content is produced by a stream writer.

The StreamContent instance is created with the stream defined in the constructor. Afterward, when serializing the HTTP message, the HTTP model runtime will pull the byte sequence from this stream and add it to the serialized message body:

[Fact]
public async Task StreamContent_can_be_used_when_content_is_in_a_stream()
{
    const string thisFileName = @"..\..\HttpContentFacts.cs";
    var stream = new FileStream(thisFileName, FileMode.Open, FileAccess.Read);
    using (var content = new StreamContent(stream))
    {
        content.Headers.ContentType = new MediaTypeHeaderValue("text/plain");

        // Assert
        var text = await content.ReadAsStringAsync();
        Assert.True(text.Contains("this string"));
    }
    Assert.Throws<ObjectDisposedException>(() => stream.Read(new byte[1], 0, 1));
}

The stream will be disposed when the wrapping StreamContent is disposed (e.g., by the Web API runtime).

There are, however, scenarios where the content is not already on a stream. Instead, the content is produced by a process that requires a stream into which to write the content. A typical example is XML serialization using a XmlWriter, which requires an output stream in which to write the serialized bytes. A solution would be to use an intermediary MemoryStream where the stream writer writes the contents, and then give this memory stream to a StreamContent instance. However, this solution implies an intermediate copy and is not well suited for streaming scenarios.

A better solution is to use the PushStreamContent class, which receives an Action<Stream, ...> and works in a push-style manner: when the runtime has a stream available (e.g., the underlying ASP.NET response context stream), it calls the action with the stream. It is the action’s responsibility to write the contents to this final stream, without any intermediate buffering:

[Fact]
public async Task
    PushStreamContent_can_be_used_when_content_is_provided_by_a_stream_writer()
{
    var xml = new XElement("root",
                                 new XElement("child1", "text"),
                                 new XElement("child2", "text")
        );
    var content = new PushStreamContent((stream, cont, ctx) =>
        {
            using (var writer = XmlWriter.Create(stream,
                new XmlWriterSettings { CloseOutput = true }))
            {
                xml.WriteTo(writer);
            }
        });
    content.Headers.ContentType =
        new MediaTypeWithQualityHeaderValue("application/xml");

    // Assert
    var text = await content.ReadAsStringAsync();
    Assert.True(text.Contains("<child1"));
}

An important aspect to highlight is that the action does not need to write all the contents synchronously. In fact, the runtime considers the contents to be completely written only when the stream is closed, not when the action returns. This means that the contents can be written by code scheduled by the action (e.g., asynchronous task or timer callback), after the action has returned. The only requirement is that the stream’s Close method be called, in order to signal that the content is completely written. Unfortunately, if an error occurs after the action returns, there is no way to signal it to the runtime. The only possible behavior is to close the stream, which does not distinguish success from failure. To address this problem, newer versions of the System.Net.Http.Formatting.dll assembly provide a PushStreamContent overload receiving a Func<Stream, HttpContent, TransportContext, Task>. This allows the asynchronous code to return a Task, providing a way to signal the ocurrences of exceptions back to the runtime. In the following example, note that the lambda expression is prefixed with async, meaning that it will return a Task:

[Fact]
public async Task PushStreamContent_can_be_used_asynchronously()
{
    const string text = "will wait for 2 seconds without blocking";
    var content = new PushStreamContent(async (stream, cont, ctx) =>
    {
        await Task.Delay(2000);
        var bytes = Encoding.UTF8.GetBytes(text);
        stream.Write(bytes, 0, bytes.Length);
        stream.Close();
    });
    content.Headers.ContentType =
        new MediaTypeWithQualityHeaderValue("text/plain");

    // Assert
    var sw = new Stopwatch();
    sw.Start();
    var receivedText = await content.ReadAsStringAsync();
    sw.Stop();
    Assert.Equal(text, receivedText);
    Assert.True(sw.ElapsedMilliseconds > 1500);
}

The previous content classes require the content to already be represented as a byte sequence. However, the new HTTP programming model also contains the ObjectContent and ObjectContent<T> classes, providing a way to define HTTP message content directly from an object. Internally, these classes use media type formatters to convert the object into the byte sequence.

The following example shows the production of a JSON representation for an anonymous object with three fields. Notice that the media type formatter used—JsonMediaTypeFormatter, in this case—must be explicitly defined in the ObjectContent constructor:

[Fact]
public async Task ObjectContent_uses_mediatypeformatter_to_produce_the_content()
{
    var representation = new
        {
         field1 = "a string",
         field2 = 42,
         field3 = true
        };
    var content = new ObjectContent(
        representation.GetType(),
        representation,
        new JsonMediaTypeFormatter());

    // Assert
    Assert.Equal("application/json",content.Headers.ContentType.MediaType);
    var text = await content.ReadAsStringAsync();
    var obj = JObject.Parse(text);
    Assert.Equal("a string", obj["field1"]);
    Assert.Equal(42, obj["field2"]);
    Assert.Equal(true, obj["field3"]);
}

The ObjectContent receives both the input object value and object type. The generic version, ObjectContent<T>, is just a simplification where the input type is given as a generic parameter.

The new programming model also has a set of extension methods, contained in multiple HttpRequestMessageExtensions classes, that aim to simplify the creation of responses from requests. For instance, you can create a response message that is automatically linked to the request message:

[Fact]
public void HttpRequestMessage_has_a_CreateResponse_extension_method()
{
    var request =
        new HttpRequestMessage(HttpMethod.Get,
            new Uri("http://www.example.net"));
    var response = request.CreateResponse(HttpStatusCode.OK);
    Assert.Equal(request, response.RequestMessage);
}

You can also use a CreateResponse overload to create a representation from an object, given a media type formatter, similarly to what you can do with ObjectContent:

public void CreateResponse_can_receive_a_formatter()
{
    var request =
        new HttpRequestMessage(HttpMethod.Get,
            new Uri("http://www.example.net"));

    var response = request.CreateResponse(
        HttpStatusCode.OK,
        new { String = "hello", AnInt = 42 },
        new JsonMediaTypeFormatter());

    Assert.Equal("application/json",
        response.Content.Headers.ContentType.MediaType);
}

The CreateResponse message is particularly useful in server-driven content negotiation scenarios, where the request information—namely, the request Accept header—is needed to decide the most appropriate media type:

[Fact]
public void CreateResponse_performs_content_negotiation()
{
    var request =
        new HttpRequestMessage(HttpMethod.Get,
            new Uri("http://www.example.net"));
    request.Headers.Accept.Add(
        new MediaTypeWithQualityHeaderValue("application/xml", 0.9));
    request.Headers.Accept.Add(
        new MediaTypeWithQualityHeaderValue("application/json", 1.0));

    var response = request.CreateResponse(
        HttpStatusCode.OK,
        "resource representation",
        new HttpConfiguration());

    Assert.Equal("application/json",
        response.Content.Headers.ContentType.MediaType);
}

Notice how the used CreateResponse overload receives an HttpConfiguration with the configured formatters.

Finally, you also have the option of producing message content by creating custom HttpContent-derived classes. However, before we present this technique, it is useful to understand how an HTTP message content length is computed.

Content length and streaming

In HTTP, there are three major ways to define the payload body length:

Explicitly, by adding the Content-Length header field with the message length
Implicitly, by using chunked transfer encoding
Implicitly, by closing the connection after all the content is transmitted (applicable only to response content)

The last option exists mainly for compatibility with HTTP 1.0 and should not be used, since an abnormal termination of the connection will result in undetected content corruption.

With chunked transfer encoding, the message body is divided in a series of chunks, each with its own size definition. This allows streaming content, where the length information is not known beforehand, to be transmitted without buffering.

The first option is the simpler one, but it requires a priori knowledge of the content length. For this purpose, the HttpContent class contains the following abstract method:

protected internal abstract bool TryComputeLength(out long length)

Each concrete content class must implement this method according to how the content is represented. For instance, the ByteArrayContent class implementation always returns true, providing the underlying array length. On the other hand, the PushStreamContent class implementation returns false, since the contents are pushed dynamically by the registered action. Notice that there is no way for the PushStreamContent class to know how many bytes will be pushed by this action. Finally, the StreamContent class implementation delegates this query to the underlying Stream, defined in the constructor: if this stream is seekable, then the TryComputeLength method uses the Stream.Length to compute the content length; otherwise, the TryComputeLength method returns false.

There is also a close relationship between the TryComputeLength method and the long? HttpContentHeaders.ContentLength property: when this property is not explicitly set, its value will query the TryComputeLength method. This means that there is no need to explicitly set the Content-Length header, except in scenarios where this information is obtained by external methods. Notice also that HttpContentHeaders.ContentLength is of type long?, allowing for the absence of value.

In Chapter 11, we will describe how this information is used by the hosting layer to determine the best way to handle the response message content (namely, to decide if buffering should be used or not).

Custom content classes

Now that we’ve seen how message content length is defined and what influences streaming, we will address the creation of custom content classes. The following code excerpt shows the definition of the FileContent class: a custom HttpContent-derived class to represent file contents:

public class FileContent : HttpContent
{
    private readonly Stream _stream;
    public FileContent(string path,
        string mediaType = "application/octet-stream")
    {
        _stream = new FileStream(path, FileMode.Open, FileAccess.Read);
        base.Headers.ContentType = new MediaTypeHeaderValue(mediaType);
    }
    protected override Task
        SerializeToStreamAsync(Stream stream, TransportContext context)
    {
        return _stream.CopyToAsync(stream);
    }
    protected override bool TryComputeLength(out long length)
    {
        if (!_stream.CanSeek)
        {
            length = 0;
            return false;
        }
        else
        {
            length = _stream.Length;
            return true;
        }
    }
    protected override void Dispose(bool disposing)
    {
        _stream.Dispose();
    }
}

Creating a custom HttpContent class requires us to define the following two abstract methods:

protected internal abstract bool TryComputeLength(out long length)
protected abstract Task SerializeToStreamAsync(
    Stream stream, TransportContext context);

As we saw in the last section, the first one—TryComputeLength—is used to try to obtain the content length. In the FileContent implementation, this method uses the Stream.CanSeek property to query if the file stream length can be computed. If so, it uses the Stream.Length property to return the content length.

The second method, SerializeToStreamAsync, is responsible for writing the contents to the passed-in Stream. This method can operate asynchronously, returning a Task before the write is concluded. This returned Task should be signaled when the write process is finally finished. This asynchronous ability is useful when the message contents are provided by another asynchronous process (e.g., reading from the filesystem or from an external system). For instance, the FileContent implementation takes advantage of the CopyToAsync method, introduced in .NET 4.5, to start the asynchronous copy and return a Task representing this operation.

Instead of deriving directly from HttpContent, you can take an alternative approach and use the StreamContent and PushStreamContent classes, either as a base class or via factory methods. The following class shows you how to build XML-based content without requiring any buffering, by creating a PushStreamContent-derived class:

public class XmlContent : PushStreamContent
{
    public XmlContent(XElement xe)
        : base(PushStream(xe), "application/xml")
    {
    }

    private static Action<Stream,HttpContent,TransportContext>
        PushStream(XElement xe)
    {
        return (stream, content, ctx) =>
            {
                using (var writer = XmlWriter.Create(stream,
                    new XmlWriterSettings(){CloseOutput = true}))
                {
                    xe.WriteTo(writer);
                }
            };
    }
}

Since the PushStreamContent constructor requires an Action<Stream, HttpContent, TransportContext>, in the previous example we use a private static method to create this action from the given XElement. Notice also the use of the XmlWriterSettings parameter in order to close the given stream. Recall that, since the action is assumed to be asynchronous, the close of the stream signals the conclusion of this process.

We can accomplish the same goal by using an extension method on XElement:

public static class XElementContentExtensions
{
    public static HttpContent ToHttpContent(this XElement xe)
    {
        return new PushStreamContent((stream, content, ctx) =>
            {
                using (var writer = XmlWriter.Create(stream,
                    new XmlWriterSettings(){CloseOutput = true}))
                {
                    xe.WriteTo(writer);
                }
            },"application/xml");
    }
}

Conclusion

In this chapter, our focus was on the new HTTP programming model, which was introduced in version 4.5 of the .NET Framework and is at the core of both Web API and the new HttpClient class. As we’ve shown, this new model provides a more usable and testable way of dealing with the core HTTP concepts of messages, headers, and content. The following chapters build upon this knowledge to provide a deeper understanding of Web API inner workings. Namely, Chapter 11 describes the interface between this model and a lower HTTP stack, such as the one provided by ASP.NET.

Get Designing Evolvable Web APIs with ASP.NET now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial