Chapter 6. Request Pipeline

When a request reaches Lift, there are a number of points where you can jump in and control what Lift does, sending back a different kind of response or controlling access. This chapter looks at the pipeline through examples of different kinds of LiftResponses and configurations.

You can get a great overview of the pipeline, including diagrams, from the Lift pipeline wiki page.

See https://github.com/LiftCookbook/cookbook_pipeline for the source code that accompanies this chapter.

Debugging a Request

Problem

You want to debug a request and see what’s arriving to your Lift application.

Solution

Add an onBeginServicing function in Boot.scala to log the request. For example:

LiftRules.onBeginServicing.append {
  case r => println("Received: "+r)
}

Discussion

The onBeginServicing call is called quite early in the Lift pipeline, before S is set up, and before Lift has the chance to 404 your request. The function signature it expects is Req => Unit. We’re just logging, but the functions could be used for other purposes.

If you want to select only certain paths, you can. For example, to track all requests starting /paypal:

LiftRules.onBeginServicing.append {
  case r @ Req("paypal" :: _), _, _) => println(r)
}

This pattern will match any request starting /paypal, and we’re ignoring the suffix on the request, if any, and the type of request (e.g., GET, POST, or so on).

There’s also LiftRules.early, which is called before onBeginServicing. It expects an HTTPRequest => Unit function, so is a little lower-level than the Req used in onBeginServicing. However, it will be called by all requests that pass through Lift. To see the difference, you could mark a request as something that the container should handle by itself:

LiftRules.liftRequest.append {
  case Req("robots" :: _, _, _) => false
}

With this in place, a request for robots.txt will be logged by LiftRules.early but won’t make it to any of the other methods described in this recipe.

If you need access to state (e.g., S), use earlyInStateful, which is based on a Box[Req] not a Req:

LiftRules.earlyInStateful.append {
  case Full(r) => // access S here
  case _ =>
}

It’s possible for your earlyInStateful function to be called twice. This will happen when a new session is being set up. You can prevent this by only matching on requests in a running Lift session:

LiftRules.earlyInStateful.append {
  case Full(r) if LiftRules.getLiftSession(r).running_? => // access S here
  case _ =>
}

Finally, there’s also earlyInStateless, which like earlyInStateful, works on a Box[Req] but in other respects is the same as onBeginServicing. It is triggered after early and before earlyInStateful.

As a summary, the functions described in this recipe are called in this order:

LiftRules.early
LiftRules.onBeginServicing
LiftRules.earlyInStateless
LiftRules.earlyInStateful

Running Code When Sessions Are Created (or Destroyed)

Problem

You want to carry out actions when a session is created or destroyed.

Solution

Make use of the hooks in LiftSession. For example, in Boot.scala:

LiftSession.afterSessionCreate ::=
 ( (s:LiftSession, r:Req) => println("Session created") )

LiftSession.onBeginServicing ::=
 ( (s:LiftSession, r:Req) => println("Processing request") )

LiftSession.onShutdownSession ::=
 ( (s:LiftSession) => println("Session going away") )

If the request path is marked as being stateless via LiftRules.statelessReqTest, this example would only execute the onBeginServicing functions.

Discussion

The hooks in LiftSession allow you to insert code at various points in the session life cycle: when the session is created, at the start of servicing the request, after servicing, when the session is about to shut down, at shutdown, etc. The pipeline diagrams mentioned at the start of this chapter are a useful guide to these stages.

The list of session hooks follows:

onSetupSession: This will be the first hook called when a session is created.
afterSessionCreate: This will be called after all onSetupSession functions have been called.
onBeginServicing: This will be called at the start of request processing.
onEndServicing: This will be called at the end of request processing.
onAboutToShutdownSession: This will be called just before a session is shut down (for example, when a session expires or the Lift application is being shut down).
onShutdownSession: This will be called after all onAboutToShutdownSession functions have been run.

If you are testing these hooks, you might want to make the session expire faster than the 30 minutes of inactivity used by default in Lift. To do this, supply a millisecond value to LiftRules.sessionInactivityTimeout:

// 30 second inactivity timeout
LiftRules.sessionInactivityTimeout.default.set(Full(1000L * 30))

There are two other hooks in LiftSession: onSessionActivate and onSessionPassivate. These may be of use if you are working with a servlet container in distributed mode and want to be notified when the servlet HTTP session is about to be serialised (passivated) and deserialised (activated) between container instances. These hooks are rarely used.

Note that the Lift session is not the same as the HTTP session. Lift bridges from the HTTP session to its own session management. This is described in some detail in Exploring Lift.

Run Code When Lift Shuts Down

Problem

You want to have some code executed when your Lift application is shutting down.

Solution

Append to LiftRules.unloadHooks:

LiftRules.unloadHooks.append( () => println("Shutting down") )

Discussion

You append functions of type () => Unit to unloadHooks, and these functions are run right at the end of the Lift handler, after sessions have been destroyed, Lift actors have been shut down, and requests have finished being handled.

This is triggered, in the words of the Java servlet specification, “by the web container to indicate to a filter that it is being taken out of service.”

Running Stateless

Problem

You want to force your application to be stateless at the HTTP level.

Solution

In Boot.scala:

LiftRules.enableContainerSessions = false
LiftRules.statelessReqTest.append { case _ => true }

All requests will now be treated as stateless. Any attempt to use state, such as via SessionVar for example, will trigger a warning in developer mode: “Access to Lift’s statefull features from Stateless mode. The operation on state will not complete.”

Discussion

HTTP session creation is controlled via enableContainerSessions, and applies for all requests. Leaving this value at the default (true) allows more fine-grained control over which requests are stateless.

Using statelessReqTest allows you to decide, based on the StatelessReqTest case class, if a request should be stateless (true) or not (false). For example:

def asset(file: String) =
  List(".js", ".gif", ".css").exists(file.endsWith)

LiftRules.statelessReqTest.append {
  case StatelessReqTest("index" :: Nil, httpReq) => true
  case StatelessReqTest(List(_, file),  _) if asset(file) => true
}

This example would only make the index page and any GIFs, JavaScript, and CSS files stateless. The httpReq part is an HTTPRequest instance, allowing you to base the decision on the content of the request (cookies, user agent, etc.).

Another option is LiftRules.statelessDispatch, which allows you to register a function that returns a LiftResponse. This will be executed without a session, and is convenient for REST-based services.

If you just need to mark an entry in SiteMap as being stateless, you can:

Menu.i("Stateless Page") / "demo" >> Stateless

A request for /demo would be processed without state.

Catch Any Exception

Problem

You want a wrapper around all requests to catch exceptions and display something to the user.

Solution

Declare an exception handler in Boot.scala:

LiftRules.exceptionHandler.prepend {
  case (runMode, request, exception) =>
    logger.error("Failed at: "+request.uri)
    InternalServerErrorResponse()
}

In this example, all exceptions for all requests at all run modes are being matched, causing an error to be logged and a 500 (internal server) error to be returned to the browser.

Discussion

The partial function you add to exceptionHandler needs to return a LiftResponse (i.e., something to send to the browser). The default behaviour is to return an XhtmlResponse, which in Props.RunModes.Development gives details of the exception, and in all other run modes simply says: “Something unexpected happened.”

You can return any kind of LiftResponse, including RedirectResponse, JsonResponse, XmlResponse, JavaScriptResponse, and so on.

The previous example just sends a standard 500 error. That won’t be very helpful to your users. An alternative is to render a custom message but retain the 500 status code that will be useful for external site-monitoring services, if you use them:

LiftRules.exceptionHandler.prepend {
  case (runMode, req, exception) =>
    logger.error("Failed at: "+req.uri)
    val content = S.render(<lift:embed what="500" />, req.request)
    XmlResponse(content.head, 500, "text/html", req.cookies)
}

Here we are sending back a response with a 500 status code, but the content is the Node that results from running src/main/webapp/template-hidden/500.html. Create that file with the message you want to show to users:

<html>
<head>
  <title>500</title>
</head>
<body data-lift-content-id="main">
<div id="main" data-lift="surround?with=default;at=content">
  <h1>Something is wrong!</h1>
  <p>It's our fault - sorry</p>
</div>
</body>
</html>

You can also control what to send to clients when processing Ajax requests. In the following example, we’re matching just on Ajax POST requests, and returning custom JavaScript to the browser:

import net.liftweb.http.js.JsCmds._

val ajax = LiftRules.ajaxPath

LiftRules.exceptionHandler.prepend {
  case (mode, Req(ajax :: _, _, PostRequest), ex) =>
    logger.error("Error handing ajax")
    JavaScriptResponse(Alert("Boom!"))
}

You could test out this handling code by creating an Ajax button that always produces an exception:

package code.snippet

import net.liftweb.util.Helpers._
import net.liftweb.http.SHtml

class ThrowsException {
  private def fail = throw new Error("not implemented")

  def render = "*" #> SHtml.ajaxButton("Press Me", () => fail)
}

This Ajax example will jump in before Lift’s default behaviour for Ajax errors. The default is to retry the Ajax command three times (LiftRules.ajaxRetryCount), and then execute LiftRules.ajaxDefaultFailure, which will pop up a dialog saying: “The server cannot be contacted at this time.”

Streaming Content

Problem

You want to stream content back to the web client.

Solution

Use OutputStreamResponse, passing it a function that will write to the OutputStream that Lift supplies.

In this example, we’ll stream all the integers from one, via a REST service:

package code.rest

import net.liftweb.http.{Req,OutputStreamResponse}
import net.liftweb.http.rest._

object Numbers extends RestHelper {

  // Convert a number to a String, and then to UTF-8 bytes
  // to send down the output stream.
  def num2bytes(x: Int) = (x + "\n") getBytes("utf-8")

  // Generate numbers using a Scala stream:
  def infinite = Stream.from(1).map(num2bytes)

  serve {
    case Req("numbers" :: Nil, _, _) =>
      OutputStreamResponse( out => infinite.foreach(out.write) )
  }
}

Scala’s Stream class is a way to generate a sequence with lazy evaluation. The values being produced by infinite are used as example data to stream back to the client.

Wire this into Lift in Boot.scala:

LiftRules.dispatch.append(Numbers)

Visiting http://127.0.0.1:8080/numbers will generate a 200 status code and start producing the integers from 1. The numbers are produced quite quickly, so you probably don’t want to try that in your web browser, but instead from something that is easier to stop, such as cURL.

Discussion

OutputStreamResponse expects a function of type OutputStream => Unit. The OutputStream argument is the output stream to the client. This means the bytes we write to the stream are written to the client. The relevant line from the example is:

OutputStreamResponse(out => infinite.foreach(out.write))

We are making use of the write(byte[]) method on out, a Java OutputStream, and sending it the Array[Byte] being generated from our infinite stream.

Be aware that OutputStreamResponse is executed outside of the scope of S. This means if you need to access anything in the session, do so outside of the function you pass to OutputStreamResponse.

For more control over status codes, headers, and cookies, there are a variety of signatures for the OutputStreamResponse object. For the most control, create an instance of the OutputStreamResponse class:

case class OutputStreamResponse(
  out: (OutputStream) => Unit,
  size: Long,
  headers: List[(String, String)],
  cookies: List[HTTPCookie],
  code: Int)

Note that setting size to -1 causes the Content-Length header to be skipped.

There are two related types of response: InMemoryResponse and StreamingResponse.

InMemoryResponse

InMemoryResponse is useful if you have already assembled the full content to send to the client. The signature is straightforward:

case class InMemoryResponse(
  data: Array[Byte],
  headers: List[(String, String)],
  cookies: List[HTTPCookie],
  code: Int)

As an example, we can modify the recipe and force our infinite sequence of numbers to produce the first few numbers as an Array[Byte] in memory:

import net.liftweb.util.Helpers._

serve {
  case Req(AsInt(n) :: Nil, _, _) =>
    InMemoryResponse(infinite.take(n).toArray.flatten, Nil, Nil, 200)
}

The AsInt helper in Lift matches on an integer, meaning that a request starting with a number matches, and we’ll return that many numbers from the infinite sequence. We’re not setting headers or cookies, and this request produces what you’d expect:

$ curl http://127.0.0.1:8080/3
1
2
3

StreamingResponse

StreamingResponse pulls bytes into the output stream. This contrasts with OutputStreamResponse, where you are pushing data to the client.

Construct this type of response by providing a class with a read method that can be read from:

case class StreamingResponse(
  data: {def read(buf: Array[Byte]): Int},
  onEnd: () => Unit,
  size: Long,
  headers: List[(String, String)],
  cookies: List[HTTPCookie],
  code: Int)

Notice the use of a structural type for the data parameter. Anything with a matching read method can be given here, including java.io.InputStream`–like objects, meaning `StreamingResponse can act as a pipe from input to output. Lift pulls 8 K chunks from your StreamingResponse to send to the client.

Your data read function should follow the semantics of Java IO and return “the total number of bytes read into the buffer, or –1 if there is no more data because the end of the stream has been reached.”

Serving a File with Access Control

Problem

You have a file on disk and you want to allow users to download it, but only if they are allowed to. If they are not allowed to, you want to explain why.

Solution

Use RestHelper to serve the file or an explanation page.

For example, suppose we have the file /tmp/important and we only want selected requests to download that file from the /download/important URL. The structure for that would be:

package code.rest

import net.liftweb.util.Helpers._
import net.liftweb.http.rest.RestHelper
import net.liftweb.http.{StreamingResponse, LiftResponse, RedirectResponse}
import net.liftweb.common.{Box, Full}
import java.io.{FileInputStream, File}

object DownloadService extends RestHelper {

  // (code explained below to go here)

  serve {
    case "download" :: Known(fileId) :: Nil Get req =>
      if (permitted) fileResponse(fileId)
      else Full(RedirectResponse("/sorry"))
  }
}

We are allowing users to download “known” files. That is, files we approve of for access. We do this because opening up the filesystem to any unfiltered end user input pretty much means your server will be compromised.

For our example, Known is checking a static list of names:

val knownFiles = List("important")

object Known {
 def unapply(fileId: String): Option[String] = knownFiles.find(_ == fileId)
}

For requests to these known resources, we convert the REST request into a Box[LiftResponse]. For permitted access we serve up the file:

private def permitted = scala.math.random < 0.5d

private def fileResponse(fileId: String): Box[LiftResponse] = for {
    file <- Box !! new File("/tmp/"+fileId)
    input <- tryo(new FileInputStream(file))
 } yield StreamingResponse(input,
    () => input.close,
    file.length,
    headers=Nil,
    cookies=Nil,
    200)

If no permission is given, the user is redirected to /sorry.html.

All of this is wired into Lift in Boot.scala with:

LiftRules.dispatch.append(DownloadService)

Discussion

By turning the request into a Box[LiftResponse], we are able to serve up the file, send the user to a different page, and also allow Lift to handle the 404 (Empty) cases.

If we added a test to see if the file existed on disk in fileResponse, that would cause the method to evaluate to Empty for missing files, which triggers a 404. As the code stands, if the file does not exist, the tryo would give us a Failure that would turn into a 404 error with a body of “/tmp/important (No such file or directory).”

Because we are testing for known resources via the Known extractor as part of the pattern for /download/, unknown resources will not be passed through to our File access code. Again, Lift will return a 404 for these.

Guard expressions can also be useful for these kinds of situations:

serve {
  case "download" :: Known(id) :: Nil Get _ if permitted => fileResponse(id)
  case "download" :: _ Get req => RedirectResponse("/sorry")
}

You can mix and match extractors, guards, and conditions in your response to best fit the way you want the code to look and work.

Access Restriction by HTTP Header

Problem

You need to control access to a page based on the value of an HTTP header.

Solution

Use a custom If in SiteMap:

val HeaderRequired = If(
  () => S.request.map(_.header("ALLOWED") == Full("YES")) openOr false,
  "Access not allowed"
)

// Build SiteMap
val entries = List(
  Menu.i("Header Required") / "header-required" >> HeaderRequired
)

In this example, header-required.html can only be viewed if the request includes an HTTP header called ALLOWED with a value of YES. Any other request for the page will be redirected with a Lift error notice of “Access not allowed.”

This can be tested from the command line using a tool like cURL:

$ curl http://127.0.0.1:8080/header-required.html -H "ALLOWED:YES"

Discussion

The If test ensures the () => Boolean function you supply as a first argument returns true before the page it applies to is shown. In this example, we’ll get true if the request contains a header called ALLOWED, and the optional value of that header is Full("YES"). This is a LocParam (location parameter) that modifies the SiteMap item. It can be appended to any menu items you want using the >> method.

Note that without the header, the test will be false. This will mean links to the page will not appear in the menu generated by Menu.builder.

The second argument to the If() is what Lift does if the test isn’t true when the user tries to access the page. It’s a () => LiftResponse function. This means you can return whatever you like, including redirects to other pages. In the example, we are making use of a convenient implicit conversation from a String (“Access not allowed”) to a notice with a redirection that will take the user to the home page.

If you visit the page without a header, you’ll see a notice saying “Access not allowed.” This will be the home page of the site, but that’s just the default. You can request that Lift show a different page by setting LiftRules.siteMapFailRedirectLocation in Boot.scala:

LiftRules.siteMapFailRedirectLocation = "static" :: "permission" :: Nil

If you then try to access header-required.html without the header set, you’ll be redirected to /static/permission and shown the content of whatever you put in that page.

Accessing HttpServletRequest

Problem

You have an API to call that requires access to the HttpServletRequest.

Solution

Cast S.request:

import net.liftweb.http.S
import net.liftweb.http.provider.servlet.HTTPRequestServlet
import javax.servlet.http.HttpServletRequest

def servletRequest: Box[HttpServletRequest] = for {
  req <- S.request
  inner <- Box.asA[HTTPRequestServlet](req.request)
} yield inner.req

You can then make your API call:

servletRequest.foreach { r => yourApiCall(r) }

Discussion

Lift abstracts away from the low-level HTTP request, and from the details of the servlet container your application is running in. However, it’s reassuring to know, if you absolutely need it, there is a way to get back down to the low level.

Note that the results of servletRequest is a Box, because there might not be a request when you evaluate servletRequest—or you might one day port to a different deployment environment and not be running on a standard Java servlet container.

As your code will have a direct dependency on the Java Servlet API, you’ll need to include this dependency in your SBT build:

"javax.servlet" % "servlet-api" % "2.5" % "provided"

Force HTTPS Requests

Problem

You want to ensure clients are using HTTPS.

Solution

Add an earlyResponse function in Boot.scala redirecting HTTP requests to HTTPS equivalents. For example:

LiftRules.earlyResponse.append { (req: Req) =>
  if (req.request.scheme != "https") {
    val uriAndQuery = req.uri +
     (req.request.queryString.map(s => "?"+s) openOr "")
    val uri = "https://%s%s".format(req.request.serverName, uriAndQuery)
    Full(PermRedirectResponse(uri, req, req.cookies: _*))
  }
  else Empty
}

Discussion

The earlyResponse call is called early on in the Lift pipeline. It is used to execute code before a request is handled and, if required, exit the pipeline and return a response. The function signature expected is Req => Box[LiftResponse].

In this example, we are testing for a request that is not https, and then formulating a new URL that starts https and appends to it the rest of the original URL and any query parameters. With this created, we return a redirection to the new URL, along with any cookies that were set.

By evaluating to Empty for other requests (i.e., HTTPS requests), Lift will continue passing the request through the pipeline as usual.

The ideal method to ensure requests are served using the correct scheme would be via web server configuration, such as Apache or Nginx. This isn’t possible in some cases, such as when your application is deployed to a Platform as a Service (PaaS) such as CloudBees.

Amazon Load Balancer

For Amazon Elastic Load Balancer, note that you need to use an X-Forwarded-Proto header to detect HTTPS. As mentioned in their Overview of Elastic Load Balancing document, “Your server access logs contain only the protocol used between the server and the load balancer; they contain no information about the protocol used between the client and the load balancer.”

In this situation, modify the test from req.request.scheme != "https" to:

req.header("X-Forwarded-Proto") != Full("https")

Get Lift Cookbook now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.

Start your free trial

Chapter 6. Request Pipeline

Debugging a Request

Problem

Solution

Discussion

See Also

Running Code When Sessions Are Created (or Destroyed)

Problem

Solution

Discussion

See Also

Run Code When Lift Shuts Down

Problem

Solution

Discussion

See Also

Running Stateless

Problem

Solution

Discussion

See Also

Catch Any Exception

Problem

Solution

Discussion

See Also

Streaming Content

Problem

Solution

Discussion

InMemoryResponse

StreamingResponse

See Also

Serving a File with Access Control

Problem

Solution

Discussion

See Also

Access Restriction by HTTP Header

Problem

Solution

Discussion

See Also

Accessing HttpServletRequest

Problem

Solution

Discussion

Force HTTPS Requests

Problem

Solution

Discussion

Amazon Load Balancer

Don’t leave empty-handed

It’s yours, free.

Check it out now on O’Reilly