There is a general lesson I’ve learned over a couple of decades of writing protocols small and large. I call this the “Cheap or Nasty” pattern: you can often split your work into two aspects or layers, and solve these separately—one using a “cheap” approach, the other using a “nasty” approach.
The key insight to making Cheap or Nasty work is to realize that many protocols mix a low-volume chatty part for control, and a high-volume asynchronous part for data. For instance, HTTP has a chatty dialog to authenticate and get pages, and an asynchronous dialog to stream data. FTP actually splits this over two ports; one port for control and one port for data.
Protocol designers who don’t separate control from data tend to make horrid protocols, because the trade-offs in the two cases are almost totally opposed. What is perfect for control is bad for data, and what’s ideal for data just doesn’t work for control. This is especially true when we want high performance at the same time as extensibility and good error checking.
Let’s break this down using a classic client/server use case. The client connects to the server and authenticates. It then asks for some resource. The server chats back, then starts to send data back to the client. Eventually, the client disconnects or the server finishes, and the conversation is over.
Now, before starting to design these messages, stop and think, and let’s compare the control dialog and the data flow:
The control dialog lasts a ...