The WebSocket Protocol – Past Travails Are To Be Avoided

The WebSocket protocol is a new facility; originally conceived as part of the HTML5 effort. Together with its applications programming interface (API), the WebSocket protocol provides a standard framework for ongoing communications between web clients and servers. The authors of the protocol deserve kudos for leveraging the existing HTTP/HTTPS infrastructure to provide an extended-lifetime link between a web browser and one or more web servers. However, the specification includes decisions that need revision and deficiencies in drafting that beg correction. In my opinion, these shortcomings constrain the specification unnecessarily while realizing no commensurate benefits.

This opinion was originally based upon a reading of the February 11, 2011 draft.^[1] In the interim, the February 25, 2011, revision of the WebSocket protocol candidate RFC has been released.^[2]

There are at least five areas of concern in the draft specification:

• the distinction between UTF-8^[3] text and binary messages is unnecessary
• Section 1.4, entitled “Closing Handshakes” has an error. While the error is not conceptual, it is a structural and drafting error that requires correction
• the consequences associated with large numbers of long-duration connections on network and host operating system infrastructure appear underappreciated. Ephemeral port saturation is but one of the possible consequences
• the lack of provision for multiplexing several sub-channels on the single connection unnecessarily aggravates the problems engendered by large numbers of open WebSocket protocol connections
• the connection of the WebSocket protocol to TCP is unnecessary and should be removed. Instead, the underlying functionality required by the WebSocket protocol should be explicitly enunciated

Many speak of how computer systems “evolve.” However, the invocation of “evolution” is a split reference. Evolution has two meanings: “unfolding over time;” or the biosciences meaning of “speciation and decimation.” Both meanings are problematic. The web community can ill afford to to try several different approaches. In the same sense, arguments for later revision are not helpful. Past experience has shown that once a protocol is adopted, it has proven all but impossible to effect change (e.g., SMTP, DNS).

Previous versions of the WebSocket protocol^[4] used delimited records (a feature noted on the WebSockets Wikipedia page as late as March 11, 2011^[5]). The revision to a counted record format correctly avoids a path already explored on numerous occasions with other protocols and found wanting. It represents a choice whose correction at a later time would be increasingly difficult to correct. Once adoption has reached critical mass, interfaces of necessity freeze. Installed base transition and the complexity of post-acceptance code bases make significant revisions difficult and painful, if not impossible, for the community. In effect, the WebSocket protocol community will have condemned itself to recapitulating problems similar in repercussions to those experienced with early versions of HTML: once a choice is adopted, the installed base's inherent inertia effectively precludes later correction.

Text/Binary Records

From a web developer's perspective, there is little reason for distinguishing between text and binary at the protocol level. I would not propose regression to naked streams, a là TCP, but rather actual records or messages with transparent content (e.g., binary text with no special characters; more akin to the Stream Control Transmission Protocol, known as “SCTP”). In limiting character sets (UTF-8) and using reserved in-stream delimiters (0x00 and 0xff), previous versions of the WebSocket protocol were embarking upon a well-traveled road that has often led to problems. Each time, recovery from this choice has been painful and expensive. It is far past the time that we should heed the George Santayana^[6] adage: “Those who cannot remember the past are condemned to repeat it.” In its current form, the WebSocket protocol has partially corrected this decision, and is now avoiding such problems. Removing the distinction between UTF-8 and binary would remove the last vestiges of this now-discarded decision.

The distinction between text and binary records and the specification of the UTF-8 character set both rest upon what I believe to be an incorrect classification of the WebSocket protocol as an applications protocol and a related incorrect interpretation of the applicable IETF standard, Policy on Character Sets – RFC 2277.^[7] The WebSocket protocol is more appropriately characterized as a providing a limited, transport-level tunnel over an existing transport (e.g., TCP) using an HTTP-compatible initial handshake. Operating over a transport protocol does not in and of itself make the WebSocket protocol an applications protocol. In this respect, a more appropriate categorization of the WebSocket protocol is similar to that of Point to Point Tunneling Protocol (PPTP; RFC 2637^[8]), which itself operates over TCP. PPTP is clearly a transport protocol. The addition of multiplex support within the WebSocket protocol only makes this classification even more manifest.

Once the categorization of the WebSocket protocol as a transport protocol is accepted, the context in which UTF-8 is relevant becomes far clearer. It is appropriate that the headers and other information used to negotiate the connection be UTF-8 encoded, a point made in RFC 2277. Once the WebSocket protocol connection has been established, however, it is just as clear that the distinction between text and binary data serves no useful purpose. It is far more efficient and effective to distinguish between binary and text at the levels of mutually orthogonal applications-level protocols, each of which uses the WebSocket protocol as an underlying transport, as I will describe in more detail later in this article.

Data communication protocols have a rich history. The early days of that history are filled with a wide cast of protocols, many of which have faded from popular use. However, the issues faced by these protocols remain. The changes protocols underwent over time trying to resolve these problems are examples worth heeding; not experiments to be repeated.

The OSI and Internet layered models for network protocols were not formulated out of whole cloth. Rather, they are answers to difficulties encountered with communications protocols without layered architectures. A protocol layered above TCP cannot replace part of TCP, it does not have the necessary access. It must live within the universe of rules imposed by TCP.^[9]

TCP implements a single bidirectional stream between two network applications. There is no secondary channel. This was a conscious design decision. It was not the only possible choice. In contrast, DECnet’s architects chose a different path.

DECnet's Network Services Protocol^[10] has two application-visible differences from TCP: message-based communication and a provision for out-of-band messages. These differences have affected the implementation of protocols and applications at higher levels. TCP connections have no corresponding construct. This is one of the reasons why FTP requires two connections: one for control and one for data. DECnet's used for FTP-like operations: the Data Access Protocol,^[11] has a far more complex scope than merely transferring files yet only requires a single network connection.

Closing Handshake

The drafting error in Section 1.4 of the draft WebSocket protocol specification,^[12] is straightforward. Currently, the text reads:

“… replace the TCP closing handshake (FIN/ACK).”

The WebSocket protocol cannot replace the TCP close message exchange. Fundamentally, the WebSocket protocol is a TCP user. It occupies an architectural level outside of TCP. The present phrasing is incorrect, in that it both conceals the real problem and misstates the appropriate corrective measure. A better statement would be:

“In the presence of proxies and other intermediaries, transport-level closing handshakes may be subject to indeterminate delays. For this reason, the WebSocket protocol includes its own closing handshake independent of the disconnect event provided by the underlying transport (e.g., TCP). In any event, the receipt of a stream close or link disconnect event from the underlying transport shall be treated in the same way as an explicit WebSocket protocol close. WebSocket protocol implementations shall properly deal with transport-level disconnect events that may occur at any arbitrary time (e.g., within a segment).”

The restated phrasing correctly represents the WebSocket protocol's relationship to its underlying transport, presently TCP (the same comment would apply to a WebSocket protocol implementation over SCTP or any other transport protocol). It properly identifies a problem posed by the behavior of the transport-level protocols, together with a solution at the Websocket protocol level. The revised wording restates this function, together with the formerly implied, but not explicitly stated, requirement that implementations of the WebSocket protocol deal with unexpected disconnects of the underlying transport at any point, regardless of cause. This is a requirement for any applications-level network protocol.

The close connection message sent via TCP cannot replace the TCP FIN/ACK sequence. It can merely provide an application with a way of signaling to its partner that the connection should be closed. It is certainly good practice, but it does not eliminate the possibility that the transport-level link will disconnect unexpectedly.

Similarly, ping/pong messages to the partner application serve a supplemental purpose to keep-alive and other functions at the transport-level, they are not replacements.

Long Duration Connection Maintenance

Successful adoption of the WebSocket protocol will shift the balance of network connections by increasing the number of long-duration TCP connections. Presently, many connections are repetitive, but of relatively short duration. This change in traffic mix represents a network infrastructure hazard. Of particular concern are the firewalls at both client organizations and server-side traffic distribution devices.

For the last two decades, HTTP (Hyper Text Transport Protocol) has been the most popular protocol on the web. By design, HTTP was not intended to maintain long-term connections. Original versions of HTTP were limited to a single page per connection.^[13] Put another way, one of the original design goals of HTTP was to limit the lifespan of the TCP connections required to transfer pages.^[14]

This decision is a tradeoff. Establishing a TCP connection to a server and the overhead of negotiating the HTTP transaction indisputably requires several messages and over 1,000 characters of message exchange. Thus, maintaining a link to an intranet quote server for stock prices that are updating several times per second is eminently logical. Maintaining a connection to an external resource for hourly updates, or even an update every couple of minutes, is a far different proposition. The problem rarely lies with the user’s web browser or with the end server itself. Rather, the problem lies in the proxy interfaces and firewalls on both sides of the network.

The underlying problem is one of resource starvation. Problems will not occur with mere handfuls of WebSocket protocol connections. Rather, problems will appear after the WebSocket protocol is broadly adopted. The problem is unlikely to appear in local contexts (e.g., trading desktops connected to local servers). Starvation (or saturation, from a slightly different perspective) is far more likely to occur with mass adoption by a broadly-used consumer application (e.g., Facebook, Twitter, Google Gmail). Substituting long-standing connections using the WebSockets protocol instead of intermittent HTTP/HTTPS connections dramatically alters the aggregate traffic and connection population.

What are the consequences of using persistent connections for page and feed updates? Persistent connections dramatically increase the number of open TCP connections in existence at any moment.

There is no single answer. How many operating systems and network stacks reasonably handle tens of thousands of connections at a time to a single host? Are there performance implications? Finally, there is the question of ephemeral port exhaustion.

Each TCP link uses two ports, one at each end.^[15] The destination port must be a known port. When a TCP connection is established for a well-known protocol (e.g., HTTP), the destination port is the assigned port while the local port is a temporary port, referred to as an ephemeral port. IANA maintains the registry of well-known ports (e.g., for HTTP, port 80; HTTP/TLS, referred to often as HTTPS uses port 443).^[16] Socket numbers are restricted to 16-bits in IPv4 and IPv6, so there are a total of 65,536 possible sockets. Of these theoretical 65,536 possible sockets, 25% (16,384; ports 49,152—65,536) are reserved for use as ephemeral ports. Since the advent of IP, this has rarely been a problem.

From security and software engineering perspectives, it is desirable for port recycling to have extended periods of disuse for individual port numbers. It guards against predictable port numbers being used in a variety of attacks.^[17] In this context, the 16,384 port population of ephemeral ports is already arguably far too small.

This shortage is not easily solved. The problem occurs in two different network locations: the Internet side of client firewalls and the DMZ/Intranet side of server firewalls. Both cases are the aggregated side of the firewall (outside for clients, inside for servers). One possible solution is to multi-home^[18] the aggregated side of the firewall. On the server side, where the firewall already acts as a traffic director and resides typically within a non-propagating IPv4 address range, this is certainly feasible.

Client firewalls are the more challenging problem. Here, the problem occurs on the public Internet-facing side of the firewall. Multi-homing is not an answer, particularly with IPv4 connections where IP address space is in short supply.

Current HTTP applications make repeated requests for update from servers. Each time, a connection is created, used, and then destroyed. Connections rarely live for more than a handful of seconds. The adoption of the WebSocket protocol will alter this state of affairs. The WebSocket protocol means that links will be open between web browsers and servers continuously for a period of time.

The nature of web applications is also a significant factor. Web applications are not purpose-developed symbiotically-paired client and server-side code. On the contrary, client-side applications increasingly take the form of dashboards composed of numerous independently scripted elements, each of which in turn reference a plethora of libraries. It is unreasonable to presume that these independently evolved code bases will coordinate the use of a single WebSocket protocol connection as currently conceived. It is far more plausible to foresee that each code base will independently open one or more WebSocket protocol links, even though many of these links will inevitably connect to the same network server, if not endpoint. This plethora of WebSocket protocol connections will cause needless consumption of system and network resources, creating bottlenecks.

Pause/Resume

A parallel approach to reducing network infrastructure usage is link pause/resume. Pause/resume is a way to maintain the applications-level context of a protocol link without actually maintaining the underlying network link.

In the WebSocket protocol context, a pause/resume facility would consist of a unique session identifier provided by the server side of the WebSocket protocol connection, together with the additions to the WebSocket protocol to allow the client to re-establish a previously paused connection at a later time by supplying the session identifier when re-establishing the connection.

Properly structured, this feature also provides a mechanism that can be used to manage load on a WebSocket protocol server. If there is a resource saturation event imminent, a WebSocket protocol server can use pause (based upon its own criteria) to reduce resource load in a controlled fashion. In concert with appropriate server software infrastructure, the same facility can be used to shift load to a different server, accommodating changes in hardware and network infrastructure without the risk of application downtime.

Multiplexing

The present WebSocket protocol specification does not provide for multiplexing. This is a weakness that will likely severely impact applications using the WebSocket protocol. Ephemeral port congestion is directly related to both the number of simultaneously open connections and the lifetime of those connections.

Multiplexing administered at the WebSocket protocol directly attacks ephemeral port saturation collapsing many parallel WebSocket protocol connections to a single WebSocket protocol connection between the web browser and a network server. It is easy to conceive of scenarios, particularly involving stock trading and other dashboards of complex information, where multiplexing could reduce ephemeral port usage significantly, in some cases by more than an order of magnitude.

Establishing a sub-channel within the context of a WebSocket protocol link requires designation of the desired remote end-point and the desired protocol within that sub-channel. While I have not sat down and designed a complete specification, certain features seem clear. Sub-channels can independently speak differing protocols layered on top of the sub-channel, and character sets (or lack thereof) and a variety of other attributes are best specified at the sub-channel level, rather than the WebSocket protocol connection itself.

In terms of naming, an approach based upon URIs, particularly TAG URIs, ^[19] would permit the use of a wide variety of standard, productized, and custom server components communicating through a single WebSocket protocol connection to an external server. This would significantly reduce the number of WebSocket protocol links which must be sustained to a single server.

I have not fully designed the mechanism to establish a sub-channel. Setting up a sub-channel would require an initial dialogue similar in character to the WebSocket protocol itself. The initial information supplied is a URI for the desired service, with a following supplemental record providing additional information required (e.g., character set, binary/non-binary, compression). Sub-channels may be established at any point, and disconnected at any time. Provisions for 256 independent sub-channels, with sub-channel creation/deletion at any time during the life of a WebSocket protocol connection would significantly reduce overhead and the potential for ephemeral port saturation.

TCP Presumption

There is no reason for tying the WebSocket protocol to TCP. On the contrary, a review of the current draft specification reveals no reason that the WebSocket protocol could not be implemented utilizing another transport-level connection (e.g., SCTP, DECnet NSP). The linkage to TCP is not a requirement, it is an accidental relic of the initial connection to HTTP, and its nominal, but not obligatory use of TCP.^[20]

A better approach would be to excise the material directly relating to TCP from the WebSocket protocol specification. In the interests of limiting specification entropy, the best choice would be to move TCP-specific material (including the specification of the schemes that indicate TCP as a transport, ws:// and wss://) to a separate, dependent RFC. This dependent RFC would be entitled “WebSockets over TCP.” Notes for implementation of the WebSocket protocol over other underlying transports would be titled similarly.

Similarly, the specifications of the applications-level protocols that use the WebSocket protocol as a transport are orthogonal both to the WebSocket protocol and to the transport protocol(s) over which the WebSocket protocol is operating.

Proposed WebSocket Protocol Family RFCs/Private Specification

It is worth noting that Simple Mail Transport Protocol (RFC 5321)^[21] does not specify TCP, it merely recites the properties required of the underlying protocol. The original Mail Transport Protocol^[22] clearly states that it is designed to be implemented over TCP, NCP (TCP's non-IP predecessor), FTP, and other protocols. Indeed, there have been implementations of SMTP over protocols other than TCP, both within the IP universe and outside of it (e.g., DECnet).

Grammar

Lastly, there is an almost grammatical question. According to the specification, the name of the protocol is “WebSocket Protocol.” There seems to be an emerging consensus to refer to the protocol as “WebSockets.” The change from “the WebSocket protocol” to “WebSockets” makes for smoother reading text when both writing and speaking. With all due respect, I suggest that the official specification be re-titled “WebSockets: A Protocol ….”

Conclusion

In this author's opinion, the WebSocket protocol is a significant step forward in browser/server communication. However, several steps should be undertaken before adoption of the WebSocket protocol; and before the WebSocket protocol is employed for applications that attract, or potentially attract, significant numbers of users:

• remove the distinction between UTF-8 text and binary records at the protocol level
• correct the drafting error in Section 1.4
• more extensive characterization of the tradeoff between the overhead of establishing a connection with the costs of maintaining an open connection using the WebSocket protocol
• definition of a cooperative mechanism to permit server and client to agree to discontinue the WebSocket transport connection; with provisions of a mechanism(s) to maintain context for later reestablishment of the connection with abbreviated handshaking at a later point in time, and thus effectively increasing the liquidity of transport-level resources (e.g., TCP ephemeral ports)
• enhance the protocol to include provisions for multiplexing multiple (preferably at least 256) streams of frames within the WebSocket protocol. This would require two opcodes: Open/Close-subStream; together with the appropriate provisions for subStream feature negotiation
• remove explicit references to TCP. Instead, the base specification for the WebSocket protocol should specify the functionality required of the underlying protocol used as a transport.
• change the title from “The WebSocket Protocol …” to “WebSockets: A Protocol for …” to produce smoother written descriptions (re-read this article substituting “WebSockets” for “the WebSocket protocol”)

With these adjustments, the WebSocket protocol will have a smaller footprint, be far more scalable, and thus may be employed in widespread production use with higher performance and a lesser chance of resource and other problems.

Notes

[1]	Ian Fette (2011, February 25) “The Web Socket protocol,” Revision 6
[2]	Ian Fette (2011, February 2) “The Web Socket protocol,” Revision 5
[3]	Francois Yergeau (2003, November) “UTF-8, a transformation format of ISO 10646” RFC 3629, STD 63
[4]	Ian Hickson (2010, May 23) “The WebSocket protocol”, pp 5
[5]	From Wikipedia page entitled “Websocket”: retrieved on March 11, 2011 “…Once established, WebSocket data frames can be sent back and forth between the client and the server in full-duplex mode. Text frames can be sent full-duplex, in either direction at the same time. The data is minimally framed with just two bytes. Each frame starts with a 0x00 byte, ends with a 0xFF byte, and contains UTF-8 data in between.[emphasis mine] Binary frames are not supported yet in the API. WebSocket text frames use a terminator, while binary frames use a length prefix. …”
[6]	George Santayana (1905) Life of Reason, Reason in Common Sense Scribner's, page 284
[7]	Harald Alvestrand (1998, January) “IETF Policy on Character Sets and Languages” RFC 2277, BCP: 18
[8]	Kory Hamzeh, Gurdeep Singh Pall, William Verthein, Jeff Taarud, W. Andrew Little, and Glen Zorn (1999, July) “Point-to-Point Tunneling Protocol (PPTP)” RFC 2637
[9]	R. Stewart, Q. Xie, K. Morneault, C. Sharp, H. Schwarzbauer, T. Taylor, I. Rytina, M. Kalla, L. Zhang, V. Paxson (2000, October) “Stream Control Transmission Protocol” RFC 2960
[10]	Digital Equipment Corporation (1984, July) “DECnet DIGITAL Network Architecture NSP Functional Specification, Phase IV, Version 4.0.1”
[11]	Digital Equipment Corporation (1980, March 28) “DECNET DIGITAL Network Architecture, Data Access Protocol (DAP) Functional Specification, Version 5.6”
[12]	Ian Fette (2011, February 8) “The Web Socket protocol” pp
[13]	R. Fielding, J. Gettys, J. Mogul, H. Frystyk, L. Masinter, P. Leach, T. Berners-Lee (1999, June) “Hypertext Transfer Protocol -- HTTP/1.1” RFC 2616
[14]	The original HTTP specifications required a rule of “one connection, one file.” RFC 2616[13] (the final HTTP V1.1 specification) relaxes this rule as a performance enhancement. The stated issue is the overhead in the handshake processing required to establish a TCP connection.
[15]	TCP is chosen for pedagogy. Other protocols (e.g., SCTP, DECnet NSP, OSI) have similar issues, although the precise terminology may differ.
[16]	IANA (2011, February 16) “Assigned Port Numbers”
[17]	Mark Allman (2009, April) “Comments on Selecting Ephemeral Ports”
[18]	A host is said to be multi-homed if it has more than a single IP address, often on the same physical adapter. This is normally done to allow a single machine to have direct access to multiple subnets without requiring a router to intervene. A side effect of this is the expansion of the potential number of available ephemeral ports.
[19]	Tim Kindberg and Sandro Hawke (2005, October) “The ‘tag’ URI Scheme” RFC 4151
[20]	R. Fielding, J. Gettys, J. Mogul, H. Frystyk, L. Masinter, P. Leach, T. Berners-Lee (1999, June) “Hypertext Transfer Protocol -- HTTP/1.1” RFC 2616, pp13
[21]	John Kleinsin (2008, October) “Simple Mail Transport Protocol” RFC 5321, pp 5
[22]	Suzanne Sluizer and Jon Postel (1980, September) “Mail Transport Protocol” RFC 772

References

Biography

Robert Gezelter has over 30 years of experience as a systems programmer/architect. He has implemented and managed data communications in a wide range of environments using protocols including TCP/IP, DECnet, BiSynch, and others. He has also architected, designed, and implemented protocols at a variety of levels from local area networks to the communications between hardware elements in custom signal processing systems and computer peripheral equipment.

He is an accomplished speaker within the United States and internationally. He is a Senior Member of the IEEE, and an alumnus of the IEEE Computer Society's Distinguished Visitor Program (2004-6).

He is the author “Internet E-Mail Architecture” in the Handbook of Information Security (H. Bidgoli, ed., 2005, Wiley), as well as a Contributing Editor for the Computer Security Handbook since 1995.

URLs for referencing this entry

• Home
• Services
  • OpenVMS
  • Windows
  • Printers
  • PostScript
  • Architectures
  • Networks
  • Embedded Systems
• Speaking
• Upcoming Events
• TechnicalCounsel^SM News Archive
• Ruminations – An IT Blog 
  • March 23, 2011
  • March 6, 2011
  • February 28, 2011
  • February 9, 2011
  • January 10, 2011
  • November 26, 2010
  • November 22 2010
  • November 2, 2010
  • October 25, 2010
  • September 28, 2010
  • August 31, 2010
  • August 23, 2010
  • June 21, 2010
  • June 14, 2010
  • May 25, 2010
  • May 12, 2010
  • March 2, 2010
  • February 8, 2010
  • December 21, 2009
  • December 7, 2009
  • November 24, 2009
  • June 25, 2009
  • May 8, 2009
  • May 5, 2009
  • March 31, 2009
  • March 18, 2009
  • February 20, 2009
  • January 21, 2009
• The OpenVMS Consultant 
• Presentations
• Publications
• Technology Demonstrations
• Bookstore
• Contact Us

http://www.rlgsc.com

+1 (718) 463 1079