Currently, I am playing around with WebSockets. This is due to an application idea I have in my mind which requires a bidirectional connection between browser and server with low latency. The communication will happen in a stream-like fashion at low bandwidth. Real network sockets using TCP/UDP are often the desired optimum for things like that, but within a browser they can only be provided by Java or Flash plugins. The future belongs to WebSockets. Implemented directly in the browser they are providing a much lower level network connection between the browser and the server than HTTP — without any plugin. WebSockets still work on a layer above TCP, but low latency and efficient data transport in both directions is warranted in a TCP-like fashion. Therefore, real-time application developers are very keen to use WebSockets. As this still is a young development, there is only few browser support and a lot of non-mature client/server libraries. Most importantly, there is a huge lack of documentation how to use these.

In this blog post, I present a very simple “echo application” where the user sends a message from his browser to the server — which in turn sends this message back to the client. Simple so far, but the main focus while realizing this is on selecting the right tools for the task, as there is already a lot of stuff out there — some very good, undocumented and hidden things, some totally overloaded things, and some bad things. I tried to fulfill the following conditions:

• Make use of a “Flashbridge” to realize a fallback when WebSockets are not available in a browser (which is true for Firefox at the moment)
• use Python on the server side
• use the best / most solid tools available
• at the same time, use the simplest tools available that do not bring along loads of stuff that you do not need for simple applications or if you want to desing your own communication protocol anyway.

Tools for the client side

First thing, the client running in the browser has to be implemented in JavaScript, of course. I do not like JavaScript, but this is the language of the browsers :-( (is there a chance to revolutionize this to Python syntax?). So, when you search the web you will at first find Socket.IO and jWebSocket. Socket.IO is consisting of the client “Socket.IO” and the server “Socket.IO-node” — designed to be used together. Yes, it is great — it seems mature, it already provides a lot of different protocols on top of WebSockets and it provides loads of fallback mechanisms if there are no WebSockets available on the client side. There even is a Python implementation of the server side. But my feeling was: Nope. I have no experience so far with WebSockets, so I would like to start implementing a basic core. I don’t want to use any communication protocol so far. Just sending messages as designed by the specification of WebSockets. Furthermore, I only want Flash fallback as an equivalently fast replacement, no slow backup techniques like xhr-multipart, xhr-polling, and jsonp-polling. Socket.IO overwhelms with a lot of functionality and few of documentation — making it difficult for newcomers to grasp the essentials.

A lot of above also applies for jWebSocket, so they have many things in common why I did not want to use them. And as it turned out while browsing through their sources, they have one very special thing in common:

web-socket-js

Socket.IO as well as jWebSocket both base their core WebSocket-Flash-switch on a very neat and small JavaScript library called web-socket-js written by Hiroshi Ichikawa. You can check this yourself by looking at Socket.IO’s repository and jWebSocket’s repository. If required, web-socket-js replaces the standard JavaScript new WebSocket constructor by its own implementation. This does the trick:

(function() {
 
  if (window.WebSocket) return;
 [...]
}

If WebSockets are supported by the browser natively, then they are used natively and web-socket-js does not do anything more than return;. If the browser lacks support, a Flash plugin Hiroshi wrote is loaded, providing the same API as the original JavaScript WebSocket. This API is implemented in the [...] of the code snippet above.

This is the way I want to start learning WebSockets: using the original API and with a lot of browser support. Decision made: on the client side, I use web-socket-js.

Tools for the server side

Answer upfront this time: gevent in combination with gevent-websocket. Gevent is a nicely performing networking library for Python, based on the brilliant idea of greenlets. It allows you to write extremely simple server code. Gevent-websocket is just a very small piece of code on top of this, basically saving you from implementing the WebSocket handshake yourself.

The code

Let’s start with websocket_echoserver.py:

from geventwebsocket.handler import WebSocketHandler
from gevent import pywsgi
import gevent
 
 
def handle_echo_client(ws):
    while True:
        msg = ws.wait()
        if msg == "quit":
            ws.close_connection()
            break
        ws.send(msg)
 
 
def app(environ, start_response):
    if environ['PATH_INFO'] == "/echo":
        handle_echo_client(environ['wsgi.websocket'])
    else:
        print "404, PATH_INFO: %s" %  environ["PATH_INFO"]
        start_response("404 Not Found", [])
        return []
 
 
server = pywsgi.WSGIServer(('', 8087), app, handler_class=WebSocketHandler)
server.serve_forever()

It listens on localhost:8087 for incoming connections. WebSocket connections are accepted and handled. If the requested resource is /echo, the greenlet handle_echo_client() deals with the client. I think it is obvious what this function does. To reach this server, the client has to connect to ws://yourhost:8087/echo.

Now the client side. Call this HTML code in your browser (make sure to set the paths to the JavaScript sources and SWF file correctly and keep everything under the same domain):

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
"http://www.w3.org/TR/html4/transitional.dtd">
<html>
<head>
<title>websocket client: send/receive test</title>
<script type="text/javascript" src=".........../web-socket-js/swfobject.js"></script>
<script type="text/javascript" src=".........../web-socket-js/web_socket.js"></script>
<script type="text/javascript">
WEB_SOCKET_SWF_LOCATION = "............/web-socket-js/WebSocketMain.swf";
function log(msg) {
    var d = new Date();
    var hour = d.getHours();
    var min = d.getMinutes();
    var sec = d.getSeconds();
    var msec = d.getMilliseconds();
    var time = hour + ":" + min + ":" + sec + ":" + msec
    document.getElementById('foo').innerHTML += time + ": " + msg + "<br />";
    }
 
function ws_init(url) {
    log("connecting to " + url + "...")
    ws = new WebSocket(url);
    ws.onopen = function() {
        log("connection established.");
        };
    ws.onmessage = function(e) {
        log("message received: '" + e.data + "'");
        };
    ws.onclose = function() {
        log("connection closed.");
        };
    }
 
function ws_send(msg) {
    log("send: " + msg);
    ws.send(msg);
    }
 
function ws_close() {
    log("closing connection..");
    ws.close();
    }
</script>
</head>
<body>
<form name="input_form">
<table>
<tr>
    <td>host/resource:</td>
    <td><input type="text" name="host" size="50" value="ws://host:port/resource"></td>
    <td><input type="button" value="connect" onclick="ws_init(document.input_form.host.value)"></td>
    <td><input type="button" value="disconnect" onclick="ws_close()"></td>
</tr>
<tr>
    <td>msg:</td>
    <td><input type="text" name="msg" size="50"></td>
    <td><input type="button" value="send" onclick="ws_send(document.input_form.msg.value)"></td>
</tr>
</table>
</form>
<div id="foo"> </div>
</body>
</html>

What is used here is the standard JavaScript WebSocket API. For me, this HTML renders as:

Now… maybe you opened this in Chrome. Then, WebSocket support is natively included and activated. In Firefox, it is disabled. There the Flashbridge would come into play. But Flash does not just send and receive data to and from anywhere. It at first asks the host who delivered the SWF file with whom it is allowed to exchange data: it expects to get a so-called socket policy file from the server, after asking him for this on port 843. Yeah, just accept this — it is not a bad idea that the internet communication of Flash plugins is restricted. Anyway, you have to set up a server listening on port 843 to provide this policy. I have made a blog post for setting up a Flash socket policy server in Python based on gevent. I just assume that you have set up such a server. Then, our HTML code from above works (at least for me) under Firefox 4+, Chrome 6+ and Internet Explorer 9. Enter your echo server’s details (ws://yourhost:8087/echo) and play around with it. For me, it looks like that:

Isn’t that great? It works. The latency times between send and receive are pretty low (at least for such small words to send): they are comparable with the ping latencies I measured against the same server. The Flashbridge transparantly does its job in Firefox and Internet Explorer and introduces around 5 ms more latency.

Conclusion

The tools used for this test (gevent, gevent-websocket and web-socket-js) are solid, perform very good and are very simple to use. The code executed in the browser uses the standard WebSocket API. But, due to web-socket-js and its nice Flashbridge, it works in all browsers supporting Flash. gevent-websocket and web-socket-js do not provide more functionality than simple message passing over WebSockets, which was desired.