screencapture-staticfuzz-com-1454195611214 — staticfuzz.com

Server-sent events efficiently sends data to clients in real-time and asynchronously. This particular setup was used for STATICFUZZ and shows you how to send an event from server/Python to client/JavaScript, plus setting up the server! This is about as full stack as it gets!

Core technologies:

JavaScript/EventSource
gevent
Flask/Python
nginx
FreeBSD

What your server will look like (endresult)

The user used for managing the web app is freebsd.
Where you’ll keep your webapp repository: /home/freebsd/exampleapp/.
Where you’ll keep the main flask app: /home/freebsd/exampleapp/exampleapp.py.
There will be an even source stream at http://example.com/stream/ which can be listened to with JavaScript’s EventSource.
The output for your app will go to /home/freebsd/exampleapp/supervisord.log.
Your web app will run as an a daemon/service, in the background, and you will be able to start/stop/restart it.
The Python virtual environment for your app will be at /home/freebsd/exampleapp/venv/bin.
The web app service will listen on port 5000, which nginx will connect to, in order to serve the app to http://example.com/.

Flask, Python

At its most basic your Flask app that has an EventSource stream will look something like this (in this example we’ll say it’s at /home/freebsd/exampleapp/exampleapp.py):

import flask
import random

from gevent.pywsgi import WSGIServer


app = flask.Flask(__name__)
app.config.from_object("config")


def event():
    """EventSource stream; server side events. Used for
    constantly making new rolls and sending that roll to
    everyone.

    Returns:
        json event (str): --

    See Also:
        stream()

    """

    while True:
        die_rolls = {u"1D6": random.randint(1, 6),
                     u"1D20": random.randint(1, 20)}
        yield "data: " + json.dumps(die_rolls) + "nn"

        with app.app_context():
            gevent.sleep(app.config['SLEEP_RATE'])


@app.route('/stream/', methods=['GET', 'POST'])
def stream():
    """SSE (Server Side Events), for an EventSource. Send
    the event of a new message.

    See Also:
        event()

    """

    return Response(event(), mimetype="text/event-stream")


if __name__ == '__main__':
    WSGIServer(('', 5000), app).serve_forever()

You’ll want to create a config.py which looks something like this:

# DEBUG is not for production!!!
DEBUG = True

# seconds
SLEEP_RATE = 0.2

Javascript: EventSource

The script below is used to listen to /stream/, waiting for the server to
specify a new event. If we fail to connect to /stream/ for listening,
we close this attempt and start a new listen attempt. Call listen to initialize actually listening for events.

function listen() {
    var source = new EventSource("/stream/");
    source.onerror = function(eventdata) {
        // relisten if fail to connect to eventsource
        this.close();
        listen();
    }
    source.onmessage = function(eventdata) {
        console.log(eventdata);
        var dice_roll = JSON.parse(eventdata["data"]);
        $.each(die_roll, function(index, dice_roll) {
            var new_li = jQuery('<li>');
            new_li.text(die_roll.1D6);
            $('#rolls').append(new_li);
        });
     }
 }

The above will add a new die roll to a list (ol, ul) with the id rolls. every time a die roll (event) is sent from the server (EventSource).

Server Config

For webapp hosting, I’m currently running FreeBSD 10.2 on DigitalOcean, highly recommended!

To get started, on FreeBSD, you’ll want to do the following short commands:

sudo pkg install nginx py27-supervisor py27-sqlite3 py27-virtualenv git
cd ~

nginx

The reverse proxy connects Internet users to the Python application server. Here’s a very minimal setup:

worker_processes  4;

events {
    worker_connections  1024;
}

http {
    include       mime.types;
    default_type  application/octet-stream;

    upstream exampleapp {
        server 127.0.0.1:5000 fail_timeout=0;
    }

    sendfile        on;

    keepalive_timeout  65;

    server {
        listen       80;
        server_name  example.com;

        location / {
            proxy_pass http://exampleapp;
            proxy_buffering off;
            proxy_cache off;
            proxy_set_header Connection '';
            chunked_transfer_encoding off;
            proxy_http_version 1.1;
        }
    }
}

Supervisor

Daemonize the application! Run sudo service supervisord restart to restart your app whenever you update your code or templates/.

Edit /usr/local/etc/supervisord.conf:

[supervisord]
logfile=/var/log/supervisord.log ; (main log file;default $CWD/supervisord.log)
logfile_maxbytes=50MB       ; (max main logfile bytes b4 rotation;default 50MB)
logfile_backups=10          ; (num of main logfile rotation backups;default 10)
loglevel=info               ; (log level;default info; others: debug,warn,trace)
pidfile=/var/run/supervisor/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
nodaemon=false              ; (start in foreground if true;default false)
minfds=1024                 ; (min. avail startup file descriptors;default 1024)
minprocs=200                ; (min. avail process descriptors;default 200)
user=freebsd                ; (default is current user, required if root)

[supervisorctl]
serverurl=unix:///var/run/supervisor/supervisor.sock ; use a unix:// URL  for a unix socket

[program:exampleapp]
autorestart = true
numprocs = 1
autostart = true
redirect_stderr = True
;stopwaitsecs = 1
startsecs = 10
priority = 99
directory = /home/freebsd/exampleapp
environment=PATH="/home/freebsd/exampleapp/venv/bin"
command = /home/freebsd/exampleapp/venv/bin/python /home/freebsd/exampleapp/exampleapp.py serve
startretries = 100
stdout_logfile = /home/freebsd/exampleapp/supervisord.log

Note that output from your script will go to stdout_logfile.

Resources, Bonus Material

What do you think?

I encourage you to reproduce this setup and post any questions or comments you have below!

Polling is a Hack: Server Sent Events (EventSource) with gevent, Flask, nginx, and FreeBSD