Migration to an internal WebSockets server

Until now, we’ve been using Pusher as the broadcasting service for the real-time collaboration features within Sprint Boards. Of course, you won’t notice anything different on your end and that’s certainly a good thing.

It won’t come as a secret for us to mention Sprint Boards is a Laravel project – we’ve made this public before. We’re proud to be using Laravel and Vue.js. One of the reasons we created Sprint Boards is because other solutions out there really lacked modern polish and used really outdated technologies. It didn’t feel right to us to be using retrospective tools that don’t follow the same high standards we demand of ourselves within our Agile team.

Laravel has out-of-the-box support for Pusher, so since January we’ve been using them to serve as the broadcasting provider. While we love Pusher, we’ll be honest by disclosing that we’ve been forced to move away because it would have been financially unjustifiable to continue using them. We tried negotiating better rates with Pusher because our projected costs over the next year were potentially as high as US$0.10 per connection, but those conversations failed to materialise into anything we could agree on, so we made the painful choice of moving away.

We try and avoid making engineering decisions for purely financial reasons, but the main function of our platform is to provide real-time collaboration, so this is quite exceptional for us.

Earlier this month, we started work to integrate our own WebSockets service using Redis. As with Pusher, Laravel makes this a breeze.

The purpose of this article is to explain some of the difficulties we came across during deployment, which we hope will help our users going through similar challenges in their own teams.

Socket.IO Server

We’re using the Laravel Echo Server by tlaverdure as the Socket.IO server. Depending on what’s most appropriate for your intended use, you can either have this sitting on the same box as your Laravel app, or have it hosted separately.

Configuring the Socket.IO Server

This couldn’t be any simpler. You simply install the NPM package globally, run laravel-echo-server init within the directory of your project (or somewhere else, if you want to store the configuration file in a specific location on your server), answer a few questions and then run laravel-echo-server start. To stop the server, just quit the process.

Automating the Start/Stop Process

Of course, in a production environment, you’re going to want to automate this process. If you use Laravel Forge, you should add a few lines to your deployment script to stop the Laravel Echo Server before a deployment begins and to start it again afterwards:

laravel-echo-server stop > /dev/null 2>&1 &

...

laravel-echo-server start > /dev/null 2>&1 &

By forwarding all output to /dev/null 2>&1, we won’t continuously see output as users start connecting to the WebSockets service. The & at the end allows us to effectively run each process in the background.

The Laravel Echo Server looks for a laravel-echo-server.json configuration file within the directory on which it is started from. If you’ve named the file differently, you can pass the --config flag like this:

laravel-echo-server start --config="laravel-echo-server.production.json" > /dev/null 2>&1 &

If you don’t want to store the Laravel Echo Server configuration file within the same directory as your project, you can store it elsewhere and pass the --dir flag to tell the Laravel Echo Server where it can find it.

Working with HTTPS Connections

If your site is served over HTTPS (and it should be), you’ll need to provide the Laravel Echo Server with the location to your SSL certificate and key for your website.

For those of you using Laravel Forge, you can retrieve the absolute path to the location of your SSL certificate within the SSL tab on the Forge front-end. The key will be located in the same directory as the certificate itself.

If you didn’t supply the location of the SSL certificate and key when you were setting up the Laravel Echo Server, you can just modify the laravel-echo-server.json configuration file directly.

Addressing the Proxy Problem

If you’re struggling to get the Laravel Echo Server working over SSL, you’ll need to proxy requests through to it instead. We use nginx, so we needed to define an upstream server to proxy requests through to. An upstream allows you to define a set of servers to proxy requests through proxy_pass.

In the nginx configuration file for your website, add the following directive to define an upstream server that WebSocket requests are passed onto:

upstream websocket {
    server 127.0.0.1:6001;
}

If you’ve configured the Laravel Echo Server to work on a port other than 6001, make sure you change the port of the upstream server defined above.

Finally, we need to add a location directive within the server{} block:

location /socket.io {
    proxy_pass https://websocket;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "Upgrade";
}

An obvious but important note, as shown above, is that the proxy_pass directive must pass the requests to the upstream server over HTTPS if your website is being served over HTTPS.

Whichever JavaScript library you are using to subscribe to channels and listen for events broadcast by Laravel, make sure you are doing so over port 80 now that you’re proxying requests from /socket.io:80 to the Socket.IO server internally.

If you’re using the Laravel Echo JavaScript library, this could be as simple as:

window.Echo = new Echo({
   broadcaster: 'socket.io',
   host: window.location.hostname,
});

Conclusions

What was supposed to be a smooth migration had a few interesting turns for us but, really, this is what keeps our work interesting. From our initial benchmarks, real-time collaboration appears to be even faster than it was before now we’re hosting our broadcasting service in-house. Pusher was no slouch either.

Beyond doubt, Laravel made this transition far less painful than it could have been. A job well done. Now for our retrospective!