Celebrating our one year anniversary

This month, we are celebrating our one year anniversary. Since last year, more than 1,900 registered users have created over 3,800 boards, with teams around the world collectively publishing more than 30,000 cards. We’d like to thank all of our users for helping us to achieve this great milestone.

As we’ve grown significantly over the last year, we need to take the next step forward by registering ourselves as a limited company. We are now incorporated as Bytewave Ltd, registered in England and Wales under company number 11764616.

If you are a paid subscriber, you’ll notice our company name and address listed in future invoices. Of course, your subscription otherwise remains the same and we’d like to thank all of our subscribers for helping us to continue this work.

We hope you enjoy using Sprint Boards as much we enjoy building it.

Ben Stones
Company Director

Migration to an internal WebSockets server

Until now, we’ve been using Pusher as the broadcasting service for the real-time collaboration features within Sprint Boards. Of course, you won’t notice anything different on your end and that’s certainly a good thing.

It won’t come as a secret for us to mention Sprint Boards is a Laravel project – we’ve made this public before. We’re proud to be using Laravel and Vue.js. One of the reasons we created Sprint Boards is because other solutions out there really lacked modern polish and used really outdated technologies. It didn’t feel right to us to be using retrospective tools that don’t follow the same high standards we demand of ourselves within our Agile team.

Laravel has out-of-the-box support for Pusher, so since January we’ve been using them to serve as the broadcasting provider. While we love Pusher, we’ll be honest by disclosing that we’ve been forced to move away because it would have been economically unjustifiable to continue using them. We tried negotiating better rates with Pusher because our projected costs over the next year were potentially as high as US$0.10 per connection, but those conversations failed to materialise into anything we could agree on, so we made the painful choice of moving away.

We try and avoid making engineering decisions on purely economic terms, but the main function of our platform is to provide real-time collaboration, so this is quite exceptional for us.

Earlier this month, we started work to integrate our own WebSockets service using Redis. As with Pusher, Laravel makes this a breeze.

The purpose of this article is to explain some of the difficulties we came across during deployment, which we hope will help our users going through similar challenges in their own teams.

Socket.IO Server

We’re using the Laravel Echo Server by tlaverdure as the Socket.IO server. Depending on what’s most appropriate for your intended use, you can either have this sitting on the same box as your Laravel app, or have it hosted separately.

Configuring the Socket.IO Server

This couldn’t be any simpler. You simply install the NPM package globally, run laravel-echo-server init within the directory of your project (or somewhere else, if you want to store the configuration file in a specific location on your server), answer a few questions and then run laravel-echo-server start. To stop the server, just quit the process.

Automating the Start/Stop Process

Of course, in a production environment, you’re going to want to automate this process. If you use Laravel Forge, you should add a few lines to your deployment script to stop the Laravel Echo Server before a deployment begins and to start it again afterwards:

laravel-echo-server stop > /dev/null 2>&1 &

...

laravel-echo-server start > /dev/null 2>&1 &

By forwarding all output to /dev/null 2>&1, we won’t continuously see output as users start connecting to the WebSockets service. The & at the end allows us to effectively run each process in the background.

The Laravel Echo Server looks for a laravel-echo-server.json configuration file within the directory on which it is started from. If you’ve named the file differently, you can pass the --config flag like this:

laravel-echo-server start --config="laravel-echo-server.production.json" > /dev/null 2>&1 &

If you don’t want to store the Laravel Echo Server configuration file within the same directory as your project, you can store it elsewhere and pass the --dir flag to tell the Laravel Echo Server where it can find it.

Working with HTTPS Connections

If your site is served over HTTPS (and it should be), you’ll need to provide the Laravel Echo Server with the location to your SSL certificate and key for your website.

For those of you using Laravel Forge, you can retrieve the absolute path to the location of your SSL certificate within the SSL tab on the Forge front-end. The key will be located in the same directory as the certificate itself.

If you didn’t supply the location of the SSL certificate and key when you were setting up the Laravel Echo Server, you can just modify the laravel-echo-server.json configuration file directly.

Addressing the Proxy Problem

If you’re struggling to get the Laravel Echo Server working over SSL, you’ll need to proxy requests through to it instead. We use nginx, so we needed to define an upstream server to proxy requests through to. An upstream allows you to define a set of servers to proxy requests through proxy_pass.

In the nginx configuration file for your website, add the following directive to define an upstream server that WebSocket requests are passed onto:

upstream websocket {
    server 127.0.0.1:6001;
}

If you’ve configured the Laravel Echo Server to work on a port other than 6001, make sure you change the port of the upstream server defined above.

Finally, we need to add a location directive within the server{} block:

location /socket.io {
    proxy_pass https://websocket;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "Upgrade";
}

An obvious but important note, as shown above, is that the proxy_pass directive must pass the requests to the upstream server over HTTPS if your website is being served over HTTPS.

Whichever JavaScript library you are using to subscribe to channels and listen for events broadcast by Laravel, make sure you are doing so over port 80 now that you’re proxying requests from /socket.io:80 to the Socket.IO server internally.

If you’re using the Laravel Echo JavaScript library, this could be as simple as:

window.Echo = new Echo({
   broadcaster: 'socket.io',
   host: window.location.hostname,
});

Conclusions

What was supposed to be a smooth migration had a few interesting turns for us but, really, this is what keeps our work interesting. From our initial benchmarks, real-time collaboration appears to be even faster than it was before now we’re hosting our broadcasting service in-house. Pusher was no slouch either.

Beyond doubt, Laravel made this transition far less painful than it could have been. A job well done. Now for our retrospective!

Avoiding rolling conversations during standups

Standups are one of the most important ceremonies in an Agile team. It keeps members up to date with the overall status of an ongoing sprint and encourages individual accountability by requiring members to discuss what they’re working on and any impediments that might be affecting them.

By discussing these issues as a team, it also provides an opportunity for colleagues to raise questions, provide solutions and to offer advice. What it should not do, however, is create rolling conversations about ongoing issues or future tasks that ought to be discussed at a separate meeting.

As a general rule, a standup should not last more than ten to fifteen minutes, or two minutes per person. We have found this to be enough time to discuss what we did yesterday, what we’re doing today and any blockers we have encountered or might encounter.

Many people suggest forcing teams to remain standing (as should be expected) helps keep these ceremonies short, but we have found this depends entirely on the environment. If members are stood staring at a project white board while taking turns to discuss what they’re working on instead of facing other members of the team, it can actually encourage ongoing conversations which are not appropriate for a standup.

By contrast, having members face each other while standing seems to reduce the risk of a short conversation turning into a rolling discussion that should ideally be parked for a separate meeting. This is because standups are usually prolonged as a result of two people discussing specific issues with each other; if they are facing other members of the team, it becomes far more obvious to them that they’re making other colleagues remain standing just to listen to their surely interesting, but irrelevant, one-to-one conversation.

Another common problem with standups is members not participating correctly. As soon as you start witnessing colleagues just pointing out what they’re working on today without properly discussing what they did yesterday and what blockers they might encounter, you are no longer participating in a standup and are violating a principle rule of Agile.

Standups are designed to enforce shared accountability. The reason it is important to discuss what you actually did yesterday is because it might differ from what you said you would do yesterday; and if that’s the case, other members may choose to raise that discrepancy with you if a reasonable explanation wasn’t provided as to why. This in itself is a form of shared accountability; it is the expectation that other members will call you out if you don’t properly explain what you did yesterday, what you are doing today and any impediments you might encounter. This increases individual responsibility to get work done. There is a reason standups are designed this way.

If you allow people to fall back to past habits, you can end up finishing sprints with surprise burndown charts that need to be explained to the rest of your organisation.

Strive for simplicity over complexity.

In most aspects of life, people seem to assume the higher you get, the more complex your life has to become. Whether that’s starting a family, running a business or accepting a promotion with more responsibilities.

The problem with this thought process is that by not challenging against the need for more complexity, you are reducing your ability to have the necessary clarity and focus on the things that actually matter the most. There’s a big difference between something that is inherently complex (like string theory) and something unnecessarily complex (like laws and regulations).

Take running a company: a lot of responsibility comes with this role. You might be reporting into multiple investors, managing several teams and be under commitments with many people across your organisation. You’ll also be responsible for the strategic direction of the company.

When faced with these kind of challenges, people often become confused about their priorities; often caused by personal pressure, illogical thinking and from distractions by those around them. The most important lesson anyone can learn is to listen to yourself and to follow your instinct. Don’t allow the thought process of others take place of yours.

An interesting problem that faces software engineers in particular is the balance between getting a product to market (the economic demand) and the desire to maintain technical quality (the intellectual demand). Agile methodology teaches us a way to balance both priorities by encouraging discussion between the stakeholders to identify the minimum viable feature set possible within the provided timescales.

This process allows teams to dissect the real requirements of a project that might have many moving parts; and by having all stakeholders involved in that process, each side has the opportunity to see and understand the challenges facing other colleagues across the organisation. By simplifying the entire process, it encourages teams to break up requirements into manageable chunks and to discourage decision-makers from making decisions without being fully aware of the consequences.

The truth is, complexity is in the eyes of the beholder. By identifying ways to break down and simplify challenges, we are far more capable at solving them. This applies to software developers in the same way as it applies to senior managers. You should always challenge the need for complexity and forever strive for the elegance of simplicity.

When work gets tough, work smarter – not harder.