Sprint Boards 2.0: here’s what’s new

We’ve been quite busy for the past several months. In late April, we announced that we would be migrating to the React framework and building an entirely new OAuth2-compatible API.

After many long hours, late nights and weekends, we are so thrilled to announce it’s now live!

You will most likely notice how much faster our site is now: that’s because we’ve migrated to a ‘single-page app’ architecture and we make aggressive use of code splitting which means only the resources you need are downloaded. It works and behaves like a pre-installed app on your device, but in your web browser.

Every line of code has been reviewed and optimised: while there aren’t many front-facing changes that you can see, the front-end has been entirely rebuilt from the ground-up and the back-end architecture is all new.

We’ve also made improvements to our deployment process: we are now able to apply bug fixes and feature improvements without taking the service offline.

However, we did manage to fit in some great front-facing improvements that we think you’ll really appreciate:

  • We’ve refined the look and feel of your boards to make better use of horizontal space: tool actions are now on the left-side (on desktops), cards have a much cleaner colour scheme and are a lot more compact and easier to read.
  • You can now right-click on cards to perform actions such as edit, delete, highlight and copy.
  • We no longer allow users to like cards multiple times and we now show you the cards you’ve liked.
  • We’ve improved Slack integration so you can now choose the channel to send board invites into.
  • We now automatically encrypt new subscriber boards: you no longer need to store your own passwords.
  • You can now customise the column titles when creating a new board. We will soon be supporting the ability to add and remove columns (and yes, also being able to add more than five – watch this space).

Send us your feedback

We’d love to hear what you think. Please email [email protected].

Upcoming changes to Sprint Boards

A short summary:

We will be:

  • building an entirely new OAuth2-compatible API
  • moving away from Vue.js in favour of React
  • discontinuing support for selecting different board languages
  • introducing support for adding and removing columns
  • improving support for liking user cards
  • changing the way we encrypt boards to simplify the user experience
  • removing boards created before February 25, 2018
  • requiring users to be registered and logged in to create new boards

The full details:

Later this summer (for those of us in the Northern Hemisphere), we will be releasing a major back-end overhaul that will result in a number of changes that we’d like to make our users aware of.

The good news is that our work today will address technical constraints on our side that’s preventing us from delivering some important features our users have been asking us for.

Firstly, we are building an entirely new OAuth2-compatible API that will allow third parties (and you!) to provide meaningful integrations into our platform. This will allow us to support a wider range of third party services that we might not otherwise intend to support ourselves.

We plan to start rolling out third party access to our API early next year. In the meantime, we’d love to hear from you about what kind of third party services you use that you’d like our platform to be supported by.

We’ve had a number of conversations with a few companies that augment what we do well and we will continue to have those conversations as we progress forward with our development track.

Secondly, we are migrating away from Vue.js in favour of React. We greatly prefer JSX over HTML templating and we will be exploring React Native in the future. There are also more software engineers experienced with React which is obviously important to us from a hiring perspective, given there is a constant shortage of software engineers.

In a recent JetBrains Developer Ecosystem Survey [1], only 33% of respondents said they use Vue.js regularly, compared with 60% for React (this increases to 85% if you include React Native in the mix). This also drives our technical decision.

Upcoming Features

We have been receiving a lot of feedback from users about being able to add and delete columns on their boards. There are several important reasons why we don’t currently support this.

When you create a board, our platform checks whether you’ve created any similar boards before and made any customisations to the default title/apperance of the columns. If you have, we re-use those customisations when you create another board belonging to the same type; so you can spend less time setting up your board and more time leading your teams’ retrospective.

The problem here is if we allow people to add and remove columns when we have fixed board types, we can end up carrying over customisations that no longer reflect the type of board you are creating. Obviously, we can address this in one way or another, but we want to avoid adding unnecessarily complexity to support the different reasons people might want to customise the default columns.

For example, some teams might not even use the Starfish Retrospective model for their retros, but they may choose to create a board using this format so they have five columns to work with. Clearly, this is stupid and not very scalable.

We’ll be addressing this by removing the different board types and instead allow you to choose how many columns you need when creating your board.

Upcoming Changes

As you may know, we allow users to create boards in either Arabic, English, French, German or Spanish. We are finding not many users are choosing to create boards in other languages, so we will be discontinuing this feature later this summer. You will still be able to conduct retrospectives in your own language, of course – you’ll just need to modify the column titles first!

Subscribers are able to take advantage of advanced features such as board encryption. Right now, using this feature requires you to supply a decryption key which then needs to be passed onto your team to gain access. From the feedback we have received, users don’t like having to needlessly store passwords for each board. We need to simplify the way this works.

We will be removing the need for users to supply their own decryption password and we will instead encrypt and decrypt these boards with our own platform key. Data will be encrypted with OpenSSL using the AES-256-CBC cipher. This way, boards can be automatically decrypted when they’re accessed by team members. Of course, subscribers will be able to take advantage of user access control to limit who can access these boards.

Unfortunately, we will need to break compatibility with existing boards that are encrypted using the current method when we make these changes live later this summer, but we will provide an utility for board administrators to automatically convert their boards to the new encryption method. Alternatively, board administrators can ask us for a copy of their encrypted data so they can decrypt them locally.

We will also be introducing better support for ‘liking’ cards by showing you what cards you’ve liked and to prevent users from liking the same item multiple times. Of course, we will continue to allow guests to like cards on boards that don’t require participants to be logged in, and they will be recorded as anonymous votes.

Finally, we will be removing boards created before February 25, 2018 as these boards were created before we had a user registration system in place. The changes we are making to our API (and to facilitate our plan to provide third party access in the future) requires all boards to belong to a user account.

You may not immediately see all the new features we’ve discussed here when we begin rolling out these changes, as we will be releasing these updates in stages.

Going Forward

We are proud to have a wide range of companies using our platform – from small startups to large organisations within the FTSE 100. Over the next year, we will be reviewing our subscription strategy to identify whether we can continue to offer a free tier for our users. Rest assured, we will provide advance notice if anything changes.

We hope you enjoy using Sprint Boards as much as we love building it!

[1] https://www.jetbrains.com/research/devecosystem-2018/javascript

Migration to an internal WebSockets server

Until now, we’ve been using Pusher as the broadcasting service for the real-time collaboration features within Sprint Boards. Of course, you won’t notice anything different on your end and that’s certainly a good thing.

It won’t come as a secret for us to mention Sprint Boards is a Laravel project – we’ve made this public before. We’re proud to be using Laravel and Vue.js. One of the reasons we created Sprint Boards is because other solutions out there really lacked modern polish and used really outdated technologies. It didn’t feel right to us to be using retrospective tools that don’t follow the same high standards we demand of ourselves within our Agile team.

Laravel has out-of-the-box support for Pusher, so since January we’ve been using them to serve as the broadcasting provider. While we love Pusher, we’ll be honest by disclosing that we’ve been forced to move away because it would have been financially unjustifiable to continue using them. We tried negotiating better rates with Pusher because our projected costs over the next year were potentially as high as US$0.10 per connection, but those conversations failed to materialise into anything we could agree on, so we made the painful choice of moving away.

We try and avoid making engineering decisions for purely financial reasons, but the main function of our platform is to provide real-time collaboration, so this is quite exceptional for us.

Earlier this month, we started work to integrate our own WebSockets service using Redis. As with Pusher, Laravel makes this a breeze.

The purpose of this article is to explain some of the difficulties we came across during deployment, which we hope will help our users going through similar challenges in their own teams.

Socket.IO Server

We’re using the Laravel Echo Server by tlaverdure as the Socket.IO server. Depending on what’s most appropriate for your intended use, you can either have this sitting on the same box as your Laravel app, or have it hosted separately.

Configuring the Socket.IO Server

This couldn’t be any simpler. You simply install the NPM package globally, run laravel-echo-server init within the directory of your project (or somewhere else, if you want to store the configuration file in a specific location on your server), answer a few questions and then run laravel-echo-server start. To stop the server, just quit the process.

Automating the Start/Stop Process

Of course, in a production environment, you’re going to want to automate this process. If you use Laravel Forge, you should add a few lines to your deployment script to stop the Laravel Echo Server before a deployment begins and to start it again afterwards:

laravel-echo-server stop > /dev/null 2>&1 &

...

laravel-echo-server start > /dev/null 2>&1 &

By forwarding all output to /dev/null 2>&1, we won’t continuously see output as users start connecting to the WebSockets service. The & at the end allows us to effectively run each process in the background.

The Laravel Echo Server looks for a laravel-echo-server.json configuration file within the directory on which it is started from. If you’ve named the file differently, you can pass the --config flag like this:

laravel-echo-server start --config="laravel-echo-server.production.json" > /dev/null 2>&1 &

If you don’t want to store the Laravel Echo Server configuration file within the same directory as your project, you can store it elsewhere and pass the --dir flag to tell the Laravel Echo Server where it can find it.

Working with HTTPS Connections

If your site is served over HTTPS (and it should be), you’ll need to provide the Laravel Echo Server with the location to your SSL certificate and key for your website.

For those of you using Laravel Forge, you can retrieve the absolute path to the location of your SSL certificate within the SSL tab on the Forge front-end. The key will be located in the same directory as the certificate itself.

If you didn’t supply the location of the SSL certificate and key when you were setting up the Laravel Echo Server, you can just modify the laravel-echo-server.json configuration file directly.

Addressing the Proxy Problem

If you’re struggling to get the Laravel Echo Server working over SSL, you’ll need to proxy requests through to it instead. We use nginx, so we needed to define an upstream server to proxy requests through to. An upstream allows you to define a set of servers to proxy requests through proxy_pass.

In the nginx configuration file for your website, add the following directive to define an upstream server that WebSocket requests are passed onto:

upstream websocket {
    server 127.0.0.1:6001;
}

If you’ve configured the Laravel Echo Server to work on a port other than 6001, make sure you change the port of the upstream server defined above.

Finally, we need to add a location directive within the server{} block:

location /socket.io {
    proxy_pass https://websocket;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection "Upgrade";
}

An obvious but important note, as shown above, is that the proxy_pass directive must pass the requests to the upstream server over HTTPS if your website is being served over HTTPS.

Whichever JavaScript library you are using to subscribe to channels and listen for events broadcast by Laravel, make sure you are doing so over port 80 now that you’re proxying requests from /socket.io:80 to the Socket.IO server internally.

If you’re using the Laravel Echo JavaScript library, this could be as simple as:

window.Echo = new Echo({
   broadcaster: 'socket.io',
   host: window.location.hostname,
});

Conclusions

What was supposed to be a smooth migration had a few interesting turns for us but, really, this is what keeps our work interesting. From our initial benchmarks, real-time collaboration appears to be even faster than it was before now we’re hosting our broadcasting service in-house. Pusher was no slouch either.

Beyond doubt, Laravel made this transition far less painful than it could have been. A job well done. Now for our retrospective!