Behind the WidgetBot

The TradeQwik users on here will have already noticed the new Trollbox, used to interact with the TQ Discord via the Website. The shiny piece of code making it possible is WidgetBot. It allows you to read Discord conversations in their full glory, with the Widget looking almost identical to Discord (embeds, reactions, edits, emojis ...). And if you want, you can even use it to send messages to the Discord channel. But, this is not for advertising, I want to talk about how it actually works!

Client

The WidgetBot "Client" (or as we call it, the "Widget"), is the browser-side component, obviously. The main library used is React (developed by Facebook). With React, your site consists of Components. Instead of appending HTML every time a new message is created, we add a new Message Component to the Message Container's Component data array. This makes it not only easier, but more efficient to work with (compared to the previous two versions 1 and 2, both written with jQuery). Other than that, we use Twemoji for emoji parsing, socket.io for communication with the server and a lot of engineering to make it look excactly like Discord. All those modules and libraries are then packed into a single file using WebPack.

But there is also the "Configurator", wich is used to manage the WidgetBot. There, you configure the Widget to your needs. The Configurator is also making use of React, though it doesn't focus on looking like Discord as much.

Server

The "Server" (we also refer to it by "Server"), is the component that receives, processes and broadcasts data from the Discord API. It is written in Node.js, which was the most logical choice for a project like this, since I am very familiar with JavaScript. Most of the spooky Discord API stuff is taken care of by disord.js (requests, caching, rate limiting, storing). Still, the server has a lot of work to do. Here is what happens when you load up the widget (note this happens within 2 seconds):

Nginx receives request and passes it to WidgetBotServer
WidgetBotServer ensures the requested channel is active and configured (including some Discord API calls)
Formats html file with some CSS and general information relative to the requested channel
Sends it to the user

Doesn't seem too complicated right? Well, at this point you would only see the WidgetBot logo flashing at you. The actual messages and any further information is transferred via the websocket, to reduce initial loading time tremendously. First, the Widget sends its current guild (discord server) and channel ID, the server performs some checks on it and then saves that data for further requests. Then, the Server packs messages, general information and configuration up and sends it to the Widget. This is by far the most stressful function of the server. It has to fetch the last 40 messages from the Discord API, and process them synchronously, exchanging names with nicknames, parsing mentions and cleaning up embeds and reactions.

Additionally, the WidgetBot listens for any Discord API events (for example reaction added, message, typing stop), formats them and broadcasts them to all Widgets registered to the channel the event was emitted by. This is one of the core functions, making the Widget act in realtime.

In the end, the Widget receives a watered-down version of the data Discord would receive. For example: Instead of having to parse every single message for mentions, the server has done that already and provides a list for it. Instead of receiving a big Reaction object, it receives an ID and a number, which is emoji name and reaction count respectively, making the Widget fast and resource-efficient.

Behind the Code

That's me and samdd. While I have founded this project back in April, Sam joined the Team in August. Now, I'm responsible for the Server and most of the JavaScript on the Widget and Configurator, while he takes care of UI and UX design. He also created a simple Widget container, called Crate, that makes it easier than before to embed the Widget on any site. Adding up to that, he went through the hassle of setting up our deployment system (GitHub -> Travis CI -> Jekyll). This way, server and client automatically update whenever we push new code. The WidgetBot is being financiated by our awesome Patrons.

I hope this was interesting to read, I'd be happy to answer any questions in the comments!