nankeen

Is This Loss? Part 1: Building a Discord Chatbot

So my schedule has been significantly emptier recently, it’s the weekend and I’m looking for a project. I thought about what I could do and came across chatbots, they’re fun to play with and not as expensive as a website. Interaction will be through basic commands, nothing fancy like natural language processing. However I will be throwing in some TensorFlow because this is going to be an objection recognition bot.

With this technology at our disposal, we must do something important, this project must matter and make a positive difference to the world. So I’ve decided that the noble cause which this bot will serve is going to be detecting loss memes. I’m writing this as I go so this series may not go smoothly or may even not be complete at all but writing this gets me going.

Registering the Discord bot

First step is to register an app on Discord, the developer portal has the option “My Apps” on the top of the left panel. I’m naming it “Is This Loss?”, keeping it simple so no description or picture for now. After creating the app, there’s an option at the bottom to turn this into a bot user, which is what I want. After converting to a bot user, the bot token can be obtained.

Adding the bot to a server

To test the bot, I’ll need a Discord server and a dummy channel where I have permissions to add and spam bots. The my apps page generously provides a Generate OAuth2 URL button, click and allow permissions to “Send Messages” and “Attach Files”. Navigating to the generated URL adds the bot to your server.

Discord Permissions

Starting the project

I like to create Python project skeletons with pyscaffold. After installing it, the skeleton can be generated with.

$ putup Is-This-Loss
$ cd Is-This-Loss

Next I’m going to create a virtual environment and download the dependencies for this part which is disco, pillow, and requests. For now I’m keeping this simple with the standard version that excludes optional dependencies, see here.

$ python3 -m venv .
$ source bin/activate
$ pip install -U disco-py
$ pip install -U pillow

Also, remember to add an appropriate gitignore so git doesn’t include all the unnecessary crap.

$ curl https://www.gitignore.io/api/venv,python -o .gitignore

Bot command

I setup my src folder like so, disco requires the bot token in config.json.

src/
  is_this_loss/
    plugins/
      __init__.py
      object_detector.py
    config.json

Here’s a template for config.json from the docs.


{
  "token": "YOUR_BOT_TOKEN",
  "bot": {
    "plugins": [
        "plugins.object_detector"
    ]
  }
}

Next in object_detector.py a Plugin class is needed to handle commands. In my case the bot will support only one command which is !loss <link> The bot will listen for that command, go to the link and download a picture from it. It will run an inference through TensorFlow to detect loss memes in it. It then draws bounding boxes and sends back the new image as an attachment.

For now it will only download and send back the picture at the link.

First the class needs a method to handle the command event, specify it with @Plugin.command.

from disco.bot import Plugin
from io import BytesIO
from PIL import Image

import requests


class ObjectDetector(Plugin):
    @Plugin.command('!loss', '<link:str...>')
    def command_detect_loss(self, event, link):
        pass

It will also need a method to load the link and return it as a file, here BytesIO is used so there’s no need to save a file. I don’t want to support too many formats, just jpgs and png, throwing an OSError if that’s not the case since it’s is the same error that Image.open throws when an invalid image is loaded.

format_map = {
    'JPEG': 'jpg',
    'PNG': 'png',
    'JPEG 2000': 'jpg'
}

def load_image_from_url(self, url):
    response = requests.get(url)
    file = BytesIO(response.content)
    image = Image.open(file)
    if image.format not in self.format_map.keys():
        raise OSError('Unrecognized format')
    return image

Retrieving and echoing the image

Next, a method to create a file from the image returned is needed for later when I add the inferences code. file.seek(0) got me stuck for a while, I was receiving attachments of size 0 until I realized that BytesIO is a stream implementation. The “cursor” needs to seek to the start in order to send the contents.

def create_attachment_from_image(self, image):
    file = BytesIO()
    image.save(file, 'PNG')
    file.seek(0)
    return file

Finally back in the handler everything can be tied together.

@Plugin.command('!loss', '<link:str...>')
def command_detect_loss(self, event, link):
    try:
        image = self.load_image_from_url(link)
    except OSError as e:
        event.msg.reply("I can't find a JPEG or PNG image at the link you've given")
        return
    file = self.create_attachment_from_image(image)
    filename = 'loss_inference.png'
    event.msg.reply(
        "Hi I found the following image at the link you've given.",
        attachments=[(filename, file)])

So now the bot has the minimal components to run and do something. Here it is in full.

object_detector.py

from disco.bot import Plugin
from io import BytesIO
from PIL import Image

import requests


class ObjectDetector(Plugin):
    format_map = {
        'JPEG': 'jpg',
        'PNG': 'png',
        'JPEG 2000': 'jpg'
    }

    def load_image_from_url(self, url):
        response = requests.get(url)
        file = BytesIO(response.content)
        image = Image.open(file)
        if image.format not in self.format_map.keys():
            raise OSError('Unrecognized format')
        return image

    def create_attachment_from_image(self, image):
        file = BytesIO()
        image.save(file, 'PNG')
        file.seek(0)
        return file

    @Plugin.command('!loss', '<link:str...>')
    def command_detect_loss(self, event, link):
        try:
            image = self.load_image_from_url(link)
        except OSError as e:
            event.msg.reply("I can't find a JPEG or PNG image at the link you've given")
            return
        file = self.create_attachment_from_image(image)
        filename = 'loss_inference.png'
        event.msg.reply(
            "Hi I found the following image at the link you've given.",
            attachments=[(filename, file)])

The following command executed in the is_this_loss directory runs the bot.

$ python3 -m disco.cli --config config.json
[INFO] 2018-06-16 16:39:12,601 - Bot:485 - Adding plugin module at path "plugins.object_detector"
[INFO] 2018-06-16 16:39:12,621 - HTTPClient:271 - GET https://discordapp.com/api/v7/gateway (None)
[INFO] 2018-06-16 16:39:15,870 - GatewayClient:147 - Opening websocket connection to URL `wss://gateway.discord.gg?v=6&encoding=json&compress=zlib-stream`
[INFO] 2018-06-16 16:39:17,725 - GatewayClient:212 - WS Opened: sending identify payload
[INFO] 2018-06-16 16:39:17,726 - GatewayClient:122 - Received HELLO, starting heartbeater...
[INFO] 2018-06-16 16:39:18,751 - GatewayClient:126 - Received READY

Let’s test the bot in Discord. Discord Test 1

Here you can see that Discord recognized an image from the website and displayed it but the bot didn’t as it requires a direct link to the image. This could be a future feature to improve the versatility of the image scraper. Discord Test 2

I’m leaving this here for now, in part 2 I plan to tie a general object detector model to this bot with TensorFlow. Then in part 3 I will train the model into a specialized loss recognizer. The repository will be made public on GitHub and GitLab after cleaning out all junk and bot tokens.

Part 2 is available here.

nankeen

Pwn, rev, and stuff.

By Kai, 2018-06-15