Port Comp_iler to Myriad (#8)

This commit is contained in:
1Computer1 2019-07-11 02:27:17 -04:00 committed by GitHub
parent 0dd7f992be
commit c72376656c
Signed by: GitHub
GPG key ID: 4AEE18F83AFDEB23
91 changed files with 156 additions and 1140 deletions

View file

@ -1,5 +1,12 @@
# Changelog
## 2.0.0
- Ported Comp_iler backend to [Myriad](https://github.com/1Computer1/myriad).
- Removed `exit` command.
- Removed evaluation options.
- Removed Python 2.
## 1.10.1
- Fixed cleanup not invalidating cache.

View file

@ -42,24 +42,10 @@ int main()
```
````
````
>harmony```js
class Foo {
bar = 1;
}
console.log(new Foo().bar);
```
````
```
>`py print('hello world')`
```
```
>e`hs (+) <$> Just 1 <*> Just 2`
```
## Supported Languages and Options
One of the following language codes is set in `lang`.
@ -68,81 +54,50 @@ Options are optionally set in `options`, which is a semicolon-delimited list of
- `apl` APL
- `bash` Bash
- `bf` Brainfuck
- `c` C (GCC)
- `c` C
- `clj` Clojure
- `cpp` C++ (G++)
- `cs` C# (Mono)
- `e` evaluates a single expression instead of a module
- `cpp` C++
- `cs` C#
- `elixir` Elixir
- `fs` F# (Mono)
- `fs` F#
- `go` Go
- `hs` Haskell (GHC)
- `e` evaluates a single expression instead of a module
- `java` Java (OpenJDK)
- `js` JavaScript (Node)
- `harmony` enables harmony features (`--harmony` on node)
- `hs` Haskell
- `java` Java
- `js` JavaScript
- `julia` Julia
- `lisp` Racket
- `lua` Lua
- `ocaml` OCaml
- `pas` Pascal (FPC)
- `pas` Pascal
- `php` PHP
- `pl` Perl5
- `prolog` Prolog (SWI-Prolog)
- `py` Python (CPython)
- `2` runs Python 2 instead of Python 3
- `prolog` Prolog
- `py` Python
- `rb` Ruby
- `rs` Rust
## How it Works
Read the source code, specifically `src/struct/LanguageHandler.js`.
In summary, for every language there is a docker image which spins up a docker container.
For every language there is a docker image which spins up a docker container.
The container is used for all evaluations of code, restarting if something goes wrong.
The container is locked down, so there is no networking, limited memory and CPU usage, and a time limit.
## Setup
0. Install Docker 18+
0. Install Node 10+
0. Install [Docker 18+](https://www.docker.com/)
0. Install [Node 10+](https://nodejs.org/)
0. Install [Myriad](https://github.com/1Computer1/myriad)
- This will require [Stack 2+](https://docs.haskellstack.org/en/stable/README/).
- You will also have to configure Myriad, see its repository.
0. Fill out `config.json`
- `owner` The owner(s) of the bot. Use an array for multiple owners.
- `token` The bot token.
- `prefix` The prefix for commands.
- `codePrefix` The prefix for code evaluation.
- `myriad` The port that Myriad is running on.
0. Run `npm i`
0. Fill out `config.json` as described in the configuration section below
## Running
0. Run `myriad --config path/to/config.dhall`
0. Run `node .`
## Configuration
### Bot
- `owner` - The owner(s) of the bot.
Use an array for multiple owners.
- `token` - The bot token.
- `prefix` - The prefix for commands.
- `codePrefix` - The prefix for code evaluation.
### Setup
- `languages` Languages to use.
The language names here are different from the user-facing ones.
Check the filenames in `src/languages/` for the language names.
Change to `null` to enable all languages.
- `prepare` Whether to start containers on setup.
Setting to true will speed up the first eval, but that language might not be used.
- `parallel` Whether to build images in parallel.
Will also setup containers in parallel if `prepare` is set.
Faster, but will take more resources.
- `cleanup` Interval in minutes to occasionally kill all containers.
Set to `0` to disable.
### Compilers
For each of these options, you can use either the expected value to set it for all compilers or an object with compiler names to the expected values.
If using an object, you can set the default with the `default` key.
The compiler names are the folder names under `docker/`.
- `memory` Max memory usage of a container.
- `cpu` Max CPU usage of a container.
- `timeout` Time limit for code in milliseconds.
- `concurrent` Number of code evaluations than can run at a time per container.
The more that can run, the more resources a container would need.
- `retries` Maximum number of retries for an evaluation.
Evaluations are retried when all concurrent evaluations fail because one failed.

View file

@ -3,17 +3,5 @@
"token": "MTU1fdsYNTRb2RT.FcD2l1ig.jIuKqwertyd432RROhF5A",
"prefix": ">",
"codePrefix": ">",
"languages": [
"haskell",
"python",
"javascript"
],
"prepare": false,
"parallel": false,
"cleanup": 60,
"memory": "256m",
"cpus": "0.25",
"timeout": 10000,
"concurrent": 10,
"retries": 2
"myriad": "8081"
}

View file

@ -1,4 +0,0 @@
FROM juergensauermann/gnu-apl
LABEL author="1Computer1"
COPY run.sh /var/run/

View file

@ -1,2 +0,0 @@
printf %s "$1" > program.apl
apl --OFF -s -f program.apl || true

View file

@ -1,4 +0,0 @@
FROM bash
LABEL author="1Computer1"
COPY run.sh /var/run/

View file

@ -1,2 +0,0 @@
printf %s "$1" > program.sh
bash program.sh || true

View file

@ -1,12 +0,0 @@
FROM alpine AS build
RUN apk update && apk add g++
COPY bf.cpp .
RUN g++ bf.cpp -o bf
FROM alpine
LABEL author="1Computer1"
RUN apk update && apk add libstdc++
COPY --from=build bf /usr/local/bin/
COPY run.sh /var/run/

View file

@ -1,119 +0,0 @@
#include <iostream>
#include <vector>
#include <string.h>
#include <string>
int main(int argc, char **argv) {
std::string ops;
if (argc == 1) {
std::string line;
while (std::getline(std::cin, line)) {
ops.append(line);
}
if (ops.empty()) {
std::cerr << "No input given";
return 1;
}
} else {
ops.assign(argv[1], strlen(argv[1]));
}
int len = ops.length();
std::vector<char> tape = { 0 };
int oix = 0;
int tix = 0;
while (oix < len) {
switch (ops[oix]) {
case '>':
tix++;
if (tix >= tape.size()) {
tape.push_back(0);
}
oix++;
break;
case '<':
tix--;
if (tix < 0) {
std::cerr << "Out of bounds";
return 1;
}
oix++;
break;
case '+':
tape[tix]++;
oix++;
break;
case '-':
tape[tix]--;
oix++;
break;
case '.':
std::cout << tape[tix];
oix++;
break;
case ',':
std::cin >> tape[tix];
oix++;
break;
case '[':
if (tape[tix] == 0) {
int ls = 0;
int rs = 0;
for (int i = oix; i < len; i++) {
switch (ops[i]) {
case '[':
ls++;
break;
case ']':
rs++;
break;
default:
break;
}
if (ls == rs) {
oix = i + 1;
break;
}
}
} else {
oix++;
}
break;
case ']':
if (tape[tix] != 0) {
int ls = 0;
int rs = 0;
for (int i = oix; i >= 0; i--) {
switch (ops[i]) {
case '[':
ls++;
break;
case ']':
rs++;
break;
default:
break;
}
if (ls == rs) {
oix = i + 1;
break;
}
}
} else {
oix++;
}
break;
default:
oix++;
}
}
return 0;
}

View file

@ -1 +0,0 @@
printf %s "$1" | bf || true

View file

@ -1,7 +0,0 @@
FROM alpine
LABEL author="1Computer1"
RUN apk update
RUN apk add gcc libc-dev
COPY run.sh /var/run/

View file

@ -1,2 +0,0 @@
printf %s "$1" > program.c
gcc program.c -o program && ./program || true

View file

@ -1,4 +0,0 @@
FROM clojure:tools-deps-alpine
LABEL author="1Computer1"
COPY run.sh /var/run/

View file

@ -1,2 +0,0 @@
printf %s "$1" > program.clj
clojure program.clj || true

View file

@ -1,7 +0,0 @@
FROM alpine
LABEL author="1Computer1"
RUN apk update
RUN apk add g++
COPY run.sh /var/run/

View file

@ -1,2 +0,0 @@
printf %s "$1" > program.cpp
g++ program.cpp -o program && ./program || true

View file

@ -1,4 +0,0 @@
FROM mono
LABEL author="1Computer1"
COPY run.sh /var/run/

View file

@ -1,6 +0,0 @@
if [ "$EVAL_EXPR" = "true" ]; then
printf %s "$1" | csharp -e
else
printf %s "$1" > program.cs
csc program.cs >/dev/null && mono program.exe || true
fi

View file

@ -1,4 +0,0 @@
FROM elixir:alpine
LABEL author="1Computer1"
COPY run.sh /var/run/

View file

@ -1,2 +0,0 @@
printf %s "$1" > program.exs
elixir program.exs || true

View file

@ -1,4 +0,0 @@
FROM fsharp
LABEL author="1Computer1"
COPY run.sh /var/run/

View file

@ -1,2 +0,0 @@
printf %s "$1" > program.fs
fsharpc --optimize- program.fs >/dev/null && mono program.exe || true

View file

@ -1,4 +0,0 @@
FROM golang:alpine
LABEL author="1Computer1"
COPY run.sh /var/run/

View file

@ -1,3 +0,0 @@
export GOCACHE=/tmp/"$CODEDIR"/cache
printf %s "$1" > program.go
go run program.go || true

View file

@ -1,18 +0,0 @@
import Data.Bool
import Data.Char
import Data.Either
import Data.Function
import Data.List
import Data.Maybe
import Data.Ord
import Control.Applicative
import Control.Monad
import Data.Map (Map)
import qualified Data.Map as Map
import Data.Set (Set)
import qualified Data.Set as Set
import Data.Sequence (Seq)
import qualified Data.Sequence as Seq
import Data.Tree (Tree)
import qualified Data.Tree as Tree

View file

@ -1,15 +0,0 @@
FROM debian:stretch
LABEL author="1Computer1"
ENV LANG C.UTF-8
RUN apt-get update && \
apt-get install -y --no-install-recommends gnupg dirmngr && \
echo 'deb http://downloads.haskell.org/debian stretch main' > /etc/apt/sources.list.d/ghc.list && \
apt-key adv --keyserver keyserver.ubuntu.com --recv-keys BA3CBA3FFE22B574 && \
apt-get update && \
apt-get install -y --no-install-recommends ghc-8.6.5
ENV PATH /opt/ghc/8.6.5/bin:$PATH
COPY .ghci $HOME/
COPY run.sh /var/run/

View file

@ -1,6 +0,0 @@
if [ "$EVAL_EXPR" = "true" ]; then
ghc -e "$1" || true
else
printf %s "$1" > program.hs
ghc -e main program.hs || true
fi

View file

@ -1,4 +0,0 @@
FROM openjdk:13-alpine
LABEL author="1Computer1"
COPY run.sh /var/run/

View file

@ -1,2 +0,0 @@
printf %s "$1" > Main.java
javac Main.java && java Main || true

View file

@ -1,4 +0,0 @@
FROM node:alpine
LABEL author="1Computer1"
COPY run.sh /var/run/

View file

@ -1,5 +0,0 @@
if [ "$EVAL_HARMONY" = "true" ]; then
printf %s "$1" | node --harmony -p || true
else
printf %s "$1" | node -p || true
fi

View file

@ -1,4 +0,0 @@
FROM julia
LABEL author="1Computer1"
COPY run.sh /var/run/

View file

@ -1 +0,0 @@
printf %s "$1" | julia

View file

@ -1,6 +0,0 @@
FROM alpine
RUN apk update
RUN apk add lua5.3
COPY run.sh /var/run/

View file

@ -1,2 +0,0 @@
printf %s "$1" > program.lua
lua5.3 program.lua || true

View file

@ -1,4 +0,0 @@
FROM frolvlad/alpine-ocaml
LABEL author="1Computer1"
COPY run.sh /var/run/

View file

@ -1,2 +0,0 @@
printf %s "$1" > program.ml
ocamlopt -cclib --static -o program program.ml && ./program || true

View file

@ -1,4 +0,0 @@
FROM frolvlad/alpine-fpc
LABEL author="1Computer1"
COPY run.sh /var/run/

View file

@ -1,10 +0,0 @@
printf %s "$1" > program.pas
# fpc does not use stderr, ld however does, capture both
res="$(fpc program.pas 2>&1)"
if [ $? -eq 0 ]; then
./program || true
else
printf %s "$res"
fi

View file

@ -1,4 +0,0 @@
FROM perl:slim
LABEL author="1Computer1"
COPY run.sh /var/run/

View file

@ -1,2 +0,0 @@
printf %s "$1" > program.pl
perl program.pl || true

View file

@ -1,4 +0,0 @@
FROM php:alpine
LABEL author="1Computer1"
COPY run.sh /var/run/

View file

@ -1,2 +0,0 @@
printf %s "$1" > program.php
php program.php || true

View file

@ -1,4 +0,0 @@
FROM swipl
LABEL author="1Computer1"
COPY run.sh /var/run/

View file

@ -1,2 +0,0 @@
printf %s "$1" > program.pl
swipl --quiet program.pl || true

View file

@ -1,4 +0,0 @@
FROM python:2-alpine
LABEL author="1Computer1"
COPY run.sh /var/run/

View file

@ -1,2 +0,0 @@
printf %s "$1" > program.py
python program.py || true

View file

@ -1,4 +0,0 @@
FROM python:3-alpine
LABEL author="1Computer1"
COPY run.sh /var/run/

View file

@ -1,2 +0,0 @@
printf %s "$1" > program.py
python program.py || true

View file

@ -1,4 +0,0 @@
FROM jackfirth/racket
LABEL author="1Computer1"
COPY run.sh /var/run/

View file

@ -1,2 +0,0 @@
printf %s "$1" > program.rkt
racket program.rkt || true

View file

@ -1,4 +0,0 @@
FROM ruby:alpine
LABEL author="1Computer1"
COPY run.sh /var/run/

View file

@ -1,2 +0,0 @@
printf %s "$1" > program.rb
ruby program.rb || true

View file

@ -1,4 +0,0 @@
FROM rust:slim
LABEL author="1Computer1"
COPY run.sh /var/run/

View file

@ -1,2 +0,0 @@
printf %s "$1" > program.rs
rustc -C opt-level=0 --color never program.rs && ./program || true

View file

@ -1,6 +1,6 @@
{
"name": "comp_iler",
"version": "1.10.1",
"version": "2.0.0",
"description": "Sandboxed eval bot",
"main": "src/index.js",
"author": "1Computer1",

View file

@ -14,6 +14,9 @@ class AboutCommand extends Command {
'Comp_iler is made by 1Computer.',
'Source code is available at <https://github.com/1Computer1/comp_iler>.',
'',
'Comp_iler runs on Myriad, a Docker-based arbitrary code evaluation server!',
'Check it out here <https://github.com/1Computer1/myriad>.',
'',
`**Guilds**: ${this.client.guilds.size}`,
`**Channels**: ${this.client.channels.size}`,
`**Users**: ${this.client.users.size}`,

View file

@ -11,7 +11,7 @@ class CleanupCommand extends Command {
async exec(message) {
await message.util.send('Cleaning up...');
await this.client.languageHandler.cleanup();
await this.client.myriad.postCleanup();
await message.util.send('Cleaned up!');
}
}

View file

@ -1,20 +0,0 @@
const { Command } = require('discord-akairo');
class ExitCommand extends Command {
constructor() {
super('exit', {
aliases: ['exit'],
ownerOnly: true,
clientPermissions: ['SEND_MESSAGES']
});
}
async exec(message) {
await message.util.send('Cleaning up...');
await this.client.languageHandler.cleanup();
await message.util.send('Exiting!');
process.exit();
}
}
module.exports = ExitCommand;

View file

@ -13,11 +13,9 @@ class HelpCommand extends Command {
return message.util.send([
'**Usage:**',
`Put a \`${codePrefix}\` before a code block or inline codeblock that starts with a language code to execute it.`,
`You can add options, separated by semicolons, after the \`${codePrefix}\`.`,
'',
`For list of enabled languages, use the \`${prefix}languages\` command.`,
'',
'See the readme for usage examples, supported languages, and options: <https://github.com/1Computer1/comp_iler>'
'See the readme for usage examples and supported languages: <https://github.com/1Computer1/comp_iler>'
]);
}
}

View file

@ -1,19 +1,21 @@
const { Command } = require('discord-akairo');
const Myriad = require('../struct/Myriad');
class LanguagesCommand extends Command {
constructor() {
super('languages', {
aliases: ['languages', 'language'],
aliases: ['languages', 'language', 'langs', 'lang'],
clientPermissions: ['SEND_MESSAGES']
});
}
exec(message) {
async exec(message) {
const languages = await this.client.myriad.getLanguages();
return message.util.send([
'**List of enabled languages (Name: Language Codes)**:',
...this.client.languageHandler.modules.map(lang => `${lang.name}: \`${lang.aliases.join('`, `')}\``),
'**List of enabled languages (Language Codes)**:',
...languages.map(lang => `\`${Myriad.Languages.get(lang).join('`, `')}\``),
'',
'See the readme for usage examples, supported languages, and options: <https://github.com/1Computer1/comp_iler>'
'See the readme for usage examples: <https://github.com/1Computer1/comp_iler>'
]);
}
}

View file

@ -1,12 +0,0 @@
const Language = require('../struct/Language');
class APL extends Language {
constructor() {
super('apl', {
name: 'APL',
aliases: ['apl']
});
}
}
module.exports = APL;

View file

@ -1,12 +0,0 @@
const Language = require('../struct/Language');
class Bash extends Language {
constructor() {
super('bash', {
name: 'Bash',
aliases: ['bash', 'sh']
});
}
}
module.exports = Bash;

View file

@ -1,12 +0,0 @@
const Language = require('../struct/Language');
class Brainfuck extends Language {
constructor() {
super('brainfuck', {
name: 'Brainfuck',
aliases: ['brainfuck', 'bf']
});
}
}
module.exports = Brainfuck;

View file

@ -1,12 +0,0 @@
const Language = require('../struct/Language');
class C extends Language {
constructor() {
super('c', {
name: 'C',
aliases: ['c']
});
}
}
module.exports = C;

View file

@ -1,12 +0,0 @@
const Language = require('../struct/Language');
class Clojure extends Language {
constructor() {
super('clojure', {
name: 'Clojure',
aliases: ['clojure', 'clj']
});
}
}
module.exports = Clojure;

View file

@ -1,12 +0,0 @@
const Language = require('../struct/Language');
class CPP extends Language {
constructor() {
super('cpp', {
name: 'C++',
aliases: ['cpp', 'c++']
});
}
}
module.exports = CPP;

View file

@ -1,23 +0,0 @@
const Language = require('../struct/Language');
class CSharp extends Language {
constructor() {
super('csharp', {
name: 'C#',
aliases: ['csharp', 'cs'],
options: {
e: () => ''
}
});
}
runWith(options) {
if (options.has('e')) {
return { env: { EVAL_EXPR: 'true' } };
}
return {};
}
}
module.exports = CSharp;

View file

@ -1,12 +0,0 @@
const Language = require('../struct/Language');
class Elixir extends Language {
constructor() {
super('elixir', {
name: 'Elixir',
aliases: ['elixir']
});
}
}
module.exports = Elixir;

View file

@ -1,12 +0,0 @@
const Language = require('../struct/Language');
class FSharp extends Language {
constructor() {
super('fsharp', {
name: 'F#',
aliases: ['fsharp', 'fs']
});
}
}
module.exports = FSharp;

View file

@ -1,12 +0,0 @@
const Language = require('../struct/Language');
class Go extends Language {
constructor() {
super('go', {
name: 'Go',
aliases: ['golang', 'go']
});
}
}
module.exports = Go;

View file

@ -1,23 +0,0 @@
const Language = require('../struct/Language');
class Haskell extends Language {
constructor() {
super('haskell', {
name: 'Haskell',
aliases: ['haskell', 'hs'],
options: {
e: () => ''
}
});
}
runWith(options) {
if (options.has('e')) {
return { env: { EVAL_EXPR: 'true' } };
}
return {};
}
}
module.exports = Haskell;

View file

@ -1,12 +0,0 @@
const Language = require('../struct/Language');
class Java extends Language {
constructor() {
super('java', {
name: 'Java',
aliases: ['java']
});
}
}
module.exports = Java;

View file

@ -1,24 +0,0 @@
const Language = require('../struct/Language');
class JavaScript extends Language {
constructor() {
super('javascript', {
name: 'JavaScript',
aliases: ['javascript', 'js'],
options: {
harmony: () => ''
}
});
}
runWith(options) {
const ret = { id: this.id, env: {} };
if (options.has('harmony')) {
ret.env.EVAL_HARMONY = 'true';
}
return ret;
}
}
module.exports = JavaScript;

View file

@ -1,12 +0,0 @@
const Language = require('../struct/Language');
class Julia extends Language {
constructor() {
super('julia', {
name: 'Julia',
aliases: ['julia']
});
}
}
module.exports = Julia;

View file

@ -1,12 +0,0 @@
const Language = require('../struct/Language');
class Lua extends Language {
constructor() {
super('lua', {
name: 'Lua',
aliases: ['lua']
});
}
}
module.exports = Lua;

View file

@ -1,12 +0,0 @@
const Language = require('../struct/Language');
class OCaml extends Language {
constructor() {
super('ocaml', {
name: 'OCaml',
aliases: ['ocaml']
});
}
}
module.exports = OCaml;

View file

@ -1,12 +0,0 @@
const Language = require('../struct/Language');
class Pascal extends Language {
constructor() {
super('pascal', {
name: 'Pascal',
aliases: ['pascal', 'pas', 'freepascal']
});
}
}
module.exports = Pascal;

View file

@ -1,12 +0,0 @@
const Language = require('../struct/Language');
class Perl extends Language {
constructor() {
super('perl', {
name: 'Perl',
aliases: ['perl', 'pl']
});
}
}
module.exports = Perl;

View file

@ -1,12 +0,0 @@
const Language = require('../struct/Language');
class PHP extends Language {
constructor() {
super('php', {
name: 'PHP',
aliases: ['php']
});
}
}
module.exports = PHP;

View file

@ -1,12 +0,0 @@
const Language = require('../struct/Language');
class Prolog extends Language {
constructor() {
super('prolog', {
name: 'Prolog',
aliases: ['prolog']
});
}
}
module.exports = Prolog;

View file

@ -1,24 +0,0 @@
const Language = require('../struct/Language');
class Python extends Language {
constructor() {
super('python', {
name: 'Python',
aliases: ['python', 'py'],
loads: ['python3', 'python2'],
options: {
2: () => ''
}
});
}
runWith(options) {
if (options.has('2')) {
return { id: 'python2' };
}
return { id: 'python3' };
}
}
module.exports = Python;

View file

@ -1,12 +0,0 @@
const Language = require('../struct/Language');
class Racket extends Language {
constructor() {
super('racket', {
name: 'Racket',
aliases: ['lisp']
});
}
}
module.exports = Racket;

View file

@ -1,12 +0,0 @@
const Language = require('../struct/Language');
class Ruby extends Language {
constructor() {
super('ruby', {
name: 'Ruby',
aliases: ['ruby', 'rb']
});
}
}
module.exports = Ruby;

View file

@ -1,12 +0,0 @@
const Language = require('../struct/Language');
class Rust extends Language {
constructor() {
super('rust', {
name: 'Rust',
aliases: ['rust', 'rs']
});
}
}
module.exports = Rust;

View file

@ -1,4 +1,5 @@
const { Listener } = require('discord-akairo');
const Myriad = require('../struct/Myriad');
const fetch = require('node-fetch');
class MessageInvalidListener extends Listener {
@ -26,95 +27,51 @@ class MessageInvalidListener extends Listener {
reaction = await message.react('📝');
}
let errored = false;
let result;
try {
({ output: result, hasStderr: errored } = await this.client.languageHandler.evalCode(parse));
} catch (e) {
errored = true;
result = e.message;
}
result = result || '';
const [ok, response] = await this.client.myriad.postEval(parse.language, parse.code);
if (!message.guild || message.channel.permissionsFor(this.client.user).has('ADD_REACTIONS')) {
if (reaction) {
reaction.users.remove();
}
if (errored) {
message.react('✖');
} else {
if (ok) {
message.react('✔');
} else {
message.react('✖');
}
}
const invalid = parse.invalid.length ? `Invalid options: ${parse.invalid.join(', ')}\n` : '';
const output = `${invalid}\`\`\`\n${result}\n\`\`\``;
const output = `\`\`\`\n${response}\n\`\`\``;
if (output.length >= 2000) {
const key = await fetch('https://hastebin.com/documents', { method: 'POST', body: result })
const key = await fetch('https://hastebin.com/documents', { method: 'POST', body: response })
.then(res => res.json())
.then(json => json.key);
return message.util.send(`${invalid}Output was too long: <https://hastebin.com/${key}.js>`);
return message.util.send(`Output was too long: <https://hastebin.com/${key}>`);
}
return message.util.send(output);
}
parseMessage(message) {
const prefix = this.client.config.codePrefix.replace(/[-[\]{}()*+?.,\\^$|#\s]/g, '\\$&');
const regex1 = new RegExp(`^\\s*${prefix}\\s*(.+?)?\\s*\`\`\`(.+?)\\n([^]+)\`\`\`\\s*$`);
const regex2 = new RegExp(`^\\s*${prefix}\\s*(.+?)?\\s*\`(.+?) \\s*([^]+)\`\\s*$`);
const match = message.content.match(regex1) || message.content.match(regex2);
const prefix = this.client.config.codePrefix;
const starts = message.content.slice(0, prefix.length).toLowerCase().startsWith(prefix.toLowerCase());
if (!starts) {
return null;
}
const regex = /^\s*(`{1,3})(.+?)\n([^]+)\1\s*$/;
const match = message.content.slice(prefix.length).match(regex);
if (!match) {
return null;
}
const language = this.parseLanguage(match[2]);
const language = Myriad.Aliases.get(match[2].toLowerCase());
if (!language) {
return null;
}
const code = match[3].trim();
const [valid, invalid] = this.parseOptions(language, match[1] || '');
return { id: message.id, language, code, options: valid, invalid };
}
parseLanguage(language) {
const match = this.client.languageHandler.findLanguage(language.trim());
if (!match) {
return null;
}
return match;
}
parseOptions(language, options) {
const kvs = options.split(';').map(opt => {
const [k, v = ''] = opt.split('=');
return [k.toLowerCase().trim(), v.trim()];
});
const valid = new Map();
const invalid = [];
for (const [key, value] of kvs) {
if (!key) {
continue;
}
if (Object.prototype.hasOwnProperty.call(language.options, key)) {
const parse = language.options[key](value);
if (parse != null) {
valid.set(key, parse);
} else {
invalid.push(key);
}
} else {
invalid.push(key);
}
}
return [valid, invalid];
return { language, code };
}
}

View file

@ -1,5 +1,5 @@
const { AkairoClient, CommandHandler, ListenerHandler } = require('discord-akairo');
const LanguageHandler = require('./LanguageHandler');
const Myriad = require('./Myriad');
const path = require('path');
class CompilerClient extends AkairoClient {
@ -24,25 +24,19 @@ class CompilerClient extends AkairoClient {
directory: path.join(__dirname, '../listeners')
});
this.languageHandler = new LanguageHandler(this, {
directory: path.join(__dirname, '../languages')
});
this.myriad = new Myriad(config.myriad);
this.config = config;
}
async start() {
start() {
this.commandHandler.useListenerHandler(this.listenerHandler);
this.listenerHandler.setEmitters({
commandHandler: this.commandHandler,
listenerHandler: this.listenerHandler,
languageHandler: this.languageHandler
listenerHandler: this.listenerHandler
});
this.commandHandler.loadAll();
this.listenerHandler.loadAll();
this.languageHandler.loadAll();
await this.languageHandler.buildDocker();
return this.login(this.config.token);
}
}

View file

@ -1,24 +0,0 @@
const { AkairoModule } = require('discord-akairo');
class Language extends AkairoModule {
constructor(id, {
category,
name,
aliases,
loads = [id],
options = {}
} = {}) {
super(id, { category });
this.name = name;
this.aliases = aliases;
this.loads = loads;
this.options = options;
}
runWith() {
return {};
}
}
module.exports = Language;

View file

@ -1,227 +0,0 @@
const { AkairoHandler } = require('discord-akairo');
const { Collection } = require('discord.js');
const Language = require('./Language');
const Queue = require('./Queue');
const childProcess = require('child_process');
const util = require('util');
const path = require('path');
const exec = util.promisify(childProcess.exec);
class LanguageHandler extends AkairoHandler {
constructor(client, {
directory,
classToHandle = Language,
extensions = ['.js', '.ts'],
automateCategories,
loadFilter = filepath =>
!this.client.config.languages || this.client.config.languages.includes(path.parse(filepath).name)
}) {
super(client, {
directory,
classToHandle,
extensions,
automateCategories,
loadFilter
});
this.aliases = new Collection();
this.containers = new Collection();
this.queues = new Collection();
}
register(language, filepath) {
super.register(language, filepath);
for (let alias of language.aliases) {
const conflict = this.aliases.get(alias.toLowerCase());
if (conflict) {
throw new TypeError(`Alias conflict of ${alias} between ${language.id} and ${conflict}`);
}
alias = alias.toLowerCase();
this.aliases.set(alias, language.id);
}
}
deregister(language) {
for (let alias of language.aliases) {
alias = alias.toLowerCase();
this.aliases.delete(alias);
}
super.deregister(language);
}
findLanguage(alias) {
return this.modules.get(this.aliases.get(alias.toLowerCase()));
}
async buildDocker() {
if (this.client.config.parallel) {
await Promise.all(this.modules.map(({ loads }) => Promise.all(loads.map(dockerID => this.buildImage(dockerID)))));
return;
}
for (const { loads } of this.modules.values()) {
for (const dockerID of loads) {
// eslint-disable-next-line no-await-in-loop
await this.buildImage(dockerID);
}
}
if (this.client.config.cleanup > 0) {
setInterval(() => this.cleanup().catch(() => null), this.client.config.cleanup * 60 * 1000);
}
}
async buildImage(dockerID) {
const folder = path.join(__dirname, '../../docker', dockerID);
await util.promisify(childProcess.exec)(`docker build -t "1computer1/comp_iler:${dockerID}" ${folder}`);
// eslint-disable-next-line no-console
console.log(`Built image 1computer1/comp_iler:${dockerID}.`);
const concurrent = this.getCompilerConfig(dockerID, 'concurrent', 'number');
this.queues.set(dockerID, { evalQueue: new Queue(concurrent), setupQueue: new Queue(1) });
if (this.client.config.prepare) {
await this.setupContainer(dockerID);
}
}
async setupContainer(dockerID) {
if (this.containers.has(dockerID)) {
return this.containers.get(dockerID);
}
const cpus = this.getCompilerConfig(dockerID, 'cpus', 'string');
const memory = this.getCompilerConfig(dockerID, 'memory', 'string');
const name = `comp_iler-${dockerID}-${Date.now()}`;
try {
await exec([
`docker run --rm --name=${name}`,
'-u1000:1000 -w/tmp/ -dt',
`--net=none --cpus=${cpus} -m=${memory} --memory-swap=${memory}`,
`1computer1/comp_iler:${dockerID} /bin/sh`
].join(' '));
await exec(`docker exec ${name} mkdir eval`);
await exec(`docker exec ${name} chmod 711 eval`);
this.containers.set(dockerID, { name });
// eslint-disable-next-line no-console
console.log(`Started container ${name}.`);
return this.containers.get(dockerID);
} catch (err) {
throw err;
}
}
evalCode({ language, code, options, retries = 0 }) {
const { id: dockerID = language.id, env = {} } = language.runWith(options);
const { evalQueue, setupQueue } = this.queues.get(dockerID);
return evalQueue.enqueue(async () => {
const { name } = await setupQueue.enqueue(() => this.setupContainer(dockerID));
const now = Date.now();
try {
await exec(`docker exec ${name} mkdir eval/${now}`);
await exec(`docker exec ${name} chmod 777 eval/${now}`);
const proc = childProcess.spawn('docker', [
'exec', '-u1001:1001', `-w/tmp/eval/${now}`,
...Object.entries(env).map(([k, v]) => `-e${k}=${v}`),
name, '/bin/sh', '/var/run/run.sh', code
]);
const timeout = this.getCompilerConfig(dockerID, 'timeout', 'number');
const result = await this.handleSpawn(proc, timeout);
await exec(`docker exec ${name} rm -rf eval/${now}`);
return result;
} catch (err) {
this.containers.delete(dockerID);
try {
await this.kill(name);
} catch (err2) {
// The container was not alive to be killed, i.e. multiple evals were occuring,
// one of them caused the container to be killed, the remaining evals all fail.
// Retry those remaining ones until they work!
if (retries < this.getCompilerConfig(dockerID, 'retries', 'number')) {
return this.evalCode({ language, code, options, retries: retries + 1 });
}
}
throw err;
}
});
}
handleSpawn(proc, timeout = null) {
return new Promise((resolve, reject) => {
let handled = false;
if (timeout !== null) {
setTimeout(() => {
handled = true;
reject(new Error('Timed out'));
}, timeout);
}
let output = '';
let hasStderr = false;
let error;
proc.stdout.on('data', chunk => {
output += chunk;
});
proc.stderr.on('data', chunk => {
hasStderr = true;
output += chunk;
});
proc.on('error', e => {
error = e;
});
proc.on('close', status => {
if (!handled) {
handled = true;
if (status !== 0 || error) {
if (!error) {
error = new Error(output || 'Something went wrong');
}
reject(error);
} else {
resolve({ output, hasStderr });
}
}
});
});
}
async kill(name) {
let cmd;
if (process.platform === 'win32') {
cmd = `docker kill --signal=9 ${name} >nul 2>nul`;
} else {
cmd = `docker kill --signal=9 ${name} >/dev/null 2>/dev/null`;
}
await exec(cmd);
// eslint-disable-next-line no-console
console.log(`Killed container ${name}.`);
}
cleanup() {
return Promise.all(this.containers.map(({ name }, dockerID) => {
this.containers.delete(dockerID);
return this.kill(name);
}));
}
getCompilerConfig(dockerID, key, type) {
const o = this.client.config[key];
return typeof o === type
? o
: o[dockerID] !== undefined
? o[dockerID]
: o.default;
}
}
module.exports = LanguageHandler;

87
src/struct/Myriad.js Normal file
View file

@ -0,0 +1,87 @@
const fetch = require('node-fetch');
class Myriad {
constructor(port) {
this.port = port;
}
url(k) {
return `http://localhost:${this.port}/${k}`;
}
getLanguages() {
return fetch(this.url('languages'), { method: 'GET' }).then(x => x.json());
}
async postEval(language, code) {
const response = await fetch(this.url('eval'), {
method: 'POST',
body: JSON.stringify({ language, code }),
headers: { 'Content-Type': 'application/json' }
});
if (!response.ok) {
if (response.status === 404 && /^Language .+? was not found$/.test(response.statusText)) {
return [false, 'Invalid language'];
}
if (response.status === 500 && response.statusText === 'Evaluation failed') {
return [false, 'Evaluation failed'];
}
if (response.status === 504 && response.statusText === 'Evaluation timed out') {
return [false, 'Evaluation timed out'];
}
throw new Error(`Unexpected ${response.status} response from ['Myriad', ${response.statusText}`);
}
const body = await response.json();
return [true, body.result || '\n'];
}
getContainers() {
return fetch(this.url('containers'), { method: 'GET' }).then(x => x.json());
}
postCleanup() {
return fetch(this.url('cleanup'), { method: 'POST' }).then(x => x.json());
}
}
const entries = [
['apl', ['apl']],
['bash', ['bash', 'sh']],
['brainfuck', ['brainfuck', 'bf']],
['c', ['c']],
['clojure', ['clojure', 'clj']],
['cpp', ['cpp']],
['csharp', ['csharp', 'cs']],
['elixir', ['elixir']],
['fsharp', ['fsharp', 'fs']],
['go', ['golang', 'go']],
['haskell', ['haskell', 'hs']],
['java', ['java']],
['javascript', ['javascript', 'js']],
['julia', ['julia']],
['lua', ['lua']],
['ocaml', ['ocaml', 'ml']],
['pascal', ['pascal', 'pas', 'freepascal']],
['perl', ['perl', 'pl']],
['php', ['php']],
['prolog', ['prolog']],
['python', ['python', 'py']],
['racket', ['lisp']],
['ruby', ['ruby', 'rb']],
['rust', ['rust', 'rs']]
];
Myriad.Languages = new Map(entries);
Myriad.Aliases = new Map();
for (const [l, xs] of entries) {
for (const x of xs) {
Myriad.Aliases.set(x, l);
}
}
module.exports = Myriad;

View file

@ -1,38 +0,0 @@
class Queue {
constructor(limit) {
this.limit = limit;
this.tasks = [];
this.ongoing = 0;
}
get length() {
return this.tasks.length;
}
enqueue(task) {
return new Promise((resolve, reject) => {
this.tasks.push({ task, resolve, reject });
if (this.ongoing < this.limit) {
this.process();
}
});
}
async process() {
this.ongoing++;
const { task, resolve, reject } = this.tasks.shift();
try {
const x = await task();
resolve(x);
} catch (e) {
reject(e);
}
this.ongoing--;
while (this.ongoing < this.limit && this.tasks.length > 0) {
this.process();
}
}
}
module.exports = Queue;