SunRed
/
haste-server
mirror of https://github.com/SunRed/haste-server.git
1
0
Fork 0
Code Issues Releases Activity

Added proper documentation

This commit is contained in:
zneix 2020-08-29 05:29:39 +02:00
parent 1efc62a8c5
commit b4f42219e5
No known key found for this signature in database
GPG Key ID: 911916E0523B22F6
9 changed files with 326 additions and 311 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

21
LICENSE Normal file
View File

@ -0,0 +1,21 @@
The MIT License
Copyright © 2011-2012 John Crepezzi
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the ‘Software’), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
of the Software, and to permit persons to whom the Software is furnished to do
so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED ‘AS IS’, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE

303
README.md
View File

@ -1,9 +1,15 @@
# DISCLAIMER
This is a continued version of hastebin server with extended development, developed alone by zneix.
Original developer abandoned this amazing project and due to pile of unmerged Pull Requests and several security issues with outdated dependencies I decided to rewrite whole project in JavaScript ES6.
**This version is heavily changed, meaning there will be breaking changes in your config if you were running outdated upstream version.**
# Haste
Haste is an open-source pastebin software written in node.js, which is easily
installable in any network. It can be backed by either redis or filesystem,
and has a very easy adapter interface for other stores. A publicly available
version can be found at [hastebin.com](http://hastebin.com)
Haste is an open-source pastebin software written in Node.JS, which is easily installable in any network.
It can be backed by either redis or filesystem and has a very easy adapter interface for other storage systems.
A publicly available version can be found at [haste.zneix.eu](https://haste.zneix.eu)
Major design objectives:
@ -11,297 +17,28 @@ Major design objectives:
* Be really simple
* Be easy to set up and use
Haste works really well with a little utility called
[haste-client](https://github.com/seejohnrun/haste-client), allowing you
to do things like:
~~Haste works really well with a little utility called [haste-client](https://github.com/seejohnrun/haste-client), allowing you to do things like:~~
`cat something | haste`
~~`cat file | haste`~~
which will output a URL to share containing the contents of `cat something`'s
STDOUT. Check the README there for more details and usages.
~~which will output a URL to paste containing the contents of `file`. Check the README there for more details and usages.~~
## Tested Browsers
There will be a rewritten version of this CLI Haste Client in Golang soon.
* Firefox 81
* Chrome 17
* Safari 5.3
## Installation
# Installation
1. Download the package, and expand it
2. Explore the settings inside of config.js, but the defaults should be good
3. `npm install`
4. `npm start`
Full installation and config instructions can be found in [`docs` directory](https://github.com/zneix/haste-server/tree/master/docs).
## Settings
* `host` - the host the server runs on (default localhost)
* `port` - the port the server runs on (default 7777)
* `keyLength` - the length of the keys to user (default 10)
* `maxLength` - maximum length of a paste (default 400000)
* `staticMaxAge` - max age for static assets (86400)
* `compressStaticAssets` - whether or not to compile static js assets (true)
* `documents` - static documents to serve (ex: http://hastebin.com/about.md)
in addition to static assets. These will never expire.
* `storage` - storage options (see below)
* `logging` - logging preferences
* `keyGenerator` - key generator options (see below)
* `rateLimits` - settings for rate limiting (see below)
## Authors
## Rate Limiting
Project continued by zneix <zzneix@gmail.com>
When present, the `rateLimits` option enables built-in rate limiting courtesy
of `express-rate-limit`. Any of the options supported by that library can be
used and set in `config.js`.
Original Code by John Crepezzi <john.crepezzi@gmail.com>
See the README for [express-rate-limit](https://www.npmjs.com/package/express-rate-limit)
for more information!
## Key Generation
### Phonetic
Attempts to generate phonetic keys, similar to `pwgen`
``` json
{
"type": "phonetic"
}
```
### Random
Generates a random key
``` json
{
"type": "random",
"keyspace": "abcdef"
}
```
The _optional_ keySpace argument is a string of acceptable characters
for the key.
## Storage
### File
To use file storage (the default) change the storage section in `config.js` to
something like:
``` json
{
"type": "file",
"path": "./data"
}
```
where `path` represents where you want the files stored.
File storage currently does not support paste expiration, you can follow [#191](https://github.com/seejohnrun/haste-server/issues/191) for status updates.
### Redis
To use redis storage you must install the `redis` package in npm, and have
`redis-server` running on the machine.
`npm install redis`
Once you've done that, your config section should look like:
``` json
{
"type": "redis",
"host": "localhost",
"port": 6379,
"db": 2
}
```
You can also set an `expire` option to the number of seconds to expire keys in.
This is off by default, but will constantly kick back expirations on each view
or post.
All of which are optional except `type` with very logical default values.
If your Redis server is configured for password authentification, use the `password` field.
### Postgres
To use postgres storage you must install the `pg` package in npm
`npm install pg`
Once you've done that, your config section should look like:
``` json
{
"type": "postgres",
"connectionUrl": "postgres://user:password@host:5432/database"
}
```
You can also just set the environment variable for `DATABASE_URL` to your database connection url.
You will have to manually add a table to your postgres database:
`create table entries (id serial primary key, key varchar(255) not null, value text not null, expiration int, unique(key));`
You can also set an `expire` option to the number of seconds to expire keys in.
This is off by default, but will constantly kick back expirations on each view
or post.
All of which are optional except `type` with very logical default values.
### MongoDB
To use mongodb storage you must install the 'mongodb' pachage in npm
`npm install mongodb`
Once you've done that, your config section should look like:
``` json
{
"type": "mongodb",
"expire": false,
"connectionUrl": "mongodb://localhost:27017/haste",
"clientOptions": {
"useUnifiedTopology": true,
"useNewUrlParser": true,
"keepAlive": true,
"keepAliveInitialDelay": 60000,
"poolSize": 30,
"socketTimeoutMS": 360000,
"connectTimeoutMS": 360000,
"auth": {
"user": "username",
"password": "password"
},
"authSource": "admin"
}
}
```
You can adjust properties in `clientOptions`, but the ones in config are considered as the most optimal defaults.
You can omit `auth` object if database has no authentication.
No further database configuration is required.
You can set `expire` option to the number of seconds in which documents will expire.
This is set to `false` by default, but will constantly kick back expirations on each view or post.
### Memcached
To use memcache storage you must install the `memcached` package via npm
`npm install memcached`
Once you've done that, your config section should look like:
``` json
{
"type": "memcached",
"host": "127.0.0.1",
"port": 11211
}
```
You can also set an `expire` option to the number of seconds to expire keys in.
This behaves just like the redis expirations, but does not push expirations
forward on GETs.
All of which are optional except `type` with very logical default values.
### RethinkDB
To use the RethinkDB storage system, you must install the `rethinkdbdash` package via npm
`npm install rethinkdbdash`
Once you've done that, your config section should look like this:
``` json
{
"type": "rethinkdb",
"host": "127.0.0.1",
"port": 28015,
"db": "haste"
}
```
In order for this to work, the database must be pre-created before the script is ran.
Also, you must create an `uploads` table, which will store all the data for uploads.
You can optionally add the `user` and `password` properties to use a user system.
### Amazon S3
To use [Amazon S3](https://aws.amazon.com/s3/) as a storage system, you must
install the `aws-sdk` package via npm:
`npm install aws-sdk`
Once you've done that, your config section should look like this:
```json
{
"type": "amazon-s3",
"bucket": "your-bucket-name",
"region": "us-east-1"
}
```
Authentication is handled automatically by the client. Check
[Amazon's documentation](https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/setting-credentials-node.html)
for more information. You will need to grant your role these permissions to
your bucket:
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Action": [
"s3:GetObject",
"s3:PutObject"
],
"Effect": "Allow",
"Resource": "arn:aws:s3:::your-bucket-name-goes-here/*"
}
]
}
```
## Author
John Crepezzi <john.crepezzi@gmail.com>
## License
(The MIT License)
Copyright © 2011-2012 John Crepezzi
Permission is hereby granted, free of charge, to any person obtaining a copy of
this software and associated documentation files (the ‘Software’), to deal in
the Software without restriction, including without limitation the rights to
use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies
of the Software, and to permit persons to whom the Software is furnished to do
so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED ‘AS IS’, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE
### Other components:
## Other components:
* jQuery: MIT/GPL license
* highlight.js: Copyright © 2006, Ivan Sagalaev

37
about.md
View File

@ -1,6 +1,6 @@
# Haste
Sharing code is a good thing, and it should be _really_ easy to do it.
Sharing code is a good thing, and it should be _really_ easy to do.
A lot of times, I want to show you something I'm seeing - and that's where we
use pastebins.
@ -8,28 +8,25 @@ Haste is the prettiest, easiest to use pastebin ever made.
## Basic Usage
Type what you want me to see, click "Save", and then copy the URL. Send that
URL to someone and they'll see what you see.
Paste your code, click "Save" and copy the URL.
Send that URL to someone and they'll see your code.
To make a new entry, click "New" (or type 'control + n')
To make a new entry, click "New" (or hit `Ctrl + N`)
## From the Console
Most of the time I want to show you some text, it's coming from my current
console session. We should make it really easy to take code from the console
and send it to people.
Sometimes I want to show you some text from my current console session.
We made it really easy to take code from the console and send it to people.
`cat something | haste` # https://hastebin.com/1238193
`cat something | haste` # https://hastebin.com/2107420
You can even take this a step further, and cut out the last step of copying the
URL with:
You can take this a step further and directly copy the URL with:
* osx: `cat something | haste | pbcopy`
* linux: `cat something | haste | xsel`
* windows: check out [WinHaste](https://github.com/ajryan/WinHaste)
After running that, the STDOUT output of `cat something` will show up at a URL
which has been conveniently copied to your clipboard.
After running that, URL to your paste will be copied to clipboard.
That's all there is to that, and you can install it with `gem install haste`
right now.
@ -38,15 +35,14 @@ right now.
## Duration
Pastes will stay for 30 days from their last view. They may be removed earlier
and without notice.
Pastes will not be removed, however I preseve all rights to make any exceptions.
Contact me directly with any issues about documents uploaded at `zzneix@gmail.com`
## Privacy
While the contents of hastebin.com are not directly crawled by any search robot
that obeys "robots.txt", there should be no great expectation of privacy. Post
things at your own risk. Not responsible for any loss of data or removed
pastes.
While the contents of haste.zneix.eu are not crawled by any search robot that
obeys "robots.txt", there should be no great expectation of privacy.
Post things at your own risk. Not responsible for any removed pastes.
## Open Source
@ -55,8 +51,9 @@ Haste can easily be installed behind your network, and it's all open source!
* [haste-client](https://github.com/seejohnrun/haste-client)
* [haste-server](https://github.com/zneix/haste-server)
## Author
## Authors
Project continued by zneix <zzneix@gmail.com>
Original Code by John Crepezzi <john.crepezzi@gmail.com>
Continued by zneix <zzneix@gmail.com>
Key Design by Brian Dawson <bridawson@gmail.com>

46
docs/generators.md Normal file
View File

@ -0,0 +1,46 @@
# Generators
Here's a list of all supported random string generators.
One of these is meant to be set in `config.json` as `keyGenerator` object.
Default type is [random](#random) with all alphanumeric characters as keyspace.
**Table of Contents**
- [Phonetic](#phonetic)
- [Random](#random)
- [Dictionary](#dictionary)
## Random
Generates a random key from set of characters in `keysapce`.
Keyspace can be left empty to use all alphanumeric characters instead.
``` json
{
"type": "random",
"keyspace": "abcdef"
}
```
## Phonetic
Generates phonetic key with a combination of vovels similar to `pwgen` command on linux.
``` json
{
"type": "phonetic"
}
```
## Dictionary
Generates a key consisting of words from file named `words.txt`, one word per line.
To avoid any issues with URL length, it is recommended to use `keyLength` 5 or shorter.
```json
{
"type": "dictionary",
"path": "./words.txt"
}
```

118
docs/install.md Normal file
View File

@ -0,0 +1,118 @@
# Short instructions
1. Install [node](https://nodejs.org/en/) and npm packages.
2. Rename `example.config.js` to `config.js`. (Defaults are enough)
3. (Optional) Change document storage system ([guide](./storage.md))
4. Run with `npm start` or `node .`
# Full installation instructions
> **NOTE:** This guide assumes you're on a Linux server, like Debian 10 or Ubuntu 20.04.
If you're using windows, you will have to improvise a bit with Autorestarts.
**Table of Contents**
- [Set up node](#set-up-node)
- [Install dependencies](#install-dependencies)
- [Edit configuration](#edit-configuration)
- [Run the server](#run-the-server)
- [Further steps](#further-steps)
## Set up node
Hastebin server runs on the JavaScript runtime called node, so you need to download it first:
```bash
curl -sL https://deb.nodesource.com/setup_14.x | bash -
apt-get install -y nodejs
```
If method above didn't work [use this guide](https://github.com/nodesource/distributions/blob/master/README.md#installation-instructions).
For Windows enviroments, you can [download and install node from its website](https://nodejs.org/en/download/current/).
Node should also automatically install [npm](https://docs.npmjs.com/about-npm/) which will help us install dependencies. You can verify your installations, with following commands
```bash
node -v
npm -v
```
## Install dependencies
Hastebin makes use of several node packages, which can be installed with npm by executing this command:
```bash
npm install
```
## Edit configuration
There is an [example configuration file](../example.config.json) available for you to copy:
```bash
cp example.config.json config.json
```
By default storage system stores haste documents as hashed files in `./data` directory.
Rest of provided defaults are enough to run the server, however, example config contains comments about what each value does and you can adjust the options to your likings.
[Guide for all supported storage systems.](./storage.md).
[Guide for all supported key generators.](./generators.md).
## Run the server
You should now be able to start the server with the following command:
```bash
npm run-script start
```
To stop server the server, hit `Ctrl + C`
## Further steps
Once you're good to go and server is running you probably want to keep it alive 24/7 - to achieve that you can choose from variety of things like: pm2, systemd, screen, etc.
Here, I will cover a basic guide to the PM2 - a program which will make sure your server is running 24/7 with autorestarts.
Download it with npm:
```bash
sudo npm install pm2 -g
```
Now assign Hastebin server to pm2's daemon:
```bash
pm2 start "npm start" --name=haste
```
To ensure haste server will automatically start after server reboot you need to retrieve startup script and execute it.
Example:
```bash
$ pm2 startup
[PM2] Init System found: systemd
[PM2] To setup the Startup Script, copy/paste the following command:
sudo env PATH=$PATH:/usr/bin /usr/lib/node_modules/pm2/bin/pm2 startup systemd -u zneix --hp /home/zneix
```
Now, just copy-paste the last line into your terminal and you should be done.
PM2 Commands:
- `pm2 list`: displays list of all pm2 processes
- `pm2 logs haste`: displays server logs
- `pm2 restart haste`: restarts Hastebin server (you will need to do that after config changes, etc.)
- `pm2 stop haste`: stops Hastebin server
- `pm2 start haste`: starts Hastebin server if it was stopped manually before

100
docs/storage.md Normal file
View File

@ -0,0 +1,100 @@
# Storage
Here's a list of all supported document store systems.
One of these is meant to be set in `config.json` as `storage` object.
Default type is [file](#file) with save directory at `./data`.
With some storage options you can set up document expiration - after which documents will no longer be available
**Table of Contents**
In alphabetical order:
- [Amazon S3](#amazon-s3)
- [File](#file)
- [Memcached](#memcached)
- [MongoDB](#mongodb)
- [Postgres](#postgres)
- [Redis](#redis)
- [RethinkDB](#rethinkdb)
## Amazon S3
Not rewritten yet, to be filled in
## File
Default storage option, with no further installation required.
Stores documents in a specified directory in files named with a md5 hash of the key to avoid any security issues.
Default path is `./data`
> **NOTE:** File storage option does not support document expiration!
Config:
```json
{
"type": "file",
"path": "./data"
}
```
## Memcached
Not rewritten yet, to be filled in
## MongoDB
Requires npm package (Tested on v3.6.0):
```bash
npm i mongodb@3.6.0
```
Stores documents in a specified database in a collection named `entries`.
Expiration property in config can be changed to a value in seconds after which entries will not be served to users.
Optimal default config:
> **NOTE:** Depending on how your MongoDB server is configured, options as connectionUri may vary.
If server has no authentication, you can omit the `auth` object.
Check [documentation](http://mongodb.github.io/node-mongodb-native/3.5/api/MongoClient.html) for more detailed explanation about available `clientOptions` properties.
```json
{
"type": "mongodb",
"expire": 0,
"connectionUri": "mongodb://localhost:27017/haste",
"clientOptions": {
"useUnifiedTopology": true,
"useNewUrlParser": true,
"keepAlive": true,
"keepAliveInitialDelay": 60000,
"poolSize": 30,
"socketTimeoutMS": 360000,
"connectTimeoutMS": 360000,
"auth": {
"user": "username",
"password": "password"
},
"authSource": "admin"
}
}
```
## Postgres
Not rewritten yet, to be filled in
## Redis
Not rewritten yet, to be filled in
## RethinkDB
Not rewritten yet, to be filled in

4
example.config.js
View File

@ -27,8 +27,8 @@ module.exports = {
"level": "info"
},
//rate limits for requests, handled by express-rate-limit
//options can be found here: https://github.com/nfriedly/express-rate-limit/blob/master/lib/express-rate-limit.js#L7-L14
//rate limits for requests, can be omitted
//handled by express-rate-limit, options can be found here: https://github.com/nfriedly/express-rate-limit/blob/master/lib/express-rate-limit.js#L7-L14
"rateLimits": {
"windowMs": 30 * 60 * 1000,
"max": 250

6
lib/document_stores/file.js
View File

@ -1,11 +1,7 @@
const winston = require('winston');
const fs = require('fs');
const crypto = require('crypto');
const winston = require('winston');
// For storing in files
// options[type] = file
// options[path] - Where to store
class FileDocumentStore {

2
lib/document_stores/mongodb.js
View File

@ -3,7 +3,7 @@ const mongodb = require('mongodb');
const MongoDocumentStore = function (config){
this.expire = config.expire;
this.MongoClient = new mongodb.MongoClient(config.connectionUrl, config.clientOptions);
this.MongoClient = new mongodb.MongoClient(config.connectionUri, config.clientOptions);
};
MongoDocumentStore.prototype.set = async function (key, data, callback, skipExpire){