diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..793f2eb --- /dev/null +++ b/LICENSE @@ -0,0 +1,21 @@ +The MIT License + +Copyright © 2011-2012 John Crepezzi + +Permission is hereby granted, free of charge, to any person obtaining a copy of +this software and associated documentation files (the ‘Software’), to deal in +the Software without restriction, including without limitation the rights to +use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies +of the Software, and to permit persons to whom the Software is furnished to do +so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED ‘AS IS’, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE \ No newline at end of file diff --git a/README.md b/README.md index 5c6a014..cdfc84a 100644 --- a/README.md +++ b/README.md @@ -1,9 +1,15 @@ +# DISCLAIMER + +This is a continued version of hastebin server with extended development, developed alone by zneix. +Original developer abandoned this amazing project and due to pile of unmerged Pull Requests and several security issues with outdated dependencies I decided to rewrite whole project in JavaScript ES6. + +**This version is heavily changed, meaning there will be breaking changes in your config if you were running outdated upstream version.** + # Haste -Haste is an open-source pastebin software written in node.js, which is easily -installable in any network. It can be backed by either redis or filesystem, -and has a very easy adapter interface for other stores. A publicly available -version can be found at [hastebin.com](http://hastebin.com) +Haste is an open-source pastebin software written in Node.JS, which is easily installable in any network. +It can be backed by either redis or filesystem and has a very easy adapter interface for other storage systems. +A publicly available version can be found at [haste.zneix.eu](https://haste.zneix.eu) Major design objectives: @@ -11,297 +17,28 @@ Major design objectives: * Be really simple * Be easy to set up and use -Haste works really well with a little utility called -[haste-client](https://github.com/seejohnrun/haste-client), allowing you -to do things like: +~~Haste works really well with a little utility called [haste-client](https://github.com/seejohnrun/haste-client), allowing you to do things like:~~ -`cat something | haste` +~~`cat file | haste`~~ -which will output a URL to share containing the contents of `cat something`'s -STDOUT. Check the README there for more details and usages. +~~which will output a URL to paste containing the contents of `file`. Check the README there for more details and usages.~~ -## Tested Browsers +There will be a rewritten version of this CLI Haste Client in Golang soon. -* Firefox 81 -* Chrome 17 -* Safari 5.3 -## Installation +# Installation -1. Download the package, and expand it -2. Explore the settings inside of config.js, but the defaults should be good -3. `npm install` -4. `npm start` +Full installation and config instructions can be found in [`docs` directory](https://github.com/zneix/haste-server/tree/master/docs). -## Settings -* `host` - the host the server runs on (default localhost) -* `port` - the port the server runs on (default 7777) -* `keyLength` - the length of the keys to user (default 10) -* `maxLength` - maximum length of a paste (default 400000) -* `staticMaxAge` - max age for static assets (86400) -* `compressStaticAssets` - whether or not to compile static js assets (true) -* `documents` - static documents to serve (ex: http://hastebin.com/about.md) - in addition to static assets. These will never expire. -* `storage` - storage options (see below) -* `logging` - logging preferences -* `keyGenerator` - key generator options (see below) -* `rateLimits` - settings for rate limiting (see below) +## Authors -## Rate Limiting +Project continued by zneix -When present, the `rateLimits` option enables built-in rate limiting courtesy -of `express-rate-limit`. Any of the options supported by that library can be -used and set in `config.js`. +Original Code by John Crepezzi -See the README for [express-rate-limit](https://www.npmjs.com/package/express-rate-limit) -for more information! -## Key Generation - -### Phonetic - -Attempts to generate phonetic keys, similar to `pwgen` - -``` json -{ - "type": "phonetic" -} -``` - -### Random - -Generates a random key - -``` json -{ - "type": "random", - "keyspace": "abcdef" -} -``` - -The _optional_ keySpace argument is a string of acceptable characters -for the key. - -## Storage - -### File - -To use file storage (the default) change the storage section in `config.js` to -something like: - -``` json -{ - "type": "file", - "path": "./data" -} -``` - -where `path` represents where you want the files stored. - -File storage currently does not support paste expiration, you can follow [#191](https://github.com/seejohnrun/haste-server/issues/191) for status updates. - -### Redis - -To use redis storage you must install the `redis` package in npm, and have -`redis-server` running on the machine. - -`npm install redis` - -Once you've done that, your config section should look like: - -``` json -{ - "type": "redis", - "host": "localhost", - "port": 6379, - "db": 2 -} -``` - -You can also set an `expire` option to the number of seconds to expire keys in. -This is off by default, but will constantly kick back expirations on each view -or post. - -All of which are optional except `type` with very logical default values. - -If your Redis server is configured for password authentification, use the `password` field. - -### Postgres - -To use postgres storage you must install the `pg` package in npm - -`npm install pg` - -Once you've done that, your config section should look like: - -``` json -{ - "type": "postgres", - "connectionUrl": "postgres://user:password@host:5432/database" -} -``` - -You can also just set the environment variable for `DATABASE_URL` to your database connection url. - -You will have to manually add a table to your postgres database: - -`create table entries (id serial primary key, key varchar(255) not null, value text not null, expiration int, unique(key));` - -You can also set an `expire` option to the number of seconds to expire keys in. -This is off by default, but will constantly kick back expirations on each view -or post. - -All of which are optional except `type` with very logical default values. - -### MongoDB - -To use mongodb storage you must install the 'mongodb' pachage in npm - -`npm install mongodb` - -Once you've done that, your config section should look like: - -``` json -{ - "type": "mongodb", - "expire": false, - "connectionUrl": "mongodb://localhost:27017/haste", - "clientOptions": { - "useUnifiedTopology": true, - "useNewUrlParser": true, - "keepAlive": true, - "keepAliveInitialDelay": 60000, - "poolSize": 30, - "socketTimeoutMS": 360000, - "connectTimeoutMS": 360000, - "auth": { - "user": "username", - "password": "password" - }, - "authSource": "admin" - } -} -``` - -You can adjust properties in `clientOptions`, but the ones in config are considered as the most optimal defaults. -You can omit `auth` object if database has no authentication. -No further database configuration is required. - -You can set `expire` option to the number of seconds in which documents will expire. -This is set to `false` by default, but will constantly kick back expirations on each view or post. - -### Memcached - -To use memcache storage you must install the `memcached` package via npm - -`npm install memcached` - -Once you've done that, your config section should look like: - -``` json -{ - "type": "memcached", - "host": "127.0.0.1", - "port": 11211 -} -``` - -You can also set an `expire` option to the number of seconds to expire keys in. -This behaves just like the redis expirations, but does not push expirations -forward on GETs. - -All of which are optional except `type` with very logical default values. - -### RethinkDB - -To use the RethinkDB storage system, you must install the `rethinkdbdash` package via npm - -`npm install rethinkdbdash` - -Once you've done that, your config section should look like this: - -``` json -{ - "type": "rethinkdb", - "host": "127.0.0.1", - "port": 28015, - "db": "haste" -} -``` - -In order for this to work, the database must be pre-created before the script is ran. -Also, you must create an `uploads` table, which will store all the data for uploads. - -You can optionally add the `user` and `password` properties to use a user system. - -### Amazon S3 - -To use [Amazon S3](https://aws.amazon.com/s3/) as a storage system, you must -install the `aws-sdk` package via npm: - -`npm install aws-sdk` - -Once you've done that, your config section should look like this: - -```json -{ - "type": "amazon-s3", - "bucket": "your-bucket-name", - "region": "us-east-1" -} -``` - -Authentication is handled automatically by the client. Check -[Amazon's documentation](https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/setting-credentials-node.html) -for more information. You will need to grant your role these permissions to -your bucket: - -```json -{ - "Version": "2012-10-17", - "Statement": [ - { - "Action": [ - "s3:GetObject", - "s3:PutObject" - ], - "Effect": "Allow", - "Resource": "arn:aws:s3:::your-bucket-name-goes-here/*" - } - ] -} -``` - -## Author - -John Crepezzi - -## License - -(The MIT License) - -Copyright © 2011-2012 John Crepezzi - -Permission is hereby granted, free of charge, to any person obtaining a copy of -this software and associated documentation files (the ‘Software’), to deal in -the Software without restriction, including without limitation the rights to -use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies -of the Software, and to permit persons to whom the Software is furnished to do -so, subject to the following conditions: - -The above copyright notice and this permission notice shall be included in all -copies or substantial portions of the Software. - -THE SOFTWARE IS PROVIDED ‘AS IS’, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR -IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, -FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE -AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER -LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, -OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE -SOFTWARE - -### Other components: +## Other components: * jQuery: MIT/GPL license * highlight.js: Copyright © 2006, Ivan Sagalaev diff --git a/about.md b/about.md index 992c303..6966b27 100644 --- a/about.md +++ b/about.md @@ -1,6 +1,6 @@ # Haste -Sharing code is a good thing, and it should be _really_ easy to do it. +Sharing code is a good thing, and it should be _really_ easy to do. A lot of times, I want to show you something I'm seeing - and that's where we use pastebins. @@ -8,28 +8,25 @@ Haste is the prettiest, easiest to use pastebin ever made. ## Basic Usage -Type what you want me to see, click "Save", and then copy the URL. Send that -URL to someone and they'll see what you see. +Paste your code, click "Save" and copy the URL. +Send that URL to someone and they'll see your code. -To make a new entry, click "New" (or type 'control + n') +To make a new entry, click "New" (or hit `Ctrl + N`) ## From the Console -Most of the time I want to show you some text, it's coming from my current -console session. We should make it really easy to take code from the console -and send it to people. +Sometimes I want to show you some text from my current console session. +We made it really easy to take code from the console and send it to people. -`cat something | haste` # https://hastebin.com/1238193 +`cat something | haste` # https://hastebin.com/2107420 -You can even take this a step further, and cut out the last step of copying the -URL with: +You can take this a step further and directly copy the URL with: * osx: `cat something | haste | pbcopy` * linux: `cat something | haste | xsel` * windows: check out [WinHaste](https://github.com/ajryan/WinHaste) -After running that, the STDOUT output of `cat something` will show up at a URL -which has been conveniently copied to your clipboard. +After running that, URL to your paste will be copied to clipboard. That's all there is to that, and you can install it with `gem install haste` right now. @@ -38,15 +35,14 @@ right now. ## Duration -Pastes will stay for 30 days from their last view. They may be removed earlier -and without notice. +Pastes will not be removed, however I preseve all rights to make any exceptions. +Contact me directly with any issues about documents uploaded at `zzneix@gmail.com` ## Privacy -While the contents of hastebin.com are not directly crawled by any search robot -that obeys "robots.txt", there should be no great expectation of privacy. Post -things at your own risk. Not responsible for any loss of data or removed -pastes. +While the contents of haste.zneix.eu are not crawled by any search robot that +obeys "robots.txt", there should be no great expectation of privacy. +Post things at your own risk. Not responsible for any removed pastes. ## Open Source @@ -55,8 +51,9 @@ Haste can easily be installed behind your network, and it's all open source! * [haste-client](https://github.com/seejohnrun/haste-client) * [haste-server](https://github.com/zneix/haste-server) -## Author +## Authors + +Project continued by zneix Original Code by John Crepezzi -Continued by zneix Key Design by Brian Dawson diff --git a/docs/generators.md b/docs/generators.md new file mode 100644 index 0000000..bc3259a --- /dev/null +++ b/docs/generators.md @@ -0,0 +1,46 @@ +# Generators + +Here's a list of all supported random string generators. +One of these is meant to be set in `config.json` as `keyGenerator` object. +Default type is [random](#random) with all alphanumeric characters as keyspace. + + +**Table of Contents** + +- [Phonetic](#phonetic) +- [Random](#random) +- [Dictionary](#dictionary) + +## Random + +Generates a random key from set of characters in `keysapce`. +Keyspace can be left empty to use all alphanumeric characters instead. + +``` json +{ + "type": "random", + "keyspace": "abcdef" +} +``` + +## Phonetic + +Generates phonetic key with a combination of vovels similar to `pwgen` command on linux. + +``` json +{ + "type": "phonetic" +} +``` + +## Dictionary + +Generates a key consisting of words from file named `words.txt`, one word per line. +To avoid any issues with URL length, it is recommended to use `keyLength` 5 or shorter. + +```json +{ + "type": "dictionary", + "path": "./words.txt" +} +``` \ No newline at end of file diff --git a/docs/install.md b/docs/install.md new file mode 100644 index 0000000..cf736c9 --- /dev/null +++ b/docs/install.md @@ -0,0 +1,118 @@ +# Short instructions + +1. Install [node](https://nodejs.org/en/) and npm packages. +2. Rename `example.config.js` to `config.js`. (Defaults are enough) +3. (Optional) Change document storage system ([guide](./storage.md)) +4. Run with `npm start` or `node .` + + + +# Full installation instructions + +> **NOTE:** This guide assumes you're on a Linux server, like Debian 10 or Ubuntu 20.04. +If you're using windows, you will have to improvise a bit with Autorestarts. + +**Table of Contents** + +- [Set up node](#set-up-node) +- [Install dependencies](#install-dependencies) +- [Edit configuration](#edit-configuration) +- [Run the server](#run-the-server) +- [Further steps](#further-steps) + + +## Set up node + +Hastebin server runs on the JavaScript runtime called node, so you need to download it first: + +```bash +curl -sL https://deb.nodesource.com/setup_14.x | bash - +apt-get install -y nodejs +``` + +If method above didn't work [use this guide](https://github.com/nodesource/distributions/blob/master/README.md#installation-instructions). + +For Windows enviroments, you can [download and install node from its website](https://nodejs.org/en/download/current/). + +Node should also automatically install [npm](https://docs.npmjs.com/about-npm/) which will help us install dependencies. You can verify your installations, with following commands + +```bash +node -v +npm -v +``` + + +## Install dependencies + +Hastebin makes use of several node packages, which can be installed with npm by executing this command: + +```bash +npm install +``` + + +## Edit configuration + +There is an [example configuration file](../example.config.json) available for you to copy: + +```bash +cp example.config.json config.json +``` + +By default storage system stores haste documents as hashed files in `./data` directory. + +Rest of provided defaults are enough to run the server, however, example config contains comments about what each value does and you can adjust the options to your likings. + +[Guide for all supported storage systems.](./storage.md). +[Guide for all supported key generators.](./generators.md). + + +## Run the server + +You should now be able to start the server with the following command: + +```bash +npm run-script start +``` + +To stop server the server, hit `Ctrl + C` + +## Further steps + +Once you're good to go and server is running you probably want to keep it alive 24/7 - to achieve that you can choose from variety of things like: pm2, systemd, screen, etc. +Here, I will cover a basic guide to the PM2 - a program which will make sure your server is running 24/7 with autorestarts. + +Download it with npm: + +```bash +sudo npm install pm2 -g +``` + +Now assign Hastebin server to pm2's daemon: + +```bash +pm2 start "npm start" --name=haste +``` + +To ensure haste server will automatically start after server reboot you need to retrieve startup script and execute it. +Example: + +```bash +$ pm2 startup +[PM2] Init System found: systemd +[PM2] To setup the Startup Script, copy/paste the following command: +sudo env PATH=$PATH:/usr/bin /usr/lib/node_modules/pm2/bin/pm2 startup systemd -u zneix --hp /home/zneix +``` + +Now, just copy-paste the last line into your terminal and you should be done. + + + +PM2 Commands: + +- `pm2 list`: displays list of all pm2 processes +- `pm2 logs haste`: displays server logs + +- `pm2 restart haste`: restarts Hastebin server (you will need to do that after config changes, etc.) +- `pm2 stop haste`: stops Hastebin server +- `pm2 start haste`: starts Hastebin server if it was stopped manually before \ No newline at end of file diff --git a/docs/storage.md b/docs/storage.md new file mode 100644 index 0000000..b507402 --- /dev/null +++ b/docs/storage.md @@ -0,0 +1,100 @@ +# Storage + +Here's a list of all supported document store systems. +One of these is meant to be set in `config.json` as `storage` object. +Default type is [file](#file) with save directory at `./data`. +With some storage options you can set up document expiration - after which documents will no longer be available + +**Table of Contents** + +In alphabetical order: + +- [Amazon S3](#amazon-s3) +- [File](#file) +- [Memcached](#memcached) +- [MongoDB](#mongodb) +- [Postgres](#postgres) +- [Redis](#redis) +- [RethinkDB](#rethinkdb) + + +## Amazon S3 + +Not rewritten yet, to be filled in + + +## File + +Default storage option, with no further installation required. +Stores documents in a specified directory in files named with a md5 hash of the key to avoid any security issues. +Default path is `./data` +> **NOTE:** File storage option does not support document expiration! + +Config: + +```json +{ + "type": "file", + "path": "./data" +} +``` + + +## Memcached + +Not rewritten yet, to be filled in + + +## MongoDB + +Requires npm package (Tested on v3.6.0): + +```bash +npm i mongodb@3.6.0 +``` + +Stores documents in a specified database in a collection named `entries`. +Expiration property in config can be changed to a value in seconds after which entries will not be served to users. + +Optimal default config: +> **NOTE:** Depending on how your MongoDB server is configured, options as connectionUri may vary. +If server has no authentication, you can omit the `auth` object. + +Check [documentation](http://mongodb.github.io/node-mongodb-native/3.5/api/MongoClient.html) for more detailed explanation about available `clientOptions` properties. + +```json +{ + "type": "mongodb", + "expire": 0, + "connectionUri": "mongodb://localhost:27017/haste", + "clientOptions": { + "useUnifiedTopology": true, + "useNewUrlParser": true, + "keepAlive": true, + "keepAliveInitialDelay": 60000, + "poolSize": 30, + "socketTimeoutMS": 360000, + "connectTimeoutMS": 360000, + "auth": { + "user": "username", + "password": "password" + }, + "authSource": "admin" + } +} +``` + + +## Postgres + +Not rewritten yet, to be filled in + + +## Redis + +Not rewritten yet, to be filled in + + +## RethinkDB + +Not rewritten yet, to be filled in diff --git a/example.config.js b/example.config.js index 3668304..aa59fb4 100644 --- a/example.config.js +++ b/example.config.js @@ -27,8 +27,8 @@ module.exports = { "level": "info" }, - //rate limits for requests, handled by express-rate-limit - //options can be found here: https://github.com/nfriedly/express-rate-limit/blob/master/lib/express-rate-limit.js#L7-L14 + //rate limits for requests, can be omitted + //handled by express-rate-limit, options can be found here: https://github.com/nfriedly/express-rate-limit/blob/master/lib/express-rate-limit.js#L7-L14 "rateLimits": { "windowMs": 30 * 60 * 1000, "max": 250 diff --git a/lib/document_stores/file.js b/lib/document_stores/file.js index 2930325..44499e5 100644 --- a/lib/document_stores/file.js +++ b/lib/document_stores/file.js @@ -1,11 +1,7 @@ +const winston = require('winston'); const fs = require('fs'); const crypto = require('crypto'); -const winston = require('winston'); - -// For storing in files -// options[type] = file -// options[path] - Where to store class FileDocumentStore { diff --git a/lib/document_stores/mongodb.js b/lib/document_stores/mongodb.js index f42e8fe..b685a52 100644 --- a/lib/document_stores/mongodb.js +++ b/lib/document_stores/mongodb.js @@ -3,7 +3,7 @@ const mongodb = require('mongodb'); const MongoDocumentStore = function (config){ this.expire = config.expire; - this.MongoClient = new mongodb.MongoClient(config.connectionUrl, config.clientOptions); + this.MongoClient = new mongodb.MongoClient(config.connectionUri, config.clientOptions); }; MongoDocumentStore.prototype.set = async function (key, data, callback, skipExpire){