Upgrade of documentation: build, config, contributing, readme
Jakub Matys
parent
81d8780a92
commit
ef12666f70
|
|
@ -0,0 +1,85 @@
|
|||
# Blockbook Contributor Guide
|
||||
|
||||
Blockbook is back-end service for Trezor wallet. Although it is open source, design and development of core packages
|
||||
is done by Trezor developers in order to keep Blockbook compatible with Trezor. If you feel you could use Blockbook
|
||||
for another purposes, we recommend you to make a fork.
|
||||
|
||||
However you can still help us find bugs or add support for new coins.
|
||||
|
||||
## Development environment
|
||||
|
||||
Instructions to set up your development environment and build Blockbook are described in separated
|
||||
[document](/docs/build.md).
|
||||
|
||||
## How can I contribute?
|
||||
|
||||
### Reporting bugs
|
||||
|
||||
### Adding coin support
|
||||
|
||||
Trezor harware wallet supports over 500 coins, see https://trezor.io/coins/. You are free to add support for any of
|
||||
them to Blockbook. Actually implemented coins are listed [here](/docs/ports.md).
|
||||
|
||||
You should follow few steps bellow to get smooth merge of your PR.
|
||||
|
||||
> Altough we are happy for support of new coins we have not enough capacity to run them all on our infrastructure.
|
||||
> Actually we can run Blockbook instances only for coins supported by Trezor wallet. If you want to have Blockbook
|
||||
> instance for your coin, you will have to deploy your own server.
|
||||
|
||||
#### Add coin definition
|
||||
|
||||
Coin definitions are stored in JSON files in *configs/coins* directory. They are single source of Blockbook
|
||||
configuration, Blockbook and back-end package definition and build metadata. Since Blockbook supports only single
|
||||
coin index per running instance, every coin (including testnet) must have single definition file.
|
||||
|
||||
All options of coin definition are described in [config.md](/docs/config.md).
|
||||
|
||||
Because most of coins are fork of Bitcoin and they have similar way to install and configure their daemon, we use
|
||||
templates to generate package definition and configuration files during build process. It is similar to build Blockbook
|
||||
package too. Templates are filled with data from coin definition. Although build process generate packages
|
||||
automatically, there is sometimes necessary see intermediate step. You can generate all files by calling
|
||||
`go run build/templates/generate.go coin` where *coin* is name of definition file without .json extension. Files are
|
||||
generated to *build/pkg-defs* directory.
|
||||
|
||||
Good examples of coin configuration are
|
||||
[*configs/coins/bitcoin.json*](configs/coins/bitcoin.json) and
|
||||
[*configs/coins/ethereum.json*](configs/coins/ethereum.json) for Bitcoin-like coins and different coins, respectively.
|
||||
|
||||
Usually you have to update only few options that differ from Bitcoin definition. At first there are base information
|
||||
about coin in section *coin* – name, alias etc. Then update port information in *port* section. We keep port series as
|
||||
listed in [port registry](/docs/ports.md). Select next port numbers in series. Port numbers must be unique across all
|
||||
port definitions.
|
||||
|
||||
In section *backend* update information how to build and configure backend service. When back-end package is built,
|
||||
build process downloads installation archive, verify and extract it. How it is done is described in
|
||||
[build guide](/docs/build.md#on-back-end-building). Naming conventions and versioning are described
|
||||
also in [build guide](/docs/build.md#on-naming-conventions-and-versioning). You have to update *package_name*,
|
||||
*package_revision*, *system_user*, *version*, *binary_url*, *verification_type*, *verification_source*, *extract_command* and
|
||||
*exclude_files*. Also update information whether service runs mainnet or testnet network in *mainnet* option.
|
||||
|
||||
In section *blockbook* update information how to build and configure Blockbook service. Usually they are only
|
||||
*package_name*, *system_user* and *explorer_url* options. Naming conventions are are described
|
||||
[here](/docs/build.md#on-naming-conventions-and-versioning).
|
||||
|
||||
Update *package_maintainer* and *package_maintainer_email* options in section *meta*.
|
||||
|
||||
Execute script *contrib/scripts/generate-port-registry.go* that will update *docs/ports.md*.
|
||||
|
||||
Now you can try generate package definitions as described above in order to check outputs.
|
||||
|
||||
##### Go template evaluation note
|
||||
|
||||
We use *text/template* package to generate package definitions and configuration files. Some options in coin definition
|
||||
are also templates and are executed inside base template. Use `{{.path}}` syntax to refer values in coin definition,
|
||||
where *.path* can be for example *.Blockbook.BlockChain.Parse*. Go uses CammelCase notation so references inside templates
|
||||
as well. Note that dot at the beginning is mandatory. Go template syntax is fully documented
|
||||
[here](https://godoc.org/text/template).
|
||||
|
||||
TODO:
|
||||
* script that checks unique port numbers
|
||||
|
||||
#### Add coin implementation
|
||||
|
||||
#### Add tests
|
||||
|
||||
#### Deploy public server
|
||||
153
README.md
153
README.md
|
|
@ -1,161 +1,40 @@
|
|||
# blockbook
|
||||
# Blockbook
|
||||
|
||||
## **blockbook is currently in the state of heavy development, do not expect this documentation to be up to date**
|
||||
> **Blockbook is currently in the state of heavy development, do not expect this documentation to be up to date**
|
||||
|
||||
## Build and install using docker
|
||||
## Build and installation instructions
|
||||
|
||||
Run in the project root
|
||||
|
||||
```
|
||||
make all
|
||||
```
|
||||
|
||||
to create blockbook debian packages.
|
||||
|
||||
## Install manually
|
||||
|
||||
Setup go environment (Debian 9):
|
||||
|
||||
```
|
||||
sudo apt-get update && apt-get install -y \
|
||||
build-essential git wget pkg-config lxc-dev libzmq3-dev libgflags-dev libsnappy-dev zlib1g-dev libbz2-dev liblz4-dev
|
||||
cd /opt
|
||||
wget https://storage.googleapis.com/golang/go1.9.2.linux-amd64.tar.gz && tar xf go1.9.2.linux-amd64.tar.gz
|
||||
sudo ln -s /opt/go/bin/go /usr/bin/go
|
||||
go help gopath
|
||||
```
|
||||
|
||||
Install RocksDB: https://github.com/facebook/rocksdb/blob/master/INSTALL.md
|
||||
and compile the static_lib and tools
|
||||
|
||||
```
|
||||
git clone https://github.com/facebook/rocksdb.git
|
||||
cd rocksdb
|
||||
make release
|
||||
```
|
||||
|
||||
Setup variables for gorocksdb: https://github.com/tecbot/gorocksdb
|
||||
|
||||
```
|
||||
export CGO_CFLAGS="-I/path/to/rocksdb/include"
|
||||
export CGO_LDFLAGS="-L/path/to/rocksdb -lrocksdb -lstdc++ -lm -lz -lbz2 -lsnappy -llz4"
|
||||
```
|
||||
|
||||
Install ZeroMQ: https://github.com/zeromq/libzmq
|
||||
|
||||
Install go-dep tool:
|
||||
```
|
||||
RUN go get github.com/golang/dep/cmd/dep
|
||||
```
|
||||
|
||||
Get blockbook sources, install dependencies, build:
|
||||
|
||||
```
|
||||
cd $GOPATH/src
|
||||
git clone https://github.com/trezor/blockbook.git
|
||||
cd blockbook
|
||||
dep ensure
|
||||
go build
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
```
|
||||
./blockbook --help
|
||||
```
|
||||
|
||||
## Example command
|
||||
To run blockbook with fast synchronization, connection to ZeroMQ and providing https and socket.io interface, with database in local directory *data* and connected to local bitcoind with configuration specified by parameter *-blockchaincfg*:
|
||||
```
|
||||
./blockbook -sync -blockchaincfg=configs/bitcoin_testnet.json -internal=127.0.0.1:8333 -public=127.0.0.1:8334 -certfile=server/testcert -logtostderr
|
||||
```
|
||||
Blockbook logs to stderr *-logtostderr* or to directory specified by parameter *-log_dir* . Verbosity of logs can be tuned by command line parameters *-v* and *-vmodule*, details at https://godoc.org/github.com/golang/glog
|
||||
Develper build guide is [here](/docs/build.md).
|
||||
|
||||
Sysadmin installation guide is [here](https://wiki.trezor.io/Blockbook).
|
||||
|
||||
# Implemented coins
|
||||
|
||||
- [Bitcoin](bchain/coins/btc/btc.md)
|
||||
- [Bitcoin Testnet](bchain/coins/btc/btctestnet.md)
|
||||
The most significant coins implemented by Blockbook are:
|
||||
|
||||
- Bitcoin
|
||||
- Bitcoin Testnet
|
||||
- Bcash
|
||||
- Bcash Testnet
|
||||
- Bgold
|
||||
- [ZCash](bchain/coins/zec/zec.md)
|
||||
- ZCash
|
||||
- ZCash Testnet
|
||||
- Dash
|
||||
- Dash Testnet
|
||||
- Litecoin
|
||||
- Litecoin Testnet
|
||||
- [Ethereum](bchain/coins/eth/eth.md)
|
||||
- [Ethereum Testnet Ropsten](bchain/coins/eth/ethropsten.md)
|
||||
- Ethereum
|
||||
- Ethereum Testnet Ropsten
|
||||
|
||||
They are also supported by Trezor wallet. List of all coins is [here](/docs/ports.md).
|
||||
|
||||
# Data storage in RocksDB
|
||||
|
||||
Blockbook stores data the key-value store RocksDB. Data are stored in binary form to save space.
|
||||
The data are separated to different column families:
|
||||
|
||||
- **default**
|
||||
|
||||
at the moment not used, will store statistical data etc.
|
||||
|
||||
- **height** - maps *block height* to *block hash*
|
||||
|
||||
*Block heigh* stored as array of 4 bytes (big endian uint32)
|
||||
*Block hash* stored as array of 32 bytes
|
||||
|
||||
Example - the first four blocks (all data hex encoded)
|
||||
```
|
||||
0x00000000 : 0x000000000933ea01ad0ee984209779baaec3ced90fa3f408719526f8d77f4943
|
||||
0x00000001 : 0x00000000b873e79784647a6c82962c70d228557d24a747ea4d1b8bbe878e1206
|
||||
0x00000002 : 0x000000006c02c8ea6e4ff69651f7fcde348fb9d557a06e6957b65552002a7820
|
||||
0x00000003 : 0x000000008b896e272758da5297bcd98fdc6d97c9b765ecec401e286dc1fdbe10
|
||||
```
|
||||
|
||||
- **outputs** - maps *output script+block height* to *array of outpoints*
|
||||
|
||||
*Output script (ScriptPubKey)+block height* stored as variable length array of bytes for output script + 4 bytes (big endian uint32) block height
|
||||
*array of outpoints* stored as array of 32 bytes for transaction id + variable length outpoint index for each outpoint
|
||||
|
||||
Example - (all data hex encoded)
|
||||
```
|
||||
0x001400efeb484a24a1c1240eafacef8566e734da429c000e2df6 : 0x1697966cbd76c75eb9fc736dfa3ba0bc045999bab1e8b10082bc0ba546b0178302
|
||||
0xa9143e3d6abe282d92a28cb791697ba001d733cefdc7870012c4b1 : 0x7246e79f97b5f82e7f51e291d533964028ec90be0634af8a8ef7d5a903c7f6d301
|
||||
```
|
||||
|
||||
- **inputs** - maps *transaction outpoint* to *input transaction* that spends it
|
||||
|
||||
*Transaction outpoint* stored as array of 32 bytes for transaction id + variable length outpoint index
|
||||
*Input transaction* stored as array of 32 bytes for transaction id + variable length input index
|
||||
|
||||
Example - (all data hex encoded)
|
||||
```
|
||||
0x7246e79f97b5f82e7f51e291d533964028ec90be0634af8a8ef7d5a903c7f6d300 : 0x0a7aa90ea0269c79f844c516805e4cac594adb8830e56fca894b66aab19136a428
|
||||
0x7246e79f97b5f82e7f51e291d533964028ec90be0634af8a8ef7d5a903c7f6d301 : 0x4303a9fcfe6026b4d33ba488df6443c9a99bca7b7fcb7c6f6cd65cea24a749b700
|
||||
```
|
||||
Blockbook stores data the key-value store RocksDB. Database format is described [here](/docs/rocksdb.md).
|
||||
|
||||
## Registry of ports
|
||||
|
||||
| coin | blockbook internal port | blockbook public port | backend rpc port | zmq port |
|
||||
|--------------------------|-------------------------|-----------------------|--------------------|----------|
|
||||
| Bitcoin | 9030 | 9130 | 8030 | 38330 |
|
||||
| Bcash | 9031 | 9131 | 8031 | 38331 |
|
||||
| Zcash | 9032 | 9132 | 8032 | 38332 |
|
||||
| Dash | 9033 | 9133 | 8033 | 38333 |
|
||||
| Litecoin | 9034 | 9134 | 8034 | 38334 |
|
||||
| Bgold | 9035 | 9135 | 8035 | 38335 |
|
||||
| Ethereum | 9036 | 9136 | 8036 ws, 8136 http | 38336* |
|
||||
| Ethereum Classic | 9037 | 9137 | 8037 | 38337* |
|
||||
| Dogecoin | 9038 | 9138 | 8038 | 38338 |
|
||||
| Namecoin | 9039 | 9139 | 8039 | 38339 |
|
||||
| Vertcoin | 9040 | 9140 | 8040 | 38340 |
|
||||
| Bitcoin Testnet | 19030 | 1913 | 18030 | 48330 |
|
||||
| Bcash Testnet | 19031 | 1913 | 18031 | 48331 |
|
||||
| Zcash Testnet | 19032 | 1913 | 18032 | 48332 |
|
||||
| Dash Testnet | 19033 | 1913 | 18033 | 48333 |
|
||||
| Litecoin Testnet | 19034 | 1913 | 18034 | 48334 |
|
||||
| Ethereum Testnet Ropsten | 19036 | 19136 | 18036 | 48336* |
|
||||
| Vertcoin Testnet | 19040 | 19140 | 18040 | 48340 |
|
||||
|
||||
\* geth listens on this port, however not as zmq service
|
||||
Reserved ports are described [here](/docs/ports.md)
|
||||
|
||||
## Todo
|
||||
|
||||
|
|
|
|||
|
|
@ -1,50 +0,0 @@
|
|||
## BTC Setup
|
||||
Get Bitcoin Core
|
||||
```
|
||||
wget https://bitcoin.org/bin/bitcoin-core-0.16.0/bitcoin-0.15.1-x86_64-linux-gnu.tar.gz
|
||||
tar -xf bitcoin-0.16.0-x86_64-linux-gnu.tar.gz
|
||||
```
|
||||
|
||||
Data are stored in */data/btc*, in folders */data/btc/bitcoin* for Bitcoin Core data, */data/btc/blockbook* for Blockbook data.
|
||||
|
||||
Create configuration file */data/btc/bitcoin/bitcoin.conf* with content
|
||||
```
|
||||
daemon=1
|
||||
server=1
|
||||
rpcuser=rpc
|
||||
rpcpassword=rpc
|
||||
rpcport=8030
|
||||
txindex=1
|
||||
```
|
||||
Create script that starts the bitcoind daemon *run-btc-bitcoind.sh* with increased rpcworkqueue and configured zeromq
|
||||
```
|
||||
#!/bin/bash
|
||||
|
||||
bitcoin-0.15.1/bin/bitcoind -datadir=/data/btc/bitcoin -rpcworkqueue=32 -zmqpubhashtx=tcp://127.0.0.1:38330 -zmqpubhashblock=tcp://127.0.0.1:38330 -zmqpubrawblock=tcp://127.0.0.1:38330 -zmqpubrawtx=tcp://127.0.0.1:38330
|
||||
```
|
||||
Run the *run-btc-bitcoind.sh* to get initial import of data.
|
||||
|
||||
Create blockchain configuration file */data/testnet/blockbook/btc.json*
|
||||
```
|
||||
{
|
||||
"rpcURL": "http://127.0.0.1:8030",
|
||||
"rpcUser": "rpc",
|
||||
"rpcPass": "rpc",
|
||||
"rpcTimeout": 25,
|
||||
"parse": true,
|
||||
"zeroMQBinding": "tcp://127.0.0.1:38330"
|
||||
}
|
||||
```
|
||||
|
||||
Create script that runs blockbook *run-btc-blockbook.sh*
|
||||
```
|
||||
#!/bin/bash
|
||||
|
||||
cd go/src/blockbook
|
||||
./blockbook -coin=btc -blockchaincfg=/data/btc/blockbook/btc.json -datadir=/data/btc/blockbook/db -sync -internal=:9030 -public=:9130 -certfile=server/testcert -explorer=https://bitcore1.trezor.io/ $1
|
||||
```
|
||||
To run blockbook with logging to file (run with nohup or daemonize or using screen)
|
||||
```
|
||||
./run-btc-blockbook.sh 2>/data/btc/blockbook/blockbook.log
|
||||
```
|
||||
|
||||
|
|
@ -1,49 +0,0 @@
|
|||
## BTC Testnet Setup
|
||||
Get Bitcoin Core
|
||||
```
|
||||
wget https://bitcoin.org/bin/bitcoin-core-0.16.0/bitcoin-0.15.1-x86_64-linux-gnu.tar.gz
|
||||
tar -xf bitcoin-0.16.0-x86_64-linux-gnu.tar.gz
|
||||
```
|
||||
Data are stored in */data/testnet*, in folders */data/testnet/bitcoin* for Bitcoin Core data, */data/testnet/blockbook* for Blockbook data.
|
||||
|
||||
Create configuration file */data/testnet/bitcoin/bitcoin.conf* with content
|
||||
```
|
||||
testnet=1
|
||||
daemon=1
|
||||
server=1
|
||||
rpcuser=rpc
|
||||
rpcpassword=rpc
|
||||
rpcport=18030
|
||||
txindex=1
|
||||
```
|
||||
Create script that starts the bitcoind daemon *run-testnet-bitcoind.sh* with increased rpcworkqueue and configured zeromq
|
||||
```
|
||||
#!/bin/bash
|
||||
|
||||
bitcoin-0.15.1/bin/bitcoind -datadir=/data/testnet/bitcoin -rpcworkqueue=32 -zmqpubhashtx=tcp://127.0.0.1:48330 -zmqpubhashblock=tcp://127.0.0.1:48330 -zmqpubrawblock=tcp://127.0.0.1:48330 -zmqpubrawtx=tcp://127.0.0.1:48330
|
||||
```
|
||||
Run the *run-testnet-bitcoind.sh* to get initial import of data.
|
||||
|
||||
Create blockchain configuration file */data/testnet/blockbook/btc-testnet.json*
|
||||
```
|
||||
{
|
||||
"rpcURL": "http://127.0.0.1:18030",
|
||||
"rpcUser": "rpc",
|
||||
"rpcPass": "rpc",
|
||||
"rpcTimeout": 25,
|
||||
"parse": true,
|
||||
"zeroMQBinding": "tcp://127.0.0.1:48330"
|
||||
}
|
||||
```
|
||||
|
||||
Create script that runs blockbook *run-testnet-blockbook.sh*
|
||||
```
|
||||
#!/bin/bash
|
||||
|
||||
cd go/src/blockbook
|
||||
./blockbook -coin=btc-testnet -blockchaincfg=/data/testnet/blockbook/btc-testnet.json -datadir=/data/testnet/blockbook/db -sync -internal=:19030 -public=:19130 -certfile=server/testcert -explorer=https://testnet-bitcore1.trezor.io $1
|
||||
```
|
||||
To run blockbook with logging to file (run with nohup or daemonize or using screen)
|
||||
```
|
||||
./run-testnet-blockbook.sh 2>/data/testnet/blockbook/blockbook.log
|
||||
```
|
||||
|
|
@ -1,25 +0,0 @@
|
|||
## Ethereum Testnet Setup
|
||||
Get Ethereum
|
||||
```
|
||||
git clone https://github.com/ethereum/go-ethereum
|
||||
cd go-ethereum/
|
||||
make geth
|
||||
```
|
||||
Data are stored in */data/eth*, in folders */data/eth/eth* for Ethereum data, */data/eth/blockbook* for Blockbook data.
|
||||
|
||||
Run geth with rpc and websocket interfaces, bound to all ip addresses - insecure! (run with nohup or daemonize or using screen)
|
||||
```
|
||||
go-ethereum/build/bin/geth --syncmode "full" --cache 1024 --datadir /data/eth/eth --port "35555" --rpc --rpcport 8545 -rpcaddr 0.0.0.0 --rpccorsdomain "*" --ws --wsaddr 0.0.0.0 --wsport 8546 --wsorigins "*" 2>/data/eth/eth/eth.log
|
||||
```
|
||||
|
||||
Create script that runs blockbook *run-eth-blockbook.sh*
|
||||
```
|
||||
#!/bin/bash
|
||||
|
||||
cd go/src/blockbook
|
||||
./blockbook -coin=eth -blockchaincfg=/data/eth/blockbook/eth.json -datadir=/data/eth/blockbook/db -sync -internal=:8555 -public=:8556 -certfile=server/testcert $1
|
||||
```
|
||||
To run blockbook with logging to file (run with nohup or daemonize or using screen)
|
||||
```
|
||||
./run-eth-blockbook.sh 2>/data/eth/blockbook/blockbook.log
|
||||
```
|
||||
|
|
@ -1,25 +0,0 @@
|
|||
## Ethereum Testnet Setup
|
||||
Get Ethereum
|
||||
```
|
||||
git clone https://github.com/ethereum/go-ethereum
|
||||
cd go-ethereum/
|
||||
make geth
|
||||
```
|
||||
Data are stored in */data/eth-testnet*, in folders */data/eth-testnet/eth* for Ethereum data, */data/eth-testnet/eth/blockbook* for Blockbook data.
|
||||
|
||||
Run geth with rpc and websocket interfaces, bound to all ip addresses - insecure! (run with nohup or daemonize or using screen)
|
||||
```
|
||||
go-ethereum/build/bin/geth --testnet --datadir /data/eth-testnet/eth --rpc --rpcport 18545 -rpcaddr 0.0.0.0 --rpccorsdomain "*" --ws --wsaddr 0.0.0.0 --wsport 18546 --wsorigins "*" 2>/data/eth-testnet/eth/eth.log
|
||||
```
|
||||
|
||||
Create script that runs blockbook *run-eth-testnet-blockbook.sh*
|
||||
```
|
||||
#!/bin/bash
|
||||
|
||||
cd go/src/blockbook
|
||||
./blockbook -coin=eth-testnet -blockchaincfg=/data/eth-testnet/blockbook/eth-testnet.json -datadir=/data/eth-testnet/blockbook/db -sync -internal=:18555 -public=:18556 -certfile=server/testcert $1
|
||||
```
|
||||
To run blockbook with logging to file (run with nohup or daemonize or using screen)
|
||||
```
|
||||
./run-eth-testnet-blockbook.sh 2>/data/eth-testnet/blockbook/blockbook.log
|
||||
```
|
||||
|
|
@ -1,57 +0,0 @@
|
|||
## Zcash Setup
|
||||
Get Zcash client
|
||||
```
|
||||
wget https://z.cash/downloads/zcash-1.0.15-linux64.tar.gz
|
||||
tar xzf zcash-1.0.15-linux64.tar.gz
|
||||
```
|
||||
|
||||
Run command to download the parameters used to create and verify shielded transactions:
|
||||
```
|
||||
zcash-1.0.15/bin/zcash-fetch-params
|
||||
```
|
||||
|
||||
Data are stored in */data/zec* , in folders */data/zec/zcash* for Zcash client data, */data/zec/blockbook* for Blockbook data.
|
||||
|
||||
Create configuration file */data/zec/zcash/zcash.conf* with content
|
||||
```
|
||||
daemon=1
|
||||
server=1
|
||||
rpcuser=rpc
|
||||
rpcpassword=rpc
|
||||
rpcport=8032
|
||||
txindex=1
|
||||
mainnet=1
|
||||
addnode=mainnet.z.cash
|
||||
```
|
||||
|
||||
Create script *run-zec-zcashd.sh* that starts the zcashd daemon with increased rpcworkqueue and configured zeromq
|
||||
```
|
||||
#!/bin/bash
|
||||
|
||||
zcash-1.0.15/bin/zcashd -datadir=/data/zec/zcash -rpcworkqueue=32 -zmqpubhashblock=tcp://127.0.0.1:38332 -zmqpubrawblock=tcp://127.0.0.1:38332 -zmqpubhashtx=tcp://127.0.0.1:38332 -zmqpubrawtx=tcp://127.0.0.1:38332
|
||||
```
|
||||
|
||||
Run the *run-zec-zcashd.sh* to get initial import of data.
|
||||
|
||||
Create blockchain configuration file */data/zec/blockbook/zec.json*
|
||||
```
|
||||
{
|
||||
"rpcURL": "http://127.0.0.1:8032",
|
||||
"rpcUser": "rpc",
|
||||
"rpcPass": "rpc",
|
||||
"rpcTimeout": 25,
|
||||
"parse": true,
|
||||
"zeroMQBinding": "tcp://127.0.0.1:38332"
|
||||
}
|
||||
```
|
||||
|
||||
Create *run-zec-blockbook.sh* script that starts blockbook
|
||||
```
|
||||
#!/bin/bash
|
||||
./blockbook -coin=zec -blockchaincfg=/data/zec/blockbook/zec.json -datadir=/data/zec/blockbook/db -sync -internal=:9032 -public=:9132 -certfile=server/testcert -explorer=https://zec-bitcore1.trezor.io $1
|
||||
```
|
||||
|
||||
To run blockbook with logging to file (run with nohup or daemonize using screen)
|
||||
```
|
||||
./run-zec-blockbook.sh 2> /data/zec/blockbook/blockbook.log
|
||||
```
|
||||
|
|
@ -8,6 +8,8 @@
|
|||
"ports": {
|
||||
"backend_rpc": 8036,
|
||||
"backend_message_queue": 0,
|
||||
"backend_p2p": 38336,
|
||||
"backend_http": 8136,
|
||||
"blockbook_internal": 9036,
|
||||
"blockbook_public": 9136
|
||||
},
|
||||
|
|
|
|||
|
|
@ -0,0 +1,14 @@
|
|||
{
|
||||
"coin": {
|
||||
"name": "Ethereum Classic",
|
||||
"shortcut": "ETC",
|
||||
"label": "Ethereum Classic",
|
||||
"alias": "ethereum_classic"
|
||||
},
|
||||
"ports": {
|
||||
"backend_rpc": 8037,
|
||||
"backend_message_queue": 0,
|
||||
"blockbook_internal": 9037,
|
||||
"blockbook_public": 9137
|
||||
}
|
||||
}
|
||||
|
|
@ -8,6 +8,7 @@
|
|||
"ports": {
|
||||
"backend_rpc": 18036,
|
||||
"backend_message_queue": 0,
|
||||
"backend_p2p": 48336,
|
||||
"blockbook_internal": 19036,
|
||||
"blockbook_public": 19136
|
||||
},
|
||||
|
|
|
|||
|
|
@ -0,0 +1,263 @@
|
|||
//usr/bin/go run $0 $@ ; exit
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"io/ioutil"
|
||||
"math"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"sort"
|
||||
"strings"
|
||||
)
|
||||
|
||||
const (
|
||||
inputDir = "configs/coins"
|
||||
outputFile = "docs/ports.md"
|
||||
)
|
||||
|
||||
type PortInfo struct {
|
||||
CoinName string
|
||||
BlockbookInternalPort uint16
|
||||
BlockbookPublicPort uint16
|
||||
BackendRPCPort uint16
|
||||
BackendServicePorts map[string]uint16
|
||||
}
|
||||
|
||||
type PortInfoSlice []*PortInfo
|
||||
|
||||
type Config struct {
|
||||
Coin struct {
|
||||
Name string `json:"name"`
|
||||
}
|
||||
Ports map[string]uint16 `json:"ports"`
|
||||
}
|
||||
|
||||
func main() {
|
||||
output := "stdout"
|
||||
if len(os.Args) > 1 {
|
||||
if len(os.Args) == 2 && os.Args[1] == "-w" {
|
||||
output = outputFile
|
||||
} else {
|
||||
fmt.Fprintf(os.Stderr, "Usage: %s [-w]\n", filepath.Base(os.Args[0]))
|
||||
fmt.Fprintf(os.Stderr, " -w write output to %s instead of stdout\n", outputFile)
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
slice, err := loadPortInfo(inputDir)
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
|
||||
sortPortInfo(slice)
|
||||
|
||||
err = writeMarkdown(output, slice)
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
}
|
||||
|
||||
func loadPortInfo(dir string) (PortInfoSlice, error) {
|
||||
files, err := ioutil.ReadDir(dir)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
items := make(PortInfoSlice, 0, len(files))
|
||||
|
||||
for _, fi := range files {
|
||||
if fi.IsDir() || fi.Name()[0] == '.' {
|
||||
continue
|
||||
}
|
||||
|
||||
path := filepath.Join(dir, fi.Name())
|
||||
f, err := os.Open(path)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%s: %s", path, err)
|
||||
}
|
||||
defer f.Close()
|
||||
|
||||
v := Config{}
|
||||
d := json.NewDecoder(f)
|
||||
err = d.Decode(&v)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("%s: json: %s", path, err)
|
||||
}
|
||||
|
||||
item := &PortInfo{CoinName: v.Coin.Name, BackendServicePorts: map[string]uint16{}}
|
||||
for k, v := range v.Ports {
|
||||
if v == 0 {
|
||||
continue
|
||||
}
|
||||
|
||||
switch k {
|
||||
case "blockbook_internal":
|
||||
item.BlockbookInternalPort = v
|
||||
case "blockbook_public":
|
||||
item.BlockbookPublicPort = v
|
||||
case "backend_rpc":
|
||||
item.BackendRPCPort = v
|
||||
default:
|
||||
if len(k) > 8 && k[:8] == "backend_" {
|
||||
item.BackendServicePorts[k[8:]] = v
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
items = append(items, item)
|
||||
}
|
||||
|
||||
return items, nil
|
||||
}
|
||||
|
||||
func sortPortInfo(slice PortInfoSlice) {
|
||||
// normalizes values in order to sort zero values at the bottom of the slice
|
||||
normalize := func(a, b uint16) (uint16, uint16) {
|
||||
if a == 0 {
|
||||
a = math.MaxUint16
|
||||
}
|
||||
if b == 0 {
|
||||
b = math.MaxUint16
|
||||
}
|
||||
return a, b
|
||||
}
|
||||
|
||||
// sort values by BlockbookPublicPort, then by BackendRPCPort and finally by
|
||||
// CoinName; zero values are sorted at the bottom of the slice
|
||||
sort.Slice(slice, func(i, j int) bool {
|
||||
a, b := normalize(slice[i].BlockbookPublicPort, slice[j].BlockbookPublicPort)
|
||||
|
||||
if a < b {
|
||||
return true
|
||||
}
|
||||
if a > b {
|
||||
return false
|
||||
}
|
||||
|
||||
a, b = normalize(slice[i].BackendRPCPort, slice[j].BackendRPCPort)
|
||||
|
||||
if a < b {
|
||||
return true
|
||||
}
|
||||
if a > b {
|
||||
return false
|
||||
}
|
||||
|
||||
return strings.Compare(slice[i].CoinName, slice[j].CoinName) == -1
|
||||
})
|
||||
}
|
||||
|
||||
func writeMarkdown(output string, slice PortInfoSlice) error {
|
||||
var (
|
||||
buf bytes.Buffer
|
||||
err error
|
||||
)
|
||||
|
||||
fmt.Fprintf(&buf, "# Registry of ports\n\n")
|
||||
|
||||
header := []string{"coin", "blockbook internal port", "blockbook public port", "backend rpc port", "backend service ports (zmq)"}
|
||||
writeTable(&buf, header, slice)
|
||||
|
||||
fmt.Fprintf(&buf, "\n> NOTE: This document is generated from coin definitions in `configs/coins`.\n")
|
||||
|
||||
out := os.Stdout
|
||||
if output != "stdout" {
|
||||
out, err = os.OpenFile(output, os.O_CREATE|os.O_WRONLY, 0644)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer out.Close()
|
||||
}
|
||||
|
||||
n, err := out.Write(buf.Bytes())
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if n < len(buf.Bytes()) {
|
||||
return io.ErrShortWrite
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func writeTable(w io.Writer, header []string, slice PortInfoSlice) {
|
||||
rows := make([][]string, len(slice))
|
||||
for i, item := range slice {
|
||||
row := make([]string, len(header))
|
||||
row[0] = item.CoinName
|
||||
if item.BlockbookInternalPort > 0 {
|
||||
row[1] = fmt.Sprintf("%d", item.BlockbookInternalPort)
|
||||
}
|
||||
if item.BlockbookPublicPort > 0 {
|
||||
row[2] = fmt.Sprintf("%d", item.BlockbookPublicPort)
|
||||
}
|
||||
if item.BackendRPCPort > 0 {
|
||||
row[3] = fmt.Sprintf("%d", item.BackendRPCPort)
|
||||
}
|
||||
|
||||
svcPorts := make([]string, 0, len(item.BackendServicePorts))
|
||||
for k, v := range item.BackendServicePorts {
|
||||
var s string
|
||||
if k == "message_queue" {
|
||||
s = fmt.Sprintf("%d", v)
|
||||
} else {
|
||||
s = fmt.Sprintf("%d %s", v, k)
|
||||
}
|
||||
svcPorts = append(svcPorts, s)
|
||||
}
|
||||
|
||||
row[4] = strings.Join(svcPorts, ", ")
|
||||
|
||||
rows[i] = row
|
||||
}
|
||||
|
||||
padding := make([]int, len(header))
|
||||
for column := range header {
|
||||
padding[column] = len(header[column])
|
||||
|
||||
for _, row := range rows {
|
||||
padding[column] = maxInt(padding[column], len(row[column]))
|
||||
}
|
||||
}
|
||||
|
||||
content := make([][]string, 0, len(rows)+2)
|
||||
|
||||
content = append(content, paddedRow(header, padding))
|
||||
content = append(content, delim("-", padding))
|
||||
|
||||
for _, row := range rows {
|
||||
content = append(content, paddedRow(row, padding))
|
||||
}
|
||||
|
||||
for _, row := range content {
|
||||
fmt.Fprintf(w, "|%s|\n", strings.Join(row, "|"))
|
||||
}
|
||||
}
|
||||
|
||||
func maxInt(a, b int) int {
|
||||
if a > b {
|
||||
return a
|
||||
}
|
||||
return b
|
||||
}
|
||||
|
||||
func paddedRow(row []string, padding []int) []string {
|
||||
out := make([]string, len(row))
|
||||
for i := 0; i < len(row); i++ {
|
||||
format := fmt.Sprintf(" %%-%ds ", padding[i])
|
||||
out[i] = fmt.Sprintf(format, row[i])
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func delim(str string, padding []int) []string {
|
||||
out := make([]string, len(padding))
|
||||
for i := 0; i < len(padding); i++ {
|
||||
out[i] = strings.Repeat(str, padding[i]+2)
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
|
@ -0,0 +1,263 @@
|
|||
# Blockbook Build Guide
|
||||
|
||||
## Setting up your development environment
|
||||
|
||||
Supported environment to develop Blockbook is Linux. Although it is possible build and run Blockbook on macOS
|
||||
or Windows our build process is not prepared for it. But you can still build Blockbook [manually](#manual-build).
|
||||
|
||||
The only dependency required to build Blockbook is Docker. You can see how to install Docker [here](https://docs.docker.com/install/linux/docker-ce/debian/).
|
||||
Manual build require additional dependencies that are described in appropriate section.
|
||||
|
||||
## Build in Docker environment
|
||||
|
||||
All build operations run in Docker container in order to keep build environment isolated. Makefile in root of repository
|
||||
define few targets used for building, testing and packaging of Blockbook. With Docker image definitions and Debian
|
||||
package templates in *build/docker* and *build/templates* respectively, they are only inputs that make build process.
|
||||
|
||||
Docker build images are created at first execution of Makefile and that information is persisted. (Actually there are
|
||||
created two files in repository – .bin-image and .deb-image – that are used as tags.) Sometimes it is necessary to
|
||||
rebuild Docker images, it is possible by executing `make build-images`.
|
||||
|
||||
### Building binary
|
||||
|
||||
Just run `make` and that is it. Output binary is stored in *build* directory. Note that although Blockbook is Go application
|
||||
it is dynamically linked with RocksDB dependencies and ZeroMQ. Therefore operating system where Blockbook will be
|
||||
executed still need that dependencies installed. See [Manual build](#manual-build) instructions below or install
|
||||
Blockbook via Debian packages.
|
||||
|
||||
### Building debug binary
|
||||
|
||||
Standard binary contains no debug symbols. Execute `make build-debug` to get binary for debugging.
|
||||
|
||||
### Testing
|
||||
|
||||
How to execute tests is described in separate document [here](/docs/testing.md).
|
||||
|
||||
### Building Debian packages
|
||||
|
||||
Blockbook and particular coin back-end are usually deployed together. They are defined in same place as well.
|
||||
So typical way to build Debian packages is build Blockbook and back-end deb packages by single command. But it is not
|
||||
mandatory, of course.
|
||||
|
||||
> Early releases of Blockbook weren't so friendly for extending. One had to define back-end package, Blockbook package,
|
||||
> back-end configuration and Blockbook configuration as well. There were many options that were duplicated across
|
||||
> configuration files and therefore error prone.
|
||||
>
|
||||
> Actually all configuration options and also build options for both Blockbook and backend are defined in single JSON
|
||||
> file and all stuff required during build is generated dynamically.
|
||||
|
||||
Makefile targets follow simple pattern, there are few prefixes that define what to build.
|
||||
|
||||
* *deb-blockbook-<coin>* – Build Blockbook package for given coin.
|
||||
|
||||
* *deb-backend-<coin>* – Build back-end package for given coin.
|
||||
|
||||
* *deb-<coin>* – Build both Blockbook and back-end packages for given coin.
|
||||
|
||||
* *all-<coin>* – Similar to deb-<coin> but clean repository and rebuild Docker image before package build. It is useful
|
||||
for production deployment.
|
||||
|
||||
* *all* – Build both Blockbook and back-end packages for all coins.
|
||||
|
||||
Which coins are possible to build is defined in *configs/coins*. Particular coin has to have JSON config file there.
|
||||
|
||||
For example we want to build some packages for Bitcoin and Bitcoin Testnet.
|
||||
|
||||
```bash
|
||||
# make all-bitcoin deb-backend-bitcoin_testnet
|
||||
...
|
||||
# ls build/*.deb
|
||||
build/backend-bitcoin_0.16.1-satoshilabs-1_amd64.deb build/backend-bitcoin-testnet_0.16.1-satoshilabs-1_amd64.deb build/blockbook-bitcoin_0.0.6_amd64.deb
|
||||
```
|
||||
|
||||
We have built two backend packages – for Bitcoin and Testnet – and Blockbook package for Bitcoin. Before build have been
|
||||
performed there was cleaned build directory and rebuilt Docker image.
|
||||
|
||||
### Extra variables
|
||||
|
||||
There are few variables that can be passed to make in order to modify build process.
|
||||
|
||||
In general, build of Blockbook binary require some dependencies. They are downloaded automatically during build process
|
||||
but if you need to build binary repeatedly it consumes a lot of time. Here comes variable *UPDATE_VENDOR* that if is
|
||||
unset says that build process uses *vendor* (i.e. dependencies) from your local repository. For example:
|
||||
`make deb-bitcoin UPDATE_VENDOR=0`. But before the command is executed there must be *vendor* directory populated,
|
||||
you can do it by calling `dep ensure --vendor-only`. See [Manual build](#manual-build) instructions below.
|
||||
|
||||
All build targets allow pass additional parameters to underlying command inside container. It is possible via ARGS
|
||||
variable. For example if you want run only subset of unit-tests, you will perform it by calling:
|
||||
`make test ARGS='-run TestBitcoinRPC' UPDATE_VENDOR=0`
|
||||
|
||||
Common behaviour of Docker image build is that build steps are cached and next time they are executed much faster.
|
||||
Although this is a good idea, when something went wrong you will need to override this behaviour somehow. Execute this
|
||||
command: `make build-images NO_CACHE=true`.
|
||||
|
||||
### On naming conventions and versioning
|
||||
|
||||
**install and data directories**
|
||||
|
||||
Both Blockbook and back-end have separated install and data directories. They use common preffix and are defined in
|
||||
*configs/environ.json* and all templates use them.
|
||||
|
||||
Back-end install directory is */opt/coins/nodes/<coin>*.
|
||||
Back-end data directory is */opt/coins/data/<coin>/backend*.
|
||||
Blockbook install directory is */opt/coins/blockbook/<coin>*.
|
||||
Blockbook data directory is */opt/coins/data/<coin>/blockbook*.
|
||||
|
||||
*coin* used above is defined in *coin.alias* in coin definition file.
|
||||
|
||||
**package names**
|
||||
|
||||
Package names are defined in *backend.package_name* and *blockbook.package_name* in coin definition file. We use
|
||||
simple pattern *<prefix>-<coin>* to name packages where *prefix* is either *blockbook* or *backend* and
|
||||
*coin* is made similarly to *coin.alias*. We use convention that coin name uses lowercase characters and dash '-' as
|
||||
a word delimiter. Testnet versions of coins must have *-testnet* suffix. That differs from *coin.alias* because
|
||||
underscore has a special meaning in Debian packaging. For example there are packages *backend-bitcoin* and
|
||||
*blockbook-bitcoin-testnet*.
|
||||
|
||||
**user names**
|
||||
|
||||
User names are defined in *backend.system_user* and *blockbook.system_user* in coin definition file. We follow common
|
||||
Linux conventions, user names use lowercase characters and dash '-' as a word delimiter.
|
||||
|
||||
Back-end user name use coin name only, including testnet services. For example there is *bitcoin* user for both
|
||||
*backend-bitcoin* and *backend-bitcoin-testnet* packages.
|
||||
|
||||
Blockbook user name has *blockbook-* prefix and coin name (made same as back-end version). For example there is
|
||||
*blockbook-bitcoin* user for both *blockbook-bitcoin* and *blockbook-bitcoin-testnet* packages.
|
||||
|
||||
**back-end versioning**
|
||||
|
||||
Since we have to distinguish version of coin distribution and version of our configuration we follow standard Debian
|
||||
package versioning rules (for details see
|
||||
[Debian policy](https://www.debian.org/doc/debian-policy/ch-controlfields.html#version)). There is upstream version
|
||||
and revision both defined in coin definition file in *backend.version* and *backend.package_revision*, respectively.
|
||||
|
||||
**blockbook versioning**
|
||||
|
||||
Blockbook versioning is much simpler. There is only one version defined in *configs/environ.json*.
|
||||
|
||||
### On back-end building
|
||||
|
||||
Because we don't keep back-end archives inside out repository we download them during build process. Build steps
|
||||
are these: download, verify and extract archive, prepare distribution and make package.
|
||||
|
||||
All configuration keys described below are in coin definition file in *configs/coins*.
|
||||
|
||||
**download archive**
|
||||
|
||||
URL from where is archive downloaded is defined in *backend.binary_url*.
|
||||
|
||||
**verify archive**
|
||||
|
||||
There are three different approaches how is archive verification done. Some projects use PGP sign of archive, some
|
||||
have signed sha256 sums and some don't care about verification at all. So there is option *backend.verification_type* that
|
||||
could be *gpg*, *gpg-sha256* or *sha256* and chooses particular method.
|
||||
|
||||
*gpg* type require file with digital sign and maintainer's public key imported in Docker build image (see below). Sign
|
||||
file is downloaded from URL defined in *backend.verification_source*. Than is passed to gpg in order to verify archvie.
|
||||
|
||||
*gpg-sha256* type require signed checksum file and maintainer's public key imported in Docker build image (see below).
|
||||
Checksum file is downloaded from URL defined in *backend.verification_source*. Then is verified by gpg and passed to
|
||||
sha256sum in order to verify archive.
|
||||
|
||||
*sha256* type is used for coins that don't support verification at all. In *backend.verification_source* is defined
|
||||
hexadecimal string that is compared with output of sha256sum. Although this solution is not secure, it avoid download
|
||||
errors and other surprises at least.
|
||||
|
||||
*gpg* and *gpg-sha256* types require maintainer's public key imported in Docker build image. It is not expected that
|
||||
maintainer's key will change requently while sing or checksum files are changed every release, so it is ideal to
|
||||
store maintainer's key within image definition. Public keys are stored in *build/docker/deb/gpg-keys* directory. Docker
|
||||
image must be rebuilt by calling `make build-images`.
|
||||
|
||||
**extract archive**
|
||||
|
||||
Extraction command is defined in *backend.extract_command*. Content of archive must be extracted to `./backend` directory.
|
||||
See bitcoin.json and vertcoin.json for different approaches.
|
||||
|
||||
**prepare distribution**
|
||||
|
||||
There are two steps in this stage – exclude unnecessary files and generate configuration.
|
||||
|
||||
Some files are not required for server deployment, some binaries have unnecessary dependencies, so it is good idea to
|
||||
extract these files from output package. Files to extract are listed in *backend.exclude_files*. Note that paths are
|
||||
relative to *backend* directory where archive is extracted.
|
||||
|
||||
Configuration is described in [config.md](/docs/config.md).
|
||||
|
||||
## Manual build
|
||||
|
||||
Instructions below are focused on Debian 9 (Stretch). If you want to use another Linux distribution or operating system
|
||||
like macOS or Windows, please read instructions specific for each project.
|
||||
|
||||
Setup go environment:
|
||||
|
||||
```
|
||||
wget https://dl.google.com/go/go1.10.3.linux-amd64.tar.gz && tar xf go1.10.3.linux-amd64.tar.gz
|
||||
sudo mv go /opt/go
|
||||
sudo ln -s /opt/go/bin/go /usr/bin/go
|
||||
# see `go help gopath` for details
|
||||
mkdir $HOME/go
|
||||
export GOPATH=$HOME/go
|
||||
export PATH=$PATH:$GOPATH/bin
|
||||
```
|
||||
|
||||
Install RocksDB: https://github.com/facebook/rocksdb/blob/master/INSTALL.md
|
||||
and compile the static_lib and tools
|
||||
|
||||
```
|
||||
sudo apt-get update && sudo apt-get install -y \
|
||||
build-essential git wget pkg-config libzmq3-dev libgflags-dev libsnappy-dev zlib1g-dev libbz2-dev liblz4-dev
|
||||
git clone https://github.com/facebook/rocksdb.git
|
||||
cd rocksdb
|
||||
CFLAGS=-fPIC CXXFLAGS=-fPIC make release
|
||||
```
|
||||
|
||||
Setup variables for gorocksdb: https://github.com/tecbot/gorocksdb
|
||||
|
||||
```
|
||||
export CGO_CFLAGS="-I/path/to/rocksdb/include"
|
||||
export CGO_LDFLAGS="-L/path/to/rocksdb -lrocksdb -lstdc++ -lm -lz -lbz2 -lsnappy -llz4"
|
||||
```
|
||||
|
||||
Install ZeroMQ: https://github.com/zeromq/libzmq
|
||||
|
||||
Install go-dep tool:
|
||||
```
|
||||
go get github.com/golang/dep/cmd/dep
|
||||
```
|
||||
|
||||
Get blockbook sources, install dependencies, build:
|
||||
|
||||
```
|
||||
cd $GOPATH/src
|
||||
git clone https://github.com/trezor/blockbook.git
|
||||
cd blockbook
|
||||
dep ensure -vendor-only
|
||||
go build
|
||||
```
|
||||
|
||||
### Example command
|
||||
|
||||
Blockbook require full node daemon as its back-end. You are responsible for proper installation. Port numbers and
|
||||
daemon configuration are defined in *configs/coins* and *build/templates/backend/config* directories. You should use
|
||||
specific installation process for particular coin you want run (e.g. https://bitcoin.org/en/full-node#other-linux-distributions for Bitcoin).
|
||||
|
||||
When you have running back-end daemon you can start Blockbook. It is highly recomended use ports described in [ports.md](/docs/ports.md)
|
||||
for both Blockbook and back-end daemon. You can use *contrib/scripts/build-blockchaincfg.sh* that will generate
|
||||
Blockbook's blockchain configuration from our coin definition files.
|
||||
|
||||
Example for Bitcoin:
|
||||
```
|
||||
contrib/scripts/build-blockchaincfg.sh
|
||||
./blockbook -sync -blockchaincfg=build/blockchaincfg.json -internal=:9030 -public=:9130 -certfile=server/testcert -logtostderr
|
||||
```
|
||||
|
||||
This command starts Blockbook with parallel synchronization and providing HTTP and Socket.IO interface, with database
|
||||
in local directory *data* and established ZeroMQ and RPC connections to back-end daemon specified in configuration
|
||||
file passed to *-blockchaincfg* option.
|
||||
|
||||
Blockbook logs to stderr (option *-logtostderr*) or to directory specified by parameter *-log_dir* . Verbosity of logs can be tuned
|
||||
by command line parameters *-v* and *-vmodule*, for details see https://godoc.org/github.com/golang/glog.
|
||||
|
||||
You can check that Blockbook is running by simple HTTP request: `curl https://localhost:9130`. Returned data is JSON with some
|
||||
run-time information. If port is closed, Blockbook is syncing data.
|
||||
|
|
@ -0,0 +1,97 @@
|
|||
# Configuration
|
||||
|
||||
Coin definitions are stored in JSON files in *configs/coins* directory. They are single source of Blockbook
|
||||
configuration, Blockbook and back-end package definition and build metadata. Since Blockbook supports only single
|
||||
coin index per running instance, every coin (including testnet) must have single definition file.
|
||||
|
||||
Because most of coins are fork of Bitcoin and they have similar way to install and configure their daemon, we use
|
||||
templates to generate package definition and configuration files during build process. It is similar to build Blockbook
|
||||
package too. Templates are filled with data from coin definition. Although build process generate packages
|
||||
automatically, there is sometimes necessary see intermediate step. You can generate all files by calling
|
||||
`go run build/templates/generate.go coin` where *coin* is name of definition file without .json extension. Files are
|
||||
generated to *build/pkg-defs* directory.
|
||||
|
||||
Good examples of coin configuration are
|
||||
[*configs/coins/bitcoin.json*](configs/coins/bitcoin.json) and
|
||||
[*configs/coins/ethereum.json*](configs/coins/ethereum.json) for Bitcoin-like coins and different coins, respectively.
|
||||
|
||||
## Description of coin definition
|
||||
|
||||
* `coin` – Base information about coin.
|
||||
* `name` – Name of coin used internally (e.g. "Bcash Testnet").
|
||||
* `shortcut` – Ticker symbol (code) of coin (e.g. "TBCH").
|
||||
* `label` – Name of coin used publicly (e.g. "Bitcoin Cash Testnet").
|
||||
* `alias` – Name of coin used in file system paths and config files. We use convention that name uses lowercase
|
||||
characters and underscore '_' as a word delimiter. Testnet versions of coins must have *_testnet*
|
||||
suffix. For example "bcash_testnet".
|
||||
|
||||
* `ports` – List of ports used by both back-end and Blockbook. Ports defined here are used in configuration templates
|
||||
and also as source for generated documentation.
|
||||
* `backend_rpc` – Port of back-end RPC that is connected by Blockbook service.
|
||||
* `backend_message_queue` – Port of back-end MQ (if used) that is connected by Blockbook service.
|
||||
* `backend_*` – Additional back-end ports can be documented here. Actually the only purpose is to get them to
|
||||
port table (prefix is removed and rest of string is used as note).
|
||||
* `blockbook_internal` – Blockbook's internal port that is used for metric collecting, debugging etc.
|
||||
* `blockbook_public` – Blockbook's public port that is used to comunicate with Trezor wallet (via Socket.IO).
|
||||
|
||||
* `ipc` – Defines how Blockbook connects its back-end service.
|
||||
* `rpc_url_template` – Template that defines URL of back-end RPC service. See note on templates below.
|
||||
* `rpc_user` – User name of back-end RPC service, used by both Blockbook and back-end configuration templates.
|
||||
* `rpc_pass` – Password of back-end RPC service, used by both Blockbook and back-end configuration templates.
|
||||
* `rpc_timeout` – RPC timeout used by Blockbook.
|
||||
* `message_queue_binding_template` – Template that defines URL of back-end's message queue (ZMQ), used by both
|
||||
Blockbook and back-end configuration template. See note on templates below.
|
||||
|
||||
* `backend` – Definition of back-end package, configuration and service.
|
||||
* `package_name` – Name of package. See convention note in [build guide](/docs/build.md#on-naming-conventions-and-versioning).
|
||||
* `package_revision` – Revision of package. See convention note in [build guide](/docs/build.md#on-naming-conventions-and-versioning).
|
||||
* `system_user` – User used to run back-end service. See convention note in [build guide](/docs/build.md#on-naming-conventions-and-versioning).
|
||||
* `version` – Upstream version. See convention note in [build guide](/docs/build.md#on-naming-conventions-and-versioning).
|
||||
* `binary_url` – URL of back-end archive.
|
||||
* `verification_type` – Type of back-end archive verification. Possible values are *gpg*, *gpg-sha256*, *sha256*.
|
||||
* `verification_source` – Source of sign/checksum of back-end archive.
|
||||
* `extract_command` – Command to extract back-end archive. It is required to extract content of archive to
|
||||
*backend* directory.
|
||||
* `exclude_files` – List of files from back-end archive to exclude. Some files are not required for server
|
||||
deployment, some binaries have unnecessary dependencies, so it is good idea to extract these files from output
|
||||
package. Note that paths are relative to *backend* directory where archive is extracted.
|
||||
* `exec_command_template` – Template of command to execute back-end node daemon. Every back-end node daemon has its
|
||||
service that is managed by systemd. Template is evaluated to *ExecStart* option in *Service* section of
|
||||
service unit. See note on templates below.
|
||||
* `logrotate_files_template` – Template that define log files rotated by logrotate daemon. See note on templates
|
||||
below.
|
||||
* `postinst_script_template` – Additional steps in postinst script. See [ZCash definition](configs/coins/zcash.json)
|
||||
for more information.
|
||||
* `service_type` – Type of service. Services that daemonize must have *forking* type and write their PID to
|
||||
*PIDFile*. Services that don't support daemonization must have *simple* type. See examples above.
|
||||
* `service_additional_params_template` – Additional parameters in service unit. See
|
||||
[ZCash definition](configs/coins/zcash.json) for more information.
|
||||
* `protect_memory` – Enables *MemoryDenyWriteExecute* option in service unit if *true*.
|
||||
* `mainnet` – Set *false* for testnet back-end.
|
||||
* `config_file` – Name of template of back-end configuration file. Templates are defined in *build/backend/config*.
|
||||
For Bitcoin-like coins it is not necessary to add extra template, most options can be added via
|
||||
*additional_params*. For coins that don't require configuration option should be empty (e.g. Ethereum).
|
||||
* `additional_params` – Object of extra parameters that are added to back-end configuration file as key=value pairs.
|
||||
Exception is *addnode* key that contains list of nodes that is expanded as addnode=item lines.
|
||||
|
||||
* `blockbook` – Definition of Blockbook package, configuration and service.
|
||||
* `package_name` – Name of package. See convention note in [build guide](/docs/build.md#on-naming-conventions-and-versioning).
|
||||
* `system_user` – User used to run Blockbook service. See convention note in [build guide](/docs/build.md#on-naming-conventions-and-versioning).
|
||||
* `internal_binding_template` – Template for *-internal* parameter. See note on templates below.
|
||||
* `public_binding_template` – Template for *-public* parameter. See note on templates below.
|
||||
* `explorer_url` – URL of blockchain explorer.
|
||||
* `additional_params` – Additional params of exec command (see [Dogecoin definition](configs/coins/dogecoin.json)).
|
||||
* `block_chain` – Configuration of BlockChain type that ensures communication with back-end service. All options
|
||||
must be tweaked for each individual coin separely.
|
||||
* `parse` – Use binary parser for block decoding if *true* else call verbose back-end RPC method that returns
|
||||
JSON. Note that verbose method is slow and not every coin support it. However there are coin implementations
|
||||
that don't support binary parsing (e.g. ZCash).
|
||||
* `mempool_workers` – Number of workers for UTXO mempool.
|
||||
* `mempool_sub_workers` – Number of subworkers for UTXO mempool.
|
||||
* `block_addresses_to_keep` – Number of blocks that are to be kept in blockaddresses column.
|
||||
* `additional_params` – Object of coin-specific params.
|
||||
|
||||
* `meta` – Common package metadata.
|
||||
* `package_maintainer` – Full name of package maintainer.
|
||||
* `package_maintainer_email` – E-mail of package maintainer.
|
||||
|
||||
|
|
@ -0,0 +1,24 @@
|
|||
# Registry of ports
|
||||
|
||||
| coin | blockbook internal port | blockbook public port | backend rpc port | backend service ports (zmq) |
|
||||
|--------------------------|-------------------------|-----------------------|------------------|-----------------------------|
|
||||
| Bitcoin | 9030 | 9130 | 8030 | 38330 |
|
||||
| Bcash | 9031 | 9131 | 8031 | 38331 |
|
||||
| Zcash | 9032 | 9132 | 8032 | 38332 |
|
||||
| Dash | 9033 | 9133 | 8033 | 38333 |
|
||||
| Litecoin | 9034 | 9134 | 8034 | 38334 |
|
||||
| Bgold | 9035 | 9135 | 8035 | 38335 |
|
||||
| Ethereum | 9036 | 9136 | 8036 | 38336 p2p, 8136 http |
|
||||
| Ethereum Classic | 9037 | 9137 | 8037 | |
|
||||
| Dogecoin | 9038 | 9138 | 8038 | 38338 |
|
||||
| Namecoin | 9039 | 9139 | 8039 | 38339 |
|
||||
| Vertcoin | 9040 | 9140 | 8040 | 38340 |
|
||||
| Testnet | 19030 | 19130 | 18030 | 48330 |
|
||||
| Bcash Testnet | 19031 | 19131 | 18031 | 48331 |
|
||||
| Zcash Testnet | 19032 | 19132 | 18032 | 48332 |
|
||||
| Dash Testnet | 19033 | 19133 | 18033 | 48333 |
|
||||
| Litecoin Testnet | 19034 | 19134 | 18034 | 48334 |
|
||||
| Ethereum Testnet Ropsten | 19036 | 19136 | 18036 | 48336 p2p |
|
||||
| Vertcoin Testnet | 19040 | 19140 | 18040 | 48340 |
|
||||
|
||||
> NOTE: This document is generated from coin definitions in `configs/coins`.
|
||||
|
|
@ -0,0 +1,44 @@
|
|||
# Data storage in RocksDB
|
||||
|
||||
Blockbook stores data the key-value store RocksDB. Data are stored in binary form to save space.
|
||||
The data are separated to different column families:
|
||||
|
||||
- **default**
|
||||
|
||||
at the moment not used, will store statistical data etc.
|
||||
|
||||
- **height** - maps *block height* to *block hash*
|
||||
|
||||
*Block heigh* stored as array of 4 bytes (big endian uint32)
|
||||
*Block hash* stored as array of 32 bytes
|
||||
|
||||
Example - the first four blocks (all data hex encoded)
|
||||
```
|
||||
0x00000000 : 0x000000000933ea01ad0ee984209779baaec3ced90fa3f408719526f8d77f4943
|
||||
0x00000001 : 0x00000000b873e79784647a6c82962c70d228557d24a747ea4d1b8bbe878e1206
|
||||
0x00000002 : 0x000000006c02c8ea6e4ff69651f7fcde348fb9d557a06e6957b65552002a7820
|
||||
0x00000003 : 0x000000008b896e272758da5297bcd98fdc6d97c9b765ecec401e286dc1fdbe10
|
||||
```
|
||||
|
||||
- **outputs** - maps *output script+block height* to *array of outpoints*
|
||||
|
||||
*Output script (ScriptPubKey)+block height* stored as variable length array of bytes for output script + 4 bytes (big endian uint32) block height
|
||||
*array of outpoints* stored as array of 32 bytes for transaction id + variable length outpoint index for each outpoint
|
||||
|
||||
Example - (all data hex encoded)
|
||||
```
|
||||
0x001400efeb484a24a1c1240eafacef8566e734da429c000e2df6 : 0x1697966cbd76c75eb9fc736dfa3ba0bc045999bab1e8b10082bc0ba546b0178302
|
||||
0xa9143e3d6abe282d92a28cb791697ba001d733cefdc7870012c4b1 : 0x7246e79f97b5f82e7f51e291d533964028ec90be0634af8a8ef7d5a903c7f6d301
|
||||
```
|
||||
|
||||
- **inputs** - maps *transaction outpoint* to *input transaction* that spends it
|
||||
|
||||
*Transaction outpoint* stored as array of 32 bytes for transaction id + variable length outpoint index
|
||||
*Input transaction* stored as array of 32 bytes for transaction id + variable length input index
|
||||
|
||||
Example - (all data hex encoded)
|
||||
```
|
||||
0x7246e79f97b5f82e7f51e291d533964028ec90be0634af8a8ef7d5a903c7f6d300 : 0x0a7aa90ea0269c79f844c516805e4cac594adb8830e56fca894b66aab19136a428
|
||||
0x7246e79f97b5f82e7f51e291d533964028ec90be0634af8a8ef7d5a903c7f6d301 : 0x4303a9fcfe6026b4d33ba488df6443c9a99bca7b7fcb7c6f6cd65cea24a749b700
|
||||
```
|
||||
|
||||
Loading…
Reference in New Issue