2018-08-01 07:49:06 -06:00
|
|
|
|
# Blockbook Contributor Guide
|
|
|
|
|
|
|
|
|
|
Blockbook is back-end service for Trezor wallet. Although it is open source, design and development of core packages
|
2018-08-29 02:12:54 -06:00
|
|
|
|
is done by Trezor developers in order to keep Blockbook compatible with Trezor.
|
2018-08-01 07:49:06 -06:00
|
|
|
|
|
2018-08-29 02:12:54 -06:00
|
|
|
|
Bug fixes and support for new coins are welcome. Please take note that non-fixing pull requests that change base
|
|
|
|
|
packages or another coin code will not be accepted. If you will have to change some of existing code, please file
|
|
|
|
|
an issue and discuss your request with Blockbook maintainers.
|
2018-08-01 07:49:06 -06:00
|
|
|
|
|
|
|
|
|
## Development environment
|
|
|
|
|
|
|
|
|
|
Instructions to set up your development environment and build Blockbook are described in separated
|
|
|
|
|
[document](/docs/build.md).
|
|
|
|
|
|
|
|
|
|
## How can I contribute?
|
|
|
|
|
|
|
|
|
|
### Reporting bugs
|
|
|
|
|
|
2018-08-29 08:06:31 -06:00
|
|
|
|
A great way to contribute to the project is to send a detailed report when you encounter an issue. We always appreciate
|
|
|
|
|
a well-written, thorough bug report, and will thank you for it!
|
2018-08-29 02:12:54 -06:00
|
|
|
|
|
2018-08-29 08:06:31 -06:00
|
|
|
|
Check that [our issue database](https://github.com/trezor/blockbook/issues) doesn't already include that problem or
|
|
|
|
|
suggestion before submitting an issue. If you find a match, you can use the "subscribe" button to get notified on
|
|
|
|
|
updates. Do not leave random "+1" or "I have this too" comments, as they only clutter the discussion, and don't help
|
|
|
|
|
resolving it. However, if you have ways to reproduce the issue or have additional information that may help resolving
|
|
|
|
|
the issue, please leave a comment.
|
|
|
|
|
|
|
|
|
|
Include information about Blockbook instance which is exposed at internal HTTP port. Ports are listed in
|
|
|
|
|
[port registry](/docs/ports.md). For example execute `curl -k https://localhost:9030` for Bitcoin.
|
|
|
|
|
|
|
|
|
|
Also include the steps required to reproduce the problem if possible and applicable. This information will help us
|
|
|
|
|
review and fix your issue faster. When sending lengthy log-files, consider posting them as a gist
|
|
|
|
|
(https://gist.github.com).
|
2018-08-29 02:12:54 -06:00
|
|
|
|
|
2018-08-01 07:49:06 -06:00
|
|
|
|
### Adding coin support
|
|
|
|
|
|
|
|
|
|
Trezor harware wallet supports over 500 coins, see https://trezor.io/coins/. You are free to add support for any of
|
|
|
|
|
them to Blockbook. Actually implemented coins are listed [here](/docs/ports.md).
|
|
|
|
|
|
|
|
|
|
You should follow few steps bellow to get smooth merge of your PR.
|
|
|
|
|
|
2018-08-29 01:47:56 -06:00
|
|
|
|
> Although we are happy for support of new coins we have not enough capacity to run them all on our infrastructure.
|
2018-08-01 07:49:06 -06:00
|
|
|
|
> Actually we can run Blockbook instances only for coins supported by Trezor wallet. If you want to have Blockbook
|
|
|
|
|
> instance for your coin, you will have to deploy your own server.
|
|
|
|
|
|
|
|
|
|
#### Add coin definition
|
|
|
|
|
|
|
|
|
|
Coin definitions are stored in JSON files in *configs/coins* directory. They are single source of Blockbook
|
|
|
|
|
configuration, Blockbook and back-end package definition and build metadata. Since Blockbook supports only single
|
|
|
|
|
coin index per running instance, every coin (including testnet) must have single definition file.
|
|
|
|
|
|
|
|
|
|
All options of coin definition are described in [config.md](/docs/config.md).
|
|
|
|
|
|
|
|
|
|
Because most of coins are fork of Bitcoin and they have similar way to install and configure their daemon, we use
|
2018-10-10 06:52:15 -06:00
|
|
|
|
templates to generate package definition and configuration files during build process. Similarly, there templates for Blockbook
|
|
|
|
|
package. Templates are filled with data from coin definition. Although normally all package definitions are generated automatically
|
|
|
|
|
during the build process, sometimes there is a reason to see them. You can create them by calling
|
|
|
|
|
`go run build/templates/generate.go coin`, where *coin* is name of definition file without .json extension. Files are
|
2018-08-01 07:49:06 -06:00
|
|
|
|
generated to *build/pkg-defs* directory.
|
|
|
|
|
|
|
|
|
|
Good examples of coin configuration are
|
|
|
|
|
[*configs/coins/bitcoin.json*](configs/coins/bitcoin.json) and
|
|
|
|
|
[*configs/coins/ethereum.json*](configs/coins/ethereum.json) for Bitcoin-like coins and different coins, respectively.
|
|
|
|
|
|
2018-10-10 06:52:15 -06:00
|
|
|
|
Usually you have to update only few options that differ from Bitcoin definition. At first there is base information
|
2018-08-01 07:49:06 -06:00
|
|
|
|
about coin in section *coin* – name, alias etc. Then update port information in *port* section. We keep port series as
|
2018-10-10 06:52:15 -06:00
|
|
|
|
listed in [the port registry](/docs/ports.md). Select next port numbers in the series. Port numbers must be unique across all
|
2018-08-01 07:49:06 -06:00
|
|
|
|
port definitions.
|
|
|
|
|
|
2018-10-10 06:52:15 -06:00
|
|
|
|
In the section *backend* update information how to build and configure back-end service. When back-end package is built,
|
|
|
|
|
build process downloads installation archive, verifies and extracts it. How it is done is described in
|
2018-08-01 07:49:06 -06:00
|
|
|
|
[build guide](/docs/build.md#on-back-end-building). Naming conventions and versioning are described
|
|
|
|
|
also in [build guide](/docs/build.md#on-naming-conventions-and-versioning). You have to update *package_name*,
|
2018-08-29 02:12:54 -06:00
|
|
|
|
*package_revision*, *system_user*, *version*, *binary_url*, *verification_type*, *verification_source*,
|
|
|
|
|
*extract_command* and *exclude_files*. Also update information whether service runs mainnet or testnet network in
|
|
|
|
|
*mainnet* option.
|
2018-08-01 07:49:06 -06:00
|
|
|
|
|
2018-10-10 06:52:15 -06:00
|
|
|
|
In the section *blockbook* update information how to build and configure Blockbook service. Usually they are only
|
2018-08-01 07:49:06 -06:00
|
|
|
|
*package_name*, *system_user* and *explorer_url* options. Naming conventions are are described
|
|
|
|
|
[here](/docs/build.md#on-naming-conventions-and-versioning).
|
|
|
|
|
|
2018-10-10 06:52:15 -06:00
|
|
|
|
Update *package_maintainer* and *package_maintainer_email* options in the section *meta*.
|
2018-08-01 07:49:06 -06:00
|
|
|
|
|
2018-10-10 01:37:14 -06:00
|
|
|
|
Execute script *go run contrib/scripts/check-and-generate-port-registry.go -w* that checks mandatory ports and
|
2018-09-03 05:25:50 -06:00
|
|
|
|
uniqueness of ports and updates registry of ports *docs/ports.md*.
|
2018-08-01 07:49:06 -06:00
|
|
|
|
|
2018-10-10 06:52:15 -06:00
|
|
|
|
Now you can try to generate package definitions as described above in order to check outputs.
|
2018-08-01 07:49:06 -06:00
|
|
|
|
|
|
|
|
|
#### Add coin implementation
|
|
|
|
|
|
2018-08-16 08:53:27 -06:00
|
|
|
|
Coin implementation is stored in *bchain/coins* directory. Each coin must implement interfaces *BlockChain* and
|
|
|
|
|
*BlockChainParser* (both defined in [bchain/types.go][/bchain/types.go]) and has registered factory function by
|
|
|
|
|
*init()* function of package *blockbook/bchain/coins* ([bchain/coins/blockchain.go](/bchain/coins/blockchain.go)).
|
|
|
|
|
|
|
|
|
|
There are several approaches how to implement coin support in Blockbook, please see examples below.
|
|
|
|
|
|
2018-10-10 06:52:15 -06:00
|
|
|
|
Bitcoin package *blockbook/bchain/coins/btc* is a reference implementation for Bitcoin-like coins. Most of the functionality
|
|
|
|
|
is usually the same so particular coin should embed it and override just different parts.
|
2018-08-16 08:53:27 -06:00
|
|
|
|
|
|
|
|
|
Bitcoin uses binary WIRE protocol thus decoding is very fast but require complex parser. Parser translate whole
|
2018-08-29 01:47:56 -06:00
|
|
|
|
pubkey-script to database ID and therefore it is usually possible store transactions without change.
|
2018-08-16 08:53:27 -06:00
|
|
|
|
|
2018-08-29 01:47:56 -06:00
|
|
|
|
ZCash package *blockbook/bchain/coins/zec* on the other side uses JSON version of RPCs therefore it doesn't require
|
2018-10-10 06:52:15 -06:00
|
|
|
|
specialized parser. Only responsibility that parser has is to translate address to Address Descriptor (used as
|
|
|
|
|
address ID in the database) and vice versa.
|
2018-08-16 08:53:27 -06:00
|
|
|
|
|
2018-10-10 06:52:15 -06:00
|
|
|
|
Ethereum package *blockbook/bchain/coins/eth* has own stand alone implementation because Ethereum uses totally
|
2018-08-16 08:53:27 -06:00
|
|
|
|
different concept than Bitcoin.
|
|
|
|
|
|
|
|
|
|
##### BlockChain interface
|
|
|
|
|
|
|
|
|
|
Type that implements *bchain.BlockChain* interface ensures communication with block chain network. Because
|
|
|
|
|
it calls node RPCs it usually has suffix RPC.
|
|
|
|
|
|
|
|
|
|
Initialization of object is separated into two stages. At first there is called factory method (details described
|
|
|
|
|
in next section) and then *bchain.BlockChain.Initialize()* method. Separated initialization method allows you call
|
|
|
|
|
inherited methods during initialization. However it is common practice override fields of embedded structure in factory
|
|
|
|
|
method.
|
|
|
|
|
|
|
|
|
|
During initialization, there is usually loaded chain information, registered message queue callback and created mempool
|
|
|
|
|
and parser objects.
|
|
|
|
|
|
|
|
|
|
BitcoinRPC uses *btc.RPCMarshaller* ([btc/codec.go](/bchain/coins/btc/codec.go)) in order to distinguish API version of
|
|
|
|
|
Bitcoin RPC. Current API (*btc.JSONMarshalerV2*) uses JSON object with method arguments. Older API (*btc.JSONMarshalerV1*)
|
|
|
|
|
uses JSON array with method arguments and some arguments are defined differently (e.g. bool vs int).
|
|
|
|
|
|
|
|
|
|
For example see [zec/zcashrpc.go](/bchain/coins/zec/zcashrpc.go).
|
|
|
|
|
|
|
|
|
|
##### BlockChain factory function
|
|
|
|
|
|
|
|
|
|
Factory function must be *coins.blockChainFactory* type ([coins/blockchain.go](/bchain/coins/blockchain.go)). It gets
|
|
|
|
|
configuration as JSON object and handler function for PUSH notifications. All factory functions have registered by
|
|
|
|
|
*init()* function of package *blockbook/bchain/coins* ([coins/blockchain.go](/bchain/coins/blockchain.go)). Coin name
|
|
|
|
|
must correspond to *coin.name* in coin definition file (see above).
|
|
|
|
|
|
|
|
|
|
Configuration passed to factory method is coin specific. For types that embed *btc.BitcoinRPC,* configuration must
|
|
|
|
|
contain at least fields referred in *btc.Configuration* ([btc/bitcoinrpc.go](/bchain/coins/btc/bitcoinrpc.go)).
|
|
|
|
|
|
|
|
|
|
For types that embed base struct it is common practise call factory method of embedded type in order to
|
|
|
|
|
create & initialize it. It is much more robust than simple struct composition.
|
|
|
|
|
|
|
|
|
|
For example see [zec/zcashrpc.go](/bchain/coins/zec/zcashrpc.go).
|
|
|
|
|
|
|
|
|
|
##### BlockChainParser interface
|
|
|
|
|
|
|
|
|
|
Type that implements *bchain.BlockChainParser* interface ensures parsing and conversions of block chain data. It is
|
|
|
|
|
initialized by *bchain.BlockChain* during initialization.
|
|
|
|
|
|
|
|
|
|
There are several groups of methods defined in *bchian.BlockChainParser*:
|
|
|
|
|
|
2018-10-10 06:52:15 -06:00
|
|
|
|
* *GetAddrDescFromVout* and *GetAddrDescFromAddress* – Convert transaction addresses to *Address Descriptor* that is used as database ID.
|
|
|
|
|
Most of coins use output script as *Address Descriptor*.
|
|
|
|
|
* *GetAddressesFromAddrDesc* and *GetScriptFromAddrDesc* – Convert *Address Descriptor* to addresses and output script. Note that
|
|
|
|
|
*btc.BitcoinParser* uses pointer to function *OutputScriptToAddressesFunc* that is called from *GetAddressesFromAddrDesc*
|
2018-08-29 01:47:56 -06:00
|
|
|
|
method in order to rewrite implementation by types embedding it.
|
2018-08-16 08:53:27 -06:00
|
|
|
|
* *PackTxid* and *UnpackTxid* – Packs txid to store in database and vice versa.
|
2018-08-29 08:06:31 -06:00
|
|
|
|
* *ParseTx* and *ParseTxFromJson* – Parse transaction from binary data or JSON and return *bchain.Tx*.
|
|
|
|
|
* *PackTx* and *UnpackTx* – Pack transaction to binary data to store in database and vice versa.
|
2018-08-16 08:53:27 -06:00
|
|
|
|
* *ParseBlock* – Parse block from binary data and return *bchain.Block*.
|
|
|
|
|
|
2018-10-10 06:52:15 -06:00
|
|
|
|
Base type of parsers is *bchain.BaseParser*. It implements method *ParseTxFromJson* that should be the same for all
|
2018-08-16 08:53:27 -06:00
|
|
|
|
Bitcoin-like coins. Also implements *PackTx* and *UnpackTx* that pack and unpack transactions using protobuf. Note
|
2018-10-10 06:52:15 -06:00
|
|
|
|
that Bitcoin stores transactions in more compact binary format.
|
2018-08-16 08:53:27 -06:00
|
|
|
|
|
|
|
|
|
*bchain.BaseParser* stores pointer to function *bchain.AddressFactoryFunc* that is responsible for making human readable
|
2018-10-10 06:52:15 -06:00
|
|
|
|
address representation. See [*bch.bcashparser*](/bchain/coins/bch/bcashparser.go) for example of implementation that uses
|
2018-08-16 08:53:27 -06:00
|
|
|
|
different approach for address representation than Bitcoin.
|
|
|
|
|
|
2018-08-01 07:49:06 -06:00
|
|
|
|
#### Add tests
|
|
|
|
|
|
2018-10-10 06:52:15 -06:00
|
|
|
|
Add unit tests and integration tests. PR without passing tests won't be accepted.
|
|
|
|
|
Tests are described [here](/docs/testing.md).
|
2018-08-18 07:53:33 -06:00
|
|
|
|
|
2018-08-01 07:49:06 -06:00
|
|
|
|
#### Deploy public server
|
2018-08-18 07:53:33 -06:00
|
|
|
|
|
2018-08-29 01:47:56 -06:00
|
|
|
|
Deploy Blockbook server on public IP address. Blockbook maintainers will check implementation before merging.
|