blockbook/docs/rocksdb.md

Data storage in RocksDB

Blockbook stores data the key-value store RocksDB. Each index is stored in its own column family.

Database content is described in golang pseudo types in the form (name type).

Operators used in the description:

• -> mapping from key to value.
• + concatenation,
• [] array

Types used in the description:

• []byte - array of bytes
• uint32 - unsigned integer, stored as array of 4 bytes in big endian*
• vint, vuint - variable length signed/unsigned int
• addrDesc - address descriptor, abstraction of an address. In all bitcoin like coins it is output script. Stored as variable length array of bytes.
• bigInt - unsigned big integer, stored as length of array (1 byte) followed by array of bytes of big int, i.e. (int_len byte)+(int_value []byte). Zero is stored as one byte containing 0.

Column families:

• default

  stores internal state in json format, under the key internalState.

  Most important internal state values are:

  • coin - which coin is indexed in DB
  • data format version - currently 3
  • dbState - closed, open, inconsistent

  Blockbook is on startup checking these values and does not allow to run against wrong coin, data format version and in inconsistent state.

• height

  maps block height to block hash and additional data about block

  (height uint32) -> (hash [32]byte)+(time uint32)+(nr_txs vuint)+(size vuint)
  

• addresses

  maps addrDesc+block height to array of outpoints (array of transactions with input/output index). Input/output is recognized by the sign of the number, output is positive, input is negative, with operation bitwise complement ^ performed on the number.

  (addrDesc []byte)+(height uint32) -> []((txid [32]byte)+(index vint))
  

• addressBalance

  maps addrDesc to number of transactions, sent amount and total balance of given address

  (addrDesc []byte) -> (nr_txs vuint)+(sent_amount bigInt)+(balance bigInt)
  

• txAddresses

  maps txid to block height and array of input addrDesc with amounts and array of output addrDesc with amounts, with flag if output is spent. In case of spent output, addrDesc_len is negative (negative sign is achieved by bitwise complement ^).

  (txid []byte) -> (height vuint)+
                   (nr_inputs vuint)+[]((addrDesc_len vuint)+(addrDesc []byte)+(amount bigInt))+
                   (nr_outputs vuint)+[]((addrDesc_len vint)+(addrDesc []byte)+(amount bigInt))
  

• blockTxs

  maps block height to an array of txids and input points in the block - only last 300 (by default) blocks are kept, the column is used in case of rollback.

  (height uint32) -> []((txid [32]byte)+(nr_inputs vuint)+[]((txid [32]byte)+(index vint)))
  

• transactions

  transaction cache, txdata is generated by coin specific parser function PackTx

  (txid []byte) -> (txdata []byte)