blockbook/fiat/coingecko.go

137 lines
3.7 KiB
Go
Raw Permalink Normal View History

Add fiat rates functionality (#316) * Add initial commit for fiat rates functionality * templates.go: use bash from current user's environment * bitcoinrpc.go: add FiatRates and FiatRatesParams to config * blockbook.go: add initFiatRatesDownloader kickoff * bitcoin.json: add coingecko API URL * rockdb.go: add FindTicker and StoreTicker functions * rocksdb_test.go: add a simple test for storing and getting FiatRate tickers * rocksdb: add FindLastTicker and convertDate, make FindTicker return strings * rocksdb: add ConvertDate function and CoinGeckoTicker struct, update tests * blockbook.go, fiat: finalize the CoinGecko downloader * coingecko.go: do not stop syncing when encountered an error * rocksdb_test: fix the exported function name * worker.go: make getBlockInfoFromBlockID a public function * public.go: apiTickers kickoff * rocksdb_test: fix the unittest comment * coingecko.go: update comments * blockbook.go, fiat: reword CoinGecko -> FiatRates, fix binary search upper bound, remove assignment of goroutine call result * rename coingecko -> fiat_rates * fiat_rates: export only the necessary methods * blockbook.go: update log message * bitcoinrpc.go: remove fiatRates settings * use CurrencyRatesTicker structure everywhere, fix time format string, update tests, use UTC time * add /api/v2/tickers tests, store rates as strings (json.Number) * fiat_rates: add more tests, metrics and tickers-list endpoint, make the "currency" parameter mandatory * public, worker: move FiatRates API logic to worker.go * fiat_rates: add a future date test, fix comments, add more checks, store time as a pointer * rocksdb_test: remove unneeded code * fiat_rates: add a "ping" call to check server availability * fiat_rates: do not return empty ticker, return nil instead if not found add a test for non-existent ticker * rocksdb_test: remove Sleep from tests * worker.go: do not propagate all API errors to the client * move InitTestFiatRates from rocksdb.go to public_test.go * public.go: fix FiatRatesFindLastTicker result check * fiat_rates: mock API server responses * remove commented-out code * fiat_rates: add comment explaining what periodSeconds attribute is used for * websocket.go: implement fiatRates websocket endpoints & add tests * fiatRates: add getFiatRatesTickersList websocket endpoint & test * fiatRates: make websocket getFiatRatesByDate accept an array of dates, add more tests * fiatRates: remove getFiatRatesForBlockID from websocket endpoints * fiatRates: remove "if test", use custom startTime instead Update tests and mock data * fiatRates: finalize websocket functionality add "date" parameter to TickerList return data timestamps where needed fix sync bugs (nil timestamp, duplicate save) * fiatRates: add FiatRates configs for different coins * worker.go: make GetBlockInfoFromBlockID private again * fiatRates: wait & retry on errors, remove Ping function * websocket.go: remove incorrect comment * fiatRates: move coingecko-related code to a separate file, use interface * fiatRates: if the new rates are the same as previous, try five more times, and only then store them * coingecko: fix getting actual rates, add a timestamp parameter to get uncached responses * vertcoin_testnet.json: remove fiat rates parameters * fiat_rates: add timestamp to log message about skipping the repeating rates
2019-12-17 02:40:02 -07:00
package fiat
import (
"encoding/json"
"errors"
"io/ioutil"
"net/http"
"strconv"
"time"
"github.com/golang/glog"
2021-04-05 12:59:11 -06:00
"spacecruft.org/spacecruft/blockbook/db"
Add fiat rates functionality (#316) * Add initial commit for fiat rates functionality * templates.go: use bash from current user's environment * bitcoinrpc.go: add FiatRates and FiatRatesParams to config * blockbook.go: add initFiatRatesDownloader kickoff * bitcoin.json: add coingecko API URL * rockdb.go: add FindTicker and StoreTicker functions * rocksdb_test.go: add a simple test for storing and getting FiatRate tickers * rocksdb: add FindLastTicker and convertDate, make FindTicker return strings * rocksdb: add ConvertDate function and CoinGeckoTicker struct, update tests * blockbook.go, fiat: finalize the CoinGecko downloader * coingecko.go: do not stop syncing when encountered an error * rocksdb_test: fix the exported function name * worker.go: make getBlockInfoFromBlockID a public function * public.go: apiTickers kickoff * rocksdb_test: fix the unittest comment * coingecko.go: update comments * blockbook.go, fiat: reword CoinGecko -> FiatRates, fix binary search upper bound, remove assignment of goroutine call result * rename coingecko -> fiat_rates * fiat_rates: export only the necessary methods * blockbook.go: update log message * bitcoinrpc.go: remove fiatRates settings * use CurrencyRatesTicker structure everywhere, fix time format string, update tests, use UTC time * add /api/v2/tickers tests, store rates as strings (json.Number) * fiat_rates: add more tests, metrics and tickers-list endpoint, make the "currency" parameter mandatory * public, worker: move FiatRates API logic to worker.go * fiat_rates: add a future date test, fix comments, add more checks, store time as a pointer * rocksdb_test: remove unneeded code * fiat_rates: add a "ping" call to check server availability * fiat_rates: do not return empty ticker, return nil instead if not found add a test for non-existent ticker * rocksdb_test: remove Sleep from tests * worker.go: do not propagate all API errors to the client * move InitTestFiatRates from rocksdb.go to public_test.go * public.go: fix FiatRatesFindLastTicker result check * fiat_rates: mock API server responses * remove commented-out code * fiat_rates: add comment explaining what periodSeconds attribute is used for * websocket.go: implement fiatRates websocket endpoints & add tests * fiatRates: add getFiatRatesTickersList websocket endpoint & test * fiatRates: make websocket getFiatRatesByDate accept an array of dates, add more tests * fiatRates: remove getFiatRatesForBlockID from websocket endpoints * fiatRates: remove "if test", use custom startTime instead Update tests and mock data * fiatRates: finalize websocket functionality add "date" parameter to TickerList return data timestamps where needed fix sync bugs (nil timestamp, duplicate save) * fiatRates: add FiatRates configs for different coins * worker.go: make GetBlockInfoFromBlockID private again * fiatRates: wait & retry on errors, remove Ping function * websocket.go: remove incorrect comment * fiatRates: move coingecko-related code to a separate file, use interface * fiatRates: if the new rates are the same as previous, try five more times, and only then store them * coingecko: fix getting actual rates, add a timestamp parameter to get uncached responses * vertcoin_testnet.json: remove fiat rates parameters * fiat_rates: add timestamp to log message about skipping the repeating rates
2019-12-17 02:40:02 -07:00
)
// Coingecko is a structure that implements RatesDownloaderInterface
type Coingecko struct {
url string
coin string
httpTimeoutSeconds time.Duration
timeFormat string
}
// NewCoinGeckoDownloader creates a coingecko structure that implements the RatesDownloaderInterface
func NewCoinGeckoDownloader(url string, coin string, timeFormat string) RatesDownloaderInterface {
return &Coingecko{
url: url,
coin: coin,
httpTimeoutSeconds: 15 * time.Second,
timeFormat: timeFormat,
}
}
// makeRequest retrieves the response from Coingecko API at the specified date.
// If timestamp is nil, it fetches the latest market data available.
func (cg *Coingecko) makeRequest(timestamp *time.Time) ([]byte, error) {
requestURL := cg.url + "/coins/" + cg.coin
if timestamp != nil {
requestURL += "/history"
}
req, err := http.NewRequest("GET", requestURL, nil)
if err != nil {
glog.Errorf("Error creating a new request for %v: %v", requestURL, err)
return nil, err
}
req.Close = true
req.Header.Set("Content-Type", "application/json")
// Add query parameters
q := req.URL.Query()
// Add a unix timestamp to query parameters to get uncached responses
currentTimestamp := strconv.FormatInt(time.Now().UTC().UnixNano(), 10)
q.Add("current_timestamp", currentTimestamp)
if timestamp == nil {
q.Add("market_data", "true")
q.Add("localization", "false")
q.Add("tickers", "false")
q.Add("community_data", "false")
q.Add("developer_data", "false")
} else {
timestampFormatted := timestamp.Format(cg.timeFormat)
q.Add("date", timestampFormatted)
}
req.URL.RawQuery = q.Encode()
client := &http.Client{
Timeout: cg.httpTimeoutSeconds,
}
resp, err := client.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
return nil, errors.New("Invalid response status: " + string(resp.Status))
}
bodyBytes, err := ioutil.ReadAll(resp.Body)
if err != nil {
return nil, err
}
return bodyBytes, nil
}
// GetData gets fiat rates from API at the specified date and returns a CurrencyRatesTicker
// If timestamp is nil, it will download the current fiat rates.
func (cg *Coingecko) getTicker(timestamp *time.Time) (*db.CurrencyRatesTicker, error) {
dataTimestamp := timestamp
if timestamp == nil {
timeNow := time.Now()
dataTimestamp = &timeNow
}
dataTimestampUTC := dataTimestamp.UTC()
ticker := &db.CurrencyRatesTicker{Timestamp: &dataTimestampUTC}
bodyBytes, err := cg.makeRequest(timestamp)
if err != nil {
return nil, err
}
type FiatRatesResponse struct {
MarketData struct {
Prices map[string]float64 `json:"current_price"`
Add fiat rates functionality (#316) * Add initial commit for fiat rates functionality * templates.go: use bash from current user's environment * bitcoinrpc.go: add FiatRates and FiatRatesParams to config * blockbook.go: add initFiatRatesDownloader kickoff * bitcoin.json: add coingecko API URL * rockdb.go: add FindTicker and StoreTicker functions * rocksdb_test.go: add a simple test for storing and getting FiatRate tickers * rocksdb: add FindLastTicker and convertDate, make FindTicker return strings * rocksdb: add ConvertDate function and CoinGeckoTicker struct, update tests * blockbook.go, fiat: finalize the CoinGecko downloader * coingecko.go: do not stop syncing when encountered an error * rocksdb_test: fix the exported function name * worker.go: make getBlockInfoFromBlockID a public function * public.go: apiTickers kickoff * rocksdb_test: fix the unittest comment * coingecko.go: update comments * blockbook.go, fiat: reword CoinGecko -> FiatRates, fix binary search upper bound, remove assignment of goroutine call result * rename coingecko -> fiat_rates * fiat_rates: export only the necessary methods * blockbook.go: update log message * bitcoinrpc.go: remove fiatRates settings * use CurrencyRatesTicker structure everywhere, fix time format string, update tests, use UTC time * add /api/v2/tickers tests, store rates as strings (json.Number) * fiat_rates: add more tests, metrics and tickers-list endpoint, make the "currency" parameter mandatory * public, worker: move FiatRates API logic to worker.go * fiat_rates: add a future date test, fix comments, add more checks, store time as a pointer * rocksdb_test: remove unneeded code * fiat_rates: add a "ping" call to check server availability * fiat_rates: do not return empty ticker, return nil instead if not found add a test for non-existent ticker * rocksdb_test: remove Sleep from tests * worker.go: do not propagate all API errors to the client * move InitTestFiatRates from rocksdb.go to public_test.go * public.go: fix FiatRatesFindLastTicker result check * fiat_rates: mock API server responses * remove commented-out code * fiat_rates: add comment explaining what periodSeconds attribute is used for * websocket.go: implement fiatRates websocket endpoints & add tests * fiatRates: add getFiatRatesTickersList websocket endpoint & test * fiatRates: make websocket getFiatRatesByDate accept an array of dates, add more tests * fiatRates: remove getFiatRatesForBlockID from websocket endpoints * fiatRates: remove "if test", use custom startTime instead Update tests and mock data * fiatRates: finalize websocket functionality add "date" parameter to TickerList return data timestamps where needed fix sync bugs (nil timestamp, duplicate save) * fiatRates: add FiatRates configs for different coins * worker.go: make GetBlockInfoFromBlockID private again * fiatRates: wait & retry on errors, remove Ping function * websocket.go: remove incorrect comment * fiatRates: move coingecko-related code to a separate file, use interface * fiatRates: if the new rates are the same as previous, try five more times, and only then store them * coingecko: fix getting actual rates, add a timestamp parameter to get uncached responses * vertcoin_testnet.json: remove fiat rates parameters * fiat_rates: add timestamp to log message about skipping the repeating rates
2019-12-17 02:40:02 -07:00
} `json:"market_data"`
}
var data FiatRatesResponse
err = json.Unmarshal(bodyBytes, &data)
if err != nil {
glog.Errorf("Error parsing FiatRates response: %v", err)
return nil, err
}
ticker.Rates = data.MarketData.Prices
return ticker, nil
}
// MarketDataExists checks if there's data available for the specific timestamp.
func (cg *Coingecko) marketDataExists(timestamp *time.Time) (bool, error) {
resp, err := cg.makeRequest(timestamp)
if err != nil {
glog.Error("Error getting market data: ", err)
return false, err
}
type FiatRatesResponse struct {
MarketData struct {
Prices map[string]interface{} `json:"current_price"`
} `json:"market_data"`
}
var data FiatRatesResponse
err = json.Unmarshal(resp, &data)
if err != nil {
glog.Errorf("Error parsing Coingecko response: %v", err)
return false, err
}
return len(data.MarketData.Prices) != 0, nil
}