RanchiMall/blockbook
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
1,924 Commits 8 Branches 2 Tags 8 MiB
76a19b7f10
Commit Graph

236 Commits

Author SHA1 Message Date
Martin Boehm
c852214407 Complete explorer/api unit tests 2018-10-19 16:37:49 +02:00
Martin Boehm
d7ed48a657 Handle not found block in rocksdb.GetBlockInfo 2018-10-19 16:37:48 +02:00
Martin Boehm
cc11ae1e57 Setup public http server test 2018-10-19 16:37:48 +02:00
Martin Boehm
caab79ed94 Extract rocksdb_test data to dbtestdata package for reusibility 2018-10-19 16:37:48 +02:00
Jakub Matys
b5a8cd2fc5 Error is correctly returned 2018-10-16 13:35:50 +02:00
Martin Boehm
d369242b48 Bump rocksdb to version 5.15.10 and use data format version 3 2018-10-14 00:29:05 +02:00
Martin Boehm
84b457b34a Use TxAddresses index in TxCache 2018-10-08 14:55:21 +02:00
Jakub Matys
503ff9582a Merge branch 'tests' 2018-10-04 16:59:39 +02:00
Martin Boehm
8e057e9e7b Fix incorrect logging formatting directives 2018-10-04 09:19:41 +02:00
Jakub Matys
e102c81577 Merge branch 'params-registration' into tests 2018-10-02 22:11:07 +02:00
Jakub Matys
8c7f383e9a Explicit registration of network parameters + cleanup in tests 2018-10-02 17:02:57 +02:00
Jakub Matys
5000c01f11 WIP: sync tests 2018-10-01 14:32:40 +02:00
Jakub Matys
c5cb1e2e54 Merge branch 'integration-tests-upgrade' into tests 2018-10-01 13:43:38 +02:00
Martin Boehm
776bebdf23 Revert "Update balances in extra goroutine during import block" 
This reverts commit 099321126f.
The parallel processing of balances did not bring any speedup
(actually it was a little bit slower)
 2018-10-01 13:28:19 +02:00
Jakub Matys
642310eb95 Added tests for single-thread connectBlocks 2018-09-26 12:19:40 +02:00
Martin Boehm
099321126f Update balances in extra goroutine during import block 2018-09-26 12:17:57 +02:00
Martin Boehm
67e47adf04 Reset BlockbookAppInfo metrics to avoid duplicate values 2018-09-26 11:26:38 +02:00
Jakub Matys
db39b5ef51 Interrupt connectBlocks on OS signal while initial sync 2018-09-25 13:40:52 +02:00
Martin Boehm
77d4e59f1b Compute stats of bulk connect block caches 2018-09-24 18:23:13 +02:00
Jakub Matys
d2d5df88f2 Parallel sync improved in order to write all blocks and handle OS signal 2018-09-24 12:13:01 +02:00
Jakub Matys
a69f79f955 Parallel sync improved in order to write all blocks and handle OS signal 2018-09-24 12:08:07 +02:00
Martin Boehm
d6b9cd496b Start public interface in limited mode before initial sync #51 2018-09-20 12:15:46 +02:00
Martin Boehm
9a33c338dd Fix error handling in rocksdb.dirSize 2018-09-20 12:06:34 +02:00
Martin Boehm
184f2e9ca6 Add maxOpenFiles rocksdb flag 2018-09-18 11:49:39 +02:00
Martin Boehm
bebddbcd11 Implement list of blocks 2018-09-14 12:10:03 +02:00
Martin Boehm
503ff4389a Merge branch 'master' into v0.0.7 2018-09-11 13:44:36 +02:00
Martin Boehm
8bdf4b0ae3 Prepare notification of block height in OnNewBlock 2018-09-11 13:37:12 +02:00
Martin Boehm
fdf1fe612f Stringer interface for AddressDescriptor 2018-09-02 21:31:33 +02:00
Martin Boehm
676aabfaa6 Make AddressDescriptor as type,expose DB/mempool methods working with it 2018-08-30 22:43:37 +02:00
Martin Boehm
be19523065 Refactor of address handling, renamed addrID to addr descriptor addrDesc 2018-08-29 15:31:19 +02:00
Martin Boehm
b1e749dab9 Add logging of memory status 2018-08-26 21:29:10 +02:00
Martin Boehm
ad5ddbd029 Write height column in bulk mode 2018-08-23 23:20:07 +02:00
Martin Boehm
328312e48a Add dbcache configuration flag, tune db options 2018-08-23 21:19:12 +02:00
Martin Boehm
2001312382 Extract bulk connect to own source file 2018-08-23 11:18:36 +02:00
Martin Boehm
a34ac14883 Allow partitioned index filter in rocksdb 2018-08-23 11:15:59 +02:00
Martin Boehm
41252d33d2 Tune RocksDB options and measure memory usage 2018-08-22 16:20:52 +02:00
Martin Boehm
59497e3c97 Get address data for explorer using index v3 - WIP 2018-08-22 00:48:53 +02:00
Martin Boehm
188eed8881 Write data to DB synchronously in bulk connect 
RocksDB has some memory leak/fragmentation problem
when inserting data in parallel
 2018-08-21 18:56:30 +02:00
Martin Boehm
c9471bf867 Store extended info about block in heigth column 2018-08-21 16:36:14 +02:00
Martin Boehm
78f6162d5c Add height to TxAddresses as required by explorer 2018-08-21 13:16:29 +02:00
Martin Boehm
bbc47db6dd Get address data for explorer using index v3 - WIP 2018-08-21 10:11:27 +02:00
Martin Boehm
c67306ad09 Make AddrBalance and TxAddresses publicly loadable from DB 2018-08-20 18:35:46 +02:00
Martin Boehm
5621ed49f3 Optimize bulk connect of blocks 2018-08-19 17:43:00 +02:00
Martin Boehm
d45d028ef2 Implement bulk connect blocks 2018-08-19 00:23:26 +02:00
Martin Boehm
878d25ea42 Fix handling of coinbase txs in db, add test for it 2018-08-17 23:48:36 +02:00
Martin Boehm
745d7af92d Implement index v3 WIP 2018-08-17 16:33:42 +02:00
Martin Boehm
f5b29b3a17 Implement index v3 WIP 2018-08-15 19:22:26 +02:00
Martin Boehm
78d2c085a9 Log expected db data version 2018-08-15 16:07:09 +02:00
Martin Boehm
8e3c7f851b Clean up the rocksdb sources and disconnect UTXO blocks WIP 2018-08-03 19:26:16 +02:00
Martin Boehm
7e11a4e615 Set LZ4HC compression for all columns 2018-08-03 15:04:49 +02:00
Martin Boehm
a04e38f0f7 Improve handling of invalid addresses 2018-08-02 17:46:23 +02:00
Martin Boehm
fb93c9ff7a Add unit test for pack/unpack txAddress 2018-08-02 16:10:28 +02:00
Martin Boehm
a2bbf3f9de Implement and test connectBlock for index v3 2018-08-02 14:30:45 +02:00
Martin Boehm
e558c10da9 Implement pack and unpack of big.Int 2018-07-27 19:46:21 +02:00
Martin Boehm
75d48376e1 Change db columns, enable compression on some columns 2018-07-27 14:08:20 +02:00
Martin Boehm
4db0895899 Check DB version on startup 2018-07-27 12:57:48 +02:00
Jakub Matys
9409907f80 Tests are classified by tags 2018-06-14 17:27:03 +02:00
Martin Boehm
d4cd2ad607 Return state information for index request on internal http server 2018-06-12 22:57:46 +02:00
Martin Boehm
4f5461da4f Report metrics of all db columns 2018-06-08 14:05:41 +02:00
Martin Boehm
fd7828a877 Compute column stats periodically 2018-06-08 13:19:57 +02:00
Martin Boehm
2535e4b6b7 Report metrics of txcache 2018-06-04 18:09:36 +02:00
Martin Boehm
4343c888a0 Update txcache stats on put/delete db operations 2018-06-04 17:11:10 +02:00
Martin Boehm
4f42d2f1d6 Add method to compute stats (rowcount, size) of all db columns 2018-06-01 16:01:58 +02:00
Martin Boehm
9147781772 Use single instance instead of global InternalState object 2018-05-29 11:37:35 +02:00
Martin Boehm
927fd0aaea Fix inSync state 2018-05-29 10:04:24 +02:00
Martin Boehm
8b20c38136 Put parameter BlockAddressesToKeep to coin configuration json 2018-05-28 14:57:44 +02:00
Martin Boehm
1c2ae8cec7 Fix rocksdb test 2018-05-23 11:07:16 +02:00
Martin Boehm
c03b3d8a1a Store coin in internal state and verify that rpc coin matches db coin 2018-05-23 10:58:07 +02:00
Martin Boehm
af064a3921 Store sync and mempool sync state to internal state 2018-05-22 17:34:37 +02:00
Martin Boehm
291e99ba5f Add internal state of the application 2018-05-22 12:56:51 +02:00
Martin Boehm
7cb8c8d3f0 Add option to disable txcache 2018-05-14 15:49:08 +02:00
Martin Boehm
83edc33b3f Fix processing of transactions in ConnectBlock 2018-05-03 01:03:20 +02:00
Martin Boehm
880e3e8025 Fix rocksdb test to handle nondeterministic order of addresses in value 2018-04-30 18:54:48 +02:00
Martin Boehm
b3b8512958 Sync using indexv2 2018-04-30 14:50:19 +02:00
Martin Boehm
df2a6b5551 Sync using indexv2 - WIP 2018-04-29 21:35:45 +02:00
Martin Boehm
0a55ca61f6 Sync using indexv2 - WIP 2018-04-29 00:17:30 +02:00
Martin Boehm
22af986121 Implement DisconnectBlocks in index v2 - WIP 2018-04-26 19:50:22 +02:00
Martin Boehm
ed027a68c9 Implement DisconnectBlocks in index v2 - WIP 2018-04-24 12:00:24 +02:00
Martin Boehm
ce485099a7 Implement DisconnectBlocks in index v2 - WIP 2018-04-23 17:11:08 +02:00
Martin Boehm
f78b050234 Fix indexing of coinbase transactions 2018-04-20 23:53:17 +02:00
Martin Boehm
d569a08cf8 Test rocksdb tx storage functionality 2018-04-20 16:03:45 +02:00
Martin Boehm
850b175940 Test rocksdb GetBestBlock, GetBlockHash, small refactor in test 2018-04-20 15:08:08 +02:00
Martin Boehm
febcba5fbe Use new db column blockaddresses to support UTXO chain block disconnect 2018-04-20 13:56:55 +02:00
Martin Boehm
9ad8a4b873 Test for GetTransactions in TestRocksDB_Index_UTXO 2018-04-19 15:11:32 +02:00
Martin Boehm
c657381d7e Change the way UTXO addresses are indexed - WIP 2018-04-19 14:28:05 +02:00
Martin Boehm
0ae9c446a0 Add TestRocksDB_Index_UTXO test - WIP 2018-04-18 23:42:38 +02:00
Martin Boehm
2ee2177970 Change the way UTXO addresses are indexed - WIP 2018-04-18 23:42:11 +02:00
Martin Boehm
296eee828f Change the way UTXO addresses are indexed - WIP 
Columns before:
outputs: saddress+block height -> outpoints
inputs: txid+vout -> spending txid+vout

Columns after change:
addresses: address+block height -> input or output outpoints
unspenttxs: txid -> addresses+indexes
 2018-04-17 23:50:01 +02:00
Martin Boehm
a117bd2abd Rework resyncIndex to handle eth rollbacks 2018-04-09 15:43:42 +02:00
Martin Boehm
a75376706e Move packing/unpacking of Txid and Block Hash to coin specific code 2018-04-08 11:24:29 +02:00
Martin Boehm
f8eca976ab Fix computation of confirmations 2018-04-06 17:13:38 +02:00
Martin Boehm
39a31d4544 Handle transactions with missing ethereum To address 2018-03-28 13:23:43 +02:00
Martin Boehm
bfb22c4f20 Fix error in chain sync by height 2018-03-28 12:09:05 +02:00
Martin Boehm
b1694b4a61 Support sync of blockchains without block.next hash 2018-03-26 16:44:54 +02:00
Martin Boehm
44fdb5528b Support indexing of non UTXO chains 2018-03-23 13:15:35 +01:00
Martin Boehm
dda80bd074 Refactor names in interface BlockChainParser, added method IsUTXOChain 2018-03-23 11:03:41 +01:00
Jakub Matys
a9effbe835 Added ZCashBlockParser 2018-03-20 17:28:03 +01:00
Jakub Matys
fbfb33cb5d Output index uses abstract identifier instead of outputScript 2018-03-20 16:01:36 +01:00
Martin Boehm
22d30ffaa1 Compute db size metrics during initial import 2018-03-14 12:40:17 +01:00
Martin Boehm
96bfdfd74e Use block data in disconnect blocks, remove disconnected txs from cache 2018-03-14 12:34:13 +01:00
Jakub Matys
d7229f8db0 added prometheus metrics 2018-03-13 11:44:50 +01:00
Martin Boehm
b55306b6eb Hide GetBlockWithoutHeader BTC optimization from BlockChain interface 2018-03-12 17:37:32 +01:00
Martin Boehm
4c51d7cb0a Add rocksdb reopen db method 
Mainly used for debugging purposes
 2018-03-12 16:28:52 +01:00
Martin Boehm
f80ae32846 Abstract bitcoinrpc in preparation for alt coins 2018-03-08 12:59:37 +01:00
Martin Boehm
0783cac294 Detect type of the block chain and use correct chain params 2018-03-07 09:38:19 +01:00
Martin Boehm
cac412527d Improve handling of os interrupt signal in connectBlocksParallel 2018-03-06 13:13:30 +01:00
Martin Boehm
d1c4c66c5f Implement TxCache stored in RocksDB 2018-03-06 12:36:24 +01:00
Martin Boehm
58205ed84a Store transactions in RocksDB 2018-03-05 18:14:41 +01:00
Martin Boehm
b3d1d449fd Fix minimum sync height 2018-03-01 20:20:50 +01:00
Martin Boehm
9bdb83deff Refactor sync code to own file 2018-03-01 18:37:01 +01:00
Martin Boehm
34400f5b2b Add context to errors returned by bitcoinrpc 2018-03-01 11:06:10 +01:00
Martin Boehm
da28595321 Fix incorrect memory handling 2018-02-28 12:24:29 +01:00
Martin Boehm
9dfef815b3 Tune RocksDB 2018-02-27 13:10:18 +01:00
Martin Boehm
386b4f1db8 Tune RocksDB 2018-02-26 18:42:55 +01:00
Martin Boehm
2375837d0a Tune RocksDB 2018-02-26 13:11:23 +01:00
Martin Boehm
7a1ff7241f Tune RocksDB. 
Remove bulk import - it consumes a lot of disk space.
Set no compression of data.
Reduce write buffer to .5GB from 2GB.
 2018-02-24 16:25:55 +01:00
Martin Boehm
af0e85ce43 Tune rocksdb options 2018-02-19 16:17:21 +01:00
Martin Boehm
d96af8e648 Change return values of index.GetSpentOutput method 2018-02-08 13:11:10 +01:00
Martin Boehm
43dec549d5 Implement index.GetSpentOutput method 2018-02-08 13:00:22 +01:00
Martin Boehm
ec87f4ed8d Use maximum size on disk trigger to invoke DB compaction 2018-02-06 09:43:54 +01:00
Martin Boehm
359d7ee1c4 Use constants instead of magic numbers 2018-02-06 09:12:50 +01:00
Martin Boehm
ee663944e2 Refresh iterator in DisconnectBlocks to avoid huge snapshot 2018-02-05 18:35:05 +01:00
Martin Boehm
d6e9b5caa4 Use special RocksDB options for outputs 2018-02-05 17:12:32 +01:00
Martin Boehm
97cc2aca17 Use PrepareForBulkLoad option for connectBlocksParallel 2018-02-05 12:27:41 +01:00
Martin Boehm
e31095d03f Compact database during connectBlocksParallel 2018-02-05 10:31:22 +01:00
Martin Boehm
1950f125e5 Ignore too large (> 512 bytes) output scripts 2018-02-04 23:17:54 +01:00
Martin Boehm
7c71876c12 Lookup input transactions in mempool 2018-02-03 19:14:27 +01:00
Martin Boehm
a403b8d0fe Rename package bitcoin to bchain 2018-01-31 15:23:17 +01:00
Martin Boehm
e22e1a946d Use glog for logging with verbosity levels 2018-01-30 18:22:25 +01:00
Martin Boehm
3dacffda59 Use output script (ScriptPubKey) as key to output transactions 2018-01-29 23:25:40 +01:00
Martin Boehm
2dd0b9fc21 RocksDB.GetTransactions returns transactions from inputs 2018-01-28 11:59:05 +01:00
Martin Boehm
9afa5b4187 resyncIndex DisconnectBlocks 2018-01-28 00:59:54 +01:00
Martin Boehm
48057b2453 GetBestBlock height and hash from DB 2018-01-24 18:02:46 +01:00
Martin Boehm
e72b437e23 rocksdb column families 2018-01-24 16:57:05 +01:00
Martin Boehm
6af0a39e61 accept base58 and hex addresses 2018-01-24 15:10:07 +01:00
Martin Boehm
441c0f9024 folder reorganization 2018-01-18 16:44:31 +01:00