RanchiMall/blockbook
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
1,872 Commits 8 Branches 2 Tags 8 MiB
8bdc3da694
Commit Graph

223 Commits

Author SHA1 Message Date
Martin Boehm
776bebdf23 Revert "Update balances in extra goroutine during import block" 
This reverts commit 099321126f.
The parallel processing of balances did not bring any speedup
(actually it was a little bit slower)
 2018-10-01 13:28:19 +02:00
Jakub Matys
642310eb95 Added tests for single-thread connectBlocks 2018-09-26 12:19:40 +02:00
Martin Boehm
099321126f Update balances in extra goroutine during import block 2018-09-26 12:17:57 +02:00
Martin Boehm
67e47adf04 Reset BlockbookAppInfo metrics to avoid duplicate values 2018-09-26 11:26:38 +02:00
Jakub Matys
db39b5ef51 Interrupt connectBlocks on OS signal while initial sync 2018-09-25 13:40:52 +02:00
Martin Boehm
77d4e59f1b Compute stats of bulk connect block caches 2018-09-24 18:23:13 +02:00
Jakub Matys
d2d5df88f2 Parallel sync improved in order to write all blocks and handle OS signal 2018-09-24 12:13:01 +02:00
Jakub Matys
a69f79f955 Parallel sync improved in order to write all blocks and handle OS signal 2018-09-24 12:08:07 +02:00
Martin Boehm
d6b9cd496b Start public interface in limited mode before initial sync #51 2018-09-20 12:15:46 +02:00
Martin Boehm
9a33c338dd Fix error handling in rocksdb.dirSize 2018-09-20 12:06:34 +02:00
Martin Boehm
184f2e9ca6 Add maxOpenFiles rocksdb flag 2018-09-18 11:49:39 +02:00
Martin Boehm
bebddbcd11 Implement list of blocks 2018-09-14 12:10:03 +02:00
Martin Boehm
503ff4389a Merge branch 'master' into v0.0.7 2018-09-11 13:44:36 +02:00
Martin Boehm
8bdf4b0ae3 Prepare notification of block height in OnNewBlock 2018-09-11 13:37:12 +02:00
Martin Boehm
fdf1fe612f Stringer interface for AddressDescriptor 2018-09-02 21:31:33 +02:00
Martin Boehm
676aabfaa6 Make AddressDescriptor as type,expose DB/mempool methods working with it 2018-08-30 22:43:37 +02:00
Martin Boehm
be19523065 Refactor of address handling, renamed addrID to addr descriptor addrDesc 2018-08-29 15:31:19 +02:00
Martin Boehm
b1e749dab9 Add logging of memory status 2018-08-26 21:29:10 +02:00
Martin Boehm
ad5ddbd029 Write height column in bulk mode 2018-08-23 23:20:07 +02:00
Martin Boehm
328312e48a Add dbcache configuration flag, tune db options 2018-08-23 21:19:12 +02:00
Martin Boehm
2001312382 Extract bulk connect to own source file 2018-08-23 11:18:36 +02:00
Martin Boehm
a34ac14883 Allow partitioned index filter in rocksdb 2018-08-23 11:15:59 +02:00
Martin Boehm
41252d33d2 Tune RocksDB options and measure memory usage 2018-08-22 16:20:52 +02:00
Martin Boehm
59497e3c97 Get address data for explorer using index v3 - WIP 2018-08-22 00:48:53 +02:00
Martin Boehm
188eed8881 Write data to DB synchronously in bulk connect 
RocksDB has some memory leak/fragmentation problem
when inserting data in parallel
 2018-08-21 18:56:30 +02:00
Martin Boehm
c9471bf867 Store extended info about block in heigth column 2018-08-21 16:36:14 +02:00
Martin Boehm
78f6162d5c Add height to TxAddresses as required by explorer 2018-08-21 13:16:29 +02:00
Martin Boehm
bbc47db6dd Get address data for explorer using index v3 - WIP 2018-08-21 10:11:27 +02:00
Martin Boehm
c67306ad09 Make AddrBalance and TxAddresses publicly loadable from DB 2018-08-20 18:35:46 +02:00
Martin Boehm
5621ed49f3 Optimize bulk connect of blocks 2018-08-19 17:43:00 +02:00
Martin Boehm
d45d028ef2 Implement bulk connect blocks 2018-08-19 00:23:26 +02:00
Martin Boehm
878d25ea42 Fix handling of coinbase txs in db, add test for it 2018-08-17 23:48:36 +02:00
Martin Boehm
745d7af92d Implement index v3 WIP 2018-08-17 16:33:42 +02:00
Martin Boehm
f5b29b3a17 Implement index v3 WIP 2018-08-15 19:22:26 +02:00
Martin Boehm
78d2c085a9 Log expected db data version 2018-08-15 16:07:09 +02:00
Martin Boehm
8e3c7f851b Clean up the rocksdb sources and disconnect UTXO blocks WIP 2018-08-03 19:26:16 +02:00
Martin Boehm
7e11a4e615 Set LZ4HC compression for all columns 2018-08-03 15:04:49 +02:00
Martin Boehm
a04e38f0f7 Improve handling of invalid addresses 2018-08-02 17:46:23 +02:00
Martin Boehm
fb93c9ff7a Add unit test for pack/unpack txAddress 2018-08-02 16:10:28 +02:00
Martin Boehm
a2bbf3f9de Implement and test connectBlock for index v3 2018-08-02 14:30:45 +02:00
Martin Boehm
e558c10da9 Implement pack and unpack of big.Int 2018-07-27 19:46:21 +02:00
Martin Boehm
75d48376e1 Change db columns, enable compression on some columns 2018-07-27 14:08:20 +02:00
Martin Boehm
4db0895899 Check DB version on startup 2018-07-27 12:57:48 +02:00
Jakub Matys
9409907f80 Tests are classified by tags 2018-06-14 17:27:03 +02:00
Martin Boehm
d4cd2ad607 Return state information for index request on internal http server 2018-06-12 22:57:46 +02:00
Martin Boehm
4f5461da4f Report metrics of all db columns 2018-06-08 14:05:41 +02:00
Martin Boehm
fd7828a877 Compute column stats periodically 2018-06-08 13:19:57 +02:00
Martin Boehm
2535e4b6b7 Report metrics of txcache 2018-06-04 18:09:36 +02:00
Martin Boehm
4343c888a0 Update txcache stats on put/delete db operations 2018-06-04 17:11:10 +02:00
Martin Boehm
4f42d2f1d6 Add method to compute stats (rowcount, size) of all db columns 2018-06-01 16:01:58 +02:00
Martin Boehm
9147781772 Use single instance instead of global InternalState object 2018-05-29 11:37:35 +02:00
Martin Boehm
927fd0aaea Fix inSync state 2018-05-29 10:04:24 +02:00
Martin Boehm
8b20c38136 Put parameter BlockAddressesToKeep to coin configuration json 2018-05-28 14:57:44 +02:00
Martin Boehm
1c2ae8cec7 Fix rocksdb test 2018-05-23 11:07:16 +02:00
Martin Boehm
c03b3d8a1a Store coin in internal state and verify that rpc coin matches db coin 2018-05-23 10:58:07 +02:00
Martin Boehm
af064a3921 Store sync and mempool sync state to internal state 2018-05-22 17:34:37 +02:00
Martin Boehm
291e99ba5f Add internal state of the application 2018-05-22 12:56:51 +02:00
Martin Boehm
7cb8c8d3f0 Add option to disable txcache 2018-05-14 15:49:08 +02:00
Martin Boehm
83edc33b3f Fix processing of transactions in ConnectBlock 2018-05-03 01:03:20 +02:00
Martin Boehm
880e3e8025 Fix rocksdb test to handle nondeterministic order of addresses in value 2018-04-30 18:54:48 +02:00
Martin Boehm
b3b8512958 Sync using indexv2 2018-04-30 14:50:19 +02:00
Martin Boehm
df2a6b5551 Sync using indexv2 - WIP 2018-04-29 21:35:45 +02:00
Martin Boehm
0a55ca61f6 Sync using indexv2 - WIP 2018-04-29 00:17:30 +02:00
Martin Boehm
22af986121 Implement DisconnectBlocks in index v2 - WIP 2018-04-26 19:50:22 +02:00
Martin Boehm
ed027a68c9 Implement DisconnectBlocks in index v2 - WIP 2018-04-24 12:00:24 +02:00
Martin Boehm
ce485099a7 Implement DisconnectBlocks in index v2 - WIP 2018-04-23 17:11:08 +02:00
Martin Boehm
f78b050234 Fix indexing of coinbase transactions 2018-04-20 23:53:17 +02:00
Martin Boehm
d569a08cf8 Test rocksdb tx storage functionality 2018-04-20 16:03:45 +02:00
Martin Boehm
850b175940 Test rocksdb GetBestBlock, GetBlockHash, small refactor in test 2018-04-20 15:08:08 +02:00
Martin Boehm
febcba5fbe Use new db column blockaddresses to support UTXO chain block disconnect 2018-04-20 13:56:55 +02:00
Martin Boehm
9ad8a4b873 Test for GetTransactions in TestRocksDB_Index_UTXO 2018-04-19 15:11:32 +02:00
Martin Boehm
c657381d7e Change the way UTXO addresses are indexed - WIP 2018-04-19 14:28:05 +02:00
Martin Boehm
0ae9c446a0 Add TestRocksDB_Index_UTXO test - WIP 2018-04-18 23:42:38 +02:00
Martin Boehm
2ee2177970 Change the way UTXO addresses are indexed - WIP 2018-04-18 23:42:11 +02:00
Martin Boehm
296eee828f Change the way UTXO addresses are indexed - WIP 
Columns before:
outputs: saddress+block height -> outpoints
inputs: txid+vout -> spending txid+vout

Columns after change:
addresses: address+block height -> input or output outpoints
unspenttxs: txid -> addresses+indexes
 2018-04-17 23:50:01 +02:00
Martin Boehm
a117bd2abd Rework resyncIndex to handle eth rollbacks 2018-04-09 15:43:42 +02:00
Martin Boehm
a75376706e Move packing/unpacking of Txid and Block Hash to coin specific code 2018-04-08 11:24:29 +02:00
Martin Boehm
f8eca976ab Fix computation of confirmations 2018-04-06 17:13:38 +02:00
Martin Boehm
39a31d4544 Handle transactions with missing ethereum To address 2018-03-28 13:23:43 +02:00
Martin Boehm
bfb22c4f20 Fix error in chain sync by height 2018-03-28 12:09:05 +02:00
Martin Boehm
b1694b4a61 Support sync of blockchains without block.next hash 2018-03-26 16:44:54 +02:00
Martin Boehm
44fdb5528b Support indexing of non UTXO chains 2018-03-23 13:15:35 +01:00
Martin Boehm
dda80bd074 Refactor names in interface BlockChainParser, added method IsUTXOChain 2018-03-23 11:03:41 +01:00
Jakub Matys
a9effbe835 Added ZCashBlockParser 2018-03-20 17:28:03 +01:00
Jakub Matys
fbfb33cb5d Output index uses abstract identifier instead of outputScript 2018-03-20 16:01:36 +01:00
Martin Boehm
22d30ffaa1 Compute db size metrics during initial import 2018-03-14 12:40:17 +01:00
Martin Boehm
96bfdfd74e Use block data in disconnect blocks, remove disconnected txs from cache 2018-03-14 12:34:13 +01:00
Jakub Matys
d7229f8db0 added prometheus metrics 2018-03-13 11:44:50 +01:00
Martin Boehm
b55306b6eb Hide GetBlockWithoutHeader BTC optimization from BlockChain interface 2018-03-12 17:37:32 +01:00
Martin Boehm
4c51d7cb0a Add rocksdb reopen db method 
Mainly used for debugging purposes
 2018-03-12 16:28:52 +01:00
Martin Boehm
f80ae32846 Abstract bitcoinrpc in preparation for alt coins 2018-03-08 12:59:37 +01:00
Martin Boehm
0783cac294 Detect type of the block chain and use correct chain params 2018-03-07 09:38:19 +01:00
Martin Boehm
cac412527d Improve handling of os interrupt signal in connectBlocksParallel 2018-03-06 13:13:30 +01:00
Martin Boehm
d1c4c66c5f Implement TxCache stored in RocksDB 2018-03-06 12:36:24 +01:00
Martin Boehm
58205ed84a Store transactions in RocksDB 2018-03-05 18:14:41 +01:00
Martin Boehm
b3d1d449fd Fix minimum sync height 2018-03-01 20:20:50 +01:00
Martin Boehm
9bdb83deff Refactor sync code to own file 2018-03-01 18:37:01 +01:00
Martin Boehm
34400f5b2b Add context to errors returned by bitcoinrpc 2018-03-01 11:06:10 +01:00
Martin Boehm
da28595321 Fix incorrect memory handling 2018-02-28 12:24:29 +01:00
Martin Boehm
9dfef815b3 Tune RocksDB 2018-02-27 13:10:18 +01:00
Martin Boehm
386b4f1db8 Tune RocksDB 2018-02-26 18:42:55 +01:00
Martin Boehm
2375837d0a Tune RocksDB 2018-02-26 13:11:23 +01:00
Martin Boehm
7a1ff7241f Tune RocksDB. 
Remove bulk import - it consumes a lot of disk space.
Set no compression of data.
Reduce write buffer to .5GB from 2GB.
 2018-02-24 16:25:55 +01:00
Martin Boehm
af0e85ce43 Tune rocksdb options 2018-02-19 16:17:21 +01:00
Martin Boehm
d96af8e648 Change return values of index.GetSpentOutput method 2018-02-08 13:11:10 +01:00
Martin Boehm
43dec549d5 Implement index.GetSpentOutput method 2018-02-08 13:00:22 +01:00
Martin Boehm
ec87f4ed8d Use maximum size on disk trigger to invoke DB compaction 2018-02-06 09:43:54 +01:00
Martin Boehm
359d7ee1c4 Use constants instead of magic numbers 2018-02-06 09:12:50 +01:00
Martin Boehm
ee663944e2 Refresh iterator in DisconnectBlocks to avoid huge snapshot 2018-02-05 18:35:05 +01:00
Martin Boehm
d6e9b5caa4 Use special RocksDB options for outputs 2018-02-05 17:12:32 +01:00
Martin Boehm
97cc2aca17 Use PrepareForBulkLoad option for connectBlocksParallel 2018-02-05 12:27:41 +01:00
Martin Boehm
e31095d03f Compact database during connectBlocksParallel 2018-02-05 10:31:22 +01:00
Martin Boehm
1950f125e5 Ignore too large (> 512 bytes) output scripts 2018-02-04 23:17:54 +01:00
Martin Boehm
7c71876c12 Lookup input transactions in mempool 2018-02-03 19:14:27 +01:00
Martin Boehm
a403b8d0fe Rename package bitcoin to bchain 2018-01-31 15:23:17 +01:00
Martin Boehm
e22e1a946d Use glog for logging with verbosity levels 2018-01-30 18:22:25 +01:00
Martin Boehm
3dacffda59 Use output script (ScriptPubKey) as key to output transactions 2018-01-29 23:25:40 +01:00
Martin Boehm
2dd0b9fc21 RocksDB.GetTransactions returns transactions from inputs 2018-01-28 11:59:05 +01:00
Martin Boehm
9afa5b4187 resyncIndex DisconnectBlocks 2018-01-28 00:59:54 +01:00
Martin Boehm
48057b2453 GetBestBlock height and hash from DB 2018-01-24 18:02:46 +01:00
Martin Boehm
e72b437e23 rocksdb column families 2018-01-24 16:57:05 +01:00
Martin Boehm
6af0a39e61 accept base58 and hex addresses 2018-01-24 15:10:07 +01:00
Martin Boehm
441c0f9024 folder reorganization 2018-01-18 16:44:31 +01:00