changelog: merge blockstore and indexer changes

CHANGELOG.md

@@ -5,9 +5,19 @@
 ### How to upgrade
 
 The way that block data is stored has changed for greater performance,
-efficiency, reliability and portability. To upgrade to the new disk layout
-it's necessary to move block data from LevelDB (e.g. `~/.bcoin/chain`) to
-a new file based block storage (e.g. `~./.bcoin/blocks`).
+efficiency, reliability and portability.
+
+- Block and undo block data has been moved from LevelDB into flat files.
+- The transaction and address indexes have been moved into separate
+  LevelDB databases.
+- The transaction has been de-duplicated, and will reduce disk usage by
+  half for those running with `txindex` enabled.
+- The `txindex` and `addrindex` can now be enabled after the initial
+  block download.
+
+To upgrade to the new disk layout it's necessary to move block data
+from LevelDB (e.g. `~/.bcoin/chain`) to a new file based block
+storage (e.g. `~./.bcoin/blocks`).
 
 To do this you can run:
 ```
@@ -22,6 +32,17 @@ Alternatively, you can also sync the chain again, however the above
 migration will be faster as additional network bandwidth won't be used
 for downloading the blocks again.
 
+For those with `txindex` and `addrindex` enabled there is an additional
+step to cleanup and regenerate the indexes.
+
+``` bash
+$ ./migrate/chaindb5to6.js /path/to/bcoin/chain
+```
+
+The indexes will be regenerated by rescanning the chain on next startup,
+this process might take a while. Please take the potential downtime in
+re-indexing into account before upgrading.
+
 ### Wallet API changes
 
 #### HTTP
@@ -76,6 +97,9 @@ for downloading the blocks again.
 - The option for `coin-cache` has been removed, this setting was causing
   issues during the sync with out-of-memory errors and was making performance
   worse instead of better.
+- The database location for indexes can be configured via the
+  `--index-prefix` option. Default locations are `prefix` + `/index`
+  (e.g. `~/.bcoin/testnet/index/tx` and `~/.bcoin/testnet/index/addr`).
 
 ### Script changes
 
@@ -97,6 +121,22 @@ for downloading the blocks again.
 - Config file `wallet.conf` won't be read during test runs that was
   causing issues with some testing environments.
 
+### Chain changes
+
+- The method `getSpentView` accepts a `TXMeta` instead of `TX`.
+- The transaction index methods are now implemented at `node.txindex`:
+  - `getMeta(hash)`
+  - `getTX(hash)`
+  - `hasTX(hash)`
+  - `getSpentView(tx)`
+- The address index methods are now implemented at `node.addrindex`:
+  - `getCoinsByAddress(addrs)`
+  - `getHashesByAddress(addrs)`
+- The following methods require `getHashesByAddress` in conjunction with
+  `node.txindex.getTX` and `node.txindex.getMeta` respectively.
+  - `getTXByAddress(addrs)`
+  - `getMetaByAddress(addrs)`
+
 ### Other changes
 
 - A new module for storing block data in files.
@@ -112,72 +152,6 @@ for downloading the blocks again.
 - Updates to dependencies including `bcrypto` to version > 3.
 - Various small fixes to run bcoin in a browser.
 
-## v1.x.x
-
-### Migration
-
-The chain indexing subsystem has been refactored to be more modular and
-flexible.
-
-A migration is required to cleanup the old indexes, if present.
-
-``` bash
-$ ./migrate/chaindb4to5.js ~/.bcoin/chain
-```
-
-**Note**: if enabled, tx and addr indexes will be regenerated by rescanning the
-chain on next startup, this process might take a while. Please take the
-potential downtime in re-indexing into account before upgrading.
-
-Indexing has been made extensible so that new indexers such as a filter index
-for BIP 157 can be implemented easily.
-
-Users can toggle any indexing on/off anytime before or after the initial sync.
-The indexer will start resyncing the chain state and replaying blocks to
-process them for indexing. Once caught up, it will just index new blocks.
-
-An index can be dropped by just deleting the corresponding database.
-
-### Notable Changes
-
-- `__lib/indexer__` `Indexer` implements the base methods which are common to
-  all indexers, including setting up the database, handling chain events such
-  as new block etc.
-
-- By default, bcoin ships `TXIndexer`, `AddrIndexer` implementations. These
-  indexers preserve all the existing indexing functionality and can be enabled
-  via the same flags i.e. `--index-tx` `--index-address`, for compatibility.
-
-- `Indexer` emits a `chain tip` with `[tip]`, where tip is an instance of
-  `BlockMeta`, when it is caught up with the chain.
-
-- Database location can be configured via `--index-prefix` config option.
-  Default locations are `prefix` + `/index` e.g.: `~/.bcoin/testnet/index/tx`,
-  `~/.bcoin/testnet/index/addr`.
-
-- `__/lib/blockchain/chain__` - `getSpentView` accepts a `TXMeta` insted of `TX`
-
-- `__/lib/blockchain/chain__` - the following methods have been moved out of
-  the chain to the indexers. Using the methods on the chain is deprecated:
-
-  `node.txindex` implements:
-
-  + `getMeta(hash)`
-  + `getTX(hash)`
-  + `hasTX(hash)`
-  + `getSpentView(tx)`
-
-  `node.addrindex` implements:
-
-  + `getCoinsByAddress(addrs)`
-  + `getHashesByAddress(addrs)`
-
-  The following methods require `getHashesByAddress` in conjunction with
-  `node.txindex.getTX` and `node.txindex.getMeta` respectively.
-
-  + `getTXByAddress(addrs)`
-  + `getMetaByAddress(addrs)`
-
 ## v1.0.0
 
 ### Migration