fcoin/CHANGELOG.md

# Bcoin release notes & changelog

## v2.0.0-dev

### How to upgrade

The way that block data is stored has changed for greater performance,
efficiency, reliability and portability.

- Block and undo block data has been moved from LevelDB into flat files.
- The transaction and address indexes have been moved into separate
  LevelDB databases.
- The transaction has been de-duplicated, and will reduce disk usage by
  half for those running with `txindex` enabled.
- The `txindex` and `addrindex` can now be enabled after the initial
  block download.
- The `addrindex` has been sorted to support querying for large sets
  of results, and will no longer cause CPU and memory exhaustion issues.

To upgrade to the new disk layout it's necessary to move block data
from LevelDB (e.g. `~/.bcoin/chain`) to a new file based block
storage (e.g. `~./.bcoin/blocks`).

To do this you can run:
```
node ./migrate/chaindb4to5.js /path/to/bcoin/chain
```

The migration will take 1-3 hours, depending on hardware. The block data
will now be stored at `/path/to/bcoin/blocks`, after the data has been moved
the chain database will be compacted to free disk space.

Alternatively, you can also sync the chain again, however the above
migration will be faster as additional network bandwidth won't be used
for downloading the blocks again.

For those with `txindex` and `addrindex` enabled there is an additional
step to cleanup and regenerate the indexes.

``` bash
$ ./migrate/chaindb5to6.js /path/to/bcoin/chain
```

The indexes will be regenerated by rescanning the chain on next startup,
this process might take a while. Please take the potential downtime in
re-indexing into account before upgrading.

### Wallet API changes

#### HTTP

- `PUT /wallet/:id` Creating a watch-only wallet now requires an `accountKey`
  argument. This is to prevent bcoin from generating keys and addresses the
  user can not spend from.
- `POST /wallet/:id/create`
  - Now has a `sign` argument for optional signing of transactions.
  - Now has a `template` option, that will skip templating inputs when
  `sign = false`, but you can enable it if necessary. It does not have an
  effect when `sign = true`.
  - Exposes `blocks`, which can will be used if there is no `rate` option.
  - Exposes `sort` (Default `true`), that can be used to disable BIP69 sorting.

#### RPC

- Bug fix addresses for the `getnewaddress` command with various networks.
- Deprecate the `ismine` and `iswatchonly` fields from the `validateaddress`
  command and add `isscript`, `iswitness`, `ischange`, `witness_version`
  and `witness_program` to partially match the v0.18.0 Bitcoin Core release
  (26a2000b0177fd2668b7d82e5aa52829cf2bfdf6)
- Add wallet RPC `getaddressinfo` to return `ismine` and `iswatchonly`
  with the correct values instead of their previous values which were
  hardcoded to false. Also returns `address`, `scriptPubKey`, `isscript`,
  `iswitness`, `witness_version` and `witness_program`.
  (a28ffa272a3c4d90d0273d9aa223a23becc08e0e)

### Node API changes

#### HTTP

Several CPU and memory exhaustion issues have been resolved with some
additional arguments for querying multiple sets of results for addresses
that have many transactions.

- `GET /tx/address/:address` has several new arguments: `after`, `reverse`
  and `limit`. The `after` argument is a txid, for querying additional results
  after a previous result. The `reverse` argument will change the order that
  results are returned, the default order is oldest to latest. The `limit`
  argument can be used to give results back in smaller sets if necessary.
- `POST /tx/address` This has been deprecated, instead query for each address
  individually with `GET /tx/address/:address` with the expectation that
  there could be _many_ results that would additionally need to be queried
  in a subsequent query using the `after` argument to request the next set.
- `POST /coin/address` and `GET /coin/address/:address` are deprecated as
  coins can be generated using results from `/tx/address/:address` and
  querying by only a range of the latest transactions to stay synchronized.
  Coins could otherwise be removed from results at any point, and thus the
  entire set of results would need to be queried every time to discover
  which coins have been spent and are currently available.

### Network changes

- Regtest params have been updated to correspond with other bitcoin
  implementations for cross testing. Key prefixes have been updated and
  segwit is always active (39684df6051b599da9ed1383cb7e3f8605806a74). The
  regtest seeds have also been cleared so that the node will not attempt
  to connect to itself.
- Testnet seeds have been updated (8b6eba165b090edfc1ff5dbc9c422758d60b6342)

### Logging changes

- Wallet and node HTTP and RPC is now more clearly identified. The wallet
  HTTP server will log with the context `wallet-http` and the RPC as
  `wallet-rpc`. The node HTTP will log with the context `node-http` and the
  RPC as `node-rpc`. There was also a reduction in the duplicate logs in
  the case of an error for the node RPC, and the addition of method logging
  for the wallet RPC.
- There is now also additional logging during wallet rescans.

### Configuration changes

- Pool options now has a `--discover` option exposed, and the `--only` node
  option will disable discovery of new nodes.
- The option for `coin-cache` has been removed, this setting was causing
  issues during the sync with out-of-memory errors and was making performance
  worse instead of better.
- The database location for indexes can be configured via the
  `--index-prefix` option. Default locations are `prefix` + `/index`
  (e.g. `~/.bcoin/testnet/index/tx` and `~/.bcoin/testnet/index/addr`).

### Script changes

- Script has been updated for policy flags. For example
  `VERIFY_CONST_SCRIPTCODE` to disable `OP_CODESEPARATOR` in non-segwit
  scripts and make a positive `findAndDelete` result invalid.
- Segwit scripts that terminate with a stack size not equal to one
  now has as error of `CLEANSTACK` instead of `EVAL_FALSE`.
- `VERIFY_UPGRADABLE_NOPS` now does not apply to `OP_CHECKLOCKTIMEVERIFY`
  and `OP_CHECKSEQUENCEVERIFY`.

### Testing changes

- Switched to use a security focused rewrite of `mocha` called
  `bmocha`, for further details see: https://github.com/bcoin-org/bmocha
- Data has been updated to be in sync with other implementations
  around policy-only script validation.
- Tests now cleanly close with all timers being cleared.
- Config file `wallet.conf` won't be read during test runs that was
  causing issues with some testing environments.

### Chain changes

- The method `getSpentView` accepts a `TXMeta` instead of `TX`.
- The transaction index methods are now implemented at `node.txindex`:
  - `getMeta(hash)`
  - `getTX(hash)`
  - `hasTX(hash)`
  - `getSpentView(tx)`
- The address index method is now implemented at `node.addrindex`:
  - `getHashesByAddress(addrs)`
- The following methods require `getHashesByAddress` in conjunction with
  `node.txindex.getTX` and `node.txindex.getMeta` respectively.
  - `getTXByAddress(addrs)`
  - `getMetaByAddress(addrs)`
- The following method has been deprecated:
  - `getCoinsByAddress(addrs)`

### Other changes

- A new module for storing block data in files.
- Use of `buffer-map` for storing hashes
  (see https://github.com/bcoin-org/bcoin/issues/533).
- Use of `bsert` for assertions.
- `SIGINT` handling will close the full node, spvnode and wallet.
- Using the bcoin library in a REPL now has auto-completion by pressing tab.
- Various documentation updates.
- Mempool fix to add non-standard (non-segwit) to reject cache.
- A lockfile is now included and all dependencies integrity verified
  with sha512 hash and not the vulnerable sha1 hash.
- Updates to dependencies including `bcrypto` to version > 3.
- Various small fixes to run bcoin in a browser.

## v1.0.0

### Migration

The latest bcoin release contains almost a total rewrite of the wallet. A
migration is required.

Bcoin has also been modularized quite a bit. Users installing globally from npm
should be sure to run:

``` bash
$ npm update -g bcoin
```

For users who cloned the respository themselves, it might be wise to simply
remove the `node_modules` directory and do a fresh `npm install`.

``` bash
$ cd bcoin/
$ git pull origin master
$ rm -rf node_modules/
$ npm install
$ npm test
```

---

Once upgrading is complete, a migration script must be executed. The migration
script resides in `./migrate/latest` as an executable file.

A bcoin __prefix directory__ must be passed as the first argument.

Example:

``` bash
$ ./migrate/latest ~/.bcoin
```

What this does:

- Renames `chain.ldb` to `chain`.
- Renames `spvchain.ldb` to `spvchain`.
- Renames `walletdb.ldb` to `wallet`.
- Runs a migration (`chaindb3to4.js`) on `chain`.
- Runs a migration (`chaindb3to4.js`) on `spvchain`.
- Runs a migration (`walletdb6to7.js`) on `wallet`.

The chain migration should be minor, and only taxing if you have
`--index-address` enabled.

If you have a testnet, regtest, or simnet directory, the migration must also be
run on these, e.g.

``` bash
$ ./migrate/latest ~/.bcoin/testnet
```

Also note that the `--prefix` argument now __always__ creates a fully-fledged
prefix directory. For example `--prefix ~/.bcoin_whatever --network testnet`
will create a directory structure of `~/.bcoin_whatever/testnet/` instead of
`~/.bcoin_whatever`. Please update your directory structure accordingly.

### Configuration changes

Wallet and Node are now separated and use different persistent configuration files.
Some configuration options have moved to `wallet.conf`, which is stored in the prefix
directory next to the existing `bcoin.conf`.

All configuration options in `wallet.conf` will have the `wallet-`
prefix added implicitly. The prefix is still needed when using CLI/ENV configuration
methods. The wallet now has it's own HTTP server, so options such as
`wallet-http-host` must also be specified along with `http-host`
if you require this setting (required for Docker networking).

Wallet-specific settings such as `api-key` and `wallet-auth`
have moved to `wallet.conf`. If using CLI/ENV options, these are prefixed with `wallet-`.
Various configuration method examples are shown below.
Note some config options (eg. `network`) are automatically passed to wallet plugin
if specified through CLI/ENV, but should be specified in `wallet.conf` for `bwallet-cli`.

Example using wallet.conf:
```
network: testnet
wallet-auth: true
api-key: hunter2
http-host: 0.0.0.0
```

Example using CLI options:
`./bin/node --network=testnet --http-host=0.0.0.0 --wallet-http-host=0.0.0.0 --wallet-api-key=hunter2 --wallet-wallet-auth=true`

Example using ENV:
`BCOIN_NETWORK=simnet BCOIN_HTTP_HOST=0.0.0.0 BCOIN_WALLET_HTTP_HOST=0.0.0.0 BCOIN_WALLET_API_KEY=hunter2 BCOIN_WALLET_WALLET_AUTH=true ./bin/node`

### Bcoin client changes

The `bcoin cli` interface has been deprecated and is now replaced with a
separate tool: `bcoin-cli`. `bcoin wallet` has also been deprecated and is
replaced with `bwallet-cli`. Both `bcoin-cli` and `bwallet-cli` can be
installed with the `bclient` package in npm.

### Wallet API changes

The main wallet API changes have to do with changes in the structure of the
HTTP server itself. The bcoin wallet HTTP server will now always listen on it's
own port. Standard ports are:

```
main: 8334
testnet: 18334
regtest: 48334
simnet: 18558
```

The default account object is no longer present on the wallet. To retrieve it,
request `/wallet/:id/account/default` instead. More minor changes to the JSON
output include the removal of `id` and `wid` properties on a number of objects.
Please update your code accordingly.

For socket.io events, all `wallet ` prefixes have been removed. For example,
`wallet join` should be replaced with simply `join`. All wallet-related events
(`tx`, `confirmed`, etc) now receive the wallet ID as the first argument. The
`tx` event will receive `[id, details]` instead of `[details]`.

The `_admin` endpoints have been moved up a level. Instead of requesting
`/wallet/_admin/backup` (for example), request `/backup` instead. If
`--wallet-auth` is enabled, a new flag `--admin-token` is accepted. Admin token
is a 64 character hex string and allows access to all admin routes. For
example, `GET /backup?token=xxx...`. It also allows access to any wallet on the
HTTP or websocket layer (websockets who use the admin token can join a wallet
of `*`, which notifies them of events on _all_ wallets).

### Notable changes

- __wallet/http__ - A `conflict` event will be correctly emitted for _all_
  double spends.
- __wallet/http__ - Account balances are now indexed and will appear on account
  objects via the API.
- __bcoin__ - Bcoin has been rewritten using class syntax. Objects will not be
  inheritable with legacy prototypal inheritance.
- __pkg__ - Bcoin is once again buildable with browserify.
- __pkg__ - Numerous modules have been moved out to separate dependencies. For
  example, the `lib/crypto` module has been moved to the `bcrypto` package (and
  also heavily optimized even further).
- __bcoin__ - The main `bcoin` module has been rewritten in terms of what it
  exposes and how. Conventions have changed (`Address` instead of `address`).
  Due to the use of class syntax, something like `bcoin.Address(str)` is no
  longer possible. Please take a look at `lib/bcoin.js` to get a better idea of
  what has changed.
- __net__ - Support for BIP151 and BIP150 has been dropped.

## v1.0.0-beta.15

### Notable changes

## v1.0.0-beta.14

### Notable changes

- __pkg__ - Ignored `bcoin*` files in npmignore have been removed. This fixes
  the npm install.

## v1.0.0-beta.13

### Notable changes

- __config__ - Options using megabyte units are now calculated properly again
  (6182df044228f9215938e7d314435f3f2640acca,
  a630d23a97b68f189a85105856fedc4e9e515754,
  7728a0047053d4c368e60426e5fc7cc812d54caf).

- __address__ - Bech32 addresses are now supported
  (6acef06cbc87a3051ba238a2fb640562e718135e). This changes the semantics of
  the `Address` object: to support bech32, `Address.fromBase58` calls should be
  replaced with `Address.fromString`.  Likewise, `addr.toBase58` calls should
  be replaced with `addr.toString`

- __rpc__ - `getblockbyheight` is now exposed via JSON-RPC. It takes the same
  parameters as the `getblock` call, requiring a height instead of block hash
  (12d3ee6f9582afa9a3ba8984c63dcbc27b8db57e).

- __bin__ - `bcoin --version` and `bcoin --help` now exit with appropriate
  messages (f2f94a800e37c5dbdda6920fa6b85fbd485c212a).

- __net__ - The p2p pool now exposes an `--only` option
  (a1d0948f2e528c5d77d6502659fafd064b1e693b).

- __mempool__ - The mempool `indexAddress` option should now work correctly
  (fba9b46d253c19bbf8e662d9d75ab03dc9e20a78).

- __rpc__ - JSON-RPC calls now properly cast booleans a la bitcoin core
  (dd49ee076196d2353783e3044185165dbac4aeb9).

- __rpc__ - Various RPC calls have been improved and updated
  (c78707ed6a71ce46c41c4e43ecb505e78a84b380,
  c1e684dc12d0a86573a905d54d4f81fce921987a,
  5bde338a53117b1bd0fd92df0abc23d95180ab32).

- __rpc__ - Retroactive pruning is now available via the `pruneblockchain` call
  (f0bc6d5925419ba4a8289fa7828efc48ecc152d4).

- __http__ - Getting block by height via the Rest API now works again
  (df4c8cc68c965bd818a5004354d2652751d4a702).

- __net__ - Peers who serve invalid orphans are now punished properly
  (0ceca23cb5a3d724c79c6bf587ede5d473df8486).

- __utils__ - An implementation of GCS filters is now supported
  (b994c278f25408886c3095d0c24123baaf07f78f).

- __http__ - The `witness` option is now properly exposed on the Rest API for
  segwit wallets (f04ad612b216becd35765c6e231f7820c7eee358).

- __deps__ - Node.js >=v7.6.0 is now a required dependency
  (a0bd9680fed07c5eb37c227d160b0868f8adaf31).

- __build__ - The browser build has switched from browserify to webpack
  (19f236f74072d473123d20282d2119f6d9130458).

- __bcoin__ - The codebase has been rewritten to use all ES6 and ES7 features
  supported by node.js
  (aa05bb5df79d9a3af53060a4c0c066226f6e9e4c,
  78d62c73b82e1953999d1cf80c90ed2035d4996e,
  e00472891df5934d8fc3aa63662f852816aa86b0,
  c53f4cf89e46d9de8ab7f65430310567558fe03f,
  8c7279518f5341a2482a79ac98f0574468541edc).

- __workers__ - The worker pool has been refactored to solve the dissonance
  between the browser and node.js (27c60ce76e57af1695d78f912227d93194812c88).

- __net__ - DNS requests should now timeout sooner
  (647b6909c6d527eb82f6d789c88a23b2f8a60126).

- __http__ - Satoshi values are now required for all JSON serialization, as
  opposed to BTC strings which were used in the past
  (2f51fd1c5066f194a5a52383f4dd45497b882706).

- __bin__ - The `--no-wallet` flag is now exposed for nodes who want to run
  without a wallet.

- __chain__ - BIP91 support is now exposed via the `--bip91` option.  The
  `segwit` and `segsignal` rules will be automatically added when calling
  `getblocktemplate`. To enable bip91 on an existing database `--force-flags`
  must be passed. This will force bip91 to be enforced, but will not invalidate
  any previous blocks (`bcoin cli reset [height]` may need to be used).

- __chain__ - BIP148 support is now exposed via the `--bip148` option. This
  likewise must be enabled with `--force-flags` on existing chains. This has
  the same potential "reset" implications as the `--bip91` option.

### Migrating

This release contains a few non-backward-compatible changes.

Bcoin now requires node.js >=7.6.0 due to the use of ES6/7 features, and for
the sake of more stability.

Bcoin's rest API now assumes satoshi values for in all JSON serialization.
__This is a breaking change__ for code that is not aware of it. All code which
hits the rest API must be updated to use only satoshi values as opposed to BTC
strings.

In other words, this:

``` json
{
  "unconfirmed": "1.12",
  "confirmed": "1.12"
}
```

Becomes this:

``` json
{
  "unconfirmed": 112000000,
  "confirmed": 112000000
}
```

## v1.0.0-beta.12

### Notable changes

- __networks__ - Fixed simnet wpkh prefix.
- __http__ - `wallet join` without wallet auth has been fixed for responses.
  This was causing a hanging issue with the client.

## v1.0.0-beta.11

### Notable changes

- __networks__ - Simnet params have been fixed.
- __cli__ - Chain reset call has been fixed.

## v1.0.0-beta.10

### Notable changes

- __wallet/http__ - Create wallet route modified
  (`POST /wallet/:id?` changed to `PUT /wallet/:id`).
- __wallet/http__ - Create account route modified
  (`POST /wallet/:id/account/:account?` changed to
  `PUT /wallet/:id/account/:account`).
- __wallet/http__ - `auth` socket.io event name for wallet auth changed to
  `wallet auth`.
- __config__ - `payout-address` option was changed to `coinbase-address`.
- __node__ - Plugin system is now exposed.
  See https://github.com/bcoin-org/bcoin/pull/156.
- __config__ - The internal API for the config object has been rewritten
  and is now more reusable, particularly by node plugins.
- __http/rpc__ - Both the HTTPBase and RPCBase objects now allow "mounting" by
  other rpc and http servers.
- __wallet__ - The wallet code has been completely removed from the node, and
  now resides entirely within one module. The wallet is exposed only as a
  plugin or a separate server.
- __rpc__ - `prioritisetransaction` is now exposed properly (`deltaFee`s are
  now tracked on mempool entries).
- __rpc__ - Proper `id` and error codes are now implemented.
- __rpc__ - Several `getblocktemplate` improvements have been implemented for
  more accuracy. e.g. `curtime` will now be updated each call.
- __mining__ - The internal miner API has been rewritten, and now mimics
  stratum in a sense.
- __chain__ - Faster verification with checkpoints.
- __net__ - Fixed a potential block stalling issue.
- __net__ - Hardcoded seeds for main added. Makes
  for better shipping with browsers.
- __wsproxy/proxysocket__ - DNS resolution is no longer exposed.
- __logger__ - Log files now trim to 20mb on boot.
- __hostlist__ - A persistent `hosts` file is now written by default.

## v1.0.0-beta.9

### Notable changes

- __mempool__ - Trimming now removes dependency chains by cumulative fee rate.
- __mempool__ - Cumulative descendant fees are now updated properly when
  removing a transaction.
- __net__ - Preliminary upnp support for adding port mappings.
- __chain/mempool/miner__ - Various atomicity fixes and extra sanity checking.
- __pool/peer__ - Peer height is now tracked and exposed on the RPC as
  `bestheight`.

## v1.0.0-beta.8

### Notable changes

- __mempool__ - Fixed critical fee estimator bug causing throwing in the
  mempool.

## v1.0.0-beta.7

### Notable changes

- __http__ - Always display spent coins in tx routes (e.g. `/tx/[txid]`).
- __mempool__ - An on-disk mempool is now exposed via `--persistent-mempool`
  (also makes fee data persistent).
- __chain__ - `chain.add` now takes a `flags` parameter to avoid POW and
  non-contextual checks if necessary.
- __net__ - HostList is now potentially persistent with a `hostLocation` option.
- __net__ - Smarter stall behavior and timeouts.

## v1.0.0-beta.6

### Notable changes

- __http__ - Better bitcoind compatability for JSON-RPC.

## v1.0.0-beta.5

### Notable changes

- __miner__ - Better fee rate comparisons.
- __deps__ - Upgrade deps, fix build on arm and windows.

## v1.0.0-beta.4

### Notable changes

- __miner__ - Optimized TX sorting.
- __rpc__ - Improved getblocktemplate to provide more
  accurate results to bitcoind.

## v1.0.0-beta.3

### Notable changes

- __miner__ - Improved fee rate sorting.
- __rpc__ - Fix incompatibilities in rpc api (getblocktemplate & submitblock).

## v1.0.0-beta.2

### Notable changes

- __pool__ - Increase max header chain failures to 500 (prevents the initial
  sync from reverting to getblocks so frequently).

## v1.0.0-beta.1

### Notable changes

- __wsproxy__: Fixed proof of work handling in websocket proxy (43c491b).
- __chain__: Optimized MTP and network target calculations (1e07d1b).
- __wallet__: Implemented "smart" coin selection (304f0e7e).
- __protocol__: Increased default network fees for main (09c2357).
- __http__: Fix for handling `DELETE` http method (393dd5d).
- __miner__: Improved handling of default reserved size and sigops (f2964e0
  and 7104e4c).

## v1.0.0-beta

### Notable changes

- Initial tagged release.