From b757bd3148b42468b1ea2aac25438b892b14cdc9 Mon Sep 17 00:00:00 2001 From: Braydon Fuller Date: Mon, 11 Apr 2016 12:59:36 -0400 Subject: [PATCH] docs: update docs for bitcoind with address indexes --- README.md | 19 +-- docs/patch.md | 6 - docs/release.md | 17 +-- docs/services.md | 155 +------------------ docs/services/address.md | 136 ----------------- docs/services/bitcoind.md | 305 ++++++++++++++++++++++++++++++-------- docs/services/db.md | 113 -------------- docs/testing.md | 21 +-- 8 files changed, 261 insertions(+), 511 deletions(-) delete mode 100644 docs/patch.md delete mode 100644 docs/services/address.md delete mode 100644 docs/services/db.md diff --git a/README.md b/README.md index 2da202a5..4b87327e 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,7 @@ Bitcore Node ============ -A Bitcoin full node for building applications and services with Node.js. A node is extensible and can be configured to run additional services. At the minimum a node has native bindings to Bitcoin Core with the [Bitcoin Service](docs/services/bitcoind.md). Additional services can be enabled to make a node more useful such as exposing new APIs, adding new indexes for addresses with the [Address Service](docs/services/address.md), running a block explorer, wallet service, and other customizations. +A Bitcoin full node for building applications and services with Node.js. A node is extensible and can be configured to run additional services. At the minimum a node has an interface to [Bitcoin Core with additional indexing](https://github.com/bitpay/bitcoin/tree/0.12-bitcore) for more advanced address queries. Additional services can be enabled to make a node more useful such as exposing new APIs, running a block explorer and wallet service. ## Install @@ -10,14 +10,13 @@ npm install -g bitcore-node bitcore-node start ``` -Note: For your convenience, we distribute binaries for x86_64 Linux and x86_64 Mac OS X. Upon npm install, the binaries for your platform will be downloaded. For more detailed installation instructions, or if you want to compile the project yourself, then please see the [Build & Install](docs/build.md) documentation to build the project from source. +Note: For your convenience, we distribute bitcoind binaries for x86_64 Linux and x86_64 Mac OS X. Upon npm install, the binaries for your platform will be downloaded. For more detailed installation instructions, or if you want to compile the project yourself, then please see the Bitcore branch of [Bitcoin Core with additional indexing](https://github.com/bitpay/bitcoin/tree/0.12-bitcore). ## Prerequisites - Node.js v0.12 or v4.2 -- ~100GB of disk storage +- ~150GB of disk storage - ~4GB of RAM -- Mac OS X >= 10.9, Ubuntu >= 12.04 (libc >= 2.15 and libstdc++ >= 6.0.16) ## Configuration @@ -32,12 +31,6 @@ bitcore-node install https://github.com/yourname/helloworld This will create a directory with configuration files for your node and install the necessary dependencies. For more information about (and developing) services, please see the [Service Documentation](docs/services.md). -To start bitcore-node as a daemon: - -```bash -bitcore-node start --daemon -``` - ## Add-on Services There are several add-on services available to extend the functionality of Bitcore: @@ -49,16 +42,12 @@ There are several add-on services available to extend the functionality of Bitco ## Documentation - [Services](docs/services.md) - - [Bitcoind](docs/services/bitcoind.md) - Native bindings to Bitcoin Core - - [Database](docs/services/db.md) - The foundation API methods for getting information about blocks and transactions. - - [Address](docs/services/address.md) - Adds additional API methods for querying and subscribing to events with bitcoin addresses. + - [Bitcoind](docs/services/bitcoind.md) - Interface to Bitcoin Core - [Web](docs/services/web.md) - Creates an express application over which services can expose their web/API content -- [Build & Install](docs/build.md) - How to build and install from source - [Testing & Development](docs/testing.md) - Developer guide for testing - [Node](docs/node.md) - Details on the node constructor - [Bus](docs/bus.md) - Overview of the event bus constructor - [Errors](docs/errors.md) - Reference for error handling and types -- [Patch](docs/patch.md) - Information about the patch applied to Bitcoin Core - [Release Process](docs/release.md) - Information about verifying a release and the release process. ## Contributing diff --git a/docs/patch.md b/docs/patch.md deleted file mode 100644 index 86c366c6..00000000 --- a/docs/patch.md +++ /dev/null @@ -1,6 +0,0 @@ -# Static Library Patch -To provide native bindings to JavaScript _(or any other language for that matter)_, Bitcoin code, itself, must be linkable. Currently, Bitcoin Core provides a JSON RPC interface to bitcoind as well as a shared library for script validation _(and hopefully more)_ called libbitcoinconsensus. There is a node module, [node-libbitcoinconsensus](https://github.com/bitpay/node-libbitcoinconsensus), that exposes these methods. While these interfaces are useful for several use cases, there are additional use cases that are not fulfilled, and being able to implement customized interfaces is necessary. To be able to do this a few simple changes need to be made to Bitcoin Core to compile as a static library. - -The patch is located at `etc/bitcoin.patch` and adds a configure option `--enable-daemonlib` to compile all object files with `-fPIC` (Position Independent Code - needed to create a shared object), exposes leveldb variables and objects, exposes the threadpool to the bindings, and conditionally includes the main function. - -Every effort will be made to ensure that this patch stays up-to-date with the latest release of Bitcoin. At the very least, this project began supporting Bitcoin Core v0.11. diff --git a/docs/release.md b/docs/release.md index 29deb20f..e2ce5313 100644 --- a/docs/release.md +++ b/docs/release.md @@ -1,11 +1,11 @@ # Release Process -Binaries for the C++ binding file (which includes libbitcoind statically linked in) are distributed for convenience. The binary binding file `bitcoind.node` is signed and published to S3 for later download and installation. Source files can also be built if binaries are not desired. +Binaries for bitcoind are distributed for convenience and built deterministically with Gitian. ## How to Verify Signatures ``` -cd build/Release -gpg --verify bitcoind.node.sig bitcoind.node +cd bin +gpg --verify bitcoin-0.12.0-linux64.tar.gz.sig bitcoin-0.12.0-linux64.tar.gz ``` To verify signatures, use the following PGP keys: @@ -14,7 +14,7 @@ To verify signatures, use the following PGP keys: - @pnagurny: [https://pgp.mit.edu/pks/lookup?op=get&search=0x0909B33F0AA53013](https://pgp.mit.edu/pks/lookup?op=get&search=0x0909B33F0AA53013) ## How to Release -Ensure you've followed the instructions in the README.md for building the project from source. When building for any platform, be sure to keep in mind the minimum supported C and C++ system libraries and build from source using this library. Example, Ubuntu 12.04 has the earliest system library for Linux that we support, so it would be easiest to build the Linux artifact using this version. You will be using node-gyp to build the C++ bindings. A script will then upload the bindings to S3 for later use. You will also need credentials for BitPay's bitcore-node S3 bucket and be listed as an author for the bitcore-node's npm module. +Ensure you've followed the instructions in the README.md for building the project from source. When building for any platform, be sure to keep in mind the minimum supported C and C++ system libraries and build from source using this library. Example, Ubuntu 12.04 has the earliest system library for Linux that we support, so it would be easiest to build the Linux artifact using this version. A script will then upload the binaries to S3 for later use. You will also need credentials for BitPay's bitcore-node S3 bucket and be listed as an author for the bitcore-node's npm module. - Create a file `.bitcore-node-upload.json` in your home directory - The format of this file should be: @@ -28,7 +28,7 @@ Ensure you've followed the instructions in the README.md for building the projec When publishing to npm, the .gitignore file is used to exclude files from the npm publishing process. Be sure that the bitcore-node directory has only the directories and files that you would like to publish to npm. You might need to run the commands below on each platform that you intend to publish (e.g. Mac and Linux). -To make a release, bump the `version` and `lastBuild` of the `package.json`: +To make a release, bump the `version` of the `package.json`: ```bash git checkout master @@ -40,13 +40,6 @@ npm run upload npm publish ``` -And then update the `version` of the `package.json` for development (e.g. "0.3.2-dev"): - -```bash -git commit -a -m "Bump development version to " -git push upstream master -``` - Create a release tag and push it to the BitPay Github repo: ```bash diff --git a/docs/services.md b/docs/services.md index 34f948f2..e4511388 100644 --- a/docs/services.md +++ b/docs/services.md @@ -10,7 +10,7 @@ The `bitcore-node.json` file describes which services will load for a node: ```json { "services": [ - "bitcoind", "db", "address", "insight-api" + "bitcoind", "web" ] } ``` @@ -37,9 +37,7 @@ If, instead, you would like to run a custom node, you can include services by in var bitcore = require('bitcore-node'); //Services -var Address = bitcore.services.Address; var Bitcoin = bitcore.services.Bitcoin; -var DB = bitcore.services.DB; var Web = bitcore.services.Web; var myNode = new bitcore.Node({ @@ -48,21 +46,11 @@ var myNode = new bitcore.Node({ name: 'livenet' }, "services": [ - { - name: "address", - module: Address, - config: {} - }, { name: 'bitcoind', module: Bitcoin, config: {} }, - { - name: 'db', - module: DB, - config: {} - }, { name: 'web', module: Web, @@ -77,8 +65,8 @@ var myNode = new bitcore.Node({ Now that you've loaded your services you can access them via `myNode.services..`. For example if you wanted to check the balance of an address, you could access the address service like so. ```js -myNode.services.address.getBalance('1HB5XMLmzFVj8ALj6mfBsbifRoD4miY36v', false, function(err, total) { - console.log(total); //Satoshi amount of this address +myNode.services.bitcoind.getBalance('1HB5XMLmzFVj8ALj6mfBsbifRoD4miY36v', false, function(err, total) { + console.log(total.balance); //Satoshi amount of this address }); ``` @@ -96,140 +84,3 @@ The `package.json` for the service module can either export the `Node.Service` d Please take a look at some of the existing services for implementation specifics. -### Adding an index -One quite useful feature exposed to services is the ability to index arbitrary data in the blockchain. To do so we make use of leveldb, a simple key-value store. As a service we can expose a 'blockHandler' function which is called each time a new block is added or removed from the blockchain. This gives us access to every new transaction received, allowing us to index them. Let's take a look at an example where we will index the time that a transaction was confirmed. - -```js -//Index prefix, so that we can determine the difference between our index -//and the indexes provided by other services -MyService.datePrefix = new Buffer('10', 'hex'); - -MyService.minPosition = new Buffer('00000', 'hex'); -MyService.maxPosition = new Buffer('99999', 'hex'); - -//This function is automatically called when a block is added or receieved -MyService.prototype.prototype.blockHandler = function(block, addOutput, callback) { - - //Determine if the block is added or removed, and therefore whether we are adding - //or deleting indexes - var databaseAction = 'put'; - if (!addOutput) { - databaseAction = 'del'; - } - - //An array of all leveldb operations we will be committing - var operations = []; - - //Timestamp of the current block - var blocktime = new Buffer(4); - blocktime.writeUInt32BE(block.header.time); - - for (var i = 0; i < block.transactions.length; i++) { - var transaction = block.transactions[i]; - var txid = new Buffer(transaction.id, 'hex'); - var position = new Buffer(('0000' + i).slice(-5), 'hex'); - - //To be able to query this txid by the block date we create an index, leading with the prefix we - //defined earlier, the the current blocktime, and finally a differentiator, in this case the index - //of this transaction in the block's transaction list - var indexOperation = { - type: databaseAction, - key: Buffer.concat([this.datePrefix, blockTime, position]), - value: txid - }; - - //Now we push this index into our list of operations that should be performed - operations.push(indexOperation); - } - - //Send the list of db operations back so they can be performed - setImmediate(function() { - callback(null, operations); - }); -}; -``` - -### Retrieving data using an index -With our block handler code every transaction in the blockchain will now be indexed. However, if we want to query this data we need to add a method to our service to expose it. - -```js - -MyService.prototype.getTransactionIdsByDate = function(startDateBuffer, endDateBuffer, callback) { - - var error; - var transactions = []; - - //Read data from leveldb which is between our startDate and endDate - var stream = this.node.services.db.store.createReadStream({ - gte: Buffer.concat([ - MyService.datePrefix, - startDateBuffer, - MyService.minPosition - ]), - lte: Buffer.concat([ - MyService.datePrefix, - endDateBuffer, - MyService.maxPosition - ]), - valueEncoding: 'binary', - keyEncoding: 'binary' - }); - - stream.on('data', function(data) { - transactions.push(data.value.toString('hex')); - }); - - stream.on('error', function(streamError) { - if (streamError) { - error = streamError; - } - }); - - stream.on('close', function() { - if (error) { - return callback(error); - } - callback(null, transactions); - }); -}; -``` - -If you're new to leveldb and would like to better understand how createReadStream works you can find [more information here](https://github.com/Level/levelup#dbcreatereadstreamoptions). - -### Understanding indexes -You may notice there are several pieces to the index itself. Let's take a look at each piece to make them easier to understand. - -#### Prefixes -Since leveldb is just a simple key-value store we need something to differentiate which keys are part of which index. If we had two services trying to index on the same key, say a txid, they would overwrite each other and their queries would return results from the other index. By introducing a unique prefix per index type that we can prepend our indexes with prevents these collisions. - -```js -//A simple example of indexing the number of inputs and ouputs given a transaction id - -/** Wrong way **/ -var index1key = new Buffer(transaction.id, 'hex'); -var index1value = transaction.inputs.length; - -//Since this key has the same value it would just overwrite index1 when we write to the db -var index2key = new Buffer(transaction.id, 'hex'); -var index2value = transaction.outputs.length; - - -/** Right way **/ -var index1prefix = new Buffer('11', 'hex'); -var index2prefix = new Buffer('12', 'hex'); - -var index1key = Buffer.concat([index1prefix, new Buffer(transaction.id, 'hex')]); -var index1value = transaction.inputs.length; - -//Now that the keys are different, this won't overwrite the index -var index2key = Buffer.concat([index2prefix, new Buffer(transaction.id, 'hex')]); -var index2value = transaction.outputs.length; -``` - -Remember that all indexes are global, so check to make sure no other services you are using make use of the same prefix you plan to use in your service. We recommend documenting which prefixes you use and that you check for collisions with popular services if you plan to release your service for others to use. - -#### Index Key -The index key is the value you want to query by. This value should be deterministic so that it can be removed in the case of a [re-org](https://en.bitcoin.it/wiki/Chain_Reorganization) resulting in a block removal. The value should be unique, as no two indexes can be the same value. If you need two indexes with the same key value, consider adding a deterministic differentiator, such as a position in an array, or instead storing multiple values within the same index data. - -#### Index Data -This is the data which is returned when you search by the index's key. This can be whatever you would like to retrieve. Try to be efficient by not storing data that is already available elsewhere, such as storing a transaction ID instead of an entire transaction. diff --git a/docs/services/address.md b/docs/services/address.md deleted file mode 100644 index 7f46d3ad..00000000 --- a/docs/services/address.md +++ /dev/null @@ -1,136 +0,0 @@ -# Address Service -The address service builds on the [Bitcoin Service](bitcoind.md) and the [Database Service](db.md) to add additional functionality for querying and subscribing to information based on bitcoin addresses. This will typically represent the core functionality for wallet applications. - -## API Documentation -These methods are exposed over the JSON-RPC interface and can be called directly from a node via: - -```js -node.services.address. -``` - -**Get Unspent Outputs** - -One of the most common uses will be to retrieve unspent outputs necessary to create a transaction, here is how to get the unspent outputs for an address: - -```js -var address = 'mgY65WSfEmsyYaYPQaXhmXMeBhwp4EcsQW'; -var includeMempool = true; -node.services.address.getUnspentOutputs(address, includeMempool, function(err, unspentOutputs) { - // see below -}); -``` - -The `unspentOutputs` will have the format: - -```js -[ - { - address: 'mgY65WSfEmsyYaYPQaXhmXMeBhwp4EcsQW', - txid: '9d956c5d324a1c2b12133f3242deff264a9b9f61be701311373998681b8c1769', - outputIndex: 1, - height: 150, - satoshis: 1000000000, - script: '76a9140b2f0a0c31bfe0406b0ccc1381fdbe311946dadc88ac', - confirmations: 3 - } -] -``` - -**View Balances** - -```js -var address = 'mgY65WSfEmsyYaYPQaXhmXMeBhwp4EcsQW'; -var includeMempool = true; -node.services.address.getBalance(address, includeMempool, function(err, balance) { - // balance will be in satoshis -}); -``` - -**View Address History** - -This method will give history of an address limited by a range of block heights by using the "start" and "end" arguments. The "start" value is the more recent, and greater, block height. The "end" value is the older, and lesser, block height. This feature is most useful for synchronization as previous history can be omitted. Furthermore for large ranges of block heights, results can be paginated by using the "from" and "to" arguments. - -```js -var addresses = ['mgY65WSfEmsyYaYPQaXhmXMeBhwp4EcsQW']; -var options = { - start: 345000, - end: 344000, - queryMempool: true -}; -node.services.address.getAddressHistory(addresses, options, function(err, history) { - // see below -}); -``` - -The history format will be: - -```js -{ - totalCount: 1, // The total number of items within "start" and "end" - items: [ - { - addresses: { - 'mgY65WSfEmsyYaYPQaXhmXMeBhwp4EcsQW': { - inputIndexes: [], - outputIndexes: [0] - } - }, - satoshis: 1000000000, - height: 150, // the block height of the transaction - confirmations: 3, - timestamp: 1442948127, // in seconds - fees: 191, - tx: // the populated transaction - } - ] -} -``` - -**View Address Summary** - -```js -var address = 'mgY65WSfEmsyYaYPQaXhmXMeBhwp4EcsQW'; -var options = { - noTxList: false -}; - -node.services.address.getAddressSummary(address, options, function(err, summary) { - // see below -}); -``` - -The `summary` will have the format (values are in satoshis): - -```js -{ - totalReceived: 1000000000, - totalSpent: 0, - balance: 1000000000, - unconfirmedBalance: 1000000000, - appearances: 1, // number of transactions - unconfirmedAppearances: 0, - txids: [ - '3f7d13efe12e82f873f4d41f7e63bb64708fc4c942eb8c6822fa5bd7606adb00' - ] -} -``` - -## Events -For details on instantiating a bus for a node, see the [Bus Documentation](../bus.md). -- Name: `address/transaction`, Arguments: `[address, address...]` -- Name: `address/balance`, Arguments: `[address, address...]` - -**Examples:** - -```js -bus.subscribe('address/transaction', ['13FMwCYz3hUhwPcaWuD2M1U2KzfTtvLM89']); -bus.subscribe('address/balance', ['13FMwCYz3hUhwPcaWuD2M1U2KzfTtvLM89']); - -bus.on('address/transaction', function(transaction) { - -}); - -bus.on('address/balance', function(balance) { - -}); -``` diff --git a/docs/services/bitcoind.md b/docs/services/bitcoind.md index 922965c4..d535d52b 100644 --- a/docs/services/bitcoind.md +++ b/docs/services/bitcoind.md @@ -1,20 +1,120 @@ # Bitcoin Service -The Bitcoin Service adds a native [Node.js](https://nodejs.org) interface to [Bitcoin Core](https://github.com/bitcoin/bitcoin) for querying information about the Bitcoin blockchain. Bindings are linked to Bitcoin Core compiled as a static library. + +The Bitcoin Service is a Node.js interface to [Bitcoin Core](https://github.com/bitcoin/bitcoin) for querying information about the bitcoin block chain. It will manage starting and stopping `bitcoind` or connect to several running `bitcoind` processes. It uses a branch of a [branch of Bitcoin Core](https://github.com/bitpay/bitcoin/tree/0.12-bitcore) with additional indexes for querying information about addresses and blocks. Results are cached for performance and there are several additional API methods added for common queries. + +## Configuration + +The default configuration will include a "spawn" configuration in "bitcoind". This defines the location of the block chain database and the location of the `bitcoind` daemon executable. The below configuration points to a local clone of `bitcoin`, and will start `bitcoind` automatically with your Node.js application. + +```json + "servicesConfig": { + "bitcoind": { + "spawn": { + "datadir": "/home/bitcore/.bitcoin", + "exec": "/home/bitcore/bitcoin/src/bitcoind" + } + } + } +``` + +It's also possible to connect to separately managed `bitcoind` processes with round-robin quering, for example: + +```json + "servicesConfig": { + "bitcoind": { + "connect": [ + { + "rpchost": "127.0.0.1", + "rpcport": 30521, + "rpcuser": "bitcoin", + "rpcpassword": "local321", + "zmqpubrawtx": "tcp://127.0.0.1:30611" + }, + { + "rpchost": "127.0.0.1", + "rpcport": 30522, + "rpcuser": "bitcoin", + "rpcpassword": "local321", + "zmqpubrawtx": "tcp://127.0.0.1:30622" + }, + { + "rpchost": "127.0.0.1", + "rpcport": 30523, + "rpcuser": "bitcoin", + "rpcpassword": "local321", + "zmqpubrawtx": "tcp://127.0.0.1:30633" + } + ] + } + } +``` + +**Note**: For detailed example configuration see [`regtest/cluster.js`](regtest/cluster.js) + ## API Documentation -These methods are currently only available via directly interfacing with a node: +Methods are available by directly interfacing with the service: ```js node.services.bitcoind. ``` +### Chain + +**Getting Latest Blocks** + +```js +// gives the block hashes within a range of timestamps +var high = 1460393372; // Mon Apr 11 2016 12:49:25 GMT-0400 (EDT) +var low = 1460306965; // Mon Apr 10 2016 12:49:25 GMT-0400 (EDT) +node.services.bitcoind.getBlockHashesByTimestamp(high, low, function(err, blockHashes) { + //... +}); + +// get the current tip of the chain +node.services.bitcoind.getBestBlockHash(function(err, blockHash) { + //... +}) +``` + +**Getting Synchronization and Node Status** + +```js +// gives a boolean if the daemon is fully synced (not the initial block download) +node.services.bitcoind.isSynced(function(err, synced) { + //... +}) + +// gives the current estimate of blockchain download as a percentage +node.services.bitcoind.syncPercentage(function(err, percent) { + //... +}); + +// gives information about the chain including total number of blocks +node.services.bitcoind.getInfo(function(err, info) { + //... +}); +``` + +**Generate Blocks** + +```js +// will generate a block for the "regtest" network (development purposes) +var numberOfBlocks = 10; +node.services.bitcoind.generateBlock(numberOfBlocks, function(err, blockHashes) { + //... +}); +``` + +### Blocks and Transactions + **Getting Block Information** -It's possible to query blocks by both block hash and by height. Blocks are given as Node.js buffers and can be parsed via Bitcore: +It's possible to query blocks by both block hash and by height. Blocks are given as Node.js Buffers and can be parsed via Bitcore: ```js var blockHeight = 0; -node.services.bitcoind.getBlock(blockHeight, function(err, blockBuffer) { +node.services.bitcoind.getRawBlock(blockHeight, function(err, blockBuffer) { if (err) { throw err; } @@ -22,36 +122,40 @@ node.services.bitcoind.getBlock(blockHeight, function(err, blockBuffer) { console.log(block); }; -// check if the block is part of the main chain -var mainChain = node.services.bitcoind.isMainChain(block.hash); -console.log(mainChain); +// get a bitcore object of the block (as above) +node.services.bitcoind.getBlock(blockHash, function(err, block) { + //... +}; -// get only the block index (including chain work and previous hash) -var blockIndex = node.services.bitcoind.getBlockIndex(blockHeight); -console.log(blockIndex); +// get only the block header and index (including chain work, height, and previous hash) +node.services.bitcoind.getBlockHeader(blockHeight, function(err, blockHeader) { + //... +}); ``` **Retrieving and Sending Transactions** -Get a transaction asynchronously by reading it from disk, with an argument to optionally not include the mempool: +Get a transaction asynchronously by reading it from disk: ```js var txid = '7426c707d0e9705bdd8158e60983e37d0f5d63529086d6672b07d9238d5aa623'; -var queryMempool = true; -node.services.bitcoind.getTransaction(txid, queryMempool, function(err, transactionBuffer) { +node.services.bitcoind.getRawTransaction(txid, function(err, transactionBuffer) { if (err) { throw err; } var transaction = bitcore.Transaction().fromBuffer(transactionBuffer); }); +// get a bitcore object of the transaction (as above) +node.services.bitcoind.getTransaction(txid, function(err, transaction) { + //... +}); // also retrieve the block timestamp and height -node.services.bitcoind.getTransactionWithBlockInfo(txid, queryMempool, function(err, info) { - console.log(info.blockHash); - console.log(info.height); - console.log(info.timestamp); // in seconds - var transaction = bitcore.Transaction().fromBuffer(transactionBuffer); +node.services.bitcoind.getTransactionWithBlockInfo(txid, function(err, transaction) { + console.log(transaction.__blockHash); + console.log(transaction.__height); + console.log(transaction.__timestamp); // in seconds }); ``` @@ -59,71 +163,154 @@ Send a transaction to the network: ```js var numberOfBlocks = 3; -var feesPerKilobyte = node.services.bitcoind.estimateFee(numberOfBlocks); // in satoshis +node.services.bitcoind.estimateFee(numberOfBlocks, function(err, feesPerKilobyte) { + //... +}); -try { - node.services.bitcoind.sendTransaction(transaction.serialize()); -} catch(err) { - // handle error +node.services.bitcoind.sendTransaction(transaction.serialize(), function(err, hash) { + //... +}); +``` + +### Addresses + +**Get Unspent Outputs** + +One of the most common uses will be to retrieve unspent outputs necessary to create a transaction, here is how to get the unspent outputs for an address: + +```js +var address = 'mgY65WSfEmsyYaYPQaXhmXMeBhwp4EcsQW'; +node.services.bitcoind.getAddressUnspentOutputs(address, options, function(err, unspentOutputs) { + // see below +}); +``` + +The `unspentOutputs` will have the format: + +```js +[ + { + address: 'mgY65WSfEmsyYaYPQaXhmXMeBhwp4EcsQW', + txid: '9d956c5d324a1c2b12133f3242deff264a9b9f61be701311373998681b8c1769', + outputIndex: 1, + height: 150, + satoshis: 1000000000, + script: '76a9140b2f0a0c31bfe0406b0ccc1381fdbe311946dadc88ac', + confirmations: 3 + } +] +``` + +**View Balances** + +```js +var address = 'mgY65WSfEmsyYaYPQaXhmXMeBhwp4EcsQW'; +node.services.bitcoind.getAddressBalance(address, options, function(err, balance) { + // balance will be in satoshis with "received" and "balance" +}); +``` + +**View Address History** + +This method will give history of an address limited by a range of block heights by using the "start" and "end" arguments. The "start" value is the more recent, and greater, block height. The "end" value is the older, and lesser, block height. This feature is most useful for synchronization as previous history can be omitted. Furthermore for large ranges of block heights, results can be paginated by using the "from" and "to" arguments. + +```js +var addresses = ['mgY65WSfEmsyYaYPQaXhmXMeBhwp4EcsQW']; +var options = { + start: 345000, + end: 344000, + queryMempool: true +}; +node.services.bitcoind.getAddressHistory(addresses, options, function(err, history) { + // see below +}); +``` + +The history format will be: + +```js +{ + totalCount: 1, // The total number of items within "start" and "end" + items: [ + { + addresses: { + 'mgY65WSfEmsyYaYPQaXhmXMeBhwp4EcsQW': { + inputIndexes: [], + outputIndexes: [0] + } + }, + satoshis: 1000000000, + height: 150, // the block height of the transaction + confirmations: 3, + timestamp: 1442948127, // in seconds + fees: 191, + tx: // the populated transaction + } + ] } ``` -Get all of the transactions in the mempool: +**View Address Summary** ```js -var mempool = node.services.bitcoind.getMempoolTransactions(); -var transactions = []; -for (var i = 0; i < mempool.length; i++) { - transactions.push(bitcore.Transaction().fromBuffer(transactions[i])); +var address = 'mgY65WSfEmsyYaYPQaXhmXMeBhwp4EcsQW'; +var options = { + noTxList: false +}; + +node.services.bitcoind.getAddressSummary(address, options, function(err, summary) { + // see below +}); +``` + +The `summary` will have the format (values are in satoshis): + +```js +{ + totalReceived: 1000000000, + totalSpent: 0, + balance: 1000000000, + unconfirmedBalance: 1000000000, + appearances: 1, // number of transactions + unconfirmedAppearances: 0, + txids: [ + '3f7d13efe12e82f873f4d41f7e63bb64708fc4c942eb8c6822fa5bd7606adb00' + ] } ``` -Determine if an output is spent (excluding the mempool): - -```js -var spent = node.services.bitcoind.isSpent(txid, outputIndex); -console.log(spent); -``` - -**Miscellaneous** -- `bitcoind.start(callback)` - Start the JavaScript Bitcoin node, the callback is called when the daemon is ready. -- `bitcoind.getInfo()` - Basic information about the chain including total number of blocks. -- `bitcoind.isSynced()` - Returns a boolean if the daemon is fully synced (not the initial block download) -- `bitcoind.syncPercentage()` - Returns the current estimate of blockchain download as a percentage. -- `bitcoind.stop(callback)` - Stop the JavaScript bitcoin node safely, the callback will be called when bitcoind is closed. This will also be done automatically on `process.exit`. It also takes the bitcoind node off the libuv event loop. If the daemon object is the only thing on the event loop. Node will simply close. - ## Events -The Bitcoin Service doesn't expose any events via the Bus, however there are a few events that can be directly registered: +The Bitcoin Service exposes two events via the Bus, and there are a few events that can be directly registered: ```js node.services.bitcoind.on('tip', function(blockHash) { - // a new block tip has been added + // a new block tip has been added, if there is a rapid update (with a second) this will not emit every tip update }); -node.services.bitcoind.on('tx', function(txInfo) { +node.services.bitcoind.on('tx', function(transactionBuffer) { // a new transaction has entered the mempool }); -node.services.bitcoind.on('txleave', function(txLeaveInfo) { +node.services.bitcoind.on('block', function(blockHash) { // a new transaction has left the mempool }); ``` -The `txInfo` object will have the format: +For details on instantiating a bus for a node, see the [Bus Documentation](../bus.md). +- Name: `bitcoind/transaction`, Arguments: `[address, address...]` +- Name: `bitcoind/balance`, Arguments: `[address, address...]` + +**Examples:** ```js -{ - buffer: , - mempool: true, // will currently always be true - hash: '7426c707d0e9705bdd8158e60983e37d0f5d63529086d6672b07d9238d5aa623' -} -``` +bus.subscribe('bitcoind/transaction', ['13FMwCYz3hUhwPcaWuD2M1U2KzfTtvLM89']); +bus.subscribe('bitcoind/balance', ['13FMwCYz3hUhwPcaWuD2M1U2KzfTtvLM89']); -The `txLeaveInfo` object will have the format: +bus.on('bitcoind/transaction', function(transaction) { + //... +}); -```js -{ - buffer: , - hash: '7426c707d0e9705bdd8158e60983e37d0f5d63529086d6672b07d9238d5aa623' -} +bus.on('bitcoind/balance', function(balance) { + //... +}); ``` diff --git a/docs/services/db.md b/docs/services/db.md deleted file mode 100644 index 73a6cbb2..00000000 --- a/docs/services/db.md +++ /dev/null @@ -1,113 +0,0 @@ -# Database Service -This service synchronizes a leveldb database with the [Bitcoin Service](bitcoind.md) block chain by connecting and disconnecting blocks to build new indexes that can be queried. Other services can extend the data that is indexed by implementing a `blockHandler` method, similar to the built-in [Address Service](address.md). - -## How to Reindex - -If you need to be able to recreate the database from historical transactions in blocks: -- Shutdown your node -- Remove the `bitcore-node.db` directory in the data directory (e.g. `~/.bitcore/bitcore-node.db`) -- Start your node again - -The database will then ask bitcoind for all the blocks again and recreate the database. This is sometimes required during upgrading as the format of the keys and values has changed. For "livenet" this can take half a day or more, for "testnet" this can take around an hour. - -## Adding Indexes -For a service to include additional block data, it can implement a `blockHandler` method that will be run to when there are new blocks added or removed. - -```js -CustomService.prototype.blockHandler = function(block, add, callback) { - var transactions = block.transactions; - var operations = []; - operations.push({ - type: add ? 'put' : 'del', - key: 'key', - value: 'value' - }); - callback(null, operations); -}; -``` - -Take a look at the Address Service implementation for more details about how to encode the key, value for the best efficiency and ways to format the keys for streaming reads. - -## API Documentation -These methods are exposed over the JSON-RPC interface and can be called directly from a node via: - -```js -node.services.db. -``` - -**Query Blocks by Date** - -One of the additional indexes created by the Database Service is querying for blocks by ranges of dates: - -```js -var newest = 1441914000; // Notice time is in seconds not milliseconds -var oldest = 1441911000; - -node.services.db.getBlockHashesByTimestamp(newest, oldest, function(err, hashes) { - // hashes will be an array of block hashes -}); -``` - -**Working with Blocks and Transactions as Bitcore Instances** - -```js - -var txid = 'c349b124b820fe6e32136c30e99f6c4f115fce4d750838edf0c46d3cb4d7281e'; -var includeMempool = true; -node.services.db.getTransaction(txid, includeMempool, function(err, transaction) { - console.log(transaction.toObject()); -}); - -var txid = 'c349b124b820fe6e32136c30e99f6c4f115fce4d750838edf0c46d3cb4d7281e'; -var includeMempool = true; -node.services.db.getTransactionWithBlockInfo(txid, includeMempool, function(err, transaction) { - console.log(transaction.toObject()); - console.log(transaction.__blockHash); - console.log(transaction.__height); - console.log(transaction.__timestamp); -}); - -var blockHash = '00000000d17332a156a807b25bc5a2e041d2c730628ceb77e75841056082a2c2'; -node.services.db.getBlock(blockHash, function(err, block) { - console.log(block.toObject()); -}); - -// contruct a transaction -var transaction = bitcore.Transaction(); - -node.services.db.sendTransaction(transaction, function(err) { - if (err) { - throw err; - } - // otherwise the transaction has been sent -}); -``` - -## Events -For details on instantiating a bus for a node, see the [Bus Documentation](../bus.md). -- Name: `db/transaction` -- Name: `db/block` - -**Examples:** - -```js -bus.subscribe('db/transaction'); -bus.subscribe('db/block'); - -bus.on('db/block', function(blockHash) { - // blockHash will be a hex string of the block hash -}); - -bus.on('db/transaction', function(txInfo) { - // see below -}); -``` - -The `txInfo` object will have the format: - -```js -{ - rejected: true, // If the transaction was rejected into the mempool - tx: // a Bitcore Transaction instance -} -``` diff --git a/docs/testing.md b/docs/testing.md index 30658bb6..a1a6844e 100644 --- a/docs/testing.md +++ b/docs/testing.md @@ -5,34 +5,19 @@ To run all of the JavaScript tests: npm run test ``` -To run tests against the bindings, as defined in `bindings.gyp` the regtest feature of Bitcoin Core is used, and to enable this feature we currently need to build with the wallet enabled _(not a part of the regular build)_. To do this, export an environment variable and recompile: - -```bash -export BITCORENODE_ENV=test -npm run build -``` - If you do not already have mocha installed: ```bash npm install mocha -g ``` -To run the integration tests: +To run the regression tests: ```bash -mocha -R spec integration/regtest.js +mocha -R spec regtest/bitcoind.js ``` -If any changes have been made to the bindings in the "src" directory, manually compile the Node.js bindings, as defined in `bindings.gyp`, you can run (-d for debug): - -```bash -$ node-gyp -d rebuild -``` - -Note: `node-gyp` can be installed with `npm install node-gyp -g` - -To be able to debug you'll need to have `gdb` and `node` compiled for debugging with gdb using `--gdb` (sometimes called node_g), and you can then run: +To be able to debug bitcoind you'll need to have `gdb` and `node` compiled for debugging with gdb using `--gdb` (sometimes called node_g), and you can then run: ```bash $ gdb --args node examples/node.js