diff --git a/docs/protocol-basics.rst b/docs/protocol-basics.rst index 7d51ccd..532545e 100644 --- a/docs/protocol-basics.rst +++ b/docs/protocol-basics.rst @@ -194,3 +194,43 @@ inputs are confirmed. 4. The :dfn:`status` of the script hash is the :func:`sha256` hash of the full string expressed as a hexadecimal string, or :const:`null` if the string is empty because there are no transactions. + + +Block Headers +------------- + +Originally Electrum clients would download all block headers and +verify the chain of hashes and header difficulty in order to confirm +the merkle roots with which to check transaction inclusion. + +With the BTC and BCH chains now past height 500,000, the headers form +over 40MB of raw data which becomes 80MB if downloaded as text from +Electrum servers. The situation is worse for testnet and coins with +more frequent blocks. Downloading and verifying all this data on +initial use would take several minutes, during which Electrum was +non-responsive. + +To facilitate a better experience for SPV clients, particularly on +mobile, protocol :ref:`version 1.4 ` introduces an +optional *cp_height* argument to the :func:`blockchain.block.header` +and :func:`blockchain.block.headers` RPC calls. + +This requests the server provide a merkle proof, to a single 32-byte +checkpoint hard-coded in the client, that the header(s) provided are +valid in the same way the server proves a transaction is included in a +block. If several consecutive headers are requested, the proof is +provided for the final header - the *prev_hash* links in the headers +are sufficient to prove the others valid. + +Using this feature client software only needs to download the headers +it is interested in up to the checkpoint. HHeaders after the +checkpoint must all be downloaded and validated as before. The RPC +calls return the merkle root, so to embed a checkpoint in a client +simply make an RPC request to a couple of trusted servers for the +greatest height to which a reorganisation of the chain is infeasible, +and confirm the returned roots match. + +.. note:: with 500,000 headers of 80 bytes each, a naïve server + implementation would require hashing approximately 88MB of data to + provide a single merkle proof. ElectrumX implements an optimization + such that it hashes only approximately 180KB of data per proof. diff --git a/docs/protocol-changes.rst b/docs/protocol-changes.rst index 9cafb0b..647301c 100644 --- a/docs/protocol-changes.rst +++ b/docs/protocol-changes.rst @@ -121,15 +121,16 @@ Deprecated methods Version 1.4 =========== -This documents the current intent for protocol version 1.4 which is -not yet implemented. It removes all support for :ref:`deserialized -headers `. +This version removes all support for :ref:`deserialized headers +`. Changes ------- * The only valid value for :func:`blockchain.headers.subscribe` argument *raw* is :const:`True` + * Optional *cp_height* argument added to + :func:`blockchain.block.header` and :func:`blockchain.block.headers` Removed methods --------------- diff --git a/docs/protocol-methods.rst b/docs/protocol-methods.rst index a13ace9..d3a0a22 100644 --- a/docs/protocol-methods.rst +++ b/docs/protocol-methods.rst @@ -2,69 +2,6 @@ Protocol Methods ================== -blockchain.block.get_header -=========================== - -Return the :ref:`deserialized header ` of the -block at the given height. - -**Signature** - - .. function:: blockchain.block.get_header(height) - .. deprecated:: 1.3 - - *height* - - The height of the block, an integer. - -**Result** - - The coin-specific :ref:`deserialized header `. - -**Example Result** - -:: - - { - "bits": 392292856, - "block_height": 510000, - "merkle_root": "297cfcc6a66e063692b20650d21cc0ac7a2a80f7277ebd7c5d6c7010a070d25c", - "nonce": 3347656422, - "prev_block_hash": "0000000000000000002292de0d9f03dfa15a04dbf09102d5d4552117b717fa86", - "timestamp": 1519083654, - "version": 536870912 - } - -blockchain.block.get_chunk -========================== - -Return a concatenated chunk of block headers from the main chain. -Typically, a chunk consists of a fixed number of block headers over -which difficulty is constant, and at the end of which difficulty is -retargeted. - -In the case of Bitcoin a chunk is 2,016 headers, each of 80 bytes, so -chunk 5 consists of the block headers from height 10,080 to 12,095 -inclusive. When encoded as hexadecimal, the result string is twice as -long, so for Bitcoin it takes 322,560 bytes, making this a -bandwidth-intensive request. - -**Signature** - - .. function:: blockchain.block.get_chunk(index) - .. deprecated:: 1.2 - - *index* - - The zero-based index of the chunk, an integer. - -**Result** - - The binary block headers as hexadecimal strings, in-order and - concatenated together. As many as headers as are available at the - implied starting height will be returned; this may range from zero - to the coin-specific chunk size. - blockchain.block.header ======================= @@ -72,16 +9,44 @@ Return the block header at the given height. **Signature** - .. function:: blockchain.block.header(height) + .. function:: blockchain.block.header(height, cp_height=0) .. versionadded:: 1.3 + .. versionchanged:: 1.4 + *cp_height* parameter added *height* - The height of the block, an integer. + The height of the block, a non-negative integer. + + *cp_height* + + Checkpoint height, a non-negative integer. Ignored if zero, + otherwise the following must hold: + + *height* <= *cp_height* **Result** - The raw block header as a hexadecimal string. + If *cp_height* is zero, the raw block header as a hexadecimal + string. + + Otherwise a dictionary with the following keys. This provides a + proof that the given header is present in the blockchain; presumably + the client has the merkle root hard-coded as a checkpoint. + + * *header* + + The raw block header as a hexadecimal string. + + * *root* + + The merkle root of all blockchain headers up to and including + *cp_height*. + + * *branch* + + The merkle branch of *header* up to *root*, deepest pairing first. + **Example Result** @@ -89,6 +54,7 @@ Return the block header at the given height. "0100000085144a84488ea88d221c8bd6c059da090e88f8a2c99690ee55dbba4e00000000e11c48fecdd9e72510ca84f023370c9a38bf91ac5cae88019bee94d24528526344c36649ffff001d1d03e477" + blockchain.block.headers ======================== @@ -96,8 +62,10 @@ Return a concatenated chunk of block headers from the main chain. **Signature** - .. function:: blockchain.block.headers(start_height, count) + .. function:: blockchain.block.headers(start_height, count, cp_height=0) .. versionadded:: 1.2 + .. versionchanged:: 1.4 + *cp_height* parameter added *start_height* @@ -107,6 +75,13 @@ Return a concatenated chunk of block headers from the main chain. The number of headers requested, a non-negative integer. + *cp_height* + + Checkpoint height, a non-negative integer. Ignored if zero, + otherwise the following must hold: + + *start_height* + (*count* - 1) <= *cp_height* + **Result** A dictionary with the following members: @@ -128,6 +103,22 @@ Return a concatenated chunk of block headers from the main chain. The maximum number of headers the server will return in a single request. + The dictionary additionally has the following keys if *count* and + *cp_height* are not zero. This provides a proof that all the given + headers are present in the blockchain; presumably the client has the + merkle root hard-coded as a checkpoint. + + * *root* + + The merkle root of all blockchain headers up to and including + *cp_height*. + + * *branch* + + The merkle branch of the last returned header up to *root*, + deepest pairing first. + + **Example Response** :: @@ -135,7 +126,7 @@ Return a concatenated chunk of block headers from the main chain. { "count": 2, "hex": "0100000000000000000000000000000000000000000000000000000000000000000000003ba3edfd7a7b12b27ac72c3e67768f617fc81bc3888a51323a9fb8aa4b1e5e4a29ab5f49ffff001d1dac2b7c010000006fe28c0ab6f1b372c1a6a246ae63f74f931e8365e15a089c68d6190000000000982051fd1e4ba744bbbe680e1fee14677ba1a3c3540bf7b1cdb606e857233e0e61bc6649ffff001d01e36299" - "max": 2016 + "max": 2016 } blockchain.estimatefee @@ -600,7 +591,7 @@ When *verbose* is :const:`True`:: blockchain.transaction.get_merkle ================================= -Return the markle branch to a confirmed transaction given its hash +Return the merkle branch to a confirmed transaction given its hash and height. **Signature** diff --git a/docs/protocol-removed.rst b/docs/protocol-removed.rst index 3e26ffb..5514865 100644 --- a/docs/protocol-removed.rst +++ b/docs/protocol-removed.rst @@ -13,8 +13,7 @@ Return the confirmed and unconfirmed balances of a bitcoin address. **Signature** .. function:: blockchain.address.get_balance(address) - .. deprecated:: 1.2 - *Removed in version 1.3.* + .. deprecated:: 1.2 removed in version 1.3 * *address* @@ -32,8 +31,7 @@ Return the confirmed and unconfirmed history of a bitcoin address. **Signature** .. function:: blockchain.address.get_history(address) - .. deprecated:: 1.2 - *Removed in version 1.3.* + .. deprecated:: 1.2 removed in version 1.3 * *address* @@ -51,8 +49,7 @@ Return the unconfirmed transactions of a bitcoin address. **Signature** .. function:: blockchain.address.get_mempool(address) - .. deprecated:: 1.2 - *Removed in version 1.3.* + .. deprecated:: 1.2 removed in version 1.3 * *address* @@ -70,8 +67,7 @@ Return an ordered list of UTXOs sent to a bitcoin address. **Signature** .. function:: blockchain.address.listunspent(address) - .. deprecated:: 1.2 - *Removed in version 1.3.* + .. deprecated:: 1.2 removed in version 1.3 * *address* @@ -89,8 +85,7 @@ Subscribe to a bitcoin address. **Signature** .. function:: blockchain.address.subscribe(address) - .. deprecated:: 1.2 - *Removed in version 1.3.* + .. deprecated:: 1.2 removed in version 1.3 *address* @@ -116,8 +111,7 @@ Subscribe to receive the block height when a new block is found. **Signature** .. function:: blockchain.numblocks.subscribe() - .. deprecated:: 1.0 - *Removed in version 1.1.* + .. deprecated:: 1.0 removed in version 1.1 **Result** @@ -139,7 +133,7 @@ Return the address paid to by a UTXO. .. function:: blockchain.utxo.get_address(tx_hash, index) - *Optional in version 1.0. Removed in version 1.1.* + *Optional in version 1.0, removed in version 1.1* *tx_hash* @@ -155,3 +149,66 @@ Return the address paid to by a UTXO. doesn't exist, the index is out of range, or the output is not paid to an address, :const:`null` must be returned. If the output is spent :const:`null` *may* be returned. + +blockchain.block.get_header +=========================== + +Return the :ref:`deserialized header ` of the +block at the given height. + +**Signature** + + .. function:: blockchain.block.get_header(height) + .. deprecated:: 1.3 removed in version 1.4 + + *height* + + The height of the block, an integer. + +**Result** + + The coin-specific :ref:`deserialized header `. + +**Example Result** + +:: + + { + "bits": 392292856, + "block_height": 510000, + "merkle_root": "297cfcc6a66e063692b20650d21cc0ac7a2a80f7277ebd7c5d6c7010a070d25c", + "nonce": 3347656422, + "prev_block_hash": "0000000000000000002292de0d9f03dfa15a04dbf09102d5d4552117b717fa86", + "timestamp": 1519083654, + "version": 536870912 + } + +blockchain.block.get_chunk +========================== + +Return a concatenated chunk of block headers from the main chain. +Typically, a chunk consists of a fixed number of block headers over +which difficulty is constant, and at the end of which difficulty is +retargeted. + +In the case of Bitcoin a chunk is 2,016 headers, each of 80 bytes, so +chunk 5 consists of the block headers from height 10,080 to 12,095 +inclusive. When encoded as hexadecimal, the result string is twice as +long, so for Bitcoin it takes 322,560 bytes, making this a +bandwidth-intensive request. + +**Signature** + + .. function:: blockchain.block.get_chunk(index) + .. deprecated:: 1.2 removed in version 1.4 + + *index* + + The zero-based index of the chunk, an integer. + +**Result** + + The binary block headers as hexadecimal strings, in-order and + concatenated together. As many as headers as are available at the + implied starting height will be returned; this may range from zero + to the coin-specific chunk size.