diff --git a/docs/HOWTO.rst b/docs/HOWTO.rst index 8bfc7f7..3faa587 100644 --- a/docs/HOWTO.rst +++ b/docs/HOWTO.rst @@ -1,4 +1,9 @@ -============= +.. _HOWTO: + +===== +HOWTO +===== + Prerequisites ============= @@ -13,16 +18,17 @@ Package Notes Python3 ElectrumX uses asyncio. Python version >= 3.6 is **required**. `aiohttp`_ Python library for asynchronous HTTP. Version >= - 1.0 required; I am using 1.0.5. + 1.0 required; I am using 3.0.1. `pylru`_ Python LRU cache package. I'm using 1.0.9. DB Engine I use `plyvel`_ 0.9, a Python interface to LevelDB. A database engine package is required but others are supported (see **Database Engine** below). -`x11_hash`_ Only required for DASH. Python X11 Hash package. Only - required if for Dash. Version 1.4 tested. ================ ======================== -You need to be running a non-pruning bitcoin daemon with:: +Some coins need an additional package, typically for their block hash +functions. For example, `x11_hash`_ is required for DASH. + +You **must** to be running a non-pruning bitcoin daemon with:: txindex=1 diff --git a/docs/index.rst b/docs/index.rst index 2d117c5..4564cf6 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -3,14 +3,73 @@ You can adapt this file completely to your liking, but it should at least contain the root `toctree` directive. -Welcome to ElectrumX's documentation! -===================================== +.. image:: https://travis-ci.org/kyuupichan/electrumx.svg?branch=master + :target: https://travis-ci.org/kyuupichan/electrumx +.. image:: https://coveralls.io/repos/github/kyuupichan/electrumx/badge.svg + :target: https://coveralls.io/github/kyuupichan/electrumx + +========= +ElectrumX +========= + +A reimplementation of Electrum-Server for a future with bigger blocks. + + :Licence: MIT + :Language: Python (>= 3.6) + :Author: Neil Booth + + +Getting Started +=============== + +See :ref:`HOWTO`. + +There is also an `installer`_ available that simplifies the +installation on various Linux-based distributions, and a `Dockerfile`_ +available . + +.. _installer: https://github.com/bauerj/electrumx-installer +.. _Dockerfile: https://github.com/lukechilds/docker-electrumx + + +Features +======== + +- Efficient, lightweight reimplementation of electrum-server +- Fast synchronization of bitcoin mainnet from Genesis. Recent + hardware should synchronize in well under 24 hours. The fastest + time to height 448k (mid January 2017) reported is under 4h 30m. On + the same hardware JElectrum would take around 4 days and + electrum-server probably around 1 month. +- The full current Electrum protocol is implemented. +- Various configurable means of controlling resource consumption and + handling denial of service attacks. These include maximum + connection counts, subscription limits per-connection and across all + connections, maximum response size, per-session bandwidth limits, + and session timeouts. +- Minimal resource usage once caught up and serving clients; tracking the + transaction mempool appears to be the most expensive part. +- Fully asynchronous processing of new blocks, mempool updates, and + client requests. Busy clients should not noticeably impede other + clients' requests and notifications, nor the processing of incoming + blocks and mempool updates. +- Daemon failover. More than one daemon can be specified, and + ElectrumX will failover round-robin style if the current one fails + for any reason. +- Peer discovery protocol removes need for IRC +- Coin abstraction makes compatible altcoin and testnet support easy. + .. toctree:: :maxdepth: 2 :caption: Contents: + ENVIRONMENT + HOWTO + PEER_DISCOVERY + RPC-INTERFACE protocol-new + ARCHITECTURE Indices and tables diff --git a/docs/mempool_fee_get_histogram.rst b/docs/mempool_fee_get_histogram.rst deleted file mode 100644 index 7debce5..0000000 --- a/docs/mempool_fee_get_histogram.rst +++ /dev/null @@ -1,15 +0,0 @@ -.. method:: mempool.get_fee_histogram() - - Return a histogram of the fee rates paid by transactions in the - memory pool, weighted by transaction size. - - **Response** - - The histogram is an array of [*fee*, *vsize*] pairs, where *vsize_n* is - the cumulative virtual size of mempool transactions with a fee rate - in the interval [*fee*_(n-1), *fee*_n)], and *fee*_(n-1) > *fee*_n. - - Fee intervals may have variable size. The choice of appropriate - intervals is currently not part of the protocol. - - .. versionadded:: 1.2 diff --git a/docs/protocol-basics.rst b/docs/protocol-basics.rst new file mode 100644 index 0000000..aee447e --- /dev/null +++ b/docs/protocol-basics.rst @@ -0,0 +1,125 @@ +Protocol Basics +=============== + +Message Stream +-------------- + +Clients and servers communicate using **JSON RPC** over an unspecified +underlying stream transport protocol, typically TCP or SSL. + +Two standards `JSON RPC 1.0 +`_ and `JSON RPC 2.0 +`_ are specified; use of version +2.0 is encouraged but not required. Server support of batch requests +is encouraged for version 1.0 but not required. Clients making batch +requests should limit their size depending on the nature of their +query, because servers will limit response size as an anti-DoS +mechanism. + +Eeach RPC call, and each response, is separated by a single newline in +their respective streams. The JSON specification does not permit +control characters within strings, so no confusion is possible there. +However it does permit newlines as extraneous whitespace between +elements; client and server MUST NOT use newlines in such a way. + +If using JSON RPC 2.0's feature of parameter passing by name, the +names shown in the description of the method in question MUST be used. + +A server advertising support for a particular protocol version MUST +support each method documented for that protocol version, unless the +method is explicitly marked optional. It may support other methods or +additional parameters with unspecified behaviour. Use of additional +parameters is discouraged as it may conflict with future versions of +the protocol. + + +Notifications +------------- + +Some RPC calls are subscriptions, which, after the initial response, +will send a JSON RPC :dfn:`notification` each time the thing +subscribed to changes. The `method` of the notification is the same +as the method of the subscription, and the `params` of the +notification (and their names) are given in the documentation of the +method. + + +Version Negotiation +------------------- + +It is desirable to have a way to enhance and improve the protocol +without forcing servers and clients to upgrade at the same time. +Protocol negotiation is not implemented in any client or server at +present to the best of my knowledge, so care is needed to ensure +current clients and servers continue to operate as expected. + +Protocol versions are denoted by dotted "a.b" strings, where *m* is +the major version number and *n* the minor version number. For +example: "1.5". + +A party to a connection will speak all protocol versions in a range, +say from `protocol_min` to `protocol_max`, which may be the same. +When a connection is made, both client and server must initially +assume the protocol to use is their own `protocol_min`. + +The client should send a :func:`server.version` RPC call as early as +possible in order to negotiate the precise protocol version; see its +description for more detail. All responses received in the stream +from and including the server's response to this call will use the +negotiated protocol version. + +.. _script hashes: + +Script Hashes +------------- + +A :dfn:`script hash` is the hash of the binary bytes of the locking +script (ScriptPubKey), expressed as a hexadecimal string. The hash +function to use is given by the "hash_function" member of +:func:`server.features` (currently "sha256" only). Like for block and +transaction hashes, when converting the big-endian binary hash to a +hexadecimal string the least-significant byte appears first, and the +most-significant byte last. + +For example, the legacy Bitcoin address from the genesis block:: + + 1A1zP1eP5QGefi2DMPTfTL5SLmv7DivfNa + +has P2PKH script:: + + 76a91462e907b15cbf27d5425399ebf6f0fb50ebb88f1888ac + +with SHA256 hash:: + + 6191c3b590bfcfa0475e877c302da1e323497acf3b42c08d8fa28e364edf018b + +which is sent to the server reversed as:: + + 8b01df4e368ea28f8dc0423bcf7a4923e3a12d307c875e47a0cfbf90b5c39161 + +By subscribing to this hash you can find P2PKH payments to that address. + +One public key for that address (the genesis block public key) is:: + + 04678afdb0fe5548271967f1a67130b7105cd6a828e03909a67962e0ea1f61deb + 649f6bc3f4cef38c4f35504e51ec112de5c384df7ba0b8d578a4c702b6bf11d5f + +which has P2PK script:: + + 4104678afdb0fe5548271967f1a67130b7105cd6a828e03909a67962e0ea1f61deb + 649f6bc3f4cef38c4f35504e51ec112de5c384df7ba0b8d578a4c702b6bf11d5fac + +with SHA256 hash:: + + 3318537dfb3135df9f3d950dbdf8a7ae68dd7c7dfef61ed17963ff80f3850474 + +which is sent to the server reversed as:: + + 740485f380ff6379d11ef6fe7d7cdd68aea7f8bd0d953d9fdf3531fb7d531833 + +By subscribing to this hash you can find P2PK payments to that public +key. + +.. note:: The Genesis block coinbase is unspendable and therefore not + indexed. It will not show with the above P2PK script hash + subscription. diff --git a/docs/protocol-methods.rst b/docs/protocol-methods.rst new file mode 100644 index 0000000..4121799 --- /dev/null +++ b/docs/protocol-methods.rst @@ -0,0 +1,268 @@ +Protocol Methods +================ + +mempool.get_fee_histogram +------------------------- + + Return a histogram of the fee rates paid by transactions in the + memory pool, weighted by transaction size. + +**Signature** + + .. function:: mempool.get_fee_histogram() + + .. versionadded:: 1.2 + +**Result** + + The histogram is an array of [*fee*, *vsize*] pairs, where |vsize_n| + is the cumulative virtual size of mempool transactions with a fee rate + in the interval [|fee_n1|, |fee_n|], and |fee_n1| > |fee_n|. + + .. |vsize_n| replace:: vsize\ :sub:`n` + .. |fee_n| replace:: fee\ :sub:`n` + .. |fee_n1| replace:: fee\ :sub:`n-1` + + Fee intervals may have variable size. The choice of appropriate + intervals is currently not part of the protocol. + +**Example Result** + + :: + + [[12, 128812], [4, 92524], [2, 6478638], [1, 22890421]] + + +server.add_peer +--------------- + + This call is intended for a new server to get itself into a server's + peers list, and should not be used by wallet clients. + +**Signature** + + .. function:: server.add_peer(features) + + .. versionadded:: 1.1 + + * **features** + + The same information that a call to the sender's + :func:`server.features` RPC call would return. + +**Result** + + A boolean indicating whether the request was tentatively accepted. + The requesting server will appear in :func:`server.peers.subscribe` + when further sanity checks complete successfully. + + +server.banner +------------- + + Return a banner to be shown in the Electrum console. + +**Signature** + + .. function:: server.banner() + +**Result** + + A string. + +**Example Result** + + :: + + "Welcome to Electrum!" + + +server.donation_address +----------------------- + + Return a server donation address. + +**Signature** + + .. function:: server.donation_address() + +**Result** + + A string. + +**Example Result** + + :: + + "1BWwXJH3q6PRsizBkSGm2Uw4Sz1urZ5sCj" + + +server.features +--------------- + + Return a list of features and services supported by the server. + +**Signature** + + .. function:: server.features() + +**Result** + + A dictionary of keys and values. Each key represents a feature or + service of the server, and the value gives additional information. + + The following features MUST be reported by the server. Additional + key-value pairs may be returned. + + * **hosts** + + A dictionary, keyed by host name, that this server can be reached + at. Normally this will only have a single entry; other entries + can be used in case there are other connection routes (e.g. Tor). + + The value for a host is itself a dictionary, with the following + optional keys: + + * **ssl_port** + + An integer. Omit or set to **null** if SSL connectivity is not + provided. + + * **tcp_port** + + An integer. Omit or set to **null** if TCP connectivity is not + provided. + + A server should ignore information provided about any host other + than the one it connected to. + + * **genesis_hash** + + The hash of the genesis block. This is used to detect if a peer + is connected to one serving a different network. + + * **hash_function** + + The hash function the server uses for :ref:`script hashing +