Update peer discovery notes
This commit is contained in:
Neil Booth
parent
5e3ab51e5e
commit
e2f1864a94
@ -6,13 +6,9 @@ Peer Discovery
|
||||
This was imlpemented in ElectrumX as of version 0.11.0. Support for
|
||||
IRC peer discovery was removed in ElectrumX version 1.2.1.
|
||||
|
||||
|
||||
Peer Database
|
||||
-------------
|
||||
|
||||
An in-memory store of peers with at least the following information
|
||||
about a peer, required for a response to the
|
||||
**server.peers.subscribe** RPC call:
|
||||
The :dfn:`peer database` is an in-memory store of peers with at least
|
||||
the following information about a peer, required for a response to the
|
||||
:func:`server.peers.subscribe` RPC call:
|
||||
|
||||
* host name
|
||||
* ip address
|
||||
@ -20,39 +16,32 @@ about a peer, required for a response to the
|
||||
* protocol version
|
||||
* pruning limit, if any
|
||||
|
||||
At present ElectrumX uses a flat file for this DB in the main database
|
||||
directory. It retains additional per-peer metadata including:
|
||||
|
||||
* time of last successful connection
|
||||
* time of last connection attempt
|
||||
* count of unsuccessful attempts since last successful one
|
||||
* source of the information stored about this peer
|
||||
Hard-coded Peers
|
||||
----------------
|
||||
|
||||
A list of hard-coded, well-known peers seeds the peer discovery
|
||||
process. Ideally it should have at least 4 servers that have shown
|
||||
commitment to reliable service.
|
||||
|
||||
In ElectrumX this is a per-coin property in `lib/coins.py
|
||||
<https://github.com/kyuupichan/electrumx/blob/master/lib/coins.py>`_.
|
||||
|
||||
|
||||
Default Peers
|
||||
-------------
|
||||
server.peers.subscribe
|
||||
----------------------
|
||||
|
||||
This is a list of hard-coded, well-known peers to seed the peer
|
||||
discovery process if the peer database is empty or corrupt. If the
|
||||
peer database is available it is not used. Ideally it should hold up
|
||||
to 10 servers that have shown commitment to reliable service.
|
||||
|
||||
In ElectrumX this is a per-coin property in `lib/coins.py`.
|
||||
|
||||
|
||||
Response to server.peers.subscribe RPC call
|
||||
-------------------------------------------
|
||||
|
||||
This RPC call is used by Electrum clients to get a list of peer
|
||||
servers, in preference to a hard-coded list of peer servers in the
|
||||
client, which it will fall back to if necessary.
|
||||
:func:`server.peers.subscribe` is used by Electrum clients to get a
|
||||
list of peer servers, in preference to a hard-coded list of peer
|
||||
servers in the client, which it will fall back to if necessary.
|
||||
|
||||
The server should craft its response in a way that reduces the
|
||||
effectiveness of sybil attacks and peer spamming.
|
||||
effectiveness of server sybil attacks and peer spamming.
|
||||
|
||||
The response should only include peers it has successfully connected
|
||||
to recently. Only reporting recent good peers ensures that those that
|
||||
have gone offline will not be passed around for long.
|
||||
have gone offline will be forgotten quickly and not be passed around
|
||||
for long.
|
||||
|
||||
In ElectrumX, "recently" is taken to be the last 24 hours. Only one
|
||||
peer from each IPv4/16 netmask is returned, and the number of onion
|
||||
@ -62,14 +51,16 @@ peers is limited.
|
||||
Maintaining the Peer Database
|
||||
-----------------------------
|
||||
|
||||
In order to keep its peer database up-to-date and fresh, if some time
|
||||
has passed since the last successful connection to a peer, an Electrum
|
||||
server should make an attempt to connect, choosing either the TCP or
|
||||
SSL port. On connecting it should issue **server.peers.subscribe**,
|
||||
**blockchain.headers.subscribe**, and **server.features** RPC calls to
|
||||
collect information about the server and its peers. If the peer seems
|
||||
to not know of you, you can issue a **server.add_peer** call to
|
||||
advertise yourself. Once this is done and replies received it should
|
||||
In order to keep its peer database up-to-date and fresh, after some
|
||||
time has passed since the last successful connection to a peer, an
|
||||
Electrum server should make another attempt to connect, choosing
|
||||
either the TCP or SSL port.
|
||||
|
||||
On connecting it should issue :func:`server.peers.subscribe`,
|
||||
:func:`blockchain.headers.subscribe`, and :func:`server.features` RPC
|
||||
calls to collect information about the server and its peers. If the
|
||||
peer seems to not know of you, you can issue a :func:`server.add_peer`
|
||||
call to advertise yourself. Once this is done and replies received,
|
||||
terminate the connection.
|
||||
|
||||
The peer database should view information obtained from an outgoing
|
||||
@ -77,13 +68,13 @@ connection as authoritative, and prefer it to information obtained
|
||||
from any other source.
|
||||
|
||||
On connecting, a server should confirm the peer is serving the same
|
||||
network, ideally via the genesis block hash of the **server.features**
|
||||
RPC call below. Also the height reported by the peer should be within
|
||||
a small number of the expected value. If a peer is on the wrong
|
||||
network it should never be advertised to clients or other peers. Such
|
||||
invalid peers should perhaps be remembered for a short time to prevent
|
||||
redundant revalidation if other peers persist in advertising them, and
|
||||
later forgotten.
|
||||
network, ideally via the genesis block hash of the
|
||||
:func:`server.features` RPC call below. Also the height reported by
|
||||
the peer should be within a small number of the expected value. If a
|
||||
peer is on the wrong network it should never be advertised to clients
|
||||
or other peers. Such invalid peers should perhaps be remembered for a
|
||||
short time to prevent redundant revalidation if other peers persist in
|
||||
advertising them, and later forgotten.
|
||||
|
||||
If a connection attempt fails, subsequent reconnection attempts should
|
||||
follow some kind of exponential backoff.
|
||||
@ -97,81 +88,31 @@ ElectrumX will connect to the SSL port if both ports are available.
|
||||
If that fails it will fall back to the TCP port. It tries to
|
||||
reconnect to a good peer at least once every 24 hours, and a failing
|
||||
after 5 minutes but with exponential backoff. It forgets a peer
|
||||
entirely if two weeks have passed since a successful connection.
|
||||
entirely if a few days have passed since a successful connection.
|
||||
ElectrumX attempts to connect to onion peers through a Tor proxy that
|
||||
can be configured or that it will try to autodetect.
|
||||
|
||||
|
||||
server.features RPC call
|
||||
------------------------
|
||||
server.features
|
||||
---------------
|
||||
|
||||
This is a new RPC call that a server can use to advertise what
|
||||
services and features it offers. It is intended for eventual use by
|
||||
Electrum clients as well as other peers. Peers will use it to gather
|
||||
peer information from the peer itself.
|
||||
:func:`server.features` is a fairly new RPC call that a server can use
|
||||
to advertise what services and features it offers. It is intended for
|
||||
use by Electrum clients as well as other peers. Peers will use it to
|
||||
gather peer information from the peer itself.
|
||||
|
||||
The call takes no arguments and returns a dictionary keyed by feature
|
||||
name whose value gives details about the feature where appropriate.
|
||||
If a key is missing the feature is presumed not to be offered.
|
||||
|
||||
Currently ElectrumX understands and returns the following keys.
|
||||
Unknown keys should be silently ignored.
|
||||
|
||||
* **hosts**
|
||||
server.add_peer
|
||||
---------------
|
||||
|
||||
An dictionary, keyed by host name, that this server can be reached
|
||||
at. Normally this will only have a single entry; other entries can
|
||||
be used in case there are other connection routes (e.g. Tor).
|
||||
:func:`server.add_peer` is intended for a new server to get itself in
|
||||
the connected set.
|
||||
|
||||
The value for a host is itself a dictionary, with the following
|
||||
optional keys:
|
||||
|
||||
* **ssl_port**
|
||||
|
||||
An integer. Omit or set to *null* if SSL connectivity is not
|
||||
provided.
|
||||
|
||||
* **tcp_port**
|
||||
|
||||
An integer. Omit or set to *null* if TCP connectivity is not
|
||||
provided.
|
||||
|
||||
A server should ignore information provided about any host other
|
||||
than the one it connected to.
|
||||
|
||||
* **genesis_hash**
|
||||
|
||||
The hash of the genesis block. This is used to detect if a peer is
|
||||
connected to one serving a different network.
|
||||
|
||||
* **server_version**
|
||||
|
||||
A string that identifies the server software. Should be the same as
|
||||
the response to **server.version** RPC call.
|
||||
|
||||
* **protocol_max**
|
||||
* **protocol_min**
|
||||
|
||||
Strings that are the minimum and maximum Electrum protocol versions
|
||||
this server speaks. Example: "1.1".
|
||||
|
||||
* **pruning**
|
||||
|
||||
An integer, the pruning limit. Omit or set to *null* if there is no
|
||||
pruning limit.
|
||||
|
||||
|
||||
server.add_peer RPC call
|
||||
------------------------
|
||||
|
||||
This call is intended for a new server to get itself in the connected
|
||||
set.
|
||||
|
||||
It takes a single parameter (named **features** if JSON RPCv2 named
|
||||
parameters are being used) which contains the same information as the
|
||||
**server.features** RPC call would return.
|
||||
|
||||
A server receiving a **server.add_peer** call should not replace
|
||||
A server receiving a :func:`server.add_peer` call should not replace
|
||||
existing information about the host(s) given, but instead schedule a
|
||||
separate connection to verify the information for itself.
|
||||
|
||||
@ -181,33 +122,33 @@ calls to this method from a single connection.
|
||||
The result should be True if accepted and False otherwise.
|
||||
|
||||
|
||||
Notes to Implementators
|
||||
-----------------------
|
||||
Notes for Implementors
|
||||
----------------------
|
||||
|
||||
* it is very important to only accept peers that appear to be on the
|
||||
same network. At a minimum the genesis hash should be compared (if
|
||||
the peer supports the *server.features* RPC call), and also that the
|
||||
peer's reported height is within a few blocks of your own server's
|
||||
height.
|
||||
* care should be taken with the *add_peer* call. Consider only
|
||||
accepting it once per connection. Clearnet peer requests should
|
||||
check the peer resolves to the requesting IP address, to prevent
|
||||
attackers from being able to trigger arbitrary outgoing connections
|
||||
from your server. This doesn't work for onion peers so they should
|
||||
be rate-limited.
|
||||
the peer supports :func:`server.features`), and also that the peer's
|
||||
reported height is within a few blocks of your own server's height.
|
||||
* care should be taken with the :func:`server.add_peer` call.
|
||||
Consider only accepting it once per connection. Clearnet peer
|
||||
requests should check the peer resolves to the requesting IP
|
||||
address, to prevent attackers from being able to trigger arbitrary
|
||||
outgoing connections from your server. This doesn't work for onion
|
||||
peers so they should be rate-limited.
|
||||
* it should be possible for a peer to change their port assignments -
|
||||
presumably connecting to the old ports to perform checks will not
|
||||
work.
|
||||
* peer host names should be checked for validity before accepting
|
||||
them; and *localhost* should probably be rejected. If it is an IP
|
||||
them; and `localhost` should probably be rejected. If it is an IP
|
||||
address it should be a normal public one (not private, multicast or
|
||||
unspecified).
|
||||
* you should limit the number of new peers accepted from any single
|
||||
source to at most a handful, to limit the effectiveness of malicious
|
||||
peers wanting to trigger arbitrary outgoing connections or fill your
|
||||
peer tables with junk data.
|
||||
* in the response to *server.peers.subscribe* calls, consider limiting
|
||||
the number of peers on similar IP subnets to protect against sybil
|
||||
attacks, and in the case of onion servers the total returned.
|
||||
* in the response to :func:`server.peers.subscribe` calls, consider
|
||||
limiting the number of peers on similar IP subnets to protect
|
||||
against sybil attacks, and in the case of onion servers the total
|
||||
returned.
|
||||
* you should not advertise a peer's IP address if it also advertises a
|
||||
hostname (avoiding duplicates).
|
||||
|
||||
Loading…
Reference in New Issue
Block a user