From 1711834fe88410923b8cf1f4bb2db9bac0158795 Mon Sep 17 00:00:00 2001 From: Neil Booth Date: Thu, 3 Nov 2016 14:46:27 +0900 Subject: [PATCH] Release preparation --- HOWTO.rst | 16 ++++++++-------- README.rst | 24 +++++++++++------------- RELEASE-NOTES | 20 ++++++++++++++++++++ samples/scripts/NOTES | 23 +++++++++++++++-------- server/version.py | 2 +- 5 files changed, 55 insertions(+), 30 deletions(-) diff --git a/HOWTO.rst b/HOWTO.rst index af5be2e..e2ae867 100644 --- a/HOWTO.rst +++ b/HOWTO.rst @@ -32,12 +32,12 @@ recommend having at least 30-40GB free space. Database Engine =============== -You can choose between either RocksDB, LevelDB or LMDB to store transaction -information on disk. Currently, the fastest seems to be RocksDB with LevelDB -being about 10% slower. LMDB seems to be the slowest but maybe that's because -of bad implementation or configuration. +You can choose between either RocksDB, LevelDB or LMDB to store +transaction information on disk. Currently, the fastest seems to be +RocksDB with LevelDB being about 10% slower. LMDB is slowest but that +is because it is not yet efficiently abstracted. -You will need to install either: +You will need to install one of: + `plyvel `_ for LevelDB + `pyrocksdb `_ for RocksDB @@ -188,7 +188,7 @@ over the LAN from a bitcoind on machine B. Machine B: a late 2012 iMac running El-Capitan 10.11.6, 2.9GHz quad-core Intel i5 CPU with an HDD and 24GB RAM. Running bitcoind on -the same machine. HIST_MB of 350, UTXO_MB of 1,600. +the same machine. HIST_MB of 350, UTXO_MB of 1,600. LevelDB. For chains other than bitcoin-mainnet sychronization should be much faster. @@ -275,5 +275,5 @@ After flush-to-disk you may see an aiohttp error; this is the daemon timing out the connection while the disk flush was in progress. This is harmless. -The ETA is just a guide and can be quite volatile. It is too optimistic -initially. +The ETA is just a guide and can be quite volatile, particularly around +flushes. It is too optimistic initially. diff --git a/README.rst b/README.rst index ea84b5e..55165a6 100644 --- a/README.rst +++ b/README.rst @@ -68,26 +68,24 @@ Roadmap ======= - test a few more performance improvement ideas -- handle client connections (half-implemented but not functional) +- handle the mempool +- implement light caching of client responses +- yield during expensive requests and/or penalize the connection +- improve DB abstraction so LMDB is not penalized +- continue to clean up the code and remove layering violations +- store all UTXOs, not just those with addresses +- implement IRC connectivity - potentially move some functionality to C or C++ -Once I get round to writing the server part, I will add DoS -protections if necessary to defend against requests for large -histories. However with asyncio it would not surprise me if ElectrumX -could smoothly serve the whole history of the biggest Satoshi dice -address with minimal negative impact on other connections; we shall -have to see. If the requestor is running Electrum client I am -confident that it would collapse under the load far more quickly that -the server would; it is very inefficient at handling large wallets -and histories. +The above are in no particular order. Database Format =============== -The database and metadata formats of ElectrumX are very likely to -change in the future which will render old DBs unusable. For now I do -not intend to provide converters as the rate of flux is high. +The database and metadata formats of ElectrumX is certain to change in +the future which will render old DBs unusable. For now I do not +intend to provide converters as the rate of flux is high. Miscellany diff --git a/RELEASE-NOTES b/RELEASE-NOTES index 295f5d5..6ffc948 100644 --- a/RELEASE-NOTES +++ b/RELEASE-NOTES @@ -1,3 +1,23 @@ +Version 0.04 +------------ + +- made the DB interface a little faster for LevelDB and RocksDB; this was + a small regression in 0.03 +- fixed a bug that prevented block reorgs from working +- implement and enable client connectivity. This is not yet ready for + public use for several reasons. Local RPC, and remote TCP and SSL + connections are all supported in the same way as Electrum-server. + ElectrumX does not begin listening for incoming connections until it + has caught up with the daemon's height. Which ports it is listening + on will appear in the logs when it starts listening. The complete + Electrum wire protocol is implemented, so it is possible to now use + as a server for your own Electrum client. Note that mempools are + not yet handled so unconfirmed transactions will not be notified or + appear; they will appear once they get in a block. Also no + responses are cached, so performance would likely degrade if used by + many clients. I welcome feedback on your experience using this. + + Version 0.03 ------------ diff --git a/samples/scripts/NOTES b/samples/scripts/NOTES index 642ce1f..091ae6e 100644 --- a/samples/scripts/NOTES +++ b/samples/scripts/NOTES @@ -1,8 +1,8 @@ The following environment variables are required: -DB_DIRECTORY - path to the database directory (if relative, to run script) -USERNAME - the username the server will run as -SERVER_MAIN - path to the server_main.py script (if relative, to run script) +DB_DIRECTORY - path to the database directory (if relative, to `run` script) +USERNAME - the username the server will run as if using `run` script +SERVER_MAIN - path to the server_main.py script (if relative, to `run` script) DAEMON_URL - the URL used to connect to the daemon. Should be of the form http://username:password@hostname:port/ Alternatively you can specify DAEMON_USERNAME, DAEMON_PASSWORD, @@ -14,10 +14,22 @@ sensible defaults if not specified. COIN - see lib/coins.py, must be a coin NAME. Defaults to Bitcoin. NETWORK - see lib/coins.py, must be a coin NET. Defaults to mainnet. +DB_ENGINE - database engine for the transaction database. Default is + leveldb. Supported alternatives are rocksdb and lmdb. + You will need to install the appropriate python packages. + Not case sensitive. REORG_LIMIT - maximum number of blocks to be able to handle in a chain reorganisation. ElectrumX retains some fairly compact undo information for this many blocks in levelDB. Default is 200. +TCP_PORT - if set will serve Electrum clients on that port +SSL_PORT - if set will serve Electrum clients over SSL on that port. + If set SSL_CERTFILE and SSL_KEYFILE must be filesystem paths +RPC_PORT - Listen on this port for local RPC connections, defaults to + 8000. +BANNER_FILE - a path to a banner file to serve to clients. The banner file + is re-read for each new client. +DONATION_ADDRESS - server donation address. Defaults to none. Your performance might change by tweaking these cache settings. Cache size is only checked roughly every minute, so the caches can grow @@ -41,8 +53,3 @@ UTXO_MB - amount of UTXO and history cache, in MB, to retain before leveldb caching and Python GC effects. However this may be very dependent on hardware and you may have different results. - -DB_ENGINE - database engine for the transaction database. Default is - leveldb. Supported alternatives are rocksdb and lmdb, - which will require installation of the appropriate python - packages. \ No newline at end of file diff --git a/server/version.py b/server/version.py index c83fa77..285207d 100644 --- a/server/version.py +++ b/server/version.py @@ -1 +1 @@ -VERSION = "ElectrumX 0.03" +VERSION = "ElectrumX 0.04"