Commit Graph

99 Commits

Author SHA1 Message Date
Neil Booth
2a461bd98c Serve sessions in a priority queue.
Add new RPC command requests.
Adjust sessions RPC command to show pending requests.
Only keep first 15 chars in version string.
Set socket timeout
Try more forceful closing of socket
Fixes #67
2016-12-09 22:06:04 +09:00
Neil Booth
1a9e8cdcd4 More work 2016-12-09 08:06:03 +09:00
Neil Booth
cbb1e504cc Cache headers. 2016-12-08 06:31:15 +09:00
Neil Booth
e40db63beb Queue requests, which have a process method. 2016-12-08 00:29:46 +09:00
Neil Booth
4e512343be Fix other JSON bugs introduced in 0.8.5
Further clean up the JSON code.
2016-12-07 19:44:36 +09:00
Neil Booth
b2e4d32de6 Remove special-casing of all_seeing_eye 2016-12-07 08:11:30 +09:00
Neil Booth
f904ce2f5d Rework the JSON layer
Batch requests are now processed and encoded incrementally.
Their bandwidth usage is also incrementally added, and so overlarge
responses are rejected as soon as they become too large.
JSON text is also more memory efficient than the python data
structures they represent.
2016-12-07 08:11:30 +09:00
Neil Booth
b116040365 Clean up param verification code 2016-12-07 08:11:29 +09:00
Neil Booth
1f1f0f42f8 Remove invalidated history cache on notifications 2016-12-06 06:36:11 +09:00
Neil Booth
643c990684 Revert "Add feature to rate limit connections"
This reverts commit a82ab0b075.
2016-12-05 23:27:36 +09:00
Neil Booth
be8adbbcd5 Fix some typos. 2016-12-04 23:41:07 +09:00
Neil Booth
a82ab0b075 Add feature to rate limit connections 2016-12-04 21:14:45 +09:00
Neil Booth
79a9e91994 Handle closing sessions a little differently
Move connection type to first flag letter.

All seeing eye is not stale.
2016-12-04 11:57:25 +09:00
Neil Booth
fcf696d3f3 Don't require 2 params for version requests 2016-12-04 10:54:22 +09:00
Neil Booth
656f749c70 Add LRU cache for history 2016-12-03 20:20:28 +09:00
Neil Booth
04369dd228 Forcefully drop stale sessions or if shutting down
Don't wait for the socket
2016-12-03 18:33:23 +09:00
Neil Booth
aaf0592f52 Fix typo 2016-12-03 15:10:57 +09:00
Neil Booth
c4e7878407 Add session ID for each session
Show it in the logs with each session-specific message.
Show the sessions ID in the sessions list with an L suffix if logging.
Add RPC commands to toggle logging of, and disconnect, a session.
Closes #55
2016-12-03 14:32:09 +09:00
Neil Booth
3d2824218b Close stale sessions
New envvar SESSION_TIMEOUT
A session with no activity is cut off after this time
Fixes #56
2016-12-03 13:06:20 +09:00
Neil Booth
c16792fb89 Only wait 30 secsonds on shutdown 2016-12-03 11:03:27 +09:00
Neil Booth
62a531b1d0 Revert "Fix ref leak in mempool updates"
This reverts commit 6c95644ae0.
2016-12-02 18:43:15 +09:00
Neil Booth
6c95644ae0 Fix ref leak in mempool updates 2016-12-02 07:22:30 +09:00
Neil Booth
0892b12036 Replace $DAEMON_VERSION and $DAEMON_SUBVERSION
in banner file.
2016-11-29 23:03:05 +09:00
Neil Booth
b65bcda504 Add per-session bandwidth limits 2016-11-29 22:19:12 +09:00
Neil Booth
c9a10be5ba Introduce incoming buffer size limit
- incoming buffered network requests limited to 150,000 bytes
  which I believe is ample for a genuine client
- if exceeded, the connection is dropped
- raise outgoing data limit for RPC connections to 5 MB - expect
  sessions calls can be long and connection is implicitly trusted
- similarly raise incoming buffered data limit to 5 MB for RPC
  connections
2016-11-29 20:24:58 +09:00
Neil Booth
c08ade5861 Add session logging facility
Move session logging code to protocol.py from electrum_rpc.py
Use it for periodic logging controlled by envvar LOG_SESSIONS
For each session, track sent transaction stats and show that
per-session instead of errors
2016-11-29 08:00:54 +09:00
Neil Booth
6aef79461f Don't shut down block processor by cancellation
The block processor needs to be able to close cleanly, and not
mid-block.  In order to be able to yield whilst processing blocks
we cannot forcefully close its coroutine with a cancellation.
2016-11-27 12:20:16 +09:00
Neil Booth
9eb701f220 Replace $VERSION in the banner file
with the ElectrumX version string
2016-11-26 18:19:53 +09:00
Neil Booth
59244cc24e Move bulk of FS flush code to db.py
More logically belongs there.
Have servers use the flushed DB height not the block processor's height.
Get all headers only from disk.
2016-11-26 13:11:59 +09:00
Neil Booth
9544170c55 Prepare release-0.7.10 2016-11-26 09:53:47 +09:00
Neil Booth
292073f2c7 Log large requests and reject them 2016-11-26 09:32:29 +09:00
Neil Booth
42c00d32d2 Rewrite lib/jsonrpc.py to be a useful client too
Use for an improved electrumx_rpc.py, so it now handles responses spread
over several packets.  Also added a timeout of 5 seconds.

Fixes #43
2016-11-25 21:19:43 +09:00
Neil Booth
de4930b96d Introduce MAX_HIST environment variable. 2016-11-24 23:30:40 +09:00
Neil Booth
9e220820aa Rework futures and event handling 2016-11-23 17:52:09 +09:00
Neil Booth
bab0d162de Merge ServerManager and BlockServer 2016-11-23 17:52:08 +09:00
Neil Booth
fa30c5529b Fix missing qualifier 2016-11-20 22:01:44 +09:00
Neil Booth
dd0cd0156f Remove debug log 2016-11-20 20:56:12 +09:00
Neil Booth
11558fd9d2 Add debug log 2016-11-20 20:52:54 +09:00
Neil Booth
1dc43b3020 Show connection total in connection log
Fix typo resulting in no mempool notifications
2016-11-20 19:29:58 +09:00
Neil Booth
5eb92007ae Remove redundant log; IRC will emit one 2016-11-20 18:18:30 +09:00
Neil Booth
98fd178c62 Make mempool truly asynchronous
Mempool updates happen in parallel asynchronously to processing of
new blocks once caught up.  This means that, e.g., during the initial
slow mempool download incoming blocks can be processed and communicated
to clients without waiting for the downloaded mempool transaction
analysis to complete.  From a client's point of view the server won't
be seen as lagging.
2016-11-20 18:17:55 +09:00
Neil Booth
dd5a31d0f4 Take a little more care cleaning up connections 2016-11-19 20:56:08 +09:00
Neil Booth
ccf24fdc71 Wait 20 secs for sockets to close
10 seems a little low
2016-11-19 20:00:38 +09:00
Neil Booth
52116539d4 Clean shutdown: wait for transports to close
We give 10 seconds for everything to be cleaned up, then close forcibly.
Fixes #30
2016-11-19 17:05:47 +09:00
Neil Booth
0b52376f23 Add subscription limits 2016-11-19 10:51:16 +09:00
Neil Booth
913d156786 Convert DaemonError to RPCError
To cleanly return errors to clients
getrawtransaction has the ugly API and still does its own thing

Fixes #32
2016-11-19 08:12:13 +09:00
Neil Booth
942d5d6b00 Fix import 2016-11-19 08:07:37 +09:00
Neil Booth
063a5469cc Move mempool to be with BlockServer
BlockProcessor doesn't really need a mempool; mempools are
only useful for servers.
Set first_sync before flushing, so it goes into DB state.
Start servers immediately on catchup; mempool fills asynchronously.

Fixes #31
2016-11-19 08:02:13 +09:00
Neil Booth
3abddf4a51 Simplify sessions call 2016-11-19 00:02:56 +09:00
Neil Booth
88185b0daa Bump log threshold up; tweak log 2016-11-18 22:20:48 +09:00