Commit Graph

117 Commits

Author SHA1 Message Date
Neil Booth
1c79a2d47e Fix delayed sessions issue for good 2016-12-14 06:07:37 +09:00
Neil Booth
ba862343b4 Fix issue for paused connections
Prepare 0.9.5
2016-12-14 00:07:26 +09:00
Neil Booth
e2f4847632 New env var MAX_SESSIONS
When the number of sessions reaches MAX_SESSIONS, which defaults
to 1,000, turn off TCP and SSL listening sockets to prevent new
connections.  When the session count falls below a low watermark,
currently 90% of MAX_SESSIONS, the listening sockets will be re-opened.
Helps prevent DoS and limit open file usage.

Bug fix: do not start serving paused connections until the buffer
socket is sufficiently drained.  Also, loop.
2016-12-13 22:47:03 +09:00
Neil Booth
067814e7d9 Remove numpeers and numsessions RPC calls
getinfo has that information and more.
2016-12-13 21:38:02 +09:00
Neil Booth
aaaa914cfa Tweak some constants to reduce group count 2016-12-13 20:09:40 +09:00
Neil Booth
f600d3bd87 Fix a couple of issues with 0.9.0
Fixes #69
2016-12-12 07:24:00 +09:00
Neil Booth
1ef6a4d785 Split mempool out into new file.
Rework mempool handling
2016-12-12 00:05:42 +09:00
Neil Booth
ed3db731c2 Prepare 0.8.12 2016-12-10 18:06:14 +09:00
Neil Booth
263e88ad57 Tweak request handling
Pause serving sessions whose socket buffer is full (anti-DoS)
Serve requests in batches of 8
Don't store the session in the request
RPC has priority 0; every other session at least 1
Periodically consolidate small session groups into 1
2016-12-10 18:06:13 +09:00
Neil Booth
b3b3f047c2 Better columns 2016-12-10 18:06:13 +09:00
Neil Booth
cfb92a139f Prefer transport.abort() 2016-12-10 13:52:05 +09:00
Neil Booth
89d83349e1 Prepare 0.8.11 2016-12-10 11:08:44 +09:00
Neil Booth
d1e314a48b Show group session count in the groups RPC call 2016-12-10 11:05:57 +09:00
Neil Booth
c181f8df82 Put a try/except around socket.shutdown 2016-12-10 10:43:01 +09:00
Neil Booth
6fcb8b0574 Fix cut-n-pasteo 2016-12-10 09:07:08 +09:00
Neil Booth
99d7e65de8 Give sockets a grace period to close
Defaults to 15 seconds for now.
Update docs on systemd and process limits.
Fixes #62
2016-12-10 08:52:24 +09:00
Neil Booth
c523064632 Improve RPC groups call
Show columns correctly for RPC sessions call
2016-12-10 08:52:23 +09:00
Neil Booth
9bdad7ad66 Fix column ordering. 2016-12-09 22:40:36 +09:00
Neil Booth
2a461bd98c Serve sessions in a priority queue.
Add new RPC command requests.
Adjust sessions RPC command to show pending requests.
Only keep first 15 chars in version string.
Set socket timeout
Try more forceful closing of socket
Fixes #67
2016-12-09 22:06:04 +09:00
Neil Booth
1a9e8cdcd4 More work 2016-12-09 08:06:03 +09:00
Neil Booth
cbb1e504cc Cache headers. 2016-12-08 06:31:15 +09:00
Neil Booth
e40db63beb Queue requests, which have a process method. 2016-12-08 00:29:46 +09:00
Neil Booth
4e512343be Fix other JSON bugs introduced in 0.8.5
Further clean up the JSON code.
2016-12-07 19:44:36 +09:00
Neil Booth
b2e4d32de6 Remove special-casing of all_seeing_eye 2016-12-07 08:11:30 +09:00
Neil Booth
f904ce2f5d Rework the JSON layer
Batch requests are now processed and encoded incrementally.
Their bandwidth usage is also incrementally added, and so overlarge
responses are rejected as soon as they become too large.
JSON text is also more memory efficient than the python data
structures they represent.
2016-12-07 08:11:30 +09:00
Neil Booth
b116040365 Clean up param verification code 2016-12-07 08:11:29 +09:00
Neil Booth
1f1f0f42f8 Remove invalidated history cache on notifications 2016-12-06 06:36:11 +09:00
Neil Booth
643c990684 Revert "Add feature to rate limit connections"
This reverts commit a82ab0b075.
2016-12-05 23:27:36 +09:00
Neil Booth
be8adbbcd5 Fix some typos. 2016-12-04 23:41:07 +09:00
Neil Booth
a82ab0b075 Add feature to rate limit connections 2016-12-04 21:14:45 +09:00
Neil Booth
79a9e91994 Handle closing sessions a little differently
Move connection type to first flag letter.

All seeing eye is not stale.
2016-12-04 11:57:25 +09:00
Neil Booth
fcf696d3f3 Don't require 2 params for version requests 2016-12-04 10:54:22 +09:00
Neil Booth
656f749c70 Add LRU cache for history 2016-12-03 20:20:28 +09:00
Neil Booth
04369dd228 Forcefully drop stale sessions or if shutting down
Don't wait for the socket
2016-12-03 18:33:23 +09:00
Neil Booth
aaf0592f52 Fix typo 2016-12-03 15:10:57 +09:00
Neil Booth
c4e7878407 Add session ID for each session
Show it in the logs with each session-specific message.
Show the sessions ID in the sessions list with an L suffix if logging.
Add RPC commands to toggle logging of, and disconnect, a session.
Closes #55
2016-12-03 14:32:09 +09:00
Neil Booth
3d2824218b Close stale sessions
New envvar SESSION_TIMEOUT
A session with no activity is cut off after this time
Fixes #56
2016-12-03 13:06:20 +09:00
Neil Booth
c16792fb89 Only wait 30 secsonds on shutdown 2016-12-03 11:03:27 +09:00
Neil Booth
62a531b1d0 Revert "Fix ref leak in mempool updates"
This reverts commit 6c95644ae0.
2016-12-02 18:43:15 +09:00
Neil Booth
6c95644ae0 Fix ref leak in mempool updates 2016-12-02 07:22:30 +09:00
Neil Booth
0892b12036 Replace $DAEMON_VERSION and $DAEMON_SUBVERSION
in banner file.
2016-11-29 23:03:05 +09:00
Neil Booth
b65bcda504 Add per-session bandwidth limits 2016-11-29 22:19:12 +09:00
Neil Booth
c9a10be5ba Introduce incoming buffer size limit
- incoming buffered network requests limited to 150,000 bytes
  which I believe is ample for a genuine client
- if exceeded, the connection is dropped
- raise outgoing data limit for RPC connections to 5 MB - expect
  sessions calls can be long and connection is implicitly trusted
- similarly raise incoming buffered data limit to 5 MB for RPC
  connections
2016-11-29 20:24:58 +09:00
Neil Booth
c08ade5861 Add session logging facility
Move session logging code to protocol.py from electrum_rpc.py
Use it for periodic logging controlled by envvar LOG_SESSIONS
For each session, track sent transaction stats and show that
per-session instead of errors
2016-11-29 08:00:54 +09:00
Neil Booth
6aef79461f Don't shut down block processor by cancellation
The block processor needs to be able to close cleanly, and not
mid-block.  In order to be able to yield whilst processing blocks
we cannot forcefully close its coroutine with a cancellation.
2016-11-27 12:20:16 +09:00
Neil Booth
9eb701f220 Replace $VERSION in the banner file
with the ElectrumX version string
2016-11-26 18:19:53 +09:00
Neil Booth
59244cc24e Move bulk of FS flush code to db.py
More logically belongs there.
Have servers use the flushed DB height not the block processor's height.
Get all headers only from disk.
2016-11-26 13:11:59 +09:00
Neil Booth
9544170c55 Prepare release-0.7.10 2016-11-26 09:53:47 +09:00
Neil Booth
292073f2c7 Log large requests and reject them 2016-11-26 09:32:29 +09:00
Neil Booth
42c00d32d2 Rewrite lib/jsonrpc.py to be a useful client too
Use for an improved electrumx_rpc.py, so it now handles responses spread
over several packets.  Also added a timeout of 5 seconds.

Fixes #43
2016-11-25 21:19:43 +09:00