Commit Graph

48 Commits

Author SHA1 Message Date
Neil Booth
135ab68f74 Simple protocol negotiation and setting of handlers
It turns out clients pass 0.10 instead of 1.0 as the protocol version.
Distinguish some handlers for 1.0 and 1.1 protocols.
Log protocol version request
Add tests of new library function
2017-09-06 16:24:56 +09:00
Neil Booth
56130e4a66 Fix uninitialized variable 2017-03-03 08:03:48 +09:00
Neil Booth
86f6a148b9 Separate async item processor per session
Improve daemon wait logic

Fixes #100
2017-02-19 19:01:41 +09:00
Neil Booth
23b7ec3fb5 get_extra_info fix 2017-02-19 10:52:34 +09:00
Neil Booth
0149f2785f A few peer handling improvements
based on suggestions from hsmiths
Set transport to None if a connection is lost
Only subscribe to peers if the peer is verified on same network
2017-02-19 10:21:17 +09:00
Neil Booth
151da40d5b Implement peer discovery protocol
Closes #104

DEFAULT_PORTS now a coin property
A Peer object maintains peer information
Revamp LocalRPC "peers" call to show a lot more information
Have lib/jsonrpc.py take care of handling request timeouts
Save and restore peers to a file
Loosen JSON RPC rules so we work with electrum-server and beancurd which don't follow the spec.
Handle incoming server.add_peer requests
Send server.add_peer registrations if peer doesn't have us or correct ports
Verify peers at regular intervals, forget stale peers, verify new peers or those with updated ports
If connecting via one port fails, try the other
Add socks.py for SOCKS4 and SOCKS5 proxying, so Tor servers can now be reached by TCP and SSL
Put full licence boilerplate in lib/ files
Disable IRC advertising on testnet
Serve a Tor banner file if it seems like a connection came from your tor proxy (see ENVIONMENT.rst)
Retry tor proxy hourly, and peers that are about to turn stale
Report more onion peers to a connection that seems to be combing from your tor proxy
Only report good peers to server.peers.subscribe; always report self if valid
Handle peers on the wrong network robustly
Default to 127.0.0.1 rather than localhost for Python <= 3.5.2 compatibility
Put peer name in logs of connections to it
Update docs
2017-02-18 12:43:45 +09:00
Neil Booth
97557fa437 Don't send batches for now 2017-01-31 20:05:40 +09:00
Neil Booth
06c8eda161 Preparations for peer discovery
DEFAULT_PORTS is a now a coin property.
Other miscellaneous updates to get things closer
2017-01-29 06:47:43 +09:00
Neil Booth
f43c08d979 Log fatal messages once only 2017-01-29 06:08:29 +09:00
Neil Booth
e4c206e123 Namespace fix 2017-01-25 20:38:28 +09:00
Neil Booth
a7462a6ea3 Complete rewrite of JSON RPC infrastructure
Aim to be easier for clients to use, because to do peer
discovery we must act as a client to other servers.

Split out JSON session concept from the asyncio protocol
concept.  This makes the JSON RPC support more easily
testable and usable as a stand-alone library.

In addition, support JSON RPC v1 and v2, and auto-detection
of peer's version.
2017-01-23 23:36:02 +09:00
Neil Booth
ed0646efbf Tidy up request and notification sending 2017-01-20 20:04:56 +09:00
Neil Booth
4eed43accb Add named-argument handling as per JSON RPC 2.0
This involved a rewrite of the JSON RPC layer.
I took the opportunity to clean up the handling of requests
in general.  It should now be easy to return nice help from
the docstrings.

Closes issue #99
2017-01-12 07:59:21 +09:00
Neil Booth
2b2909f1c4 Move to hashX
The hashX of a script is the first 11 bytes of its sha256 hash.

Closes #72.
2017-01-07 11:51:01 +09:00
Neil Booth
50f02747de Throttle abusive logging 2016-12-21 19:47:25 +09:00
Neil Booth
378372e666 Logs: show IPv6 address + port correctly
Closes #79
2016-12-18 08:07:28 +09:00
Neil Booth
1ebebf08d9 Improve logging 2016-12-17 15:26:17 +09:00
Neil Booth
263e88ad57 Tweak request handling
Pause serving sessions whose socket buffer is full (anti-DoS)
Serve requests in batches of 8
Don't store the session in the request
RPC has priority 0; every other session at least 1
Periodically consolidate small session groups into 1
2016-12-10 18:06:13 +09:00
Neil Booth
cfb92a139f Prefer transport.abort() 2016-12-10 13:52:05 +09:00
Neil Booth
c6093639b5 Don't set socket timeout 2016-12-10 13:39:46 +09:00
Neil Booth
99d7e65de8 Give sockets a grace period to close
Defaults to 15 seconds for now.
Update docs on systemd and process limits.
Fixes #62
2016-12-10 08:52:24 +09:00
Neil Booth
2a461bd98c Serve sessions in a priority queue.
Add new RPC command requests.
Adjust sessions RPC command to show pending requests.
Only keep first 15 chars in version string.
Set socket timeout
Try more forceful closing of socket
Fixes #67
2016-12-09 22:06:04 +09:00
Neil Booth
1a9e8cdcd4 More work 2016-12-09 08:06:03 +09:00
Neil Booth
cbb1e504cc Cache headers. 2016-12-08 06:31:15 +09:00
Neil Booth
e40db63beb Queue requests, which have a process method. 2016-12-08 00:29:46 +09:00
Neil Booth
4e512343be Fix other JSON bugs introduced in 0.8.5
Further clean up the JSON code.
2016-12-07 19:44:36 +09:00
Shane Moore
e17a9ee2c7 Update RPC client for JSON rework and add missing arg in json_request_text
Fixes #60
2016-12-07 00:25:43 -08:00
Neil Booth
f904ce2f5d Rework the JSON layer
Batch requests are now processed and encoded incrementally.
Their bandwidth usage is also incrementally added, and so overlarge
responses are rejected as soon as they become too large.
JSON text is also more memory efficient than the python data
structures they represent.
2016-12-07 08:11:30 +09:00
Neil Booth
b116040365 Clean up param verification code 2016-12-07 08:11:29 +09:00
Neil Booth
be8adbbcd5 Fix some typos. 2016-12-04 23:41:07 +09:00
Neil Booth
ab9d9f7c07 Reduce bandwith usage over the bandwith interval
So if e.g. your limit is 10MB per hour, then every minute
your cumulative usage will be reduced by 1/6MB.
2016-12-03 23:32:20 +09:00
Neil Booth
c4e7878407 Add session ID for each session
Show it in the logs with each session-specific message.
Show the sessions ID in the sessions list with an L suffix if logging.
Add RPC commands to toggle logging of, and disconnect, a session.
Closes #55
2016-12-03 14:32:09 +09:00
Neil Booth
3d2824218b Close stale sessions
New envvar SESSION_TIMEOUT
A session with no activity is cut off after this time
Fixes #56
2016-12-03 13:06:20 +09:00
Neil Booth
625508f5c6 Send no ID for batch request errors
Fixes #54
2016-12-03 10:56:55 +09:00
Neil Booth
a412531ccb Fix typos in docs 2016-11-30 21:06:25 +09:00
Neil Booth
15a88600eb Upped read buffer limit to 1m bytes
This accomodates large tx sends of up to 500k bytes.
However they are unlikely to be propogated by your daemon as the
default relay limit it imposes is 100K.
2016-11-30 07:26:34 +09:00
Neil Booth
b65bcda504 Add per-session bandwidth limits 2016-11-29 22:19:12 +09:00
Neil Booth
c9a10be5ba Introduce incoming buffer size limit
- incoming buffered network requests limited to 150,000 bytes
  which I believe is ample for a genuine client
- if exceeded, the connection is dropped
- raise outgoing data limit for RPC connections to 5 MB - expect
  sessions calls can be long and connection is implicitly trusted
- similarly raise incoming buffered data limit to 5 MB for RPC
  connections
2016-11-29 20:24:58 +09:00
Neil Booth
b2032b0a66 Fix 2 JSON RPC issues 2016-11-27 21:54:05 +09:00
Neil Booth
292073f2c7 Log large requests and reject them 2016-11-26 09:32:29 +09:00
Neil Booth
42c00d32d2 Rewrite lib/jsonrpc.py to be a useful client too
Use for an improved electrumx_rpc.py, so it now handles responses spread
over several packets.  Also added a timeout of 5 seconds.

Fixes #43
2016-11-25 21:19:43 +09:00
Neil Booth
cebf93cdc3 Handle abuse better
- close connections on any encoding error
- if the connection is closing do not process buffered requests
2016-11-23 11:52:17 +09:00
Neil Booth
11ed1c4cae Don't log decoding errors any more
The can be frequent.
Close connections once they reach 10 errors.
2016-11-22 06:45:22 +09:00
Neil Booth
95bb1588ab Add missing await 2016-11-14 19:15:36 +09:00
Neil Booth
41fb143947 Fix typo 2016-11-13 23:18:04 +09:00
Neil Booth
0c2b4afbdc Remove a debug log 2016-11-13 22:40:26 +09:00
Neil Booth
57dd4ece9c Avoid annoying exception on shutdown 2016-11-13 17:52:08 +09:00
Neil Booth
3b6ab77e47 Break out JSONRPC into own file
- improved handling of JSON 2.0 RPC protocol
- permits batched requests (not yet supported by Electrum client)
2016-11-13 13:39:37 +09:00