RanchiMall/electrumx
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
1,093 Commits 4 Branches 138 Tags 3.9 MiB
2e12dcfdef
Commit Graph

216 Commits

Author SHA1 Message Date
Neil Booth
86f6a148b9 Separate async item processor per session 
Improve daemon wait logic

Fixes #100
 2017-02-19 19:01:41 +09:00
Neil Booth
23b7ec3fb5 get_extra_info fix 2017-02-19 10:52:34 +09:00
Neil Booth
0149f2785f A few peer handling improvements 
based on suggestions from hsmiths
Set transport to None if a connection is lost
Only subscribe to peers if the peer is verified on same network
 2017-02-19 10:21:17 +09:00
Neil Booth
9d6b353eec Fix typo pointed out by hsmiths, thanks 2017-02-18 23:42:24 +09:00
Neil Booth
3f35bc0298 More PEP8 stuff 2017-02-18 13:05:26 +09:00
Neil Booth
33cdfa4fc8 Merge branch 'nmarley-pep8' into develop 2017-02-18 12:54:01 +09:00
Neil Booth
9bd7b5a9f9 Merge branch 'pep8' of https://github.com/nmarley/electrumx into nmarley-pep8 2017-02-18 12:51:58 +09:00
Neil Booth
151da40d5b Implement peer discovery protocol 
Closes #104

DEFAULT_PORTS now a coin property
A Peer object maintains peer information
Revamp LocalRPC "peers" call to show a lot more information
Have lib/jsonrpc.py take care of handling request timeouts
Save and restore peers to a file
Loosen JSON RPC rules so we work with electrum-server and beancurd which don't follow the spec.
Handle incoming server.add_peer requests
Send server.add_peer registrations if peer doesn't have us or correct ports
Verify peers at regular intervals, forget stale peers, verify new peers or those with updated ports
If connecting via one port fails, try the other
Add socks.py for SOCKS4 and SOCKS5 proxying, so Tor servers can now be reached by TCP and SSL
Put full licence boilerplate in lib/ files
Disable IRC advertising on testnet
Serve a Tor banner file if it seems like a connection came from your tor proxy (see ENVIONMENT.rst)
Retry tor proxy hourly, and peers that are about to turn stale
Report more onion peers to a connection that seems to be combing from your tor proxy
Only report good peers to server.peers.subscribe; always report self if valid
Handle peers on the wrong network robustly
Default to 127.0.0.1 rather than localhost for Python <= 3.5.2 compatibility
Put peer name in logs of connections to it
Update docs
 2017-02-18 12:43:45 +09:00
Neil Booth
251dca4da9 Update match groups 2017-02-17 21:42:34 +09:00
Neil Booth
4dcd7f25a5 Further fix regex for hex chars 2017-02-17 07:59:49 +09:00
Neil Booth
82d57dc90f Accept IPv6 addresses in DAEMON_URL 
Fixes #126
 2017-02-16 08:00:08 +09:00
Nathan Marley
a03665696c recommendations from pycodestyle (pep8 style) 2017-02-07 20:12:50 -08:00
Neil Booth
6db9cb967c Require attributes to be specified for a coin 2017-02-06 19:48:30 +09:00
shsmith
169f5df91f update hostnames (#120) 2017-02-06 07:28:52 +09:00
Neil Booth
f1251766b6 Merge branch 'master' of github.com:kyuupichan/electrumx 2017-02-05 09:50:29 +09:00
Neil Booth
1856cbe6c0 Coin fixes/cleanup (erasmospunk) 
2 out of 3 from #118
 2017-02-05 09:41:27 +09:00
Neil Booth
a019fde853 Show uptime in RPC getinfo result 2017-02-04 10:35:36 +09:00
TheLazieR Yip
0e51c80156 Add PEERS for DASH (#117) 
* Add PEERS for DASH

* Update PEER_DEFAULT_PORTS for DASH Testnet
 2017-02-02 11:32:38 +09:00
Neil Booth
97557fa437 Don't send batches for now 2017-01-31 20:05:40 +09:00
Neil Booth
06c8eda161 Preparations for peer discovery 
DEFAULT_PORTS is a now a coin property.
Other miscellaneous updates to get things closer
 2017-01-29 06:47:43 +09:00
Neil Booth
f43c08d979 Log fatal messages once only 2017-01-29 06:08:29 +09:00
Neil Booth
e4c206e123 Namespace fix 2017-01-25 20:38:28 +09:00
Neil Booth
551c04a3bc Remove all uses of "except:" 2017-01-25 20:28:19 +09:00
Neil Booth
a7462a6ea3 Complete rewrite of JSON RPC infrastructure 
Aim to be easier for clients to use, because to do peer
discovery we must act as a client to other servers.

Split out JSON session concept from the asyncio protocol
concept.  This makes the JSON RPC support more easily
testable and usable as a stand-alone library.

In addition, support JSON RPC v1 and v2, and auto-detection
of peer's version.
 2017-01-23 23:36:02 +09:00
Petr Kracík
5343c1a286 Changed litecoin genesis block hash (#109) 2017-01-22 23:41:45 +09:00
Neil Booth
ed0646efbf Tidy up request and notification sending 2017-01-20 20:04:56 +09:00
Neil Booth
766da5ed79 Don't maintain a sub_count in controller 
Just do a tally occasionally.
It's too tricky to keep it accurate as it double-counts resubscribing to
the same address, for example
 2017-01-20 08:09:33 +09:00
Neil Booth
5f9e337457 Improve RocksDB iterator 2017-01-17 07:43:06 +09:00
Neil Booth
4eed43accb Add named-argument handling as per JSON RPC 2.0 
This involved a rewrite of the JSON RPC layer.
I took the opportunity to clean up the handling of requests
in general.  It should now be easy to return nice help from
the docstrings.

Closes issue #99
 2017-01-12 07:59:21 +09:00
Neil Booth
15051124af Make flushes and reorgs async 
Apart from the flush on shutdown and the flush when caught up,
neither of which matter, this makes flushes asynchronous.

Also, block processing for reorgs is now asynchronous.

This also removes the FORCE_REORG debug envvar; I want to
put that into the RPC interface.

Closes #102
 2017-01-09 16:15:17 +09:00
Neil Booth
540479ad23 Tweaks for prior; update README. 2017-01-08 12:10:56 +09:00
protonn
1aa55f2976 add digibyte and argentum (#96) 
* add DigiByte and Argentum
 2017-01-08 12:00:41 +09:00
Neil Booth
dcc2c92bd6 Rename NETWORK envvar to NET 
For consistency with lib/coins.py

SegWit testnet needs its own NET
 2017-01-08 10:07:58 +09:00
Neil Booth
4d62019b07 Check Genesis hash 
Testnet RPC_PORT is 18332
 2017-01-08 10:07:57 +09:00
Neil Booth
852753cb94 Implement deserialization of SegWit transactions 
tx_hash needs to be that of the prior serialization, so
need to change internal read_block API.

Bitcoin core 0.13.1 broke backwards compat of the RPC interface.
Closes #92
 2017-01-08 10:07:57 +09:00
Neil Booth
7f9cb9cd9d Generic header length 
If header lengths change in the future, only one function needs
to change in coins.py.
 2017-01-07 11:51:12 +09:00
Neil Booth
c8c4199503 New VirtualFile abstraction 
Use for headers, tx counts and tx hashes.
 2017-01-07 11:51:11 +09:00
Neil Booth
2b2909f1c4 Move to hashX 
The hashX of a script is the first 11 bytes of its sha256 hash.

Closes #72.
 2017-01-07 11:51:01 +09:00
Neil Booth
de201a247f Double fetch size of mempool. 
Set testnet genesis hash.
 2017-01-07 11:51:01 +09:00
Neil Booth
8bd2e7ee06 Add constants for bitcoin testnet 2017-01-07 11:51:01 +09:00
Neil Booth
50f02747de Throttle abusive logging 2016-12-21 19:47:25 +09:00
Neil Booth
378372e666 Logs: show IPv6 address + port correctly 
Closes #79
 2016-12-18 08:07:28 +09:00
Neil Booth
1ebebf08d9 Improve logging 2016-12-17 15:26:17 +09:00
Neil Booth
f600d3bd87 Fix a couple of issues with 0.9.0 
Fixes #69
 2016-12-12 07:24:00 +09:00
Neil Booth
1ef6a4d785 Split mempool out into new file. 
Rework mempool handling
 2016-12-12 00:05:42 +09:00
Neil Booth
263e88ad57 Tweak request handling 
Pause serving sessions whose socket buffer is full (anti-DoS)
Serve requests in batches of 8
Don't store the session in the request
RPC has priority 0; every other session at least 1
Periodically consolidate small session groups into 1
 2016-12-10 18:06:13 +09:00
Neil Booth
cfb92a139f Prefer transport.abort() 2016-12-10 13:52:05 +09:00
Neil Booth
c6093639b5 Don't set socket timeout 2016-12-10 13:39:46 +09:00
Neil Booth
99d7e65de8 Give sockets a grace period to close 
Defaults to 15 seconds for now.
Update docs on systemd and process limits.
Fixes #62
 2016-12-10 08:52:24 +09:00
Neil Booth
2a461bd98c Serve sessions in a priority queue. 
Add new RPC command requests.
Adjust sessions RPC command to show pending requests.
Only keep first 15 chars in version string.
Set socket timeout
Try more forceful closing of socket
Fixes #67
 2016-12-09 22:06:04 +09:00
Neil Booth
1a9e8cdcd4 More work 2016-12-09 08:06:03 +09:00
Neil Booth
cbb1e504cc Cache headers. 2016-12-08 06:31:15 +09:00
Neil Booth
e40db63beb Queue requests, which have a process method. 2016-12-08 00:29:46 +09:00
Neil Booth
4e512343be Fix other JSON bugs introduced in 0.8.5 
Further clean up the JSON code.
 2016-12-07 19:44:36 +09:00
Shane Moore
e17a9ee2c7 Update RPC client for JSON rework and add missing arg in json_request_text 
Fixes #60
 2016-12-07 00:25:43 -08:00
Neil Booth
f904ce2f5d Rework the JSON layer 
Batch requests are now processed and encoded incrementally.
Their bandwidth usage is also incrementally added, and so overlarge
responses are rejected as soon as they become too large.
JSON text is also more memory efficient than the python data
structures they represent.
 2016-12-07 08:11:30 +09:00
Neil Booth
b116040365 Clean up param verification code 2016-12-07 08:11:29 +09:00
Neil Booth
be8adbbcd5 Fix some typos. 2016-12-04 23:41:07 +09:00
Neil Booth
ab9d9f7c07 Reduce bandwith usage over the bandwith interval 
So if e.g. your limit is 10MB per hour, then every minute
your cumulative usage will be reduced by 1/6MB.
 2016-12-03 23:32:20 +09:00
Neil Booth
c4e7878407 Add session ID for each session 
Show it in the logs with each session-specific message.
Show the sessions ID in the sessions list with an L suffix if logging.
Add RPC commands to toggle logging of, and disconnect, a session.
Closes #55
 2016-12-03 14:32:09 +09:00
Neil Booth
3d2824218b Close stale sessions 
New envvar SESSION_TIMEOUT
A session with no activity is cut off after this time
Fixes #56
 2016-12-03 13:06:20 +09:00
Neil Booth
625508f5c6 Send no ID for batch request errors 
Fixes #54
 2016-12-03 10:56:55 +09:00
Neil Booth
a412531ccb Fix typos in docs 2016-11-30 21:06:25 +09:00
Neil Booth
15a88600eb Upped read buffer limit to 1m bytes 
This accomodates large tx sends of up to 500k bytes.
However they are unlikely to be propogated by your daemon as the
default relay limit it imposes is 100K.
 2016-11-30 07:26:34 +09:00
Neil Booth
b65bcda504 Add per-session bandwidth limits 2016-11-29 22:19:12 +09:00
Neil Booth
c9a10be5ba Introduce incoming buffer size limit 
- incoming buffered network requests limited to 150,000 bytes
  which I believe is ample for a genuine client
- if exceeded, the connection is dropped
- raise outgoing data limit for RPC connections to 5 MB - expect
  sessions calls can be long and connection is implicitly trusted
- similarly raise incoming buffered data limit to 5 MB for RPC
  connections
 2016-11-29 20:24:58 +09:00
Neil Booth
c6be118bab Tweak Litecoin settings 2016-11-29 07:28:09 +09:00
Santzi
f1911140ef Updated Litecoin mainnet parameters 2016-11-27 19:13:38 +02:00
Neil Booth
b2032b0a66 Fix 2 JSON RPC issues 2016-11-27 21:54:05 +09:00
Neil Booth
292073f2c7 Log large requests and reject them 2016-11-26 09:32:29 +09:00
Neil Booth
59733e4609 Move bitcoin-specific coin defaults to Bitcoin 2016-11-26 08:32:30 +09:00
TheLazieR Yip
dfaf36ae16 Remove header_hashes 2016-11-25 07:25:57 -05:00
TheLazieR Yip
6b63ecb439 Add header_hash and header_prevhash class methods 2016-11-25 07:25:56 -05:00
Neil Booth
42c00d32d2 Rewrite lib/jsonrpc.py to be a useful client too 
Use for an improved electrumx_rpc.py, so it now handles responses spread
over several packets.  Also added a timeout of 5 seconds.

Fixes #43
 2016-11-25 21:19:43 +09:00
Neil Booth
cebf93cdc3 Handle abuse better 
- close connections on any encoding error
- if the connection is closing do not process buffered requests
 2016-11-23 11:52:17 +09:00
TheLazieR Yip
f075f0bb5f Add coin's IRC parameters 2016-11-23 08:15:03 +07:00
Neil Booth
11ed1c4cae Don't log decoding errors any more 
The can be frequent.
Close connections once they reach 10 errors.
 2016-11-22 06:45:22 +09:00
TheLazieR Yip
5aae539476 Add DEFAULT_RPC_PORT for DASH 2016-11-20 19:11:22 +07:00
Neil Booth
87cdd2709d Implement daemon failover 
Daemon URLs can be comma-separated in the DAEMON_URL env var.
Surrounding whitespace is stripped.
http:// is preprended if missing.
The coin's default port is supplied if missing.
A trailing / is supplied if missing.
Closes #33
 2016-11-20 20:52:54 +09:00
Neil Booth
3d87e299ea Move formatted_time to library 2016-11-19 15:42:06 +09:00
Neil Booth
66c493dbf4 Make the default reorg_limit a coin property 2016-11-18 19:21:38 +09:00
Neil Booth
95bb1588ab Add missing await 2016-11-14 19:15:36 +09:00
Neil Booth
41fb143947 Fix typo 2016-11-13 23:18:04 +09:00
Neil Booth
0c2b4afbdc Remove a debug log 2016-11-13 22:40:26 +09:00
Neil Booth
9ac4718218 OP_RETURN can be spent inside false conditions 
See tx 928ed84cd4c48beb0d3494ccc17cc1e06b1473f9dc118db9bb56972395ede461
 2016-11-13 21:55:18 +09:00
Neil Booth
85786b87a2 Save all UTXOs 
Change the DB version
 2016-11-13 20:35:03 +09:00
Neil Booth
57dd4ece9c Avoid annoying exception on shutdown 2016-11-13 17:52:08 +09:00
Neil Booth
fdb5b33fba Prepare 0.4.2 2016-11-13 15:37:51 +09:00
Neil Booth
ecb70fcef9 Merge remote-tracking branch 'github/master' into release-0.4.2 2016-11-13 15:31:54 +09:00
Neil Booth
3b6ab77e47 Break out JSONRPC into own file 
- improved handling of JSON 2.0 RPC protocol
- permits batched requests (not yet supported by Electrum client)
 2016-11-13 13:39:37 +09:00
TheLazieR Yip
66bf2570e5 Move dash stuff into Dash class 
Modify DashTestnet class to use Dash class
 2016-11-13 08:56:26 +07:00
TheLazieR Yip
ff834519aa Update for DASH 
- Update information for both mainnet and testnet
- Use x11_hash module when network name is Dash to process header_hashes
 2016-11-13 00:51:53 +07:00
Neil Booth
8e6e8329ac Remove dead code. 
Also use native python3 functions to speed up 2 lib functions
 2016-11-08 19:14:52 +09:00
Neil Booth
9acd91615c Merge remote-tracking branch 'github/master' into release-0.2.1 
Include's bauerj's storage.py improvements.
 2016-11-07 23:11:34 +09:00
Johann Bauer
f6f674577f Fix RocksDB and LMDB 2016-11-06 18:16:43 +01:00
Neil Booth
e7a516c46e Need to catch script errors 2016-11-06 16:01:02 +09:00
Neil Booth
e2ef9dceaf Rework assertion logic in coins.py 2016-11-06 15:34:20 +09:00
Neil Booth
3ab07c1fb6 Speed up script parsing for ~3% faster throughput 
Also improves the coin abstraction
 2016-11-06 14:32:27 +09:00
Neil Booth
0ff579604c Speed up tx.py; giving ~ 5% higher tx/s in memory 2016-11-06 12:07:53 +09:00
Neil Booth
6946863879 Add setup.py 
Fix electrum_rpc.py
Rename server_main.py to electrumx_server.py
 2016-11-05 18:00:03 +09:00
Neil Booth
c0568daec3 Rework the DB API a bit 2016-11-03 19:09:21 +09:00
Neil Booth
be2475f617 Handle utxo.get_address 2016-11-03 19:08:25 +09:00
Neil Booth
ceecdc54ac Server work 
Avoid touching the block preprocessor hot loop for now
 2016-11-03 17:26:50 +09:00
Neil Booth
d2ebb80fac Extend copyright notice; improve comments 2016-11-03 16:45:06 +09:00
Neil Booth
2b45698962 Remove Outpoint as a separate object 
Hopefully this is a little more efficient
 2016-10-28 07:35:18 +09:00
Neil Booth
30d1b7f77c Add proper block chaining check 2016-10-27 07:38:39 +09:00
Neil Booth
6ad8c16c47 Handle a couple more requests. 2016-10-27 07:25:34 +09:00
Neil Booth
334ffdaa4f Begin work on proper server 2016-10-17 20:02:42 +09:00
Neil Booth
965b15023e Update notes / comments 2016-10-15 14:45:32 +09:00
Neil Booth
6b513a91d0 CACHE_MB is now the environment var. 2016-10-14 07:56:25 +09:00
Neil Booth
3fce298835 Better time est. A couple of fixes. 2016-10-12 18:59:15 +09:00
Neil Booth
d8e9eb7796 Accurate cache accounting 2016-10-11 17:38:03 +09:00
Neil Booth
6644102441 Use hash168 to distinguish address types in DB 2016-10-10 11:42:33 +09:00
Neil Booth
ca11710fdd Print balance in query tool 2016-10-09 09:14:59 +09:00
Neil Booth
64b7554b73 Fix some typos 2016-10-08 17:34:12 +09:00
Neil Booth
a3dbc68614 Initial revision 2016-10-08 17:17:43 +09:00