RanchiMall/electrumx
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
1,093 Commits 4 Branches 138 Tags 3.9 MiB
2e12dcfdef
Commit Graph

641 Commits

Author SHA1 Message Date
Neil Booth
9abc1dc11e REPORT_HOST no longer defaults to HOST 
Cleanup of identity handling.  It is now possible to specify
a Tor identity and no clearnet identity.
 2017-04-01 12:03:53 +09:00
Neil Booth
abba36ac6c Relax the get_chunk restriction based on client 
Closes #162
 2017-04-01 11:17:57 +09:00
Neil Booth
55cedfea9c Have Daemon work with aiohttp 1 and 2 
Fixes #163
 2017-04-01 10:43:34 +09:00
Neil Booth
594b66236f Prepare 1.0.5 2017-03-28 12:30:06 +09:00
Neil Booth
8e00affc1a Fix the diagnostic looping in PeerSession 
Fixes #160
 2017-03-28 11:14:50 +09:00
Neil Booth
23a408c572 More logging 2017-03-27 13:04:59 +09:00
Neil Booth
860a4e8e93 Prepare 1.0.4 2017-03-26 11:51:18 +09:00
Neil Booth
2650459012 Peer discovery fixes 
Change last_connect to mean last connection as its name implies,
not last connection that wasn't bad.
Keep bad peers around for 3 tries.

Improve diagnostic
 2017-03-26 11:42:23 +09:00
Neil Booth
79bce5335e Fix missing self 2017-03-25 20:52:26 +09:00
Neil Booth
fa1a5bd3e4 Prepare 1.0.3 2017-03-25 20:27:33 +09:00
Neil Booth
3236989165 Fix bad peer looping 
JWU42 pointed out an issue where peer discovery could get in a
failure loop for bad peers; this fixes the the root cause and
the immediate retries
 2017-03-25 20:22:38 +09:00
Neil Booth
7b263c5c49 Prepare 1.0.2 2017-03-25 13:28:43 +09:00
Neil Booth
e4947cb9ef Sanitize the maximum number of sessions 
Reduce the maximum number of sessions to permit if running with
a tight rlimit, to avoid hitting open file limits.  Log when
doing so.

Closes #158
 2017-03-25 13:28:42 +09:00
Neil Booth
f04ff6f5b3 Don't add_peer to ourself 2017-03-25 13:28:42 +09:00
Neil Booth
8236aaf234 Be more strict on form of features dictionary 2017-03-25 13:28:42 +09:00
Neil Booth
be5397a853 Validate hostnames before accepting a new peer 
Don't retain non-public addresses

Closes #157
 2017-03-25 11:58:54 +09:00
Neil Booth
7a2f29aabe Be stricter accepting add_peer requests 
- rate-limit onion add_peer requests
- for clearnet peers only accept if the peer resolves to the
  source address
 2017-03-25 11:36:14 +09:00
Neil Booth
84c201f665 Improve diagnostic 2017-03-25 11:35:56 +09:00
John L. Jegutanis
a820829e0e Dynamic header support 
Block headers can have a dynamic size that is being indexed on a
new meta file "headers_offsets".

The offsets are 64 bits in order to accommodate coins with big
headers that will accumulate GBs of header data after some years.

Closes #128
 2017-03-25 11:35:55 +09:00
Neil Booth
37602d4eae A couple of tweaks to the prior commit. 2017-03-25 11:35:55 +09:00
John L. Jegutanis
ed7d8a319d Refactor block parsing API 2017-03-25 11:35:55 +09:00
Neil Booth
127b4de745 Add new RPC method: add_peer 2017-03-25 11:35:54 +09:00
Neil Booth
060d32211f Check height for all peers 
Closes #152
 2017-03-25 11:35:54 +09:00
Neil Booth
d198b95798 Reduce new peers per source limit to 2 by default 2017-03-25 11:35:54 +09:00
Neil Booth
9238fe397d Drop ports from top level of features 2017-03-25 11:35:54 +09:00
Neil Booth
e0fd64d29a Rate-limit add_peer calls randomly 
Prepare 1.0.1
 2017-03-22 07:59:56 +09:00
Neil Booth
294212d421 Fix discovery of base of reorgs 2017-03-21 22:17:01 +09:00
Neil Booth
5f56689e9c Don't permit common invalid REPORT_HOST values 2017-03-21 22:17:01 +09:00
Neil Booth
30bb832ef8 Prepare 1.0 2017-03-05 13:22:09 +09:00
Neil Booth
90d1d2641b Prepare 0.99.4 2017-03-04 11:08:03 +09:00
Neil Booth
27f94cfb04 Take up to 2 peers randomly from each bucket 
Bumped up from 1.
Restore STALE_SECS to 1 day
 2017-03-01 20:20:30 +09:00
Neil Booth
51c1e9a59b Prepare 0.99.3 2017-03-01 07:32:28 +09:00
Neil Booth
39bcdb1b6a Require Python 3.5.3 
3.5.2 has various socket and API issues affecting peer discovery

Closes #135
 2017-03-01 07:23:11 +09:00
Neil Booth
be9d063457 Rename function 2017-03-01 06:44:38 +09:00
Neil Booth
31b978f21e Remove debug logging and peers semaphore. 
Increase STALE_SECS to 4 hrs.  Will revert to 1 day soon.
 2017-02-28 07:27:36 +09:00
Neil Booth
ddbc13653f Merge branch 'develop' 2017-02-27 19:20:04 +09:00
Neil Booth
2c1481859f More verbose peer logging 
Force more reconnects.  Restrict semaphore to 2.
 2017-02-27 19:15:07 +09:00
Neil Booth
9bc326430d Import JSONRPC 2017-02-27 19:14:56 +09:00
John L. Jegutanis
472c738479 fix crash in IRC initialization (#141) 2017-02-27 07:24:20 +09:00
Neil Booth
be95697002 Prepare 0.99.2 2017-02-26 15:29:52 +09:00
Neil Booth
e2d640dcb3 Don't announce self if not public 
Show all hosts add_peer gives
 2017-02-26 15:26:11 +09:00
Neil Booth
b3005fbb3e Fix typo 2017-02-26 13:18:06 +09:00
Neil Booth
b0039a25a7 Prepare 0.99.1 2017-02-26 10:49:27 +09:00
Neil Booth
0145098165 Add debug output 
Idea is to help understand #135
 2017-02-26 10:42:32 +09:00
Neil Booth
d36d222489 Maintain more than one self-identity 
Rather than just retaining our clearnet identity from the environment,
maintain any tor one as well.  This is a little cleaner and avoids
confusion over port assignment when converting to real names.

Fixes #136
 2017-02-26 10:22:22 +09:00
Neil Booth
04e7dbe99e Only serve chunks in forward direction 
This disconnects bad clients that are on the wrong network
for some reason, before they consume enormous bandwidth.

Closes #132
 2017-02-26 10:16:09 +09:00
fr3aker
4c8b5a6439 fix tor coin peers always scheduled for check (#139) 
fixes #138
 2017-02-26 10:16:08 +09:00
Neil Booth
5bcbdea16a Only set last_connect if the connection was good 2017-02-20 05:37:53 +09:00
Neil Booth
6f32f99a06 Prepare 0.99 2017-02-19 19:42:26 +09:00
Neil Booth
5662f1fa88 Add facility to disable peer discovery 
and/or self announcement.
 2017-02-19 19:02:01 +09:00
Neil Booth
86f6a148b9 Separate async item processor per session 
Improve daemon wait logic

Fixes #100
 2017-02-19 19:01:41 +09:00
Neil Booth
f6a60d9b94 Prepare 0.11.4 2017-02-19 10:25:02 +09:00
Neil Booth
0149f2785f A few peer handling improvements 
based on suggestions from hsmiths
Set transport to None if a connection is lost
Only subscribe to peers if the peer is verified on same network
 2017-02-19 10:21:17 +09:00
Neil Booth
9d6b353eec Fix typo pointed out by hsmiths, thanks 2017-02-18 23:42:24 +09:00
Neil Booth
589f695aaf Prepare 0.11.2 2017-02-18 23:20:07 +09:00
Neil Booth
68a8835db6 Implement script hash subscriptions 
Best considered experimental

Closes #124
 2017-02-18 23:16:20 +09:00
Neil Booth
eefa86ffbe Remove dead code 2017-02-18 17:03:10 +09:00
Neil Booth
140304460d Remove debug line 2017-02-18 16:23:39 +09:00
Neil Booth
a99e8bf45f Prepare 0.11.1 2017-02-18 16:14:26 +09:00
Neil Booth
f3cdd97ff9 Report unconfirmed parents correctly. 
Also, send a notification to the client if the unconfirmed status
of any parent changes.

Fixes #129
 2017-02-18 16:10:20 +09:00
Neil Booth
d29c07f0eb Prepare 0.11.0 2017-02-18 13:29:52 +09:00
Neil Booth
790755b630 Reduce upper limit on blocks fetched at once 
Testnet is lumpy and 2500 can blow up memory
 2017-02-18 13:17:51 +09:00
Neil Booth
3f35bc0298 More PEP8 stuff 2017-02-18 13:05:26 +09:00
Neil Booth
33cdfa4fc8 Merge branch 'nmarley-pep8' into develop 2017-02-18 12:54:01 +09:00
Neil Booth
9bd7b5a9f9 Merge branch 'pep8' of https://github.com/nmarley/electrumx into nmarley-pep8 2017-02-18 12:51:58 +09:00
Neil Booth
151da40d5b Implement peer discovery protocol 
Closes #104

DEFAULT_PORTS now a coin property
A Peer object maintains peer information
Revamp LocalRPC "peers" call to show a lot more information
Have lib/jsonrpc.py take care of handling request timeouts
Save and restore peers to a file
Loosen JSON RPC rules so we work with electrum-server and beancurd which don't follow the spec.
Handle incoming server.add_peer requests
Send server.add_peer registrations if peer doesn't have us or correct ports
Verify peers at regular intervals, forget stale peers, verify new peers or those with updated ports
If connecting via one port fails, try the other
Add socks.py for SOCKS4 and SOCKS5 proxying, so Tor servers can now be reached by TCP and SSL
Put full licence boilerplate in lib/ files
Disable IRC advertising on testnet
Serve a Tor banner file if it seems like a connection came from your tor proxy (see ENVIONMENT.rst)
Retry tor proxy hourly, and peers that are about to turn stale
Report more onion peers to a connection that seems to be combing from your tor proxy
Only report good peers to server.peers.subscribe; always report self if valid
Handle peers on the wrong network robustly
Default to 127.0.0.1 rather than localhost for Python <= 3.5.2 compatibility
Put peer name in logs of connections to it
Update docs
 2017-02-18 12:43:45 +09:00
Neil Booth
09b2df1179 Prepare 0.10.19 2017-02-16 08:01:45 +09:00
Nathan Marley
a03665696c recommendations from pycodestyle (pep8 style) 2017-02-07 20:12:50 -08:00
Neil Booth
09c4bdc5c3 Prepare 0.10.18 2017-02-06 19:48:54 +09:00
Neil Booth
6db9cb967c Require attributes to be specified for a coin 2017-02-06 19:48:30 +09:00
Neil Booth
13cf80a326 Re-enable bandwidth throttling 2017-02-06 19:29:20 +09:00
Neil Booth
ba45e001fa Prepare 0.10.17 2017-02-05 09:49:46 +09:00
Neil Booth
1856cbe6c0 Coin fixes/cleanup (erasmospunk) 
2 out of 3 from #118
 2017-02-05 09:41:27 +09:00
Neil Booth
a019fde853 Show uptime in RPC getinfo result 2017-02-04 10:35:36 +09:00
Neil Booth
184fc615bd Improve clarity about current daemon URL 2017-02-04 10:35:35 +09:00
Neil Booth
2d126f7436 Prepare 0.10.16 2017-01-31 20:09:57 +09:00
Neil Booth
60711956d5 Prepare 0.10.15 2017-01-31 08:17:10 +09:00
Neil Booth
616bcf6bfd Add daemon_url RPC call and document it. 
Closes #111
 2017-01-31 08:13:25 +09:00
Neil Booth
b23b19473c Sanity check on basic IRC reported settings 
Closes #110
 2017-01-31 07:58:19 +09:00
Neil Booth
eff5d3d7e4 Force JSONRPCv2 
Temporary backwards compat hack for old Coinomi wallets
 2017-01-31 07:48:18 +09:00
John L. Jegutanis
fd19bddeb1 fix crash due to a missing env variable in irc.py and a typo fix (#112) 
Thank you!
 2017-01-30 23:52:56 +09:00
Neil Booth
4a8cec9988 Prepare 0.10.14 2017-01-29 08:29:41 +09:00
Neil Booth
06c8eda161 Preparations for peer discovery 
DEFAULT_PORTS is a now a coin property.
Other miscellaneous updates to get things closer
 2017-01-29 06:47:43 +09:00
Neil Booth
8bc9abf9c1 Prepare 0.10.13 2017-01-25 20:30:02 +09:00
Neil Booth
e56f188816 Implement peer subscriptions 
Incremental updates are passed.
Also implement a server.features RPC
 2017-01-25 20:28:19 +09:00
Neil Booth
551c04a3bc Remove all uses of "except:" 2017-01-25 20:28:19 +09:00
Neil Booth
76b6899cf2 Try to avoid asyncio log spew on shutdown 
Closes #106.
This is a hacky workaround to an issue that needs to be
fixed in Python's asyncio library (where I filed issue 487
on github)
 2017-01-25 20:27:33 +09:00
Neil Booth
cb0160901f Unify executor and futures logic 2017-01-24 21:25:37 +09:00
Neil Booth
9b5cb105d5 Prepare 0.10.12 2017-01-24 07:57:29 +09:00
Neil Booth
0ac36aeca0 Make a copy of sessions to loop over 2017-01-24 07:55:52 +09:00
Neil Booth
1a48ad136c Prepare 0.10.11 2017-01-23 23:39:49 +09:00
Neil Booth
a7462a6ea3 Complete rewrite of JSON RPC infrastructure 
Aim to be easier for clients to use, because to do peer
discovery we must act as a client to other servers.

Split out JSON session concept from the asyncio protocol
concept.  This makes the JSON RPC support more easily
testable and usable as a stand-alone library.

In addition, support JSON RPC v1 and v2, and auto-detection
of peer's version.
 2017-01-23 23:36:02 +09:00
Neil Booth
d0d4c0d75f Prepare 0.10.10 2017-01-23 07:28:13 +09:00
Neil Booth
05a6da1920 Move peer management to peers.py from irc 
It's cleaner and will be useful for peer-to-peer comms later
 2017-01-22 23:34:44 +09:00
Neil Booth
ed0646efbf Tidy up request and notification sending 2017-01-20 20:04:56 +09:00
Neil Booth
766da5ed79 Don't maintain a sub_count in controller 
Just do a tally occasionally.
It's too tricky to keep it accurate as it double-counts resubscribing to
the same address, for example
 2017-01-20 08:09:33 +09:00
Neil Booth
112e0a12a9 Prepare 0.10.9 2017-01-18 20:35:24 +09:00
Neil Booth
4729ba2e21 Tweak the shutdown process 
Clean it up a bit and make it harder to do wrongly.
 2017-01-18 20:28:57 +09:00
Neil Booth
9a8c598fa8 The session needs to handle the version command. 2017-01-18 19:50:26 +09:00
Neil Booth
b03a44ca5a Prepare 0.10.8 2017-01-18 08:04:14 +09:00
Neil Booth
23c4616711 Fix testcase 2017-01-18 08:02:45 +09:00
Neil Booth
0dfdfaba7e Prepare 0.10.7 2017-01-17 08:13:57 +09:00
Neil Booth
f05dd6c2a3 Use snappy compression 
Seems to make ~4.5% faster (5281 tx/s -> 5516 tx/s)
Shrinks UTXO DB by ~15% and hist DB by ~5%
 2017-01-17 07:59:50 +09:00
Neil Booth
5f9e337457 Improve RocksDB iterator 2017-01-17 07:43:06 +09:00
Neil Booth
5abe4faa8d JSON results are returned with HTTP status 500 2017-01-17 07:23:58 +09:00
Neil Booth
c958b3af49 Remove LMDB 
Performance of history writes was very poor even with
the code rewritten to accommodate the environment concept
(see db_abstraction branch)
 2017-01-16 22:03:45 +09:00
Neil Booth
ffb4d4f158 Don't print response; might have auth info 2017-01-16 21:27:30 +09:00
Neil Booth
1a9ac22fa0 Prepare 0.10.6 2017-01-16 21:24:59 +09:00
Neil Booth
65f927122b daemon: properly check for HTTP status codes 2017-01-16 21:22:41 +09:00
Neil Booth
131601a7b2 Rework clean shutdown logic 
So that the main block processor future is cancellable.
We wait for the executor and then flush anything unflushed.
Resolves the rest of the second part of #100
 2017-01-16 21:22:24 +09:00
Neil Booth
73945c51d1 Prepare 0.10.5 2017-01-16 19:34:17 +09:00
Neil Booth
b01933913f Add traceback for daemon errors 2017-01-16 19:32:22 +09:00
Neil Booth
8d58d2a0e7 Clean up shutdown logic and logging 
Add RPC "stop" command, and document it.

Fixes the 2nd part of #100
 2017-01-16 19:31:57 +09:00
Neil Booth
0845f26ec2 Prepare 0.10.4 2017-01-12 22:45:28 +09:00
Neil Booth
9e6de7a554 Merge branch 'json-named-args' into develop 2017-01-12 22:41:18 +09:00
Neil Booth
d2da494120 Clarify README; no need to resync. 
Update HOWTO and docs
 2017-01-12 22:40:21 +09:00
Neil Booth
4eed43accb Add named-argument handling as per JSON RPC 2.0 
This involved a rewrite of the JSON RPC layer.
I took the opportunity to clean up the handling of requests
in general.  It should now be easy to return nice help from
the docstrings.

Closes issue #99
 2017-01-12 07:59:21 +09:00
Neil Booth
5784412393 Write out undo info with the UTXO flushes 
Cleaner and slightly more efficient.
Closes #101
 2017-01-09 18:58:39 +09:00
Neil Booth
a8a82224e7 Prepare 0.10.3 2017-01-09 17:19:15 +09:00
Neil Booth
2ad64f6243 Add an RPC call to force a reorg at run-time 
This required a large rework block processor and prefetcher
interaction, and led to various cleanups

Closes #103
 2017-01-09 17:14:06 +09:00
Neil Booth
15051124af Make flushes and reorgs async 
Apart from the flush on shutdown and the flush when caught up,
neither of which matter, this makes flushes asynchronous.

Also, block processing for reorgs is now asynchronous.

This also removes the FORCE_REORG debug envvar; I want to
put that into the RPC interface.

Closes #102
 2017-01-09 16:15:17 +09:00
Neil Booth
89209632ae Clean up daemon.py 2017-01-08 18:48:32 +09:00
Neil Booth
4bd474bf57 Prepare 0.10.2 2017-01-08 10:07:58 +09:00
Neil Booth
dcc2c92bd6 Rename NETWORK envvar to NET 
For consistency with lib/coins.py

SegWit testnet needs its own NET
 2017-01-08 10:07:58 +09:00
Neil Booth
4d62019b07 Check Genesis hash 
Testnet RPC_PORT is 18332
 2017-01-08 10:07:57 +09:00
Neil Booth
852753cb94 Implement deserialization of SegWit transactions 
tx_hash needs to be that of the prior serialization, so
need to change internal read_block API.

Bitcoin core 0.13.1 broke backwards compat of the RPC interface.
Closes #92
 2017-01-08 10:07:57 +09:00
Neil Booth
7392306e68 Prepare 0.10.1 2017-01-07 15:25:22 +09:00
Neil Booth
18af57059f Improve group handling. 
I believe this fixes #94
 2017-01-07 15:21:45 +09:00
Neil Booth
fe70a6db30 Prepare 0.10.0 2017-01-07 11:54:52 +09:00
Neil Booth
91ca5fc14a Split out history into its own DB. 2017-01-07 11:51:12 +09:00
Neil Booth
7f9cb9cd9d Generic header length 
If header lengths change in the future, only one function needs
to change in coins.py.
 2017-01-07 11:51:12 +09:00
Neil Booth
c8c4199503 New VirtualFile abstraction 
Use for headers, tx counts and tx hashes.
 2017-01-07 11:51:11 +09:00
Neil Booth
427183ae01 Place metadata files in meta/, db in db/ 2017-01-07 11:51:11 +09:00
Neil Booth
2b2909f1c4 Move to hashX 
The hashX of a script is the first 11 bytes of its sha256 hash.

Closes #72.
 2017-01-07 11:51:01 +09:00
Neil Booth
de201a247f Double fetch size of mempool. 
Set testnet genesis hash.
 2017-01-07 11:51:01 +09:00
Neil Booth
0f43ce564a Fix log message 2017-01-07 11:51:00 +09:00
Neil Booth
4b0ba2fcd3 Prepare 0.9.22 
Documentation updates only
 2017-01-03 17:08:55 +09:00
Neil Booth
7d899e07ae Fix version 2017-01-02 20:12:08 +09:00
Neil Booth
533c97a575 Clean up DB open handling 
Closes #89
 2017-01-02 18:11:17 +09:00
Neil Booth
ddfefd7f36 Prepare 0.9.20 2017-01-02 17:38:01 +09:00
Neil Booth
517f78858c Fix IRC flood issues 
Tweaked version of suggestions from Valesi, petrkr, and bauerj.
Fixes #93
 2017-01-02 17:30:32 +09:00
Neil Booth
5cbe878cb8 Move ServerManager to controller.py 
rename Controller
 2017-01-02 17:30:32 +09:00
Neil Booth
67908b3541 Slightly cleaner semaphore fix 2017-01-02 17:30:32 +09:00
shsmith
62eded55bb move sleep outside semaphore (issue 88) 2016-12-26 09:15:22 -08:00
Neil Booth
b1e8012b56 Prepare 0.9.18 2016-12-22 06:37:04 +09:00
Neil Booth
a81b8917d8 Logging tweaks 2016-12-21 21:15:48 +09:00
Neil Booth
d45e174e03 Prepare 0.9.17 2016-12-21 20:33:09 +09:00
Neil Booth
4c4cd92c54 Banner: add $DONATION_ADDRESS meta variable 
Replaced with the DONATION_ADDRESS environment variable
 2016-12-21 20:31:42 +09:00
Neil Booth
7a3b0830e1 Make all storage use fsync 
Hopefully costs little and increases robustness.
Closes #75
 2016-12-21 20:30:56 +09:00
Neil Booth
99d8cbfbf6 Permit RPC requests up to 1MB 
Fixes #85

Prepare 0.9.16
 2016-12-21 19:47:27 +09:00