RanchiMall/electrumx
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
1,613 Commits 4 Branches 138 Tags 3.9 MiB
07ca48ad58
Commit Graph

289 Commits

Author SHA1 Message Date
Neil Booth
4329724b98 Implement TX serialization 
This will be used in other tests
 2018-08-11 16:15:34 +09:00
Neil Booth
8b34de53f9 Add some utility functions and tests 2018-08-11 16:15:34 +09:00
Neil Booth
8861ebac8f Placate pycodestyle 2018-08-11 16:15:34 +09:00
Neil Booth
635ffed42b flush_data() must be called with the lock held 2018-08-10 11:41:02 +09:00
Neil Booth
967b2de60d Separate the block processor from the DB 
- BP no longer inherits from the DB, but is passed it
 2018-08-10 11:10:13 +09:00
Neil Booth
27b31746f8 Remove remaining flush-related logic to db.py 2018-08-10 11:10:13 +09:00
Neil Booth
a50d17c5b9 Clear data by reference as it's flushed 2018-08-10 11:10:13 +09:00
Neil Booth
dc445e2a54 Move catch-up stats to db.py 2018-08-10 11:10:13 +09:00
Neil Booth
891730e78f Move flush_backup() to db.py 2018-08-10 11:10:13 +09:00
Neil Booth
42c3a308db Move to flush_dbs in db.py 2018-08-10 11:10:13 +09:00
Neil Booth
d3f9ba386c Move flush_state() to db.py 2018-08-10 11:10:13 +09:00
Neil Booth
d1510b1192 Move bulk of UTXO flush logic to db.py 2018-08-10 11:10:13 +09:00
Neil Booth
9515e1a1e4 Improve flow for opening DBs 2018-08-10 11:10:13 +09:00
Neil Booth
aac84ade75 Sleep at shutdown 2018-08-10 11:10:13 +09:00
Neil Booth
c9631f3438 Move history flushing to DB.flush_history() 2018-08-10 11:10:13 +09:00
Neil Booth
11c6c919a6 Move fs_flush to db.py and merge with fs_update 2018-08-10 11:10:13 +09:00
Neil Booth
d87c3dedcf Move assert_flushed DB logic to db.py 2018-08-10 11:10:13 +09:00
Neil Booth
70319bb22d Merge branch 'master' into devel 2018-08-10 11:06:49 +09:00
Neil Booth
dd67b57c30 Catch and log failures in the main peers task 
Closes #565
 2018-08-10 09:52:49 +09:00
Neil Booth
afae1b4a42 Fix rare race condition 
Closes #567
 2018-08-10 09:20:56 +09:00
Neil Booth
83813ff1ac Extend SUPPRESS_MESSAGE_REGEX 
Fixes #521
 2018-08-10 06:25:27 +09:00
Neil Booth
7f28677549 Merge branch 'master' into devel 2018-08-09 18:19:16 +09:00
Neil Booth
9dff85c027 Prepare 1.8.2 2018-08-09 18:16:18 +09:00
Neil Booth
028374ede4 Shield the taking of the lock, otherwise it is lost 2018-08-09 17:28:34 +09:00
Neil Booth
e95f556234 Merge branch 'master' into devel 2018-08-09 15:40:07 +09:00
Neil Booth
e2d3f655fb Set retry_event on correct object 
Fixes #564
 2018-08-09 15:29:10 +09:00
Neil Booth
9a2dfa9ab0 Merge branch 'master' into devel 2018-08-09 06:36:24 +09:00
Neil Booth
87d55c3fc5 Prepare 1.8.1 2018-08-08 23:18:06 +09:00
Neil Booth
9b22fd3a15 Peer verification needs to be serial... 2018-08-08 20:32:51 +09:00
Neil Booth
e0ccf0cce3 Populate the header merkle cache in a thread 
- It can take a while
- Client requests that need it will block until it's done
- It's a function of FS state so move it to the DB
- Tweak docs

Fixes #558
 2018-08-08 17:26:34 +09:00
Neil Booth
0a8045278a Merge branch 'no-shutdown-hacks' into devel 2018-08-08 16:34:21 +09:00
Neil Booth
2803ef913e Merge branch 'master' into devel 2018-08-08 16:33:11 +09:00
Neil Booth
28ea9ae2f4 Remove hackery 2018-08-08 14:58:52 +09:00
Neil Booth
12c49bbe75 Suppress accept_connection2 asyncio log spew 2018-08-08 12:48:22 +09:00
Neil Booth
931b227618 Verify a peer concurrently 2018-08-08 12:40:48 +09:00
Neil Booth
9b4276c68d Have peers.py use its own task group 2018-08-08 12:35:58 +09:00
Neil Booth
9c5d59e997 Use new Session interface 2018-08-08 12:19:26 +09:00
Neil Booth
2d66a4f5aa aiorpcX 0.7.0 has better string text for TaskTimeout 2018-08-08 12:19:18 +09:00
Neil Booth
6697614427 Use close from aiorpcX 0.7.0 2018-08-08 12:19:09 +09:00
Neil Booth
962b4679d7 Require aiorpcX 0.7.0 2018-08-08 12:18:57 +09:00
Neil Booth
a3afab83d6 Better handle bad input to query RPC call 
Based on #559.  Also:

- remove unused import
- restore timeout to 15s
- handle invalid input by catching at a higher level
  and converting to RPCError
 2018-08-07 21:21:27 +09:00
Neil Booth
74e6fe416f Move close() up to the base class 
Fixes #557
 2018-08-07 10:46:12 +09:00
Neil Booth
955a8e927d Put flushing-to-DB in a thread 
- flush() and backup_flush() are now async
 2018-08-06 22:28:25 +09:00
Neil Booth
1efc8cb8ec Make the merkle cache and read_headers async 
read_headers runs in a thread to avoid blocking
 2018-08-06 21:49:26 +09:00
Neil Booth
db5d516756 Make raw_header async 2018-08-06 21:49:26 +09:00
Neil Booth
f24b022fa4 Remove dead code 2018-08-06 21:49:26 +09:00
Neil Booth
12dbf2c74a Rename get_history to limited_history 
- make it async and run in a thread
 2018-08-06 21:49:26 +09:00
Neil Booth
a036a2eb3f Rename get_utxos to all_utxos. 
- no longer takes a limit
- runs in a thread to avoid blocking
 2018-08-06 21:38:53 +09:00
Neil Booth
0c8d5ddf63 Mark as dev branch 2018-08-06 21:38:53 +09:00
Neil Booth
78388999a7 Prepare 1.8 2018-08-06 18:55:47 +09:00
Neil Booth
c141bfffd9 Remove some dead debug code 2018-08-06 18:17:46 +09:00
Neil Booth
55da063f6b Tweak exception output to use !r 2018-08-06 18:04:20 +09:00
Neil Booth
1956b9d659 Use a regex for message suppression 2018-08-06 14:55:25 +09:00
Neil Booth
09e840de3c Fix spawn call 2018-08-06 14:47:53 +09:00
Neil Booth
4bb78ecbbb Move history cache to session_mgr 2018-08-06 14:34:44 +09:00
Neil Booth
790d462b55 Use a list for order 2018-08-06 11:59:55 +09:00
Neil Booth
e5b2268387 Fix groestlcoin address parsing 
Baesd on erasmospunk's PR #551
 2018-08-06 08:39:44 +09:00
Neil Booth
2afb95930f Update aiorpcX requirements 2018-08-06 06:43:15 +09:00
John L. Jegutanis
dbfe9096c2 Fix reorg crash due a missing await (#555) 2018-08-05 18:49:50 +09:00
Neil Booth
02a34d05c2 async/await fixes 2018-08-05 12:10:34 +09:00
Neil Booth
51c9988b81 Various fixes 
- Fix pycodestyle
- Change session.close API
- Fix logging
 2018-08-05 11:06:36 +09:00
Neil Booth
4c2834f899 Prepare 1.8 beta 2018-08-05 10:17:35 +09:00
Neil Booth
2c7d56d097 Update to aiorpcX 0.6 2018-08-05 10:17:35 +09:00
John L. Jegutanis
0815ff8e24 Add Decred support (#550) 
* Refactor reorg_hashes function

* Add Decred support
 2018-08-02 22:20:36 +09:00
Neil Booth
898e2eea00 Make pycodestyle happy 2018-08-02 16:17:30 +09:00
Neil Booth
9185198703 Rework electrumx_rpc; add "query" command 2018-08-02 15:37:11 +09:00
Neil Booth
147989a0a6 Get query.py working 2018-08-02 10:56:21 +09:00
Kefkius
9e3893b065 Add Groestlcoin (#544) 
* Add Groestlcoin

* GRS: Add tests
 2018-08-02 07:12:14 +09:00
Neil Booth
4c7ab221a7 Allow per-class tx hash functions 
Idea from erasmospunk; see #548
 2018-08-01 12:08:01 +09:00
Neil Booth
aa1f43b6be Prepare 1.7.3 2018-08-01 10:49:42 +09:00
Neil Booth
6c3cce5705 Remove dead code 2018-07-31 06:48:05 +08:00
Neil Booth
2c07939120 Temporarily remove one task group from peers.py 
Fixes #538
 2018-07-29 23:46:58 +08:00
Neil Booth
7b9cd944be Require aiorpcX 0.5.9 2018-07-29 05:58:44 +08:00
Neil Booth
0b4f6b3d6d Prepare 1.7.1 2018-07-28 17:13:54 +08:00
Neil Booth
d962c97ef1 Have one peer monitoring task per peer 2018-07-28 16:29:02 +08:00
Neil Booth
751f9917a4 Switch to curio primitives 
Gives much clearer code
 2018-07-28 14:35:54 +08:00
Neil Booth
55ef1ab157 Rework server_base to spawn a single task 2018-07-28 12:32:08 +08:00
Neil Booth
5ee5a54f5a Session manager owns peer manager 2018-07-28 09:22:19 +08:00
Neil Booth
010ef08320 Catch SOCKSError 
Fixes #534
 2018-07-27 09:08:05 +08:00
ghost43
5304080220 server list: add to btc testnet (#535) 2018-07-27 06:24:23 +08:00
cipig
3cdf453663 change BTX address format (#533) 2018-07-26 14:43:25 +08:00
Neil Booth
bef04750d7 Handle the case there are no blocks 2018-07-26 06:05:00 +08:00
Neil Booth
9b23fbc747 Prepare 1.7 2018-07-25 23:16:32 +08:00
Neil Booth
7a3938be23 Make mempool histogram refresh frequency coin-specific 2018-07-25 23:16:32 +08:00
Neil Booth
5e4586ce54 Create a task to start_servers() 
That way we start waiting on the shutdown event no matter what
so we can always be shut down
 2018-07-25 22:02:00 +08:00
Neil Booth
365c217031 header_mc might be None 2018-07-25 21:36:23 +08:00
Neil Booth
68dbf9fad2 Tweak task handling so all errors are logged 2018-07-25 21:35:05 +08:00
Neil Booth
3fbd4992ce Rework prefetch logic 
This also fixes a recent reorg bug...
 2018-07-25 20:40:25 +08:00
Neil Booth
d008707330 Fix bug recently introduced in reorgs 
Fixes #532
 2018-07-25 15:25:40 +08:00
Neil Booth
db8ee48c2e await on send_request directly 2018-07-25 10:40:50 +08:00
Neil Booth
8d3f98d3fd Don't check task exceptions ourselves 2018-07-25 10:40:50 +08:00
Neil Booth
fa84b0f54b Use asyncio.gather to propogate exceptions 2018-07-25 10:37:13 +08:00
Neil Booth
918592fbff Remove RequestError; clean up a bit 2018-07-25 09:47:07 +08:00
Neil Booth
e1309836ca Fix typo in prior 2018-07-24 21:38:50 +08:00
Neil Booth
eac214e508 Rework peer discovery 
Make it more naturally async
 2018-07-24 21:22:07 +08:00
Neil Booth
ba607544b9 Avoid callback in peer manager 
Spawn separate tasks for each peer we test
 2018-07-24 14:26:14 +08:00
Neil Booth
12c024f3ac Log proxy detection failure 2018-07-24 14:02:31 +08:00
Neil Booth
79b98b2b54 Distinguish private and public methods of peer mgr 
Give private methods a leading _
Remove dead code
 2018-07-24 13:52:40 +08:00
Neil Booth
92ddb52f63 Update various comments 2018-07-24 11:41:29 +08:00
joesixpack
83732af4fe Support for Minexcoin (#529) 2018-07-24 09:41:03 +08:00
Neil Booth
221f1222a6 Defer peer discovery 2018-07-24 09:20:45 +08:00
Neil Booth
316f642a08 Reduce memory footprint 
- Use tuples in preference to lists
  They are always at least 11% smaller memory
  footprint, rising to 35% for short arrays
 2018-07-24 09:15:35 +08:00
Neil Booth
7485d0aaf0 Retain the prevouts with each MemPoolTx 2018-07-24 08:21:58 +08:00
ghost43
45ccf8a64e fix mempool tx fee calculation typo (#531) 2018-07-23 20:31:52 +08:00
Neil Booth
7c01acdbf2 Remove comment 2018-07-23 17:26:51 +08:00
Neil Booth
b05cc4e78b Clean up fee histogram implementation 
Much more efficient to just calculate it outright than try
and maintain datastructures
 2018-07-23 17:26:42 +08:00
Neil Booth
3e535e47f4 Start peer discovery whilst syncing the mempool 2018-07-23 15:18:49 +08:00
Neil Booth
0963ce5230 Completely overhaul mempool sync logic 
- highly concurrent and a lot more efficient than previously
- initial mempool sync should be much faster (feedback please)
- mempool processing no longer blocks client session handling
- uses less memory to store the mempool
- fixes an obscure bug where sometimes txs were dropped
- more robust, clean and easy to understand

Fixes #433
 2018-07-23 15:18:04 +08:00
Neil Booth
21604cad15 Create a MemPoolTx object and use it 2018-07-22 23:19:35 +08:00
Neil Booth
4b3ceff0d2 Deserialize mempool txs in a thread 2018-07-22 22:18:54 +08:00
Neil Booth
9bd9476a54 Make mempool processing more properly asynchronous 2018-07-22 21:56:22 +08:00
Neil Booth
4da95dd853 Simplify refresh hashes loop 2018-07-22 16:57:30 +08:00
Neil Booth
821f907971 Put stats logging into its own task 2018-07-22 16:54:32 +08:00
Neil Booth
2c51b127de Clarify mempool's exported interface 2018-07-22 13:25:13 +08:00
Neil Booth
a9a69acaf3 Pass the daemon not chain_state to mempool constructor 
The mempool as its own process would maintain its own daemon
 2018-07-22 12:04:48 +08:00
Neil Booth
35506f6054 Make notifications properly async 2018-07-22 10:38:37 +08:00
Neil Booth
e568ee2488 Improve comments 2018-07-21 20:51:42 +08:00
Neil Booth
c715ae6249 Rework mempool and notification code 
Clarifies the mempool interface to look more like what it
would in its own process
 2018-07-21 20:45:27 +08:00
Neil Booth
3e8c413b77 In ChainState prepend private members with an underscore 2018-07-21 20:45:27 +08:00
Neil Booth
6cf7b049fa Require aiohttp >= 2.0 2018-07-21 20:45:27 +08:00
Neil Booth
74c5c3f817 Don't accept onion peers if we have no proxy 2018-07-20 19:03:34 +08:00
Neil Booth
f7c95986bf Fix failing test 2018-07-20 17:14:47 +08:00
Neil Booth
0c28136739 Remove dead code 2018-07-20 17:05:30 +08:00
Neil Booth
8e99d076b6 Logging tweaks 2018-07-20 16:43:15 +08:00
Neil Booth
ca3ba2b2d8 Rework mempool and BP<->prefretcher communication 2018-07-20 16:43:15 +08:00
Neil Booth
431989f0ea Label 1.6a 2018-07-20 15:07:23 +08:00
Neil Booth
530c7cac6f Clean up shutdown process of the block processor 
- enables removal of executor code
- clarify flush guarantees
 2018-07-20 15:06:28 +08:00
Neil Booth
41e734caeb Clean up controller interface with other parts 2018-07-20 12:40:57 +08:00
Neil Booth
bd636a75ad Clean up the process of opening the DBs, make asynchronous 2018-07-20 12:26:59 +08:00
Neil Booth
a019656a5e Make it clear where BlockProcessor is instantiated 2018-07-20 12:26:49 +08:00
Omar Cruz
227696636d Added Motion Coin mainnet (#528) 
* Motion added
 2018-07-19 17:41:20 +08:00
Neil Booth
023e463748 Prepare 1.6 2018-07-19 11:07:02 +08:00
Neil Booth
94d1f7a651 Save raw blocks to disk for reorg purposes 
We used to rely on the daemon being able to return the
orphaned blocks, but some old daemon codebases are buggy
and fail to do so.

Fixes #258 #315 #479
 2018-07-18 16:09:19 +08:00
Neil Booth
7ce91b1a43 Minor tweaks 2018-07-18 14:22:40 +08:00
Neil Booth
f5926cf865 Label as 1.6pre3 2018-07-18 14:07:08 +08:00
Neil Booth
391e69b66c Introduce ChainState object 
- reduces the tangle of object dependencies
- rationalizes responsibilities
 2018-07-18 14:05:39 +08:00
Neil Booth
53425ce585 Move task logic to Tasks object 
This helps to rationalize the inter-object
dependencies.
 2018-07-18 11:30:33 +08:00
Neil Booth
22b01d4dfb Move masternode cache. 
It was mistakenly made a pylru cache...
Don't force non-Dash coins to carry the cache.
 2018-07-18 11:30:33 +08:00
Neil Booth
9cf3b5358f Move raw_header to session manager 
Ultimately this stuff belongs in a ChainState object.
 2018-07-18 11:30:33 +08:00
Neil Booth
b7572ce9c0 More refactoring of controller 
- remove the header cache - not needed for higher protocol versions
- simplify session notification; move to session manager
- move history cache to session manager
 2018-07-18 09:20:14 +08:00
Neil Booth
c8c896a14c Minor tweaks. 2018-07-18 08:46:37 +08:00
ghost43
c7f3f7ac6d new protocol method: blockchain.transaction.id_from_pos (#524) 2018-07-18 08:35:34 +08:00
Neil Booth
60435c0c06 Fix peer discovery for protocol >= 1.4 2018-07-17 22:45:50 +08:00
Neil Booth
c7f6f3ede6 Move get_utxos to session 2018-07-17 22:12:41 +08:00
Neil Booth
c712d4be32 Fix session_count invocation 
(thanks to nijel)
 2018-07-17 20:29:14 +08:00
Neil Booth
a3d3bbe9a7 Move more code out of controller to sessions 
Disable a test that will require significant work
 2018-07-17 20:20:28 +08:00
Neil Booth
0b55a3f7f2 Mark as pre2 2018-07-17 18:23:53 +08:00
Neil Booth
e41d1db25c Create SessionManager class 
Break out controller functionality for session management.
 2018-07-17 18:14:55 +08:00
Neil Booth
c35c062780 Move more code to session class from controller 2018-07-17 16:47:00 +08:00
Neil Booth
063f1b2eff Move block_get_header and estimate_fee to session 2018-07-17 14:45:55 +08:00