Commit Graph

159 Commits

Author SHA1 Message Date
Neil Booth
22c75a6216 Remove deprecated "with await" syntax 2018-02-11 22:10:07 +08:00
Neil Booth
cb33dd115f Clean up client notifications
- mempool informed of new block; it notifies controller synchronously
- controller notifies sessions synchronously
- sessions are notified of new height synchronously.  Any address touch
notifications are returned to the controller and scheduled
asynchronously.

Also, remove a redundant notification of height on initial header
subscriptions - the subscription response gives the current height;
we also used to send a notification as we didn't update our idea
of notified height.
2017-11-27 18:22:32 +09:00
Neil Booth
3eef0ba4cf Rename block_full to block.
Retain the raw block in the named tuple.
2017-07-30 20:41:08 +09:00
Neil Booth
2ade5fce9c Revert "More logical mempool hash handling"
This reverts commit 87d24f38dc.
2017-07-23 14:55:03 +09:00
Neil Booth
87d24f38dc More logical mempool hash handling
Fixes the issue whereby notifications weren't sent as long as new blocks
kept coming in.  Now a new height notification, with an appropriate mempool
update, is sent after each batch of blocks is processed.
2017-07-19 23:50:44 +09:00
Neil Booth
858bac217d Move DB UTXO code into one place. 2017-07-10 14:12:27 +09:00
Neil Booth
2f26e81629 Implement history compression with tests.
Still to do: running compression in background when the flush
count reaches a certain level
2017-06-18 22:59:30 +09:00
Neil Booth
37602d4eae A couple of tweaks to the prior commit. 2017-03-25 11:35:55 +09:00
John L. Jegutanis
ed7d8a319d Refactor block parsing API 2017-03-25 11:35:55 +09:00
Neil Booth
294212d421 Fix discovery of base of reorgs 2017-03-21 22:17:01 +09:00
Neil Booth
790755b630 Reduce upper limit on blocks fetched at once
Testnet is lumpy and 2500 can blow up memory
2017-02-18 13:17:51 +09:00
Neil Booth
3f35bc0298 More PEP8 stuff 2017-02-18 13:05:26 +09:00
Nathan Marley
a03665696c recommendations from pycodestyle (pep8 style) 2017-02-07 20:12:50 -08:00
Neil Booth
6db9cb967c Require attributes to be specified for a coin 2017-02-06 19:48:30 +09:00
Neil Booth
1856cbe6c0 Coin fixes/cleanup (erasmospunk)
2 out of 3 from #118
2017-02-05 09:41:27 +09:00
Neil Booth
cb0160901f Unify executor and futures logic 2017-01-24 21:25:37 +09:00
Neil Booth
4729ba2e21 Tweak the shutdown process
Clean it up a bit and make it harder to do wrongly.
2017-01-18 20:28:57 +09:00
Neil Booth
131601a7b2 Rework clean shutdown logic
So that the main block processor future is cancellable.
We wait for the executor and then flush anything unflushed.
Resolves the rest of the second part of #100
2017-01-16 21:22:24 +09:00
Neil Booth
b01933913f Add traceback for daemon errors 2017-01-16 19:32:22 +09:00
Neil Booth
8d58d2a0e7 Clean up shutdown logic and logging
Add RPC "stop" command, and document it.

Fixes the 2nd part of #100
2017-01-16 19:31:57 +09:00
Neil Booth
5784412393 Write out undo info with the UTXO flushes
Cleaner and slightly more efficient.
Closes #101
2017-01-09 18:58:39 +09:00
Neil Booth
2ad64f6243 Add an RPC call to force a reorg at run-time
This required a large rework block processor and prefetcher
interaction, and led to various cleanups

Closes #103
2017-01-09 17:14:06 +09:00
Neil Booth
15051124af Make flushes and reorgs async
Apart from the flush on shutdown and the flush when caught up,
neither of which matter, this makes flushes asynchronous.

Also, block processing for reorgs is now asynchronous.

This also removes the FORCE_REORG debug envvar; I want to
put that into the RPC interface.

Closes #102
2017-01-09 16:15:17 +09:00
Neil Booth
4d62019b07 Check Genesis hash
Testnet RPC_PORT is 18332
2017-01-08 10:07:57 +09:00
Neil Booth
852753cb94 Implement deserialization of SegWit transactions
tx_hash needs to be that of the prior serialization, so
need to change internal read_block API.

Bitcoin core 0.13.1 broke backwards compat of the RPC interface.
Closes #92
2017-01-08 10:07:57 +09:00
Neil Booth
91ca5fc14a Split out history into its own DB. 2017-01-07 11:51:12 +09:00
Neil Booth
7f9cb9cd9d Generic header length
If header lengths change in the future, only one function needs
to change in coins.py.
2017-01-07 11:51:12 +09:00
Neil Booth
2b2909f1c4 Move to hashX
The hashX of a script is the first 11 bytes of its sha256 hash.

Closes #72.
2017-01-07 11:51:01 +09:00
Neil Booth
0f43ce564a Fix log message 2017-01-07 11:51:00 +09:00
Neil Booth
533c97a575 Clean up DB open handling
Closes #89
2017-01-02 18:11:17 +09:00
Neil Booth
67908b3541 Slightly cleaner semaphore fix 2017-01-02 17:30:32 +09:00
shsmith
62eded55bb move sleep outside semaphore (issue 88) 2016-12-26 09:15:22 -08:00
Neil Booth
efe3ebd9ee Remove None from touched when backing up.
Fixes #84
2016-12-20 06:50:55 +09:00
Neil Booth
e91f49101b Don't start processing mempool until caught up
Print server manager settings once servers start
2016-12-19 23:25:37 +09:00
Neil Booth
0665e1ac5c Clear the touched addresses if not caught up 2016-12-18 00:02:16 +09:00
Neil Booth
39af7a7463 Rework main block processor loop
It's less awkward and more explicit.
This brings back the efficiency lost in the 0.9.x series.
It also removes the special case hack: both when syncing and
caught up, block processing is done in the executor.

Fixes #58
2016-12-17 21:45:44 +09:00
Neil Booth
3f64a8dd6e Log block processing time only if not syncing
Prepare 0.9.10
2016-12-17 15:31:30 +09:00
Neil Booth
1ebebf08d9 Improve logging 2016-12-17 15:26:17 +09:00
Neil Booth
5c80b96d0f Prioritize mempool processing of sent txs
Closes #73
2016-12-15 17:22:19 +09:00
Neil Booth
ac48695db8 daemon: getting height optionally gets mempool
Improve daemon startup log message
2016-12-15 15:38:22 +09:00
Neil Booth
5fe49bb261 Synchronize daemon height and mempool fetching
Cleanup and simplify touched handling and its event, which is
now controlled and owned by the mempool.
The daemon object owns the set of current mempool hashes.
Clean up and simplify the mempool main loop.

Fixes #70.
2016-12-15 12:08:14 +09:00
Neil Booth
e2f4847632 New env var MAX_SESSIONS
When the number of sessions reaches MAX_SESSIONS, which defaults
to 1,000, turn off TCP and SSL listening sockets to prevent new
connections.  When the session count falls below a low watermark,
currently 90% of MAX_SESSIONS, the listening sockets will be re-opened.
Helps prevent DoS and limit open file usage.

Bug fix: do not start serving paused connections until the buffer
socket is sufficiently drained.  Also, loop.
2016-12-13 22:47:03 +09:00
Neil Booth
1ef6a4d785 Split mempool out into new file.
Rework mempool handling
2016-12-12 00:05:42 +09:00
Neil Booth
7399452294 Temporary hack to fix initial sync slowdown 2016-12-08 22:47:29 +09:00
Neil Booth
238b9e0285 Process blocks in the executor
This should eliminate latency when new blocks come in
and prevent sessions being dropped whilst they are processed.
2016-12-03 23:32:20 +09:00
Neil Booth
6b551973e8 queue -> tasks 2016-11-29 23:21:22 +09:00
Neil Booth
f9cc21807f Further optimize the inner loop 2016-11-27 14:20:36 +09:00
Neil Booth
5c0b026158 Tasks queue just has null entries for now 2016-11-27 13:35:09 +09:00
Neil Booth
6aef79461f Don't shut down block processor by cancellation
The block processor needs to be able to close cleanly, and not
mid-block.  In order to be able to yield whilst processing blocks
we cannot forcefully close its coroutine with a cancellation.
2016-11-27 12:20:16 +09:00
Neil Booth
1b95bcd8ac Open DB differently depending on if syncing
If syncing, use a high max_open_files, otherwise lower it.
2016-11-27 07:25:28 +09:00