Commit Graph

84 Commits

Author SHA1 Message Date
Neil Booth
bc6093a8fe Fully integrate aiorpcX 2018-04-08 10:16:36 +09:00
Roman Zeyde
19565a136d Fix small typo in server/db.py (#412) 2018-03-12 17:15:55 +08:00
Neil Booth
b5e97b6d37 Log supported protocol versions 2018-03-09 08:01:31 +08:00
Neil Booth
49ee008346 Relax the constraints on read_headers
They were really for fs_block_hashes; that still enforces
the full constraint.  Simplifies get_chunk.
2018-03-03 18:39:24 +08:00
Neil Booth
22c75a6216 Remove deprecated "with await" syntax 2018-02-11 22:10:07 +08:00
Neil Booth
56d03625f6 Keep P2PK scripts separate from P2PKH scripts
This makes the DB incompatible with prior versions,
so the DB version is bumped.
2017-10-12 14:02:22 +09:00
Kefkius
59d8421c87 Fix KeyError on incorrect genesis hash (#224)
Fix KeyError that occurs when creating an exception for an incorrect genesis hash.
2017-08-20 05:31:55 +07:00
Justin Turner Arthur
28764936c4 Remove semicolons and a couple unused variables. (#212) 2017-08-02 15:55:44 +09:00
Neil Booth
858bac217d Move DB UTXO code into one place. 2017-07-10 14:12:27 +09:00
Neil Booth
2f26e81629 Implement history compression with tests.
Still to do: running compression in background when the flush
count reaches a certain level
2017-06-18 22:59:30 +09:00
Neil Booth
2940ea3199 Add new function get_history_txnums 2017-06-17 16:33:15 +09:00
John L. Jegutanis
a820829e0e Dynamic header support
Block headers can have a dynamic size that is being indexed on a
new meta file "headers_offsets".

The offsets are 64 bits in order to accommodate coins with big
headers that will accumulate GBs of header data after some years.

Closes #128
2017-03-25 11:35:55 +09:00
John L. Jegutanis
ed7d8a319d Refactor block parsing API 2017-03-25 11:35:55 +09:00
Neil Booth
151da40d5b Implement peer discovery protocol
Closes #104

DEFAULT_PORTS now a coin property
A Peer object maintains peer information
Revamp LocalRPC "peers" call to show a lot more information
Have lib/jsonrpc.py take care of handling request timeouts
Save and restore peers to a file
Loosen JSON RPC rules so we work with electrum-server and beancurd which don't follow the spec.
Handle incoming server.add_peer requests
Send server.add_peer registrations if peer doesn't have us or correct ports
Verify peers at regular intervals, forget stale peers, verify new peers or those with updated ports
If connecting via one port fails, try the other
Add socks.py for SOCKS4 and SOCKS5 proxying, so Tor servers can now be reached by TCP and SSL
Put full licence boilerplate in lib/ files
Disable IRC advertising on testnet
Serve a Tor banner file if it seems like a connection came from your tor proxy (see ENVIONMENT.rst)
Retry tor proxy hourly, and peers that are about to turn stale
Report more onion peers to a connection that seems to be combing from your tor proxy
Only report good peers to server.peers.subscribe; always report self if valid
Handle peers on the wrong network robustly
Default to 127.0.0.1 rather than localhost for Python <= 3.5.2 compatibility
Put peer name in logs of connections to it
Update docs
2017-02-18 12:43:45 +09:00
John L. Jegutanis
fd19bddeb1 fix crash due to a missing env variable in irc.py and a typo fix (#112)
Thank you!
2017-01-30 23:52:56 +09:00
Neil Booth
06c8eda161 Preparations for peer discovery
DEFAULT_PORTS is a now a coin property.
Other miscellaneous updates to get things closer
2017-01-29 06:47:43 +09:00
Neil Booth
131601a7b2 Rework clean shutdown logic
So that the main block processor future is cancellable.
We wait for the executor and then flush anything unflushed.
Resolves the rest of the second part of #100
2017-01-16 21:22:24 +09:00
Neil Booth
5784412393 Write out undo info with the UTXO flushes
Cleaner and slightly more efficient.
Closes #101
2017-01-09 18:58:39 +09:00
Neil Booth
4d62019b07 Check Genesis hash
Testnet RPC_PORT is 18332
2017-01-08 10:07:57 +09:00
Neil Booth
852753cb94 Implement deserialization of SegWit transactions
tx_hash needs to be that of the prior serialization, so
need to change internal read_block API.

Bitcoin core 0.13.1 broke backwards compat of the RPC interface.
Closes #92
2017-01-08 10:07:57 +09:00
Neil Booth
91ca5fc14a Split out history into its own DB. 2017-01-07 11:51:12 +09:00
Neil Booth
7f9cb9cd9d Generic header length
If header lengths change in the future, only one function needs
to change in coins.py.
2017-01-07 11:51:12 +09:00
Neil Booth
c8c4199503 New VirtualFile abstraction
Use for headers, tx counts and tx hashes.
2017-01-07 11:51:11 +09:00
Neil Booth
427183ae01 Place metadata files in meta/, db in db/ 2017-01-07 11:51:11 +09:00
Neil Booth
2b2909f1c4 Move to hashX
The hashX of a script is the first 11 bytes of its sha256 hash.

Closes #72.
2017-01-07 11:51:01 +09:00
Neil Booth
533c97a575 Clean up DB open handling
Closes #89
2017-01-02 18:11:17 +09:00
Neil Booth
b8fc739590 Fix ordering in get_utxos 2016-11-27 21:54:05 +09:00
Neil Booth
6aef79461f Don't shut down block processor by cancellation
The block processor needs to be able to close cleanly, and not
mid-block.  In order to be able to yield whilst processing blocks
we cannot forcefully close its coroutine with a cancellation.
2016-11-27 12:20:16 +09:00
Neil Booth
1b95bcd8ac Open DB differently depending on if syncing
If syncing, use a high max_open_files, otherwise lower it.
2016-11-27 07:25:28 +09:00
Neil Booth
95a2de6dae Move DB undo code to db.py 2016-11-26 13:18:10 +09:00
Neil Booth
8d253c81d3 Move clean_db() to db.py 2016-11-26 13:15:45 +09:00
Neil Booth
59244cc24e Move bulk of FS flush code to db.py
More logically belongs there.
Have servers use the flushed DB height not the block processor's height.
Get all headers only from disk.
2016-11-26 13:11:59 +09:00
Neil Booth
59733e4609 Move bitcoin-specific coin defaults to Bitcoin 2016-11-26 08:32:30 +09:00
TheLazieR Yip
e987510432 Replace double_sha256 with header_hash from coin 2016-11-25 07:25:57 -05:00
Neil Booth
aa9f106865 Add missing self; use max_secs where intended. 2016-11-20 23:11:35 +09:00
Neil Booth
8617c82ec2 Floor disk_count at zero. 2016-11-19 20:56:08 +09:00
Neil Booth
3d87e299ea Move formatted_time to library 2016-11-19 15:42:06 +09:00
Neil Booth
ffd6cd63b0 Fix thinko 2016-11-19 08:16:20 +09:00
Neil Booth
d6fa4a1564 More logging improvements
- log software and DB version at startup
- log more DB info at startup
- log software and DB version when first synced

Fixes #29
2016-11-19 07:29:17 +09:00
Neil Booth
f4651c0ca4 Store tx_num in 'h' table key so keys are unique 2016-11-17 07:59:40 +09:00
Neil Booth
c5c7b6f438 Have all expensive requests yield. 2016-11-17 06:50:46 +09:00
Neil Booth
85786b87a2 Save all UTXOs
Change the DB version
2016-11-13 20:35:03 +09:00
Neil Booth
1b9a9b8927 Start saving DB version and checking it 2016-11-12 07:31:56 +09:00
Neil Booth
f98dc9c351 Practise optimism like before 2016-11-11 21:03:09 +09:00
Neil Booth
311f7f9ffd Better optimised storage of UTXO set 2016-11-10 20:04:33 +09:00
Neil Booth
094fd44fc2 Merge UTXO cache into BlockProcessor 2016-11-10 07:49:01 +09:00
Neil Booth
06b6d695c2 Only attempt to flush if cancelled 2016-11-10 07:32:39 +09:00
Neil Booth
3972e18e98 Move the rest of fs_cache into BlockProcessor 2016-11-08 23:30:30 +09:00
Neil Booth
ad2dabf751 Remove the FS cache
Really belongs with BlockProcessor now
2016-11-08 20:06:06 +09:00
Neil Booth
5736e9cb70 Move utxo_cache to BlockProcessor where it belongs 2016-11-08 19:35:21 +09:00