The synchronizer's work is done from the network proxy's main loop.
A minor problem with the old synchronizer was that it considered itself
out of date if the network was out of date. This was too generic: the
network can have pending requests unrelated to the synchronizer. This
resulted in the synchronizer often unnecessarily flipping the wallet
between up-to-date and not-up-to-date, and causing unnecessary calls
to wallet.save_transactions(). This was observable when opening the
network dialog box: frequently just opening it would cause a wallet
status change and transaction flush, simply because the network dialog
sends a get_parameters() request. This rework of the synchronizer does
not have that issue.
This is a layering violation - the SocketPipe doesn't own
the socket and provides no other way to close the socket, leading
to unnecessary complexity like that in interface.py.
I looked at deamon.py and NetworkProxy - the two other users,
and they don't close the sockets explicitly, just let them be
garbage collected.
Set timeout and socket options on all simple sockets. At present
some code paths can miss it, such as when the SSL certificate is
CA-signed.
Add a missing check for failure.
Remove interface communication out of blockchain.py
into network.py. network.py handles protocol requests
for headers and chunks. blockchain.py continues to
handle their analysis and verification.
If an interface provides a header chain that doesn't
connect, it is dismissed, as per a previous TODO comment.
This removes a thread and another source of timeouts.
I see no performance issues with this when truncating the
blockchain.
Rename 'result' to 'header' for clarity.
This allows us to distinguish between connecting and connected
state in interface.py (used to be done in network.py but that
had other issues).
This means we don't switch to a connecting server, and get_interfaces()
does not report connecting ones.
Two callers of get_spendable_coins were removing frozen addrs
before calling. Put that functionality in the function.
We shouldn't be able to send_from a frozen address. This was
possible in the current release because logic assumed a two-element
tuple was returned when it is now three-element. Fix that too.
Command line options listunspent and createrawtransaction currently
ignore frozen addresses. I'm not sure that's right but I've preserved
that behaviour.
With this patch only the wallet class refers to self.frozen_addresses;
other clients use is_frozen() now.
Speedup mainly from writing to storage only once.
Make frozen_addresses a set in memory, as sets give cleaner
code and are more efficient.
Minor change in behaviour: command line freezing used to return
False if the address isn't in the wallet OR the address was already
frozen. Now it returns more like a success code: it returns False
only if the address isn't in the wallet regardless of frozen state.
Similarly for unfreezing.
interfaces we have created.
Existing code has the concept of pending servers, where a connection
thread is started but has not sent a connection notification, and
and interfaces which have received the notification.
This separation caused a couple of minor bugs, and given the cleaner
semantics of unifying the two I don't think the separation is beneficial.
The bugs:
1) When stopping the network, we only stopped the connected interface
threads, not the pending ones. This would leave Python hanging
on exit if we don't make them daemon threads.
2) start_interface() did not check pending servers before starting
a new thread. Some of its callers did, but not all, so it was
possible to initiate two threads to one server and "lose" one thread.
Apart form fixing the above two issues, unification causes one more
change in semantics: we are now willing to switch to a connection
that is pending (we don't switch to failed interfaces). I don't
think that is a problem: if it times out we'll just switch
again when we receive the disconnect notification, and previously the
fact that an interface was in the interaces dictionary wasn't a
guarantee the connection was good anyway: we might not have processed
a pending disconnection notification.
As discussed on #electrum yesterday.
Increase change gap limit to 6.
Choose the next unused change address, if any, otherwise pick
one at random from the gap limit.
Fees should be recalculated when send_from changes.
Fees should be recalculated when editing fee preference, but
only save to storage when leaving the fee per kb widget.
No need to emit a textEdited signal; the widget does that already
(with the effect that we used to call update_fee() twice).
Three places used to test for lagging and switch. Commonize the
code. We test for lagging before autoconnect so lagging
diagnostics are output if not auto-connect.
Lagging diagnotics moved to server_is_lagging().
SimpleConfig claims to handle configuration options in priority
command line, user config, system config, which makes sense.
In fact it appears it used priority command line, system config,
user config.
Also, from the priority ordering, it would seem correct that a
value should be unmodifiable if and only if it's set on the command
line. Previously anything in the system config would be unmodifiable.
This patch fixes these and cleans the code up a bit. I noticed this
whilst attempting to unify the 'auto_cycle' setting.
Fixup tests accordingly.
1) For new proxy or protocol, restart the network and default to
the requested server.
2) Otherwise if we aren't using the requested server, switch to it
3) Otherwise choose a random server if the requested server is
stale and auto_connect is True
As switch_to_interface() now has another user, move the logic
there whereby we close the old interface in order to terminate
subscriptions, in order to have it in one place.
Whilst it's not a good idea to have two electrum instances
open on the same wallet, we should avoid throwing an
exception. Also note how the old code's handling of the
exception (caused by both renaming the file almost at the
same time, rather than a non-POSIX system not supporting
the atomic rename) can lead to the wallet file being lost
enirely because os.remove(self.path) succeeds and the
rename of the temporary no-longer-existing file then fails.
switch_to_random_interface used to do is_connected()
checks and remove interfaces when switching. This
is redundant: interfaces are only added to self.interfaces
once in is_connected() state from process_if_notification().
When they are disconnected, a notification is also received
and process_if_notification() performs the removal. Furthermore,
two other callers of switch_to_interface do not do explicit
is_connected() checks before calling it.
Traceback (most recent call last):
File "src/electrum/lib/network.py", line 411, in process_request
out['result'] = f(*params)
UnboundLocalError: local variable 'f' referenced before assignment
[Network] network error local variable 'f' referenced before assignment
switch_to_interface() becomes the common place where
self.interface is set; therefore self.interface is not None
precisely when self.default_server is connected. Previously some
places required it to be connected and some didn't. Also an
interface change now sends the 'updated' notification
consistently - previously some did and some didn't.
Have network_start() call start_interfaces() - this also means
network restarts now do this.
Fix apparent off-by-one in start_interfaces()
Prefer to use safer self.is_connected() as it checks interface
is not None.
Using common code gives small observable changes in behaviour:
- slightly different print_error() messages
- network restarts now enter status 'connecting' which they
didn't before, which seems correct
- status 'connecting' is done with set_status() rather than
simply assigning the status, which seems more correct. Now
that the response_queue is available in the constructor this
works; it used to fail with 'response_queue is not defined'
This is somewhat cleaner as the proxy's pipe and network setup
was awkwardly interleaved. It also means network's constructor
is free to use both; currently some code is working around the
fact that the response queue doesn't exist in the constructor.
Found these issues while trying to create, sign, and broadcast a raw transaction.
* createrawtransaction was using old signature for Transaction constructor
* Signwithwallet and decoderawtransaction need to call deserialize on tx before they can access inputs and outputs. (Maybe adding getInputs() and getOutputs() which deserializes if needed might be nicer)
A couple of changes
1) Old electrum wallets seemed to save labels in latin1, when you call json.dumps on line 83/84 it fails silently, which causes the label import to fail. Whenever electrum saves, it then overwrites your default wallet with no labels - essentially deleting all your labels. The solution to this is iterating over all the labels for old wallets decoding anything that fails to unicode() as latin1, and then unicoding it :)
2) Failing to import data and then deleting it is bad. So I'm raising an exception to avoid data being lost.
First, close the socket from the thread itself rather than from
the stop() function. This prevents another thread closing the
socket that the interface thread is simultaneously using.
Second, it occasionally would happen that the parent thread such as
network.py start() an interface, do a send_request() and timeout
waiting for a response (timeouts are 0.1s). It would check
is_connected(), get False, and assume the connection has failed.
In fact the thread hadn't even been scheduled or gotten around to
completing the socket connection. Fix by having self.connected
start out True. If the connection fails or times out, we set
connected to False soon enough.
Finally for correctness we need to deepcopy a send_request() rather
than take a reference to it.
The verifier will retain responsibility for verification, but will no longer
hold the transaction sets itself.
Change requires_fee to take a wallet.
Add new function add_unverified_tx()
Move get_confirmations() to the wallet from the verifier.
