Instead of only incrementing it, allow to be able to set it to any
value. This will be useful for instance to set the derivation to the
latest used onchain, if another wallet is much further down the
derivation tree than we are (or after a rescan).
32f3bdeb16b66878e23ba71a6f2c401c5d1f601a bitcoind: check the rescan was successful, retry at most 10 times (Antoine Poinsot)
337f422283e3392b6cea94df6211fd6c736772ef bitcoind: add the timestamp to the 'listdescriptors' entry (Antoine Poinsot)
3c75e2e944638fdfe7ae6986aee9af83e0e2a532 bitcoind: disable retry logic for noreply requests (Antoine Poinsot)
55cf21fa2aa7a66d998061684630a32cfb972a35 qa: unflake the getinfo RPC functional test (Antoine Poinsot)
96b634b69c3b059c0711a35bac580ffcbf34af57 bitcoind: make the rescan starting interface failible (Antoine Poinsot)
85cd261fcd50e87ad8c3b6a5443e7e38266ff655 bitcoin: remove the BitcoinError enum, use String as error instead (Antoine Poinsot)
9b253e7ea71e3da38922e35a9fdc182e85a15bcd bitcoind: rescan with a range, handle spurious error when rescanning (Antoine Poinsot)
ff753fecca5c1c77c44a4627a565e580aefc9446 bitcoind: get more info from 'listdescriptors' (Antoine Poinsot)
4a7fd1af2999622af7e6421c0bea611688150cae commands: check if the Bitcoin backend is already rescanning (Antoine Poinsot)
8a22f5e8c944447572bad60fa7e187058483ed47 qa: functional tests for block chain rescans (Antoine Poinsot)
925df283a62af32d816bcf2e0b2d20c7f08a3a31 bitcoin: check for ongoing rescans in the poller (Antoine Poinsot)
073cdd0a89c8688b2cdad73666b71a79d9f81ae1 commands: add a 'rescan_progress' field to 'getinfo' (Antoine Poinsot)
7866ff46cfcf02f75f0d5fd996434ef647940882 commands: add a new 'startrescan' command (Antoine Poinsot)
7e83bfad55504e758e2380ba76e48ce7f868b837 db: the interface to store the state of an ongoing rescan (Antoine Poinsot)
bd4de0b87a0db999ead08de335566df462251ca6 bitcoin: interface for rescanning the chain on the Bitcoin backend (Antoine Poinsot)
c32f714a2ed12135f07a2dd77951b15c3f0fcfe5 bitcoind: add a 'noreply' client (Antoine Poinsot)
45d601282fd4e722ad8558243c4b8c9c1ad11345 bitcoind: only sleep on error if there is >0 retries (Antoine Poinsot)
51ff7d6734db353e0aab65c45d322c45d7e3aa8b bitcoind: a utility to find a block height by block timestamp (Antoine Poinsot)
6323ae0d0fb2726e95a97ceda52b133027297719 bitcoind: add time and MTP to BlockStats (Antoine Poinsot)
80803c78a4579641b604c110e411556588ebdf24 bitcoind: handle missing previousblockhash from getblockheader (Antoine Poinsot)
Pull request description:
This implements the ability for a user to rescan the block chain from an earlier timestamp to find transactions the wallet isn't currently aware of.
The main difficulty arises with the fact there are two rescans to be performed:
1. The block chain rescan on the Bitcoin backend. For the bitcoind implementation this means we need to rescan the bitcoind watchonly wallet down to the given timestamp (or height). We can imagine for an indexer backend such as an Electrum backend it could be almost instant.
2. The update of our state with the new state of the Bitcoin backend. In the poller we scan blocks linearly, we will never scan a block that we already scanned and that includes blocks anterior to the first block we scanned. In order to implement this rescan of past blocks after a Bitcoin backend rescan was performed we reused our current chain reorganization logic: we rewind our state down to the first block that was rescanned by the backend. This is overkill (we don't need to erase existing information), but sufficient for now.
The implementation is not as robust as it could be. Notably, in order to rescan the chain on the bitcoind implementation of the backend i use a trick to not make the connection block for the entire duration of the rescan when calling `importdescriptors`: i introduce a new client that has a timeout of 1 second and that ignores the response as well as timeout errors. Obviously this is problematic since it makes it more complicated to know whether the call actually succeeded.
~~A more robust solution would be to use `rescanblockchain` which starts the rescan without blocking but~~
1. We should be storing a height in the database, not a timestamp
2. The timestamp of the descriptors in the bitcoind watchonly wallet wouldn't be updated
EDIT: i'm dumb, `rescanblockchain` does block too!
I'm happy to have feedback on this. I think the current implementation is fine for now but eventually we should move to using `rescanblockchain` and get rid of the "noreply" client hack.
Also, some functional tests for edge cases we could encounter are included but happy to have some more suggestions.
Fixes#28.
ACKs for top commit:
darosior:
ACK 32f3bdeb16b66878e23ba71a6f2c401c5d1f601a
Tree-SHA512: adca6345329f60d23ed85f0f6abc8f445b10d0c24b5a555a852d0507822598abc7de78742381b1a47b54820f97b037bb4d1ddd0e0b600921ef16c5128ba84950
Give the range to 'importdescriptors' when re-importing a descriptor for
rescanning. This is because the range must include the range of the
descriptor being updated.
Secondly, it is possible that the combination of our timeout-to-1s hack
and our retry logic trigger an edge case: we would retry after
successfully triggering a rescan, and therefore panic on a "a rescan is
already ongoing" error. Instead check before starting that we aren't
rescanning already, and assume that such an error after triggering the
rescan is because we succeeded. That's racy but only reasonably so (as
long as we don't crash, which isn't the case here).
We'll need to store in persistent storage if a rescan was requested by a
user, and if so from what date.
For the SQLite implementation we introduce a rescan_timestamp to the
wallet table.
The query we are going to use to rescan, 'importdescriptors' will block
until the rescan is finished. Stopping the connection through a timeout
immediately after triggering the rescan is a workaround to make it
asynchronous.
Eventually we should probably use 'rescanblockchain' but then it
wouldn't update the timestamps of the descriptors in the watchonly
wallet...
e9fa816be7c6ddbf64413dc4df6940d0f90c1f87 Delete install target datadir in case of failure (edouard)
Pull request description:
close#20
ACKs for top commit:
edouardparis:
Self-ACK e9fa816be7c6ddbf64413dc4df6940d0f90c1f87
Tree-SHA512: 21b78bdad3a3e5127703211e5d119dc9ec88e4bc881ab744e1f6e20d51000af8b5aac4a12f19c775576c69ce958cac7d905378ec8fa4d69e4398fb5fedb61a05
We are passed a timestamp and we want to know down to what height we
should rescan / roll back. That's helpful for it. It was a bit tricky
although it's a simple binary search.. So to make sure i got it right i
made it a standalone function in a utils/ module to be able to unit test
it.
See the TODO, it's theoretically not precisely entirely correct. But
it's good enough for now.
It seems that internally bitcoind might temporarily not have a pprev
pointer for a block. This will result in the optional
"previousblockhash" field to be null and would previously make us crash.
Handle that gracefully.
08505b25452e61a92d5b7eb4a02c893f0d585a91 commands: actually use the change descriptor when creating a Spend (Antoine Poinsot)
Pull request description:
Fixes#78
ACKs for top commit:
darosior:
ACK 08505b25452e61a92d5b7eb4a02c893f0d585a91
Tree-SHA512: 4cccb6636f4e95831a9b3555f6a35cff85d855758b4c26c83b3db8086ea3012af6552ab0db485e982757b68c5c40d6b6696a95bca28c5b5b2f78339bfd800636
3b2c9109dd2c16e3812feb06f322ff0e6b6a61a0 Clippy fixes (Antoine Poinsot)
Pull request description:
Clippy just broke under our feets by enabling more checks and making CI fail...
ACKs for top commit:
darosior:
ACK 3b2c9109dd2c16e3812feb06f322ff0e6b6a61a0
Tree-SHA512: 973f7ae94609209038626a3b6f9ff940d9e488c8588865b439a37a46eec1a5df995013ac98be0fc63e50009339e78238b3a317c568c89677fcb0facf08736874
