1 2017-05-12 00:35:22	0|instagibbs|sipa, could you plot as CDF?
  2 2017-05-12 00:36:26	0|sipa|instagibbs: https://bitcoin.sipa.be/depths.xz
  3 2017-05-12 00:36:29	0|sipa|have fun :)
  4 2017-05-12 00:38:44	0|gmaxwell|I've been chewing on that data, and I don't think it really supports the 144 block target very much.
  5 2017-05-12 01:02:12	0|gmaxwell|sipa: do you have more detailed data?
  6 2017-05-12 01:08:47	0|sipa|i have timestamps + depth for each block transfer
  7 2017-05-12 01:10:00	0|gmaxwell|logically we'd want to put our breakpoints at peaks in the second derivative of the total blocks served (the cumulative version of that data).  Which would in peaks of the derivative of your data.. but it's measurements so getting a useful derivative is hard.   working from a smoothed spline it looks like the peaks are at 29, 199, and then there is so much periodic behavior that its hard to see
  8 2017-05-12 01:10:06	0|gmaxwell|any other trend.   What I think I'm seeing is 24 hour-ish cycles,  but they're spread out in block count due to difficulty changing.
  9 2017-05-12 01:10:30	0|gmaxwell|e.g. if you want to fully capture a 24-hour sycle then perhaps you need 200 blocks, because there can be 200 blocks in a day.
 10 2017-05-12 01:12:23	0|petertodd|gmaxwell: when have I last responded to a post that you think wasn't worth responding too?
 11 2017-05-12 01:22:58	0|gmaxwell|petertodd: I can't think of a recent example, remark wasn't intended to criticize your behavior. Sorry if it came off that way.
 12 2017-05-12 01:47:42	0|gmaxwell|sipa: so from this data, I'm not seeing a lot of justification for anything but a single level at around 200 blocks +plus a dozen or so blocks for headroom (so, e.g. 288 would be fine).    If I use a cumulative maximum from the end to estimate how many transfers are due to fullsync-- there are 107, if then subtract those off, 90% of the cumulative transfer is met by depth 244, and 99% by 410.
 13 2017-05-12 02:39:00	0|bitcoin-git|[13bitcoin] 15instagibbs opened pull request #10390: [wallet] remove minimum total fee option (06master...06killminfee) 02https://github.com/bitcoin/bitcoin/pull/10390
 14 2017-05-12 04:16:11	0|bitcoin-git|[13bitcoin] 15luke-jr opened pull request #10391: OP_CHECKBLOCKATHEIGHT anti-replay (BIP 115; logic only) (06master...06cbah) 02https://github.com/bitcoin/bitcoin/pull/10391
 15 2017-05-12 06:10:47	0|jonasschnelli|gmaxwell: Agree on what you said about the bitcoin-dev list. I refuse to answer.
 16 2017-05-12 06:16:27	0|dcousens|jonasschnelli: here I was thinking about the "2048" of coin selection
 17 2017-05-12 06:16:54	0|jonasschnelli|2048?
 18 2017-05-12 06:17:04	0|dcousens|https://gabrielecirulli.github.io/2048/
 19 2017-05-12 06:18:33	0|jonasschnelli|I wasn't aware of that game,...
 20 2017-05-12 06:18:56	0|dcousens|jonasschnelli: ha, my bad
 21 2017-05-12 06:19:03	0|jonasschnelli|;-)
 22 2017-05-12 06:23:09	0|sipa|jonasschnelli: oh god, i lost about 6 months of my life to that game
 23 2017-05-12 10:45:23	0|petertodd|gmaxwell: no worries, I was just wondering if there was something I missed :)
 24 2017-05-12 10:45:50	0|petertodd|gmaxwell: I'm sure I've done that in the far past! :)
 25 2017-05-12 12:00:52	0|SopaXorzTaker|the possibility of ASICBOOST is a vulnerability in the PoW
 26 2017-05-12 12:00:55	0|SopaXorzTaker|let's fix it now
 27 2017-05-12 12:01:13	0|SopaXorzTaker|then Bitmain would finally move their fat lazy... uhm and accept SegWit
 28 2017-05-12 12:02:08	0|SopaXorzTaker|I mean, that's suggested on Reddit and is definitely about the Core
 29 2017-05-12 12:55:14	0|jonasschnelli|sipa: what do you think about ryanofsky comment here? https://github.com/bitcoin/bitcoin/pull/8501#discussion_r115811850
 30 2017-05-12 12:56:45	0|jonasschnelli|I guess the 26bytes may be padded to 32... so I guess ryanofsky is right here.
 31 2017-05-12 13:57:59	0|SopaXorzTaker|jonasschnelli, hey
 32 2017-05-12 13:58:08	0|SopaXorzTaker|could I ask for some offtopic help?
 33 2017-05-12 14:01:48	0|vedochiaro|ciao
 34 2017-05-12 14:01:59	0|vedochiaro|musica anni 80
 35 2017-05-12 14:41:01	0|jonasschnelli|SopaXorzTaker: shoot
 36 2017-05-12 14:41:29	0|jonasschnelli|If its offtopic use a on-topic channel or PM me
 37 2017-05-12 14:45:51	0|cchadwicka|i need to find an online wallet that allows me to buy bitcoin from within the wallet with a credit card, like coinbase, but a different one
 38 2017-05-12 14:45:54	0|cchadwicka|any ideas
 39 2017-05-12 14:46:09	0|instagibbs|cchadwicka, #bitcoin
 40 2017-05-12 14:46:09	0|jonasschnelli|cchadwicka: please no cross posts... use #bitcoin
 41 2017-05-12 14:47:23	0|cchadwicka|i am banned from bitcoin
 42 2017-05-12 14:47:38	0|cchadwicka|im new here and they kicked me out yesterday
 43 2017-05-12 14:47:43	0|cchadwicka|because i was off topic
 44 2017-05-12 14:48:00	0|sipa|and if you keep posting offtopic things here, i'll ban you too
 45 2017-05-12 14:55:14	0|sipa|jonasschnelli: absolutely, you need to pack the bytes if you want to save space on it
 46 2017-05-12 14:55:37	0|sipa|jonasschnelli: but don't bother, saving 2 bytes in such a structure is negligablr
 47 2017-05-12 16:20:22	0|kanzure|jonasschnelli: fundrawtransaction complains about minimum fee policy when i set feeRate to 0; i have an output with the fee and i was just going to delete the output later...
 48 2017-05-12 16:22:40	0|kanzure|"Note that all inputs selected must be of standard form and P2SH scripts must be in the wallet using importaddress or addmultisigaddress (to calculate fees)."
 49 2017-05-12 16:23:52	0|kanzure|does that mean that if i'm spending p2sh inputs that my wallet doesn't know about (except by importaddress p2sh-address) in my fundrawtransaction input transaction, do i need to first run importaddress redeemscript?
 50 2017-05-12 16:26:19	0|kanzure|really i just want it to ignore my existing inputs-- i could easily pass my estimate of the final transaction size, if necessary.
 51 2017-05-12 16:38:52	0|kanzure|this seems related https://github.com/bitcoin/bitcoin/pull/9965
 52 2017-05-12 16:39:46	0|kanzure|this claims to override minimum estimated fee but i get error "Transaction too large for fee policy" with feeRate=0 https://github.com/bitcoin/bitcoin/pull/7967
 53 2017-05-12 16:52:16	0|instagibbs|kanzure, ah, yes that's a check I've always wondered about, that sounds like a sharp edge
 54 2017-05-12 16:52:43	0|instagibbs|It's checking the fee you're getting against the entire transaction size once constructed
 55 2017-05-12 16:52:49	0|kanzure|sure.
 56 2017-05-12 16:58:21	0|instagibbs|I don't totally get the check's reasoning tbh.
 57 2017-05-12 16:58:23	0|instagibbs|// because we must be at the maximum allowed fee.
 58 2017-05-12 16:58:23	0|instagibbs|/ If we made it here and we aren't even able to meet the relay fee on the next pass, give up
 59 2017-05-12 17:02:04	0|kanzure|workaround: create a fake transaction of the same size as the estimated final size of my unsigned watchonly-p2sh-spending transaction, and then use fundrawtransaction and i'll pass a realistic feeRate value.
 60 2017-05-12 17:02:33	0|instagibbs|I think the check is just supposed to be reversed.
 61 2017-05-12 17:02:42	0|kanzure|and then i'll delete the fake inputs/outputs from the fundrawtransaction output.
 62 2017-05-12 17:05:10	0|kanzure|specifically my problem is that fundrawtransaction gives me "Signing transaction failed" when i directly pass my unsigned watcholy-p2sh-spending transaction as the input to fundrawtransaction.  so i figured hey i'll just set feeRate to 0 on a dummy transaction, add a change output that i'll delete later, and then use fundrawtransaction... which also has problems.
 63 2017-05-12 17:05:32	0|kanzure|*watchonly
 64 2017-05-12 17:07:18	0|instagibbs|possibly related: https://github.com/bitcoin/bitcoin/pull/10202
 65 2017-05-12 17:13:45	0|kanzure|replied, https://github.com/bitcoin/bitcoin/pull/10202#issuecomment-301134038
 66 2017-05-12 17:24:06	0|kanzure|instagibbs: okay here is my new workaround.   createrawtransaction with a change output spending back to myself the total fee that i would like to spend. fundrawtransaction with reasonable feeRate.  using fundrawtransaction output dictionary, i'll switch the "change" output amount to be the fundrawtransaction fee amount, so ultimately there is no extra fee and my original "fee" request shou...
 67 2017-05-12 17:24:12	0|kanzure|...ld be satisfied.
 68 2017-05-12 17:25:02	0|kanzure|and the final transaction will have two change outputs (unless i consolidate).
 69 2017-05-12 18:02:32	0|paveljanik|Are GetDustThreshold and IsDust expected to be called with dustRelayFee other than ::dustRelayFee?
 70 2017-05-12 18:11:48	0|instagibbs|It seems surprising behavior to me to have the wallet simply reduce the fee if it hits maxTxFee, other than panic and abort.
 71 2017-05-12 18:13:05	0|instagibbs|rather than*
 72 2017-05-12 18:26:31	0|kanzure|huh, signrawtransaction also gives me an error ("Operation not valid with the current stack size")
 73 2017-05-12 18:26:54	0|Chris_Stewart_5|Hmm, I think I have had that problem when I forgot to call fundrawtransaction first?
 74 2017-05-12 18:27:38	0|kanzure|i have definitely called fundrawtransaction; i excised the inputs and outputs, and added them to my unsigned p2sh-spending transaction.
 75 2017-05-12 18:31:03	0|Chris_Stewart_5|with watch-only p2sh spending txs the redeemScript is imported right?
 76 2017-05-12 18:32:29	0|kanzure|i am spending watch-only p2sh transactions, however i did not pass includeWatching to fundrawtransaction -- i supplied an unsigned transaction to start with.
 77 2017-05-12 18:33:14	0|kanzure|also, i did not import the redeemscripts, is that important
 78 2017-05-12 18:33:40	0|kanzure|(why would that be important for signrawtransaction? i want to sign only the inputs i'm able to sign.)
 79 2017-05-12 18:38:49	0|kanzure|oh, that's a misleading error message, that's how it reports the transaction is incompletely signed?
 80 2017-05-12 18:39:40	0|Chris_Stewart_5|No, it returns a bool indicating if it is fully signed: https://bitcoin.org/en/developer-reference#signrawtransaction
 81 2017-05-12 18:40:36	0|Chris_Stewart_5|Why are you adding a watch-only output to a tx you are creating? You can't fullfill the spending conditions of it by definition
 82 2017-05-12 18:41:33	0|Chris_Stewart_5|fundrawtransaction will add inputs to the tx until it *fully* funds the outputs of that tx
 83 2017-05-12 18:44:43	0|kanzure|the transaction is fully funded by my inputs, to my knowledge, prior to calling fundrawtransaction with the exception of any extra fee i'm trying to add.
 84 2017-05-12 18:45:56	0|kanzure|Chris_Stewart_5: yeah i was overly focused on "errors" that i overlooked "complete: False" and that it had a new scriptsig :)
 85 2017-05-12 18:47:06	0|Chris_Stewart_5|so fundrawtransaction isn't giving you a large enough fee?
 86 2017-05-12 18:47:37	0|kanzure|fundrawtransaction is giving me an error "Signing transaction failed" if i try to directly call it with my transaction
 87 2017-05-12 18:47:58	0|kanzure|probably because it's using dummy signatures somewhere-- i dunno- and yeah i have not imported the redeemscripts (if that's necessary- which is still unclear to me)
 88 2017-05-12 18:48:29	0|Chris_Stewart_5|Well, if you (or some one else) is trying to spend a p2sh output the redeem script must be provided in the scriptSig
 89 2017-05-12 18:48:49	0|kanzure|i may not have put the reedemScripts in the scriptSigs yet.
 90 2017-05-12 18:49:00	0|Chris_Stewart_5|But since it seems like the p2sh output isn't yours, your counterparty will have to sign the p2sh output and provide the redeem script
 91 2017-05-12 18:49:19	0|instagibbs|kanzure, that would explain it as it's expecting more stack items
 92 2017-05-12 18:49:33	0|Chris_Stewart_5|^
 93 2017-05-12 18:49:35	0|kanzure|i'd like to add fee first, sign for the fee, then sign the other outputs-- it's exceedingly inconvenient for me to do this in another order.
 94 2017-05-12 18:49:41	0|kanzure|*sign for the other inputs
 95 2017-05-12 18:50:18	0|Chris_Stewart_5|You should be able to do it that way, then just pass the partially signed tx to your counterparty
 96 2017-05-12 18:50:36	0|kanzure|i think we've already confirmed that i can't because of the error?
 97 2017-05-12 18:51:17	0|Chris_Stewart_5|oh, yes.. hmm..
 98 2017-05-12 18:51:56	0|kanzure|anyway, my workaround seems to be working for me, where i use createrawtransaction with an output that represents my fee, then later i remove the output, copy the inputs and any extra outputs added by fundrawtransaction to my actual transaction, and then i call signrawtransaction.
 99 2017-05-12 18:53:10	0|kanzure|(also i'm adding the "fee" reported by fundrawtransaction to the change output reported by fundrawtransaction, since i have deleted the output that has my actual fee)
100 2017-05-12 18:53:38	0|Chris_Stewart_5|kanzure: Can't your counter party just add the p2sh output to the transaction? If you remove that outpoint your wallet can sign the tx right?
101 2017-05-12 18:54:12	0|kanzure|er, maybe. but keep in mind that i was calling fundrawtransaction not signrawtransaction-- it's something about dummy sigs.
102 2017-05-12 18:54:50	0|sipa|is the problem that fundrawtransactrion can't determine the feerate of your overall transaction as it does not know how large the scriptSigs of some inputs will need to be?
103 2017-05-12 18:54:53	0|kanzure|so you're saying that i have a partial outpoint and that fundrawtransaction should not report "Signing transaction failed" once i remove the incompete outpoint?
104 2017-05-12 18:54:56	0|Chris_Stewart_5|but in the hex transaction you provided to fundrawtransaction you had specified the watch-only p2sh output?
105 2017-05-12 18:55:04	0|sipa|and as a result can't determine how much fee to use?
106 2017-05-12 18:55:30	0|kanzure|sipa: right, that seems likely to me. i am also interested in confirming/denying with someone whether i need to "importpubkey redeemscript" to overcome that?
107 2017-05-12 18:56:13	0|sipa|or importscript
108 2017-05-12 18:56:44	0|sipa|if fundrawtransaction doesn't know what kind of redeemscript a P2SH output that's being spent has, it can't determine feerate
109 2017-05-12 18:57:01	0|sipa|it may need both, unsure
110 2017-05-12 18:57:26	0|kanzure|i added a comment here explaining that perhaps a better design would be to let user specify the final transaction size (if the user knows it) or allowing feeRate=0 if the user plans to delete an output later https://github.com/bitcoin/bitcoin/pull/10202#issuecomment-301134038
111 2017-05-12 18:59:16	0|kanzure|perhaps CreateTransaction is overloaded at this point :p
112 2017-05-12 19:00:01	0|sipa|just slightly
113 2017-05-12 19:08:25	0|SopaXorzTaker|sipa, can you provide some secp256k1 test cases?
114 2017-05-12 19:09:09	0|SopaXorzTaker|such as, "for private key 0xdeadbeef, z=0xabcd, k=0x1337, r=0x1234 and s=0x5678
115 2017-05-12 21:40:32	0|sipa|general question: how useful is gettxoutsetinfo's serialized_bytes field?
116 2017-05-12 21:40:52	0|sipa|it does not correspond to actual disk usage, and is highly database dependent
117 2017-05-12 21:41:29	0|sipa|it's also impossible to even give a reasonable estimate for after pertxout, except by iterating over the whole database (which I'd like to get rid of)
118 2017-05-12 21:49:44	0|gmaxwell|I think it's not useful, or rather the only thing I've ever used it for is taking about how big the utxo set is on reddit, and an on-disk size would be _better_ for that.
119 2017-05-12 22:09:24	0|BlueMatt|sipa: heh, I have some reivew comments noting it seems out of sync with everything....
120 2017-05-12 22:09:35	0|BlueMatt|sipa: gah, why did you kill Accessors?
121 2017-05-12 22:13:24	0|sipa|BlueMatt: i didn't
122 2017-05-12 22:13:34	0|sipa|how do you mean out of sync?
123 2017-05-12 22:20:04	0|BlueMatt|ehh, meant Modifiers not Accessors
124 2017-05-12 22:20:29	0|BlueMatt|sipa: what is serialized_bytes /supposed/ to show? it seems to not represent anything that actually means something (both before and especially, as you note, after)
125 2017-05-12 22:21:05	0|sipa|BlueMatt: i'm very happy to get rid of Modifiers :D
126 2017-05-12 22:21:12	0|sipa|they're so complicated to reason about
127 2017-05-12 22:21:28	0|BlueMatt|they were there to prevent footgun incidents :(
128 2017-05-12 22:21:46	0|sipa|yes, if you have modifiable references
129 2017-05-12 22:21:50	0|BlueMatt|now you can take a AccessCoins, then add something to the cache, then blow your face off
130 2017-05-12 22:21:58	0|sipa|AccessCoins returns a const reference
131 2017-05-12 22:21:59	0|BlueMatt|AccessCoins gives you such a reference, though?
132 2017-05-12 22:22:11	0|sipa|the only way to modify is through AddCoin and SpendCoin (and BatchWrite)
133 2017-05-12 22:22:23	0|BlueMatt|yes, but if you add something to the cache the reference is still invalid? or are unordered_map refs still valid?
134 2017-05-12 22:22:50	0|BlueMatt|ofc if you remove from cache the same element you've still blown your face off
135 2017-05-12 22:23:14	0|sipa|adding something to the cache may always invalidate references
136 2017-05-12 22:23:20	0|sipa|Modifiers didn't help with that
137 2017-05-12 22:23:31	0|BlueMatt|didnt add assert(!fHasModifier)?
138 2017-05-12 22:23:40	0|BlueMatt|which is what you want if you're returning refs from inside the object
139 2017-05-12 22:23:50	0|BlueMatt|(or, it should have, if it didnt)
140 2017-05-12 22:23:56	0|sipa|yes, there could be at most be one modifier at a time
141 2017-05-12 22:24:05	0|sipa|but that didn't prevent any concurrent references
142 2017-05-12 22:24:26	0|BlueMatt|well we could trivially fix that - reintroduce modifiers and allow multiple ones, but assert when you actually modify the cache that there are no other ones
143 2017-05-12 22:24:39	0|sipa|that won't do anything
144 2017-05-12 22:24:48	0|BlueMatt|they did prevent some footguns, though, like you couldnt take two at a time because that implies you're possibly modifying the cache
145 2017-05-12 22:25:00	0|BlueMatt|sipa: I meant modifier here as accessor, really, from AccessCoins
146 2017-05-12 22:25:14	0|BlueMatt|since its returning a possibly-invalidated-by-other-action reference
147 2017-05-12 22:25:21	0|sipa|they prevented exactly the same footguns as are now prevented by just having explicit modification methods
148 2017-05-12 22:25:24	0|BlueMatt|easy to accidentally have one stick around too long
149 2017-05-12 22:25:37	0|sipa|yes, we could make AccessCoin return something like a modifier... but that seems an independent change
150 2017-05-12 22:25:38	0|BlueMatt|fair
151 2017-05-12 22:25:59	0|BlueMatt|I'd like to see that (or just not return a ref, have you checked the performance overhead of doing that copy everywhere?)
152 2017-05-12 22:27:36	0|sipa|i really don't like copying potentially large data structures where it's unneeded
153 2017-05-12 22:27:58	0|sipa|it's also imcompatible with reducing some of the duplicate output lookups during block/tx validation
154 2017-05-12 22:28:02	0|BlueMatt|fair, I am worried about the footgun potential there, though
155 2017-05-12 22:28:17	0|BlueMatt|maybe we can do the copy for now and introduce an accessor in a separate pr?
156 2017-05-12 22:28:25	0|sipa|please
157 2017-05-12 22:28:31	0|sipa|you don't have to
158 2017-05-12 22:28:40	0|sipa|nothing has changed wrt that case
159 2017-05-12 22:28:57	0|sipa|nothing that was prevented is no longer prevent
160 2017-05-12 22:29:04	0|sipa|and nothing that wasn't prevent is prevented now
161 2017-05-12 22:29:07	0|BlueMatt|hmm, maybe i missed part of how it used to work, though i vaguely recall you doing some AccessCoins in place of old modifiers
162 2017-05-12 22:29:08	0|BlueMatt|maybe im wrong
163 2017-05-12 22:30:25	0|sipa|modifiers were only used when a modification was expected
164 2017-05-12 22:33:36	0|sipa|ok, so in the rpc signing code and in bitcoin-tx, there used to be a Modifier that was both used check whether an output already existed, and then add it
165 2017-05-12 22:33:50	0|sipa|which is now turned into an AccessCoin + AddCoin
166 2017-05-12 22:35:22	0|BlueMatt|yea, i mean its probably no big deal, just more shit to review, would be good to also add some kind of checker later so that it doesnt blow up in our face
167 2017-05-12 22:35:31	0|BlueMatt|cause it'd be easy to slip a change in later that breaks that
168 2017-05-12 22:37:44	0|sipa|FWIW, i'm adding a commit that adds a disk_size to gettexoutsetinfo instead, which just asks LevelDB for the actual size
169 2017-05-12 22:37:59	0|BlueMatt|that sounds much better
170 2017-05-12 22:40:29	0|sipa|BlueMatt: std::unordered_map references are only invalidated when the corresponding entry is deleted
171 2017-05-12 22:40:44	0|sipa|they remain valid under addition, including when rehashing occurs
172 2017-05-12 22:43:27	0|BlueMatt|oh, heh, i thought you said no earlier, ok
173 2017-05-12 22:43:31	0|BlueMatt|yea, makes sense
174 2017-05-12 22:44:13	0|sipa|iterators are invalidated, but iterators from the underlying CCoinsViewCache are never exposed anymore
175 2017-05-12 22:44:53	0|BlueMatt|ohoh, ok