1 2016-04-11 01:22:08	0|BlueMatt|anyone around who might have a node which restarts occasionally, could you grep for "prev block not found" in debug.log and report the frequency?
  2 2016-04-11 01:24:13	0|achow101|BlueMatt: I've got it twive
  3 2016-04-11 01:24:18	0|achow101|*twice
  4 2016-04-11 01:24:25	0|BlueMatt|across how many days/restarts?
  5 2016-04-11 01:26:17	0|GitHub121|[13bitcoin] 15gmaxwell opened pull request #7856: Only send one GetAddr response per connection. (06master...06addr_once) 02https://github.com/bitcoin/bitcoin/pull/7856
  6 2016-04-11 01:26:39	0|achow101|since march 10th and 51 restarts
  7 2016-04-11 01:27:13	0|BlueMatt|achow101: thanks
  8 2016-04-11 07:03:40	0|jonasschnelli|Holly! The Github traffic on weekends is extremely intense. Catch up after a weekend (people where supposed to not work on weekend,.. once) needs 1-2h! :)
  9 2016-04-11 07:04:22	0|sipa|weekends are highly overratee
 10 2016-04-11 07:04:25	0|sipa|weekends are highly overrated
 11 2016-04-11 07:08:23	0|jonasschnelli|Yah! Tell that my wife. :)
 12 2016-04-11 07:26:08	0|GitHub84|[13bitcoin] 15promag opened pull request #7857: Add fee to fundrawtransaction (06master...06enhancement/add-fee-to-fundrawtransaction) 02https://github.com/bitcoin/bitcoin/pull/7857
 13 2016-04-11 08:01:08	0|BlueMatt|hmm, the sendheaders stuff is pretty bitcoin-core-specific
 14 2016-04-11 08:01:38	0|BlueMatt|we DoS a node for sending us a header that doesnt connect to our chain, because bitcoin core keeps track of what it thinks its peers have for headers (and assumes they never throw away headers)
 15 2016-04-11 08:01:42	0|BlueMatt|sipa: ^
 16 2016-04-11 08:02:38	0|BlueMatt|if, for example, a peer didnt want to keep track of what it thought we had in our block-headers-tree, it couldnt implement sendheaders
 17 2016-04-11 08:05:36	0|sipa|you can always send all headers along the new branch you're switching to
 18 2016-04-11 08:06:06	0|BlueMatt|also, we assume a node doesnt have a given set of headers based on only the most recently received or sent hash/header...this works fine for non-forked chains, but if the chain forks at the tip, we may very well fall back to sending invs
 19 2016-04-11 08:06:40	0|BlueMatt|sipa: but we dont do this
 20 2016-04-11 08:06:54	0|BlueMatt|sipa: also, why not just remove the DoS?
 21 2016-04-11 08:07:09	0|BlueMatt|sipa: and if it doesnt connect, just send a getheaders for further-back headers
 22 2016-04-11 08:07:18	0|sipa|which case are you talking about, a non-core peer sending headers to core, or the other way around?
 23 2016-04-11 08:07:32	0|BlueMatt|in this case I'm talking about core-to-core
 24 2016-04-11 08:07:52	0|BlueMatt|from my (not so in depth) reading, this looks like it can very easily fall back to invs if the chain is forked near the tip
 25 2016-04-11 08:08:11	0|sipa|it should send all new headers along the new branch
 26 2016-04-11 08:08:26	0|sipa|unless there are too many, in which case it intentionally falls back to using invs
 27 2016-04-11 08:08:27	0|BlueMatt|eg we just got a new block on our chaintip B (based on A), but we only know they have C (which we may not even have, but lets say its based on A)
 28 2016-04-11 08:08:54	0|sipa|then we send the header for B
 29 2016-04-11 08:09:01	0|BlueMatt|I dont think it will? It will only ever send invs/headers that were added to vBlockHashesTonnounce
 30 2016-04-11 08:09:19	0|BlueMatt|no, because B wont be in vBlockHashesToAnnounce
 31 2016-04-11 08:09:34	0|BlueMatt|only D (based on B) will be
 32 2016-04-11 08:10:47	0|BlueMatt|in any case, it is much more flexible to handle the headers-msg-that-didnt-connect case by requesting a headers with our locator and see what they come back with instead of marking that peer a DoS
 33 2016-04-11 08:11:06	0|BlueMatt|would make it much easier for other implementors (they can skip all block inv sending, though that would be a bit less effecient)
 34 2016-04-11 08:12:10	0|BlueMatt|sipa: eg https://github.com/TheBlueMatt/bitcoin/commits/1daf94f540592345dca5818321a28996fe7e04fd
 35 2016-04-11 08:12:11	0|BlueMatt|?
 36 2016-04-11 08:14:33	0|sipa|hmm, i'd still like to understand the use case
 37 2016-04-11 08:15:59	0|BlueMatt|sipa: the use-case is simpler code for other implementors (or maybe us in the future)
 38 2016-04-11 08:16:06	0|sipa|we have D-B-A as tip, a peer has C-A as tip
 39 2016-04-11 08:16:08	0|BlueMatt|sipa: they could even skip the inv logic
 40 2016-04-11 08:16:13	0|BlueMatt|sipa: that is a separate bug
 41 2016-04-11 08:16:22	0|sipa|let's talk one thing at a timr
 42 2016-04-11 08:17:18	0|sipa|but the peer will have the headers for D and B already
 43 2016-04-11 08:17:25	0|sipa|eh, B
 44 2016-04-11 08:17:31	0|sipa|so just sending D should be fine?
 45 2016-04-11 08:18:02	0|BlueMatt|this only works if either the last header we sent them or the last header they sent us is B
 46 2016-04-11 08:18:37	0|sipa|it's possible that B was sent just as an inv
 47 2016-04-11 08:18:51	0|sipa|and the peer hasn't getdata'd yet
 48 2016-04-11 08:18:58	0|BlueMatt|no, its worse
 49 2016-04-11 08:19:07	0|BlueMatt|they could have had B already from another peer
 50 2016-04-11 08:19:19	0|BlueMatt|well, yea, i suppose it would be an inv, then
 51 2016-04-11 08:19:22	0|BlueMatt|but they wouldnt request it
 52 2016-04-11 08:19:34	0|sipa|we would still have either sent an inv or a header, indeed
 53 2016-04-11 08:20:11	0|BlueMatt|anyway, minor bug, just means if chain is forked when you first connect to a peer, you will probably send block as inv instead of header
 54 2016-04-11 08:21:11	0|BlueMatt|butttt, this does feed into my more general protocol suggestion/question here, which is that we seem to have half-assed the sendheaders stuff....I can get that if you're sending a lot you might prefer invs over headers, but if you're only sending a few, this semi-duplicated logic doesnt make sense to me
 55 2016-04-11 08:21:26	0|BlueMatt|why not just always use headers and if you got headers that you cant connect, treat them as invs and request more headers
 56 2016-04-11 08:22:58	0|BlueMatt|anyway, bedtime for me
 57 2016-04-11 09:32:08	0|GitHub33|[13bitcoin] 15jl2012 opened pull request #7858: Add jl2012 public key for gitian build (06master...06patch-9) 02https://github.com/bitcoin/bitcoin/pull/7858
 58 2016-04-11 10:46:42	0|wumpus|oh, that's too bad, I saw a "Merge button" setting under repository settings, I thought because we use a custom script we could disable it entirely. Unfortunately "You must select at least one option"
 59 2016-04-11 10:46:50	0|wumpus|going to request this
 60 2016-04-11 10:49:31	0|sipa|wumpus: that will be interesting :)
 61 2016-04-11 10:51:40	0|wumpus|jonasschnelli: hah! and remember that if you take all time zones into account, I think there's only one full day it's 'weekend' everywhere
 62 2016-04-11 10:52:03	0|wumpus|yes it'd be nice
 63 2016-04-11 10:57:52	0|GitHub183|13bitcoin/06master 1464c22be 15Johnson Lau: Add jl2012 public key for gitian build
 64 2016-04-11 10:57:52	0|GitHub183|[13bitcoin] 15laanwj pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/065c6b443f3e...934f2b5e7693
 65 2016-04-11 10:57:53	0|GitHub183|13bitcoin/06master 14934f2b5 15Wladimir J. van der Laan: Merge #7858: Add jl2012 public key for gitian build...
 66 2016-04-11 10:58:02	0|GitHub114|[13bitcoin] 15laanwj closed pull request #7858: Add jl2012 public key for gitian build (06master...06patch-9) 02https://github.com/bitcoin/bitcoin/pull/7858
 67 2016-04-11 11:02:13	0|GitHub100|13bitcoin/060.12 14de7c34c 15BtcDrak: Add missing link to BIP113
 68 2016-04-11 11:02:13	0|GitHub100|[13bitcoin] 15laanwj pushed 2 new commits to 060.12: 02https://github.com/bitcoin/bitcoin/compare/465d30915cd3...9779e1e1f320
 69 2016-04-11 11:02:13	0|GitHub192|[13bitcoin] 15laanwj closed pull request #7852: [0.12] Add missing reference to release notes (060.12...06bip113) 02https://github.com/bitcoin/bitcoin/pull/7852
 70 2016-04-11 11:02:14	0|GitHub100|13bitcoin/060.12 149779e1e 15Wladimir J. van der Laan: Merge #7852: [0.12] Add missing reference to release notes...
 71 2016-04-11 11:59:20	0|instagibbs|wumpus, you pointed to rc1 in the source on mailing list, FYI :P
 72 2016-04-11 11:59:35	0|wumpus|yes I sent a mail after it correcting it right?
 73 2016-04-11 11:59:52	0|instagibbs|maybe my email is slow
 74 2016-04-11 12:00:03	0|wumpus|https://lists.linuxfoundation.org/pipermail/bitcoin-core-dev/2016-April/000002.html
 75 2016-04-11 12:00:17	0|instagibbs|oh i see it, maybe I checked email while asleep :)
 76 2016-04-11 12:06:43	0|wumpus|:)
 77 2016-04-11 12:10:17	0|sipa|oh, you're a sleepmailer too?
 78 2016-04-11 12:46:55	0|instagibbs|with new baby you never quite know when you checked things fully awake
 79 2016-04-11 13:53:43	0|GitHub23|[13bitcoin] 15jonasschnelli closed pull request #7831: [CLI] refactor wallets RPC JSON conversions (06master...062016/04/cli_conversion) 02https://github.com/bitcoin/bitcoin/pull/7831
 80 2016-04-11 14:46:56	0|jonasschnelli|hmm... these guys: https://twitter.com/kawalgrover/status/719535585153716224
 81 2016-04-11 14:57:40	0|Chris_Stewart_5|Is CScriptNum(0) bitwise equivalent to OP_0?
 82 2016-04-11 14:58:01	0|Chris_Stewart_5|aka CScriptNum(0) OP_0 OP_EQUAL would push an OP_TRUE on the stack?
 83 2016-04-11 14:58:24	0|Chris_Stewart_5|sanity check for my own sake..
 84 2016-04-11 15:00:27	0|sipa|CScriptNum(0) is a C++ expression
 85 2016-04-11 15:00:35	0|sipa|OP_0 is a script operator
 86 2016-04-11 15:01:12	0|sipa|OP_0 pushes an empty bytearray onto the execution stack
 87 2016-04-11 15:01:51	0|sipa|which, if fed as input to CScriptNum will indeed produce the same as feeding it the number 0
 88 2016-04-11 15:03:30	0|Chris_Stewart_5|interesting. Is that just something we have to live with since Satoshi put it their or have we put that in after the fact?
 89 2016-04-11 15:03:51	0|Chris_Stewart_5|but the little script I gave about would evaluate to OP_FALSE then since an empty byte vector != 0 right?
 90 2016-04-11 15:04:35	0|Chris_Stewart_5|Seems like CScriptNum(OP_0) is a little strange.. surprised that even type checks
 91 2016-04-11 15:08:05	0|sipa|Chris_Stewart_5: "CScriotNum(0) OP_0 OP_EQUAL" has no meaning, i cannot answer your question
 92 2016-04-11 15:09:03	0|sipa|CScriotNum(0) is C++
 93 2016-04-11 15:09:32	0|sipa|it is not something that makes sense in a bitcoin script
 94 2016-04-11 15:19:08	0|sipa|Chris_Stewart_5: CScriptNum(OP_0) unfortunately works due to C++ type conversion (OP_0 is part of the opcodes enum, and gets automatically converted to an int when passing to CScriptNum's constructor
 95 2016-04-11 15:23:05	0|Chris_Stewart_5|sipa: is the number 0 and the empty byte vector bitwise equivalent in Script? I think that is what I was really trying to ask
 96 2016-04-11 15:23:22	0|Chris_Stewart_5|I'm just trying to understand how exactly it is handled, it seems in the interpreter these lines are how it is interpreted
 97 2016-04-11 15:23:25	0|Chris_Stewart_5|https://github.com/bitcoin/bitcoin/blob/master/src/script/interpreter.cpp#L290-L294
 98 2016-04-11 15:24:07	0|Chris_Stewart_5|I'm not sure how the idea of 0 is handled in most PLs, is it represented as 0x00 or just an empty byte vector... I guess i'll have to more research on that later :-)
 99 2016-04-11 15:26:42	0|sipa|Chris_Stewart_5: in most languages, numbers and byte arrays are different data types, so the question does not even arise
100 2016-04-11 15:27:01	0|sipa|but in Scriot, yes, an empty stack elemented is interpreted as tje number 0, when fed to ancoperator that expects a number
101 2016-04-11 15:28:04	0|Chris_Stewart_5|Thanks!
102 2016-04-11 15:41:50	0|sipa|there are other valid serializations for 0, though.
103 2016-04-11 15:56:11	0|Chris_Stewart_5|sipa: Are you planning on doing something similar to tx_valid/invalid.json as you did with your recent pull request?
104 2016-04-11 15:57:17	0|Chris_Stewart_5|wrt to script_valid/invalid.json
105 2016-04-11 18:57:42	0|sdaftuar|gmaxwell: sipa: i was looking at 7840, and i think i must not understand gmaxwell's comment about "by eliminating queued entries from the mempool response and responding only at trickle time, this makes the mempool no longer leak transaction arrival order information..."
106 2016-04-11 18:58:29	0|sdaftuar|it seemed to me that if we are responding to mempool messages only at trickle time, then we might as well include everything and clear the relay queue for that peer instead?
107 2016-04-11 18:59:53	0|sipa|sdaftuar: that would allow you to bypass the rate limiting by continuously requesting mempool
108 2016-04-11 19:00:08	0|gmaxwell|you already bypass the rate limit via the mempool results itself though.
109 2016-04-11 19:00:10	0|sdaftuar|ah
110 2016-04-11 19:00:40	0|sdaftuar|but with my approach, you can learn about more than just 35 transactions or whatever
111 2016-04-11 19:01:17	0|gmaxwell|At least my view is that it's not necessary to enforce the limit in that case. The limit is there to soften traffic surges when floods of new transactions come in. If you're making a mempool request, you clearly don't care. ... the alernative would be to add the mempool to the inv to send set and subject it to the rate limiter... but that would make mempool requests rather slow. :)
112 2016-04-11 19:01:42	0|sipa|that makes sense
113 2016-04-11 19:05:56	0|sdaftuar|so if we're not concerned about rate limiting being weakened by a mempool-attacker, is there any downside to responding with everything?
114 2016-04-11 19:07:28	0|gmaxwell|I think it should respond with everything*, and remove them from the set-to-send as it goes.   (*ignoring my general complaint about there existing any P2P message that sends megabytes of response to a single query)
115 2016-04-11 19:08:13	0|sdaftuar|ok that makes sense to me, though i don't know if there's any point to relaying something that has been evicted in between deciding to relay and trying to relay
116 2016-04-11 19:08:33	0|sdaftuar|it's an unlikely edge case regardless
117 2016-04-11 19:08:51	0|gmaxwell|It probably shouldn't relay it.
118 2016-04-11 19:09:45	0|sdaftuar|actually, with the rate limiting, we should probably remove things that are mined or evicted prior to relay, right?
119 2016-04-11 19:09:56	0|sdaftuar|it's conceivable that you could have a big backlog if someone dumped their mempool at you
120 2016-04-11 19:11:37	0|gmaxwell|sdaftuar: related to that, I'm looking to moving all the filtering to just-in-time  https://people.xiph.org/~greg/temp/0001-Just-in-time-inventory-filtering.patch
121 2016-04-11 19:11:57	0|gmaxwell|Not a PR yet because I wanted pieter's input before going and ripping out the feerate from the RelayTransaction prototype everywhere.
122 2016-04-11 19:12:23	0|gmaxwell|So that applies the bloom and fee-filter right before relaying, not on the queue... so that effects of changing these filters is not delayed.
123 2016-04-11 19:12:45	0|sdaftuar|that seems nice, and makes the code clearer too i suspect
124 2016-04-11 19:13:12	0|sdaftuar|i guess at the expense of slightly more memory, especially for bloom-filtering peers?
125 2016-04-11 19:13:58	0|sdaftuar|i'm not sure i understand the privacy gain though
126 2016-04-11 19:14:01	0|gmaxwell|Very slightly, but it doesn't increase the worst case.
127 2016-04-11 19:14:09	0|sdaftuar|agreed
128 2016-04-11 19:14:17	0|gmaxwell|sdaftuar: without that change, I can rotate my filtering to 'time tag' arrivals.
129 2016-04-11 19:14:39	0|gmaxwell|So even though the queuing destroys timing information, by modulating my filtering I could restore some of it.
130 2016-04-11 19:15:05	0|sdaftuar|assuming a single peer connection?
131 2016-04-11 19:15:18	0|sdaftuar|or multiple?
132 2016-04-11 19:15:41	0|gmaxwell|If it's done with just a single, it means you'll miss some transactions. More effective with multiple.
133 2016-04-11 19:17:56	0|gmaxwell|But it's justifyable just on the basis of making the relays more accurate. Likewise, something that was evicted shouldn't get relayed. Though thats corner case enough I don't know if it's worth checking.
134 2016-04-11 19:18:35	0|sdaftuar|ok i think i see what you're talking about, makes sense.
135 2016-04-11 19:20:32	0|sdaftuar|do you think it's worth trying to drain a peer's tx relay set faster if there's a big backlog?  i'm a little worried that this thing is bounded only by the size of the mempool, and drains at a fixed rate
136 2016-04-11 19:21:13	0|sdaftuar|100k transactions would take 4 hours or so, and in the meantime you're using a bunch of memory and slowing down the heapify, i suppose
137 2016-04-11 19:22:18	0|gmaxwell|I'd rather drop stragglers than drain much faster.
138 2016-04-11 19:22:38	0|sdaftuar|yeah that seems better
139 2016-04-11 19:22:54	0|sdaftuar|otherwise a peer who dumps you garbage ruins relay for everyone...
140 2016-04-11 19:29:37	0|gmaxwell|right now someone flooding at the boundary of what you'll accept turns you into a big relay amplifier, spamming people with junk. The fee filter will help that a lot, but not completely. I was thinking about using a map on id->{depth,feerate,insert time}, and using the first two for sorting (without taking the mempool lock), the feerate for filtering (again no mempool lock), and the insert time
141 2016-04-11 19:29:43	0|gmaxwell|to expire things that sat around too long.
142 2016-04-11 19:48:47	0|gribble|I have not seen bramc.
143 2016-04-11 19:48:47	0|JackH|!seen bramc
144 2016-04-11 20:10:32	0|GitHub88|[13bitcoin] 15sdaftuar opened pull request #7862: Use txid as key in mapAlreadyAskedFor (06master...06inv-to-txid-mapalreadyaskedfor) 02https://github.com/bitcoin/bitcoin/pull/7862
145 2016-04-11 21:12:17	0|Chris_Stewart_5|Why does the DISCOURAGE_UPGRADABLE_NOPS flag allow a OP_NOP?
146 2016-04-11 21:12:47	0|Chris_Stewart_5|i.e. this test case ["1", "NOP", "P2SH,STRICTENC,DISCOURAGE_UPGRADABLE_NOPS", "OK", "Discourage NOPx flag allows OP_NOP"]
147 2016-04-11 21:31:24	0|instagibbs|OP_NOP isn't in the set of NOPx
148 2016-04-11 21:31:33	0|instagibbs|although that probably doesn't answer your question
149 2016-04-11 21:32:06	0|instagibbs|OP_NOP is 0x61 while the other NOPS are 0xb0, so they were probably created for different reasons
150 2016-04-11 21:32:55	0|instagibbs|Meaning OP_NOP will probably not be relied on for future softforks, especially if they aren't currently discouraged