1 2016-09-01 00:25:51 0|GitHub77|[13bitcoin] 15fanquake opened pull request #8640: [depends] Remove Qt46 package (06master...06remove-qt46-depends) 02https://github.com/bitcoin/bitcoin/pull/8640
2 2016-09-01 01:52:47 0|instagibbs|am I the only one who never types in those types of commands in chat windows? :)
3 2016-09-01 01:54:22 0|gmaxwell|yea, man, who does that. Next thing we're going to find out that someone used their same password on github and the bitcoin core slack. :P
4 2016-09-01 01:55:38 0|achow101|not me! :D Changed all those passwords
5 2016-09-01 01:56:28 0|instagibbs|hypothetically speaking of course >_>
6 2016-09-01 01:58:59 0|kanzure|we should probably tell achow101 that he should use better passwords :)
7 2016-09-01 02:00:29 0|achow101|It was for my computer actually (synergy screwed something up when I typed it). I don't actually use passwords like that since I use a password manager to randomly generate all of my other passwords.
8 2016-09-01 02:00:32 0|warren|That's the combination on my luggage!
9 2016-09-01 03:22:28 0|luke-jr|are we actively trying to remove boost in non-consensus code, or is it okay to use?
10 2016-09-01 03:26:14 0|gmaxwell|We are now using C++11-- so could you consider using that instead? I don't think we're quite at 'actively trying to remove' yet.
11 2016-09-01 03:49:05 0|luke-jr|gmaxwell: C++11 doesn't include all of boost, for better or worse. Particularly, I'm looking at boost::any for #7289
12 2016-09-01 03:52:26 0|gmaxwell|ugh, do you really want boost::any? we like static typing..
13 2016-09-01 03:54:21 0|luke-jr|maybe not. I was thinking having each option-definition parse the param string to its correct type in a parse virtual, and then call that result into a check/set. The reason boost::any is needed with this design, is that the caller to parse+set doesn't necessarily need to know the types it's parsing
14 2016-09-01 03:54:30 0|luke-jr|but perhaps there's a way to workaround this
15 2016-09-01 03:55:15 0|luke-jr|(eg, move the parse internal to the setter, and set by string)
16 2016-09-01 07:11:55 0|GitHub15|[13bitcoin] 15rebroad reopened pull request #8622: [WIP] Clarify variable names (06master...06ClarifyVariableNames) 02https://github.com/bitcoin/bitcoin/pull/8622
17 2016-09-01 08:05:12 0|GitHub2|[13bitcoin] 15sipa closed pull request #8597: Move code from VERACK to VERSION (since VERACK is not requied) (06master...06NotDependentOnVerack) 02https://github.com/bitcoin/bitcoin/pull/8597
18 2016-09-01 08:05:36 0|GitHub45|[13bitcoin] 15sipa closed pull request #8622: [WIP] Clarify variable names (06master...06ClarifyVariableNames) 02https://github.com/bitcoin/bitcoin/pull/8622
19 2016-09-01 08:25:05 0|GitHub22|[13bitcoin] 15rebroad opened pull request #8642: Less reliance on node version (06master...06LessRelianceOnNodeVersion) 02https://github.com/bitcoin/bitcoin/pull/8642
20 2016-09-01 08:25:20 0|luke-jr|sigh
21 2016-09-01 08:28:26 0|gmaxwell|I dunno whats going on there.
22 2016-09-01 08:28:38 0|gmaxwell|I feel like I am being fuzz tested.
23 2016-09-01 08:28:43 0|luke-jr|lol
24 2016-09-01 08:30:25 0|gmaxwell|we need pulltester tests that test against a fixed point, so that if you break the protocol in a self-compatible way, the CI tests will still fail.
25 2016-09-01 08:31:02 0|gmaxwell|those changes will throughly break communications with at least some versions.
26 2016-09-01 08:32:48 0|sipa|gmaxwell: it seems intentional
27 2016-09-01 08:33:31 0|gmaxwell|it's completely wildly handled though, sending garbage messages to peers that will tell who knows what malfunction.
28 2016-09-01 08:35:25 0|sipa|one of the commits mentions that a protocol change introduced in 2010 should be supported by everyone
29 2016-09-01 08:35:38 0|sipa|but he kills pre-bip31 nodes as well, which dates from 2012
30 2016-09-01 08:38:49 0|gmaxwell|yea, I was looking at the BIP31 changes and scratching my head.
31 2016-09-01 08:39:15 0|luke-jr|I guess there's a safe way one /could/ do that.. not sure it's worth the effort though
32 2016-09-01 08:39:30 0|luke-jr|(if vRecv has nonce data, read and use it; otherwise, don't)
33 2016-09-01 08:43:34 0|gmaxwell|I don't see why we'd touch old, working, functional code-- if there was some big reworking of those parts for other reasons, then sure it might be justifyable to break compatiblity at least back to the point where compatability is already likely broken for other reasons.
34 2016-09-01 08:51:06 0|warren|Big reworking of the network layer is happening in the rewrite. It makes no sense to change anything before the rewrite, especially working code.
35 2016-09-01 08:52:34 0|sipa|can someone else other than me comment there?
36 2016-09-01 08:54:27 0|gmaxwell|last comment from his is you're right.
37 2016-09-01 08:56:55 0|warren|I don't understand why he's citing nodes that are INTENDED to be incompatible as reason to change anything.
38 2016-09-01 10:21:06 0|GitHub97|[13bitcoin] 15sipa pushed 4 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/84decb54f259...19b0f33de0ef
39 2016-09-01 10:21:07 0|GitHub97|13bitcoin/06master 14ab48c5e 15Nicolas DORIER: Unit test for sighash caching
40 2016-09-01 10:21:07 0|GitHub97|13bitcoin/06master 14d2c5d04 15Pieter Wuille: Precompute sighashes...
41 2016-09-01 10:21:08 0|GitHub97|13bitcoin/06master 1435fe039 15Pieter Wuille: Rename to PrecomputedTransactionData
42 2016-09-01 10:21:12 0|GitHub100|[13bitcoin] 15sipa closed pull request #8524: Precompute sighashes (06master...06noprecomputecachedhashes) 02https://github.com/bitcoin/bitcoin/pull/8524
43 2016-09-01 10:25:40 0|gmaxwell|\O/
44 2016-09-01 10:37:08 0|jl2012|great, one big step closer to 0.13.1
45 2016-09-01 11:56:41 0|GitHub89|13bitcoin/06master 142663e51 15Wladimir J. van der Laan: Merge #8640: [depends] Remove Qt46 package...
46 2016-09-01 11:56:41 0|GitHub89|[13bitcoin] 15laanwj pushed 1 new commit to 06master: 02https://github.com/bitcoin/bitcoin/commit/2663e5149dc06dedb8c29672abe4b1c645768e52
47 2016-09-01 11:56:51 0|GitHub131|[13bitcoin] 15laanwj closed pull request #8640: [depends] Remove Qt46 package (06master...06remove-qt46-depends) 02https://github.com/bitcoin/bitcoin/pull/8640
48 2016-09-01 12:42:27 0|GitHub162|13bitcoin/06master 1433d15a3 15Pavel JanÃÂk: Do not shadow LOCK's criticalblock variable for LOCK inside LOCK
49 2016-09-01 12:42:27 0|GitHub162|[13bitcoin] 15laanwj pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/2663e5149dc0...0e563d89c075
50 2016-09-01 12:42:28 0|GitHub162|13bitcoin/06master 140e563d8 15Wladimir J. van der Laan: Merge #8472: Do not shadow LOCK's criticalblock variable for LOCK inside LOCK...
51 2016-09-01 12:42:37 0|GitHub159|[13bitcoin] 15laanwj closed pull request #8472: Do not shadow LOCK's criticalblock variable for LOCK inside LOCK (06master...0620160806_Wshadow_LOCK) 02https://github.com/bitcoin/bitcoin/pull/8472
52 2016-09-01 13:40:41 0|GitHub84|[13bitcoin] 15laanwj closed pull request #7376: Payments: Allow zero value OP_RETURN in PaymentRequests (06master...06zeropayments) 02https://github.com/bitcoin/bitcoin/pull/7376
53 2016-09-01 13:57:29 0|GitHub140|[13bitcoin] 15laanwj pushed 4 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/0e563d89c075...44691f3c51c9
54 2016-09-01 13:57:30 0|GitHub140|13bitcoin/06master 14fa917f6 15MarcoFalke: [contrib] verifybinaries: Keep downloads by default
55 2016-09-01 13:57:30 0|GitHub140|13bitcoin/06master 14fab1f92 15MarcoFalke: [contrib] verifybinaries: Adjust parsing to new rc path
56 2016-09-01 13:57:31 0|GitHub140|13bitcoin/06master 14faaed88 15MarcoFalke: [contrib] verifybinaries: Mention mandatory preparation step
57 2016-09-01 13:57:43 0|GitHub143|[13bitcoin] 15laanwj closed pull request #8557: [contrib] Rework verifybinaries (06master...06Mf1608-verifyBins) 02https://github.com/bitcoin/bitcoin/pull/8557
58 2016-09-01 13:59:03 0|GitHub10|13bitcoin/06master 14f012a85 15djpnewton: rest.cpp: change HTTP_INTERNAL_SERVER_ERROR to HTTP_BAD_REQUEST
59 2016-09-01 13:59:03 0|GitHub10|[13bitcoin] 15laanwj pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/44691f3c51c9...f061415d12ce
60 2016-09-01 13:59:04 0|GitHub10|13bitcoin/06master 14f061415 15Wladimir J. van der Laan: Merge #8638: rest.cpp: change HTTP_INTERNAL_SERVER_ERROR to HTTP_BAD_REQUEST...
61 2016-09-01 13:59:13 0|GitHub78|[13bitcoin] 15laanwj closed pull request #8638: rest.cpp: change HTTP_INTERNAL_SERVER_ERROR to HTTP_BAD_REQUEST (06master...06patch-2) 02https://github.com/bitcoin/bitcoin/pull/8638
62 2016-09-01 14:25:58 0|GitHub130|[13bitcoin] 15laanwj pushed 5 new commits to 060.12: 02https://github.com/bitcoin/bitcoin/compare/15502d7b2543...ec0afbd52b78
63 2016-09-01 14:25:58 0|GitHub29|[13bitcoin] 15laanwj closed pull request #8176: [0.12.x]: Versionbits: GBT support (060.12...06gbt_bip9-0.12.x) 02https://github.com/bitcoin/bitcoin/pull/8176
64 2016-09-01 14:25:59 0|GitHub130|13bitcoin/060.12 1440e81f5 15Luke Dashjr: qa/rpc-tests: bip9-softforks: Add tests for getblocktemplate versionbits updates
65 2016-09-01 14:25:59 0|GitHub130|13bitcoin/060.12 14ddd8c01 15Luke Dashjr: Implement BIP 9 GBT changes...
66 2016-09-01 14:26:00 0|GitHub130|13bitcoin/060.12 1465ee332 15Luke Dashjr: getblocktemplate: Explicitly handle the distinction between GBT-affecting softforks vs not
67 2016-09-01 14:28:28 0|paveljanik|sipa, PrecomputedTransactionData is defined as struct, but: src/main.h:class PrecomputedTransactionData;
68 2016-09-01 14:29:30 0|sipa|will fix
69 2016-09-01 14:29:37 0|paveljanik|ok, thanks.
70 2016-09-01 16:52:29 0|jtimon|it seems travis is failing walletbackup.py no matter what I push https://travis-ci.org/bitcoin/bitcoin/jobs/156853453#L2310
71 2016-09-01 17:02:39 0|jtimon|mhmm, no, just randomly it seems, nothing to see here, sorry
72 2016-09-01 17:13:12 0|wumpus|jtimon: travis is a tad flakey at the moment
73 2016-09-01 17:15:04 0|jtimon|I see
74 2016-09-01 17:48:48 0|jeremyrubin|Yeah my lockfree checkqueue work should be passing all tests now; but has failures due to flaky pulltester :/
75 2016-09-01 17:51:27 0|jeremyrubin|Ah
76 2016-09-01 17:52:08 0|jeremyrubin|actually I did pass them on the last run; but I hadn't really made any changes that should have affected pass/fail
77 2016-09-01 18:59:21 0|wumpus|meeting time?
78 2016-09-01 18:59:29 0|helo|<3
79 2016-09-01 18:59:35 0|jonasschnelli|yes
80 2016-09-01 18:59:37 0|luke-jr|big storm here, I may lose power
81 2016-09-01 19:00:09 0|paveljanik|Hi
82 2016-09-01 19:00:13 0|sipa|ohai
83 2016-09-01 19:00:21 0|achow101|hi
84 2016-09-01 19:00:24 0|gmaxwell|#bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr btcdrak sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier
85 2016-09-01 19:00:30 0|kanzure|greetz.
86 2016-09-01 19:00:34 0|sdaftuar|hi
87 2016-09-01 19:00:39 0|cfields|here
88 2016-09-01 19:00:43 0|gmaxwell|luke-jr: I miss thunderstorms.
89 2016-09-01 19:01:02 0|btcdrak|here
90 2016-09-01 19:01:42 0|sipa|there
91 2016-09-01 19:01:58 0|lightningbot|Meeting started Thu Sep 1 19:01:57 2016 UTC. The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
92 2016-09-01 19:01:58 0|lightningbot|Useful Commands: #action #agreed #help #info #idea #link #topic.
93 2016-09-01 19:01:58 0|wumpus|#startmeeting
94 2016-09-01 19:02:21 0|wumpus|proposed topics?
95 2016-09-01 19:02:47 0|sipa|remaining 0.13.1 issues
96 2016-09-01 19:02:49 0|jtimon|hi
97 2016-09-01 19:03:00 0|instagibbs|present
98 2016-09-01 19:03:03 0|wumpus|there are lots of 0.13.1-tagged pull requests open, any that should be prioritized for review?
99 2016-09-01 19:03:04 0|jeremyrubin|present
100 2016-09-01 19:03:17 0|sipa|wumpus: in fact, some conflict
101 2016-09-01 19:03:19 0|wumpus|(e.g. what should go in first)
102 2016-09-01 19:03:24 0|wumpus|yes, I thought so
103 2016-09-01 19:03:26 0|jonasschnelli|https://github.com/bitcoin/bitcoin/milestones/0.13.1
104 2016-09-01 19:03:57 0|wumpus|#link https://github.com/bitcoin/bitcoin/milestones/0.13.1
105 2016-09-01 19:04:04 0|sipa|8393 (segwit + compact blocks) is a big blocker
106 2016-09-01 19:04:07 0|jeremyrubin|I'd like to mention that my Checkqueue Lockfree is passing tests, would like to hear what people need to see for it to merge (will be restructuing my commits later today.tomorrow)
107 2016-09-01 19:04:09 0|wumpus|#topic remaining 0.13.1 issues
108 2016-09-01 19:04:33 0|jeremyrubin|(also can't stick around meeting past 12:15 FYI)
109 2016-09-01 19:04:38 0|sipa|beyond that, we still need to choose a solution to the mempool dos issue around segwit
110 2016-09-01 19:04:51 0|instagibbs|yes cb and then dos issues are probably 2 biggest?
111 2016-09-01 19:04:58 0|sipa|this has been brought uo several times the past few meetings
112 2016-09-01 19:05:10 0|sdaftuar|sipa: do we think that the BIP for cb+segwit is basically in final substantive form? i saw there were some language discussions
113 2016-09-01 19:05:10 0|wumpus|#action #8393 (segwit + compact blocks) is a blocker, needs review testing and be merged
114 2016-09-01 19:05:36 0|sipa|sdaftuar: matt had more comments on the bip, but just on wording, not on semantics
115 2016-09-01 19:05:41 0|jtimon|sorry what was cb ?
116 2016-09-01 19:05:45 0|instagibbs|compact blocks
117 2016-09-01 19:05:46 0|sipa|compact blocks
118 2016-09-01 19:05:47 0|wumpus|jeremyrubin: great to hear it is passing the tests, we can talk about the topic later, but not if you have to leave that soon probably
119 2016-09-01 19:05:51 0|jtimon|compact blocks
120 2016-09-01 19:05:54 0|jtimon|sorry
121 2016-09-01 19:05:56 0|sdaftuar|sipa: ok, i intend to review/test 8393 soon
122 2016-09-01 19:06:00 0|sipa|thank you
123 2016-09-01 19:06:05 0|sipa|beyond that, the dos issue
124 2016-09-01 19:06:24 0|sipa|i'm not confortable with the previous suggestion of fully validating everything
125 2016-09-01 19:06:35 0|sipa|as a change for a minor point release
126 2016-09-01 19:06:46 0|jeremyrubin|wumpus: I'll try to swing back in the last few minutes of the meeting if possible?
127 2016-09-01 19:07:18 0|wumpus|it's a major change to how things have been done for the last few versions, that's for sure
128 2016-09-01 19:07:37 0|sdaftuar|so in a previous IRC meeting morcos had suggested mandatory feefilter + rejection cache only for non-witness tx, is that right?
129 2016-09-01 19:07:50 0|sipa|so that leaves us with not storing witness txn in the rejection cache, feefilter (possibly mandatory), and heuristics to detect invalid bloating before validation
130 2016-09-01 19:07:54 0|jonasschnelli|shouldn't we tag https://github.com/bitcoin/bitcoin/issues/8279 for 0.13.1?
131 2016-09-01 19:08:14 0|sipa|sdaftuar: yes, there have been other suggestions in other talks to
132 2016-09-01 19:09:02 0|luke-jr|I feel like I don't fully understand the issue, but couldn't the rejection cache use the [witness] hash instead of txid?
133 2016-09-01 19:09:20 0|sdaftuar|luke-jr: that's the approach i prefer, but it requires redoing tx relay to use wtxid, which is probably a big change
134 2016-09-01 19:10:03 0|sdaftuar|(and i don't think anyone has started work on it)
135 2016-09-01 19:10:08 0|sipa|yes, that's a bigger change, and there are complications
136 2016-09-01 19:10:13 0|sipa|like duplicating several indexes
137 2016-09-01 19:10:30 0|sipa|and always risking downloading twice, because old nodes may announce using the txid
138 2016-09-01 19:10:50 0|sipa|twice is better than $CONNECTIONS times, but still
139 2016-09-01 19:11:04 0|luke-jr|old nodes won't relay witness transactions at all.. (or if they do, they'll be incomplete)
140 2016-09-01 19:11:07 0|gmaxwell|On CB, I'm due to test the latest version-- I had tested the earlier version.. just been testing other things recently. Those things are now merged.
141 2016-09-01 19:11:42 0|sipa|i very much like the idea of mandatory feefilter
142 2016-09-01 19:11:48 0|CodeShark|me too
143 2016-09-01 19:11:57 0|sipa|moving the responsibiloty of resource cost to the sender
144 2016-09-01 19:12:19 0|CodeShark|it's the least hackish approach
145 2016-09-01 19:12:19 0|jtimon|can someone summarize the idea?
146 2016-09-01 19:12:21 0|sdaftuar|would we get rid of free tx relay at the same time, or not necessarily?
147 2016-09-01 19:12:45 0|gmaxwell|I think mandatory feefilter should be done, though I think it is not as much of a silverbullet as some thing. Feerate is only one of several resource related limitations that get imposed.
148 2016-09-01 19:12:47 0|sipa|but are we fine for 0.13.1 with only rejection cache for non-witness, and heuristics for detecting invalid witness bloating for the most common transaction types
149 2016-09-01 19:12:51 0|BlueMatt|sdaftuar: would be nice to get rid of that code
150 2016-09-01 19:13:01 0|sdaftuar|BlueMatt: i don't object in theory, but it's a sudden change for a point release
151 2016-09-01 19:13:03 0|gmaxwell|s/thing/think/
152 2016-09-01 19:13:08 0|CodeShark|what sort of heuristics?
153 2016-09-01 19:13:09 0|sipa|sdaftuar: agree
154 2016-09-01 19:13:11 0|BlueMatt|sdaftuar: sure, i wouldnt recommend it for that
155 2016-09-01 19:13:16 0|jtimon|just stop allowing free transactions?
156 2016-09-01 19:13:24 0|gmaxwell|sdaftuar: in practice it's not enabled in any case.
157 2016-09-01 19:13:28 0|jtimon|s/allowing/relaying/
158 2016-09-01 19:13:30 0|BlueMatt|jtimon: we effectively already do...
159 2016-09-01 19:13:35 0|instagibbs|jtimon, it's about particular feerate, not just free
160 2016-09-01 19:13:38 0|sipa|CodeShark: check whether the witness program's embedded scriot hash matches the hash of the witness script, for example
161 2016-09-01 19:13:39 0|gmaxwell|jtimon: they're already not relayed on nodes that are up for more than a few hours.
162 2016-09-01 19:13:49 0|sdaftuar|jtimon: because mempools are full
163 2016-09-01 19:13:55 0|jtimon|so what's the mandatory feefilter changing?
164 2016-09-01 19:13:57 0|instagibbs|sipa, yes that would be an easy fix, early on
165 2016-09-01 19:14:08 0|sipa|instagibbs: there is a PR for this
166 2016-09-01 19:14:19 0|gmaxwell|"mempoolminfee": 0.00001812
167 2016-09-01 19:14:19 0|instagibbs|is that jls? I can't remember
168 2016-09-01 19:14:24 0|jtimon|if it's too long to summarize that's fine
169 2016-09-01 19:14:40 0|sipa|instagibbs: yes, on a phonr, i can't seatch the pr number
170 2016-09-01 19:14:50 0|instagibbs|#8593
171 2016-09-01 19:15:01 0|sipa|jtimon: mandatory feefilter is that you can ban a node if it does not obey the feefilter you sent it
172 2016-09-01 19:15:03 0|sdaftuar|so mandatory fee filter would only apply to witness transactions?
173 2016-09-01 19:15:11 0|sdaftuar|er, NODE_WITNESS peers?
174 2016-09-01 19:15:18 0|sipa|sdaftuar: that's a possibility
175 2016-09-01 19:15:23 0|instagibbs|he moves full input validation up front in #8539
176 2016-09-01 19:15:30 0|sdaftuar|it doesn't make sense otherwise, we can't stop relaying transactions from older clients right
177 2016-09-01 19:15:33 0|sipa|instagibbs: oh, no, not that one
178 2016-09-01 19:15:43 0|instagibbs|would be abusive to disconnect older nodes imo
179 2016-09-01 19:15:52 0|gmaxwell|sdaftuar: It has to be negoiated somehow, node witness would be one way.. though a bit disruptive on testnet.
180 2016-09-01 19:16:04 0|gmaxwell|instagibbs: no one is going to do that.
181 2016-09-01 19:16:04 0|sipa|instagibbs: full validation has some kinks to work out; something for 0.14 imho
182 2016-09-01 19:16:11 0|luke-jr|mandatory feefilter seems liable to cause issues with 1) diverging fee policies; 2) ancestor feerate (CPFP) relay stuff
183 2016-09-01 19:16:14 0|jtimon|sipa: I see, thanks, like an additional filter per peer
184 2016-09-01 19:16:16 0|BlueMatt|sdaftuar: well you could gate it on something else (protocol version/message/whatever), but I'm not against fucking up testnet to do it for NODE_WITNESS
185 2016-09-01 19:16:32 0|instagibbs|sipa, I don't see the PR you are talking about then
186 2016-09-01 19:17:12 0|sdaftuar|conceptually though: the idea is that we only download witness transactions from peers with whom we've negotiated mandatory feefilter semantics?
187 2016-09-01 19:17:25 0|jonasschnelli|instagibbs: i think there is no PR right now for that proposal
188 2016-09-01 19:17:31 0|gmaxwell|sdaftuar: that would be fine too.
189 2016-09-01 19:17:37 0|petertodd|sdaftuar: yes
190 2016-09-01 19:17:38 0|sipa|instagibbs: 8499
191 2016-09-01 19:17:47 0|sipa|yes there is
192 2016-09-01 19:17:50 0|instagibbs|oh that one
193 2016-09-01 19:18:02 0|jonasschnelli|Indeed: https://github.com/bitcoin/bitcoin/pull/8499/files
194 2016-09-01 19:18:21 0|sipa|so, i would like to request review on 8499 and 8525
195 2016-09-01 19:18:35 0|sipa|neither of those is a policy change
196 2016-09-01 19:18:36 0|instagibbs|wumpus, can you tag that PR please #8499
197 2016-09-01 19:18:47 0|sdaftuar|sipa: will do
198 2016-09-01 19:19:07 0|wumpus|sure
199 2016-09-01 19:19:15 0|instagibbs|I will review 8499
200 2016-09-01 19:19:39 0|sipa|and untag 8593
201 2016-09-01 19:20:16 0|wumpus|done
202 2016-09-01 19:20:39 0|sipa|so the only remaining thing is enforcing feefilter stronger as sdaftuar suggests only for witness peers
203 2016-09-01 19:20:53 0|BlueMatt|I'd ACK that
204 2016-09-01 19:20:55 0|gmaxwell|luke-jr: CPFP relay is already inhibited to the maximum extent that it could be by feefilter in the current non-mandatory form. Improving that will require some kind of package relay. Mandatory fee filter won't make it worse.
205 2016-09-01 19:20:58 0|sipa|does that sound reasonable for 0.13.1?
206 2016-09-01 19:21:08 0|gmaxwell|FWIW, mandatory fee filter would couple well with 8525, rejection cache mostly exists due to a lack of feefilter.
207 2016-09-01 19:21:23 0|sipa|besides fixing known bugs, obviously
208 2016-09-01 19:21:24 0|petertodd|gmaxwell: yup
209 2016-09-01 19:21:29 0|sdaftuar|sipa: yes
210 2016-09-01 19:21:37 0|petertodd|sipa: sure
211 2016-09-01 19:21:45 0|sipa|great
212 2016-09-01 19:21:56 0|sdaftuar|you started a PR that did that, yes?
213 2016-09-01 19:22:05 0|sipa|sdaftuar: not yet, i will
214 2016-09-01 19:22:12 0|sdaftuar|ok
215 2016-09-01 19:22:20 0|gmaxwell|I'd like feelers (and my addrman insert fix) to get backported. (I'm happy to do the 'backport') I think it will be important to have feelers to compensate for any loss of network density for witness enabled nodes.
216 2016-09-01 19:22:30 0|instagibbs|gmaxwell, ack
217 2016-09-01 19:22:35 0|sdaftuar|sounds reasonable
218 2016-09-01 19:22:51 0|sipa|ack on backporting feelers
219 2016-09-01 19:22:59 0|sdaftuar|curious to know if anyone has experimented with that on testnet and has any results?
220 2016-09-01 19:23:17 0|sipa|i think it will take a while for network wide effects of feelers to become visible
221 2016-09-01 19:23:41 0|sdaftuar|oh maybe i misunderstood, i thought your own node would benefit from that by itself?
222 2016-09-01 19:23:43 0|sipa|and testnet is not particularly in a sane p2p state
223 2016-09-01 19:23:50 0|CodeShark|if a node with low feefilter connects to peers that all have a higher feefilter the higher filters would dominate. is that a problem?
224 2016-09-01 19:23:54 0|gmaxwell|we had repeated minor problems with isoloation on testnet early on when we deployed segwit there. I expect less of those on mainnet, and more proactive efforts to avoid them (eg spinning up nodes), but belt and suspenders is good.
225 2016-09-01 19:24:01 0|sipa|right, but indirectly the results of addr messages will get better too from feelers
226 2016-09-01 19:24:13 0|sdaftuar|sipa: ah, yes
227 2016-09-01 19:24:13 0|sipa|if the overall quality of addrmans improves
228 2016-09-01 19:24:15 0|gmaxwell|CodeShark: the filters are one way. "Here is what you can send me"
229 2016-09-01 19:24:21 0|instagibbs|https://github.com/bitcoin/bitcoin/pull/8282
230 2016-09-01 19:25:00 0|CodeShark|gmaxwell: sure...but your peers would filter stuff you wanted to see
231 2016-09-01 19:25:11 0|sipa|next topic?
232 2016-09-01 19:25:12 0|gmaxwell|CodeShark: has nothing to do with fee filter.
233 2016-09-01 19:25:14 0|sipa|i'm in a mildly unconfortable position here (irl meetup in milan, where BlueMatt spoke)
234 2016-09-01 19:25:21 0|gmaxwell|CodeShark: peers already filter anything they won't relay.
235 2016-09-01 19:26:04 0|BlueMatt|sipa: you can irc from my laptop if you prefer
236 2016-09-01 19:26:07 0|jtimon|its kind of "please, don't bother sending me things below X or I'll disconnect from you"
237 2016-09-01 19:26:39 0|luke-jr|gmaxwell: no particular p2p protocol changes would be needed right now afaik (would just need to send the children first, and then teach the receiver to use the feerate including orphan txs)
238 2016-09-01 19:27:41 0|luke-jr|if we're not careful, mandatory feefilter could in practice make fee policy code as sensitive as consensus code
239 2016-09-01 19:27:58 0|petertodd|luke-jr: how?
240 2016-09-01 19:28:08 0|jtimon|luke-jr: I'm not following, how does the peer know your rate X
241 2016-09-01 19:28:11 0|jtimon|?
242 2016-09-01 19:28:43 0|luke-jr|petertodd: if you change your fee policy code, you could be partitioned off from the other peers, no?
243 2016-09-01 19:28:49 0|instagibbs|suggested topic: "Extra" softforks low_s and nulldummy
244 2016-09-01 19:28:49 0|sdaftuar|luke-jr: no
245 2016-09-01 19:28:59 0|luke-jr|what am I missing?
246 2016-09-01 19:29:07 0|sdaftuar|only if you lie about it to your peers
247 2016-09-01 19:29:14 0|sipa|instagibbs: ack topic
248 2016-09-01 19:29:26 0|jtimon|I believe low_s was discussed already?
249 2016-09-01 19:29:30 0|BlueMatt|am I missing something? isnt the feefilter potentially rather deanonymizing? we should probably round/randomize the amount somewhat, no? (or do we, I probably wouldnt know)
250 2016-09-01 19:29:34 0|gmaxwell|luke-jr: e.g. later we can add a package fee filter and nodes would signal a package fee filter of X, and a fee filter of 0.
251 2016-09-01 19:29:42 0|sipa|i believe it needs rediscussing (low_s)
252 2016-09-01 19:29:46 0|jtimon|BlueMatt: good point
253 2016-09-01 19:29:48 0|instagibbs|BlueMatt, that's a good point
254 2016-09-01 19:29:48 0|sdaftuar|BlueMatt: we do, but worht looking at again
255 2016-09-01 19:29:50 0|gmaxwell|BlueMatt: we do.
256 2016-09-01 19:30:09 0|BlueMatt|ok, nvm, ignore my ramblings
257 2016-09-01 19:30:43 0|jtimon|BlueMatt: please keep rumbling out loud, I hadn't even thought about it, now seems obvious
258 2016-09-01 19:30:47 0|gmaxwell|BlueMatt: but also, we really can make no guarentees that a single node with multiple interfaces can't be distinguished as the same node. There are several ways to do this. (obviously I'm happy to reduce them, but it's not a property we currently provide)
259 2016-09-01 19:30:56 0|sipa|(can we move on to next topic?)
260 2016-09-01 19:31:01 0|gmaxwell|jtimon: it was specfically addressed in the original feefilter PR.
261 2016-09-01 19:31:02 0|BlueMatt|gmaxwell: indeed, shame we dont
262 2016-09-01 19:31:04 0|BlueMatt|also, what sipa said
263 2016-09-01 19:31:04 0|jtimon|ack next topic
264 2016-09-01 19:31:17 0|sipa|wumpus: ring topic bell? :)
265 2016-09-01 19:31:26 0|gmaxwell|we put wumpus to sleep.
266 2016-09-01 19:31:41 0|paveljanik|;-)
267 2016-09-01 19:31:48 0|petertodd|gmaxwell: ring wumpus bell?
268 2016-09-01 19:31:52 0|sipa|ok, i'll go on anyway
269 2016-09-01 19:32:03 0|jtimon|go sipa go
270 2016-09-01 19:32:03 0|paveljanik|I think that wumpus never sleeps
271 2016-09-01 19:32:07 0|sipa|so, the nulldummy and low_s softfork proposals
272 2016-09-01 19:32:31 0|btcdrak|#topic nulldummy and low_s softfork proposals
273 2016-09-01 19:32:36 0|wumpus|#topic nulldummy and low_s softfork proposals
274 2016-09-01 19:32:40 0|sipa|it was discovered by jl that low_ has a really strange implementation issue leaked into the semantics
275 2016-09-01 19:32:44 0|jtimon|I don't remember any objections on low_s last time we discussed it?
276 2016-09-01 19:32:57 0|sipa|which is not an issue for standardness, but for consensus i think we prefer clean sema tics
277 2016-09-01 19:33:02 0|instagibbs|jtimon, https://github.com/bitcoin/bitcoin/pull/8533#issuecomment-243973512
278 2016-09-01 19:33:17 0|gmaxwell|sipa: are clean semantics even possible?
279 2016-09-01 19:33:25 0|jtimon|instagibbs: thanks
280 2016-09-01 19:33:38 0|sipa|gmaxwell: yes, if we also do the only-empty-signature-in-invalid-checksig sf
281 2016-09-01 19:33:53 0|gmaxwell|sipa: oh indeed okay, yes that would be clean
282 2016-09-01 19:33:58 0|sipa|which is why i'd propose that we do low_s later, together with that empty sigs rule
283 2016-09-01 19:34:07 0|sipa|and only bundle nulldummy with 0.13.1
284 2016-09-01 19:34:20 0|sipa|s/0.13.1/segwit/
285 2016-09-01 19:34:23 0|gmaxwell|The clenlyness issue is that lowS test fails for some encodings of invalid encodings.
286 2016-09-01 19:34:37 0|gmaxwell|er encodings of invalid signatures.
287 2016-09-01 19:34:45 0|sipa|yes, some illegal signatures are exempt from low_s
288 2016-09-01 19:34:50 0|BlueMatt|would be a shame, but seems like a reasonable approach...has anyone checked for the #/frequency of invalid sigs in the chain?
289 2016-09-01 19:34:59 0|BlueMatt|that are non-0-length, at least
290 2016-09-01 19:35:04 0|sipa|such signatures are NOT valid
291 2016-09-01 19:35:15 0|BlueMatt|but can bt OP_NOT'd, no?
292 2016-09-01 19:35:23 0|sipa|yes, but nobody sane does that
293 2016-09-01 19:35:30 0|sdaftuar|has NULLDUMMY been discussed on the mailing list yet? i see one mention of it in passing about the LOW_S BIP.
294 2016-09-01 19:35:33 0|BlueMatt|sure, but /has/ anyone ever done so?
295 2016-09-01 19:35:39 0|sipa|"yes, this output can be spent UNLESS BlueMatt likes it"
296 2016-09-01 19:35:46 0|jtimon|no objections on delaying low_s enforcement
297 2016-09-01 19:35:54 0|gmaxwell|BlueMatt: with the exception of intentional insanity, any invalid signature could be malleated to a zero length one, in any case. I had looked previously and seen no ongoing checksig-not activity, obvious.
298 2016-09-01 19:35:58 0|BlueMatt|I might suggest that it is not crazy to propose doing it in 0.13.1 if it has never happened
299 2016-09-01 19:36:29 0|BlueMatt|indeed, I'm asking with the assumption that someone might have done intentional insantiy as a test-case or so
300 2016-09-01 19:36:35 0|sipa|BlueMatt: the only opractical difference is that the BIP to specify LOW_S would contain a rule that is pointless and complicating
301 2016-09-01 19:36:40 0|sipa|i think we should aboid that
302 2016-09-01 19:36:43 0|BlueMatt|(wait, it is non-stad to do non-0-length, correct?)
303 2016-09-01 19:36:47 0|gmaxwell|BlueMatt: a checksig not has been done at least once in the chain history.
304 2016-09-01 19:36:54 0|jtimon|BlueMatt: good question, petertodd has anybody done that? :p
305 2016-09-01 19:36:57 0|gmaxwell|By someone triggering a fork off of bitcoin ruby.
306 2016-09-01 19:37:04 0|sipa|petertodd: have you done that?
307 2016-09-01 19:37:11 0|BlueMatt|gmaxwell: sure, but with 0-length, or with a non-emtpy sig
308 2016-09-01 19:37:20 0|gmaxwell|yea, don't recall, we could check.
309 2016-09-01 19:37:35 0|petertodd|sipa: me personally, probably not - I'm a fine arts grad :P
310 2016-09-01 19:37:38 0|gmaxwell|But I don't think the test here should be none at all. If there was an obvious test someplace in the past, who cares?
311 2016-09-01 19:37:46 0|BlueMatt|(this would be immaterial if it is not already nonstd...which I do not recall)
312 2016-09-01 19:37:48 0|sipa|jl also pointed out that the benefit of low_s is lower, as it can only be used.for mutating, but not for bloating
313 2016-09-01 19:38:26 0|BlueMatt|I was informed that non-0-length invalid sigs is not nonstd
314 2016-09-01 19:38:31 0|BlueMatt|I would propose we do so in 0.13.1
315 2016-09-01 19:38:45 0|gmaxwell|It is non-standard for segwit. (unless I am on drugs.)
316 2016-09-01 19:38:56 0|sipa|gmaxwell: you're on drugs
317 2016-09-01 19:38:58 0|gmaxwell|I am on drugs then. Okay. Good to know.
318 2016-09-01 19:39:00 0|jtimon|are we still talking low_S ?
319 2016-09-01 19:39:03 0|sipa|yes
320 2016-09-01 19:39:04 0|BlueMatt|I was assuming it was nonstd, and thus thought it might be reasonable for a softfork, but withdraw my insanity
321 2016-09-01 19:39:08 0|gmaxwell|wlel thats insane, we should fix that at least.
322 2016-09-01 19:39:15 0|BlueMatt|gmaxwell: ack
323 2016-09-01 19:39:21 0|gmaxwell|yes, it needs to be non-standard first. pronto.
324 2016-09-01 19:39:24 0|BlueMatt|0.13.1 should make invalid sigs which are non-0-length nonstd
325 2016-09-01 19:39:25 0|sipa|agree
326 2016-09-01 19:39:30 0|sdaftuar|agree
327 2016-09-01 19:39:35 0|wumpus|yes, absolutely
328 2016-09-01 19:39:43 0|CodeShark|+1
329 2016-09-01 19:39:51 0|sipa|great
330 2016-09-01 19:40:02 0|jtimon|this needs a bip, no?
331 2016-09-01 19:40:16 0|sipa|jtimon: yes, but we just suggest it as nonstd now
332 2016-09-01 19:40:22 0|jtimon|sure
333 2016-09-01 19:40:26 0|sipa|(which certainly needs ML discussion too)
334 2016-09-01 19:40:31 0|gmaxwell|there is copy for null dummy as part of the old malleability bip, no?
335 2016-09-01 19:40:51 0|gmaxwell|nulldummy and fixed invalid?
336 2016-09-01 19:40:55 0|sipa|null dummy is about the ignored arg for CMS
337 2016-09-01 19:41:02 0|gmaxwell|thus my follow up.
338 2016-09-01 19:41:12 0|sipa|empty invalid was not part of bip62
339 2016-09-01 19:41:20 0|gmaxwell|okay.
340 2016-09-01 19:41:41 0|sipa|because it wasn't needed fir standard txn at the time
341 2016-09-01 19:41:54 0|sipa|ok, next topic?
342 2016-09-01 19:42:00 0|kanzure|jeremyrubin's stuff see above
343 2016-09-01 19:42:14 0|kanzure|12:04 < jeremyrubin> I'd like to mention that my Checkqueue Lockfree is passing tests, would like to hear what people need to see for it to merge (will be restructuing my commits later today.tomorrow)
344 2016-09-01 19:42:23 0|gmaxwell|jeremyrubin is busy making validation great again. I'm super excited about this.
345 2016-09-01 19:42:29 0|sdaftuar|sorry before we move topics -- we should repropose NULLDUMMY by itself on the mailing list, yes?
346 2016-09-01 19:42:37 0|sipa|sdaftuar: ack
347 2016-09-01 19:42:37 0|wumpus|if he's here, at least
348 2016-09-01 19:42:58 0|BlueMatt|gmaxwell: ACK
349 2016-09-01 19:42:59 0|petertodd|so, one mitigation to this malleability stuff I think we should also consider doing, is having transaction replacement trigger if the tx has a higher fee rate than the old tx (over a certain ratio)
350 2016-09-01 19:43:03 0|instagibbs|link to jeremyrubin's stuff?
351 2016-09-01 19:43:26 0|sipa|petertodd: that would be a side effect of switching to wtzid based indexing, actually
352 2016-09-01 19:43:31 0|kanzure|instagibbs: https://github.com/bitcoin/bitcoin/pull/8464
353 2016-09-01 19:43:31 0|sdaftuar|https://github.com/bitcoin/bitcoin/pull/8464
354 2016-09-01 19:44:09 0|jonasschnelli|It's a very invasive change...
355 2016-09-01 19:44:18 0|BlueMatt|jonasschnelli: its worth it, by a lot
356 2016-09-01 19:44:18 0|jonasschnelli|though, I like the concept
357 2016-09-01 19:44:27 0|btcdrak|sdaftuar there is a NULLDUMMY only PR as well https://github.com/bitcoin/bitcoin/pull/8636
358 2016-09-01 19:44:29 0|petertodd|sipa: not quite - I'm proposing we re-broadcast such replacements to our peers
359 2016-09-01 19:45:06 0|BlueMatt|(for those interested, when it was first posted I went and reimplemented it as lockfree but with a single atomic which was contested between threads a decent amount.....my reimplementation might not have been ideal, but was only marginally better than master...jeremy's work is something like 10-20%)
360 2016-09-01 19:45:16 0|petertodd|sipa: need to pick a reasonable ratio, geometric series, so something like 75% would be absolute worst case a 4x bandwidth increase, while allowing no more than 25% bloat by a malleability attacker
361 2016-09-01 19:45:31 0|sdaftuar|petertodd: you're suggesting different replacement semantics than we have under BIP 125?
362 2016-09-01 19:45:32 0|sipa|ah, i see
363 2016-09-01 19:46:07 0|petertodd|sdaftuar: yeah, obvious one would be to just do replacements if vout #1 == vout #2 regardless of BIP125, which handles malleability just fine
364 2016-09-01 19:47:07 0|petertodd|sdaftuar: mainly proposing this as a belt and suspenders to limit the damage of malleability attacks
365 2016-09-01 19:47:13 0|gmaxwell|petertodd: may just be better for mempool reconcillation to handle that.
366 2016-09-01 19:47:28 0|petertodd|gmaxwell: sure - when's that coming? :)
367 2016-09-01 19:47:54 0|gmaxwell|petertodd: interested in helping define it? :P
368 2016-09-01 19:48:09 0|sipa|i believe we're drifting off topic
369 2016-09-01 19:48:17 0|gmaxwell|Who was phone?
370 2016-09-01 19:48:26 0|petertodd|gmaxwell: heh, seems like I should do a pull-req of this ratio thing for now - that's just a few lines of code
371 2016-09-01 19:48:36 0|gmaxwell|fair
372 2016-09-01 19:49:01 0|sipa|other topics?
373 2016-09-01 19:49:11 0|gmaxwell|petertodd: if you do it right, it will interact well with mempool batching, and the batching will eliminate a lot of the bandwidth overhead.
374 2016-09-01 19:49:28 0|wumpus|yes, any other topic proposals?
375 2016-09-01 19:49:32 0|gmaxwell|s/mempool batching/relay batching/
376 2016-09-01 19:49:45 0|petertodd|gmaxwell: oh, batching? I'm unfamilar with that idea
377 2016-09-01 19:50:02 0|BlueMatt|10 min bell
378 2016-09-01 19:50:05 0|gmaxwell|petertodd: relay behavior changed in 0.13, I think you're aware.
379 2016-09-01 19:50:13 0|gmaxwell|petertodd: will talk after meeting.
380 2016-09-01 19:50:17 0|petertodd|gmaxwell: ack
381 2016-09-01 19:50:42 0|sdaftuar|quick travis discussion?
382 2016-09-01 19:50:47 0|wumpus|seems that there are no other meeting topics proposals, so we can close
383 2016-09-01 19:51:06 0|wumpus|sure
384 2016-09-01 19:51:07 0|GitHub133|[13bitcoin] 15jonasschnelli opened pull request #8643: [0.13 backport] Added feeler connections increasing good addrs in the tried table. (060.13...062016/08/feeler_013) 02https://github.com/bitcoin/bitcoin/pull/8643
385 2016-09-01 19:51:09 0|gmaxwell|Non-discussion: 0.13 deployment seems to be trouble free <knock on wood>, congrats everyone.
386 2016-09-01 19:51:10 0|wumpus|#topic travis discussion
387 2016-09-01 19:51:14 0|sdaftuar|i've been away for the last couple weeks, can anyone summarize the state of our testsuite these days?
388 2016-09-01 19:51:19 0|jeremyrubin|Yes
389 2016-09-01 19:51:24 0|sdaftuar|i feel like i've seen lots of people complaining
390 2016-09-01 19:51:32 0|wumpus|yes, congrats everyone on the 0.13.0 release
391 2016-09-01 19:51:35 0|jeremyrubin|Basically theres some failing rpctests
392 2016-09-01 19:51:40 0|sdaftuar|and my own local runs are failing a lot too, but i havent figured out why yet
393 2016-09-01 19:51:45 0|jeremyrubin|and the make check code is slow
394 2016-09-01 19:51:46 0|gmaxwell|what is this backupwallet test thing that keeps failing?
395 2016-09-01 19:51:48 0|wumpus|there are randomly failing RPC tests, and also some random timeouts
396 2016-09-01 19:51:51 0|cfields|yes, there are a few racy conditions
397 2016-09-01 19:51:56 0|sipa|segwit.py also fails sometimes
398 2016-09-01 19:52:09 0|jeremyrubin|I've also seen an abandonconflict failure once
399 2016-09-01 19:52:31 0|cfields|i believe i tracked down the segwit issue, which may be the root cause of some other issues
400 2016-09-01 19:52:37 0|sdaftuar|oh! good to know
401 2016-09-01 19:52:39 0|gmaxwell|I saw some indication before that this was actually misbehavior on the part of the travis infra, that it was occassionally running too slow and timing out.
402 2016-09-01 19:52:42 0|sipa|cfields: elobarate?
403 2016-09-01 19:52:43 0|cfields|(if that's the same as the one that was worsened by the timing change)
404 2016-09-01 19:53:12 0|cfields|sipa: need to check if it's the same thing and gather my notes, take it up after meeting?
405 2016-09-01 19:53:13 0|jonasschnelli|can we not just move the segwith test to the end/last item?
406 2016-09-01 19:53:20 0|sipa|cfields: ok
407 2016-09-01 19:53:26 0|sdaftuar|sounds good
408 2016-09-01 19:53:27 0|wumpus|there's an issue open about txn_doublespend and segwit.py, it was worse and made batter by reverting a timeout change, but not solved (as MarcoFalke says)
409 2016-09-01 19:53:30 0|jonasschnelli|s/segwith/segwit
410 2016-09-01 19:53:37 0|wumpus|the thing is, the tests never fail locally at least here
411 2016-09-01 19:53:52 0|wumpus|so yes I also suspect the travis infrastructure for some failiures
412 2016-09-01 19:54:09 0|jeremyrubin|There's also an open issue with travis race conditions internally
413 2016-09-01 19:54:10 0|wumpus|anyhow the issue is https://github.com/bitcoin/bitcoin/issues/8532
414 2016-09-01 19:54:11 0|jtimon|I suspect some tests fail in a non-reproducible way, I don't have local travis, but when the tests doesn't fail locally I just change the last commit id and force push, it works most times
415 2016-09-01 19:54:13 0|sipa|sdaftuar says he sometimes sees the failures locally?
416 2016-09-01 19:54:26 0|jeremyrubin|e.g. if you push a commit before the last push finished testing
417 2016-09-01 19:54:26 0|sdaftuar|sipa: yes, with an error condition i don't understand at all, let me find it
418 2016-09-01 19:54:35 0|cfields|ok, same one then. the issue there is that the node heights are in sync, but the wallet hasn't necessarily updated with their txs. So sync_all() followed by a balance check is racy.
419 2016-09-01 19:55:04 0|wumpus|so we need a better sync call
420 2016-09-01 19:55:16 0|sipa|cfields: so switch to a while loop until balances are an equal/expected value, with a certain timeout?
421 2016-09-01 19:55:17 0|wumpus|a kind of fence
422 2016-09-01 19:55:47 0|wumpus|sipa: that would imply updating all the balance checks to loops
423 2016-09-01 19:55:54 0|gmaxwell|run a ping and check the ping time on all the node.s
424 2016-09-01 19:56:09 0|jonasschnelli|gmaxwell: from the RPC API?
425 2016-09-01 19:56:22 0|sipa|getpeerinfo, i oresume
426 2016-09-01 19:56:26 0|gmaxwell|yes, the ping will act as a barrier in the current p2p protocol, though perhaps cfields is about to kill me
427 2016-09-01 19:56:42 0|sipa|i hope cfields isn't changing that barrier effect
428 2016-09-01 19:56:46 0|BlueMatt|gmaxwell: iirc breaking that breaks things
429 2016-09-01 19:56:47 0|wumpus|yes, ping will work for that, but it's unclear how to use that in RPC tests
430 2016-09-01 19:56:51 0|cfields|i was thinking about some rpc call like "wait_for_[height|block]" that would block until reached and finished processing everywhere. then we wouldn't need to loop
431 2016-09-01 19:56:57 0|gmaxwell|we had that discussion with cfields before, which is why I mention it. :P
432 2016-09-01 19:56:58 0|cfields|i suppose that just introduces its own issues, though
433 2016-09-01 19:57:03 0|cfields|heh
434 2016-09-01 19:57:14 0|wumpus|no, the barrier effect certainly shouldn't be removed, unless a better mechanism is introduced
435 2016-09-01 19:57:35 0|BlueMatt|(I'd actually kinda like a test which relies on pings being barriers, since some software assumes that for the p2p network, and, in fact, for some things you have to)
436 2016-09-01 19:57:40 0|gmaxwell|(I actually like ping being head of line blocked for the reason that it lets you measure processing delays of your remote peers)
437 2016-09-01 19:57:40 0|wumpus|and that's a huge protocol change, so probably not worth it right now
438 2016-09-01 19:57:55 0|sipa|yes, that's out of scope
439 2016-09-01 19:57:58 0|gmaxwell|sorry, I started a tangent.
440 2016-09-01 19:58:01 0|cfields|well, as a nasty short-term fix, we can just throw some sleeps in after sync. that should at least shut travis up while we work on a fix
441 2016-09-01 19:58:09 0|sipa|for a "get the damn unit tests to work again", at least
442 2016-09-01 19:58:10 0|gmaxwell|in any case, maybe ping can be used to sync for these tests.
443 2016-09-01 19:58:15 0|BlueMatt|2 min bell
444 2016-09-01 19:58:31 0|wumpus|sleeps are pretty terrible, it's easy to get those wrong and travis has a very unpredictable load pattern
445 2016-09-01 19:58:32 0|gmaxwell|sleeps for now sound fine to me. We could all use more sleep.
446 2016-09-01 19:58:37 0|wumpus|hah
447 2016-09-01 19:58:49 0|sdaftuar|i was seeing this a bunch, but i haven't looked at the latest fails yet to confirm if they're the same: https://0bin.net/paste/nvDO+4yPU0w5332j#Y4BWnYpKcTTIW6ePHglWpEwpE4XtfA+I-NP2M3ObMkp
448 2016-09-01 19:58:54 0|jonasschnelli|sleeps will just kick the problem a bit further down...
449 2016-09-01 19:58:55 0|BlueMatt|cfields: if we commit to reverting it within X days, ACK
450 2016-09-01 19:58:56 0|wumpus|but indeed this started when the timeouts were lowered all over the place
451 2016-09-01 19:58:58 0|gmaxwell|tests randomly faily though is ... not good.
452 2016-09-01 19:59:07 0|jtimon|cfields: go with the sleeps, better solutions later
453 2016-09-01 19:59:08 0|gmaxwell|jonasschnelli: I'm not suggesting delting fixing things at all.
454 2016-09-01 19:59:08 0|jonasschnelli|and a sleep in sync_all will have a performance impact on the test-duration-tim
455 2016-09-01 19:59:28 0|jonasschnelli|though, ack on the sleep for now
456 2016-09-01 19:59:36 0|sipa|sgtm
457 2016-09-01 19:59:38 0|wumpus|right
458 2016-09-01 19:59:39 0|cfields|BlueMatt: +1 for that. Sleeps are ripped out at next meeting if they're still there, and the tests fails again
459 2016-09-01 19:59:48 0|gmaxwell|we must not habituate ourselves to test failures being orinary, already that we weren't responding to failures as a potential emergency indicates we're in a bad state.
460 2016-09-01 19:59:49 0|BlueMatt|ACK
461 2016-09-01 19:59:55 0|sdaftuar|gmaxwell: yep
462 2016-09-01 20:00:04 0|BlueMatt|ACK to gmaxwell and cfields
463 2016-09-01 20:00:09 0|BlueMatt|also, dingdingding
464 2016-09-01 20:00:09 0|sipa|POING
465 2016-09-01 20:00:10 0|lightningbot|Meeting ended Thu Sep 1 20:00:10 2016 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
466 2016-09-01 20:00:10 0|wumpus|#endmeeting
467 2016-09-01 20:00:11 0|lightningbot|Log: http://www.erisian.com.au/meetbot/bitcoin-core-dev/2016/bitcoin-core-dev.2016-09-01-19.01.log.html
468 2016-09-01 20:00:11 0|lightningbot|Minutes: http://www.erisian.com.au/meetbot/bitcoin-core-dev/2016/bitcoin-core-dev.2016-09-01-19.01.html
469 2016-09-01 20:00:11 0|lightningbot|Minutes (text): http://www.erisian.com.au/meetbot/bitcoin-core-dev/2016/bitcoin-core-dev.2016-09-01-19.01.txt
470 2016-09-01 20:00:25 0|wumpus|gmaxwell: yes it's not a good thing that travis failures are a normal thing now
471 2016-09-01 20:00:38 0|wumpus|I tend to rely on local tests more and more
472 2016-09-01 20:00:59 0|wumpus|if that continues, travis just becomes a distraction
473 2016-09-01 20:01:05 0|sipa|;;tslb
474 2016-09-01 20:01:07 0|gribble|Time since last block: 31 minutes and 15 seconds
475 2016-09-01 20:01:19 0|cfields|an alternative is to avoid giving up cs_main while the wallet syncs, but i suppose we shouldn't neuter real usage for the sake of tests :p
476 2016-09-01 20:01:45 0|wumpus|cfields: yes, and adding a flag to do that for the tests kind of defeats the purpose of tests too
477 2016-09-01 20:01:55 0|wumpus|let's try to solve it properly
478 2016-09-01 20:02:11 0|petertodd|gmaxwell: I'm aware that we changed relay behavior; forget the exact details there
479 2016-09-01 20:02:41 0|cfields|wumpus: sure, that wasn't a real suggestion
480 2016-09-01 20:02:49 0|gmaxwell|petertodd: regarding batching, 0.13 changed relay behavior so every peer has a possion timer (5 seconds expected or 2 seconds for outbound), no relaying happens on a peer until the timer hits again. when relaying happens, the transactions are checked to see if they're still in mempool and still passing fee filter.. and relayed in topology+feerate sorted order
481 2016-09-01 20:03:07 0|jeremyrubin|(a "stopgap" measure till better things are worked out could be to run the rpc tests 3 times and take a best of three; that way we at least get rpc_tests not blocking probably working code)
482 2016-09-01 20:03:14 0|gmaxwell|petertodd: so I'm pointing out that if the replacement arrives soon, and the original is removed from the mempool, it will never get offered to most peers.
483 2016-09-01 20:03:33 0|wumpus|cfields: adding a wait RPC specific to testing to wait for completion of some ongoing things would be fine, though
484 2016-09-01 20:04:00 0|jeremyrubin|(that would at least help us catalouge a definitely non-deterministic failure)
485 2016-09-01 20:04:18 0|gmaxwell|jeremyrubin: I don't think thats better than increasing the sync sleeping... it would conceal real issue of kinds that we've encountered before, and it would make travis much slower. :)
486 2016-09-01 20:04:19 0|petertodd|gmaxwell: right, so in the malleability case, if we get a 1-byte-less transaction, but haven't sent out the previous one yet to many of our peers, it'd be perfectly reasonable to replace, for instance
487 2016-09-01 20:04:21 0|cfields|wumpus: i've often thought that would be very helpful. Not sure how to avoid long timeouts when things really do fail, though
488 2016-09-01 20:04:40 0|jtimon|agreed, if we get used to "that's probably a travis thing, just push again" then it becomes kind of worthless, when it fails you should be able to get the same error at home
489 2016-09-01 20:05:07 0|wumpus|cfields: well we already have plenty of timeouts, can't become that much worse, most important is that everything is logged so that it can be troublehsooted
490 2016-09-01 20:05:11 0|cfields|wumpus: so maybe not wait-for-completion, but wait-for-a-while?
491 2016-09-01 20:05:11 0|gmaxwell|petertodd: right. in general I think the replacement should always happen in the mempool, and if we relay or not is a seperate decision.
492 2016-09-01 20:05:32 0|jeremyrubin|gmaxwell: you can still have it fail; but at least being able to have an estimate of if it is deterministic fail or spurrious would be useful
493 2016-09-01 20:05:33 0|gmaxwell|petertodd: and for CB we should have a rejection/eviction cache.
494 2016-09-01 20:05:39 0|cfields|wumpus: fair point
495 2016-09-01 20:05:51 0|petertodd|gmaxwell: CB? compact blocks?
496 2016-09-01 20:05:54 0|sdaftuar|gmaxwell: why do we need an eviction cache, if we're using feefilter?
497 2016-09-01 20:05:55 0|gmaxwell|jeremyrubin: okay, potentially useful. Though the time is a concern.
498 2016-09-01 20:05:59 0|sdaftuar|oh
499 2016-09-01 20:06:01 0|sdaftuar|i misunderstood
500 2016-09-01 20:06:26 0|jeremyrubin|gmaxwell: can have further runs only happen if the last one fails
501 2016-09-01 20:06:44 0|jeremyrubin|gmaxwell: (up to a bound)
502 2016-09-01 20:06:46 0|cfields|wumpus: wait_for_block(height, hash) seems pretty obvious. If either one is hit without matching, it returns failure. That at least eliminates some potential timeouts
503 2016-09-01 20:06:46 0|gmaxwell|jeremyrubin: too bad there is probably no way to report the failure right away and keep running. :P
504 2016-09-01 20:06:54 0|petertodd|gmaxwell: this could solve this SIGHASH_ANYONECANPAY issue w/o having to put in specific support for SIGHASH_ANYONECANPAY
505 2016-09-01 20:07:48 0|wumpus|cfields: what about a 'wait for a change in either height or hash'?
506 2016-09-01 20:07:57 0|wumpus|cfields: the client can then check the details
507 2016-09-01 20:08:03 0|jeremyrubin|gmaxwell: after_failure hook? Not sure when those get run w.r.t. ui updates :)
508 2016-09-01 20:08:06 0|cfields|jeremyrubin: seems to me that behavior belongs in the test-script, so that it can be done locally as well
509 2016-09-01 20:08:30 0|jeremyrubin|cfields: fair.
510 2016-09-01 20:08:42 0|gmaxwell|petertodd: my current mental model for how all this stuff should work is that you should make your mempool the best, and then have a bandwidth limited process that at every timestep uses the available bandwidth in the best way you can to bring your peers closer to your idea of the best. And for the purpose of CB you should keep around recent second bests for a little while.
511 2016-09-01 20:08:54 0|cfields|wumpus: makes sense. and in that case, we block until everything is done processing rather than asap
512 2016-09-01 20:08:56 0|jeremyrubin|cfields: proposal: failed test gets re-run once after all tests complete?
513 2016-09-01 20:08:58 0|cfields|wumpus: i like that.
514 2016-09-01 20:09:01 0|wumpus|cfields: doesn't matter to loop a bit in the client, after all
515 2016-09-01 20:09:02 0|wumpus|cfields: right
516 2016-09-01 20:09:14 0|jtimon|gmaxwell: oh, even with the parallel rpc tests you still wait for all to fail without seeing anything...yuck
517 2016-09-01 20:09:44 0|petertodd|gmaxwell: makes sense to me too
518 2016-09-01 20:09:55 0|cfields|wumpus: i should think that would speed tests up a pretty good bit as well. avoids tons of msec waits
519 2016-09-01 20:10:13 0|petertodd|gmaxwell: also, at some point you want to be transmitting tx replacement deltas, rather than full txs, but that's an advanced topic...
520 2016-09-01 20:10:42 0|cfields|wumpus: ok, that sounds simple enough. only complication would be waiting for the wallet
521 2016-09-01 20:11:09 0|gmaxwell|I don't know that that would be worth the effort. maybe. once you compress the tx format, most of the size comes from high entropy parts in signatures which would change on most modifications.
522 2016-09-01 20:11:18 0|petertodd|gmaxwell: a simple first step might be to have a boolean flag in AcceptToMemPool the decides whether or not we'll relay this newly accepted transaction
523 2016-09-01 20:11:56 0|petertodd|gmaxwell: in the transaction replacement case, often you'll just be changing small parts of a tx - sigs don't come into it (especially if it's malleability that lead to the replacement)
524 2016-09-01 20:11:56 0|wumpus|cfields: yes, it needs to be a wallet call
525 2016-09-01 20:12:10 0|sdaftuar|petertodd: for context, are you talking about a replacement policy specifically for when txids match existing ones (ie malleated segwit transactions), or something more general?
526 2016-09-01 20:12:29 0|jtimon|gmaxwell: yep, your idea of the ideal mempool/relay management is great and respectful with diverse policies, I was scared when I read the first "the best" and breathed with "your idea of the best"
527 2016-09-01 20:12:30 0|gmaxwell|petertodd: I just mean that if the modification changes outputs, it will normally change signatures...
528 2016-09-01 20:12:35 0|petertodd|sdaftuar: slightly more general; I'd do it so long as the vout is the same
529 2016-09-01 20:12:58 0|petertodd|gmaxwell: normally :) but not always, e.g. SIGHASH_ANYONECANPAY
530 2016-09-01 20:13:17 0|sdaftuar|oh, so if someone adds an input, you can remove it
531 2016-09-01 20:13:27 0|sdaftuar|i think you guys were discussing that on some PR?
532 2016-09-01 20:14:10 0|petertodd|sdaftuar: yup
533 2016-09-01 20:14:15 0|petertodd|sdaftuar: https://github.com/bitcoin/bitcoin/pull/8543
534 2016-09-01 20:14:17 0|sdaftuar|thanks
535 2016-09-01 20:14:31 0|jtimon|note that the "second best" pool may not be "second best" at all, maybe stuff that passes your minimums and your peers seem to repeat for some unkown reason (they should know that's not the best, but at least they agree on being wrong on the same weird way)
536 2016-09-01 20:14:46 0|gmaxwell|jtimon: well unlikely luke I don't actually believe that different policies (beyond straight resource limits) are actually virtuious, but in a hetrogenious network, with softforks and whatever, its simply unrealistic to expect equal policy even though I think it would be ideal to have equal policy.
537 2016-09-01 20:15:27 0|gmaxwell|jtimon: yea by 'second best' I really meant something more like "data you previously had, prioritized by how shocked you'd be to see it in a block"
538 2016-09-01 20:15:47 0|gmaxwell|e.g. you wouldn't keep invalid transactions, but you would keep non-standard ones.
539 2016-09-01 20:16:08 0|gmaxwell|s/unlikely/unlike/
540 2016-09-01 20:16:42 0|jtimon|I guess it's kind of a philosophically pedantic way of saying "hey, this is not consensus code", but terms...we agree on the overall design
541 2016-09-01 20:17:04 0|jtimon|gmaxwell: lol, let's call it "the shockpool"
542 2016-09-01 20:18:10 0|jtimon|but I really don't know what criteria would I use there
543 2016-09-01 20:19:51 0|cfields|wumpus: looking closer, if we subscribe to a signal rather than just hammering on chainActive.Tip(), the wallet will have already been sync'd. So that's easy. doing it up now.
544 2016-09-01 20:20:32 0|sdaftuar|cfields: ah, good idea
545 2016-09-01 20:21:26 0|gmaxwell|jtimon: well obviously you just use machine learning to predict what will end up in blocks... :P
546 2016-09-01 20:22:29 0|petertodd|gmaxwell: better, yet, use hashing power to make your predictions come true...
547 2016-09-01 20:22:35 0|jtimon|gmaxwell: of course, I'm ashamed I hadn't thought of a neural network, the fact that txs are variable size shouldn't stop us...
548 2016-09-01 20:23:00 0|gmaxwell|jtimon: hah well actually running a model on the tx data itself is not likely to work well without a very big model.
549 2016-09-01 20:23:07 0|gmaxwell|feature vector is [feerate, tx version, vector of failed is standard rules, time in the shockpool] and the model outputs a ranking that tells you how long you should retain it. :P
550 2016-09-01 20:23:21 0|gmaxwell|I've suggested similar things for utxo cache management in the past.
551 2016-09-01 20:24:27 0|jeremyrubin|Can I ask for some thoughts on the checkqueue stuff?
552 2016-09-01 20:24:57 0|jtimon|gmaxwell: model? what's wrong with feeding them binary data?
553 2016-09-01 20:25:14 0|jtimon|</sarcasm>
554 2016-09-01 20:25:37 0|jeremyrubin|My basic plan for restructuring was to add the benchmarks for the earlier version and then add the new version. I wasn't going to make my tests work for the old code, but could do it if that's neccessary for review
555 2016-09-01 20:25:46 0|gmaxwell|jtimon: nothing, except the result will require a large, deep, complex and hard to train network. at it has to learn transaction parsing. The result would be low performance enough that I think it would preclude actually using for anything except a toy. :)
556 2016-09-01 20:25:50 0|gmaxwell|oh okay.
557 2016-09-01 20:25:52 0|gmaxwell|:P
558 2016-09-01 20:25:59 0|jeremyrubin|(also if anyone would care to try testing it out would be great help for me)
559 2016-09-01 20:26:28 0|jeremyrubin|(for reference, PR is here https://github.com/bitcoin/bitcoin/pull/8464)
560 2016-09-01 20:29:26 0|Chris_Stewart_5|jeremyrubin: Might be worth while to look at #8469 and write some properties for testing
561 2016-09-01 20:30:03 0|gmaxwell|jonasschnelli: thanks for that backport.
562 2016-09-01 20:30:48 0|jeremyrubin|Chris_Stewart_5: looks cool; I have a pretty good test suite I added already though. When that gets merged I'd be happy to
563 2016-09-01 20:34:15 0|jtimon|gmaxwell: yep, it would need to learn script parsing, for all I know, random search may be as good as anything else for training in this case
564 2016-09-01 20:49:38 0|gmaxwell|PR #8212 needs backport too.
565 2016-09-01 20:49:49 0|gmaxwell|er I meant PR #8612.
566 2016-09-01 20:52:45 0|instagibbs|what's the New Improved Way(TM) to get uint256 as string
567 2016-09-01 20:53:07 0|instagibbs|still working my way around git blaming in reverse, failing me right now
568 2016-09-01 20:54:34 0|gmaxwell|instagibbs: do you mean as hex?
569 2016-09-01 20:54:42 0|instagibbs|yeah hex string
570 2016-09-01 20:57:32 0|instagibbs|oh weight, nevermind me, just in .cpp now, and complaining
571 2016-09-01 20:57:36 0|instagibbs|wait even
572 2016-09-01 20:57:48 0|instagibbs|probably just fat fingered
573 2016-09-01 21:14:37 0|gmaxwell|interesting:
574 2016-09-01 21:14:39 0|gmaxwell|2016-09-01 15:35:29.537118 replacing tx 528f6cbebe29fa6bdab3be077a48754c2371df2fbf4cbb8fbc11992fdeb26470 with
575 2016-09-01 21:14:42 0|gmaxwell|922a0895c7246b46533d1d356152bde2477cc9987c96b78e02da4d5869e4b7cc for 0.00008873 BTC additional fees, -194 delta bytes
576 2016-09-01 21:14:45 0|gmaxwell|2016-09-01 15:35:29.537194 replacing tx 308a560e8e90ccaf64af9f5cd3b1dcfe4bccf8d49d73b0460b95b1616bb26c9d with
577 2016-09-01 21:14:48 0|gmaxwell|922a0895c7246b46533d1d356152bde2477cc9987c96b78e02da4d5869e4b7cc for 0.00008873 BTC additional fees, -194 delta bytes
578 2016-09-01 21:14:51 0|gmaxwell|2016-09-01 15:35:50.780139 Successfully reconstructed block 000000000000000002f65cc5054fefb1b8fe75ebb685ac6b51f05dac0b84ee3d with 1 txn
579 2016-09-01 21:14:54 0|gmaxwell|prefilled, 2465 txn from mempool and 2 txn requested 2016-09-01 15:35:50.780208 Reconstructed block
580 2016-09-01 21:14:57 0|gmaxwell|000000000000000002f65cc5054fefb1b8fe75ebb685ac6b51f05dac0b84ee3d required tx
581 2016-09-01 21:15:00 0|gmaxwell|528f6cbebe29fa6bdab3be077a48754c2371df2fbf4cbb8fbc11992fdeb26470
582 2016-09-01 21:15:03 0|gmaxwell|2016-09-01 15:35:50.780253 Reconstructed block 000000000000000002f65cc5054fefb1b8fe75ebb685ac6b51f05dac0b84ee3d required tx
583 2016-09-01 21:15:06 0|gmaxwell|308a560e8e90ccaf64af9f5cd3b1dcfe4bccf8d49d73b0460b95b1616bb26c9d
584 2016-09-01 21:18:43 0|gmaxwell|other recent CB extra round trips were: a single txn that failed minfee 30 minutes before the block, a pair of transactions that were rejected due to being mempool conflicts ~5-10 minutes before the block, and an orphan that was in my orphan pool at the time.
585 2016-09-01 21:22:02 0|luke-jr|gmaxwell: presumably the parent of the orphan was missing in your mempool? was that one of the conflicts?
586 2016-09-01 21:23:05 0|gmaxwell|those were three steperate blocks.
587 2016-09-01 21:23:42 0|luke-jr|so what was the cause of the orphan not being in the mempool (or rejected)?
588 2016-09-01 21:23:53 0|gmaxwell|yea, trying to figure that out.
589 2016-09-01 21:24:24 0|gmaxwell|maybe it actually had been evicted by the time the parent came around.
590 2016-09-01 21:24:41 0|gmaxwell|2016-09-01 11:50:23.982337 stored orphan tx aeb9dac61cf2cc7fd535f2b4b685070b0dce2690c9f78d9987335cbadedf5aa3 (mapsz 101 outsz 110)
591 2016-09-01 21:25:14 0|gmaxwell|2016-09-01 11:53:45.913417 Successfully reconstructed block 000000000000000004e70299cd0284b177412b63144fc4b9c6080c65545e875b with 1 txn prefilled, 1868 txn from mempool and 1 txn requested
592 2016-09-01 21:25:27 0|gmaxwell|2016-09-01 11:53:45.913505 Reconstructed block 000000000000000004e70299cd0284b177412b63144fc4b9c6080c65545e875b required tx aeb9dac61cf2cc7fd535f2b4b685070b0dce2690c9f78d9987335cbadedf5aa3
593 2016-09-01 21:26:03 0|luke-jr|hmm
594 2016-09-01 21:26:29 0|gmaxwell|looking up the parents now.
595 2016-09-01 21:28:22 0|luke-jr|FWIW, looking through my logs, most blocks seem to require a round trip. not particularly surprising (spam I assume)
596 2016-09-01 21:28:33 0|gmaxwell|luke-jr: wow, no way.
597 2016-09-01 21:29:29 0|luke-jr|http://0bin.net/paste/nELoeFqIuT4891qP#P8ugYLrvtlhggdD80jGS5NDOuhJYpJ37JgDgHiGF1CF
598 2016-09-01 21:30:09 0|gmaxwell|grep econs ~/.bitcoin/debug.log | tail -n 100 | awk '{aa+=$16>1} END {print aa/NR}'
599 2016-09-01 21:30:12 0|gmaxwell|0.11
600 2016-09-01 21:30:14 0|gmaxwell|11% here.
601 2016-09-01 21:31:00 0|luke-jr|0.661538
602 2016-09-01 21:31:05 0|gmaxwell|how long have you been up?
603 2016-09-01 21:31:49 0|gmaxwell|my past expirence is that it takes a good 24 hour uptime before it really behaves normally (and I think I saw measureable improvement for up to three days, though much of that was due to orphans, which should be less bad since the other changes.)
604 2016-09-01 21:32:52 0|gmaxwell|also, if you have an atypical mempool policy, thats going to slow you down.. until we get some kind of rejects cache going.
605 2016-09-01 21:33:12 0|luke-jr|about 1.5 hours now, but quite a bit longer before that restart
606 2016-09-01 21:33:27 0|luke-jr|yes, that's why I said not surprising ;)
607 2016-09-01 21:33:43 0|gmaxwell|oh... 'spam I assume'
608 2016-09-01 21:33:47 0|gmaxwell|indeed. okay.
609 2016-09-01 21:35:55 0|luke-jr|sub-second round-trip for 3 txn, so I assume it's still a gain
610 2016-09-01 21:37:04 0|gmaxwell|luke-jr: yes, I found that it was in testing.