1 2016-11-23 05:41:54	0|wumpus|instagibbs: yes there are testcases for zmq and rest
  2 2016-11-23 06:14:42	0|bitcoin-git|[13bitcoin] 15laanwj pushed 3 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/ac489b24453c...4d8558a2871e
  3 2016-11-23 06:14:43	0|bitcoin-git|13bitcoin/06master 14240189b 15John Newbery: add testcases for getrawtransaction
  4 2016-11-23 06:14:43	0|bitcoin-git|13bitcoin/06master 14ce2bb23 15jnewbery: getrawtransaction should take a bool for verbose
  5 2016-11-23 06:14:44	0|bitcoin-git|13bitcoin/06master 144d8558a 15Wladimir J. van der Laan: Merge #9025: getrawtransaction should take a bool for verbose...
  6 2016-11-23 06:18:47	0|bitcoin-git|13bitcoin/06master 142e44893 15Jonas Schnelli: Move -salvagewallet, -zap(wtx) to where they belong
  7 2016-11-23 06:18:47	0|bitcoin-git|[13bitcoin] 15laanwj pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/4d8558a2871e...40022fe5f2b5
  8 2016-11-23 06:18:48	0|bitcoin-git|13bitcoin/06master 1440022fe 15Wladimir J. van der Laan: Merge #9142: Move -salvagewallet, -zap(wtx) to where they belong...
  9 2016-11-23 06:18:57	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #9142: Move -salvagewallet, -zap(wtx) to where they belong (06master...062016/11/wallet_init) 02https://github.com/bitcoin/bitcoin/pull/9142
 10 2016-11-23 06:19:05	0|wumpus|can people please notify me if there is a pull which is clearly ready for merging? (e.g. 9142 had three utacks and was ready for merge for ~7 days already)
 11 2016-11-23 06:19:50	0|wumpus|there are too many open for me to keep track of so I really need help in that
 12 2016-11-23 06:25:10	0|bitcoin-git|13bitcoin/06master 144512550 15Jonas Schnelli: Remove unnecessary calls to CheckFinalTx
 13 2016-11-23 06:25:10	0|bitcoin-git|[13bitcoin] 15laanwj pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/40022fe5f2b5...5ea5e0401cb3
 14 2016-11-23 06:25:11	0|bitcoin-git|13bitcoin/06master 145ea5e04 15Wladimir J. van der Laan: Merge #9141: Remove unnecessary calls to CheckFinalTx...
 15 2016-11-23 06:25:20	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #9141: Remove unnecessary calls to CheckFinalTx (06master...062016/11/istrusted) 02https://github.com/bitcoin/bitcoin/pull/9141
 16 2016-11-23 06:29:55	0|cfields|wumpus: sure
 17 2016-11-23 06:30:08	0|wumpus|cfields: thanks
 18 2016-11-23 06:30:09	0|cfields|wumpus: would a "looks ready" label help?
 19 2016-11-23 06:30:29	0|cfields|not sure if that would just be noisy
 20 2016-11-23 06:32:11	0|shangzhou|wumpus: #9037 per author's last comment
 21 2016-11-23 06:32:13	0|gribble|https://github.com/bitcoin/bitcoin/issues/9037 | net: Add test-before-evict discipline to addrman by EthanHeilman · Pull Request #9037 · bitcoin/bitcoin · GitHub
 22 2016-11-23 06:33:32	0|wumpus|well, setting a label or even a github notification is easy to miss too :)
 23 2016-11-23 06:33:57	0|cfields|sure, i just figured it'd be an easy filter
 24 2016-11-23 06:34:05	0|wumpus|shangzhou: will take a look
 25 2016-11-23 06:34:35	0|wumpus|shangzhou: it has no (ut)ACKs at all,certainly still needs review
 26 2016-11-23 06:35:47	0|wumpus|cfields: btw good call in #9128 on not cleansing memory after every message anymore, it's always been kind of silly
 27 2016-11-23 06:35:49	0|gribble|https://github.com/bitcoin/bitcoin/issues/9128 | net: Decouple CConnman and message serialization by theuni · Pull Request #9128 · bitcoin/bitcoin · GitHub
 28 2016-11-23 06:36:39	0|cfields|wumpus: yea, i suspect that's actually a major part of the sending overhead
 29 2016-11-23 06:37:03	0|cfields|it showed to be a majority when profiling, though i built at -O0 or -O1 for that, so it probably wasn't realistic
 30 2016-11-23 06:37:10	0|fanquake|It'd be handy if you could change the default PR ordering to something like that.
 31 2016-11-23 06:37:10	0|wumpus|well overwriting memory with zeros shouldn't be too bad, though apparently it gets the openssl lock which is bad
 32 2016-11-23 06:37:29	0|cfields|wumpus: see the suggestion in https://github.com/bitcoin/bitcoin/issues/9198 as well
 33 2016-11-23 06:38:01	0|cfields|or maybe that's what reminded you :)
 34 2016-11-23 06:38:37	0|wumpus|cfields: yes that's what reminded me
 35 2016-11-23 06:39:08	0|wumpus|I had seen it before and wondered wtf, but seemed too invasive to change at the time
 36 2016-11-23 06:39:26	0|cfields|wumpus: well the zero-write goes out of its way to be a pain-in-the-ass, and therefor slower than it should be. Otherwise it'd just be optimized out
 37 2016-11-23 06:39:49	0|wumpus|now that things have beeen disentangled a bit it is easier
 38 2016-11-23 06:39:56	0|cfields|so i think much of the hit just comes from having any non-trivial dtor, regardless of what it is
 39 2016-11-23 06:40:34	0|cfields|but again, that's coming from profiling data in the non-optimized case. Much of that probably goes away at -O2 or -O3
 40 2016-11-23 06:40:44	0|wumpus|yes could be
 41 2016-11-23 06:40:53	0|cfields|so yea, the locks probably account for most of it at that point
 42 2016-11-23 06:40:55	0|wumpus|right, profiling results at O0 are useless :)
 43 2016-11-23 06:42:23	0|cfields|wumpus: i can't think of where cleansing is really necessary other than private keys and passphrases?
 44 2016-11-23 06:43:04	0|cfields|though i'd like to be sufficiently paranoid if i'm going to be the one PRing the removals :)
 45 2016-11-23 06:44:24	0|wumpus|I think so, I'm not entirely sure
 46 2016-11-23 06:45:56	0|wumpus|if that is the case we wouldn't even need the zero_after_free allocator, just the lockedpool one used for key data
 47 2016-11-23 06:46:39	0|wumpus|but in any case yes such a PR would have to evalutate all uses of zero after free individually
 48 2016-11-23 06:47:05	0|cfields|that makes sense. Hard to make the case for something sensitive enough to zero but not memlock
 49 2016-11-23 06:47:40	0|fanquake|Does anyone have some inputs to share for testing #9172
 50 2016-11-23 06:47:41	0|gribble|https://github.com/bitcoin/bitcoin/issues/9172 | An error has occurred and has been logged. Please contact this bot's administrator for more information.
 51 2016-11-23 06:48:16	0|wumpus|fanquake: yes I do now, let me upload
 52 2016-11-23 06:48:19	0|cfields|ah damn, that one slipped away from me again
 53 2016-11-23 06:49:45	0|cfields|will review in the morning. The build-side looked fine, I just wanted to find something to plug in to test
 54 2016-11-23 06:50:14	0|wumpus|fanquake: https://download.visucore.com/bitcoin/bitcoin_fuzzy_in.tar.xz
 55 2016-11-23 06:50:39	0|fanquake|cfields talking about 9172? Did you install afl via brew or download and install?
 56 2016-11-23 06:50:55	0|fanquake|wumpus thanks
 57 2016-11-23 06:51:30	0|cfields|fanquake: not tested yet, just a quick review of the build changes
 58 2016-11-23 06:52:38	0|fanquake|Ok. I'll try using brew'd afl, seems to be up to date (2.35b)
 59 2016-11-23 06:57:09	0|wumpus|cfields: "Note that each message should be padded in the front by at least CSerializedNetMsg::header_pad_size bytes. This allows for a header to be inserted without requring a copy." awesome, we're getting our own skb's :)
 60 2016-11-23 06:57:52	0|wumpus|(reminds me of http://vger.kernel.org/~davem/skb_data.html)
 61 2016-11-23 07:01:10	0|cfields|wumpus: hah! that's exactly what it's like. Wish I would've known to google "skb" earlier, would've saved me some head-scratching. Good to know it's a reasonable approach, though :)
 62 2016-11-23 07:06:08	0|wumpus|instagibbs: eh in #9073 you changed the bitcoin-cli message but it's confusing now: error: couldn't connect to server (make sure server is running and you are connecting to the correct RPC port: -1 unknown)
 63 2016-11-23 07:06:09	0|gribble|https://github.com/bitcoin/bitcoin/issues/9073 | Trivial: Add common failure cases for rpc server connection failure by instagibbs · Pull Request #9073 · bitcoin/bitcoin · GitHub
 64 2016-11-23 07:06:30	0|wumpus|instagibbs: that's as if the *port* is unknown. The error code and message should be on the first line, not the second :)
 65 2016-11-23 07:06:59	0|wumpus|cfields: yes it's a great way of handling packet processing
 66 2016-11-23 07:09:29	0|wumpus|btw RPC named arguments is feature-complete now (support in bitcoin-cli landed) and ready for testing https://github.com/bitcoin/bitcoin/pull/8811
 67 2016-11-23 07:15:03	0|cfields|wumpus: thanks. I'd really like to see that. Pushing on my review stack as well.
 68 2016-11-23 07:18:29	0|fanquake|wumpus did you need to set a higher (>50mb) memory limit when running the fuzzer?
 69 2016-11-23 07:18:57	0|wumpus|fanquake: no,I did not change any afl settings
 70 2016-11-23 07:19:27	0|wumpus|may make sense though
 71 2016-11-23 07:19:29	0|wumpus|cfields: thanks
 72 2016-11-23 07:20:22	0|bitcoin-git|13bitcoin/06master 14a33b169 15Pieter Wuille: Do not fully sort all nodes for addr relay...
 73 2016-11-23 07:20:22	0|bitcoin-git|[13bitcoin] 15laanwj pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/5ea5e0401cb3...791b58d148ab
 74 2016-11-23 07:20:23	0|bitcoin-git|13bitcoin/06master 14791b58d 15Wladimir J. van der Laan: Merge #8690: Do not fully sort all nodes for addr relay...
 75 2016-11-23 07:22:16	0|fanquake|wumpus: Ok, I must have a different issue, as I've upped the memory limit to 12GB and it's still warning that may be to restrictive.
 76 2016-11-23 07:25:46	0|wumpus|heh
 77 2016-11-23 07:28:49	0|jonasschnelli|I think this is ready: https://github.com/bitcoin/bitcoin/pull/9196
 78 2016-11-23 07:39:22	0|wumpus|jonasschnelli: looking
 79 2016-11-23 07:40:44	0|bitcoin-git|13bitcoin/06master 1467c6326 15Russell Yanofsky: Send tip change notification from invalidateblock...
 80 2016-11-23 07:40:44	0|bitcoin-git|[13bitcoin] 15laanwj pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/791b58d148ab...7e2bfd62419f
 81 2016-11-23 07:40:45	0|bitcoin-git|13bitcoin/06master 147e2bfd6 15Wladimir J. van der Laan: Merge #9196: Send tip change notification from invalidateblock...
 82 2016-11-23 07:40:58	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #9196: Send tip change notification from invalidateblock (06master...06pr/notify-tip) 02https://github.com/bitcoin/bitcoin/pull/9196
 83 2016-11-23 07:41:37	0|bitcoin-git|13bitcoin/06master 14f004e67 15Greg Walker: Minor change to comment above new NODE_WITNESS service flag to keep it consitent with existing comment structure. Helps with readability.
 84 2016-11-23 07:41:37	0|bitcoin-git|[13bitcoin] 15laanwj pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/7e2bfd62419f...5e8631b6cb5b
 85 2016-11-23 07:41:38	0|bitcoin-git|13bitcoin/06master 145e8631b 15Wladimir J. van der Laan: Merge #9205: Minor change to comment for consistency....
 86 2016-11-23 07:41:47	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #9205: Minor change to comment for consistency. (06master...06master) 02https://github.com/bitcoin/bitcoin/pull/9205
 87 2016-11-23 07:42:10	0|luke-jr|[07:28:49] <jonasschnelli> I think this is ready: https://github.com/bitcoin/bitcoin/pull/9196 <-- am I missing something? what if the new best isn't pprev?
 88 2016-11-23 07:42:42	0|bitcoin-git|13bitcoin/06master 14918b126 15instagibbs: fix CreateTransaction error messages
 89 2016-11-23 07:42:42	0|bitcoin-git|[13bitcoin] 15laanwj pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/5e8631b6cb5b...74ced54b7e7b
 90 2016-11-23 07:42:43	0|bitcoin-git|13bitcoin/06master 1474ced54 15Wladimir J. van der Laan: Merge #9204: Clarify CreateTransaction error messages...
 91 2016-11-23 07:42:50	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #9204: Clarify CreateTransaction error messages (06master...06nonneg) 02https://github.com/bitcoin/bitcoin/pull/9204
 92 2016-11-23 07:42:55	0|jonasschnelli|Luke-Jr: how could that be the case in InvalidateBlock()?
 93 2016-11-23 07:43:19	0|luke-jr|jonasschnelli: a->b->c->d->e vs a->b->c->x; invalidate d
 94 2016-11-23 07:46:20	0|jonasschnelli|Luke-Jr: You mean what happens if you invalidate block x) in a->b->c->d->e? (block thats not part of the main-chain)?
 95 2016-11-23 07:47:01	0|luke-jr|jonasschnelli: no, I mean invalidate block D
 96 2016-11-23 07:47:04	0|jonasschnelli|if you invalidate d) in a->b->c->d->e, pindex==d and pprev should match
 97 2016-11-23 07:47:12	0|luke-jr|it should end up on block X
 98 2016-11-23 07:47:17	0|luke-jr|X > C
 99 2016-11-23 07:48:21	0|jonasschnelli|Hmm... let me test this though...
100 2016-11-23 07:53:13	0|bitcoin-git|[13bitcoin] 15AmirAbrams opened pull request #9207: [Doc] Add missing bash # comment characters (06master...06patch-2) 02https://github.com/bitcoin/bitcoin/pull/9207
101 2016-11-23 08:21:06	0|jonasschnelli|luke-jr: If I call invalidateblock(d) on a-b-c-d-e, the tip will chain will result in a-b-c
102 2016-11-23 08:21:30	0|luke-jr|jonasschnelli: it shouldn't be, if there is an alternate chain a-b-c-x
103 2016-11-23 08:23:25	0|luke-jr|to test, send a-b-c-x then reorg to -d-e then invalidate d
104 2016-11-23 08:29:58	0|BlueMatt|wumpus: might want to cancel meeting on thursday in light of american thanksgiving
105 2016-11-23 08:30:11	0|wumpus|BlueMatt: good point
106 2016-11-23 08:30:38	0|BlueMatt|wumpus: also, before you get too excited about #9128, I think sipa, cfields and I need to get togehter (or at a meeting) and figure out what we want to do about where-messages-get-hashed
107 2016-11-23 08:30:40	0|gribble|https://github.com/bitcoin/bitcoin/issues/9128 | net: Decouple CConnman and message serialization by theuni · Pull Request #9128 · bitcoin/bitcoin · GitHub
108 2016-11-23 08:32:14	0|wumpus|BlueMatt: cfields could we possibly factor out the controversial part into a seperate pull?
109 2016-11-23 08:32:53	0|BlueMatt|wumpus: I think we just need to find a few minutes to discuss when we're all here...it should be easy to come to agreement (hence why i came to the conclusion that this week's meeting wont work :( )
110 2016-11-23 08:32:56	0|wumpus|most of the changes are no-brainers w/ regard to desirability
111 2016-11-23 08:33:02	0|BlueMatt|yea
112 2016-11-23 08:33:50	0|wumpus|but sure we can hold off on that one until you discussed
113 2016-11-23 08:34:25	0|BlueMatt|wumpus: I keep remembering after cfields goes to sleep :(
114 2016-11-23 08:34:44	0|gmaxwell|wumpus: ACK re-pings. I have seen some sit ready for a while and didn't want to nag.
115 2016-11-23 08:34:46	0|jonasschnelli|luke-jr: I think that would be a general improvement in InvalidateBlock ...
116 2016-11-23 08:38:12	0|jonasschnelli|luke-jr: heh, InvalidateBlock(getbestblockhash()) on a fresh regtest setup (no block mined) results in a funny state
117 2016-11-23 08:38:39	0|luke-jr|does it really not work? :O
118 2016-11-23 08:39:20	0|jonasschnelli|It disconnects the genesis block but missed the undo-data
119 2016-11-23 08:39:23	0|jonasschnelli|generate 1 will fail afterwards
120 2016-11-23 08:39:39	0|luke-jr|no, I mean a-b-c-x
121 2016-11-23 08:40:12	0|luke-jr|I mean, it's rare you'd want to invalidate the tip without a competing chain to reorg to
122 2016-11-23 08:40:14	0|jonasschnelli|I think InvalidateBlock is only used during RegTests/RPC tests... are there real world mainnet usecases?
123 2016-11-23 08:40:24	0|BlueMatt|so I could PR relay-compact-block-before-full-validation (https://github.com/TheBlueMatt/bitcoin/commits/2016-11-compact-fast-relay) but now I'm kinda thinking I dont want to do that prior to there being multithreaded ProcessMessages with some smarts to respond to getblocktxn without a lock for the latest block...thoughts?
124 2016-11-23 08:40:27	0|BlueMatt|(maybe gmaxwell)
125 2016-11-23 08:40:32	0|luke-jr|it was added so a fiasco like 0.8.0 could be manually dealt with
126 2016-11-23 08:46:03	0|jonasschnelli|luke-jr: generating a-b-c-d-e, invalidating e, mining -d-x-y, reconsidering e, invalidate x results in a-b-c-d
127 2016-11-23 08:46:14	0|jonasschnelli|So reorg with InvalidateBlock seems to work
128 2016-11-23 08:46:27	0|jonasschnelli|Maybe the pprev call is not ideal
129 2016-11-23 08:47:10	0|luke-jr|that sequence doesn't make sense. you mined d twice?
130 2016-11-23 08:47:25	0|luke-jr|if a-b-c-d-e is valid, tip should never be D
131 2016-11-23 08:48:03	0|jonasschnelli|I invalidated e
132 2016-11-23 08:48:27	0|luke-jr|but then reconsidered it
133 2016-11-23 08:48:33	0|luke-jr|which should make it valid
134 2016-11-23 08:48:46	0|jonasschnelli|wait... let me write it again
135 2016-11-23 08:49:21	0|jonasschnelli|generating a-b-c-d-e, invalidating e, mining (-d)-x-y, reconsidering e, invalidate x results in a-b-c-d-e
136 2016-11-23 08:50:09	0|luke-jr|ok, good; but does this call NotifyTip with D or E?
137 2016-11-23 08:50:27	0|jonasschnelli|I'm checking now the signal... maybe its wrong.
138 2016-11-23 08:50:28	0|luke-jr|it should only call NotifyTip with E I think, but certainly both and E last
139 2016-11-23 08:50:40	0|luke-jr|err, certainly E last*
140 2016-11-23 08:50:56	0|luke-jr|(whether it calls it with D before E is a debatable question)
141 2016-11-23 08:53:43	0|gmaxwell|BlueMatt: I think it's fine to send it first without fancy getblocktxn handling. Most blocks relay without a getblocktxn.
142 2016-11-23 08:54:21	0|gmaxwell|BlueMatt: on the balance for both bandwidth usage and latency handling using the orphan pool and having an eviction pool be more important that faster getblocktxn handling.
143 2016-11-23 08:56:54	0|gmaxwell|BlueMatt: I think sipa and I convinced cfields that what you were doing was reasonable.
144 2016-11-23 08:59:14	0|gmaxwell|luke-jr: jonasschnelli: I'd be really surprised if notify behavior was sensible around invalidate block, it's a hidden and undocumented command intenteded for development and for fringe emergencies. (Though I'd certantly support making it more sensible-- AFAIK it really hasn't been tested to the level that I'd expect for a first tier user feature).
145 2016-11-23 08:59:28	0|gmaxwell|It also is massively slow at undoing blocks, enough to degrade its utility.
146 2016-11-23 08:59:58	0|luke-jr|gmaxwell: jonasschnelli just "fixed" it, but I think the fix is wrong
147 2016-11-23 09:00:23	0|jonasschnelli|luke-jr: I don't fixed it,... I merged a fix. :)
148 2016-11-23 09:00:35	0|jonasschnelli|But I guess luke-jr is right... it could be wrong.
149 2016-11-23 09:01:04	0|jonasschnelli|But my test just showed me, the signal gets called with the reconnected blocks.. so the UI state is correct... but looking deeper into it now
150 2016-11-23 09:02:01	0|gmaxwell|One challenge is that invalidateblock can cause a best block tip with lower work than the best signaled previously. There is no other way for that to happen in the codebase, so in that sense it breaks an interface invarient.
151 2016-11-23 09:03:33	0|luke-jr|well, at least that case is GIGO
152 2016-11-23 09:03:44	0|jonasschnelli|luke-jr: I think the change is correct
153 2016-11-23 09:03:47	0|gmaxwell|true. :)
154 2016-11-23 09:04:28	0|jonasschnelli|pprev = chainActive.Tip()
155 2016-11-23 09:04:51	0|jonasschnelli|And in the RPC call InvalidateBlock, we have a call ActivateBestChain(state, Params(), NULL);
156 2016-11-23 09:05:01	0|jonasschnelli|Which calls the signal again with the new tip
157 2016-11-23 09:05:04	0|jonasschnelli|so.. all good IMO
158 2016-11-23 09:06:03	0|luke-jr|ah, so the internal InvalidateBlock doesn't complete the reorg(s), and the RPC does that in a subsequent step?
159 2016-11-23 09:06:25	0|jonasschnelli|Yes
160 2016-11-23 09:06:37	0|luke-jr|I guess that makes sense. Maybe better to change it, but outside the scope of this.
161 2016-11-23 10:11:19	0|bitcoin-git|13bitcoin/06master 1469bc8e7 15Amir Abrams: [Doc] Move comments above bash command
162 2016-11-23 10:11:19	0|bitcoin-git|[13bitcoin] 15laanwj pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/74ced54b7e7b...fa1f944107f9
163 2016-11-23 10:11:20	0|bitcoin-git|13bitcoin/06master 14fa1f944 15Wladimir J. van der Laan: Merge #9207: [Doc] Move comments above bash command in build-unix...
164 2016-11-23 10:11:33	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #9207: [Doc] Move comments above bash command in build-unix (06master...06patch-2) 02https://github.com/bitcoin/bitcoin/pull/9207
165 2016-11-23 10:41:12	0|bitcoin-git|13bitcoin/06master 1409dc406 15BtcDrak: Make test constant consistent with consensus.h
166 2016-11-23 10:41:12	0|bitcoin-git|[13bitcoin] 15MarcoFalke pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/fa1f944107f9...e662d281b837
167 2016-11-23 10:41:13	0|bitcoin-git|13bitcoin/06master 14e662d28 15MarcoFalke: Merge #9206: Make test constant consistent with consensus.h...
168 2016-11-23 10:41:24	0|bitcoin-git|[13bitcoin] 15MarcoFalke closed pull request #9206: Make test constant consistent with consensus.h (06master...06consistency) 02https://github.com/bitcoin/bitcoin/pull/9206
169 2016-11-23 10:42:35	0|bitcoin-git|[13bitcoin] 15sdaftuar opened pull request #9208: [WIP] Improve DisconnectTip performance (06master...06faster-disconnect-rebased) 02https://github.com/bitcoin/bitcoin/pull/9208
170 2016-11-23 10:42:44	0|sdaftuar|gmaxwell: ^ This should fix the invalidateblock performance issues
171 2016-11-23 11:58:20	0|brhood9|is anyone there? I need help please and I'm willing to pay.
172 2016-11-23 12:01:17	0|Victorsueca|brhood9: what with?
173 2016-11-23 12:01:55	0|bitcoin-git|[13bitcoin] 15MarcoFalke closed pull request #9139: Change sync_blocks to pick smarter maxheight (on top of #9196) (06master...06sync-height) 02https://github.com/bitcoin/bitcoin/pull/9139
174 2016-11-23 12:02:41	0|bitcoin-git|13bitcoin/06master 141126c85 15Russell Yanofsky: [qa] Change sync_blocks to pick smarter maxheight...
175 2016-11-23 12:02:41	0|bitcoin-git|[13bitcoin] 15MarcoFalke pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/e662d281b837...0de7fd36de57
176 2016-11-23 12:02:42	0|bitcoin-git|13bitcoin/06master 140de7fd3 15MarcoFalke: Merge #9139: Change sync_blocks to pick smarter maxheight (on top of #9196)...
177 2016-11-23 12:02:53	0|bitcoin-git|[13bitcoin] 15MarcoFalke closed pull request #9139: Change sync_blocks to pick smarter maxheight (on top of #9196) (06master...06sync-height) 02https://github.com/bitcoin/bitcoin/pull/9139
178 2016-11-23 12:04:06	0|brhood9_|Is there any way that you all could help me figure out what is going on with my bitcoin transaction? I have asked and looked and read everywhere. The transaction that you need to look at is this one.
179 2016-11-23 12:04:06	0|brhood9_|This is about unconfirmed transactions that need to be fixed.
180 2016-11-23 12:05:08	0|brhood9_|https://blockchain.info/tx/6bef9829ddfba24ec2e613fa1a8239f09acfc36ab057e890f192863298b648b8?show_adv=true
181 2016-11-23 12:05:54	0|brhood9_|is anyone there?
182 2016-11-23 12:06:03	0|Victorsueca|brhood9_: yes
183 2016-11-23 12:06:07	0|Victorsueca|i'm looking at it
184 2016-11-23 12:06:23	0|Victorsueca|also this is off-topic here
185 2016-11-23 12:06:29	0|Victorsueca|please move to #bitcoin
186 2016-11-23 12:27:48	0|takashi|Why "maker" was added to transaction instead of increment "version" for segwit?
187 2016-11-23 12:28:40	0|bitcoin-git|[13bitcoin] 15MarcoFalke opened pull request #9211: [0.12.2] Backports (060.12...06Mf1611-back12) 02https://github.com/bitcoin/bitcoin/pull/9211
188 2016-11-23 13:20:40	0|instagibbs|is there a flag I have to set to compile zmq/rest
189 2016-11-23 13:21:21	0|jonasschnelli|instagibbs: rest should compile with RPC (so not optional)
190 2016-11-23 13:21:29	0|jonasschnelli|ZMQ will compile if libzmq is present
191 2016-11-23 13:21:35	0|jonasschnelli|Rest needs to be enabled during startup
192 2016-11-23 13:21:36	0|jonasschnelli|-rest=1
193 2016-11-23 13:21:41	0|jonasschnelli|or just -rest
194 2016-11-23 13:21:53	0|instagibbs|ok thanks. I had zmq code fail to compile in Travis :)
195 2016-11-23 13:21:53	0|jonasschnelli|also check the docs/REST.md
196 2016-11-23 13:22:07	0|instagibbs|wanted to avoid this
197 2016-11-23 13:22:12	0|jonasschnelli|I think one of the ZMQ instance compiles zmq? Not?
198 2016-11-23 13:22:25	0|jonasschnelli|(or even all of them?)
199 2016-11-23 13:22:29	0|instagibbs|code I changed*
200 2016-11-23 13:22:47	0|jonasschnelli|In the REST.md, theres also a curl command that works "out-of-the-box". :)
201 2016-11-23 13:27:20	0|instagibbs|I don't see a REST.md?
202 2016-11-23 13:27:28	0|instagibbs|perhaps search fail on my part....
203 2016-11-23 13:33:42	0|paveljanik|while testing on testnet, I have just received: Assertion failed: (nSendVersion != 0), function GetSendVersion, file ./net.h, line 775.
204 2016-11-23 13:34:08	0|paveljanik|on current master
205 2016-11-23 13:40:06	0|jonasschnelli|instagibbs: REST-interface.md
206 2016-11-23 13:40:08	0|jonasschnelli|Sry: https://github.com/bitcoin/bitcoin/blob/master/doc/REST-interface.md
207 2016-11-23 17:45:07	0|cfields|sdaftuar: interesting
208 2016-11-23 17:45:25	0|cfields|(the staller logic log)
209 2016-11-23 18:39:59	0|Chris_Stewart_5|There isn't an easy way to replace a tx in the mempool if the user had all sequence numbers set to max, correct?
210 2016-11-23 18:40:16	0|Chris_Stewart_5|and the parent isn't RBF either
211 2016-11-23 18:42:43	0|gmaxwell|in practice you just create a replacement and keep broadcasting it. miners get restarted enough that it's pretty likely to get replaced eventually.
212 2016-11-23 18:43:03	0|gmaxwell|(assuming the prior was low fee)
213 2016-11-23 18:45:35	0|Chris_Stewart_5|Yes, it happened before this latest 'influx' or txs. Does Core's algorithm rejects txs from the mempool if it already has a tx that spends that particular output? I'm guessing it doesn't match on txids since those will be unique assuming
214 2016-11-23 18:45:44	0|Chris_Stewart_5|some sort of malleability or reordering of inputs
215 2016-11-23 18:53:15	0|BlueMatt|jonasschnelli: if you're gonna ack #9183, please also ack its dependencies, #8930
216 2016-11-23 18:53:17	0|gribble|https://github.com/bitcoin/bitcoin/issues/9183 | Final Preparation for main.cpp Split by TheBlueMatt · Pull Request #9183 · bitcoin/bitcoin · GitHub
217 2016-11-23 18:53:18	0|gribble|https://github.com/bitcoin/bitcoin/issues/8930 | Move orphan processing to ActivateBestChain by TheBlueMatt · Pull Request #8930 · bitcoin/bitcoin · GitHub
218 2016-11-23 19:14:12	0|Chris_Stewart_5|Is there a command line flag to enable RBF by default when creating a tx with core's wallet?
219 2016-11-23 19:14:28	0|luke-jr|Chris_Stewart_5: AFAIK there are nodes miners who will RBF without any flag
220 2016-11-23 19:14:56	0|luke-jr|Chris_Stewart_5: -walletrbf but I don't know what Core version it's in
221 2016-11-23 19:29:34	0|Chris_Stewart_5|Thanks luke-jr
222 2016-11-23 19:34:25	0|Victorsueca|luke-jr: did you check with the pillow why my ASIC didn't work with bfgminer or still no idea?
223 2016-11-23 20:12:33	0|cfields|BlueMatt: want to argue about #9128 for a little bit?
224 2016-11-23 20:12:35	0|gribble|https://github.com/bitcoin/bitcoin/issues/9128 | net: Decouple CConnman and message serialization by theuni · Pull Request #9128 · bitcoin/bitcoin · GitHub
225 2016-11-23 20:12:50	0|BlueMatt|cfields: sure
226 2016-11-23 20:14:01	0|BlueMatt|cfields: see https://github.com/TheBlueMatt/bitcoin/commit/7a42a0c95d04add8cff0cf7cf95df273169f8fa3 ?
227 2016-11-23 20:14:02	0|cfields|BlueMatt: so, before we end up in tangential discussions, i think the key thing to decide on is where encryption state should be held
228 2016-11-23 20:14:09	0|BlueMatt|cfields: fix the TODO :p
229 2016-11-23 20:14:33	0|BlueMatt|especially, send the message without hashing the message repeatedly
230 2016-11-23 20:15:50	0|cfields|BlueMatt: and if some nodes are using encryption?
231 2016-11-23 20:15:54	0|gmaxwell|BlueMatt: you're not thinking you can reuse the authentication value across multiple peers when encryption is in use, are you?
232 2016-11-23 20:16:29	0|BlueMatt|gmaxwell: no, but for current-style peers you can :p
233 2016-11-23 20:16:32	0|sipa|well i think cfields's design was based on the goal of being able to construct a message once, and then send it to multiple peers
234 2016-11-23 20:16:55	0|cfields|BlueMatt: fwiw, i had a discussion with sipa/gmaxwell about my grand plans for dedup, which, they made me realize doesn't make much sense
235 2016-11-23 20:17:15	0|BlueMatt|cfields: in any case, it seems massively layer-violating to have any knowledge of the structure of bytes-on-the-wire in CConnman
236 2016-11-23 20:17:32	0|sipa|BlueMatt: i don't see that
237 2016-11-23 20:17:52	0|BlueMatt|sipa: my impression is that the goal of CConnman is to be a network management library
238 2016-11-23 20:17:52	0|sipa|CConnman shouldn't know about how a block or inv message is structured
239 2016-11-23 20:17:59	0|BlueMatt|not a bitcoin-p2p-management library
240 2016-11-23 20:18:08	0|sipa|but it certainly should be able to construct and dissect network messages?
241 2016-11-23 20:18:13	0|BlueMatt|why?
242 2016-11-23 20:18:19	0|sipa|where else does that logic belong?
243 2016-11-23 20:18:29	0|BlueMatt|what if we want to change the header for encryption
244 2016-11-23 20:18:33	0|cfields|BlueMatt: at minimum, it has to have knowledge of when a message is complete
245 2016-11-23 20:18:41	0|BlueMatt|the logic belongs int he message-serialization/-deserialization logic
246 2016-11-23 20:18:41	0|cfields|well it doesn't have to, but for efficiency, it should
247 2016-11-23 20:18:48	0|gmaxwell|I really don't think it's worth the savings to conserve message hashing for the non-encrypted case, lets get everyone to encryption. :)
248 2016-11-23 20:18:54	0|BlueMatt|cfields: yes, the MessageDeserializer class thinggy we have now works great :)
249 2016-11-23 20:19:10	0|cfields|BlueMatt: which?
250 2016-11-23 20:19:44	0|gmaxwell|Serialize once is fine...
251 2016-11-23 20:19:51	0|BlueMatt|cfields: CNetMessage
252 2016-11-23 20:19:51	0|cfields|BlueMatt: wait... let's stop there. I was trying to avoid getting this far without answering the first question...
253 2016-11-23 20:20:00	0|cfields|<cfields> BlueMatt: so, before we end up in tangential discussions, i think the key thing to decide on is where encryption state should be held
254 2016-11-23 20:20:31	0|BlueMatt|cfields: right beside current protocol versioning info in CNode?
255 2016-11-23 20:20:57	0|sipa|i think we conceptually have 3 layers 1) network bytes 2) p2p messages 3) contents of p2p messages
256 2016-11-23 20:20:58	0|cfields|BlueMatt: that stuff is now ready to be extracted, just waiting on CNodeState lock cleanups
257 2016-11-23 20:21:02	0|BlueMatt|cfields: CNetMessage is an object which is going to have to be peer-specific
258 2016-11-23 20:21:06	0|sipa|i think BlueMatt is trying to separate 1 and 2
259 2016-11-23 20:21:10	0|sipa|cfields is separating 2 and 3
260 2016-11-23 20:21:15	0|BlueMatt|sipa: yes
261 2016-11-23 20:21:37	0|sipa|i'm of the opinion that separating 2 and 3 is the more interesting problem now
262 2016-11-23 20:21:42	0|BlueMatt|cfields: yes, I dont think it should be in CNode, if you want CNode to be a "dumb thing that CConnman handles"
263 2016-11-23 20:21:46	0|BlueMatt|but it should be in the same place
264 2016-11-23 20:22:08	0|BlueMatt|sipa: p2p messages and contents of p2p message/processing thereof seem already relatively separated, no?
265 2016-11-23 20:22:33	0|cfields|BlueMatt: imo, CNode should be just the stuff related to connection state. And current iv's fall into that category.
266 2016-11-23 20:22:52	0|BlueMatt|cfields: a corollary to your question is how does CNetMessage become peer-specific?
267 2016-11-23 20:22:53	0|sipa|so the question is whether CConnman should do the serialization of the contents of messages
268 2016-11-23 20:23:01	0|BlueMatt|cfields: "ivs"?
269 2016-11-23 20:23:30	0|sipa|if CNetMessage is peer-specific, it perhaps should be just hidden behind CConnman?
270 2016-11-23 20:23:42	0|sipa|like it currently is
271 2016-11-23 20:24:08	0|sipa|that makes reuse of messages across peers impossible, but maybe we're not interested if it can't be done at the same time as hashing/encrypting
272 2016-11-23 20:24:44	0|gmaxwell|for example, some peers might use compacted transactions some might not. But is that the correct layer for it?  I don't think we'd propose implementing BIP152 block compaction inside CConnman.
273 2016-11-23 20:24:53	0|cfields|sipa: to be specific, message reuse is possible, it's only the header attachment that can't be shared
274 2016-11-23 20:25:03	0|BlueMatt|sipa: the way i see it: processing of messages and the peer handling is already well-separated, or at least somewhat separated...we currently have bitcoin-specific and general-networking code intermixed in a bunch of places, and that also needs to be separated
275 2016-11-23 20:25:42	0|sipa|cfields: i mean, the whole point of 9128 is the introduction of a serializer abstraction... that seems unnecessary if we don't plan on ever reusing network messages
276 2016-11-23 20:26:31	0|sipa|cfields: while if it remains inside CConnman, it could do hashing/encryption on the fly during serialization
277 2016-11-23 20:26:34	0|BlueMatt|sipa: the serializer-abstraction in 9128 is definitely useable even if we encrypt network messages, etc
278 2016-11-23 20:26:34	0|cfields|sipa: it's necessary as a layer abstraction... it's only a shortcut for callers
279 2016-11-23 20:26:42	0|BlueMatt|sipa: see my branch for early-sending-compact-blocks
280 2016-11-23 20:27:09	0|sipa|cfields: i guess i don't see why
281 2016-11-23 20:27:52	0|cfields|BlueMatt: fwiw, my first version used a shared_ptr for dedup, and cached the hash, so that the same message could be pushed to a bunch of peers without re-serializing or re-hashing. If you think that's worth putting back, i can.
282 2016-11-23 20:28:17	0|cfields|heh, i'm not sure which conversation to be having here :)
283 2016-11-23 20:28:21	0|BlueMatt|cfields: why shared_ptrs? that seems strange to me, but, then, I havent seen that version
284 2016-11-23 20:28:26	0|BlueMatt|cfields: heh, we have fractured....
285 2016-11-23 20:28:55	0|sipa|cfields: oops, i forgot the distinction between CSerializedNetMsg and CNetMsgMaker, ignore me
286 2016-11-23 20:29:41	0|cfields|sipa: right. CSerializedNetMsg is intended to be a dumb payload. CConnman wraps it as necessary, then sends it with no knowledge of the contents. That's the layer separation change.
287 2016-11-23 20:30:32	0|sipa|but i don't think we answered the question whether CNetMsgMaker should be node-specific or not
288 2016-11-23 20:30:43	0|sipa|if we want it to do encryption on the fly, it should be
289 2016-11-23 20:30:55	0|sipa|but that reintroduces the dependency on cconnman
290 2016-11-23 20:31:14	0|cfields|sipa: it's only send-version specific, since that's all the serializers care about
291 2016-11-23 20:31:16	0|BlueMatt|sipa: i mean thats the whole point of the discussion - is the thing we pass into CConnman a blind bytes array to send on the wire, or a serialized message, which needs its header attached, etc
292 2016-11-23 20:31:52	0|BlueMatt|cfields: note that the way it currently is it mixes things...CNetMsgMaker adds most of the header, CConnman adds the hash
293 2016-11-23 20:31:54	0|sipa|my impression is that 9128 intends CSerializedNetMsg to be the unencrypted thing, which is reusable across connections
294 2016-11-23 20:32:00	0|BlueMatt|which seems like a layer-split
295 2016-11-23 20:32:06	0|cfields|so we're back to the original question again of who holds the current encryption state :)
296 2016-11-23 20:32:27	0|BlueMatt|cfields: first question - where are we putting current protocol version?
297 2016-11-23 20:32:30	0|cfields|sipa: yes
298 2016-11-23 20:32:44	0|BlueMatt|cfields: it was my impression you were planning on pulling that out of CNode and putting it in the State() stuff?
299 2016-11-23 20:32:54	0|sipa|cfields: but it somehow still has an optimization of having padding space preallocated
300 2016-11-23 20:33:03	0|sipa|cfields: which does not technically belong in that layer
301 2016-11-23 20:33:06	0|cfields|BlueMatt: yes
302 2016-11-23 20:33:25	0|sipa|cfields: hmm, but the State should be processing-module specific
303 2016-11-23 20:33:34	0|BlueMatt|cfields: I believe encyption state should go right there...encryption state is something negotiated on connect, just like nVersion, and should be in the same layer
304 2016-11-23 20:33:39	0|BlueMatt|sipa: it is
305 2016-11-23 20:33:59	0|sipa|BlueMatt: disagree there - i don't think every network message handler should reimplement encryption
306 2016-11-23 20:34:01	0|cfields|sipa: agreed. That's intended to be a "max padding necessary". I added it to make BlueMatt happy with zero-copy, at the cost of having to stick it at the wrong layer
307 2016-11-23 20:34:02	0|sipa|it's a meta layer
308 2016-11-23 20:34:32	0|sipa|there can be one module that deals with encryption, and once it's done negotiating, tells connman that from that point on encrypt all messages, including those from other modules
309 2016-11-23 20:34:34	0|BlueMatt|sipa: " every network message handler should reimplement encryption" huh?
310 2016-11-23 20:34:55	0|sipa|BlueMatt: my view is that there will be separate message handling modules, each with their own lock and their own state
311 2016-11-23 20:35:07	0|sipa|BlueMatt: one from ping/pong, one for block processing, one for tx relay, ...
312 2016-11-23 20:35:14	0|cfields|BlueMatt: i have a proposal for bip151 that would make it so that encryption isn't reliant on the messaging processing layer, but i haven't written it up yet
313 2016-11-23 20:35:21	0|sipa|BlueMatt: and they install handlers for specific messages
314 2016-11-23 20:35:45	0|BlueMatt|sipa: ok, and the place they hook in the handlers does the encryption, but certainly not CConnman
315 2016-11-23 20:35:46	0|sipa|BlueMatt: but all of their sent messages should get encrypted, not only those from the encryption module
316 2016-11-23 20:36:24	0|sipa|BlueMatt: i think you're envisioning a global processing layer in between the specific message processing and CConnman?
317 2016-11-23 20:36:25	0|BlueMatt|sipa: what if we want to use CConnman for some other protocol - does it now have to have the same encryption protocol?
318 2016-11-23 20:36:46	0|BlueMatt|sipa: CConnman should be a networking library, not a bitcoin networking library
319 2016-11-23 20:37:08	0|sipa|BlueMatt: heh, i thought that the networkling library would be libevent or some lower level wrapper
320 2016-11-23 20:37:11	0|BlueMatt|sipa: I mean right now its that way - the net message maker thing is a library that the processing modules all use
321 2016-11-23 20:37:17	0|sipa|CConnman is exactly the bitcoin specific thing
322 2016-11-23 20:37:30	0|sipa|cfields: opinion?
323 2016-11-23 20:37:34	0|cfields|BlueMatt: that was my original intent. But it became clear as the months went on that tighter integration is needed...
324 2016-11-23 20:37:51	0|cfields|BlueMatt: if we could get this stuff merged, we could move on to ripping out the lower-level stuff :)
325 2016-11-23 20:38:08	0|cfields|after this comes the libevent+async changes, which is maybe closer to what you want to see changed?
326 2016-11-23 20:38:14	0|sipa|BlueMatt: i would think that the separation you're talking about is maybe a later step... if we first want a net protocol, it just gets integrated into CConnman
327 2016-11-23 20:38:19	0|BlueMatt|cfields: well I dont think you mean that tighter-integration is needed between bitcoin logic and libevent/async
328 2016-11-23 20:38:42	0|BlueMatt|but clearly there is some integration needed where the net_processing stuff is aware of how to serialize a message
329 2016-11-23 20:38:48	0|BlueMatt|or at least has an api like CNetMessageMaker
330 2016-11-23 20:39:15	0|cfields|BlueMatt: yes, that's another possibility. One that sipa suggested a while back, i think
331 2016-11-23 20:39:29	0|BlueMatt|cfields: what is?
332 2016-11-23 20:39:42	0|cfields|callers could pass in abstract classes to handle the message wrapping (header)
333 2016-11-23 20:40:45	0|BlueMatt|cfields: ultimately I think there /are/ clearly three layers here, as sipa said, just a quesition of how we split them up....I think for now its much easier to dump things in net_processing and make CConnman just a "dumb pipe" or at least "semi-dumb-pipe"
334 2016-11-23 20:41:00	0|BlueMatt|because there are lots of changes that need to happen in just the network-library layer
335 2016-11-23 20:41:23	0|cfields|BlueMatt: for context, I first wrote a completely low-level, no-bitcoin network layer. Plugging that into bitcoin proved to require a bunch of bitcoin-specific hacks. In the end, I didn't think it was an improvement over what we had.
336 2016-11-23 20:42:19	0|BlueMatt|cfields: bitcoin-specific hacks like what?
337 2016-11-23 20:42:40	0|sipa|BlueMatt: in your hypothetical new network protocol, are there still network messages of the form (12-byte-command, data) that are dealt with in the same way in main etc?
338 2016-11-23 20:42:44	0|kanzure|was the networking library you were working on thrown out, cfields?
339 2016-11-23 20:43:28	0|BlueMatt|sipa: hmm? I dunno, do we need to stick to the same header format in 151?
340 2016-11-23 20:43:36	0|BlueMatt|sipa: I dont think we should be restricted to that?
341 2016-11-23 20:43:59	0|cfields|kanzure: not thrown out, but i kinda decided to converge with our current stuff instead
342 2016-11-23 20:44:01	0|BlueMatt|(plus I strongly thing having additional networking libraries would be welcome)
343 2016-11-23 20:44:01	0|sipa|i see bip151 simply as a change to the pipe, not to the messages in it
344 2016-11-23 20:44:25	0|cfields|BlueMatt: there's a strong reason to keep the header size, imo
345 2016-11-23 20:44:41	0|sipa|and i think CConnman is what is common between all network protocols that are just pipes for delivering messages of that form
346 2016-11-23 20:44:47	0|BlueMatt|sipa: indeed, but are message headers a part of the pipe, or the messages?
347 2016-11-23 20:44:48	0|cfields|though i'm aware there are plenty of drawbacks, and i wouldn't be heartbroken if it were abandoned
348 2016-11-23 20:44:55	0|sipa|BlueMatt: messages
349 2016-11-23 20:45:08	0|BlueMatt|sipa: well putting them in CConnman is pipe....
350 2016-11-23 20:45:09	0|sipa|BlueMatt: or you'd need to change every message handler too when the pipe changes
351 2016-11-23 20:46:26	0|BlueMatt|(note that clearly the headers wont all be the same - bip 151 will at least change the definition of the hash field)
352 2016-11-23 20:46:38	0|cfields|BlueMatt: i suppose we're looking at this from different sides. I originally rewrote the net layer and tried to plug it into bitcoin. I didn't like the result. I started instead trying to rip bitcoin out of our net layer, and that's where we are now. The next step would be to split the resulting CConnman into high and low levels. That's next.
353 2016-11-23 20:46:53	0|sipa|BlueMatt: i see messages as (12-byte-command, payload) things... that excludes the checksum
354 2016-11-23 20:47:12	0|sipa|bip151 changes the pipe, but the things being sent over it are still the same messages
355 2016-11-23 20:48:14	0|cfields|sipa: note that if it remains with start bytes (different ones) up front, and padded somehow to 24 bytes, the net layer can handle decryption without having to count on the messaging layer for initialization
356 2016-11-23 20:48:19	0|BlueMatt|cfields: ok, well if CConnman is gonna split, and the hashing/encryption/etc go in the high level, thats ok, but then CNetMsgMaker has to turn into what sipa said and have no knowledge of message-header
357 2016-11-23 20:49:03	0|BlueMatt|(which I think is an easy change from 9128 as it stands now?)
358 2016-11-23 20:50:05	0|cfields|BlueMatt: no knowledge as-in it's done in CConnman?
359 2016-11-23 20:50:28	0|BlueMatt|cfields: its done in CConnman-high
360 2016-11-23 20:50:34	0|BlueMatt|:p
361 2016-11-23 20:50:47	0|sipa|i think it's fine for CSerializedMessage to have some padding space before/after for undefined purposes, which CConnman(-high) just happens to conveniently use
362 2016-11-23 20:51:05	0|BlueMatt|sipa: I see no advantage to doing so?
363 2016-11-23 20:51:07	0|cfields|right, that was only added for the zero-copy case
364 2016-11-23 20:51:18	0|sipa|BlueMatt: not needing to copy it into a single buffer afterwards
365 2016-11-23 20:51:31	0|BlueMatt|cfields: I dont think you need that? CConnman-low takes a list of vectors...just give it two instead of one
366 2016-11-23 20:51:34	0|BlueMatt|its still zero-copy
367 2016-11-23 20:51:45	0|sipa|hmm
368 2016-11-23 20:51:48	0|sipa|i guess
369 2016-11-23 20:52:09	0|cfields|alternatively we could just send twice, that was an original TODO to investigate
370 2016-11-23 20:52:14	0|cfields|BlueMatt: is that what you're suggesting?
371 2016-11-23 20:52:30	0|BlueMatt|cfields: "send twice"?
372 2016-11-23 20:53:01	0|cfields|2 send() calls. or a sendmsg.
373 2016-11-23 20:53:18	0|BlueMatt|yes
374 2016-11-23 20:53:19	0|BlueMatt|thats ok
375 2016-11-23 20:53:27	0|BlueMatt|(and not performance-busting)
376 2016-11-23 20:53:31	0|cfields|(i'm pretty sure libevent uses scatter/gather anyway, so that should be freeish)
377 2016-11-23 20:53:44	0|BlueMatt|even two syscalls wont be performance-bustingn
378 2016-11-23 20:53:46	0|BlueMatt|ng
379 2016-11-23 20:53:59	0|BlueMatt|well, do we still disable nagle?
380 2016-11-23 20:54:02	0|BlueMatt|will have to look into that
381 2016-11-23 20:54:06	0|cfields|yes
382 2016-11-23 20:54:19	0|BlueMatt|have to use the MSG_MORE thing for the header, then
383 2016-11-23 20:54:19	0|sipa|use writev(2) :)
384 2016-11-23 20:54:20	0|cfields|BlueMatt: i was more concerned about possible awkwardness if you managed to push out exactly only the header
385 2016-11-23 20:54:45	0|BlueMatt|before what? the connection dies? so what?
386 2016-11-23 20:55:19	0|sipa|i do like this idea... just remove the message padding optimization out of CSerializedMessage, and use separate writes for header/...
387 2016-11-23 20:55:40	0|cfields|ok, i figured someone would complain about that :)
388 2016-11-23 20:55:45	0|cfields|easy, then
389 2016-11-23 20:56:33	0|cfields|i'm not going to bother with writev/sendmsg though, since libevent will handle that internally
390 2016-11-23 20:56:42	0|BlueMatt|thats fine
391 2016-11-23 20:56:57	0|BlueMatt|if you switch to libevent and I run an strace and you're wrong, I'll be upset, though :p
392 2016-11-23 20:57:04	0|cfields|heh
393 2016-11-23 20:57:30	0|cfields|BlueMatt: so to be clear, the next steps are to begin making things async-friendly. I have a branch which adds OnConnect(), OnDisconnect(), etc
394 2016-11-23 20:57:43	0|BlueMatt|cfields: cool, sounds good :)
395 2016-11-23 20:58:00	0|cfields|once we're there, we can begin the separation, and have a virtual class (high level) that responds to those things
396 2016-11-23 20:58:16	0|BlueMatt|sounds good
397 2016-11-23 20:58:18	0|sipa|cfields: yay
398 2016-11-23 20:58:21	0|sipa|concept ack
399 2016-11-23 20:59:42	0|cfields|in the end, i think it will end up looking much like my original lib. We'll just have gone from core->lib rather than the other way around.
400 2016-11-23 21:00:52	0|cfields|anyway, after this PR, I think we can segregate the files, and no one will have to concern themselves with the churn in the net code anymore :)
401 2016-11-23 21:01:19	0|BlueMatt|cfields: yay, then I can split main :)
402 2016-11-23 21:01:55	0|cfields|BlueMatt: heh, if we've done this right, we can do them in parallel without stomping on eachother :)
403 2016-11-23 21:02:21	0|BlueMatt|cfields: nailed the timing too...you'll get your last change into main a week before it splits :p
404 2016-11-23 21:02:36	0|cfields|hehe
405 2016-11-23 21:03:13	0|cfields|BlueMatt: so to clarify, drop the pad hack, send() twice, and you're happy with the rest?
406 2016-11-23 21:03:59	0|BlueMatt|cfields: yea, I'm ok with it as a next-step :)
407 2016-11-23 21:04:33	0|cfields|BlueMatt: ok, thanks.
408 2016-11-23 21:04:39	0|cfields|and thanks sipa as well
409 2016-11-23 21:10:36	0|sipa|great
410 2016-11-23 21:48:11	0|cfields|BlueMatt: i'm not familiar enough, so maybe you would know... any risk of the SyncTransaction having an effect on the downstream listeners? zmq, in particular
411 2016-11-23 21:48:23	0|cfields|*SyncTransaction change
412 2016-11-23 21:50:45	0|cfields|oh nevermind, i guess they don't actually see any difference
413 2016-11-23 22:29:29	0|BlueMatt|cfields: yea, point is now the orphan map is limited to p2p logic now :)
414 2016-11-23 22:52:14	0|cfields|BlueMatt: ready for the big move after #8930? Or am I missing one?
415 2016-11-23 22:52:15	0|gribble|https://github.com/bitcoin/bitcoin/issues/8930 | Move orphan processing to ActivateBestChain by TheBlueMatt · Pull Request #8930 · bitcoin/bitcoin · GitHub
416 2016-11-23 22:52:56	0|cfields|ah nm, #9183
417 2016-11-23 22:52:57	0|gribble|https://github.com/bitcoin/bitcoin/issues/9183 | Final Preparation for main.cpp Split by TheBlueMatt · Pull Request #9183 · bitcoin/bitcoin · GitHub
418 2016-11-23 23:06:35	0|BlueMatt|cfields: yea, ready after 9183
419 2016-11-23 23:15:41	0|dhill|addrmap.cpp, in CAddrInfo::GetChance(), nSinceLastSeen is unused...
420 2016-11-23 23:27:31	0|fubu|ssij
421 2016-11-23 23:36:47	0|fubu|xD
422 2016-11-23 23:43:39	0|rusty|Anyone thought about removing the dust limit?  In a feemarket world, it seems a little useless, and for things like lightning it complicates things (we have to always keep everything above the dust limit otherwise risk holding a non-standard tx).
423 2016-11-23 23:44:49	0|gmaxwell|If we consistently were operating in a functioning market I'd agree strongly.
424 2016-11-23 23:45:34	0|gmaxwell|Unfortunately there are many blocks that aren't and so they'd mine huge wads of dust advertisment payments sitting around.  I dunno if segwit will make this better or worse. (increases utxo costs, but also increases capacity.)
425 2016-11-23 23:45:45	0|gmaxwell|rusty: so "someday hopefully soon"?
426 2016-11-23 23:46:12	0|gmaxwell|already one large miner bypasses it, so perhaps we should give up.
427 2016-11-23 23:47:03	0|gmaxwell|here is a week of "mempool fees available" immediately before and immediately after a block, two-ish weeks ago: https://people.xiph.org/~greg/temp/fee_avail.png
428 2016-11-23 23:47:15	0|gmaxwell|you can see there are many periods of effectively no backpressure.
429 2016-11-23 23:48:37	0|rusty|gmaxwell: makes sense.  In practice, lightning has a "don't bother making any outputs below this" for economic reasons, but version 0 might have skipped that if not for the standardness issue.
430 2016-11-23 23:48:41	0|rusty|gmaxwell: nice graph!
431 2016-11-23 23:49:15	0|gmaxwell|here is a few days before that graph, where there was some nicer behavior: https://people.xiph.org/~greg/temp/fee_avail2.png
432 2016-11-23 23:50:17	0|rusty|gmaxwell: it would be nice to have these nicely generated and running constantly somewhere, BTW.  Did you have to hack bitcoind to get the data?
433 2016-11-23 23:50:25	0|fubu|u
434 2016-11-23 23:50:48	0|gmaxwell|$ cat fun.sh
435 2016-11-23 23:50:48	0|gmaxwell|rusty: no, set the target blocksize to 1mb... run
436 2016-11-23 23:50:49	0|gmaxwell|echo `date -u +%s` `./bitcoin-cli getblockcount`  `./bitcoin-cli getblocktemplate | grep \"fee\" | awk '{aa+=$2} END {print aa}'`
437 2016-11-23 23:50:52	0|gmaxwell|every couple seconds.
438 2016-11-23 23:51:17	0|gmaxwell|I have it running on my desktop but I keep disrupting it to tinker with things.. though the disruptions are less disruptive now that we have mempool saving in master.
439 2016-11-23 23:51:58	0|gmaxwell|the graph is just a little bit of postprocessing that data to get the before/after numbers for every block change.
440 2016-11-23 23:52:44	0|gmaxwell|sipa was working on some patches that would give a "CDF" like report for the mempool. (size vs fees) multiple blocks out.
441 2016-11-23 23:53:09	0|gmaxwell|one nice thing about this data is you can see all the miners that aren't doing CPFP mining yet... when they produce blocks that pay a lot less fee than my node would produce.