1 2016-11-16 00:19:24	0|thrasher`|thanks wumpus & sdaftuar :)
  2 2016-11-16 05:47:53	0|achow101|do any of the rpcs return how many blocks have signalled support for a soft fork?
  3 2016-11-16 05:48:12	0|sipa|getblockchaininfo
  4 2016-11-16 05:48:19	0|sipa|though i'm not sure it supports bip9
  5 2016-11-16 05:48:57	0|achow101|it only gives a "since" block, not actual counts
  6 2016-11-16 06:18:00	0|sipa|achow101: http://bitcoin.sipa.be/ver9-10k.png
  7 2016-11-16 06:18:21	0|sipa|it's not counted exactly according to consensus
  8 2016-11-16 08:15:10	0|wumpus|bitcoin: where even the configure scripts yell at you
  9 2016-11-16 08:19:24	0|wumpus|you think the build system is agressive? We'll introduce you to some of our community members :p
 10 2016-11-16 08:21:35	0|sipa|we have this guy called Travis who continuously yells at very active developers
 11 2016-11-16 08:22:13	0|wumpus|which reminds me, there is this guy called 'Coveralls' in some projects, maybe we should invite him too
 12 2016-11-16 08:22:17	0|luke-jr|lol
 13 2016-11-16 08:22:49	0|wumpus|I do think he's a bit on the spammy side tho, like our old pulltester
 14 2016-11-16 08:24:19	0|jonasschnelli|It someone encourages contributors to write more tests (even if coverage does not prove correctness)
 15 2016-11-16 08:27:07	0|wumpus|yeah coverage is especially good in languages such as python where many typos and such are only found by executing some code
 16 2016-11-16 08:27:23	0|gmaxwell|speaking of yelling, would people be opposed to me adding a ifdef check so that if daemon is not either defined or failed to be detected the compiler errors out?  I keep running into not-supported-on-your-OS when switching between master and 0.13. :) That fact that it completes the build and only fails at runtime is annoying (esp because the build takes a long time on my laptop)
 17 2016-11-16 08:27:46	0|wumpus|though it helps for C too I guess, having coverage in the first place is essential for complete tests, just not sufficient
 18 2016-11-16 08:28:15	0|gmaxwell|C++ makes smart coverage harder because all the boilerplate stuff often adds unreachable code. :(
 19 2016-11-16 08:29:03	0|wumpus|gmaxwell: no problem
 20 2016-11-16 08:29:32	0|wumpus|though you *really* should make a habit of re-running autogen when changing branches
 21 2016-11-16 08:30:33	0|wumpus|also saved me from committing a patch against the wrong branch at least once :)
 22 2016-11-16 08:31:02	0|gmaxwell|libsecp256k1's tests are something like 99.2% line coverage, 95.3% branch coverage. ... a few branches can't be reached without solving intractable crypto problems. alas. :(
 23 2016-11-16 08:31:27	0|gmaxwell|wumpus: I figure if it keeps hitting me it will hit random users who find it more surprising. :)
 24 2016-11-16 08:32:39	0|wumpus|I wish autoconf would just yell if your generated files are out of date with the build system files
 25 2016-11-16 08:32:47	0|jonasschnelli|For C: high coverage together with a CI valgrind memleak check is desirable IMO
 26 2016-11-16 08:33:10	0|wumpus|and fuzzing, of course
 27 2016-11-16 08:33:30	0|wumpus|gmaxwell: anyhow, no problem with adding that, it'd also catch problems with people forgetting to include configure.h
 28 2016-11-16 08:33:37	0|gmaxwell|memleak? seldom have memleaks in interesting code. Valgrind's undefined behavior checking, OTOH. Is super great. :)
 29 2016-11-16 08:33:47	0|gmaxwell|wumpus: thanks okay, I'll do so.
 30 2016-11-16 08:34:57	0|gmaxwell|for bitcoin I do think we need DRD run on it more often, it's just a bit frustrating because its so slow.
 31 2016-11-16 08:35:32	0|wumpus|at least make sure you run valgrind against the optimized executable not the -O0 debugging one :-)
 32 2016-11-16 08:36:52	0|sipa|http://bitcoin.sipa.be/ver9-10k.png
 33 2016-11-16 08:36:55	0|wumpus|but yes valgrind is slow in any case, though it's surprisingly fast if you realize what it does in the background, converting executed code blocks to VEX and back to machine instructions on the fly
 34 2016-11-16 08:37:08	0|gmaxwell|yes, it's amazing.
 35 2016-11-16 08:37:11	0|sipa|(-ever, -50k, -2k also exist)
 36 2016-11-16 08:37:53	0|wumpus|sipa: nice!
 37 2016-11-16 08:39:06	0|TD-Linux|sipa, having them on the index page would also be nice :)
 38 2016-11-16 08:39:26	0|gmaxwell|there is an index page?
 39 2016-11-16 08:39:38	0|sipa|TD-Linux: that would involve loading my html knowledge back from swap space
 40 2016-11-16 08:40:19	0|sipa|gmaxwell: http://bitcoin.sipa.be
 41 2016-11-16 08:41:06	0|sipa|did we cross 2 exahash/s?
 42 2016-11-16 08:41:59	0|jonasschnelli|sipa: come one! s/hash rate</a></li>/hash rate</a></li><li class="hilight"><a accesskey="h" href="http://bitcoin.sipa.be/ver9-10k.png">ver9-10k</a></li>/
 43 2016-11-16 08:42:24	0|jonasschnelli|*come on :-)
 44 2016-11-16 08:42:27	0|sipa|jonasschnelli: i'll want a separate page with thumbnails and stuffs
 45 2016-11-16 08:42:37	0|wumpus|yes I was about to say, someone has got to have that knowledge in their ready memory and could provide a patch
 46 2016-11-16 08:43:06	0|sipa|https://github.com/sipa/bitcoin-stats
 47 2016-11-16 08:43:16	0|jonasschnelli|Yeah. Migrate the non-pngs html stuff to bitcoincore.org
 48 2016-11-16 08:43:45	0|wumpus|not me though, my html knowledge is in the 2000-2004 archive in deep storage
 49 2016-11-16 08:43:46	0|jonasschnelli|Yeah. Perl. How did I missed this.
 50 2016-11-16 08:44:08	0|sipa|also C
 51 2016-11-16 08:44:11	0|sipa|and bash
 52 2016-11-16 08:44:19	0|sipa|and bash that generates gnuplot
 53 2016-11-16 08:46:15	0|sipa|;;nethash
 54 2016-11-16 08:46:16	0|gribble|2049301157.56
 55 2016-11-16 08:50:47	0|CubicEarth|13.1, Ubuntu.... my client is passing transaction data, synced several dozen blocks on startup, but then stalled for almost 10 minutes before downloading the most recent blocks. Is it just that none of the connected peers were providing the blocks it needed?  Or is it something else?
 56 2016-11-16 08:51:27	0|sipa|how many blocks do you have in total?
 57 2016-11-16 08:51:48	0|CubicEarth|I synced it this morning, so it was only about 10 hours behind... then it stalled at about 6 hours behind
 58 2016-11-16 08:52:15	0|CubicEarth|chewed though the first 4 or 5 hours in 30 seconds
 59 2016-11-16 08:52:49	0|sipa|if you had an abnormal shutdown before, that's normal
 60 2016-11-16 08:53:04	0|sipa|well, not normal, but known
 61 2016-11-16 08:53:49	0|CubicEarth|interesting.  Would just one abnormal shutdown ever be enough?  The most recent one was normal (so far as I could tell)
 62 2016-11-16 08:54:01	0|sipa|just the last one
 63 2016-11-16 08:54:14	0|sipa|it was busy processing the blocks it had on disk but not applied to the database, and does that in the background while connecting to other peers, ignoring their block announcements
 64 2016-11-16 08:54:19	0|sipa|s/was/would be/
 65 2016-11-16 08:54:36	0|gmaxwell|an unclean shutdown will delay the initial headers fetch?
 66 2016-11-16 08:55:07	0|sipa|no, it will cause skipping it
 67 2016-11-16 08:55:10	0|sipa|we should fix that
 68 2016-11-16 08:59:43	0|CubicEarth|I wasn't looking at cpu usage... but in the debug window the current number of blocks was not advancing either. I thought the 'current number of blocks' displayed the total that had been validated and added to the chain. Does it actually count them before they are fully processed and appended?
 69 2016-11-16 09:01:48	0|bitcoin-git|13bitcoin/06master 1470266e9 15Cory Fields: build: fix qt5.7 build under macOS...
 70 2016-11-16 09:01:48	0|bitcoin-git|[13bitcoin] 15laanwj pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/6eeac6e30d65...918ea16dc08d
 71 2016-11-16 09:01:49	0|bitcoin-git|13bitcoin/06master 14918ea16 15Wladimir J. van der Laan: Merge #9169: build: fix qt5.7 build under macOS...
 72 2016-11-16 09:02:03	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #9169: build: fix qt5.7 build under macOS (06master...06fix-objcxx-std) 02https://github.com/bitcoin/bitcoin/pull/9169
 73 2016-11-16 09:02:41	0|sipa|CubicEarth: no
 74 2016-11-16 09:03:13	0|sipa|CubicEarth: my theory is that your node downloaded a number of blocks, and stored them on disk, but didn't apply them to the main chain before crashing
 75 2016-11-16 09:03:32	0|sipa|after restarting, it noticed those blocks, and applied them immediately in the background
 76 2016-11-16 09:04:46	0|bitcoin-git|13bitcoin/06master 14fa80ef8 15MarcoFalke: [qa] proxy_test: Calculate hardcoded port numbers instead
 77 2016-11-16 09:04:46	0|bitcoin-git|[13bitcoin] 15laanwj pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/918ea16dc08d...4333b1c4ea74
 78 2016-11-16 09:04:47	0|bitcoin-git|13bitcoin/06master 144333b1c 15Wladimir J. van der Laan: Merge #9151: [qa] proxy_test: Calculate hardcoded port numbers...
 79 2016-11-16 09:05:01	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #9151: [qa] proxy_test: Calculate hardcoded port numbers (06master...06Mf1611-qaPort) 02https://github.com/bitcoin/bitcoin/pull/9151
 80 2016-11-16 09:07:03	0|CubicEarth|I'll watch cpu utilization next time I boot my node... I think this pattern of stalling has been happening often.
 81 2016-11-16 09:07:28	0|gmaxwell|CubicEarth: do you use p2pool by any chance?
 82 2016-11-16 09:08:01	0|CubicEarth|gmaxwell: No. I'm not mining.
 83 2016-11-16 09:08:23	0|gmaxwell|okay, p2pool often ends up gobbling the initial headers sync attempt for me.
 84 2016-11-16 09:10:20	0|bitcoin-git|[13bitcoin] 15laanwj pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/4333b1c4ea74...62af16463833
 85 2016-11-16 09:10:21	0|bitcoin-git|13bitcoin/06master 14079142b 15Jonas Schnelli: fNetworkActive is not protected by a lock, use an atomic
 86 2016-11-16 09:10:21	0|bitcoin-git|13bitcoin/06master 1462af164 15Wladimir J. van der Laan: Merge #9131: fNetworkActive is not protected by a lock, use an atomic...
 87 2016-11-16 09:10:30	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #9131: fNetworkActive is not protected by a lock, use an atomic (06master...062016/11/net_toggle) 02https://github.com/bitcoin/bitcoin/pull/9131
 88 2016-11-16 09:12:04	0|bitcoin-git|13bitcoin/06master 1479f755d 15Alex Morcos: Unset fImporting for loading mempool
 89 2016-11-16 09:12:04	0|bitcoin-git|[13bitcoin] 15laanwj pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/62af16463833...434e683f7be6
 90 2016-11-16 09:12:05	0|bitcoin-git|13bitcoin/06master 14434e683 15Wladimir J. van der Laan: Merge #9133: Unset fImporting for loading mempool...
 91 2016-11-16 09:12:20	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #9133: Unset fImporting for loading mempool (06master...06noImportLoadMempool) 02https://github.com/bitcoin/bitcoin/pull/9133
 92 2016-11-16 09:17:57	0|CubicEarth|separate question: does anyone know about a standard for multi-sig interoperability? Something so that wallets from different providers could talk to each other?
 93 2016-11-16 09:25:38	0|wumpus|CubicEarth: this is not a channel for general bitcoin related questions, please use #bitcoin
 94 2016-11-16 09:28:09	0|wumpus|I know of no such standard, though.
 95 2016-11-16 09:28:36	0|CubicEarth|wumpus: ok.  thanks.
 96 2016-11-16 09:30:49	0|bitcoin-git|[13bitcoin] 15jonasschnelli opened pull request #9171: Introduce out-of-band block requests (06master...062016/11/spv_step_1) 02https://github.com/bitcoin/bitcoin/pull/9171
 97 2016-11-16 09:33:52	0|wumpus|jonasschnelli: what is 'out of band' about out-of-band block requests? don't you mean something like 'asynchronous'?
 98 2016-11-16 09:34:29	0|jonasschnelli|wumpus: not sure if its the right term. But with out-of-band, I mean not-in-the-IBD-sequence...
 99 2016-11-16 09:34:32	0|wumpus|jonasschnelli: (just a question about terminology, I haven't seen 'out of band' used a lot except for some weird network protocols)
100 2016-11-16 09:34:43	0|gmaxwell|I had the same thought, fwiw.
101 2016-11-16 09:35:03	0|jonasschnelli|I see. Any better wordings?
102 2016-11-16 09:35:11	0|jonasschnelli|Asynchronous is probably also not ideal.
103 2016-11-16 09:35:23	0|jonasschnelli|Independent-block-requests?
104 2016-11-16 09:35:43	0|jonasschnelli|prioritized-block-downloads?
105 2016-11-16 09:35:57	0|gmaxwell|"Third distinct block downloading mechenism" :P   I would have called it 'unordered block fetching' perhaps.
106 2016-11-16 09:36:01	0|jonasschnelli|Though, its not always a download
107 2016-11-16 09:36:16	0|jonasschnelli|I like "unordered block fetching"
108 2016-11-16 09:36:43	0|gmaxwell|even that is confusing since the normal fetch isn't in strict order. :)
109 2016-11-16 09:37:03	0|jonasschnelli|Indeed...
110 2016-11-16 09:37:16	0|jonasschnelli|Seems to be hard to nail it in two or three words.
111 2016-11-16 09:37:24	0|wumpus|so this provides a separate interface to block downloading
112 2016-11-16 09:37:43	0|jonasschnelli|wumpus: not really,.. it still uses the internal block download mechanism
113 2016-11-16 09:38:01	0|jonasschnelli|It just priories individual blocks.
114 2016-11-16 09:38:44	0|wumpus|yes it uses the same mechanism, but provides an interface that that other parts of the software (not directly associated to validation) can use
115 2016-11-16 09:38:45	0|jonasschnelli|And you have certainty that all these requested blocks get passed through the signals in the correct order
116 2016-11-16 09:39:05	0|gmaxwell|Block on Demand.
117 2016-11-16 09:39:26	0|jonasschnelli|Blocks on Demand?
118 2016-11-16 09:39:43	0|jonasschnelli|Otherwise people think this blocks something. :P
119 2016-11-16 09:39:56	0|gmaxwell|My node has hot and cold running blocks.
120 2016-11-16 09:40:06	0|jonasschnelli|heh
121 2016-11-16 09:49:56	0|jonasschnelli|Someone mentioned auxiliary-block-requests?
122 2016-11-16 09:50:10	0|wumpus|sgtm
123 2016-11-16 09:52:09	0|bitcoin-git|13bitcoin/06master 14307acdd 15mrbandrews: [qa] add assert_raises_message to check specific error message
124 2016-11-16 09:52:09	0|bitcoin-git|[13bitcoin] 15MarcoFalke pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/434e683f7be6...0a6d48d9ed60
125 2016-11-16 09:52:10	0|bitcoin-git|13bitcoin/06master 140a6d48d 15MarcoFalke: Merge #9168: [qa] add assert_raises_message to check specific error message...
126 2016-11-16 09:52:22	0|bitcoin-git|[13bitcoin] 15MarcoFalke closed pull request #9168: [qa] add assert_raises_message to check specific error message (06master...06ba-assert-raises) 02https://github.com/bitcoin/bitcoin/pull/9168
127 2016-11-16 09:52:41	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #8747: [rpc] Fix transaction size comments and RPC help text. (06master...06rpc_comments) 02https://github.com/bitcoin/bitcoin/pull/8747
128 2016-11-16 10:06:24	0|bitcoin-git|13bitcoin/06master 1407ede5d 15Brian Deery: update comments for tx weight
129 2016-11-16 10:06:24	0|bitcoin-git|[13bitcoin] 15MarcoFalke pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/0a6d48d9ed60...cb2ed300a89e
130 2016-11-16 10:06:25	0|bitcoin-git|13bitcoin/06master 14cb2ed30 15MarcoFalke: Merge #9155: [trivial] update comments for tx weight...
131 2016-11-16 10:06:40	0|bitcoin-git|[13bitcoin] 15MarcoFalke closed pull request #9155: [trivial] update comments for tx weight (06master...06master) 02https://github.com/bitcoin/bitcoin/pull/9155
132 2016-11-16 10:33:58	0|bitcoin-git|[13bitcoin] 15laanwj opened pull request #9172: Resurrect pstratem's "Simple fuzzing framework" (06master...062016_11_fuzzing_framework) 02https://github.com/bitcoin/bitcoin/pull/9172
133 2016-11-16 10:51:49	0|BLTS|Hi all, I need to outsource a BTC payment and validation program!
134 2016-11-16 10:52:26	0|jonasschnelli|BLTS: what do you mean with outsource?
135 2016-11-16 10:55:29	0|BLTS|I need to pay someone to help develop a BTC payment and verification software
136 2016-11-16 10:56:03	0|BLTS|jonasschnelli  I need to pay someone to help develop a BTC payment and verification software
137 2016-11-16 10:56:28	0|jonasschnelli|BLTS: maybe #bitcoin or #bitcoin-dev but not #bitcoin-core-dev I guess
138 2016-11-16 10:57:07	0|BLTS|thank you
139 2016-11-16 11:09:38	0|wumpus|travis is having issues
140 2016-11-16 11:10:49	0|wumpus|(nothing we can help, it doesn't even manage apt-get)
141 2016-11-16 11:19:59	0|wumpus|phantomcircuit: do you happen to have your directory of AFL inputs for #7940 somewhere?
142 2016-11-16 11:20:01	0|gribble|https://github.com/bitcoin/bitcoin/issues/7940 | [WIP] Fuzzing framework by pstratem · Pull Request #7940 · bitcoin/bitcoin · GitHub
143 2016-11-16 11:22:24	0|[b__b]|You're doing good work, travis!
144 2016-11-16 11:22:24	0|MarcoFalke|!m travis
145 2016-11-16 11:22:25	0|gribble|Error: "m" is not a valid command.
146 2016-11-16 11:25:23	0|MarcoFalke|seems to work, now
147 2016-11-16 12:54:14	0|jonasschnelli|wumpus: yes. I encountered the travis issue on other repos too
148 2016-11-16 13:10:13	0|luke-jr|sigh, looks like Travis is totally broken
149 2016-11-16 13:14:36	0|wumpus|yep.
150 2016-11-16 13:45:09	0|jonasschnelli|Hmm... utf8 in not mandatory for JSON.. right?
151 2016-11-16 13:45:27	0|jonasschnelli|RFC 4627 does state: "JSON text SHALL be encoded in Unicode.  The default encoding is
152 2016-11-16 13:45:27	0|jonasschnelli|UTF-8."?
153 2016-11-16 13:54:45	0|luke-jr|I thought it was.. :x
154 2016-11-16 14:10:48	0|wumpus|yes, utf-8 is mandatory for JSON
155 2016-11-16 14:11:20	0|wumpus|and that is how everyone uses it. We're not going to support non-UTF-8 JSON in bitcoin core.
156 2016-11-16 14:17:16	0|wumpus|I'm 99% sure that the non-utf8 data he managed to get into the wallet database in #9166 is from the wx GUI, not from any JSON client
157 2016-11-16 14:17:17	0|gribble|https://github.com/bitcoin/bitcoin/issues/9166 | listtransactions returns invalid JSON when comment contain non-UTF8 special chars · Issue #9166 · bitcoin/bitcoin · GitHub
158 2016-11-16 14:18:15	0|wumpus|qt handles everything as UTF-8, but the wx GUI did not, the stable wx (at that point) didn't even support unicode IIRC
159 2016-11-16 14:18:35	0|timothy|who still uses wx?
160 2016-11-16 14:18:53	0|wumpus|no one,but wallets grandfathered in from that time still exist
161 2016-11-16 14:28:13	0|jonasschnelli|A fix would be to add a UTF8 filter at the deserialization of an CAccountingEntry
162 2016-11-16 14:28:37	0|jonasschnelli|Somewhere here: https://github.com/bitcoin/bitcoin/blob/master/src/wallet/wallet.h#L524
163 2016-11-16 14:28:48	0|jonasschnelli|But not going to do this. It's just not worth...
164 2016-11-16 14:29:44	0|wumpus|I understand. THis is better handled on a per-case basis probably
165 2016-11-16 14:30:41	0|luke-jr|wumpus: wxBitcoin didn't support the stable wx, and required unicode XD
166 2016-11-16 14:31:07	0|wumpus|luke-jr: ok. No clue then how he managed then :)
167 2016-11-16 14:31:24	0|luke-jr|did the RPC really prohibit it?
168 2016-11-16 14:31:49	0|wumpus|no, but no one uses JSON without unicode, the sane languages prohibit it
169 2016-11-16 14:31:56	0|luke-jr|CLI?
170 2016-11-16 14:32:17	0|wumpus|isn't cli usually unicode too?
171 2016-11-16 14:32:28	0|luke-jr|not always
172 2016-11-16 14:32:49	0|wumpus|no, not always, so it's possible, it'll just be very rare on all accounts
173 2016-11-16 14:33:14	0|luke-jr|I think it's more commonly non-Unicode outside the English-speaking area
174 2016-11-16 14:33:25	0|luke-jr|or was until some years ago
175 2016-11-16 14:33:46	0|luke-jr|but nobody else has reported a problem.. so maybe not
176 2016-11-16 14:34:01	0|wumpus|we're not talking about the 90's here
177 2016-11-16 14:34:32	0|wumpus|or original VT-100 terminals or whatnot :) bitcoin isn't that old
178 2016-11-16 14:35:03	0|wumpus|but yes it'll obviously be more common outside ENglish-speaking areas
179 2016-11-16 14:38:27	0|jonasschnelli|I located the "issue" in the code.
180 2016-11-16 14:38:43	0|jonasschnelli|Its in univalue_read.cpp
181 2016-11-16 14:38:50	0|jonasschnelli|Univalue can't handle non UTF8
182 2016-11-16 14:38:57	0|jonasschnelli|(even if its allowed by the JSON RFC)
183 2016-11-16 14:39:06	0|wumpus|that's fine, we don't want it to
184 2016-11-16 14:39:08	0|jonasschnelli|the read() function will resturn false
185 2016-11-16 14:39:14	0|jonasschnelli|Yes. We don't want this
186 2016-11-16 14:40:26	0|wumpus|univalue, as well as bitcoind, will acquire lots of cruft if you want full character set handling. We don't need that as no one uses JSON that way. E.g. the "JSON is a minefield" tests assume the JSON parser strictly checks UTF-8
187 2016-11-16 14:41:46	0|wumpus|(neither did any of the previous JSON libraries that we used, but they didn't do any input validation, which is quite a bad thing)
188 2016-11-16 15:03:47	0|bitcoin-git|[13bitcoin] 15laanwj reopened pull request #8747: [rpc] Fix transaction size comments and RPC help text. (06master...06rpc_comments) 02https://github.com/bitcoin/bitcoin/pull/8747
189 2016-11-16 15:11:38	0|bitcoin-git|[13bitcoin] 15demodun opened pull request #9173: demo (06master...06master) 02https://github.com/bitcoin/bitcoin/pull/9173
190 2016-11-16 15:12:23	0|bitcoin-git|[13bitcoin] 15jonasschnelli closed pull request #9173: demo (06master...06master) 02https://github.com/bitcoin/bitcoin/pull/9173
191 2016-11-16 15:43:57	0|cfields|morcos: ping. gentle reminder about the prevector bench.
192 2016-11-16 15:44:03	0|instagibbs|in what situations would ENABLE_WALLET be defined but pwalletMain be null? Or any other combination.
193 2016-11-16 15:45:11	0|cfields|instagibbs: ./configure --enable-wallet && ./bitcoind --disablewallet
194 2016-11-16 15:45:13	0|cfields|(i think)
195 2016-11-16 15:45:33	0|instagibbs|cfields, ah that seems likely. Thanks!
196 2016-11-16 19:19:31	0|theymos|fyi, some apparently-popular antivirus is flagging Bitcoin Core 0.13.1: https://www.virustotal.com/en/file/c1726ccc50635795c942c7d7e51d979c4f83a3d17f8982e9d02a114a15fef419/analysis/
197 2016-11-16 19:34:24	0|arubi|"Additional information" -> "VirusTotal metadata" -> "File names" -> "KMS V.1.2.3.exe"
198 2016-11-16 19:34:27	0|arubi|weird.
199 2016-11-16 19:58:08	0|morcos|cfields: sorry, will have to wait a bit longer.   my initial results seem to show that maybe its a little bit of an improvement, but that the previous branch was a more significant improvement
200 2016-11-16 19:58:20	0|morcos|so i'd like to run longer tests to get rid of the noise...
201 2016-11-16 19:59:30	0|cfields|morcos: ok, np. thanks for testing
202 2016-11-16 20:00:17	0|cfields|morcos: i suppose it's possible that all of the changes just slowly added up to a noticeable speedup. I figured it was more likely that there was a single silver bullet.
203 2016-11-16 20:01:46	0|gmaxwell|btcdrak: any interest in trying to get that spurrious AV warning fixed?
204 2016-11-16 20:03:48	0|Victorsueca|theymos: Baidu lol
205 2016-11-16 20:04:05	0|Victorsueca|isn't that a Chinese page?
206 2016-11-16 20:04:41	0|Victorsueca|would bet Chinese government is behind that
207 2016-11-16 20:16:34	0|morcos|cfields: yeah i'll keep running tests, but actually when i tried your old branch, i accidentally did it without the removal of the CScript copy constructor (b/c i didn't want to use the Transaction stuff in that second commit)
208 2016-11-16 20:17:17	0|morcos|once i re-removed the CScript copy contructor from the copy-move branch, its clearly much faster than the prevector-move branch
209 2016-11-16 20:19:44	0|cfields|morcos: interesting, that points to the speedup coming from less time copying
210 2016-11-16 20:19:56	0|btcdrak|gmaxwell: a job for a native chinese speaker I think.
211 2016-11-16 20:19:58	0|cfields|which is strange, because last i looked, there were very few of those copies
212 2016-11-16 20:21:36	0|cfields|morcos: hmm, though i suppose it could also come from construction/destruction. prevector as-is is pretty heavy on those
213 2016-11-16 20:22:34	0|cfields|grr, i should just bite the bullet and do a quick specialization of prevector for unsigned char. It should be simple enough.
214 2016-11-16 20:30:33	0|sipa|cfields: can't use use a my_is_trivially_copyable predicate class instead, and define it just manually for char for old compilers, and use std::is_trivially_copyable for new ones?
215 2016-11-16 20:31:55	0|cfields|sipa: yes, but i'm pretty sure it could be sped up substantially if written specifically for packed bytes
216 2016-11-16 20:33:16	0|cfields|sipa: for ex, no alignment concerns
217 2016-11-16 20:34:35	0|gmaxwell|maybe time would be better spent working on flat transactions (which rrequires finishing making them immutable first)?
218 2016-11-16 20:35:46	0|cfields|flat transactions?
219 2016-11-16 20:36:06	0|sipa|cfields: one malloc
220 2016-11-16 20:36:38	0|cfields|oh, accessing serialized fields on the fly, or so?
221 2016-11-16 20:37:05	0|gmaxwell|doesn't have to be (and probably shouldn't be) in the seralized form we use on the wire.
222 2016-11-16 20:37:55	0|cfields|right, you mentioned this the other day
223 2016-11-16 20:40:34	0|cfields|gmaxwell: you mentioned you'd doodled some notes about a possible format. Happen to have those around?
224 2016-11-16 20:42:24	0|gmaxwell|Well what I was working on is on the other side, improved serialization for disk and network that is more compact.
225 2016-11-16 20:43:14	0|gmaxwell|For use in memory, one would have no varints, for example, just just character arrays and primitive types, and a set of pointers to allow random access to every field even though some parts are variable length.
226 2016-11-16 20:44:39	0|sipa|doesn't sound hard
227 2016-11-16 20:45:01	0|sipa|concatenate all scripts into a single byte array, and have (begin_ptr, size) tuples to refer to them
228 2016-11-16 20:45:35	0|sipa|we'd need to modify the script interpreter to no longer use CScript anymore (which seems trivial, it doesn't need any of CScript's representation - even iteration is done byte by byte)
229 2016-11-16 20:45:36	0|gmaxwell|no, but it needs transaction to be immutable though.. since you can't make changes that would change the size or count of any scripts.
230 2016-11-16 20:45:44	0|sipa|of course
231 2016-11-16 20:46:27	0|gmaxwell|for the more effficent disk/wire format what I'd written up was https://people.xiph.org/~greg/compacted_txn.txt  though one thing it doesn't achieve, which would be nice though I don't know if its realistic, is a very fast procedure to decide how much memory would be needed for a flat in memory rep... which would facilitate some later optimizations.
232 2016-11-16 20:50:05	0|cfields|thanks, will take a look
233 2016-11-16 20:51:33	0|Chris_Stewart_5|Why does this test fail for EVAL_FALSE and not WITNESS_PROGRAM_MISMATCH? https://github.com/bitcoin/bitcoin/blob/master/src/test/data/script_tests.json#L1257
234 2016-11-16 20:51:34	0|cfields|yes, i suppose in any scheme where all scripts are cat'd in memory, there'd be no need for something like prevector
235 2016-11-16 20:55:04	0|sipa|cfields: sure there is. prevector's primary benefit is in CCoins
236 2016-11-16 20:57:29	0|gmaxwell|but thats why I suggested that flattening transactions may be a better time investment than expanding prevector's use in transactions. I think with transaction const there is no reason to not flatten it all the way except that a lot of code may need to be adjusted to twiddle how accesses work. (Though perhaps enough C++ magic could keep the interface almost identical-- beyond my pay grade).  Not
237 2016-11-16 20:57:35	0|gmaxwell|that prevector isn't useful, but just for const transactions its a half-step.
238 2016-11-16 21:00:00	0|cfields|sipa: i figured CCoins would use some similar monster allocation structure and indicies. But I suppose you'd run into crazy fragmentation issues quickly
239 2016-11-16 21:00:22	0|cfields|*indices
240 2016-11-16 21:00:27	0|sipa|cfields: for CCoins i want to move to a per-output model instead of per-tx
241 2016-11-16 21:06:16	0|sipa|Chris_Stewart_5: because 6e340b9cffb37a989ca544e6bb780a2c78901d3fb33738768511a30617afa01d is the hash of OP_0 (0x00) and not of OP_TRUE (0x51)
242 2016-11-16 21:06:42	0|sipa|Chris_Stewart_5: so it's a perfectly valid p2wsh output, for the OP_FALSE witness script (which is unspendable)
243 2016-11-16 21:07:20	0|cfields|makes sense, though admittedly i'm not very familiar with that code.
244 2016-11-16 21:07:50	0|sipa|cfields: it needs design, implementation, and benchmarking
245 2016-11-16 21:08:37	0|sipa|it's not trivial to do, but the current system has an ugly O(n^2) behaviour, where a transaction with n outputs needs O(n) database writes (one for each spend) each of size O(n) (because all unspent outputs are rewritten every time)
246 2016-11-16 21:09:18	0|cfields|ah, i see
247 2016-11-16 21:09:47	0|sipa|the easiest solution is storing height/coinbaseness in each output
248 2016-11-16 21:09:52	0|sipa|and txid
249 2016-11-16 21:09:57	0|sipa|but that's a lot of duplication
250 2016-11-16 21:10:53	0|sipa|though leveldb does deduplication of identical prefixes in the database
251 2016-11-16 21:11:18	0|sipa|an alternative is a txid->{height,coinbase,id} map, with id a short sequentially-assigned local id
252 2016-11-16 21:11:28	0|sipa|and then use (id,index)->{utxo} map for the utxos
253 2016-11-16 21:11:33	0|sipa|but that's an extra indirection
254 2016-11-16 21:16:53	0|cfields|well the first potentially brings a nice read speedup, no? Since the outputs are then somewhat sorted by hotness
255 2016-11-16 21:17:55	0|sipa|leveldb sorts by key
256 2016-11-16 21:18:40	0|sipa|and the way we're using leveldb i think hardly results in actual levels
257 2016-11-16 21:18:51	0|sipa|the batches are so large we always trigger compactions
258 2016-11-16 21:19:05	0|cfields|heh
259 2016-11-16 21:20:31	0|Chris_Stewart_5|sipa: Thanks for the help, mistakenly assumed it was still HASH160(SHA256()), is there a reason it is only 1 SHA256()?
260 2016-11-16 21:21:46	0|sipa|Chris_Stewart_5: yes, 160 bits is too little
261 2016-11-16 21:22:10	0|sipa|(it's too little for P2SH as well, but we weren't aware of the collision attack against it at the time)
262 2016-11-16 21:22:46	0|Chris_Stewart_5|sipa: Is it a HF to change P2SH to a SHA256?
263 2016-11-16 21:23:24	0|sipa|Chris_Stewart_5: of course
264 2016-11-16 21:24:48	0|sipa|when done naively, at least
265 2016-11-16 21:25:00	0|sipa|just defining something new P2SH-like could be done
266 2016-11-16 21:25:11	0|sipa|... but that's essentially what P2WSH is
267 2016-11-16 21:28:51	0|Chris_Stewart_5|and with a versioning system now we can totally redefine the semantics of any witness scripts between versions, correct?
268 2016-11-16 21:29:24	0|sipa|yes
269 2016-11-16 21:30:27	0|sipa|Chris_Stewart_5: my answer was wrong. any interpretation for "changing P2SH to SHA256" i can come up with would be a soft fork
270 2016-11-16 21:30:37	0|sipa|it would however also break any existing p2sh users
271 2016-11-16 21:31:25	0|Chris_Stewart_5|sipa: Isn't the P2SH flag a required flag at this point in script? Would it have to do with redefining the IsScriptHash function?
272 2016-11-16 21:32:14	0|sipa|that's an implementation detail
273 2016-11-16 21:33:37	0|sipa|you can just add an IsScriptHash2 function, and add a new flag that assigns sha256-p2sh behaviour to IsScriptHash2, and outlaws any older IsScriptHash results
274 2016-11-16 21:34:03	0|Chris_Stewart_5|Why would you need to outlaw?
275 2016-11-16 21:34:19	0|sipa|because that's how i interpret 'moving to sha256' :)
276 2016-11-16 21:34:27	0|Chris_Stewart_5|ahhh ok
277 2016-11-16 21:34:38	0|sipa|if you would have said 'adding p2sh-sha256 feature', yes :)
278 2016-11-16 21:34:48	0|sipa|but that's pretty much what p2wsh is... in a better way
279 2016-11-16 21:35:08	0|Chris_Stewart_5|... but I want to make p2sh great again. I'll see myself out.
280 2016-11-16 21:36:56	0|sipa|hahaha