1 2018-06-11 02:03:04	0|echeveria|2018-06-11 02:03:03.384975 Verifying last 3 blocks at level 3
  2 2018-06-11 02:03:04	0|echeveria|2018-06-11 02:03:23.676793 No coin database inconsistencies in last 4 blocks (6564 transactions)
  3 2018-06-11 02:03:08	0|echeveria|off by one?
  4 2018-06-11 02:33:25	0|sipa|echeveria: possibly!
  5 2018-06-11 04:17:47	0|kallewoof|Looks like it checks one more block than suggested. `if (pindex->nHeight < chainActive.Height()-nCheckDepth) break;` should probably be `<=`.
  6 2018-06-11 04:50:50	0|sipa|kallewoof: agree
  7 2018-06-11 05:27:39	0|bitcoin-git|[13bitcoin] 15kallewoof opened pull request #13428: validation: check the specified number of blocks (off-by-one) (06master...06validation-off-by-one) 02https://github.com/bitcoin/bitcoin/pull/13428
  8 2018-06-11 05:27:54	0|bitcoin-git|[13bitcoin] 15Empact opened pull request #13429: Return the script type from Solver (06master...06solver-return) 02https://github.com/bitcoin/bitcoin/pull/13429
  9 2018-06-11 05:49:01	0|gmaxwell|sipa: you created a 32-byte version of the specialized 1-way SSE4 asm?
 10 2018-06-11 07:04:26	0|bitcoin-git|[13bitcoin] 15kallewoof opened pull request #13430: use IsBlockPruned() where appropriate (06master...06use-isblockpruned) 02https://github.com/bitcoin/bitcoin/pull/13430
 11 2018-06-11 07:15:36	0|bitcoin-git|[13bitcoin] 15kallewoof opened pull request #13431: validation: update pindexState for check level < 3 (06master...06verifydb_pindexstate_lvl0-2) 02https://github.com/bitcoin/bitcoin/pull/13431
 12 2018-06-11 08:57:59	0|ossifrage|FYI, bitcoin-qt from the head I built today won't start if you have "daemon=0" in the config file, so you can't use the same config for either bitcoind or bitcoin-qt
 13 2018-06-11 08:59:32	0|ossifrage|Seems like bitcoin-qt should ignore this option?
 14 2018-06-11 09:00:42	0|bitcoin-git|[13bitcoin] 15ken2812221 opened pull request #13434: Set default CFLAGS, CXXFLAGS to empty (06master...06enable_debug) 02https://github.com/bitcoin/bitcoin/pull/13434
 15 2018-06-11 09:05:23	0|provoostenator|ossifrage: probably caused by 13112. Another problem is disablewallet=1 will prevent a launch if you compile bitcoind without wallet. It probably needs to be relaxed slightly.
 16 2018-06-11 09:05:35	0|provoostenator|#13112
 17 2018-06-11 09:05:40	0|gribble|https://github.com/bitcoin/bitcoin/issues/13112 | Throw an error for unknown args by achow101 · Pull Request #13112 · bitcoin/bitcoin · GitHub
 18 2018-06-11 09:10:19	0|bitcoin-git|[13bitcoin] 15ken2812221 closed pull request #13434: Set default CFLAGS, CXXFLAGS to empty (06master...06enable_debug) 02https://github.com/bitcoin/bitcoin/pull/13434
 19 2018-06-11 09:30:12	0|wumpus|rc2 executables up https://bitcoincore.org/bin/bitcoin-core-0.16.1/test.rc2/, sorry for the delay
 20 2018-06-11 09:46:35	0|bitcoin-git|[13bitcoin] 15Empact opened pull request #13435: When build fails due to lib missing, indicate which one (06master...06lib-missing) 02https://github.com/bitcoin/bitcoin/pull/13435
 21 2018-06-11 09:47:31	0|kallewoof|Regarding #13434, shouldn't --enable-debug automatically do --disable-maintainer-mode? Currently --enable-debug is useless at least on macs, as it still generates optimized code so lldb barfs.
 22 2018-06-11 09:47:33	0|gribble|https://github.com/bitcoin/bitcoin/issues/13434 | Set default CFLAGS, CXXFLAGS to empty by ken2812221 · Pull Request #13434 · bitcoin/bitcoin · GitHub
 23 2018-06-11 11:24:25	0|wumpus|kallewoof: maintainer mode affects optimization?
 24 2018-06-11 11:26:01	0|wumpus|provoostenator: I think that one makes sense, if you compile without wallet the program has no way to know about wallet options, so will reject them
 25 2018-06-11 11:26:21	0|wumpus|provoostenator: the alternative would be to move knowledge of wallet options into the core, that'd be even worse
 26 2018-06-11 11:27:01	0|rafalcpp|wumpus: I wonder how githubmerge could support git submodules. The submodules are linked by sha1 which is useless.
 27 2018-06-11 11:27:39	0|rafalcpp|perhaps we should recursivly calculate our tree-sha512 of a submodule, and that checksum will be placed as data (next to blobs) of the tree of parent
 28 2018-06-11 11:27:47	0|provoostenator|Not full knowledge, just any option that disables these features, like upnp=0, disablewallet=1. That way if you forget to leave them out of compilation, those features don't just turn themselves on.
 29 2018-06-11 11:28:07	0|wumpus|hmm okay that sounds better at least
 30 2018-06-11 11:29:24	0|wumpus|rafalcpp: sounds like a lot of hassle, indeed
 31 2018-06-11 11:30:40	0|wumpus|agree
 32 2018-06-11 11:31:53	0|wumpus|I think he keeps insisting that 'commit hashes are not a security feature', fair enough, but many people need a way to authenticate repositories securely
 33 2018-06-11 11:32:04	0|rafalcpp|yeap
 34 2018-06-11 11:32:06	0|wumpus|and the commit hash is the weakest link in that
 35 2018-06-11 11:40:34	0|rafalcpp|someone should ask him does he again want CIA to if (uid = 0) ... him like they did old CVS, but this time with false sense of security
 36 2018-06-11 11:42:48	0|wumpus|well it's open source so someone that cares should do it,instead of trying to convince Linus, that's how these things work. I wish I had the energy and time.
 37 2018-06-11 11:46:49	0|wumpus|whoa is bitcoinacks correct here https://bitcoinacks.com/?flt1_closed_empty=1&flt0_merged_empty=1&sort=7 none of the open PRs that are non high priority for review have any ACK/NACKs? or is it glitching somehow
 38 2018-06-11 11:52:59	0|wumpus|nm, got the sorting wrong, phew
 39 2018-06-11 11:56:59	0|rafalcpp|that Tree-SHA512 format, is something that bitcoin invented, or is it strictly based on some agreed upon convention? (ordering and format of data that is the material to be hashed)?
 40 2018-06-11 11:57:27	0|wumpus|it's something that was invented for our script AFAIK
 41 2018-06-11 12:20:46	0|bitcoin-git|13bitcoin/06master 14cbede7d 15Sjors Provoost: [qt] OptionsDialog: add prune setting
 42 2018-06-11 12:20:46	0|bitcoin-git|[13bitcoin] 15laanwj pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/56f69360dc98...6e249e46789f
 43 2018-06-11 12:20:47	0|bitcoin-git|13bitcoin/06master 146e249e4 15Wladimir J. van der Laan: Merge #13043: [qt] OptionsDialog: add prune setting...
 44 2018-06-11 12:21:25	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #13043: [qt] OptionsDialog: add prune setting (06master...062018/04/qt-prune) 02https://github.com/bitcoin/bitcoin/pull/13043
 45 2018-06-11 12:33:08	0|promag|wumpus: are you available for merges?
 46 2018-06-11 12:37:57	0|bitcoin-git|[13bitcoin] 15laanwj pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/6e249e46789f...531a0337ca93
 47 2018-06-11 12:37:58	0|bitcoin-git|13bitcoin/06master 14531a033 15Wladimir J. van der Laan: Merge #13421: qa: Remove portseed_offset from test runner...
 48 2018-06-11 12:37:58	0|bitcoin-git|13bitcoin/06master 14fa6edfe 15MarcoFalke: qa: Remove portseed_offset from test runner
 49 2018-06-11 12:38:43	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #13421: qa: Remove portseed_offset from test runner (06master...06Mf1806-qaPortseedOffset) 02https://github.com/bitcoin/bitcoin/pull/13421
 50 2018-06-11 12:43:50	0|bitcoin-git|13bitcoin/06master 14f68049d 15Cory Fields: crypto: cleanup sha256 build...
 51 2018-06-11 12:43:50	0|bitcoin-git|[13bitcoin] 15laanwj pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/531a0337ca93...70a03c635b73
 52 2018-06-11 12:43:51	0|bitcoin-git|13bitcoin/06master 1470a03c6 15Wladimir J. van der Laan: Merge #13408: crypto: cleanup sha256 build...
 53 2018-06-11 12:44:36	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #13408: crypto: cleanup sha256 build (06master...06sha2-cleanup) 02https://github.com/bitcoin/bitcoin/pull/13408
 54 2018-06-11 13:06:05	0|bitcoin-git|13bitcoin/06master 14a426098 15practicalswift: Fix compiler warnings emitted when compiling under stock OpenBSD 6.3
 55 2018-06-11 13:06:05	0|bitcoin-git|[13bitcoin] 15laanwj pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/70a03c635b73...26c93edf1de9
 56 2018-06-11 13:06:06	0|bitcoin-git|13bitcoin/06master 1426c93ed 15Wladimir J. van der Laan: Merge #13294: Fix compiler warnings emitted when compiling under stock OpenBSD 6.3...
 57 2018-06-11 13:06:53	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #13294: Fix compiler warnings emitted when compiling under stock OpenBSD 6.3 (06master...06openbsd-warnings) 02https://github.com/bitcoin/bitcoin/pull/13294
 58 2018-06-11 13:20:42	0|bitcoin-git|[13bitcoin] 15laanwj pushed 3 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/26c93edf1de9...3f0f39415bd7
 59 2018-06-11 13:20:43	0|bitcoin-git|13bitcoin/06master 1467e0e04 15John Newbery: [wallet] [docs] Update release notes for removing `getlabeladdress`
 60 2018-06-11 13:20:43	0|bitcoin-git|13bitcoin/06master 148160817 15John Newbery: [wallet] [rpc] Remove getlabeladdress RPC...
 61 2018-06-11 13:20:44	0|bitcoin-git|13bitcoin/06master 143f0f394 15Wladimir J. van der Laan: Merge #13060: [wallet] [rpc] Remove getlabeladdress RPC...
 62 2018-06-11 13:21:20	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #13060: [wallet] [rpc] Remove getlabeladdress RPC (06master...06remove_getlabeladdress) 02https://github.com/bitcoin/bitcoin/pull/13060
 63 2018-06-11 13:51:34	0|wumpus|promag: yes
 64 2018-06-11 14:15:02	0|promag|wumpus: #12151
 65 2018-06-11 14:15:05	0|gribble|https://github.com/bitcoin/bitcoin/issues/12151 | rpc: Remove cs_main lock from blockToJSON and blockheaderToJSON by promag · Pull Request #12151 · bitcoin/bitcoin · GitHub
 66 2018-06-11 14:16:11	0|promag|or #13160
 67 2018-06-11 14:16:13	0|gribble|https://github.com/bitcoin/bitcoin/issues/13160 | wallet: Unlock spent outputs by promag · Pull Request #13160 · bitcoin/bitcoin · GitHub
 68 2018-06-11 14:17:38	0|MarcoFalke|promag: #12151 was just pushed, imo not ready for merge
 69 2018-06-11 14:17:41	0|gribble|https://github.com/bitcoin/bitcoin/issues/12151 | rpc: Remove cs_main lock from blockToJSON and blockheaderToJSON by promag · Pull Request #12151 · bitcoin/bitcoin · GitHub
 70 2018-06-11 14:18:58	0|wumpus|thanks, I'll have a look at those
 71 2018-06-11 14:19:25	0|promag|MarcoFalke: right, should be re-ack
 72 2018-06-11 14:20:29	0|promag|MarcoFalke: if you can take a look at 13160 too
 73 2018-06-11 14:25:25	0|bitcoin-git|[13bitcoin] 15laanwj pushed 10 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/3f0f39415bd7...43ae5ee9e4c2
 74 2018-06-11 14:25:26	0|bitcoin-git|13bitcoin/06master 1446847d6 15Karl-Johan Alm: mempool: Fix max descendants check...
 75 2018-06-11 14:25:26	0|bitcoin-git|13bitcoin/06master 14b9ef21d 15Karl-Johan Alm: mempool: Add explicit max_descendants...
 76 2018-06-11 14:25:27	0|bitcoin-git|13bitcoin/06master 14475a385 15Karl-Johan Alm: Add GetTransactionAncestry to CTxMemPool for general purpose chain limit checking
 77 2018-06-11 14:25:56	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #12634: [refactor] Make TransactionWithinChainLimit more flexible (06master...06txmempool-chain-limit-value) 02https://github.com/bitcoin/bitcoin/pull/12634
 78 2018-06-11 14:47:20	0|wumpus|#13160 has only my utACK
 79 2018-06-11 14:47:24	0|gribble|https://github.com/bitcoin/bitcoin/issues/13160 | wallet: Unlock spent outputs by promag · Pull Request #13160 · bitcoin/bitcoin · GitHub
 80 2018-06-11 14:47:35	0|wumpus|I don't think that's enough for a change in behavior like that
 81 2018-06-11 14:47:52	0|wumpus|even though code-wise it's very simple to review
 82 2018-06-11 15:08:59	0|sipa|MarcoFalke: you mentioned that fetching PR information from github was the bottleneck for drahtbot? are you using the git interface for PRs?
 83 2018-06-11 15:09:50	0|MarcoFalke|I do. But I also need to get the open pull requests and metadata for them.
 84 2018-06-11 15:11:29	0|sipa|hmm
 85 2018-06-11 15:12:20	0|ken2812221|Hi all. In order to fix #13103, is it allowed to add additional functions and methods like #13426 for Windows, and add macros for other OS? (The first and second commits)
 86 2018-06-11 15:12:21	0|gribble|https://github.com/bitcoin/bitcoin/issues/13103 | Invalid wallet path with Chinese characters in windows · Issue #13103 · bitcoin/bitcoin · GitHub
 87 2018-06-11 15:12:23	0|gribble|https://github.com/bitcoin/bitcoin/issues/13426 | [WIP, bugfix] Add u8path and u8string to boost to fix #13103 by ken2812221 · Pull Request #13426 · bitcoin/bitcoin · GitHub
 88 2018-06-11 15:13:19	0|ken2812221|additional function and method in boost header
 89 2018-06-11 15:13:24	0|MarcoFalke|And the "Needs rebase" task of DrahtBot is purely based on the api
 90 2018-06-11 15:14:06	0|MarcoFalke|Since GitHub hasn't yet disclosed which merge tool they use
 91 2018-06-11 15:18:19	0|sipa|MarcoFalke: seems reasonablw that it's pretty much vanilla git
 92 2018-06-11 15:18:39	0|sipa|as our merge tool compares the local merge with the github merge
 93 2018-06-11 15:18:56	0|sipa|and afaik never failed for noticing a differencw
 94 2018-06-11 15:19:18	0|MarcoFalke|They advertise as merge conflict when one of the files got moved
 95 2018-06-11 15:19:26	0|MarcoFalke|vanilla git doen't afaict
 96 2018-06-11 15:20:03	0|sipa|hmm
 97 2018-06-11 16:00:22	0|echeveria|https://pastebin.com/Ar8cT76q
 98 2018-06-11 16:00:35	0|echeveria|bitcoind seems to get very sad sometimes if you have a low maxconnections
 99 2018-06-11 16:00:35	0|ryanofsky|i have a hacky script that checks for merge conflicts reasonably quickly with "git diff | git apply" https://github.com/ryanofsky/home/blob/master/src/pr.sh#L558
100 2018-06-11 16:17:09	0|promag|wumpus: I can't reproduce #13111 failures locally
101 2018-06-11 16:17:12	0|gribble|https://github.com/bitcoin/bitcoin/issues/13111 | Add unloadwallet RPC by promag · Pull Request #13111 · bitcoin/bitcoin · GitHub
102 2018-06-11 16:17:26	0|promag|should retry travis job?
103 2018-06-11 16:25:58	0|bitcoin-git|[13bitcoin] 15ken2812221 closed pull request #13107: Fix Windows locale problem (06master...06win-enc) 02https://github.com/bitcoin/bitcoin/pull/13107
104 2018-06-11 16:46:03	0|Purple7|Hi everyone :)
105 2018-06-11 16:46:55	0|Purple7|can someone here help me create a bitcoin script. I just installed Bitcoin core node and have tried bitcoin-cli commands
106 2018-06-11 16:47:12	0|echeveria|I don't know what you imagine bitcoin script to be capable of.
107 2018-06-11 16:47:20	0|Purple7|I want to create a script but not sure how to create one and how to run one
108 2018-06-11 16:47:22	0|wumpus|promag: I had some segfaults locally with it, but seems hard to reproduce
109 2018-06-11 16:47:31	0|Purple7|to carry out transactions
110 2018-06-11 16:47:35	0|echeveria|Purple7: lets do this in #bitcoin.
111 2018-06-11 16:48:49	0|promag|wumpus: yes I think I can reproduce now
112 2018-06-11 16:49:02	0|Purple7|Thanks echeveria
113 2018-06-11 16:49:04	0|Purple7|:)
114 2018-06-11 16:49:30	0|promag|wumpus: delete regtest folder, launch, then rpc stop
115 2018-06-11 16:49:51	0|promag|looks like that way always segfaults
116 2018-06-11 16:52:55	0|Chris_Stewart_5|Is there any ordering required for the rpcport config option? We are seeing behavior where it is being read if passed in as a command line arg but not being recogonized inside of a bitcoin.conf file
117 2018-06-11 16:55:10	0|Chris_Stewart_5|we are reading other config options from the bitcoin.conf file
118 2018-06-11 16:55:15	0|Chris_Stewart_5|just not rpcport it seems
119 2018-06-11 16:55:47	0|sipa|Chris_Stewart_5: what version of the code?
120 2018-06-11 16:55:54	0|Chris_Stewart_5|0.16.0
121 2018-06-11 16:56:12	0|wumpus|rpcport should work fine in the bitcoin.conf, I've had it in there for ages
122 2018-06-11 16:56:43	0|wumpus|note that master has per-network config sections, but 0.16 does not
123 2018-06-11 16:56:51	0|Chris_Stewart_5|yeah it is pretty bizzare. We are reading other options like daemon=1 in the conf file. Just not the rpcport for some reason
124 2018-06-11 16:57:20	0|wumpus|are you really sure? as said, I have a setup myself that uses that
125 2018-06-11 16:57:39	0|Chris_Stewart_5|just a sec, I'm double checking he didn't compile from master
126 2018-06-11 16:58:37	0|Chris_Stewart_5|nvm, he is on 0.16.99.
127 2018-06-11 16:59:30	0|sipa|how do you observe it's not using rpcport?
128 2018-06-11 16:59:35	0|wumpus|ok on master you need to put it in a network-specific section, this is described in the doc/release-notes.md
129 2018-06-11 16:59:44	0|sipa|wumpus: oh!
130 2018-06-11 16:59:55	0|wumpus|it should also warn in the log
131 2018-06-11 17:01:23	0|sipa|wumpus: nothing here: https://github.com/bitcoin/bitcoin/blob/master/doc/release-notes.md
132 2018-06-11 17:01:26	0|Chris_Stewart_5|basically it was always binding to 18433 if we set rpcport via the command line
133 2018-06-11 17:01:52	0|Chris_Stewart_5|unless we set it via the command line* sorry
134 2018-06-11 17:03:12	0|provoostenator|Anyone else noticed bitcoin-cli crashing on macOS 10.13.5? I had to recompile. It's quite possible I did something stupid unrelated, but seeing c-lightning and random Github projects struggling with macOS 10.13.5, thought I'd check...
135 2018-06-11 17:04:48	0|provoostenator|What's stranger - and why I initially assume I did something else wrong: it didn't happen immediately after the upgrade but a few days later, so it may have been a homebrew or xcode update.
136 2018-06-11 17:06:02	0|wumpus|sipa: release-notes-pr12823.md apparently
137 2018-06-11 17:06:23	0|sipa|oh, i forgot we added pr-specific files
138 2018-06-11 17:07:03	0|wumpus|it does make things harder to find
139 2018-06-11 17:08:43	0|sipa|wumpus: any opinion about converting the existing sse4 assembly code (which was in 0.16) into intrinsics based code?
140 2018-06-11 17:09:03	0|sipa|downside: possibly a bit slower (i've seen up to 4% slower) and compiler dependent
141 2018-06-11 17:09:17	0|sipa|upside: more easily reusable, readable, and works on 32-bit systems
142 2018-06-11 17:13:07	0|promag|wumpus: found the problem, will push a fix in a separate commit for now
143 2018-06-11 17:17:35	0|echeveria|yeah, that's a weird one. I have hardcoded `connect` peers but it doesn't sync.
144 2018-06-11 17:20:02	0|echeveria|random peers, syncs perfectly, hardcoded it doesn't sync, even with the same peers.
145 2018-06-11 17:31:52	0|wumpus|sipa: no strong opinion on that; 32 bit x86 is very rare so I wouldn't do it for that, given that it's also 4% slower, "do nothing" sounds like a good option to me
146 2018-06-11 17:33:53	0|gmaxwell|22:50:10 < gmaxwell> sipa: you created a 32-byte version of the specialized 1-way SSE4 asm?
147 2018-06-11 17:34:24	0|sipa|gmaxwell: no, not specialized
148 2018-06-11 17:35:09	0|sipa|we just have a benchmark for SHA256 with 32-byte inputs, and that benchmark shows a slowdown when going from asm-based 1-way SSE4 to intrin-based 1-way SSE4
149 2018-06-11 17:39:20	0|sipa|gmaxwell: also, the speedup from the 1-way SSE4 is pretty neat: it does the expansion for rounds X+16...X+19 in SSE registers simultaneously, interleaved with an otherwise totally ordinary implementation for rounds X..X+3
150 2018-06-11 17:42:13	0|wumpus|I wonder, is anyone using 32-bit x86 for bitcoin nodes? the last 32-bit-only x86 CPU was sold in 2008 or so, 10 years ago now
151 2018-06-11 17:42:36	0|sipa|wumpus: ah, but those don't even have SSE4 :)
152 2018-06-11 17:42:53	0|sipa|the only use for SSE4 in 32-bit mode is people running a 32-bit binary on 64-bit hardware
153 2018-06-11 17:42:54	0|wumpus|too bad we don't have statistics how much those get downloaded
154 2018-06-11 17:43:15	0|wumpus|right
155 2018-06-11 17:43:49	0|sipa|my goal in converting to intrinsics was making it reusable for specialized 32-byte or 64-byte inputs; we can leave the original untouched of course, and only have intrinsics based versions for the specialized versions
156 2018-06-11 17:45:46	0|wumpus|for that it certainly makes sense, easier to specialize code w/ intrinsics than manual register allocation
157 2018-06-11 17:47:22	0|sipa|i also wonder what to do about CI for special hardware
158 2018-06-11 17:47:32	0|sipa|SHA-NI is unlikely to be available on any Travis system
159 2018-06-11 17:47:48	0|sipa|and POWER9 would surprise me even more :D
160 2018-06-11 17:50:11	0|luke-jr|test these things more carefully when they change?
161 2018-06-11 17:50:23	0|wumpus|it's the same case as platforms such as FreeBSD and OpenBSD, we'll have to rely on people regularly compilng and testing master with them
162 2018-06-11 17:50:41	0|luke-jr|well, at least the platform-specific stuff in this case is isolated
163 2018-06-11 17:50:54	0|sipa|i'm also going to extend the self-test code for these
164 2018-06-11 17:51:07	0|sipa|so at least it will fail at startup rather than arbitrarily
165 2018-06-11 17:51:32	0|wumpus|it's notrealistic to have CI for all OS/architecture combinations, not even all OSes and architectures separately
166 2018-06-11 17:52:12	0|wumpus|I still have a PR open to run travis on at least one bigendian platform, but even that seems untenable
167 2018-06-11 17:52:16	0|luke-jr|maybe if someone were to volunteer to maintain a CI system for us that does all that, but yeah, probably not with Travis
168 2018-06-11 17:52:34	0|wumpus|it's not realistic to do that for every PR, sure, some daily cron job would work
169 2018-06-11 18:00:06	0|MarcoFalke|Apparently we have  some jenkins running somewhere, maybe jamesob can see if that would work with FreeBSD/OpenBSD
170 2018-06-11 18:00:21	0|MarcoFalke|SHA-NI and POWER9 should already be tested by devs regularly, no?
171 2018-06-11 18:02:24	0|gmaxwell|with jenkins it would be as simple as adding a ssh key on a host and telling jenkins to use it as a remote. I guess travis doesn't have a similar facility for non-travis build hosts?
172 2018-06-11 18:03:09	0|jamesob|gmaxwell: travis is limited to execution on their infra AFAICT
173 2018-06-11 18:03:21	0|MarcoFalke|I think so as well ^
174 2018-06-11 18:04:53	0|gmaxwell|in any case, I assume jenkins hosts will have sha-ni sometime, for now we'll just have to count on developers (lol, pieter is the only person I know with it right now.) catching it.
175 2018-06-11 18:04:59	0|wumpus|in theory it's possible to ssh out from travis, but that sounds horribly fragile
176 2018-06-11 18:05:08	0|wumpus|(also, key management is a problem)
177 2018-06-11 18:05:52	0|gmaxwell|for jenkins the integration is super nice, it's not just 'sshing out' but once you give jenkins access it does all the stuff to properly use it as a build host automagically. (wow, java actually useful for something)
178 2018-06-11 18:07:22	0|wumpus|right, that sounds better
179 2018-06-11 18:08:54	0|bitcoin-git|[13bitcoin] 15MarcoFalke opened pull request #13437: wallet: Erase wtxOrderd wtx pointer on removeprunedfunds (06master...06Mf1806-walletPrunedFundsSegfault) 02https://github.com/bitcoin/bitcoin/pull/13437
180 2018-06-11 18:08:56	0|luke-jr|MarcoFalke: I should be moving my development to my POWER9 system in this next week
181 2018-06-11 18:09:11	0|MarcoFalke|luke-jr: Good to hear
182 2018-06-11 18:09:23	0|MarcoFalke|Also BlueMatt switched to POWER9
183 2018-06-11 18:10:02	0|cfields|sipa: fwiw, I now fully understand the 1way sha2 optims for sse4/avx. I started working on the intrinsics over the weekend
184 2018-06-11 18:10:17	0|sipa|cfields: too late :)
185 2018-06-11 18:10:20	0|cfields|sipa: just thought I'd mention in case you were also looking at it
186 2018-06-11 18:10:32	0|cfields|oh?
187 2018-06-11 18:10:41	0|cfields|heh, did I miss a PR?
188 2018-06-11 18:10:43	0|sipa|cfields: i have intrinsics based 1-way sse4 working
189 2018-06-11 18:10:46	0|sipa|not PRed yet
190 2018-06-11 18:11:17	0|sipa|sorry, didn't know you were planning to work on it so soon as well
191 2018-06-11 18:11:36	0|cfields|sipa: no worries. Now I can pick yours apart :p
192 2018-06-11 18:12:22	0|BlueMatt|yea, POWER9 should get pretty good testing, sdaftuar and I have already been using it for some time now
193 2018-06-11 18:12:27	0|provoostenator|cfields: or work on #13401
194 2018-06-11 18:12:28	0|gribble|https://github.com/bitcoin/bitcoin/issues/13401 | ARMv8 sha2 support · Issue #13401 · bitcoin/bitcoin · GitHub
195 2018-06-11 18:12:30	0|provoostenator|:-)
196 2018-06-11 18:12:41	0|BlueMatt|and I think we might be getting a shared test box on it soonish if people want to get an ssh key to have access to it
197 2018-06-11 18:14:23	0|gmaxwell|sipa: would the 1-way sha2 be more efficient for longer inputs if it processed more blocks at once?
198 2018-06-11 18:14:37	0|gmaxwell|(e.g. could it be expanding the next group)
199 2018-06-11 18:17:26	0|sipa|gmaxwell: hmm, yes!
200 2018-06-11 18:17:28	0|cfields|sipa: did you include the lane duplication optim for quicker rotates?
201 2018-06-11 18:17:52	0|sipa|cfields: i literally translated the asm code instruction per instruction to intrinsics
202 2018-06-11 18:18:05	0|sipa|and then restructured it into functions that make sense
203 2018-06-11 18:18:13	0|provoostenator|What does "1-way" mean in the context of a SHA256 hash?
204 2018-06-11 18:18:23	0|cfields|sipa: ah, I think some of that will pessimize avx though, no?
205 2018-06-11 18:18:29	0|cfields|I guess I'll wait to see it
206 2018-06-11 18:18:50	0|gmaxwell|provoostenator: computing one hash at a time, as opposted to concurrently computing many hashes.
207 2018-06-11 18:19:09	0|sipa|cfields: recompiling this code with -mavx gains virtually no advantage
208 2018-06-11 18:19:41	0|sipa|provoostenator: say you have 8 64-byte vectors, and want to independently compute the 8 32-byte hashes of those
209 2018-06-11 18:19:49	0|sipa|provoostenator: we have a specialized function that does that
210 2018-06-11 18:19:56	0|sipa|that's 8-way
211 2018-06-11 18:20:27	0|cfields|sipa: hmm, that's really surprising.
212 2018-06-11 18:20:34	0|provoostenator|Ah ok, so that's useful if you're able to do some of the validation stuff in parallel?
213 2018-06-11 18:21:49	0|sipa|provoostenator: it's currently used (in master) for merkle tree root computations
214 2018-06-11 18:24:58	0|bitcoin-git|[13bitcoin] 15laanwj pushed 4 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/43ae5ee9e4c2...7c32b414b632
215 2018-06-11 18:24:59	0|bitcoin-git|13bitcoin/06master 14906bee8 15practicalswift: Use bracket syntax includes ("#include <foo.h>")
216 2018-06-11 18:25:00	0|bitcoin-git|13bitcoin/06master 146d10f43 15practicalswift: Enforce the use of bracket syntax includes ("#include <foo.h>")
217 2018-06-11 18:25:01	0|bitcoin-git|13bitcoin/06master 1416e3cd3 15practicalswift: Clarify include recommendation
218 2018-06-11 18:25:30	0|gmaxwell|in theory we could also use them for hashing a bunch of transactions at once, but handling the variable length stuff is slightly gnarly.
219 2018-06-11 18:25:32	0|provoostenator|sipa: why not in script validation? Not worth it or still on todo lists?
220 2018-06-11 18:25:35	0|cfields|sipa: no clue if it's still state-of-the-art, but this is what I was using as a basis for understanding/implementing multi-block speedup: https://eprint.iacr.org/2012/067.pdf
221 2018-06-11 18:25:40	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #13230: Simplify include analysis by enforcing the developer guide's include syntax (06master...06bracket-syntax-includes) 02https://github.com/bitcoin/bitcoin/pull/13230
222 2018-06-11 18:25:45	0|cfields|gmaxwell: ^^
223 2018-06-11 18:25:56	0|sipa|provoostenator: todo list
224 2018-06-11 18:26:21	0|gmaxwell|provoostenator: yes, it could be done, but it requires having things aranged so there are multiple messages to hash all ready to hash at once.
225 2018-06-11 18:26:39	0|gmaxwell|So easiest was hashtrees, since that all gets computed at once.
226 2018-06-11 18:26:44	0|provoostenator|Right now script validation can is done in parallel on _multiple_ cpu's, rather than on a single one?
227 2018-06-11 18:27:22	0|cfields|(ofc that's separate from the per-block optimizations)
228 2018-06-11 18:27:35	0|cfields|provoostenator: yes
229 2018-06-11 18:29:13	0|provoostenator|But if you verify e.g. 8 scripts in parallel on the same core to take advantage of that 256 optimzation, I guess the risk is that these cores are just sitting there waiting for these 8 threads to be ready?
230 2018-06-11 18:29:40	0|provoostenator|If they're not otherwise identical.
231 2018-06-11 18:29:59	0|provoostenator|But 1-way doesn't change anything
232 2018-06-11 18:32:23	0|gmaxwell|provoostenator: using n-at-a-time hashing requires restructing code so that it actually has n things to hash available at a time.  Script validation doesn't work that way today and would need significant overhalls to make use of it.
233 2018-06-11 18:58:04	0|cfields|sipa: would you be ok with a separate avx-optimized version of those intrinsics if the speedup was as significant as before?
234 2018-06-11 19:02:36	0|sipa|cfields: the avx speedup is for the 4-way sse4, not the 1-way sse4 code
235 2018-06-11 19:02:39	0|sipa|cfields: and yes
236 2018-06-11 19:03:18	0|cfields|sipa: there are definitely 1way speedups due to non-destructive xor's
237 2018-06-11 19:05:52	0|sipa|cfields: ah, the compiler may not be smart enough to use those?
238 2018-06-11 19:06:21	0|cfields|sipa: I really would've expected it to, since there's not a separate intrinsic for them
239 2018-06-11 19:06:30	0|cfields|it might be how the other registers are setup, though
240 2018-06-11 19:07:57	0|cfields|the sigma1 implementation for sse4, for example, duplicates data in 2 registers so that it can do quicker but destructive rotates. Avx doesn't need to do that, so the duplication is strictly a slowdown.
241 2018-06-11 19:08:36	0|sipa|cfields: i'll cleanup my code and PR
242 2018-06-11 19:09:28	0|cfields|ok
243 2018-06-11 19:09:42	0|bitcoin-git|[13bitcoin] 15sipa opened pull request #13438: Improve coverage of SHA256 SelfTest code (06master...06201806_selftestsha) 02https://github.com/bitcoin/bitcoin/pull/13438
244 2018-06-11 19:10:22	0|cfields|sipa: thanks for that :)
245 2018-06-11 19:11:07	0|bitcoin-git|[13bitcoin] 15MarcoFalke closed pull request #13395: rpc: Avoid "duplicate" return value for invalid submitblock (06master...06Mf1806-rpcMiningSubmitblock) 02https://github.com/bitcoin/bitcoin/pull/13395
246 2018-06-11 19:20:51	0|provoostenator|Any other live metrics I should collect on my slow pruned AWS node? https://github.com/bitcoin/bitcoin/pull/12404#issuecomment-396356384
247 2018-06-11 19:23:06	0|gmaxwell|cfields: so somewhere intel has published sse4/avx/avx2 code (which is where our 1-way sse4 comes from) when we previously benchmarked the avx code we found that it was a couple percent faster on intel but a lot slower on AMD.
248 2018-06-11 19:23:44	0|gmaxwell|cfields: I don't see any problem with having a seperate AVX version but we would need to deal with not choosing it on AMD somehow, if its slower there.
249 2018-06-11 19:23:55	0|cfields|gmaxwell: do you mean intel's modified Sigma0/Sigma1 ?
250 2018-06-11 19:24:33	0|gmaxwell|I don't know what optimizations it had inside it, IIRC it was the same code intel submitted for the linux kernel (the kernel can realistically benchmark to decide what code to use, not as reasonable for us)
251 2018-06-11 19:24:36	0|cfields|if so, yea, most of the code I've found in the wild opts out of those for that reason
252 2018-06-11 19:25:04	0|cfields|gmaxwell: I'm guessing it's what I PR'd, which we closed for that reason :(
253 2018-06-11 19:25:46	0|gmaxwell|for us we could decide to use some AVX version that was faster on intel by string matching the cpu version and only using it on intel chips.  But if it's only a couple percent it might not be worth it.
254 2018-06-11 19:25:59	0|gmaxwell|ESP since anything with AVX is pretty quick regardless.
255 2018-06-11 19:26:53	0|bitcoin-git|[13bitcoin] 15TheBlueMatt opened pull request #13439: rpc: Avoid "duplicate" return value for invalid submitblock (06master...062018-06-marcos-submitblock-fix) 02https://github.com/bitcoin/bitcoin/pull/13439
256 2018-06-11 19:30:12	0|cfields|gmaxwell: agreed. I'd rather avoid per-vendor optims unless it's really really significant
257 2018-06-11 19:30:27	0|cfields|might be harder to avoid that if we get into arm, though
258 2018-06-11 19:30:55	0|gmaxwell|for arm it's more worth it...
259 2018-06-11 19:31:15	0|gmaxwell|even a small percantage change is a large absolute time there.
260 2018-06-11 19:33:51	0|cfields|fair point.
261 2018-06-11 19:34:09	0|cfields|gmaxwell: #13400 is what I was referencing, btw. Hopefully that's the same one as the kernel discussion you had in mind.
262 2018-06-11 19:34:11	0|gribble|https://github.com/bitcoin/bitcoin/issues/13400 | sha256: small speedup for sse4 path. by theuni · Pull Request #13400 · bitcoin/bitcoin · GitHub
263 2018-06-11 19:34:20	0|gmaxwell|cfields: I guess for AVX ... Intel x86_64 cpus with AVX but without AVX2 or SHA-NI are _very_ common. I'd guess they outnumber all other kinds in our deployment by a large margin.
264 2018-06-11 19:36:26	0|cfields|huh, I figured my Sandy Bridge was the outlier
265 2018-06-11 19:39:12	0|gmaxwell|AVX2 is only haswell and later on intel, and the xeons lag desktops in microarch by a couple years.
266 2018-06-11 20:23:16	0|bitcoin-git|[13bitcoin] 15MarcoFalke opened pull request #13440: qa: Log as utf-8 (06master...06Mf1806-qaLogUtf8) 02https://github.com/bitcoin/bitcoin/pull/13440
267 2018-06-11 20:52:35	0|edgar_|hello
268 2018-06-11 21:26:52	0|bitcoin-git|[13bitcoin] 15achow101 opened pull request #13441: Prevent shared conf files from failing with different available options in different binaries (06master...06gargs-disabled-options) 02https://github.com/bitcoin/bitcoin/pull/13441
269 2018-06-11 21:27:28	0|achow101|ossifrage: provoostenator: ^ that PR should fix the problem with the options
270 2018-06-11 23:08:08	0|sipa|gmaxwell: got it faster than the asm version now
271 2018-06-11 23:08:18	0|sipa|3.68 ms vs 3.77 ms
272 2018-06-11 23:08:24	0|sipa|on i7
273 2018-06-11 23:35:50	0|bitcoin-git|[13bitcoin] 15sipa opened pull request #13442: Convert the 1-way SSE4 SHA256 code from asm to intrinsics (06master...06201806_sse4intrin) 02https://github.com/bitcoin/bitcoin/pull/13442
274 2018-06-11 23:36:40	0|cfields|sipa: nice
275 2018-06-11 23:37:22	0|cfields|sipa: and it looks like it should be friendlier to avx now. Was that purposeful, or nice side-effect?
276 2018-06-11 23:37:34	0|sipa|cfields: i don't know why or how
277 2018-06-11 23:37:40	0|sipa|elaborate please :)
278 2018-06-11 23:38:57	0|sipa|cfields: it's crazy how much the compiler affects things, though
279 2018-06-11 23:39:09	0|sipa|in one place changing a constant from const to static const made it 10% slower
280 2018-06-11 23:39:23	0|sipa|in another place changing it from static const to const made it 3% slower
281 2018-06-11 23:39:31	0|cfields|sipa: yea, I was going to ask you if the ordering really matters, I assume the compiler will reorder as it sees fit
282 2018-06-11 23:39:43	0|cfields|huh
283 2018-06-11 23:40:25	0|sipa|generally reordering seems to have only a marginal affect, to none at all
284 2018-06-11 23:40:35	0|sipa|changing the types of constants and variables has a large affect
285 2018-06-11 23:40:58	0|cfields|interesting, I would've thought the interleaving timing was the crucial part
286 2018-06-11 23:41:02	0|sipa|(the trick with Ws, as opposed to just keeping it as a __m128i and extracting the elements from it, does have a different)
287 2018-06-11 23:41:15	0|sipa|oh, the interleaving is absolutely crucial
288 2018-06-11 23:41:16	0|cfields|...I guess it probably is, but you have to fight the compiler to get there first
289 2018-06-11 23:41:21	0|sipa|but the compiler reorders things anyway
290 2018-06-11 23:41:29	0|sipa|regardless of how you write it... to an extent
291 2018-06-11 23:42:35	0|cfields|sipa: by avx-friendly, I mean that you pass variables in which may or may not be overwritten, based on what instructions are available
292 2018-06-11 23:42:40	0|cfields|for example: XTMP0 = Palignr(X3, X2, 4);
293 2018-06-11 23:43:12	0|sipa|cfields: heh, SSA transformation will destroy that
294 2018-06-11 23:43:29	0|sipa|don't assume the compiler will map every variable to a single register all of the time
295 2018-06-11 23:43:52	0|cfields|ofc, I just mean that it has the option with avx
296 2018-06-11 23:43:57	0|sipa|even if you write things in a A = f(A, B) form everywhere, the compiler may still store the resulting A in a different register than the source A
297 2018-06-11 23:44:06	0|sipa|yes, but the way the code is written shouldn't affect that at all
298 2018-06-11 23:44:32	0|sipa|the SSA transform will effectively turn every assignment to a variable into a new variable, from the compiler's perspective
299 2018-06-11 23:45:20	0|cfields|I see
300 2018-06-11 23:47:38	0|sipa|with -msse4.1: 3.68ms
301 2018-06-11 23:47:45	0|sipa|with -mavx: 3.54ms
302 2018-06-11 23:48:46	0|cfields|sipa: huh, odd. I get a massive speedup with avx
303 2018-06-11 23:48:59	0|cfields|SHA256D64_1024, 20, 7400, 123.736, 0.000835992, 0.000836151, 0.000836061
304 2018-06-11 23:49:09	0|cfields|SHA256D64_1024, 20, 7400, 73.563, 0.000496887, 0.000497227, 0.000497053
305 2018-06-11 23:52:46	0|sipa|cfields: oh, i was looking at the SHA256 benchmark