1 2017-04-24 00:43:42	0|bitcoin-git|[13bitcoin] 151mbtxn opened pull request #10266: Update policy.h (06master...06patch-1) 02https://github.com/bitcoin/bitcoin/pull/10266
  2 2017-04-24 00:45:34	0|gmaxwell|the github review thing does not have a 'diapprove'
  3 2017-04-24 00:46:59	0|BlueMatt|lol
  4 2017-04-24 00:47:07	0|BlueMatt|wut
  5 2017-04-24 00:49:21	0|gmaxwell|apparently everyone gets a gold star, you can only comment, approve, or request changes...
  6 2017-04-24 00:49:47	0|BlueMatt|can you request that they close it?
  7 2017-04-24 00:50:06	0|BlueMatt|:p
  8 2017-04-24 00:54:20	0|bitcoin-git|[13bitcoin] 15fanquake closed pull request #10266: Update policy.h (06master...06patch-1) 02https://github.com/bitcoin/bitcoin/pull/10266
  9 2017-04-24 01:10:01	0|bitcoin-git|[13bitcoin] 15kallewoof opened pull request #10267: New -readconfig argument for including external configuration files (06master...06feature-config-readconfig) 02https://github.com/bitcoin/bitcoin/pull/10267
 10 2017-04-24 06:59:42	0|wumpus|"using deep learning frameworks in bitcoin mining", that's hilarious, what bottom-of-the-barrel buzzword factory did that come out of
 11 2017-04-24 07:00:48	0|gmaxwell|lol
 12 2017-04-24 07:06:50	0|jonasschnelli|heh
 13 2017-04-24 08:37:05	0|emucode|what if... that was the deep learning that discovered asicboost? :o
 14 2017-04-24 12:29:11	0|bitcoin-git|[13bitcoin] 15laanwj pushed 6 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/1b25b6df0f08...342b9bc3907e
 15 2017-04-24 12:29:12	0|bitcoin-git|13bitcoin/06master 14663fbae 15Pieter Wuille: FastRandom benchmark
 16 2017-04-24 12:29:12	0|bitcoin-git|13bitcoin/06master 14c21cbe6 15Pieter Wuille: Introduce FastRandomContext::randbool()
 17 2017-04-24 12:29:13	0|bitcoin-git|13bitcoin/06master 14e04326f 15Pieter Wuille: Add ChaCha20
 18 2017-04-24 12:29:24	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #9792: FastRandomContext improvements and switch to ChaCha20 (06master...06chacha) 02https://github.com/bitcoin/bitcoin/pull/9792
 19 2017-04-24 12:52:22	0|BlueMatt|wumpus: the same bottom-of-the-barrel buzzword bingo that came up with "using normal computers to simulate quantum computers to do bitcoin mining"
 20 2017-04-24 12:54:00	0|wumpus|BlueMatt: ah yes the "apply all the buzzwords to bitcoin mining, see what sticks" approach
 21 2017-04-24 12:54:08	0|sipa|BlueMatt: i'm almost done splitting up pertxoutcache into more reasonable pieces... prepare to review 25 commits :)
 22 2017-04-24 12:54:50	0|wumpus|"bitcoin mining on mars"
 23 2017-04-24 12:55:45	0|jtimon|uff, none of those are preparation commits that could be merged beforehand?
 24 2017-04-24 12:56:25	0|BlueMatt|sipa: god damn it
 25 2017-04-24 12:56:43	0|BlueMatt|jtimon: several already have been split out into separate prs, I think :)
 26 2017-04-24 12:57:10	0|BlueMatt|sipa: anyway, you're gonna have to wait until i spend two days on morcos' fee estimation stuff first, I think
 27 2017-04-24 12:57:16	0|jtimon|BlueMatt: what about using deep learning to come up with software optimizations that beat sha256d asics using gpgpu?
 28 2017-04-24 12:57:36	0|BlueMatt|sipa: I keep getting distracted because your stuff is easier for me to review, and I need to spend a day figuring out wtf fee estimation does
 29 2017-04-24 12:58:05	0|BlueMatt|jtimon: sounds good, lets raise 10 mill and then tell investors it didnt work out the next day?
 30 2017-04-24 12:58:10	0|BlueMatt|can split the 5 even?
 31 2017-04-24 12:59:09	0|BlueMatt|or is that kind of overt scam reserved for ICOs now? :/
 32 2017-04-24 13:00:59	0|fanquake|Not sure what an ICO is, but I heard you could raise >10 times that just by building an internet connected juicer
 33 2017-04-24 13:01:51	0|BlueMatt|fanquake: I figured they just got so used to squeezing investors for money all they knew how to build was something that squeezed the contents out of bags
 34 2017-04-24 13:01:59	0|BlueMatt|*rimshot*
 35 2017-04-24 13:03:09	0|wumpus|fanquake BlueMatt lol
 36 2017-04-24 13:04:12	0|wumpus|the bags weren't even blockchain connected smart property :')
 37 2017-04-24 13:09:10	0|BlueMatt|wumpus: clearly we can further optimize their buzzword-compliance
 38 2017-04-24 13:14:08	0|fanquake|Since we seem to be giving Boost removal a good shot for 0.15, does anyone have suggestions for replacing GetNumCores? https://github.com/bitcoin/bitcoin/blob/master/src/util.cpp#L881
 39 2017-04-24 13:14:26	0|fanquake|There is std::thread::hardware_concurrency(), but that seems to count virtual cores, which I don't think we want.
 40 2017-04-24 13:14:51	0|BlueMatt|fanquake: I doubt we'll do boost removal for 0.15
 41 2017-04-24 13:14:58	0|BlueMatt|shit like BOOST_FOREACH, sure
 42 2017-04-24 13:15:07	0|BlueMatt|but all of boost? doubtful, there are still things we need
 43 2017-04-24 13:16:36	0|fanquake|Yea sorry, not the whole lot, but we can remove a decent chunk. Just looking into what else needs to be done to replace some of the less involved Boost usage.
 44 2017-04-24 13:16:43	0|BlueMatt|fair
 45 2017-04-24 13:17:14	0|wumpus|yes, it makes sense to plan ahead a bit, without immediately doing it
 46 2017-04-24 13:18:12	0|wumpus|right, don't count virtual cores, that used to be the case but it makes no sense for our usage
 47 2017-04-24 13:19:15	0|wumpus|it'd create a swarm of threads overwhelming any machine with hyperthreading (+accompanying thread stack overhead), for script validation, and there was no gain at all for that
 48 2017-04-24 13:20:03	0|sipa|BlueMatt: don't worry, there is no hurry
 49 2017-04-24 13:59:09	0|morcos|wumpus: i don't think that is correct
 50 2017-04-24 13:59:23	0|morcos|suppose you have 4 cores (8 virtual cores)
 51 2017-04-24 13:59:24	0|wumpus|fanquake: indeed seems that std has no equivalent to physical_concurrency, on any standard. That's annoying as it is non-trivial to implement
 52 2017-04-24 13:59:35	0|morcos|i think running par=8 (if it let you) would be notably faster
 53 2017-04-24 13:59:59	0|morcos|jeremyrubin and i discussed this at length a while back... i think i commented about it on irc at the time
 54 2017-04-24 14:00:21	0|wumpus|morcos: I think the conclusion at the time was that it made no difference, but sure would make sense to benchmark
 55 2017-04-24 14:00:39	0|morcos|perhaps historical testing on the virtual vs actual cores was polluted by concurrency issues that have now improved
 56 2017-04-24 14:00:47	0|wumpus|I think there are not more ALUs, so there is not really a point in having more threads
 57 2017-04-24 14:01:40	0|wumpus|hyperthreads are basically just a stored register state right?
 58 2017-04-24 14:02:23	0|sipa|wumpus: yes but it helps the scheduler
 59 2017-04-24 14:02:27	0|wumpus|in which case the only speedup using "number of cores" threads would give you is, possibly, excluding other software from running on the cores on the same time
 60 2017-04-24 14:02:36	0|morcos|well this is where i get out of my depth
 61 2017-04-24 14:02:49	0|sipa|if one of the threads is waiting on a read from ram, the other can use the arithmetic unit for example
 62 2017-04-24 14:02:54	0|morcos|wumpus: i'm pretty sure though that the speed up is considerably more than what you might expect from that
 63 2017-04-24 14:02:59	0|wumpus|sipa: ok, I back down, I didn't want to argue this at all
 64 2017-04-24 14:03:34	0|morcos|the reason i haven't tested it myself, is the machine i usually use has 16 cores... so not easy due to remaining concurrency issues to get much more speedup
 65 2017-04-24 14:03:35	0|wumpus|I'm fine with restoring it to number of virtual threads if that's faster
 66 2017-04-24 14:03:54	0|morcos|we should have somene with 4 cores (and 8) actually test it though, i agree
 67 2017-04-24 14:03:57	0|sipa|i would expect (but we should benchmark...) that if 8 scriot validation threads instead of 4 on a quadcore hyperthreading is not faster, it's due to lock contention
 68 2017-04-24 14:04:20	0|morcos|sipa: yeah thats my point, i think lock contention isn't that bad with 8 now
 69 2017-04-24 14:04:22	0|wumpus|on 64-bit systems the additional thread overhead wouldn't be important at least
 70 2017-04-24 14:04:23	0|gmaxwell|I previously benchmarked, a long time ago, it was faster.
 71 2017-04-24 14:04:33	0|gmaxwell|(to use the HT core count)
 72 2017-04-24 14:04:44	0|wumpus|why was this changed at all then?
 73 2017-04-24 14:04:47	0|wumpus|I'm confused
 74 2017-04-24 14:05:04	0|sipa|good question!
 75 2017-04-24 14:05:06	0|gmaxwell|I had no idea we changed it.
 76 2017-04-24 14:05:25	0|wumpus|sigh :(
 77 2017-04-24 14:05:54	0|gmaxwell|What PR changed it?
 78 2017-04-24 14:06:50	0|gmaxwell|In any case, on 32-bit it's probably a good tradeoff... the extra ram overhead is worth avoiding.
 79 2017-04-24 14:07:22	0|wumpus|https://github.com/bitcoin/bitcoin/pull/6361
 80 2017-04-24 14:07:28	0|gmaxwell|PR 6461 btw.
 81 2017-04-24 14:07:37	0|gmaxwell|er lol at least you got it right.
 82 2017-04-24 14:07:45	0|wumpus|the complaint was that systems became unsuably slow when using that many thread
 83 2017-04-24 14:07:51	0|wumpus|so at least I got one thing right, woohoo
 84 2017-04-24 14:07:54	0|sipa|seems i even acked it!
 85 2017-04-24 14:07:57	0|BlueMatt|wumpus: there are more alus
 86 2017-04-24 14:08:38	0|BlueMatt|but we need to improve lock contention first
 87 2017-04-24 14:08:39	0|morcos|anywya, i think in the past the lock contention made 8 threads regardless of cores a bit dicey.. now that is much better (although more still to be done)
 88 2017-04-24 14:09:01	0|BlueMatt|or we can just merge #10192, thats fee
 89 2017-04-24 14:09:04	0|gribble|https://github.com/bitcoin/bitcoin/issues/10192 | Cache full script execution results in addition to signatures by TheBlueMatt · Pull Request #10192 · bitcoin/bitcoin · GitHub
 90 2017-04-24 14:09:10	0|BlueMatt|s/fee/free/
 91 2017-04-24 14:09:20	0|morcos|no, we do not need to improve lock contention first.   but we should probably do that before we increase the max beyond 16
 92 2017-04-24 14:09:26	0|BlueMatt|then we can toss concurrency issues out the window and get more speedup anyway
 93 2017-04-24 14:09:35	0|gmaxwell|wumpus: yea, well in QT I thought we also diminished the count by 1 or something?  but yes, if the motivation was to reduce how heavily the machine was used, thats fair.
 94 2017-04-24 14:09:56	0|sipa|the benefit of using HT cores is certainly not a factor 2
 95 2017-04-24 14:09:58	0|wumpus|gmaxwell: for the default I think this makes a lot of sense, yes
 96 2017-04-24 14:10:10	0|gmaxwell|morcos: right now on my 24/28 physical core hosts going beyond 16 still reduces performance.
 97 2017-04-24 14:10:11	0|wumpus|gmaxwell: do we also restrict the maximum par using this? that'd make less sense
 98 2017-04-24 14:10:50	0|wumpus|if someone *wants* to use the virtual cores they should be able to by setting -par=
 99 2017-04-24 14:10:51	0|BlueMatt|sipa: sure, but the shared cache helps us get more out of it than some others, as morcos points out
100 2017-04-24 14:11:30	0|BlueMatt|(because it means our thread contention issues are less)
101 2017-04-24 14:12:05	0|morcos|gmaxwell: yeah i've been bogged down in fee estimation as well (and the rest of life) for a while now.. otherwise i would have put more effort into jeremy's checkqueue
102 2017-04-24 14:12:36	0|BlueMatt|morcos: heh, well now you can do other stuff while the rest of us get bogged down in understanding fee estimation enough to review it :p
103 2017-04-24 14:12:36	0|wumpus|[to answer my own question: no, the limit for par is MAX_SCRIPTCHECK_THREADS, or 16]
104 2017-04-24 14:12:54	0|morcos|but to me optimizing for more than 16 cores is pretty valuable as miners could use beefy machines and be less concerned by block validation time
105 2017-04-24 14:14:37	0|BlueMatt|morcos: i think you may be surprised by the number of mining pools that are on VPSes that do not have 16 cores :/
106 2017-04-24 14:15:34	0|gmaxwell|I assume right now most of the time block validation is bogged in the parts that are not as concurrent. simple because caching makes the concurrent parts so fast. (and soon to hopefully increase with bluematt's patch)
107 2017-04-24 14:17:54	0|gmaxwell|improving sha2 speed, or transaction malloc overhead are probably bigger wins now for connection at the tip than parallelism beyond 16 (though I'd like that too).
108 2017-04-24 14:18:21	0|BlueMatt|sha2 speed is big
109 2017-04-24 14:18:27	0|morcos|yeah lots of things to do actually...
110 2017-04-24 14:18:57	0|gmaxwell|BlueMatt: might be a tiny bit less big if we didn't hash the block header 8 times for every block. :P
111 2017-04-24 14:21:27	0|BlueMatt|ehh, probably, but I'm less rushed there
112 2017-04-24 14:21:43	0|BlueMatt|my new cache thing is about to add a bunch of hashing
113 2017-04-24 14:21:50	0|BlueMatt|1 sha round per tx
114 2017-04-24 14:22:25	0|BlueMatt|and sigcache is obviously a ton
115 2017-04-24 14:47:43	0|bitcoin-git|[13bitcoin] 15laanwj pushed 7 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/342b9bc3907e...fa1ac2881f2a
116 2017-04-24 14:47:44	0|bitcoin-git|13bitcoin/06master 14071c955 15Wladimir J. van der Laan: wallet: Get rid of fFileBacked...
117 2017-04-24 14:47:44	0|bitcoin-git|13bitcoin/06master 1471afe3c 15Wladimir J. van der Laan: wallet: Introduce database handle wrapper...
118 2017-04-24 14:47:45	0|bitcoin-git|13bitcoin/06master 14be9e1a9 15Wladimir J. van der Laan: wallet: Reduce references to global bitdb environment
119 2017-04-24 14:47:58	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #9951: Wallet database handling abstractions/simplifications (06master...062017_03_wallet_dbwrapper) 02https://github.com/bitcoin/bitcoin/pull/9951
120 2017-04-24 14:58:28	0|wumpus|fanquake: re: #10260, how is homebrew versioned? should the instructions always apply to only the latest version?
121 2017-04-24 14:58:29	0|gribble|https://github.com/bitcoin/bitcoin/issues/10260 | [doc] Minor corrections to osx dependencies by fanquake · Pull Request #10260 · bitcoin/bitcoin · GitHub
122 2017-04-24 20:37:02	0|achow101|gmaxwell: any update on publishing that alert key?
123 2017-04-24 23:08:06	0|bitcoin-git|[13bitcoin] 15sipa pushed 3 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/fa1ac2881f2a...c73af5416b66
124 2017-04-24 23:08:07	0|bitcoin-git|13bitcoin/06master 14344a2c4 15Pieter Wuille: Add support for std::unordered_{map,set} to memusage.h
125 2017-04-24 23:08:07	0|bitcoin-git|13bitcoin/06master 14e6756ad 15Pieter Wuille: Switch CCoinsMap from boost to std unordered_map
126 2017-04-24 23:08:08	0|bitcoin-git|13bitcoin/06master 14c73af54 15Pieter Wuille: Merge #10249: Switch CCoinsMap from boost to std unordered_map...
127 2017-04-24 23:08:26	0|bitcoin-git|[13bitcoin] 15sipa closed pull request #10249: Switch CCoinsMap from boost to std unordered_map (06master...06stdcoinmap) 02https://github.com/bitcoin/bitcoin/pull/10249