1 2018-03-12 00:59:48	0|bitcoin-git|[13bitcoin] 15fanquake closed pull request #12672: Consensus: Remove unused dependencies (06master...06removed-unused-deps) 02https://github.com/bitcoin/bitcoin/pull/12672
  2 2018-03-12 06:03:48	0|bitcoin-git|[13bitcoin] 15luke-jr opened pull request #12674: RPC: Support addnode onetry without making the connection priviliged (06master...06rpc_onetry_nonpriv) 02https://github.com/bitcoin/bitcoin/pull/12674
  3 2018-03-12 06:32:37	0|eklitzke|what is the motivatoin for obfuscating values with the obfuscate key in the leveldb databases?
  4 2018-03-12 06:33:27	0|sipa|av software :(
  5 2018-03-12 06:34:04	0|eklitzke|strange
  6 2018-03-12 06:37:45	0|achow101|eklitzke: some transactions in the blockchain contain virus signatures which AVs will catch and then remove from the chainstate database thus corrupting it. So obfuscating the data in the db prevents that from happening
  7 2018-03-12 06:38:10	0|eklitzke|huh, that is really fascinating
  8 2018-03-12 06:39:33	0|achow101|AVs are routinely a source of problems with running core in windows. More recently it's been AVs blocking connections to and from port 8333
  9 2018-03-12 06:44:49	0|Randolf|Anti-spyware software is getting more aggressive against anything that looks like it could be doing cryptocurrency mining.  And unfortunately it's often classified as Bitcoin Mining Riskware.
 10 2018-03-12 06:45:43	0|Randolf|Sadly, many users haven't even a clue that it's possible to add exceptions into their anti-virus and anti-spyware software's configuration.
 11 2018-03-12 07:16:28	0|eklitzke|i just wrote some code to dump the utxo database, the compression stuff is wild
 12 2018-03-12 07:16:30	0|eklitzke|https://gist.github.com/eklitzke/03b1c799408d4bcceca9c57ab1ff8c9f
 13 2018-03-12 07:16:50	0|eklitzke|i had no idea so much weird stuff went into making the values small
 14 2018-03-12 07:19:06	0|sipa|eklitzke: it started out as an experiment in seeing how small the utxo set could get
 15 2018-03-12 07:19:50	0|sipa|and ended up being the actual database format
 16 2018-03-12 07:19:56	0|sipa|look at how the txouts were encoded before per-txout, it close to simiarly crazy
 17 2018-03-12 07:20:33	0|eklitzke|i was originally just trying to get the height of everything in the utxo database, but once i saw the serialization code for amounts i had to add it
 18 2018-03-12 07:20:41	0|eklitzke|very cool
 19 2018-03-12 07:20:42	0|sipa|hah
 20 2018-03-12 07:21:10	0|sipa|in retrospect, it's very much overkill
 21 2018-03-12 07:23:29	0|sipa|https://github.com/bitcoin/bitcoin/blob/v0.14.0/src/coins.h#L22
 22 2018-03-12 07:33:40	0|kallewoof|sipa: rumor has it overkill is your middle name
 23 2018-03-12 07:44:06	0|kallewoof|I did some time profiling on current master (ran for about 30 mins) and found an oddity. May not be important, but 14.4% of the CPU time (9.48s) was spent in CCoinsViewDB::Upgrade(), in the pcursor->Seek call at the top (which is used to determine if an upgrade should happen or not).
 24 2018-03-12 07:45:25	0|kallewoof|Also noticed 45.4% of the time is spent in leveldb::. More than main thread (main thread = 32.4%)..
 25 2018-03-12 07:49:58	0|eklitzke|did you filter out the bgthread
 26 2018-03-12 07:52:07	0|kallewoof|eklitzke: No filtering, no.
 27 2018-03-12 07:57:38	0|eklitzke|leveldb creates a background worker thread (you'll see BGThread in the stack trace) which tends to use a lot of cpu/disk time
 28 2018-03-12 07:57:52	0|eklitzke|it's kind of confusing because it's a thread created by leveldb itself, not managed by bitcoin
 29 2018-03-12 08:04:04	0|bitcoin-git|[13bitcoin] 15kentang8228 opened pull request #12675: litecoin (06master...06master) 02https://github.com/bitcoin/bitcoin/pull/12675
 30 2018-03-12 08:04:56	0|bitcoin-git|[13bitcoin] 15fanquake closed pull request #12675: litecoin (06master...06master) 02https://github.com/bitcoin/bitcoin/pull/12675
 31 2018-03-12 08:09:14	0|kallewoof|Yep, that's the one using 45.4%.
 32 2018-03-12 08:26:05	0|kallewoof|Hum. I tried adding snappy support and leveldb dropped a ton. Still need to verify results, but looks pretty interesting.
 33 2018-03-12 08:28:12	0|eklitzke|what platform?
 34 2018-03-12 08:29:06	0|eklitzke|there is something broken with how we're using the block cache right now (which is actually affected by whether you have snappy compression on or not), waiting to get my max_open_files change in before sending out that pr
 35 2018-03-12 08:31:17	0|eklitzke|in general though i would not expect snappy compression to help since the keys and values are already compressed
 36 2018-03-12 08:41:08	0|kallewoof|eklitzke: Yeah, I'm seeing roughly the same performance now. Good point on already compressed.
 37 2018-03-12 08:42:41	0|kallewoof|Actually I think performance dropped a bit with snappy, but very imprecise measurement (total CPU time over total time for separate runs).
 38 2018-03-12 09:56:08	0|George_e30|Hello everyone!
 39 2018-03-12 09:56:34	0|bitcoin-git|[13bitcoin] 15dexX7 opened pull request #12676: Show "replaceable" flag, when decoding raw transactions (06master...06rpc-raw-replaceable-flag) 02https://github.com/bitcoin/bitcoin/pull/12676
 40 2018-03-12 09:58:44	0|George_e30|I install b2x-node, connect to it with stratum, and try to mine. But I have error in log:
 41 2018-03-12 09:59:33	0|George_e30|ERROR Bitcoin2x Block decode failed 10:23:23: *** REJECTED :( Bitcoin2x block 543500 1 txs
 42 2018-03-12 10:00:17	0|George_e30|what`s wrong am I do? maybe something in .conf?
 43 2018-03-12 10:00:38	0|George_e30|P.S. sorry for my English
 44 2018-03-12 10:02:45	0|George_e30|and now they contined:
 45 2018-03-12 10:02:47	0|George_e30|10:53:29: *** REJECTED :( Bitcoin2x block 543520 7 txs 2018-03-12 10:53:29: REJECTED B2X block 543520 10:53:31: ERROR Bitcoin2x Block decode failed 10:53:31: *** REJECTED :( Bitcoin2x block 543520 7 txs 2018-03-12 10:53:31: REJECTED B2X block 543520
 46 2018-03-12 10:04:55	0|ctrlbreak|Hey George, this may not be the blockchain you are looking for.  The Bitcoin mainnet is currently at block 513162.
 47 2018-03-12 10:13:54	0|George_e30|not bitcoin, bitcoin2x
 48 2018-03-12 10:15:19	0|kinlo|George_e30: wrong channel, this channel is only for bitcoin core, not anything else
 49 2018-03-12 11:22:33	0|AShoker|Hi guys, any pointers to a reply to this blog post? https://medium.com/@jonaldfyookball/mathematical-proof-that-the-lightning-network-cannot-be-a-decentralized-bitcoin-scaling-solution-1b8147650800
 50 2018-03-12 13:04:04	0|dongcarl|AShoker: This is a dev channel
 51 2018-03-12 13:46:27	0|contrapumpkin|launching 0.16.0 for the first time (upgrading from 0.15.1) and it thinks for a while then shows a scary error that a fatal error occurs and dies. The log says "ReadBlockFromDisk: Deserialize or I/O error -CAutoFile::read:fread failed: unspecified iostream_category error at CBlockDiskPos(nFile=1207, npos=133025664)". I assume this just means some of my block data is somehow corrupt? Can I pass in a magic flag to reuse as
 52 2018-03-12 13:46:27	0|contrapumpkin|much of it as possible while redownloading only the broken stuff?
 53 2018-03-12 13:49:07	0|contrapumpkin|right now it just refuses to do anything at all and quits after displaying the fatal error
 54 2018-03-12 14:03:55	0|esotericnonsense|you can use -reindex and that should use as much as possible of the non-corrupt data I believe. I'm not sure there's a more efficient way to do it
 55 2018-03-12 14:04:00	0|esotericnonsense|whether there's8
 56 2018-03-12 17:06:43	0|eklitzke|by my math 0.4% of utxos are 1 day old (or newer), does that sound approximately right? just trying to sanity check my results
 57 2018-03-12 17:09:17	0|sipa|that seems plausible
 58 2018-03-12 17:09:25	0|sipa|that's around 200k UTXOs
 59 2018-03-12 17:10:05	0|eklitzke|yeah that seams reasonable given 144 blocks per day
 60 2018-03-12 17:10:27	0|sipa|or 1400 utxos per block
 61 2018-03-12 17:11:36	0|eklitzke|i'm trying to figure out for a new utxo, the probability distribution of it being spent within N blocks
 62 2018-03-12 17:39:46	0|jimpo|eklitzke: Over what range of UTXOS (like UTXOs created since when)?
 63 2018-03-12 17:39:52	0|jimpo|I can pull that for you if you need
 64 2018-03-12 17:43:38	0|eklitzke|i wrote a python script that dumped my chainstate directory, this is what i got https://monad.io/height-pdf.png
 65 2018-03-12 17:43:43	0|eklitzke|which i think is sufficient
 66 2018-03-12 17:43:53	0|eklitzke|that's the probability distribution for any utxo being at a given chain height
 67 2018-03-12 17:44:15	0|eklitzke|it's cool i can dump the utxo set in a minute or two
 68 2018-03-12 17:45:50	0|jimpo|neat. y-axis is probability that UTXO is spent within a day?
 69 2018-03-12 17:46:41	0|eklitzke|it's the proportion of utxos at that chain height
 70 2018-03-12 17:47:42	0|jimpo|er, right
 71 2018-03-12 18:01:52	0|bitcoin-git|[13bitcoin] 15luke-jr opened pull request #12677: RPC: Add ancestor{count,size,fees} to listunspent output (06master...06listunspent_ancestorinfo) 02https://github.com/bitcoin/bitcoin/pull/12677
 72 2018-03-12 18:32:12	0|cfields|<sipa> sdaftuar: seems my benchmark was off, and the jacobi symbol implementation in GMP takes 2us rather than 0.2us
 73 2018-03-12 18:32:17	0|cfields|:(
 74 2018-03-12 18:34:04	0|cfields|sipa: i guess you're ready to bail on EC for set hashing, then?
 75 2018-03-12 18:36:42	0|cfields|oh, I guess so if ECMH == elliptic curve multi-set hash
 76 2018-03-12 18:51:23	0|sipa|cfields: well 9.3us isn't horrible
 77 2018-03-12 18:53:48	0|sipa|it means maybe a 20-30% CPU increase overall for full validation, though
 78 2018-03-12 18:57:19	0|cfields|sipa: is it the case that gmp's jacobi impl isn't as far away from others as you had expected? Or would it still be substantially worse if we rolled our own?
 79 2018-03-12 18:57:54	0|sipa|cfields: so, there are two ways of doing hashing onto curves
 80 2018-03-12 18:59:01	0|sipa|A: do { x = generate 32 bytes; compute y=sqrt(x^3+7); if it exists return (x,y); }
 81 2018-03-12 18:59:29	0|sipa|B: do { x = generate 32 bytes; compute jacobi(x^3,7); if it is 1, return (x, sqrt(x^3+7)); }
 82 2018-03-12 19:00:04	0|sipa|given that around 50% of x values are acceptable, you on average need 2 iterations
 83 2018-03-12 19:00:22	0|sipa|so method A does 2 sqrts, method B does 2 jacobis + 1 sqrt
 84 2018-03-12 19:00:42	0|sipa|in other words, B is better than A when jacobi is more than 2x as fast as a sqrt
 85 2018-03-12 19:01:01	0|cfields|ok
 86 2018-03-12 19:01:49	0|sipa|with sqrt=5us and jacobi=0.2us, B was clearly worth it
 87 2018-03-12 19:02:04	0|sipa|but it seems that was wrong, and gmp's jacobi is more like 2us
 88 2018-03-12 19:02:15	0|cfields|ok, i'm with you now. thanks.
 89 2018-03-12 19:02:49	0|sipa|however, with gmp not being as crazy fast as i believed, it may be feasible to write a jacobi implementation that is faster than GMP
 90 2018-03-12 19:02:51	0|bitcoin-git|[13bitcoin] 15vasild opened pull request #12678: Scripts and tools: Fix a few compilation issues with Clang 7 and -Werror (06master...06master-compilation-fixes-with-clang7-werror) 02https://github.com/bitcoin/bitcoin/pull/12678
 91 2018-03-12 19:02:59	0|sipa|we had previous given up on that, as 0.2us seemed impossible to beat
 92 2018-03-12 19:03:35	0|cfields|heh yes, i see now why it was written off so easily
 93 2018-03-12 19:03:58	0|sipa|but it is diminishing returns... we're not going to beat 6us or so overall
 94 2018-03-12 19:04:28	0|cfields|right
 95 2018-03-12 19:05:04	0|cfields|is this the WIP doc about gmp's jacobi symbol impl that you mentioned? https://gmplib.org/list-archives/gmp-bugs/attachments/20170821/bfac90c6/attachment-0001.pdf
 96 2018-03-12 19:05:53	0|sipa|possibly
 97 2018-03-12 19:06:13	0|sipa|muhash would overall run in maybe 3-4us per update
 98 2018-03-12 19:06:40	0|sipa|so it's only a small constant factor difference
 99 2018-03-12 19:08:31	0|cfields|ah, i figured it was more substantial than that. ok, so a ~2x slowdown is reasonable to consider.
100 2018-03-12 19:09:10	0|intcat|(not sure if correct channel to ask) i am trying to set up bitcoin core gitian build for 0.16.0, but dependency resolution fails on expat; it attempts to download from https://bitcoincore.org/depends-sources but gets 404
101 2018-03-12 19:09:56	0|cfields|intcat: it tries to download from the upstream source first, and only hits bitcoincore.org if that fails. Most likely you don't have net access inside your container/vm.
102 2018-03-12 19:10:14	0|intcat|cfields: actually, yes i do
103 2018-03-12 19:10:25	0|intcat|a lot of other depends are downloaded correctly before this one
104 2018-03-12 19:10:35	0|cfields|intcat: sec, let's check if upstream is down atm
105 2018-03-12 19:10:50	0|intcat|i noticed master has a different upstream url than 0.16.0 tag
106 2018-03-12 19:11:59	0|intcat|sourceforge vs github. and that kind of thing can happen, but it seems that the fallback is failing too
107 2018-03-12 19:27:10	0|cfields|intcat: not sure if it's down, but I'm fixing up the links at bitcoincore.org anyway, since it needed to be done
108 2018-03-12 19:27:35	0|intcat|:)
109 2018-03-12 19:59:46	0|intcat|cfields: seems the container im trying to build in borks on the sourceforge redirect
110 2018-03-12 20:00:10	0|cfields|intcat: not surprising, sf redirects several times
111 2018-03-12 20:00:55	0|intcat|running the same command on another machine works, though
112 2018-03-12 20:01:09	0|intcat|but still, would be nice to have working fallback
113 2018-03-12 20:12:26	0|provoostenator|I'm looking at the accounts -> labels PR by ryanofsky: https://github.com/bitcoin/bitcoin/pull/11536/files#diff-df7d84ff2f53fcb2a0dc15a3a51e55ceR3849
114 2018-03-12 20:12:42	0|provoostenator|I'm confused what causes "getaccountaddress" to not show up in bitcoin-cli help
115 2018-03-12 20:13:59	0|ryanofsky|the help code dedups by function pointer, so there are aliases to the same function it is not listed twice
116 2018-03-12 20:14:21	0|provoostenator|I figured it had to be something like that. Thanks.
117 2018-03-12 20:15:49	0|ryanofsky|np, check is here if you're curious: https://github.com/bitcoin/bitcoin/blob/d8d9162f5bad39b2720dd2b2da237c6159e4755f/src/rpc/server.cpp#L179
118 2018-03-12 20:54:06	0|provoostenator|ryanofsky how did you rebase this, but it doesn't show your name on the commit: https://github.com/ryanofsky/bitcoin/commit/55c5289acbaa4c3a510527df4e55db20619b47d4
119 2018-03-12 21:00:33	0|ryanofsky|not intentional in this case, but cherry-pick with GIT_COMMITTER_NAME set (https://github.com/ryanofsky/home/blob/df082664c2e45bb27f22fce706002a0971504c74/src/git-pr.sh#L177)
120 2018-03-12 23:28:35	0|achow101|In the unit tests, is there a way to test that an assert was hit?
121 2018-03-12 23:47:12	0|cfields|intcat: should be good now
122 2018-03-12 23:47:42	0|meshcollider|achow101: I don't think so, I think asserts aren't supposed to be hit at all in theory ;)
123 2018-03-12 23:48:18	0|intcat|cfields: https://bitcoincore.org/depends-sources/expat-2.2.5.tar.bz2 works :)
124 2018-03-12 23:48:33	0|intcat|https://bitcoincore.org/depends-sources/expat-2.2.1.tar.bz2 unfortunately doesn't though, so my v0.16.0 build will still fail
125 2018-03-12 23:48:53	0|cfields|meshcollider: right. achow101 relying on catching an assert implies that it has side-effects in code.
126 2018-03-12 23:49:04	0|cfields|intcat: sec, will do a one-time sync of 0.16.
127 2018-03-12 23:49:16	0|achow101|cfields: meshcollider: I got my logic backwards
128 2018-03-12 23:49:38	0|achow101|I don't want it to assert, but to make sure my test works, I need to make sure the assert was hit in the failure case
129 2018-03-12 23:50:32	0|sipa|achow101: you can replace them with an exception (some of the utxo logic throw std::logic_error for this reason)
130 2018-03-12 23:51:38	0|meshcollider|yeah boost can check for exceptions being thrown, with the likes of BOOST_CHECK_THROW. Just not asserts
131 2018-03-12 23:54:02	0|cfields|intcat: done
132 2018-03-12 23:54:16	0|intcat|cfields: many thanks
133 2018-03-12 23:54:59	0|cfields|intcat: np, thanks for the reminder. I've been meaning to fix the bridge for months.