1 2017-04-20 02:13:17	0|luke-jr|Does this look like a malicious peer? https://drive.google.com/file/d/0ByBPjWWp3nRqTHd2T2JqeVlGLUk/view
  2 2017-04-20 03:58:10	0|bitcoin-git|[13bitcoin] 15NicolasDorier closed pull request #10184: [Wallet] Worst case performance improvement on KeyPool filtering (06master...06hdinternalperf) 02https://github.com/bitcoin/bitcoin/pull/10184
  3 2017-04-20 04:48:52	0|gmaxwell|morcos: that writeup is pretty much just what I hoped. I thougth there was also an element of using the current mempool queue too?
  4 2017-04-20 06:06:05	0|wumpus|luke-jr: I don't think we usually backport help texts, butit's fine w/ fe
  5 2017-04-20 06:06:56	0|wumpus|indeed, I've enabled the auto-cancel feature in travis where superceded jobs are automatically cancelled
  6 2017-04-20 06:07:26	0|wumpus|if this doesn't work properly I could disable it again, but in general speeding up tests by avoiding unncessary work is a good thing
  7 2017-04-20 06:09:21	0|wumpus|however travis admits it's still in beta stage
  8 2017-04-20 06:19:18	0|bitcoin-git|13bitcoin/060.14 1451c787d 15Warren Togami: Clarify importprivkey help text with example of blank label without rescan...
  9 2017-04-20 06:19:18	0|bitcoin-git|[13bitcoin] 15laanwj pushed 1 new commit to 060.14: 02https://github.com/bitcoin/bitcoin/commit/51c787dfb4963d2a59dc8944f45e065be0a06613
 10 2017-04-20 06:26:56	0|jonasschnelli|[08:26:20]  <luke-jr>	[21:56:40] Travis jobs are randomly cancelling?
 11 2017-04-20 06:27:13	0|jonasschnelli|I sometime manually cancel (if I force push a couple of times)
 12 2017-04-20 06:27:26	0|jonasschnelli|no need to build obsolete branches
 13 2017-04-20 06:46:09	0|NicolasDorier|sipa: Have you considered https://github.com/bitcoin/bitcoin/pull/10148#issuecomment-295599772 ? I think it would make the whole thing a lot easier, with less risk.
 14 2017-04-20 06:56:46	0|NicolasDorier|edited it
 15 2017-04-20 07:30:40	0|sipa|NicolasDorier: we already have a maximum size on the cache
 16 2017-04-20 07:30:49	0|sipa|it's what -dbcache controls
 17 2017-04-20 07:30:56	0|NicolasDorier|I know
 18 2017-04-20 07:31:10	0|sipa|then i don't understand your suggestion
 19 2017-04-20 07:32:38	0|NicolasDorier|sipa: basically following the same thing as done here
 20 2017-04-20 07:32:39	0|NicolasDorier|https://github.com/bitcoin/bitcoin/blob/2584925077f9658b3953ad931b74779006e59807/src/validation.cpp#L1982
 21 2017-04-20 07:32:59	0|NicolasDorier|with a fCacheCriticalBatchSize
 22 2017-04-20 07:33:11	0|NicolasDorier|the problem that I found out by trying that
 23 2017-04-20 07:33:23	0|NicolasDorier|is that I was not expecting a flush to wipeout the cache clean
 24 2017-04-20 07:33:48	0|NicolasDorier|I thought only dirty entries in the cache would be commited into a batch
 25 2017-04-20 07:33:49	0|sipa|i think you misunderstand what this does
 26 2017-04-20 07:34:14	0|sipa|it's not about limiting the number of dirty entries
 27 2017-04-20 07:34:22	0|NicolasDorier|This flush the CoinViewCache when certain condition arise
 28 2017-04-20 07:34:25	0|NicolasDorier|right ?
 29 2017-04-20 07:34:34	0|sipa|it's splitting them up over different batches
 30 2017-04-20 07:34:52	0|sipa|we want to limit the size of the batches, as they cause a memory blowup
 31 2017-04-20 07:35:00	0|NicolasDorier|I know
 32 2017-04-20 07:35:21	0|NicolasDorier|My point is that we can prevent a batch to be too big at the CoinViewCache level
 33 2017-04-20 07:35:27	0|sipa|no
 34 2017-04-20 07:36:13	0|NicolasDorier|I was assuming CoinViewCache was not deleting all cached coin during a flush
 35 2017-04-20 07:36:22	0|NicolasDorier|I was wrong on that
 36 2017-04-20 07:36:24	0|sipa|yeah, that's weird
 37 2017-04-20 07:36:44	0|sipa|but we've tried mny things to change that, and they all make things slower
 38 2017-04-20 07:36:50	0|NicolasDorier|if it was not deleting all cached coin during a flush, then we could flush when the CoinViewCache know that a batch size would be too big
 39 2017-04-20 07:37:00	0|NicolasDorier|ha I see
 40 2017-04-20 07:37:07	0|sipa|that's an eventual goal here
 41 2017-04-20 07:37:35	0|sipa|but this is doing something much more basic: not requiring that flushes are consistent with blocks
 42 2017-04-20 07:38:11	0|sipa|let me take a step back
 43 2017-04-20 07:38:57	0|NicolasDorier|I understood your PR and the goal of it
 44 2017-04-20 07:39:23	0|NicolasDorier|I thought there was a simpler way of doing, but I relied on the assumption that a flush did not throw away all the cachedCoins
 45 2017-04-20 07:39:52	0|NicolasDorier|if my assumption was right, fixing the problem you attempt to solve would have been way more easy.
 46 2017-04-20 07:40:08	0|sipa|the reason why we have the cache, and go through all this effort (as opposed to just using leveldb's caching), is that we can do an awesome optimization: if a utxo is created in the cache, and spent before it is ever flushed, we can just delete it from the cache, preventing it from ever being serialized or written to disk
 47 2017-04-20 07:40:24	0|NicolasDorier|yes I am aware of it
 48 2017-04-20 07:40:40	0|sipa|that means that we must maximize the time a utxo is in the cache before being flushed
 49 2017-04-20 07:41:10	0|sipa|what you're suggesting is limiting the amount of dirty entries in the cache
 50 2017-04-20 07:41:24	0|NicolasDorier|right
 51 2017-04-20 07:41:28	0|sipa|that would radically reduce our ability to use that optimization above
 52 2017-04-20 07:41:36	0|NicolasDorier|yes because at every flush
 53 2017-04-20 07:41:42	0|NicolasDorier|you throw away all the cachedCoins
 54 2017-04-20 07:41:45	0|sipa|no
 55 2017-04-20 07:41:57	0|sipa|that has nothing to do with it
 56 2017-04-20 07:42:03	0|sipa|it's more fundamental
 57 2017-04-20 07:42:37	0|NicolasDorier|ah yes let me think
 58 2017-04-20 07:42:40	0|sipa|you're suggesting more frequent flushes (which don't wipe the cache, just mark the written entries as non-dirty, roght?)
 59 2017-04-20 07:42:45	0|NicolasDorier|yes
 60 2017-04-20 07:43:14	0|sipa|more frequent flushes would reduce the time between a utxo is created and the time they hit disk
 61 2017-04-20 07:43:21	0|NicolasDorier|aaaaah
 62 2017-04-20 07:43:23	0|NicolasDorier|got it
 63 2017-04-20 07:43:31	0|sipa|at that point, it's too late
 64 2017-04-20 07:43:46	0|NicolasDorier|yes, I understand now
 65 2017-04-20 07:43:52	0|NicolasDorier|I remove my comment
 66 2017-04-20 07:44:06	0|NicolasDorier|removed
 67 2017-04-20 07:44:10	0|sipa|so my eventual goal is to have continuous flushing that indeed only writes small amounts of entries
 68 2017-04-20 07:44:27	0|sipa|but not by limiting the amount of dirty entries
 69 2017-04-20 07:44:49	0|NicolasDorier|mmh so how ?
 70 2017-04-20 07:45:00	0|sipa|just have sort of a rolling window... once an entry has been in the cache without being written, flush it
 71 2017-04-20 07:45:26	0|sipa|however, that requires that the disk cache is allowed to be in an inconsistent state
 72 2017-04-20 07:45:40	0|sipa|and needs replay to correct at startup
 73 2017-04-20 07:46:00	0|sipa|which this PR is the first step towards (and a great memory usage improvement on its own)
 74 2017-04-20 07:46:06	0|NicolasDorier|I see
 75 2017-04-20 07:47:06	0|sipa|it's a very usual cache structure, which only works because we mostly create and delete recently created entries
 76 2017-04-20 07:47:21	0|sipa|and only write them once, read them once, delete them once
 77 2017-04-20 07:47:32	0|sipa|most caches are optimized for many reads
 78 2017-04-20 07:48:36	0|NicolasDorier|ok this is clear thanks. So I am fine with 10148, would just like to see a simple python test though
 79 2017-04-20 07:49:40	0|sipa|agree, if you have ideas for what it should do, let me know
 80 2017-04-20 07:51:12	0|NicolasDorier|sipa: A dbbatchsize of 1 bytes, dbcrashratio of 10. Check if the node can eventually sync 200 blocks and have the same utxoset hash than the other node.
 81 2017-04-20 07:51:26	0|NicolasDorier|does not test the forking logic though
 82 2017-04-20 07:51:32	0|sipa|ah, yes
 83 2017-04-20 07:52:13	0|sipa|well we could construct forks as well, i guess
 84 2017-04-20 07:52:42	0|NicolasDorier|yes, I don't think this is very difficult to test. Just the dbbatchsize being in MB make it inconvenient
 85 2017-04-20 07:52:56	0|NicolasDorier|maybe have a magic number just for the tests be enough
 86 2017-04-20 07:52:58	0|sipa|agree
 87 2017-04-20 07:53:09	0|sipa|it can be in bytes
 88 2017-04-20 07:53:23	0|sipa|it's a test-only option really
 89 2017-04-20 07:53:31	0|NicolasDorier|yes indeed
 90 2017-04-20 08:06:53	0|xinxi|gmaxwell: Please check my PM.
 91 2017-04-20 09:03:47	0|wumpus|indeed, no specific reason that abortrescan should not allowed in safe mode, though should anything that triggers rescan be allowed?
 92 2017-04-20 09:04:00	0|wumpus|safe mode = the block chain is in uncertain state
 93 2017-04-20 09:05:01	0|wumpus|I guess it cannot really hurt
 94 2017-04-20 09:28:58	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #10232: [0.14] release-notes: Accurately explain getblocktemplate improvements (060.14...060.14_relnotes_mining) 02https://github.com/bitcoin/bitcoin/pull/10232
 95 2017-04-20 09:41:55	0|gmaxwell|sipa: I think for testing that atomic flushing what we really need is a few bugs to insert in the code, and see that the tests catch them... at least that would give some feel for how adequate the tests are.
 96 2017-04-20 09:46:59	0|jonasschnelli|gmaxwell: You once mentioned that there are better filters for "block filters" then bloom. I think what mostly matters is the compactness. What filter type would you recommend?
 97 2017-04-20 09:47:47	0|bitcoin-git|[13bitcoin] 15laanwj pushed 9 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/c91ca0ace9bd...a987def4f629
 98 2017-04-20 09:47:48	0|bitcoin-git|13bitcoin/06master 1423e6e64 15John Newbery: Allow disconnectnode() to be called with node id...
 99 2017-04-20 09:47:48	0|bitcoin-git|13bitcoin/06master 14d6564a2 15John Newbery: [tests] fix nodehandling.py flake8 warnings
100 2017-04-20 09:47:49	0|bitcoin-git|13bitcoin/06master 14e367ad5 15John Newbery: [tests] rename nodehandling to disconnectban
101 2017-04-20 09:48:07	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #10143: [net] Allow disconnectnode RPC to be called with node id (06master...06disconnect_node_by_id) 02https://github.com/bitcoin/bitcoin/pull/10143
102 2017-04-20 09:48:26	0|sipa|jonasschnelli: the optimal datastructure is something like a bloom filter, but with only 1 hash function, and then instead of storing the bits directly, store the distances between the 1s
103 2017-04-20 09:48:53	0|jonasschnelli|sipa: okay.. I try to parse your answer.. give me some days. :)
104 2017-04-20 09:50:07	0|sipa|it's 44% smaller than bloom filters, afaik
105 2017-04-20 09:50:19	0|sipa|but much much slower to look things up
106 2017-04-20 09:50:36	0|sipa|as you need to decompress the data
107 2017-04-20 09:52:07	0|jonasschnelli|So just use a single MurmurHash? I need to understand what you mean with "store the distances between the 1s". Let me think a bit about it.
108 2017-04-20 09:53:16	0|sipa|if you only have 1 hash function, there will be a few 1s and many 0s in your filter
109 2017-04-20 09:54:08	0|sipa|which can be compressed using run length encoding
110 2017-04-20 09:54:55	0|jonasschnelli|Okay. Got that part.
111 2017-04-20 09:54:58	0|gmaxwell|jonasschnelli: I think this stuff is a waste of time. even with commited filters the users privacy is significantly harmed. Saving 14kb/s hardly seems worth it.
112 2017-04-20 09:55:28	0|jonasschnelli|gmaxwell: You propose to just scan all blocks?
113 2017-04-20 09:55:40	0|gmaxwell|sipa: as far as slower, I doubt it, even a pretty huge filter the decompression time would be insignificant compared to transfer time.
114 2017-04-20 09:55:54	0|gmaxwell|jonasschnelli: scan blocks since the creation of the keys.
115 2017-04-20 09:56:15	0|sipa|gmaxwell: i mean cpu time
116 2017-04-20 09:56:31	0|jonasschnelli|gmaxwell: Yes. But catching up two weeks on a cellphone would result in 144*14MB = 2GB of data...
117 2017-04-20 09:56:45	0|jonasschnelli|Resulting in users stick to the current BF SPV model
118 2017-04-20 09:57:33	0|gmaxwell|users don't use that in any large number now.
119 2017-04-20 09:57:43	0|gmaxwell|jonasschnelli: already almost no one uses multibit/android wallet due to poor performance, even though it completely destroys the user's privacy. You cannot compete with server based lookup which is what almost everyone uses.
120 2017-04-20 09:58:29	0|gmaxwell|also, why would the cellphone be two weeks out of date? should be catching up in the background...
121 2017-04-20 09:58:34	0|jonasschnelli|Hmm... well, .. SPV BF is pretty fast and I think it's used by a large usergroup... though, I don't have reliable numbers.
122 2017-04-20 09:59:01	0|sipa|what is BF?
123 2017-04-20 09:59:03	0|jonasschnelli|gmaxwell: Catching up data in the background puts your app in a different app-group
124 2017-04-20 09:59:06	0|jonasschnelli|BF = bloom filter
125 2017-04-20 09:59:28	0|jonasschnelli|gmaxwell: the group that has the significant warning about battery consumption. :)
126 2017-04-20 09:59:51	0|sipa|what is an app group?
127 2017-04-20 10:00:09	0|gmaxwell|jonasschnelli: http://luke.dashjr.org/programs/bitcoin/files/charts/software.html  BF using wallets are relatively rare I seldom see more than one connected.
128 2017-04-20 10:00:52	0|jonasschnelli|sipa: It may be different on Android. But on apples iOS, if you use backgroup activity, you'll end up in a different app-group resulting to different reviews,... I think you need to add warnings to your app description, etc.
129 2017-04-20 10:01:05	0|gmaxwell|sipa: I fail to see how cpu time alone is interesting for what is being discussed. :)
130 2017-04-20 10:01:13	0|sipa|gmaxwell: that chart is just reachable nodes
131 2017-04-20 10:01:15	0|sipa|no?
132 2017-04-20 10:01:17	0|gmaxwell|sipa: no.
133 2017-04-20 10:01:25	0|gmaxwell|sipa: the opposite of that, in fact.
134 2017-04-20 10:01:36	0|sipa|gmaxwell: i was discussing the data structure, not the use case
135 2017-04-20 10:02:01	0|gmaxwell|ah well, you wouldn't use it for any of the things you normally use a bloomfilter for.
136 2017-04-20 10:02:50	0|gmaxwell|Though FWIW, the cuckoo-like filter also is close to capacity achieving (at least if you high enough N to allow a high fill rate) and works incrementally.
137 2017-04-20 11:08:04	0|jonasschnelli|Hmm... is it entirely stupid to only extend and check the kepool-keyrange (HD restore) if the wallet's bestblock lacks some blocks behind the chaintip? I think a check during init could give an indication if the wallet is in restore mode or not.
138 2017-04-20 11:08:28	0|jonasschnelli|Alternatively we could add an option hdalwayscheckkeypool=1
139 2017-04-20 11:11:48	0|wumpus|what would be the rationale behind doing that, then only?
140 2017-04-20 11:21:38	0|jonasschnelli|wumpus: a) performance for unencrypted wallets, b) [more important] encrypted wallets would require a unlock during the time of the scan
141 2017-04-20 11:22:32	0|wumpus|"encrypted wallets would require a unlock during the time of the scan" but only if the scan notices that new keys are needed, right?
142 2017-04-20 11:22:35	0|jonasschnelli|for bitcoind, there is the problem how to want the user if the gap limit is reached, because, at this point, he would need to unlock the wallet in order to continue scanning
143 2017-04-20 11:22:44	0|jonasschnelli|wumpus: Right.
144 2017-04-20 11:23:10	0|wumpus|yes I understand there's a notificatino problem there. The GUI could just pop up a dialog, not so much for bitcoind
145 2017-04-20 11:23:17	0|jonasschnelli|And... to do a precaution scan, you probably should use a gap limit of 100 (configurable).
146 2017-04-20 11:24:02	0|jonasschnelli|Yes. The GUI way is much simpler.. but even there. Do we always want to extend up to a default gap limit (even in normal operations)?
147 2017-04-20 11:24:05	0|wumpus|so yes it may make sense to have a seprate 'wallet is reconstructing' mode
148 2017-04-20 11:24:29	0|jonasschnelli|Because someone may have handed out 100+ addresses and want to make sure he catches all of them in a HD rescan
149 2017-04-20 11:24:44	0|jonasschnelli|Though, most people probably dont want to auto-extend their keypool over 100+
150 2017-04-20 11:24:44	0|wumpus|this flag could also be stored in the wallet (like the reindex flag in the utxo db) instead of determining this based on the wallet's bestblock
151 2017-04-20 11:25:32	0|jonasschnelli|but if you load an initial HD wallet backup, you probably can only identify the possible backup by comparing the bestblock against the chaintip
152 2017-04-20 11:25:46	0|wumpus|one argument against treating reconstruction specially would be simultaneous use of the wallet on different machines. I know, we don't support this, but then it could be detected.
153 2017-04-20 11:26:02	0|wumpus|jonasschnelli: that's true
154 2017-04-20 11:26:03	0|jonasschnelli|Yes. That.
155 2017-04-20 11:26:25	0|jonasschnelli|Asking the user make sense... (GUI).
156 2017-04-20 11:26:37	0|wumpus|yes
157 2017-04-20 11:26:54	0|jonasschnelli|"Wallets is out of sync, do you want to restore a backup?"
158 2017-04-20 11:27:05	0|jonasschnelli|Then extend the keypool +1000 or ask about the previous usage
159 2017-04-20 11:27:32	0|wumpus|#10231 gives me a compile error : bitcoin/src/qt/clientmodel.h:85:30: error: implicit instantiation of undefined template
160 2017-04-20 11:27:32	0|wumpus|'std::atomic<int>'
161 2017-04-20 11:27:34	0|gribble|https://github.com/bitcoin/bitcoin/issues/10231 | [Qt] Reduce a significant cs_main lock freeze by jonasschnelli · Pull Request #10231 · bitcoin/bitcoin · GitHub
162 2017-04-20 11:27:45	0|wumpus|needs a header probably
163 2017-04-20 11:27:55	0|jonasschnelli|Oh... missed <atomic> include...
164 2017-04-20 11:28:10	0|jonasschnelli|Thanks. Let me fix this
165 2017-04-20 11:29:52	0|jonasschnelli|Added a commit. But why did travis not complain?!
166 2017-04-20 11:32:41	0|wumpus|my report is on ubuntu 16.04 at least, maybe it's different for 14.04
167 2017-04-20 11:33:04	0|wumpus|most likely cause: different boost versions
168 2017-04-20 11:33:22	0|wumpus|or qt
169 2017-04-20 11:38:40	0|wumpus|anyhow with your change it all compiles
170 2017-04-20 11:41:43	0|bitcoin-git|[13bitcoin] 15laanwj pushed 6 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/a987def4f629...987a6c09562e
171 2017-04-20 11:41:44	0|bitcoin-git|13bitcoin/06master 147148f5e 15Jonas Schnelli: Reduce cs_main locks during modal overlay by adding an atomic cache
172 2017-04-20 11:41:44	0|bitcoin-git|13bitcoin/06master 14cf92bce 15Jonas Schnelli: Update the remaining blocks left in modaloverlay at init.
173 2017-04-20 11:41:45	0|bitcoin-git|13bitcoin/06master 14610a917 15Jonas Schnelli: Declare headers height/time cache mutable, re-set the methods const
174 2017-04-20 11:42:08	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #10231: [Qt] Reduce a significant cs_main lock freeze (06master...062017/04/qt_freeze) 02https://github.com/bitcoin/bitcoin/pull/10231
175 2017-04-20 13:36:26	0|morcos|gmaxwell: fee estimation currently does not use mempool queue (nor in the improvements for 0.15)  it's an idea that i've been contemplating since the beginning, but i never settled on a design that i thought met all the criteria
176 2017-04-20 13:36:56	0|morcos|balancing performance, usefulness, and security is hard.
177 2017-04-20 13:41:25	0|bitcoin-git|[13bitcoin] 15jonasschnelli opened pull request #10238: Change setKeyPool to hold flexible entries (06master...062017/04/keypool_fix_a) 02https://github.com/bitcoin/bitcoin/pull/10238
178 2017-04-20 13:47:10	0|bitcoin-git|[13bitcoin] 15sipa opened pull request #10239: Make Boost use std::atomic internally (06master...06boost_std_atomic) 02https://github.com/bitcoin/bitcoin/pull/10239
179 2017-04-20 13:48:50	0|BlueMatt|sipa: hmm...why does boost prefer its own impls? just for compat?
180 2017-04-20 13:49:18	0|sipa|BlueMatt: only bug report i found about it was "gcc's atomics were not so good in the past... it's probably better now... discussion dies"
181 2017-04-20 13:49:46	0|BlueMatt|lol, yay boost
182 2017-04-20 13:49:59	0|sipa|these macros were only added in 1.54 though... i don't know what versions of boost we're using everywhere
183 2017-04-20 13:51:07	0|sipa|http://boost.2283326.n4.nabble.com/compiling-boost-using-C-11-atomics-td4687878.html
184 2017-04-20 13:51:52	0|BlueMatt|well i suppose thats good...if you have a new boost you'll probably have a new gcc which has good atomics as well :)
185 2017-04-20 13:52:55	0|sipa|well we already rely on std::atomics anywhere
186 2017-04-20 13:53:53	0|BlueMatt|indeed
187 2017-04-20 15:19:01	0|luke-jr|wumpus: FYI I get a different result of update-translations than rc2 has: specifically, bitcoin_af is not created
188 2017-04-20 15:20:14	0|bitcoin-git|[13bitcoin] 15jnewbery reopened pull request #10198: [tests] Remove is_network_split from functional test framework (06master...06remove_is_network_split) 02https://github.com/bitcoin/bitcoin/pull/10198
189 2017-04-20 15:30:53	0|bitcoin-git|[13bitcoin] 15jonasschnelli opened pull request #10240: [WIP] Add basic HD wallet restore functionality (06master...062017/04/hd_rescan) 02https://github.com/bitcoin/bitcoin/pull/10240
190 2017-04-20 17:25:43	0|wumpus|luke-jr: did I add bitcoin_af for rc2?
191 2017-04-20 17:26:12	0|wumpus|adding languages is something I do manually, when I think there's enough in a .ts file to warrant it
192 2017-04-20 17:26:35	0|luke-jr|wumpus: I don't know about added, but it's not created by the translation scripts anymore
193 2017-04-20 17:26:40	0|wumpus|but I think _af was a while ago
194 2017-04-20 17:26:50	0|wumpus|interesting, maybe it was deleted on transifex
195 2017-04-20 17:26:57	0|luke-jr|when updating translations, I delete *.ts (except en) first
196 2017-04-20 17:27:00	0|wumpus|I don't track lanugage deletions, only adding
197 2017-04-20 17:27:32	0|wumpus|unless someone notified me that it was removed for a good reason (e.g. the fake austrian(?) translation that was there at some point)
198 2017-04-20 17:28:04	0|wumpus|but I'll check that for next language update, thanks
199 2017-04-20 17:29:29	0|luke-jr|np
200 2017-04-20 18:45:46	0|jonasschnelli|Why do test fail when they are successful:  :/ https://travis-ci.org/bitcoin/bitcoin/jobs/224004681#L5249
201 2017-04-20 18:46:13	0|jonasschnelli|A stderr warning leads always to a test failure
202 2017-04-20 18:48:25	0|wumpus|stderr output in itself results in test failure?
203 2017-04-20 18:48:32	0|wumpus|I don't think so, it's based o nthe return code
204 2017-04-20 18:48:34	0|jonasschnelli|wumpus: Yes. It looks like.
205 2017-04-20 18:48:42	0|jonasschnelli|They pass locally...
206 2017-04-20 18:48:56	0|jonasschnelli|and they pass on travis... but test runner markes them as failed
207 2017-04-20 18:49:02	0|jonasschnelli|wumpus: https://travis-ci.org/bitcoin/bitcoin/jobs/224004681#L5249
208 2017-04-20 18:49:47	0|jonasschnelli|IMO at least we should flag them as passed if stderr contains only warnings...
209 2017-04-20 18:49:53	0|jonasschnelli|otherwise we can't test warnings
210 2017-04-20 18:50:16	0|wumpus|test fail/pass shouldn't be based on stderr output at all
211 2017-04-20 18:50:32	0|luke-jr|jonasschnelli: if an early step fails, travis keeps running the rest and still marks it as failed
212 2017-04-20 18:50:32	0|wumpus|if it is, that's kind of weird
213 2017-04-20 18:51:19	0|jonasschnelli|luke-jr: I think its not that: check the signmessage test: https://travis-ci.org/bitcoin/bitcoin/jobs/224004681#L5243
214 2017-04-20 18:51:30	0|wumpus|where does it check stderr output?
215 2017-04-20 18:51:41	0|luke-jr|looks like a test_runner.py thing
216 2017-04-20 18:51:42	0|jonasschnelli|(INFO): Tests successful... but signmessages.py failed, Duration: 3 s
217 2017-04-20 18:52:55	0|jonasschnelli|if proc.returncode == TEST_EXIT_PASSED and stderr == "":
218 2017-04-20 18:53:03	0|jonasschnelli|the later if statement
219 2017-04-20 18:53:30	0|jonasschnelli|or at least split by newline and pass if all lines start with /Warning/
220 2017-04-20 18:53:43	0|jonasschnelli|(or a clever regex)
221 2017-04-20 18:54:07	0|wumpus|stderr == "" should go
222 2017-04-20 18:54:20	0|wumpus|return code should be what determines whether a test passed
223 2017-04-20 18:54:31	0|wumpus|anything else is insane
224 2017-04-20 18:54:44	0|jonasschnelli|I think so. Tests may by successful is there is something in stderr
225 2017-04-20 18:55:07	0|jonasschnelli|Okay. I'll PR
226 2017-04-20 18:58:07	0|bitcoin-git|[13bitcoin] 15jonasschnelli opened pull request #10241: Allow tests to pass even when stderr got populated (06master...062017/04/test_stderr) 02https://github.com/bitcoin/bitcoin/pull/10241
227 2017-04-20 18:59:08	0|gmaxwell|I think the sanitizer stuff is only useful in our current test harnesses because we fail on stderr output.
228 2017-04-20 18:59:21	0|luke-jr|ah
229 2017-04-20 19:00:03	0|wumpus|sanitizer stuff?
230 2017-04-20 19:00:23	0|gmaxwell|TSAN/ASAN/UBSAN.
231 2017-04-20 19:01:00	0|wumpus|do we use that in travis?
232 2017-04-20 19:01:41	0|jonasschnelli|Well, we could add a test_runner argument (fail_on_stderr) if someone wants to use that with sanitizer
233 2017-04-20 19:02:04	0|sdaftuar|meeting time?
234 2017-04-20 19:02:06	0|wumpus|but ok, at least I understand why the stderr check is there now, it's for private test runs with sanitizer?
235 2017-04-20 19:02:07	0|jonasschnelli|however.. meeting
236 2017-04-20 19:02:12	0|wumpus|#startmeeting
237 2017-04-20 19:02:13	0|lightningbot|Meeting started Thu Apr 20 19:02:12 2017 UTC.  The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
238 2017-04-20 19:02:13	0|lightningbot|Useful Commands: #action #agreed #help #info #idea #link #topic.
239 2017-04-20 19:02:14	0|luke-jr|IIRC one of the sanitisers used to require a special env var to cause an exit
240 2017-04-20 19:02:20	0|gmaxwell|Not yet, only with some not yet merged PRs are we finally TSAN clean, but many of us run it locally and it has found real bugs.   I'm not protesting, but just bringing up the one thing I remember that interacts with that assumption.
241 2017-04-20 19:02:23	0|luke-jr|but I can't find that now (some do need an extra build option tho)
242 2017-04-20 19:02:34	0|gmaxwell|#bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr btcdrak sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier
243 2017-04-20 19:02:40	0|instagibbs|present
244 2017-04-20 19:02:44	0|wumpus|gmaxwell: yes I think it's a good point, not trying to disparage it, but we should document things like this
245 2017-04-20 19:02:45	0|kanzure|hi.
246 2017-04-20 19:02:50	0|cfields|hi
247 2017-04-20 19:02:56	0|jtimon|hola
248 2017-04-20 19:03:06	0|wumpus|topics?
249 2017-04-20 19:03:08	0|gmaxwell|wumpus: my thinking on seeing the comments above was "oh oh ... that interacts with something ... what was it? what was it?"
250 2017-04-20 19:03:21	0|gmaxwell|wumpus: 0.14.x release?
251 2017-04-20 19:03:29	0|wumpus|#topic 0.14.1 release
252 2017-04-20 19:03:34	0|wumpus|let's push the button?
253 2017-04-20 19:03:38	0|luke-jr|k
254 2017-04-20 19:04:11	0|jonasschnelli|Okay for me... to bad #10231 missed 0.14.1
255 2017-04-20 19:04:12	0|gribble|https://github.com/bitcoin/bitcoin/issues/10231 | [Qt] Reduce a significant cs_main lock freeze by jonasschnelli · Pull Request #10231 · bitcoin/bitcoin · GitHub
256 2017-04-20 19:04:23	0|gmaxwell|https://www.youtube.com/watch?v=rLMCjuge6oE "Turn your key SIR"
257 2017-04-20 19:04:32	0|cfields|hooray!
258 2017-04-20 19:04:34	0|luke-jr|jonasschnelli: I can put it in Knots 0.14.1
259 2017-04-20 19:04:53	0|jonasschnelli|luke-jr: Yes. Do that.
260 2017-04-20 19:04:53	0|wumpus|jonasschnelli: is that even tagged for backport? anyhow, tag it for 0.14.2 I'd say
261 2017-04-20 19:05:13	0|jonasschnelli|wumpus: Yeah. I tagged (not the project though).. 0.14.2 is good IMO.
262 2017-04-20 19:05:42	0|wumpus|jonasschnelli: ok!
263 2017-04-20 19:06:14	0|wumpus|next topic?
264 2017-04-20 19:07:23	0|wumpus|jonasschnelli: if it's any consolation, many projects had a similar issue with a central lock
265 2017-04-20 19:07:52	0|wumpus|I was thinkinkg about the Big Kernel Lock, but yes, python is guilty too
266 2017-04-20 19:07:55	0|jonasschnelli|wumpus: Yes. I guess there is much room for optimisation.
267 2017-04-20 19:08:36	0|gmaxwell|There has been some interesting discussion in github related to the wallets handling of address reuse and dust and what not. anyone interested in that subject might want to check out the discussion on #10233  and PRs linked from there.
268 2017-04-20 19:08:38	0|gribble|https://github.com/bitcoin/bitcoin/issues/10233 | Wallet: Support not reusing addresses by jet0 · Pull Request #10233 · bitcoin/bitcoin · GitHub
269 2017-04-20 19:08:49	0|wumpus|#topic blocker PRs for review
270 2017-04-20 19:09:20	0|gmaxwell|jonasschnelli: on locking we need some better lock profiling. If we have some instrumention that yelled anytime lock contention caused >100ms delays, we'd probably find a number of things to fix.
271 2017-04-20 19:09:35	0|jonasschnelli|gmaxwell: Yes. That!
272 2017-04-20 19:09:47	0|gmaxwell|I don't think cs_main is itself really the issue there... just not carefully avoiding it via things like caches.
273 2017-04-20 19:09:56	0|jonasschnelli|gmaxwell: I was printf profiling yesterday
274 2017-04-20 19:09:59	0|luke-jr|I looked into disabling address reuse and it looks harder than I'd like :/
275 2017-04-20 19:10:10	0|morcos|i would like to briefly discuss fee estimation (maybe as separate topic)
276 2017-04-20 19:10:29	0|gmaxwell|on the blocker PRs-- I'm kinda lost where we are with non-atomic writes.
277 2017-04-20 19:10:30	0|jonasschnelli|blocker PR: https://github.com/bitcoin/bitcoin/projects/8
278 2017-04-20 19:10:37	0|gribble|https://github.com/bitcoin/bitcoin/issues/10179 | Give CValidationInterface Support for calling notifications on the CScheduler Thread by TheBlueMatt · Pull Request #10179 · bitcoin/bitcoin · GitHub
279 2017-04-20 19:10:50	0|BlueMatt|gm2053: i think its ready for review now?
280 2017-04-20 19:10:54	0|morcos|gmaxwell: #10148 in its current form without multi head just needs more review i think
281 2017-04-20 19:10:56	0|gribble|https://github.com/bitcoin/bitcoin/issues/10148 | [WIP] Use non-atomic flushing with block replay by sipa · Pull Request #10148 · bitcoin/bitcoin · GitHub
282 2017-04-20 19:11:26	0|morcos|and maybe more tests?
283 2017-04-20 19:11:35	0|jonasschnelli|sipa still has the chacha20 rnd as blocker #9792 ...
284 2017-04-20 19:11:38	0|gribble|https://github.com/bitcoin/bitcoin/issues/9792 | FastRandomContext improvements and switch to ChaCha20 by sipa · Pull Request #9792 · bitcoin/bitcoin · GitHub
285 2017-04-20 19:11:56	0|wumpus|#10179 is already on the list BlueMatt
286 2017-04-20 19:11:57	0|gribble|https://github.com/bitcoin/bitcoin/issues/10179 | Give CValidationInterface Support for calling notifications on the CScheduler Thread by TheBlueMatt · Pull Request #10179 · bitcoin/bitcoin · GitHub
287 2017-04-20 19:12:30	0|BlueMatt|wumpus: oh, wasnt sure if it got switched after the last merge, sorry
288 2017-04-20 19:12:32	0|wumpus|adding 10148
289 2017-04-20 19:12:33	0|gmaxwell|9792 isn't hard to review, FWIW, in my expirence.
290 2017-04-20 19:12:52	0|sdaftuar|gmaxwell: i've become more comfortable conceptually with the non-atomic writes (it did take me a while to come around to it being worth the effort).  i'd like to review and test more.
291 2017-04-20 19:13:01	0|morcos|ditto
292 2017-04-20 19:13:25	0|wumpus|9792 is also already on the list
293 2017-04-20 19:13:43	0|BlueMatt|#9942 can get merged, I think...
294 2017-04-20 19:13:45	0|gribble|https://github.com/bitcoin/bitcoin/issues/9942 | Refactor CBlockPolicyEstimator by morcos · Pull Request #9942 · bitcoin/bitcoin · GitHub
295 2017-04-20 19:13:55	0|jtimon|#8855 isn't hard to review either...
296 2017-04-20 19:13:57	0|gribble|https://github.com/bitcoin/bitcoin/issues/8855 | Use a proper factory for creating chainparams by jtimon · Pull Request #8855 · bitcoin/bitcoin · GitHub
297 2017-04-20 19:14:04	0|wumpus|if there's something that an be merged you should tell me, preferably outside the meeting :)
298 2017-04-20 19:14:14	0|morcos|yeah wumpus i think that is now just wasting review cycles, it has more than enouch ACK's (we have told you a couple times.. :) )
299 2017-04-20 19:14:15	0|luke-jr|multiwallet is rebased and nits fixed btw
300 2017-04-20 19:14:26	0|gmaxwell|sdaftuar: will let us effectively double the dbcache size being a first benefit... plus it should allow some really nince improvements later post per-txo. Sorry if it wasn't communicated well, the value was more obvious to pieter and I perhaps because we've been hammering on caching policy changes based on per-txo for a while.
301 2017-04-20 19:14:34	0|luke-jr|CWalletDB still needs some serious refactoring, but IMO that's something to do outside multiwallet's PR
302 2017-04-20 19:15:06	0|jonasschnelli|luke-jr: agree
303 2017-04-20 19:15:18	0|wumpus|morcos: I don't remember
304 2017-04-20 19:15:50	0|morcos|wumpus: no problem, i just wasnt telling you because i didn't want to tell you too many times..  in any case i think its ready (9942 that is)
305 2017-04-20 19:16:04	0|bitcoin-git|[13bitcoin] 15luke-jr closed pull request #7289: [WIP] Make arguments reconfigurable at runtime via RPC (06master...06rpc_setarg) 02https://github.com/bitcoin/bitcoin/pull/7289
306 2017-04-20 19:16:31	0|sdaftuar|gmaxwell: yeah, makes sense to me now -- there are a lot of steps of "why don't we do X simpler thing instead" that i know you guys have tried/thought through already, that i needed to think through myself
307 2017-04-20 19:17:49	0|morcos|fee estimation?
308 2017-04-20 19:17:52	0|jonasschnelli|ack
309 2017-04-20 19:18:04	0|bitcoin-git|[13bitcoin] 15laanwj pushed 8 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/987a6c09562e...14c948987f0b
310 2017-04-20 19:18:05	0|bitcoin-git|13bitcoin/06master 14ae7327b 15Alex Morcos: Make feeEstimator its own global instance of CBlockPolicyEstimator
311 2017-04-20 19:18:05	0|bitcoin-git|13bitcoin/06master 14f6187d6 15Alex Morcos: Make processBlockTx private.
312 2017-04-20 19:18:06	0|bitcoin-git|13bitcoin/06master 14dbb9e36 15Alex Morcos: Give CBlockPolicyEstimator it's own lock
313 2017-04-20 19:18:15	0|morcos|Thanks!
314 2017-04-20 19:18:23	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #9942: Refactor CBlockPolicyEstimator (06master...06moveTxConfirmStats) 02https://github.com/bitcoin/bitcoin/pull/9942
315 2017-04-20 19:18:40	0|BlueMatt|morcos: yes, fee estimation
316 2017-04-20 19:18:47	0|morcos|I wrote this to describe the existing algorithm https://gist.github.com/morcos/d3637f015bc4e607e1fd10d8351e9f41 , which I'm happy to discuss if anyone has any questions on it.
317 2017-04-20 19:18:55	0|gmaxwell|morcos: thanks for that fee estimation writeup, I guess I understood it better than I thought I did, I think I thought more of the discussed things were actually implemented.
318 2017-04-20 19:18:59	0|bitcoin-git|[13bitcoin] 15ryanofsky opened pull request #10242: [qt] Don't call method on null WalletModel object (06master...06pr/rbfnull) 02https://github.com/bitcoin/bitcoin/pull/10242
319 2017-04-20 19:19:33	0|gmaxwell|morcos: I think that writeup is good and should go into the codebase.
320 2017-04-20 19:19:36	0|morcos|And then I wrote #10199 with a bunch of improvements.  I suppose it makes sense to add another section the gist that provides a high level overview of the improvements?
321 2017-04-20 19:19:38	0|gribble|https://github.com/bitcoin/bitcoin/issues/10199 | Better fee estimates by morcos · Pull Request #10199 · bitcoin/bitcoin · GitHub
322 2017-04-20 19:19:46	0|gmaxwell|morcos: that would be good.
323 2017-04-20 19:20:32	0|gmaxwell|I think the estimatior is a complex enough machine that we should maintain a seperate description of it, if not an actual spec.   Just like we do for many major protocol features.
324 2017-04-20 19:20:49	0|morcos|But what I would like to do is err on the side of merging 10199 early and then if there are small bugs or fixes, we can fix them in master
325 2017-04-20 19:20:54	0|sipa|oops, forgot about meeting
326 2017-04-20 19:21:01	0|gmaxwell|morcos: the writeup could use some more details about the reliablity estimates and how it merges bins.
327 2017-04-20 19:21:04	0|morcos|it takes 2 weeks of continuous up time to even explore all the code paths
328 2017-04-20 19:21:11	0|gmaxwell|sipa: the meeting did not forget about you.
329 2017-04-20 19:21:34	0|morcos|jtimon: yes, i have a plan to do that that builds off BlueMatt's CValidationInterface.  The groundwork is laid in 9942 that was just merged
330 2017-04-20 19:21:51	0|gmaxwell|morcos: are we not saving enough data between restarts that we really do need two weeks continious uptime to hit it all?
331 2017-04-20 19:21:58	0|morcos|reliability estimates?  reliability OF estimates?
332 2017-04-20 19:22:14	0|gmaxwell|morcos: I know that if there aren't many samples in a bin it doesn't use the bin.
333 2017-04-20 19:22:38	0|morcos|gmaxwell: well if you want to know how much fee it'll take to be confirmed in a week, you sure as hell better wait at least a week  (but yes once you've done that once, you may not need to do it again on a restart)
334 2017-04-20 19:23:02	0|morcos|gmaxwell: some of that stuff is changed in 10199 (for the better, obviously i guess)
335 2017-04-20 19:23:10	0|luke-jr|if anyone else acks #10242, maybe mention the meeting going on in a P.S. :p
336 2017-04-20 19:23:12	0|gribble|https://github.com/bitcoin/bitcoin/issues/10242 | An error has occurred and has been logged. Please contact this bot's administrator for more information.
337 2017-04-20 19:23:12	0|jtimon|morcos: I know, I reveiwed it yesterday and linked to similar PRs of my own, at the time you only wanted to decouple the mempool from the estimator (9942 just did it), but not the estimator from the mempool, happy if you changed your mind and want both like me now
338 2017-04-20 19:23:37	0|gmaxwell|I guess in general a thing to keep in mind for this sort of description is that we should try to make it detailed enough that if an academic showed up and wrote a paper on it based on only the description (which they will) would their results be likely useful to us or not. :)
339 2017-04-20 19:23:57	0|morcos|but there are some open questions in 10199 that would be helpful to get feedback on
340 2017-04-20 19:24:06	0|morcos|such as starting with not being able to upgrade from the old estimates
341 2017-04-20 19:24:15	0|wumpus|imo the most important about the description is that we understand it
342 2017-04-20 19:24:21	0|gmaxwell|morcos: we should think about saving more of its state in the future.  I have nodes that don't spend more than a few minutes down per month, but don't often make it to two weeks up.
343 2017-04-20 19:25:12	0|luke-jr|morcos: how complicated is the upgrading? we only would need it for one version at most IMO
344 2017-04-20 19:25:15	0|morcos|gmaxwell: i think thats problematic for being able to predict really long time horizons...
345 2017-04-20 19:25:44	0|wumpus|upgrading isn't much of an issue, if the estimation algorithm changes, feel free to throw away the data from the previous one
346 2017-04-20 19:25:55	0|wumpus|just make sure it doesn't crash on upgrading/downgrading
347 2017-04-20 19:26:23	0|gmaxwell|well, to really advance I think what we would probably want is a simulator (perhaps based on historical data) and a metric of success.
348 2017-04-20 19:26:38	0|wumpus|yes
349 2017-04-20 19:26:42	0|morcos|luke-jr: the complicated part is deciding what we want to do, implementing it probably isn't that bad...  but for instance the new estimates are smart about whether your estimates file is stale...  but should it just dumbly use your old estimates until it has new estimates...  what if the new estimates for 5 blocks which you do have is lower than the old estimate for 25 (which you dont' have a new estimate for)
350 2017-04-20 19:26:48	0|morcos|etc.
351 2017-04-20 19:26:49	0|gmaxwell|I think it's more or less fine to toss out data on algo changes. we could worry about doing better when the algo is stable for a long time.
352 2017-04-20 19:27:13	0|wumpus|the difficult part, as with coin selection, is evaluationg algorithms
353 2017-04-20 19:27:31	0|morcos|the historical data is useless... the question is whether you use the old estimates until your new estimates are warmed up (by calculating them before you throw away the data)
354 2017-04-20 19:27:58	0|luke-jr|huh, someone's playing malleability games on testnet.
355 2017-04-20 19:28:10	0|gmaxwell|Electrum does some things with using static estimates when it doesn't have data to estimate on its own.  I think there are a lot of interesting tradeoffs we could make to hotstart. But I don't think starting speed is at all our biggest concern.
356 2017-04-20 19:28:17	0|gmaxwell|luke-jr: they have been for months.
357 2017-04-20 19:28:47	0|gmaxwell|The purely retrospective algorithim is really slow to update to changing network conditions, in particular, it doesn't track the weekly load cycle well.
358 2017-04-20 19:28:49	0|morcos|gmaxwell: right so that is the question..  my preference would be to merge as is.. and then if we get around to it before 0.15 we add a smarter hotstart
359 2017-04-20 19:29:06	0|luke-jr|morcos: well, if it's  not already implemented, I'd say it's not important enough to spend time implementing
360 2017-04-20 19:29:43	0|luke-jr|(upgrading, that is)
361 2017-04-20 19:29:56	0|morcos|gmaxwell: yes... one of the main problems the new design was meant to address...  still using only a purely retrospective algorithm, so the problem fundamentally remains, but in practice its much more responsive  (b/c it looks at different time horizons simultaneously)
362 2017-04-20 19:31:00	0|jcorgan|clearly this calls for Deep Fee Estimation
363 2017-04-20 19:31:08	0|gmaxwell|die
364 2017-04-20 19:31:36	0|jcorgan|tell me what you *really* think
365 2017-04-20 19:31:40	0|wumpus|hehe, deep fee estimation
366 2017-04-20 19:31:56	0|luke-jr|no no, Xtreme Deep Fee Estimation!
367 2017-04-20 19:32:07	0|morcos|anyway, ok for now i'll update the gist with a high level description of the algorithm
368 2017-04-20 19:32:11	0|gmaxwell|I have a lovely algorithim for an efficient limited memory 2D exponentially weighed moving average somewhere....
369 2017-04-20 19:32:18	0|gmaxwell|morcos: great.
370 2017-04-20 19:32:32	0|sipa|Xthin fees
371 2017-04-20 19:32:36	0|luke-jr|XD
372 2017-04-20 19:32:46	0|morcos|but my basic point here is that ideally we need weeks/months of testing in master to uncover possible edge cases
373 2017-04-20 19:33:15	0|morcos|i'm relatively confident that overall this is better, but thats not the same thing as saying it doesn' thave problems that need fixing...
374 2017-04-20 19:33:16	0|wumpus|yes, it shouldn't be merged last minute
375 2017-04-20 19:33:34	0|gmaxwell|morcos: well get your description up soon, and I'll review shortly after.  I think fee estimation is self contained enough we could merge something and back it out if we don't like it... but we need to have more than you understanding what we're doing at least. :)
376 2017-04-20 19:33:59	0|morcos|gmaxwell: yes basically my point..  ok sounds good
377 2017-04-20 19:34:08	0|gmaxwell|(if for no other reason than we need to understand it better to spot failures with it.)
378 2017-04-20 19:34:12	0|jonasschnelli|morcos: maybe it was asked already, how fast are the estimations available after startup? Does it work with prune=550?
379 2017-04-20 19:34:26	0|morcos|prune is irrelevant
380 2017-04-20 19:34:50	0|morcos|it can give you an estimate for a target of N once it has been caught up for 2*N blocks...
381 2017-04-20 19:35:01	0|gmaxwell|jonasschnelli: Estimations for depth X need to at least see some multiple of X blocks.
382 2017-04-20 19:35:22	0|morcos|but then it saves that
383 2017-04-20 19:35:24	0|gmaxwell|Becuase you have a moving window of analysis, and no data for longer windows.
384 2017-04-20 19:35:54	0|jonasschnelli|So for a conf target of 2 you need to wait ~40min after startup?
385 2017-04-20 19:35:57	0|morcos|so if you stop and restart you're starting over again for increasing your max possible target, but you still have access for up to that max possible target
386 2017-04-20 19:36:34	0|morcos|jonasschnelli: correct, but again, only the first time  (or if you have been down for more than 6 weeks i think)
387 2017-04-20 19:36:37	0|gmaxwell|jonasschnelli: well not really, because you save the results. so the first time, yes. But that goes back to the hot start question.. and there are lots of ways we could hot start these things, if we really had something that was working well otherwise.
388 2017-04-20 19:36:50	0|jtimon|jcorgan: after alphago took go away from me I was looking for other problem to solve with https://github.com/jtimon/preann as an excuse to run it again </offtopic spam>
389 2017-04-20 19:37:42	0|jonasschnelli|Okay. Thanks... I'll test 10199 with the mainnet GUI then a bit (before of after merging)
390 2017-04-20 19:37:51	0|bitcoin-git|[13bitcoin] 15ryanofsky opened pull request #10244: [qt] Add abstraction layer for accessing node and wallet functionality from gui (06master...06pr/ipc-local) 02https://github.com/bitcoin/bitcoin/pull/10244
391 2017-04-20 19:37:53	0|jonasschnelli|*or
392 2017-04-20 19:38:44	0|morcos|gmaxwell: in the meantime the PR description in 10199 covers the changes pretty close to what i will write up on the gist
393 2017-04-20 19:40:09	0|morcos|i guess i need to do a quick rebase now that 9942 is done
394 2017-04-20 19:44:32	0|wumpus|any other topics?
395 2017-04-20 19:44:54	0|gmaxwell|I want to talk to luke some about the address reuse thing, but it can be post meeting.
396 2017-04-20 19:45:06	0|wumpus|time to tag 0.14.1 final and start gitian building
397 2017-04-20 19:45:54	0|wumpus|gmaxwell: well there's time
398 2017-04-20 19:45:58	0|wumpus|#topic address reuse thing
399 2017-04-20 19:46:48	0|gmaxwell|so a serious privacy problem which has been actively exploited for a long time is that people make near-dust payments to addresses once they've been spent from, so that you spend from them again in new txn, creating snowballing that links all your transactions togeather.
400 2017-04-20 19:46:54	0|wumpus|* [new tag]         v0.14.1 -> v0.14.1
401 2017-04-20 19:47:32	0|instagibbs|https://www.reddit.com/r/BitcoinBeginners/comments/66az3b/why_do_i_keep_getting_000000001_deposits/ for example
402 2017-04-20 19:47:35	0|gmaxwell|Latest discussions seem to be driven by a user that runs a gambling site and whom cares about this because his customers get running into issues transactions that link back to him.
403 2017-04-20 19:47:42	0|wumpus|do we need a 'block transacton' functionality against such transaction abuse?
404 2017-04-20 19:47:46	0|gmaxwell|but it's a general concern for everyone.
405 2017-04-20 19:48:16	0|gmaxwell|So there have been discussion about some very heavy handed manual methods, but I think I have a suggestion that could potentially be a default behavior:
406 2017-04-20 19:48:28	0|gmaxwell|but I'm interested in hearing if other people think I'm crazy.
407 2017-04-20 19:48:32	0|wumpus|wouldn't be the first time I have an UTXO I just want to ignore
408 2017-04-20 19:48:37	0|morcos|stop the suspense!
409 2017-04-20 19:48:52	0|BlueMatt|morcos: ack
410 2017-04-20 19:48:57	0|gmaxwell|First create a seperate quarenteened balance. Any address or specfic txo could be manually quarenteened or unquarenteed at any itme.
411 2017-04-20 19:49:30	0|gmaxwell|Then adjust coin selection to always spend all payments to a particular address at once (+/- some filtering with dust that might be needed to prevent dust attacks).
412 2017-04-20 19:49:56	0|gmaxwell|Then once an address has been spent from, it's automatically added to tue quarenteen list (with any outputs that weren't spent, e.g. whatever failed the dust filtering).
413 2017-04-20 19:50:21	0|wumpus|I think the quarantaine is a good idea, not so sure about adding things automatically though
414 2017-04-20 19:50:26	0|gmaxwell|If users want to intentionally reuse an address, I suppose they'd need a way to prevent them from being reblocked.
415 2017-04-20 19:50:35	0|morcos|i like the general idea
416 2017-04-20 19:50:42	0|luke-jr|gmaxwell: and quarantined funds are excluded from balance or tx list somehow?
417 2017-04-20 19:51:05	0|BlueMatt|morcos: I might prefer if we were Quarantine ing things and not quarantaine or quarenteened :p
418 2017-04-20 19:51:05	0|gmaxwell|Well I think the attacks will continue unless we could come up with something that was close to automatic... Could be something that gives a GUI user a choice the first time it happens or if the Q balance becomes non-negligble.
419 2017-04-20 19:51:07	0|morcos|but what if you have 10 10btc utxos at the same address and you need to pay someone 1 btc
420 2017-04-20 19:51:10	0|morcos|you spend them all?
421 2017-04-20 19:51:28	0|gmaxwell|luke-jr: they'd be shown in the tx list, but skipped for spending, and shown as a seperate balances. Like confirmed vs unconfirmed balance.
422 2017-04-20 19:51:59	0|BlueMatt|gmaxwell: I'm somewhat unsold as default policy
423 2017-04-20 19:52:03	0|gmaxwell|morcos: yep. and create a big change. Which I think is fine.  I think a seperate issue is that we should auto-split very large change. But thats 90% independant.
424 2017-04-20 19:52:04	0|BlueMatt|it seems to be a somewhat-surprising break
425 2017-04-20 19:52:19	0|jcorgan|and the name 'quarantined' might be a bit heavy handed
426 2017-04-20 19:52:27	0|gmaxwell|BlueMatt: the current privacy trashing is itself a very surprising break.
427 2017-04-20 19:52:32	0|BlueMatt|fair
428 2017-04-20 19:52:36	0|luke-jr|it might be less confusing if only the first receive ever was displayed/accepted, and all subsequent ones got quarantined
429 2017-04-20 19:52:49	0|gmaxwell|jcorgan: well I came up with that on the spot, on github they're calling it frozen, which I think is super misleading (bank froze my funds!). :P
430 2017-04-20 19:52:56	0|jcorgan|reserved?
431 2017-04-20 19:53:11	0|luke-jr|suspicious? :P
432 2017-04-20 19:53:13	0|wumpus|trash can
433 2017-04-20 19:53:17	0|Chris_Stewart_5|extraneous?
434 2017-04-20 19:53:19	0|gmaxwell|luke-jr: first recieved leads to an immediate attack: dust spammer races the payment then you get the dust and not the payment. :)
435 2017-04-20 19:53:22	0|morcos|gmaxwell: hmm...   i do like the idea of auto-quarantining spent address or dust left over in mostly spent addresses, but not sure i like default spending all the inputs and possibly giving you large change
436 2017-04-20 19:53:35	0|luke-jr|gmaxwell: first confirmed with a larger value?
437 2017-04-20 19:53:47	0|gmaxwell|morcos: really I'm surprised at that. That change alone is something I've wanted to do for a while (and was carrying patches for it for a bit)
438 2017-04-20 19:53:50	0|morcos|quarantine is a good name, but lets not bikeshed that
439 2017-04-20 19:54:14	0|luke-jr|gmaxwell: maybe auto-quarantine dust too
440 2017-04-20 19:54:26	0|instagibbs|morcos, assuming spending the dust is worthwhile, what's the concern?
441 2017-04-20 19:54:27	0|BlueMatt|gmaxwell: I'm not sold on non-end-user wallets here. it seems like it woul dbreak many merchant workflows that use bitcoind
442 2017-04-20 19:54:29	0|sdaftuar|auto-spending all the funds with a given address makes sense to me as well
443 2017-04-20 19:54:42	0|wumpus|morcos: let's quarantine the bikeshed
444 2017-04-20 19:54:49	0|luke-jr|BlueMatt: merchant workflows that reuse addresses are broken anyway
445 2017-04-20 19:54:53	0|BlueMatt|(eg you receive half a payment, your coin selection spends from that addr, then you receive the other half, and now you dont realize you got paid?)
446 2017-04-20 19:54:53	0|luke-jr|wumpus: lol
447 2017-04-20 19:54:56	0|gmaxwell|BlueMatt: why? (thats why I'm asking.) -- obviously it would be configurable.
448 2017-04-20 19:55:25	0|jtimon|re bikeshedding: the class managing this obviously needs to be called quarantiner
449 2017-04-20 19:55:33	0|gmaxwell|BlueMatt: how often are merchants doing that?  I mean you can get into advanced things like biasing selection to chose SPK that have least recently recieved funds to avoid that.
450 2017-04-20 19:55:56	0|BlueMatt|gmaxwell: I assume most merchants at least support multiple txn to complete your payment?
451 2017-04-20 19:56:02	0|BlueMatt|most guis ive read seem to imply that
452 2017-04-20 19:56:36	0|morcos|yeah i suppose if its configurable, an option that autospends everythign from any address that gets spent makes sense
453 2017-04-20 19:56:41	0|gmaxwell|BlueMatt: kinda. but they also require them to be recieved at effectively the same time. I think it's managable.
454 2017-04-20 19:56:45	0|instagibbs|I'm not sure they accept multiple txn as policy
455 2017-04-20 19:56:50	0|instagibbs|err automatically
456 2017-04-20 19:57:01	0|gmaxwell|Just the autospend alone would radically improve privacy, and would almost be enough except for the malicious dust creation.
457 2017-04-20 19:57:03	0|BlueMatt|gmaxwell: to make it compatible you'd have to never spend outputs newer than X, where X is merchant time frame
458 2017-04-20 19:57:32	0|gmaxwell|BlueMatt: ya, which would be a trivial 'first try without x' in the current framework.
459 2017-04-20 19:57:35	0|BlueMatt|i agree in principal, but it sounds like you'd break some folks' workflow in subtle ways. adding the option and defaulting off for non-gui users, maybe?
460 2017-04-20 19:58:06	0|sipa|i don't see how autospending would break anything?
461 2017-04-20 19:58:11	0|luke-jr|or just tweak how RPC shows quarantine
462 2017-04-20 19:58:13	0|gmaxwell|well it could evolve over time, too-- I do think it's not worth our time to do things here that we don't think could be on for a majority of users eventually in some form.
463 2017-04-20 19:58:22	0|instagibbs|At a minimum you could make near-dust be quarantined
464 2017-04-20 19:58:26	0|morcos|i would think there could be some threshold for auto-un-quarantining too right?  like if your quarantine address receives 1 BTC?  or maybe not, maybe it just becomes common sense to check that
465 2017-04-20 19:58:35	0|wumpus|in the first version this is introduced it should be disabled by default in any case, I think, let's present it as a security feature first. Could always be enabled by default later but that should not be the initial goal.
466 2017-04-20 19:58:43	0|gmaxwell|Because already people who are super aware of privacy can and will already manually do coin selection to achieve ends like this.
467 2017-04-20 19:58:50	0|BlueMatt|gmaxwell: agreed in principal, but there are also easier fixes we can do initially. eg bias coin selection towards this with fallbacks?
468 2017-04-20 19:58:51	0|luke-jr|wumpus: true
469 2017-04-20 19:58:55	0|gmaxwell|wumpus: absolutely.
470 2017-04-20 19:59:08	0|wumpus|anything that potentially 'disappears' funds shouldn't be enabled lightly
471 2017-04-20 19:59:09	0|luke-jr|it's harmless to add if it's disabled by default initially
472 2017-04-20 19:59:23	0|morcos|ok, so this sounds like general agreement that this is a good idea and has degenerated into arguing about defaults.   all development discussion in a nutshell!
473 2017-04-20 19:59:23	0|wumpus|luke-jr: exactly
474 2017-04-20 19:59:24	0|BlueMatt|"disabled by default" can also mean "if something fails, fall back to the current behavior"
475 2017-04-20 19:59:31	0|gmaxwell|Just in principle I don't think the resource investment is worth if it we don't think that the end goal couldn't be default-ish (e.g. GUI) use.
476 2017-04-20 19:59:33	0|BlueMatt|morcos: yup
477 2017-04-20 19:59:34	0|wumpus|I'd enable it personally
478 2017-04-20 19:59:54	0|wumpus|it's worth the resource investment if we think it's useful to have
479 2017-04-20 20:00:13	0|sdaftuar|so perhaps a first simple step would be to enable auto-spending of all funds from a given address in the coin selection logic?
480 2017-04-20 20:00:19	0|gmaxwell|I think users should be okay with 'multiple balances' we already have confirmed vs unconfirmed, and normal bank accounts have multiple balances.
481 2017-04-20 20:00:20	0|luke-jr|IMO the end goal should be to treat address reuse as something that just doesn't work, and have a quarantine people can dig out lost funds if necessary
482 2017-04-20 20:00:22	0|BlueMatt|sdaftuar: ack
483 2017-04-20 20:00:26	0|BlueMatt|DONG
484 2017-04-20 20:00:35	0|jtimon|I would start with the quarantine thing as only usable manually, which we all seem to like, and then propose automatic things
485 2017-04-20 20:00:36	0|gmaxwell|sdaftuar: yea, that would be a good first step with minimal impact, we might have to add some extra features with it, like automatic change splitting.
486 2017-04-20 20:00:36	0|wumpus|#endmeeting
487 2017-04-20 20:00:37	0|lightningbot|Log:            http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-04-20-19.02.log.html
488 2017-04-20 20:00:37	0|lightningbot|Meeting ended Thu Apr 20 20:00:36 2017 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
489 2017-04-20 20:00:37	0|lightningbot|Minutes:        http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-04-20-19.02.html
490 2017-04-20 20:00:37	0|lightningbot|Minutes (text): http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-04-20-19.02.txt
491 2017-04-20 20:00:42	0|morcos|i do think it could be default on..  i just think its a matter of avoiding surprised users who are wondering why money appears to be lost, but that can be solved with informing them
492 2017-04-20 20:00:51	0|wumpus|luke-jr: yeah, for the long long term I agree on that
493 2017-04-20 20:00:52	0|BlueMatt|morcos: or fallbacks
494 2017-04-20 20:00:53	0|luke-jr|jtimon: +1
495 2017-04-20 20:01:07	0|BlueMatt|"wait, i hit spend and it says not enough money" can be prevented while still using this
496 2017-04-20 20:01:35	0|gmaxwell|sdaftuar: I used to have a patch that basically post processes coin selection to add all other inputs that were in the same "address group" (listaddressgroupings) as anything selected.
497 2017-04-20 20:02:02	0|gmaxwell|sdaftuar: but it was kind of suboptimal, and with the existance of coinjoin it's less important to use a whole group rather than a whole address.
498 2017-04-20 20:02:08	0|wumpus|no one would send 1 BTC to an address to breach someones privacy
499 2017-04-20 20:02:16	0|wumpus|it's always small amounts
500 2017-04-20 20:02:43	0|luke-jr|wumpus: if they did, many people would be happy to give up their privacy XD
501 2017-04-20 20:02:52	0|gmaxwell|wumpus: well never say never, but "champaign problems"
502 2017-04-20 20:02:54	0|wumpus|luke-jr: xD
503 2017-04-20 20:03:19	0|sdaftuar|gmaxwell: one thing just occurred to me, if you receive lots of payments to the same address, and issue lots of payments yourself, then this will tie up lots of your utxo's, which could be operationally annoying?
504 2017-04-20 20:03:29	0|gmaxwell|unfortunately it can be realistic to send several dollars to do so, no problem, but there is only so much we can do.
505 2017-04-20 20:03:35	0|sdaftuar|ie you'll be generating lots of big unconfirmed chains, and run out of utxos
506 2017-04-20 20:03:45	0|sdaftuar|could*
507 2017-04-20 20:03:46	0|gmaxwell|sdaftuar: thus the comment about change splitting.
508 2017-04-20 20:03:51	0|sdaftuar|that doesn't help?
509 2017-04-20 20:03:56	0|sdaftuar|oh, some
510 2017-04-20 20:04:00	0|gmaxwell|yes it does, they'll go to different addresses.
511 2017-04-20 20:04:06	0|jcorgan|jtimon: i'll take a look at it
512 2017-04-20 20:04:20	0|morcos|also the threshold as to what is too small to include in the spend and leave quarantined is tricky...
513 2017-04-20 20:04:20	0|sdaftuar|well i was thinking that you're still spending an unconfirmed output, which will be under the descendant limit for the parent... eh, not sure how it would work out.
514 2017-04-20 20:04:38	0|wumpus|in any case I agree on the long run, address reuse should be seen as something suspicious unless the user opted in to it (e.g. a publically published address)
515 2017-04-20 20:04:54	0|gmaxwell|morcos: well I think there is a simple objective measure: at the current target feerate, what is the break even level?
516 2017-04-20 20:05:36	0|gmaxwell|monero seems to get along okay with protocol prohibited address reuse, we're maybe too conservative on some of these things. :)
517 2017-04-20 20:05:44	0|morcos|lets say you have 1mBTC that would cost 0.5mBTC to add as an input at the fee rate you are proposing for this tx...  maybe you prefer to leave that quarantined and send it separately at a lower fee rate  ..   certainly if you have multiple ones liek that that could be combined
518 2017-04-20 20:05:51	0|jtimon|jcorgan: unfortunately the documentation for the university had to be in spanish and I never bothered translating it https://github.com/jtimon/preann/blob/master/doc/pfc-jorge-timon.org (there's a latex generated from that and can give you a pdf as well) </sorry offtopic again>
519 2017-04-20 20:06:07	0|wumpus|so isn't such a bad idea to 'forget' old addresses after they've been used too long ago, or auto-quarantaine at least
520 2017-04-20 20:06:09	0|gmaxwell|morcos: if only we had a fee estimator that could give us a reasonable floor feerate! :)
521 2017-04-20 20:06:20	0|morcos|we do!
522 2017-04-20 20:06:30	0|morcos|2 sat/byte will get you confirmed within 500 blocks right now
523 2017-04-20 20:06:31	0|wumpus|but shouldn't be enabled by default at this point
524 2017-04-20 20:07:04	0|morcos|ahh, it went up to 3.3 since i last checked.. :)
525 2017-04-20 20:07:27	0|wumpus|I'd like my transactions to be confirmed within 100 years at least :)
526 2017-04-20 20:08:21	0|gmaxwell|morcos: right so you could use a floor feerate. But also I think it would be reasonable for us to have behavior that cleans up the UTXO set some at the users (small) expense, I think most users would support that, especially when it has privacy benefits.
527 2017-04-20 20:08:30	0|wumpus|otherwise I'd have to restart my node first to prevent 64 bit node ids from overflowing
528 2017-04-20 20:08:55	0|instagibbs|wumpus, no patience eh?
529 2017-04-20 20:09:11	0|gmaxwell|There is a whole layer of extra features we could think about what to do with the quarantined funds... but I think that should be future work.
530 2017-04-20 20:09:17	0|jcorgan|jtimon: eso no es problema
531 2017-04-20 20:09:29	0|jtimon|hahaha, estupendo!
532 2017-04-20 20:10:29	0|gmaxwell|e.g. if later we have some kind of coinjoin intergration, they could be preferentially sent into that.
533 2017-04-20 20:11:33	0|gmaxwell|too many things going on, we've responded to this too slowly. :( oh well, in any case, I think that this will prevent a lot of dust from getting created.
534 2017-04-20 20:12:28	0|gmaxwell|so it's kind of counter intutive, you might worry that the quarantine would result in UTXO bloat, but I suspect the opposite, at least if we're able to make this default-ish: with the incentive to make the tracing payments gone, they'll stop being created.
535 2017-04-20 20:12:49	0|wumpus|add an 'empty trash can' that sends the quarantained funds into devnull
536 2017-04-20 20:13:25	0|morcos|participate in network spam attack using quarantined funds button
537 2017-04-20 20:13:31	0|gmaxwell|I've mused about the idea about having some shred wallet feature that creates some long timelocked spend of any remaining coins and gives them over to fees... then sends them off somewhere.
538 2017-04-20 20:14:06	0|wumpus|'send wallet to wumpus'
539 2017-04-20 20:14:16	0|sipa|ACK
540 2017-04-20 20:14:24	0|gmaxwell|because I am somewhat pained by all the utxo bloat created by people who end up with 0.00001 BTC in a wallet, in 100 inputs, and then they just delete the file because its effectively worthless.
541 2017-04-20 20:14:51	0|gmaxwell|yea, wumpus is fine too. tricky part is timelocking it so that they have some time to reconsider their decision. :P
542 2017-04-20 20:15:07	0|gmaxwell|personally I wouldn't want it, you'll get clowns using it as a backup service and demanding their funds back. :P
543 2017-04-20 20:15:35	0|wumpus|gmaxwell: yes, there are certainly some practical issues :p
544 2017-04-20 20:15:36	0|morcos|that reminds me, we should revisit before 0.15 the dust level the wallet will create
545 2017-04-20 20:16:32	0|wumpus|sending to fees would be a better idea
546 2017-04-20 20:16:43	0|wumpus|especially those small amounts...
547 2017-04-20 20:16:55	0|gmaxwell|morcos: if we had a really good lower bound fee estimat it would be sensible to use that. e.g. don't create any change output where more than half its value would be lost in fees.
548 2017-04-20 20:17:29	0|morcos|but it depends on what you mean by that
549 2017-04-20 20:17:49	0|wumpus|morcos: what would you like to revisit it to?
550 2017-04-20 20:18:23	0|morcos|wumpus: not change the network definition, but make the wallet smarter about not creating (ever) outputs just about the network standard limit
551 2017-04-20 20:18:32	0|morcos|for instance like #9343 does
552 2017-04-20 20:18:34	0|gribble|https://github.com/bitcoin/bitcoin/issues/9343 | An error has occurred and has been logged. Please contact this bot's administrator for more information.
553 2017-04-20 20:18:52	0|morcos|gmaxwell: the problem is historically the lower bound fee estimate is 1 sat/byte
554 2017-04-20 20:19:17	0|morcos|i think any transaction ever created which paid more than that could have been mined by now, some probably weren't because they were collectively forgotten about
555 2017-04-20 20:19:40	0|morcos|but the lowest feerate mined on the weekend often drops that low
556 2017-04-20 20:19:50	0|morcos|people are just in a hurry to be confirmed quickly
557 2017-04-20 20:20:10	0|wumpus|morcos: tagging #9343 for 0.15
558 2017-04-20 20:20:12	0|gribble|https://github.com/bitcoin/bitcoin/issues/9343 | An error has occurred and has been logged. Please contact this bot's administrator for more information.
559 2017-04-20 20:20:34	0|morcos|tag 10199 too please
560 2017-04-20 20:20:59	0|wumpus|done
561 2017-04-20 21:01:23	0|sipa|nanotube: can we have the bot not produce output instead of "An error has occurred" ?
562 2017-04-20 21:08:07	0|BlueMatt|why does it fial so much in the first place?
563 2017-04-20 21:24:21	0|luke-jr|is 0.14.1 supposed to be building a new Qt?
564 2017-04-20 21:24:47	0|wumpus|relative to 0.14.1rcX? no
565 2017-04-20 21:24:57	0|bitcoin-git|[13bitcoin] 15laanwj pushed 3 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/14c948987f0b...86ea3c2ff247
566 2017-04-20 21:24:58	0|bitcoin-git|13bitcoin/06master 141c897fc 15Jorge Timón: Missing includes
567 2017-04-20 21:24:58	0|bitcoin-git|13bitcoin/06master 14a1fd450 15Jorge Timón: Trivial: Remove unneeded includes from .h:...
568 2017-04-20 21:24:59	0|bitcoin-git|13bitcoin/06master 1486ea3c2 15Wladimir J. van der Laan: Merge #10181: Include cleanup...
569 2017-04-20 21:25:15	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #10181: Include cleanup  (06master...062017-04-10-includes) 02https://github.com/bitcoin/bitcoin/pull/10181
570 2017-04-20 21:25:47	0|wumpus|doesn't seem to be doing that here, win already finished buildling
571 2017-04-20 21:26:14	0|luke-jr|wumpus: relative to 0.14.0
572 2017-04-20 21:26:19	0|luke-jr|I missed the RCs
573 2017-04-20 21:27:06	0|wumpus|I'm not sure of that
574 2017-04-20 21:27:48	0|cfields|yes
575 2017-04-20 21:27:55	0|cfields|due to zlib bump
576 2017-04-20 21:28:01	0|luke-jr|ah, since zlib is a dep?
577 2017-04-20 21:28:10	0|cfields|yep
578 2017-04-20 21:43:53	0|jtimon|regarding https://github.com/bitcoin/bitcoin/pull/10193 I'm still failing at replacing BOOST_REVERSE_FOREACH and I don't understand why, maybe I should reduce the scope of the PR (I moved from working stuff to trying to fully remove boost/foreach.hpp by popular demand)
579 2017-04-20 21:44:07	0|jtimon|?
580 2017-04-20 21:46:16	0|wumpus|zlib is at the bottom of the food chain, dependency-wise, not surprised it triggers rebuild of everything
581 2017-04-20 21:46:51	0|wumpus|jtimon: what's the problem with BOOST_REVERSE_FOREACH?
582 2017-04-20 21:49:11	0|jtimon|wumpus: I'm trying to replace it with https://github.com/bitcoin/bitcoin/pull/10193/commits/1e59de57eecc1a61b036d7f89ff6bf918c2c7f67, which I found in the interwebs but although I thought I fully understood, it seems I don't
583 2017-04-20 21:49:32	0|nanotube|sipa: yes, though it's easier to just fix the problem :) which is that github decided to add fancy middle-dots to its title, which make the existing code barf with unicode errors >_<
584 2017-04-20 21:50:03	0|luke-jr|lol
585 2017-04-20 21:50:07	0|luke-jr|nanotube: you're alive!
586 2017-04-20 21:51:36	0|nanotube|luke-jr: o/ :)
587 2017-04-20 21:52:38	0|jtimon|question
588 2017-04-20 21:52:38	0|jtimon|wumpus: it seems to interfere with prevector templates in prevector_tests.cpp, see https://github.com/bitcoin/bitcoin/pull/10193/commits/cfef34884684e71c6f43ef3e4f2e87590fc87c9e for the trick to make the PR pass travis. probably I should remove that commit already, but I wanted to make sure the commits after removing BOOST_REVERSE_FOREACH weren't breaking something else, plus that commit is kind of the link that maybe answers your
589 2017-04-20 21:53:47	0|jtimon|I would really prefer to just solve the problem but I'm kind of lost
590 2017-04-20 21:54:10	0|wumpus|jtimon: it's sad that c++11 doesn't provide an as-is replacement
591 2017-04-20 21:54:44	0|jtimon|wumpus: yep, it seems things get slightly better on c
592 2017-04-20 21:54:48	0|cfields|wumpus: does reverse_iterate not end up being a dangling reference? I'm not sure what lifetime is expected for the container in a range-based-for
593 2017-04-20 21:55:01	0|jtimon|c++0.14, I mean...c++14
594 2017-04-20 21:55:38	0|cfields|(rather, I don't know if the loop extends the lifetime of the container)
595 2017-04-20 21:55:53	0|cfields|er, that was for jtimon, sorry
596 2017-04-20 21:56:19	0|wumpus|jtimon: ok, so it's just a matter of waiting a few years (hey at least not 100 years :p)
597 2017-04-20 21:56:42	0|wumpus|c++17 is pretty nice too, esp std::optional
598 2017-04-20 21:57:08	0|nanotube|testing gribble title fix #9343
599 2017-04-20 21:57:10	0|gribble|https://github.com/bitcoin/bitcoin/issues/9343 | Dont create change at dust limit by morcos · Pull Request #9343 · bitcoin/bitcoin · GitHub
600 2017-04-20 21:57:17	0|nanotube|\o/
601 2017-04-20 21:57:26	0|wumpus|woohoo
602 2017-04-20 21:57:32	0|jtimon|no, this can be certainly solved in c++11, but maybe not in a very clean way, perhaps we want our own macro to replace it
603 2017-04-20 21:58:41	0|wumpus|I'd say replacing it with another macro would not be worth it
604 2017-04-20 21:59:39	0|wumpus|there are still significant impediments to ditching boost wholesale, and until we have replacements for those, there's no use in rolling our own for the simpler things. Certainly not if they're not clean and simple.
605 2017-04-20 21:59:59	0|jtimon|or just get rid of BOOST_FOREACH, ¿Q_FOREACH? and PAIRTYPE for now, but not BOOST_REVERSE_FOREACH or #include <boost/foreach.hpp> for now (although the include would only be used for BOOST_REVERSE_FOREACH now)
606 2017-04-20 22:00:28	0|wumpus|but maybe we can get your reverse_iterator to work
607 2017-04-20 22:00:46	0|wumpus|I would be surprised if it's not possible
608 2017-04-20 22:02:12	0|jtimon|I am already surprised that this is not working, it's working for all the other cases using BOOST_REVERSE_FOREACH, just not compiling prevector_tests for reasons beyond me. it seems the templating is colliding somehow
609 2017-04-20 22:03:06	0|jtimon|but the option of reducing the scope and leave fully removing boost/foreach.hpp for a later PR is also there, that's why I ask
610 2017-04-20 22:03:31	0|jtimon|btw, sorry for asking again, but any blockers for  #10189 ?
611 2017-04-20 22:03:33	0|gribble|https://github.com/bitcoin/bitcoin/issues/10189 | devtools/net: add a verifier for scriptable changes. Use it to make CNode::id private. by theuni · Pull Request #10189 · bitcoin/bitcoin · GitHub
612 2017-04-20 22:05:22	0|TD-Linux|gmaxwell, you could have a checkbox when generating an address that indicates it's supposed to be public
613 2017-04-20 22:05:30	0|jtimon|nevermind, just remembered cfields needs to enforce a commit tittle prefix
614 2017-04-20 22:05:54	0|cfields|jtimon: that's done
615 2017-04-20 22:05:57	0|TD-Linux|it seems more useful than the "reuse an existing address" checkbox there now
616 2017-04-20 22:06:07	0|cfields|I suppose I should comment
617 2017-04-20 22:06:33	0|gmaxwell|TD-Linux: there is a reuse checkbox? I don't recall that.
618 2017-04-20 22:07:08	0|TD-Linux|gmaxwell, maybe it's gone in the latest version, let me check. regardless I don't understand the use case
619 2017-04-20 22:07:19	0|luke-jr|gmaxwell: I have a PR open to remove it..
620 2017-04-20 22:08:14	0|jtimon|cfields: oops, great! thanks
621 2017-04-20 22:08:30	0|gmaxwell|is that just the thing that brings up the dislog that shows you existing addresses.
622 2017-04-20 22:12:09	0|jtimon|let me rebase and remove the travis-cheating commit for everyone to see the error without having to build locally, maybe it's obvious to someone else
623 2017-04-20 22:15:16	0|TD-Linux|https://github.com/bitcoin/bitcoin/pull/3716
624 2017-04-20 22:17:32	0|gmaxwell|oh well generating a QR code for an older address still seems useful.
625 2017-04-20 22:17:42	0|gmaxwell|should probably be elsewhere though.
626 2017-04-20 22:18:34	0|TD-Linux|I'd rather have it accessible via a right click in the list below but I guess that list is only so long
627 2017-04-20 22:41:41	0|bitcoin-git|[13bitcoin] 15shigeya opened pull request #10245: Minor fix in build documentation for FreeBSD 11 (06master...06freebsd-11-build-doc-fix) 02https://github.com/bitcoin/bitcoin/pull/10245
628 2017-04-20 23:02:42	0|bitcoin-git|[13bitcoin] 15practicalswift opened pull request #10246: Silence "warning: "MSG_DONTWAIT" redefined" when compiling under Linux (06master...06silence-msg_dontwait-warning) 02https://github.com/bitcoin/bitcoin/pull/10246
629 2017-04-20 23:06:15	0|jtimon|also I insist if you don't find #8855 interesting to review, maybe you do find #8994 interesting
630 2017-04-20 23:06:18	0|gribble|https://github.com/bitcoin/bitcoin/issues/8855 | Use a proper factory for creating chainparams by jtimon · Pull Request #8855 · bitcoin/bitcoin · GitHub
631 2017-04-20 23:06:19	0|gribble|https://github.com/bitcoin/bitcoin/issues/8994 | Testchains: Introduce custom chain whose constructor... by jtimon · Pull Request #8994 · bitcoin/bitcoin · GitHub
632 2017-04-20 23:21:51	0|bitcoin-git|[13bitcoin] 15practicalswift closed pull request #10246: Silence "warning: "MSG_DONTWAIT" redefined" when compiling under Linux (06master...06silence-msg_dontwait-warning) 02https://github.com/bitcoin/bitcoin/pull/10246
633 2017-04-20 23:36:27	0|luke-jr|y'all slow today; I had to build the new deps and still beat everyone but achow101 :P