1 2018-05-24 08:03:49	0|wumpus|MarcoFalke: agree re 13253
  2 2018-05-24 08:05:56	0|wumpus|getting the 0.16.1 release out should probably be priority now
  3 2018-05-24 08:11:14	0|wumpus|moneyball: hahahahaha "double unicorn"
  4 2018-05-24 08:16:11	0|wumpus|moneyball: but good to know they're making progress. A really persistent one was jtimon's PR that I merged yesterday, #10757.
  5 2018-05-24 08:16:18	0|gribble|https://github.com/bitcoin/bitcoin/issues/10757 | RPC: Introduce getblockstats to plot things by jtimon · Pull Request #10757 · bitcoin/bitcoin · GitHub
  6 2018-05-24 08:25:03	0|fanquake|wumpus there are still a couple unclean/complicated backports to do.
  7 2018-05-24 08:25:06	0|fanquake|Wondering if the people that PR'd the changes to master want to handle the backporting?
  8 2018-05-24 08:25:56	0|fanquake|wumpus Also if you're happy with it, I think #13246 is ready. I've tested, and it's a good cleanup.
  9 2018-05-24 08:25:58	0|gribble|https://github.com/bitcoin/bitcoin/issues/13246 | doc: Bump to Ubuntu Bionic 18.04 in build-windows.md by ken2812221 · Pull Request #13246 · bitcoin/bitcoin · GitHub
 10 2018-05-24 08:31:21	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #13253: [0.16] Further Backports (060.16...060-16-further-backports) 02https://github.com/bitcoin/bitcoin/pull/13253
 11 2018-05-24 08:36:38	0|wumpus|fanquake: I think it's preferable if the people that PRed the changes to master do that, yes
 12 2018-05-24 08:37:03	0|wumpus|at the very least they'd need to review it carefully
 13 2018-05-24 08:44:47	0|bitcoin-git|13bitcoin/06master 149d4f942 15Chun Kuan Lee: doc: Bump to Ubuntu Bionic 18.04 in build-windows.md...
 14 2018-05-24 08:44:47	0|bitcoin-git|[13bitcoin] 15laanwj pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/7f4db9a7c354...5c41b6008079
 15 2018-05-24 08:44:48	0|bitcoin-git|13bitcoin/06master 145c41b60 15Wladimir J. van der Laan: Merge #13246: doc: Bump to Ubuntu Bionic 18.04 in build-windows.md...
 16 2018-05-24 08:45:15	0|fanquake|wumpus I've added them to the "Blockers" in the High Prio list
 17 2018-05-24 08:45:33	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #13246: doc: Bump to Ubuntu Bionic 18.04 in build-windows.md (06master...06patch-2) 02https://github.com/bitcoin/bitcoin/pull/13246
 18 2018-05-24 08:45:35	0|fanquake|I think we can just about untag that one for 0.15.2
 19 2018-05-24 08:47:08	0|kallewoof|Is there a good link to the high prio list? I always have a hard time finding it and keep a tab of https://github.com/bitcoin/bitcoin/pulls?q=is%3Aopen+is%3Apr+project%3Abitcoin%2Fbitcoin%2F8 open, but not sure that URL changes at some point.
 20 2018-05-24 08:47:24	0|kallewoof|"bitcoin/bitcoin/8" doesn't seem particularly permanent.
 21 2018-05-24 08:48:22	0|wumpus|fanquake: you mean the GUI settings dialog crash? yes, probably, I don't think it's so likely that we'll do a 0.15.x release anyway
 22 2018-05-24 08:49:13	0|wumpus|kallewoof: pretty much all github URLs are permanent (for some definitions of permanent at least)
 23 2018-05-24 08:49:30	0|fanquake|wumpus yep, I'll untag it
 24 2018-05-24 08:49:37	0|wumpus|fanquake: already did
 25 2018-05-24 08:49:43	0|fanquake|:o
 26 2018-05-24 08:50:10	0|wumpus|kallewoof: '8' is the project ID which is not supposed to change
 27 2018-05-24 08:50:13	0|fanquake|kallewoof I think https://github.com/bitcoin/bitcoin/projects/8 is your best bet for now
 28 2018-05-24 08:52:28	0|kallewoof|OK, thanks
 29 2018-05-24 09:11:39	0|aj|kallewoof: https://github.com/bitcoin/bitcoin/projects/8 ?
 30 2018-05-24 09:12:08	0|aj|ah, helps if you don't get lost in scrollback
 31 2018-05-24 09:16:42	0|promag|wumpus: #13063
 32 2018-05-24 09:16:45	0|gribble|https://github.com/bitcoin/bitcoin/issues/13063 | Use shared pointer to retain wallet instance by promag · Pull Request #13063 · bitcoin/bitcoin · GitHub
 33 2018-05-24 09:26:55	0|wumpus|apparently there's a new "checks" thing on github PRs?
 34 2018-05-24 09:27:55	0|wumpus|promag: thanks
 35 2018-05-24 09:29:14	0|fanquake|wumpus yes, not quite sure what that's for yet
 36 2018-05-24 09:29:20	0|promag|also #13160 could have more feedback
 37 2018-05-24 09:29:21	0|gribble|https://github.com/bitcoin/bitcoin/issues/13160 | wallet: Unlock spent outputs by promag · Pull Request #13160 · bitcoin/bitcoin · GitHub
 38 2018-05-24 09:29:32	0|fanquake|I wonder if it's for the linter type things we are currently running on Travis
 39 2018-05-24 09:29:44	0|promag|fanquake: I guess integrations must update first?
 40 2018-05-24 09:31:33	0|fanquake|promag Do you mean they will run before travis does?
 41 2018-05-24 09:33:20	0|promag|fanquake: I mean apps (travis for instance) must update first to use checks
 42 2018-05-24 09:33:36	0|promag|or we must update .travis.yml?
 43 2018-05-24 09:34:15	0|fanquake|promag ah ok. I'll have a read up https://blog.github.com/2018-05-07-introducing-checks-api/
 44 2018-05-24 09:34:15	0|promag|https://help.github.com/articles/about-status-checks/#checks
 45 2018-05-24 09:35:31	0|promag|but looks like a nice feature
 46 2018-05-24 09:36:25	0|fanquake|Also https://blog.travis-ci.com/2018-05-07-announcing-support-for-github-checks-api-on-travis-ci-com
 47 2018-05-24 09:37:15	0|fanquake|Also Also, if anyone wasn't aware https://docs.travis-ci.com/user/open-source-on-travis-ci-com
 48 2018-05-24 09:39:04	0|wumpus|seems like a nice feature, if it can list the failed checks in the PR instead of having to dig into the travis log every time
 49 2018-05-24 09:40:59	0|wumpus|although having to look for 'apps' in a 'marketplace' seems kind of sily
 50 2018-05-24 09:41:55	0|fanquake|heh
 51 2018-05-24 09:42:00	0|wumpus|"
 52 2018-05-24 09:42:02	0|wumpus|Organization owners and users with push access to a repository can create checks and statuses with GitHub's API. For more information, see "Checks" and "Statuses" in the GitHub Developer documentation."
 53 2018-05-24 09:46:34	0|wumpus|so we could push *our own* statuses, funny. Though agree with promag it would be useful if this was integrated into, say, travis, to not have to run parallel checking infrastructure.
 54 2018-05-24 09:58:13	0|bitcoin-git|13bitcoin/06master 1480b4910 15João Barbosa: wallet: Use shared pointer to retain wallet instance
 55 2018-05-24 09:58:13	0|bitcoin-git|[13bitcoin] 15laanwj pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/5c41b6008079...6378eef18f61
 56 2018-05-24 09:58:14	0|bitcoin-git|13bitcoin/06master 146378eef 15Wladimir J. van der Laan: Merge #13063: Use shared pointer to retain wallet instance...
 57 2018-05-24 09:58:49	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #13063: Use shared pointer to retain wallet instance (06master...062018-04-wallet-sharedptr) 02https://github.com/bitcoin/bitcoin/pull/13063
 58 2018-05-24 10:03:11	0|promag|wumpus: thanks, #13111 on high priority?
 59 2018-05-24 10:03:12	0|gribble|https://github.com/bitcoin/bitcoin/issues/13111 | [WIP] Add unloadwallet RPC by promag · Pull Request #13111 · bitcoin/bitcoin · GitHub
 60 2018-05-24 10:04:29	0|wumpus|I think we should try to keep high-priority discussion in the meetings
 61 2018-05-24 10:05:20	0|wumpus|that makes sure at least everyone has an idea of what is added. I'm okay with adding one inbetween when there is really a hurry, but this even has a [WIP] tag still
 62 2018-05-24 10:06:47	0|wumpus|also I think high priority == 0.16.1 now
 63 2018-05-24 10:10:14	0|promag|wumpus: when is 0.17 feature freeze?
 64 2018-05-24 10:12:11	0|wumpus|#12624
 65 2018-05-24 10:12:12	0|gribble|https://github.com/bitcoin/bitcoin/issues/12624 | Release schedule for 0.17.0 · Issue #12624 · bitcoin/bitcoin · GitHub
 66 2018-05-24 10:58:06	0|promag|tks
 67 2018-05-24 12:12:08	0|fanquake|wumpus #13284 should be able to go in. Pretty trivial fix.
 68 2018-05-24 12:12:10	0|gribble|https://github.com/bitcoin/bitcoin/issues/13284 | gui: fix visual "overflow" of amount input. by brandonrninefive · Pull Request #13284 · bitcoin/bitcoin · GitHub
 69 2018-05-24 12:29:12	0|wumpus|fanquake: will look at it, thanks
 70 2018-05-24 13:07:12	0|bitcoin-git|13bitcoin/06master 14c865ee1 15Wladimir J. van der Laan: Fix FreeBSD build by including utilstrencodings.h...
 71 2018-05-24 13:07:12	0|bitcoin-git|[13bitcoin] 15MarcoFalke pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/6378eef18f61...a9b6957383a7
 72 2018-05-24 13:07:13	0|bitcoin-git|13bitcoin/06master 14a9b6957 15MarcoFalke: Merge #13314: Fix FreeBSD build by including utilstrencodings.h...
 73 2018-05-24 13:08:01	0|bitcoin-git|[13bitcoin] 15MarcoFalke closed pull request #13314: Fix FreeBSD build by including utilstrencodings.h (06master...062018_05_freebsd_build) 02https://github.com/bitcoin/bitcoin/pull/13314
 74 2018-05-24 13:11:46	0|bitcoin-git|13bitcoin/06master 1497c112d 15Ben Woosley: Declare TorReply parsing functions in torcontrol_tests...
 75 2018-05-24 13:11:46	0|bitcoin-git|[13bitcoin] 15MarcoFalke pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/a9b6957383a7...536120ec39be
 76 2018-05-24 13:11:47	0|bitcoin-git|13bitcoin/06master 14536120e 15MarcoFalke: Merge #13291: test: Don't include torcontrol.cpp into the test file...
 77 2018-05-24 13:12:32	0|bitcoin-git|[13bitcoin] 15MarcoFalke closed pull request #13291: test: Don't include torcontrol.cpp into the test file (06master...06tor-reply) 02https://github.com/bitcoin/bitcoin/pull/13291
 78 2018-05-24 13:52:07	0|bitcoin-git|13bitcoin/06master 145f3cbde 15Brandon Ruggles: Increased max width of amount field to prevent number overflow bug.
 79 2018-05-24 13:52:07	0|bitcoin-git|[13bitcoin] 15laanwj pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/536120ec39be...f8be43413368
 80 2018-05-24 13:52:08	0|bitcoin-git|13bitcoin/06master 14f8be434 15Wladimir J. van der Laan: Merge #13284: gui: fix visual "overflow" of amount input....
 81 2018-05-24 13:52:53	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #13284: gui: fix visual "overflow" of amount input. (06master...06ui_amount_overflow_fix) 02https://github.com/bitcoin/bitcoin/pull/13284
 82 2018-05-24 13:55:18	0|fanquake|jonasschnelli I probably should have tested on Windows, but I've almost had enough of VMs for the day
 83 2018-05-24 13:55:32	0|fanquake|If Windows is broken somehow it'll be the odd one out
 84 2018-05-24 13:58:01	0|jonasschnelli|fanquake: heh!
 85 2018-05-24 14:04:48	0|wumpus|ideally it'd use the font size, instead of a fixed width
 86 2018-05-24 14:05:08	0|wumpus|but an improvement is an improvement...
 87 2018-05-24 14:06:27	0|wumpus|(but there is a qt function to compute font extents, which we use in some places, if this ever comes up as issue again we should use that)
 88 2018-05-24 14:10:16	0|bitcoin-git|[13bitcoin] 15FeedTheWeb opened pull request #13315: remove ZMQ message limit (06master...06master) 02https://github.com/bitcoin/bitcoin/pull/13315
 89 2018-05-24 14:11:17	0|bitcoin-git|13bitcoin/06master 14fa865ef 15MarcoFalke: qa: Fix wallet_listreceivedby race
 90 2018-05-24 14:11:17	0|bitcoin-git|[13bitcoin] 15MarcoFalke pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/f8be43413368...610f4dd719ad
 91 2018-05-24 14:11:18	0|bitcoin-git|13bitcoin/06master 14610f4dd 15MarcoFalke: Merge #13304: qa: Fix wallet_listreceivedby race...
 92 2018-05-24 14:12:11	0|bitcoin-git|[13bitcoin] 15MarcoFalke closed pull request #13304: qa: Fix wallet_listreceivedby race (06master...06Mf1805-qaWalletRace) 02https://github.com/bitcoin/bitcoin/pull/13304
 93 2018-05-24 14:26:35	0|jnewbery|moneyball, jimpo: thanks for continuing to chase Github!
 94 2018-05-24 14:27:00	0|jnewbery|I also got an email from Ben at Github with another workaround in case people are still having issues: https://0bin.net/paste/4maKKIx04UweS5rA#EvAA3aPuNEb8fRr8x+csVRwfjjS5VCmbP5Mn7r3OGvz
 95 2018-05-24 14:27:34	0|jnewbery|append ?timeline_per_page=20 to the end of any PR URL
 96 2018-05-24 14:35:25	0|wumpus|github has a way to restrict the number of posts per page? that's good to know
 97 2018-05-24 16:39:19	0|promag|jnewbery: #13111 begs for your review =)
 98 2018-05-24 16:39:21	0|gribble|https://github.com/bitcoin/bitcoin/issues/13111 | Add unloadwallet RPC by promag · Pull Request #13111 · bitcoin/bitcoin · GitHub
 99 2018-05-24 16:47:15	0|jnewbery|promag: Of course I will, in good time. I've just reviewed #13100, and I don't think there's a huge rush to review all the load/unload wallet PRs in parallel
100 2018-05-24 16:47:18	0|gribble|https://github.com/bitcoin/bitcoin/issues/13100 | gui: Add menu entry to open wallet by promag · Pull Request #13100 · bitcoin/bitcoin · GitHub
101 2018-05-24 16:47:44	0|promag|jnewbery: what should come first? menu entries or unload?
102 2018-05-24 16:58:41	0|jnewbery|I don't think it matters. I'm more concerned about not hogging reviewer/maintainer time.
103 2018-05-24 16:58:58	0|jnewbery|I got another update from Ben at Github: One more quick update, the performance improvement I mentioned also made it into production moments ago, so hopefully the URL argument workaround should be less-and-less necessary as well.
104 2018-05-24 17:40:22	0|bitcoin-git|[13bitcoin] 15MarcoFalke opened pull request #13317: [0.16.1] Remainig backports (060.16...06Mf1805-016ForBackport) 02https://github.com/bitcoin/bitcoin/pull/13317
105 2018-05-24 18:46:49	0|promag|jnewbery: I prefer unloading first, then UI actions can be added at the same time for both load and unload
106 2018-05-24 18:51:28	0|sipa|jimpo: in the raw data about bip 158, the second but last column is the number of entries in the basic filter
107 2018-05-24 18:52:06	0|sipa|is that the number of unique elements, or the total number? (i.e. does it count multiple identical scriptPubKeys once or multiple times?)
108 2018-05-24 18:53:08	0|jimpo|unique elements
109 2018-05-24 18:53:47	0|jimpo|if you want the raw data for the sub-filters (as being discussed on the mailing list), I have those too
110 2018-05-24 18:54:22	0|sipa|it seems the byte size is almost unbelievably close to the theoretical limit
111 2018-05-24 18:54:46	0|jimpo|yeah, 21 is the limit
112 2018-05-24 18:55:23	0|jimpo|I have a graph somewhere of average bitsize per element vs # of elements in a filter
113 2018-05-24 18:55:57	0|jimpo|it drops off very fast. at around 200 elements or so, I think it's already at 22 or 23 (need to double check that)
114 2018-05-24 18:56:43	0|sipa|jimpo: the limit is actually higher
115 2018-05-24 18:57:09	0|jimpo|oh? how's that?
116 2018-05-24 18:57:59	0|MarcoFalke|I cherry-picked the remaining backports here: #13317
117 2018-05-24 18:58:01	0|gribble|https://github.com/bitcoin/bitcoin/issues/13317 | [0.16.1] Remainig backports by MarcoFalke · Pull Request #13317 · bitcoin/bitcoin · GitHub
118 2018-05-24 18:58:29	0|MarcoFalke|I believe the conflicts were only due to refactoring (variable names changed) and one adjacent documentation change
119 2018-05-24 18:58:47	0|sipa|jimpo: let's discuss after meeting
120 2018-05-24 18:59:12	0|wumpus|#startmeeting
121 2018-05-24 18:59:13	0|lightningbot|Meeting started Thu May 24 19:00:06 2018 UTC.  The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
122 2018-05-24 18:59:13	0|lightningbot|Useful Commands: #action #agreed #help #info #idea #link #topic.
123 2018-05-24 18:59:19	0|promag|hi
124 2018-05-24 18:59:37	0|jimpo|hi
125 2018-05-24 18:59:38	0|murch1|hello
126 2018-05-24 18:59:47	0|jcorgan|lurking
127 2018-05-24 18:59:55	0|wumpus|#bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr btcdrak sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier jl2012 achow101 meshcollider jnewbery maaku fanquake promag provoostenator
128 2018-05-24 19:00:03	0|achow101|hi
129 2018-05-24 19:00:07	0|jonasschnelli|hi
130 2018-05-24 19:00:38	0|wumpus|I guess the topic of priority today is 0.16.1
131 2018-05-24 19:00:50	0|jamesob|hi
132 2018-05-24 19:00:56	0|jnewbery|hello
133 2018-05-24 19:01:02	0|wumpus|#topic 0.16.1
134 2018-05-24 19:01:09	0|wumpus|there's a few backports left to do
135 2018-05-24 19:01:37	0|cfields|hi
136 2018-05-24 19:01:37	0|wumpus|or does #13317 include all of them?
137 2018-05-24 19:01:38	0|gribble|https://github.com/bitcoin/bitcoin/issues/13317 | [0.16.1] Remainig backports by MarcoFalke · Pull Request #13317 · bitcoin/bitcoin · GitHub
138 2018-05-24 19:01:47	0|MarcoFalke|wumpus: I left out the qt one
139 2018-05-24 19:01:59	0|MarcoFalke|I think it changes translations. I might do it in a separate pull
140 2018-05-24 19:02:10	0|wumpus|I'll add that one to "high priority for review" at least
141 2018-05-24 19:02:13	0|wumpus|MarcoFalke: ok, thanks!
142 2018-05-24 19:02:51	0|wumpus|so please all review that PR ^^
143 2018-05-24 19:03:31	0|wumpus|especially the non-trivial backports
144 2018-05-24 19:03:48	0|MarcoFalke|With regard to the other high priority prs in the project: I think we can remove the one from BlueMatt for now
145 2018-05-24 19:03:57	0|jtimon|https://github.com/bitcoin/bitcoin/pull/12172 is a small thing, but it is a bugfix, perhaps it makes sense to backport it?
146 2018-05-24 19:03:58	0|MarcoFalke|And add next week when the rebase is done
147 2018-05-24 19:04:15	0|sipa|#12172
148 2018-05-24 19:04:18	0|gribble|https://github.com/bitcoin/bitcoin/issues/12172 | Bugfix: RPC: savemempool: Dont save until LoadMempool() is finished by jtimon · Pull Request #12172 · bitcoin/bitcoin · GitHub
149 2018-05-24 19:04:40	0|wumpus|I think we should focus on 0.16.1 now, we'll get around to the other high priority stuff again next week
150 2018-05-24 19:04:42	0|MarcoFalke|jtimon: that lead to a ton of issues in our tests
151 2018-05-24 19:04:52	0|MarcoFalke|I'd prefer we didn't backport that one
152 2018-05-24 19:05:16	0|wumpus|I don't think it's terribly urgent to backport, maybe for 0.16.2
153 2018-05-24 19:05:19	0|jamesob|is #12431 worth a backport?
154 2018-05-24 19:05:21	0|gribble|https://github.com/bitcoin/bitcoin/issues/12431 | Only call NotifyBlockTip when chainActive changes by jamesob · Pull Request #12431 · bitcoin/bitcoin · GitHub
155 2018-05-24 19:05:32	0|MarcoFalke|jamesob: What bug does it fix?
156 2018-05-24 19:06:06	0|jamesob|potentially incorrect behavior in waitforblockheight (and anything else relying on rpc/blockchain.cpp:latestblock)
157 2018-05-24 19:06:08	0|jtimon|MarcoFalke: ok, no hurry, good to know someone tried it, now I'm curious about the issues in the tests and I'll play with travis and the backport
158 2018-05-24 19:06:26	0|MarcoFalke|jamesob: waitforblockheight is a hidden tests-only rpc
159 2018-05-24 19:06:29	0|wumpus|potentially how bad?
160 2018-05-24 19:06:31	0|wumpus|oh
161 2018-05-24 19:07:27	0|jnewbery|jtimon: #12863
162 2018-05-24 19:07:29	0|gribble|https://github.com/bitcoin/bitcoin/issues/12863 | mempool_persist.py failing in travis · Issue #12863 · bitcoin/bitcoin · GitHub
163 2018-05-24 19:08:05	0|jtimon|jnewbery: thanks
164 2018-05-24 19:08:48	0|wumpus|other topics?
165 2018-05-24 19:09:43	0|sipa|i want to bring up #13298 briefly
166 2018-05-24 19:09:45	0|gribble|https://github.com/bitcoin/bitcoin/issues/13298 | Net: Random delays *per network group* to obfuscate transaction time by naumenkogs · Pull Request #13298 · bitcoin/bitcoin · GitHub
167 2018-05-24 19:09:45	0|wumpus|anything else important for 0.16.1 that still needs to be done?
168 2018-05-24 19:09:55	0|sipa|(not for 0.16.1, to be clear)
169 2018-05-24 19:10:10	0|wumpus|(no problem, that was an orthogonal question)
170 2018-05-24 19:10:14	0|wumpus|#topic Random delays *per network group* to obfuscate transaction time
171 2018-05-24 19:10:51	0|sipa|i want to bring it up because it's a possibly significant effect on P2P transaction relay
172 2018-05-24 19:11:05	0|sipa|and it needs review beyond "does the code work"
173 2018-05-24 19:11:08	0|wumpus|I haven't had chance to look at it yet
174 2018-05-24 19:11:23	0|sipa|but it's also just local policy and not something that warrants a BIP imho
175 2018-05-24 19:11:42	0|wumpus|agree
176 2018-05-24 19:12:01	0|sipa|maybe there's not much more to say about that, just hoping to get people to think about it a bit
177 2018-05-24 19:12:39	0|wumpus|#action look at PR 13298
178 2018-05-24 19:13:00	0|wumpus|it's three days old, so people are excused not having an opinion on it yet :-)
179 2018-05-24 19:13:15	0|sipa|end topic :)
180 2018-05-24 19:13:21	0|cfields|thanks for bringing it up, I'll have a look as well...
181 2018-05-24 19:13:29	0|cfields|it'd be nice to have a tool to model that kind of change, though.
182 2018-05-24 19:13:29	0|wumpus|other topics?
183 2018-05-24 19:13:53	0|provoostenator|Topic suggestion: GUI prune setting
184 2018-05-24 19:14:29	0|wumpus|cfields: makes sense, maybe propose it in the PR
185 2018-05-24 19:14:42	0|wumpus|#topic GUI prune setting (provoostenator)
186 2018-05-24 19:14:44	0|provoostenator|AKA #13043
187 2018-05-24 19:14:47	0|gribble|https://github.com/bitcoin/bitcoin/issues/13043 | [qt] OptionsDialog: add prune setting by Sjors · Pull Request #13043 · bitcoin/bitcoin · GitHub
188 2018-05-24 19:15:29	0|jonasschnelli|Looks good,.. maybe orthogonal, but the prune settings should be in the intro...
189 2018-05-24 19:15:32	0|provoostenator|There's some confusion around whether using QT settings is appropriate for this, and I see three ways out.
190 2018-05-24 19:15:36	0|jonasschnelli|That's where it is probably most valuable
191 2018-05-24 19:16:05	0|provoostenator|1. ignore the problem
192 2018-05-24 19:16:08	0|wumpus|I'm ok with most solutions, except writing to bitcoin.conf
193 2018-05-24 19:16:18	0|provoostenator|2. go the writable config file route
194 2018-05-24 19:16:19	0|jonasschnelli|what wumpus sais
195 2018-05-24 19:16:20	0|kanzure|hi.
196 2018-05-24 19:16:34	0|provoostenator|3. interpret a lack of prune= setting differently
197 2018-05-24 19:17:21	0|jonasschnelli|Can't the GUI settings not just override what is already set?
198 2018-05-24 19:17:22	0|instagibbs|(3) is interesting
199 2018-05-24 19:17:39	0|achow101|there currently are a few settings that are saved to qt settings that are shared between qt and bitcoind
200 2018-05-24 19:17:46	0|achow101|but bitcoind can't access
201 2018-05-24 19:17:50	0|provoostenator|If we go for (2) then I'd like to nominate #11082 for priority review
202 2018-05-24 19:17:52	0|gribble|https://github.com/bitcoin/bitcoin/issues/11082 | Add new bitcoin_rw.conf file that is used for settings modified by this software itself by luke-jr · Pull Request #11082 · bitcoin/bitcoin · GitHub
203 2018-05-24 19:18:02	0|jonasschnelli|If prune is set via confi/startip, disallow access in the GUI settings (only display it)
204 2018-05-24 19:18:03	0|achow101|so if we follow what has been done previously, then we ignore it
205 2018-05-24 19:18:07	0|wumpus|yes an additional writable config file would be fine
206 2018-05-24 19:18:37	0|jonasschnelli|Do we want four(!) levels of configuration?
207 2018-05-24 19:18:42	0|jimpo|What about augmenting the GUI to help you generate a default config file when none already exists?
208 2018-05-24 19:18:43	0|jonasschnelli|And eventually importing conf-files?
209 2018-05-24 19:18:46	0|wumpus|there have also been plans to add RPCs to change configuration settings, that'd require a similar thing, just don't write the -conf file it's just as likely to be in an read-only directory
210 2018-05-24 19:19:29	0|jonasschnelli|Would the rw_conf file be replacing the QSettings layer?
211 2018-05-24 19:19:37	0|provoostenator|Right, I'd also like to get rid of QT settings completely and use a read-write (seperate) config file.
212 2018-05-24 19:19:37	0|wumpus|jonasschnelli: probably
213 2018-05-24 19:19:44	0|wumpus|jonasschnelli: long term, at least
214 2018-05-24 19:19:46	0|jonasschnelli|That would be acceptable
215 2018-05-24 19:20:04	0|provoostenator|I wrote a migration away from QTSettings here: #12833
216 2018-05-24 19:20:07	0|gribble|https://github.com/bitcoin/bitcoin/issues/12833 | [qt] move QSettings to bitcoin.conf where possible by Sjors · Pull Request #12833 · bitcoin/bitcoin · GitHub
217 2018-05-24 19:20:14	0|jonasschnelli|But please not conf<->startup-cli-params<->QTSettings<->level4_rw_conf
218 2018-05-24 19:20:20	0|wumpus|nice
219 2018-05-24 19:20:41	0|jonasschnelli|Thanks provoostenator for working on that!
220 2018-05-24 19:20:42	0|achow101|since this is a problem that effects multiple options, can we just ignore the problem for now and deal with them all together at the same time with a better solution?
221 2018-05-24 19:20:46	0|wumpus|yes storing some of the settings in a different place has been problematic
222 2018-05-24 19:21:00	0|wumpus|(at least in the QSettings - because bitcoind can't get there)
223 2018-05-24 19:21:06	0|instagibbs|users could by and large be migrated over the rw unless they have a need for read-only
224 2018-05-24 19:21:13	0|instagibbs|for simplicity
225 2018-05-24 19:21:15	0|wumpus|the only thing that would idally be stored in the QSettings would be the data directory
226 2018-05-24 19:21:28	0|wumpus|well the bitcoin.conf is for human editing
227 2018-05-24 19:21:33	0|provoostenator|This migration would also work if it's done _after_ we add prune stuff to QTSettings.
228 2018-05-24 19:21:41	0|wumpus|the _rw is machine writable, all comments will be discarded etc
229 2018-05-24 19:21:47	0|instagibbs|ah hm
230 2018-05-24 19:21:51	0|jonasschnelli|yes
231 2018-05-24 19:21:54	0|provoostenator|But if we can get the proper solution over with, that might be better.
232 2018-05-24 19:22:40	0|jonasschnelli|For 12833 I'm also unsure about the term "Limit".
233 2018-05-24 19:22:41	0|wumpus|on the other hand, having things be dependent on each other is usually a bad idea, just draws out things
234 2018-05-24 19:22:43	0|jonasschnelli|Since this is not true
235 2018-05-24 19:23:31	0|provoostenator|jonasschnelli: what "Limit"?
236 2018-05-24 19:24:15	0|jonasschnelli|provoostenator: 12833 currently tells user "Limit block storage to: " which is not true... though the tooptip hints towards the right handling.
237 2018-05-24 19:24:34	0|wumpus|how is it not true?
238 2018-05-24 19:24:53	0|provoostenator|That's #13043
239 2018-05-24 19:24:55	0|gribble|https://github.com/bitcoin/bitcoin/issues/13043 | [qt] OptionsDialog: add prune setting by Sjors · Pull Request #13043 · bitcoin/bitcoin · GitHub
240 2018-05-24 19:25:19	0|provoostenator|Which I fixed, but screenshot is outdated.
241 2018-05-24 19:25:34	0|provoostenator|It's now "Prune &amp;block storage to"
242 2018-05-24 19:25:52	0|jonasschnelli|wumpus: AFAIK the 550MB assumption (if one sets 550) is still on 1MB blocks
243 2018-05-24 19:26:02	0|jonasschnelli|But 288min blocks & undos are enforced
244 2018-05-24 19:26:37	0|wumpus|jonasschnelli: right, that's an issue for the command line help too though, I think?
245 2018-05-24 19:26:42	0|jonasschnelli|but maybe we should fix that (if it turns out to be a problem) rathern then changing the word "limit" :)
246 2018-05-24 19:26:48	0|provoostenator|I'm not too worried about details of the text and minimum size. It's what we need to do about saving settings that seems to cause things to get stuck.
247 2018-05-24 19:26:55	0|jonasschnelli|Indeed
248 2018-05-24 19:27:41	0|wumpus|other topics?
249 2018-05-24 19:27:47	0|jonasschnelli|I also hoped we could educate the user withing the GUI a bit more about pruning... but can be done later via extending the intro.cpp
250 2018-05-24 19:28:33	0|wumpus|right, let's focus on getting the functionality in first. I think educating the user is a good thing, but having the PR depend on all those things being worked out is going to put it way past 0.17 probably.
251 2018-05-24 19:29:00	0|jonasschnelli|I have a short topic: sipa raised concerns about the scantxoutset RPC command
252 2018-05-24 19:29:30	0|jonasschnelli|(if that is something we want to discuss here)
253 2018-05-24 19:29:51	0|wumpus|so the feature freeze for 0.17 is 2018-07-16, less than two months away
254 2018-05-24 19:30:08	0|provoostenator|Also note that I don't think Mac shows the intro dialog.
255 2018-05-24 19:30:14	0|wumpus|#topic scantxoutset RPC command (jonasschnelli)
256 2018-05-24 19:30:20	0|jonasschnelli|provoostenator: it does at least for me
257 2018-05-24 19:30:24	0|wumpus|provoostenator: it should!
258 2018-05-24 19:30:30	0|jonasschnelli|https://github.com/bitcoin/bitcoin/pull/12196#pullrequestreview-121570579
259 2018-05-24 19:30:46	0|provoostenator|jonasschnelli: ok, maybe I need to delete more stuff for it to appear.
260 2018-05-24 19:31:02	0|jonasschnelli|Before continuing on #12196 we may want to discuss it it makes sense to have a scantxoutset command
261 2018-05-24 19:31:05	0|gribble|https://github.com/bitcoin/bitcoin/issues/12196 | Add scantxoutset RPC method by jonasschnelli · Pull Request #12196 · bitcoin/bitcoin · GitHub
262 2018-05-24 19:31:09	0|wumpus|you need to delete -whereever it stores the qsettings on Mac-
263 2018-05-24 19:31:37	0|wumpus|on UNIX you can pass the flag -resetguisettings but that's not easy on Mac I think
264 2018-05-24 19:31:49	0|jonasschnelli|(works on mac as well)
265 2018-05-24 19:32:06	0|wumpus|good :)
266 2018-05-24 19:32:28	0|provoostenator|Resetting gui settings didn't do it for me, but will debug some other time.
267 2018-05-24 19:32:30	0|achow101|jonasschnelli: what is the command supposed to do?
268 2018-05-24 19:32:32	0|jonasschnelli|The scan functionality allows utxo sweeping (rawsweeptransaction) with no block scanning
269 2018-05-24 19:32:55	0|jonasschnelli|You can pass in n pubkeys/addresses or even xpubs with a lookup window and it gives you back all unspents
270 2018-05-24 19:33:02	0|sipa|yeah i just mentioned that we preferably don't commit to having functionality that's hard to maintain in the future
271 2018-05-24 19:33:04	0|jonasschnelli|And eben a rawsweeptransaction to a single address
272 2018-05-24 19:33:23	0|sipa|fun.
273 2018-05-24 19:33:24	0|instagibbs|err what
274 2018-05-24 19:33:27	0|wumpus|services massacre
275 2018-05-24 19:33:29	0|cfields|irc unicorns...
276 2018-05-24 19:34:01	0|cfields|let's move to slack!
277 2018-05-24 19:34:05	0|cfields|(/s)
278 2018-05-24 19:34:12	0|wumpus|:-(
279 2018-05-24 19:35:33	0|sipa|back to topic?
280 2018-05-24 19:35:35	0|sipa|yeah i just mentioned that we preferably don't commit to having functionality that's hard to maintain in the future
281 2018-05-24 19:35:37	0|jonasschnelli|Yes. I think sipa's point is a valid point. Do we want to maintain something that may be incompatible with future utxo handling (or new model=
282 2018-05-24 19:35:38	0|provoostenator|Such functionality seems quite useful for watch-only stuff
283 2018-05-24 19:35:48	0|provoostenator|Without actually having to import things into a wallet.
284 2018-05-24 19:35:59	0|sipa|which isn't a problem if it were implemented on top of an optional index
285 2018-05-24 19:36:42	0|jimpo|sipa: is the optional index a full scriptPubKey index?
286 2018-05-24 19:36:58	0|provoostenator|But without caching of some sort (or an index?) I'm guessing it'd be very slow.
287 2018-05-24 19:37:03	0|jimpo|or something less than that?
288 2018-05-24 19:37:16	0|jonasschnelli|sipa: what if we allow it for now an mention in the RN it may later require an optional index?
289 2018-05-24 19:37:47	0|sipa|yes, perhaps
290 2018-05-24 19:37:50	0|jonasschnelli|provoostenator: 30seconds for the whole index with an xpub & 1000 keys lookup window on a SSD/fastCPU machine
291 2018-05-24 19:37:57	0|jimpo|Or just have an explicit flag enabling the RPC now, even if it requires no additional index at present?
292 2018-05-24 19:38:00	0|jonasschnelli|*whole set
293 2018-05-24 19:38:21	0|sipa|yeah, that seems fine
294 2018-05-24 19:38:36	0|sipa|i do see the usefulness of scanning the UTXO set
295 2018-05-24 19:38:44	0|provoostenator|All the way back to the genesis block?
296 2018-05-24 19:38:54	0|jonasschnelli|jimpo: yes. But feels a bit after artificially holding back functions due to possible future work (which may never happen)
297 2018-05-24 19:38:57	0|sipa|provoostenator: it scans the UTXO set, not the blockchain
298 2018-05-24 19:38:59	0|wumpus|yes, scanning the utxo set would be incredibly useful
299 2018-05-24 19:39:09	0|wumpus|I've wanted this functionality since forever really
300 2018-05-24 19:39:24	0|provoostenator|Ah OK, it just gets unspends, not transaction history. Well that's still quite useful indeed.
301 2018-05-24 19:39:33	0|jonasschnelli|I wanted that in master before the fork-coins happend. :)
302 2018-05-24 19:40:00	0|wumpus|even if slow it's so much faster than scanning the entire chain
303 2018-05-24 19:40:07	0|wumpus|(without index)
304 2018-05-24 19:40:13	0|jonasschnelli|it would also allow importing wallets (no rescan required) if one don't care about the tx history
305 2018-05-24 19:40:45	0|wumpus|using importprunedfunds I guess?
306 2018-05-24 19:41:12	0|jonasschnelli|Yes..
307 2018-05-24 19:42:09	0|jonasschnelli|conclusion? Hide behind an artificial block-setting or risk it will not maintainable over time?
308 2018-05-24 19:42:54	0|jimpo|I think it's probably fine to risk it not being maintainable over time without an explicit index
309 2018-05-24 19:43:25	0|jonasschnelli|Yes. I would feel okay with that...
310 2018-05-24 19:43:51	0|jonasschnelli|Let me mention that risk in the RN and fix the PR in general /topic
311 2018-05-24 19:43:56	0|wumpus|so it will just become slower due to the linear scan?
312 2018-05-24 19:43:59	0|provoostenator|By "not being maintainable over time" do you mean if the UTXO set gets really large or is it a code maintenance things?
313 2018-05-24 19:44:12	0|jonasschnelli|from sipa:
314 2018-05-24 19:44:13	0|jonasschnelli|Overall, I'm unsure about this. This is functionality that is more easily provided by software that maintains a UTXO index by script, and is not possible in general if we'd move to a design like UHF (see mailinglist) or other UTXO avoidance techniques. Those are far away of course, and features like this can be made optional (like txindex is) if needed. I'm just generally unconvinced a full node is the best place to put
315 2018-05-24 19:44:13	0|jonasschnelli|this.
316 2018-05-24 19:44:16	0|wumpus|or is this 'unmaintainable' as in 'will give wumpus more headaches'?
317 2018-05-24 19:44:29	0|jonasschnelli|no.. changes of the general model
318 2018-05-24 19:44:45	0|sipa|wumpus: in a UHF model, without indexes, implementing a scan of the UTXO set requires going through the blockchain
319 2018-05-24 19:45:09	0|sipa|(where we store hashes of UTXOs rather than the UTXOs themselves)
320 2018-05-24 19:45:13	0|instagibbs|sipa, searchable index has been tried a few times, and sadly dropped, something like 3 times
321 2018-05-24 19:45:18	0|wumpus|sipa: so that's a far future thing right?
322 2018-05-24 19:45:21	0|instagibbs|not saying it's not the right way
323 2018-05-24 19:45:34	0|sipa|instagibbs: yes, my preference is that's implemented in other software
324 2018-05-24 19:45:57	0|jcorgan|i bet there's a dozen private implementations of tx/addr external indexing for xpub related things
325 2018-05-24 19:46:16	0|wumpus|I'd really like a way to scan UTXOs, my own appraoch was to stream the UTXO set over HTTP and do it client-side, but that ran into problems with the libevent http server :(
326 2018-05-24 19:46:32	0|jonasschnelli|Yes. But there is no fast access to the UTXO set from outside of our code-base IMO
327 2018-05-24 19:46:42	0|jcorgan|i do it with zmq notifications and the REST interface
328 2018-05-24 19:47:02	0|jcorgan|you're welcome :-)
329 2018-05-24 19:47:06	0|gribble|https://github.com/bitcoin/bitcoin/issues/7759 | [WIP] rest: Stream entire utxo set by laanwj · Pull Request #7759 · bitcoin/bitcoin · GitHub
330 2018-05-24 19:47:24	0|achow101|I've taken to modifying gettxoutsetinfo whenever I need to scan the utxo set. takes like 20 minutes though to scan the whole thing
331 2018-05-24 19:47:27	0|jonasschnelli|interesting... almost forgotten
332 2018-05-24 19:47:34	0|provoostenator|In this future where "we store hashes of UTXOs rather than the UTXOs themselves", wouldn't there simply be an optional "index" with the UTXO, which this RPC method could then move to?
333 2018-05-24 19:47:49	0|jonasschnelli|achow101: only if you are in debug mode? right... takes 30secs here
334 2018-05-24 19:47:51	0|wumpus|provoostenator: yes... exactly... I say it's a problem/option for then
335 2018-05-24 19:48:18	0|provoostenator|We'd have to make it clear that in the future the method would / might require an index.
336 2018-05-24 19:48:32	0|achow101|jonasschnelli: I think it was the operations I was doing, or my slow hdd
337 2018-05-24 19:48:34	0|sipa|it's the same issue as we had with txindex
338 2018-05-24 19:48:34	0|wumpus|achow101: on ARM it's pretty slow (but still faster than scanning the whole chain!)
339 2018-05-24 19:48:47	0|sipa|people build solution that assume txindex is always there... then we moved to a UTXO model
340 2018-05-24 19:49:14	0|sipa|and txindex became an inefficient optional thing
341 2018-05-24 19:49:14	0|wumpus|we can even deprecate the RPC then
342 2018-05-24 19:49:20	0|achow101|sipa: then it will be the same situation as getrawtransaction
343 2018-05-24 19:49:55	0|jimpo|with #13243, it should become less costly to build future indexes in the background...
344 2018-05-24 19:49:56	0|wumpus|I don't think we should reject useful, optional, functionality just because of some future data structure change
345 2018-05-24 19:49:57	0|gribble|https://github.com/bitcoin/bitcoin/issues/13243 | Make reusable base class for auxiliary indices by jimpo · Pull Request #13243 · bitcoin/bitcoin · GitHub
346 2018-05-24 19:49:58	0|sipa|achow101: my concern is not the incompatibility
347 2018-05-24 19:50:13	0|sipa|achow101: my concern is people building an ecosystem that assumes it's always possible and cheap to do
348 2018-05-24 19:50:49	0|sipa|but okay, i agree with the points here
349 2018-05-24 19:51:13	0|wumpus|sipa: I agree with that in general, but I'm not sure here
350 2018-05-24 19:51:18	0|jonasschnelli|jimpo: great work!
351 2018-05-24 19:51:23	0|jcorgan|if we want to encourage people to treat bitcoind as the "ground truth", instead of baking up their own stuff, giving them easier access to the "database" would help
352 2018-05-24 19:51:23	0|provoostenator|The fact that it takes 30 seconds is helpful in that case. :-)
353 2018-05-24 19:51:55	0|sipa|jcorgan: yes... except that the ground truth in the future may not be the UTXO set
354 2018-05-24 19:52:12	0|provoostenator|jcorgan: that too, would be nice to be able to get easy to export dumps of useful things, though not should what format.
355 2018-05-24 19:52:14	0|jcorgan|sure, but anything can happen in the future
356 2018-05-24 19:52:23	0|wumpus|sure, but anything can happen in the future <- that
357 2018-05-24 19:52:27	0|jonasschnelli|jimpo: the question is, if we not want to build a base for an external indexing daemon (outside of the Core project)
358 2018-05-24 19:52:51	0|wumpus|in any case this is hard to do out-of-process right now!
359 2018-05-24 19:52:54	0|wumpus|even harder than indexing
360 2018-05-24 19:52:58	0|jcorgan|i'd be happy with that, too, instead of making everyone else recreate it
361 2018-05-24 19:53:11	0|jonasschnelli|wumpus: Indeed
362 2018-05-24 19:53:16	0|sipa|it's also less a concern to add optional indexes now with jimpo's background index work
363 2018-05-24 19:53:30	0|sipa|before, new indexes always required ugly hacks all over the validation code
364 2018-05-24 19:53:36	0|wumpus|nice!
365 2018-05-24 19:53:38	0|jimpo|My guess is there will be ongoing tension between adding RPC functionality and keeping the node requirements small unless there are more options for users
366 2018-05-24 19:53:39	0|jcorgan|yes
367 2018-05-24 19:53:52	0|echeveria|it's a really bad idea to add an address index.
368 2018-05-24 19:53:53	0|provoostenator|Not too mention that you couldn't turn on/off txindex without reindexing everything.
369 2018-05-24 19:53:58	0|instagibbs|jimpo, an rpc call in every garage!
370 2018-05-24 19:54:02	0|jcorgan|i used to maintain the addrindex patch set and it got uglier and uglier over time
371 2018-05-24 19:54:04	0|echeveria|it means people will willyfully build insane systems.
372 2018-05-24 19:54:08	0|jimpo|haha
373 2018-05-24 19:54:11	0|sipa|echeveria: yes :(
374 2018-05-24 19:54:12	0|jonasschnelli|echeveria: agree
375 2018-05-24 19:54:18	0|sipa|but i fear they'll do so anyway
376 2018-05-24 19:54:21	0|echeveria|rather than sane, scalable things that don't require an ever growing index.
377 2018-05-24 19:54:47	0|jimpo|I tend to agree a better solution is to have a separate indexing service that doesn't do consensus but maintains the full chain state
378 2018-05-24 19:54:48	0|jonasschnelli|The question is, is it faster to index everything or to have Core running with 10k wallets
379 2018-05-24 19:55:00	0|jcorgan|+1 jimpo
380 2018-05-24 19:55:00	0|jimpo|and gets blocks via zmq
381 2018-05-24 19:55:05	0|sipa|decent wallet software shouldn't ever need to scan
382 2018-05-24 19:55:10	0|sipa|except for recovery
383 2018-05-24 19:55:15	0|jcorgan|i want to let bitcoind do the hard stuff (validation)
384 2018-05-24 19:55:33	0|jcorgan|but then i want to easily get at all the validated data
385 2018-05-24 19:55:44	0|wumpus|maybe it never *needs* to, but there are legit situations in which it's useful as a tool
386 2018-05-24 19:55:49	0|sipa|sure!
387 2018-05-24 19:57:17	0|jonasschnelli|jimpo: fork Core, ripout the validation stuff and you have an indexing daemon you can connect to your core node over p2p
388 2018-05-24 19:57:37	0|sipa|having a way to disable validation would also help with that :)
389 2018-05-24 19:57:46	0|sipa|anyway, this is turning into a philosophical discussion
390 2018-05-24 19:57:47	0|wumpus|jonasschnelli: why not use your indexing daemon?
391 2018-05-24 19:58:03	0|wumpus|yes, I think we're through the meeting
392 2018-05-24 19:58:12	0|jonasschnelli|wumpus: maybe. Not sure if we want another p2p library introduced
393 2018-05-24 19:58:27	0|wumpus|jonasschnelli: not into bitcoin core, I mean as external thing
394 2018-05-24 19:58:33	0|lightningbot|Log:            http://www.erisian.com.au/meetbot/bitcoin-core-dev/2018/bitcoin-core-dev.2018-05-24-19.00.log.html
395 2018-05-24 19:58:33	0|lightningbot|Meeting ended Thu May 24 19:59:27 2018 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
396 2018-05-24 19:58:33	0|lightningbot|Minutes:        http://www.erisian.com.au/meetbot/bitcoin-core-dev/2018/bitcoin-core-dev.2018-05-24-19.00.html
397 2018-05-24 19:58:33	0|lightningbot|Minutes (text): http://www.erisian.com.au/meetbot/bitcoin-core-dev/2018/bitcoin-core-dev.2018-05-24-19.00.txt
398 2018-05-24 19:58:33	0|wumpus|#endmeeting
399 2018-05-24 19:58:45	0|jonasschnelli|wumpus: yes. But even there we may want to reuse the primitives...
400 2018-05-24 19:59:12	0|jonasschnelli|or maybe its good to use another library to find things we would not find using the same core code
401 2018-05-24 20:01:06	0|jimpo|yeah, I think diversity at the P2P layer is healthy, using the same validation engine
402 2018-05-24 20:01:17	0|jimpo|and some can have features requiring additional indexes
403 2018-05-24 20:01:52	0|jimpo|speaking of which, what were you saying about the BIP 158 compression ratios, sipa?
404 2018-05-24 20:02:14	0|sipa|jimpo: sec
405 2018-05-24 20:03:36	0|wumpus|jimpo: rust P2P layer? *ducks*
406 2018-05-24 20:03:45	0|jimpo|of course
407 2018-05-24 20:03:50	0|jimpo|:-)
408 2018-05-24 20:05:13	0|sipa|jimpo: so say you have a set of N elements
409 2018-05-24 20:05:24	0|sipa|(after deduplication)
410 2018-05-24 20:05:53	0|sipa|this means you have a list of N entries, each in range 0..2^20*N-1, whose order does not matter
411 2018-05-24 20:06:12	0|sipa|the number of combinations for that is (2^20*N)^N / N!
412 2018-05-24 20:06:45	0|sipa|(approximately; this formula ignores that there may be duplicates among those N, but that chance is low)
413 2018-05-24 20:07:42	0|sipa|this means that information theoretically, you need at least log2((2^20*N)^N / N!) bits in total
414 2018-05-24 20:07:53	0|sipa|otherwise you couldn't express every combination
415 2018-05-24 20:08:59	0|sipa|for N=10000, that number is 214419 bits, or 21.4419 bits per element
416 2018-05-24 20:09:41	0|provoostenator|When I do "bitcoin-cli -config=/the/usual/place -datadir=/some/other/place getblockheight"  it creates a folder /some/other/place/wallets
417 2018-05-24 20:10:00	0|sipa|jimpo: a GCS implementation will use 21.5819 bits on average
418 2018-05-24 20:10:11	0|jimpo|oh, interesting
419 2018-05-24 20:10:12	0|provoostenator|Even if the RPC connection fails.
420 2018-05-24 20:10:39	0|jimpo|(damn it small graphs on Wolfram Alpha)
421 2018-05-24 20:11:25	0|sipa|jimpo: what i don't know is if there's isn't another probabilistic data structure that has a 1/2^P false postive rate which needs less information
422 2018-05-24 20:11:48	0|sipa|but at least any construction that follows from compressing a list of N elements in range 0...2^20*N will at least need 21.4419
423 2018-05-24 20:12:35	0|jimpo|thanks for the explanation
424 2018-05-24 20:13:22	0|sipa|but this is *really* good
425 2018-05-24 20:13:32	0|jimpo|yeah, seems GCS is doing very well
426 2018-05-24 20:13:43	0|sipa|less than 1% overhead above the theoretical minimum
427 2018-05-24 20:14:39	0|jimpo|I've been thinking a bit about tuning the P value. Kind of unfortunate that it's static.
428 2018-05-24 20:14:50	0|sipa|even at just 100 elements, GCS has less than 1% overhead
429 2018-05-24 20:17:27	0|sipa|at a false positive rate of 1/2^10 it's close to 1.5% overhead
430 2018-05-24 20:19:12	0|sipa|gmaxwell and i were looking at whether a better custom entropy coder could do better than GCS, but then he pointed out to me that the limit isn't 20 bits per element, and that your numbers looked suspiciously close to the limit already
431 2018-05-24 20:19:49	0|jimpo|what is a custom entropy coder?
432 2018-05-24 20:20:08	0|sipa|not using golomb-rice coding but something custom that could get us closer to the theoretical limit
433 2018-05-24 20:20:33	0|sipa|it's certainly possible with a range coder
434 2018-05-24 20:20:48	0|sipa|though that would be complex and computationally expensive
435 2018-05-24 20:21:34	0|sipa|and now that it looks like golomb coding is so close to optimal already, it doesn't look worth looking into
436 2018-05-24 20:21:43	0|jimpo|out of curiosity, have you looked at PFOR/FastPFOR for integer sequence compression?
437 2018-05-24 20:21:55	0|sipa|i have never heard about those
438 2018-05-24 20:21:56	0|jimpo|I was experimenting with it yesterday for compression header values
439 2018-05-24 20:22:07	0|jimpo|and it gets pretty amazing results
440 2018-05-24 20:24:40	0|jimpo|if you take all off the versions in a 3,000 header range in a sequence, and do the same for timestamps, bits, and nonces, it compresses all of them together by ~90%
441 2018-05-24 20:24:54	0|jimpo|can even compress sequences of nonces 75% on average
442 2018-05-24 20:26:56	0|echeveria|that seems kind of unlikely.
443 2018-05-24 20:28:16	0|jimpo|yeah, that's what I thought. but I tried decompressing some of the ranges and it seems to work?
444 2018-05-24 20:34:13	0|bitcoin-git|[13bitcoin] 15TronBlack opened pull request #13318: Remove legacy verification (06master...06Remove-legacy-verification) 02https://github.com/bitcoin/bitcoin/pull/13318
445 2018-05-24 20:34:35	0|echeveria|lol
446 2018-05-24 20:35:03	0|echeveria|what on earth happened there. 30 second old pull request with 2 month old comments.
447 2018-05-24 20:35:23	0|harding|echeveria: the comments are on the specific commits, not the PR itself.
448 2018-05-24 20:35:43	0|echeveria|ah, github presents that confusingly.
449 2018-05-24 20:39:00	0|bitcoin-git|[13bitcoin] 15TronBlack closed pull request #13318: Remove legacy verification (06master...06Remove-legacy-verification) 02https://github.com/bitcoin/bitcoin/pull/13318
450 2018-05-24 20:47:16	0|bitcoin-git|[13bitcoin] 15MarcoFalke opened pull request #13319: gui: Backport bech32 checkbox (060.16...06Mf1805-016ForBackportGui) 02https://github.com/bitcoin/bitcoin/pull/13319
451 2018-05-24 21:24:45	0|phantomcircuit|i've got a reeeeally old wallet
452 2018-05-24 21:25:01	0|phantomcircuit|the result of getbalance doesn't match the sum of all amounts in listtransactions "*" 1000000
453 2018-05-24 21:25:09	0|phantomcircuit|seems like a bug
454 2018-05-24 21:26:09	0|sipa|any unconfirmed transactions/
455 2018-05-24 21:27:34	0|phantomcircuit|sipa, nope
456 2018-05-24 21:28:03	0|phantomcircuit|possibly it's cause it's all the dust that someone was flooding a while ago?
457 2018-05-24 21:29:30	0|sipa|you're calling getbalance, not getbalance "", right/
458 2018-05-24 21:30:54	0|sipa|jimpo: it seems PFOR assumes integers are sorted?
459 2018-05-24 21:31:48	0|sipa|oh, or just the benchmark
460 2018-05-24 21:34:19	0|phantomcircuit|sipa, im running git master so the accounts stuff has been removed
461 2018-05-24 21:34:25	0|phantomcircuit|and im using the multi wallet support
462 2018-05-24 21:34:32	0|phantomcircuit|i guess i need to test this on stable
463 2018-05-24 21:40:20	0|phantomcircuit|sipa, hmm listunspent matches getbalance of course
464 2018-05-24 21:50:05	0|phantomcircuit|sipa, that wasn't it
465 2018-05-24 21:51:47	0|phantomcircuit|the sum of the amounts field in listtransactions nominally should match getbalance i believe
466 2018-05-24 21:52:48	0|gmaxwell|coinjoins fuxor that no?
467 2018-05-24 21:54:32	0|phantomcircuit|gmaxwell, shouldn't i dont think
468 2018-05-24 21:55:25	0|gmaxwell|when you 'sum amounts' how are you handling fees?
469 2018-05-24 21:56:00	0|phantomcircuit|hmm maybe it's that let me check
470 2018-05-24 21:57:49	0|phantomcircuit|derp yeah that was it
471 2018-05-24 21:57:57	0|gmaxwell|Next customer!
472 2018-05-24 21:58:17	0|sipa|queue = rotl(queue, 1)