1 2017-07-27 06:25:29	0|gmaxwell|::sigh:: https://www.reddit.com/r/Bitcoin/comments/6pu13q/whats_up_with_bitcoin_core/  so github seems to be distributing tar files for our tags that have nothing to do with our distribution, and it sounds like netbsd's stuff was using them and is not happy that the checksums change (presumably because file order or compressor changed)
  2 2017-07-27 06:33:26	0|sipa|they've complained about that before
  3 2017-07-27 06:33:34	0|sipa|there was an issue about that years ago
  4 2017-07-27 06:34:13	0|jonasschnelli|Can we disable tar download via github? I guess no...
  5 2017-07-27 06:36:28	0|sipa|#2476
  6 2017-07-27 06:36:30	0|gribble|https://github.com/bitcoin/bitcoin/issues/2476 | no separate source code tarball download available · Issue #2476 · bitcoin/bitcoin · GitHub
  7 2017-07-27 06:39:35	0|wumpus|we've been creating stable tarballs for years now, for distributions and such, what's wrong with them?
  8 2017-07-27 06:40:26	0|wumpus|more than two years, seeing that comment
  9 2017-07-27 06:43:03	0|gmaxwell|yea, just read. For some reason it seems that after nagging the crap out of us, they didn't switch to using the tarballs they asked us to provide.
 10 2017-07-27 06:43:50	0|sipa|almost 4 years ago
 11 2017-07-27 06:44:45	0|wumpus|well if there's something wrong with the tarballs from their pov we'd like to hear, but wtf post it to reddit
 12 2017-07-27 06:45:33	0|gmaxwell|well the person posting is probably an enduser, I guess
 13 2017-07-27 07:21:14	0|luke-jr|gmaxwell: it's because git decided to change git-archive output in some recentish version
 14 2017-07-27 07:21:31	0|luke-jr|they made the substituted hash abbreviation longer
 15 2017-07-27 07:21:53	0|luke-jr|wumpus: the stable tarballs are missing a bunch of stuff still :/
 16 2017-07-27 07:22:16	0|sipa|luke-jr: like?
 17 2017-07-27 07:22:39	0|gmaxwell|how could they be missing stuff ... they're just make dists
 18 2017-07-27 07:23:07	0|luke-jr|sipa: https://github.com/bitcoin/bitcoin/issues/6753
 19 2017-07-27 07:23:19	0|luke-jr|gmaxwell: `make dist` is missing the files
 20 2017-07-27 07:23:54	0|gmaxwell|Only in from-git/: .gitattributes
 21 2017-07-27 07:23:55	0|gmaxwell|Only in from-git/: .gitignore
 22 2017-07-27 07:23:59	0|gmaxwell|don't belong in the distribution
 23 2017-07-27 07:24:09	0|sipa|how does git checkdist succeed if things are missing?
 24 2017-07-27 07:24:14	0|sipa|or distcheck
 25 2017-07-27 07:24:22	0|gmaxwell|but the readmes and stuff are not good.
 26 2017-07-27 07:24:34	0|sipa|ah, agree
 27 2017-07-27 07:25:21	0|gmaxwell|should be trivial to add DISTFILES to the autotools stuff...
 28 2017-07-27 07:26:05	0|luke-jr|IIRC there's a bunch of contrib now missing
 29 2017-07-27 07:26:09	0|luke-jr|like init scripts and such
 30 2017-07-27 07:26:19	0|luke-jr|haven't done a full check recently
 31 2017-07-27 07:26:56	0|gmaxwell|why didn't you PR a fix
 32 2017-07-27 07:34:39	0|luke-jr|I don't know a good solution
 33 2017-07-27 07:39:38	0|gmaxwell|luke-jr: you list the files in EXTRA_DIST in makefile.am
 34 2017-07-27 07:39:56	0|gmaxwell|(though don't add the git specific stuff)
 35 2017-07-27 07:41:47	0|luke-jr|it's just going to get outdated again :/
 36 2017-07-27 07:42:01	0|gmaxwell|luke-jr: thats what review is for.
 37 2017-07-27 07:42:06	0|luke-jr|isn't there some way to get git to build the list?
 38 2017-07-27 07:42:26	0|gmaxwell|among other things, we don't want every file in git to end up in the dist.
 39 2017-07-27 07:43:20	0|luke-jr|the .git* files maybe not, but everything else we do
 40 2017-07-27 07:53:40	0|gmaxwell|luke-jr: in some other projects I've worked on, there were some machine generated source files that weren't checked in but were just generated and included as extra_dist, and some of the build scripts for those didn't get extra_disted.
 41 2017-07-27 07:55:16	0|bitcoin-git|[13bitcoin] 15vadimVoloshanov opened pull request #10940: Add files via upload (06master...06master) 02https://github.com/bitcoin/bitcoin/pull/10940
 42 2017-07-27 07:55:54	0|bitcoin-git|[13bitcoin] 15fanquake closed pull request #10940: Add files via upload (06master...06master) 02https://github.com/bitcoin/bitcoin/pull/10940
 43 2017-07-27 07:56:14	0|luke-jr|gmaxwell: IMO the latter would be a bug.. source should be buildable even if modified
 44 2017-07-27 07:56:31	0|luke-jr|looking at an updated list, the big things are subtrees..
 45 2017-07-27 07:57:00	0|gmaxwell|well subtrees should be handled in their sub makefiles.
 46 2017-07-27 07:57:07	0|luke-jr|https://0bin.net/paste/-R85vsle0Z7ygcL8#Hb9C8u3Y95x1d8iXMm3mC8oPrkZyrKO4tYoGT6UWZsC
 47 2017-07-27 07:58:43	0|gmaxwell|luke-jr: does our dist tarball even build?!
 48 2017-07-27 08:06:13	0|cfields|travis builds from tarball
 49 2017-07-27 08:06:23	0|cfields|(or something close to it)
 50 2017-07-27 08:07:05	0|luke-jr|talking with cfields, I'm thinking list the files individually, and add a script for Travis to complain if we add a new file that isn't either included or explicitly excluded from dist
 51 2017-07-27 08:08:07	0|cfields|that seems like more work to me, i'd rather just add "check for excluded files" as a step in the release process
 52 2017-07-27 08:08:33	0|luke-jr|cfields: I'll do it
 53 2017-07-27 08:12:35	0|gmaxwell|cfields: human steps get missed. (see also the last N RCs where versions didn't get bumped. :) )
 54 2017-07-27 08:13:15	0|cfields|heh
 55 2017-07-27 08:37:38	0|wumpus|luke-jr: which files are missing that would be required for a distro?
 56 2017-07-27 08:38:07	0|wumpus|and yes, the dist tarball builds
 57 2017-07-27 08:38:26	0|wumpus|it works fine, there have been no complaints about it at all, except from luke-jr apparently
 58 2017-07-27 08:39:43	0|wumpus|stuff not included is not part of 'make install', so generally not relevant to distros
 59 2017-07-27 08:39:49	0|wumpus|(such as developer tools)
 60 2017-07-27 08:42:40	0|gmaxwell|well there are files that are missing which should be included, including internal documentation, and contrib stuff like linearize.  (docs including things like INSTALL.md and README.md that tell you things about building the software)
 61 2017-07-27 08:42:57	0|wumpus|if that's so, it should probably be part of 'make install'
 62 2017-07-27 08:43:12	0|wumpus|if it's not installed somewhere, how aer people normally supposed to use it?
 63 2017-07-27 08:43:17	0|gmaxwell|normally build instructions don't end up in make install though.. right.
 64 2017-07-27 08:43:25	0|gmaxwell|er right?
 65 2017-07-27 08:43:39	0|wumpus|that's maybe the only exception
 66 2017-07-27 08:44:05	0|gmaxwell|development script would be another.
 67 2017-07-27 08:44:15	0|wumpus|but linearize? I'm not sure that's generally useful to end-users, even if it should be, it's not in a fool proof state suited for that
 68 2017-07-27 08:44:26	0|gmaxwell|yes, linearize should get installed.
 69 2017-07-27 08:44:38	0|wumpus|then it should be maintained and tested too
 70 2017-07-27 08:45:14	0|gmaxwell|ya, and if it's not fool proof enough, thats a reason to not install it right now I guess.
 71 2017-07-27 08:45:36	0|wumpus|most developer scripts make no sense if you don't have a git checkout
 72 2017-07-27 08:46:07	0|wumpus|a lot even rely on it explicitly (to find the root for pulling transifex translations, for example)
 73 2017-07-27 08:46:45	0|wumpus|in any case I think being somewhat selective about what to include in the dist is fine
 74 2017-07-27 08:47:11	0|wumpus|though I agree just a git tarball would make things a lot easier
 75 2017-07-27 08:48:01	0|gmaxwell|verify-binaries, are quite useful and very much part of our source. I don't really think we gain anything by leaving something out even if it might not be that useful,  except for pure git things like gitignore files.
 76 2017-07-27 08:48:12	0|wumpus|well then
 77 2017-07-27 08:48:16	0|wumpus|thinking of it, why don't we just skip the make dist and include everything
 78 2017-07-27 08:48:19	0|wumpus|zip up the git repository
 79 2017-07-27 08:48:30	0|wumpus|the tarball is the smallest file
 80 2017-07-27 08:48:40	0|gmaxwell|the make dist eliminates the endusers need for a full and compatible autotools setup.
 81 2017-07-27 08:48:55	0|wumpus|it's not like we have to worry about a few extra source files in the downlaod or storage
 82 2017-07-27 08:49:11	0|wumpus|wouldn't running autogen.sh before zipping uphave the same effect?
 83 2017-07-27 08:49:30	0|wumpus|(maybe with some special options)
 84 2017-07-27 08:49:41	0|gmaxwell|Yes +/- some temp files I think autogen leaves behind.
 85 2017-07-27 08:49:52	0|wumpus|we efiniltely don't need the selective part of 'make dist' then
 86 2017-07-27 08:50:02	0|wumpus|yes, well that's not too bad
 87 2017-07-27 08:50:53	0|gmaxwell|I don't think I have know any counterarguments to that point.
 88 2017-07-27 08:51:07	0|gmaxwell|maybe someone who knows autotools better would be aware of something we're missing.
 89 2017-07-27 08:51:32	0|wumpus|it would be the minimum energy expenditure to finally getting this discussion out of the way at least
 90 2017-07-27 08:54:07	0|wumpus|I am kind of surprised verify-binaries isn't included
 91 2017-07-27 08:55:14	0|wumpus|cfields: any reason not to do the above? e.g. make the tarball simply a repo dump + build system autogenerated files
 92 2017-07-27 08:55:22	0|gmaxwell|looking at the list, I think its just accidental. stuff got added without anyone throwing it into extra dist because it's easy to forget. seems there are things in libsecp256k1 where we'd do that too.
 93 2017-07-27 08:55:50	0|cfields|wumpus: i suppose that's fine
 94 2017-07-27 08:56:15	0|wumpus|otherwise we're bound to forget things and have this "the tarball is useless!" discussion again and agian
 95 2017-07-27 08:56:50	0|cfields|the 'make dist' stuff is kinda outdated. It's intended to be an explicit list of files so that the release tarball doesn't get polluted by the builder's working dir...
 96 2017-07-27 08:57:00	0|cfields|but with our build process, it's really kinda moot
 97 2017-07-27 08:57:21	0|wumpus|which could also be avoided by taking the snapshot before starting building
 98 2017-07-27 08:57:25	0|wumpus|yeah
 99 2017-07-27 08:58:00	0|cfields|yea, tar it up after bootstrap but before anything else
100 2017-07-27 08:59:22	0|wumpus|I understand the reasoning behind being selective about what to include, but we don't have the people nor attention to carefully decide that every time, so erring on the side of 'include everything' makes sense for this project. For others (like secp256k1) which are more focused it's probably different.
101 2017-07-27 09:00:12	0|luke-jr|fixing make dist is proving to be a real pain (hi depends/ … ) so I like the idea of not using it
102 2017-07-27 09:00:42	0|wumpus|depends/ is another joy in that regard, yes :)
103 2017-07-27 09:00:48	0|cfields|agreed
104 2017-07-27 09:38:30	0|wumpus|sigh, no, I still don't think it's acceptable to mention bitcoin-cli specifically in RPC error messages, it should be aimed at people using the JSON-RPC API, not people using bitcoin-cli only
105 2017-07-27 09:38:46	0|wumpus|there's no point in talking about a specific client in server error messages
106 2017-07-27 09:39:05	0|wumpus|that's just wrong, and won't help someone struggling with this interface in say python-bitcoinrpc
107 2017-07-27 09:42:59	0|wumpus|why is it so much to ask that documentation for bitcoin-cli arguments belongs in bitcoin-cli?
108 2017-07-27 09:46:38	0|wumpus|it doesn't belong in two places, no it just belongs in bitcoin-cli
109 2017-07-27 10:11:29	0|wumpus|gah, coredev.tech is september, scaling bitcoin in november, both in SF area, would have been better to plan it closer together :/
110 2017-07-27 10:22:52	0|wumpus|phew, at least 'make dist' includes the function test suite
111 2017-07-27 10:24:55	0|sipa|wumpus: just stay :)
112 2017-07-27 10:25:50	0|wumpus|sipa: if it was a two weeks in between instead of two months I would
113 2017-07-27 10:26:17	0|sipa|yeah :)
114 2017-07-27 10:28:42	0|wumpus|indeed, the ESTA allows staying 90 days, so theoretically it'd be possible, but just too much hassle
115 2017-07-27 10:50:38	0|jonasschnelli|wumpus: Yeah. The SB guy contacted me after I have booked the room.
116 2017-07-27 10:50:48	0|jonasschnelli|He asked to move it closer... :)
117 2017-07-27 10:50:56	0|jonasschnelli|But was already too late.
118 2017-07-27 10:51:08	0|jonasschnelli|Too bad he did not contact me before announcing the date
119 2017-07-27 10:51:17	0|jonasschnelli|I bet the knew weeks ago
120 2017-07-27 11:11:14	0|bitcoin-git|[13bitcoin] 15promag opened pull request #10941: Add blocknotify functional test (06master...062017-07-blocknotify-functional-test) 02https://github.com/bitcoin/bitcoin/pull/10941
121 2017-07-27 11:14:56	0|promag|#10941 makes sense right? should add tests for -walletnotify too right?
122 2017-07-27 11:14:57	0|gribble|https://github.com/bitcoin/bitcoin/issues/10941 | Add blocknotify functional test by promag · Pull Request #10941 · bitcoin/bitcoin · GitHub
123 2017-07-27 11:15:16	0|wumpus|sure!
124 2017-07-27 11:15:35	0|promag|will do
125 2017-07-27 11:15:39	0|wumpus|ideally there should be tests for everything
126 2017-07-27 11:16:04	0|promag|right, makes reviewing easy
127 2017-07-27 11:16:32	0|promag|ideally changing code should have some impact in tests
128 2017-07-27 11:17:03	0|promag|most PR's don't change tests
129 2017-07-27 11:18:14	0|wumpus|right, apart from pure refactors and optimizations
130 2017-07-27 11:19:39	0|promag|even those
131 2017-07-27 11:19:58	0|bitcoin-git|13bitcoin/06master 14f228b8e 15Marko Bencun: remove some unused functions...
132 2017-07-27 11:19:58	0|bitcoin-git|[13bitcoin] 15laanwj pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/5c8eb7916de7...8a99fe053a5d
133 2017-07-27 11:19:59	0|bitcoin-git|13bitcoin/06master 148a99fe0 15Wladimir J. van der Laan: Merge #10501: remove some unused functions...
134 2017-07-27 11:20:04	0|wumpus|if the behavior stays the same, the tests can also stay the same
135 2017-07-27 11:20:18	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #10501: remove some unused functions (06master...06unusedfuncs) 02https://github.com/bitcoin/bitcoin/pull/10501
136 2017-07-27 11:21:09	0|promag|refactors can impact tests, for instance, a test uses a lib which has new signatures..
137 2017-07-27 11:21:24	0|promag|in terms of functional tests, yes you're right
138 2017-07-27 11:23:34	0|wumpus|you could have tests for e.g. optimizations, that check benchmarks for inner functions such as sha256 etc, but it'd require a stable dedicated platform, certainly not on our current infrastructure
139 2017-07-27 11:25:30	0|wumpus|also something that automatically syncs from a local node and keeps timings, daily w/ most recent master could be useful, but it'd certailny need a deterministic environment or there'd be too much noise
140 2017-07-27 11:31:40	0|promag|right
141 2017-07-27 11:31:56	0|promag|I'm for sure interested in improve that
142 2017-07-27 11:32:10	0|promag|it's a great way to get deeper in the project
143 2017-07-27 11:32:27	0|wumpus|it'd need infrastructure, not so much coding work
144 2017-07-27 11:32:55	0|wumpus|with an available server it'd be quite easy to set it up
145 2017-07-27 11:38:08	0|jonasschnelli|promag: I could provide you the necessary infrastructure (server hosted remotly, ideally for test/benchmarks)
146 2017-07-27 12:00:59	0|bitcoin-git|13bitcoin/06master 1472f0060 15Dag Robole: Replace traditional for with ranged for in primitives
147 2017-07-27 12:00:59	0|bitcoin-git|[13bitcoin] 15laanwj pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/8a99fe053a5d...ba1bbb049b8f
148 2017-07-27 12:01:00	0|bitcoin-git|13bitcoin/06master 14ba1bbb0 15Wladimir J. van der Laan: Merge #10892: Replace traditional for with ranged for in block and transaction primitives...
149 2017-07-27 12:01:31	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #10892: Replace traditional for with ranged for in block and transaction primitives (06master...0620170721-rangedfor-primitives) 02https://github.com/bitcoin/bitcoin/pull/10892
150 2017-07-27 15:58:11	0|wumpus|re: installing linearize, I think the main thing that needs to be done to make it more user friendly is to make it one command, instead of the split between generating the blocks list and writing them out. These could be subcommands/options for when doing them separately.
151 2017-07-27 15:59:52	0|wumpus|at the least cookie authentication was already implemented, that's nice, used to be that it required always manually setting up the authentication
152 2017-07-27 16:59:00	0|bitcoin-git|[13bitcoin] 15laanwj pushed 3 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/ba1bbb049b8f...0b11a0784875
153 2017-07-27 16:59:01	0|bitcoin-git|13bitcoin/06master 14df389bc 15Russell Yanofsky: Change wallet method disabled error text...
154 2017-07-27 16:59:01	0|bitcoin-git|13bitcoin/06master 14e526b3d 15Russell Yanofsky: Fix misleading "Method not found" multiwallet errors...
155 2017-07-27 16:59:02	0|bitcoin-git|13bitcoin/06master 140b11a07 15Wladimir J. van der Laan: Merge #10931: Fix misleading "Method not found" multiwallet errors...
156 2017-07-27 16:59:36	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #10931: Fix misleading "Method not found" multiwallet errors (06master...06pr/multierr) 02https://github.com/bitcoin/bitcoin/pull/10931
157 2017-07-27 18:31:49	0|morcos|Can we merge #10758 before I fill up all my disks with genesis blocks?
158 2017-07-27 18:31:51	0|gribble|https://github.com/bitcoin/bitcoin/issues/10758 | Fix some chainstate-init-order bugs. by TheBlueMatt · Pull Request #10758 · bitcoin/bitcoin · GitHub
159 2017-07-27 18:37:38	0|wumpus|morcos: and fix the error messages later? fine with me
160 2017-07-27 18:54:19	0|BlueMatt|wumpus: morcos lol, give me a few minutes, I'm testing the on-the-fly compaction stuff
161 2017-07-27 18:54:27	0|BlueMatt|(which, on my test system, reliably makes it worse....)
162 2017-07-27 18:55:42	0|gmaxwell|https://en.wikipedia.org/wiki/Repeated_sequence_(DNA)
163 2017-07-27 18:55:56	0|BlueMatt|?
164 2017-07-27 18:57:14	0|wumpus|"In many organisms, a significant fraction of the genomic DNA is highly repetitive, with over two-thirds of the sequence consisting of repetitive elements in human." well at least that means it could be compressed efficiently :p
165 2017-07-27 18:58:44	0|BlueMatt|???
166 2017-07-27 18:59:17	0|wumpus|probably responding to the genesis block that gets added again and again
167 2017-07-27 18:59:36	0|gmaxwell|yes
168 2017-07-27 18:59:37	0|BlueMatt|ah
169 2017-07-27 19:00:28	0|lightningbot|Meeting started Thu Jul 27 19:00:27 2017 UTC.  The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
170 2017-07-27 19:00:28	0|lightningbot|Useful Commands: #action #agreed #help #info #idea #link #topic.
171 2017-07-27 19:00:28	0|wumpus|#startmeeting
172 2017-07-27 19:00:36	0|jonasschnelli|hi
173 2017-07-27 19:00:38	0|gmaxwell|#bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr btcdrak sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier
174 2017-07-27 19:00:44	0|gmaxwell|achow101:
175 2017-07-27 19:00:49	0|kanzure|hi.
176 2017-07-27 19:00:51	0|jonasschnelli|+promag
177 2017-07-27 19:00:52	0|achow101|hi
178 2017-07-27 19:00:59	0|morcos|here for 30
179 2017-07-27 19:01:01	0|Murch|hi
180 2017-07-27 19:01:04	0|BlueMatt|oh, thats today?
181 2017-07-27 19:01:06	0|jtimon|hi
182 2017-07-27 19:01:08	0|wumpus|I think it's most useful to discuss the still-open issues for 0.15 today
183 2017-07-27 19:01:14	0|wumpus|https://github.com/bitcoin/bitcoin/milestones/0.15.0
184 2017-07-27 19:01:45	0|jtimon|yep, sounds like a good first topic (that will probably take most of the meeting)
185 2017-07-27 19:02:04	0|wumpus|FWIW rc1 is planned for 2017-08-06
186 2017-07-27 19:02:14	0|wumpus|ok, I'll just follow the list
187 2017-07-27 19:02:45	0|wumpus|#topic test_bitcoin fails valgrind Tests (#9278)
188 2017-07-27 19:02:46	0|gribble|https://github.com/bitcoin/bitcoin/issues/9278 | test_bitcoin fails valgrind · Issue #9278 · bitcoin/bitcoin · GitHub
189 2017-07-27 19:02:51	0|wumpus|this is the oldest issue in the list
190 2017-07-27 19:03:06	0|achow101|I checked last week that this was still a problem
191 2017-07-27 19:03:24	0|wumpus|ok, so this still needs to be fixed?
192 2017-07-27 19:03:44	0|gmaxwell|cfields: are you working on that? if you don't have time, someone else can pick it up
193 2017-07-27 19:03:45	0|BlueMatt|its assigned cfields....
194 2017-07-27 19:03:55	0|achow101|it should probably be fixed, but I don't know if it is a blocker or how urgent a fix would be required
195 2017-07-27 19:04:18	0|gmaxwell|Looks like it should be a trivial fix (famous last words)
196 2017-07-27 19:04:18	0|morcos|He's only had 8 months cut him a break
197 2017-07-27 19:04:20	0|achow101|it was already pushed from 0.14 to 0.15, so I'm guessing it isn't all that urgent
198 2017-07-27 19:04:24	0|wumpus|I don't think uninitalized state in a test is important enough to block 0.15, though it could result in intermittent faliures which are always annoying
199 2017-07-27 19:04:35	0|jonasschnelli|Yes. Moving to 0.16 would be possible.
200 2017-07-27 19:04:44	0|wumpus|though it might hide real issues
201 2017-07-27 19:04:56	0|wumpus|(if the tests are not valgrind clean)
202 2017-07-27 19:05:02	0|gmaxwell|I'm guessing cfields and sipa are not in the meeting today due to timezones.
203 2017-07-27 19:05:20	0|gmaxwell|I'll follow up and make sure it gets fixed (either extract something cfields hasn't submitted yet or fix it myself)
204 2017-07-27 19:05:42	0|wumpus|ok, thanks
205 2017-07-27 19:05:48	0|gmaxwell|(or realize it isn't actually trivial and ask for a punt to the next version)
206 2017-07-27 19:05:55	0|wumpus|#topic sendtoaddress subtractfeefromamount=true does not respect paytxfee values (#10034)
207 2017-07-27 19:05:56	0|gribble|https://github.com/bitcoin/bitcoin/issues/10034 | sendtoaddress subtractfeefromamount=true does not respect paytxfee values · Issue #10034 · bitcoin/bitcoin · GitHub
208 2017-07-27 19:06:14	0|morcos|Are we going to do the wiki thing again for release notes?  If so can we put a link to it on the top of #9889.  I kept having trouble finding that link last release
209 2017-07-27 19:06:16	0|gribble|https://github.com/bitcoin/bitcoin/issues/9889 | TODO for release notes 0.15.0 · Issue #9889 · bitcoin/bitcoin · GitHub
210 2017-07-27 19:06:21	0|wumpus|I'm not sure about this one, don't think it has been looked at much
211 2017-07-27 19:06:41	0|wumpus|morcos: yes, good idea
212 2017-07-27 19:06:44	0|gmaxwell|Lets decide if we're doing the wiki or not, last time the wiki merge killed release notes written directly. :)
213 2017-07-27 19:07:00	0|wumpus|but it helped a lot with actually getting release notes written
214 2017-07-27 19:07:01	0|gmaxwell|I've held off on writing release notes due to lack of clarity on this point.
215 2017-07-27 19:07:14	0|wumpus|and getting the style consistent etc
216 2017-07-27 19:07:15	0|gmaxwell|I'm fine with doing it, I just need to know where to put them.
217 2017-07-27 19:07:26	0|wumpus|well right now you simply have to create a PR
218 2017-07-27 19:07:31	0|gmaxwell|I don't want to write any outside of the wiki if we're using the wiki.
219 2017-07-27 19:07:35	0|wumpus|as soon as the wiki is open, do it there
220 2017-07-27 19:07:37	0|morcos|I recommend we do the wiki, but we need a good code of conduct, where people should not make potentially controversial changes without flagging them somehow
221 2017-07-27 19:07:47	0|gmaxwell|wumpus: I did that last time and they got stomped by the wiki merge. :)
222 2017-07-27 19:08:02	0|morcos|Let's do the wiki.
223 2017-07-27 19:08:08	0|jonasschnelli|+1
224 2017-07-27 19:08:10	0|wumpus|gmaxwell: I know, we should avoid that, but that's no reason to completely abandon a good idea
225 2017-07-27 19:08:25	0|gmaxwell|So on this feefrom amount, that sounds like the known overpayment bug with no change that we fixed though I don't recall if we fixed it in the feefrom amount case.
226 2017-07-27 19:08:33	0|gmaxwell|wumpus: I'm all for the wiki.
227 2017-07-27 19:08:46	0|achow101|we should do the wiki and probably open it now
228 2017-07-27 19:08:55	0|gmaxwell|Just someone point me to it and I'll write a bunch of release notes.
229 2017-07-27 19:09:01	0|wumpus|gmaxwell: I think we should put the commit id that the wiki 'forked' from on that page, so that it can be compared if there haven't been changes since when mering it back
230 2017-07-27 19:09:12	0|achow101|+1
231 2017-07-27 19:09:14	0|gmaxwell|that would help too.
232 2017-07-27 19:09:24	0|wumpus|(this was missing last time, so I had no reference point)
233 2017-07-27 19:09:50	0|wumpus|re: 10034, I'm not sure why that's labelled 0.15
234 2017-07-27 19:09:57	0|jtimon|wumpus: ack on commit id on the wiki
235 2017-07-27 19:10:04	0|kanzure|wumpus: if you mean github wiki, those are actually git repositories, so they do literally have forks/branches (even if github tosses them).
236 2017-07-27 19:10:05	0|achow101|is #10034 actually a problem or is it just an instance of an already known issue?
237 2017-07-27 19:10:06	0|gribble|https://github.com/bitcoin/bitcoin/issues/10034 | sendtoaddress subtractfeefromamount=true does not respect paytxfee values · Issue #10034 · bitcoin/bitcoin · GitHub
238 2017-07-27 19:10:26	0|morcos|I'll take a look at 10034, but there have been multiple changes to this code for 0.15, so not clear that even if there was a bug it would still be there
239 2017-07-27 19:10:28	0|wumpus|kanzure: true, you can fork and edit locally, but I mean the bitcoin core master revision
240 2017-07-27 19:10:41	0|wumpus|morcos: thanks
241 2017-07-27 19:11:11	0|jonasschnelli|Also, if 10034 is still a bug, it does not deserve the "medium" tag IMO
242 2017-07-27 19:11:23	0|gmaxwell|achow101: I think 10034 is the known thing that we fixed at least in the non-fee-from-value case.
243 2017-07-27 19:11:47	0|gmaxwell|jonasschnelli: Do you believe it deserves high.. overpaying fees should be high. IMO.
244 2017-07-27 19:12:16	0|morcos|gmaxwell: it seems to have some determinism in the description that bears a closer look.. but i'm not expecting to find anything
245 2017-07-27 19:12:17	0|wumpus|the dirty secret is that we don't really use the priority tags for issues/prs for anything :)
246 2017-07-27 19:12:35	0|jonasschnelli|heh
247 2017-07-27 19:12:36	0|gmaxwell|yea...
248 2017-07-27 19:12:50	0|gmaxwell|morcos: thanks for looking at it.
249 2017-07-27 19:13:40	0|jtimon|priority medium seems specially useless, if it's not tagged low or high, then it's medium, no?
250 2017-07-27 19:13:45	0|wumpus|#topic Force on-the-fly compaction during pertxout upgrade  UTXO Db and Indexes (#10526)
251 2017-07-27 19:13:46	0|gribble|https://github.com/bitcoin/bitcoin/issues/10526 | Force on-the-fly compaction during pertxout upgrade by sipa · Pull Request #10526 · bitcoin/bitcoin · GitHub
252 2017-07-27 19:13:58	0|gmaxwell|jtimon: untagged is a possible state (not evaluated yet).
253 2017-07-27 19:14:04	0|wumpus|jtimon: I think in practice only priority high could be useful, if someone paid attention to it
254 2017-07-27 19:14:10	0|gmaxwell|wumpus: as morocos noted on the PR, it's not shrinking on its own.
255 2017-07-27 19:14:29	0|gmaxwell|5.8G    .
256 2017-07-27 19:14:29	0|gmaxwell|$ du -sh .
257 2017-07-27 19:14:39	0|wumpus|jtimon: then again, I think a year ago I checked "high priority" issues and some were >5 years old
258 2017-07-27 19:14:41	0|jtimon|gmaxwell: right, but then you have to tag them all with something about priority once you evaluate them
259 2017-07-27 19:14:48	0|BlueMatt|see my most recent comment on it: that pr makes upgrade time a bunch faster for me, but also doesnt actually save disk space directly (but its done later upon next-db-reopen cause i guess leveldb realizes it has a bunch of tables it doesnt need)
260 2017-07-27 19:15:03	0|BlueMatt|gmaxwell: I believe it may clean it up over time, but not quickly
261 2017-07-27 19:15:10	0|gmaxwell|So I think we really need to do that... but IIRC we had a report from wumpus saying that it didn't work for him.
262 2017-07-27 19:15:24	0|BlueMatt|anyway, yes, we should take 10526 for 15 imo, morcos had one code issue on it, but it doesnt appear to change behavior for me
263 2017-07-27 19:15:24	0|gmaxwell|BlueMatt: I've had two weeks and many startup and shutdowns on this node... so ... really not quickly.
264 2017-07-27 19:15:38	0|wumpus|gmaxwell: it doesn't work reliably for me, no
265 2017-07-27 19:16:13	0|gmaxwell|wumpus: could it be what bluematt says.. it works but doesn't take effect until after another restart?
266 2017-07-27 19:16:18	0|BlueMatt|gmaxwell: see my later comment, i meant that it'll do it when tables fill, ie when you connect more blocks and so
267 2017-07-27 19:16:27	0|wumpus|gmaxwell: sometimes after running upgrade on that test set it it is 2.1G sometimes 4.3G, apparently random, after another restart it's always down
268 2017-07-27 19:16:31	0|wumpus|gmaxwell: yes, that could well be
269 2017-07-27 19:16:55	0|gmaxwell|okay, well thats still better than behavior in master.. which I've tried on several hosts and it results in a persistant 5.8 gb state.
270 2017-07-27 19:17:01	0|morcos|well if it consistently shrinks at the very latest after another restart, that's a clear improvement
271 2017-07-27 19:17:04	0|wumpus|well the, we should merge it, better is better
272 2017-07-27 19:17:19	0|BlueMatt|if we want we can probably simulate restart by closing and re-opening db :p
273 2017-07-27 19:17:23	0|morcos|needs a bug fix though, commented on pr
274 2017-07-27 19:18:02	0|wumpus|yes
275 2017-07-27 19:18:08	0|achow101|I hope it does actually work. my node running master has a chainstate of 8.2 GB right now (it was upgraded from a previous 0.14 master)
276 2017-07-27 19:18:09	0|gmaxwell|BlueMatt: yes though that doesn't get rid of the peak usage, we'll need to release note that upgrading will require an extra 3GB free disk space.
277 2017-07-27 19:18:13	0|achow101|I can test it too
278 2017-07-27 19:18:28	0|gmaxwell|uh.. 5GB disk space.
279 2017-07-27 19:18:28	0|wumpus|achow101: yes, please test
280 2017-07-27 19:18:39	0|gmaxwell|achow101: well it doesn't fix already bloated ones.
281 2017-07-27 19:18:42	0|BlueMatt|gmaxwell: yes, please note that on the release notes todo
282 2017-07-27 19:18:57	0|morcos|how'd you get 8.2, that's yuuuge
283 2017-07-27 19:19:03	0|achow101|i don't know
284 2017-07-27 19:19:09	0|gmaxwell|morcos: don't make fun of how I pronounce huge. :(
285 2017-07-27 19:19:47	0|BlueMatt|heh
286 2017-07-27 19:20:04	0|gmaxwell|Maybe we should sneak in a hidden rpc to compact the whole database. (I think there is just a leveldb call to do that)
287 2017-07-27 19:20:13	0|wumpus|8.2 is definitely abnormal, if it's from a very old version maybe something left behind
288 2017-07-27 19:20:37	0|BlueMatt|gmaxwell: seems reasonable to me
289 2017-07-27 19:20:41	0|achow101|wumpus: its the same datadir from a 0.10 master that has been periodically upgraded over the years
290 2017-07-27 19:20:54	0|achow101|to different random master builds
291 2017-07-27 19:21:07	0|wumpus|(leveldb changed some things also, over time, e.g. the extension for database files at some point even)
292 2017-07-27 19:21:12	0|gmaxwell|achow101: it would be interesting to see if leveldb compaction shrinks it, or if it actually has zombie records in it due to some kind of corruption.
293 2017-07-27 19:21:22	0|wumpus|achow101: can you send me a pastebinned 'ls'?
294 2017-07-27 19:21:40	0|wumpus|zombie records, or zombie files even
295 2017-07-27 19:22:03	0|gmaxwell|achow101: you should also try gettxoutsetinfo to compare its hash with another non-bloaty host.
296 2017-07-27 19:22:21	0|wumpus|adding a hidden RPC to compact databases is fine with me
297 2017-07-27 19:22:26	0|gmaxwell|I think we know what to do to go forward on this one then.
298 2017-07-27 19:22:31	0|wumpus|ye
299 2017-07-27 19:22:35	0|wumpus|#topic Fix some chainstate-init-order bugs (#10758)
300 2017-07-27 19:22:36	0|gribble|https://github.com/bitcoin/bitcoin/issues/10758 | Fix some chainstate-init-order bugs. by TheBlueMatt · Pull Request #10758 · bitcoin/bitcoin · GitHub
301 2017-07-27 19:22:50	0|BlueMatt|lots of bugs, lots of complicated
302 2017-07-27 19:23:08	0|BlueMatt|#10919 fixes even more bugs!
303 2017-07-27 19:23:09	0|gribble|https://github.com/bitcoin/bitcoin/issues/10919 | Fix more init bugs. by TheBlueMatt · Pull Request #10919 · bitcoin/bitcoin · GitHub
304 2017-07-27 19:23:11	0|wumpus|i think that one's ready for merge, I found some silly error message discrepancies but seems was re-pushed after
305 2017-07-27 19:23:15	0|achow101|wumpus: https://pastebin.com/hAeYMjRW
306 2017-07-27 19:23:15	0|BlueMatt|and has lots of more complicated
307 2017-07-27 19:23:19	0|gmaxwell|This is what fixes the genesis block grey goo?
308 2017-07-27 19:23:27	0|BlueMatt|gmaxwell: among other issues, yes
309 2017-07-27 19:23:46	0|wumpus|yep
310 2017-07-27 19:25:09	0|gmaxwell|LGTM. I'll test it if it doesn't get merged first.
311 2017-07-27 19:25:15	0|wumpus|we'll get to 10919 later
312 2017-07-27 19:25:18	0|gribble|https://github.com/bitcoin/bitcoin/issues/10778 | update assumevalid and minimum chain work · Issue #10778 · bitcoin/bitcoin · GitHub
313 2017-07-27 19:25:18	0|wumpus|#topic update assumevalid and minimum chain work (#10778)
314 2017-07-27 19:25:45	0|BlueMatt|i guess only question is do we need be concerned about doing it pre-/post- fork(s), I think not
315 2017-07-27 19:25:49	0|gmaxwell|wumpus: when do you want me to PR that.. I can do it at any time.
316 2017-07-27 19:25:49	0|wumpus|assigning that one to gmaxwell
317 2017-07-27 19:25:56	0|wumpus|gmaxwell: asap imo
318 2017-07-27 19:26:02	0|BlueMatt|why not next week?
319 2017-07-27 19:26:05	0|achow101|best before august 1st?
320 2017-07-27 19:26:06	0|wumpus|BlueMatt: yes that is a good point
321 2017-07-27 19:26:07	0|BlueMatt|later the better
322 2017-07-27 19:26:17	0|gmaxwell|Generally the later the better.
323 2017-07-27 19:26:23	0|gmaxwell|But I don't want to miss it.
324 2017-07-27 19:26:34	0|BlueMatt|worst case it misses rc1 but makes it in for rc2
325 2017-07-27 19:26:36	0|gmaxwell|It needs review before merge (since you'll need to check it matches your chain)
326 2017-07-27 19:26:36	0|wumpus|yes though I doubt a week makes that much difference, ignoring possible fork scenario
327 2017-07-27 19:26:56	0|wumpus|yes, so please make a PR asap
328 2017-07-27 19:26:58	0|BlueMatt|true, i dont care, gmaxwell up to you
329 2017-07-27 19:27:11	0|wumpus|it can always be updated
330 2017-07-27 19:27:20	0|gmaxwell|Okay, I'll PR soon.
331 2017-07-27 19:27:21	0|wumpus|(maybe it has to be, if reveiewers have problems with it)
332 2017-07-27 19:27:28	0|wumpus|ok, thanks
333 2017-07-27 19:27:56	0|gmaxwell|A week makes a measureable amount of synctime difference, unfortunately. :(  but not enough to worry about debating it much.
334 2017-07-27 19:28:00	0|gmaxwell|:)
335 2017-07-27 19:28:05	0|jtimon|yep, later better, but merged better too
336 2017-07-27 19:28:44	0|wumpus|yeah, I don't care about the exact day, let's just make sure that there is PR open that can be merged before we do the 0.15 branch
337 2017-07-27 19:28:49	0|wumpus|#topic Fix addwitnessaddress by replacing ismine with producesignature (#10788)
338 2017-07-27 19:28:51	0|gribble|https://github.com/bitcoin/bitcoin/issues/10788 | [RPC] Fix addwitnessaddress by replacing ismine with producesignature by achow101 · Pull Request #10788 · bitcoin/bitcoin · GitHub
339 2017-07-27 19:29:23	0|wumpus|this seems to require some work still, according to sdaftuar
340 2017-07-27 19:29:31	0|morcos|I just started looking at that one, but I like sipa's suggestion for addressing sdaftuars concern
341 2017-07-27 19:29:42	0|gmaxwell|Urgency of fixing it increased by probablity of segwit activation.
342 2017-07-27 19:29:45	0|achow101|I'm not exactly sure what sdaftuar is concerned about
343 2017-07-27 19:29:48	0|morcos|I haven't thought about the performance of any of these things
344 2017-07-27 19:30:08	0|morcos|achow101: just wants it to be clear what code is responsible for accomplishing what requirements
345 2017-07-27 19:30:18	0|gmaxwell|achow101: sounds like a general concern about the safty of further code changes.
346 2017-07-27 19:30:20	0|achow101|ah, I see. So sipa's suggestion should be fine
347 2017-07-27 19:30:24	0|morcos|functionally it'll be no different after change, but it'll be a lot harder to accidentally break
348 2017-07-27 19:30:28	0|achow101|right
349 2017-07-27 19:30:50	0|wumpus|being hard to accidentally break is good
350 2017-07-27 19:30:56	0|gmaxwell|achow101: in addition to sipa's suggestion a strategic comment or two would help.
351 2017-07-27 19:31:08	0|gmaxwell|Purpose of source code is communication between programmers, after all.
352 2017-07-27 19:31:24	0|achow101|ok
353 2017-07-27 19:32:02	0|wumpus|going to skip #10851, as cfields is not here and it's a build system issue
354 2017-07-27 19:32:03	0|gribble|https://github.com/bitcoin/bitcoin/issues/10851 | depends: bump fontconfig to 2.12.4 by theuni · Pull Request #10851 · bitcoin/bitcoin · GitHub
355 2017-07-27 19:32:25	0|jtimon|gmaxwell: isn't the purpose of source code that people can read it and learn how to program?
356 2017-07-27 19:32:31	0|wumpus|#topic Keypool topup (#10882)
357 2017-07-27 19:32:33	0|gribble|https://github.com/bitcoin/bitcoin/issues/10882 | Keypool topup by jnewbery · Pull Request #10882 · bitcoin/bitcoin · GitHub
358 2017-07-27 19:32:40	0|gmaxwell|10882 I need to sync up with the current state, it looked like it was going in a good direction. I see the latest comments have some improvements still needed.
359 2017-07-27 19:33:01	0|wumpus|jtimon: no, of course not, the purpose is that they can copy it and change the attribution and claim it's theirs :-)
360 2017-07-27 19:33:04	0|gmaxwell|but it seems like people know what to do...
361 2017-07-27 19:33:09	0|gmaxwell|wumpus: lol
362 2017-07-27 19:33:11	0|jtimon|wumpus: lol
363 2017-07-27 19:33:14	0|achow101|lol
364 2017-07-27 19:33:53	0|morcos|gmaxwell: i think 10882 is pretty much done
365 2017-07-27 19:34:05	0|BlueMatt|its very close, yes
366 2017-07-27 19:34:19	0|gmaxwell|\O/
367 2017-07-27 19:34:26	0|gmaxwell|I'll review today.
368 2017-07-27 19:34:39	0|jonasschnelli|Great
369 2017-07-27 19:34:58	0|gmaxwell|(as an aside, I'm really happy that people found solutions to issues I raised that I had no idea how to fix)
370 2017-07-27 19:35:15	0|wumpus|ah the instructions for manual wallet top-up were added, great
371 2017-07-27 19:35:19	0|wumpus|yeah
372 2017-07-27 19:35:24	0|gmaxwell|(in particular the two thresholds, so you can bypass critical shutdown)
373 2017-07-27 19:36:29	0|wumpus|lol github things the diff of wallet.cpp is so huge it hides it by default
374 2017-07-27 19:37:08	0|gmaxwell|I hate that feature.
375 2017-07-27 19:37:21	0|jonasschnelli|Yes. Make searching really painful
376 2017-07-27 19:37:25	0|jonasschnelli|*Makes
377 2017-07-27 19:37:26	0|gmaxwell|Caused me to screw up reviews before.
378 2017-07-27 19:37:38	0|BlueMatt|lol just dont review on github directly :)
379 2017-07-27 19:37:38	0|gmaxwell|(because I searched for X and then complained a PR couldn't work because it didn't touch X)
380 2017-07-27 19:37:41	0|BlueMatt|next pr?
381 2017-07-27 19:37:42	0|wumpus|gmaxwell: well they got complaints from people like me that browsers crashed/run out of memory if the page becomes very large, but I agree this is suboptimal too
382 2017-07-27 19:37:58	0|gmaxwell|obviously we need to split that file. :P
383 2017-07-27 19:38:11	0|gmaxwell|lets move on
384 2017-07-27 19:38:20	0|wumpus|#topic Reject invalid wallets (#10885)
385 2017-07-27 19:38:22	0|gribble|https://github.com/bitcoin/bitcoin/issues/10885 | Reject invalid wallets by promag · Pull Request #10885 · bitcoin/bitcoin · GitHub
386 2017-07-27 19:38:33	0|BlueMatt|i think its good
387 2017-07-27 19:38:52	0|BlueMatt|i saw john's comments but havent had a chance to read the responses or do another review myself
388 2017-07-27 19:38:57	0|BlueMatt|but is also very close
389 2017-07-27 19:39:07	0|wumpus|yes, I think it's pretty close
390 2017-07-27 19:39:42	0|wumpus|I think what's left is mostly comments onthe tests
391 2017-07-27 19:39:50	0|BlueMatt|thats it, no?
392 2017-07-27 19:40:14	0|wumpus|#topic Fix more init bugs (#10919)
393 2017-07-27 19:40:15	0|gribble|https://github.com/bitcoin/bitcoin/issues/10919 | Fix more init bugs. by TheBlueMatt · Pull Request #10919 · bitcoin/bitcoin · GitHub
394 2017-07-27 19:40:25	0|wumpus|no @BlueMatt ^^ moar init bugs
395 2017-07-27 19:40:34	0|BlueMatt|thats just #10758 part deux
396 2017-07-27 19:40:36	0|gribble|https://github.com/bitcoin/bitcoin/issues/10758 | Fix some chainstate-init-order bugs. by TheBlueMatt · Pull Request #10758 · bitcoin/bitcoin · GitHub
397 2017-07-27 19:41:00	0|BlueMatt|to get the first one merged cause people didnt want me to have a huge ever-growing pile of fixes in one pr
398 2017-07-27 19:41:11	0|jtimon|probably obvious but why not symlinks?
399 2017-07-27 19:42:13	0|wumpus|jtimon: berkeleydb doesn't gracefully support them, especially if you link outside the context directory
400 2017-07-27 19:42:27	0|jtimon|thanks
401 2017-07-27 19:42:37	0|wumpus|so it's very, very dangerous
402 2017-07-27 19:42:44	0|wumpus|BlueMatt: ok
403 2017-07-27 19:42:57	0|wumpus|#topic segwit wallet release (gmaxwell)
404 2017-07-27 19:43:54	0|gmaxwell|soon that exposes a segwit wallet to the user, and maybe supports sending
405 2017-07-27 19:43:54	0|gmaxwell|So, segwit is going to activate. It would be nice to get a version out
406 2017-07-27 19:43:57	0|gmaxwell|to BIP173 style addresses. Maybe we could consider short cycling 0.16
407 2017-07-27 19:44:00	0|gmaxwell|with primarily just this functionality, or something like that?
408 2017-07-27 19:44:02	0|gmaxwell|Since we technically have support but don't wire it up by default, (except
409 2017-07-27 19:44:05	0|gmaxwell|via the RPCs) I believe it's a short effort; mostly a testing concern.
410 2017-07-27 19:44:08	0|gmaxwell|Another alternative would be to branch, off a 0.15+segwit and cut
411 2017-07-27 19:44:10	0|gmaxwell|expiremental releases for people.
412 2017-07-27 19:44:13	0|gmaxwell|There are some political implications, since getting adoption of SW
413 2017-07-27 19:44:15	0|gmaxwell|will tone down some mania in the industry.
414 2017-07-27 19:44:32	0|jonasschnelli|Agree
415 2017-07-27 19:44:38	0|BlueMatt|seems reasonable
416 2017-07-27 19:44:47	0|BlueMatt|i vote for at 15.segwit
417 2017-07-27 19:44:54	0|achow101|perhaps start off by giving out the p2sh nested addresses first and then move on to bech32?
418 2017-07-27 19:44:56	0|wumpus|no problem with doing 0.16 faster (create a specific segwit-themed feature release, if it is realistic to do it in less than 6 months but still takes significant time (say, 3 months)
419 2017-07-27 19:45:03	0|gmaxwell|Well I threw out two ideas: 0.16 fast and 0.15+segwit as a seperate downloadable thing...
420 2017-07-27 19:45:20	0|achow101|why not just a 0.15.x release?
421 2017-07-27 19:45:27	0|gmaxwell|achow101: the point on 173 is support for sending to. We likely wouldn't actually issue those addresses for >1 year.
422 2017-07-27 19:45:27	0|jonasschnelli|So 0.15.1 + 0.15.1+SW?
423 2017-07-27 19:45:28	0|wumpus|achow101: it's a feature
424 2017-07-27 19:45:39	0|BlueMatt|15.segwit seems reasonable, while working on 16 still
425 2017-07-27 19:45:50	0|gmaxwell|If we did 0.15+sw as a full release we should call that 0.16. But thats just a naming thing.
426 2017-07-27 19:45:56	0|BlueMatt|well, ok
427 2017-07-27 19:45:56	0|wumpus|so is it really a small set of changes? or are you underestimating it?
428 2017-07-27 19:46:00	0|jtimon|achow101: yeah, wondering the same
429 2017-07-27 19:46:08	0|gmaxwell|but we could also release it as 0.15+expiremental segwit, an explicit alternative.
430 2017-07-27 19:46:10	0|BlueMatt|point being lets not freeze master again soon just for this
431 2017-07-27 19:46:25	0|BlueMatt|gmaxwell: all you're saying is support sending to segwit addresses, right?
432 2017-07-27 19:46:35	0|BlueMatt|not gui showing them as a receive address?
433 2017-07-27 19:46:55	0|gmaxwell|BlueMatt: if we do BIP173 support it would just be sending to.
434 2017-07-27 19:46:56	0|wumpus|yeah full GUI support is significant work
435 2017-07-27 19:47:11	0|BlueMatt|wumpus: hmm?
436 2017-07-27 19:47:21	0|BlueMatt|i mean we're just talking about a send-only address format, no?
437 2017-07-27 19:47:27	0|gmaxwell|There are two questions:  segwit wallet, and sending to BIP173. I consider the second less important, it's also more work (because it isn't already dormant in the codebase).
438 2017-07-27 19:47:40	0|wumpus|it's very easy to underestimate the changes needed, the tests, reviews, iterations
439 2017-07-27 19:47:50	0|achow101|gmaxwell: what do you mean by segwit wallet?
440 2017-07-27 19:47:53	0|jtimon|well, with the short release (let's say, 3 months) we don't necessarily need to "freeze master", those would be just high priority features for the release but we can still keep doing other things, no?
441 2017-07-27 19:48:04	0|gmaxwell|Segwit wallet basically means making getnewaddress return p2sh embedded segwit addresses.
442 2017-07-27 19:48:15	0|gmaxwell|(and the GUI of course)
443 2017-07-27 19:48:16	0|BlueMatt|mmm, yea, thats...trickier
444 2017-07-27 19:48:29	0|gmaxwell|BlueMatt: well technically it's just a few lines changed, if we ignore the testing burden. :)
445 2017-07-27 19:48:38	0|BlueMatt|heh, yea, well testing
446 2017-07-27 19:48:46	0|wumpus|building it on top of 0.15 and calling it 0.16 that's possible too, though a big change from how releases have always been done
447 2017-07-27 19:48:53	0|gmaxwell|because the wallet does support segwit today... just doesn't use it without dorking with the rpc.
448 2017-07-27 19:48:56	0|wumpus|would be the first major release not branched from master
449 2017-07-27 19:49:03	0|BlueMatt|i mean its not a 2 week project, but i also dont think its worth freezing master to do it (and given it'd be nice to get it out, it'd be nice to have *only* it as changes)
450 2017-07-27 19:49:24	0|wumpus|and it still needs to land in master anyhow, no way around that
451 2017-07-27 19:49:32	0|gmaxwell|And in _theory_ the segwit support in the wallet is "tested".
452 2017-07-27 19:49:39	0|jtimon|yeah, nack on working on 0.15 and calling it 0.16, why not simply 0.15.1 ?
453 2017-07-27 19:49:50	0|gmaxwell|So there is a scope question and a naming question. Lets not conflate them.
454 2017-07-27 19:50:04	0|BlueMatt|fair, scope wise lets not creep
455 2017-07-27 19:50:09	0|BlueMatt|which means keep it on the 15 branch
456 2017-07-27 19:50:20	0|BlueMatt|naming, i dont feel strongly about, either way wfm
457 2017-07-27 19:50:37	0|wumpus|ok
458 2017-07-27 19:50:47	0|jonasschnelli|would the 0.15.SW always generate p2sh embedded sw addresses or optional?
459 2017-07-27 19:51:01	0|BlueMatt|by default, i assume
460 2017-07-27 19:51:05	0|gmaxwell|So there are two things I think are potential for scope, SW wallet (getnewaddress returns p2sh embedded SW), and BIP173 sending.  We don't have to do them at the same time but it would be nice to be able to say that any SW enabled wallet can send to 173 style addresses.
461 2017-07-27 19:51:24	0|gmaxwell|jonasschnelli: I assume we'd make it gated by a config option like usehd.
462 2017-07-27 19:51:41	0|gmaxwell|Unless we ran into hairballs that we couldn't resolve.
463 2017-07-27 19:51:43	0|jtimon|if the scope is smal it seems reasonable to do it on 0.15.1/0.15.segwit
464 2017-07-27 19:51:46	0|wumpus|it certainly needs to be optional, default behavior can't change in a minor version
465 2017-07-27 19:51:49	0|jonasschnelli|BIP173 sending only is kinda hard(er) to test without the rec. part
466 2017-07-27 19:52:14	0|jonasschnelli|wumpus: Yes. That makes sense
467 2017-07-27 19:52:33	0|gmaxwell|jonasschnelli: we can recieve BIP173 payments with some hidden options. (I believe the code is still there-- we have tests for that already)
468 2017-07-27 19:52:34	0|wumpus|sending obvs is optional in itself, you don't *need* tos send to BIP173 addresses, and if you do you know what you're doing. I mean about receiving.
469 2017-07-27 19:53:10	0|gmaxwell|jonasschnelli: BIP173 is native segwit, we have tests for native segwit... what BIP173 adds is just an address encoding for it, so that it can be made user accessible.
470 2017-07-27 19:53:26	0|jonasschnelli|Indeed
471 2017-07-27 19:53:41	0|gmaxwell|But actually generating 173 addresses won't be useful for a long time... since you don't want to do that until basically everyone can send to it, took 2-3 years for p2sh.
472 2017-07-27 19:53:51	0|wumpus|creating bip173 receiving addresses should be optional, to not confuse too much, especially as long as vrutally no other wallets have support for it
473 2017-07-27 19:54:40	0|gmaxwell|I could see 173 adoption going faster, but it'll still take a while, and I don't see any reason to push it. But we should get started soon.
474 2017-07-27 19:54:53	0|wumpus|yes
475 2017-07-27 19:55:00	0|gmaxwell|(also pushing for rapid 173 adoption has a downside of making people think it's needed for segwit, it's not)
476 2017-07-27 19:56:06	0|wumpus|so anyhow, we'll branch off a special branch from 0.15 after the 0.15.0 release, for this work to be done?
477 2017-07-27 19:56:32	0|gmaxwell|I would be okay with that. We don't need to decide quite yet, but I wanted to get people thinking about it.
478 2017-07-27 19:56:42	0|jtimon|so what about 0.15.1 -> segwit wallet, 0.15.2 - > send to bip173, optionally receive ?
479 2017-07-27 19:56:45	0|BlueMatt|wumpus: that seems reasonable to me, ultimately i dont care too much and its up to you
480 2017-07-27 19:56:45	0|wumpus|or, I guess development could still be done on master w/ backports as normal
481 2017-07-27 19:56:53	0|BlueMatt|oh, yea, I'd vote backports
482 2017-07-27 19:56:58	0|gmaxwell|wumpus: that would be better IMO
483 2017-07-27 19:57:04	0|BlueMatt|whether its on the 15 branch or on a 15.segwit branch I dont care
484 2017-07-27 19:57:06	0|achow101|ack backports
485 2017-07-27 19:57:07	0|wumpus|(in the other case you'd have to forward-port)
486 2017-07-27 19:57:10	0|achow101|so we can have it in 0.16 too
487 2017-07-27 19:57:50	0|wumpus|yes
488 2017-07-27 19:58:08	0|gmaxwell|I really think the segwit-wallet part will be pretty minimal effort, except perhaps for more testing... just because it's virtually already done.
489 2017-07-27 19:58:44	0|jtimon|2 mins
490 2017-07-27 19:58:51	0|wumpus|I was thinking people wanted to work on the 0.15 branch for a moment, which would make sense if we expect master/0.16 to drift apart very quickly, but I don't think that's very true for the wallet (though there's more multi-wallet changes waiting for merge after the branch)
491 2017-07-27 19:59:02	0|gmaxwell|But who knows whatever dusty corners we'll expose.
492 2017-07-27 19:59:18	0|gmaxwell|wumpus: well it might favor delaying some 0.16 development to happen after this.
493 2017-07-27 19:59:35	0|wumpus|but forward and backporting isn't much different effort and it has to end up in master too
494 2017-07-27 19:59:39	0|gmaxwell|but I don't think there are many PRs already open that would get in the way.
495 2017-07-27 19:59:50	0|jonasschnelli|What about delaying 0.15 for P2SH(P2WPKH) support?
496 2017-07-27 19:59:55	0|wumpus|gmaxwell: in that case we're doing the 0.16 freeze that BlueMatt didn't want :)
497 2017-07-27 19:59:59	0|gmaxwell|jonasschnelli: I'd rather not.
498 2017-07-27 20:00:14	0|wumpus|I'd also prefer not
499 2017-07-27 20:00:16	0|wumpus|0.15 is almost ready
500 2017-07-27 20:00:18	0|gmaxwell|jonasschnelli: if we wanted to do that, I'd rather we just do 0.15 and then release 0.16 a couple weeks later with it.
501 2017-07-27 20:00:24	0|wumpus|0.15.1 is better
502 2017-07-27 20:00:39	0|jonasschnelli|Okay
503 2017-07-27 20:00:57	0|wumpus|would be very bad to delay 0.15.0 on a *feature* now, so long after the feature freeze
504 2017-07-27 20:01:03	0|wumpus|espeically one that doens't even have a PR open
505 2017-07-27 20:01:14	0|BlueMatt|*DONG*
506 2017-07-27 20:01:17	0|lightningbot|Log:            http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-07-27-19.00.log.html
507 2017-07-27 20:01:17	0|lightningbot|Meeting ended Thu Jul 27 20:01:16 2017 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
508 2017-07-27 20:01:17	0|lightningbot|Minutes:        http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-07-27-19.00.html
509 2017-07-27 20:01:17	0|lightningbot|Minutes (text): http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-07-27-19.00.txt
510 2017-07-27 20:01:17	0|wumpus|#endmeeting
511 2017-07-27 20:01:22	0|gmaxwell|we ran over!
512 2017-07-27 20:01:27	0|wumpus|shit
513 2017-07-27 20:01:59	0|jtimon|yeah, nack on delaying 0.15
514 2017-07-27 20:02:01	0|BlueMatt|well, clearly we have failed, we should give up having meetings
515 2017-07-27 20:02:24	0|gmaxwell|that meeting was invalid, gonna have to drop it and make a new one
516 2017-07-27 20:02:26	0|achow101|so about the large chainstate I have, here's the pastebin in case you missed it: https://pastebin.com/hAeYMjRW
517 2017-07-27 20:02:30	0|wumpus|we should have a meeting about better timing of meetings
518 2017-07-27 20:02:42	0|wumpus|achow101: ah yes, thanks, forgot about that one
519 2017-07-27 20:03:12	0|gmaxwell|BlueMatt: so why do you think it's not a two week project.. barring testing, I think we just shim getnewaddress to add the witness address. And then it will just work(tm).
520 2017-07-27 20:03:20	0|wumpus|achow101: .fuse_hidden files?!
521 2017-07-27 20:03:27	0|achow101|dunno wtf those are
522 2017-07-27 20:03:30	0|BlueMatt|gmaxwell: i dunno, i dont expect anything to get done in 2 weeks
523 2017-07-27 20:03:52	0|BlueMatt|gmaxwell: just review of wallet stuff is usually slow, plus testing and release cycle
524 2017-07-27 20:03:53	0|wumpus|achow101: what size do you get if you only measure LOG and .ldb files?
525 2017-07-27 20:03:56	0|gmaxwell|BlueMatt: well sure, but I think it's like a 1 day change plus N weeks of discussion and tweaking.
526 2017-07-27 20:04:06	0|wumpus|achow101: I think those are your junk files
527 2017-07-27 20:04:27	0|wumpus|achow101: fuse_hidden files are part of filesystem management anyhow, not of the leveldb database
528 2017-07-27 20:04:40	0|BlueMatt|gmaxwell: yea, that
529 2017-07-27 20:04:45	0|gmaxwell|BIP173 is worse becuase it's just not implemented at all... I wouldn't even suggest it except for the nice property of being able to say "any segwit version can send to BC addresses".
530 2017-07-27 20:04:46	0|wumpus|achow101: sometimes they're left behind
531 2017-07-27 20:04:49	0|jcorgan|what fs is dir in
532 2017-07-27 20:04:50	0|achow101|oh
533 2017-07-27 20:05:28	0|wumpus|sshfs?
534 2017-07-27 20:05:34	0|achow101|it's 5.8 of .ldb files
535 2017-07-27 20:05:40	0|achow101|*5.8 GB
536 2017-07-27 20:06:15	0|wumpus|achow101: that's more or  less the normal size after upgrade and before compaction
537 2017-07-27 20:07:08	0|gmaxwell|oh you said 8GB before, which was concerning.
538 2017-07-27 20:07:10	0|achow101|yeah, that makes sense
539 2017-07-27 20:07:18	0|achow101|gmaxwell: that's the entire chainstate folder
540 2017-07-27 20:07:29	0|gmaxwell|hm my entire chainstate folder is 5.8
541 2017-07-27 20:07:31	0|achow101|jcorgan: I think its an NTFS drive
542 2017-07-27 20:07:37	0|jcorgan|ah ok
543 2017-07-27 20:07:48	0|jcorgan|so ntfs-fuse then
544 2017-07-27 20:07:51	0|achow101|(I use it with windows occasionally)
545 2017-07-27 20:07:54	0|BlueMatt|ntfs? eww
546 2017-07-27 20:08:03	0|achow101|it started out as a windows drive
547 2017-07-27 20:08:05	0|wumpus|gmaxwell: it turns out to be full of fuse fs ghost files
548 2017-07-27 20:08:12	0|achow101|too lazy to change it
549 2017-07-27 20:08:30	0|jcorgan|those .fuse_hidden files are files deleted but someone still has an open file handle to it
550 2017-07-27 20:08:43	0|wumpus|I also had my datadir for one node on a ntfs-formatted external HDD for a long time, it seems to work surprisingly well
551 2017-07-27 20:08:46	0|gmaxwell|zoinks, well thats one mystery solved.
552 2017-07-27 20:08:57	0|achow101|jcorgan: can they be safely deleted?
553 2017-07-27 20:08:59	0|wumpus|jcorgan: yep, and sometimes it forgets about them and leaves them behind
554 2017-07-27 20:09:06	0|jcorgan|yes and yes
555 2017-07-27 20:09:24	0|wumpus|achow101: yes, maybe close bitcoind first just in case
556 2017-07-27 20:09:31	0|achow101|too late
557 2017-07-27 20:09:47	0|wumpus|(nothing *else* will be holding on handles to the chainstate files)
558 2017-07-27 20:10:09	0|jcorgan|the dates on those were from last september, you've probably rebooted since then :)
559 2017-07-27 20:10:19	0|wumpus|lol, yes
560 2017-07-27 20:10:33	0|achow101|I probably have. uptime says 43 days
561 2017-07-27 20:10:48	0|wumpus|or at least restarted bitcoind
562 2017-07-27 20:11:13	0|jcorgan|yeah, then they were just a bunch of fuse turds
563 2017-07-27 20:11:38	0|achow101|so how about compacting the database after upgrade (and not upgrading with 10526)?
564 2017-07-27 20:11:57	0|achow101|or is that "you did something stupid so deal with it yourself" kind of thing?
565 2017-07-27 20:12:31	0|wumpus|achow101: gmaxwell was talking about adding a hidden RPC to do database compactions, seems like a reasonable idea
566 2017-07-27 20:12:59	0|achow101|would it be a bad idea to just compact every startup?
567 2017-07-27 20:13:04	0|wumpus|yes, slow
568 2017-07-27 20:13:15	0|achow101|I figured
569 2017-07-27 20:13:40	0|wumpus|leveldb does it automatically once in a while anyway
570 2017-07-27 20:13:50	0|wumpus|(depending on some metric)
571 2017-07-27 20:14:18	0|wumpus|normally you shouldn't want to bother with it, unless you just rewrote the entire database like with the upgrade
572 2017-07-27 20:25:37	0|morcos|Murch: how deterministic is the existing coin selection algorithm?  if you call it twice with the same nValueToSelect, how likely is it to return the same inputs?
573 2017-07-27 20:30:40	0|Murch|morcos: I have not explicitly checked that, but since it heavily biases towards the largest input available and removes the last when it overshoots, my gut feeling would be that the vast majority of the 1000 attempts would get the same result.
574 2017-07-27 20:32:00	0|Murch|or at least the vast majority would reach a very small set of results.
575 2017-07-27 20:33:02	0|morcos|ok, thanks, i'm just looking at #10034 and there is a some braindead behavior where we stupidly call SelectCoins a second time for no reason if we are subtracting from recipient
576 2017-07-27 20:33:03	0|gribble|https://github.com/bitcoin/bitcoin/issues/10034 | sendtoaddress subtractfeefromamount=true does not respect paytxfee values · Issue #10034 · bitcoin/bitcoin · GitHub
577 2017-07-27 20:33:09	0|Murch|~50% of the results include the first utxo below target for starters. 25+% include the second utxo (depending on whether the first and second both fit at the same time)
578 2017-07-27 20:33:49	0|morcos|but i'm not seeing why it would consistently lead to significant overpaying
579 2017-07-27 20:34:17	0|Murch|morcos: The overpaying behavior comes from locking in the expected fees from the previous run.
580 2017-07-27 20:34:50	0|morcos|Murch: yes i'm aware of that problem, but that should be quite rare, not something that is seen over and over again
581 2017-07-27 20:35:18	0|morcos|and should be even less likely with subtractfeefromamount, b/c then you are trying to select the same value on the second run, so unlikly to choose far fewer inputs
582 2017-07-27 20:39:24	0|Murch|morcos:  every time when one round fails for selecting too little fees and then the next one succeeds with fewer inputs.
583 2017-07-27 20:39:43	0|sipa|so i think #10924 should be tagged for 0.15; i'll update with some more information
584 2017-07-27 20:39:44	0|gribble|https://github.com/bitcoin/bitcoin/issues/10924 | [RPC][Wallet][Segwit] Bug: Transaction sent to imported P2WSH does not appear in listtransaction. · Issue #10924 · bitcoin/bitcoin · GitHub
585 2017-07-27 20:40:03	0|sipa|it's possible to make gettransaction and listtransactions output diverge
586 2017-07-27 20:40:06	0|Murch|I think you changed something that took a minimal amount of fee from the change output when too little fee was selected. That should fix it. right?
587 2017-07-27 20:41:01	0|sipa|by importing a native segwit output script with importaddress, it getd detected as an ismine transaction, but the receiving end is treated as change
588 2017-07-27 20:41:02	0|morcos|Murch: my fixes didn't apply when you were subtracting fees from recipients, but in theory that case is the simplest to solve, just subtract fee from recipients. :)  But not sure we need that now for 0.15 or not
589 2017-07-27 20:41:38	0|Murch|morcos: I described the original problem on page twenty in my thesis: http://murch.one/wp-content/uploads/2016/11/erhardt2016coinselection.pdf
590 2017-07-27 20:41:57	0|gribble|https://github.com/bitcoin/bitcoin/issues/10924 | [RPC][Wallet][Segwit] Bug: Transaction sent to imported P2WSH does not appear in listtransaction. · Issue #10924 · bitcoin/bitcoin · GitHub
591 2017-07-27 20:41:57	0|sipa|the fix for #10924 is easy: we need to add P2WSH/P2WPKH to CTxDestination, which is needed anyway for bip173 support
592 2017-07-27 20:42:57	0|Murch|I seem to remember that you fixed at least the accumulative fee issue, but I'm not sure.
593 2017-07-27 20:59:38	0|MarcoFalke|happy editing: https://github.com/bitcoin-core/bitcoin-devwiki/wiki/0.15.0-Release-notes
594 2017-07-27 21:00:07	0|promag|wumpus: should I make the change https://github.com/bitcoin/bitcoin/pull/10885#issuecomment-318478956 ?
595 2017-07-27 21:27:50	0|bitcoin-git|[13bitcoin] 15morcos opened pull request #10942: Eliminate fee overpaying edge case when subtracting fee from recipients (06master...06subtractfee) 02https://github.com/bitcoin/bitcoin/pull/10942
596 2017-07-27 21:40:21	0|promag|morcos: is fSubtractFeeFromAmount something really used?
597 2017-07-27 21:40:44	0|gmaxwell|Yes, people use it.
598 2017-07-27 21:41:17	0|gmaxwell|It's a pretty useful way, for example, to transfer funds to another account you control; like an exchange, when you want to spend a given amount.
599 2017-07-27 21:42:43	0|promag|ah I see, ty
600 2017-07-27 21:45:54	0|promag|however we could do the same without having that hassle in the algorithm: do as normal and then subtract the fee from those outputs and add it to the change
601 2017-07-27 21:46:40	0|gmaxwell|promag: one of the applications is completely emptying a wallet.
602 2017-07-27 21:47:04	0|gmaxwell|without subtract fee from amount there is basically no way to do that (other than manually with raw transactions)
603 2017-07-27 21:47:30	0|promag|I surrender
604 2017-07-27 21:47:38	0|gmaxwell|:)
605 2017-07-27 21:47:48	0|gmaxwell|It's a good instinct to want to simplify.
606 2017-07-27 21:48:17	0|promag|it would be nice to have a test failing before https://github.com/bitcoin/bitcoin/pull/10942/files
607 2017-07-27 21:48:48	0|promag|I wonder if morcos is able to reproduce
608 2017-07-27 21:52:59	0|morcos|promag: irrelevangt of whether this fixes a bug or not, its a better design to just pick the inputs once and then subtract the fee.  That's what this PR accomplishes
609 2017-07-27 21:53:23	0|morcos|granted it does it in a kind of roundabout way, but all this code is going in the rubbish bin the day 0.15 is released
610 2017-07-27 21:53:40	0|morcos|so no reason to go crazy trying to make it pretty right now
611 2017-07-27 21:53:58	0|promag|yeah dejavu
612 2017-07-27 21:54:42	0|morcos|even if it doesn't look better, the logic is far clearer and smarter now than it was in 0.14
613 2017-07-27 22:03:46	0|promag|morcos: some comments