1 2017-06-15 06:12:51	0|wumpus|morcos: I think 0.14.2 is ready to go, but I asked yesterday and got no respnses at all
  2 2017-06-15 06:13:29	0|wumpus|(oh that's not true, fanquake replied)
  3 2017-06-15 06:14:55	0|wumpus|I'd like to get your display issue in, but that would mean another week and another rc, and I'm not sure it warrants it
  4 2017-06-15 06:17:05	0|wumpus|but we can discuss at the meeting
  5 2017-06-15 06:35:28	0|gmaxwell|wumpus: you missed morcos commenting several hours earler about it.
  6 2017-06-15 06:35:53	0|wumpus|gmaxwell: okay
  7 2017-06-15 06:43:30	0|wumpus|(didn't use to be the case here as it was a quite small channel, but a lot of people have joined here since, welcome everyone!)
  8 2017-06-15 07:09:05	0|wumpus|jonasschnelli: sorry if I'm completely off with https://github.com/bitcoin/bitcoin/pull/10251#issuecomment-308647720, has been a while I've touched that code, but I have some concerns
  9 2017-06-15 07:09:54	0|jonasschnelli|wumpus: you mean the comment is wrong?
 10 2017-06-15 07:10:21	0|wumpus|no, I think it's correct, that's why I'm worried
 11 2017-06-15 07:10:31	0|wumpus|if you can convince me it's wrong that would be great
 12 2017-06-15 07:10:32	0|wumpus|:)
 13 2017-06-15 07:10:40	0|jonasschnelli|okay... let me see
 14 2017-06-15 07:11:09	0|wumpus|but unconditional lock-taking on cs_main should be avoided as much as possible in the GUI thread
 15 2017-06-15 07:11:45	0|wumpus|and the timer isn't some separate thread, and neither is the signal handler for updateBalance
 16 2017-06-15 07:14:36	0|jonasschnelli|wumpus: the PR 10251 would remove the timer... and..
 17 2017-06-15 07:14:48	0|jonasschnelli|wumpus: the wallet would push balance updates to the GUI
 18 2017-06-15 07:15:12	0|wumpus|so where would the balance computation happen?
 19 2017-06-15 07:15:39	0|wumpus|functions like GetUnconfirmedWatchOnlyBalance are called from the GUI thread
 20 2017-06-15 07:15:57	0|wumpus|so if the balance is dirty, the computation happens in the GUI thread, after aquiring cs_main and cs_wallet unconditionally
 21 2017-06-15 07:16:25	0|jonasschnelli|Hmm... yes. I see
 22 2017-06-15 07:16:39	0|wumpus|up to six times - for every type of balance, this could take ~6 minutes on a slower machine during heavy lock contention
 23 2017-06-15 07:17:04	0|jonasschnelli|But only if a tx gets added or block connected/disconnected, right?
 24 2017-06-15 07:17:25	0|wumpus|yes, which is very often
 25 2017-06-15 07:17:52	0|wumpus|ideally the GUI thread would *never* hang itself based on notifications
 26 2017-06-15 07:17:56	0|jonasschnelli|You think the TRY_LOCK in a poll routine performs better?
 27 2017-06-15 07:17:57	0|wumpus|this is currently the case for that timer
 28 2017-06-15 07:18:12	0|jonasschnelli|Yeah.. it should not be in the GUI thread
 29 2017-06-15 07:18:19	0|wumpus|it optimizes for a different user experience - it's slower, but it never hangs the GUI thread
 30 2017-06-15 07:18:41	0|wumpus|it wouldn't be a big deal if wallet functions didn't need cs_main
 31 2017-06-15 07:18:45	0|wumpus|taking the wallet lock itself is ok
 32 2017-06-15 07:19:15	0|wumpus|but the cs_main lock is terrible, never grab it from the GUI thread if possible, and if you need to, and it's just to update information, use TRY_LOCK
 33 2017-06-15 07:19:36	0|jonasschnelli|That was my main intention (remove cs_main from GUI), but I guess I have made it worse
 34 2017-06-15 07:20:05	0|jonasschnelli|I thought pushing would perform better then constant polling (even with a TRY_LOCK)
 35 2017-06-15 07:20:40	0|jonasschnelli|Because, if you can LOCK in a TRY_LOCK, I would have expected to also block further calls that want to aquire the LOCK during the time you calculate the balances
 36 2017-06-15 07:20:44	0|bitcoin-git|[13bitcoin] 15luke-jr opened pull request #10595: Bugfix: RPC/Mining: Use pre-segwit sigops and limits, when working with non-segwit GBT clients (06master...06gbt_nosegwit_fix) 02https://github.com/bitcoin/bitcoin/pull/10595
 37 2017-06-15 07:21:00	0|wumpus|(for user-initiated things such as sending a transaction, it's more acceptable, though there too I'd prefer a solution that doesn't block the GUI thread)
 38 2017-06-15 07:21:37	0|jonasschnelli|Can we recalculate the balance during MarkDirty (connect/disconnect block, etc.)?
 39 2017-06-15 07:21:53	0|wumpus|the whole point of the dirty stuff is not to do that
 40 2017-06-15 07:22:17	0|wumpus|e.g.: if you're going to recalculate the balance *every* time it gets dirty, you don't need a cache
 41 2017-06-15 07:22:22	0|jonasschnelli|The idea behind the dirty/caching is to no recalculate it at every call,.. right?
 42 2017-06-15 07:23:10	0|wumpus|the markdirty is lazy evaluation: we know that the balance might have changed, but computing it can be expensive, so only compute it when necessary
 43 2017-06-15 07:23:19	0|jonasschnelli|Ideally the balance calls return the cache and the calls that have the power to invalidate the cache could recalculate?
 44 2017-06-15 07:23:33	0|jonasschnelli|I see..
 45 2017-06-15 07:23:45	0|jonasschnelli|[..] only compute it when necessary <-- make sense
 46 2017-06-15 07:23:59	0|jonasschnelli|the PR does that,.. but on the GUI thread
 47 2017-06-15 07:24:55	0|luke-jr|wumpus: I may not be able to make the meeting, but FWIW I am unsure if 10595 should be a blocker for v0.14.2 or not
 48 2017-06-15 07:25:39	0|jonasschnelli|wumpus: what about calling the getBalance() calls within a QThread and Q_EMIT balanceChanged once it's done?
 49 2017-06-15 07:25:39	0|wumpus|luke-jr: if we're unsure, it probably shouldn't be
 50 2017-06-15 07:26:59	0|wumpus|jonasschnelli: but then we're adding a thread
 51 2017-06-15 07:27:48	0|jonasschnelli|or use CScheduler?
 52 2017-06-15 07:27:49	0|wumpus|jonasschnelli: I do think eventually that's a better solution though: have a thread to communicate with the core
 53 2017-06-15 07:28:04	0|wumpus|never block in the GUI thread, have the GUI send commands/receive notifications from that thread
 54 2017-06-15 07:28:34	0|jonasschnelli|Yes. Some calls are synchronous though,.. they need redesign. But that must be done at some point
 55 2017-06-15 07:28:35	0|wumpus|but not a thread especially for updating balances, that's overkill, the current solution works ok
 56 2017-06-15 07:29:33	0|jonasschnelli|I think a tool where we can see what lock gets triggered how much and how long it has spent time in there would be of great value for the GUI
 57 2017-06-15 07:29:40	0|wumpus|the poll timer also offers congestion control: if a lot of transactions come in, it won't recompute the balance every time
 58 2017-06-15 07:32:54	0|jonasschnelli|wumpus: thanks for having a look into this and it seems to have be a waste of time... let me work towards the GUI<->Core communication thread
 59 2017-06-15 07:33:19	0|wumpus|jonasschnelli: yes, sorry for being encouraging at first then backpedalling, but I hadn't seen this
 60 2017-06-15 07:33:38	0|jonasschnelli|Yes. Great review. I'm happy you brought that up
 61 2017-06-15 07:33:58	0|gmaxwell|luke-jr: the gbt thing?  I don't see how its a potential issue unless you are adding _more_ transactions to your block than the template gave you, which I doubt anyone does right now.
 62 2017-06-15 07:35:00	0|luke-jr|gmaxwell: or a lot of sigops in the generation tx possibly, but a very unlikely case
 63 2017-06-15 07:35:09	0|luke-jr|not sure even Eligius or p2pool would hit it
 64 2017-06-15 07:36:34	0|luke-jr|yeah, seems like it shouldn't be a practical problem thinking about it more
 65 2017-06-15 07:36:53	0|wumpus|jonasschnelli: and in general design direction you're right - direct notification is better than polling, but it works best if all the information can be passed in through the signal so that there's no need for a round-trip
 66 2017-06-15 07:37:17	0|wumpus|jonasschnelli: in this case that won't work due to lazy evaluation
 67 2017-06-15 07:37:36	0|jonasschnelli|Yes. I see that point now...
 68 2017-06-15 08:18:22	0|bitcoin-git|[13bitcoin] 15benma opened pull request #10596: Add vConnect to CConnman::Options (06master...06connmanoptions_connect) 02https://github.com/bitcoin/bitcoin/pull/10596
 69 2017-06-15 09:39:13	0|bitcoin-git|[13bitcoin] 15practicalswift opened pull request #10597: scripted-diff: Make use of C++11:s improved handling of two consecutive right angle brackets (06master...06right-angle-brackets) 02https://github.com/bitcoin/bitcoin/pull/10597
 70 2017-06-15 10:44:42	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #10588: doc: Note preexisting bug in display of fee calculation in coin control (060.14...06notebug) 02https://github.com/bitcoin/bitcoin/pull/10588
 71 2017-06-15 10:52:47	0|bitcoin-git|[13bitcoin] 15laanwj pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/228c319a944b...7c72fb99afba
 72 2017-06-15 10:52:48	0|bitcoin-git|13bitcoin/06master 147c72fb9 15Wladimir J. van der Laan: Merge #10582: Pass in smart fee slider value to coin control dialog...
 73 2017-06-15 10:52:48	0|bitcoin-git|13bitcoin/06master 14e9cd778 15Alex Morcos: Pass in smart fee slider value to coin control dialog...
 74 2017-06-15 10:53:18	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #10582: Pass in smart fee slider value to coin control dialog (06master...06fixcoincontrolfee) 02https://github.com/bitcoin/bitcoin/pull/10582
 75 2017-06-15 10:57:18	0|gribble|https://github.com/bitcoin/bitcoin/issues/10504 | GUI unresponsive during slow operations · Issue #10504 · bitcoin/bitcoin · GitHub
 76 2017-06-15 10:57:18	0|ryanofsky|wumpus, jonasschnelli: related to this topic, not sure if you saw #10504 (also had a question for you in #10244)
 77 2017-06-15 10:57:19	0|gribble|https://github.com/bitcoin/bitcoin/issues/10244 | [qt] Add abstraction layer for accessing node and wallet functionality from gui by ryanofsky · Pull Request #10244 · bitcoin/bitcoin · GitHub
 78 2017-06-15 10:57:42	0|wumpus|ryanofsky: no, hadn't seen yet, will take a look
 79 2017-06-15 11:39:08	0|fanquake|wumpus going to build/release 14.2 tonight?
 80 2017-06-15 11:41:15	0|wumpus|fanquake: the release notes are so depressing now!
 81 2017-06-15 11:41:45	0|wumpus|but yes, I think we should just get it done
 82 2017-06-15 11:41:50	0|wumpus|0.14.3 will be more exciting, I promise
 83 2017-06-15 11:46:14	0|wumpus|so, one last time: did anyone hear of any issues with rc2?
 84 2017-06-15 12:05:57	0|paveljanik|silent night, ...
 85 2017-06-15 12:06:14	0|paveljanik|no issues
 86 2017-06-15 12:08:18	0|wumpus|agreed
 87 2017-06-15 12:08:24	0|wumpus|well, there we go
 88 2017-06-15 12:09:04	0|wumpus|* [new tag]         v0.14.2 -> v0.14.2
 89 2017-06-15 12:20:35	0|bitcoin-git|13bitcoin/06master 141bebfc8 15Alex Morcos: Output Fee Estimation Calculations in CreateTransaction
 90 2017-06-15 12:20:35	0|bitcoin-git|[13bitcoin] 15laanwj pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/7c72fb99afba...c2ab38bdd57a
 91 2017-06-15 12:20:36	0|bitcoin-git|13bitcoin/06master 14c2ab38b 15Wladimir J. van der Laan: Merge #10284: Always log debug information for fee calculation in CreateTransaction...
 92 2017-06-15 12:21:00	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #10284: Always log debug information for fee calculation in CreateTransaction (06master...06debugEstimates) 02https://github.com/bitcoin/bitcoin/pull/10284
 93 2017-06-15 12:45:15	0|bitcoin-git|[13bitcoin] 15paveljanik opened pull request #10598: Supress struct/class mismatch warnings introduced in #10284 (06master...0620170615_FeeCalculation_structclass) 02https://github.com/bitcoin/bitcoin/pull/10598
 94 2017-06-15 12:52:13	0|fanquake|wumpus heh have to follow up with a quick 0.14.3 then.
 95 2017-06-15 12:53:11	0|wumpus|yes - 0.14.2 just has to get out because of the upnp vuln.
 96 2017-06-15 13:16:55	0|jonasschnelli|ryanofsky: Yes. Saw it... as soon as I have a bit more time (currently occupied with DigitalBitbox work) i'll give you response...
 97 2017-06-15 13:17:06	0|jonasschnelli|The GUI responsiveness is ugly right now...
 98 2017-06-15 13:17:52	0|jonasschnelli|I wonder moving everything to ZMQ/RPC would be a viable approach (fully detach the GUI)
 99 2017-06-15 13:18:12	0|jonasschnelli|Ideally a clone of qt/ (to /qtdetatched)
100 2017-06-15 13:18:28	0|jonasschnelli|Then it would allow to remove some functions which are not possible via ZMQ/RPC
101 2017-06-15 13:22:46	0|ryanofsky|is that different than what I am doing with 10244?
102 2017-06-15 13:23:16	0|ryanofsky|10244 removes direct calls from qt -> node wallet and replaces then with calls through interfaces which could be implemented locally or through any rpc mechanism
103 2017-06-15 13:23:35	0|bitcoin-git|[13bitcoin] 15NicolasDorier reopened pull request #9991: listreceivedbyaddress Filter Address (06master...06listreceivedbyaddress-filtered) 02https://github.com/bitcoin/bitcoin/pull/9991
104 2017-06-15 13:24:03	0|wumpus|ryanofsky: but in itself that won't improve responsiveness, or does it also make callbacks asynchronous?
105 2017-06-15 13:26:06	0|ryanofsky|it doesn't affect responsiveness, though i do think the improved code organization would make improvements easier
106 2017-06-15 13:26:06	0|wumpus|I still have to look at it in detail, I think the idea makes sense though
107 2017-06-15 13:27:17	0|ryanofsky|10244 by itself doesn't change any behavior, all it does is replace direct calls with calls through interfaces
108 2017-06-15 13:27:25	0|wumpus|right
109 2017-06-15 13:27:33	0|wumpus|which makes sense
110 2017-06-15 13:28:36	0|wumpus|and when there's interfaces, we can add signals to them, to subscribe to to make the UI updates asynchronous
111 2017-06-15 13:29:15	0|wumpus|why rename CFeeBumper to FeeBumper though?
112 2017-06-15 13:29:47	0|wumpus|it would be better if this was separated into GUI changes, and core changes, I think in current state this is way too large
113 2017-06-15 13:29:56	0|ryanofsky|or even if we don't do that, it makes it easier to identify where blocking calls are happening, because they will all look like m_ipc_node->something() or m_ipc_wallet->something()
114 2017-06-15 13:31:04	0|ryanofsky|the feebumper commit is the one commit that actually makes substantial core changes, i can pull that into a separate pr
115 2017-06-15 13:31:32	0|ryanofsky|the other commits are almost entirely changes in src/qt and additions to src/ipc
116 2017-06-15 13:31:32	0|wumpus|but why rename it?
117 2017-06-15 13:32:48	0|ryanofsky|i renamed it because the commit had to update every single reference to it anyway, so i figured it'd be good to follow the current naming convention
118 2017-06-15 13:33:33	0|wumpus|it seems a bit too much to try to do all that in one PR, in my opinion, but don't know what others think
119 2017-06-15 13:33:35	0|ryanofsky|anyway happy to pull that commit into a separate pr
120 2017-06-15 13:34:01	0|ryanofsky|happy to break it down
121 2017-06-15 13:34:11	0|ryanofsky|but just know that one commit is an anomoly
122 2017-06-15 13:34:45	0|ryanofsky|the other commits are uniform mechanical changes
123 2017-06-15 13:34:53	0|wumpus|I don't personally particularly care if there's a large GUI change, but core changes need to be reviewable easily, so combining functionality changes with a rename will make it harder
124 2017-06-15 13:35:09	0|wumpus|ok, that's good
125 2017-06-15 13:35:22	0|ryanofsky|ok, will pull that commit out
126 2017-06-15 13:42:43	0|ryanofsky|there is another smaller change which affects locking in wallet.cpp. will make a separate pr for that too. after this 10244 should only touch src/qt/ and src/ipc/ files, no core files
127 2017-06-15 13:45:31	0|ryanofsky|jonasschnelli, still curious to hear more about your qtdetached idea, and if 10244 would help with that
128 2017-06-15 13:46:21	0|jonasschnelli|10244 can probably help, but my first hurdle is: +2,217 −1,131
129 2017-06-15 13:46:22	0|jonasschnelli|:)
130 2017-06-15 13:47:48	0|jonasschnelli|ryanofsky: The general abstraction may be good... but it may be also possible though wallet-/clientmodel
131 2017-06-15 13:48:15	0|ryanofsky|i guess i'd really appreciate feedback on the first commit: https://github.com/bitcoin/bitcoin/pull/10244/commits/7aeea1e0e54a773f8283605e9e4f9051cbf1ea87
132 2017-06-15 13:48:17	0|jonasschnelli|My (probably dumb) concept would be to have RPC calls in there...
133 2017-06-15 13:48:32	0|ryanofsky|because all the other commits (except feebumper which i will pull out) follow the same pattern as first commit
134 2017-06-15 13:49:32	0|jonasschnelli|I guess I would first tackle the asynchrony in the GUI layer...
135 2017-06-15 13:49:57	0|jonasschnelli|That needs to be done anyways and can have a quicker end user benefit when we would run all node communication in a designated thread
136 2017-06-15 13:50:00	0|wumpus|my preferred priority would also be improving responsiveness first
137 2017-06-15 13:50:09	0|jonasschnelli|Yes.
138 2017-06-15 13:50:24	0|jonasschnelli|For that, we first need to analze what functions/locks are the worst
139 2017-06-15 13:50:29	0|wumpus|though I think moving things to interfaces can help with that
140 2017-06-15 13:50:42	0|ryanofsky|can you guys explain your reasoning? to me it seems like this is basically unrelated, but could only make things easier not harder
141 2017-06-15 13:50:47	0|wumpus|as then it's clearer what the interface to the core thread should be
142 2017-06-15 13:50:49	0|jonasschnelli|Turn those expensive calls into async and use a general NODE<->GUI thread?
143 2017-06-15 13:51:02	0|ryanofsky|is there some way that my changes could make async improvments harder that i'm not seeing?
144 2017-06-15 13:51:21	0|jonasschnelli|No... i don't say you change is not good
145 2017-06-15 13:51:33	0|wumpus|not AFAIK
146 2017-06-15 13:51:38	0|jonasschnelli|I'm just arguing about priorities
147 2017-06-15 13:51:55	0|jonasschnelli|To get a +2,217 −1,131 change in, my experience tells me, ~0.5-1y
148 2017-06-15 13:53:47	0|jonasschnelli|but ryanofsky change makes sense.. the additional layer *IPC* <-> *wallet/client-model* <-> GUI is probably okay...
149 2017-06-15 13:55:50	0|ryanofsky|well hopefully it will take less than 0.5y to decide whether it's okay or not :)
150 2017-06-15 13:56:21	0|jonasschnelli|hehe... I think the concept is good.
151 2017-06-15 13:57:52	0|ryanofsky|feedback so far is helpful though, definitely makes sense to keep all core changes out of the pr and make it qt-only
152 2017-06-15 13:58:26	0|jonasschnelli|Yes. Ideally... this would reduce some risks
153 2017-06-15 14:00:22	0|wumpus|agree with doing it after the 0.15 split-off
154 2017-06-15 14:02:44	0|fanquake|to early for a 0.16 tag heh
155 2017-06-15 14:03:50	0|bitcoin-git|[13bitcoin] 15practicalswift closed pull request #10597: scripted-diff: Make use of the improved handling of two consecutive right angle brackets in C++11 (06master...06right-angle-brackets) 02https://github.com/bitcoin/bitcoin/pull/10597
156 2017-06-15 14:04:12	0|jonasschnelli|ryanofsky wumpus: is it only me or is the responsiveness in the current GUI worse then 0.13?
157 2017-06-15 14:04:42	0|jonasschnelli|ryanofsky: I guess some minor fixes could speed up the GUI and there would still be time for 0.15
158 2017-06-15 14:04:51	0|fanquake|jonasschnelli in 0.14.2 or master?
159 2017-06-15 14:05:17	0|jonasschnelli|fanquake: I always run master and I often have some freezes...
160 2017-06-15 14:05:33	0|jonasschnelli|I though my balance fix would reduce some... but sadly no
161 2017-06-15 14:06:03	0|ryanofsky|i can't say because i didn't really use the gui in 0.13, and even now tend to only use gui with small wallets, few transactions
162 2017-06-15 14:06:25	0|jonasschnelli|Would a LogPrint (with microseconds) in our LOCK macro make sense to analyse the lock behavior better?
163 2017-06-15 14:07:55	0|ryanofsky|that seems like would probably tell you where freezes are happening, maybe there is also a way to hook into qt to warn about event handlers that take a long time to run
164 2017-06-15 14:08:17	0|fanquake|jonasschnelli during any particular actions?
165 2017-06-15 14:08:41	0|jonasschnelli|fanquake: the freezes or the log print?
166 2017-06-15 14:08:45	0|jonasschnelli|ryanofsky: Yes...
167 2017-06-15 14:08:53	0|fanquake|the freezes
168 2017-06-15 14:09:20	0|jonasschnelli|fanquake: Yes. Calling generate 1 followed by a sendtoaddress feels blockish
169 2017-06-15 14:10:47	0|wumpus|I think there's too much overhead from logging to make that useful, also logging itself also locks
170 2017-06-15 14:11:01	0|wumpus|so you'd have to exclude that; but you could try
171 2017-06-15 14:14:22	0|jonasschnelli|I guess i'd use printf for a short hackish solution
172 2017-06-15 14:15:24	0|wumpus|in any case, I'm not sure how much you'd learn, all LOCKs in GUI code and calls to core functions that take locks on cs_main are a problem. THough possibly you can find the ones that occur most frequently if you also log where it's taken.
173 2017-06-15 14:45:55	0|Lauda|aruby
174 2017-06-15 14:46:00	0|Lauda|oops wrong channel
175 2017-06-15 14:56:03	0|bitcoin-git|[13bitcoin] 15ryanofsky opened pull request #10600: Make feebumper class stateless (06master...06pr/ipc-bump) 02https://github.com/bitcoin/bitcoin/pull/10600
176 2017-06-15 15:35:43	0|bitcoin-git|[13bitcoin] 15practicalswift opened pull request #10602: Make clang-format use C++11 features (e.g. A<A<int>> instead of A<A<int> >) (06master...06clang-format-cpp11) 02https://github.com/bitcoin/bitcoin/pull/10602
177 2017-06-15 17:35:54	0|jnewbery|Is anyone using the 'comparison' part of the comparison test framework? Any objections to me retiring it? #10603
178 2017-06-15 17:35:55	0|gribble|https://github.com/bitcoin/bitcoin/issues/10603 | Retire the comparison test framework · Issue #10603 · bitcoin/bitcoin · GitHub
179 2017-06-15 17:45:13	0|wumpus|jnewbery: I'm fairly sure people are using it
180 2017-06-15 17:45:26	0|wumpus|@(sdaftuar morcos BlueMatt)
181 2017-06-15 17:46:13	0|sdaftuar|wumpus: i've talked to jnewbery about it offline a bit; i mostly think we don't really use it
182 2017-06-15 17:46:24	0|wumpus|okay
183 2017-06-15 17:46:30	0|sdaftuar|the intention was to be able to compare many versions of the software
184 2017-06-15 17:46:44	0|wumpus|but that isn't that useful in practice?
185 2017-06-15 17:46:50	0|sdaftuar|but i don't think anyone (outside my proof of concept, way back when) actually tried to do that?
186 2017-06-15 17:47:45	0|wumpus|would be a better question to ask then "should someone be doing those tests?"
187 2017-06-15 17:48:00	0|sdaftuar|yeah i think that's the good question
188 2017-06-15 17:48:18	0|jnewbery|The way those test cases are actually used in practice is that test_runner.py runs them with a single node. In theory they could be run with multiple nodes and the states compared, but as far as I'm aware no-one does that
189 2017-06-15 17:48:26	0|wumpus|I mean we could always add tests, maybe even in a travis crontab
190 2017-06-15 17:48:57	0|sdaftuar|in practice, it seems like we've never really written tests where the outcome wasn't fixed and known, and therefore hardcoded in the test itself
191 2017-06-15 17:49:26	0|sdaftuar|i guess it's possibel we could introduce a change to bitcoind and the corresponding test and accidentally introduce a comparison failure
192 2017-06-15 17:50:21	0|sdaftuar|p2p-fullblocktest doesn't change very often though
193 2017-06-15 17:50:34	0|sdaftuar|the risk seems like it'd be in consensus changes not yet deployed, eg something like segwit
194 2017-06-15 17:50:40	0|sdaftuar|but those tests haven't changed much either
195 2017-06-15 17:52:56	0|jnewbery|I think the risk seems low. The individual tests are written quite prescriptively, so in order to break a 'comparison' without breaking running the test with an individual node, someone would probably need to update the individual test case to change the expectation
196 2017-06-15 17:53:24	0|jnewbery|Anyway, I'd be interested to hear whether anyone out there is using them or has any input. Issue 10603
197 2017-06-15 17:55:14	0|jnewbery|10457
198 2017-06-15 17:55:14	0|jnewbery|The tests that use the comparison test framework are:
199 2017-06-15 17:55:24	0|jnewbery|bip65-cltv-p2p.py bip68-112-113-p2p.py bip9-softforks.py bipdersig-p2p.py invalidblockrequest.py invalidtxrequest.py p2p-fullblocktest.py
200 2017-06-15 18:01:03	0|gmaxwell|sdaftuar: we used it extensively in the past.
201 2017-06-15 18:01:19	0|gmaxwell|rather the old bitcoinj tool. (to be clear)
202 2017-06-15 18:01:31	0|sdaftuar|gmaxwell: right, we're still going to have p2p-fullblocktest
203 2017-06-15 18:01:40	0|sdaftuar|which is approx. the same as the old bitcoinj tool
204 2017-06-15 18:02:02	0|sdaftuar|the question is just, do we try to support/improve the ComparisonTestFramework thing as the implementation
205 2017-06-15 18:02:44	0|sdaftuar|which i think has mostly been a not very helpful infrastructure... the tests in that framework are hard to read
206 2017-06-15 18:02:57	0|gmaxwell|OK.
207 2017-06-15 18:03:01	0|sdaftuar|and working around p2p changes that the ComparisonTestFramework isn't designed for is pretty hacky
208 2017-06-15 18:03:14	0|bitcoin-git|[13bitcoin] 15jnewbery opened pull request #10604: Expose multiwallet in getwalletinfo and add multiwallet test (06master...06multiwallet_test) 02https://github.com/bitcoin/bitcoin/pull/10604
209 2017-06-15 18:05:49	0|gmaxwell|sorry, I tuned in to the end of the conversation. I mostly started commenting because I strongly disagree with the claim that that the existing test cases were at all close enough to comprehensive to guarentee consensus consistency with another implementation. We don't even come close to 100% branch coverage in script.
210 2017-06-15 18:06:46	0|sdaftuar|oh, yeah definitely agree with that
211 2017-06-15 18:07:06	0|sdaftuar|i think the question is, should we be working towards comparison style tests, along the lines of what the comparisontestframework was intended for?
212 2017-06-15 18:07:49	0|kanzure|branch coverage would be a good set of tests..
213 2017-06-15 18:07:59	0|kanzure|er, script coverage. what is missing?
214 2017-06-15 18:11:52	0|gmaxwell|sdaftuar: I think they're secondary to other kinds of tests.  Comparison is primarily useful if we have good random behavior generation and can verify consistency.
215 2017-06-15 18:13:06	0|sdaftuar|that makes sense to me...  my thought was that it would be more helpful to rewrite our comparison tests (which are currently just evaluating a single node anyway) in an imperative style, to make them easier to maintain and debug
216 2017-06-15 18:14:31	0|sdaftuar|and in retrospect, i think if we did want a comparison test framework, it probably makes more sense to scrap trying to do it over p2p, and just test consensus with submitblock or something.  less prone to maintenance headache
217 2017-06-15 18:14:48	0|gmaxwell|er. I don't agree (sadly)--
218 2017-06-15 18:16:05	0|gmaxwell|for two reasons:  p2p means its more duarable across versions (e.g. you can test an alternative implementation), and p2p means its testing the interface that actually matters; it would catch thing like failures in the block fetching state machine.
219 2017-06-15 18:16:50	0|gmaxwell|Comparing other implementations can increase the sensitivity of the tests even if we'd never expect someone to use the other implementation.
220 2017-06-15 18:17:25	0|sdaftuar|that's true in theory, but i think that has mostly failed in practice
221 2017-06-15 18:18:04	0|sdaftuar|at least, we have never implemented the full range of p2p behaviors in a single piece of python test harness code
222 2017-06-15 18:18:20	0|wumpus|what I'm not sure about is how the comparision test is better than just running the tests against the other implementation
223 2017-06-15 18:19:31	0|sdaftuar|so for instance, we end up implementing kludges and workarounds in the python test code in order to eg ensure block delivery to a particular implementation. see for instance how we handle announcing and relaying a block, after we turned off direct fetch on inv's in bitcoind
224 2017-06-15 18:19:47	0|sipa|wumpus: i guess the advantage is that you can randomly generate scenarios and compare them, rather than needing a test writer who knows what the result is supposed to be
225 2017-06-15 18:19:48	0|gmaxwell|wumpus: see my comment about randomly generated behavior.  If you have a random sequence of operations you can test certian invariants, but the random generator doesn't know the exact result-- but it can let us know when two different implementations behaved differently.
226 2017-06-15 18:20:00	0|sipa|but that advantage does go away if you're just running a static scenario
227 2017-06-15 18:20:02	0|jnewbery|gmaxwell: I think really for stability across versions, you want to have a known good node that you use RPCs on to create blocks and transactions, then connect that node to nodes-under-test of different versions. Otherwise you need to update the tests whenever any p2p logic changes (eg headers first, etc)
228 2017-06-15 18:20:07	0|gmaxwell|It's an orthorgonal and very powerful method of testing.
229 2017-06-15 18:20:09	0|sdaftuar|it seems to me that the more durable thing is to have specific p2p tests that exercise the code paths we want to test
230 2017-06-15 18:20:12	0|wumpus|right, if you can somehow generate random scenarios, it'd be useful
231 2017-06-15 18:20:14	0|ProfMac|Is this the best place to ask questions re: setting up the virtualbox / debian / gitian build environment according to https://github.com/bitcoin/bitcoin/blob/master/doc/gitian-building.md#create-a-new-virtualbox-vm  The first question is at #installing-gitian, when I do the git clone https://github.com/bitcoin/bitcoin should I be in ~ or in vm-builder-0.12.4+bzr494
232 2017-06-15 18:20:19	0|gmaxwell|And yes, for static tests is not useful IMO.
233 2017-06-15 18:20:31	0|wumpus|yes, then it's like fuzzing two implementations at the same time and comparing
234 2017-06-15 18:20:51	0|gmaxwell|Rather if we get interesting patterns out of a comparison, we could seralize those into a static test (and add the exact comparison points)
235 2017-06-15 18:21:19	0|wumpus|ProfMac: where doesn't matter, as long as you consistently use same the directory, though cloning bitcoin under the vm-builder directory likely is not what you want
236 2017-06-15 18:21:34	0|gmaxwell|This is, e.g. how we tested the DER parser in libsecp256k1-- a harness with three implementations.
237 2017-06-15 18:21:36	0|jnewbery|ok, so how about I change those static tests to use the standard test framework? I won't delete the comparison test framework code so if someone wants to use it for randomized testing, they can still do that
238 2017-06-15 18:22:23	0|ProfMac|Thanks wumpus.  I am doing a 2nd install from scratch, yesterday I could not get it to work.
239 2017-06-15 18:22:47	0|sipa|jnewbery: if all they are doing is testing a scenario where it's obvious from the test code what the expected behaviour is, sure
240 2017-06-15 18:23:38	0|gmaxwell|Would that mean moving the test stimulus for those things from p2p to rpc?
241 2017-06-15 18:23:45	0|sdaftuar|no, it shouldn't
242 2017-06-15 18:23:49	0|gmaxwell|okay!
243 2017-06-15 18:24:26	0|wumpus|that would mean that the comparison test code doesn't get tested itself anymore, and thus will code-rot
244 2017-06-15 18:24:36	0|sdaftuar|wumpus: i fear it already has rotted :(
245 2017-06-15 18:24:54	0|ProfMac|So I'm going to continue with the 3 git clone commands in "~/opt/gitian", anyone see any red flags or offer a better naming scheme?
246 2017-06-15 18:24:56	0|wumpus|ok...
247 2017-06-15 18:25:56	0|sdaftuar|i mean that in the sense, we already have hacky workarounds in eg p2p-fullblocktest to deal with shortcomings in the framework
248 2017-06-15 18:50:56	0|gmaxwell|meeting in 10 minutes.
249 2017-06-15 19:00:04	0|jonasschnelli|hi
250 2017-06-15 19:00:08	0|lightningbot|Meeting started Thu Jun 15 19:00:07 2017 UTC.  The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
251 2017-06-15 19:00:08	0|lightningbot|Useful Commands: #action #agreed #help #info #idea #link #topic.
252 2017-06-15 19:00:08	0|wumpus|#startmeeting
253 2017-06-15 19:00:25	0|wumpus|PSA: v0.14.2 has been tagged, please start your gitian builders
254 2017-06-15 19:00:30	0|wumpus|topics?
255 2017-06-15 19:00:34	0|gmaxwell|#bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr btcdrak sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier
256 2017-06-15 19:00:34	0|wumpus|#bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr btcdrak sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier jl2012 instagibbs achow101
257 2017-06-15 19:00:36	0|kanzure|hi.
258 2017-06-15 19:00:41	0|paveljanik|Hi
259 2017-06-15 19:00:42	0|achow101|hi
260 2017-06-15 19:00:43	0|cfields|jonasschnelli: i just wrote a quick mutex locktime reporter, ping me after meeting if you'd like to discuss
261 2017-06-15 19:00:45	0|sdaftuar|here
262 2017-06-15 19:00:46	0|sipa|yow
263 2017-06-15 19:00:50	0|gmaxwell|(I guess I should update my list)
264 2017-06-15 19:00:58	0|jonasschnelli|cfields: awesome!
265 2017-06-15 19:01:04	0|sipa|wumpus: you're requesting the presence of 2 instagibbses?
266 2017-06-15 19:01:07	0|gmaxwell|cfields: finally someone did that, awesome.
267 2017-06-15 19:01:20	0|cfields|well it's very dumb, but it's something :)
268 2017-06-15 19:01:21	0|wumpus|sipa: yes!
269 2017-06-15 19:01:34	0|sipa|cfields: -DDEBUG_LOCKCONTENTION?
270 2017-06-15 19:01:39	0|jonasschnelli|cfields: dumb is good
271 2017-06-15 19:01:39	0|luke-jr|lol
272 2017-06-15 19:01:52	0|wumpus|#topic high priority for review
273 2017-06-15 19:02:05	0|gmaxwell|https://github.com/bitcoin/bitcoin/projects/8
274 2017-06-15 19:02:09	0|sipa|#10148 plz *puppyeyes*
275 2017-06-15 19:02:11	0|gribble|https://github.com/bitcoin/bitcoin/issues/10148 | Use non-atomic flushing with block replay by sipa · Pull Request #10148 · bitcoin/bitcoin · GitHub
276 2017-06-15 19:02:27	0|wumpus|10148 is already on the list, I'm testing it
277 2017-06-15 19:02:31	0|sipa|cool
278 2017-06-15 19:02:33	0|sdaftuar|i'm working on an rpc test as well
279 2017-06-15 19:02:40	0|sipa|sdaftuar: awesome!
280 2017-06-15 19:02:43	0|sipa|can i help?
281 2017-06-15 19:03:01	0|sdaftuar|i'll let you know!
282 2017-06-15 19:03:06	0|sdaftuar|i think i almos thave it
283 2017-06-15 19:04:08	0|sipa|having thought more about it, i don't think #10339 will have any significant performance impact
284 2017-06-15 19:04:12	0|gribble|https://github.com/bitcoin/bitcoin/issues/10339 | Optimization: Calculate block hash less times by jtimon · Pull Request #10339 · bitcoin/bitcoin · GitHub
285 2017-06-15 19:04:23	0|luke-jr|basic multiwallet was merged. I hope to have the next step (RPC support) later today. ACK to add to priority then?
286 2017-06-15 19:04:49	0|jonasschnelli|RPC support would be great
287 2017-06-15 19:04:54	0|sipa|agree
288 2017-06-15 19:04:56	0|achow101|luke-jr: yes please
289 2017-06-15 19:04:56	0|jonasschnelli|You mean addressing wallet via RPC endpoint?
290 2017-06-15 19:05:09	0|luke-jr|jonasschnelli: I mean each username has a different single wallet
291 2017-06-15 19:05:23	0|jonasschnelli|luke-jr: hmm...
292 2017-06-15 19:05:24	0|wumpus|sipa: ok, removing it from high priority then
293 2017-06-15 19:05:27	0|jonasschnelli|I'd prefere endpoints
294 2017-06-15 19:05:39	0|jonasschnelli|AUTH for wallet switching seems hackish
295 2017-06-15 19:05:42	0|luke-jr|jonasschnelli: endpoints can be done later; I'm just trying to get the simplest stuff done first
296 2017-06-15 19:05:51	0|jonasschnelli|endpoint is 10lines of code
297 2017-06-15 19:05:54	0|wumpus|I'd also prefer endpoints, makes it easier to move wallets to external processes etc just by changing the url
298 2017-06-15 19:05:58	0|jonasschnelli|I can post it to you later
299 2017-06-15 19:06:01	0|luke-jr|jonasschnelli: not securely ;p
300 2017-06-15 19:06:10	0|luke-jr|jonasschnelli: I don't want JoinMarket to have access to my main wallet
301 2017-06-15 19:06:21	0|jonasschnelli|there is no security in our RPC implementation :/
302 2017-06-15 19:06:25	0|sipa|yay, wallet ACLs
303 2017-06-15 19:06:31	0|wumpus|please don't use auth, it's not supposed to be a multi-user authentication mult-wallet, that just adds another nightmare difficult to support like accounts
304 2017-06-15 19:06:32	0|achow101|you could do some combination of both?
305 2017-06-15 19:06:32	0|luke-jr|wumpus: different username is also a simple change to the URI
306 2017-06-15 19:06:44	0|wumpus|we already support different usernames/passwords
307 2017-06-15 19:06:50	0|jonasschnelli|use a passphase as wallet name
308 2017-06-15 19:06:56	0|jonasschnelli|same security as basic auth?
309 2017-06-15 19:06:56	0|wumpus|it's an authentication feature, should not affect the wallet
310 2017-06-15 19:07:23	0|sipa|i think the first step should be either endpoint or a generic optional named parameter to select the wallet
311 2017-06-15 19:07:25	0|luke-jr|wumpus: I see no distinction
312 2017-06-15 19:07:29	0|gmaxwell|endpoints won't be ten lines of code. After all, we'll need to add support for them to bitcoin-cli, the test framework, etc.
313 2017-06-15 19:07:29	0|wumpus|sipa: yes
314 2017-06-15 19:07:34	0|achow101|sipa: agreed
315 2017-06-15 19:07:43	0|sipa|the choice of which user can access which wallets is orthogonal, i think
316 2017-06-15 19:07:51	0|wumpus|gmaxwell: which is easy, the underlying stuff (rpcproxy, libevent) obviously supports it
317 2017-06-15 19:07:56	0|sipa|but i would prefer not to tie users to wallets at the auth level
318 2017-06-15 19:07:57	0|jonasschnelli|gmaxwell: yes Indeed
319 2017-06-15 19:08:48	0|gmaxwell|obviously we do want to have username/wallet binding, right?  This lets you be more confident e.g. that your joinmarket install isn't going to screw up your ordinary wallet, for example.
320 2017-06-15 19:08:48	0|jonasschnelli|first stage, each user should be able to access each wallet....
321 2017-06-15 19:08:49	0|wumpus|sipa: me neither, it just seems a level violation, and causes wrong expectations that giving access to RPC to one wallet is secure in any way
322 2017-06-15 19:08:55	0|gmaxwell|(eventually)
323 2017-06-15 19:08:57	0|luke-jr|jonasschnelli: no -.-
324 2017-06-15 19:09:12	0|wumpus|gmaxwell: I really think that's going too far
325 2017-06-15 19:09:24	0|gmaxwell|then why are we bothering?
326 2017-06-15 19:09:28	0|wumpus|securing RPC for multiple users is absolutely a nightmare
327 2017-06-15 19:09:33	0|jonasschnelli|luke-jr: the first logical extendable step would be that, no? Adding wallet selecting via AUTH is something you need to throw away later
328 2017-06-15 19:09:50	0|luke-jr|well, if I can't isolate JoinMarket this way, I have no interest in doing it.. so I can just move on to GUI and leave RPC support in Knots only
329 2017-06-15 19:09:52	0|sipa|wumpus: i think it's inevitable that we'll need that
330 2017-06-15 19:09:53	0|wumpus|anyhow a security layer could always be added could be later if endpoint-based multiwallet is in place
331 2017-06-15 19:09:59	0|wumpus|sipa: I think it's a mistake
332 2017-06-15 19:10:06	0|wumpus|sipa: just like accounts was
333 2017-06-15 19:10:14	0|wumpus|it's something that bitcoind shouldn't handle
334 2017-06-15 19:10:27	0|gmaxwell|I think what luke would like to accomplish is making multiwallet immediately useful for the application of combining multiple applicatoins onto one bitcoind; rather than having to run seperate bitcoinds for each thing that needs a wallet that you're running.
335 2017-06-15 19:10:31	0|luke-jr|jonasschnelli: no?
336 2017-06-15 19:11:00	0|sipa|gmaxwell: i think that's an interesting use case; i don't think it should be the first step
337 2017-06-15 19:11:05	0|wumpus|that's just inviting bugs, there's no way we can make that secure, the RPC is not a secure endpoint and is regarded as compeltely  trusted
338 2017-06-15 19:11:35	0|wumpus|it would escalate a bug in e.g. a single RPC command to a security issue, right now RPC access = fully trusted
339 2017-06-15 19:11:37	0|jonasschnelli|luke-jr: I don't know the JoinMarket use case very well.. but if you give it access to your node, it could shutdown, add peers, etc. (in case you don't trust that software)
340 2017-06-15 19:11:46	0|gmaxwell|This is also important to us at blockstream and we will end up maintaining a fork of Bitcoin with it. (though luke wasn't doing this work at our request).
341 2017-06-15 19:11:59	0|luke-jr|jonasschnelli: even if we add endpoint multiwallet and ACLs later, we still want a way to select a default wallet for each user
342 2017-06-15 19:12:21	0|luke-jr|wumpus: then why do we have auth at all?
343 2017-06-15 19:12:33	0|jonasschnelli|I really think we should keep hands away from multi-user/multi-wallet setup
344 2017-06-15 19:12:35	0|wumpus|luke-jr: to gain access
345 2017-06-15 19:12:48	0|wumpus|jonasschnelli: me too... seems something that needs to be a level on top, not handled by bitcoind itself
346 2017-06-15 19:12:56	0|jonasschnelli|For now we should focus on single-user/multi-wallet (1:n)
347 2017-06-15 19:13:16	0|jonasschnelli|n:n smells like a account-like-problem-re-incarnation
348 2017-06-15 19:13:21	0|wumpus|anyhow if we have endpoint multi-wallet access, it'spossible to slap on a wallet/user auth mapping later
349 2017-06-15 19:13:35	0|luke-jr|or vice-versa..
350 2017-06-15 19:13:39	0|wumpus|that's "just" a matter of access control
351 2017-06-15 19:13:45	0|jonasschnelli|yes. n:n may make sense.. but endpoint first seems much more logical
352 2017-06-15 19:13:49	0|wumpus|yes
353 2017-06-15 19:13:52	0|gmaxwell|jonasschnelli: I do not follow your comment with account like problems. The problem with accounts is that they weren't wallets but users expected them to be and treated them like ones.
354 2017-06-15 19:14:07	0|jonasschnelli|gmaxwell: Yes. Not directly related.
355 2017-06-15 19:14:10	0|sipa|i don't think access control is necessarily that complicated; have a global permission and wallet specific permission; configure which users have which
356 2017-06-15 19:14:14	0|wumpus|I just think that making bitcoind multi-user is a grave mistake
357 2017-06-15 19:14:18	0|wumpus|but I"ll shut up about it...
358 2017-06-15 19:14:24	0|jonasschnelli|I think the complexity is huge,.. leads to permission groups, etc.
359 2017-06-15 19:14:36	0|wumpus|yes, exactly, some people wnat everything in bitcoind
360 2017-06-15 19:14:45	0|gmaxwell|jonasschnelli: what? no it doesn't.
361 2017-06-15 19:14:48	0|sipa|well it seems that multiple people want multiwallet for multiple reasons
362 2017-06-15 19:14:54	0|sipa|i don't think that's a problem
363 2017-06-15 19:15:04	0|sipa|and should not be a blocker for the basic functionality
364 2017-06-15 19:15:08	0|wumpus|this is one the reason why the wallet should have been split off to a separate process / library I guess... now it all needs to be compounded
365 2017-06-15 19:15:11	0|wumpus|making bitcoind some kind of systemd
366 2017-06-15 19:15:12	0|jonasschnelli|if we start to use n:n, enterprises will probably use it for multi-user wallet backends...
367 2017-06-15 19:15:33	0|luke-jr|wumpus: user:wallet makes a split off later simpler
368 2017-06-15 19:15:34	0|jonasschnelli|and removing – if it gets to complicated – is hard or even impossible (like the accounting)
369 2017-06-15 19:15:35	0|wumpus|jonasschnelli: yes exactly... and what if there's a bug in that
370 2017-06-15 19:15:38	0|luke-jr|endpoints makes splitting off later complex
371 2017-06-15 19:15:54	0|wumpus|it moves all the (perceived) responsiblity for managing multi-user setups secure to us
372 2017-06-15 19:16:01	0|jonasschnelli|luke-jr: endpoint would even work if each wallet runs in its own process
373 2017-06-15 19:16:05	0|gmaxwell|How do we split wallets if we are using endpoints?
374 2017-06-15 19:16:09	0|jonasschnelli|(though auth probably also)
375 2017-06-15 19:16:18	0|luke-jr|jonasschnelli: huh? not really..?
376 2017-06-15 19:16:24	0|jtimon|but multi-wallet doesn't imply multi-user, does it?
377 2017-06-15 19:16:26	0|wumpus|gmaxwell: what do you mean with "split wallets"?
378 2017-06-15 19:16:37	0|gmaxwell|split wallets int oseperate processes
379 2017-06-15 19:16:51	0|wumpus|gmaxwell: different wallets have different URLs then
380 2017-06-15 19:17:05	0|wumpus|gmaxwell: so it's just another change: change the port...
381 2017-06-15 19:17:18	0|luke-jr|wumpus: ALL of these options are simple URI changes..
382 2017-06-15 19:17:28	0|achow101|how would different endpoints work with bitcoin-cli or the debug console?
383 2017-06-15 19:17:28	0|luke-jr|although some tools don't allow changing the URI right now
384 2017-06-15 19:17:32	0|wumpus|http://127.0.0.1:8333/wallet1 versus http://127.0.0.1:8334/wallet2
385 2017-06-15 19:17:40	0|gmaxwell|achow101: thats why it isn't ten lines of code.
386 2017-06-15 19:17:42	0|jonasschnelli|achow101: you can add: -wallet=filename
387 2017-06-15 19:17:56	0|wumpus|achow101: just add an option
388 2017-06-15 19:18:04	0|sipa|s/filename/name/
389 2017-06-15 19:18:10	0|jonasschnelli|endpoints in bitcoin-cli is not really complex...
390 2017-06-15 19:18:11	0|luke-jr|achow101: debug console isn't via RPC anyway
391 2017-06-15 19:18:12	0|jonasschnelli|sipa: yes
392 2017-06-15 19:18:14	0|achow101|but it certainly wouldn't work with debug console
393 2017-06-15 19:18:14	0|gmaxwell|sipa +1
394 2017-06-15 19:18:40	0|gmaxwell|luke-jr: I think we'll need endpoints in any case regardless of auth to set a default.
395 2017-06-15 19:18:40	0|wumpus|as for debug console: you could ask the same question about authentication
396 2017-06-15 19:18:52	0|luke-jr|gmaxwell: maybe
397 2017-06-15 19:18:53	0|jonasschnelli|achow101: the whole GUI has no multiwallet interface
398 2017-06-15 19:18:58	0|gmaxwell|luke-jr: because that will be what you need to make it usable to work with multiple wallets as a single user.
399 2017-06-15 19:19:00	0|wumpus|debug console is not authenticated at all - so adding endpoint/auth support is likely the similar amount of work
400 2017-06-15 19:19:25	0|wumpus|if access control is implemented, a single user could want to have access to multiple wallets anyhow
401 2017-06-15 19:19:26	0|gmaxwell|debug console should perhaps get a dropdown, and yes, it will be the same work either way.. probably easier with endpoints.
402 2017-06-15 19:19:32	0|wumpus|so user=wallet is a bad abstraction
403 2017-06-15 19:19:37	0|luke-jr|I already have the GUI done BTW
404 2017-06-15 19:19:43	0|gmaxwell|yes you'll want to have access to multiple wallets from a single user regardless.
405 2017-06-15 19:19:45	0|luke-jr|it's just based on the RPC branch
406 2017-06-15 19:19:51	0|sipa|luke-jr: how does it let you select the wallet?
407 2017-06-15 19:19:53	0|gmaxwell|luke-jr: how does gui handle the debug console?
408 2017-06-15 19:19:57	0|jonasschnelli|A very hackish (and very old) endpoint impl
409 2017-06-15 19:19:58	0|luke-jr|sipa: comboboxes
410 2017-06-15 19:20:04	0|jonasschnelli|A very hackish (and very old) endpoint impl for bitcoin-cli: https://github.com/jonasschnelli/bitcoin/blob/2015/05/corewallet/src/bitcoin-cli.cpp#L134
411 2017-06-15 19:20:05	0|luke-jr|one in the main window, and one in the debug window
412 2017-06-15 19:20:21	0|gmaxwell|sounds more or less okay.
413 2017-06-15 19:20:57	0|gmaxwell|luke-jr: so why not implement endpoints first?  surely even if your own use needs account you can carry a 5 line patch to allow accounts to select the default wallet.
414 2017-06-15 19:20:57	0|luke-jr|wumpus: even if a single user can access multiple wallets, we still want a way to choose the default
415 2017-06-15 19:21:06	0|gmaxwell|see above
416 2017-06-15 19:21:21	0|luke-jr|gmaxwell: it's more code, and not done yet
417 2017-06-15 19:21:24	0|wumpus|well the default wallet could depend on the user, I don't really care
418 2017-06-15 19:21:32	0|luke-jr|I can implement it, but IMO it will delay things to make it the next step
419 2017-06-15 19:21:33	0|wumpus|though I'd prefer to get rid of 'default wallet', in time
420 2017-06-15 19:21:36	0|jonasschnelli|maybe the GUI should have a node window (network, peers) and a wallet-window per wallet...
421 2017-06-15 19:21:53	0|sipa|jonasschnelli: ugh
422 2017-06-15 19:21:59	0|luke-jr|the user=>wallet stuff is literally done and well-tested (in Knots), just needs to be rebased
423 2017-06-15 19:22:00	0|gmaxwell|jonasschnelli: that doesn't sound like a good UI. :P
424 2017-06-15 19:22:08	0|sipa|jonasschnelli: /me remembers browsers before tabs
425 2017-06-15 19:22:09	0|gmaxwell|at least not mandatory.
426 2017-06-15 19:22:14	0|gmaxwell|what sipa says. :P
427 2017-06-15 19:22:21	0|jonasschnelli|yeah... I like windows.. but I'm pretty alone nowadays with that
428 2017-06-15 19:22:32	0|jonasschnelli|Yeah. Tabs make more sense I guess.
429 2017-06-15 19:22:38	0|sipa|anyway, separate discussion
430 2017-06-15 19:23:05	0|sipa|i would really prefer endpoints or optional named argument to select a wallet, and deal with the authentication question later
431 2017-06-15 19:23:16	0|jonasschnelli|sipa: +1
432 2017-06-15 19:23:17	0|gmaxwell|luke-jr: in any case, seems to me the path forward is to do the endpoints thing, and having auth pick default is a simple change which is either sufficiently non-objectionable or at least a trivial patch to carry.
433 2017-06-15 19:23:22	0|achow101|sipa: ack
434 2017-06-15 19:23:22	0|wumpus|sipa: same for me
435 2017-06-15 19:23:27	0|luke-jr|would anyone NACK if I go forward with user->wallet mappings since they're basically ready, and then do endpoints based on that?
436 2017-06-15 19:24:07	0|achow101|probably
437 2017-06-15 19:24:10	0|gmaxwell|luke-jr: does it also support one user with many wallets?
438 2017-06-15 19:24:13	0|wumpus|well as we determined above, a user may want to have access to multiple wallets, so a single user->wallet mapping just doesn't cut it, even if you want to add access control
439 2017-06-15 19:24:20	0|gmaxwell|what wumpus says.
440 2017-06-15 19:24:29	0|luke-jr|gmaxwell: the current code does not, but there's no reason the endpoints couldn't add that
441 2017-06-15 19:24:30	0|wumpus|I really think we should just start with endpoints as sipa says
442 2017-06-15 19:24:53	0|jnewbery|Isn't rpcuser deprecated anyway?
443 2017-06-15 19:24:55	0|jonasschnelli|Yes. Let's start with endpoint.. I'll can write it next week because I already did once...
444 2017-06-15 19:24:58	0|wumpus|I'm not going to NACK anything that makes progress though
445 2017-06-15 19:24:59	0|luke-jr|jnewbery: it's rpcauth
446 2017-06-15 19:25:00	0|sipa|jnewbery: rpcauth isn't
447 2017-06-15 19:25:02	0|jonasschnelli|*I can
448 2017-06-15 19:25:29	0|gmaxwell|jnewbery: rpcuser is, but this is rpcauth (rpcuser doesn't even have multiple users)
449 2017-06-15 19:25:30	0|jtimon|jnewbery: is rpcuser deprecated? since when?
450 2017-06-15 19:25:38	0|gmaxwell|jtimon: a year?
451 2017-06-15 19:25:39	0|achow101|jtimon: it's deprecated since a long time ago
452 2017-06-15 19:25:43	0|gmaxwell|it prints out a notice!
453 2017-06-15 19:25:51	0|wumpus|rpcuser is deprecated, people are encouraged to use either rpcauth or cookie auth
454 2017-06-15 19:26:00	0|wumpus|we won't remove it just yet ofcourse
455 2017-06-15 19:26:10	0|jtimon|oh, deprecated as in "we want to remove this", but it actually still works, no?
456 2017-06-15 19:26:14	0|luke-jr|right
457 2017-06-15 19:26:14	0|paveljanik|26 minutes...
458 2017-06-15 19:26:16	0|achow101|yes
459 2017-06-15 19:26:18	0|wumpus|that is what deprecated means, yes
460 2017-06-15 19:26:29	0|jtimon|yeah, sorry
461 2017-06-15 19:26:30	0|instagibbs|paveljanik, 34 to go :P
462 2017-06-15 19:26:44	0|wumpus|paveljanik: what's so special about 26?
463 2017-06-15 19:26:49	0|sipa|jonasschnelli: any progress on GUI for database upgrade?
464 2017-06-15 19:26:59	0|wumpus|#topic GUI for database upgrade?
465 2017-06-15 19:27:01	0|gmaxwell|And we've wasted a perfectly good half hour. :P  luke should put up patches and we can yell at him on github, but I think I would really prefer if the first cut does multiple wallets for a user... (if nothing else, that is the easiest thing to test)
466 2017-06-15 19:27:01	0|jonasschnelli|sipa: I sadly had only little time last and this week
467 2017-06-15 19:27:08	0|luke-jr|wumpus: 21 is half of 42
468 2017-06-15 19:27:18	0|jonasschnelli|sipa: I looked into it and wanted to ask you how I get the max size of a db cursor (to calc progress)
469 2017-06-15 19:27:43	0|sipa|jonasschnelli: it's not hard to estimate as txids are randomly distributed
470 2017-06-15 19:27:52	0|gmaxwell|so you just look at the txid...
471 2017-06-15 19:27:54	0|sipa|i can add code for that
472 2017-06-15 19:27:56	0|gmaxwell|they're done in order.
473 2017-06-15 19:28:08	0|gmaxwell|if it's at 0x01... then it's done 1/256 of it.
474 2017-06-15 19:28:12	0|sipa|txid -> arith_uint256 -> * 100/2^256
475 2017-06-15 19:28:30	0|jonasschnelli|Okay. The rest is simple (debug.log non newline [10%] progress   /  GUI splash screen progress with abort)
476 2017-06-15 19:28:39	0|wumpus|BTW: jnewbery rebased the label API pull (#7729), a lot of thanks for that
477 2017-06-15 19:28:41	0|gribble|https://github.com/bitcoin/bitcoin/issues/7729 | rpc: introduce label API for wallet by laanwj · Pull Request #7729 · bitcoin/bitcoin · GitHub
478 2017-06-15 19:28:51	0|sipa|jonasschnelli: i'll write code for the progress estimation
479 2017-06-15 19:29:09	0|jonasschnelli|sipa: Okay. Pass me over a commit and I'll finish the rest
480 2017-06-15 19:29:15	0|jnewbery|wumpus: no problem. I wanted to test drive it :)
481 2017-06-15 19:30:39	0|gmaxwell|wumpus: \O/ on the label rebase.
482 2017-06-15 19:30:54	0|wumpus|must have been a nightmare
483 2017-06-15 19:31:27	0|wumpus|any other topics?
484 2017-06-15 19:31:52	0|jonasschnelli|what about the rpc splitting,... has that been discussed so far?
485 2017-06-15 19:32:05	0|jnewbery|There had been quite a few refactors. I think the rebase was good, but reviewers should look out for anything that looks off
486 2017-06-15 19:32:07	0|jonasschnelli|signrawwithkey, etc.?
487 2017-06-15 19:32:10	0|wumpus|rpc splitting?
488 2017-06-15 19:32:27	0|achow101|the PRs I made to unfuck signrawtx and validateaddress
489 2017-06-15 19:32:43	0|wumpus|ah https://github.com/bitcoin/bitcoin/pull/10583
490 2017-06-15 19:32:48	0|wumpus|#10583
491 2017-06-15 19:32:49	0|gribble|https://github.com/bitcoin/bitcoin/issues/10583 | [RPC] Split part of validateaddress into getaddressinfo by achow101 · Pull Request #10583 · bitcoin/bitcoin · GitHub
492 2017-06-15 19:32:49	0|jonasschnelli|achow101: yes. Those..
493 2017-06-15 19:33:05	0|wumpus|#topic split off wallet functionality from mixed wallet/non-wallet RPC calls
494 2017-06-15 19:33:09	0|wumpus|or something
495 2017-06-15 19:33:23	0|achow101|#10570
496 2017-06-15 19:33:24	0|gribble|https://github.com/bitcoin/bitcoin/issues/10570 | [RPC] Split signrawtransaction into multiple distinct RPCs · Issue #10570 · bitcoin/bitcoin · GitHub
497 2017-06-15 19:33:46	0|wumpus|concept ack (haven't gotten around to reviewing anything)
498 2017-06-15 19:33:48	0|gmaxwell|I think those changes all make sense.  Someone commented about breaking compatibility, but its for a new major version and it will be easy for callers to update their behavior.
499 2017-06-15 19:34:00	0|morcos|i'm here now.. haven't caught up on backlog
500 2017-06-15 19:34:07	0|sipa|it breaks compatibility with a never documented or advertized feature :)
501 2017-06-15 19:34:08	0|wumpus|well we could allow both, for one major version
502 2017-06-15 19:34:11	0|gmaxwell|Though we might want to rename the old calls at the same time. (a suggestion for discussion)
503 2017-06-15 19:34:21	0|sipa|(concatenating multiple tx hex strings, yuck)
504 2017-06-15 19:34:26	0|wumpus|wut?
505 2017-06-15 19:34:32	0|wumpus|ok, we're talking about different things
506 2017-06-15 19:34:38	0|gmaxwell|sipa: it was certantly advertised and known.
507 2017-06-15 19:34:38	0|instagibbs|aside from mixing wallet/nonwallet, what's the issue with validateaddress?
508 2017-06-15 19:34:41	0|wumpus|I mean validateaddress/getaddressinfo
509 2017-06-15 19:34:43	0|instagibbs|(or is that the issue)
510 2017-06-15 19:34:50	0|achow101|instagibbs: that's the issue
511 2017-06-15 19:34:51	0|sipa|wumpus: ah, i'm talking about signrawtransaction
512 2017-06-15 19:34:52	0|gmaxwell|wumpus: he's talking about the signraw split to create the combine call.
513 2017-06-15 19:34:52	0|wumpus|instagibbs: that is the issue
514 2017-06-15 19:34:58	0|instagibbs|ok, are we killing off getinfo then too:
515 2017-06-15 19:35:00	0|instagibbs|:)
516 2017-06-15 19:35:06	0|achow101|hopefully :D
517 2017-06-15 19:35:11	0|wumpus|instagibbs: yes, but that's not the topic now
518 2017-06-15 19:35:17	0|gmaxwell|I would miss getinfo. all the other commands take more typing. :P
519 2017-06-15 19:35:18	0|wumpus|we're already confusing two things, let's add more!
520 2017-06-15 19:35:26	0|gmaxwell|wumpus: okay!
521 2017-06-15 19:35:26	0|instagibbs|eh, we're talking about blowing away validateaddress as is, sorry
522 2017-06-15 19:35:26	0|wumpus|gmaxwell: hey I have a pull that implements it client side
523 2017-06-15 19:35:47	0|gmaxwell|wumpus: lol.  can you name the rpc call "gi" :P even less typing.
524 2017-06-15 19:35:48	0|jonasschnelli|IMO having a non wallet sign rawtx where priv keys are passed aroung in a shell over a possible TCP channel is not ideal.. but we already have it... I though instead of splitting it off, move it to the tool
525 2017-06-15 19:35:56	0|jonasschnelli|But I see the point with getting the UTXOs
526 2017-06-15 19:36:02	0|sipa|let's rename all RPCs to get*info... for example s/sendtoaddress/getnewpaymenttxid/
527 2017-06-15 19:36:09	0|gmaxwell|jonasschnelli: we need the UTXOs or its all awfulsauce.
528 2017-06-15 19:36:12	0|instagibbs|sipa, lol
529 2017-06-15 19:36:17	0|wumpus|#8843
530 2017-06-15 19:36:18	0|gribble|https://github.com/bitcoin/bitcoin/issues/8843 | rpc: Handle `getinfo` client-side in bitcoin-cli w/ `-getinfo` by laanwj · Pull Request #8843 · bitcoin/bitcoin · GitHub
531 2017-06-15 19:36:19	0|luke-jr|lol
532 2017-06-15 19:36:29	0|luke-jr|getsignedtransaction
533 2017-06-15 19:36:32	0|jonasschnelli|gmaxwell: I though the node RPC can spitout what you need to pass it into bitcoin-tx or so...
534 2017-06-15 19:36:48	0|jonasschnelli|but I know... very inconvinient
535 2017-06-15 19:36:49	0|luke-jr|getthistransactionbroadcast
536 2017-06-15 19:36:50	0|luke-jr|:p
537 2017-06-15 19:36:52	0|sipa|jonasschnelli: sure it can; it's just more convenient to not need that
538 2017-06-15 19:36:52	0|wumpus|eventually closed it because the only person responding was luke-jr and he kept arguing against it
539 2017-06-15 19:37:09	0|gmaxwell|jonasschnelli: and that adds steps to the process.. which is already long enough that it's prone to error.
540 2017-06-15 19:37:14	0|sipa|jonasschnelli: it's called listunspent
541 2017-06-15 19:37:17	0|jonasschnelli|It's just another source how people can shoot themselfs with exposing priv keys
542 2017-06-15 19:37:29	0|gmaxwell|luke-jr: why do you hate freedom?
543 2017-06-15 19:37:33	0|luke-jr|gmaxwell: !⁈
544 2017-06-15 19:37:36	0|gmaxwell|haha
545 2017-06-15 19:38:30	0|sipa|jonasschnelli: but the functionality already exists, and i very much like removing it from the wallet (so people at least won't accidentally mix up privkey based operations with wallet stuff)
546 2017-06-15 19:38:54	0|instagibbs|sipa, ok I see the motivation there
547 2017-06-15 19:38:59	0|gmaxwell|anyways, I can just say that I have tried to stop using getinfo and failed. Mostly because typing getnetworkinfo getblockchainfo getfooooooooooinfo and then wading through a bunch of things when I want to see: How many connections, which block am I at, and what wallet am I running (which I can tell via the balance). :P  just personal feedback.
548 2017-06-15 19:39:01	0|luke-jr|gmaxwell: I didn't even NACK it :o
549 2017-06-15 19:39:05	0|jonasschnelli|Yes... I guess that makes sense. I kinda hoped once we touch that we could move it away from the node into a sep. process
550 2017-06-15 19:39:22	0|luke-jr|gmaxwell: use the GUI for that! :p
551 2017-06-15 19:39:32	0|sipa|jonasschnelli: i have a vague proposal for that too, but it's more complicated
552 2017-06-15 19:39:38	0|gmaxwell|luke-jr: again, why do you hate freedom? :P
553 2017-06-15 19:39:51	0|sipa|jonasschnelli: and involves a new format for partially signed transactions...
554 2017-06-15 19:39:58	0|wumpus|gmaxwell: anyhow we can easily reopen and rebase that PR, bitcoin-cli hardly changed since then
555 2017-06-15 19:40:02	0|sipa|wumpus: ack
556 2017-06-15 19:40:04	0|jonasschnelli|sipa: that contains everything you need to sign?
557 2017-06-15 19:40:09	0|luke-jr|gmaxwell: make a shell alias to all the calls ;)
558 2017-06-15 19:40:23	0|gmaxwell|it's so much easier to have a signing blob thing post segwit. :(
559 2017-06-15 19:40:38	0|bitcoin-git|[13bitcoin] 15laanwj reopened pull request #8843: rpc: Handle `getinfo` client-side in bitcoin-cli w/ `-getinfo` (06master...062016_09_getinfo_clientside) 02https://github.com/bitcoin/bitcoin/pull/8843
560 2017-06-15 19:40:39	0|sipa|jonasschnelli: yes, contains amounts, change info, prevouts being spent, ...
561 2017-06-15 19:40:40	0|wumpus|yes, let's activate segwit
562 2017-06-15 19:40:46	0|gmaxwell|luke-jr: I don't like customizing my expirence of bitcoin too much because then I'll just patch around everything that stinks.
563 2017-06-15 19:41:10	0|jonasschnelli|sipa: That's also something we could re-use for the detatched signing standard (a.k.a hardware wallet standard)
564 2017-06-15 19:41:12	0|luke-jr|we could just leave getinfo how it is ;)
565 2017-06-15 19:41:27	0|gmaxwell|in any case, we're offtopic. I think that achow's PRs are all nice incremental improvements and we should take them (after review)
566 2017-06-15 19:41:29	0|sipa|jonasschnelli: pre-segwit however, it also needs to contain the full spent transactions :(
567 2017-06-15 19:41:34	0|luke-jr|bitcoin-cli -getinfo only handles 1 sortof-use-case, and leaves the other 2 supported use cases unaddressed
568 2017-06-15 19:41:35	0|sipa|gmaxwell: agree
569 2017-06-15 19:41:38	0|wumpus|after promising to deprecate it for years... yeah, of course....
570 2017-06-15 19:41:53	0|sipa|(disclaimer: achow101'w my intern this summer, i asked him to work on those)
571 2017-06-15 19:42:02	0|jonasschnelli|sipa: [full spent transactions], I guess that's okay.
572 2017-06-15 19:42:26	0|sipa|jonasschnelli: bitcoind unfortunately can't do that generically (you need the wallet for that)
573 2017-06-15 19:42:26	0|wumpus|getinfo is going away, there's no going back now
574 2017-06-15 19:42:45	0|luke-jr|let's add a getallinfo then
575 2017-06-15 19:42:46	0|luke-jr|/s
576 2017-06-15 19:42:47	0|wumpus|I've exactly documented what information you can find on what get*info command
577 2017-06-15 19:42:55	0|gmaxwell|I am fine with it going away, but I don't believe we replaced it as well as we thought we did.
578 2017-06-15 19:43:02	0|sipa|i have no problem with removing it, with or without 8843
579 2017-06-15 19:43:02	0|wumpus|and the client-side getinfo is ther for user friendlyness, if people want it
580 2017-06-15 19:43:16	0|luke-jr|wumpus: it doesn't work in the debug window
581 2017-06-15 19:43:17	0|sipa|i'm sure i'll curse a bit that getinfo isn't around anymore, and then change my habits
582 2017-06-15 19:43:35	0|gmaxwell|sipa: I tried blocking it, you won't. the replacements right now are not usable.
583 2017-06-15 19:43:43	0|luke-jr|:/
584 2017-06-15 19:43:51	0|gmaxwell|But thats okay, wumpus suggestion would be fine, though luke has a point about the debug console.
585 2017-06-15 19:43:54	0|wumpus|luke-jr: isn't all the information in the debug window *without* typing anything?
586 2017-06-15 19:43:58	0|achow101|removing getinfo will mess with a ton of things that use getinfo for basic rpc connection checking too...
587 2017-06-15 19:44:12	0|wumpus|achow101: and you're starting to bring that up *now*?
588 2017-06-15 19:44:13	0|achow101|but I think it should be removed anyways
589 2017-06-15 19:44:13	0|sipa|let's merge the getuptime rpc thing
590 2017-06-15 19:44:14	0|gmaxwell|wumpus: if it isn't we should make it. problem solved.
591 2017-06-15 19:44:22	0|bitcoin-git|[13bitcoin] 15ryanofsky opened pull request #10605: Add AssertLockHeld assertions in CWallet::ListCoins (06master...06pr/listlock) 02https://github.com/bitcoin/bitcoin/pull/10605
592 2017-06-15 19:44:23	0|instagibbs|achow101, move to dumpprivkey obv
593 2017-06-15 19:44:34	0|wumpus|gmaxwell: the first tab of the debug window pretty much shows everything, and indeed, if it isn't it could be added
594 2017-06-15 19:44:50	0|sipa|oh, another topic: non-hardened key derivation
595 2017-06-15 19:44:52	0|achow101|wumpus: well most of those things are old website scripts that were written once and never touched again by the authors
596 2017-06-15 19:44:55	0|gmaxwell|luke-jr: I think ^ is how we should handle the gui.  (also important to make it copy/pasteable if it isn't.)
597 2017-06-15 19:44:55	0|instagibbs|sipa, ACK
598 2017-06-15 19:45:01	0|wumpus|I'm really disappointed that years after deciding to deprecate getinfo we're still having this discussion
599 2017-06-15 19:45:13	0|luke-jr|wumpus: lol maybe :D
600 2017-06-15 19:45:15	0|wumpus|anyhow next topic
601 2017-06-15 19:45:21	0|gmaxwell|wumpus: sometimes you have to try things out to know their effects completely!
602 2017-06-15 19:45:31	0|wumpus|#topic non-hardened key derivation
603 2017-06-15 19:45:33	0|sipa|so
604 2017-06-15 19:45:54	0|sipa|non-hardened key derivation has many use cases in addition to hardened
605 2017-06-15 19:46:03	0|jonasschnelli|I guess NicolasDorier made good work there
606 2017-06-15 19:46:08	0|wumpus|achow101: we should carefully note it in the release notes of course
607 2017-06-15 19:46:13	0|sipa|however, they also have a gaping wide security hole when child private keys are exposed
608 2017-06-15 19:46:24	0|wumpus|achow101: we could even make getinfo fail with a custom message
609 2017-06-15 19:46:30	0|instagibbs|that plus dumpwallet will give you sads
610 2017-06-15 19:46:31	0|sipa|thus, suggestion: allow a new wallet to be created with either harderned or unhardened keys
611 2017-06-15 19:46:41	0|sipa|when you choose unhardened, dumpprivkey is disabled
612 2017-06-15 19:46:45	0|instagibbs|xpub is only accessible through dumpwallet right now AFAIK
613 2017-06-15 19:46:46	0|achow101|wumpus: returning null would probably not mess with anything
614 2017-06-15 19:46:50	0|sipa|(but dumpmasterkey or whatever is still available)
615 2017-06-15 19:47:12	0|achow101|sipa: there is no dumpmasterkey
616 2017-06-15 19:47:12	0|jonasschnelli|instagibbs: dumpprivkey must be disabled anyways
617 2017-06-15 19:47:19	0|luke-jr|sipa: I'd want to be able to mix them..
618 2017-06-15 19:47:23	0|jonasschnelli|achow101: dumpmasterkey must be added
619 2017-06-15 19:47:31	0|gmaxwell|this is a lot more interesting with multiwallet support in place, since the cases where you want that are mostly secondary wallets (like incoming payments with keys generated by your webserver)
620 2017-06-15 19:47:31	0|instagibbs|luke-jr, .... why
621 2017-06-15 19:47:38	0|wumpus|dumpmasterkey, isn't there a PR for that?
622 2017-06-15 19:47:42	0|jonasschnelli|no
623 2017-06-15 19:47:48	0|luke-jr|instagibbs: to generate reusable payment tokens
624 2017-06-15 19:47:50	0|sipa|luke-jr: i guess that's fine; just disable dumpwallet, and dumpprivkey selectively for keys derived in a non-hardened fashion
625 2017-06-15 19:47:52	0|gmaxwell|sipa said or whatever for a reason. :P
626 2017-06-15 19:47:55	0|achow101|jonasschnelli: I'm stil lwaiting #9504
627 2017-06-15 19:47:56	0|gribble|https://github.com/bitcoin/bitcoin/issues/9504 | [RPC] dumpmasterprivkey command by achow101 · Pull Request #9504 · bitcoin/bitcoin · GitHub
628 2017-06-15 19:48:05	0|jonasschnelli|oh...
629 2017-06-15 19:48:06	0|luke-jr|sipa: sgtm
630 2017-06-15 19:48:10	0|wumpus|see, I wasn't crazy
631 2017-06-15 19:48:26	0|gmaxwell|wumpus: well, technically it doesn't prove that...
632 2017-06-15 19:48:27	0|jonasschnelli|achow101: how can I not be aware of that PR... sorry for the missinfo
633 2017-06-15 19:48:35	0|wumpus|gmaxwell: true
634 2017-06-15 19:48:37	0|sipa|getmissinfo
635 2017-06-15 19:48:48	0|jonasschnelli|:/
636 2017-06-15 19:48:57	0|jonasschnelli|(I even commited on the PR ^^)
637 2017-06-15 19:49:00	0|jonasschnelli|commented
638 2017-06-15 19:49:16	0|wumpus|getmisinfo would really fit with the spirit of the times
639 2017-06-15 19:49:20	0|gmaxwell|Anyways, I think that sounds okay. These applications will likely need a couple of extra RPCs too. no need to design here however.  (e.g. it will need to export the extended public key.)
640 2017-06-15 19:49:34	0|sipa|not much more to say about the topic - just pointing out that if we disable dumping child private keys, my concern with non-hardered derivation largely goes away
641 2017-06-15 19:49:48	0|wumpus|anyhow, no problem with non-hardened key support
642 2017-06-15 19:49:59	0|jonasschnelli|I guess with allowing xpub derivation, flexible keypath would be welcome..
643 2017-06-15 19:50:05	0|jonasschnelli|People want to use BIP44
644 2017-06-15 19:50:07	0|wumpus|as an option, not as default
645 2017-06-15 19:50:09	0|gmaxwell|w/ disabling and it not being a default thing. sounds great to me.
646 2017-06-15 19:50:25	0|achow101|sgtm
647 2017-06-15 19:50:51	0|jonasschnelli|Yes. Would be a great change...
648 2017-06-15 19:51:00	0|gmaxwell|also so long as we do the extra rpcs to make it actually useful (like extract the extended public keys, and whatever else is needed to handle an external address generator.)
649 2017-06-15 19:51:13	0|jonasschnelli|I guess keypool handling would be much simpler with xpub derivation
650 2017-06-15 19:51:22	0|gmaxwell|I doubt it?
651 2017-06-15 19:51:31	0|instagibbs|I think it's the same-ish
652 2017-06-15 19:51:34	0|jonasschnelli|Why would you need a keypool with xpub derivation?
653 2017-06-15 19:51:42	0|gmaxwell|for scanning.
654 2017-06-15 19:51:42	0|jonasschnelli|you derive when you need
655 2017-06-15 19:51:50	0|instagibbs|you have master seed already, you can already do that
656 2017-06-15 19:51:50	0|jonasschnelli|you always derive the lookup window in mem
657 2017-06-15 19:51:54	0|gmaxwell|you need to scan forward to know when you're paid.
658 2017-06-15 19:52:06	0|jonasschnelli|+1000 keys in mem should be ackish
659 2017-06-15 19:52:18	0|jonasschnelli|(pubkeys)
660 2017-06-15 19:52:26	0|gmaxwell|yes, the keypool doesn't technically need to be saved on disk, though it may be faster to do so then to generate thousands of addresses at every wallet load.
661 2017-06-15 19:52:40	0|gmaxwell|but otherwise I think its the same.
662 2017-06-15 19:52:57	0|jonasschnelli|I'd say loading the keypool at wallet load takes almost the same (or longer)
663 2017-06-15 19:53:28	0|jtimon|sipa: why would people want to dump child private keys?
664 2017-06-15 19:53:33	0|gmaxwell|okay, this would make the change much more intrusive then I think.
665 2017-06-15 19:53:40	0|instagibbs|jtimon, you want to stop them from doing so
666 2017-06-15 19:53:48	0|gmaxwell|jtimon: same reason some people eat paste.  (they don't know better)
667 2017-06-15 19:53:53	0|jonasschnelli|heh
668 2017-06-15 19:53:57	0|sipa|jonasschnelli: the reason creating new keys is slow, is because we flush them all individually to disk
669 2017-06-15 19:54:11	0|wumpus|yes...
670 2017-06-15 19:54:18	0|jtimon|gmaxwell: I see
671 2017-06-15 19:54:22	0|jonasschnelli|Yes. In-mem non-bdb "keypool" would be much faster
672 2017-06-15 19:54:42	0|gmaxwell|that flushing is not required anymore with hdwallets regardless, I believe.
673 2017-06-15 19:54:46	0|sipa|gmaxwell: exactly
674 2017-06-15 19:55:01	0|jonasschnelli|With xpub derivation, would there be really a need to write the pool to disk? I doubt it
675 2017-06-15 19:55:08	0|gmaxwell|(we will still need a flush to know how many we've given out...)
676 2017-06-15 19:55:09	0|wumpus|gmaxwell: good point!
677 2017-06-15 19:55:29	0|jonasschnelli|All you need is the seed
678 2017-06-15 19:55:44	0|gmaxwell|jonasschnelli: no, not a need, though why result in more distinct codepaths?  I would expect it to also make loading faster.
679 2017-06-15 19:55:48	0|achow101|you just need to write the path of the most recent key
680 2017-06-15 19:55:49	0|jonasschnelli|gmaxwell: yes. thats a good point
681 2017-06-15 19:56:12	0|jonasschnelli|achow101: Yes.
682 2017-06-15 19:56:15	0|gmaxwell|achow101: yes, every new address still needs to do a wallet flush.  But on bulk key creation we do not need a flush for each operation.
683 2017-06-15 19:56:39	0|gmaxwell|e.g. starting up for the first time with a keypoool of 10000.
684 2017-06-15 19:58:26	0|wumpus|two minutes to go
685 2017-06-15 19:58:34	0|gmaxwell|jonasschnelli: did you ever get around to implementing the change where any key that is noticed used on the blockchain marks all the earlier keys in its chain as used?
686 2017-06-15 19:58:52	0|jonasschnelli|gmaxwell: HD restore?
687 2017-06-15 19:59:13	0|jonasschnelli|https://github.com/bitcoin/bitcoin/pull/10240
688 2017-06-15 19:59:16	0|gmaxwell|it's part of your restore pr? okay.
689 2017-06-15 19:59:26	0|jonasschnelli|That makes all keys as used up to the one found
690 2017-06-15 19:59:31	0|gmaxwell|great.
691 2017-06-15 19:59:35	0|jonasschnelli|It can even temp. halt the sync if you prune, etc.
692 2017-06-15 19:59:40	0|jonasschnelli|Needs overhaul and rebase...
693 2017-06-15 19:59:47	0|jonasschnelli|It's already (too) big
694 2017-06-15 19:59:55	0|jonasschnelli|ryanofsky gave me a hard time
695 2017-06-15 20:00:01	0|sipa|#getmeetingendinfo
696 2017-06-15 20:00:13	0|lightningbot|Log:            http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-06-15-19.00.log.html
697 2017-06-15 20:00:13	0|lightningbot|Meeting ended Thu Jun 15 20:00:12 2017 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
698 2017-06-15 20:00:13	0|lightningbot|Minutes:        http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-06-15-19.00.html
699 2017-06-15 20:00:13	0|lightningbot|Minutes (text): http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-06-15-19.00.txt
700 2017-06-15 20:00:13	0|wumpus|#endmeeting
701 2017-06-15 20:07:35	0|gmaxwell|wumpus: sorry to bog you with the getinfo stuff. :(
702 2017-06-15 20:08:56	0|gribble|https://github.com/bitcoin/bitcoin/issues/8843 | rpc: Handle `getinfo` client-side in bitcoin-cli w/ `-getinfo` by laanwj · Pull Request #8843 · bitcoin/bitcoin · GitHub
703 2017-06-15 20:08:56	0|wumpus|gmaxwell: yes, no problem, don't mind resurrecting #8843
704 2017-06-15 20:09:08	0|gmaxwell|I think the main challenges for it in an interactive setting is that the replacements spam the screen with information which is not what I'm interested in, and it takes about three commands to check on them.  Thats not reason to not get rid of getinfo at all, and I support getting rid of it, but think we need something like your PR.  I'll give it a look.
705 2017-06-15 20:09:08	0|wumpus|gmaxwell: just don't want to talk with luke-jr about it :p
706 2017-06-15 20:09:13	0|gmaxwell|haha
707 2017-06-15 20:09:22	0|gmaxwell|Well, he does hate freedom, after all.
708 2017-06-15 20:09:54	0|sipa|gmaxwell: perhaps you should write a getmyinfo.sh with just queries for the things you're typically interested in :)
709 2017-06-15 20:09:59	0|wumpus|I mean the reason for deprecating it is that we didn't want to add any more info to it, also it combines wallet and non-wallet information which doesn't work with multiwallet
710 2017-06-15 20:10:36	0|wumpus|sipa: I have tons of those scripts
711 2017-06-15 20:10:48	0|wumpus|e.g. to show # connections per interface in/out
712 2017-06-15 20:11:05	0|wumpus|getinfo's output isn't *that* useful
713 2017-06-15 20:11:06	0|luke-jr|maybe -getinfo should take a -getinfodata which can go in the config file, to tell it what data to fetch from what RPCs ;)
714 2017-06-15 20:11:19	0|wumpus|luke-jr: well in that case I'd prefer just a python script
715 2017-06-15 20:11:52	0|ProfMac|What git version introduced the file bitcoin/contrib/gitian-build.sh ?
716 2017-06-15 20:12:06	0|wumpus|oh wait I'm talking to luke-jr about getinfo
717 2017-06-15 20:12:14	0|luke-jr|:<
718 2017-06-15 20:12:40	0|morcos|at the risk of irritating everyone, i use getinfo all the time
719 2017-06-15 20:13:16	0|gmaxwell|sipa: so then I'm happy while typical users suffer, I've tried to avoid doing that.
720 2017-06-15 20:13:19	0|achow101|ProfMac: check the history in github
721 2017-06-15 20:13:24	0|ProfMac|morcos, yep, me too.
722 2017-06-15 20:13:39	0|gmaxwell|same reason I don't use the ncurses thing that I like.
723 2017-06-15 20:13:47	0|luke-jr|gmaxwell: throw it in contrib/?
724 2017-06-15 20:13:59	0|wumpus|e.g. scripts like this work, it's not a work of art but meh https://gist.github.com/laanwj/27241686db0db712cd94f692a491ad78
725 2017-06-15 20:14:47	0|wumpus|typical users may have very different wants, though
726 2017-06-15 20:14:48	0|ProfMac|achow101, I've been in Google, github, for days.  Thought someone would just actually answer the question.
727 2017-06-15 20:14:58	0|instagibbs|I've been weaning myself off of getinfo
728 2017-06-15 20:15:18	0|wumpus|in any case, I'm all for keeping -getinfo client-side in bitcoin-cli, I'm rebasing 8843
729 2017-06-15 20:15:44	0|achow101|ProfMac: it was introduced in this commit: https://github.com/bitcoin/bitcoin/commit/eda4cfb992d140c8471f6cc2bf16c4a106439225
730 2017-06-15 20:16:09	0|Chris_Stewart_5|Why don't we return fee estimation information in sat/byte?
731 2017-06-15 20:16:24	0|ProfMac|Thanks, achow101
732 2017-06-15 20:16:48	0|jonasschnelli|gmaxwell: Yes. 20 is to little...
733 2017-06-15 20:16:58	0|instagibbs|ProfMac, you can always `git log -p -M --follow -- file/path` to track those kind of questions
734 2017-06-15 20:16:59	0|jonasschnelli|I guess I took BIP44 for a start.. but it's a single const
735 2017-06-15 20:17:07	0|instagibbs|Chris_Stewart_5, because legacy, mostly
736 2017-06-15 20:17:32	0|wumpus|instagibbs: hah, 99% of the time that's the right answer to any 'why' question
737 2017-06-15 20:18:43	0|Chris_Stewart_5|I suppose that is hard to change too for fear of people paying large fees if they don't get the memo
738 2017-06-15 20:20:38	0|wumpus|once a custom is ingrained, it usually turns out the cost and risk of change doesn't weigh up against any possible benefits
739 2017-06-15 20:21:13	0|ProfMac|Thanks instagibbs.  I'm just spinning up to speed with git.  I made a deterministic vm description and a Ubuntu preseed file de novo and am now making a local git repository, then spinning up the deterministic bitcoin build.  Lots of coffee and new thought patterns.
740 2017-06-15 20:21:16	0|instagibbs|Any new interface certainly should be sat/byte
741 2017-06-15 20:21:46	0|Chris_Stewart_5|^
742 2017-06-15 20:21:57	0|wumpus|and then you have multiple different conventions on different RPC calls
743 2017-06-15 20:24:59	0|sipa|nanobitcoins per Hart
744 2017-06-15 20:25:16	0|wumpus|some of the output of getinfo is also blatantly deceptive by now, e.g. it shows one proxy, while every network can have its own proxy
745 2017-06-15 20:25:28	0|sipa|do we still show hashrate?
746 2017-06-15 20:25:47	0|wumpus|other things are only marginally useful; how many times have you wanted to know the the wallet version?
747 2017-06-15 20:26:08	0|wumpus|or "testnet":false, which is also false for regtest
748 2017-06-15 20:26:30	0|wumpus|sipa: no, that seems to be gone :
749 2017-06-15 20:27:11	0|instagibbs|new interface meaning RPC2.0 or whatever... not new calls
750 2017-06-15 20:27:59	0|gmaxwell|To be clear I am joking about luke hating freedom.
751 2017-06-15 20:28:59	0|wumpus|oh, I thought you were serious!
752 2017-06-15 20:29:13	0|cfields|jonasschnelli / gmaxwell / sipa: https://pastebin.com/raw/eKhEiJR3
753 2017-06-15 20:29:21	0|wumpus|https://github.com/bitcoin/bitcoin/pull/8843 rebased
754 2017-06-15 20:29:38	0|jonasschnelli|cfields: ahh.. that look like a beautiful girl
755 2017-06-15 20:29:57	0|cfields|:)
756 2017-06-15 20:29:59	0|jonasschnelli|cfields: I expected a code diff at the bottom.... :)
757 2017-06-15 20:30:04	0|cfields|guess that's what you had in mind, then?
758 2017-06-15 20:30:09	0|cfields|heh, 1 sec
759 2017-06-15 20:30:20	0|jonasschnelli|Yes. Have you tried to run it with the GUI?
760 2017-06-15 20:30:21	0|cfields|it's per-thread/per-mutex/per-lock
761 2017-06-15 20:30:32	0|cfields|no, just simple tests so far
762 2017-06-15 20:31:19	0|cfields|ok, i'll push to a branch
763 2017-06-15 20:31:19	0|cfields|other than that, seems to work as expected
764 2017-06-15 20:31:19	0|cfields|something about threadnames are busted, and it's not sorted yet. and the only threshold for display is that the time is non-zero
765 2017-06-15 20:31:19	0|jonasschnelli|To bed my wife calls me to bed.. I'll play with the code soon
766 2017-06-15 20:31:26	0|jonasschnelli|thanks
767 2017-06-15 20:31:36	0|cfields|np, nnite
768 2017-06-15 20:31:48	0|wumpus|cfields: nice!
769 2017-06-15 20:34:21	0|cfields|https://github.com/theuni/bitcoin/commit/be49a294a240ec81a901af1aaabbba2172d38dc1
770 2017-06-15 20:35:07	0|cfields|jonasschnelli: for tomorrow ^^
771 2017-06-15 20:48:35	0|cfields|er, that should s/locked/contended/ everywhere. it's showing any non-zero time taken to grab a lock.
772 2017-06-15 20:58:30	0|bitcoin-git|[13bitcoin] 15benma opened pull request #10607: scripted-diff: stop using the gArgs wrappers (06master...06gargs_wrappers) 02https://github.com/bitcoin/bitcoin/pull/10607
773 2017-06-15 21:20:03	0|ProfMac|pastebin or google docs?
774 2017-06-15 21:36:29	0|ProfMac|I am stuck trying to do a gitian build.  I have made a Pastebin  https://pastebin.com/eXPaQsLf