1 2017-04-13 00:17:45	0|jtimon|sipa: I'm getting some errors related to prevector templating, perhaps you can help with https://github.com/bitcoin/bitcoin/pull/10193#issuecomment-293742166
  2 2017-04-13 00:32:33	0|sdaftuar|gmaxwell: is this what you have in mind? https://github.com/sdaftuar/bitcoin/tree/2017-04-braindead-simple-mining
  3 2017-04-13 00:33:00	0|sdaftuar|i'm happy to simulate it for you on the same data set i've been using, just tell me what parameters you'd like to see
  4 2017-04-13 00:33:27	0|sdaftuar|i'm not a fan of the model though
  5 2017-04-13 01:19:08	0|gmaxwell|sdaftuar: 10 seconds and .0005 BTC should be fine for a test run.
  6 2017-04-13 01:22:47	0|gmaxwell|sdaftuar: I would gladly agree that it's a hack. But -- it's surely a simple one, if there is nothing else that could be said about it..  CreateNewBlock performance and simplicity counts for a lot.
  7 2017-04-13 01:24:56	0|gmaxwell|sdaftuar: also because of how BIP152 works it is still preferable to have less new data. E.g. having a binary "contains new stuff" vs "not" isn't ideal--- because once prefilling is in use up to 10kb will be prefilled and if there is more than that, we're likely to prefill noting (under the assumption that the block is too inconsistent to be salvagable)
  8 2017-04-13 01:25:21	0|sdaftuar|gmaxwell: that seems gameable though?  if i just rbf a transaction every 10 seconds with fee 50000, 50000+incremental_relay_fee, ... you'll always produce a block with a recent transaction, and i'll give up a negligible amount of money?
  9 2017-04-13 01:26:54	0|gmaxwell|sdaftuar: 10 seconds is a non-trivial amount of time to allow propagation however.
 10 2017-04-13 01:27:28	0|sdaftuar|gmaxwell: i agree about the prefilling concern... that was part of the reason i was skeptical of this approach at all, but i figured that for now, "needs a roundtrip" is pretty different from not needing a roundtrip, so might as well capture that.
 11 2017-04-13 01:28:10	0|sdaftuar|even if i rbf every 5 seconds, its a trivial amount of money
 12 2017-04-13 01:29:44	0|gmaxwell|I did calculations before that were hand-wave-handwave about the cost of orphaning relative to the subsidy based on block propagation observations and concluded a pretty low fee pays for the risk.
 13 2017-04-13 01:30:10	0|gmaxwell|(mentioned somewhere in the logs here)
 14 2017-04-13 01:31:17	0|sdaftuar|yeah i have no idea what the actual stale rates people see are
 15 2017-04-13 01:32:08	0|sdaftuar|(and really, my model requires converting two numbers into 1 -- stale rate without recent txs/stale rate with recent txs -> my parameter.  not a difficult calculation, but requires understanding
 16 2017-04-13 01:32:47	0|sdaftuar|for clarity:  that "/" above is not meant to be division
 17 2017-04-13 01:34:07	0|gmaxwell|well my figuring didn't use stale rates but measurements of delays between stratum updates at pools as a function of blocksize, so I was able to come up with a constant + factor*size model for orphaning risk.  This was before BIP152-- and I think we can't really measure it anymore, but I figure the before bip152 figures give a baseline for the marginal impact. my model basically assumed the comp
 18 2017-04-13 01:34:13	0|gmaxwell|etition was sending an empty block.
 19 2017-04-13 01:51:14	0|sdaftuar|gmaxwell: for the 0.14.1 release notes, did you want to specifically mention that the 0.14.0 defaults could result in the utxo cache using 1.2GB of memory due to mempool sharing?
 20 2017-04-13 01:52:14	0|sdaftuar|i just pushed an update to #10185 but didn't call out that fact specifically, let me know if you'd like me to add it
 21 2017-04-13 01:52:15	0|gribble|https://github.com/bitcoin/bitcoin/issues/10185 | [0.14] Mention dbcache memory changes in release notes by sdaftuar · Pull Request #10185 · bitcoin/bitcoin · GitHub
 22 2017-04-13 01:52:40	0|gmaxwell|sdaftuar: I don't remember now, I don't think I wanted to do that. did I?  uh. I don't think we should bother mentioning the 0.14.0 usage. Did you mention it before but only say it could use 600 or something?
 23 2017-04-13 01:53:16	0|sdaftuar|gmaxwell: i just explained that the 300MiB default could result in 600MiB of usage, so that people would know to double their values if they wanted to, say, keep the whole thing cached still
 24 2017-04-13 01:53:42	0|sdaftuar|ah, i guess my language is potentially confusing
 25 2017-04-13 01:55:06	0|gmaxwell|sdaftuar: but with the mempool the defaults could be 1200 which is what I was trying to express.. the dbcache just get doubled, the mempool did too.
 26 2017-04-13 01:55:12	0|sdaftuar|right
 27 2017-04-13 01:55:21	0|sdaftuar|all i wanted to explain was "multiply by 2!"
 28 2017-04-13 01:55:28	0|sdaftuar|maybe just delete my parenthetical?
 29 2017-04-13 01:58:48	0|sdaftuar|ok, got rid of it -- hopefully the sentence about only accounting for half the peak utilization will be enough for people to figure out what they want to do.
 30 2017-04-13 08:29:34	0|wumpus|"oh dear god lets not default to running random peoples' commitmsg scripts on wumpus' computer" yes that's not going to happen, I'm already paranoid with github-merge.py itself, I use a version outside the repository and only update it if I carefully reviewed changes inside the repo
 31 2017-04-13 08:31:02	0|gmaxwell|lol, please only on pulltester which is already vulnerable as heck.
 32 2017-04-13 08:31:15	0|MarcoFalke|The script could be run in a vm. Though, it comes with an overhead...
 33 2017-04-13 08:31:45	0|gmaxwell|There are a LOT of vm escape exploits.
 34 2017-04-13 08:32:18	0|wumpus|in any case I don't like the idea of adding more functionality to github-merge.py, please just keep it a merge script and not add any other hooks
 35 2017-04-13 08:32:20	0|MarcoFalke|Running it on travis is not enough. travis is just an indicator that should not be trusted
 36 2017-04-13 08:32:28	0|sipa|i doubt you can trigger any of them from a shell script in a way that it's obvious to cursory review, though
 37 2017-04-13 08:32:38	0|gmaxwell|most recent one in QEMU was awful, with the instruction decoder. Esp since previously I'd intentionally avoided kvm to try to lower risk but that made it trivially exploitable.
 38 2017-04-13 08:32:53	0|MarcoFalke|Just need to make sure the script is reviewed on every (ut)ACK
 39 2017-04-13 08:33:08	0|gmaxwell|sipa: doubt it enough to put a bounty on it though? :P
 40 2017-04-13 08:50:36	0|bitcoin-git|[13bitcoin] 15laanwj pushed 3 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/de01da7cad32...c9ff4f8ee601
 41 2017-04-13 08:50:37	0|bitcoin-git|13bitcoin/06master 14714e4ad 15John Newbery: AddToWalletIfInvolvingMe should test pIndex, not posInBlock
 42 2017-04-13 08:50:37	0|bitcoin-git|13bitcoin/06master 14d0cd0bd 15John Newbery: Make CWallet::SyncTransactions() interface friendlier
 43 2017-04-13 08:50:38	0|bitcoin-git|13bitcoin/06master 14c9ff4f8 15Wladimir J. van der Laan: Merge #10186: Remove SYNC_TRANSACTION_NOT_IN_BLOCK magic number...
 44 2017-04-13 08:51:04	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #10186: Remove SYNC_TRANSACTION_NOT_IN_BLOCK magic number (06master...06remove_SYNC_TRANSACTION_NOT_IN_BLOCK_magic_number) 02https://github.com/bitcoin/bitcoin/pull/10186
 45 2017-04-13 10:07:16	0|bitcoin-git|13bitcoin/060.14 14b7caa30 15Suhas Daftuar: Mention dbcache memory changes in 0.14.1 release notes
 46 2017-04-13 10:07:16	0|bitcoin-git|[13bitcoin] 15laanwj pushed 2 new commits to 060.14: 02https://github.com/bitcoin/bitcoin/compare/55f641ca194b...06909df179e7
 47 2017-04-13 10:07:17	0|bitcoin-git|13bitcoin/060.14 1406909df 15Wladimir J. van der Laan: Merge #10185: [0.14] Mention dbcache memory changes in release notes...
 48 2017-04-13 10:08:47	0|bitcoin-git|[13bitcoin] 15laanwj pushed 5 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/c9ff4f8ee601...70f6f56e9dde
 49 2017-04-13 10:08:48	0|bitcoin-git|13bitcoin/06master 14e78bc45 15NicolasDorier: [Wallet] Decouple CInputCoin from CWalletTx
 50 2017-04-13 10:08:48	0|bitcoin-git|13bitcoin/06master 14fd44ac1 15NicolasDorier: [Wallet] Rename std::pair<const CWalletTx*, unsigned int> to CInputCoin
 51 2017-04-13 10:08:49	0|bitcoin-git|13bitcoin/06master 14f597dcb 15NicolasDorier: [Wallet] Simplify code using CInputCoin
 52 2017-04-13 10:09:18	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #10165: [Wallet] Refactoring by using CInputCoin instead of std::pair (06master...06inputcoin) 02https://github.com/bitcoin/bitcoin/pull/10165
 53 2017-04-13 10:30:37	0|wumpus|gmaxwell: one problem is that x86 is such a complex beast to emulate (or even write a instruction decoder for). If you induce the overhead of full emulation for security reasons, might as wll choose a simpler  platform to emulate
 54 2017-04-13 10:37:38	0|wumpus|may also help to restrict what qemu can do through e.g. apparmor, so if a process manages to escape from the vm it isn't immediately able to do much more than the VM could. libvirt does that by default AFAIK
 55 2017-04-13 12:21:35	0|sdaftuar|gmaxwell: i ran the sim, with 10 seconds / 50000 satoshi fee threshold, roughly 95% of CNB invocations included a recent transaction.
 56 2017-04-13 13:38:07	0|sdaftuar|gmaxwell: for comparison, i re-ran my patch with a setting of 10seconds, 0.2% fee drop, and less than 2% of CNB calls included recent transactions
 57 2017-04-13 13:45:56	0|bitcoin-git|[13bitcoin] 15mariodian opened pull request #10201: pass Consensus::Params& to ReceivedBlockTransactions() (06master...06consensusparams-receivedblocktransactions) 02https://github.com/bitcoin/bitcoin/pull/10201
 58 2017-04-13 13:47:09	0|bitcoin-git|[13bitcoin] 15NicolasDorier closed pull request #10068: [Wallet] FundRawTransaction accepts preset non-wallet inputs (06master...06nonwalletinputs) 02https://github.com/bitcoin/bitcoin/pull/10068
 59 2017-04-13 13:48:32	0|bitcoin-git|[13bitcoin] 15NicolasDorier opened pull request #10202: [Wallet] FundRawTransaction can fund a transaction with preset inputs found in the CoinView (06master...06fundrawtransaction) 02https://github.com/bitcoin/bitcoin/pull/10202
 60 2017-04-13 14:01:43	0|BlueMatt|cfields: hmm, whats the status of #7522?
 61 2017-04-13 14:01:45	0|gribble|https://github.com/bitcoin/bitcoin/issues/7522 | Bugfix: Only use git for build info if the repository is actually the right one by luke-jr · Pull Request #7522 · bitcoin/bitcoin · GitHub
 62 2017-04-13 14:03:18	0|BlueMatt|luke-jr: are you going to rebase #7289?
 63 2017-04-13 14:03:19	0|gribble|https://github.com/bitcoin/bitcoin/issues/7289 | [WIP] Make arguments reconfigurable at runtime via RPC by luke-jr · Pull Request #7289 · bitcoin/bitcoin · GitHub
 64 2017-04-13 14:08:57	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #7148: [NO MERGE] Travis: Run nightly test suite (06master...06travis/nightly) 02https://github.com/bitcoin/bitcoin/pull/7148
 65 2017-04-13 14:15:04	0|wumpus|I agree it's a bit strange but I can't be bothered to change it, it's not as if incidents ever get reported to it
 66 2017-04-13 14:15:55	0|wumpus|eh, wrong key
 67 2017-04-13 14:16:18	0|BlueMatt|wumpus: you have a key which types whole sentences? wow!
 68 2017-04-13 14:16:20	0|BlueMatt|:p
 69 2017-04-13 14:18:18	0|wumpus|yes this smart-keyboard has analyzed all of the text I've entered on it, put it into a  markov chain, and now generated it's own text with just one keypress :p
 70 2017-04-13 14:20:46	0|BlueMatt|wumpus: you jest, but thats what fucking mobile phones do :(
 71 2017-04-13 14:21:56	0|jnewbery|and your markov chain predicts that your most common response is "I agree it's a bit strange but I can't be bothered to change it". Seems pretty accurate :)
 72 2017-04-13 14:21:58	0|wumpus|ah yes that's always the first thing I disable
 73 2017-04-13 14:24:32	0|wumpus|jnewbery: yep, the keyboard can almost replace me already :-)
 74 2017-04-13 14:27:14	0|michagogo|Yeah, the keyboard auto correct is the best app that I can use on my iPhone and my phone is a very quiet app and the fact is that it doesn't have a plan to use Facebook to access the web for the past couple weeks
 75 2017-04-13 14:28:03	0|michagogo|(I typed the first 4 words of that and selected the rest from the 3 suggestions)
 76 2017-04-13 14:32:38	0|sipa|wumpus: it's annoying to create PRs against the bitcoin core leveldb repository
 77 2017-04-13 14:32:56	0|sipa|because it's marked as 'forked from' the upstream google repo (which didn't exist at the time ours was created)
 78 2017-04-13 14:33:01	0|sipa|*isn't marked
 79 2017-04-13 14:33:11	0|sipa|seems the only way to fix that is to delete it and recreate it
 80 2017-04-13 14:33:31	0|wumpus|that'll lose all the current issues and PRs though
 81 2017-04-13 14:34:00	0|sipa|true
 82 2017-04-13 14:34:07	0|sipa|maybe we can contact support to just fix it for us?
 83 2017-04-13 14:34:19	0|wumpus|yes I think they can reparent repositories
 84 2017-04-13 14:36:32	0|bitcoin-git|13bitcoin/06master 14c851be4 15Cory Fields: net: define NodeId as an int64_t...
 85 2017-04-13 14:36:32	0|bitcoin-git|[13bitcoin] 15laanwj pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/70f6f56e9dde...cf8a8b1028d6
 86 2017-04-13 14:36:33	0|bitcoin-git|13bitcoin/06master 14cf8a8b1 15Wladimir J. van der Laan: Merge #10176: net: gracefully handle NodeId wrapping...
 87 2017-04-13 14:36:58	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #10176: net: gracefully handle NodeId wrapping (06master...06nodeid-no-wrap) 02https://github.com/bitcoin/bitcoin/pull/10176
 88 2017-04-13 14:37:15	0|wumpus|are you going to contact support or should I?
 89 2017-04-13 14:37:41	0|sipa|i can
 90 2017-04-13 15:12:53	0|sdaftuar|wumpus: i just noticed #9665 has been sitting around for a couple of months, it has a few ACKs and I think could be merged
 91 2017-04-13 15:12:54	0|gribble|https://github.com/bitcoin/bitcoin/issues/9665 | Use cached [compact] blocks to respond to getdata messages by TheBlueMatt · Pull Request #9665 · bitcoin/bitcoin · GitHub
 92 2017-04-13 15:18:45	0|wumpus|sdaftuar: thanks
 93 2017-04-13 15:22:49	0|bitcoin-git|13bitcoin/06master 14efc135f 15Matt Corallo: Use cached [compact] blocks to respond to getdata messages
 94 2017-04-13 15:22:49	0|bitcoin-git|[13bitcoin] 15laanwj pushed 4 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/cf8a8b1028d6...eab00d96dfe9
 95 2017-04-13 15:22:50	0|bitcoin-git|13bitcoin/06master 14b49ad44 15Matt Corallo: Add comment about cs_most_recent_block coverage
 96 2017-04-13 15:22:50	0|bitcoin-git|13bitcoin/06master 14c47f5b7 15Matt Corallo: Cache witness-enabled state with recent-compact-block-cache
 97 2017-04-13 15:23:05	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #9665: Use cached [compact] blocks to respond to getdata messages (06master...062017-02-processgetdata-cache) 02https://github.com/bitcoin/bitcoin/pull/9665
 98 2017-04-13 15:24:28	0|gribble|https://github.com/bitcoin/bitcoin/issues/9942 | Refactor CBlockPolicyEstimator by morcos · Pull Request #9942 · bitcoin/bitcoin · GitHub
 99 2017-04-13 15:24:30	0|gribble|https://github.com/bitcoin/bitcoin/issues/9480 | De-duplicate SignatureCacheHasher by JeremyRubin · Pull Request #9480 · bitcoin/bitcoin · GitHub
100 2017-04-13 16:08:13	0|sdaftuar|sipa: i was trying to review #9743, and i'm confused by the lockstack cleanup commit.  the boost documentation i'm reading suggests that by default, delete should be called already?
101 2017-04-13 16:08:15	0|gribble|https://github.com/bitcoin/bitcoin/issues/9743 | Fix several potential issues found by sanitizers by sipa · Pull Request #9743 · bitcoin/bitcoin · GitHub
102 2017-04-13 16:10:53	0|sipa|sdaftuar: hmm
103 2017-04-13 16:12:22	0|sdaftuar|i'm looking at this: http://www.boost.org/doc/libs/1_63_0/doc/html/thread/thread_local_storage.html
104 2017-04-13 16:12:35	0|sdaftuar|there is this comment: "Note: on some platforms, cleanup of thread-specific data is not performed for threads created with the platform's native API. On those platforms such cleanup is only done for threads that are started with boost::thread unless boost::on_thread_exit() is called manually from that thread.
105 2017-04-13 16:13:21	0|sipa|sdaftuar: but that would mean that the custom cleanup function is also not called?
106 2017-04-13 16:13:30	0|sdaftuar|sipa: that was my guess, but i wasn't sure!
107 2017-04-13 16:14:05	0|sipa|i seem to recall that fixed things
108 2017-04-13 17:13:38	0|bitcoin-git|13bitcoin/06master 14f9c8807 15Jeremy Rubin: Deduplicate SignatureCacheHasher...
109 2017-04-13 17:13:38	0|bitcoin-git|[13bitcoin] 15sipa pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/eab00d96dfe9...b7365f0545b1
110 2017-04-13 17:13:39	0|bitcoin-git|13bitcoin/06master 14b7365f0 15Pieter Wuille: Merge #9480: De-duplicate SignatureCacheHasher...
111 2017-04-13 17:13:53	0|bitcoin-git|[13bitcoin] 15sipa closed pull request #9480: De-duplicate SignatureCacheHasher (06master...06refactor-signaturecachehasher-visibility) 02https://github.com/bitcoin/bitcoin/pull/9480
112 2017-04-13 17:33:41	0|cfields_|sipa: for leveldb merge, whenever it's needed: https://github.com/theuni/bitcoin/commit/2cb7dda13884e44105f33c16e7e2c1a9aed46295
113 2017-04-13 17:35:48	0|cfields_|BlueMatt: hmm, i thought 7522 was merged a long time ago. Having a look.
114 2017-04-13 18:06:45	0|luke-jr|hm, thought I was late; am I early?
115 2017-04-13 18:07:32	0|gmaxwell|you are early
116 2017-04-13 18:07:46	0|luke-jr|cool
117 2017-04-13 18:12:09	0|aantonop|gmaxwell: apologies for the trolling by an "aantonop" imposter the other day. I hadn't turned on Nickserv enforcement. It's on now.
118 2017-04-13 18:13:24	0|gmaxwell|aantonop: oh no problem, people should have just ignored him. :) trolls show up all the time.
119 2017-04-13 18:31:04	0|spudowiar|Is there a good way to get a transaction input and find the BIP32 derivation path?
120 2017-04-13 18:31:20	0|spudowiar|I don't think the keystore stores BIP32 derivation information
121 2017-04-13 18:33:01	0|luke-jr|pretty sure it does..
122 2017-04-13 18:33:46	0|spudowiar|luke-jr: Well, I was looking in src/keystore.h but it seems I can only get a CKey or CPubKey out which I don't think stores derivation information
123 2017-04-13 18:37:31	0|sipa|CKeyMetaData or something stores ot
124 2017-04-13 18:37:33	0|sipa|*it
125 2017-04-13 18:39:58	0|spudowiar|sipa: How do I get one of those from a public key?
126 2017-04-13 19:00:03	0|wumpus|meeting time?
127 2017-04-13 19:00:12	0|jonasschnelli|Yes. Hi all.
128 2017-04-13 19:00:18	0|lightningbot|Meeting started Thu Apr 13 19:00:17 2017 UTC.  The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
129 2017-04-13 19:00:18	0|lightningbot|Useful Commands: #action #agreed #help #info #idea #link #topic.
130 2017-04-13 19:00:18	0|wumpus|#startmeeting
131 2017-04-13 19:00:26	0|morcos|roger
132 2017-04-13 19:00:28	0|sdaftuar|hello
133 2017-04-13 19:00:30	0|CodeShark|hi
134 2017-04-13 19:00:33	0|wumpus|#bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr btcdrak sdaftuar jtimon cfields petertodd kanzure blue
135 2017-04-13 19:00:33	0|wumpus|matt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier jl2012 instagibbs
136 2017-04-13 19:00:39	0|kanzure|hi.
137 2017-04-13 19:00:39	0|wumpus|hello
138 2017-04-13 19:00:45	0|cfields_|hi
139 2017-04-13 19:00:51	0|instagibbs|hi
140 2017-04-13 19:00:55	0|phantomcircuit|hi
141 2017-04-13 19:00:56	0|wumpus|topics?
142 2017-04-13 19:01:02	0|gmaxwell|Hi!
143 2017-04-13 19:01:14	0|wumpus|#topic 0.14.1
144 2017-04-13 19:01:20	0|wumpus|anyone had report of bugs with the rc1?
145 2017-04-13 19:01:29	0|cfields_|topic suggestion (after 0.14.1): scripted-diffs
146 2017-04-13 19:01:33	0|gmaxwell|Push the button?
147 2017-04-13 19:01:47	0|gmaxwell|wumpus: you know that the bug reports only come after releasing it. :P
148 2017-04-13 19:01:56	0|petertodd|hi
149 2017-04-13 19:02:13	0|wumpus|gmaxwell: the serious ones, yes :p
150 2017-04-13 19:02:42	0|wumpus|but apparently no one heard anything, so time to tag -final
151 2017-04-13 19:03:26	0|cfields_|wumpus: wait for ^^ ? doing now.
152 2017-04-13 19:04:37	0|wumpus|would be nice but I think that's not a regression since 0.14.0, so I'm not sure it warrants another rc
153 2017-04-13 19:05:15	0|gmaxwell|can talk about post meeting, certantly nothing but that is in the air.
154 2017-04-13 19:05:41	0|cfields_|ok
155 2017-04-13 19:05:41	0|wumpus|if you think it's really important to do we can do another rc, no problem
156 2017-04-13 19:06:26	0|gmaxwell|I really don't want to delay 0.14.1 further.
157 2017-04-13 19:06:37	0|cfields_|wumpus: it was more of a since-we're-releasing-anyway kind of thing
158 2017-04-13 19:06:47	0|wumpus|I'd prefer not to either; we should fix the wrapping, but could just was well be included for 0.14.2
159 2017-04-13 19:07:12	0|wumpus|I agree it's something that should be backported to the 0.14 branch
160 2017-04-13 19:07:26	0|morcos|no opinion either way here
161 2017-04-13 19:07:55	0|wumpus|ok
162 2017-04-13 19:07:58	0|BlueMatt|wumpus: I think we should do it in 0.14
163 2017-04-13 19:08:00	0|BlueMatt|.1
164 2017-04-13 19:08:10	0|BlueMatt|because its free, we're already doing 0.14.1 and delaying 1 week isnt gonna kill us
165 2017-04-13 19:08:43	0|luke-jr|cfields_: new fixes = new rc required
166 2017-04-13 19:08:50	0|luke-jr|so not worth it in that scenario
167 2017-04-13 19:09:26	0|BlueMatt_|But delaying 1 week isn't too bad
168 2017-04-13 19:09:40	0|BlueMatt|wait, who is BlueMatt_ ?
169 2017-04-13 19:10:31	0|instagibbs|:/
170 2017-04-13 19:10:47	0|kanzure|different timeline, carry on.
171 2017-04-13 19:10:53	0|luke-jr|whois says it's Matt Corallo
172 2017-04-13 19:11:04	0|jcorgan|digital ocean ip
173 2017-04-13 19:11:12	0|BlueMatt|not me
174 2017-04-13 19:11:42	0|gmaxwell|wumpus: shoot the T1000 (BlueMatt_) and lets move on.
175 2017-04-13 19:12:01	0|sipa|BlueMatt_: this statement is false
176 2017-04-13 19:12:13	0|spudowiar|gmaxwell: I think that's the fake aantonop IP
177 2017-04-13 19:12:28	0|luke-jr|sipa: :D
178 2017-04-13 19:12:37	0|spudowiar|2017-04-11 21:37:35 -- Mode #bitcoin [+b *!*@178.62.68.75] by gmaxwell
179 2017-04-13 19:12:45	0|gmaxwell|We've got lots more to discuss.
180 2017-04-13 19:12:50	0|BlueMatt|yes, lets move on
181 2017-04-13 19:13:04	0|wumpus|so have we decided anything?
182 2017-04-13 19:13:12	0|sipa|0.14.1 yay
183 2017-04-13 19:13:56	0|wumpus|#topic scripted-diffs
184 2017-04-13 19:14:16	0|cfields_|yes, let's finish discussing after the meeting
185 2017-04-13 19:14:46	0|cfields_|ok, for reference: #10189
186 2017-04-13 19:14:47	0|gribble|https://github.com/bitcoin/bitcoin/issues/10189 | devtools/net: add a verifier for scriptable changes. Use it to make CNode::id private. by theuni · Pull Request #10189 · bitcoin/bitcoin · GitHub
187 2017-04-13 19:14:47	0|luke-jr|mv meeting/* after-meeting/
188 2017-04-13 19:15:01	0|wumpus|haven't had a time to look at that one yet
189 2017-04-13 19:15:30	0|cfields_|the idea is for changes that are basically just search/replace, to allow a short script to be placed in the commit message, and verified by Travis
190 2017-04-13 19:15:49	0|cfields_|if they don't transform the old commit to the new one exactly, travis will reject
191 2017-04-13 19:15:59	0|cfields_|the aim is to make those kinds of changes easier to review
192 2017-04-13 19:16:03	0|BlueMatt|is ther emuch to discuss? I think its a good idea
193 2017-04-13 19:16:18	0|cfields_|example: https://github.com/bitcoin/bitcoin/pull/10189/commits/d04198309e7e9b21de1604cc4321148b37a46347
194 2017-04-13 19:16:21	0|luke-jr|IMO we shouldn't trust Travis
195 2017-04-13 19:16:33	0|gmaxwell|I'm fine with it so long as we don't expect commiters to be running it.  I think someone should write a list of examples, because when I've made mass programatic changes I haven't done it in ways that would look too tidy in a commit message.
196 2017-04-13 19:16:34	0|wumpus|would that execute arbitrary scripts in commit messages?
197 2017-04-13 19:16:37	0|luke-jr|which is basicaly what you're suggesting)
198 2017-04-13 19:16:38	0|cfields_|BlueMatt: well, you brought up a good point about the safety of random scripts
199 2017-04-13 19:16:48	0|wumpus|seems kind of dangerous, I certainly wouldn't want to run that locally
200 2017-04-13 19:16:51	0|cfields_|so i thought it might be worth discussing possibly constraining. Say to sed or so.
201 2017-04-13 19:17:00	0|BlueMatt|gmaxwell: there are already two examplesa
202 2017-04-13 19:17:03	0|wumpus|I'm really careful what scripts I run, and wouldn't want anything to be run automatically
203 2017-04-13 19:17:14	0|BlueMatt|and, yes, agree, jtimon suggested that they only be run if there is a scripted prefix in the commit's title
204 2017-04-13 19:17:15	0|spudowiar|What sort of damage can be done with, say, sed or awk?
205 2017-04-13 19:17:17	0|BlueMatt|to make it much more obvious
206 2017-04-13 19:17:28	0|gmaxwell|spudowiar: arbritary damage.
207 2017-04-13 19:17:28	0|petertodd|wumpus: granted, if you ever compile bitcoin you're running stuff within the repo anyway
208 2017-04-13 19:17:31	0|spudowiar|I don't think you can do much with sed, not too sure about awk
209 2017-04-13 19:17:43	0|spudowiar|gmaxwell: I mean dangerous code execution
210 2017-04-13 19:17:48	0|cfields_|wumpus: sure, agreed. Travis can run them, and they can ofc be run manually as well.
211 2017-04-13 19:17:51	0|sipa|try applying sed to /etc/init.d files
212 2017-04-13 19:17:58	0|BlueMatt|see-also #10193
213 2017-04-13 19:17:59	0|gribble|https://github.com/bitcoin/bitcoin/issues/10193 | scripted-diff: Remove #include foreach.hpp> by jtimon · Pull Request #10193 · bitcoin bitcoin · GitHub
214 2017-04-13 19:18:02	0|BlueMatt|(which is an example)
215 2017-04-13 19:18:10	0|jcorgan|maybe there should only be a predefined set of opcodes in the script that can run, with a stack...oh wait
216 2017-04-13 19:18:14	0|wumpus|just make it create a ~/.bashrc, runs next time
217 2017-04-13 19:18:22	0|gmaxwell|in any case, if it's a travis mostly thing I think thats ducky. I would attempt to use it.
218 2017-04-13 19:18:28	0|wumpus|I'd really prefer not to do this
219 2017-04-13 19:18:55	0|luke-jr|I don't see the value in a Travis-"mostly" thing.
220 2017-04-13 19:18:55	0|spudowiar|What are the benefits?
221 2017-04-13 19:18:56	0|gmaxwell|I guess the reason to just not commit the script is that they're one time things?
222 2017-04-13 19:18:58	0|cfields_|gmaxwell: that was my intention, yes.
223 2017-04-13 19:19:06	0|luke-jr|it gives a false sense of review
224 2017-04-13 19:19:10	0|wumpus|if there's a script then reviewers can manually verify it too, after reviewing the script
225 2017-04-13 19:19:13	0|BlueMatt|gmaxwell: yes, if they're one-time things its annoying to commit them, also annoying to lose them
226 2017-04-13 19:19:36	0|sipa|when would these scripts be ru
227 2017-04-13 19:19:38	0|sipa|n
228 2017-04-13 19:20:03	0|cfields_|yes, i got the idea from bac5c9cf643e9333479ac667426d0b70f8f3aa7f
229 2017-04-13 19:20:06	0|luke-jr|I wouldn't mind including the scripts in non-first-line commit msg, and having a dev tool to run them, but IMO better if Travis *doesn't* do it
230 2017-04-13 19:20:17	0|gmaxwell|luke-jr: but reviewers can test it. After reviewing the script. The challenge there is just that you normally don't critically review commit messages in that way. but it's kind of perfect-- in that its one time information about the change.
231 2017-04-13 19:20:24	0|cfields_|https://github.com/bitcoin/bitcoin/commit/bac5c9cf643e9333479ac667426d0b70f8f3aa7f
232 2017-04-13 19:20:56	0|gmaxwell|luke-jr: the purpose of travis doing it is for feedback that you've formated it right and it runs on computers other than yours. Not as a review step.
233 2017-04-13 19:20:59	0|cfields_|if we're going to include a transform in the message like that, I figured we may as well standardize it
234 2017-04-13 19:21:00	0|luke-jr|gmaxwell: as long as reviewers do test it, and don't trust Travis
235 2017-04-13 19:21:03	0|wumpus|cfields_: the idea of that was to give reviewers a way to verify it, as well as make it easy for myself to repeat the work if it needs rebase
236 2017-04-13 19:21:04	0|spudowiar|You could create format like `*.cpp *.h | s/boost::filesystem/fs/g`
237 2017-04-13 19:22:26	0|sipa|spudowiar: little bobby tables will haunt you
238 2017-04-13 19:23:02	0|luke-jr|lol
239 2017-04-13 19:23:03	0|petertodd|gmaxwell: note how this is rather like how a merkle sum tree works... the script is a function to transform input to output, and both input and final output are committed via the git commit object
240 2017-04-13 19:23:15	0|cfields_|ok, put a different way, then: if it could be done so that it worked on nothing but a dummy git index, with no access to the filesystem, would that be more acceptable?
241 2017-04-13 19:23:38	0|wumpus|cfields_: if a transformation of the files could be perfectly sandboxed , I'd be all for it, but I'm kind of afraid of anything that will automatically execute scripts from commit messages, especially as most people review the code changes but not so much the messages :)
242 2017-04-13 19:23:46	0|petertodd|cfields_: I mean, with travis the compilation process can do anything anyway, so I don't see how this is a security risk on top of anything else
243 2017-04-13 19:23:53	0|gmaxwell|I don't know that sandboxing this is worth the effort. If you want to do that, knock yourself out, I guess?
244 2017-04-13 19:23:54	0|petertodd|cfields_: that's true for your local dev setup too
245 2017-04-13 19:24:03	0|wumpus|I don't think it's worth the effort, no
246 2017-04-13 19:24:18	0|BlueMatt|wumpus: that was my concern, hence why I like jtimon's suggestion of only doing it if, eg, the commit title starts with "AUTO-SCRIPT: "
247 2017-04-13 19:24:19	0|BlueMatt|or something
248 2017-04-13 19:24:24	0|BlueMatt|then its obvious from the first line of the commit
249 2017-04-13 19:24:24	0|gmaxwell|petertodd: the only concern I have wrt security is that there is a new thing you need to review before running things. Not just the git diff but the commit messages.
250 2017-04-13 19:24:25	0|cfields_|petertodd: i agree. The concern here (I guess) is that we'd get a malicious script committed, then someone would run it locally.
251 2017-04-13 19:24:26	0|BlueMatt|and you will see it
252 2017-04-13 19:24:58	0|petertodd|cfields_: but that's already a problem anyway: I can commit a malicious commit that adds rm -rf / to the makefile
253 2017-04-13 19:25:16	0|gmaxwell|cfields_: I dunno about your workflow, but I could be tricked by this, because I'll pull and git diff <where I was before>  which won't show me the commit message(s).
254 2017-04-13 19:25:32	0|gmaxwell|otherwise I'd argue there is no change.
255 2017-04-13 19:25:45	0|cfields_|gmaxwell: but it wouldn't touch anything in that case. You'd have to actively run it.
256 2017-04-13 19:25:51	0|petertodd|gmaxwell: so long as devs never have git hooks that run this automatically I think we're covered
257 2017-04-13 19:25:51	0|wumpus|petertodd: you can, but changes to the makefile would be apparent to me at least
258 2017-04-13 19:26:02	0|spudowiar|Why not have a script that manually does these checks, prints out each commit message beginning with "AUTO-SCRIPT: " and the commands it would run and asks if you want to run them?
259 2017-04-13 19:26:08	0|gmaxwell|petertodd: yea, I'm not too concerned.
260 2017-04-13 19:26:08	0|petertodd|wumpus: right, but see my reply to gmaxwell above
261 2017-04-13 19:26:11	0|wumpus|yes, as long as it's not executed automatically outside of travis it's fine
262 2017-04-13 19:26:22	0|wumpus|I don't care what is done in travis, that is already somebody else's sandbox
263 2017-04-13 19:26:25	0|cfields_|yes, that's the case as-is.
264 2017-04-13 19:26:27	0|petertodd|wumpus: +1
265 2017-04-13 19:26:28	0|gmaxwell|oh actually too spudowiar's suggestion is good  unless run with --force the script could ask for confirmation.
266 2017-04-13 19:26:31	0|cfields_|nothing is touched locally.
267 2017-04-13 19:26:51	0|wumpus|my point was just that I just don't want e.g. github_merge.py to execute it automatically
268 2017-04-13 19:26:55	0|wumpus|great
269 2017-04-13 19:27:08	0|petertodd|wumpus: yup, that'd be a bit crazy :)
270 2017-04-13 19:27:10	0|cfields_|wait, please back up, I'm not sure if there's a communication breakdown here.
271 2017-04-13 19:27:49	0|cfields_|as-is, this is travis-only. Nothing runs automatically anywhere but there.
272 2017-04-13 19:27:56	0|wumpus|cfields_: as said I haven't seen the actual PR, just rumors from this channel
273 2017-04-13 19:28:11	0|spudowiar|Well, Travis can run it with --force, github_merge.py can run it without --force?
274 2017-04-13 19:28:14	0|wumpus|yes makes sense to me then cfields_
275 2017-04-13 19:28:26	0|spudowiar|Without --force, no actual commands are run
276 2017-04-13 19:28:30	0|spudowiar|Unless you interactively say so
277 2017-04-13 19:28:34	0|gmaxwell|spudowiar: no need to make merge run it at all. just an extra review step available.
278 2017-04-13 19:28:35	0|cfields_|spudowiar: there's nothing to run "it".
279 2017-04-13 19:28:44	0|wumpus|no, github_merge.py won't run it at all, that is a merge script and shouldn't do anything else
280 2017-04-13 19:29:03	0|wumpus|(I run that on the machine that has the gpg key, so I'm paranoid about it)
281 2017-04-13 19:29:10	0|gmaxwell|But I do think it would be nice if when you manually run that review step it shows you what it's going to run.. to avoid my workflow issue example.
282 2017-04-13 19:29:13	0|spudowiar|cfields_: I'm talking about a hypothetical script
283 2017-04-13 19:29:18	0|spudowiar|wumpus: Buy a PGP smartcard :)
284 2017-04-13 19:29:47	0|petertodd|spudowiar: adversary still can cause the PGP smartcard to sign something you didn't aprove
285 2017-04-13 19:29:50	0|wumpus|ok. seems a communication breakdown here.
286 2017-04-13 19:29:59	0|wumpus|let's just review the PR, and talk about it again next week
287 2017-04-13 19:30:01	0|gmaxwell|okay
288 2017-04-13 19:30:04	0|cfields_|heh, yes please :)
289 2017-04-13 19:30:12	0|sdaftuar|+1!
290 2017-04-13 19:30:18	0|wumpus|#action review #10193
291 2017-04-13 19:30:20	0|gribble|https://github.com/bitcoin/bitcoin/issues/10193 | scripted-diff: Remove #include foreach.hpp> by jtimon · Pull Request #10193 · bitcoin bitcoin · GitHub
292 2017-04-13 19:30:21	0|morcos|good thing our trolling coordination doesn't suffer from these communication problems
293 2017-04-13 19:30:24	0|wumpus|other topics?
294 2017-04-13 19:31:11	0|wumpus|#action review #10189
295 2017-04-13 19:31:13	0|gribble|https://github.com/bitcoin/bitcoin/issues/10189 | devtools/net: add a verifier for scriptable changes. Use it to make CNode::id private. by theuni · Pull Request #10189 · bitcoin/bitcoin · GitHub
296 2017-04-13 19:31:19	0|sdaftuar|topic suggestion: goals for 0.15
297 2017-04-13 19:31:40	0|cfields_|good one.
298 2017-04-13 19:31:43	0|BlueMatt|ok, so ryanofsky wanted to talk about multi-process, too, i think
299 2017-04-13 19:31:49	0|BlueMatt|:)
300 2017-04-13 19:31:49	0|BlueMatt|and, ofc, pr review targets for the week
301 2017-04-13 19:31:57	0|wumpus|#topic goals for 0.15
302 2017-04-13 19:32:28	0|BlueMatt|sdaftuar: my interpretation of "goals for 0.15" is "everyone has their own goals, and should mention the PRs they want reviewed to further their goals as review priorities, and we should all help them :)"
303 2017-04-13 19:32:53	0|gmaxwell|Per-txo dbcache and non-atomic flushing please...
304 2017-04-13 19:32:58	0|wumpus|agree with BlueMatt, though people could state what their goals for 0.15 are
305 2017-04-13 19:33:02	0|jonasschnelli|My goals for 0.15 are: HD auto-restore, Qt feebumper and multiwallet
306 2017-04-13 19:33:15	0|spudowiar|Watch only HD wallets sound good for 0.15 :)
307 2017-04-13 19:33:21	0|gmaxwell|and multiwallet.
308 2017-04-13 19:33:21	0|jonasschnelli|Oh.. yes. That!
309 2017-04-13 19:33:27	0|jonasschnelli|(hd watch only)
310 2017-04-13 19:33:27	0|sdaftuar|gmaxwell: agreed, also i think it would be good to get the new fee estimation in place
311 2017-04-13 19:33:40	0|BlueMatt|multithreaded net_processing (and wallet) with CValidationInterface generating callbacks into it
312 2017-04-13 19:33:54	0|BlueMatt|ie disconnecting validation and everything else with CValidationInterface in betwene :)
313 2017-04-13 19:34:02	0|sipa|when is 0.15 feature freeze?
314 2017-04-13 19:34:05	0|wumpus|luke-jr: are you planning to update the multiwallet prepare pull according to review comments, or should I take it over?
315 2017-04-13 19:34:07	0|achow101|is replacing accounts with labels planned for 0.15?
316 2017-04-13 19:34:13	0|gmaxwell|sdaftuar: I was going to bring that up. I think that we really need a high level description of the algorithim that we could give to non-developers (e.g. academics) to review.  I don't know if that should hold the improvements but we need to get to that.
317 2017-04-13 19:34:21	0|wumpus|luke-jr: it's kind of blocking other things right now I think
318 2017-04-13 19:34:32	0|morcos|yes, re: fee estimation.. i understand its a giant pain in the ass to review...  and for little perceived gain.  but i do think its worth it, and merging it sooner rather than later is better in case there are any edge case improvements needed.
319 2017-04-13 19:34:44	0|luke-jr|wumpus: when I get home
320 2017-04-13 19:34:46	0|BlueMatt|I do NOT think its little perceived gain
321 2017-04-13 19:34:51	0|cfields_|my big goals are: finally move to libevent for connman, deterministic toolchain creator (so we can drop our reliance on ubuntu's toolchain)
322 2017-04-13 19:34:58	0|wumpus|#link Release schedule for 0.15.0 https://github.com/bitcoin/bitcoin/issues/9961
323 2017-04-13 19:35:00	0|sipa|cfields_: woooh!
324 2017-04-13 19:35:06	0|BlueMatt|morcos: one of the topics discussed at the wallet dev meetup thing was about how shit fee estimates across the ecosystem are
325 2017-04-13 19:35:09	0|wumpus|yes libevent please
326 2017-04-13 19:35:10	0|BlueMatt|and bitcoin core is a big part of that
327 2017-04-13 19:35:11	0|morcos|gmaxwell: heh... it's more art than science i think
328 2017-04-13 19:35:15	0|gmaxwell|jonasschnelli: I don't know about HD watch only, we have a lot of overhang in the HD change that isn't done yet. We really need to hammer out all these corner cases before we go adding more major new features in key management.
329 2017-04-13 19:35:24	0|wumpus|I'd also really like to get the UNIX p2p/rpc stuff in
330 2017-04-13 19:35:36	0|jonasschnelli|gmaxwell: can you explain "a lot"?
331 2017-04-13 19:35:42	0|BlueMatt|morcos: your previous warrants to me about weekend fee estimates in your new design being good represent a huge win for the ecosystem
332 2017-04-13 19:36:08	0|luke-jr|wumpus: (tomorrow)
333 2017-04-13 19:36:10	0|wumpus|it's all very low-risk and shouldn't affect non-unix-socket use cases
334 2017-04-13 19:36:11	0|sdaftuar|BlueMatt: I also think the ability to return non-conservative (ie, lower) fee estimates is important
335 2017-04-13 19:36:17	0|gmaxwell|jonasschnelli: all the things related to recovery are broken and we even don't know how to fix some of them.
336 2017-04-13 19:36:19	0|morcos|BlueMatt: yes, but exactly the kind of thing that need real world experimentation, b/c they change the real world conditions
337 2017-04-13 19:36:19	0|wumpus|luke-jr: thanks
338 2017-04-13 19:36:22	0|sipa|i want pertxoutcache, remove memory peak at flushing, better dbcache eviction policy, ...
339 2017-04-13 19:36:45	0|jonasschnelli|gmaxwell: Yes. I'm working on the recover (one of my 0.15 goals: "HD auto-restore")
340 2017-04-13 19:36:53	0|BlueMatt|OKOK, so review priorities towards all these amazing things? whats up this week :)
341 2017-04-13 19:36:55	0|sipa|oh, and segwit activated? pretty please?
342 2017-04-13 19:36:58	0|gmaxwell|morcos: right but there are incentives and security implications in the design, that deserve wider review than we'll get from people who will infer it from the code. Even if the description wasn't completely precise it would help.
343 2017-04-13 19:37:01	0|BlueMatt|sipa: lol
344 2017-04-13 19:37:08	0|cfields_|wumpus: yes. I've looked over that and it kinda clashes with my libevent changes. But it makes sense to get yours in first, then I can adapt mine.
345 2017-04-13 19:37:24	0|gribble|https://github.com/bitcoin/bitcoin/issues/10179 | Give CValidationInterface Support for calling notifications on the CScheduler Thread by TheBlueMatt · Pull Request #10179 · bitcoin/bitcoin · GitHub
346 2017-04-13 19:37:28	0|wumpus|cfields_: the RPC pull (the first one) shouldn't collide
347 2017-04-13 19:37:29	0|gmaxwell|sipa: would likely help to get 0.14.1 out...
348 2017-04-13 19:37:40	0|wumpus|cfields_: I'm fine with including that one only for now, it's most of the code
349 2017-04-13 19:37:41	0|sipa|gmaxwell: ack
350 2017-04-13 19:37:47	0|cfields_|wumpus: no, the other. But it's a non-issue either way.
351 2017-04-13 19:38:10	0|wumpus|cfields_: the p2p-over-unix-socket is a very small patch on top of that, Im fine with doing that after libevent which means it can share the code with the RPC path
352 2017-04-13 19:38:30	0|morcos|gmaxwell: yes, writing up the high level design is actually simpler than explaining the actual implementation.  the high level design is pretty simple.. maybe i'll put more effort into the last commit which adds commentary
353 2017-04-13 19:38:31	0|wumpus|cfields_: though it depends on the time frame I guess
354 2017-04-13 19:38:40	0|gmaxwell|morcos: I think that would be really helpful.
355 2017-04-13 19:38:47	0|cfields_|sounds good.
356 2017-04-13 19:39:00	0|gmaxwell|morcos: Also for general review, I've lost track of the overall design of the estimator.
357 2017-04-13 19:39:12	0|sipa|morcos: i would like to see a 1-page document explaining what it tries to accomplish
358 2017-04-13 19:39:21	0|gmaxwell|morcos: in any case I strongly support improving it, the current one is dumb on weekends.
359 2017-04-13 19:39:25	0|cfields_|sipa: let's activate segwit after the meeting. We only have 20 min left :p
360 2017-04-13 19:39:32	0|gmaxwell|cfields_: ack
361 2017-04-13 19:39:37	0|wumpus|#action activate segwit
362 2017-04-13 19:39:41	0|morcos|sipa: ok
363 2017-04-13 19:39:45	0|gmaxwell|jinx
364 2017-04-13 19:39:59	0|sipa|(i have no clue how the estimator works, or ever worked, beside "something with buckets")
365 2017-04-13 19:40:04	0|morcos|gmaxwell: yes this is much better on weekends (optionally)
366 2017-04-13 19:40:07	0|instagibbs|ack on explanation, I was asking really dumb questions about it yesterday, may help
367 2017-04-13 19:40:22	0|gmaxwell|So we should bring up again per-txo and non-atomic flushing.  These changes are straight forward but call for a lot of heroic crash testing.
368 2017-04-13 19:41:02	0|BlueMatt|gmaxwell: I'm waiting for non-atomic to support disconnects
369 2017-04-13 19:41:05	0|BlueMatt|then I'll re-review :)
370 2017-04-13 19:41:42	0|bitcoin-git|[13bitcoin] 15jnewbery opened pull request #10204: [rpc] rename disconnectnode argument (06master...06rename_disconnect_node_argument) 02https://github.com/bitcoin/bitcoin/pull/10204
371 2017-04-13 19:42:01	0|sipa|gmaxwell: agree, it needs to support reorgs
372 2017-04-13 19:42:08	0|sipa|also... even harder to test
373 2017-04-13 19:42:29	0|cfields_|high-level question about per-txo: what's the desired/expected outcome when downgrading from 0.15 to 0.14 ?
374 2017-04-13 19:42:34	0|gmaxwell|sipa: hm? with the current code we finish the flush all at once.
375 2017-04-13 19:42:51	0|gmaxwell|needing to support reorgs is only an issue with the changes we discussed post per-txo.
376 2017-04-13 19:43:20	0|MarcoFalke|wumpus: The pull jnewbery just opened could go into 0.14.1 if we do another rc anyway
377 2017-04-13 19:43:33	0|sipa|gmaxwell: a crash in the middle a flush after a disconnect cannot be recovered from with the current code
378 2017-04-13 19:43:33	0|wumpus|MarcoFalke: yep
379 2017-04-13 19:43:37	0|gmaxwell|cfields_: per-txo cannot be downgraded. reindex. Gotta take the cost at some point.  Non-atomic is downgradable on clean shutdown, reindex otherwise.
380 2017-04-13 19:43:49	0|jnewbery|MarcoFalke has suggested that I split off the non-backwards compatible rpc argument name change from #10143 into its own PR so it can be backported
381 2017-04-13 19:43:50	0|gribble|https://github.com/bitcoin/bitcoin/issues/10143 | [net] Allow disconnectnode RPC to be called with node id by jnewbery · Pull Request #10143 · bitcoin/bitcoin · GitHub
382 2017-04-13 19:43:58	0|gmaxwell|sipa: gotcha.
383 2017-04-13 19:44:31	0|gmaxwell|cfields_: We're going to have to take a non-downgradable change for per-txo at some point, I don't think never doing it is a realistic option, the improvement is too significant.
384 2017-04-13 19:44:34	0|wumpus|jnewbery: makes sense, it's still fairly safe to rename named arguments now, but they should be backported
385 2017-04-13 19:45:16	0|gmaxwell|cfields_: the overall changes to the dbcache are too burdensom to realisitcally support a version that can support both.
386 2017-04-13 19:45:36	0|sipa|i still wish for non-atomic flush in 0.14.2
387 2017-04-13 19:45:38	0|cfields_|gmaxwell: sure, makes sense. is it worth trying to slip in a bit of smarts into 0.14.1/0.14.2 gracefully saying that (in the event of an attempted downgrade) the format has changed?
388 2017-04-13 19:46:03	0|gmaxwell|cfields_: yes, that would be more reasonable than getting a generic corruption message.
389 2017-04-13 19:46:11	0|wumpus|cfields_: yes, something for 0.14.2
390 2017-04-13 19:46:13	0|sipa|cfields_: we could add a change in 0.14 that would make explicit downgrade easy
391 2017-04-13 19:46:14	0|morcos|sipa: too big a change
392 2017-04-13 19:46:18	0|wumpus|no more new things for 0.14.1 please
393 2017-04-13 19:46:19	0|cfields_|.2, ignore for .1.
394 2017-04-13 19:46:33	0|gmaxwell|"Your database format is from the future, so sad for you. Please run reindex."
395 2017-04-13 19:46:36	0|cfields_|wumpus: sorry, realized my mistake too late :)
396 2017-04-13 19:46:48	0|sipa|morcos: i said wish, not demand :)
397 2017-04-13 19:47:24	0|sdaftuar|sipa: feature freeze for 0.15 is 2017-07-16
398 2017-04-13 19:47:31	0|luke-jr|sh AI sipa in 0.14.2
399 2017-04-13 19:47:44	0|gmaxwell|T1000 is back.
400 2017-04-13 19:47:46	0|sipa|sdaftuar: yeah, i saw, wumpus gave link
401 2017-04-13 19:48:06	0|luke-jr|I wish*
402 2017-04-13 19:48:34	0|gmaxwell|/mode #bitcoin-dev +b *!*@178.62.68.75
403 2017-04-13 19:48:40	0|luke-jr_|:(
404 2017-04-13 19:48:53	0|luke-jr_|I can't think of a good nick :(
405 2017-04-13 19:49:01	0|wumpus|apparently we need to make meetings invite-to-speak only
406 2017-04-13 19:49:17	0|wumpus|stupid childish trolls are why we can't have nice things
407 2017-04-13 19:49:18	0|kanzure|you used wrong ip address last time
408 2017-04-13 19:49:36	0|spudowiar|wumpus: If you did that, I wouldn't be able to participate :(
409 2017-04-13 19:49:40	0|gmaxwell|wumpus: lets discuss in the meta meeting. :P
410 2017-04-13 19:49:46	0|luke-jr_|spudowiar: Well, no one gives a fuck
411 2017-04-13 19:49:53	0|cfields_|kanzure: no need, they're easy to spot. just ignore.
412 2017-04-13 19:49:56	0|gmaxwell|(after the meeting)
413 2017-04-13 19:50:20	0|gmaxwell|in any case, lots of good things in flight. I think progress for 15 is good right now.
414 2017-04-13 19:50:27	0|BlueMatt|ok, so 10 minutes left
415 2017-04-13 19:50:29	0|BlueMatt|ryanofsky: ?
416 2017-04-13 19:50:34	0|BlueMatt|did you want to talk about multi-proces
417 2017-04-13 19:50:44	0|BlueMatt|also please folks list your review-priority for this week :)
418 2017-04-13 19:50:56	0|spudowiar|If that gets merged I will hug ryanofsky :)
419 2017-04-13 19:50:59	0|wumpus|spudowiar: I don't see why not; would just make it more work to invite everyone
420 2017-04-13 19:51:12	0|ryanofsky|not sure, what there is to say, #10102 is not complete, but the code that's there could use some review
421 2017-04-13 19:51:13	0|gribble|https://github.com/bitcoin/bitcoin/issues/10102 | bitcoin-qt: spawn bitcoind and communicate over pipe (Experimental, WIP) by ryanofsky · Pull Request #10102 · bitcoin/bitcoin · GitHub
422 2017-04-13 19:51:20	0|spudowiar|wumpus: No one would invite me :(
423 2017-04-13 19:51:37	0|sipa|i'm not sure what the appeal of multi process is
424 2017-04-13 19:51:38	0|wumpus|#topic multiprocess
425 2017-04-13 19:51:48	0|wumpus|process isolation
426 2017-04-13 19:52:01	0|sipa|i'd like to see the wallet go into a separate process from the networking
427 2017-04-13 19:52:03	0|jonasschnelli|sipa: #10102
428 2017-04-13 19:52:05	0|gribble|https://github.com/bitcoin/bitcoin/issues/10102 | bitcoin-qt: spawn bitcoind and communicate over pipe (Experimental, WIP) by ryanofsky · Pull Request #10102 · bitcoin/bitcoin · GitHub
429 2017-04-13 19:52:19	0|BlueMatt|sipa: then review my PRs, those do the first step of splitting it off neatly
430 2017-04-13 19:52:26	0|cfields_|ryanofsky: I've been wanting to split out net into a separate process, but the barrier was creating an IPC. So I'm all for it.
431 2017-04-13 19:52:30	0|BlueMatt|then we can use ryanofsky's multiprocess stuff to do the process :)
432 2017-04-13 19:52:32	0|sipa|that that's qt from the rest
433 2017-04-13 19:52:32	0|wumpus|not having the gui and the rest in one address space would be a good start
434 2017-04-13 19:52:50	0|sipa|that seems like a pointless gimmick to me
435 2017-04-13 19:53:02	0|jonasschnelli|Yes. The wallet seems to be more important.
436 2017-04-13 19:53:06	0|wumpus|sigh
437 2017-04-13 19:53:11	0|gmaxwell|If it could disconect and reconnect it would have some value.
438 2017-04-13 19:53:16	0|ryanofsky|agreed the wallet is more important
439 2017-04-13 19:53:30	0|jonasschnelli|not saying the Qt part is not important
440 2017-04-13 19:53:42	0|jnewbery|I'd like one process per wallet for multi-wallet
441 2017-04-13 19:53:42	0|ryanofsky|reason for starting with qt is that wallet work will depend on matt's changes
442 2017-04-13 19:53:44	0|spudowiar|Oh, I thought that was wallet as well (my offer of hug has been withdrawn, ryanofsky)
443 2017-04-13 19:53:54	0|wumpus|the gui has more pressing problems though: too much happens synchronous with the core in the GUI loop
444 2017-04-13 19:54:06	0|jonasschnelli|wumpus: thats a good point
445 2017-04-13 19:54:10	0|ryanofsky|and qt seemed like it might be a less controversial place to start since we already have separate bitcoind and bitcoin-qt executables
446 2017-04-13 19:54:19	0|wumpus|ryanofsky: yes I agree
447 2017-04-13 19:55:04	0|sipa|hmm, i withdraw my comment about it being pointless
448 2017-04-13 19:55:04	0|wumpus|ryanofsky: would make sense to not have them share all that code
449 2017-04-13 19:55:11	0|gmaxwell|I've read the PR.
450 2017-04-13 19:55:35	0|wumpus|also having one example of IPC between process could make other splits easier
451 2017-04-13 19:55:41	0|gmaxwell|But this is not a good design.
452 2017-04-13 19:55:47	0|gmaxwell|as such an example.
453 2017-04-13 19:55:58	0|jnewbery|5 minutes left.
454 2017-04-13 19:55:59	0|jnewbery|Suggested topic: PR review requests
455 2017-04-13 19:56:12	0|wumpus|gmaxwell: have you commented about the design in the PR?
456 2017-04-13 19:56:20	0|wumpus|#topic high-priority PR review requests
457 2017-04-13 19:56:34	0|BlueMatt|#10179
458 2017-04-13 19:56:34	0|gribble|https://github.com/bitcoin/bitcoin/issues/10179 | Give CValidationInterface Support for calling notifications on the CScheduler Thread by TheBlueMatt · Pull Request #10179 · bitcoin/bitcoin · GitHub
459 2017-04-13 19:56:35	0|BlueMatt|for me
460 2017-04-13 19:57:10	0|sipa|#9792 plz
461 2017-04-13 19:57:12	0|gribble|https://github.com/bitcoin/bitcoin/issues/9792 | FastRandomContext improvements and switch to ChaCha20 by sipa · Pull Request #9792 · bitcoin/bitcoin · GitHub
462 2017-04-13 19:57:12	0|morcos|wumpus: #9942 can be merged i think which will at least make the rest of the fee estimation changes smaller to review
463 2017-04-13 19:57:14	0|gribble|https://github.com/bitcoin/bitcoin/issues/9942 | Refactor CBlockPolicyEstimator by morcos · Pull Request #9942 · bitcoin/bitcoin · GitHub
464 2017-04-13 19:57:19	0|spudowiar|wumpus: For external signing, currently messing about with external command, using stdio but other IPC example would be useful
465 2017-04-13 19:57:25	0|spudowiar|Oops, topic's moved on, sorry
466 2017-04-13 19:57:27	0|gmaxwell|(What this PR does is pulls in a compat library to just break the function boundaries, to work across processes, it is not an actual IPC.  And for just making the GUI more concurrent and perhaps reconnectable that fine.  But that is not how we should seperate the wallet.
467 2017-04-13 19:57:32	0|gmaxwell|)
468 2017-04-13 19:57:43	0|gmaxwell|wumpus: I can comment more if I haven't.
469 2017-04-13 19:58:11	0|spudowiar|Wallet communicating over the JSON-RPC interface would be quite good, but then you have the issue that wallet functionality is provided over the JSON-RPC interface?
470 2017-04-13 19:58:24	0|wumpus|morcos: BlueMatt: ok added them to the project  https://github.com/bitcoin/bitcoin/projects/8
471 2017-04-13 19:58:27	0|jnewbery|I'd like some review on #10044. The concept had some ACKs in the meeting a few weeks ago.
472 2017-04-13 19:58:28	0|gribble|https://github.com/bitcoin/bitcoin/issues/10044 | Run functional tests in `make check` by jnewbery · Pull Request #10044 · bitcoin/bitcoin · GitHub
473 2017-04-13 19:58:42	0|wumpus|also sipa's
474 2017-04-13 19:58:47	0|morcos|wumpus: ok, in 9942 case i think it has enough acks already
475 2017-04-13 19:59:04	0|cfields_|jnewbery: will do.
476 2017-04-13 19:59:17	0|wumpus|jnewbery: also added
477 2017-04-13 19:59:36	0|luke-jr|spudowiar: JSON-RPC already provides wallet..
478 2017-04-13 20:00:01	0|BlueMatt|DONG
479 2017-04-13 20:00:04	0|wumpus|#endmeeting
480 2017-04-13 20:00:05	0|lightningbot|Log:            http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-04-13-19.00.log.html
481 2017-04-13 20:00:05	0|lightningbot|Meeting ended Thu Apr 13 20:00:04 2017 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
482 2017-04-13 20:00:05	0|lightningbot|Minutes:        http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-04-13-19.00.html
483 2017-04-13 20:00:05	0|lightningbot|Minutes (text): http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-04-13-19.00.txt
484 2017-04-13 20:00:13	0|BlueMatt|gmaxwell: so is your objection that qt and other things dont need separated, or that this isnt the right approach?
485 2017-04-13 20:00:16	0|spudowiar|luke-jr: That's the point I was making. If you have the wallet talking to the daemon over JSON-RPC, how can you provide the wallet functionality over JSON-RPC?
486 2017-04-13 20:00:24	0|spudowiar|luke-jr: I'm just really bad at wording stuff :)
487 2017-04-13 20:00:33	0|jnewbery|cfields_ wumpus thanks
488 2017-04-13 20:00:38	0|spudowiar|petertodd: That's why we need OpenPGP Card 4.0 :)
489 2017-04-13 20:00:52	0|spudowiar|petertodd: So that the smart card can request bits of the data, etc. and display it on the display
490 2017-04-13 20:00:53	0|petertodd|spudowiar: it'll have to have a display at least... fudnemental probelm
491 2017-04-13 20:01:00	0|spudowiar|petertodd: TREZOR
492 2017-04-13 20:01:09	0|spudowiar|petertodd: I have a working OpenPGP implementation for TREZOR
493 2017-04-13 20:01:13	0|spudowiar|Only does ECDSA signing though
494 2017-04-13 20:01:17	0|petertodd|spudowiar: heh, git support for trezor...
495 2017-04-13 20:01:17	0|spudowiar|Rewriting it in Rust though
496 2017-04-13 20:01:30	0|spudowiar|petertodd: I signed a Git commit with it :)
497 2017-04-13 20:01:42	0|petertodd|spudowiar: oh, theres a rust impl for the trezor uc?
498 2017-04-13 20:02:04	0|gmaxwell|BlueMatt: QT being seperated can improve concurrency and ... if done right allow it to turn off and on, while the daemon keeps running. Which I think are moderately useful things.   And the way that it is done here is more or less okay for just accomplishing that.  But it not a way that we should seperate the wallet, it is HIGHLY trusting, it wouldn't support multiple concurrent users, it doesn'
499 2017-04-13 20:02:08	0|spudowiar|petertodd: Rust runs on pretty much anything (there's an LLVM backend for STM32*)
500 2017-04-13 20:02:10	0|gmaxwell|t provide for a stable interface. etc.
501 2017-04-13 20:02:26	0|petertodd|spudowiar: nice!
502 2017-04-13 20:02:31	0|luke-jr|bbl
503 2017-04-13 20:02:33	0|gmaxwell|wumpus: metametting we should +v all the regulars and then if there is noise we can set the channel +m and +v the few extra non-regulars that aren't causing trouble.
504 2017-04-13 20:02:36	0|petertodd|spudowiar: I've got a big project I'm doing in rust right now
505 2017-04-13 20:02:44	0|[b__b]|You're doing good work, petertodd!
506 2017-04-13 20:02:44	0|gribble|Error: "m" is not a valid command.
507 2017-04-13 20:02:44	0|spudowiar|!m petertodd
508 2017-04-13 20:02:51	0|spudowiar|Go away gribble
509 2017-04-13 20:03:08	0|petertodd|heh
510 2017-04-13 20:03:31	0|spudowiar|petertodd: https://pgp.mit.edu/pks/lookup?op=vindex&search=0xEDD78A88C5D03B56 is the key generated by my TREZOR
511 2017-04-13 20:03:38	0|luke-jr|(not sure if anyone's fixed 0.14.1 relnotes confusing ppl re miner signalling.. but it should probably get fixed befoe firal)
512 2017-04-13 20:03:57	0|petertodd|spudowiar: nice!
513 2017-04-13 20:04:04	0|spudowiar|https://github.com/trezor/trezor-mcu/pull/133 is the firmware code (written in C)
514 2017-04-13 20:04:08	0|jonasschnelli|spudowiar: Yeah. Nice stuff...
515 2017-04-13 20:04:41	0|spudowiar|Only thing stopping me from getting this all finished is school :(
516 2017-04-13 20:05:21	0|jonasschnelli|skip school
517 2017-04-13 20:05:24	0|spudowiar|lol
518 2017-04-13 20:05:25	0|spudowiar|Wait...
519 2017-04-13 20:05:33	0|spudowiar|I actually have a video of this :)
520 2017-04-13 20:05:53	0|jcorgan|of you skipping school?
521 2017-04-13 20:05:58	0|petertodd|spudowiar: I dropped out of a physics degree for bitcoin (though I was doing it part time while working at a startup...)
522 2017-04-13 20:05:59	0|wumpus|gmaxwell: yes, seems a good idea
523 2017-04-13 20:06:45	0|cfields_|petertodd: you had that lucrative art degree as a fallback, though :p
524 2017-04-13 20:06:57	0|petertodd|cfields_: lol
525 2017-04-13 20:07:47	0|spudowiar|https://twitter.com/spudowiar/status/784429631545958400
526 2017-04-13 20:07:50	0|spudowiar|NO, THAT IS THE WRONG VIDEO
527 2017-04-13 20:07:54	0|spudowiar|https://twitter.com/spudowiar/status/802960101992759296
528 2017-04-13 20:07:55	0|spudowiar|Sorry
529 2017-04-13 20:08:20	0|jcorgan|petertodd: worst case you can always rely on bridgette's money
530 2017-04-13 20:08:22	0|spudowiar|Someone asked me if it worked on Windows so I recorded that for them :)
531 2017-04-13 20:09:02	0|petertodd|jcorgan: heh
532 2017-04-13 20:09:16	0|spudowiar|Wait... petertodd liked that video? When...?
533 2017-04-13 20:09:26	0|spudowiar|Oh wait, late Twitter notifications
534 2017-04-13 20:09:35	0|petertodd|spudowiar: just now :)
535 2017-04-13 20:11:14	0|spudowiar|petertodd: The firmware in the Pong video was written in Rust
536 2017-04-13 20:12:49	0|petertodd|spudowiar: nice! I'm rather excited for rust on embedded; I used to do a bunch of asm and C uC stuff, and it always kinda sucked
537 2017-04-13 20:17:08	0|BlueMatt|re: splitting qt process: can we use our own serialization instead of capnproto, ryanofsky?
538 2017-04-13 20:18:51	0|ryanofsky|BlueMatt: yes we can, i'm working on splitting up the change into two prs, one adding ipc interface and another adding capnproto glue to show how separation works
539 2017-04-13 20:18:53	0|wumpus|the gui has more pressing problems though: too much happens synchronous with the core in the GUI loop <- in any case, this needs to be addressed first, or as part of the move to a seprate process
540 2017-04-13 20:19:06	0|wumpus|the GUI thread should never block on a IPC request
541 2017-04-13 20:20:16	0|wumpus|(currently it blocks on calls to core, or while taking the cs_main lock, resulting in lack of user response for significant time)
542 2017-04-13 20:21:04	0|spudowiar|sipa: If I'm parsing a scriptPubKey using Solver, I get a CPubKey. How to get to a CKeyMetadata from there? (to get hdMasterKeyID)
543 2017-04-13 20:21:30	0|gmaxwell|wumpus: Can you explain shimming in the IPC address those blocks?-- if none of the dataflow is changed.
544 2017-04-13 20:22:06	0|ryanofsky|i'm not changing gui locking / blocking in my pr. if there are cases where gui is freezing up, the only way to address those is individually
545 2017-04-13 20:22:31	0|wumpus|gmaxwell: shimming?
546 2017-04-13 20:23:18	0|ryanofsky|the only impact my will have on gui responsiveness is there are a lot of ipc calls being called in a tight loop where they might be noticablely slower than current function calls
547 2017-04-13 20:23:37	0|ryanofsky|or ipc calls that have to serialize / deserialize a large amount of data
548 2017-04-13 20:23:37	0|wumpus|ryanofsky: that's kind of my problem with it; it may compound thecurrent issues
549 2017-04-13 20:23:41	0|gmaxwell|my read on ryanofsky's patch is that it just inserts IPC dispatch at these communication points. The software would still block no less as a result.  By shimming I mean replacing  function() with  ipc->function->ipc.
550 2017-04-13 20:23:59	0|ryanofsky|wumpus, yes, we will have to see what performance will be
551 2017-04-13 20:23:59	0|wumpus|gmaxwell: that's why I say that should be addressed first
552 2017-04-13 20:24:26	0|wumpus|gmaxwell: one everything the GUI does is asynchronous, making it work over IPC is also a lot easier
553 2017-04-13 20:24:30	0|ryanofsky|but i do think any performance problems that are uncovered can be dealt with
554 2017-04-13 20:24:39	0|wumpus|gmaxwell: that was my original plan for the GUI but I never had time to finish it :(
555 2017-04-13 20:24:45	0|gmaxwell|wumpus: ah!
556 2017-04-13 20:24:57	0|ryanofsky|it seems to me a lot easier to fix performance problems that come up than "make everything asyncrhonous"
557 2017-04-13 20:25:07	0|wumpus|easier, but not better
558 2017-04-13 20:25:10	0|gmaxwell|(I was thinking you thought the ipc changes did this already and thought I was confused).
559 2017-04-13 20:25:29	0|wumpus|no, ideally IPC chnanges would do that
560 2017-04-13 20:25:36	0|wumpus|but these don't, no
561 2017-04-13 20:25:39	0|gmaxwell|ryanofsky: is it your thought that eventually the GUI could be started and stopped independantly of the backend?
562 2017-04-13 20:26:13	0|wumpus|making everything asynchronous would improve the user experience a lot
563 2017-04-13 20:26:35	0|wumpus|as well as avoid silly things like the gui not coming up because it's blocked on one value that it needs to get under a cs_main
564 2017-04-13 20:26:40	0|ryanofsky|gmaxwell, supporting that would be pretty easy, but i don't know what the use-cases would be
565 2017-04-13 20:27:00	0|wumpus|ryanofsky: the typical windows service idea: have a GUI to manage it, that runs only part of the time
566 2017-04-13 20:27:19	0|gmaxwell|If not then I really do not understand the purpose of IPC seperating it at all. Making it (largely)-async is an independant issue.  The kind of approach here is tightly coupled, so it couldn't really support a GUI in a seperate codebase, multiple GUI projects, or concurrent multiple GUIs.
567 2017-04-13 20:27:33	0|gmaxwell|The advantage of being able to start and stop is so the GUI doesn't run when it's not needed.
568 2017-04-13 20:27:39	0|spudowiar|Oh, wait, I can call CWallet::LoadKeyMetadata!
569 2017-04-13 20:27:49	0|wumpus|well the cap'n'proto thing could theoretically work with a separate process
570 2017-04-13 20:27:57	0|wumpus|codebase*
571 2017-04-13 20:28:09	0|wumpus|they only have to share the interface definition
572 2017-04-13 20:28:20	0|ryanofsky|gmaxwell, i think the approach can support all of those things. but my initial plan for the pr is just to change the infrastructure without exposing new features
573 2017-04-13 20:28:23	0|gmaxwell|wumpus: yes but without a designed and clean interface you can't reasonably support that.
574 2017-04-13 20:28:38	0|wumpus|gmaxwell: maybe not in the first version
575 2017-04-13 20:28:42	0|gmaxwell|And you get the ZMQ like "everything has to run exactly the same version of ZMQ" issue.
576 2017-04-13 20:28:57	0|wumpus|gmaxwell: but I think you want too much at the same time
577 2017-04-13 20:29:28	0|ryanofsky|gmaxwell, my pr defines an interface
578 2017-04-13 20:29:42	0|gmaxwell|ISTM if the goal is those things, what is needed is a RPC.  This just seems like a 170 degree turn from existing plans on seperating the wallet as a SPV client with special messages for mempool and whatnot.
579 2017-04-13 20:29:51	0|wumpus|baby steps sometimes make sense so that there is at least progress, the interface he defines may not be super clean yet but it's at least a start
580 2017-04-13 20:30:01	0|gmaxwell|and replacing it with something that we'll never be able to precisely define or secure.
581 2017-04-13 20:30:20	0|wumpus|anyhow I don't think we'll ever make progress here this way, we just all want too different things
582 2017-04-13 20:30:22	0|sipa|spudowiar: no
583 2017-04-13 20:30:31	0|ryanofsky|how is the interface not precisely designed & secure?
584 2017-04-13 20:30:46	0|ryanofsky|i mean defined not designed
585 2017-04-13 20:30:54	0|sipa|spudowiar: that method is for loading metadata into the wallet
586 2017-04-13 20:31:31	0|spudowiar|sipa: Yeah, I just saw that
587 2017-04-13 20:31:44	0|gmaxwell|ryanofsky: where is a wire specification for captin proto that can be independantly implemented?
588 2017-04-13 20:32:14	0|ryanofsky|oh this is just an objection to capnproto?
589 2017-04-13 20:32:17	0|wumpus|ryanofsky: even then, you're not currently opening it up, this first experimental version in the PR is internal only
590 2017-04-13 20:32:43	0|ryanofsky|if capnproto is a real problem, a different protocol could be dropped in instead
591 2017-04-13 20:33:01	0|gmaxwell|ryanofsky: yea, a lot of my concern is the approach of taking an internal boundary and then wrapping it in captinproto (which I have specific concerns about which might be outdated).
592 2017-04-13 20:33:02	0|wumpus|once you have a better idea of what you need you can improve the protocol, then eventually open it up to other applications maybe
593 2017-04-13 20:33:11	0|ryanofsky|most of the work i'm doing here is in making qt not call wallet/node functions directly
594 2017-04-13 20:33:38	0|ryanofsky|if you have suggestions for another wire format to use instead of capnproto, that'd be helpful
595 2017-04-13 20:33:42	0|gmaxwell|ryanofsky: that stuff all sounds great to me independant of the rest.
596 2017-04-13 20:34:16	0|wumpus|what I like about cap'n'proto is that the interface is cleanly defined in the .capnp file, other software could in principle use that interface definition and just communicate with it
597 2017-04-13 20:34:39	0|ryanofsky|there is a lot i like about capnproto, but so far it actually requires a lot more boilerplate than i was expected, and i'm not at all wedded to it
598 2017-04-13 20:34:39	0|wumpus|e.g. https://github.com/bitcoin/bitcoin/pull/10102/files#diff-1832023b80d9337ec49f60646c7a9c5dR1
599 2017-04-13 20:35:37	0|gmaxwell|wumpus: yes so long as they also take all of captin proto, of the same vintage.
600 2017-04-13 20:35:59	0|wumpus|gmaxwell: yes, they have to use the same IPC library
601 2017-04-13 20:36:12	0|wumpus|gmaxwell: you'd want an independent implementation of the protocol?
602 2017-04-13 20:37:10	0|wumpus|I vaguely remember the wire format for capnp is documented, but can't find it quickly
603 2017-04-13 20:37:14	0|gmaxwell|that comes back to IPC vs RPC.  I believe the wallet seperation should be a RPC.  I don't really have any opinion in GUI IPC seperation, beyond being able to independantly start and stop it, I don't see the value-- but I acknoweldge and respect that people who know more about the GUI parts may see value in it that I don't.
604 2017-04-13 20:37:51	0|sipa|well it would be nice to just have bitcoind running, and occasionally connect your qt frontend to it
605 2017-04-13 20:37:59	0|gmaxwell|If it truly just is an IPC then I don't care much about requring the same versions or an interface that wasn't really designed and documented and supported as a public interface.
606 2017-04-13 20:37:59	0|wumpus|https://capnproto.org/encoding.html
607 2017-04-13 20:38:03	0|gmaxwell|sipa: agreed.
608 2017-04-13 20:38:22	0|ryanofsky|i don't really see a significant difference between ipc / rpc here. either way you have bytes going across a socket. the only difference is whether you bind the socket to a port or not
609 2017-04-13 20:38:43	0|sipa|ryanofsky: i think what gmaxwell means is whether it's a well defined interface
610 2017-04-13 20:38:53	0|sipa|will the gui and server always be the same version
611 2017-04-13 20:39:04	0|sipa|or do they need to interact across different versions
612 2017-04-13 20:39:12	0|wumpus|at first: no
613 2017-04-13 20:39:13	0|ryanofsky|sipa, that's up to us
614 2017-04-13 20:39:14	0|gmaxwell|Well defined and stable, right.
615 2017-04-13 20:39:42	0|sipa|ryanofsky: and i think gmaxwell means that for the Qt split, no well defined and stable interface is needed
616 2017-04-13 20:39:42	0|wumpus|there's no reason that couldn't be added, but it doesn't seem required for doing this in the first place
617 2017-04-13 20:39:49	0|sipa|but for the wallet separation there is
618 2017-04-13 20:40:17	0|gmaxwell|Also how much do you trust the code you're talking to.  With captin proto, if the far end does something unexpected (much less malicious) access to the objects long after the IPC completes will throw exceptions.
619 2017-04-13 20:40:19	0|ryanofsky|capnproto supports extending interfaces in backwards compatible ways, that's the reason for all the field numbers & interface method numbers in the schema file
620 2017-04-13 20:40:35	0|wumpus|ryanofsky: indeed, they have thought of this
621 2017-04-13 20:41:10	0|gmaxwell|And thats fine for gui/rest since it's all one codebase, from one group.. and of course function calls also do crazy things if you get anything wrong.
622 2017-04-13 20:41:18	0|wumpus|but yes secure IPC for sandboxed code is difficult, and maybe cap'\n'proto is not the best suited for that
623 2017-04-13 20:41:19	0|gmaxwell|But that isn't what we want for wallet seperation.
624 2017-04-13 20:41:51	0|sipa|i think for wallet separation we should just use p2p
625 2017-04-13 20:42:17	0|sipa|and have the wallet side implement spv
626 2017-04-13 20:42:27	0|gmaxwell|That is what we've always planned, and I think it is good and as a side effect mostly written already.
627 2017-04-13 20:42:40	0|sipa|and designed for
628 2017-04-13 20:44:04	0|wumpus|looked at chromium: for IPC they use a serialization system (like ours) called pickle, and serialize/deserialize using that in the IPC entry points where needed, no interface compiler
629 2017-04-13 20:44:48	0|gmaxwell|thats totally not confusing with python. :P
630 2017-04-13 20:45:20	0|BlueMatt|wumpus: i think you can remove the merged PRs from https://github.com/bitcoin/bitcoin/projects/8
631 2017-04-13 20:45:51	0|BlueMatt|luke-jr: where are you on #8694? looks like you have a bunch of comments to fix?
632 2017-04-13 20:45:53	0|gribble|https://github.com/bitcoin/bitcoin/issues/8694 | Basic multiwallet support by luke-jr · Pull Request #8694 · bitcoin/bitcoin · GitHub
633 2017-04-13 20:46:28	0|ryanofsky|wumpus, chromium is in the middle of transitioning between ipc systems
634 2017-04-13 20:46:49	0|wumpus|communication (on unix) is done over a SOCK_DGRAM or SOCK_SEQPACKET socketpair, they support some advanced things like sending over file descriptors
635 2017-04-13 20:46:57	0|wumpus|ryanofsky: oh?
636 2017-04-13 20:46:58	0|ryanofsky|older one is based on c macros, newer one is a stripped down fork of https://github.com/domokit/mojo
637 2017-04-13 20:48:13	0|ryanofsky|mojo is similar to capnproto, but supports passing file descriptors & shared memory over the socket, not just sending bytes
638 2017-04-13 20:48:37	0|wumpus|didn't know about mojo
639 2017-04-13 20:49:09	0|ryanofsky|this page describes the older chromium ipc mechanism: https://www.chromium.org/developers/design-documents/inter-process-communication
640 2017-04-13 20:49:49	0|ryanofsky|yeah i actually looked into using mojo before capnproto
641 2017-04-13 20:50:08	0|ryanofsky|but the mojo project itself is actually dead
642 2017-04-13 20:50:53	0|ryanofsky|there are now two forks of mojo, one is internal to chromium, and the other is internal to google's fuschia os project
643 2017-04-13 20:51:03	0|wumpus|so they don't use the project itself, just copied some files to their project?
644 2017-04-13 20:51:20	0|ryanofsky|copied & stripped out parts they didn't need
645 2017-04-13 20:51:38	0|ryanofsky|https://chromium.googlesource.com/chromium/src/+/master/mojo
646 2017-04-13 20:57:08	0|wumpus|yes that seems quite close to cap'n'proto
647 2017-04-13 21:19:55	0|bitcoin-git|[13bitcoin] 15theuni opened pull request #10205: net: define NodeId as an int64_t (060.14...0610176-backport) 02https://github.com/bitcoin/bitcoin/pull/10205
648 2017-04-13 21:42:11	0|bitcoin-git|[13bitcoin] 15dramaticbulgarian opened pull request #10206: Chainparams: In ReceivedBlockTransactions() (06master...06chainparams) 02https://github.com/bitcoin/bitcoin/pull/10206
649 2017-04-13 22:14:27	0|TheV01D|lo
650 2017-04-13 23:17:06	0|bitcoin-git|[13bitcoin] 15dramaticbulgarian closed pull request #10206: Chainparams: In ReceivedBlockTransactions() (06master...06chainparams) 02https://github.com/bitcoin/bitcoin/pull/10206