1 2016-04-04 00:19:49	0|midnightmagic|is someone still mirroring the PRs, etc?
  2 2016-04-04 01:00:10	0|gmaxwell|morcos: I am boggle at +class CompareTxMemPoolEntryByScore
  3 2016-04-04 01:00:38	0|gmaxwell|oh no I'm not.
  4 2016-04-04 01:00:54	0|gmaxwell|turns out that b and a look a lot like each other on the screen.
  5 2016-04-04 07:06:25	0|wumpus|does anyone know of a profiling/monitoring tool that can how how much % of the time a lock is held and by what threads?
  6 2016-04-04 07:09:45	0|wumpus|(for linux)
  7 2016-04-04 07:13:24	0|wumpus|thanks for all the work on the tests MarcoFalke
  8 2016-04-04 07:21:26	0|Lightsword|wumpus, I think valgrind may have something for that
  9 2016-04-04 07:21:44	0|wumpus|thanks
 10 2016-04-04 07:22:30	0|Lightsword|http://valgrind.org/docs/manual/drd-manual.html#drd-manual.lock-contention
 11 2016-04-04 07:24:25	0|sipa|if contention is really significant i expect it to show up in profilimg
 12 2016-04-04 07:25:53	0|wumpus|yes the problem I usually have with profiling is that it gives way too much information, where I just need to know a specific thing :)
 13 2016-04-04 07:26:14	0|sipa|haha
 14 2016-04-04 07:26:43	0|wumpus|linux has about a gazillion tools to monitor the system, or a process, but I can hardly ever find the one I need when I need it
 15 2016-04-04 07:28:20	0|wumpus|something that shows a timeline *wtf is this process doing now*, per thread, in a kind of GANTT graph would be awesome, we had something like that for Sun at ASML but I have no clue whether it exists for linux
 16 2016-04-04 07:28:37	0|Lightsword|dtrace?
 17 2016-04-04 07:28:49	0|wumpus|yes it was a proprietary tool based on dtrace
 18 2016-04-04 07:29:28	0|Lightsword|https://github.com/dtrace4linux/linux
 19 2016-04-04 07:29:57	0|wumpus|yes saw that one, there's also systemtap, and oprofile, which have similar aims but use slightly different mechanisms
 20 2016-04-04 07:30:24	0|wumpus|oh and sysdig!
 21 2016-04-04 07:33:06	0|Lightsword|because who doesn’t like to have 10 different profiling tools that work slightly differently :P
 22 2016-04-04 07:33:12	0|wumpus|:-)
 23 2016-04-04 07:34:09	0|wumpus|well I suppose people who specialize in this appreciate the flexibility, but if you just want to investigate something quickly it's easy to get stuck in the "what tool to choose... oh wtf I'll just roll something myself" stage
 24 2016-04-04 07:36:24	0|wumpus|oh yes don't get me started on programming languages :) we have to rewrite every single thing and library in every programming language, sometimes it's good but sometimes it's also a waste, what could we accomplish if we just cooporated on a single version of *thing* that works awesomely
 25 2016-04-04 07:36:37	0|wumpus|"we" is general, especially the open source community
 26 2016-04-04 07:37:34	0|wumpus|for c++ we even want multiple version of *thing* based on what the favourite combination of dependencies of the day is
 27 2016-04-04 07:37:59	0|Lightsword|heh, seems that’s why c is so common with linux stuff, since those libraries are pretty much compatible with any language
 28 2016-04-04 07:44:15	0|jonasschnelli|Any final feelings about RBF for createrawtx (https://github.com/bitcoin/bitcoin/pull/7159/files)?
 29 2016-04-04 07:44:28	0|jonasschnelli|Should I remove the general opt-in boolean?
 30 2016-04-04 07:44:33	0|jonasschnelli|And add a seq-nr per input?
 31 2016-04-04 07:44:51	0|jonasschnelli|I like both approaches.
 32 2016-04-04 07:45:34	0|jonasschnelli|The per-tx-global opt-in bool is a higher level function. The seq-nr per input is something we should have anyways.
 33 2016-04-04 07:45:42	0|jonasschnelli|Maybe we can have both?
 34 2016-04-04 07:49:53	0|jonasschnelli|Wait. The sequence option was added already.
 35 2016-04-04 07:50:38	0|gmaxwell|needs nlock too.. globals could be used to autopopulate sequence/nlock if not overridden.
 36 2016-04-04 07:52:01	0|sipa|hmm, i'm unsure
 37 2016-04-04 07:52:15	0|sipa|the question is what createrawtransaction's goal os
 38 2016-04-04 07:52:37	0|sipa|just only using explicit values would be more in line with its name
 39 2016-04-04 07:52:44	0|gmaxwell|it bothers me that rawtxn are trivially distinguishable from txn created by bitcoin core itself for no good reason... and it's weird that there is _less_ functionality via rawtxn now.
 40 2016-04-04 07:53:26	0|sipa|but given that it can now be used in conjunction in fundrawtransaction to just choose outputs, maybe there is a need fir slightly higher level behaviour
 41 2016-04-04 07:53:49	0|gmaxwell|why make it gratitiously less useful?
 42 2016-04-04 07:53:51	0|jonasschnelli|I'm also not sure if passing a CREATE_TX_RBF_OPT_IN - flag to FundTransaction is the way to go: https://github.com/bitcoin/bitcoin/pull/7159/files#diff-12635a58447c65585f51d32b7e04075bR747
 43 2016-04-04 07:58:50	0|GitHub77|13bitcoin/06master 148efed3b 15Jonas Schnelli: [Qt] Support for abandoned/abandoning transactions
 44 2016-04-04 07:58:50	0|GitHub77|[13bitcoin] 15jonasschnelli pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/e662a7628801...a9149688f87c
 45 2016-04-04 07:58:51	0|GitHub77|13bitcoin/06master 14a914968 15Jonas Schnelli: Merge #7707: [RPC][QT] UI support for abandoned transactions...
 46 2016-04-04 07:59:00	0|GitHub144|[13bitcoin] 15jonasschnelli closed pull request #7707: [RPC][QT] UI support for abandoned transactions (06master...062016/03/abandon_ui) 02https://github.com/bitcoin/bitcoin/pull/7707
 47 2016-04-04 08:02:48	0|sipa|gmaxwell: i dislike feature creep
 48 2016-04-04 08:03:26	0|gmaxwell|I dislike having functionality which doesn't get used but for small shortcoming that take away its utility.
 49 2016-04-04 08:03:26	0|sipa|making createrawtransaction do things automagically may be confusing "i was making a raw transaction... why did it put things in the transaction i didn't tell it to?"
 50 2016-04-04 08:05:25	0|sipa|i just don't know how to reconcile it being "raw" with the suggested functionality
 51 2016-04-04 08:05:55	0|gmaxwell|raw is the fact that it returns a rawtransaction to you.
 52 2016-04-04 08:06:06	0|sipa|my preference would be just allowing setting nlocktime and nsequence
 53 2016-04-04 08:06:22	0|sipa|it also doesn't create change outputs automatocally
 54 2016-04-04 08:06:24	0|gmaxwell|Createrawtransaction is the only way to do manual coin selection from the command-line, it works great for that, and I use it that way several times per week.  It has a nice workflow that allows offline signing and allows review of the completed transaction before signing.
 55 2016-04-04 08:07:25	0|sipa|but have another RPC for higher level operations like this
 56 2016-04-04 08:07:53	0|gmaxwell|it would just be an exact copy of createrawtransaction, with extra flags and or slightly different defaults.
 57 2016-04-04 08:10:55	0|sipa|so you would suggest having the ability to set nlocktime and nsequence specifically, and also have options to change defaults based on rbf, transaction sniping, relative locktime, ...?
 58 2016-04-04 08:11:00	0|gmaxwell|I wouldn't object to just having another RPC like that... but it seems a waste.
 59 2016-04-04 08:11:34	0|gmaxwell|sipa: yes.
 60 2016-04-04 08:12:25	0|gmaxwell|the problem with 'global defaults' however, is that I don't see a clean way to make it work with fundrawtransaction unless side information is added to the raw transaction.
 61 2016-04-04 08:12:43	0|sipa|so what if you want the rbf/sniping/rellocktime/... logic, but not createrawtransaction?
 62 2016-04-04 08:12:57	0|gmaxwell|sounds like arguments for sendmany.
 63 2016-04-04 08:13:20	0|sipa|so we'd add those to both sendmany and createrawtransaction?
 64 2016-04-04 08:14:53	0|sipa|just trying to think this through
 65 2016-04-04 08:14:56	0|gmaxwell|See any reason why we wouldn't? sendmany's workflow requires that the transaction it creates be valid now, that isn't so for the raw workflow.
 66 2016-04-04 08:15:10	0|gmaxwell|e.g. totally reasonable to use the rawworkflow to make a nlocked transaction for the future.
 67 2016-04-04 08:16:10	0|sipa|it's just so ugly to have both high level logic and the ability to override
 68 2016-04-04 08:16:28	0|sipa|say we didn't add csv support now, just optinrbf or not
 69 2016-04-04 08:16:53	0|sipa|someone starts using csv by setting the sequence number explicitly, but also passes optinrbf=no
 70 2016-04-04 08:17:06	0|sipa|and expects a transaction that is not replacable
 71 2016-04-04 08:18:18	0|gmaxwell|I get what you're saying, but a sequenced transaction implies at least sequence succession relplacability. But yes, ... that would not by far be the biggest footguns around that interface.
 72 2016-04-04 08:19:06	0|sipa|if createrawtransaction was instead just something that took an existing raw tx (empty be default?) and allowed adding inputs and outputs
 73 2016-04-04 08:33:55	0|jonasschnelli|sipa: the question is, do we think that rawtx users need to know the MAXINT-2 RBF opt-in seqnr? I think we should abstract these magic numbers from the rpc-users.
 74 2016-04-04 08:34:40	0|jonasschnelli|IMO it's on a different level then the rawtx abstraction
 75 2016-04-04 08:35:12	0|sipa|jonasschnelli: createrawtransaction users also need to know that creating less outouts than inputs will send their life savings to miners
 76 2016-04-04 08:35:29	0|jonasschnelli|sipa: not if they use fundraw. :)
 77 2016-04-04 08:35:54	0|sipa|i totally agree that there should be a way for users to need to learn all the ugly details ofnthe semantics of all fields
 78 2016-04-04 08:36:23	0|sipa|but so far, createrawtransaction has been the way to actually do control the exact values, and not the high level behaviour
 79 2016-04-04 08:36:55	0|sipa|and mixing them may be seen as a recommendation for people to use createrawtransaction when it actually cannot guarantee safety without learning the details
 80 2016-04-04 08:36:56	0|jonasschnelli|sipa: I agree. But knowing also the magic numbers we use for nSequence is another "level":
 81 2016-04-04 08:37:22	0|jonasschnelli|At least we should offer some nSequence=BIP125 abstraction.
 82 2016-04-04 08:37:35	0|sipa|yes, agree
 83 2016-04-04 08:37:40	0|jonasschnelli|The per-tx-general opt-in-to-Bip125 is probably to high. I agree.
 84 2016-04-04 08:37:41	0|gmaxwell|the magic nsquence is really MAX-1 which is not replacable when it logically should be (after all, the transaction is locktimed and non-final)
 85 2016-04-04 08:38:22	0|sipa|but you should not be using createrawtransaction if you don't know what the semantics of the raw transaction fields are, or you'll shoot yourself in the foot
 86 2016-04-04 08:38:30	0|jonasschnelli|Yes. Agree.
 87 2016-04-04 08:38:46	0|jonasschnelli|But the BIP125 MAXINT-2 is a policy we should offer directly.
 88 2016-04-04 08:39:17	0|jonasschnelli|{"txid": <txid>, "vout" : 0, "sequence": "BIP12"} <--- feels ugly
 89 2016-04-04 08:39:19	0|sipa|what will you do to add csv supoort to createrawtransaction?
 90 2016-04-04 08:40:04	0|sipa|i really dislike mixing magic smart behaviour with raw overriding
 91 2016-04-04 08:40:29	0|gmaxwell|I don't really like mixing types in a single argument name.
 92 2016-04-04 08:40:29	0|sipa|that feels like the rpc is trying to solve things at different levels
 93 2016-04-04 08:41:03	0|jonasschnelli|flags per input? FLAG_BIP125_RBF, SEQUENCE_LOCKTIME_TYPE_FLAG?
 94 2016-04-04 08:41:08	0|gmaxwell|e.g. sequence: 12345  / and seqtype: "BIP12" and have their use be mutually exclusive would seem preferable to me.
 95 2016-04-04 08:41:35	0|jonasschnelli|no. flags per input is bad.
 96 2016-04-04 08:41:41	0|sipa|i'd almost argue for a computensequencevalue RPC
 97 2016-04-04 08:41:55	0|sipa|which just gives you the number to use, which you can put in crt
 98 2016-04-04 08:42:08	0|sipa|but that's cumbersome for human users...
 99 2016-04-04 08:42:20	0|jonasschnelli|RPC is not made for "human" users.
100 2016-04-04 08:42:25	0|jonasschnelli|I think its fine...
101 2016-04-04 08:42:38	0|jonasschnelli|Chains of commands is something that increase the modularity
102 2016-04-04 08:43:41	0|gmaxwell|the RPC is our commandline interface as well, and its usefulness aids in discoverability which makes it much easier to use as an RPC too.
103 2016-04-04 08:43:52	0|gmaxwell|like having an REPL makes python accessible to people.
104 2016-04-04 08:43:55	0|jonasschnelli|A "setoptinrbf <rawtx>" is probably a bad design and leads to overriding ther nsequence magics
105 2016-04-04 08:44:02	0|sipa|jonasschnelli: agree
106 2016-04-04 08:44:15	0|sipa|optinrbf should be something you choose per tx
107 2016-04-04 08:44:24	0|jonasschnelli|gmaxwell: imo RPC != commandline, .. bitcoin-cli == commandline
108 2016-04-04 08:44:32	0|sipa|some receivers may not want such transactions
109 2016-04-04 08:44:33	0|jonasschnelli|bitcoin-cli could combine stuff
110 2016-04-04 08:45:00	0|jonasschnelli|sipa: "setoptinrbf" would be per rawtx.
111 2016-04-04 08:45:09	0|jonasschnelli|It would just set the nSequence number of all inputs to MAXINT-2
112 2016-04-04 08:45:12	0|gmaxwell|jonasschnelli: that would harm discoverability, I think it is unfortunate to create wildly different interfaces.
113 2016-04-04 08:45:20	0|jonasschnelli|But I don't like the override character.
114 2016-04-04 08:45:33	0|gmaxwell|jonasschnelli: I hope it wouldn't do that, I hope it would move anything from MAXINT/-1 to -2 and leave the rest alone.
115 2016-04-04 08:45:42	0|jonasschnelli|gmaxwell: Yes. It would add another level.. which I agree its bad.
116 2016-04-04 08:46:14	0|jonasschnelli|gmaxwell: Yes. That could be nice (move MAXINT/-1 to -2)
117 2016-04-04 08:46:41	0|jonasschnelli|but would "optinrbf <rawtx>" not be to prominent for a policy flag?
118 2016-04-04 08:46:42	0|gmaxwell|(and unix has worked for decades where the programmatic interface to unix commands is the same as the human one)
119 2016-04-04 08:47:37	0|sipa|our 'default' of using positional arguments does not help
120 2016-04-04 08:47:53	0|gmaxwell|jonasschnelli: I don't think that it's "policy" makes it any less useful to set it. From a api fanout perspective it might be better to have a tweakrawtxn that is multimodal.
121 2016-04-04 08:48:05	0|sipa|multimodal?
122 2016-04-04 08:48:13	0|CodeShark|https://bitcoincore.org/en/2016/04/04/announcing_sponsorship_programme/
123 2016-04-04 08:48:49	0|gmaxwell|positional arguments are sadness. but the json style of arguments is sadness squared. Thats a place where I think bitcoin-cli translating unix style arguments to json might be OK, as long as it was done consistently so someone could just learn the cli to rpc mapping.
124 2016-04-04 08:49:14	0|gmaxwell|sipa: as it tweakrawtxn "hex" optinrbf
125 2016-04-04 08:49:30	0|sipa|gmaxwell: ah, reimplementing bitcoin-tx as an rpc? ;)
126 2016-04-04 08:49:48	0|gmaxwell|Ha. Indeed.
127 2016-04-04 08:49:51	0|jonasschnelli|hehe..
128 2016-04-04 08:50:32	0|jonasschnelli|what speaks against having these tweak-flags in crt?
129 2016-04-04 08:50:40	0|gmaxwell|right now the api has excessive fanout. seperate top level functions for varioations on the same thing, side effect of positional arguments and legacy.  It means when looking up a command you have to go through long lists.
130 2016-04-04 08:51:04	0|gmaxwell|crt?
131 2016-04-04 08:51:07	0|jonasschnelli|createrawtransaction
132 2016-04-04 08:51:19	0|jonasschnelli|createrawtransaction <inputs> <outputs> <flag|flag>
133 2016-04-04 08:51:22	0|gmaxwell|the existance of fundraw for one.
134 2016-04-04 08:51:36	0|jonasschnelli|fundrawtx interacts with the wallet
135 2016-04-04 08:52:01	0|gmaxwell|the point is that if you set flags in createraw, they won't be visible in fundraw and it might violate them.
136 2016-04-04 08:52:03	0|jonasschnelli|<flag|flag> could be {"bip125rbf":true, etc.}
137 2016-04-04 08:52:18	0|jonasschnelli|gmaxwell: good point.
138 2016-04-04 08:52:22	0|gmaxwell|where as createraw | fundraw | tweak | signraw | sendraw is a fine pipeline.
139 2016-04-04 08:52:55	0|jonasschnelli|hmm.. not bad. Fundraw could then remove the opt-in-rbf flag (because it can add inputs).
140 2016-04-04 08:53:27	0|jonasschnelli|alternative names for "tweak"?
141 2016-04-04 08:53:33	0|sipa|mutate
142 2016-04-04 08:53:35	0|jonasschnelli|alter?
143 2016-04-04 08:53:36	0|sipa|alter
144 2016-04-04 08:53:38	0|sipa|modify
145 2016-04-04 08:53:52	0|jonasschnelli|modify is probably most understandable... but that blitcoin-cli in RPC!
146 2016-04-04 08:53:55	0|sipa|in the bitcoin-tx source code they are called mutate
147 2016-04-04 08:54:02	0|gmaxwell|spindletx
148 2016-04-04 08:54:12	0|sipa|transmogrifyrawtx
149 2016-04-04 08:54:19	0|gmaxwell|Presotchangotx
150 2016-04-04 08:54:25	0|gmaxwell|alterrawtx is probably fine.
151 2016-04-04 08:54:44	0|jonasschnelli|+1 for alterrawttransaction
152 2016-04-04 08:55:05	0|jonasschnelli|We could start with RBF...
153 2016-04-04 08:55:10	0|jonasschnelli|add CSV..
154 2016-04-04 08:55:13	0|gmaxwell|I guess thats more consistent. (the fact that transaction is spelled out is one of the sadder parts of the interface)
155 2016-04-04 08:55:19	0|jonasschnelli|and someone could also implement adding ins and outs.
156 2016-04-04 08:55:39	0|jonasschnelli|But once we said keep the utilities outside of the RPC level
157 2016-04-04 08:55:50	0|gmaxwell|We sometimes made made choices.
158 2016-04-04 08:56:17	0|jonasschnelli|I agree. RPC is very handy for utitilites...
159 2016-04-04 09:06:20	0|jonasschnelli|gmaxwell, sipa: If you interested to review the enc/auth BIP again, a new version is here https://github.com/bitcoin/bips/pull/362/files
160 2016-04-04 09:10:50	0|gmaxwell|64-bit MAC, really?  This leaks message lengths.
161 2016-04-04 09:12:32	0|gmaxwell|probably should remove the performance claims from the document? they'll just invite debate over the document text. AES-128-GCM is quite a bit faster than chacha when AES-NI is used. (I'm not arguing it here, just pointing out the argument)
162 2016-04-04 09:13:42	0|gmaxwell|Is there any need to limit the rekeying that strictly? if it's just a hash operation, someone can just send us transactions to make us hash over and over again.
163 2016-04-04 09:13:50	0|wumpus|when AES-NI is used <- but our odroid C2 doesn't have AES-NI :o
164 2016-04-04 09:14:10	0|jonasschnelli|gmaxwell: why 64bit mac? Because of the SHA512?
165 2016-04-04 09:14:20	0|gmaxwell|hah I know, I was advocating before that we only use chacha here... (the alternative is to support both and negoiate when both ends have aes-ni).
166 2016-04-04 09:15:22	0|wumpus|agree, let's just settle on that. I don't think the performance considerations in the cipher are the problem here so I also agree with not making it primary in the document
167 2016-04-04 09:15:26	0|jonasschnelli|gmaxwell: the rekeying is local policy, but should not be under 600 seconds to avoid dos (https://github.com/bitcoin/bips/pull/362/files#diff-0bd7a3b80179294f4bcb38cae13c8534R142)
168 2016-04-04 09:15:30	0|gmaxwell|jonasschnelli: it's using a 4byte length, which is very wasteful. (on average 2.5 bytes will be wasted) but then only an 8 byte mac which is quite weak.
169 2016-04-04 09:15:52	0|sipa|jonasschnelli: poly1305 does not have 256-bit security, and certainly not when you truncate the tag to 64 bits!
170 2016-04-04 09:16:07	0|gmaxwell|jonasschnelli: if the timeout is 600 seconds, then a sender cannot rekey for some multiple of that for fear that the far end has a different idea of the arrival time.
171 2016-04-04 09:16:46	0|jonasschnelli|sipa: Chacha20 offers 256bit security. I though the poly MAC does not affect the security itself,... only the authentication? Not?
172 2016-04-04 09:17:03	0|sipa|jonasschnelli: authentication is part of the security
173 2016-04-04 09:17:12	0|sipa|it protects against different types of tlattacks
174 2016-04-04 09:17:15	0|sipa|*attacks
175 2016-04-04 09:17:23	0|jonasschnelli|Okay... I see.
176 2016-04-04 09:17:26	0|gmaxwell|jonasschnelli: the cipher and authentication security go hand in hand, if you can compromise the authentication you can usually make the endpoints leak information about the content.
177 2016-04-04 09:18:10	0|jonasschnelli|But 256bit security does not reflect the overall robustness....
178 2016-04-04 09:18:16	0|jonasschnelli|But right. Let me better remove this line.
179 2016-04-04 09:18:43	0|jonasschnelli|gmaxwell: So you think truncate the TAG to 8 byte is still to long?
180 2016-04-04 09:18:49	0|gmaxwell|no it's too short.
181 2016-04-04 09:19:05	0|gmaxwell|I would suggest the length be made variable length, self-delimiting, encrypted like in the ssh spec... and that tag length be increased.
182 2016-04-04 09:20:08	0|jonasschnelli|Okay. So using the non-truncated full 128bit poly1305 tag?
183 2016-04-04 09:20:29	0|gmaxwell|much of the cost of a longer tag could be paid by making the length shorter.  There are some things using 12 byte tags, but I'm not sure what I think about that.
184 2016-04-04 09:20:47	0|jonasschnelli|gmaxwell: length variable length -> do you mean varint serialization?
185 2016-04-04 09:21:07	0|sipa|gmaxwell: i think varint serialization is overkill
186 2016-04-04 09:21:56	0|jonasschnelli|The current message structure also has a int32 for the length. Maybe we keep the varint outside of the message header.
187 2016-04-04 09:22:27	0|gmaxwell|sipa: better than truncating the MAC to 8 bytes.
188 2016-04-04 09:22:51	0|wumpus|varint is slightly inconvenient, it's nice to be able to read network headers at once
189 2016-04-04 09:23:05	0|wumpus|how does ssh handle this?
190 2016-04-04 09:23:23	0|gmaxwell|uses a fixed width interger, it's encrypted however.
191 2016-04-04 09:23:48	0|gmaxwell|The varint suggestion was trying to recover the overhead from the longer mac tag, I'm not wed to the idea.
192 2016-04-04 09:24:02	0|wumpus|I don't think we should couple the MAC size discussion to the packet length field discussion in any case, at most you'd save ~2 bytes for the typical packet
193 2016-04-04 09:24:08	0|gmaxwell|Many of the messages in bitcoin are quite small, so the extra lengths do make a meaningful impact.
194 2016-04-04 09:24:24	0|wumpus|yes, but reading one byte of a time from a TCP stream :-(
195 2016-04-04 09:24:50	0|gmaxwell|average message size in bitcoin p2p is someting like 100 bytes right now.
196 2016-04-04 09:24:52	0|wumpus|let's increase the MAC size and leave the length at four bytes for now
197 2016-04-04 09:24:58	0|gmaxwell|yup
198 2016-04-04 09:25:04	0|jonasschnelli|Okay. Will do.
199 2016-04-04 09:25:23	0|wumpus|gmaxwell: well you had the idea to collate P2P packets into encryption packets ...
200 2016-04-04 09:25:38	0|wumpus|(I know, futur work.)
201 2016-04-04 09:25:41	0|sipa|wumpus: which is also in the bip, btw
202 2016-04-04 09:25:44	0|jonasschnelli|gmaxwell: re-keying minimum of 600 seconds is to large?
203 2016-04-04 09:25:45	0|wumpus|oh!
204 2016-04-04 09:25:53	0|gmaxwell|yes, though there is 4 byte lengths at each level. At least that helps with mac overhead.
205 2016-04-04 09:26:03	0|wumpus|well in the inner packet I'm not opposed to varints
206 2016-04-04 09:26:07	0|wumpus|those packets are in memory already
207 2016-04-04 09:26:08	0|gmaxwell|jonasschnelli: I'd make it 10 seconds or something very small.
208 2016-04-04 09:26:30	0|sipa|rekey every 10 seconds?
209 2016-04-04 09:26:38	0|jonasschnelli|sipa: min. 10 sec
210 2016-04-04 09:26:43	0|jonasschnelli|flexible local policy.
211 2016-04-04 09:26:57	0|jonasschnelli|A node could detect repeating re-keys and ban
212 2016-04-04 09:27:25	0|jonasschnelli|I just want a minimum timespan between re-key in the BIP.
213 2016-04-04 09:27:51	0|gmaxwell|I don't think a minimum timespan is needed, but if one is there it shouldn't be 600 seconds.
214 2016-04-04 09:28:46	0|jonasschnelli|gmaxwell: Yes. We could also leave that open to the implementation. But the most obvious attack vectors maybe should be covered by the BIP?
215 2016-04-04 09:28:51	0|sipa|if we care about overhead, the first thing to consider would be making those 12-byte message names varlen :)
216 2016-04-04 09:29:06	0|jonasschnelli|Indeed.
217 2016-04-04 09:29:12	0|wumpus|yes
218 2016-04-04 09:29:26	0|jonasschnelli|We could do this for the encrypted message stucture.
219 2016-04-04 09:29:35	0|sipa|jonasschnelli: rekeying every second will still be much lower overhead than normal transaction relay...
220 2016-04-04 09:29:56	0|wumpus|+1 for making those varlen, that padding is ugly and people have accidentally leaked memory contents into them in the past :)
221 2016-04-04 09:30:03	0|jonasschnelli|sipa: Hmm... right. Its just a 2xSHA256.
222 2016-04-04 09:30:17	0|sipa|jonasschnelli: no, it's an ecdh
223 2016-04-04 09:30:31	0|sipa|which is similar to a signature validation in time
224 2016-04-04 09:30:33	0|jonasschnelli|sipa: No. The ECDH is done already,... its only hash(old_key)?
225 2016-04-04 09:30:42	0|sipa|jonasschnelli: rekey means doing ecdh again
226 2016-04-04 09:30:48	0|gmaxwell|it shouldn't.
227 2016-04-04 09:30:54	0|gmaxwell|it doesn't in the spec.
228 2016-04-04 09:31:26	0|sipa|ah
229 2016-04-04 09:31:31	0|gmaxwell|sipa: it's logically just cranking the initial KDF csprng to the next state.
230 2016-04-04 09:33:11	0|jonasschnelli|okay. 1.) command varlen 2.) AEAD-tag 128bits, 3.) remove 256bit security mentioning.
231 2016-04-04 09:33:47	0|gmaxwell|on auth, this protocol looks like it would result in a bitcoin daemon announcing a persistant identity to everyone that connects to it?
232 2016-04-04 09:34:14	0|jonasschnelli|According to https://www.ietf.org/proceedings/88/slides/slides-88-tls-1.pdf AES-128-GCM is slower then ChaCha20+Poly1305
233 2016-04-04 09:34:42	0|gmaxwell|Is there a reason not to use the scheme I suggested where the client sends an auth challenge which is H(session-id||server-pubkey)  and if the server reconizes one of his keys, he responds with a signature?
234 2016-04-04 09:34:52	0|sipa|jonasschnelli: aes-gcm is slowish without hardware support, very fast with
235 2016-04-04 09:35:08	0|gmaxwell|jonasschnelli: only if you don not have AES-NI. If you do AES-128-GCM is maybe 7 times faster.
236 2016-04-04 09:35:20	0|sipa|jonasschnelli: aes-gcm can get to 1 cycle/byte on modern hardware with asm code
237 2016-04-04 09:35:51	0|sipa|jonasschnelli: chacha20-poly1305 is hard to get under 4ish, i think
238 2016-04-04 09:35:53	0|jonasschnelli|a... i see. AES-NI: 892 MB/s, ChaCha20+Poly1305, -march=native = 560 MB/s
239 2016-04-04 09:36:41	0|jonasschnelli|gmaxwell: the identity would only be revealed by the requesting peer.
240 2016-04-04 09:36:52	0|jonasschnelli|So the requesting peer would know who he wants to give its identity price.
241 2016-04-04 09:37:22	0|jonasschnelli|The responding peers only reveals its identity if the requesting peers identity "was accepted".
242 2016-04-04 09:37:52	0|sipa|jonasschnelli: gmaxwell's protocol never reveals identity, and only works if both sides know the pubkey beforehand
243 2016-04-04 09:38:16	0|jonasschnelli|Hmm... yes: H(session-id||server-pubkey)  makes sense.
244 2016-04-04 09:38:41	0|sipa|jonasschnelli: those numbers are certainly not the best possible ones
245 2016-04-04 09:38:53	0|jonasschnelli|okay.
246 2016-04-04 09:38:59	0|gmaxwell|then it hast to be mutual, but what if you just have a trusted node and many things that connect to it. You don't want to do per-peer setup on each side (if you do-- you could setup symmetric keys).  The downside of my protocol is that if you have many different identities you'd have to try all of them, but hashing is cheap, and I don't see a reason to have a huge number of alternative identitie
247 2016-04-04 09:39:05	0|gmaxwell|s.
248 2016-04-04 09:39:58	0|jonasschnelli|hmm.. so we assume the requesting peer can link a IP to a pubkey
249 2016-04-04 09:40:01	0|sipa|i'd prefer focussing on encryption first :)
250 2016-04-04 09:40:19	0|gmaxwell|I'd also made a suggestion that auth trigger a rekeying. with the pubkey in the rekeying. Because this has a cute property of being forever forward secure even if ecc is broken, if the public keys are kept private.
251 2016-04-04 09:41:02	0|gmaxwell|jonasschnelli: well he hopefully knows who he thinks he's connecting to-- more like the other way around, he has something he wants to connect to (pubkey), and has an IP he thinks belongs to it.
252 2016-04-04 09:41:24	0|gmaxwell|but yea, maybe hammer out encryption first.
253 2016-04-04 09:41:27	0|jonasschnelli|Could the responding peer answer with a signature of the received hash(received-hash || session-id||server-pubkey)?
254 2016-04-04 09:41:55	0|jonasschnelli|I think auth and enc has some overlaps.
255 2016-04-04 09:42:07	0|jonasschnelli|(at least in thinking about a solution)
256 2016-04-04 09:43:11	0|gmaxwell|All he should need to do is sign the session-id.
257 2016-04-04 09:49:07	0|Anduck|who answers from contact@bitcoincore.org?
258 2016-04-04 09:49:22	0|Anduck|i see lots of "contact us" etc. but no names
259 2016-04-04 09:49:27	0|Anduck|like who actually run it
260 2016-04-04 10:01:27	0|GitHub148|[13bitcoin] 15gmaxwell opened pull request #7805: Eliminate TX trickle bypass, sort TX invs for privacy and priority. (06master...06sorted_invs) 02https://github.com/bitcoin/bitcoin/pull/7805
261 2016-04-04 11:05:18	0|jonasschnelli|sipa: I agree that encryption should be made first. But I just want to avoid that people start using it, and think: "okay, now everything is encrypted, let me use minimum BF FPR", but MITM is still trivial. Auth would allow to reduce the MITM scenario.
262 2016-04-04 11:08:59	0|GitHub1|[13bitcoin] 15laanwj closed pull request #7543: [0.12] Backport BIP9, BIP68 and BIP112 with softfork (060.12...06dot12_backport_bip68) 02https://github.com/bitcoin/bitcoin/pull/7543
263 2016-04-04 11:08:59	0|GitHub144|[13bitcoin] 15laanwj pushed 24 new commits to 060.12: 02https://github.com/bitcoin/bitcoin/compare/c5f94f6584cb...834aaef7bd37
264 2016-04-04 11:09:00	0|GitHub144|13bitcoin/060.12 140d09af7 15Suhas Daftuar: Add RPC test exercising BIP68 (mempool only)
265 2016-04-04 11:09:00	0|GitHub144|13bitcoin/060.12 1415ba08c 15Alex Morcos: Implement SequenceLocks functions...
266 2016-04-04 11:09:01	0|GitHub144|13bitcoin/060.12 140a79c04 15Alex Morcos: Bug fix to RPC test
267 2016-04-04 11:10:29	0|sipa|jonasschnelli: bf for?
268 2016-04-04 11:10:36	0|sipa|bf fpr?
269 2016-04-04 11:10:45	0|jonasschnelli|bloomfilter false positive rate
270 2016-04-04 11:10:57	0|jonasschnelli|(in case you want to connect a SPV wallet to a node over enc. channels)
271 2016-04-04 11:11:07	0|sipa|jonasschnelli: i don't mean that we should encryption running first in production
272 2016-04-04 11:11:23	0|sipa|but just have it designed and perhaps implemented first
273 2016-04-04 11:11:31	0|sipa|as it interacts heavily with network code
274 2016-04-04 11:11:32	0|jonasschnelli|gmaxwell: H(session-id||server-pubkey) would have once downside: extending to an concept where you can connect to a peer where you don't have pre-shared identity-keys
275 2016-04-04 11:11:59	0|jonasschnelli|sipa: okay. Yes. That make sense.
276 2016-04-04 11:12:10	0|sipa|jonasschnelli: if you don't have pre-shared keys, what are you trying to accomplish?
277 2016-04-04 11:12:46	0|jonasschnelli|sipa: I agree. It's a different security level. But it would allow to make sure that further connections are going to the same node.
278 2016-04-04 11:13:04	0|sipa|jonasschnelli: that's by definition fingerprinting the node
279 2016-04-04 11:13:37	0|sipa|i'm unconvinced that's something we should aim for
280 2016-04-04 11:14:10	0|sipa|at least, it's a very different thing from what we've been doing recently, namely trying to avoid fingerprinting
281 2016-04-04 11:14:50	0|sipa|maybe there is some mechanism possible where the key is based on the ip, and you never leak identity information for other incoming network addresses
282 2016-04-04 11:15:10	0|jonasschnelli|sipa: Yes. Probably. Well, .. i though you could connect to a node, reveal you identity, get the pubkey if the remote node wants to show its identity, .. further MITM would be detectable. But agree. If MITM would be there in first place you have lost anyways.
283 2016-04-04 11:15:34	0|sipa|jonasschnelli: yes, that's called tofu (trust on first use)
284 2016-04-04 11:16:10	0|jonasschnelli|Ah.. that's the tofu. Thanks.
285 2016-04-04 11:16:54	0|sipa|but that may not something you want in general... for example if a node is available over tor and over a normal iov4, you don't want it to have the same identity on both
286 2016-04-04 11:18:08	0|jonasschnelli|sipa: hmm.. this would mean one identity per net-type?
287 2016-04-04 11:18:14	0|sipa|maybe something you only want for servers on static ips, and only on the listening side
288 2016-04-04 11:19:28	0|sipa|so i think it's an orthogonal use case
289 2016-04-04 11:20:46	0|sipa|not just per net type
290 2016-04-04 11:21:24	0|sipa|if you receive a connection on localhost, what key do you use? that may depend on whether it's a trusted local connection, or it's coming via a proxied tor
291 2016-04-04 11:35:08	0|jonasschnelli|If the authack signature would also contain the encryption-session-id (for the opposite com. direction) and the identity-pubkey from the responding peer, that would probably avoid an auth/authack initiated from the responding peer.
292 2016-04-04 11:35:40	0|jonasschnelli|auth: H(enc-session-id-A || indenity-pubkey)-A
293 2016-04-04 11:36:21	0|jonasschnelli|auth: H(enc-session-id-A || indenity-pubkey-A), authack: sig(H(enc-session-id-B || indenity-pubkey-B))
294 2016-04-04 11:36:34	0|jonasschnelli|no wait...
295 2016-04-04 11:36:46	0|jonasschnelli|auth: H(enc-session-id-A || indenity-pubkey-A), authack: sig(H(auth-request-hash || enc-session-id-B || indenity-pubkey-B))
296 2016-04-04 11:37:08	0|sipa|where does the pubkey of the other side come from?
297 2016-04-04 11:37:56	0|jonasschnelli|sipa: the identity pubkey from the other side.
298 2016-04-04 11:38:19	0|jonasschnelli|(should be node by the requesting peer because we assume pre-shared keys)
299 2016-04-04 11:38:26	0|jonasschnelli|*pre-shared pubkey*
300 2016-04-04 11:40:46	0|sipa|let's go over the use cases one by one
301 2016-04-04 11:40:48	0|sipa|i know 3
302 2016-04-04 11:41:18	0|sipa|1) light client connecting to a trusted full node; in this case the light client needs to have the pubkey of the trusted node
303 2016-04-04 11:42:03	0|sipa|2) full node only provides (part of) its services to known peers, for example bloom filtering, or block download, or whitelisting, ...; in this case the full node needs to have the pubkey of the client
304 2016-04-04 11:42:30	0|sipa|3) tofu security between any nodes on the network
305 2016-04-04 11:42:37	0|sipa|i think 1 and 2 are fundamentally different from 3
306 2016-04-04 11:42:57	0|jonasschnelli|I think we should focus for 1/2.
307 2016-04-04 11:42:58	0|sipa|because 1 and 2 never need to reveal identities, only provide proof when requested
308 2016-04-04 11:43:17	0|jonasschnelli|Do you think 1 without 2 is something we should aim for?
309 2016-04-04 11:43:33	0|sipa|i think 1 and 2 are orthogonal
310 2016-04-04 11:43:41	0|jonasschnelli|Yes. I agree.
311 2016-04-04 11:43:43	0|sipa|and they can use the exact same code, just in the other direction
312 2016-04-04 11:44:05	0|jonasschnelli|Then we probably need: auth: H(enc-session-id-A || indenity-pubkey-A), authack: sig(requesting-hash)
313 2016-04-04 11:44:09	0|jonasschnelli|For both sides.
314 2016-04-04 11:45:06	0|sipa|i don't think that adds anything over signing just the session id
315 2016-04-04 11:45:39	0|sipa|Schnorr signatures internally compute a message hash, which is based on the message and the signing pubkey
316 2016-04-04 11:45:48	0|sipa|so they already do that internally
317 2016-04-04 11:46:27	0|jonasschnelli|if we assume ECDSA, would you then recover the pubkey from the sig to lookup the identity?
318 2016-04-04 11:47:13	0|sipa|? you asked for it, you know it already
319 2016-04-04 11:47:31	0|sipa|"hey, are you X?" - "yes, here is proof"
320 2016-04-04 11:48:16	0|sipa|we could use bitcoin script for the signatures, so you can do multisig auth :p
321 2016-04-04 11:48:24	0|jonasschnelli|heh.
322 2016-04-04 11:48:39	0|sipa|"hey, are you X, Y, or Z?" - "yes, here is a scriptSig"
323 2016-04-04 11:49:04	0|sipa|no good use case for that though, so let's not exaggerate
324 2016-04-04 11:50:00	0|jonasschnelli|For your usecase 2) when or how would the responding peer ask for "hey, are you X?" - "yes, here is proof"?
325 2016-04-04 11:50:20	0|jonasschnelli|A new message-type from the requesting peer?
326 2016-04-04 11:50:36	0|jonasschnelli|Or as soon as the requesting peer accesses a restrictied service?
327 2016-04-04 11:57:16	0|sipa|let's say there are two new messages "areyou"(H(peer-pubkey || receive-session-id)) and "yesiam"(H(my-pubkey || send-session-id), sig(key=my-privkey, msg=send-session-id))
328 2016-04-04 11:58:57	0|sipa|if you're making/accepting a connection to/from someone and want them to authenticate, you send the areyou as first message inside the encrypted channel, before version
329 2016-04-04 11:59:28	0|sipa|there should probably be an explicit "i don't need you to authenticate" ?
330 2016-04-04 12:00:04	0|sipa|actually, no
331 2016-04-04 12:00:31	0|sipa|you either send an 'areyou' (in which case you wait for a yesiam first), or you send a version yourself
332 2016-04-04 12:05:25	0|jonasschnelli|sipa: Hmm.. a requesting peer that seeks access to "filtering" would first send a "areyou"-message to hope the responding peer will also request auth over a "areyou" message?
333 2016-04-04 12:05:58	0|jonasschnelli|Or could the requesting peer not just send a "yesiam"(H(my-pubkey || send-session-id)
334 2016-04-04 12:06:26	0|sipa|the protocol doesn't continue if they don't respond
335 2016-04-04 12:08:36	0|jonasschnelli|sipa: for your usecase 2) the requesting peer first needs to authenticate the responding peer... i think thats fine.
336 2016-04-04 12:08:59	0|jonasschnelli|But I don't see a way to do 2) without 1)
337 2016-04-04 12:11:15	0|jonasschnelli|but however, let me think a bit about it. My brain usually needs some minutes for processing the input. :)
338 2016-04-04 12:12:05	0|sipa|jonasschnelli: 1 and 2 are exactly the same!
339 2016-04-04 12:12:32	0|sipa|except initiated by the one listening instead of the one connecting
340 2016-04-04 12:13:11	0|sipa|1 is "are you the server I know? if not, i won't tell you anything about myself!"
341 2016-04-04 12:13:25	0|sipa|2 is "are you the client I know? if not, i won't tell you anything about myself!"
342 2016-04-04 12:13:45	0|sipa|but the p2p protocol has no distinction between the one connecting and the one listening
343 2016-04-04 12:13:57	0|jonasschnelli|sipa: yes. But for 2) you might not want to ask every peer for authentication.
344 2016-04-04 12:14:21	0|sipa|ah, i see
345 2016-04-04 12:14:50	0|jonasschnelli|I think in practice, 2 makes only sense with 1
346 2016-04-04 12:15:40	0|jonasschnelli|Though, it could be policy (ask every peer, ask only peer where you have authacked).
347 2016-04-04 12:16:24	0|sipa|let's get encryption flushed out first :)
348 2016-04-04 12:16:42	0|sipa|that's a part that can be shared across all use cases
349 2016-04-04 12:17:00	0|jonasschnelli|okay... lets do that.
350 2016-04-04 12:17:23	0|jonasschnelli|I'll update the auth bip with all the stuff we have disusses but focus on the enc BIP
351 2016-04-04 12:41:22	0|GitHub28|[13bitcoin] 15laanwj pushed 2 new commits to 060.12: 02https://github.com/bitcoin/bitcoin/compare/834aaef7bd37...e3341aa94e1f
352 2016-04-04 12:41:23	0|GitHub158|[13bitcoin] 15laanwj closed pull request #7800: [0.12] Update release notes (060.12...06docs) 02https://github.com/bitcoin/bitcoin/pull/7800
353 2016-04-04 12:41:23	0|GitHub28|13bitcoin/060.12 14e10c044 15BtcDrak: [0.12] Update release notes
354 2016-04-04 12:41:23	0|GitHub28|13bitcoin/060.12 14e3341aa 15Wladimir J. van der Laan: Merge #7800: [0.12] Update release notes...
355 2016-04-04 13:23:49	0|wumpus|oh wow re: tracing and profiling https://github.com/brendangregg/FlameGraph
356 2016-04-04 15:20:46	0|GitHub163|[13bitcoin] 15instagibbs closed pull request #7784: miner_tests number clarification (06master...06patch-4) 02https://github.com/bitcoin/bitcoin/pull/7784
357 2016-04-04 15:20:51	0|GitHub6|[13bitcoin] 15instagibbs opened pull request #7807: Fixed miner test values, gave constants for less error-prone values. (06master...06mintest) 02https://github.com/bitcoin/bitcoin/pull/7807
358 2016-04-04 17:56:00	0|Luke-Jr|FWIW, it sounds like bc.i is making bogus problems for 0.12 txns
359 2016-04-04 17:56:11	0|Luke-Jr|the fee sniping chang
360 2016-04-04 17:56:13	0|Luke-Jr|change*
361 2016-04-04 17:56:57	0|gmaxwell|making bogus problems?
362 2016-04-04 17:58:42	0|btcdrak|?
363 2016-04-04 17:58:51	0|gmaxwell|sipa:  the client could send [h(session_id||serverkey), h(session_id||clientkey)]  and the server could respond with a signature, and then "ah, you claim to be h(sessionid||clientkey||2), prove it with a signature."
364 2016-04-04 17:59:33	0|gmaxwell|sipa: and if the client doesn't wish to identify itself/not configured for mutual auth it just sends a random number in the clientkey field. Server can't meet that challenge, so server doesn't learn anything about client identity.
365 2016-04-04 17:59:54	0|gmaxwell|sipa: so this lets you do mutual auth without leaking client identities to people who don't already know them.
366 2016-04-04 23:48:35	0|GitHub124|[13bitcoin] 15theuni opened pull request #7809: depends: some base fixes/changes (06master...06depends-updates) 02https://github.com/bitcoin/bitcoin/pull/7809