1 2016-12-05 00:00:05	0|BlueMatt|oh, what? you told sipa that the "it updates the storage"
  2 2016-12-05 00:00:05	0|jtimon|but let me check that commit
  3 2016-12-05 00:02:12	0|jtimon|I mean it updates what you store, through the api (ie the library handles reorgs and updates things)
  4 2016-12-05 00:02:47	0|BlueMatt|well of course, you tell the library about a block and it connects what it can, possibly asking the library owner for blocks that it previously saw but (ofc) didnt store
  5 2016-12-05 00:03:01	0|BlueMatt|I dont see how else you'd possibly do it
  6 2016-12-05 00:03:33	0|BlueMatt|gmaxwell: see, eg, the bitcoinj stuff, where all the contributions to "full mode" since it was added have been to add it to database X, so that people can put it in their own db
  7 2016-12-05 00:03:35	0|jtimon|see the backlog for when I say for people answer the 4 possible combinations to 2 yes/no questions
  8 2016-12-05 00:05:38	0|jtimon|you and me agree on abstracting storage
  9 2016-12-05 00:05:57	0|jtimon|I mean, if I understood everyone's position correctly
 10 2016-12-05 00:06:39	0|gmaxwell|BlueMatt: without using a highly efficient format like ours, I'm dubious that the system can stay in sync without insane hardware.
 11 2016-12-05 00:06:46	0|jtimon|the other question is only check that a block is valid or also handle reorgs, update tables etc
 12 2016-12-05 00:06:52	0|BlueMatt|gmaxwell: thats not our problem, thats theirs
 13 2016-12-05 00:06:58	0|BlueMatt|gmaxwell: hell, they can use leveldb if they want
 14 2016-12-05 00:07:02	0|jtimon|ie verifyBlock vs processBlock
 15 2016-12-05 00:07:20	0|BlueMatt|gmaxwell: and, if we prefer, its also easy to add eg a single flag like "use X MB of utxo cache, because I dont want to implement that myself"
 16 2016-12-05 00:07:24	0|gmaxwell|BlueMatt: not just about leveldb or not, but the compressed ccoins representation is worthless for queries.
 17 2016-12-05 00:08:27	0|BlueMatt|gmaxwell: the compressed ccoins thing doesnt matter all that much if you're talking about an actually-high-performance db on high-performance hardware....if folks need their shit in their own db and are willing to pay $$$$$ for it to run, more power to them
 18 2016-12-05 00:08:34	0|gmaxwell|As far as 'their problem' goes, we shouldn't waste our resources (or code base clarity, or performance) supporting functionality that won't be useful to anyone (or to anyone beyond a couple centeralized API services).
 19 2016-12-05 00:08:45	0|jtimon|we can add a wrapper with our own implementation of the interfaces, beyond that, right, it is their problem
 20 2016-12-05 00:08:56	0|BlueMatt|gmaxwell: ok, then what is the point of libconsensus at all?
 21 2016-12-05 00:09:20	0|BlueMatt|and whats your answer to folks like btcd/the new javascript one, who do have dbs that are performant enough to stay in sync
 22 2016-12-05 00:09:20	0|sipa|BlueMatt: i don't care about libconsensus. i care about abstracting consensus logic out
 23 2016-12-05 00:09:21	0|gmaxwell|The way it was pitched to me is so that people could make wallets and other similar applications without having to reimplement consensus logic.
 24 2016-12-05 00:09:35	0|jtimon|right
 25 2016-12-05 00:09:53	0|BlueMatt|gmaxwell: sure, but that doesnt mean we have to handle db logic ourselves
 26 2016-12-05 00:09:59	0|sipa|well, the hardest part of that is already available: we expose script verification
 27 2016-12-05 00:10:03	0|jtimon|how is that incompatible with allowing them to chose their database implementation?
 28 2016-12-05 00:10:14	0|BlueMatt|sipa: I dont think thats the only hard part, really
 29 2016-12-05 00:10:16	0|sipa|but i also don't think it's very hard to abstract utxo storage out... so if there is a use case, sure
 30 2016-12-05 00:10:45	0|BlueMatt|sipa: indeed, abstracting out utxo/block storage is also abstracting consensus logic out of other crap
 31 2016-12-05 00:10:45	0|sipa|maybe it is.
 32 2016-12-05 00:11:12	0|sipa|something like changing to per-output rather than per-tx utxo model would be impossible with a stable utxo storage abstraction
 33 2016-12-05 00:11:38	0|jtimon|I assume the use cases come from the fact that they want to reuse that database for some of their logic somehow
 34 2016-12-05 00:11:45	0|sipa|s/impossible/inefficient or complicated/
 35 2016-12-05 00:11:47	0|gmaxwell|I'm not opposed to it being abstractable-- but I don't see how this is related to that goal-- it's the opposite of it, the blockchain storage and utxo set is consensus and may even be quite normative in its behavior (e.g. if we have a commited utxo set of some kind), and if it trashes performance or code clarity then it's not a good move.
 36 2016-12-05 00:12:53	0|BlueMatt|sipa: oh? if you query per-utxo then per-tx can be hidden on the backend and could still be pretty performant....indeed, the other way around doesnt really work
 37 2016-12-05 00:12:54	0|gmaxwell|and for an example of the kind of complexity it creates: if you think you can just query a utxo database it means we can't have writeback caching internally.
 38 2016-12-05 00:13:03	0|BlueMatt|gmaxwell: sure, no one wants to do anything that ends up introducing performance regressions
 39 2016-12-05 00:13:42	0|sipa|abstracting storage is more to avoid dependencies rather than it being reusable
 40 2016-12-05 00:13:56	0|gmaxwell|I think callers that want a database probably don't want to replace the database used for consensus-- what they want is a node that maintains an external database for their application.
 41 2016-12-05 00:14:10	0|gmaxwell|Which is probably not the same thing due to consistency requirements.
 42 2016-12-05 00:15:19	0|jtimon|gmaxwell: it shouldn't trash performance or code clarity, I agree
 43 2016-12-05 00:15:32	0|BlueMatt|gmaxwell: it might be worse performance, but syncing after every block after ibd (ie having a flag to sync) isnt all that hard, either....
 44 2016-12-05 00:15:43	0|BlueMatt|eg ProcessNewBlock(block); SyncToDB();
 45 2016-12-05 00:16:11	0|sipa|BlueMatt: but for a full node, you probably don't want to sync after every block
 46 2016-12-05 00:16:12	0|BlueMatt|gmaxwell: and its easy to do that without introducing our own performance regressions....just dont call Sync after every block....
 47 2016-12-05 00:16:22	0|BlueMatt|sipa: depending on your application, maybe you do
 48 2016-12-05 00:16:30	0|jtimon|sipa: once abstracted out, changing the interface in certain ways may be painful, correct
 49 2016-12-05 00:16:33	0|BlueMatt|at least after IBD
 50 2016-12-05 00:16:38	0|gmaxwell|Not writing out the utxo set constantly is critical for performance. Leveldb is slow.
 51 2016-12-05 00:16:47	0|sipa|BlueMatt: but it'd be using our block validation code, which has known performance characteristics
 52 2016-12-05 00:17:05	0|sipa|BlueMatt: and if you don't care about that, you wouldn't be using libconsensus at all
 53 2016-12-05 00:17:15	0|BlueMatt|sipa: as long as we dont drop the cache when we flush (like we do now, which we already need to fix), I dont see a performance issue there?
 54 2016-12-05 00:17:31	0|sipa|BlueMatt: fair enough
 55 2016-12-05 00:17:43	0|sipa|BlueMatt: that's a good point - but the current code doesn't do that
 56 2016-12-05 00:17:53	0|gmaxwell|I can't imagine an application which needs to muck around storing the utxo database in an application format which wouldn't be equally or better served by block processing callback that maintains an external database that isn't used for validation.
 57 2016-12-05 00:17:57	0|BlueMatt|suresure, but these are all minor issues that are trivial to fix
 58 2016-12-05 00:17:58	0|jtimon|avoiding dependencies is also a great gain, although I think we should have a version depending on levelDB and our own implementation too
 59 2016-12-05 00:18:00	0|sipa|again, i'm not against abstracting things out
 60 2016-12-05 00:18:17	0|gmaxwell|Widespread application visiblity into the actual utxo database would be pretty toxic for commited utxo or stxo improvements.
 61 2016-12-05 00:18:22	0|BlueMatt|gmaxwell: duplicate databases? people who run shit on modern services where you literally have no local persistent storage?
 62 2016-12-05 00:18:25	0|sipa|i very much feel that utxo storage is one of the things that is abstractable
 63 2016-12-05 00:18:45	0|sipa|but that doesn't mean it's necessarily useful for sharing that information with other purposes
 64 2016-12-05 00:18:50	0|sipa|it also doesn't mean it's not
 65 2016-12-05 00:19:16	0|jtimon|yeah, I assume one use case could be having everything in memory
 66 2016-12-05 00:19:26	0|gmaxwell|BlueMatt: basically any standard database approach would have horrible performance to the point that it would only be useable on very high end hardware.  Having two copies of the utxo set would hardly be a consideration there, our copy is only about 2GB data.
 67 2016-12-05 00:20:00	0|BlueMatt|gmaxwell: I seriously dont believe that...maybe it takes 10 seconds or 30 seconds to sync a block to the db, but so what? you just dont flush all the time during IBD and then wait it out
 68 2016-12-05 00:20:19	0|BlueMatt|(and, yes, I get that in most dbs it actually will take 30 seconds, but it wont take much longer than that)
 69 2016-12-05 00:20:26	0|BlueMatt|or a minute with segwit blocks
 70 2016-12-05 00:20:47	0|gmaxwell|BlueMatt: see also the electrum channel with people complaing about their servers falling multiple blocks behind.
 71 2016-12-05 00:20:57	0|gmaxwell|and two minute updates.
 72 2016-12-05 00:22:06	0|BlueMatt|gmaxwell: ok...? that doesnt mean its impossible to build a db that can store the utxo set with reasonable performance, even if not blazing fast performance?
 73 2016-12-05 00:22:23	0|gmaxwell|I just feel that the purpose of these changes is no longer clear. Expecting the user to implement complex interfaces with bitcoin specific and consensus critical behavior is at odds with what I understood to be the stated goal.
 74 2016-12-05 00:22:39	0|jtimon|it is also possible to build something faster than our solution, isn't it?
 75 2016-12-05 00:23:03	0|BlueMatt|gmaxwell: I highly disagree that selecting a sane DB (ie not using some external SQL thing) is "implementing complex interfaces"
 76 2016-12-05 00:23:08	0|gmaxwell|And if maintaining a database for a block explorer is a goal-- then it can be done in a better way then also trying to use that database for consensus... just run it in parallel, the resource overhead will be moderate.
 77 2016-12-05 00:23:28	0|BlueMatt|and I dont expect every exchange to do so, I'd expect folks like btcd/bitcoinj/javascript thinggy to do it
 78 2016-12-05 00:23:30	0|BlueMatt|and people to use it
 79 2016-12-05 00:23:45	0|BlueMatt|gmaxwell: go look at the bitcoinj users...people actually use its full validation shitshow so that they can do exactly this
 80 2016-12-05 00:23:47	0|jtimon|gmaxwell: we don't need to expect the user to reimplement it, we should provide our own implementation to the interfaces
 81 2016-12-05 00:23:48	0|BlueMatt|and its slow, but they dont care
 82 2016-12-05 00:23:55	0|BlueMatt|(and the interface is trivial)
 83 2016-12-05 00:24:11	0|gmaxwell|BlueMatt: Complex interfaces is that you need to actually pass the data as fields and not opaque blobs.   And what happens when the set of data needed for consensus changes (as sequence locktiming did).
 84 2016-12-05 00:24:16	0|BlueMatt|its like 4 functions and only a single non-trivial requirement for which there are unit tests (the utxo-replacement-thing)
 85 2016-12-05 00:24:40	0|jtimon|again, you keep discussing the slow case, what if I'm faster than our implementation?
 86 2016-12-05 00:24:54	0|gmaxwell|BlueMatt: that usage doesn't even need full node support at all, run your bitcoinj behind a full node, have it log blocks to a database. Hurray.
 87 2016-12-05 00:25:10	0|BlueMatt|gmaxwell: yes, except people dont do that
 88 2016-12-05 00:25:20	0|sipa|maybe we should write a simple daemon that maintains the utxo set in SQL
 89 2016-12-05 00:25:22	0|BlueMatt|people prefer to run full validation shit in btcd or bitcoinj, despite knowingly putting themselves at risk
 90 2016-12-05 00:25:27	0|sipa|(without validation)
 91 2016-12-05 00:25:28	0|jtimon|or are we discarding that as a possibility?
 92 2016-12-05 00:25:31	0|gmaxwell|sipa: that is what I was saying.
 93 2016-12-05 00:25:39	0|gmaxwell|BlueMatt: they don't know they're putting themselves at risk.
 94 2016-12-05 00:26:13	0|BlueMatt|gmaxwell: ok, well either way I dont see an alternative great solution here? most developers do want a library that handles all the background validation shit for them
 95 2016-12-05 00:26:19	0|BlueMatt|and as long as such things exist, they will use them
 96 2016-12-05 00:26:20	0|gmaxwell|jtimon: because there exists nothing faster currently, or else we'd be using it.
 97 2016-12-05 00:26:25	0|BlueMatt|proxy-nodes be damned
 98 2016-12-05 00:26:43	0|gmaxwell|BlueMatt: the alternative is to have support for maintaining a database synced with the blockchain-- that doesn't mean inserting things into the consensus logic.
 99 2016-12-05 00:27:16	0|gmaxwell|It just means having a simple set of hooks that run post tip change and update the database.
100 2016-12-05 00:27:40	0|BlueMatt|gmaxwell: you missed my previous point about how lots of "modern" developers run shit on services with ~no persistent storage
101 2016-12-05 00:27:48	0|BlueMatt|anyway, I actually need to run, have an apt to keep
102 2016-12-05 00:27:58	0|sipa|run, BlueMatt, run
103 2016-12-05 00:28:10	0|jtimon|there's no storage solution optimized for having everything in memory that outperforms leveldb or offers some other advantage?
104 2016-12-05 00:28:11	0|sipa|apt-keep BlueMatt
105 2016-12-05 00:28:25	0|sipa|jtimon: -dbcache=8000
106 2016-12-05 00:28:31	0|gmaxwell|We already have that built in.
107 2016-12-05 00:28:32	0|sipa|jtimon: leveldb won't be used at all
108 2016-12-05 00:29:11	0|jtimon|sipa: right, I know, but I doubt there's nothing better than levle db with unbounded cache
109 2016-12-05 00:29:23	0|jtimon|sipa: levledb is what we're using, no?
110 2016-12-05 00:29:39	0|gmaxwell|jtimon: that doesn't use leveldb (except to persist across restarts)
111 2016-12-05 00:29:54	0|jtimon|oh
112 2016-12-05 00:30:09	0|jtimon|I see, I didn't know that, thanks
113 2016-12-05 00:32:09	0|jtimon|even if we only expose the version with our implementation, I think it would be good to abstract consensus storage even for bitcoin core rewardless of libconsensus users
114 2016-12-05 00:32:48	0|gmaxwell|our storage is already abstracted.
115 2016-12-05 00:34:16	0|jtimon|I really think not trashing preformance for our own implementation should be the priority at first, if the interface needs to go through a few iterations not to trash other implementations too, I think that's fine since we will be offering the "don't use storage abstractions" version too anyway
116 2016-12-05 00:35:35	0|jtimon|yes, our storage is abstracted in more ways than we would want to expose in an storage-independent libconsensus C API
117 2016-12-05 00:37:15	0|jtimon|I mean, since I'm in favor of exposing both, one storage independent and one that is not, I'm fine with starting with the one that is not, I'm just more interested technically in the other one
118 2016-12-05 00:39:27	0|jtimon|can we talk a little bit about the other question?
119 2016-12-05 00:42:55	0|jtimon|ie whether libconsensus should fully validate a given block, or also accept it and update the state, manage reorgs, etc
120 2016-12-05 00:44:12	0|jtimon|again I'm ok with offering both but I'm more interested in the smaller one
121 2016-12-05 00:47:42	0|jtimon|or at least that's what I have been imagining all this time, I wasn't counting reorgs or updating the tip as part of the validation
122 2016-12-05 01:05:31	0|morcos|gmaxwell: sdaftuar: to return to the fee bumping, in suhas' example, where you first try to bump tx1,2,3 with tx4, and then you try again.
123 2016-12-05 01:05:54	0|morcos|i can see how you could invent a way to prevent tx4 from getting bumped, but how are you stopping 1,2,3 from being bumped again?
124 2016-12-05 01:06:31	0|morcos|or more generally, lets say you manually had tried to bump and tx1 and tx1a both are trying to pay the same guy (maybe you were smart enought to conflict, maybe not)
125 2016-12-05 01:06:34	0|gmaxwell|They should be marked abandoned when the bump is created.
126 2016-12-05 01:06:40	0|morcos|how do you stop a bumpunconfirmed from bumping both
127 2016-12-05 01:06:48	0|gmaxwell|(but even if they weren't they're conflicted at that point)
128 2016-12-05 01:06:55	0|morcos|they aren't conflicted
129 2016-12-05 01:07:03	0|gmaxwell|by tx4, which is in your mempool.
130 2016-12-05 01:07:07	0|morcos|conflicted means the conflicting tx is in the block
131 2016-12-05 01:07:24	0|morcos|a conflicting tx in the mempool is something you are not even aware of
132 2016-12-05 01:08:03	0|morcos|which brings me to the second point i wanted to make, aobut what you said about people abandoning wallets they think are empty
133 2016-12-05 01:08:14	0|morcos|thats a fantastic point, that i wish had been made a while ago
134 2016-12-05 01:08:31	0|gmaxwell|you're right, I'd been assuming it would be conflicted without thinking carefully what that test actually means right now.
135 2016-12-05 01:08:54	0|gmaxwell|I know for sure people do abandon wallets that they think are empty (or even 'almost empty')
136 2016-12-05 01:09:09	0|morcos|before we made the change to the confliction logic (for 0.12 ?)  then if your spend was not in your mempool, it was considered conflicted (regardless of whether it was by an in-block, in-mempool tx, or nothing)
137 2016-12-05 01:09:24	0|morcos|so it would be kind of rare for you to think you were out of money but not
138 2016-12-05 01:09:31	0|morcos|but now, for sure that might happen
139 2016-12-05 01:10:01	0|morcos|now if you issue a tx that never makes it into a block or for some reason can't ever make it into your own mempool
140 2016-12-05 01:10:13	0|gmaxwell|I seem to vaguely recall that something else would still prevent us from doublespending the inputs on txn that weren't in the mempool, even then.
141 2016-12-05 01:10:16	0|morcos|regardless of any conflicting txs, its uses up your balance until you abandon it
142 2016-12-05 01:10:26	0|morcos|i don't think so
143 2016-12-05 01:11:05	0|morcos|it seems like we need another notion of balance (or maybe 2 more)
144 2016-12-05 01:11:53	0|morcos|potential balance (which is not reduced by non-in chain (6 deep?) spends) and maybe even a pending receive balance (although i guess that hasn't been a problem historically, that hasnt' changed)
145 2016-12-05 01:15:16	0|gmaxwell|Yes, agreed. I worry about the use 'balance' for a number which will go down without user interaction. :)
146 2016-12-05 01:16:13	0|gmaxwell|More like "pending outgoing payments: outbound payments which are not N confirms deep yet" and "pending incoming payments".
147 2016-12-05 01:16:27	0|Taek|Would it be enough to have a [confirmed balance] and an [unconfirmed diff]?
148 2016-12-05 01:16:45	0|morcos|if this is really the only use case, it would be easy enough to make a rpc call that just give you a report on how "empty" your wallet is
149 2016-12-05 01:17:15	0|morcos|Taek: we already have that..  , oh yeah, nm, thats the second thing i was talking abou tthen, the pending received balance
150 2016-12-05 01:17:23	0|morcos|getunconfirmedbalance
151 2016-12-05 01:17:50	0|Taek|if I'm following correctly, the worry is about coins that become unconfirmed due to e.g. change outputs?
152 2016-12-05 01:18:17	0|morcos|i think the primary worry is that if you spend some coins , but your spend never makes it into a block
153 2016-12-05 01:18:27	0|morcos|your wallet still deducts that spend from your balance
154 2016-12-05 01:18:30	0|morcos|forever
155 2016-12-05 01:19:05	0|morcos|until you manually mark it as abandoned (which is sort of an advanced feature, that we don't often recommend)
156 2016-12-05 01:19:38	0|Taek|technically some adversary could un-abandon any transaction that hasn't been double-spent
157 2016-12-05 01:20:05	0|morcos|exactly why its an advanced manual feature
158 2016-12-05 01:20:06	0|Taek|imo your confirmed balance should not change until the tx is in the blockchain
159 2016-12-05 01:20:55	0|Taek|and the confirmed balance should be what is presented to the user as the primary balance
160 2016-12-05 01:21:04	0|morcos|perhaps confirmed balance is the wrong word for what getbalance returns...  it returns your spendable balance..  which certainly should be decremented for spends that haven't yet made it into the blockchain
161 2016-12-05 01:21:17	0|morcos|and i think thats what people expect to see when they ask their balance
162 2016-12-05 01:22:53	0|Taek|I don't think it's safe enough to show the user just one number.
163 2016-12-05 01:23:14	0|Taek|simply because the whole unconfirmed uncertainty is unseparatable from the blockchain way of doing thigns
164 2016-12-05 01:23:30	0|Taek|(well, lightning doesn't really have this issue)
165 2016-12-05 01:25:41	0|BlueMatt|gmaxwell: thinking about it more, the way we'd probably do it is, initially (ie v1) you make the libconsensus consumer provide a k-v store api, and we use that the same way we use leveldb, and then add functionality to parse the blobs we provide the k-v store into things like scriptPubKeys later
166 2016-12-05 01:25:42	0|jtimon|perhaphs both confirmed and spendable balances should be shown?
167 2016-12-05 01:25:53	0|BlueMatt|this provides functionality, without breaking our ability to change the format to add new things
168 2016-12-05 01:26:56	0|jtimon|I understand k-v is key-value kind of storage
169 2016-12-05 01:27:30	0|jtimon|BlueMatt: if so, what would the values be? C structs ?
170 2016-12-05 01:27:48	0|BlueMatt|the values are binary blobs that libconsensus provides which the user does not have any visibility into
171 2016-12-05 01:28:02	0|BlueMatt|(in our case its the serialization of CCoins or whatever with our compression stuff)
172 2016-12-05 01:28:20	0|sipa|BlueMatt: yup, that's what i imagined
173 2016-12-05 01:28:33	0|BlueMatt|if the user wants to know whats inside, there is some api which can parse it into a c struct or whatever
174 2016-12-05 01:28:49	0|sipa|a batch key-value write operation, and a key read operation
175 2016-12-05 01:29:05	0|jtimon|kind of like https://github.com/bitcoin/bitcoin/pull/8493/commits/ad3ac37387b231378573f4996c59467247fccd44 ?
176 2016-12-05 01:30:15	0|jtimon|^ for the "binary blobs the caller doesn't know about"
177 2016-12-05 01:31:00	0|sipa|jtimon: i don't see such a thing at all
178 2016-12-05 01:31:22	0|sipa|that commit is about bitcoinconsensus_create_consensus_parameters
179 2016-12-05 01:32:06	0|jtimon|well, yeah, sorry, this are void pointers the other it's just data like in the tx for current verifyScript
180 2016-12-05 01:33:06	0|jtimon|in this case we would use the serialize lib to interpret and produce the "blobs", correct?
181 2016-12-05 01:33:55	0|sipa|if there needs to be a way to view the utxo set, i'd just provide a separate api for that
182 2016-12-05 01:34:00	0|sipa|not a parser for the database
183 2016-12-05 01:34:10	0|sipa|and not in the first stage
184 2016-12-05 01:36:02	0|jtimon|I am extremely interested in hearing in what other people's next steps, or more feedback on my own proposed next step
185 2016-12-05 01:38:30	0|jtimon|sipa: I'm still not sure if you prefer to expose verifyBlock or processBlock
186 2016-12-05 01:39:41	0|sipa|jtimon: i don't think we can do so right now anyway, without having a way to abstract state out
187 2016-12-05 01:40:13	0|sipa|imho the first step is just continuing refactoring so that consensus logic and other things become better separated internally
188 2016-12-05 01:40:24	0|sipa|and not focus on exposing things
189 2016-12-05 01:40:50	0|sipa|but others may disagree - i think wumpus prefers first having a clear idea of what will be exposed
190 2016-12-05 01:41:29	0|sipa|even a verifyBlock will need a way to pass in the utxo set and the block index
191 2016-12-05 01:42:27	0|sipa|the only difference is that a processBlock doesn't need a way to update set utxo set, and doesn't need to be able to request other blocks in case of a reorg
192 2016-12-05 01:44:08	0|jtimon|althought I tend to agree, I feel that's very vague and doesn't help on clarifying priorities, thinking of the next thing to expose, I think, helps clarify what the goal of the refactors should be and where are we supposed to be moving towards to
193 2016-12-05 01:44:36	0|sipa|you know my opinion - i don't care about exposing anything at all at this point, so i'm the wrong person to ask
194 2016-12-05 01:44:47	0|jtimon|yes verifyBlock would need an interface to access data from the utxo
195 2016-12-05 01:44:52	0|sipa|i think we have harder problems to solve before exposing even comes into question
196 2016-12-05 01:45:52	0|jtimon|dcousens: proposal was to pass all required data explicitly for the block you were validating
197 2016-12-05 01:46:15	0|sipa|essentially, i think we should first introduce clean abstractions between certain modules inside bitcoin core, in such a way that it's effectively bitcoind using a consensus library already, without it being exposed
198 2016-12-05 01:46:31	0|sipa|when it's good enough for us to use, we can think about exposing it to others
199 2016-12-05 01:47:06	0|sipa|(but again, others may see things differently)
200 2016-12-05 01:48:01	0|jtimon|right, and I think the module that should be a priority to cleanly separate is the part of the code that is required to fully validate whether a block is valid or not from everything else
201 2016-12-05 01:48:55	0|sipa|but that's very tightly coupled with validation of the whole chain, through CBlockIndex
202 2016-12-05 01:49:27	0|jtimon|right, it basically depends on chain.o and coins.o
203 2016-12-05 01:50:10	0|sipa|you can't validate a block without knowing its CBlockIndex
204 2016-12-05 01:50:25	0|jtimon|more specifically on two existing classes on them, an API for that is not hard
205 2016-12-05 01:50:26	0|sipa|so i'm not sure whether "single block validation" is a useful abstraction on its own
206 2016-12-05 01:50:48	0|sipa|transaction validation may be useful
207 2016-12-05 01:51:36	0|jtimon|CBlockIndex is the storage interface I abstract (or abstract from its own exsiting abstraction) in 8493
208 2016-12-05 01:51:49	0|sipa|i don't want an abstraction for CBlockIndex
209 2016-12-05 01:52:25	0|jtimon|my proposed next steps are single header validation or single tx validation
210 2016-12-05 01:52:34	0|jtimon|but without policy checks
211 2016-12-05 01:53:35	0|sipa|as i said, i don't think we should focus on exposing interfaces now, but on separating modules
212 2016-12-05 01:53:49	0|sipa|and i think separating block validation from chain validation is hard
213 2016-12-05 01:54:04	0|BlueMatt|jtimon: I'm with sipa here - The Main Split was the first of many steps that make sense on their own to abstract out consensus and non-consensus code
214 2016-12-05 01:54:19	0|BlueMatt|jtimon: the few commits I sent you earlier form the very tiny beginning of what I think are the next steps
215 2016-12-05 01:54:48	0|jtimon|separating network things was absolutely brilliant, thanks again
216 2016-12-05 01:54:50	0|BlueMatt|jtimon: ie having a state object internally which keeps chainstate in it and calls out to things for disk access and has ProcessNewBlock as a member function
217 2016-12-05 01:55:10	0|sipa|BlueMatt: chainstate include mapBlockIndex?
218 2016-12-05 01:55:25	0|BlueMatt|and its ~no code changes, just some function splitting and putting ClassName:: in front of them
219 2016-12-05 01:55:35	0|BlueMatt|sipa: yes, mapBlockIndex and chainActive and related variables
220 2016-12-05 01:55:52	0|BlueMatt|sipa: but calling out for ReadBlockFromDisk, and pcoinsTip is just a pointer that is passed to it
221 2016-12-05 01:56:03	0|sipa|BlueMatt: got it
222 2016-12-05 01:56:09	0|sipa|seems like an easy first step
223 2016-12-05 01:56:34	0|jtimon|sipa: if you don't care about exposing, that's fine, let's talk about dependencies, I want the consensus module to fully verify a single tx and a single header and a single block without depending on coins.o or chain.o
224 2016-12-05 01:56:35	0|BlueMatt|yea, should be pretty clean...I dont have time to do it for the next week or two...do you want to take it up jtimon?
225 2016-12-05 01:56:50	0|BlueMatt|I also want to work on splitting up net_processing more so that we can multithread ProcessMessages
226 2016-12-05 01:57:08	0|BlueMatt|jtimon: I dont see how thats possibel?
227 2016-12-05 01:57:20	0|sipa|yes, i think those are possible
228 2016-12-05 01:57:22	0|jtimon|I don't know what you want to do, how can I pick it up?
229 2016-12-05 01:57:24	0|sipa|(don't
230 2016-12-05 01:57:36	0|BlueMatt|jtimon: literally the point of "fully validating" a tx is to validate it against a CCoins-holding UTXO db
231 2016-12-05 01:57:51	0|BlueMatt|jtimon: did you look at the top commit on the branch I sent you?
232 2016-12-05 01:57:53	0|sipa|efficiency of validation is highly dependent on low-level access to coins and chain
233 2016-12-05 01:57:57	0|jtimon|sigh, I thought I had proved it was possible repeated times...
234 2016-12-05 01:58:06	0|BlueMatt|jtimon: https://github.com/TheBlueMatt/bitcoin/commit/54c967e73a1d1abba8f426fce0f14c5eeaf277e6
235 2016-12-05 01:58:09	0|sipa|it's possible if you introduce abstractions everywhere
236 2016-12-05 01:58:23	0|jtimon|is it possible to fully validate a header without depending on chain.o?
237 2016-12-05 01:58:34	0|sipa|no
238 2016-12-05 01:59:00	0|sipa|(unless you abstract it out, of course)
239 2016-12-05 01:59:03	0|jtimon|then what's happening in 8493
240 2016-12-05 01:59:06	0|jtimon|right
241 2016-12-05 01:59:33	0|sipa|but i think such abstraction are both a performance issue and an unnecessary code complication
242 2016-12-05 02:00:22	0|jtimon|well, more than half of 8493 is purely for demonstrating the exposed api and without benchmarking of any kind
243 2016-12-05 02:03:50	0|jtimon|my goal was to separate the code the verify a full block, depending on chain.o and coins.o (but only on those related to storage) [maybe put it all in the consensus folder? or wait for later?] but not putting it in the consensus module until you want to expose more and abstract it from chain and coins
244 2016-12-05 02:05:43	0|jtimon|anyway, I'm happy reviewing any related refactors, please ping me
245 2016-12-05 02:07:01	0|jtimon|sipa: does the GetConsensusFlag make any sense to you at a first glance? at least more than the previous version?
246 2016-12-05 02:08:22	0|jtimon|without exposing anything, just as a refactor (note that calling GetConsensusFlag inside ContextualCheckBlock is painful performance-wise)
247 2016-12-05 05:45:51	0|bitcoin-git|[13bitcoin] 15jtimon opened pull request #9279: Consensus: Move CFeeRate out of libconsensus (06master...060.13-consensus-dust-out-minimal) 02https://github.com/bitcoin/bitcoin/pull/9279
248 2016-12-05 05:46:15	0|bitcoin-git|[13bitcoin] 15jtimon closed pull request #7820: Consensus: Policy: Move CFeeRate out of consensus module and create CPolicy interface (06master...060.12.99-consensus-dust-out) 02https://github.com/bitcoin/bitcoin/pull/7820
249 2016-12-05 06:49:00	0|gmaxwell|including some clown at 138.68.10.138 who looks like he's connected three times to everyone; while pretending to be android wallet (he's not).
250 2016-12-05 07:06:15	0|bitcoin-git|[13bitcoin] 15laanwj pushed 5 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/4d955fc5824b...46904ee5d2ce
251 2016-12-05 07:06:17	0|bitcoin-git|13bitcoin/06master 14a188353 15Pieter Wuille: Switch GetTransaction to returning a CTransactionRef
252 2016-12-05 07:06:17	0|bitcoin-git|13bitcoin/06master 14c3f5673 15Pieter Wuille: Make CWalletTx store a CTransactionRef instead of inheriting
253 2016-12-05 07:06:18	0|bitcoin-git|13bitcoin/06master 1442fd8de 15Pieter Wuille: Make DecodeHexTx return a CMutableTransaction
254 2016-12-05 07:08:09	0|bitcoin-git|[13bitcoin] 15laanwj pushed 7 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/46904ee5d2ce...d04aebaec7bb
255 2016-12-05 07:08:10	0|bitcoin-git|13bitcoin/06master 146fdd43b 15Matt Corallo: Add struct to track block-connect-time-generated info for callbacks
256 2016-12-05 07:08:10	0|bitcoin-git|13bitcoin/06master 14fd9d890 15Matt Corallo: Keep blocks as shared_ptrs, instead of copying txn in ConnectTip
257 2016-12-05 07:08:11	0|bitcoin-git|13bitcoin/06master 14ae4db44 15Matt Corallo: Create a shared_ptr for the block we're connecting in ActivateBCS
258 2016-12-05 07:08:21	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #9014: Fix block-connection performance regression (06master...062016-10-fix-tx-regression) 02https://github.com/bitcoin/bitcoin/pull/9014
259 2016-12-05 07:44:07	0|sipa|BlueMatt: i can't build flto with -O3, but i can with -O2
260 2016-12-05 08:00:51	0|BlueMatt|sipa: strange...I was not doing that, iirc
261 2016-12-05 08:01:05	0|BlueMatt|I think I tried -march=native originally, but turned it off when it didnt work
262 2016-12-05 08:01:09	0|BlueMatt|(and ccache and all that shit)
263 2016-12-05 08:02:04	0|BlueMatt|gmaxwell: this is why I auto-ban on strange-ass nodes like that (and had already banned that ip)
264 2016-12-05 08:02:35	0|BlueMatt|and why my public node is behind stupid-ridiculous ddos protection :)
265 2016-12-05 08:04:51	0|gmaxwell|sipa: why does O3 break it?!
266 2016-12-05 08:05:10	0|sipa|gmaxwell: sense it makes none
267 2016-12-05 08:05:30	0|sipa|a linker error with ~boost::filesystem::path
268 2016-12-05 08:07:18	0|wumpus|never tried flto with -O3, no issues with -O2
269 2016-12-05 08:09:18	0|sipa|(which afaik was what BlueMatt was seeing as well)
270 2016-12-05 08:10:03	0|BlueMatt|sipa: yes, it was, though I thought I went back and disabled all the strange optimizations for testing when it failed
271 2016-12-05 08:10:07	0|BlueMatt|actually, pretty sure I did
272 2016-12-05 08:17:14	0|gmaxwell|I have a newly updated mass-connector/spy banlist which added 13 new entries:
273 2016-12-05 08:17:17	0|gmaxwell|https://0bin.net/paste/iSGeBTgtmA8cSHGX#VBd97-ME4hUiIqJIOsd24oLfM57UXP3F3QfgzuoVHAD  -- with commandline syntax
274 2016-12-05 08:17:20	0|gmaxwell|https://0bin.net/paste/zplzkwGmn6oeP6Ec#oTFWsZFpGQ1zR10Ofv3mJbIOGCnoshxQBQBJoWhSaip -- with debug console syntax
275 2016-12-05 08:21:39	0|wumpus|thanks
276 2016-12-05 08:30:57	0|Lightsword|gmaxwell, maybe 138.197.197.164 as well
277 2016-12-05 08:31:45	0|Lightsword|and 138.197.197.132 and 180.173.203.229 and 138.197.197.108
278 2016-12-05 08:31:49	0|bitcoin-git|[13bitcoin] 15jonasschnelli opened pull request #9280: [Qt] Show ModalOverlay by pressing the progress bar, allow hiding (06master...062016/12/qt_modal) 02https://github.com/bitcoin/bitcoin/pull/9280
279 2016-12-05 08:32:38	0|gmaxwell|Lightsword: I've only been including ones that show up on all my input hosts, unfortunately since everyone is at limits, that conceals a few.
280 2016-12-05 08:35:06	0|BlueMatt|gmaxwell: just take the unlimited-connection-slots patch?
281 2016-12-05 08:37:26	0|sipa|you can set -maxconnections=1000 without any patches
282 2016-12-05 08:41:42	0|BlueMatt|sipa: huh? I thought that sets you at 125?
283 2016-12-05 08:42:10	0|sipa|no
284 2016-12-05 08:42:16	0|sipa|125 is just the default
285 2016-12-05 08:42:30	0|BlueMatt|sipa: its limited by available sockets
286 2016-12-05 08:42:31	0|wumpus|depends on what the fd lmiit is
287 2016-12-05 08:42:35	0|BlueMatt|which can be super low, because select()
288 2016-12-05 08:42:58	0|BlueMatt|or this used to be the case
289 2016-12-05 08:43:01	0|sipa|Warning: Reducing -maxconnections from 1000 to 873, because of system limitations.
290 2016-12-05 08:43:04	0|sipa|ok, 873.
291 2016-12-05 08:43:11	0|BlueMatt|oh, 873, hum, I thought it was lower
292 2016-12-05 08:43:13	0|wumpus|select() can handle 1024 on most systems, that's pretty much enough for most cases
293 2016-12-05 08:43:23	0|BlueMatt|whatever, I carry a patch to use poll() to make it actually higher....
294 2016-12-05 08:44:25	0|wumpus|I guess we've held up switching to poll because we expected to switch to libevent any day, that's taking a bit longer than expected :)
295 2016-12-05 08:44:42	0|sipa|Soon! (tm)
296 2016-12-05 08:44:48	0|wumpus|yea :-)
297 2016-12-05 08:44:52	0|gmaxwell|IIRC matt's patch is darn near trivial.
298 2016-12-05 08:46:27	0|BlueMatt|gmaxwell: yes, but given that its 873 not 1XX as I'd thought, probably not worth it
299 2016-12-05 08:46:40	0|BlueMatt|and libevent is actually sooner now
300 2016-12-05 09:13:44	0|bitcoin-git|[13bitcoin] 15kallewoof opened pull request #9281: Refactor: Removed using namespace <xxx> from bench/ & test/ sources (06master...06no-using-namespace-bench-test) 02https://github.com/bitcoin/bitcoin/pull/9281
301 2016-12-05 09:59:04	0|bitcoin-git|[13bitcoin] 15laanwj pushed 3 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/d04aebaec7bb...613bda418f83
302 2016-12-05 09:59:05	0|bitcoin-git|13bitcoin/06master 14605d701 15Pieter Wuille: Merge in LevelDB 1.19 changes
303 2016-12-05 09:59:05	0|bitcoin-git|13bitcoin/06master 14634ad51 15Pieter Wuille: Squashed 'src/leveldb/' changes from 20ca81f..a31c8aa...
304 2016-12-05 09:59:06	0|bitcoin-git|13bitcoin/06master 14613bda4 15Wladimir J. van der Laan: Merge #8613: LevelDB 1.19...
305 2016-12-05 10:03:08	0|sipa|wumpus: yeah, i didn't mean to say we should use reuse_logs, just that the feature over time may be interesting to us
306 2016-12-05 10:03:13	0|sipa|*roght now
307 2016-12-05 10:05:15	0|wumpus|sipa: right
308 2016-12-05 10:09:52	0|bitcoin-git|13bitcoin/06master 142efc438 15Pieter Wuille: Align struct COrphan definition
309 2016-12-05 10:09:52	0|bitcoin-git|[13bitcoin] 15laanwj pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/613bda418f83...43e8150ef690
310 2016-12-05 10:09:53	0|bitcoin-git|13bitcoin/06master 1443e8150 15Wladimir J. van der Laan: Merge #9269: Align struct COrphan definition...
311 2016-12-05 10:10:05	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #9269: Align struct COrphan definition (06master...06oneorphan) 02https://github.com/bitcoin/bitcoin/pull/9269
312 2016-12-05 10:16:09	0|bitcoin-git|[13bitcoin] 15paveljanik opened pull request #9282: CMutableTransaction is defined as struct (06master...0620161205_CMutableTransaction_is_struct) 02https://github.com/bitcoin/bitcoin/pull/9282
313 2016-12-05 10:36:48	0|bitcoin-git|[13bitcoin] 15sipa opened pull request #9283: A few more CTransactionRef optimizations (06master...06sharedblock2) 02https://github.com/bitcoin/bitcoin/pull/9283
314 2016-12-05 11:47:55	0|bitcoin-git|13bitcoin/06master 14ea83d00 15instagibbs: SendMoney: use already-calculated balance
315 2016-12-05 11:47:55	0|bitcoin-git|[13bitcoin] 15laanwj pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/43e8150ef690...c01f16aaa0ed
316 2016-12-05 11:47:56	0|bitcoin-git|13bitcoin/06master 14c01f16a 15Wladimir J. van der Laan: Merge #9165: SendMoney: use already-calculated balance...
317 2016-12-05 11:48:05	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #9165: SendMoney: use already-calculated balance (06master...06triv-curbal) 02https://github.com/bitcoin/bitcoin/pull/9165
318 2016-12-05 11:55:26	0|bitcoin-git|13bitcoin/06master 14c4b6fa8 15Pavel Janík: CMutableTransaction is defined as struct.
319 2016-12-05 11:55:26	0|bitcoin-git|[13bitcoin] 15laanwj pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/c01f16aaa0ed...7d5d44969b4a
320 2016-12-05 11:55:27	0|bitcoin-git|13bitcoin/06master 147d5d449 15Wladimir J. van der Laan: Merge #9282: CMutableTransaction is defined as struct...
321 2016-12-05 11:55:42	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #9282: CMutableTransaction is defined as struct (06master...0620161205_CMutableTransaction_is_struct) 02https://github.com/bitcoin/bitcoin/pull/9282
322 2016-12-05 13:02:45	0|bitcoin-git|[13bitcoin] 15jonasschnelli opened pull request #9284: Suppress some annoying deprecation warnings (OSX) (06master...062016/12/osx_warnings) 02https://github.com/bitcoin/bitcoin/pull/9284
323 2016-12-05 13:38:57	0|jonasschnelli|And comments on our keypoolrefil RPC call behavior?
324 2016-12-05 13:38:57	0|jonasschnelli|https://github.com/bitcoin/bitcoin/blob/master/qa/rpc-tests/keypool.py#L45
325 2016-12-05 13:39:10	0|jonasschnelli|The tests proof, that nodes[0].keypoolrefill(3) result in 4 available keys.
326 2016-12-05 13:39:22	0|jonasschnelli|But reading the API docs, it should be 3.
327 2016-12-05 13:39:35	0|jonasschnelli|IMO the +1 is wrong here
328 2016-12-05 13:39:55	0|jonasschnelli|(I'd ask because I'd like to fix this with the HD split in ext/int chain)
329 2016-12-05 13:56:31	0|dcousens|hmm, CTransaction assignment was totally removed aye
330 2016-12-05 14:00:52	0|dcousens|sipa: was removing CTransaction& operator=(const CTransaction& tx); necessary, or just a safety precaution?
331 2016-12-05 14:08:47	0|dcousens|meh, I guess I can just CTransactionRef anyway
332 2016-12-05 14:21:20	0|dcousens|eh, nvm, rebased all my local code, tl;dr was juts changing CTransaction to CTransactionRef, .vout to ->vout ... and thats it.
333 2016-12-05 14:21:26	0|dcousens|LGTM :)
334 2016-12-05 14:25:28	0|instagibbs|dcousens, I love happy endings
335 2016-12-05 14:49:45	0|dcousens|instagibbs: not so happy yet ha
336 2016-12-05 14:52:41	0|dcousens|trying a fresh-recompile,  but master seems to just lock up for me atm
337 2016-12-05 17:29:56	0|jl2012|in what situation, the "Warning: We do not appear to fully agree with our peers! You may need to upgrade, or other nodes may need to upgrade" will be shown?
338 2016-12-05 17:40:10	0|morcos|jonasschnelli: you around?
339 2016-12-05 17:41:33	0|morcos|re: #8501, I agree with not fixing the frequency..  But i'm unsure about the no duplicating the same value over and over again..
340 2016-12-05 17:41:35	0|gribble|https://github.com/bitcoin/bitcoin/issues/8501 | Add mempool statistics collector by jonasschnelli · Pull Request #8501 · bitcoin/bitcoin · GitHub
341 2016-12-05 17:42:14	0|morcos|It might depend on the use case, but for instance it might be valuable to know that it stayed the same for a while and then incremented all at once, as opposed to not being able to tell that it just hadn't been simpled in between
342 2016-12-05 17:43:04	0|morcos|my thought was if we saved having to record the time stamp, we might be able to put up with lots of duplicate values..  especially if we're saving for instance only 1000 data points at second frequency, it just won't use all that much memory
343 2016-12-05 17:43:27	0|morcos|anywya, sorry, dont mean to redesign your whole PR months after you opened it
344 2016-12-05 18:46:54	0|Chris_Stewart_5|Does -txindex significantly impact performance of IBD? I tried to sync last night and only synced ~10K blocks, which seems slow. Is that reasonable?
345 2016-12-05 18:48:57	0|sipa|that's totally unreasonable
346 2016-12-05 18:49:05	0|sipa|is it stuck?
347 2016-12-05 18:49:09	0|sipa|or just slow?
348 2016-12-05 18:52:29	0|Chris_Stewart_5|extremely slow it seems. I'm using out of box settings on 0.13.1 with -txindex.
349 2016-12-05 18:52:53	0|sipa|does increasing dbcache help?
350 2016-12-05 18:53:43	0|Chris_Stewart_5|I'll try it later and report back, default is 2GB?
351 2016-12-05 18:54:33	0|sipa|default is 300MB
352 2016-12-05 18:55:20	0|Chris_Stewart_5|mmm that is probably why. I thought it was significantly higher. How long does IBD take other people with that setting as default?
353 2016-12-05 18:59:03	0|sipa|what height are you at now?
354 2016-12-05 19:07:53	0|Chris_Stewart_5|403817
355 2016-12-05 20:33:43	0|Chris_Stewart_5|sipa: I should have been more clear, I have been trying to do IBD over the course of a few nights, with results like I said ~10k blocks a night.
356 2016-12-05 20:34:47	0|Chris_Stewart_5|the first ~250k blocks went relatively fast (a couple hour period) but I think some might have already been on disk? Perhaps i'm using the term IBD a little too loosely
357 2016-12-05 20:34:59	0|Chris_Stewart_5|but it is a major sync
358 2016-12-05 20:40:11	0|instagibbs|sdaftuar_, why would you want to spend a coin from you wallet that has descendants(already spent)? I'm surely thinking of this wrong
359 2016-12-05 20:41:49	0|instagibbs|or are descendants calculated from a tx point of view, ie other output has been spent in a chain, therefore that adds to that count
360 2016-12-05 20:42:23	0|sdaftuar_|instagibbs: oh, yeah i meant in-mempool descendants
361 2016-12-05 20:42:35	0|sdaftuar_|say you have a tx that has 2 outputs, you send me money and give yourself change.
362 2016-12-05 20:42:40	0|sdaftuar_|then i chain 24 transactions off it
363 2016-12-05 20:42:48	0|sdaftuar_|you try to spend your change, but that'll fail
364 2016-12-05 20:42:54	0|instagibbs|ok, didnt think of the fact that outputs are linked re:descendants
365 2016-12-05 20:43:21	0|sdaftuar_|that does seem to be a confusing property of the mempool limiting :)
366 2016-12-05 20:43:31	0|instagibbs|but that is obv in hindsight. Ok, well one issue is you might have asymmetrical limits.
367 2016-12-05 20:43:40	0|sdaftuar|yeah, i suggested using min()
368 2016-12-05 20:43:51	0|gmaxwell|it might have been better if that limit was split across outputs.
369 2016-12-05 20:44:07	0|sdaftuar|gmaxwell: that would then fail to capture the issue at hand, i think
370 2016-12-05 20:44:09	0|instagibbs|sdaftuar, hmm says max on my screen
371 2016-12-05 20:44:22	0|gmaxwell|e.g. A can have up to 24 decendants, it has two outputs, each can have 12 under it.
372 2016-12-05 20:44:35	0|sdaftuar|sorry, max(tx->ancestor, tx->descendants()) should be less than min(ancestorlimit, descendantlimit)
373 2016-12-05 20:44:40	0|instagibbs|oh i see nvm
374 2016-12-05 20:45:02	0|sdaftuar|gmaxwell: oh, hm.
375 2016-12-05 20:47:05	0|gmaxwell|in the worst case though it reduces your maximum to a log_outputs(depth), which isn't awesome.
376 2016-12-05 20:47:17	0|sdaftuar|maybe doable, but kind of yuck to implement i think
377 2016-12-05 20:48:01	0|gmaxwell|but it would prevent other people from chewing up your limit. I think this hasn't actually been a problem, though I could imagine it being one in certian kinds of transaction protocols.
378 2016-12-05 20:48:42	0|sdaftuar|what kinds of protocols do you have in mind?
379 2016-12-05 20:49:39	0|gmaxwell|In the abstract, protocols where someone delaying your transaction can allow the party to cheat like atomic swaps. Not that big of a concern since unless the head transaction is confirmed those protocols are not secure against miners.
380 2016-12-05 20:52:28	0|sdaftuar|right, if someone comes up with a use case that does rely on the parent not necessarily being confirmed, then that should alter our thinking
381 2016-12-05 23:50:22	0|dcousens|hmph
382 2016-12-05 23:50:30	0|dcousens|So I'm running master, no changes at all
383 2016-12-05 23:50:54	0|dcousens|And my bitcoind finishes up to verify, then just sits there on 100% CPU usage (probably forever, but who knows)
384 2016-12-05 23:51:13	0|gmaxwell|'up to verify'?
385 2016-12-05 23:51:15	0|dcousens|It fails to open up the RPC, or start synchronizing
386 2016-12-05 23:51:20	0|dcousens|checkblocks
387 2016-12-05 23:51:28	0|gmaxwell|what is the last log entry?
388 2016-12-05 23:51:34	0|gmaxwell|can you attach GDB?
389 2016-12-05 23:51:48	0|dcousens|It still keeps logging, but solely the tor control messages
390 2016-12-05 23:52:26	0|gmaxwell|that sounds like a deadlock then.
391 2016-12-05 23:52:29	0|dcousens|What do I need to do to attach GDB? Happy to do it
392 2016-12-05 23:52:39	0|gmaxwell|dcousens: what OS are you on?
393 2016-12-05 23:53:48	0|dcousens|just collecting info, sec
394 2016-12-05 23:54:28	0|gmaxwell|on *nix:   ps aux | grep bitcoin   to get the bitcoind pid    then gdb -p <pid> to attach.     then run thread apply all bt full   to get backtraces from every thread, and 0bin that to me,  and then you can type q<enter> to quit
395 2016-12-05 23:58:38	0|dcousens|http://pastie.org/private/a40ppfohw2kxxxundn1iw - for the info, doing gdb now