1 2016-10-23 00:57:59	0|gmaxwell|2016-10-23 00:57:16 - Disconnect block: 1651.55ms
  2 2016-10-23 01:10:07	0|gmaxwell|:( actually the disconnectblock message undersates it, seeing on a fast machine 97 seconds between update tips while disconnecting blocks.
  3 2016-10-23 01:15:16	0|morcos|gmaxwell: yeah disconnects are slow..  but those are on my list once i'm done banging my head against a wall with connects...   i tried to look up that on my nodes and i didn't see one
  4 2016-10-23 01:16:07	0|gmaxwell|I'm trying to roll back to a couple months ago for some tests.
  5 2016-10-23 01:16:48	0|gmaxwell|and it's going to take hours. I'm not sure if we regressed or it's just increased load. It was always slow but when invalidate block was first written I tested it by reorging back all the way to the start... and I know it didn't take me months. :)
  6 2016-10-23 01:21:21	0|sipa|18:20:59 <sipa> for every undo that creates a UTXO that doesn't exist yet (because it's an undo of the last spend from one txid), it's trying to 'modify' that entry, looking for it on disk
  7 2016-10-23 01:21:25	0|sipa|18:21:14 <sipa> not realizing that it needs to create a new one, and thus can avoid the lookup
  8 2016-10-23 01:21:58	0|sipa|so it causes on average 1 disk lookup for each txid being rolled back
  9 2016-10-23 01:22:04	0|sipa|over long periods of time
 10 2016-10-23 01:24:25	0|sipa|it would be an easy fix except for the fact that we want to use the more thorough current behaviour in the start-up consistency check (ah! that explains why that check is so slow as well...)
 11 2016-10-23 01:31:48	0|gmaxwell|in addition to that it looks like it's spending most of its time twiddling with the mempool. setting the relay fee to 1btc/kb and the mempool size to minimum has it going fast.
 12 2016-10-23 01:32:12	0|gmaxwell|not blindingly fast but fast enough that I wouldn't have commented (maybe 4 blocks per second or so)
 13 2016-10-23 01:32:39	0|gmaxwell|sampling the backtrace seems to show a lot of it under UpdateForDescendants
 14 2016-10-23 01:40:45	0|gmaxwell|so I have a node configured with connect=0 (what I've historically done when wanting no connections) and I see that it's managing to connect to itself over and over again...
 15 2016-10-23 01:40:49	0|gmaxwell|2016-10-23 01:40:11 trying connection 0 lastseen=0.0hrs
 16 2016-10-23 01:40:52	0|gmaxwell|2016-10-23 01:40:11 Added connection peer=721
 17 2016-10-23 01:40:54	0|gmaxwell|2016-10-23 01:40:11 Added connection peer=722
 18 2016-10-23 01:40:57	0|gmaxwell|2016-10-23 01:40:11 send version message: version 70014, blocks=435494, us=[::]:0, peer=721
 19 2016-10-23 01:41:00	0|gmaxwell|2016-10-23 01:40:11 connection from 127.0.0.1:53402 accepted
 20 2016-10-23 01:41:02	0|gmaxwell|2016-10-23 01:40:11 sending: version (103 bytes) peer=721
 21 2016-10-23 01:41:05	0|gmaxwell|2016-10-23 01:40:11 received: version (103 bytes) peer=722
 22 2016-10-23 01:41:07	0|gmaxwell|2016-10-23 01:40:11 connected to self at 127.0.0.1:53402, disconnecting
 23 2016-10-23 05:10:15	0|luke-jr|is there some reason we're not using SVG images in the GUI?
 24 2016-10-23 06:30:22	0|GitHub8|[13bitcoin] 15luke-jr opened pull request #8996: Network activity toggle (06master...06networkactive) 02https://github.com/bitcoin/bitcoin/pull/8996
 25 2016-10-23 09:56:04	0|gmaxwell|why is getblocktemplate returning txn results for me on a 0.13.1rc1 node?
 26 2016-10-23 09:56:39	0|gmaxwell|I thought as soon as segwit was defined it needed the capability?
 27 2016-10-23 09:57:08	0|gmaxwell|... if thats only triggered on activation, then uh we would expect some portion of hashpower who hasn't correctly prepared to simply drop off at that point.
 28 2016-10-23 09:57:27	0|gmaxwell|Where did I desync?
 29 2016-10-23 10:28:04	0|sdaftuar_|gmaxwell: possible I'm misremembering but I think gbt just won't signal for segwit until the capability is specified, but it will still return successfully
 30 2016-10-23 10:28:52	0|sdaftuar_|There should be tests for the intended behavior in segwit.py or p2p-segwit.py
 31 2016-10-23 10:30:35	0|gmaxwell|oh thats okay too then!
 32 2016-10-23 11:01:06	0|wumpus|luke-jr: because including the svg rendering engine would introduce an extra dependency, and also qt4/qt5 differences IIRC
 33 2016-10-23 11:02:00	0|wumpus|also drawing SVG is generally slower than just drawing pixmaps, unless you have some smart caching layer, I have no clue where Qt is in that regard
 34 2016-10-23 11:02:24	0|wumpus|tldr it's just a big risky change and things work pretty well as they do
 35 2016-10-23 11:03:41	0|wumpus|maybe it makes sense when, if ever, moving from qt widgets to qml quick or such based gui. I have no relevant experience with any of the qt newness in the last few years though.
 36 2016-10-23 11:06:25	0|wumpus|in any case I doubt we'll accept a patch that just changes image rendering to svg and has zero visual changes for the user, even though it would be 'neater' way of doing things in some sense, it's just not worth the review and testing overhead
 37 2016-10-23 11:07:13	0|wumpus|on the other hand a snappy GUI-redo that blows everyone away and happens to need SVG rendering, well, sure
 38 2016-10-23 12:06:29	0|nsh|there were a couple issues identified in NCC audit by NCC group which might be relevant to bitcoin-core. not sure if they made it upstream.
 39 2016-10-23 12:06:41	0|nsh|NCC-2016-015 - Out-of-bounds Read in Boost date Class #1459 - https://github.com/zcash/zcash/issues/1459
 40 2016-10-23 12:06:50	0|nsh|NCC-2016-008 - Potential uninitialized reads #1464  - https://github.com/zcash/zcash/issues/1464
 41 2016-10-23 12:09:19	0|wumpus|I've seen the report, but thanks for the update
 42 2016-10-23 12:09:44	0|wumpus|the respective fixes haven't made it upstream yet
 43 2016-10-23 12:21:52	0|jtimon|gmaxwell: wumpus btcdrak sorry I missed the rest conversation in https://botbot.me/freenode/bitcoin-core-dev/2016-10-22/?msg=75272893&page=1
 44 2016-10-23 12:22:55	0|jtimon|I know btcdrak hates "cascading PRs", but I think it makes sense here
 45 2016-10-23 12:23:39	0|jtimon|I plan to finish https://github.com/jtimon/bitcoin/compare/0.13-new-testchain...jtimon:0.13-blocksign hopefully on monday
 46 2016-10-23 12:23:49	0|btcdrak|nice!
 47 2016-10-23 12:24:32	0|jtimon|but I think they could really just use this temporarily and I'm afraid the block signing part will require a lot more review and testing and will take longer to merge
 48 2016-10-23 12:24:59	0|jtimon|they can -chain=custom -chainpetname=lightning, new chain
 49 2016-10-23 12:25:46	0|btcdrak|I'm ok with two PRs. blocksign will be rebased on the first PR anyway right?
 50 2016-10-23 12:26:12	0|jtimon|if an idiot spends a lot of energy mining that testchain, you can just -chain=custom -chainpetname=lightning2 and show him your middle finger, you can always -listen=0 -connect=... etc
 51 2016-10-23 12:27:20	0|jtimon|btcdrak: yeah blocksigning will be on top of this one since to create a new blocksigning chain you need to first create a new chain
 52 2016-10-23 12:28:00	0|jtimon|reading from a file was a later addition, but it was like 4 lines extra
 53 2016-10-23 12:30:06	0|jtimon|at least reusing ReadConfigFile, if we really prefer a json file over a conf file that may be more extra work, not sure how strongly wumpus wants json, I personally don't see the gain
 54 2016-10-23 12:31:44	0|btcdrak|I think json makes sense, there are potentially a lot of chain params to configure.
 55 2016-10-23 12:33:04	0|jtimon|regarding "what they asked for" rusty asked for providing the genesishash manually instead of it being calculated dynamically  from the genesis block (you can force a new genesis just changing -chainpetname) like this PR does, so I wanted his feedback on that (it's true I didn't need to open the PR already for that)
 56 2016-10-23 12:34:03	0|jtimon|btcdrak: the same number as with a conf file, but you're just not able to reuse util to handle them
 57 2016-10-23 12:35:06	0|jtimon|I don't think CChainParams::UpdateFromArgs() will get smaller by using json
 58 2016-10-23 12:36:38	0|btcdrak|It's also easier to share chain details with a json file.
 59 2016-10-23 12:37:53	0|jtimon|regarding "a pointless feature that we've already lowered the quality of the codebase to enable" I strongly disagree but I'm not entirely sure what gmaxwell is refering to
 60 2016-10-23 12:38:42	0|jtimon|he mentions Params().GetConsensus().getdarnaninterger() inside a loop
 61 2016-10-23 12:39:09	0|wumpus|let's bury that part of the discussion please
 62 2016-10-23 12:39:16	0|wumpus|both gmaxwell and me went out of line there
 63 2016-10-23 12:39:31	0|wumpus|blame it on stress, or whatever
 64 2016-10-23 12:42:00	0|jtimon|as opposed to chainparams.GetConsensus().getdarnaninterger() so I assume he is not complaining about changes related to #7829 . If he is complaining about the GetConsensus part, that was only for libconsensus to avoid the chainparams dependency which contains globals. Of course I agree that shouldn't be in a loop, it should be called once and made a local variable outside the loop, if it is the only chainparams that the function
 65 2016-10-23 12:42:00	0|jtimon|needs, it should take darninteger directly as parameter instead of the whole CChainParams
 66 2016-10-23 12:42:25	0|jtimon|ok, not trying to revive any fire, just trying to make sure we're on the same page
 67 2016-10-23 12:42:59	0|wumpus|I explained the reasoning behind it afterward, and what will be the future direction, no need to rake up anything
 68 2016-10-23 12:43:02	0|wumpus|right
 69 2016-10-23 12:43:21	0|jtimon|btcdrak: how is sharing mychain.json any easier than sharing mychain.conf ?
 70 2016-10-23 12:43:43	0|wumpus|if there's a bla().bla().bla() in a loop we can just factor it out of the loop, this is not rocketscience :)
 71 2016-10-23 12:44:32	0|gmaxwell|putting things in a file does not make them configurable.
 72 2016-10-23 12:44:41	0|gmaxwell|please keep that in mind.
 73 2016-10-23 12:44:43	0|wumpus|jtimon: json may be a more convenient format for automatic generation from the tests / handling nested structures, but I don't care deeply
 74 2016-10-23 12:44:47	0|jtimon|wumpus: what are your thoughts on json vs conf since you brought that up?
 75 2016-10-23 12:44:57	0|gmaxwell|many of our constants tie deeply into algorithims and protocol assumptions.
 76 2016-10-23 12:44:57	0|jtimon|I see
 77 2016-10-23 12:44:59	0|btcdrak|what wumpus said
 78 2016-10-23 12:45:28	0|wumpus|yes, there is compromise somewhere on what constants whould be configurable, I think the ones curently in ChainParams make sense though
 79 2016-10-23 12:45:45	0|wumpus|this doesn't mean the entire algorithm should be micro-manageable though the configuration file
 80 2016-10-23 12:45:58	0|wumpus|unless you want to replace it with JITed LUA or so, but that's clearly not a goal here
 81 2016-10-23 12:46:36	0|gmaxwell|yep.
 82 2016-10-23 12:46:57	0|gmaxwell|(as I said, just something to keep in mind.)
 83 2016-10-23 12:47:04	0|wumpus|it may be possible to switch between discretely defined algorithms in the config file though, e.g. between a PoW or "centrally signed blocks"
 84 2016-10-23 12:47:16	0|gmaxwell|fRequireStandard = false;
 85 2016-10-23 12:47:21	0|gmaxwell|sure
 86 2016-10-23 12:47:47	0|jtimon|once they're in ChainParams they are not constants, but we may have abused ChainParams and maybe we want to try to turn some back into constants (if testnet and regtest have the same values at least)
 87 2016-10-23 12:48:21	0|wumpus|well they are constants after reading them from the configuration file
 88 2016-10-23 12:48:30	0|wumpus|they don't change at runtime, that would be madness
 89 2016-10-23 12:48:43	0|jtimon|the way I was planning to expose the blocksigning was through a variable like -blocksignscript, maybe a boolean too
 90 2016-10-23 12:48:55	0|wumpus|sure, at the language leven they're not constants, but they already are not because they're fetched from a structure...
 91 2016-10-23 12:49:01	0|wumpus|leveL
 92 2016-10-23 12:49:29	0|jtimon|wumpus: right, they init once and then ChainParams should be always passed as const
 93 2016-10-23 12:49:36	0|gmaxwell|there are a at least some things that the values in chain params for testnet/regtest are at odds with the code. don't assume the paramters for testnet or regtest were especially well thought out. :) there may be places where we want to reduce their divergence with mainnet in the future... as their differences are a continued source of time-waste.
 94 2016-10-23 12:50:35	0|wumpus|gmaxwell: that's our whole point, though, there may be use for testnets that are more close to mainnet, or further away from it, dependeing on the specific testing
 95 2016-10-23 12:50:37	0|jtimon|I fear a cleanup may require testnet4 and regtest2
 96 2016-10-23 12:50:55	0|gmaxwell|I am wtfing at regtest2.
 97 2016-10-23 12:51:14	0|wumpus|regtest2 makes no sense
 98 2016-10-23 12:51:33	0|gmaxwell|but duh sure, improving things may mean some incompatiblities. thats fine.
 99 2016-10-23 12:51:35	0|wumpus|there is no shared 'regtest' network
100 2016-10-23 12:52:07	0|wumpus|although compatibility between versions may be useful for comparison testing
101 2016-10-23 12:52:10	0|wumpus|(!)
102 2016-10-23 12:52:46	0|jtimon|well, maybe if you want to make some values more similar to the mainnet to make them constants again, but I really don't know, I've thought more about testnet4, particularly in the context of bip70 which maybe gets replaced or something
103 2016-10-23 12:53:14	0|gmaxwell|E.g. I don't think when we created testnet or regtest anyone tought of the idea of inserting an optional bit mask in the target comparison function, so that high value blocks could be seen as meeting much lower targets.  I think if we'd thought of that we could have avoided some of the special difficulty rules there.
104 2016-10-23 12:53:53	0|jtimon|I particularly hate the fact that in bip70 testnet3 is called "test" instead of "testnet3" but that's a tiny detail
105 2016-10-23 12:54:44	0|wumpus|well a new testnet would need a new bip70 identifier I guess so that can be fixed then... but it's a minor inconsequential thing
106 2016-10-23 12:55:20	0|jtimon|and I dislike testnet's special case for pow too, but matt said it was quite useful (I don't really know)
107 2016-10-23 12:55:45	0|wumpus|well it will always need a special case for PoW, the question is can we do better than testnet3
108 2016-10-23 12:56:17	0|jtimon|not sure I understand gmaxwell's point about the bit mask
109 2016-10-23 12:57:07	0|wumpus|without special case for PoW a miner entering it and exiting it will just kill it, this happened before and was the reason for adding it to testnet3 in the first place. That doesn't mean it's the perfect solution that shoudl be used forever, but just reverting to plain PoW would be stupid.
110 2016-10-23 12:57:20	0|gmaxwell|regtest's 'special casing' requires difficulty go below one, which causes a large amount of divergence in the code.
111 2016-10-23 12:57:57	0|jtimon|wumpus: maybe a different difficulty filter would help more, but that's strong special-casing too
112 2016-10-23 12:58:03	0|wumpus|I sometimes wonder why doesn't just ignore PoW completely
113 2016-10-23 12:58:06	0|wumpus|regtest*
114 2016-10-23 12:58:21	0|wumpus|that would mean the PoW checking is regression tested less, ofcourse
115 2016-10-23 12:58:51	0|wumpus|jtimon: my point is just that testnet requires special casing there, the type of special casing is open for proposals though
116 2016-10-23 12:58:52	0|jtimon|gmaxwell: I see, what about making regtest just always pass the pow check?
117 2016-10-23 12:59:10	0|gmaxwell|that could have been better accomplished with a if (params->weakpow) memset(blockhash,0,4);  in the pow comparison.
118 2016-10-23 12:59:28	0|jtimon|I see
119 2016-10-23 12:59:34	0|gmaxwell|jtimon: there are tests that need pow to not be bypassed. Or at least there have been in the past, e.g. feeding lower difficulty blocks.
120 2016-10-23 13:00:06	0|wumpus|yes, that would make sense
121 2016-10-23 13:00:35	0|jtimon|gmaxwell: yeah just passing pow would need those tests to move to mainnet or testnet, but your solution seems less disruptive
122 2016-10-23 13:00:44	0|gmaxwell|the testnet getting stuckthing, even the current setup has problems with that, part of the call for the availability of signed block testnets.
123 2016-10-23 13:00:53	0|wumpus|regtest as it is now was a compromise back in the day and it works pretty well for regression testing, most trivial alternatives are probably worse
124 2016-10-23 13:01:33	0|gmaxwell|it also existed as a patch at first, it wasn't quite so designed out. Did it's roll fine, but with expirence better could be done now.
125 2016-10-23 13:01:48	0|jtimon|gmaxwell: right, so maybe after adding signed blocks it makes more sense to remove testnet3's mindif special case
126 2016-10-23 13:02:02	0|gmaxwell|It might also be possible to set the rest of the paramters like normal (2016 blocks, yadda yadda, just make it cheaper to mine if its not fast enough)
127 2016-10-23 13:02:07	0|gmaxwell|jtimon: potentially.
128 2016-10-23 13:03:05	0|gmaxwell|Basically we should either have divergences that _really_ aid testing (signed blocks) or otherwise minimize them, we really have lost of a lot of troubleshooting time due to testnet having issues that mainnet would never have, not all due to paramter differences, but many.
129 2016-10-23 13:04:26	0|jtimon|right
130 2016-10-23 13:06:23	0|jtimon|regarding block signing, I was thinking making it optional at compile time and just have blocks having both nBits-nNonce and scriptChallenge-scriptSolution in blocks (that's a hit on memory requirements that shouldn't be imposed on production nodes)
131 2016-10-23 13:07:00	0|jtimon|previously thought of union, but even that would be a hit if you cannot disable it at compile time
132 2016-10-23 13:07:22	0|jtimon|how does that sound?
133 2016-10-23 13:08:19	0|jtimon|I mean, even an extra pointer and polymorphism would be 4 or 8 extra bytes per header (apart from polymorphism performance concerns)
134 2016-10-23 13:08:20	0|wumpus|where would be the hit in memory requirements? header stoage?
135 2016-10-23 13:08:29	0|jtimon|yep, header storage
136 2016-10-23 13:09:12	0|jtimon|in elements we just remove nBits and nNonce, but obviously we cannot do that here
137 2016-10-23 13:09:12	0|wumpus|ok yes bleh, that structure is already too fat
138 2016-10-23 13:09:42	0|jtimon|so ack on compile time option for signed blocks?
139 2016-10-23 13:09:45	0|wumpus|using an union sounds better then
140 2016-10-23 13:10:02	0|wumpus|well compile time option does mean it cannot be used for the normal tests
141 2016-10-23 13:10:17	0|wumpus|and probably won't be enabled by default in releases
142 2016-10-23 13:10:45	0|wumpus|I'd prefer not to do that, unless this is a rarely used, memory hogging option, but then who would enable it in practice?
143 2016-10-23 13:10:48	0|jtimon|mhmm, if you allow the option you compile the tests that need it too?
144 2016-10-23 13:11:02	0|sipa|i don't think you can use a union with non-trivial C++ types in it
145 2016-10-23 13:11:08	0|jtimon|only devs I was assuming
146 2016-10-23 13:11:22	0|wumpus|this is about trivial C++ types such as ints right? nBits, nNonce etc
147 2016-10-23 13:11:26	0|sipa|c+11 relaxed the requirements a bit, though
148 2016-10-23 13:11:44	0|wumpus|if not that's a dangerous minefile
149 2016-10-23 13:11:47	0|wumpus|minefield*
150 2016-10-23 13:11:49	0|sipa|for block signing the signature is a CScript
151 2016-10-23 13:11:56	0|gmaxwell|compile time is pretty sad, how about an auxiliray datastructure that only gets created if using signed blocks, e.g. a second index?
152 2016-10-23 13:11:56	0|sipa|in CBlockHeader
153 2016-10-23 13:12:09	0|wumpus|that needs to be stored for every block, permanently?
154 2016-10-23 13:12:22	0|sipa|yes, instead of a nonce you have a signature
155 2016-10-23 13:12:45	0|jtimon|I was thinking ethier a union of structs PowProof and ScriptProof or an IntOrScript union for both nBits and nNonce
156 2016-10-23 13:12:45	0|wumpus|gmaxwell: yes
157 2016-10-23 13:12:57	0|gmaxwell|well technically it's a block witness...
158 2016-10-23 13:13:16	0|gmaxwell|segregate all the witnesses. :P
159 2016-10-23 13:13:18	0|wumpus|that makes sense; with using an union you could even take the additional memory requirement to 0
160 2016-10-23 13:13:28	0|wumpus|union a pointer with nBits,nNonce
161 2016-10-23 13:14:03	0|jtimon|I mean, the challenge field could be just constant per chain instead of being included in every block, I was just thinking that maybe someone could get creative with CalculateNextScriptChallenge or something
162 2016-10-23 13:14:25	0|wumpus|(i mean the additonal memory requirement when not using the feature, which is what we care about here)
163 2016-10-23 13:14:58	0|jtimon|this is it's actually in both CBlockHeader and CBlockIndex
164 2016-10-23 13:15:58	0|jtimon|wumpus: yeah, at a minimum union a pointer would be an extra pointer per block, that's my worry
165 2016-10-23 13:16:18	0|sipa|ah, it seems you can have non-trivial classes in a union now
166 2016-10-23 13:16:36	0|sipa|but it requires placement new and explicit destructor invocations
167 2016-10-23 13:16:49	0|jtimon|yep IntOrScript seemed to compile
168 2016-10-23 13:17:30	0|wumpus|jtimon: I don't understand that. You'd only need the pointer when using block signing, youd' only need nBits+nNonce when using PoW, those could be in the same memory space right?
169 2016-10-23 13:18:08	0|jtimon|so it would be a pointer to a struct that is either nBits+nNonce or a script (or a couple of them)
170 2016-10-23 13:18:20	0|sipa|wumpus: nBits is the "challenge" which can in theory also be replaced with a "block scriptPubKey"
171 2016-10-23 13:18:38	0|wumpus|jtimon: in the first case it'd just be nBits+nNonce in-place, in the second case it'd be a pointer to a script
172 2016-10-23 13:18:41	0|sipa|wumpus: so you can allow rules about how the signer(s) can change over time
173 2016-10-23 13:18:50	0|wumpus|sipa: okay
174 2016-10-23 13:18:55	0|sipa|however, that seems overkill here
175 2016-10-23 13:18:59	0|gmaxwell|a bit out of scope here but not incompatable.
176 2016-10-23 13:19:01	0|jtimon|sipa: right, but we can also remove that if the challenge script is going to be always constant
177 2016-10-23 13:19:14	0|sipa|as the block challenge can just be a chain wide constant as jtimon says
178 2016-10-23 13:19:18	0|gmaxwell|the union is the 64 bits of nbits+nonce or a pointer to an extension datastructure.
179 2016-10-23 13:20:05	0|jtimon|wumpus: I see, yeah, that's better and removes the need for the compile time option, thanks!
180 2016-10-23 13:20:37	0|sipa|perhaps we want the union to be between {nBits,nNonce} on the onenhand, and CScript *scriptSig on the other
181 2016-10-23 13:20:54	0|wumpus|sipa: that's what both gmaxwell and me are saying , yes :)
182 2016-10-23 13:21:09	0|sipa|note the *
183 2016-10-23 13:21:22	0|jtimon|or we could put the script in the coinbase with the other witnesses, but I'm not really sure I like that
184 2016-10-23 13:21:22	0|wumpus|I had assumed the *
185 2016-10-23 13:21:39	0|wumpus|I have no clue what the size of CScript is, but I'd guess it's larger than 8
186 2016-10-23 13:21:53	0|wumpus|so yes that should be a pointer
187 2016-10-23 13:21:54	0|sipa|24
188 2016-10-23 13:22:01	0|sipa|on 64-bit
189 2016-10-23 13:22:16	0|sipa|actually, it's a prevector, so much larger
190 2016-10-23 13:22:17	0|jtimon|yep, now I'm embarrased I didn't thought of the union being like that myself, but thanks guys, good call
191 2016-10-23 13:22:53	0|sipa|for some reason i am very scared of using unions
192 2016-10-23 13:23:12	0|wumpus|in this case the way it is used depends on a global setting, so I'm okay with it
193 2016-10-23 13:23:21	0|wumpus|I'm scared of tagged/per-case unions though
194 2016-10-23 13:23:25	0|jtimon|sipa: yeah that motivated my run to the compile option too
195 2016-10-23 13:23:43	0|sipa|but the CBlockHeader destructor will need to know which of the two cases is being used
196 2016-10-23 13:23:48	0|sipa|that's very ugly
197 2016-10-23 13:24:00	0|sipa|how will it know? a global?
198 2016-10-23 13:24:08	0|wumpus|maybe have two different CBlockHeader types?
199 2016-10-23 13:24:12	0|jtimon|oh, right, that's ugly
200 2016-10-23 13:24:25	0|jtimon|a static field in CBlockHeader maybe
201 2016-10-23 13:24:34	0|sipa|jtimon: that's just a global
202 2016-10-23 13:24:43	0|wumpus|CPoWBlockHeader CSignedBlockHeader .. but yeah that moves the problem up :)
203 2016-10-23 13:24:45	0|jtimon|sipa: yep
204 2016-10-23 13:24:57	0|sipa|wumpus: then you need to templatize all the block logoc
205 2016-10-23 13:25:07	0|jtimon|wumpus: CPoWBlockHeader CSignedBlockHeader is way too disruptive
206 2016-10-23 13:25:14	0|wumpus|yes...
207 2016-10-23 13:25:33	0|wumpus|no, never mind, that would be stupid in c++ :)
208 2016-10-23 13:26:01	0|jtimon|I mean, it's CBlockHeader<proofType> or whatever, but still, not an option I think
209 2016-10-23 13:26:08	0|wumpus|this is pretty much the place where the inflexibilty of C languages starts to show
210 2016-10-23 13:27:00	0|gmaxwell|only if the header itself owns the data, and it isn't just an index into a seperate data structure.
211 2016-10-23 13:27:15	0|wumpus|either you have to template everything, or do some crazy union hack, both seem like bad choices...
212 2016-10-23 13:27:38	0|sipa|it is in fact much easier (in terms of code changes) to make it tagged
213 2016-10-23 13:27:53	0|jtimon|well, I guess the less disruptive option would be to make CBlockHeader the base and use polymorphism, but I think we want to avoid that too
214 2016-10-23 13:27:58	0|gmaxwell|e.g. you could have a header-extradata structure, and the header just gets indexes into it. iirc we don't ever delete headers one accepted.
215 2016-10-23 13:28:06	0|wumpus|sipa: yes, but the tag takes up extra space, which was what we wre trying to avoid in the first place :)
216 2016-10-23 13:28:24	0|sipa|wumpus: i know, but far less than just always having both cases
217 2016-10-23 13:28:34	0|gmaxwell|so the extradata structure would own its own memory and be responsable for freeing it on shutdown.
218 2016-10-23 13:28:48	0|sipa|gmaxwell: the extra data is not just in CBlockIndex
219 2016-10-23 13:28:48	0|wumpus|gmaxwell: yes, indeed
220 2016-10-23 13:29:06	0|sipa|it's part of every CBlockHeader we send and receive
221 2016-10-23 13:29:25	0|jtimon|I think the best candidates are union and a compile time option
222 2016-10-23 13:29:28	0|sipa|it sounds very hard to avoid a memory leak if you do not deal with deletion
223 2016-10-23 13:29:40	0|jtimon|right, plus CBlock extends from CBlockHeader
224 2016-10-23 13:30:13	0|wumpus|what I like least about this is that it changes consensus data structures
225 2016-10-23 13:30:34	0|wumpus|for something that is just for testing
226 2016-10-23 13:31:59	0|jtimon|yep, that may be another argument in favor of the compile time option (which I realise is independent from using the union or not)
227 2016-10-23 13:32:43	0|gmaxwell|Compile time option would greatly diminish the utility of this. (especially considering the size of our static binaries)
228 2016-10-23 13:32:56	0|wumpus|with compile time option you could *guarantee* that the normal case remains unchanged
229 2016-10-23 13:33:14	0|gmaxwell|Yes.
230 2016-10-23 13:33:17	0|gmaxwell|at that point, might as well just make it a seperate repo and resync to core periodically, I think.
231 2016-10-23 13:33:23	0|wumpus|which is, imo, the only acceptable way to do this
232 2016-10-23 13:33:45	0|wumpus|yes, probably. It's effectively an altcoin at that point :)
233 2016-10-23 13:34:19	0|gmaxwell|Right, so why take noise in the codebase for it if we can't even make it as integrated as testnet? it's at least a pretty clean patch.
234 2016-10-23 13:35:37	0|wumpus|yes, it seems it's not practically doable at this time
235 2016-10-23 13:35:54	0|wumpus|in our current codebase and structure
236 2016-10-23 13:38:09	0|jtimon|well, I will try with the union and without compile time option and see how it looks like
237 2016-10-23 13:39:17	0|jtimon|if we do it as a constanly rebased branch, at least merging the custom chain patch would make the blocksigning one more maintainable
238 2016-10-23 13:40:18	0|sipa|jtimon: agree
239 2016-10-23 13:40:47	0|sipa|i was surprised there was not a resurgence of coingen like sites after chainparams was merged :)
240 2016-10-23 13:44:25	0|jtimon|sipa: you where also surprised there wasn't an elements_alpha_with_pow_back altcoin I asume, I guess you forget that the most important part of an altcoin is the logo not the features :p
241 2016-10-23 13:44:25	0|wumpus|proabably coingen didn't work too well as a business model
242 2016-10-23 13:45:12	0|wumpus|making it even easier to make altcoins by just changing one source file undermines that further, who would pay for it anymore :)
243 2016-10-23 13:45:27	0|sipa|wumpus, jtimon: we can of course trying the separate-branch approach first, merging only the unlikely-to-affect-consensus patches in mainline, to see how much interest there is in it
244 2016-10-23 13:45:46	0|wumpus|yes
245 2016-10-23 13:45:53	0|sipa|sure, not having it integrated inline may hurt adoption of such a chain
246 2016-10-23 13:46:19	0|sipa|but on the other hand, it would be a pity tongo throigh all the work of plugging this in if it doesn't get used
247 2016-10-23 13:46:28	0|sipa|*to go through
248 2016-10-23 13:48:16	0|jtimon|in any case, I think chainparams is older than coingen , isn't it? I don't remember not being a chainparams
249 2016-10-23 13:51:29	0|gmaxwell|not at all.
250 2016-10-23 13:55:11	0|sipa|chainparams was only in june 2013
251 2016-10-23 13:56:10	0|sipa|before that, we just had "if (fTestnet)" all over the place
252 2016-10-23 13:57:30	0|sipa|and testnet itself was october 2010, 2 months before i ever looked at the code
253 2016-10-23 14:00:07	0|wumpus|also altcoins, inherently driven by speculation, have always tended to fork from what is the speculation hotness of the day, at one point this was litecoin, after that the "PoS" coins, now it's probably ethereum-ism things. A profitable coingen would have to follow all that :p
254 2016-10-23 14:01:37	0|tulip|for a long time most alt coins were, and still are 0.6 based branches. the original proof of stake patches were never rebased onto modern Bitcoin Core until quite late in the crazy. the original lfnet IRC channels still have hundreds of alt coin nodes (but only 2-3 wxBitcoin).
255 2016-10-23 14:02:15	0|wumpus|yes :)
256 2016-10-23 14:02:52	0|sipa|many earlier ones forked off namecoin, was was based on 0.3.24 afaik
257 2016-10-23 14:02:57	0|tulip|up until recently there was a 0.3 and a 0.4 node still connected to #bitcoin00 on lfnet, one of the two still had 8333 routed. I'd love to know where that was running to be still up, but obviously well behind the chain this number of years on.
258 2016-10-23 14:03:21	0|wumpus|it's a lemons market, flooded with even more lemons every day, quite interesting from a psychology point of view not so much from a technological :)
259 2016-10-23 14:08:56	0|tulip|wumpus: given there's >200 name coin clients on lfnet I assume they never rebased past 0.5? surprised it even functions, they must be missing some serious patches by this point.
260 2016-10-23 14:15:40	0|wumpus|namecoin did fairly recently rebase on top of newer bitcoin core (not sure what version). But yes most coins do not, it's not like they're a big target for attacks, nor actively maintained. The only way we notice is that sometimes people file bugs/build system issues against bitcoincore that have been fixed years ago, not realizing we've moved on since
261 2016-10-23 14:17:16	0|wumpus|I'm going to try to get this gcbpay person banned from our github, this can no longer be accidental: https://github.com/bitcoin/bitcoin/issues?utf8=%E2%9C%93&q=is%3Aissue%20author%3Agcbpay%20
262 2016-10-23 14:17:47	0|luke-jr|wumpus: go the organization settings
263 2016-10-23 14:17:55	0|luke-jr|there's a "tab" for banning users
264 2016-10-23 14:25:23	0|sipa|wumpus: he has a bunch of repositories, but none contain code, and all issues are self created and look like nonsense as well
265 2016-10-23 14:25:39	0|sipa|it may be just someone who has no clue
266 2016-10-23 14:28:52	0|sipa|but if they're a nuisance and not responsive to comments, yes ban
267 2016-10-23 14:31:51	0|wumpus|Blocking a user prevents the following on all your repositories: opening or commenting on issues or pull requests, starring, forking, or watching,  adding or editing wiki pages
268 2016-10-23 14:32:05	0|wumpus|they can still download the source, or check it out
269 2016-10-23 14:32:10	0|wumpus|which is great
270 2016-10-23 14:32:32	0|wumpus|I don't expect any useful contributions from them, but they won't lose access completely, so this is the right thing to do
271 2016-10-23 17:06:35	0|arubi|in mempool too.  now, I can not get either A or B to mine this tx, no matter if hours pass or if I mine a thousand blocks at a time.  it just stays in both mempools.
272 2016-10-23 17:06:35	0|arubi|something weird on regtest (did not try testnet), I have two nodes addnode'd to eachother (A and B).  A first mines 750 blocks, sends the sum in one output to B, and mines block 751.  then B creates a 1 input to 607 p2wsh(15-of-15 multisig) outputs (to itself) tx and relays it to A which then mines a block.  B now has 607 15-of-15 p2wsh utxos, it combines them all a 607 p2wsh(15-of-15) -> 1 p2wpkh output tx, and relays to A which now has it
273 2016-10-23 17:06:59	0|arubi|the same process with a 606 15-of-15 outputs works fine.  still trying other types of scripts.  I couldn't get this to happen with p2pkh or p2sh(15-of-15).  it either aborts because the tx is too large, or too many sigops.
274 2016-10-23 17:07:15	0|arubi|606 and below*
275 2016-10-23 17:08:12	0|achow101|arubi: is it related to https://github.com/bitcoin/bitcoin/pull/8499#issuecomment-252420342
276 2016-10-23 17:08:25	0|achow101|the new policy limits for p2wsh?
277 2016-10-23 17:09:08	0|arubi|is 15 of 15 multisig affected?  looking
278 2016-10-23 17:12:28	0|sipa|how large is the 607 p2wsh output?
279 2016-10-23 17:12:56	0|arubi|well the script is 513 bytes
280 2016-10-23 17:13:36	0|arubi|signatures + pushes.. 1110 bytes?  I think
281 2016-10-23 17:13:46	0|arubi|can check, moment
282 2016-10-23 17:14:02	0|sipa|eh, i mean the size and weight of the transaction that does not get mined
283 2016-10-23 17:14:47	0|arubi|oh heh, sec.
284 2016-10-23 17:15:14	0|arubi|size": 999515, vsize": 268602,
285 2016-10-23 17:16:05	0|jl2012|vsize over 100000 is nonstandard?
286 2016-10-23 17:16:45	0|arubi|hm, so it's relayed around because regtest?  then not mined because it's too exotic?
287 2016-10-23 17:17:26	0|jl2012|not tested. but if it is accepted to mempool, it should also be mined?
288 2016-10-23 17:17:47	0|sipa|yes, being relayd/accepted but not mined sounds like a bug
289 2016-10-23 17:18:24	0|jl2012|arubi: are you sure your blocks are synced between the 2 nodes?
290 2016-10-23 17:18:31	0|arubi|yep
291 2016-10-23 17:18:45	0|jl2012|have you tried to do everything with only 1 node?
292 2016-10-23 17:19:09	0|arubi|I can try, though neither will mine it
293 2016-10-23 17:19:12	0|arubi|(trying)
294 2016-10-23 17:19:36	0|jl2012|are you doing it by hand or automated?
295 2016-10-23 17:20:03	0|jl2012|if automated, would you please share the script?
296 2016-10-23 17:20:19	0|arubi|a lot is automated, the script is a monstrosity
297 2016-10-23 17:20:39	0|arubi|I'm rewriting it, making it useful for more complex stuff.  mast in mind
298 2016-10-23 17:21:20	0|arubi|maybe it's time..  I'll get a gitlab account tomorrow and put it there
299 2016-10-23 17:22:36	0|jl2012|size: 999515 is for 606 or 607 inputs?
300 2016-10-23 17:23:11	0|arubi|607
301 2016-10-23 17:23:18	0|jl2012|i think there is a setting about the block size/weight, forgot the name
302 2016-10-23 17:23:26	0|jl2012|maybe you hit the limit
303 2016-10-23 17:23:39	0|arubi|maxblocksize=1m , maxblockweight=4m with my nodes
304 2016-10-23 17:23:56	0|jl2012|try with only maxblockweight=4m ?
305 2016-10-23 17:23:57	0|arubi|unless weight can go higher, I didn't assume it could
306 2016-10-23 17:24:07	0|arubi|hm.  alright
307 2016-10-23 17:24:17	0|sipa|do you literally set '1m' as value?
308 2016-10-23 17:24:21	0|arubi|oh no no
309 2016-10-23 17:24:25	0|jl2012|i think the problem is maxblocksize=1m
310 2016-10-23 17:24:27	0|sipa|ok, just making sure
311 2016-10-23 17:24:41	0|sipa|yes, maxblocksize=1000000 may be too much
312 2016-10-23 17:24:54	0|arubi|yea no worries.  so just not set it?
313 2016-10-23 17:24:54	0|jl2012|too small, you mean?
314 2016-10-23 17:24:54	0|sipa|i think we always stay 1000 bytes below the limit
315 2016-10-23 17:25:10	0|sipa|you can set both weight and size to 4000000
316 2016-10-23 17:25:43	0|arubi|re-running with 1 node now, waiting for it to finish.  sipa, oh really?  I really didn't think that's the case
317 2016-10-23 17:26:21	0|sipa|maxblocksize is about the total serialized size, including witness
318 2016-10-23 17:27:36	0|arubi|and weight?  seems like the default value for maxsize is 750000, so maybe that's why I assumed
319 2016-10-23 17:28:22	0|arubi|and default weight is 3m so really seemed like the x4 factor at the time, now I remember
320 2016-10-23 17:28:34	0|sipa|weight is (total_size + 3*size_without_witness)
321 2016-10-23 17:30:40	0|arubi|I see, thanks.  jl2012, single node same behavior.  trying w/ setting maxsize to 4m as well
322 2016-10-23 17:31:06	0|jl2012|quite sure it's because of the max size
323 2016-10-23 17:31:14	0|jl2012|i tried before
324 2016-10-23 17:32:13	0|arubi|you mean without setting it at all, or setting it to 4m?
325 2016-10-23 17:32:50	0|sipa|if you only specify maxblockweight, the maxblocksize is implicitly 4M
326 2016-10-23 17:32:53	0|jl2012|i think i just set max weight
327 2016-10-23 17:32:55	0|GitHub114|[13bitcoin] 15theuni opened pull request #9000: miner debugging: faux-mining (06master...06faux-mining) 02https://github.com/bitcoin/bitcoin/pull/9000
328 2016-10-23 17:33:19	0|jl2012|#9000
329 2016-10-23 17:33:53	0|arubi|ah, so is it correct to say maxblockweight supersedes maxblocksize?  congrats! :)
330 2016-10-23 17:34:04	0|arubi|bitcoin is officially over 9000
331 2016-10-23 17:34:16	0|sipa|not _over_ 9000.
332 2016-10-23 17:34:59	0|jl2012|arubi: if you set both, i guess it always take the effective lower one
333 2016-10-23 17:35:12	0|sipa|jl2012: if you set both, it respects both
334 2016-10-23 17:35:26	0|jl2012|make sense
335 2016-10-23 17:36:49	0|arubi|sipa, programmers of all people don't start counting at 1 :P
336 2016-10-23 17:37:43	0|sipa|arubi: ok, so we're at 8999 even.
337 2016-10-23 17:39:09	0|arubi|sipa, thanks :(
338 2016-10-23 17:54:43	0|arubi|\o/ jl2012 , sipa , thank you!  setting only weight=4m cleared a 607 inputs tx.  I'll play around with both size and weight, got a better idea what they mean now
339 2016-10-23 18:03:24	0|luke-jr|we are well over 9000 byte blocks, and even 9000 MB blockchain
340 2016-10-23 21:42:02	0|GitHub52|[13bitcoin] 15TheBlueMatt closed pull request #7903: Fix help text around importaddress and rename it to importscript (06master...0616-04-importaddress-helptext) 02https://github.com/bitcoin/bitcoin/pull/7903