1 2016-12-06 00:04:42	0|dcousens|gmaxwell: https://0bin.net/paste/GS3r5T8zbK5PWDmo#a97Ti6KI4ePCUa6tCJC0RWTDYa3oMxrM4dYJASGuga9 for gdb
  2 2016-12-06 00:05:16	0|dcousens|let me know if you need anything else
  3 2016-12-06 00:05:39	0|dcousens|I was considering compiling it without clang, just to see
  4 2016-12-06 00:15:48	0|dcousens|I forgot how much slower gcc builds :S
  5 2016-12-06 00:16:33	0|gmaxwell|dcousens: build with O1.
  6 2016-12-06 00:16:57	0|gmaxwell|Clang default optimization levels are less effective than GCC's it compiles faster but for most code produces slower binaries.
  7 2016-12-06 00:17:24	0|dcousens|gmaxwell: did you need me to build with O1 for debugging? Or just future reference
  8 2016-12-06 00:17:26	0|sipa|i sometimes configure with -O1 to compile faster
  9 2016-12-06 00:17:35	0|gmaxwell|no, just if the compile times bother you-- :)
 10 2016-12-06 00:17:39	0|sipa|you could compile with "-Og -ggdb3"
 11 2016-12-06 00:18:27	0|dcousens|brb, making breakfast ha
 12 2016-12-06 00:18:43	0|gmaxwell|I see things waiting on cs_main in that backtrace... supports the locking theory but I don't see where.
 13 2016-12-06 00:26:43	0|cfields|BlueMatt: around?
 14 2016-12-06 00:27:33	0|cfields|BlueMatt: something I'm hacking on just started coming together nicely, but would like to make sure we're on the same page before I get in too deep
 15 2016-12-06 00:27:42	0|dcousens|behaviour unchanged with gcc
 16 2016-12-06 00:27:58	0|dcousens|wait no
 17 2016-12-06 00:28:02	0|dcousens|no, RPC is responding now
 18 2016-12-06 00:28:43	0|bitcoin-git|[13bitcoin] 15gmaxwell opened pull request #9286: Time-based expiration in LimitOrphanTxSize only when map is full. (06master...06defer_orphan_exp) 02https://github.com/bitcoin/bitcoin/pull/9286
 19 2016-12-06 00:28:54	0|dcousens|though, only on requests not waiting CS_MAIN :P
 20 2016-12-06 00:29:01	0|gmaxwell|hah
 21 2016-12-06 00:29:08	0|gmaxwell|dcousens: do you know how to compile with debug lockorder?
 22 2016-12-06 00:29:20	0|cfields|dcousens: any chance it's sitting in an fread?
 23 2016-12-06 00:29:42	0|gmaxwell|My WAG is that something about your configuration has revealed a lock inversion introduced -- likely by one of Matt's recent race condition fixes -- but other people aren't seeing it.
 24 2016-12-06 00:29:47	0|dcousens|gmaxwell: no, a few commands away from doing it
 25 2016-12-06 00:29:59	0|cfields|dcousens: if so, i get that on one of my machines when i use a certain combo of clang/libc and hardening enabled
 26 2016-12-06 00:30:07	0|dcousens|gmaxwell: did you want me to start culling params?
 27 2016-12-06 00:30:22	0|gmaxwell|dcousens: no, I'd like you to build with the lockorder debugging enabled.
 28 2016-12-06 00:30:35	0|gmaxwell|if it is an inversion like I'm guessing it should immediately tell us exactly where.
 29 2016-12-06 00:31:14	0|cfields|dcousens: oh, nm, i see the bt
 30 2016-12-06 00:31:41	0|dcousens|gmaxwell: what do I need to do to enable it?
 31 2016-12-06 00:32:35	0|gmaxwell|dcousens: run ./configure CXXFLAGS="-DDEBUG_LOCKORDER -g"  then build like normal
 32 2016-12-06 00:33:00	0|dcousens|OK, interestingly this doesn't happen on a fresh .bitcoin (tried locally)
 33 2016-12-06 00:33:09	0|dcousens|running the configure now
 34 2016-12-06 00:35:13	0|dcousens|scratch that, still happens on a fresh .bitcoin :) (yay)
 35 2016-12-06 00:38:59	0|BlueMatt|cfields: yea, I can take a look
 36 2016-12-06 00:39:32	0|cfields|BlueMatt: well, not much to show yet, just an idea...
 37 2016-12-06 00:39:43	0|BlueMatt|sure
 38 2016-12-06 00:40:03	0|cfields|BlueMatt: using https://github.com/theuni/bitcoin/commit/3f598dbe7100c7c6c7bfb7e10210585327ed9d31 as a starting point
 39 2016-12-06 00:40:36	0|BlueMatt|cfields: didnt you already show me this?
 40 2016-12-06 00:40:38	0|cfields|with that, CConnman holds an instance of the message processor interface. Taking that a step further...
 41 2016-12-06 00:41:01	0|cfields|BlueMatt: yea, just giving the context
 42 2016-12-06 00:41:05	0|BlueMatt|ok
 43 2016-12-06 00:41:21	0|dcousens|while its compiling, i've binary searched the arguments to when it doesn't happen...
 44 2016-12-06 00:41:29	0|dcousens|and, I have the argument that causes it :P
 45 2016-12-06 00:41:45	0|dcousens|-minrelaytxfee=0.00000001
 46 2016-12-06 00:41:59	0|cfields|BlueMatt: er, jump to pm?
 47 2016-12-06 00:42:08	0|dcousens|remove that argf
 48 2016-12-06 00:42:10	0|dcousens|everything works
 49 2016-12-06 00:42:26	0|BlueMatt|cfields: ok
 50 2016-12-06 00:42:29	0|dcousens|ping gmaxwell
 51 2016-12-06 00:42:53	0|gmaxwell|dcousens: sweet
 52 2016-12-06 00:43:10	0|dcousens|and the lock order is compiled
 53 2016-12-06 00:43:15	0|dcousens|want a full bt?
 54 2016-12-06 00:43:27	0|gmaxwell|just run it.. is it spitting out any errors?
 55 2016-12-06 00:45:14	0|dcousens|gmaxwell: no errors yet
 56 2016-12-06 00:45:42	0|dcousens|nothing seen, would it be a hard crash or a debug log error?
 57 2016-12-06 00:45:59	0|dcousens|in any case, neither
 58 2016-12-06 00:46:45	0|gmaxwell|It logs with the normal logging process.
 59 2016-12-06 00:47:27	0|dcousens|gmaxwell: for what its worth, you could probably check it out with just the above arg, a fresh compile off master causes that deadlock with that arg
 60 2016-12-06 00:47:51	0|gmaxwell|yep. will be doing as soon as it compiles here.
 61 2016-12-06 00:47:53	0|dcousens|like, fresh compile, fresh .bitcoin,  deadlocks on -minrelaytxfee=0.00000001
 62 2016-12-06 00:48:07	0|gmaxwell|(takes like 15 minutes to comile bitcoin on my laptop)
 63 2016-12-06 00:48:23	0|dcousens|haha, the joy
 64 2016-12-06 00:49:08	0|gmaxwell|(and all my other systems are busy with longer running expirements at the moment)
 65 2016-12-06 01:06:20	0|gmaxwell|okay longer than 15 minutes.
 66 2016-12-06 01:09:06	0|gmaxwell|dcousens: your misbehavior doesn't reproduce for me. :(
 67 2016-12-06 01:09:20	0|dcousens|gmaxwell: :( - really?
 68 2016-12-06 01:09:36	0|dcousens|Dumb question
 69 2016-12-06 01:10:14	0|gmaxwell|dcousens: well it could require interaction with some other option, or some library/compiler version.
 70 2016-12-06 01:10:19	0|dcousens|It definitely fixes my issue, I mean, all my nodes are back up haha
 71 2016-12-06 01:10:39	0|dcousens|but, I'll retry local repro
 72 2016-12-06 01:14:15	0|dcousens|rm -rf ~/.bitcoin, git clean -xdf, ./autogen.sh, ./configure --disable-bench --disable-tests --disable-wallet --without-miniupnpc, make -j4, ./src/bitcoind -minrelaytxfee=0.00000001 ---- that was my previous steps, I'm retrying them now
 73 2016-12-06 01:16:19	0|dcousens|Otherwise, I guess I could maybe grab a list of some local package versions for you if you want, or try it out on a fresh ubuntu install
 74 2016-12-06 01:16:52	0|dcousens|but, I probably don't have that time atm :(
 75 2016-12-06 01:16:58	0|dcousens|(for the fresh install etc)
 76 2016-12-06 01:18:54	0|dcousens|gmaxwell: reproduced with the above commands
 77 2016-12-06 01:19:35	0|gmaxwell|hm. disable wallet. I could try that.
 78 2016-12-06 01:19:43	0|dcousens|commit 7d5d449
 79 2016-12-06 01:22:20	0|dcousens|gmaxwell: interesting
 80 2016-12-06 01:22:30	0|dcousens|-minrelaytxfee=0.1 does not deadlock
 81 2016-12-06 01:22:49	0|dcousens|-minrelaytxfee=0.00000001 does
 82 2016-12-06 01:23:45	0|dcousens|-minrelaytxfee=0.00000002 does not deadlock
 83 2016-12-06 01:23:48	0|dcousens|lol
 84 2016-12-06 01:24:05	0|dcousens|for god sake, I reckon the binary just has it in for my configuration
 85 2016-12-06 01:25:53	0|dcousens|rounding error I suppose?
 86 2016-12-06 01:26:07	0|dcousens|Want me to bisect back to the source commit?
 87 2016-12-06 01:29:52	0|gmaxwell|dcousens: would be useful, since only you can break it right now.
 88 2016-12-06 01:30:40	0|dcousens|gmaxwell: must be platform related, since the hardware is completely different (EC2 instance, RPi, and my Laptop)
 89 2016-12-06 01:30:45	0|dcousens|but they all run Arch
 90 2016-12-06 01:30:47	0|dcousens|:P
 91 2016-12-06 01:31:39	0|dcousens|and yeah, only in the last... 1.5 weeks of dev, if that
 92 2016-12-06 01:42:08	0|gmaxwell|dcousens: disable wallet didn't break it for me.
 93 2016-12-06 01:42:37	0|dcousens|gmaxwell: awesome,  the fact this happens with me for both gcc+clang makes me think it must be some other dependency?
 94 2016-12-06 01:42:46	0|dcousens|And cross-hardware
 95 2016-12-06 01:53:07	0|dcousens|man, net_processing touches so many files haha
 96 2016-12-06 01:53:55	0|dcousens|none the less, found the source of the bug so far, seems to be that static FeeFilterRounder
 97 2016-12-06 01:54:06	0|dcousens|if I change that to a constant fee != 1, it doesnt deadlock
 98 2016-12-06 01:54:22	0|dcousens|the investigation continues...
 99 2016-12-06 01:55:00	0|sipa|dcousens: try with https://github.com/bitcoin/bitcoin/pull/9268/files
100 2016-12-06 01:56:51	0|dcousens|sipa: success
101 2016-12-06 01:56:53	0|dcousens|that fixes the issue
102 2016-12-06 01:57:41	0|sipa|dcousens: can you briefly comment on the PR?
103 2016-12-06 01:57:46	0|dcousens|sipa: did
104 2016-12-06 01:57:47	0|sipa|ah, i'm too late
105 2016-12-06 01:58:45	0|dcousens|gmaxwell: welp, should I investigate further into why it deadlocks for only me or?
106 2016-12-06 02:00:49	0|dcousens|I feel like this is probably a red-herring
107 2016-12-06 02:01:25	0|dcousens|Because nothing in that very small code block aludes to a deadlock issue, from what I can see
108 2016-12-06 02:10:18	0|morcos|sipa: do you have any suspicion of what's going on?  that doesn't make much sense
109 2016-12-06 02:10:37	0|dcousens|morcos: not at this point, but, it repros quite solidly
110 2016-12-06 02:10:49	0|dcousens|-- on my machines, ha
111 2016-12-06 02:13:39	0|morcos|that first stack trace looks kind of crazy
112 2016-12-06 02:14:56	0|gmaxwell|I see the bug.
113 2016-12-06 02:15:17	0|dcousens|gmaxwell: oh?
114 2016-12-06 02:15:28	0|gmaxwell|well one sec. might have spoke too soon.
115 2016-12-06 02:15:38	0|sipa|morcos: my guess would be something related to initialization order
116 2016-12-06 02:15:52	0|gmaxwell|yea, okay not what I thought.
117 2016-12-06 02:15:55	0|gmaxwell|{ CAmount minFeeLimit = minIncrementalFee.GetFeePerK() / 2;
118 2016-12-06 02:15:55	0|gmaxwell|FeeFilterRounder::FeeFilterRounder(const CFeeRate& minIncrementalFee)
119 2016-12-06 02:16:21	0|gmaxwell|so his rate will go to zero there, and then I expect to find a loop immediately after it with that rate as an increment.
120 2016-12-06 02:16:25	0|gmaxwell|but it wasn't so. :)
121 2016-12-06 02:16:44	0|gmaxwell|oh wait, there it goes.
122 2016-12-06 02:16:46	0|dcousens|gmaxwell: still sounds plausible
123 2016-12-06 02:17:01	0|gmaxwell|for (double bucketBoundary = minFeeLimit; bucketBoundary <= MAX_FEERATE; bucketBoundary *= FEE_SPACING) {
124 2016-12-06 02:17:24	0|gmaxwell|so there you go bucketBoundary = 0 ; bucketBoundary <= MAX_FEERATE; bucketBoundary *= FEE_SPACING.. infinite loop.
125 2016-12-06 02:17:58	0|gmaxwell|though this doesn't explain why he's seeing it and I'm not.
126 2016-12-06 02:18:01	0|morcos|oops
127 2016-12-06 02:18:26	0|morcos|hmm
128 2016-12-06 02:18:33	0|dcousens|gmaxwell: and now the fun starts haha
129 2016-12-06 02:18:35	0|morcos|that's backwrds then
130 2016-12-06 02:18:57	0|gmaxwell|oh perhaps some artifact of mempool saving, which I wasn't clearing?
131 2016-12-06 02:19:09	0|morcos|oh nm
132 2016-12-06 02:19:12	0|gmaxwell|hm. no we don't save the feerates.
133 2016-12-06 02:19:15	0|dcousens|gmaxwell: mine was loading mempools from previous builds
134 2016-12-06 02:19:19	0|dcousens|that wouldn't be it
135 2016-12-06 02:19:22	0|morcos|ok so right now it interprets the command line argument and 9268 makes it not
136 2016-12-06 02:19:25	0|morcos|ok
137 2016-12-06 02:19:35	0|morcos|your explanation makes sense
138 2016-12-06 02:19:47	0|gmaxwell|right and apparently it used to not.  So the unexplained part is why is it not happening to everyone.
139 2016-12-06 02:20:16	0|gmaxwell|dcousens: can you insert some instrumentation in that function to log the minFeeLimit before the loop and bucketBoundary inside the loop?
140 2016-12-06 02:20:24	0|gmaxwell|just to confirm its really infinitely looping there?
141 2016-12-06 02:20:32	0|dcousens|gmaxwell: sure
142 2016-12-06 02:21:22	0|dcousens|304 or 490? 490 right
143 2016-12-06 02:21:27	0|dcousens|(line number0
144 2016-12-06 02:21:28	0|dcousens|)
145 2016-12-06 02:21:50	0|morcos|yeah 490
146 2016-12-06 02:22:39	0|morcos|gmaxwell: yeah wait, why would it get rounded to 0
147 2016-12-06 02:22:53	0|morcos|oh, b/c its an integer number of satoshis?
148 2016-12-06 02:23:17	0|gmaxwell|1/2 = 0  yes. CAmounts are integers.
149 2016-12-06 02:26:05	0|dcousens|gmaxwell: thats the deadlock
150 2016-12-06 02:26:37	0|dcousens|bucketBou;w
151 2016-12-06 02:26:44	0|dcousens|ah, ignore that
152 2016-12-06 02:27:04	0|dcousens|anyway, yeah, that is the deadlock*, minFeeLimit == 0, bucketBoundary = 0.0000
153 2016-12-06 02:27:22	0|gmaxwell|dcousens: great, but now why you and not anyone else.
154 2016-12-06 02:28:44	0|dcousens|gmaxwell: let me know if you need me to check anything,  have to return to a different task but happy to run anything
155 2016-12-06 02:29:00	0|gmaxwell|dcousens: thanks so much for all your help. I'm sure we've got it from here.
156 2016-12-06 02:31:00	0|morcos|dcousens: yes, thanks and sorry
157 2016-12-06 02:31:16	0|dcousens|morcos: all good, no need to be sorry, I run master for this exact reason :)
158 2016-12-06 02:40:24	0|morcos|gmaxwell: compiler isn't allowed to optimize that out is it?
159 2016-12-06 02:41:14	0|morcos|nm, stupid question, it can't get out of the loop...
160 2016-12-06 02:41:17	0|gmaxwell|morcos: if the compiler is able to prove a loop is infinite it can.
161 2016-12-06 02:41:29	0|gmaxwell|But it can't prove that loop is infinite, it depends on the argument.
162 2016-12-06 02:41:42	0|gmaxwell|but yea, actually you're probably right.
163 2016-12-06 02:42:11	0|gmaxwell|The loop uses floating point, god knows there is probably some way to rewrite that loop which is accurate except for 0.
164 2016-12-06 02:42:41	0|gmaxwell|but I've reproduced it.
165 2016-12-06 02:42:45	0|gmaxwell|so theres that.
166 2016-12-06 02:42:58	0|morcos|oh good
167 2016-12-06 02:43:12	0|gmaxwell|I yanked everything out of my configuration except txindex=1, and dcousens's setting.
168 2016-12-06 02:43:49	0|morcos|oh , so you sometimes don't get it still...  that is super weird
169 2016-12-06 02:43:58	0|gmaxwell|sipa: thoughts on merging https://github.com/bitcoin/bitcoin/pull/9268  to unbust master while the actual algorithim here gets fixed?
170 2016-12-06 02:44:13	0|gmaxwell|morcos: I think it might be a product of which peers I connect to.
171 2016-12-06 02:44:21	0|gmaxwell|e.g. doesn't get triggered until I send a fee filter.
172 2016-12-06 02:44:39	0|morcos|should we fix it by doing anything other than max(1,)
173 2016-12-06 02:46:08	0|gmaxwell|I don't see any reason to do anything but that. But thats not my code. :) I'd have to understand it to answer!
174 2016-12-06 02:47:02	0|morcos|yeah , i mean we could do something else entirely, its probably not the cleanest way to accomplish the goal, but i don't know that its worth doing anyting other than that, i'll pr
175 2016-12-06 02:48:37	0|gmaxwell|we could keep the behavior "this is what you get for setting you relayfee to 10 nanobitcoin and spamming other folks" :P
176 2016-12-06 02:50:23	0|morcos|Its an optimization itself, this is your nodes expected performance profile
177 2016-12-06 02:52:19	0|gmaxwell|"We've decided to save you the bandwidth and just give you a simulation of the expected cpu usage."
178 2016-12-06 02:53:12	0|bitcoin-git|[13bitcoin] 15morcos opened pull request #9288: Fix a bug if the min fee is 0 for FeeFilterRounder (06master...06fixFFRbug) 02https://github.com/bitcoin/bitcoin/pull/9288
179 2016-12-06 02:54:03	0|morcos|How snarky would it be if that was just BlueMatt's way of pointing out the underlying bug
180 2016-12-06 02:54:39	0|gmaxwell|well true, that fact that your node is down makes you pretty distinguishable from other users.
181 2016-12-06 02:57:21	0|gmaxwell|I guess its good? news that we already had a PR open that would have covered this bug up...
182 2016-12-06 03:02:09	0|bitcoin-git|[13bitcoin] 15theuni closed pull request #8631: Nuke boost::thread and boost::thread_group (06master...06no-interrupt-threads) 02https://github.com/bitcoin/bitcoin/pull/8631
183 2016-12-06 03:02:27	0|bitcoin-git|[13bitcoin] 15theuni opened pull request #9289: net: drop boost::thread_group (06master...06connman-threads) 02https://github.com/bitcoin/bitcoin/pull/9289
184 2016-12-06 03:05:23	0|dcousens|gmaxwell: haha
185 2016-12-06 03:05:50	0|dcousens|good to know you repro'd it :)
186 2016-12-06 03:06:09	0|gmaxwell|really since the introduction of fee filter, low minrelay fees aren't so anti-social anymore at least.
187 2016-12-06 03:07:28	0|dcousens|gmaxwell: IMHO, all I really would want as policy is IsStandard, and a maximum in-memory size, sorted by fee descending (accounting for CPFP etc)
188 2016-12-06 03:08:20	0|dcousens|hence, the setting of 1 haha
189 2016-12-06 03:10:33	0|dcousens|gmaxwell: I wouldn't mind seeing double spends in the mempool too, but, I guess that would annoy too many people
190 2016-12-06 03:11:25	0|dcousens|and probably opens too many DOS vectors
191 2016-12-06 03:14:47	0|gmaxwell|well double spends in the mempool would break varrious assumptions in the code. :)
192 2016-12-06 03:15:04	0|gmaxwell|dcousens: thats not unreasonable since the advent of feefilter.
193 2016-12-06 03:16:27	0|dcousens|morcos: line 304 could probably be bounded too, just incase MIN_FEERATE was changed
194 2016-12-06 06:21:11	0|wumpus|so did anyone compile bitcoin for risc-v yet? :)
195 2016-12-06 06:49:58	0|bitcoin-git|[13bitcoin] 15gmaxwell opened pull request #9290: Make RelayWalletTransaction attempt to AcceptToMemoryPool. (06master...06resend_retries_mempool) 02https://github.com/bitcoin/bitcoin/pull/9290
196 2016-12-06 07:08:45	0|bitcoin-git|[13bitcoin] 15sipa opened pull request #9291: Remove mapOrphanTransactionsByPrev from DoS_tests (06master...06reallyoneorphan) 02https://github.com/bitcoin/bitcoin/pull/9291
197 2016-12-06 07:10:36	0|warren|wumpus: there's machines available for it?
198 2016-12-06 07:11:12	0|gmaxwell|Does anyone recall why we decided to not backport #9053 ? I was just thinking about maybe it should be backported, then saw it was tagged and untagged.
199 2016-12-06 07:11:14	0|gribble|https://github.com/bitcoin/bitcoin/issues/9053 | IBD using chainwork instead of height and not using header timestamps by gmaxwell · Pull Request #9053 · bitcoin/bitcoin · GitHub
200 2016-12-06 07:11:24	0|wumpus|warren: I just ordered a hifive board, but that's arduino-class, nothing that can handle bitcoind IIRC. There's a toolchain and qemu (also linux user) though so it'd be possible to compile and run it
201 2016-12-06 07:11:46	0|sipa|gmaxwell: i think we considered it for 0.13.1, but decided it wasn't important enough for that?
202 2016-12-06 07:11:47	0|wumpus|warren: maybe if you use a RiscV core on one of the high-end FPGAs
203 2016-12-06 07:12:36	0|gmaxwell|TD-Linux: was telling me recently he planned on having an actual RiscV chip soon... but not one that would be anywhere near powerful enough to run bitcoin core. :P
204 2016-12-06 07:13:14	0|wumpus|gmaxwell: yeah, if there's no discussion on the issue it probably means it was discussed in a meeting
205 2016-12-06 07:13:42	0|wumpus|nov 3's meeting I'd guess
206 2016-12-06 07:14:15	0|jonasschnelli|<*highlight>	[18:40:09] <morcos:#bitcoin-core-dev> jonasschnelli: you around?
207 2016-12-06 07:14:15	0|wumpus|gmaxwell: yep this one probably: https://www.sifive.com/products/hifive1/  ordered one as well
208 2016-12-06 07:14:17	0|jonasschnelli|Now. You?
209 2016-12-06 07:14:24	0|gmaxwell|12:57 <@wumpus> I think personally I'd prefer to keep it for 0.14, so the new rule/logic can prove itself a while
210 2016-12-06 07:14:29	0|jonasschnelli|ref: [18:40:09] <morcos:#bitcoin-core-dev> jonasschnelli: you around?
211 2016-12-06 07:14:31	0|gmaxwell|okay, yep. sorry, hamster brain here.
212 2016-12-06 07:14:56	0|wumpus|gmaxwell: well it has been a while in master now ...
213 2016-12-06 07:15:09	0|wumpus|gmaxwell: I think that was just last-minute before a release
214 2016-12-06 07:16:34	0|jonasschnelli|gmaxwell: I have a solution for the HD chain-split (95% complete). I think it could be backwards compatible. Means, if a 0.13 wallet will be loaded, only external (m/0'/0'/k) will be generated. Creating a new wallet will respect the 0'/|1'/ switch.
215 2016-12-06 07:16:36	0|jonasschnelli|https://github.com/bitcoin/bitcoin/compare/master...jonasschnelli:2016/12/hd_split?expand=1
216 2016-12-06 07:16:38	0|gmaxwell|I don't have a really strong opinion, but I did just have a 0.13.1 testnet node act goofy today in a way that change fixes, so it came back to mind.
217 2016-12-06 07:17:12	0|jonasschnelli|Loading the 0.14 wallet in 0.13 will result in using only the external chain.. or we disable that.
218 2016-12-06 07:17:17	0|jonasschnelli|(probably better to disable)
219 2016-12-06 07:17:19	0|gmaxwell|jonasschnelli: but if you go back to an older version do funds silently disappear? or .. I guess we'll just not be backwards compatible?
220 2016-12-06 07:17:33	0|wumpus|I do think we should only backport critical things, not things that 'would be nice' but are risky. Also if it *only* affected testnet it'd be a no-brainer.
221 2016-12-06 07:17:47	0|jonasschnelli|Once the keypool is filled, the keypath doesn't matter.
222 2016-12-06 07:18:31	0|jonasschnelli|I think we should probably disable loading 0.14 (hd splitI) wallets in 0.13. Becuase once you load them in 0.13, you take the risk of having chain outputs in the external chain.
223 2016-12-06 07:18:34	0|jonasschnelli|This can be confusing.
224 2016-12-06 07:18:52	0|wumpus|this also has an effect on mainnet, it'd be a pain if it introduced some regression, though that chance is lower than at Nov 3 which was two days after it was openened :)
225 2016-12-06 07:19:06	0|gmaxwell|wumpus: well it fixes bad behavior that is possible on mainnet too just really unlikely and not something we're currently observing.
226 2016-12-06 07:19:53	0|wumpus|it's been merged in master for more than a month and no one at least observed any wacky behavior due to it, so I'd trust it for backport now
227 2016-12-06 07:22:07	0|sipa|jonasschnelli: it's 2:22 am for morcos
228 2016-12-06 07:22:20	0|bitcoin-git|[13bitcoin] 15laanwj pushed 3 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/7d5d44969b4a...ed8d693c71b0
229 2016-12-06 07:22:21	0|bitcoin-git|13bitcoin/06master 14412bab2 15instagibbs: Adapt ZMQ/rest serialization to take rpcserialversion arg
230 2016-12-06 07:22:21	0|bitcoin-git|13bitcoin/06master 14bc7ff8d 15Gregory Sanders: Add option to return non-segwit serialization via rpc
231 2016-12-06 07:22:22	0|bitcoin-git|13bitcoin/06master 14ed8d693 15Wladimir J. van der Laan: Merge #9194: Add option to return non-segwit serialization via rpc...
232 2016-12-06 07:22:35	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #9194: 	Add option to return non-segwit serialization via rpc (06master...06nonswserialrpc) 02https://github.com/bitcoin/bitcoin/pull/9194
233 2016-12-06 07:22:35	0|wumpus|sipa: ugh, your comment on #9194 was just too late
234 2016-12-06 07:22:37	0|gribble|https://github.com/bitcoin/bitcoin/issues/9194 | Add option to return non-segwit serialization via rpc by instagibbs · Pull Request #9194 · bitcoin/bitcoin · GitHub
235 2016-12-06 07:22:55	0|sipa|wumpus: can easily be fixed later
236 2016-12-06 07:23:03	0|wumpus|had already hit enter then I saw it appear at the same time :p
237 2016-12-06 07:23:27	0|wumpus|yes luckily just a help message
238 2016-12-06 07:23:56	0|sipa|well the comment is more about showing an error when an unknown value is passed
239 2016-12-06 07:24:22	0|sipa|i'll PR my suggested change
240 2016-12-06 07:25:17	0|jonasschnelli|sipa: Oh. Yes.
241 2016-12-06 07:27:55	0|jonasschnelli|For the chain-split, what would be preferable, a keypool=100 results in 50 internal and 50 external keys or in 100/100?
242 2016-12-06 07:28:08	0|bitcoin-git|[13bitcoin] 15sipa opened pull request #9292: Complain when unknown rpcserialversion is specified (06master...06nofutureserial) 02https://github.com/bitcoin/bitcoin/pull/9292
243 2016-12-06 07:28:22	0|jonasschnelli|sipa: I guess its 2.30am you as well?
244 2016-12-06 07:28:22	0|sipa|jonasschnelli: internal keys need way fewer
245 2016-12-06 07:28:38	0|sipa|jonasschnelli: no, i'm on the west coast, 23:28 here
246 2016-12-06 07:28:40	0|gmaxwell|no, the US is a big place.
247 2016-12-06 07:28:50	0|jonasschnelli|Ah. Right. He's NY.
248 2016-12-06 07:28:59	0|gmaxwell|US timezones span 7 hours IIRC.
249 2016-12-06 07:29:43	0|sipa|there's a tiny part of the US on the other side of the international date line
250 2016-12-06 07:30:12	0|jonasschnelli|sipa: But in general, should the keypool=<value> results in <value>=<internal>+<external> or <value>=<internalamout>, <value>=<externalamount>?
251 2016-12-06 07:30:41	0|gmaxwell|yea, I wasn't counting guam.
252 2016-12-06 07:30:55	0|gmaxwell|in that case they span like 20 hours. :P
253 2016-12-06 07:31:47	0|sipa|gmaxwell: you don't need to count territories. there is a tiny part of alaska across the date line
254 2016-12-06 07:33:00	0|sipa|ah, but it still uses western time. ignore me
255 2016-12-06 07:51:07	0|sipa|https://github.com/bitcoin/bitcoin/pulse/monthly
256 2016-12-06 07:51:11	0|sipa|that's pretty cool
257 2016-12-06 07:51:20	0|sipa|Excluding merges, 33 authors have pushed 160 commits to master and 175 commits to all branches. On master, 288 files have changed and there have been 13,187 additions and 10,679 deletions.
258 2016-12-06 07:53:11	0|wumpus|nice!
259 2016-12-06 07:53:31	0|wumpus|BlueMatt is clearly winning this month :)
260 2016-12-06 07:54:43	0|sipa|also the last week and the last 3 days
261 2016-12-06 07:55:20	0|sipa|but over a month... 112 PRs merged, and 50 PRs opened
262 2016-12-06 07:55:23	0|sipa|that's pretty good
263 2016-12-06 08:57:53	0|bitcoin-git|[13bitcoin] 15gmaxwell opened pull request #9293: [0.13 Backport] IBD using chainwork instead of height and not using header timestamp (#9053) (060.13...069053_backport) 02https://github.com/bitcoin/bitcoin/pull/9293
264 2016-12-06 08:58:50	0|gmaxwell|based on the prior discussion ^, if we decide we don't want to-- I won't be offened. But I figured I wouldn't ask about it then fail to do the work.
265 2016-12-06 08:59:45	0|gmaxwell|(it wasn't any work, however)
266 2016-12-06 09:00:05	0|wumpus|great
267 2016-12-06 10:08:32	0|bitcoin-git|[13bitcoin] 15jonasschnelli opened pull request #9294: Use internal HD chain for change outputs (hd split) (06master...062016/12/hd_split) 02https://github.com/bitcoin/bitcoin/pull/9294
268 2016-12-06 10:13:59	0|bitcoin-git|[13bitcoin] 15jonasschnelli closed pull request #9238: Ignore BIP35 mempool command by default (06master...062016/11/dis_mempool) 02https://github.com/bitcoin/bitcoin/pull/9238
269 2016-12-06 10:15:17	0|bitcoin-git|[13bitcoin] 15jonasschnelli closed pull request #8182: [Qt] Add simple opt-in-RBF support (06master...062016/04/qt_rbf_set_new) 02https://github.com/bitcoin/bitcoin/pull/8182
270 2016-12-06 10:15:49	0|bitcoin-git|[13bitcoin] 15jonasschnelli closed pull request #7685: Add bloom filter usage statistics (06master...062016/03/bf_stats) 02https://github.com/bitcoin/bitcoin/pull/7685
271 2016-12-06 10:48:07	0|MarcoFalke|wumpus: Can you add instagibbs to our team?
272 2016-12-06 10:48:28	0|MarcoFalke|https://github.com/orgs/bitcoin/people
273 2016-12-06 10:48:55	0|timothy|MarcoFalke: core developers?
274 2016-12-06 10:49:21	0|MarcoFalke|Just as a member, so I can select pull requests by name :)
275 2016-12-06 10:50:15	0|MarcoFalke|I think it is some read-only group
276 2016-12-06 10:50:23	0|wumpus|sure
277 2016-12-06 10:53:45	0|jonasschnelli|wumpus: Yes. The fundrawtx exception is strange. Currently checking...
278 2016-12-06 10:54:31	0|wumpus|MarcoFalke: I've invited him to bitcoin and bitcoin-core (as read-only member), team page should still be updated
279 2016-12-06 10:56:08	0|MarcoFalke|ok, thx. Often I know who opened a pull request and to check it I use the dropdown on the pull request page.
280 2016-12-06 10:58:19	0|jonasschnelli|Interesting, can you call fundrawtx on a locked wallet?!
281 2016-12-06 10:58:55	0|wumpus|jonasschnelli: yes you should be able to, only signing requires unlocking
282 2016-12-06 10:59:25	0|wumpus|funding requires the wallet's utxo set which is not encrypted
283 2016-12-06 10:59:43	0|jonasschnelli|Not only, also retrieving a change output key
284 2016-12-06 10:59:51	0|jonasschnelli|Keypool can be empty, needs unlocking.
285 2016-12-06 11:00:29	0|wumpus|oh sure - if the keypool is empty it needs new keys, and for that the wallet needs to be unlocked
286 2016-12-06 11:01:17	0|jonasschnelli|But there is nothing that prevents that. Only an assertion
287 2016-12-06 11:01:24	0|wumpus|then can you either generate new keys and re-lock the wallet or call the RPC with wallet unlocked - there's more RPCs which work like that
288 2016-12-06 11:01:42	0|wumpus|it should just raise an exception in that case
289 2016-12-06 11:02:01	0|wumpus|which IIRC trying to get a new key with empty keypool (and locked wallet) does
290 2016-12-06 11:02:23	0|jonasschnelli|https://github.com/bitcoin/bitcoin/blob/master/src/wallet/wallet.cpp#L2392
291 2016-12-06 11:02:34	0|jonasschnelli|If no key is available, the app crashes. :)
292 2016-12-06 11:02:40	0|jonasschnelli|(over the assertion)
293 2016-12-06 11:03:05	0|jonasschnelli|And I can't find a code block that ensures we check if the wallet is unlocked first.
294 2016-12-06 11:09:12	0|bitcoin-git|[13bitcoin] 15MarcoFalke pushed 3 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/ed8d693c71b0...919db037f1f5
295 2016-12-06 11:09:13	0|bitcoin-git|13bitcoin/06master 14fa2ecc4 15MarcoFalke: [qa] pruning: Use cached utxo set to run faster
296 2016-12-06 11:09:13	0|bitcoin-git|13bitcoin/06master 14fab1af3 15MarcoFalke: [qa] maxuploadtarget: Use cached utxo set
297 2016-12-06 11:09:14	0|bitcoin-git|13bitcoin/06master 14919db03 15MarcoFalke: Merge #9274: [qa] Use cached utxo set to fix performance regression...
298 2016-12-06 11:09:26	0|bitcoin-git|[13bitcoin] 15MarcoFalke closed pull request #9274: [qa] Use cached utxo set to fix performance regression (06master...06Mf1612-qaPruningCacheUtxos) 02https://github.com/bitcoin/bitcoin/pull/9274
299 2016-12-06 11:37:49	0|wumpus|jonasschnelli: yeah that comment is definitely wrong
300 2016-12-06 11:38:01	0|wumpus|jonasschnelli: it should raise an exception, not crash the application
301 2016-12-06 11:48:57	0|jonasschnelli|wumpus: Yes. Will fix that soon,
302 2016-12-06 12:03:54	0|bitcoin-git|13bitcoin/06master 1489a3723 15Jonas Schnelli: [Qt] Show ModalOverlay by pressing the progress bar, disabled show() in sync mode
303 2016-12-06 12:03:54	0|bitcoin-git|[13bitcoin] 15jonasschnelli pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/919db037f1f5...e15660c16f75
304 2016-12-06 12:03:55	0|bitcoin-git|13bitcoin/06master 14e15660c 15Jonas Schnelli: Merge #9280: [Qt] Show ModalOverlay by pressing the progress bar, allow hiding...
305 2016-12-06 12:04:05	0|bitcoin-git|[13bitcoin] 15jonasschnelli closed pull request #9280: [Qt] Show ModalOverlay by pressing the progress bar, allow hiding (06master...062016/12/qt_modal) 02https://github.com/bitcoin/bitcoin/pull/9280
306 2016-12-06 12:51:32	0|bitcoin-git|[13bitcoin] 15jonasschnelli opened pull request #9295: [Wallet] Bugfix: Fundrawtransaction: don't terminate when keypool is empty (06master...062016/12/fix_frt) 02https://github.com/bitcoin/bitcoin/pull/9295
307 2016-12-06 12:51:51	0|MarcoFalke|jonasschnelli: Does this fix the segfault on master?
308 2016-12-06 12:52:12	0|jonasschnelli|Which Segfault?
309 2016-12-06 12:52:20	0|jonasschnelli|It should fix an assert terminiation
310 2016-12-06 12:52:27	0|jonasschnelli|not sure if this can lead to a segfault.
311 2016-12-06 12:53:01	0|MarcoFalke|oh, maybe unrelated.
312 2016-12-06 12:53:10	0|MarcoFalke|I can't send any coins: bitcoin-qt: ./primitives/transaction.h:331: void SerializeTransaction(const TxType&, Stream&) [with Stream = CDataStream; TxType = CTransaction]: Assertion `tx.wit.vtxinwit.size() <= tx.vin.size()' failed.
313 2016-12-06 12:53:55	0|jonasschnelli|Hmm... that one probably needs further investigation.
314 2016-12-06 13:00:17	0|paveljanik|MarcoFalke, I can confirm the same problem here.
315 2016-12-06 13:05:56	0|jonasschnelli|paveljanik MarcoFalke: do we have steps to reproduce?
316 2016-12-06 13:06:04	0|jonasschnelli|Or a failing rpc test snipped?
317 2016-12-06 13:06:18	0|MarcoFalke|Interestingly the rpc tests do not fail, it seems
318 2016-12-06 13:06:26	0|paveljanik|jonasschnelli, only in GUI.
319 2016-12-06 13:06:27	0|MarcoFalke|only the gui, really odd
320 2016-12-06 13:06:36	0|paveljanik|Just Send something somewhere.
321 2016-12-06 13:06:50	0|paveljanik|after waiting for Yes(2) ... Yes. Click on Yes, BUMP
322 2016-12-06 13:10:14	0|afk11|there are some others that wind up exposed through bitcoinconsensus.h.. anyone wish to advise if we should have those functions check if an impossible situation is about to arise before calling VerifyScript?
323 2016-12-06 13:10:18	0|afk11|or consumers of bitcoinconsensus.h?
324 2016-12-06 13:10:27	0|afk11|sorry, others = assertions
325 2016-12-06 13:11:36	0|jonasschnelli|paveljanik MarcoFalke: wait, it happens >always< on any tx in the GUI?
326 2016-12-06 13:12:20	0|paveljanik|both two my tries finished with SEIGSEGV
327 2016-12-06 13:13:09	0|jonasschnelli|I can send to myself.. but not directly on master (some commits behind)
328 2016-12-06 13:13:16	0|jonasschnelli|compiling now
329 2016-12-06 13:13:24	0|paveljanik|testnet e.g...
330 2016-12-06 13:14:06	0|paveljanik|it is #8580
331 2016-12-06 13:14:08	0|gribble|https://github.com/bitcoin/bitcoin/issues/8580 | Make CTransaction actually immutable by sipa · Pull Request #8580 · bitcoin/bitcoin · GitHub
332 2016-12-06 13:15:50	0|jonasschnelli|Ah. I'm regtesting..
333 2016-12-06 13:16:31	0|jonasschnelli|paveljanik, MarcoFalke: has someone opened an issue for that?
334 2016-12-06 13:17:14	0|paveljanik|not yet IIRC.
335 2016-12-06 13:17:29	0|paveljanik|I can't right now, about to leave..
336 2016-12-06 13:17:48	0|jonasschnelli|Okay. Sure.
337 2016-12-06 13:18:02	0|paveljanik|https://github.com/bitcoin/bitcoin/pull/8580#pullrequestreview-11582301
338 2016-12-06 13:18:08	0|paveljanik|but leaving now, sorry
339 2016-12-06 13:39:56	0|morcos|jonasschnelli: paveljanik: found the bug.  walletmodel.cpp:338
340 2016-12-06 13:40:08	0|morcos|i'm just going to do a quick scan for other occurences and then PR a fix
341 2016-12-06 14:02:21	0|jonasschnelli|morcos: great! Thanks.
342 2016-12-06 14:03:02	0|bitcoin-git|[13bitcoin] 15morcos opened pull request #9296: Fix missed change to WalletTx structure (06master...06fixwallettx) 02https://github.com/bitcoin/bitcoin/pull/9296
343 2016-12-06 14:04:15	0|morcos|wumpus: we should get a quick review of 9296 and merge that or whatever the correct fix is
344 2016-12-06 14:04:44	0|morcos|Since anyone using master will have transactions committed and then crash, it could inadverntantly lead to multiple payments or something
345 2016-12-06 14:05:12	0|jonasschnelli|MarcoFalke: Oops, I think i change the tags you made on https://github.com/bitcoin/bitcoin/pull/9295
346 2016-12-06 14:06:06	0|MarcoFalke|The milestone says what is the lowest version the patch should go to.
347 2016-12-06 14:06:27	0|jonasschnelli|morcos: we haven't really discussed #8501...
348 2016-12-06 14:06:29	0|gribble|https://github.com/bitcoin/bitcoin/issues/8501 | Add mempool statistics collector by jonasschnelli · Pull Request #8501 · bitcoin/bitcoin · GitHub
349 2016-12-06 14:06:30	0|morcos|I panicked a little because I thought it was potentially corrupting the wallet at first, but what is getting committed to the wallet is fine, it was just serialization for the GUI that was messed up..
350 2016-12-06 14:06:31	0|MarcoFalke|So "needs backport" and "0.14.0" does not make sense :)
351 2016-12-06 14:07:08	0|morcos|jonasschnelli: yes, i left a few comments on irc yesterday re: 8501.
352 2016-12-06 14:07:40	0|jonasschnelli|morcos: I think there can be a problem with storing to many timestamp... is that what you thought?
353 2016-12-06 14:08:09	0|morcos|it just seemed to me that the idea of storing 1000 seconds, 1000 minutes, 1000 hours and 1000 days (with no time stamps) is perfectly adequate for any stat, but maybe not?  (or 2000)
354 2016-12-06 14:08:53	0|jonasschnelli|Yes. But that would required fix frequency and active lock occupy. Not?
355 2016-12-06 14:09:00	0|morcos|i think the question would be what to do if some number of seconds go by before you record a new data point... i would argue that you want to differentiate between recording a repeated data point and not updating the stat
356 2016-12-06 14:09:11	0|morcos|jonasschnelli: not the way i'm imagining it
357 2016-12-06 14:09:44	0|morcos|lets say you had a way to indicate no value recorded.
358 2016-12-06 14:10:10	0|jonasschnelli|morcos: Yes. But I guess you still blow memory up to the size of the samples structure?...
359 2016-12-06 14:10:14	0|morcos|and look for a moment and the 1000 seconds data points... you have a circular buffer of 1000 entries
360 2016-12-06 14:10:37	0|morcos|and every time you call it, say you call it 17 seconds later.. you put in 16 NA's and then the latest data point
361 2016-12-06 14:10:46	0|morcos|so then graphing can be smart about interpolating
362 2016-12-06 14:10:48	0|jonasschnelli|hm...
363 2016-12-06 14:11:09	0|morcos|but if you call it every second with the same data point, it would put it in , you would distinguish between a step function and slow increase
364 2016-12-06 14:11:11	0|jonasschnelli|So you would not care if the 17second samples was done at 17.5s?
365 2016-12-06 14:11:15	0|morcos|yes, it would use constant memory
366 2016-12-06 14:11:22	0|morcos|but not that much
367 2016-12-06 14:11:26	0|morcos|a few MB
368 2016-12-06 14:12:20	0|jonasschnelli|I think the possible time offset could matter at the 1000 hours circular buffer.
369 2016-12-06 14:12:40	0|morcos|right, so you'd have to figure out how to cover that.. but i think thats doable..   suppose the data point for 12:14:16 means the latest data point that you have before 12:14:16.00000000
370 2016-12-06 14:12:51	0|jonasschnelli|Assume sample 1 was done at 0h, sample 2 and 0.6h (=1h sample), sample three at 1.4.
371 2016-12-06 14:13:16	0|jonasschnelli|What speaks against storing a sample with a delta timestamp (1byte) to the last sample?
372 2016-12-06 14:13:25	0|jonasschnelli|Could be something like a linked list with NA's
373 2016-12-06 14:13:41	0|jonasschnelli|*linked list
374 2016-12-06 14:13:42	0|morcos|then if you call it at 14.93 (that goes in the 12:14:15 data point) then you save just the latest call every time if you call it at 15.03, 15.37, 15.87, and then when you call it at 16.05, you put the 15.87 data point in for 12:14:16
375 2016-12-06 14:14:48	0|morcos|i think you could also do linked lists with delta time stamps, but i think the delta time stamps don't add much
376 2016-12-06 14:15:21	0|morcos|i guess i'm envisioning that for things with 1h samplign or 1day sampling you'll be calling it much more frequently anyway
377 2016-12-06 14:15:28	0|jonasschnelli|Maybe the advantage of delta timestamps is: more accurate (no interpolation in the recorded data), flexible memory cap-
378 2016-12-06 14:15:32	0|morcos|so you'll get a data point pretty close to the cut off
379 2016-12-06 14:16:19	0|jonasschnelli|sipa once had the idea to remove in-between samples once they are old and stats-memory gets drained.
380 2016-12-06 14:16:19	0|morcos|but are you envisioning that the part of the program thats making the call to record the sample is doing it separately for each timescale?
381 2016-12-06 14:17:01	0|morcos|well the design i'm trying to describe would never even keep "in-between" samples
382 2016-12-06 14:17:02	0|jonasschnelli|morcos: IMO the stats infrastructure should not actively record stuff. It should offer an interface to store records based
383 2016-12-06 14:17:20	0|morcos|yes, i understand
384 2016-12-06 14:17:24	0|jonasschnelli|"in-between" samples is probably the wrong term. Sorry.
385 2016-12-06 14:17:34	0|morcos|so in your mempool code, you call stats.record(SomeStats)
386 2016-12-06 14:17:39	0|jonasschnelli|Let's assume you record samples in a 1s frequency...
387 2016-12-06 14:17:48	0|jonasschnelli|then you reserved stats memory gets low
388 2016-12-06 14:17:49	0|morcos|but i'm saying that you can then record those however often you want from the mempool
389 2016-12-06 14:17:59	0|jonasschnelli|You could remove some of the old 1s samples...
390 2016-12-06 14:18:08	0|morcos|and the stats class will save the last 1000 seconds, 1000 mins, 1000 hrs and 1000 days of data poitns for you
391 2016-12-06 14:18:23	0|morcos|it removes the old 1s samples b/c its a circular buffer
392 2016-12-06 14:18:58	0|jonasschnelli|Yes. But maybe someone once to keep 1s stats longer.. in 8501 I introduces a -statsmaxmemory
393 2016-12-06 14:19:24	0|morcos|ok, well that was my argument, is that no one would care about keeping 1s stats for longer than say 1000 or 2000 secs
394 2016-12-06 14:19:32	0|morcos|if they wanted to look at a longer time period, they would look at minutes
395 2016-12-06 14:19:37	0|jonasschnelli|Thats a point. Right
396 2016-12-06 14:19:39	0|morcos|if that is false, then i agree
397 2016-12-06 14:20:14	0|jonasschnelli|My experience tells me, record as much as possible, :-)
398 2016-12-06 14:20:29	0|jonasschnelli|I would even say a stats to disk dumper would be great
399 2016-12-06 14:20:45	0|jonasschnelli|Especially stuff that can't be reconstructred
400 2016-12-06 14:20:50	0|jonasschnelli|(freerates, etc.)
401 2016-12-06 14:21:11	0|morcos|the concern i had about your design was that if we are recording lots of stats
402 2016-12-06 14:21:24	0|morcos|then we lose the ability to look over a long time period without using tons of memory
403 2016-12-06 14:21:32	0|morcos|i don't really care how we do it
404 2016-12-06 14:21:42	0|jonasschnelli|Yes. That is not done well in 8501
405 2016-12-06 14:21:54	0|morcos|but i think the stats class should automatically keep less fine-grained data for a longer time in order to keep the memory limited
406 2016-12-06 14:22:00	0|morcos|i was just presenting one way to do that
407 2016-12-06 14:22:06	0|jonasschnelli|The idea was to slowly stretch the frequency over time, depending on the available stats ram.
408 2016-12-06 14:22:24	0|jonasschnelli|Not just cut the back
409 2016-12-06 14:22:26	0|morcos|but don't you think you might want to look at both?
410 2016-12-06 14:22:36	0|jonasschnelli|Yes.
411 2016-12-06 14:22:42	0|morcos|high frequency recent and low frequency long time horizon
412 2016-12-06 14:22:49	0|morcos|ok...  so thats my goal...
413 2016-12-06 14:22:52	0|jonasschnelli|I think you convinced me to do the 1000s 1000m 1000h 1000d approach.
414 2016-12-06 14:23:02	0|jonasschnelli|maybe the 1000 is configurable.
415 2016-12-06 14:23:06	0|morcos|doesn't matter to me how we do it... i think a delta version coudl be just as good
416 2016-12-06 14:23:16	0|morcos|and you could just be smart about trimming the delta list or something
417 2016-12-06 14:23:31	0|morcos|yes, 1000 should be configurable i thik... actually maybe isn't enough for a default
418 2016-12-06 14:23:47	0|jonasschnelli|also, I liked the configurability of the buffer in MB.
419 2016-12-06 14:23:54	0|jonasschnelli|That's what you probably care about.
420 2016-12-06 14:23:59	0|jonasschnelli|Not the 1000
421 2016-12-06 14:24:02	0|morcos|1000 secs is just 16 minutes...  you would not want to have to only have 16 data points
422 2016-12-06 14:24:15	0|jonasschnelli|You would say, I reserve 300MB for stats.
423 2016-12-06 14:24:39	0|jonasschnelli|Right... just en
424 2016-12-06 14:24:47	0|jonasschnelli|just as an example
425 2016-12-06 14:25:17	0|jonasschnelli|So,.. you convinced me for high frequency recent and low frequency long time horizon,...
426 2016-12-06 14:25:23	0|morcos|ok cool...   any approach that automatically keeps both recent fine-grained and long time horizon bigger step, is fine with me
427 2016-12-06 14:25:28	0|jonasschnelli|And I think the delta timeoffset thing could work.
428 2016-12-06 14:25:28	0|morcos|thanks
429 2016-12-06 14:25:40	0|jonasschnelli|It can be tricky if you cut sample not at the back
430 2016-12-06 14:25:52	0|jonasschnelli|You need to re-adjust the delta.
431 2016-12-06 14:26:14	0|jonasschnelli|Losing a sample will corrupt the timestream
432 2016-12-06 14:26:43	0|morcos|i'll let you worry about that.  :)
433 2016-12-06 14:26:48	0|jonasschnelli|heh.
434 2016-12-06 14:27:00	0|jonasschnelli|Yes. I try to overhaul 8501
435 2016-12-06 14:27:19	0|jonasschnelli|But lets first fix Bitcoin
436 2016-12-06 14:42:08	0|bitcoin-git|13bitcoin/06master 1428f8ae8 15Alex Morcos: Fix missed change to WalletTx structure
437 2016-12-06 14:42:08	0|bitcoin-git|[13bitcoin] 15laanwj pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/e15660c16f75...fde7d99c4d7d
438 2016-12-06 14:42:09	0|bitcoin-git|13bitcoin/06master 14fde7d99 15Wladimir J. van der Laan: Merge #9296: Fix missed change to WalletTx structure...
439 2016-12-06 14:42:21	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #9296: Fix missed change to WalletTx structure (06master...06fixwallettx) 02https://github.com/bitcoin/bitcoin/pull/9296
440 2016-12-06 14:54:36	0|kanzure|reproducible builds for windows https://github.com/jasonwhite/ducible
441 2016-12-06 15:02:24	0|timothy|kanzure: you can still build using mingw32 on linux :P
442 2016-12-06 16:09:02	0|instagibbs|is there a written down explanation of the lock macros anywhere
443 2016-12-06 17:40:53	0|Chris_Stewart_5|sipa: re: yesterdays convo about syncing, seems like another guy had a similar experience that I did in #bitcoin, not sure if this is a concern or not
444 2016-12-06 17:50:17	0|sipa|Chris_Stewart_5: that sync is slow? of course it is a concern
445 2016-12-06 17:50:40	0|sipa|but we can't just go double our memory requirements
446 2016-12-06 17:54:03	0|gmaxwell|the speeds Chris_Stewart_5 were reporting sounded reasonably slow, but yes-- the processing is slowing down as the chain grows and will continue to do so.
447 2016-12-06 17:54:50	0|gmaxwell|We have been fighting that effect for years and will continue to do so, but nothing changes the fact that more state means more work.
448 2016-12-06 18:02:56	0|Chris_Stewart_5|gmaxwell: Yes, I was wondering if this was average or extremely slow. I thought I read something about it being on the order of magnitude of 12 hours -- everything you read online is accurate right? :P
449 2016-12-06 18:08:53	0|owowo|I synced today, was about 4k block behind, but it was quite fast.
450 2016-12-06 20:58:23	0|morcos|gmaxwell: can we duel about SelectCoinsMinConf(.., #ancestors, ..)?
451 2016-12-06 20:58:51	0|morcos|both sdaftuar and i think that the most important thing for your goal is to try low values for number of ancestors
452 2016-12-06 20:59:33	0|morcos|and lets for the moment just worry abou tthe devault limit of 25...  then you are suggesting 1,6,13,25
453 2016-12-06 21:01:13	0|morcos|sorry 0,6,13,25  (eh, 0 or 1 doesn't mean anything anyway though b/c you skip anything >= n and everything has at least 1 , it counts itself)
454 2016-12-06 21:02:30	0|morcos|so i think emphasizing more tries in the low limits helps keep any individual chain small and therefore helps avoid polluting small chains by combining them with big ones
455 2016-12-06 21:03:10	0|morcos|so trying 2,3,4,25  might even be better than 2,6,13,25
456 2016-12-06 21:03:27	0|morcos|anyway thats the tradeoff sdaftuar and i were trying to suggest, emphasize trying the low numbers
457 2016-12-06 21:04:27	0|gmaxwell|morcos: I agree with you, I didn't perform that bit of shed painting myself because I didn't think it mattered _that_ much.   The only reason I commented there was because instagibbs changed it in a way that was clearly wrong for some values. :)
458 2016-12-06 21:04:54	0|morcos|yeah but all it is is a bit inefficient for not default limits
459 2016-12-06 21:04:58	0|morcos|no one has non-default limits
460 2016-12-06 21:05:00	0|morcos|who cares
461 2016-12-06 21:05:03	0|morcos|its not broken
462 2016-12-06 21:05:25	0|gmaxwell|I don't care about inefficient, but when he changed the patch he harcoded one to 3 which could be higher than your limit.
463 2016-12-06 21:05:37	0|morcos|but that doesn't hurt anything
464 2016-12-06 21:06:46	0|gmaxwell|say you've set your limit to two, it will cause you to go past it.
465 2016-12-06 21:06:53	0|morcos|it just lets you potentially pick coins which will let you create a tx which will fail the final test..  thats already a possibility anyway..
466 2016-12-06 21:07:52	0|morcos|the actual test is way more restrictive, none of your ancestors can have more than the descendant limit
467 2016-12-06 21:08:01	0|gmaxwell|Personally, if I wrote it, I would have done 1,2,3,max-2,max-1,max or something like that.
468 2016-12-06 21:08:04	0|morcos|we're nto checking that in this quick heuristic anyway
469 2016-12-06 21:08:23	0|morcos|so you always might be picking coins with which you have no chance , you just don't know
470 2016-12-06 21:08:40	0|morcos|but better to have multiple tries with things which have a good heuristic
471 2016-12-06 21:08:46	0|gmaxwell|you might be, but that isn't the pratical case people are encountering.
472 2016-12-06 21:09:04	0|gmaxwell|I don't know where we disagree.
473 2016-12-06 21:09:34	0|morcos|mostly i think its fine as is, and your suggestion is a slight downgrade..  but i agree its mostly bikeshedding i suppose
474 2016-12-06 21:10:52	0|gmaxwell|hm. if the limit is set to 2.. why should it check 3?  would you agree that min(3,limit) wouldn't be better?
475 2016-12-06 21:11:30	0|morcos|sure
476 2016-12-06 21:11:53	0|gmaxwell|I think it absolutely should spend more tries on low numbers, because say it runs with limit/4,  then it combines two inputs a depth 2 and a depth limit/2 .. great, now you've just trashed a short chain.
477 2016-12-06 21:12:04	0|morcos|exactly
478 2016-12-06 21:13:03	0|gmaxwell|but for that e.g. spacing my a multiplicave factor at least won't screw you by more than that factor.
479 2016-12-06 21:13:09	0|morcos|i'm fine adding a min, i just think making it work well for a limit of 25 is fine, as long as its not horribly broken for other limits, which it wont' be b/c it'll still try 2 first
480 2016-12-06 21:13:36	0|gmaxwell|okay we agree then.
481 2016-12-06 21:14:08	0|morcos|2, min(4,limit), something in between, limit
482 2016-12-06 21:14:24	0|morcos|:)
483 2016-12-06 21:18:02	0|gmaxwell|I posted  min(2,limit/4), min(4, limit/3), limit/2
484 2016-12-06 21:18:07	0|gmaxwell|, limit
485 2016-12-06 21:18:20	0|instagibbs|is it safe to come out of my bikeshelter :)
486 2016-12-06 21:18:28	0|sipa|cfields: i've been playing with -flto lately, and you can use -flto=jobserver at link time to make gcc itself perform the linking step in multiple processes... the problem is that this requires prefixing the compiler invocation in make with a '+' so make knows to expose its jobserver interface to it
487 2016-12-06 21:18:47	0|gmaxwell|instagibbs: haha
488 2016-12-06 21:18:49	0|sipa|cfields: any idea how to accomplish that in automake. it doesn't seem that CXX="+g++" works
489 2016-12-06 21:19:49	0|gmaxwell|morcos: I really try to not bikeshed this stuff too much, because it's just a hop-skip-and-jump away from arguing that we should be linking GLPK.
490 2016-12-06 21:20:09	0|morcos|heh
491 2016-12-06 21:20:38	0|morcos|well sdaftuar and i played around with testing the PR, and it kind of performed poorly
492 2016-12-06 21:20:49	0|morcos|if you started with two inputs
493 2016-12-06 21:21:02	0|morcos|instead of getting 50 txs, you only got 30 (whereas before you got 25)
494 2016-12-06 21:21:53	0|morcos|this was because we were sending to ourselves so the outputs were being combined quickly and we were trashing chains, so led us to prefer this small change approach.. obviously as sdaftuar reported, it works much better when the test doesn't pay yourself
495 2016-12-06 21:22:24	0|morcos|s/small change approach/emphasis on small limits approach/
496 2016-12-06 21:23:28	0|gmaxwell|I'm glad you tested it-- I anticipated that result but I was trying to resist bikesheding it and making it not go into 0.13.2.
497 2016-12-06 21:24:33	0|morcos|i thik with a large set of coins though , it'll really shine as it'll stop you from just being stupid
498 2016-12-06 21:25:13	0|morcos|btw, it would be helpful for me if more people went through and tagged things for 0.14, i could review some more things, and would like to know where to concentrate...
499 2016-12-06 21:25:25	0|morcos|i think i've done what i can on the existing milestones
500 2016-12-06 21:26:23	0|instagibbs|ok, putting final touches on the selection, thanks for all the feedback
501 2016-12-06 21:26:35	0|instagibbs|(also am I the only one getting spammed to hell by phising PMs?)
502 2016-12-06 21:27:42	0|gmaxwell|instagibbs: yes.. you can /mode instagibbs +R  to turn off pms from unregistered users.
503 2016-12-06 21:28:01	0|instagibbs|ah thx
504 2016-12-06 21:28:18	0|gmaxwell|remember to turn it back off again later, if you want j-random-people to pm you later.
505 2016-12-06 21:28:53	0|sipa|what about j-invariant-people?
506 2016-12-06 21:30:14	0|gmaxwell|sipa: thats complex.
507 2016-12-06 21:33:24	0|bitcoin-git|13bitcoin/06master 149b9324e 15Matt Corallo: Fix rounding privacy leak introduced in #9260
508 2016-12-06 21:33:24	0|bitcoin-git|[13bitcoin] 15sipa pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/fde7d99c4d7d...09c4fd157c5b
509 2016-12-06 21:33:25	0|bitcoin-git|13bitcoin/06master 1409c4fd1 15Pieter Wuille: Merge #9268: Fix rounding privacy leak introduced in #9260...
510 2016-12-06 21:33:40	0|bitcoin-git|[13bitcoin] 15sipa closed pull request #9268: Fix rounding privacy leak introduced in #9260 (06master...062016-12-feefilterrounder) 02https://github.com/bitcoin/bitcoin/pull/9268
511 2016-12-06 23:26:10	0|bitcoin-git|[13bitcoin] 15Mirobit opened pull request #9297: Various RPC help outputs updated (06master...06rpchelp12/16) 02https://github.com/bitcoin/bitcoin/pull/9297