1 2018-02-01 00:05:40	0|GitHub131|[13bitcoin-detached-sigs] 15jonasschnelli opened pull request #1: 0.16: osx signatures for 0.16.0rc1 (060.16...060.16) 02https://github.com/bitcoin-core/bitcoin-detached-sigs/pull/1
  2 2018-02-01 01:16:45	0|GitHub8|[13bitcoin-detached-sigs] 15theuni closed pull request #1: 0.16: osx signatures for 0.16.0rc1 (060.16...060.16) 02https://github.com/bitcoin-core/bitcoin-detached-sigs/pull/1
  3 2018-02-01 01:17:15	0|bitcoin-git|[13bitcoin] 15fivepiece opened pull request #12321: p2wsh address in decodescript (06master...06decodescript-p2wsh) 02https://github.com/bitcoin/bitcoin/pull/12321
  4 2018-02-01 01:17:33	0|meshcollider|mryandao: see #12254
  5 2018-02-01 01:17:35	0|gribble|https://github.com/bitcoin/bitcoin/issues/12254 | BIP 158: Compact Block Filters for Light Clients by jimpo · Pull Request #12254 · bitcoin/bitcoin · GitHub
  6 2018-02-01 01:18:09	0|meshcollider|Oh oops, I'm too late :)
  7 2018-02-01 01:27:13	0|achow101|cfields: I saw that the detached sigs were pushed, but they don't seem to be working rn
  8 2018-02-01 01:27:29	0|cfields|achow101: not quite yet, still working on it
  9 2018-02-01 01:27:51	0|achow101|oh, ok
 10 2018-02-01 01:27:55	0|cfields|this is the first release with the new key, and the first that's not just me signing. So it's a bit kludgy
 11 2018-02-01 01:28:25	0|achow101|we haven't done the mpc rsa thing yet, have we?
 12 2018-02-01 01:29:18	0|cfields|no, didn't make it in time
 13 2018-02-01 01:29:23	0|cfields|ok, pushed the tag. should work now
 14 2018-02-01 01:29:33	0|cfields|off to find food before everything closes, bbl
 15 2018-02-01 01:29:57	0|cfields|gitian builders: v0.16.0rc1 detached sigs are pushed. Please ping me if there are any issues
 16 2018-02-01 04:45:23	0|dx25|with 0.16.0rc1 on exit i'm seeing "IO Error: ...chainstate/244997.ldb: Bad file descriptor", system error while flushing: Database I/O error.
 17 2018-02-01 04:45:36	0|dx25|Does not seem to be happening with v0.15.2 or older version
 18 2018-02-01 07:06:37	0|gmaxwell|dx25: can you tell us some about your system? What OS, etc.
 19 2018-02-01 07:07:00	0|gmaxwell|dx25: we believe thats a new issue, which will likely block the release until we fix it.
 20 2018-02-01 07:07:42	0|gmaxwell|dx25: we've had one developer hit it but don't have a reliable reproduction yet, and maybe there is something in common between systems that have hit it which would help track it down
 21 2018-02-01 07:07:55	0|gmaxwell|e.g. what OS, kernel version, etc.
 22 2018-02-01 09:56:24	0|bitcoin-git|[13bitcoin] 15murrayn opened pull request #12322: Docs: Remove step making cloned repository world-writable for Windows build. (06master...06doc_change) 02https://github.com/bitcoin/bitcoin/pull/12322
 23 2018-02-01 10:53:50	0|dafuq|a PR change to the --help CLI options would fall under what category?
 24 2018-02-01 10:58:18	0|wumpus|docs
 25 2018-02-01 11:04:40	0|dafuq|ok, seems most suitable but does involve code changes
 26 2018-02-01 11:05:11	0|wumpus|yes, that doesn't matter, as long as it is message changes
 27 2018-02-01 11:05:36	0|dafuq|thanks
 28 2018-02-01 11:27:40	0|wumpus|gmaxwell:  dx25: I've created an issue for the problem, please add more information if you have there https://github.com/bitcoin/bitcoin/issues/12323
 29 2018-02-01 11:52:12	0|bitcoin-git|[13bitcoin] 15AkioNak opened pull request #12324: speed up Unserialize_impl for prevector (06master...06unserialize) 02https://github.com/bitcoin/bitcoin/pull/12324
 30 2018-02-01 12:49:46	0|provoostenator|I'm getting "Fatal Internal Error" during IBD on testnet3 with 0.16.0rc1, twice. Will investigate.
 31 2018-02-01 12:50:10	0|bitcoin-git|[13bitcoin] 15kekimusmaximus opened pull request #12325: Use dynamic_cast for downcasting instead of static_cast. (06master...06use_dynamic_cast_to_downcast) 02https://github.com/bitcoin/bitcoin/pull/12325
 32 2018-02-01 12:50:32	0|provoostenator|Might be a disk permission thing. "Pre-allocating up to position 0x3000000 in blk00002.dat" "ERROR: WriteBlockToDisk: ftell failed" "*** Failed to write block"
 33 2018-02-01 12:53:20	0|wumpus|disk full?
 34 2018-02-01 12:53:21	0|provoostenator|Also seeing a bunch of "socket recv error Bad file descriptor (9)" messages, not sure what that's about.
 35 2018-02-01 12:53:36	0|wumpus|oof bad file descriptor
 36 2018-02-01 12:53:39	0|provoostenator|No, plenty of space. It's an external SSD though, so can't rule out a hardware problem.
 37 2018-02-01 12:53:45	0|wumpus|provoostenator: see  https://github.com/bitcoin/bitcoin/issues/12323
 38 2018-02-01 12:54:03	0|dx25|mine's also an external drive, not ssd tho
 39 2018-02-01 12:54:11	0|wumpus|no, it's possibly a regression in 0.16
 40 2018-02-01 12:55:11	0|provoostenator|Getting this crash every other minute now on testnet. Will try local drive just to rule out hardware issue.
 41 2018-02-01 12:55:21	0|dx25|i'm on qubes, fedora25 vm, 4.9.56 kernel
 42 2018-02-01 13:00:20	0|wumpus|I really wonder why the bad file descriptor thing happens, normally that happens if either a fd is used that was not returned by open, or a file descriptor that was already closed
 43 2018-02-01 13:01:25	0|provoostenator|Same "bad file descriptor" on a fresh testnet3 IBD (MacOS) on the built in harddisk which has plenty of space. Waiting for it to crash.
 44 2018-02-01 13:03:25	0|wumpus|I seriously doubt this is a hardware issue
 45 2018-02-01 13:03:39	0|wumpus|are you doing anything special with the node? regular RPC requests, for example?
 46 2018-02-01 13:04:10	0|wumpus|or any fd related settings in bitcoin.conf?
 47 2018-02-01 13:04:57	0|provoostenator|I was running QT production and testnet at the same time for different users on my system, bother in seperate directories. I also suspended the computer (though it should keep syncing in that case). So trying to reproduce with fewer variables now.
 48 2018-02-01 13:05:50	0|provoostenator|server=1  rpcuser=... rpcpassword=... listen=0
 49 2018-02-01 13:06:20	0|provoostenator|I used a symlink to point to the SSD drive.
 50 2018-02-01 13:06:27	0|dx25|i have some weird walletnotify and alertnotify thing calling curl for some reason i can't remember.  haven't tried turning that off yet.
 51 2018-02-01 13:06:30	0|provoostenator|Ah there we go again: crash.
 52 2018-02-01 13:07:32	0|provoostenator|(no symlink involved this time, nor an external drive). Crashed happened at or after block 64709 (testnet)
 53 2018-02-01 13:07:37	0|dx25|i was doing no rpc stuff
 54 2018-02-01 13:07:37	0|wumpus|provoostenator: so it happens even without listening
 55 2018-02-01 13:08:10	0|wumpus|provoostenator: that rules out some p2p races I guess, but what can it be then...
 56 2018-02-01 13:08:32	0|wumpus|provoostenator: you're able to reliably reproduce this? could try git bisecting, if it was ok with 0.15.1
 57 2018-02-01 13:08:51	0|provoostenator|Meanwhile I had production QT running all night without a problem, so I suspect it's sync related (my testnet node was doing an IBD the first time it crashed)
 58 2018-02-01 13:08:59	0|wumpus|(or find some later commit where the issue doesn't exist)
 59 2018-02-01 13:09:31	0|provoostenator|I just down the other mainnet QT instance, will try once more. Then I'll compare versions after that.
 60 2018-02-01 13:09:34	0|dx25|also sync related here, but on mainnet
 61 2018-02-01 13:09:37	0|provoostenator|Pretty reliable so far
 62 2018-02-01 13:16:40	0|provoostenator|It was built with: ./configure --disable-tests --disable-bench --with-miniupnpc=no
 63 2018-02-01 13:17:09	0|provoostenator|Hooray, another crash. Ok, should be easy to bisect.
 64 2018-02-01 13:17:38	0|provoostenator|Height 8089 (testnet)
 65 2018-02-01 13:19:14	0|wumpus|yes, would make a backup  of the data directory to make sure you start with the same state every time
 66 2018-02-01 13:19:25	0|wumpus|at hight 8089 at least that isn't too much data
 67 2018-02-01 13:20:08	0|provoostenator|I get this crash with a fresh testnet3 directory.
 68 2018-02-01 13:20:29	0|provoostenator|I'll keep a copy for forensics
 69 2018-02-01 13:22:59	0|wumpus|I'll hold up on uploading executables for rc1 for now
 70 2018-02-01 13:26:55	0|provoostenator|Mmm, I just found a zombie lightningd instance in the background. Maybe it was making RPC requests, not sure.  I'm going to kill it just in case.
 71 2018-02-01 13:29:57	0|provoostenator|(no difference)\
 72 2018-02-01 13:30:12	0|provoostenator|Height 401 :-)
 73 2018-02-01 13:30:18	0|wumpus|I wouldn't expect it to be so predictable in that case
 74 2018-02-01 13:47:07	0|provoostenator|Other than ccache and skipping tests and bench, any hints on how to make it compile faster?
 75 2018-02-01 13:48:16	0|wumpus|do only 'make -j<x> src/bitcoind'
 76 2018-02-01 13:48:26	0|wumpus|you don't really need to rebuild cli and such
 77 2018-02-01 13:56:08	0|provoostenator|Right, I should test if this is QT related first, and otherwise just build bitcoind
 78 2018-02-01 14:02:39	0|wumpus|yes, you could also only built -qt with a similar command, but it takes much longer
 79 2018-02-01 14:04:12	0|provoostenator|I know, I was doing that (make src/qt/bitcoin-qt)
 80 2018-02-01 14:04:32	0|provoostenator|Is there a reason the binaries end up in the /src path rather than in e.g. /dist?
 81 2018-02-01 14:09:55	0|wumpus|that's common for automake build systems, if you want to build somewhere else you can do an out of tree build, if you want to put the binaries somewhere set a prefix and do `make install`<
 82 2018-02-01 14:15:15	0|morcos|just to clarify we think the crash is caused by what?  a Bad file descriptor issue occuring on block write?
 83 2018-02-01 14:15:28	0|morcos|We've also seen it on ldb which causes a crash
 84 2018-02-01 14:15:37	0|morcos|and on socket send/recv which doesn't
 85 2018-02-01 14:16:30	0|provoostenator|Mmm, bitcoind doesn't show "Bad file descriptor" messages (with -debug=1). It does exit with " A fatal internal error occurred, see debug.log for details"
 86 2018-02-01 14:19:17	0|provoostenator|But http://termbin.com/83iva
 87 2018-02-01 14:19:40	0|provoostenator|I wonder what "Interrupting HTTP server" is about.
 88 2018-02-01 14:21:41	0|wumpus|there is no error there - you didn't simply send a stop command?
 89 2018-02-01 14:22:17	0|provoostenator|I didn't. And a stop command wouldn't explain bitcoind exiting with "Error: Error: A fatal internal error occurred, see debug.log for details"
 90 2018-02-01 14:22:34	0|provoostenator|I'm blaming gremlins. Bisecting now.
 91 2018-02-01 14:23:17	0|wumpus|no, that's true, normally that's accompanied by an error being logged, this looks like a succesful shutdown
 92 2018-02-01 14:23:37	0|wumpus|you did paste the right debug.log? :)
 93 2018-02-01 14:26:18	0|provoostenator|Not sure actually, double checking
 94 2018-02-01 14:28:25	0|provoostenator|Default og locations changed between 0.15 and 0.16. Will try again.
 95 2018-02-01 14:30:07	0|wumpus|default log location changed?!
 96 2018-02-01 14:30:47	0|wumpus|should not be the case, it's still <datadir>/debug.log, sounds more likely you've set a different datadir for -qt
 97 2018-02-01 14:32:29	0|provoostenator|Or accidentally used mainnet for bitcoind.
 98 2018-02-01 14:37:00	0|provoostenator|Ok, now I'm seeing the "Bad file descriptor" messages again. Will wait for crash and upload correct debug log. Then continue with bisect.
 99 2018-02-01 14:37:12	0|BlueMatt|provoostenator: if you're trying to bisect, I'd recommend focusing on any changes to net
100 2018-02-01 14:37:39	0|BlueMatt|eg #11363
101 2018-02-01 14:37:41	0|gribble|https://github.com/bitcoin/bitcoin/issues/11363 | net: Split socket create/connect by theuni · Pull Request #11363 · bitcoin/bitcoin · GitHub
102 2018-02-01 14:37:48	0|BlueMatt|and #10663
103 2018-02-01 14:37:50	0|gribble|https://github.com/bitcoin/bitcoin/issues/10663 | net: split resolve out of connect by theuni · Pull Request #10663 · bitcoin/bitcoin · GitHub
104 2018-02-01 14:38:04	0|BlueMatt|though I dont see anything obvious in net on master that should be causing this
105 2018-02-01 14:38:55	0|provoostenator|BlueMatt: bisecting everything involves less thinking :-)
106 2018-02-01 14:39:55	0|provoostenator|Although I'm not sure yet how long without a crash is long enough. So far it crashes within a few minutes though if it does.
107 2018-02-01 14:40:49	0|provoostenator|Maybe "Bad file descriptor" is a good enough proxy.
108 2018-02-01 14:46:07	0|morcos|cfields asked me to add some asserts for debugging this
109 2018-02-01 14:46:11	0|morcos|https://0bin.net/paste/X-6vyuCUUlN+XY1l#5VoMIrtAP6S0Yex-7eAwDNLJl/7UT2e7medmcgsVtvA
110 2018-02-01 14:46:15	0|gribble|https://github.com/bitcoin/bitcoin/issues/5 | Make the version number the protocol version and not the client version · Issue #5 · bitcoin/bitcoin · GitHub
111 2018-02-01 14:46:45	0|morcos|I just tried a fresh sync of testnet3 with these asserts patched on 0.16.0rc1 and the middle assert triggered
112 2018-02-01 14:47:28	0|morcos|last line in debug log
113 2018-02-01 14:47:30	0|morcos|2018-02-01 14:40:00.528984 socket select error Bad file descriptor (9)
114 2018-02-01 14:49:05	0|cfields|BlueMatt: that seems like a really good candidate, yes. I went through it earlier this week already, but maybe i keep missing something
115 2018-02-01 14:49:42	0|morcos|here is the backtrace from that thread:
116 2018-02-01 14:49:46	0|morcos|https://0bin.net/paste/OmSDtVV-u0EwBaEp#Muh7dQfn6+kzip0Ib-b5brUozDZtCZblT1ev2lNtfdT
117 2018-02-01 14:49:52	0|wumpus|also going to test with those asserts
118 2018-02-01 14:50:22	0|cfields|morcos: can you do a 'thread apply all bt' ?
119 2018-02-01 14:50:44	0|cfields|if it's a leak in something like 11363, it won't show up in a bt though :(
120 2018-02-01 14:51:29	0|cfields|though also, a leak should be obvious after a quick peek at /proc
121 2018-02-01 14:51:56	0|wumpus|would be nice if it was able to rewind the process to see what closed fd 9
122 2018-02-01 14:52:05	0|morcos|https://0bin.net/paste/FotU51A-Z98LdS0u#aWgxDyxhAn5L4XIex7Ko+kjMJ+qkAL0CJ77NubzV6io
123 2018-02-01 14:52:08	0|morcos|cfields: ^^
124 2018-02-01 14:52:31	0|cfields|thanks
125 2018-02-01 14:52:42	0|cfields|wumpus: i think 9 is EBADFD, not the fd#
126 2018-02-01 14:52:54	0|morcos|yeah i hope so, b/c it's always 9
127 2018-02-01 14:53:05	0|wumpus|cfields: oh, right it doesn't print the actual fd number
128 2018-02-01 14:53:55	0|provoostenator|http://termbin.com/h7jy
129 2018-02-01 14:54:02	0|provoostenator|"System error: CAutoFile::write: write failed: unspecified iostream_category error"
130 2018-02-01 14:54:26	0|provoostenator|It then happily processes a few more blocks and shuts down
131 2018-02-01 14:54:40	0|wumpus|so the buckshot hit CAutoFile's descriptor this time
132 2018-02-01 14:55:06	0|wumpus|this is so weird, looks like some evil background thread is randomly closing fds
133 2018-02-01 14:55:35	0|provoostenator|Unfortunately that took almost 20 minutes to crash, so this bisect will take a while, but probably worth it.
134 2018-02-01 14:55:57	0|cfields|maybe some callback isn't taking cs_main while touching block files?
135 2018-02-01 14:56:11	0|cfields|provoostenator: it'd be great if you could catch it in gdb
136 2018-02-01 14:56:26	0|cfields|that should allow a 'freeze' long enough to ld your fd's
137 2018-02-01 14:56:31	0|cfields|*ls
138 2018-02-01 14:56:41	0|provoostenator|gdb?
139 2018-02-01 14:56:55	0|cfields|provoostenator: debugger
140 2018-02-01 14:56:58	0|wumpus|cfields: right, it could well be something else than net calls, I remember there is a PR that changes locking around block files
141 2018-02-01 14:57:36	0|provoostenator|Since morcos is able to reproduce, maybe it's easier if he looks at the debugger, while I just try to pinpoint which commit caused this.
142 2018-02-01 14:58:01	0|cfields|sure
143 2018-02-01 14:58:06	0|provoostenator|That should also provide more assurance that the fix actually fixes it.
144 2018-02-01 14:58:13	0|morcos|cfields: now i hit the first assert
145 2018-02-01 14:58:45	0|cfields|morcos: if it's some fd leak, it'd make sense that you'd get EBADFD randomly, all over the place
146 2018-02-01 14:58:56	0|cfields|morcos: any chance you can catch it in gdb?
147 2018-02-01 14:59:02	0|morcos|yeah, ok so if i run in gdb, then you want what, the list of what's in /proc/pid, or what
148 2018-02-01 14:59:11	0|cfields|yea
149 2018-02-01 14:59:43	0|cfields|try to break on assert (might be _assert or __assert) in gdb, so it hangs before exit is called
150 2018-02-01 15:03:49	0|wumpus|cfields: maybe #11281 / ccd8ef65f93ed82a87cee634660bed3ac17d9eb5?
151 2018-02-01 15:03:55	0|gribble|https://github.com/bitcoin/bitcoin/issues/11281 | Avoid permanent cs_main/cs_wallet lock during RescanFromTime by jonasschnelli · Pull Request #11281 · bitcoin/bitcoin · GitHub
152 2018-02-01 15:04:06	0|morcos|it's on to me, it's not going to crash now
153 2018-02-01 15:04:31	0|wumpus|cfields: it changes locking around block file reading, at least
154 2018-02-01 15:04:32	0|cfields|morcos: heh, gdb tends to throw timings just enough so that everything works perfectly
155 2018-02-01 15:05:04	0|cfields|wumpus: good find, taking a look
156 2018-02-01 15:06:15	0|cfields|morcos: wait, you said you hit the first one? that should be much more interesting
157 2018-02-01 15:06:25	0|cfields|i guess th core file is blown away already? :(
158 2018-02-01 15:06:45	0|wumpus|cfields: the one I was thinking of https://github.com/bitcoin/bitcoin/pull/11913 is not merged :)
159 2018-02-01 15:07:31	0|cfields|wumpus: ah, heh
160 2018-02-01 15:07:36	0|wumpus|no problems with testnet sync here w/ cfields's assertions
161 2018-02-01 15:07:52	0|morcos|cfield: no i saved it
162 2018-02-01 15:07:53	0|wumpus|(at least at block 319000)
163 2018-02-01 15:08:17	0|cfields|morcos: could you do a 'thread apply all bt' for that one?
164 2018-02-01 15:08:57	0|morcos|how do i easily output that to a file
165 2018-02-01 15:09:05	0|cfields|the send is more interesting because it may be an optimistic send. in that case, it'd be coming from the message handler thread rather than the sockethandler, so it might show a little more
166 2018-02-01 15:09:12	0|wumpus|gdb logging
167 2018-02-01 15:09:27	0|wumpus|morcos: https://sourceware.org/gdb/onlinedocs/gdb/Logging-Output.html
168 2018-02-01 15:15:05	0|morcos|some serious user error there
169 2018-02-01 15:15:09	0|morcos|https://0bin.net/paste/43cNNOSrrPlIyVqt#BE1tyObep4Y9aLIpDDVC4kwVFJYxctxlhMJTE3xF08j
170 2018-02-01 15:15:57	0|cfields|thanks
171 2018-02-01 15:15:58	0|wumpus|"Hello. CloseSocket may be called with hSocket uninitialised, at net.cpp:448 (not confirmed to be the cause of this bug, but it seems likely)"
172 2018-02-01 15:16:08	0|wumpus|(https://github.com/bitcoin/bitcoin/issues/12323#issuecomment-362296158)
173 2018-02-01 15:17:58	0|cfields|I don't think it can be called unitialized?
174 2018-02-01 15:18:25	0|wumpus|closing an uninitialized fd would explain this perfectly, though
175 2018-02-01 15:18:55	0|wumpus|if you can reproduce this, maybe log all CloseSocket calls and see if it ends up with any funny data
176 2018-02-01 15:18:59	0|cfields|yes it would
177 2018-02-01 15:19:24	0|wumpus|(or at least those for which the close() fails, because we ignore closes return value right now)
178 2018-02-01 15:19:37	0|cfields|and it matches provoostenator's log as well
179 2018-02-01 15:19:40	0|wumpus|yep
180 2018-02-01 15:20:50	0|cfields|oh wait, that's an else if(), not an else
181 2018-02-01 15:20:56	0|cfields|maybe that can happen
182 2018-02-01 15:21:15	0|morcos|yeah seems like you are missing a catch all else
183 2018-02-01 15:21:55	0|cfields|looks like it'd happen as a result of a dns seed handing out a bad/local ip
184 2018-02-01 15:22:03	0|morcos|nice find by david60
185 2018-02-01 15:22:25	0|cfields|morcos: do you see dns queries before crashes in your logs?
186 2018-02-01 15:22:26	0|morcos|cfields: that would match my having it happen a lot when i noticed i was using dns seeds a lot
187 2018-02-01 15:22:49	0|morcos|i did previously, i don't know if i exclusively checked , but i had observed it was doing a lot of dns querying
188 2018-02-01 15:23:05	0|cfields|I'll pr a fix for that right now either way. Seems like a really good candidate, though
189 2018-02-01 15:23:19	0|cfields|ok
190 2018-02-01 15:23:44	0|cfields|maybe you could force it by setting up a phony seed and returning only 127.0.0.1
191 2018-02-01 15:24:17	0|cfields|i think 2 phony entries in /etc/hosts would work
192 2018-02-01 15:25:53	0|morcos|2?  any 2?  one ip4 and one ip6?
193 2018-02-01 15:25:57	0|provoostenator|That would be nice, in that case you can even write a test for it?
194 2018-02-01 15:27:36	0|cfields|morcos: 127.0.0.1 seed.bitcoin.sipa.be
195 2018-02-01 15:27:46	0|cfields|i think that should do it?
196 2018-02-01 15:28:21	0|morcos|i think i tried that, let me see what happened, it didn't crash yet
197 2018-02-01 15:29:26	0|provoostenator|I've never had a crash before or during headers sync, always during block downloads.
198 2018-02-01 15:31:47	0|cfields|morcos: ok, 127.0.0.1 would still pass IsValid, we'll have to use a more busted value
199 2018-02-01 15:33:23	0|morcos|i got it to crash again not sure if it was because of those entries or what
200 2018-02-01 15:33:26	0|morcos|in gdb
201 2018-02-01 15:34:12	0|bitcoin-git|[13bitcoin] 15theuni opened pull request #12326: net: initialize socket to avoid closing random fd's (06master...06fix-socket-init) 02https://github.com/bitcoin/bitcoin/pull/12326
202 2018-02-01 15:35:01	0|cfields|morcos: dns query at the end of your log?
203 2018-02-01 15:35:08	0|morcos|https://0bin.net/paste/3T-vAUCXmHtlqJ9X#1UvKDgv7oZR4uZEEV0WqIGFDNdUQFTW3xFHkE0R-UOe
204 2018-02-01 15:35:10	0|gribble|https://github.com/bitcoin/bitcoin/issues/1 | JSON-RPC support for mobile devices ("ultra-lightweight" clients) · Issue #1 · bitcoin/bitcoin · GitHub
205 2018-02-01 15:35:45	0|provoostenator|gribble: lol
206 2018-02-01 15:35:59	0|cfields|morcos: ok, so you're not running through fd's. Closing a random one makes way more sense
207 2018-02-01 15:35:59	0|morcos|oh that was stupid, i changed the mainnet dns seeds but was running on testnet
208 2018-02-01 15:36:18	0|cfields|heh
209 2018-02-01 15:37:18	0|cfields|morcos: try setting it to 0.0.0.0 instead. Not sure if the resolver will actually hand that out, though
210 2018-02-01 15:37:47	0|cfields|actually, if that's the case, I should be able to repro too instead of asking you to :p
211 2018-02-01 15:37:55	0|cfields|off to test
212 2018-02-01 15:38:17	0|morcos|cfields: ok yeah it did load dns seeds before this crash though
213 2018-02-01 15:38:26	0|wumpus|well it also depends on what is in the uninitialized memory
214 2018-02-01 15:39:03	0|morcos|btw, before i forget, it seemed that running in testnet was reading peers.dat from .bitcoin and not testnet3
215 2018-02-01 15:39:10	0|wumpus|if there happen to be zeroes there, or some value that is larger than max fd, it will go unniticed, it still doesn't have to trigger every time
216 2018-02-01 15:39:11	0|cfields|wumpus: true, but an assert on a successfull close() should point it out quickly i should think
217 2018-02-01 15:39:11	0|morcos|i deleted both of them to force dnsseeds
218 2018-02-01 15:39:48	0|cfields|wumpus: wouldn't anything other than -1 in memory cause a problem?
219 2018-02-01 15:40:26	0|wumpus|cfields: it might, though closing fd 0 (stdin) is harmless in our case
220 2018-02-01 15:40:54	0|cfields|hmm
221 2018-02-01 15:41:53	0|wumpus|at least the first time. Once you close stdin, the next time you use open() you might get that fd, and if it then randomly gets closed again, it will still interfere. So, yeah. THe only harmless values would be very large ones that can't be a fd, ever.
222 2018-02-01 15:43:52	0|cfields|makes sense
223 2018-02-01 15:43:56	0|cfields|morcos: fyi, there's -forcednsseed
224 2018-02-01 15:47:11	0|morcos|cfields: I added an assert in netbase.cpp CloseSocket that ret != error and I hit it
225 2018-02-01 15:47:37	0|cfields|morcos: great! can you print hSocket there ?
226 2018-02-01 15:49:29	0|morcos|0?
227 2018-02-01 15:49:34	0|morcos|looks like addrConnect is 0
228 2018-02-01 15:49:46	0|morcos|looks like this is all a test from provoostenator as its coming from his seed
229 2018-02-01 15:50:12	0|cfields|morcos: you didn't set yours in /etc/hosts ?
230 2018-02-01 15:50:21	0|provoostenator|Interesteing, is my testnet seed doing something funny?
231 2018-02-01 15:50:36	0|morcos|actually i'm not sure abou tthat, since i'm not familiar with this code
232 2018-02-01 15:50:50	0|morcos|0x00005555555ed33a in CConnman::ConnectNode (this=this@entry=0x555556bc8530, addrConnect=..., pszDest=0x0, pszDest@entry=0x7fffac08c150 "seed.testnet.bitcoin.sprovoost.nl", fCountFailure=fCountFailure@entry=false) at net.cpp:448
233 2018-02-01 15:51:20	0|provoostenator|Mmm, it might actually be down. Let me check
234 2018-02-01 15:51:24	0|morcos|cfields i did make changes in /etc/hosts, but isn't seed.bitcoin.sipa.be a mainnet seed?
235 2018-02-01 15:51:47	0|cfields|morcos: yea, but you said you had all of em. didn't know what you set in there
236 2018-02-01 15:51:51	0|sdaftuar|i don't know if this is related, but i have a lot of lines like this in my debug.log: "trying connection seed.testnet.bitcoin.sprovoost.nl lastseen=0.0hrs"
237 2018-02-01 15:52:04	0|cfields|provoostenator: it's down for me...
238 2018-02-01 15:52:19	0|provoostenator|For me as well.
239 2018-02-01 15:52:25	0|cfields|but that should return 0 addresses and not try a connection. It shouldn't end up trying to connect to 0...
240 2018-02-01 15:52:31	0|provoostenator|I still need to setup monitoring.
241 2018-02-01 15:52:34	0|cfields|provoostenator: leave it down while we're testing :)
242 2018-02-01 15:52:38	0|morcos|here is the bt from that thread
243 2018-02-01 15:52:42	0|morcos|https://0bin.net/paste/X7-3MRaRJlLP8PoG#ApzQJmsY+N3H64RbQ-42PwxlbdJhXwpSvnx1BoINmZ+
244 2018-02-01 15:52:52	0|provoostenator|cfields: will do, just ping me when you want me to bring it back
245 2018-02-01 15:53:01	0|morcos|let me know if you want me to look at any particular value
246 2018-02-01 15:55:09	0|cfields|weird, i don't hit an assert there
247 2018-02-01 15:55:09	0|morcos|isn't it telling that in my /proc/pid/fd i didn't have a FD 0 in the earlier paste
248 2018-02-01 15:55:14	0|morcos|i also don't have one in this paste
249 2018-02-01 15:55:28	0|morcos|sorry not paste, example or whatever
250 2018-02-01 15:55:33	0|wumpus|morcos: that's very telling
251 2018-02-01 15:55:52	0|cfields|morcos: very good point. that would explain the environment difference too
252 2018-02-01 15:56:21	0|morcos|didn't follow that last part
253 2018-02-01 15:57:08	0|cfields|morcos: if something about your OS/mem/etc. makes 0 a more likely value for you than everyone else
254 2018-02-01 15:57:29	0|provoostenator|My seed died with: https://0bin.net/paste/2bx9a3iijDdBgWXg#dlXDiXwy9dVkyvR0yuCyjzH4scHaL+6GHT-ll14wOrD
255 2018-02-01 15:58:43	0|sdaftuar|i just added an else {} clause in net.cpp (before the suspicious line 448), and it triggered for me on -testnet when running with -forcednsseeds
256 2018-02-01 15:59:36	0|morcos|huh, the fix didn't fix it?
257 2018-02-01 16:00:03	0|sdaftuar|wasn't running with the fix -- just verifying that hSocket could indeed be uninitialized
258 2018-02-01 16:00:08	0|cfields|or are you saying that you've verified that you can hit the else branch?
259 2018-02-01 16:00:13	0|sdaftuar|^ that
260 2018-02-01 16:00:16	0|cfields|ok, great
261 2018-02-01 16:00:49	0|cfields|morcos: have you been on testnet every time you've hit this?
262 2018-02-01 16:01:20	0|cfields|i realize that a mainnet seed could've been returning 0 as well, but that doesn't seem like it'd affect a mainnnet node that's been up for more than a few minutes
263 2018-02-01 16:01:50	0|cfields|but i guess that does jive with your complaints that we're querying the seeds too often
264 2018-02-01 16:02:12	0|cfields|heh, in fact, you would've been noticing that because there'd be an entry at the end of every log file
265 2018-02-01 16:02:18	0|morcos|no all the prior times were mainnet
266 2018-02-01 16:02:22	0|provoostenator|Doesn't it pick a seed at random?
267 2018-02-01 16:02:35	0|morcos|but yes i was querying dns seeds occasionally
268 2018-02-01 16:02:45	0|morcos|i don't know what you mean about seing an etnry at the end of every log file
269 2018-02-01 16:02:45	0|provoostenator|Or does it ping them all? Because I didn't get crashes only 1 in ~5 times, I got them all the time, with a fresh datadir.
270 2018-02-01 16:03:09	0|morcos|the problem is if there is a dnsseed that is returning garbage somehow, it'll periodically retry it right?
271 2018-02-01 16:03:18	0|morcos|so if everytime it does that, it results in me close fd 0
272 2018-02-01 16:03:26	0|morcos|which at that point has been reused for soemthing else
273 2018-02-01 16:03:34	0|morcos|it'll cause an error
274 2018-02-01 16:04:12	0|morcos|but the socket errors aren't fatal, so its only the leveldb or blockwriting errors that cause a crash and show up at the end of the log
275 2018-02-01 16:04:16	0|morcos|but maybe thats what you're saying
276 2018-02-01 16:04:28	0|cfields|morcos: it only hits the seeds if we don't have enough peers
277 2018-02-01 16:04:53	0|cfields|it shouldn't keep retrying anything just because it failed
278 2018-02-01 16:05:12	0|cfields|wait, it might now!
279 2018-02-01 16:06:13	0|provoostenator|Should there be a functional test for dealing with broken DNS seeds?
280 2018-02-01 16:06:41	0|cfields|provoostenator: does your seed support filtering?
281 2018-02-01 16:07:14	0|cfields|it would make sense if a connection was tried because it was a oneshot()...
282 2018-02-01 16:07:17	0|morcos|cfields: it looked to me that it adds it to oneshot
283 2018-02-01 16:07:29	0|morcos|isn't that what it does with dnsseeds
284 2018-02-01 16:07:34	0|morcos|assuming you need them
285 2018-02-01 16:07:38	0|cfields|that's a new change, sec
286 2018-02-01 16:08:06	0|cfields|so we're not differentiating between a failed resolve, and a resolve with 0 results
287 2018-02-01 16:08:53	0|morcos|cfields: yeah can you look at line 390 in net.cpp
288 2018-02-01 16:09:04	0|morcos|i don't know what that does, but what happens if it fails
289 2018-02-01 16:09:11	0|cfields|#11512
290 2018-02-01 16:09:14	0|gribble|https://github.com/bitcoin/bitcoin/issues/11512 | Use GetDesireableServiceFlags in seeds, dnsseeds, fixing static seed adding by TheBlueMatt · Pull Request #11512 · bitcoin/bitcoin · GitHub
291 2018-02-01 16:09:17	0|morcos|if (Lookup(pszDest, resolved,  default_port, fNameLookup && !HaveNameProxy(), 256) && !resolved.empty()) {
292 2018-02-01 16:14:24	0|provoostenator|cfields: I use sipa's tool with the default settings, see my 0bin past above for full command
293 2018-02-01 16:16:28	0|cfields|morcos: ok, red herring. I see what's happening.
294 2018-02-01 16:16:45	0|cfields|it fails on the filtered resolve, so it does a oneshot for the unfiltered one. working as intended
295 2018-02-01 16:17:06	0|cfields|but they're both down, so you get a random socket
296 2018-02-01 16:18:35	0|cfields|wumpus: i'm more and more confident that this is the issue
297 2018-02-01 16:19:03	0|wumpus|cfields: great!
298 2018-02-01 16:19:18	0|cfields|and very sorry that i introduced it :(
299 2018-02-01 16:19:22	0|wumpus|means we can do rc2 soon
300 2018-02-01 16:19:38	0|wumpus|heh no worries
301 2018-02-01 16:20:15	0|wumpus|happy if it's this and some problem deep in leveldb
302 2018-02-01 16:20:24	0|morcos|provoostenator is right though we should have a test for bad dns seed.
303 2018-02-01 16:22:22	0|cfields|we could add that to the travis cron job
304 2018-02-01 16:22:45	0|wumpus|I agree, would be somewhat tricky to spin up a fake dns server in the test framework though, though maybe python has something easy for that I don't know
305 2018-02-01 16:23:14	0|cfields|oh, i thought the concern was not knowing that dns seeds were down
306 2018-02-01 16:23:28	0|sdaftuar|i think the concern is making sure we handle it when they are?  or really both i guess...
307 2018-02-01 16:24:09	0|provoostenator|I'll keep the bisect going just in case. Leaving it running for 25 minutes until I "git bisect good" a commit, so it will take few hours.
308 2018-02-01 16:24:18	0|cfields|2018-02-01 16:23:47 Closing bad socket: 1266668816
309 2018-02-01 16:24:47	0|cfields|2018-02-01 16:24:30 Closing bad socket: 100640016
310 2018-02-01 16:24:57	0|cfields|yep, that's it
311 2018-02-01 16:39:42	0|provoostenator|Mmm, if it's DNS related bisect might end up finding the PR where my seed was merged. Anyway, we'll see.
312 2018-02-01 17:00:21	0|bitcoin-git|[13bitcoin] 15promag opened pull request #12327: [gui] Defer coin control instancing (06master...062018-02-fix-12312) 02https://github.com/bitcoin/bitcoin/pull/12327
313 2018-02-01 17:03:05	0|instagibbs|promag, how come that error doesn't result in *all* qt coin control settings getting ignored?
314 2018-02-01 17:04:49	0|promag|instagibbs: I think the only affected are `change_type = g_change_type;` and `signalRbf = fWalletRbf;`
315 2018-02-01 17:05:17	0|promag|because these are set with argument
316 2018-02-01 17:07:44	0|promag|and CoinControlDialog::coinControl->SetNull() is only when the "Enable coin control features" is unchecked
317 2018-02-01 17:08:01	0|promag|instagibbs: yeah didn't look at that
318 2018-02-01 17:12:31	0|provoostenator|^ created the above to track my bisect progress
319 2018-02-01 17:12:53	0|provoostenator|Well, below: #12328
320 2018-02-01 17:12:55	0|gribble|https://github.com/bitcoin/bitcoin/issues/12328 | Consistent crashes for v0.16.0rc1 · Issue #12328 · bitcoin/bitcoin · GitHub
321 2018-02-01 17:15:01	0|instagibbs|promag, your analysis looks correct, those two should be those effected
322 2018-02-01 17:53:05	0|provoostenator|Produced a crash using an older version: https://github.com/bitcoin/bitcoin/commit/16bac24f60fa3ae27cb2d9d89dfdd245694445d4
323 2018-02-01 17:53:10	0|provoostenator|Four more bisect steps to go
324 2018-02-01 17:53:34	0|provoostenator|Well, that's just 7 days ago...
325 2018-02-01 17:54:10	0|provoostenator|I added the log to the above issue.
326 2018-02-01 17:54:13	0|ProfMac|"should" the DNS discover use IPv4 when onlynet=IPv6?
327 2018-02-01 17:56:00	0|wumpus|AFAIK there's no way in the libc API to do DNS resolving only over either IPv4 or IPv6
328 2018-02-01 17:57:16	0|wumpus|and as many modern linux distros run a DNS cache on localhost, on the IPv4 loopback, that'd effectively mean that DNS seeding cannot be used when onlynet=ipv6
329 2018-02-01 18:04:11	0|wumpus|(hm, how many ISPs give out IPv6 DNS servers in the first place?)
330 2018-02-01 18:06:28	0|wumpus|probably only those that only give clients a IPv6 address
331 2018-02-01 18:07:05	0|wumpus|I don't know, it's an interesting though experiment, but I think in practice it's good that use of dns seeding or not is a separate optoin
332 2018-02-01 18:21:49	0|cfields|wumpus: you can kinda fudge it with AI_ADDRCONFIG, as we do
333 2018-02-01 18:22:16	0|bitcoin-git|13bitcoin/06master 1496dbd38 15Cory Fields: net: initialize socket to avoid closing random fd's
334 2018-02-01 18:22:16	0|bitcoin-git|[13bitcoin] 15laanwj pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/895fbd768f0c...84291d18dd69
335 2018-02-01 18:22:17	0|bitcoin-git|13bitcoin/06master 1484291d1 15Wladimir J. van der Laan: Merge #12326: net: initialize socket to avoid closing random fd's...
336 2018-02-01 18:22:29	0|cfields|oh, you mean the resolver itself
337 2018-02-01 18:22:51	0|bitcoin-git|13bitcoin/060.16 14e54c1ac 15Cory Fields: net: initialize socket to avoid closing random fd's...
338 2018-02-01 18:22:51	0|bitcoin-git|[13bitcoin] 15laanwj pushed 1 new commit to 060.16: 02https://github.com/bitcoin/bitcoin/commit/e54c1ac110664efd58b7351139da55284f58f2ca
339 2018-02-01 18:23:12	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #12326: net: initialize socket to avoid closing random fd's (06master...06fix-socket-init) 02https://github.com/bitcoin/bitcoin/pull/12326
340 2018-02-01 18:23:39	0|cfields|wumpus: i think there's another issue there, introduced by (my suggestion in) 11512
341 2018-02-01 18:24:02	0|wumpus|cfields: yes, I was assuming he meant the network used by the resolver itself, selecting only one kind of results sounds feasible
342 2018-02-01 18:24:04	0|cfields|I believe that failed resolves will end up forever connecting as oneshots, since failed oneshots get re-added
343 2018-02-01 18:24:59	0|wumpus|whoops
344 2018-02-01 18:25:13	0|cfields|looking into it
345 2018-02-01 18:48:59	0|cfields|actually, anyone know why oneshots are re-added in the first place? https://github.com/bitcoin/bitcoin/blob/master/src/net.cpp#L1685
346 2018-02-01 18:49:15	0|cfields|sipa maybe ?
347 2018-02-01 18:49:37	0|wumpus|I don't know, I woudln't have expected that either
348 2018-02-01 18:49:51	0|wumpus|if it's one-shot - if it fails, that was the shot
349 2018-02-01 18:50:04	0|cfields|right
350 2018-02-01 18:50:37	0|wumpus|I'm fairly sure I've used 'addnode ... onetry' in the past to probe if a certain node was up, not expecting it to try forever
351 2018-02-01 18:51:46	0|BlueMatt|cfields: if your network is done
352 2018-02-01 18:51:51	0|BlueMatt|or something like that
353 2018-02-01 18:52:04	0|BlueMatt|there was some pr that made things more robust if your net is down
354 2018-02-01 18:52:08	0|BlueMatt|maybe that is related?
355 2018-02-01 18:52:18	0|cfields|BlueMatt: wouldn't that same logic apply to everything, not just oneshots?
356 2018-02-01 18:52:33	0|BlueMatt|uhhhh, uhhhhh
357 2018-02-01 18:52:36	0|BlueMatt|?
358 2018-02-01 18:53:16	0|wumpus|for non-oneshot connections I could understand it better
359 2018-02-01 18:53:23	0|wumpus|maybe the logic is the wrong way around?
360 2018-02-01 18:54:12	0|cfields|looks like it's been this way since they were introduced:  478b01d9a797f3ea
361 2018-02-01 18:56:01	0|cfields|heh, when provoostenator gets his seed back up and running, he'll likely be DDoS'd like crazy
362 2018-02-01 18:56:08	0|wumpus|maybe it's been the wrong way around since the beginning, and no one ever reasoned this far
363 2018-02-01 18:56:55	0|wumpus|well at least there haven't been rc1 binaries, so the scale of DoS is likely limited :)
364 2018-02-01 18:57:33	0|cfields|heh
365 2018-02-01 18:59:34	0|cfields|i'll go ahead and PR the removal, maybe someone will chime in with a valid reason otherwise
366 2018-02-01 18:59:44	0|wumpus|agreed
367 2018-02-01 19:00:11	0|lightningbot|Meeting started Thu Feb  1 19:00:10 2018 UTC.  The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
368 2018-02-01 19:00:11	0|lightningbot|Useful Commands: #action #agreed #help #info #idea #link #topic.
369 2018-02-01 19:00:11	0|wumpus|#startmeeting
370 2018-02-01 19:00:26	0|provoostenator|hi
371 2018-02-01 19:00:33	0|wumpus|#bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr btcdrak sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier jl2012 achow101 meshcollider jnewbery maaku fanquake promag provoostenator
372 2018-02-01 19:00:35	0|achow101|hi
373 2018-02-01 19:00:36	0|jonasschnelli|hi
374 2018-02-01 19:00:39	0|cfields|hi
375 2018-02-01 19:00:53	0|sdaftuar|ack
376 2018-02-01 19:00:58	0|jcorgan|hey folks
377 2018-02-01 19:01:05	0|meshcollider|hi
378 2018-02-01 19:01:08	0|luke-jr|hi
379 2018-02-01 19:01:14	0|BlueMatt|0.16!
380 2018-02-01 19:01:19	0|meshcollider|\o/
381 2018-02-01 19:01:23	0|wumpus|so with regard for 0.16.0 status, there already have been some issues that came up with rc1, so I think it makes sense to skip uploading binaries for that and go to rc2 soon
382 2018-02-01 19:01:31	0|BlueMatt|ack
383 2018-02-01 19:01:32	0|achow101|what issues?
384 2018-02-01 19:01:32	0|cfields|agreed
385 2018-02-01 19:01:48	0|cfields|achow101: see backlog for the last ~3hrs
386 2018-02-01 19:02:02	0|wumpus|https://github.com/bitcoin/bitcoin/milestone/30
387 2018-02-01 19:02:18	0|wumpus|the serious issue is https://github.com/bitcoin/bitcoin/issues/12323
388 2018-02-01 19:02:46	0|achow101|ok
389 2018-02-01 19:03:25	0|wumpus|there's another issue with onetry connections being re-tried forever, resulting in potential DoS on DNS seeds in the case they temporarily fail
390 2018-02-01 19:03:34	0|wumpus|cfields is working on a patch for that
391 2018-02-01 19:03:42	0|instagibbs|oh right, thursday
392 2018-02-01 19:03:59	0|wumpus|https://github.com/bitcoin/bitcoin/pull/12327 fixes a more minor issue with coin control and the change type setting
393 2018-02-01 19:04:06	0|wumpus|in the gui
394 2018-02-01 19:04:20	0|jonasschnelli|by the way, is it a policy that a DNS seed also runs a node (same ip) for the oneshot?
395 2018-02-01 19:04:34	0|wumpus|jonasschnelli: no, that's not necessary
396 2018-02-01 19:04:48	0|wumpus|jonasschnelli: it looks up the DNS seed which will return a (the first?) node
397 2018-02-01 19:04:51	0|jonasschnelli|wumpus: okay. My seeders will refuse connections to 8333
398 2018-02-01 19:05:02	0|jonasschnelli|wumpus: okay.
399 2018-02-01 19:05:35	0|wumpus|jonasschnelli: that is not the IP of the DNS server. I was confused about that too at some point in the past.
400 2018-02-01 19:05:50	0|jonasschnelli|I think also has something do to with tor mode
401 2018-02-01 19:05:54	0|provoostenator|Using A records is what makes it confusing
402 2018-02-01 19:05:55	0|cfields|yea, it's just some random peer
403 2018-02-01 19:06:18	0|BlueMatt|jonasschnelli: yes, you'd have to include your own ip in the dnsseed to (maybe) get the oneshot to be you, but that would be bad, and a violation of dnsseed policy (IIRC)
404 2018-02-01 19:06:32	0|jonasschnelli|BlueMatt: sure.
405 2018-02-01 19:06:42	0|wumpus|yes, in tor mode no resolving is used to get  multiple results (that'd require some SOCKS5 extension being used), so it uses a one shot to a random node as replacement
406 2018-02-01 19:06:54	0|jonasschnelli|But in tor only mode, don't we do a oneshot to the seeds?
407 2018-02-01 19:06:58	0|wumpus|no
408 2018-02-01 19:06:59	0|provoostenator|BlueMatt: and an effective way to DDOS yourself
409 2018-02-01 19:07:13	0|jonasschnelli|wumpus: okay. Thanks... never looked that up properly
410 2018-02-01 19:07:31	0|wumpus|it never looks up the IP of the DNS server at all, that's all happening below the libc abstraction
411 2018-02-01 19:07:49	0|wumpus|any topics?
412 2018-02-01 19:10:03	0|jonasschnelli|Everyone already back at work it seems
413 2018-02-01 19:10:04	0|wumpus|ok.. seems not... well please review anything under the 0.16.0 milestone, and anything added in the next day too
414 2018-02-01 19:10:10	0|sdaftuar|if we don't have more pressing things to discuss, i'd like to solicit feedback on #11739 (backdating p2sh /segwit v0 script rules) to genesis
415 2018-02-01 19:10:13	0|gribble|https://github.com/bitcoin/bitcoin/issues/11739 | Enforce SCRIPT_VERIFY_P2SH and SCRIPT_VERIFY_WITNESS from genesis by sdaftuar · Pull Request #11739 · bitcoin/bitcoin · GitHub
416 2018-02-01 19:10:15	0|wumpus|then we'll tag rc2 after that
417 2018-02-01 19:10:43	0|wumpus|and try to find the next round of bugs :)
418 2018-02-01 19:10:50	0|bitcoin-git|[13bitcoin] 15theuni opened pull request #12329: net: don't retry failed oneshot connections forever (06master...06no-infinite-oneshot) 02https://github.com/bitcoin/bitcoin/pull/12329
419 2018-02-01 19:10:51	0|wumpus|sdaftuar: okay
420 2018-02-01 19:11:03	0|sdaftuar|mostly i want to know if there are any concept NACKs
421 2018-02-01 19:11:14	0|wumpus|#topic enforce SCRIPT_VERIFY_P2SH and SCRIPT_VERIFY_WITNESS from genesis (sdaftuar)
422 2018-02-01 19:12:17	0|sdaftuar|and i guess the other question is confirming whether/how such a change should be documented
423 2018-02-01 19:12:19	0|cfields|+0
424 2018-02-01 19:12:40	0|cfields|sdaftuar: going forward, you mean? or this one?
425 2018-02-01 19:12:44	0|sdaftuar|both?
426 2018-02-01 19:12:49	0|sdaftuar|i drafted a BIP for this one
427 2018-02-01 19:13:05	0|cfields|well going forward, i think we could specify this intention as part of a soft-fork bip?
428 2018-02-01 19:13:06	0|wumpus|no NACK from me, if the code can be simplified that way then it's great
429 2018-02-01 19:13:29	0|BlueMatt|+1
430 2018-02-01 19:13:34	0|wumpus|it doesn't change the rules enforced for current blocks, does it?
431 2018-02-01 19:13:38	0|wumpus|how is it a softfork?
432 2018-02-01 19:13:42	0|sdaftuar|no effect on current blocks
433 2018-02-01 19:13:56	0|wumpus|if it's a softfork I am misunderstanding
434 2018-02-01 19:14:04	0|BlueMatt|its a soft spoon - only prevents a 6-month reorg from removing segwit :p
435 2018-02-01 19:14:07	0|luke-jr|it restricts the rules on older blocks
436 2018-02-01 19:14:07	0|sdaftuar|it's a softfork under a technical definition
437 2018-02-01 19:14:11	0|BlueMatt|not a fork
438 2018-02-01 19:14:19	0|sdaftuar|of making valid things now invalid
439 2018-02-01 19:14:22	0|cfields|sorry, my fault.
440 2018-02-01 19:14:29	0|morcos|+1 as well..  but i do have concerns about how we could do this on a going forward basis
441 2018-02-01 19:14:30	0|wumpus|oh, right
442 2018-02-01 19:14:38	0|provoostenator|Or just Buried Deployment?
443 2018-02-01 19:14:40	0|wumpus|but it makes no difference bencause the old blocks all qualify
444 2018-02-01 19:14:42	0|luke-jr|the question comes down to, are we limiting soft/hardfork definitions to only ones that affect future blocks?
445 2018-02-01 19:14:43	0|morcos|it seems like if this is always our intention, then as soon as we announce a future soft fork
446 2018-02-01 19:14:53	0|morcos|some jack ass is going to mine violations just to make us annoyed
447 2018-02-01 19:15:01	0|BlueMatt|luke-jr: yes, we should start calling buried deployments spoons
448 2018-02-01 19:15:02	0|luke-jr|or do we consider this an implementation detail?
449 2018-02-01 19:15:14	0|cfields|morcos: true
450 2018-02-01 19:15:17	0|sdaftuar|morcos: i think it's not really worth worrying about that
451 2018-02-01 19:15:22	0|wumpus|I see this as an implementation detail to validation
452 2018-02-01 19:15:32	0|wumpus|there's no need to cause a lot of rufus about it
453 2018-02-01 19:15:42	0|wumpus|if you call it softfork you'll have the miners in arms, and whatnot
454 2018-02-01 19:15:44	0|cfields|luke-jr: well if there's an absolutely massive reorg, it's not just an implementation detail, no?
455 2018-02-01 19:15:48	0|morcos|well it addresses cfields point about having the original BIP specify the intention.  i think we should always only consider backdating after the fact
456 2018-02-01 19:16:03	0|sdaftuar|morcos: oh i see your point
457 2018-02-01 19:16:08	0|wumpus|because then it also needs to be signaled some way, I guess
458 2018-02-01 19:16:10	0|luke-jr|cfields: if there's an absolutely massive reorg, it's unclear what the outcome will be period
459 2018-02-01 19:16:24	0|luke-jr|cfields: for example, Knots has a checkpoint on a Segwit block
460 2018-02-01 19:16:31	0|cfields|well isn't the intention here to clarify that outcome?
461 2018-02-01 19:16:33	0|BlueMatt|if there's a 6 month reorg there will be debate as to whether to follow it...whether we follow it or not ends up being a community question anyway :p
462 2018-02-01 19:16:35	0|wumpus|if there's a reorg that big that all segwit blocks are reorged out, well...
463 2018-02-01 19:16:54	0|sdaftuar|i think in this case, it's clear that segwit transactors do not intend for their funds to be spendable on a segwit-inactive chain
464 2018-02-01 19:17:00	0|wumpus|yes, I'm sure there will be discussion enough in that case
465 2018-02-01 19:17:15	0|sdaftuar|so backdating the segwit rules matches consensus, in that sense
466 2018-02-01 19:17:16	0|BlueMatt|so, definitely not a fork
467 2018-02-01 19:17:23	0|wumpus|right
468 2018-02-01 19:18:11	0|luke-jr|in which case, I don't see that we need a BIP for it. I suggest we make a new repo for Core-specific documentation like this.
469 2018-02-01 19:18:26	0|cfields|morcos: i half agree about after-the-fact. Not mentioning backdating with the intention of doing so anyway is a bit... iffy
470 2018-02-01 19:18:31	0|luke-jr|BIPs are for cross-software standards, which doesn't really include implementation details
471 2018-02-01 19:18:34	0|BlueMatt|seems fine to me, I also appreciate gmaxwell's partially-joking suggestion of calling it a spoon
472 2018-02-01 19:18:48	0|wumpus|hehe
473 2018-02-01 19:18:51	0|luke-jr|(actually, we have a repo for gitian docs already, right?)
474 2018-02-01 19:18:52	0|sdaftuar|i personally think that it's helpful to put it in a BIP, because it affects the implementation of existing BIPs
475 2018-02-01 19:18:53	0|BlueMatt|and then doing a bip and just saying "Soft Spoon"
476 2018-02-01 19:19:06	0|sdaftuar|but i don't feel strongly
477 2018-02-01 19:19:19	0|BlueMatt|i mean we use BIPs for things that are core-specific anyway, like getblocktemplate
478 2018-02-01 19:19:23	0|wumpus|but in the end it doesn't matter whether people implement this BIP
479 2018-02-01 19:19:27	0|wumpus|because it's an implementation detail
480 2018-02-01 19:19:28	0|luke-jr|BlueMatt: GBT isn't Core-specific
481 2018-02-01 19:19:29	0|sdaftuar|wumpus: agreed
482 2018-02-01 19:19:32	0|sdaftuar|it's an informational BIP
483 2018-02-01 19:19:37	0|cfields|luke-jr: there are several post-mortem BIPs
484 2018-02-01 19:19:38	0|wumpus|BlueMatt: well that's an interface! interfaces need documentation
485 2018-02-01 19:19:43	0|kanzure|hi.
486 2018-02-01 19:20:29	0|luke-jr|maybe we can put it in an annex for the BIPs it affects or something? just seems like it will get old to have two BIPs for every fork
487 2018-02-01 19:20:38	0|wumpus|if a softspoon drops at 300000 blocks deep and no one hears it, did it happen at all?
488 2018-02-01 19:20:45	0|luke-jr|one for the deployment and implementation, and another for the reinterpretation of the deployment
489 2018-02-01 19:20:54	0|sdaftuar|luke-jr: that seems reasonable to me as well, if the BIP authors agree?
490 2018-02-01 19:22:08	0|wumpus|luke-jr: agree
491 2018-02-01 19:22:26	0|luke-jr|none of the authors appear to be here now, but I doubt they'd object
492 2018-02-01 19:22:35	0|luke-jr|at least for Segwit
493 2018-02-01 19:23:33	0|provoostenator|And the winner of the git bisect game is....
494 2018-02-01 19:23:38	0|provoostenator|bluematt! https://github.com/bitcoin/bitcoin/commit/62e764219b25f5d5a4de855e53f62c43130ec918
495 2018-02-01 19:23:57	0|BlueMatt|we already decided it was cfields' fault
496 2018-02-01 19:24:07	0|sdaftuar|i knew it was both of you
497 2018-02-01 19:24:10	0|sdaftuar|:)
498 2018-02-01 19:24:23	0|cfields|heh it was me for sure
499 2018-02-01 19:24:42	0|cfields|the busted part of 62e7642 was even my suggestion!
500 2018-02-01 19:25:11	0|wumpus|it's everyone's fault for not finding the fault in review! :)
501 2018-02-01 19:25:17	0|sdaftuar|+1
502 2018-02-01 19:25:33	0|cfields|well, sorry everyone. I'm glad it didn't make it into a release.
503 2018-02-01 19:25:53	0|luke-jr|I'm glad someone noticed it before a release XD
504 2018-02-01 19:25:55	0|wumpus|the rc process, it works
505 2018-02-01 19:26:16	0|cfields|yea, the surge of reports in the last ~day is actually really nice to see
506 2018-02-01 19:26:19	0|provoostenator|Bad linux skills on my part (causing the seed not to restart): it works
507 2018-02-01 19:26:38	0|sdaftuar|good thing it was down or we never would have found this before release!
508 2018-02-01 19:26:44	0|sdaftuar|(which is very disturbing)
509 2018-02-01 19:26:54	0|wumpus|we do need tests for the DNS seed code
510 2018-02-01 19:27:02	0|meshcollider|clearly
511 2018-02-01 19:27:04	0|provoostenator|+1 for tests there
512 2018-02-01 19:27:16	0|cfields|agreed. I'll add some.
513 2018-02-01 19:27:21	0|BlueMatt|lol, maybe keep your dnsseed down until after rc cycle? :P
514 2018-02-01 19:27:57	0|wumpus|any other topics?
515 2018-02-01 19:28:08	0|cfields|or we could just add a dummy seednode to test.com or something :p
516 2018-02-01 19:30:04	0|wumpus|testing that code is not entirely trivial as-is as you somehow need to redirect DNS resolving
517 2018-02-01 19:30:23	0|wumpus|if no other topics, I'm closing the meeting early
518 2018-02-01 19:30:44	0|lightningbot|Log:            http://www.erisian.com.au/meetbot/bitcoin-core-dev/2018/bitcoin-core-dev.2018-02-01-19.00.log.html
519 2018-02-01 19:30:44	0|lightningbot|Meeting ended Thu Feb  1 19:30:43 2018 UTC.  Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
520 2018-02-01 19:30:44	0|lightningbot|Minutes:        http://www.erisian.com.au/meetbot/bitcoin-core-dev/2018/bitcoin-core-dev.2018-02-01-19.00.html
521 2018-02-01 19:30:44	0|lightningbot|Minutes (text): http://www.erisian.com.au/meetbot/bitcoin-core-dev/2018/bitcoin-core-dev.2018-02-01-19.00.txt
522 2018-02-01 19:30:44	0|wumpus|#endmeeting
523 2018-02-01 19:30:45	0|cfields|sgtm
524 2018-02-01 19:30:48	0|luke-jr|noooooooooooooooooo
525 2018-02-01 19:30:50	0|sdaftuar|ack
526 2018-02-01 19:30:53	0|gmaxwell|damnit
527 2018-02-01 19:31:07	0|gmaxwell|missed the meeting by 30 seconds.
528 2018-02-01 19:31:09	0|luke-jr|lol
529 2018-02-01 19:31:28	0|gmaxwell|Without reading the logs, can we apply the fixes for that fd issue and RC2 like lightning?
530 2018-02-01 19:31:38	0|sdaftuar|i believe that is the plan
531 2018-02-01 19:31:47	0|wumpus|yes, that is the plan
532 2018-02-01 19:32:04	0|gmaxwell|Good.
533 2018-02-01 19:32:48	0|wumpus|merge+backport #12323 and #12312 and tag rc2, and just skip executables for rc1
534 2018-02-01 19:32:49	0|gribble|https://github.com/bitcoin/bitcoin/issues/12323 | File descriptor problem, causing leveldb crash · Issue #12323 · bitcoin/bitcoin · GitHub
535 2018-02-01 19:32:50	0|gribble|https://github.com/bitcoin/bitcoin/issues/12312 | QT ignores -changetype=bech32 when coin control features are enabled · Issue #12312 · bitcoin/bitcoin · GitHub
536 2018-02-01 19:34:21	0|provoostenator|Meanwhile I'm checking if #12326 actually makes the crash go away now, as well as whether removing my seed from v0.16.0rc1 makes it go away (very likely yes).
537 2018-02-01 19:34:24	0|gribble|https://github.com/bitcoin/bitcoin/issues/12326 | net: initialize socket to avoid closing random fds by theuni · Pull Request #12326 · bitcoin/bitcoin · GitHub
538 2018-02-01 19:35:27	0|jcorgan|hey guys, just fyi, i've passed on the management/maintainer/architect roles in gnuradio to new hands (after 12 years), to focus on bitcoin-related work full-time. part of that will be getting back into core dev process. very much looking forward to the change of pace.
539 2018-02-01 19:35:43	0|gmaxwell|jcorgan: awesome!
540 2018-02-01 19:35:57	0|cfields|wumpus: and 12329 ?
541 2018-02-01 19:36:08	0|wumpus|jcorgan: congratulations!
542 2018-02-01 19:36:11	0|gmaxwell|(I mean, nooo sucks for gnuradio; and all my SDR projects)
543 2018-02-01 19:36:43	0|cfields|jcorgan: very cool. 12 years is a long time
544 2018-02-01 19:37:03	0|wumpus|cfields: huh I meant that one
545 2018-02-01 19:37:15	0|jcorgan|i think that's half over some people here's lifetime
546 2018-02-01 19:37:19	0|jcorgan|*of
547 2018-02-01 19:37:23	0|wumpus|oh cool maybe I can go to gnu radio now
548 2018-02-01 19:37:41	0|sdaftuar|wait what
549 2018-02-01 19:37:42	0|cfields|jcorgan: you're Bitcoin Core maintainer now! congrats.
550 2018-02-01 19:37:46	0|achow101|jcorgan: more than half
551 2018-02-01 19:38:47	0|instagibbs|\o/
552 2018-02-01 19:39:07	0|instagibbs|who here is best to bug about gitian? I'd hate to blow away my gitian directory yet again...
553 2018-02-01 19:39:28	0|sdaftuar|who is going to answer yes to that question
554 2018-02-01 19:39:36	0|instagibbs|someone who loves... pain
555 2018-02-01 19:39:38	0|achow101|instagibbs: probably cfields
556 2018-02-01 19:39:53	0|sdaftuar|lol
557 2018-02-01 19:39:55	0|cfields|instagibbs: what's the issue?
558 2018-02-01 19:40:14	0|instagibbs|ill DM, it's likely intractible
559 2018-02-01 19:40:24	0|wumpus|it's surprising in how many different ways gitian can fail
560 2018-02-01 19:40:42	0|cfields|this is going to be the least sexy slide into a DM ever...
561 2018-02-01 19:40:58	0|promag|o/ late..
562 2018-02-01 19:41:03	0|achow101|gitian is kinda outdated. it looks like its doing things that aren't really supported anymore
563 2018-02-01 19:41:09	0|jcorgan|anyway, carry on, i have to go fly a plane in the sky
564 2018-02-01 19:42:18	0|instagibbs|cfields, lol
565 2018-02-01 19:42:28	0|wumpus|lol indeed
566 2018-02-01 19:43:27	0|meshcollider|0.16 crashes my gitian build too grr 0.15.1 was ok
567 2018-02-01 19:43:38	0|meshcollider|I probably need to upgrade something
568 2018-02-01 19:43:41	0|instagibbs|trying as far back as 0.14.1, not working
569 2018-02-01 19:43:46	0|wumpus|strange, nothing should have changed in that regard
570 2018-02-01 19:43:52	0|instagibbs|(something I know this VM did correctly once)
571 2018-02-01 19:43:59	0|wumpus|still uses the same version of ubuntu for buildling, still uses the same packages, etc
572 2018-02-01 19:44:12	0|achow101|I bet some package updated and that just broke everything
573 2018-02-01 19:51:43	0|instagibbs|cfields fixed my issue... was looking at my even older version of gitian that I had failed to rename to something sensible
574 2018-02-01 19:53:59	0|bitcoin-git|13bitcoin/06master 146558f8a 15João Barbosa: [gui] Defer coin control instancing...
575 2018-02-01 19:53:59	0|bitcoin-git|[13bitcoin] 15jonasschnelli pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/84291d18dd69...41363fe11df5
576 2018-02-01 19:54:00	0|bitcoin-git|13bitcoin/06master 1441363fe 15Jonas Schnelli: Merge #12327: [gui] Defer coin control instancing...
577 2018-02-01 19:54:54	0|bitcoin-git|[13bitcoin] 15jonasschnelli closed pull request #12327: [gui] Defer coin control instancing (06master...062018-02-fix-12312) 02https://github.com/bitcoin/bitcoin/pull/12327
578 2018-02-01 19:56:28	0|gmaxwell|Would it be entirely crazy to make debug=1 not enable all debugging options, but a "many" option, that turns off absurdly chatty stuff (leveldb internals would be the one thing now).
579 2018-02-01 19:56:32	0|gmaxwell|?
580 2018-02-01 19:56:48	0|provoostenator|+1
581 2018-02-01 19:57:11	0|gmaxwell|I don't think any of us have ever found the leveldb internals logging useful so far. And I know I always disable it and curse when I accidentally level it on.
582 2018-02-01 19:57:25	0|sdaftuar|that seems to be everyone's experience afaik
583 2018-02-01 19:57:48	0|gmaxwell|er leave it on.
584 2018-02-01 19:58:09	0|gmaxwell|I suppose I've learned a bit about how much background stuff leveldb does due to it. :)
585 2018-02-01 19:58:22	0|wumpus|I don't like the chatty net logging either
586 2018-02-01 19:58:48	0|gmaxwell|At least I've found that stuff useful. I'd be okay with it off in debug=1 though I'd often turn it on.
587 2018-02-01 19:58:59	0|wumpus|which is why I opened #12219, I think we need a DEBUG..ERROR axis as well
588 2018-02-01 19:59:00	0|gribble|https://github.com/bitcoin/bitcoin/issues/12219 | More granular net logging · Issue #12219 · bitcoin/bitcoin · GitHub
589 2018-02-01 19:59:08	0|sdaftuar|yeah we should tackle that
590 2018-02-01 19:59:23	0|provoostenator|The flickering makes it hard to watch an IBD in real time.
591 2018-02-01 19:59:44	0|gmaxwell|or at least chatty net doesn't bother me so much since upgrading nodes to 1TB ssds...
592 2018-02-01 19:59:47	0|wumpus|apart from the category we also need a log level, that'd make logging things more sane without singling out specific categories
593 2018-02-01 19:59:50	0|wumpus|or splitting up categories
594 2018-02-01 20:00:50	0|wumpus|I think debug=1 should remain 'log everything possible', which doesn't rule out more selective logging options
595 2018-02-01 20:00:56	0|gmaxwell|I'd buy a debug vs error sort of axis though I'm doubtful about a really granular level, as people will irritatingly set them pretty subjectively.
596 2018-02-01 20:01:39	0|gmaxwell|or I suppose if we just had a debug=many or debug=most that would be fine too.  I seem to screw up debug=1 and then excluding for some reason.
597 2018-02-01 20:02:00	0|wumpus|or something like debug=1 debug=-leveldb
598 2018-02-01 20:02:03	0|gmaxwell|also having a single setting is more useful when I'm asking users to set things.
599 2018-02-01 20:02:06	0|wumpus|allowing categories to be disabled
600 2018-02-01 20:02:23	0|cfields|gmaxwell: agreed. -debug and -debug=all don't have to be the same thing
601 2018-02-01 20:02:34	0|gmaxwell|we have -debugexclude
602 2018-02-01 20:02:41	0|wumpus|but yes it is subjective and dpeends on what you want to debug
603 2018-02-01 20:02:54	0|wumpus|which is why making a single selection for -debug=1 seems weird to me
604 2018-02-01 20:03:20	0|wumpus|some messages might be less interesting for what you're debugging, but that's why we have categories in the first place
605 2018-02-01 20:03:25	0|gmaxwell|Though also for some of this user stuff, I think it would be very useful to have a circular buffer in ram that always gets a MUCH higher debug level than what goes to disk (e.g. every debug option that isn't computationally expensive)
606 2018-02-01 20:03:48	0|gmaxwell|and on crashes we could make a best effort attempt to dump the circular buffer to a file in a crash handler.
607 2018-02-01 20:04:06	0|wumpus|debug=1 is super-overkill last resort for when you really don't know where to look
608 2018-02-01 20:04:26	0|wumpus|gmaxwell: yes, we need that too
609 2018-02-01 20:04:57	0|gmaxwell|wumpus: or when round trips are expensive. if I have a user reporting an issue I don't want to iterate on a bunch of options. I just want all the info that might be relevant, but the leveldb stuff is so far useless and very bloaty.
610 2018-02-01 20:04:58	0|promag|-debug-net -debug-foo (-debug enables all)?
611 2018-02-01 20:05:20	0|wumpus|we already have -debug=net -debug=foo
612 2018-02-01 20:05:24	0|gmaxwell|promag: you can already -debug=category to add categories and -debugexclude to remove them.
613 2018-02-01 20:05:44	0|wumpus|gmaxwell: still you need a selection of categories then
614 2018-02-01 20:05:48	0|promag|but you can't have levels right?
615 2018-02-01 20:05:50	0|gmaxwell|right now I tell users to debug=1 and debugexclude leveldb.
616 2018-02-01 20:06:02	0|wumpus|gmaxwell: it's unlikely that you want debugging for ,say, torcontrol, even though that logging is extremely useful if you're debugging that
617 2018-02-01 20:06:14	0|gmaxwell|wumpus: tor control isn't that chatty however.
618 2018-02-01 20:06:24	0|gmaxwell|net is chatty but one of the most useful thing to log.
619 2018-02-01 20:06:31	0|wumpus|but it might become so, or another chatty category could be added
620 2018-02-01 20:06:52	0|wumpus|that's pretty subjective though
621 2018-02-01 20:06:55	0|wumpus|because you're interested in net
622 2018-02-01 20:07:14	0|gmaxwell|probably if I could grab a circular buffer with everything the question wrt support would go away.
623 2018-02-01 20:07:58	0|wumpus|currently I have the problem that I'm interested in high-level network stuff, e.g. incoming connections, outgoing connections, what clients are connecting, what are their IPs, when do they disconnect. I don't need to see every single packet.
624 2018-02-01 20:08:12	0|wumpus|but debug=net is way too chatty
625 2018-02-01 20:08:14	0|gmaxwell|I agree that net messages and net-activity should be split.
626 2018-02-01 20:08:41	0|gmaxwell|I do frequently like net-activity for debugging because from that I can more or less trace the state that the node is in.
627 2018-02-01 20:09:04	0|wumpus|sure, I don't mwan that detailed net logging should go away or such
628 2018-02-01 20:09:19	0|wumpus|it certainly has good uses when you're debugging network things
629 2018-02-01 20:09:46	0|wumpus|in any case I think a log level would resolve some of these problems
630 2018-02-01 20:10:00	0|wumpus|'I want to see all categories, but only at INFO levell, not DEBUG and below'
631 2018-02-01 20:10:08	0|wumpus|where DEBUG would be the chatty stuff
632 2018-02-01 20:10:13	0|promag|hence -debug-net=X
633 2018-02-01 20:10:14	0|gmaxwell|Please lets not make more than three levels though.
634 2018-02-01 20:10:15	0|wumpus|then if you want net DEBUG, you enable net debug
635 2018-02-01 20:10:37	0|wumpus|I agre,but let's not get into bikeshedding about the number of levels
636 2018-02-01 20:11:22	0|gmaxwell|my concern there is if there are a dozen levels, people will argue over the levels, or worse not argue over them and set them randomly and then I'll just have to be debugging everything to avoid inexplicably missing log entries.
637 2018-02-01 20:11:52	0|wumpus|ERROR is clear, INFO/DEBUG can be set depend on chattiness
638 2018-02-01 20:11:59	0|wumpus|I don't think we need more
639 2018-02-01 20:12:02	0|gmaxwell|At least my expirence is that I usually want almost everything or nothing.  (basically, everything that isn't so chatty that it makes handling the logs a burden)
640 2018-02-01 20:12:16	0|gmaxwell|Yes, thats why I said three. I think three we can handle easily.
641 2018-02-01 20:12:45	0|gmaxwell|I guess one question is about "error", there are "peer violated the protocol" sorts of errors, and "omg our state is corrupted" sorts of errors.
642 2018-02-01 20:13:04	0|wumpus|ERROR would be potentially dangerous but not fatal errors, maybe WARNING is a better name
643 2018-02-01 20:13:20	0|wumpus|peer violated the protocol is INFO imo
644 2018-02-01 20:13:26	0|wumpus|it's not dangerous to us
645 2018-02-01 20:13:42	0|gmaxwell|We might want to adapt our language to call the first things "abnormal" (e.g. info level log, and the log text should not use the word error but perhaps use the word abnormal).
646 2018-02-01 20:14:46	0|wumpus|right
647 2018-02-01 20:15:23	0|provoostenator|I'm switching my testnet seed back on tomorrow unless something really surprising happens.
648 2018-02-01 20:16:21	0|provoostenator|Or I can do it in 20 minutes if we want rc1 complaints to stop coming in.
649 2018-02-01 20:16:21	0|wumpus|but anyhow, if you come up with a specific combination of categories that would be useful as single -debug= option, I wouldn't be against it
650 2018-02-01 20:17:33	0|wumpus|provoostenator: we should just tag r2
651 2018-02-01 20:18:34	0|provoostenator|I'll know in ~10 minutes whether todays fixex made my crash go away (fairly certain it did).
652 2018-02-01 20:20:26	0|gmaxwell|wumpus: okay.  Well right now I think all minus leveldb internals is useful, at that seems like what a lot of us are using much of the time.
653 2018-02-01 20:20:58	0|wumpus|gmaxwell: I think it needs a better definition though, something like debug=lowtraffic
654 2018-02-01 20:21:20	0|wumpus|not everythingbutleveldb lol
655 2018-02-01 20:21:42	0|wumpus|it also gives guidance for people to add or not categories to that in the future
656 2018-02-01 20:22:00	0|wumpus|or to remove categories once they become noisy
657 2018-02-01 20:22:10	0|gmaxwell|well net is high traffic but useful, leveldb is high traffic but so far I've not found it useful.
658 2018-02-01 20:22:32	0|wumpus|but what is the rationale of the combination then?
659 2018-02-01 20:22:55	0|gmaxwell|omit useless chatty things.
660 2018-02-01 20:23:33	0|wumpus|let's kill leveldb logging completely if it's so useless
661 2018-02-01 20:24:20	0|wumpus|libevent too, probably
662 2018-02-01 20:24:27	0|meshcollider|it seems my gitian issue lies when it is trying to download zeromq-4.2.2.tar.gz for bitcoincore.org/depends-sources
663 2018-02-01 20:24:28	0|wumpus|it can be even more hilariously useless
664 2018-02-01 20:24:37	0|gmaxwell|It's at least of conjectural use e.g. if we were chasing some leveldb internal bug.
665 2018-02-01 20:25:08	0|wumpus|oh sure it can be useful, esp when adding custom debugging to leveldb to troubleshoot some issue
666 2018-02-01 20:25:17	0|wumpus|that's why a bridge exists
667 2018-02-01 20:25:22	0|cfields|meshcollider: you probably had all other sources already and your lxc is failing to make any net connections
668 2018-02-01 20:25:31	0|gmaxwell|I mean we could also leave the code for the bridge there but remove the logging level.
669 2018-02-01 20:25:49	0|meshcollider|cfields: that's true, was the version of zeromq bumped for 0.16
670 2018-02-01 20:26:01	0|cfields|believe so
671 2018-02-01 20:26:10	0|cfields|yes, it was
672 2018-02-01 20:26:36	0|gmaxwell|I opened the conversation basically with the idea of having leveldb (and maybe later other things) being log categories you never get unless you explicitly ask for them.
673 2018-02-01 20:26:48	0|gmaxwell|maybe libevent would fall into that too.
674 2018-02-01 20:27:14	0|gmaxwell|I've noticed it being useless too but just never had it be chatty enough to bother me.
675 2018-02-01 20:27:18	0|wumpus|so I still think =lowtraffic makes sense
676 2018-02-01 20:27:37	0|gmaxwell|I suppose we could lowtraffic then turn back on net-messages.
677 2018-02-01 20:28:12	0|gmaxwell|(I mean to achieve my normal desired logging config which is basically low traffic things plus net messages)
678 2018-02-01 20:28:12	0|wumpus|we should just include net int hat
679 2018-02-01 20:28:38	0|gmaxwell|I'm imagining net divided into per-message logging and the rest (e.g. connections)
680 2018-02-01 20:28:40	0|wumpus|with the future remark that if there is a low-traffic net category, that should be in instead
681 2018-02-01 20:28:46	0|gmaxwell|right
682 2018-02-01 20:29:09	0|meshcollider|cfields: yep you're right, it works fine if I make the depends directory beforehand, the new zeromq downloaded successfully
683 2018-02-01 20:29:33	0|cfields|great
684 2018-02-01 20:29:49	0|meshcollider|cfields: thanks :)
685 2018-02-01 20:29:56	0|cfields|np
686 2018-02-01 20:30:20	0|gmaxwell|Another thing to think about is the performance impacts of logging, some of our logs cause computation that is probably pretty bad for performance.
687 2018-02-01 20:32:32	0|wumpus|I think that's true for libevent and leveldb logging too, they require a special enabling flag, which causes those libraries to send the messages at all
688 2018-02-01 20:33:37	0|wumpus|gmaxwell: that'd only be problematic for high-volume messages, I'm sure e.g. computing the BENCH messages takes some cycles, but they only happen once per block
689 2018-02-01 20:34:04	0|wumpus|and in the total validation tme that's probably neglible
690 2018-02-01 20:34:46	0|gmaxwell|The leveldb stuff looks kind of expensive.
691 2018-02-01 20:34:49	0|wumpus|I don't think we have low-traffic messages that take significant computation
692 2018-02-01 20:35:18	0|gmaxwell|And I recall that there were some moderate traffic messages that did stuff like an extra iteration over all inputs in transactions or something.
693 2018-02-01 20:35:18	0|wumpus|for high traffic, yes, I woudln't be surprised if the net logging slowed some things down
694 2018-02-01 20:35:52	0|wumpus|if you have to write a message for every incoming packet to a file, it becomes disk bound
695 2018-02-01 20:36:35	0|wumpus|oh I didn't know that
696 2018-02-01 20:39:23	0|zelest|sorry for asking in here, but I did some quick googling and it seems like OneFixt is known in here? May I ask who he/she is? :o
697 2018-02-01 20:40:24	0|wumpus|can we have some review on #12329 please, I'd like to tag rc2 before I go to bed
698 2018-02-01 20:40:26	0|gribble|https://github.com/bitcoin/bitcoin/issues/12329 | net: dont retry failed oneshot connections forever by theuni · Pull Request #12329 · bitcoin/bitcoin · GitHub
699 2018-02-01 20:50:42	0|wumpus|and that's the last one
700 2018-02-01 21:51:13	0|sipa|oops, seems i accidentally left this channel and forgot about the meeting as well
701 2018-02-01 22:13:57	0|luke-jr|sipa: there was discussion of how to handle Segwit-back-to-the-start type stuff, and I thought perhaps it would be better as an annex to the Segwit BIP(s) rather than an entirely new BIP (making 2 BIPs for each fork); would you be okay with that, as an author of the BIP in question?
702 2018-02-01 22:16:08	0|gmaxwell|and then you're gonna add (hardfork) to the segwit bips and collect your cheque from Ver? :P
703 2018-02-01 22:17:21	0|jnewbery|re logging - we already have an alias -debug=all which aliases to -debug=1 (~0). Doesn't seem like to much of a stretch to add a new alias debug=gmax* which aliases to all the categories that greg wants to see (*name TBD)
704 2018-02-01 22:18:01	0|jnewbery|perhaps -debug=standard , -debug=default, -debug=...
705 2018-02-01 22:18:55	0|jnewbery|I also agree that logging should log to a circular buffer in memory and then have a background thread flushing to disk. I bet there are plenty of places where we're logging to disk while holding cs_main for example
706 2018-02-01 22:21:10	0|jnewbery|(in fact I started implementing that a few months ago, but never finished)
707 2018-02-01 22:22:34	0|gmaxwell|I'd like to get to a state where our logs by default can log virutally nothing, and on fault we can dump a good amount of context.
708 2018-02-01 22:24:01	0|gmaxwell|this also would address a lot of privacy concerns, where we avoid logging detailed data that would make node logs attractive for trying to trace transactions.
709 2018-02-01 22:24:11	0|luke-jr|gmaxwell: as an annex, it's clearer to consider it an implementation detail ;)
710 2018-02-01 22:24:50	0|luke-jr|ie, "you can tighten the rules using method A or method B, and the outcome is the same"