1 2017-10-24 02:35:26 0|mryandao|hey, my bitcoind just crashed while its processing datadir, i attached gdb to the process and saw the following system call.
2 2017-10-24 02:35:29 0|mryandao|__pthread_disable_asynccancel () at ../sysdeps/unix/sysv/linux/x86_64/cancellation.S:88
3 2017-10-24 02:35:32 0|mryandao|88 ../sysdeps/unix/sysv/linux/x86_64/cancellation.S: No such file or directory.
4 2017-10-24 02:35:35 0|mryandao|is this the right place to report this?
5 2017-10-24 02:49:46 0|achow101|mryandao: make an issue
6 2017-10-24 02:52:21 0|mryandao|ok
7 2017-10-24 02:52:36 0|mryandao|i'll run bitcoind again and see if crashes again
8 2017-10-24 06:24:55 0|gmaxwell|in #11534, is there a reason to use connect time rather than highest node id?
9 2017-10-24 06:24:56 0|gribble|https://github.com/bitcoin/bitcoin/issues/11534 | Evict outbound peers if tip is stale by sdaftuar ÷ Pull Request #11534 ÷ bitcoin/bitcoin ÷ GitHub
10 2017-10-24 06:25:18 0|gmaxwell|node ID should be monotone and increasing, so it should if anything be a better latest peer criteria than connect time.
11 2017-10-24 11:49:06 0|sdaftuar|gmaxwell: thanks, that sounds better
12 2017-10-24 11:49:48 0|sdaftuar|gmaxwell: i thought of another problem with that patch, that i'm not sure how to best address
13 2017-10-24 11:50:29 0|sdaftuar|it seems suboptimal to disconnect a peer that hasn't been connected very long -- eg perhaps your tip is stale, but the peer you're choosing to evict is one you just connected to
14 2017-10-24 11:51:38 0|sipa|sdaftuar: years ago i experimented with a outbound connection rotation system, which kicked just based on chances that depend on connection time
15 2017-10-24 11:51:40 0|sdaftuar|or, perhaps your network was down for a few hours (say, you're running on a laptop) and then when you start up, you detect a stale tip -- so you immediately flag one of your initial 8 peers for eviction, for basically no good reason
16 2017-10-24 11:52:19 0|sipa|i believe a good approach was (by simulations, but i've lost all the code so it's hard to reproduce) to make the chance to disconnect proportional to time_connected^-0.8
17 2017-10-24 11:54:24 0|sipa|sdaftuar: well if we're considering something like peer rotation anyway (at least, i am), then occasionally randomly disconnecting someone for no reason isn't too bad
18 2017-10-24 11:54:25 0|sdaftuar|sipa: i haven't given a ton of thought to how to measure success/failure of general-purpose algorithms like that...
19 2017-10-24 11:54:55 0|sipa|at least if it only happens occasionally
20 2017-10-24 11:55:19 0|sdaftuar|ah, ok
21 2017-10-24 11:55:43 0|sdaftuar|well, i think my PR won't behave *terribly*, under the assumption that it's okay for an occasional spurious disconnect
22 2017-10-24 11:55:52 0|sdaftuar|though it could certainly use testing
23 2017-10-24 11:56:08 0|sipa|and in general, your most recent connection is the least likely peer to be your best
24 2017-10-24 11:56:56 0|sdaftuar|right, agreed. i think my PR does have an edge-case bug, where eg your first peer could be the one disconnected (in the situation where your network is down, then comes back up, and you briefly have just one peer that is fully connected--
25 2017-10-24 11:56:58 0|sipa|so if we're aiming for simplicity, i don't think it'll be terrible under any circumstances
26 2017-10-24 11:57:12 0|sdaftuar|it could be flagged for eviction)
27 2017-10-24 11:58:11 0|sdaftuar|i was considering another approach to the same problem, where i just connect outbound to a 9th peer when the tip looks like it might be stale, and then at some point after that disconnect one
28 2017-10-24 11:58:20 0|sdaftuar|with the idea that if no new info was learned from the 9th, then it should be the one disconnected
29 2017-10-24 11:58:57 0|sdaftuar|i think that approach has the advantage of (a) being able to trigger more often, because it seems safer to connect to an extra peer than disconnect an existing one, so you can test for stale-ness with less conservatism
30 2017-10-24 11:59:14 0|sipa|right
31 2017-10-24 11:59:24 0|sdaftuar|and (b) it seems like the outbound peer selection logic will perform better if you connect to a new peer before disconnecting an old peer (due to the netgroup diversity requirement? if i understand it right)
32 2017-10-24 11:59:52 0|sdaftuar|but i haven't quite worked out the implementation details yet
33 2017-10-24 12:20:04 0|bitcoin-git|[13bitcoin] 15ryanofsky opened pull request #11552: Improve wallet-accounts test (06master...06pr/acctt) 02 https://github.com/bitcoin/bitcoin/pull/11552
34 2017-10-24 13:55:10 0|aj|jtimon: hmm, googling old issues doesn't seem like there's much consensus on changing RPC from decimal BTC values? #9855 was the most recent i saw, i think
35 2017-10-24 13:55:12 0|gribble|https://github.com/bitcoin/bitcoin/issues/9855 | RPC: Use integer satoshis instead BTC with decimals by jtimon ÷ Pull Request #9855 ÷ bitcoin/bitcoin ÷ GitHub
36 2017-10-24 13:57:07 0|jtimon|perhaps that's a good topic for the next meeting
37 2017-10-24 13:57:55 0|jtimon|it was my understanding that everybody considered using BTC instead of satoshis a mistake, but perhaps it's just me and I'm just assuming the opinions of others
38 2017-10-24 14:00:39 0|aj|jtimon: if i'm understanding what they're saying, sipa and wumpus seem to think string vs number is more important than btc vs satoshi
39 2017-10-24 14:01:03 0|aj|jtimon: https://github.com/bitcoin/bitcoin/pull/3759/commits/f3f8460fe51be719231612a921dd37af638df46a seems like a good idea to me?
40 2017-10-24 16:34:38 0|jtimon|aj: I also tried https://github.com/bitcoin/bitcoin/pull/9855 but it wasn't very well received
41 2017-10-24 16:35:21 0|jtimon|also I think string vs numbers is more important when you have decimals. for integer satoshis I don't think anyone has a problem with them being numbers
42 2017-10-24 16:35:29 0|jtimon|perhaps I'm wrong there too, not sure
43 2017-10-24 16:36:02 0|jtimon|but for the getblockstats pr, I really think using integer satoshis is the right thing to do
44 2017-10-24 16:36:16 0|jtimon|or the best option
45 2017-10-24 17:02:11 0|bitcoin-git|[13bitcoin] 15Varunram opened pull request #11553: [P2P] Throw Warning if -peerbloomfilters is enabled (06master...06peerfilters) 02 https://github.com/bitcoin/bitcoin/pull/11553
46 2017-10-24 17:03:57 0|Varunram|Hey guys! This is my first PR, please pardon me if I've done something wrong or haven't abided by any rules. Looking forward to your comments :)
47 2017-10-24 18:12:25 0|bitcoin-git|[13bitcoin] 15TheBlueMatt opened pull request #11554: Sanity-check script sizes in bitcoin-tx (06master...062017-10-bitcoin-tx-script-sizes) 02 https://github.com/bitcoin/bitcoin/pull/11554
48 2017-10-24 18:14:34 0|BlueMatt|Varunram: heh, thanks for the pr, though sadly I dont think we can do that yet :(
49 2017-10-24 18:15:37 0|BlueMatt|Varunram: note that the -peerbloomfilters option is mostly to prevent dos issues for nodes which cannot take the potential latency increases, and not to encourage people to stop providing bloom filters until we have a viable alternative :/
50 2017-10-24 18:18:48 0|Varunram|BlueMatt: haha, no worries :) I was going through the issues with "good first issue" labels on them and this seemed to be a good place to start. And yeah, we certainly don't want people to stop providing bloom filters, that would take a hit on SPV clients as a whole
51 2017-10-24 18:19:21 0|BlueMatt|heh, yea, maybe we should be better about tags...
52 2017-10-24 18:19:29 0|BlueMatt|someone should probably remove the "good first issue" tag there....
53 2017-10-24 18:19:36 0|Varunram|I'll rummage through the issues repo and contribute something more useful, meanwhile do I close the PR or?
54 2017-10-24 18:20:19 0|BlueMatt|I mean you dont have to, but it'll sit for 6/12 months if you dont "waiting on someone to add a viable alternative"....
55 2017-10-24 18:20:19 0|Varunram|I had a slight notion that it might be controversial, but gave it a shot (yolo lol)
56 2017-10-24 18:20:22 0|BlueMatt|so probably easier to...
57 2017-10-24 18:21:39 0|Varunram|got it, thanks!
58 2017-10-24 18:25:43 0|bitcoin-git|[13bitcoin] 15Varunram closed pull request #11553: [P2P] Throw Warning if -peerbloomfilters is enabled (06master...06peerfilters) 02 https://github.com/bitcoin/bitcoin/pull/11553
59 2017-10-24 18:29:26 0|jb55|seconded on adding more good first issue tags, I'm also interested in getting my feet wet..
60 2017-10-24 19:37:16 0|cfields|sdaftuar: sorry for the late review
61 2017-10-24 19:59:25 0|luke-jr|dnsseed.bitcoin.dashjr.org is now servicebit-filter-enabled
62 2017-10-24 19:59:46 0|sdaftuar|cfields: thanks for taking a look!
63 2017-10-24 20:00:47 0|cfields|sdaftuar: don't kill me for that last comment... I know 11534 was my suggestion :\
64 2017-10-24 20:02:06 0|sdaftuar|no problem, thanks for thinking about it. not sure if you saw the scrollback, but i'm actually working on an alternate implementation where we'd add a new peer first, and then figure out who to disconnect
65 2017-10-24 20:02:31 0|sdaftuar|i *think* that will turn out to be simpler... but the code isn't done yet, so too early for me to be sure
66 2017-10-24 20:02:43 0|sdaftuar|testing this stuff is all awful of course :(
67 2017-10-24 20:04:06 0|sdaftuar|anyway i think #11490 is closer to done, so i'll address your comments there and hopefully we can get some final acks soon
68 2017-10-24 20:04:09 0|gribble|https://github.com/bitcoin/bitcoin/issues/11490 | Disconnect from outbound peers with bad headers chains by sdaftuar ÷ Pull Request #11490 ÷ bitcoin/bitcoin ÷ GitHub
69 2017-10-24 20:04:19 0|cfields|no, i didn't see that
70 2017-10-24 20:04:23 0|cfields|reading
71 2017-10-24 20:07:07 0|gmaxwell|sdaftuar: I like adding a new peer first; then we won't potentially spend a long time with one fewer peer.
72 2017-10-24 20:07:27 0|cfields|I see. The idea being that you may disconnect a node just to connect to his neighbor. But if you have a new connection first, you can make a better decision
73 2017-10-24 20:08:18 0|sdaftuar|cfields: yeah. i don't know how compelling that reasoning is, but that plus not being able to be more aggressive about checking whether your tip is stale both seem like good reasons
74 2017-10-24 20:08:36 0|sdaftuar|and what gmaxwell said ^
75 2017-10-24 20:21:42 0|gmaxwell|this was, fwiw, how I always envisioned outbound rotation working, temporarily allowing one extra peer, then deciding which one to axe after you've talked to the new one.
76 2017-10-24 20:56:07 0|cfields|sdaftuar: so should i let #11534 rest until you've tried out your new approach?
77 2017-10-24 20:56:09 0|gribble|https://github.com/bitcoin/bitcoin/issues/11534 | Evict outbound peers if tip is stale by sdaftuar ÷ Pull Request #11534 ÷ bitcoin/bitcoin ÷ GitHub
78 2017-10-24 20:57:28 0|sdaftuar|cfields: yeah maybe give me another day or so to try to get a new approach working, and then we can compare and see which looks more viable?
79 2017-10-24 20:57:36 0|cfields|ok, np
80 2017-10-24 20:57:37 0|sdaftuar|so you don't waste your time on an approach we abandon
81 2017-10-24 20:59:37 0|cfields|also, if our p2p was more spec'd out, I'd grumpily ask for a bip for #11490. But I guess it's not worth it since we already make a bunch of assumptions.
82 2017-10-24 20:59:39 0|gribble|https://github.com/bitcoin/bitcoin/issues/11490 | Disconnect from outbound peers with bad headers chains by sdaftuar ÷ Pull Request #11490 ÷ bitcoin/bitcoin ÷ GitHub
83 2017-10-24 21:00:33 0|sdaftuar|i'm not averse to documentation, but do you think it needs a bip? i feel like peer selection is something that doesn't really require coordination across implementations
84 2017-10-24 21:01:36 0|cfields|nah, i don't think it's really needed
85 2017-10-24 21:02:05 0|cfields|not the peer selection though, the "you must send MAX_HEADERS_RESULTS or be disconnected" rule.
86 2017-10-24 21:02:33 0|sdaftuar|oh, i forgot about that
87 2017-10-24 21:03:19 0|gmaxwell|We alrady have a requirement for the max results, since our headers syncinc chaining depends on it, no?
88 2017-10-24 21:03:37 0|sdaftuar|gmaxwell: yeah, that's basically how i looked at it
89 2017-10-24 21:04:26 0|sdaftuar|i think matt observed that if we added a one-off getheaders message (eg with a hashstop) before we'd completed chain sync from a peer for some reason, then that would interact badly with the new logic
90 2017-10-24 21:04:41 0|gmaxwell|since it's automatic outbound only it wouldn't harm monitoring tools or such that don't really implement the protocol correctly.
91 2017-10-24 21:04:43 0|sdaftuar|but we don't do that... and i don't know why we would
92 2017-10-24 21:05:05 0|cfields|ah, so it does
93 2017-10-24 21:32:19 0|gmaxwell|sdaftuar: another point why its important that we don't evict half our outbound peers in 11490. Consider, say there is a new softfork we don't know about. At the moment, a chain violating it is a block ahead. We don't want the non-upgraded network to completely partition the upgraded network, because eventually the upgraded network will get ahead and we will switch to it.
94 2017-10-24 21:32:54 0|sdaftuar|gmaxwell: great point. i should add a comment
95 2017-10-24 21:33:47 0|sdaftuar|hmm, do you think protecting the first 4 who give us headers is reasonable then?
96 2017-10-24 21:34:02 0|sdaftuar|(afk, back later)
97 2017-10-24 21:37:56 0|BlueMatt|google translate appears to think #11555 is the guy asking how to create his own altcoin....someone wanna close it?
98 2017-10-24 21:37:57 0|gribble|https://github.com/bitcoin/bitcoin/issues/11555 | ßÞÃÅÞÃâÃËâÃâ¢!!!!!!! ÷ Issue #11555 ÷ bitcoin/bitcoin ÷ GitHub
99 2017-10-24 21:39:14 0|gmaxwell|sdaftuar: yes, probably.
100 2017-10-24 21:39:47 0|gmaxwell|sdaftuar: to be fair on my point, inbound connections would heal the partition, but I feel better if we can justify the behavior even in the impossible network of only hosts with outbound connections.
101 2017-10-24 21:58:21 0|meshcollider|BlueMatt: heh I like his/Google's phrasing, "altcoin come alive and go free swimming"
102 2017-10-24 22:05:28 0|esotericnonsense|does anyone have a good way to reproduce slow RPC behaviour? all I can think of at the moment is continually resyncing which doesn't seem like a good idea for my poor SSD. perhaps running a seperate thread that hits RPC with heavy queries over and over?
103 2017-10-24 22:35:40 0|aj|jtimon: i think the drawback for numbers vs strings is that even without a decimal point, "integers" are treated as doubles in javascript which makes it ambiguous at best in json generally
104 2017-10-24 22:38:02 0|gmaxwell|some json implementations manage to use floats for numbers too! :(
105 2017-10-24 22:38:28 0|jtimon|aj: yeah, perhaps all data in getblocksstats should be returned as strings, I still prefer satoshis over in BTC in that case
106 2017-10-24 22:41:26 0|jb55|I find myself using sed to wrap numbers in strings before I process them with jq...
107 2017-10-24 22:46:36 0|jtimon|gmaxwell: so what do you think should be done for https://github.com/bitcoin/bitcoin/pull/10757 ? all string satoshis, all string BTC ?
108 2017-10-24 22:51:05 0|jtimon|with that PR, I'm also kind of stuck with testing...I need to add at least one segwit tx to test some of the functionality, but some size and feerate functionality is currently not being tested due to sigs not being deterministic in size,I thought about putting hardcoded private keys in the test, but nobody confirmed that it makes any sense
109 2017-10-24 22:59:28 0|jb55|jtimon: wouldn't it make more sense to just to dump N verbose blocks and build analysis tools externally? I guess that would be a bit slower...
110 2017-10-24 23:04:58 0|jtimon|jb55: mhmm, I'm talking about the tests, but yeah, didn't thought about that...hardcoded blocks should do the trick and additionally also solve any potential concern with coin selection being non deterministic too, in case more complex tests are needed in the future. Great suggestion, now I feel bad about not thinking about it myself, but good about you not waiting for me to discover that possibility on my own
111 2017-10-24 23:08:20 0|jb55|just came to mind because I was trying to write a script that gave me the average block times the other day...
112 2017-10-24 23:13:05 0|jtimon|are there any other tests that use hardcoded blocks that I can copy from ? I guess the answer is no since https://github.com/bitcoin/bitcoin/pull/8994 which changes the genesis block for all functional tests is passing, but it was recently brought to my attention that hardcoding txs (including coinbases) should be enough even to construct a compatible even with a hf that completely changes the proof of work rules from genesis
113 2017-10-24 23:13:05 0|jtimon|block (such as regtest/custom) [isn't rusty around? it was him who reminded me in a completely unrelated conversation, I vaguely remind having known this before, or perhaps was just a deja vu]
114 2017-10-24 23:16:22 0|jtimon|or perhaps it's been too long since last time I tried the extended tests (including pruning)...
115 2017-10-24 23:16:48 0|jtimon|(for that PR)
116 2017-10-24 23:19:21 0|jtimon|I'll stop overtly and subconsciously review begging now...