1 2017-01-02 02:55:40	0|da2ce7|Happy new year! :)
  2 2017-01-02 06:48:05	0|btcdrak|wumpus: can we tag 0.13.2? :-)
  3 2017-01-02 08:37:32	0|wumpus|btcdrak: yes
  4 2017-01-02 08:37:46	0|bitcoin-git|13bitcoin/06master 141d2d676 15Wladimir J. van der Laan: qt: Set transifex slug to 0.14...
  5 2017-01-02 08:37:46	0|bitcoin-git|[13bitcoin] 15laanwj pushed 1 new commit to 06master: 02https://github.com/bitcoin/bitcoin/commit/1d2d67692c74a5539f3736754d84f0aa6a702c56
  6 2017-01-02 08:43:46	0|bitcoin-git|[13bitcoin] 15jonasschnelli pushed 4 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/1d2d67692c74...53442af0aac3
  7 2017-01-02 08:43:47	0|bitcoin-git|13bitcoin/06master 1409aefb5 15Cory Fields: build: Fix 'make deploy' for OSX...
  8 2017-01-02 08:43:47	0|bitcoin-git|13bitcoin/06master 14b01667c 15Jonas Schnelli: Mention RSVG dependency when creating the disk image on OSX
  9 2017-01-02 08:43:48	0|bitcoin-git|13bitcoin/06master 142fb98f6 15Don Patterson: Fix bug in dmg builder so that it actually reads in the configuration file
 10 2017-01-02 08:44:03	0|bitcoin-git|[13bitcoin] 15jonasschnelli closed pull request #9412: build: Fix 'make deploy' for OSX (06master...062016/12/fix_mac_deploy) 02https://github.com/bitcoin/bitcoin/pull/9412
 11 2017-01-02 08:55:13	0|bitcoin-git|13bitcoin/060.13 140d71914 15Wladimir J. van der Laan: doc: Remove ... from release notes
 12 2017-01-02 08:55:13	0|bitcoin-git|[13bitcoin] 15laanwj pushed 1 new commit to 060.13: 02https://github.com/bitcoin/bitcoin/commit/0d719145b018e28d48d35c2646a5962b87c60436
 13 2017-01-02 08:57:36	0|wumpus|* [new tag]         v0.13.2 -> v0.13.2
 14 2017-01-02 09:09:48	0|jonasschnelli|\o/
 15 2017-01-02 09:11:18	0|wumpus|*almost* new year tag :p
 16 2017-01-02 09:24:29	0|jouke_|I updated a .10 node to .13.1. It connects to an other .13.1 node, but it doesn't seem to sync the blockchain. But when I look at the blockheight, it might already got "stuck" when it was a .10 node. I updated it, because some rpc calls were a bit slow.
 17 2017-01-02 09:27:51	0|jouke_|In debug log I see that it is receiving "invs".
 18 2017-01-02 09:35:38	0|gmaxwell|jouke_: if you were stuck it is presumably because corruption made you reject a block. you can try running 'reconsiderblock <block hash after your current height>'
 19 2017-01-02 09:40:10	0|jouke_|I retreived the blockhash from an other node, but that doesn't seem to do anything.
 20 2017-01-02 09:42:24	0|gmaxwell|jouke_: I assume you don't have historic logs from the point where it got stuck?
 21 2017-01-02 09:42:41	0|jouke_|no, I don't unfortunately
 22 2017-01-02 09:46:16	0|jouke_|It just received information about the latest block and I see it is retreiving and receiving the headers of that block, but that's it.
 23 2017-01-02 09:52:21	0|jouke_|Hmm, we use that node to gather information about mempool size, and it stopped accepting blocks at around the 29th, but it did receive transactions which it added to it's mempool.
 24 2017-01-02 09:53:09	0|gmaxwell|jouke_: do you have logs that cover the 29th?
 25 2017-01-02 09:54:23	0|jouke_|no
 26 2017-01-02 09:59:08	0|gmaxwell|the reconsiderblock did nothing? can you try invalidateblock <thatblock> then reconsider?
 27 2017-01-02 09:59:46	0|gmaxwell|can you also run getchaintips and pastebin the results?
 28 2017-01-02 09:59:57	0|jouke_|And thatblock = currentheight+1 right?
 29 2017-01-02 10:00:12	0|gmaxwell|right what is your current height?
 30 2017-01-02 10:00:33	0|jouke_|It's stuck at 445623
 31 2017-01-02 10:01:16	0|gmaxwell|can you getblockhash for 445623 ?
 32 2017-01-02 10:02:13	0|jouke_|000000000000000002c45ed9057608b268a447499fcd8bea8e7930bfed11eaaf
 33 2017-01-02 10:03:29	0|gmaxwell|jouke_: okay run reconsiderblock 000000000000000000fc97c4cd1f956f46a420536ca4851c31ad9d32f4e8ec0c
 34 2017-01-02 10:03:58	0|jouke_|already did that
 35 2017-01-02 10:04:03	0|jouke_|Didn't do anything
 36 2017-01-02 10:04:36	0|gmaxwell|jouke_: can you try doing a getblock 000000000000000000fc97c4cd1f956f46a420536ca4851c31ad9d32f4e8ec0c and see if it says anything?  grep your logs for that hash and see if there is any mention?
 37 2017-01-02 10:05:08	0|gmaxwell|unfortunately I think there are types of rejection that reconsider will not reconsider, it's really just intended to be the compliment to invalidateblock
 38 2017-01-02 10:05:43	0|jouke_|Can't read block from disk
 39 2017-01-02 10:05:43	0|jouke_|error code: -32603
 40 2017-01-02 10:05:43	0|jouke_|error message:
 41 2017-01-02 10:06:26	0|jouke_|chaintips: https://www.zerobin.net/?70af58e5db340709#1i/w7FQhQE0ubu1+9QO5ANJPRSJ+V24C0LgiYVl15gU=
 42 2017-01-02 10:06:27	0|gribble|https://github.com/bitcoin/bitcoin/issues/1 | JSON-RPC support for mobile devices ("ultra-lightweight" clients) · Issue #1 · bitcoin/bitcoin · GitHub
 43 2017-01-02 10:07:30	0|gmaxwell|jouke_: okay your blockchain data is corrupted. the block isn't on your disk (or is corrupted on disk) but is in the blockindex.
 44 2017-01-02 10:08:43	0|jouke_|Hmm. nothing strange in syslog about file corruption or anything.
 45 2017-01-02 10:09:54	0|gmaxwell|any chance you had a unclean power loss? the block isn't added into the block index until its written to disk and fsynced, but if your system is ignoring fsyncs or there is some other misbehavior in your storage it could be lost during a power failure.
 46 2017-01-02 10:10:31	0|jouke_|no, system has been up for a whie
 47 2017-01-02 10:10:32	0|jouke_|while
 48 2017-01-02 10:10:55	0|gmaxwell|oh this was 0.10 before?
 49 2017-01-02 10:10:58	0|jouke_|yes
 50 2017-01-02 10:11:04	0|gmaxwell|ah ha.
 51 2017-01-02 10:11:25	0|gmaxwell|I believe we actually had a bug that could cause that before.
 52 2017-01-02 10:12:11	0|jouke_|I was already looking through the issues, but I did not come across it yet.
 53 2017-01-02 10:13:29	0|gmaxwell|jouke_: in any case, I believe there is no other resolution right now to your condition but to reindex.  setting a higher dbcache than default, if you have the ram will make it run much faster.
 54 2017-01-02 10:14:34	0|jouke_|Shame I don't have the logs. I am making a note to push all bitcoin node logs to a log server.
 55 2017-01-02 10:14:48	0|jouke_|gmaxwell: thanks for the help!
 56 2017-01-02 11:36:13	0|bitcoin-git|[13bitcoin] 15MarcoFalke opened pull request #9457: WIP: [qt] translate bitcoin.cpp and libbitcoin_common_a_SOURCES (06master...06Mf1701-qtTrans) 02https://github.com/bitcoin/bitcoin/pull/9457
 57 2017-01-02 12:52:09	0|jonasschnelli|Hmm... full block SPV with random peers is horrible slow. 10-12 blocks per minute around height 445100
 58 2017-01-02 12:52:21	0|jonasschnelli|(on a 100MBit link)
 59 2017-01-02 12:52:48	0|jonasschnelli|Maybe I just had bad luck with remote peers
 60 2017-01-02 12:52:52	0|gmaxwell|increase your socket recieve buffer maximum size.
 61 2017-01-02 12:53:10	0|gmaxwell|by like a factor of 10, you'll likely see it much faster.
 62 2017-01-02 12:53:37	0|gmaxwell|Does your fetching code fetch in parallel?
 63 2017-01-02 12:54:19	0|jonasschnelli|Yes. Same code.
 64 2017-01-02 12:54:28	0|jonasschnelli|Maybe I need to unset NODE_NETWORK?
 65 2017-01-02 12:54:38	0|jonasschnelli|debug=net shows plenty of tx invs
 66 2017-01-02 12:55:50	0|gmaxwell|increase your socket recieve buffer maximum size.
 67 2017-01-02 12:56:52	0|gmaxwell|One problem (no doubt there is more) is that it cannot achieve high throughput for transfers because the recieve buffers are so small that almost any processing delay and they fill.
 68 2017-01-02 12:57:15	0|jonasschnelli|gmaxwell: Thanks. I'll try that.
 69 2017-01-02 13:23:25	0|jonasschnelli|gmaxwell: -maxreceivebuffer=50000 = 1block per second. Better,... need to profile.. but I guess SyncTransaction with IsMine, etc. is the bottleneck
 70 2017-01-02 13:26:05	0|gmaxwell|thanks for checking and, it sounds like, validating my theory.  Not just how long sync takes, but the block deseralization isn't fast, ans I assume your sync runs in order, which means that when blocks arrive out of order and eventually the missing one shows up, it is probably processing many blocks at once, which creates a long delay in the message handling loop which will stall the recieve buf
 71 2017-01-02 13:26:11	0|gmaxwell|fers.
 72 2017-01-02 13:27:25	0|jonasschnelli|The blockdownload im using (#9171) does download in parallel, then store them on the disk and, process them in order (unserialize again probably).
 73 2017-01-02 13:27:27	0|gribble|https://github.com/bitcoin/bitcoin/issues/9171 | Introduce auxiliary block requests by jonasschnelli · Pull Request #9171 · bitcoin/bitcoin · GitHub
 74 2017-01-02 13:29:28	0|gmaxwell|right, and say that you recieve blocks 0 2 3 4 5 6 7 8 9 1 will it end up processing 9 of them at the moment it recieves block 1 in the message handler? if so... it is not emptying the recieve buffers during that time.
 75 2017-01-02 13:30:46	0|jonasschnelli|gmaxwell: I think it does empty the receive buffer,... ProcessNewBlock gets called for all blocks where only the header gets verified, the stored to the disk. I guess this drains the buffer...
 76 2017-01-02 13:30:56	0|jonasschnelli|*then
 77 2017-01-02 13:32:15	0|gmaxwell|the recieve buffer is drained as the blocks are saved to disk. it saves 1 to disk but then processes it and then will connect 1 2 3 4 5 6 7 8 9  all within that function call.
 78 2017-01-02 13:32:26	0|gmaxwell|and during that time the message handler is not handling any messages. :)
 79 2017-01-02 13:32:37	0|gmaxwell|so the recieve buffers fill.
 80 2017-01-02 13:33:42	0|jonasschnelli|hmmm...
 81 2017-01-02 13:33:57	0|jonasschnelli|https://github.com/bitcoin/bitcoin/pull/9171/files#diff-6905024871a5ac748c4465bb25973c39R37
 82 2017-01-02 13:36:51	0|jonasschnelli|Ah. Right... all the processing (LoadBlock, SyncTransaction, etc.) is done in thread "bitcoin-msghandler".
 83 2017-01-02 13:39:02	0|gmaxwell|which fine, but not if it's processing a bunch of blocks at once.
 84 2017-01-02 13:39:36	0|gmaxwell|e.g. it should just process two and then return control to the message handler loop.
 85 2017-01-02 13:39:58	0|jonasschnelli|Okay.. I'll try that. Is 16 atm.
 86 2017-01-02 13:40:02	0|jonasschnelli|*It's
 87 2017-01-02 13:41:04	0|gmaxwell|where is it limited to 16?  like.. if you get 0 2 3 4 5 ... 999 1 doesn't end up processing 1000 at once?
 88 2017-01-02 13:41:17	0|jonasschnelli|I guess an explicit thread "process spv" (or similar) doesn't make that much sense. SyncTransaction does hold cs_main anyways.
 89 2017-01-02 13:41:48	0|jonasschnelli|16 is the limit for the max amount of blocks per auxiliary request
 90 2017-01-02 13:42:03	0|jonasschnelli|If block 1 comes in last, it processes 15 blocks in a row.
 91 2017-01-02 13:42:22	0|jonasschnelli|Once the 16 are processed, it continues with the next batch of 16
 92 2017-01-02 13:42:32	0|jonasschnelli|(through findNextBlocksToDownload())
 93 2017-01-02 13:42:51	0|jonasschnelli|Ah. no, MAX_BLOCK_TO_PROCESS_PER_ITERATION == 5
 94 2017-01-02 13:44:49	0|gmaxwell|oh that isn't many blocks to have in flight at once.
 95 2017-01-02 13:45:03	0|gmaxwell|it should be possible to have many in flight but only process a few at a time.
 96 2017-01-02 13:45:28	0|gmaxwell|if you cut the number in flight your throughput will decrease due to round trip delays.
 97 2017-01-02 13:46:48	0|jonasschnelli|I just got in 16 blocks nice in order. It took more then 5s per block. I think its a slow peer.
 98 2017-01-02 13:47:34	0|gmaxwell|thats one of the reasons you want to have lots of blocks in flight at once. :)
 99 2017-01-02 13:48:23	0|jonasschnelli|Okay. Let me try that.
100 2017-01-02 13:48:39	0|jonasschnelli|16 is max per peer. Maybe the request should contain 16*16 or similar.
101 2017-01-02 13:49:20	0|gmaxwell|yes 16 max per peer is probably fine.
102 2017-01-02 13:56:37	0|jonasschnelli|gmaxwell: Oh yes. That was the bottleneck, not enough blocks in flight, parallelism was not ideal
103 2017-01-02 13:56:52	0|jonasschnelli|now 450 blocks in 1min.
104 2017-01-02 13:57:24	0|jonasschnelli|(at height 445761)
105 2017-01-02 14:02:30	0|MarcoFalke|wumpus: No, the translations were never added back
106 2017-01-02 14:02:54	0|MarcoFalke|As of today we are missing the ones that were moved to warnings.cpp
107 2017-01-02 14:03:03	0|MarcoFalke|(additionally)
108 2017-01-02 14:03:49	0|MarcoFalke|I don't understand the translation process code enough to fix this, but something like #9457 should do it
109 2017-01-02 14:03:50	0|gribble|https://github.com/bitcoin/bitcoin/issues/9457 | WIP: [qt] translate bitcoin.cpp and libbitcoin_common_a_SOURCES by MarcoFalke · Pull Request #9457 · bitcoin/bitcoin · GitHub
110 2017-01-02 14:16:45	0|gmaxwell|oh interesting, I didn't really realize translations were file specific, I thought they were string to string conversions?
111 2017-01-02 14:17:49	0|MarcoFalke|It is an issue with some files are not picked up by the translation process code
112 2017-01-02 14:20:28	0|gmaxwell|oh I see.
113 2017-01-02 15:06:05	0|gribble|https://github.com/bitcoin/bitcoin/issues/9289 | net: drop boost::thread_group by theuni · Pull Request #9289 · bitcoin/bitcoin · GitHub
114 2017-01-02 15:06:06	0|gribble|https://github.com/bitcoin/bitcoin/issues/9419 | Stop Using cs_main for CNodeState/State() by TheBlueMatt · Pull Request #9419 · bitcoin/bitcoin · GitHub
115 2017-01-02 15:06:08	0|gribble|https://github.com/bitcoin/bitcoin/issues/9375 | Relay compact block messages prior to full block connection by TheBlueMatt · Pull Request #9375 · bitcoin/bitcoin · GitHub
116 2017-01-02 15:26:24	0|gmaxwell|Is there a reason that make check doesn't run qa/pull-tester/rpc-tests.py  ?
117 2017-01-02 15:26:40	0|gmaxwell|the tests in that are far more meaningful than most of the unit tests, IMO.
118 2017-01-02 15:27:57	0|BlueMatt|someone was complaining the other day that there is no way (afaik) to run rpc-tests from make ...
119 2017-01-02 16:58:07	0|f0nky|hi all, can i ask u guys a question?
120 2017-01-02 20:29:25	0|achow101|are the 0.13.2 detached sigs ready yet?
121 2017-01-02 20:41:48	0|cfields|achow101: running sanity tests. eta ~10 min.
122 2017-01-02 20:50:27	0|cfields|gitian builders: 0.13.2 detached sigs pushed
123 2017-01-02 20:56:35	0|luke-jr|crap, forgot to save a copy of the intermediate files
124 2017-01-02 22:00:09	0|bitcoin-git|[13bitcoin] 15isle2983 opened pull request #9459: Improvements to copyright_header.py and some minor copyright header tweaks. (06master...06PR-copyright-script-improve) 02https://github.com/bitcoin/bitcoin/pull/9459
125 2017-01-02 23:01:44	0|cfields|BlueMatt: right, so does making CNodeState::cs static not enforce the only-one-nodestate-at-once rule?