1 2016-10-29 01:45:48	0|GitHub115|[13bitcoin] 15sipa opened pull request #9039: Various serialization simplifcations and optimizations (06master...06simpleserial) 02https://github.com/bitcoin/bitcoin/pull/9039
  2 2016-10-29 09:59:43	0|Bi|Guys, if I have a block with only the coinbase tx and one segwit tx. It takes sha256->sha256 on 64 bytes of data to calculate the witness' merkle - right?
  3 2016-10-29 09:59:59	0|Bi|And the first 32 bytes are zero?
  4 2016-10-29 10:00:35	0|Bi|While the other 32 bytes is sha256->sha256 of the raw segwit transaction?
  5 2016-10-29 10:01:36	0|Bi|It seems very simple, but I must be doing something wrong, beacuse my merkle roots dont match the expected values
  6 2016-10-29 10:02:12	0|Bi|Can someone please help me?
  7 2016-10-29 10:07:30	0|Bi|69baaa750c70eb352ebeadb07da0ff5bbd642cb285895ee43f2889dd12642ca0ff9b2489bd82d4d5ca287c4ebdf509f1bcd02d6fa720542e2900000000
  8 2016-10-29 10:07:30	0|Bi|Or maybe you could at leat tell me what is wtxid of this transaction: 01000000000101f15a2db447e8316e9f20073ab8bdeac53df1a53d415fbd41e73929699b889e7a0000000017160014fb459699ce9ca78de6d0790121dfb215883acb64ffffffff0100111024010000001976a914fb459699ce9ca78de6d0790121dfb215883acb6488ac02483045022100c5feadba5fbcc28afb940c7ee664bba81830f19a5407ae49cf75f565534b149702200a7c2b6b1b5846a37c434ffe0cae168d6b797de9087a21176f2c9ef6fa9f47aa014104b848ab6ac853cd
  9 2016-10-29 10:08:03	0|Bi|it is from this block: https://testnet.blockexplorer.com/block/0000000000000009fdda232a7dfb09682aeb35e04d1437b4ed6a08e9abb479e0
 10 2016-10-29 10:34:04	0|Victorsueca|Bi:  your transaction is supposed to have 2 IDs, the txid as it has always been and the new wtxid
 11 2016-10-29 10:34:55	0|Victorsueca|the txid should be sha256(sha256(original serialization format))
 12 2016-10-29 10:35:58	0|Victorsueca|the wtxid should be sha256(sha256(new serialization format with witness data))
 13 2016-10-29 10:36:52	0|Victorsueca|if your transaction does not have any witness data the txid and the wtxid should be the same
 14 2016-10-29 10:41:25	0|Bi|Victorsueca, yes I know all of this
 15 2016-10-29 10:42:07	0|Bi|Bu my merkle roots dont match :(
 16 2016-10-29 10:42:46	0|Victorsueca|hmmmm
 17 2016-10-29 10:43:49	0|Bi|If someone could give me the expected wtxid for the given raw transaction?
 18 2016-10-29 10:49:09	0|Victorsueca|weird.... can't decode your hex string using decoderawtransaction
 19 2016-10-29 10:49:51	0|Victorsueca|also tried blockchain.info transaction parses but I wouldn't be surprised if that failed anyway
 20 2016-10-29 10:50:06	0|arubi|the transaction is fine
 21 2016-10-29 10:50:10	0|Bi|This is segwit transaction
 22 2016-10-29 10:50:23	0|Bi|it decodes properly here: http://n.bitcoin.ninja/checktx
 23 2016-10-29 10:50:46	0|Bi|It is a real tx from testent block #895231
 24 2016-10-29 10:51:06	0|Victorsueca|why does bitcoin-cli decoderawtransaction not work then?
 25 2016-10-29 10:51:46	0|Bi|I dont know
 26 2016-10-29 10:51:59	0|arubi|it works for me..
 27 2016-10-29 10:52:10	0|Bi|Make sure to remove all the spaces
 28 2016-10-29 10:52:22	0|Victorsueca|ahhh yeah
 29 2016-10-29 10:52:28	0|Bi|So what is its wtxid?
 30 2016-10-29 10:52:35	0|Victorsueca|IRC added a line jump, forgot about that
 31 2016-10-29 10:54:10	0|arubi|Bi, as reported by bitcoin-cli, it's 508e4409c2cdff3d08b54370147cf6bf4d328f8248e4c60941a528b4d0375159 . not sure how much that'll help you
 32 2016-10-29 10:54:26	0|Bi|ok, thank you - I have the same one
 33 2016-10-29 10:54:44	0|Bi|But then it doesnt create a correct merkle hash
 34 2016-10-29 10:55:01	0|Bi|if you look into this block: https://testnet.blockexplorer.com/block/0000000000000009fdda232a7dfb09682aeb35e04d1437b4ed6a08e9abb479e0
 35 2016-10-29 10:55:28	0|arubi|I am
 36 2016-10-29 10:55:42	0|Bi|... it expects segwit merkle root of 554949ea7454928f42158d3554817f70c1fa8ba75e769faa58128ac05d333d28
 37 2016-10-29 10:56:34	0|arubi|looks like it, right.  hopefully the hash in the op_return output is actually reversed like you say..  I'm not sure
 38 2016-10-29 10:56:47	0|Bi|and when you hash null hash (wtxid of coinbase) followed by this 508e4409c2cdff3d08b54370147cf6bf4d328f8248e4c60941a528b4d0375159 - I get something else
 39 2016-10-29 10:57:21	0|Bi|I was trying swapping and not swapping - no success :(
 40 2016-10-29 10:58:03	0|Victorsueca|Bi: here's your transaction in JSON format https://softnet.homenet.org/zerobin/?11d833ece4506b4e#IYNmeQ7+/uvT/uZUW6TbikcrQus5TJa8AJruObsa3bY=
 41 2016-10-29 10:58:05	0|arubi|but what's in that return output is not just the wtxid merkle root
 42 2016-10-29 10:58:21	0|Victorsueca|and here's your block in JSON https://softnet.homenet.org/zerobin/?4e1d5fd2c883f51c#BKcO0qKshmbruiyexoJltMZIMqf4p7OMbzch05rnkQA=
 43 2016-10-29 10:58:36	0|Victorsueca|jus queried those from my node
 44 2016-10-29 10:58:38	0|arubi|the merkle root is appended the witness reserved value and then hashed hash256 again
 45 2016-10-29 10:59:11	0|Bi|oh?
 46 2016-10-29 10:59:13	0|arubi|at least that's what I gathered from bip141
 47 2016-10-29 10:59:13	0|Bi|is it?
 48 2016-10-29 10:59:50	0|Bi|No, it says "A witness root hash is calculated with all those wtxid as leaves, in a way similar to the hashMerkleRoot in the block header."
 49 2016-10-29 10:59:59	0|arubi|right, and then...
 50 2016-10-29 11:00:09	0|arubi|Double-SHA256(witness root hash|witness reserved value
 51 2016-10-29 11:00:13	0|arubi|)
 52 2016-10-29 11:01:04	0|Bi|right - thank you!
 53 2016-10-29 11:01:07	0|Bi|this must be it
 54 2016-10-29 11:01:09	0|arubi|also, pretty sure that at least with the wtxid I gave you, you /should/ reverse the bytes.  and also treat the bytes like they are in the commitment hash
 55 2016-10-29 11:01:33	0|arubi|(so not reversed like you pasted)
 56 2016-10-29 11:02:13	0|Bi|yes, reversing I knew about
 57 2016-10-29 11:02:29	0|Bi|Double-SHA256(witness root hash|witness reserved value) - was the part I missed
 58 2016-10-29 11:02:35	0|Bi|thanks again
 59 2016-10-29 11:02:40	0|arubi|cheers Bi
 60 2016-10-29 11:05:13	0|Bi|Yes, it works no!
 61 2016-10-29 11:05:17	0|Bi|Cheers
 62 2016-10-29 11:05:20	0|arubi|yep, tried it too :)
 63 2016-10-29 11:05:23	0|Bi|now*
 64 2016-10-29 14:16:20	0|tonikt|Is the presence of the witness root field (in the coinbase output) mandatory, after SEGWIT activation?
 65 2016-10-29 14:17:08	0|tonikt|(assuming there are any segwit txs inside the block)
 66 2016-10-29 14:20:00	0|tonikt|Sorry, I found it. It's the "unexpected-witness" code :)
 67 2016-10-29 15:01:01	0|sipa|tonikt: not, not mandatoy
 68 2016-10-29 15:03:33	0|tonikt|sipa: I think it becomes mandatory, after the activation.
 69 2016-10-29 15:03:38	0|tonikt|yes?
 70 2016-10-29 15:05:42	0|tonikt|.. it would go to "if (!fHaveWitness) ..." in ContextualCheckBlock()
 71 2016-10-29 15:06:32	0|tonikt|.. and then it would find a tx with segwit, and then it would return error "unexpected-witness"
 72 2016-10-29 15:44:53	0|Victorsueca|tonikt: non-segwit blocks are supposed to still be valid even after activation
 73 2016-10-29 16:05:16	0|tonikt|Sure. I was just wondering, after the activation, whether I have to reject blocks without the segwit root
 74 2016-10-29 16:05:54	0|tonikt|It seemed logical to reject them, but could not find that part of code
 75 2016-10-29 16:11:24	0|sipa|tonikt: no, if no transaction contains a witness, there is no requirement for a witness commitment
 76 2016-10-29 16:12:02	0|tonikt|yes - only if there is at least one witness transaction
 77 2016-10-29 16:12:07	0|sipa|of course
 78 2016-10-29 16:12:15	0|sipa|then it is mandatory
 79 2016-10-29 16:12:23	0|tonikt|but I think it also counts coinbase transaction as a possible witness-type?
 80 2016-10-29 16:15:21	0|tonikt|I have another, more high-level question...
 81 2016-10-29 16:15:52	0|tonikt|I am on testnet network now and trying to sync up the chain and all my current peers are pre 0.13.0
 82 2016-10-29 16:16:07	0|tonikt|And I am already on block #893669
 83 2016-10-29 16:16:28	0|sipa|tonikt: the coinbase witness is not committed to anywhere excet by the commitment in the output
 84 2016-10-29 16:16:45	0|tonikt|I should not be taking any block data from these peers, should I?
 85 2016-10-29 16:16:46	0|sipa|tonikt: which means that if you don't have a commitment, you can just remove the coinbase witness
 86 2016-10-29 16:17:18	0|sipa|without invalidating anything
 87 2016-10-29 16:17:28	0|sipa|we only download blocks from witness peers, indeed
 88 2016-10-29 16:18:25	0|tonikt|sipa: OK - that's interesting. I forgot that removing the witness from CB would not change txid, so it would still validate the block :)
 89 2016-10-29 16:19:35	0|sipa|indeed
 90 2016-10-29 16:20:04	0|sipa|if there is no witness commitment, it means that no tx has a witness
 91 2016-10-29 16:20:12	0|sipa|including the coinbase
 92 2016-10-29 16:20:43	0|tonikt|OK - all clear now
 93 2016-10-29 16:20:46	0|tonikt|thanks
 94 2016-10-29 16:21:40	0|tonikt|do you know any running testnet node where I could connect to fetch some segwith blocks?
 95 2016-10-29 16:21:59	0|tonikt|all my peers are < 0.13.0 now :)
 96 2016-10-29 16:22:34	0|sipa|use x9.seed.bitcoin.sipa.be
 97 2016-10-29 16:22:46	0|sipa|eh, that's mainnet
 98 2016-10-29 16:23:04	0|sipa|for testnet i think jonasschnelli's and petertodd's seed support filtering
 99 2016-10-29 16:23:37	0|tonikt|if you have a running node, would you just give me an IP, please?
100 2016-10-29 16:24:19	0|sipa|37.34.48.17
101 2016-10-29 16:25:16	0|tonikt|thank you!
102 2016-10-29 17:13:05	0|BlueMatt|uhhhh, preferential peering is quite impressive... https://imgur.com/a/jwH7R
103 2016-10-29 17:44:28	0|sipa|BlueMatt: 174 connections here
104 2016-10-29 18:14:38	0|Victorsueca|used to have 70-80 peers, I have 121 right now
105 2016-10-29 18:18:57	0|sipa|i temporarily increased my max connections
106 2016-10-29 18:25:43	0|BlueMatt|sipa: i had something like 250 on that node
107 2016-10-29 18:26:21	0|BlueMatt|(with some blacklisting of bad nodes)
108 2016-10-29 18:33:44	0|sipa|seems my dns seed has a bug... it's using 400% cpu and is not responding to dns requests
109 2016-10-29 18:34:04	0|sipa|yay for multiple implementations
110 2016-10-29 19:19:31	0|sipa|fixed, i think
111 2016-10-29 19:21:55	0|sipa|jonasschnelli, petertodd, luke-jr: pushed a bugfix to bitcoin-seeder which could cause infinite loops in the dns handling thread
112 2016-10-29 19:22:27	0|luke-jr|sipa: I'm intentionally holding back my version for a little while just to avoid us all using the same code immediately
113 2016-10-29 19:22:42	0|luke-jr|is it an exploit, or am I probably okay?
114 2016-10-29 19:22:59	0|sipa|luke-jr: you're probably already stuck in an infinite loop
115 2016-10-29 19:23:14	0|sipa|is the 'DNS requests' number going up?
116 2016-10-29 19:23:17	0|luke-jr|CPU is 100% idle, so I doubt it?
117 2016-10-29 19:23:23	0|luke-jr|yes
118 2016-10-29 19:23:24	0|sipa|indeed, then i doubt it
119 2016-10-29 19:23:26	0|luke-jr|[16-10-29 19:23:24] 591/5717464 available (5713103 tried in 132614s, 3081 new, 1280 active), 4288200 banned; 1851875 DNS requests, 316442 db queries
120 2016-10-29 19:23:59	0|sipa|did you upgrade to have filtering support?
121 2016-10-29 19:24:06	0|luke-jr|-rw-r--r-- 1 bitcoinpool bitcoinpool 849M Oct 29 18:49 dnsseed.dat <-- is this right? O.o
122 2016-10-29 19:24:09	0|luke-jr|no
123 2016-10-29 19:24:34	0|luke-jr|(the size)
124 2016-10-29 19:25:02	0|sipa|with 11M addresses in your database, i guess yes
125 2016-10-29 19:25:09	0|sipa|the question is why do you have so many addresses in there
126 2016-10-29 19:25:18	0|sipa|i have 2.5M
127 2016-10-29 19:26:04	0|luke-jr|plaintext dump is only 117 MB
128 2016-10-29 19:26:48	0|sipa|the dump doesn't contain banned/terrible IPs
129 2016-10-29 19:27:26	0|luke-jr|would it matter that I dropped crawler threads to 20 for a long time?
130 2016-10-29 19:27:51	0|sipa|160M -rw-rw-r-- 1 pw pw 160M Oct 29 12:27 dnsseed.dat 54M -rw-r--r-- 1 pw pw  54M Oct 29 12:27 dnsseed.dump
131 2016-10-29 19:28:10	0|sipa|it seems you're spending a huge amount of time retrying IPs that are gone
132 2016-10-29 19:28:23	0|sipa|your node needs 2 days to cycle through everything
133 2016-10-29 19:28:32	0|sipa|it's 15 minutes here
134 2016-10-29 19:28:36	0|luke-jr|:o
135 2016-10-29 19:28:58	0|luke-jr|I guess variety is worth the overhead?
136 2016-10-29 19:29:11	0|sipa|i would say so
137 2016-10-29 19:29:25	0|sipa|but i'd still like to understand why yours behaves so differently from mine
138 2016-10-29 19:30:10	0|luke-jr|only difference in mine from 11e935b72020607e5c3ce85a88209bc34e427a06 is REQUIRE_VERSION 70001
139 2016-10-29 19:30:41	0|luke-jr|./dnsseed -h dnsseed.bitcoin.dashjr.org -n jun.dashjr.org -p 1053 -t 80
140 2016-10-29 19:32:58	0|luke-jr|sipa: some years ago, my server was blackholed semi-regularly for a while due to an upstream/backbone router hating the crawler's access patterns; could blackholing do it?
141 2016-10-29 19:33:12	0|luke-jr|ie, maybe it thinks all IPs are sometimes terrible
142 2016-10-29 19:33:56	0|sipa|you only have 591 good IPs
143 2016-10-29 19:34:00	0|sipa|i have 4000
144 2016-10-29 19:34:26	0|sipa|wow, only 3000 suddenly... perhaps due to being offline for a while as a result of that bug
145 2016-10-29 19:34:36	0|sipa|usually it's around 4000