1 2016-10-29 01:45:48 0|GitHub115|[13bitcoin] 15sipa opened pull request #9039: Various serialization simplifcations and optimizations (06master...06simpleserial) 02 https://github.com/bitcoin/bitcoin/pull/9039
2 2016-10-29 09:59:43 0|Bi|Guys, if I have a block with only the coinbase tx and one segwit tx. It takes sha256->sha256 on 64 bytes of data to calculate the witness' merkle - right?
3 2016-10-29 09:59:59 0|Bi|And the first 32 bytes are zero?
4 2016-10-29 10:00:35 0|Bi|While the other 32 bytes is sha256->sha256 of the raw segwit transaction?
5 2016-10-29 10:01:36 0|Bi|It seems very simple, but I must be doing something wrong, beacuse my merkle roots dont match the expected values
6 2016-10-29 10:02:12 0|Bi|Can someone please help me?
7 2016-10-29 10:07:30 0|Bi|69baaa750c70eb352ebeadb07da0ff5bbd642cb285895ee43f2889dd12642ca0ff9b2489bd82d4d5ca287c4ebdf509f1bcd02d6fa720542e2900000000
8 2016-10-29 10:07:30 0|Bi|Or maybe you could at leat tell me what is wtxid of this transaction: 01000000000101f15a2db447e8316e9f20073ab8bdeac53df1a53d415fbd41e73929699b889e7a0000000017160014fb459699ce9ca78de6d0790121dfb215883acb64ffffffff0100111024010000001976a914fb459699ce9ca78de6d0790121dfb215883acb6488ac02483045022100c5feadba5fbcc28afb940c7ee664bba81830f19a5407ae49cf75f565534b149702200a7c2b6b1b5846a37c434ffe0cae168d6b797de9087a21176f2c9ef6fa9f47aa014104b848ab6ac853cd
9 2016-10-29 10:08:03 0|Bi|it is from this block: https://testnet.blockexplorer.com/block/0000000000000009fdda232a7dfb09682aeb35e04d1437b4ed6a08e9abb479e0
10 2016-10-29 10:34:04 0|Victorsueca|Bi: your transaction is supposed to have 2 IDs, the txid as it has always been and the new wtxid
11 2016-10-29 10:34:55 0|Victorsueca|the txid should be sha256(sha256(original serialization format))
12 2016-10-29 10:35:58 0|Victorsueca|the wtxid should be sha256(sha256(new serialization format with witness data))
13 2016-10-29 10:36:52 0|Victorsueca|if your transaction does not have any witness data the txid and the wtxid should be the same
14 2016-10-29 10:41:25 0|Bi|Victorsueca, yes I know all of this
15 2016-10-29 10:42:07 0|Bi|Bu my merkle roots dont match :(
16 2016-10-29 10:42:46 0|Victorsueca|hmmmm
17 2016-10-29 10:43:49 0|Bi|If someone could give me the expected wtxid for the given raw transaction?
18 2016-10-29 10:49:09 0|Victorsueca|weird.... can't decode your hex string using decoderawtransaction
19 2016-10-29 10:49:51 0|Victorsueca|also tried blockchain.info transaction parses but I wouldn't be surprised if that failed anyway
20 2016-10-29 10:50:06 0|arubi|the transaction is fine
21 2016-10-29 10:50:10 0|Bi|This is segwit transaction
22 2016-10-29 10:50:23 0|Bi|it decodes properly here: http://n.bitcoin.ninja/checktx
23 2016-10-29 10:50:46 0|Bi|It is a real tx from testent block #895231
24 2016-10-29 10:51:06 0|Victorsueca|why does bitcoin-cli decoderawtransaction not work then?
25 2016-10-29 10:51:46 0|Bi|I dont know
26 2016-10-29 10:51:59 0|arubi|it works for me..
27 2016-10-29 10:52:10 0|Bi|Make sure to remove all the spaces
28 2016-10-29 10:52:22 0|Victorsueca|ahhh yeah
29 2016-10-29 10:52:28 0|Bi|So what is its wtxid?
30 2016-10-29 10:52:35 0|Victorsueca|IRC added a line jump, forgot about that
31 2016-10-29 10:54:10 0|arubi|Bi, as reported by bitcoin-cli, it's 508e4409c2cdff3d08b54370147cf6bf4d328f8248e4c60941a528b4d0375159 . not sure how much that'll help you
32 2016-10-29 10:54:26 0|Bi|ok, thank you - I have the same one
33 2016-10-29 10:54:44 0|Bi|But then it doesnt create a correct merkle hash
34 2016-10-29 10:55:01 0|Bi|if you look into this block: https://testnet.blockexplorer.com/block/0000000000000009fdda232a7dfb09682aeb35e04d1437b4ed6a08e9abb479e0
35 2016-10-29 10:55:28 0|arubi|I am
36 2016-10-29 10:55:42 0|Bi|... it expects segwit merkle root of 554949ea7454928f42158d3554817f70c1fa8ba75e769faa58128ac05d333d28
37 2016-10-29 10:56:34 0|arubi|looks like it, right. hopefully the hash in the op_return output is actually reversed like you say.. I'm not sure
38 2016-10-29 10:56:47 0|Bi|and when you hash null hash (wtxid of coinbase) followed by this 508e4409c2cdff3d08b54370147cf6bf4d328f8248e4c60941a528b4d0375159 - I get something else
39 2016-10-29 10:57:21 0|Bi|I was trying swapping and not swapping - no success :(
40 2016-10-29 10:58:03 0|Victorsueca|Bi: here's your transaction in JSON format https://softnet.homenet.org/zerobin/?11d833ece4506b4e#IYNmeQ7+/uvT/uZUW6TbikcrQus5TJa8AJruObsa3bY=
41 2016-10-29 10:58:05 0|arubi|but what's in that return output is not just the wtxid merkle root
42 2016-10-29 10:58:21 0|Victorsueca|and here's your block in JSON https://softnet.homenet.org/zerobin/?4e1d5fd2c883f51c#BKcO0qKshmbruiyexoJltMZIMqf4p7OMbzch05rnkQA=
43 2016-10-29 10:58:36 0|Victorsueca|jus queried those from my node
44 2016-10-29 10:58:38 0|arubi|the merkle root is appended the witness reserved value and then hashed hash256 again
45 2016-10-29 10:59:11 0|Bi|oh?
46 2016-10-29 10:59:13 0|arubi|at least that's what I gathered from bip141
47 2016-10-29 10:59:13 0|Bi|is it?
48 2016-10-29 10:59:50 0|Bi|No, it says "A witness root hash is calculated with all those wtxid as leaves, in a way similar to the hashMerkleRoot in the block header."
49 2016-10-29 10:59:59 0|arubi|right, and then...
50 2016-10-29 11:00:09 0|arubi|Double-SHA256(witness root hash|witness reserved value
51 2016-10-29 11:00:13 0|arubi|)
52 2016-10-29 11:01:04 0|Bi|right - thank you!
53 2016-10-29 11:01:07 0|Bi|this must be it
54 2016-10-29 11:01:09 0|arubi|also, pretty sure that at least with the wtxid I gave you, you /should/ reverse the bytes. and also treat the bytes like they are in the commitment hash
55 2016-10-29 11:01:33 0|arubi|(so not reversed like you pasted)
56 2016-10-29 11:02:13 0|Bi|yes, reversing I knew about
57 2016-10-29 11:02:29 0|Bi|Double-SHA256(witness root hash|witness reserved value) - was the part I missed
58 2016-10-29 11:02:35 0|Bi|thanks again
59 2016-10-29 11:02:40 0|arubi|cheers Bi
60 2016-10-29 11:05:13 0|Bi|Yes, it works no!
61 2016-10-29 11:05:17 0|Bi|Cheers
62 2016-10-29 11:05:20 0|arubi|yep, tried it too :)
63 2016-10-29 11:05:23 0|Bi|now*
64 2016-10-29 14:16:20 0|tonikt|Is the presence of the witness root field (in the coinbase output) mandatory, after SEGWIT activation?
65 2016-10-29 14:17:08 0|tonikt|(assuming there are any segwit txs inside the block)
66 2016-10-29 14:20:00 0|tonikt|Sorry, I found it. It's the "unexpected-witness" code :)
67 2016-10-29 15:01:01 0|sipa|tonikt: not, not mandatoy
68 2016-10-29 15:03:33 0|tonikt|sipa: I think it becomes mandatory, after the activation.
69 2016-10-29 15:03:38 0|tonikt|yes?
70 2016-10-29 15:05:42 0|tonikt|.. it would go to "if (!fHaveWitness) ..." in ContextualCheckBlock()
71 2016-10-29 15:06:32 0|tonikt|.. and then it would find a tx with segwit, and then it would return error "unexpected-witness"
72 2016-10-29 15:44:53 0|Victorsueca|tonikt: non-segwit blocks are supposed to still be valid even after activation
73 2016-10-29 16:05:16 0|tonikt|Sure. I was just wondering, after the activation, whether I have to reject blocks without the segwit root
74 2016-10-29 16:05:54 0|tonikt|It seemed logical to reject them, but could not find that part of code
75 2016-10-29 16:11:24 0|sipa|tonikt: no, if no transaction contains a witness, there is no requirement for a witness commitment
76 2016-10-29 16:12:02 0|tonikt|yes - only if there is at least one witness transaction
77 2016-10-29 16:12:07 0|sipa|of course
78 2016-10-29 16:12:15 0|sipa|then it is mandatory
79 2016-10-29 16:12:23 0|tonikt|but I think it also counts coinbase transaction as a possible witness-type?
80 2016-10-29 16:15:21 0|tonikt|I have another, more high-level question...
81 2016-10-29 16:15:52 0|tonikt|I am on testnet network now and trying to sync up the chain and all my current peers are pre 0.13.0
82 2016-10-29 16:16:07 0|tonikt|And I am already on block #893669
83 2016-10-29 16:16:28 0|sipa|tonikt: the coinbase witness is not committed to anywhere excet by the commitment in the output
84 2016-10-29 16:16:45 0|tonikt|I should not be taking any block data from these peers, should I?
85 2016-10-29 16:16:46 0|sipa|tonikt: which means that if you don't have a commitment, you can just remove the coinbase witness
86 2016-10-29 16:17:18 0|sipa|without invalidating anything
87 2016-10-29 16:17:28 0|sipa|we only download blocks from witness peers, indeed
88 2016-10-29 16:18:25 0|tonikt|sipa: OK - that's interesting. I forgot that removing the witness from CB would not change txid, so it would still validate the block :)
89 2016-10-29 16:19:35 0|sipa|indeed
90 2016-10-29 16:20:04 0|sipa|if there is no witness commitment, it means that no tx has a witness
91 2016-10-29 16:20:12 0|sipa|including the coinbase
92 2016-10-29 16:20:43 0|tonikt|OK - all clear now
93 2016-10-29 16:20:46 0|tonikt|thanks
94 2016-10-29 16:21:40 0|tonikt|do you know any running testnet node where I could connect to fetch some segwith blocks?
95 2016-10-29 16:21:59 0|tonikt|all my peers are < 0.13.0 now :)
96 2016-10-29 16:22:34 0|sipa|use x9.seed.bitcoin.sipa.be
97 2016-10-29 16:22:46 0|sipa|eh, that's mainnet
98 2016-10-29 16:23:04 0|sipa|for testnet i think jonasschnelli's and petertodd's seed support filtering
99 2016-10-29 16:23:37 0|tonikt|if you have a running node, would you just give me an IP, please?
100 2016-10-29 16:24:19 0|sipa|37.34.48.17
101 2016-10-29 16:25:16 0|tonikt|thank you!
102 2016-10-29 17:13:05 0|BlueMatt|uhhhh, preferential peering is quite impressive... https://imgur.com/a/jwH7R
103 2016-10-29 17:44:28 0|sipa|BlueMatt: 174 connections here
104 2016-10-29 18:14:38 0|Victorsueca|used to have 70-80 peers, I have 121 right now
105 2016-10-29 18:18:57 0|sipa|i temporarily increased my max connections
106 2016-10-29 18:25:43 0|BlueMatt|sipa: i had something like 250 on that node
107 2016-10-29 18:26:21 0|BlueMatt|(with some blacklisting of bad nodes)
108 2016-10-29 18:33:44 0|sipa|seems my dns seed has a bug... it's using 400% cpu and is not responding to dns requests
109 2016-10-29 18:34:04 0|sipa|yay for multiple implementations
110 2016-10-29 19:19:31 0|sipa|fixed, i think
111 2016-10-29 19:21:55 0|sipa|jonasschnelli, petertodd, luke-jr: pushed a bugfix to bitcoin-seeder which could cause infinite loops in the dns handling thread
112 2016-10-29 19:22:27 0|luke-jr|sipa: I'm intentionally holding back my version for a little while just to avoid us all using the same code immediately
113 2016-10-29 19:22:42 0|luke-jr|is it an exploit, or am I probably okay?
114 2016-10-29 19:22:59 0|sipa|luke-jr: you're probably already stuck in an infinite loop
115 2016-10-29 19:23:14 0|sipa|is the 'DNS requests' number going up?
116 2016-10-29 19:23:17 0|luke-jr|CPU is 100% idle, so I doubt it?
117 2016-10-29 19:23:23 0|luke-jr|yes
118 2016-10-29 19:23:24 0|sipa|indeed, then i doubt it
119 2016-10-29 19:23:26 0|luke-jr|[16-10-29 19:23:24] 591/5717464 available (5713103 tried in 132614s, 3081 new, 1280 active), 4288200 banned; 1851875 DNS requests, 316442 db queries
120 2016-10-29 19:23:59 0|sipa|did you upgrade to have filtering support?
121 2016-10-29 19:24:06 0|luke-jr|-rw-r--r-- 1 bitcoinpool bitcoinpool 849M Oct 29 18:49 dnsseed.dat <-- is this right? O.o
122 2016-10-29 19:24:09 0|luke-jr|no
123 2016-10-29 19:24:34 0|luke-jr|(the size)
124 2016-10-29 19:25:02 0|sipa|with 11M addresses in your database, i guess yes
125 2016-10-29 19:25:09 0|sipa|the question is why do you have so many addresses in there
126 2016-10-29 19:25:18 0|sipa|i have 2.5M
127 2016-10-29 19:26:04 0|luke-jr|plaintext dump is only 117 MB
128 2016-10-29 19:26:48 0|sipa|the dump doesn't contain banned/terrible IPs
129 2016-10-29 19:27:26 0|luke-jr|would it matter that I dropped crawler threads to 20 for a long time?
130 2016-10-29 19:27:51 0|sipa|160M -rw-rw-r-- 1 pw pw 160M Oct 29 12:27 dnsseed.dat 54M -rw-r--r-- 1 pw pw 54M Oct 29 12:27 dnsseed.dump
131 2016-10-29 19:28:10 0|sipa|it seems you're spending a huge amount of time retrying IPs that are gone
132 2016-10-29 19:28:23 0|sipa|your node needs 2 days to cycle through everything
133 2016-10-29 19:28:32 0|sipa|it's 15 minutes here
134 2016-10-29 19:28:36 0|luke-jr|:o
135 2016-10-29 19:28:58 0|luke-jr|I guess variety is worth the overhead?
136 2016-10-29 19:29:11 0|sipa|i would say so
137 2016-10-29 19:29:25 0|sipa|but i'd still like to understand why yours behaves so differently from mine
138 2016-10-29 19:30:10 0|luke-jr|only difference in mine from 11e935b72020607e5c3ce85a88209bc34e427a06 is REQUIRE_VERSION 70001
139 2016-10-29 19:30:41 0|luke-jr|./dnsseed -h dnsseed.bitcoin.dashjr.org -n jun.dashjr.org -p 1053 -t 80
140 2016-10-29 19:32:58 0|luke-jr|sipa: some years ago, my server was blackholed semi-regularly for a while due to an upstream/backbone router hating the crawler's access patterns; could blackholing do it?
141 2016-10-29 19:33:12 0|luke-jr|ie, maybe it thinks all IPs are sometimes terrible
142 2016-10-29 19:33:56 0|sipa|you only have 591 good IPs
143 2016-10-29 19:34:00 0|sipa|i have 4000
144 2016-10-29 19:34:26 0|sipa|wow, only 3000 suddenly... perhaps due to being offline for a while as a result of that bug
145 2016-10-29 19:34:36 0|sipa|usually it's around 4000