1 2012-08-16 00:01:16 [\\\] has quit (Ping timeout: 245 seconds)
   2 2012-08-16 00:01:54 devrandom has quit (Remote host closed the connection)
   3 2012-08-16 00:03:29 QbY has quit (Quit: QbY)
   4 2012-08-16 00:03:55 Mobius_ has joined
   5 2012-08-16 00:03:58 MobiusLoop has quit (Ping timeout: 276 seconds)
   6 2012-08-16 00:08:31 D34TH has quit (Ping timeout: 276 seconds)
   7 2012-08-16 00:09:08 D34TH has joined
   8 2012-08-16 00:12:02 devrandom has joined
   9 2012-08-16 00:13:16 D34TH has quit (Client Quit)
  10 2012-08-16 00:16:35 d4de has joined
  11 2012-08-16 00:16:36 d4de has quit (Changing host)
  12 2012-08-16 00:16:36 d4de has joined
  13 2012-08-16 00:20:55 Gladamas has quit (Read error: Connection reset by peer)
  14 2012-08-16 00:22:10 Gladamas has joined
  15 2012-08-16 00:23:05 copumpkin has quit (Quit: Computer has gone to sleep.)
  16 2012-08-16 00:25:50 QbY has joined
  17 2012-08-16 00:28:40 Mobius_ has quit (Ping timeout: 276 seconds)
  18 2012-08-16 00:28:51 one_zero has joined
  19 2012-08-16 00:30:28 da2ce769 has joined
  20 2012-08-16 00:31:58 QbY has quit (Read error: Connection reset by peer)
  21 2012-08-16 00:35:32 Gladamas has quit (Ping timeout: 240 seconds)
  22 2012-08-16 00:37:38 da2ce769 has quit (Ping timeout: 245 seconds)
  23 2012-08-16 00:39:36 BCBot has quit (Ping timeout: 245 seconds)
  24 2012-08-16 00:39:52 Marf has quit (Ping timeout: 244 seconds)
  25 2012-08-16 00:41:14 BCBot has joined
  26 2012-08-16 00:42:01 copumpkin has joined
  27 2012-08-16 00:46:13 dlb76 has joined
  28 2012-08-16 00:48:12 ThomasV has joined
  29 2012-08-16 00:50:35 Marf has joined
  30 2012-08-16 00:53:14 Joric has joined
  31 2012-08-16 00:56:54 user has joined
  32 2012-08-16 01:04:36 BCBot has quit (Ping timeout: 245 seconds)
  33 2012-08-16 01:06:26 BCBot has joined
  34 2012-08-16 01:07:32 [\\\] has joined
  35 2012-08-16 01:07:38 phantomcircuit has quit (Quit: Leaving)
  36 2012-08-16 01:08:21 nonick has quit (Remote host closed the connection)
  37 2012-08-16 01:08:30 sneak has quit (Ping timeout: 272 seconds)
  38 2012-08-16 01:08:40 sneak has joined
  39 2012-08-16 01:08:40 sneak has quit (Changing host)
  40 2012-08-16 01:08:40 sneak has joined
  41 2012-08-16 01:08:48 nonick has joined
  42 2012-08-16 01:09:53 bcb_ has joined
  43 2012-08-16 01:12:24 phantomcircuit has joined
  44 2012-08-16 01:12:41 TAiS46 has quit (Quit: This computer has gone to sleep)
  45 2012-08-16 01:13:58 darkee has joined
  46 2012-08-16 01:16:07 ehash_ has quit (Ping timeout: 276 seconds)
  47 2012-08-16 01:17:25 nonick has quit (Ping timeout: 276 seconds)
  48 2012-08-16 01:18:31 ehash has joined
  49 2012-08-16 01:25:34 theymos has joined
  50 2012-08-16 01:28:58 Marf has quit (Ping timeout: 248 seconds)
  51 2012-08-16 01:29:02 toffoo has joined
  52 2012-08-16 01:33:46 Mobius_ has joined
  53 2012-08-16 01:44:31 caedes has quit (Remote host closed the connection)
  54 2012-08-16 01:47:29 phantomcircuit has quit (Quit: Leaving)
  55 2012-08-16 01:48:54 phantomcircuit has joined
  56 2012-08-16 01:56:47 kish is now known as sojourn
  57 2012-08-16 01:56:49 Joric has quit ()
  58 2012-08-16 01:57:16 sojourn is now known as Guest52544
  59 2012-08-16 02:00:11 devrandom has quit (Remote host closed the connection)
  60 2012-08-16 02:00:29 devrandom has joined
  61 2012-08-16 02:21:38 root2 has joined
  62 2012-08-16 02:22:49 jrmithdobbs has quit (Quit: quit)
  63 2012-08-16 02:24:21 jrmithdobbs has joined
  64 2012-08-16 02:26:24 ThomasV has quit (Ping timeout: 268 seconds)
  65 2012-08-16 02:29:46 jrmithdobbs has quit (Quit: quit)
  66 2012-08-16 02:32:16 jrmithdobbs has joined
  67 2012-08-16 02:36:12 minimoose has left ()
  68 2012-08-16 02:40:03 ThomasV has joined
  69 2012-08-16 02:42:56 gmaxwell has joined
  70 2012-08-16 02:53:35 <CodeLion> Can someone please explain to me what practical difference there is between blocks and shares and how the pooling system manages to distribute the mining process into shares?
  71 2012-08-16 02:54:24 <gmaxwell> shares are attempted blocks which meet a lower difficulty standard.
  72 2012-08-16 02:55:09 <gmaxwell> The process of mining is stochastic; it's trivially distributed. Participants are given work that pays to a different public key, or has a different extranonce, or a different timestamp... and tada. distributed.
  73 2012-08-16 02:55:37 <sipa> you may want to read https://en.bitcoin.it/wiki/Pooled_mining
  74 2012-08-16 02:56:19 <CodeLion> Thanks sipa that looks like what I need.
  75 2012-08-16 02:56:35 <CodeLion> Trying to wrap my head around the rest of the system, besides the actual hashing xD
  76 2012-08-16 02:56:53 <gmaxwell> sipa: I think this is an extra argument for peer rotation: http://arxiv.org/abs/1208.2534  , weakening transaction source localization by making the connectivity graph less easy to learn.
  77 2012-08-16 02:59:08 RainbowDashh has quit (Read error: Connection reset by peer)
  78 2012-08-16 03:04:03 bcb_ has quit (Quit: Page closed)
  79 2012-08-16 03:07:23 Z0rZ0rZ0r_ has joined
  80 2012-08-16 03:09:46 Z0rZ0rZ0r has quit (Ping timeout: 248 seconds)
  81 2012-08-16 03:11:36 user has quit (Quit: Leaving)
  82 2012-08-16 03:11:46 MC-Eeepc has quit (Ping timeout: 244 seconds)
  83 2012-08-16 03:15:00 theymos has quit (Remote host closed the connection)
  84 2012-08-16 03:18:10 Mobius_ has quit (Remote host closed the connection)
  85 2012-08-16 03:21:06 Mobius_ has joined
  86 2012-08-16 03:24:04 torsthaldo has quit (Read error: Connection reset by peer)
  87 2012-08-16 03:24:16 torsthaldo has joined
  88 2012-08-16 03:24:57 dlb76 has quit (Ping timeout: 245 seconds)
  89 2012-08-16 03:25:18 sneak has quit (Ping timeout: 272 seconds)
  90 2012-08-16 03:25:27 mndrix has quit (Ping timeout: 245 seconds)
  91 2012-08-16 03:25:34 mndrix has joined
  92 2012-08-16 03:25:35 mndrix has quit (Changing host)
  93 2012-08-16 03:25:35 mndrix has joined
  94 2012-08-16 03:25:35 mndrix has quit (Changing host)
  95 2012-08-16 03:25:35 mndrix has joined
  96 2012-08-16 03:25:35 sneak has joined
  97 2012-08-16 03:25:35 sneak has quit (Changing host)
  98 2012-08-16 03:25:35 sneak has joined
  99 2012-08-16 03:28:41 <weex> From the wiki: Coins generated aren't confirmed until 100 blocks. Then it says bitcoin-qt requires 120 blocks. Which is right?
 100 2012-08-16 03:28:54 <[Tycho]> Both.
 101 2012-08-16 03:28:55 <weex> not really bitcoin-qt at the time it was written i guess
 102 2012-08-16 03:29:22 <[Tycho]> The network requires around 100-101 and the satoshi client waits for 120
 103 2012-08-16 03:29:54 <weex> so the protocol is 100 or 101 and the specific client is 120
 104 2012-08-16 03:30:13 MiningBuddy- has joined
 105 2012-08-16 03:30:16 <[Tycho]> Yes.
 106 2012-08-16 03:30:20 <gmaxwell> right. if you send right at the limit then peers that haven't quite caught up yet will drop your transactions.
 107 2012-08-16 03:30:22 copumpkin is now known as pumpkin
 108 2012-08-16 03:30:38 <gmaxwell> So it's inadvisable to send right at the threshold.. (though 20 is a bit conservative)
 109 2012-08-16 03:31:10 <[Tycho]> I'm frequently sending at 101, usually works fine.
 110 2012-08-16 03:31:22 talso has joined
 111 2012-08-16 03:31:45 <[Tycho]> Even tried sending before, but it worked anyway, just was confirmed after maturing.
 112 2012-08-16 03:31:52 pumpkin is now known as copumpkin
 113 2012-08-16 03:31:58 <gmaxwell> I've gotten txn stuck doing it. I suspect if it happens to you, you end up solving the next block anways. :)
 114 2012-08-16 03:32:01 <[Tycho]> (I was not sure if it will work)
 115 2012-08-16 03:32:21 enquirer has joined
 116 2012-08-16 03:32:25 <gmaxwell> [Tycho]: it absolutely will not relay, and peers will not request it again.
 117 2012-08-16 03:32:43 <[Tycho]> Yes, but TXes are reannounced.
 118 2012-08-16 03:32:53 <gmaxwell> Yes, that will work.
 119 2012-08-16 03:33:13 t4ls0 has quit (Ping timeout: 268 seconds)
 120 2012-08-16 03:33:14 malaimo_ has quit (Ping timeout: 268 seconds)
 121 2012-08-16 03:33:15 malaimo has joined
 122 2012-08-16 03:33:27 <[Tycho]> I can't remember clearly what happened, but initially I thought that this TX got in trouble, but later it went thru.
 123 2012-08-16 03:38:47 TheSeven has quit (Disconnected by services)
 124 2012-08-16 03:38:53 [7] has joined
 125 2012-08-16 03:39:47 torsthaldo has quit (Excess Flood)
 126 2012-08-16 03:40:07 eoss has quit (Remote host closed the connection)
 127 2012-08-16 03:41:54 torsthaldo has joined
 128 2012-08-16 03:42:24 <sipa> gmaxwell: sure, i'm quite convinced peer rotation is good idea; just haven't gotten to implementing it
 129 2012-08-16 03:43:36 <CodeLion> Question because nobody else knows... I know this isn't actually bitcoin directly but does anyone know of a BTC or LTC client designed to run on android? I need to manage my transactions and keep my wallet without a computer. Mine died.
 130 2012-08-16 03:44:43 <sipa> sure
 131 2012-08-16 03:44:47 <sipa> several, i think
 132 2012-08-16 03:45:17 <CodeLion> I would hug you if it wasn't weird. Any chance for a link?
 133 2012-08-16 03:46:46 <sipa> just search for bitcoin wallet in google play
 134 2012-08-16 03:47:10 <sipa> it's by andreas schildbach
 135 2012-08-16 03:47:22 <CodeLion> Thanks bro
 136 2012-08-16 03:48:24 <CodeLion> I'll have to study that source and see if I can hack it up to run litecoin
 137 2012-08-16 03:48:34 <CodeLion> >.< too many projects not enough me.
 138 2012-08-16 03:50:25 eoss has joined
 139 2012-08-16 03:50:25 eoss has quit (Changing host)
 140 2012-08-16 03:50:25 eoss has joined
 141 2012-08-16 03:50:48 <[Tycho]> CodeLion: why would you need litecoin ?
 142 2012-08-16 03:52:23 <CodeLion> I'm poor and I run a litecoin miner on my tablet for funz.
 143 2012-08-16 03:52:47 <CodeLion> I'm working on a bitcoin board though and I should have moneys soon from selling ti
 144 2012-08-16 03:52:48 <CodeLion> *it
 145 2012-08-16 03:52:59 <CodeLion> Now I have to go though, much work to do
 146 2012-08-16 03:54:25 da2ce7 has joined
 147 2012-08-16 03:55:30 CodeLion has left ("Leaving")
 148 2012-08-16 04:00:47 brwyatt is now known as brwyatt|Away
 149 2012-08-16 04:03:27 finway has joined
 150 2012-08-16 04:03:31 ThomasV has quit (Ping timeout: 246 seconds)
 151 2012-08-16 04:04:16 <finway> Hey,
 152 2012-08-16 04:04:18 <finway> The Bitcoin Project will be making a major announcement in September that should contribute some stability, he said. Still, he predicted things will continue to be exciting in the world of Bitcoin with "continued controversy and chaos."
 153 2012-08-16 04:04:22 <finway> What's that?
 154 2012-08-16 04:04:28 <finway> 2-factor-walklet ?
 155 2012-08-16 04:04:34 <finway> 2-factor-wallet ?
 156 2012-08-16 04:04:50 <finway> The Bitcoin Project will be making a major announcement in September that should contribute some stability
 157 2012-08-16 04:04:57 <finway> What's the major announcement ?
 158 2012-08-16 04:05:28 <gmaxwell> If it were announced already it would be announced already!
 159 2012-08-16 04:05:54 <finway> gmaxwell, is it a secret ?
 160 2012-08-16 04:06:01 enquirer_ has joined
 161 2012-08-16 04:06:18 enquirer has quit (Ping timeout: 248 seconds)
 162 2012-08-16 04:06:25 enquirer_ is now known as enquirer
 163 2012-08-16 04:07:22 <finway> gmaxwell, what is it ?
 164 2012-08-16 04:08:11 <weex> exciting.
 165 2012-08-16 04:08:25 <finway> weex, what's that ? do you konw ?
 166 2012-08-16 04:08:46 <weex> a new economic stimulus feature?
 167 2012-08-16 04:08:55 <finway> oh, fee
 168 2012-08-16 04:08:57 <finway> i guess
 169 2012-08-16 04:09:04 ThomasV has joined
 170 2012-08-16 04:09:39 <gmaxwell> Dunno, I suppose it could be several yet unannounced things. We'll know when its announced!
 171 2012-08-16 04:10:24 <finway> gmaxwell, can't be
 172 2012-08-16 04:10:51 denisx has quit (Remote host closed the connection)
 173 2012-08-16 04:11:13 <weex> finway: where did you read that? Verge?
 174 2012-08-16 04:11:17 denisx has joined
 175 2012-08-16 04:11:37 <finway> weex, yeah,  http://www.theverge.com/2012/8/15/3243200/bitcoin-ponzi-schemes-savings-and-trust
 176 2012-08-16 04:14:14 eoss has quit (Remote host closed the connection)
 177 2012-08-16 04:27:00 nhodges has joined
 178 2012-08-16 04:27:50 ByteUnits has quit (Ping timeout: 265 seconds)
 179 2012-08-16 04:27:58 <Luke-Jr> finway: it's mainstream Tonal support of course
 180 2012-08-16 04:28:34 <finway> luke-jr, i suppose gavin was talking about fee , right ?
 181 2012-08-16 04:28:51 <gmaxwell> finway: you shouldn't make guesses.
 182 2012-08-16 04:29:58 <finway> gmaxwell, i missed this channel for about 1 month,i have to guess...
 183 2012-08-16 04:30:51 denisx_ has joined
 184 2012-08-16 04:33:12 denisx has quit (Ping timeout: 246 seconds)
 185 2012-08-16 04:33:12 denisx_ is now known as denisx
 186 2012-08-16 04:36:45 * nanotube on edge of seat. ;)
 187 2012-08-16 04:37:37 * Luke-Jr pushes nanotube off his seat :P
 188 2012-08-16 04:37:55 * nanotube plops down
 189 2012-08-16 04:38:01 <Luke-Jr> I still see vulnerable versions on SourceForge! :P
 190 2012-08-16 04:38:18 <Luke-Jr> including 0.6.x ones :x
 191 2012-08-16 04:38:40 * Luke-Jr thinks vulnerable releases should probably be hidden, if not removed
 192 2012-08-16 04:39:32 <weex> just wait until September
 193 2012-08-16 04:43:57 meLon has joined
 194 2012-08-16 04:43:57 meLon has quit (Changing host)
 195 2012-08-16 04:43:57 meLon has joined
 196 2012-08-16 04:44:35 finway has quit (Ping timeout: 245 seconds)
 197 2012-08-16 04:45:20 <midnightmagic> Luke-Jr, sipa: there can be a significant cost to seeking to the end of a large file.
 198 2012-08-16 04:47:19 <Luke-Jr> O.o
 199 2012-08-16 04:48:17 <midnightmagic> for a few hundred GB, for example, depending on FS, it's possible that seeking could take on the order of seconds..
 200 2012-08-16 04:52:53 leotreasure has quit (Quit: leotreasure)
 201 2012-08-16 05:02:57 brwyatt is now known as Away!~brwyatt@brwyatt.net|brwyatt
 202 2012-08-16 05:03:14 CodeLion has joined
 203 2012-08-16 05:03:38 <CodeLion> Hey what would occur if a miner accidentally reported a "winning" hash which was infact produced by an error?
 204 2012-08-16 05:03:41 ehash has quit ()
 205 2012-08-16 05:06:34 <weex> it would be caught and dropped
 206 2012-08-16 05:08:25 denisx has quit (Quit: denisx)
 207 2012-08-16 05:10:56 <Luke-Jr> midnightmagic: you're referring to the OS seek syscall?
 208 2012-08-16 05:11:36 sirk390 has quit (Quit: Leaving.)
 209 2012-08-16 05:12:37 <midnightmagic> Luke-Jr: yes.
 210 2012-08-16 05:15:27 OneEyed has quit (Quit: WeeChat 0.3.8)
 211 2012-08-16 05:21:29 denisx has joined
 212 2012-08-16 05:22:52 ThomasV has quit (Quit: Quitte)
 213 2012-08-16 05:24:27 <CodeLion> Weex: Is there any penalty or something?
 214 2012-08-16 05:24:44 <CodeLion> Trying to decide if it is worth it to implement error checking of if I can afford to just ignore errors
 215 2012-08-16 05:30:04 sneak has quit (Ping timeout: 272 seconds)
 216 2012-08-16 05:30:21 sneak has joined
 217 2012-08-16 05:30:21 sneak has quit (Changing host)
 218 2012-08-16 05:30:21 sneak has joined
 219 2012-08-16 05:34:38 _W has joined
 220 2012-08-16 05:35:40 abracadabopoulos has joined
 221 2012-08-16 05:35:43 <denisx> CodeLion: your users would complain about to many rejected shares
 222 2012-08-16 05:35:50 _W_ has quit (Ping timeout: 260 seconds)
 223 2012-08-16 05:35:50 Detritus has quit (Read error: Connection reset by peer)
 224 2012-08-16 05:35:50 ForceMajeure_ has joined
 225 2012-08-16 05:35:50 BLZbubba_ has quit (Ping timeout: 260 seconds)
 226 2012-08-16 05:35:50 <[7]> there is no penalty
 227 2012-08-16 05:35:50 <[7]> and nobody will care if you produce a few of them occasionally
 228 2012-08-16 05:35:50 <[7]> but if you expect >1% of them I'd implement it
 229 2012-08-16 05:35:50 <[7]> that also allows you to keep track of the errors and possibly nail down their cause
 230 2012-08-16 05:35:50 ForceMajeure has quit (Ping timeout: 260 seconds)
 231 2012-08-16 05:35:50 abracadabra has quit (Ping timeout: 260 seconds)
 232 2012-08-16 05:36:25 BLZbubba has joined
 233 2012-08-16 05:36:40 Clipse has quit (Ping timeout: 260 seconds)
 234 2012-08-16 05:37:15 forrestv has joined
 235 2012-08-16 05:37:22 ForceMajeure_ is now known as ForceMajeure
 236 2012-08-16 05:37:48 brwyatt is now known as brwyatt|Away
 237 2012-08-16 05:37:52 ovidiusoft has joined
 238 2012-08-16 05:39:37 <[7]> denisx: not sure if he's looking at this from a pool's perspective
 239 2012-08-16 05:39:49 <denisx> [7]: but I do! ;)
 240 2012-08-16 05:41:14 <[7]> well if someone makes a pool that accepts invalids he'll get lots of share uploads from me :P
 241 2012-08-16 05:41:55 <denisx> [7]: start with duplicates on deepbit! ;)
 242 2012-08-16 05:41:59 <Luke-Jr> lol
 243 2012-08-16 05:43:48 <[7]> i doubt they count them
 244 2012-08-16 05:44:00 <[7]> they'll just ignore them
 245 2012-08-16 05:45:11 <[Tycho]> Tell me that worker name and I'll find out.
 246 2012-08-16 05:45:25 <denisx> [Tycho]: sorry, I forgot to ask
 247 2012-08-16 05:45:26 <[Tycho]> I think they are just counted as "error".
 248 2012-08-16 05:45:37 <denisx> [Tycho]: but I try today!
 249 2012-08-16 05:45:38 <CodeLion> Denisx: I am looking at a pool
 250 2012-08-16 05:45:51 <[Tycho]> CodeLion: try Deepbit :)
 251 2012-08-16 05:45:56 <denisx> [7]: help me to remember it!
 252 2012-08-16 05:46:10 <CodeLion> Well the only reason there would be errors would be if the user was pushing the clock signal beyond the safe limit
 253 2012-08-16 05:46:15 <CodeLion> so... no problems.
 254 2012-08-16 05:46:18 <[Tycho]> No only.
 255 2012-08-16 05:46:30 <denisx> CodeLion: clock signal?
 256 2012-08-16 05:46:34 <CodeLion> Long story
 257 2012-08-16 05:46:42 <[Tycho]> FPGAs can do this too, but usually software filters them out
 258 2012-08-16 05:46:47 <CodeLion> mmhm.
 259 2012-08-16 05:47:04 <CodeLion> I was wondering wether such software would be worth the effort
 260 2012-08-16 05:47:08 <CodeLion> probably not
 261 2012-08-16 05:47:10 <[7]> [Tycho]: apparently cgminer with ztex boards sends dupes
 262 2012-08-16 05:47:31 <[Tycho]> Shares from wrong getwork are also rejected or declined.
 263 2012-08-16 05:48:25 <[7]> one of your miners said that cgminer didn't notice them being rejected on deep it, but did see duplicate rejects on other pools
 264 2012-08-16 05:48:27 <CodeLion> ah so it would look like a stale?
 265 2012-08-16 05:48:45 <[7]> deepbit*
 266 2012-08-16 05:48:51 <[7]> damn phone keyboars
 267 2012-08-16 05:49:01 <[7]> keyboard* :P
 268 2012-08-16 05:49:02 <[Tycho]> [7]: other pools may implement rejection reason.
 269 2012-08-16 05:49:03 <denisx> my autocorrection changed it to deppbit ;)
 270 2012-08-16 05:49:59 <[7]> CodeLion: you'll need to detect the invalids yourself to drive your clocking strategy properly
 271 2012-08-16 05:50:04 <[Tycho]> dep bit... May be, considering "dep" is "beautiful" in vietnamese.
 272 2012-08-16 05:50:14 <[7]> and the code for that isn't all that complicated
 273 2012-08-16 05:50:58 <CodeLion> Yeah I know
 274 2012-08-16 05:51:05 <[Tycho]> MPBM sometimes says something like "/dev/xxx sent K-not-zero share a5170080"
 275 2012-08-16 05:51:08 <CodeLion> But it takes cycles which I don't have
 276 2012-08-16 05:51:10 <CodeLion> lol
 277 2012-08-16 05:51:24 <[Tycho]> But what is that "K" ?
 278 2012-08-16 05:51:44 <[7]> the last word of the sha256 result
 279 2012-08-16 05:51:51 <[Tycho]> Oh.
 280 2012-08-16 05:51:53 <[Tycho]> Got it
 281 2012-08-16 05:52:19 <[7]> no idea why it's called like that
 282 2012-08-16 05:52:29 <CodeLion> [7]: I know it isn't complicated but considering I only have about 32 Mhz on my non-hashing processor, it would take 2 seconds to verify a share.
 283 2012-08-16 05:52:31 <denisx> [7]: 16bit words?
 284 2012-08-16 05:52:33 <CodeLion> actually 4 seconds
 285 2012-08-16 05:52:39 <[7]> a through h would have made sense actually
 286 2012-08-16 05:53:05 <denisx> [7]: in pushpool it is h
 287 2012-08-16 05:53:16 <[7]> but some pools put k-not-zero as the reject reason, which i followed
 288 2012-08-16 05:53:19 <[Tycho]> CodeLion: verify for what ?
 289 2012-08-16 05:53:28 <CodeLion> Error correction
 290 2012-08-16 05:53:44 <[Tycho]> [7]: MPBM is yours ?
 291 2012-08-16 05:53:46 <[7]> 4 seconds!? on what?
 292 2012-08-16 05:53:48 <[7]> yes
 293 2012-08-16 05:53:55 <[Tycho]> [7]: cool. Thanks.
 294 2012-08-16 05:53:57 <CodeLion> A 32 MHz microcontroller
 295 2012-08-16 05:54:03 <CodeLion> xD
 296 2012-08-16 05:54:06 <denisx> CodeLion: even on 32MHz it should be faster than 2 seconds
 297 2012-08-16 05:54:07 <[7]> that should be milliseconds
 298 2012-08-16 05:54:16 <CodeLion> Oooh yeah
 299 2012-08-16 05:54:19 <CodeLion> *facepalms
 300 2012-08-16 05:54:23 <CodeLion> lool
 301 2012-08-16 05:54:28 <[Tycho]> May be he is using some managed code ?
 302 2012-08-16 05:54:30 <[7]> especially if you have the midstate already
 303 2012-08-16 05:54:45 <[Tycho]> Kind of VERY managed.
 304 2012-08-16 05:54:52 <[7]> i doubt his mcu has enough ram for that kind of bloat
 305 2012-08-16 05:54:55 <denisx> [7]: midstate is deprecated!
 306 2012-08-16 05:54:55 <CodeLion> Yeah I meant 4 miliseconds I think
 307 2012-08-16 05:55:18 <[7]> but you need the midstate to mine
 308 2012-08-16 05:55:18 <CodeLion> ok thats not bad
 309 2012-08-16 05:55:27 <[7]> so he must have it somewhere
 310 2012-08-16 05:55:34 <CodeLion> [7] Why do you need the midstate?
 311 2012-08-16 05:55:50 * CodeLion is still confused by all the terms. and might know about the midstate but not remember what it is called
 312 2012-08-16 05:55:56 <[7]> wanna redo it for every bruteforce attempt?
 313 2012-08-16 05:55:57 <[Tycho]> To save you from hashing the static part
 314 2012-08-16 05:56:18 B0g4r7__ has joined
 315 2012-08-16 05:56:22 <CodeLion> Wait I've never heard of this?
 316 2012-08-16 05:56:29 <Luke-Jr> [7]: erm, never heard of pools using K
 317 2012-08-16 05:56:30 <Luke-Jr> it's H
 318 2012-08-16 05:56:36 <[Tycho]> CodeLion: do you know how hashing works ?
 319 2012-08-16 05:56:40 RainbowDashh has joined
 320 2012-08-16 05:56:50 <CodeLion> Tycho: Yeah I've done extensive research about SHA.
 321 2012-08-16 05:56:52 <[7]> hm, should probably change that then
 322 2012-08-16 05:57:05 <CodeLion> It didn't include midstate though that I remember
 323 2012-08-16 05:57:16 <[Tycho]> CodeLion: source string is split into blocks.
 324 2012-08-16 05:57:19 <[7]> CodeLion: every sha round gets a data input, and a state carryover from the last round
 325 2012-08-16 05:57:41 <[7]> and as only the last block changes for the next bruteforce attempt, you only have to redo the last round
 326 2012-08-16 05:57:43 <CodeLion> oooh so you can just restart hashing from the begining of the nonce instead of rehashing the rest of the protocol?
 327 2012-08-16 05:57:44 <Luke-Jr> CodeLion: midstate is just the SHA256 state after the first block is processed
 328 2012-08-16 05:57:58 <CodeLion> I should have thought of that >><<
 329 2012-08-16 05:58:18 <[7]> actually (although that is deprecated) the mining pools will feed you the midstate directly
 330 2012-08-16 05:58:23 <CodeLion> oh
 331 2012-08-16 05:58:33 <Luke-Jr> [7]: not all
 332 2012-08-16 05:58:38 <CodeLion> Can anyone post a link about this stuff or something?
 333 2012-08-16 05:58:38 <Guest54468> [7]: don't call it rounds, call it blocks, like [Tycho] said. "rounds" means something different.
 334 2012-08-16 05:58:38 <Perlboy> is rpcpassword= in the config file meant to be plain text?
 335 2012-08-16 05:58:52 B0g4r7 has quit (Ping timeout: 276 seconds)
 336 2012-08-16 05:58:52 B0g4r7__ is now known as B0g4r7
 337 2012-08-16 05:58:58 <Luke-Jr> [7]: I'd feed it to known legacy miners ;)
 338 2012-08-16 05:59:14 <[7]> Luke-Jr: to everything not advertising the midstate extension?
 339 2012-08-16 05:59:37 <weex> Perlboy: it is plain text
 340 2012-08-16 06:00:13 <CodeLion> I've read alot of the wiki but I must have missed the midstate. Anyone have documentation about it so I don't have to follow the IRC? (which can get quite confusing with 2 or more convos going)
 341 2012-08-16 06:00:37 <[7]> CodeLion: what are you trying to do in the first place?
 342 2012-08-16 06:00:48 <[7]> implement some kind of frontend for an FPGA-based device?
 343 2012-08-16 06:01:38 <CodeLion> I'm not prepared to disclose all the details but it's something like that, yeah
 344 2012-08-16 06:01:59 <CodeLion> For all practical purposes,  I'm writing a frontend for an FPGA
 345 2012-08-16 06:02:20 <denisx> with 32MHz? atxmega?
 346 2012-08-16 06:02:40 phantomcircuit_ has joined
 347 2012-08-16 06:03:15 <Luke-Jr> CodeLion: just implement it yourself however you want; don't expect pools to give it to you
 348 2012-08-16 06:03:50 <[Tycho]> If you are feeding FPGA, you'll need midstate anyway.
 349 2012-08-16 06:04:03 <CodeLion> Luke-Jr: yeah I plan to now, but knowing nothing about it besides a basic concept isn't very good. which is why I was hoping for documentation about it.
 350 2012-08-16 06:04:11 <[7]> well, you'll need it for just about anything that mines
 351 2012-08-16 06:04:42 phantomcircuit has quit (Disconnected by services)
 352 2012-08-16 06:04:42 <[7]> I don't get why people still use those crappy mcus though
 353 2012-08-16 06:04:46 phantomcircuit_ is now known as phantomcircuit
 354 2012-08-16 06:04:59 <[7]> given we have cheap 100+ MHz ARM MCUs these days
 355 2012-08-16 06:05:01 <CodeLion> OK rather than practical purposes shall we say I am learning about mining concepts and designing an "imaginary" miner for fun and learning purposes?
 356 2012-08-16 06:05:10 phantomcircuit_ has joined
 357 2012-08-16 06:05:12 <[Tycho]> ARM is way too complicated.
 358 2012-08-16 06:05:25 <CodeLion> Cheap as in... how much [7]?
 359 2012-08-16 06:05:34 <[7]> complicated in terms of what?
 360 2012-08-16 06:05:37 <CodeLion> [Tycho]: I have experience with Arm though
 361 2012-08-16 06:05:41 <denisx> [7]: and until some while ago the gpios were too slow on it
 362 2012-08-16 06:05:56 <[7]> CodeLion: between $3 and $15 depending on memory requirements etc.
 363 2012-08-16 06:06:01 <[Tycho]> [7]: I don't know, never tried it.
 364 2012-08-16 06:06:06 <CodeLion> Holy cow awesome
 365 2012-08-16 06:06:09 <Luke-Jr> [7]: 100+ MHz don't use more power?
 366 2012-08-16 06:06:17 <[Tycho]> Most of my contraptions use Z8
 367 2012-08-16 06:06:24 <[7]> well you can surely clock them slower if power is an issue
 368 2012-08-16 06:06:29 <[7]> but in most cases it isn't
 369 2012-08-16 06:06:39 <CodeLion> Raspberry Pi to driver a miner. instant win.
 370 2012-08-16 06:06:52 <[7]> guess what people are doing...
 371 2012-08-16 06:07:18 <[Tycho]> Installing a linux there.
 372 2012-08-16 06:07:35 <[7]> CodeLion: I personally recommend the STM32F2/F4 series, but LPC13xx is also interesting :)
 373 2012-08-16 06:07:53 * [Tycho] is planning to try something much more exotic
 374 2012-08-16 06:07:56 <CodeLion> Linux on raspberry pi must be incredible. I personally want it to hook up to an xbox kinect to build a 3D cad model projector
 375 2012-08-16 06:08:54 CodesInChaos has joined
 376 2012-08-16 06:09:19 <Luke-Jr> [7]: would be nice to just load BFGMiner directly onto the MCU :P
 377 2012-08-16 06:09:37 <[7]> well, not really bfgminer
 378 2012-08-16 06:09:48 <[7]> but I think this guy is about to try something similar
 379 2012-08-16 06:09:58 <[Tycho]> Would be nice to use some kind of bus or ethernet for mining devices.
 380 2012-08-16 06:10:00 <CodeLion> I hate it when companies have an awesome product, but the minimal order is like 80 units and the quantity available is 0
 381 2012-08-16 06:10:13 <[Tycho]> Those hundreds of USB cables are annoying.
 382 2012-08-16 06:10:22 <[7]> CodeLion: can you disclose whether whatever you're hacking up right now is going to be an entirely new fpga-based solution, or just a frontend for pre-existing fpga boards? (just out of curiosity)
 383 2012-08-16 06:10:25 <CodeLion> Have yall been lurking on #Ozcoin? lol
 384 2012-08-16 06:10:48 <[7]> [Tycho]: we're on it
 385 2012-08-16 06:10:50 <CodeLion> [7] New solution entirely
 386 2012-08-16 06:10:57 <[7]> our next-gen boards will have that
 387 2012-08-16 06:11:01 <[Tycho]> [7]: who ?
 388 2012-08-16 06:11:06 <[7]> fpga mining llc
 389 2012-08-16 06:11:12 <[7]> the makers of the x6500
 390 2012-08-16 06:11:17 <[Tycho]> ztex ?
 391 2012-08-16 06:11:21 <denisx> no
 392 2012-08-16 06:11:28 RazielZ has joined
 393 2012-08-16 06:12:40 <[Tycho]> [7]: aren't you afraid of ASICs ?
 394 2012-08-16 06:14:07 <CodeLion> ASIC is always going to be overpriced
 395 2012-08-16 06:14:09 <CodeLion> always
 396 2012-08-16 06:14:24 <[Tycho]> Not always.
 397 2012-08-16 06:14:37 <CodeLion> Well... untill they have enough demand to be truly mass produced.
 398 2012-08-16 06:14:54 <CodeLion> until then the cost points will stay up compared to other tech
 399 2012-08-16 06:15:03 <Luke-Jr> CodeLion: you mean the 3.5 GH/s I can buy for $150 and have before 2013?
 400 2012-08-16 06:15:33 <[Tycho]> Before 2013 ? Are you sure ?
 401 2012-08-16 06:15:56 <[Tycho]> Price per GH/s will be instantly lower than FPGA
 402 2012-08-16 06:15:57 <CodeLion> Wait what?
 403 2012-08-16 06:16:03 <CodeLion> yeah thats nuts
 404 2012-08-16 06:16:05 <Luke-Jr> well, someone ordering today probably can't get it before 2013 :p
 405 2012-08-16 06:16:07 <[Tycho]> And I can imagine even selling at a loss.
 406 2012-08-16 06:16:16 <CodeLion> link or it isn't true
 407 2012-08-16 06:16:51 <[Tycho]> CodeLion: there are at least 3 or more ASIC attempts in development.
 408 2012-08-16 06:16:53 <Luke-Jr> http://www.butterflylabs.com/products/
 409 2012-08-16 06:17:03 <Luke-Jr> "Our ASIC based products ranging from 3.5 GH/s to 1,000 GH/s are currently scheduled for availability in October, 2012."
 410 2012-08-16 06:17:03 <CodeLion> Yeah I know tycho.
 411 2012-08-16 06:17:22 <CodeLion> Luke-Jr how is that price number so low?
 412 2012-08-16 06:17:42 <CodeLion> thats well over 20 MH/s per dollar
 413 2012-08-16 06:17:42 <Luke-Jr> CodeLion: BFL has always had the lowest prices
 414 2012-08-16 06:18:11 <CodeLion> hmmm...
 415 2012-08-16 06:18:15 * CodeLion scratches chin
 416 2012-08-16 06:18:18 <CodeLion> Back to work then.
 417 2012-08-16 06:18:28 <CodeLion> Must. finish. project!
 418 2012-08-16 06:18:30 CodeInChaos has joined
 419 2012-08-16 06:18:33 * CodeLion chugs a monster
 420 2012-08-16 06:18:43 ultrixx has joined
 421 2012-08-16 06:19:57 CodesInChaos has quit (Ping timeout: 240 seconds)
 422 2012-08-16 06:22:54 <denisx> [Tycho]: ultrix is the one using cgminer on your pool and has no duplicates
 423 2012-08-16 06:23:35 <denisx> .oO(the hackers I know don't get up before 2pm, what has happened)
 424 2012-08-16 06:24:02 <ultrixx> i use btcminer atm
 425 2012-08-16 06:24:19 <ultrixx> from ztex's site
 426 2012-08-16 06:24:37 <[7]> denisx: I haven't even gone to sleep yet
 427 2012-08-16 06:25:04 <[7]> [Tycho]: tbh I'm not really afraid of BFL
 428 2012-08-16 06:25:14 <[7]> the other asic projects might get dangerous
 429 2012-08-16 06:25:27 <[7]> especially if they decide to mine for themselves instead of selling the chips
 430 2012-08-16 06:25:54 <Luke-Jr> denisx: ultrixx: wait, BTCMiner *also* gets the dupes?
 431 2012-08-16 06:26:04 <denisx> Luke-Jr: yes
 432 2012-08-16 06:26:06 <ultrixx> to my knowledge, no
 433 2012-08-16 06:26:15 * Luke-Jr facepalms
 434 2012-08-16 06:26:33 <denisx> Luke-Jr: someone ported exactly the code with the bugs? ;)
 435 2012-08-16 06:26:42 <[Tycho]> [7]: no, I was talking about ASICs as competitors to your new boards.
 436 2012-08-16 06:26:53 <Luke-Jr> denisx: not unlikely
 437 2012-08-16 06:27:05 <[7]> I doubt BFL will really take off this year
 438 2012-08-16 06:27:11 <[Tycho]> ultrixx: PM me your login name
 439 2012-08-16 06:27:18 <[7]> and no idea when the other asics will hit market
 440 2012-08-16 06:27:23 <jgarzik> ASIC mini-rig for $30,000
 441 2012-08-16 06:27:29 * jgarzik wonders where to get the other $25,000...
 442 2012-08-16 06:27:43 <[7]> jgarzik: and wait until 2014 for it to be shipped
 443 2012-08-16 06:28:38 <[7]> [Tycho]: and fpgas will always have the advantage of reprogrammability
 444 2012-08-16 06:28:46 <[7]> i.e. they can be optimized after the fact
 445 2012-08-16 06:28:48 <denisx> even GPU mining is still lucrative
 446 2012-08-16 06:29:18 <denisx> yeah, I hope someone breaks sha-256 the day the ASICs arrive ;)
 447 2012-08-16 06:29:25 <[7]> and FPGAs use a much more modern silicon process than whatever the other projects will produce
 448 2012-08-16 06:29:34 <[7]> so there's a performance advantage from that as well
 449 2012-08-16 06:29:47 <jgarzik> reprogram-ability is not an asset if the FPGA is far slower than ASIC
 450 2012-08-16 06:29:53 <[Tycho]> No, that's not advantage.
 451 2012-08-16 06:30:33 <[7]> according to my sources it is fairly likely for next-generation FPGAs to beat BFL
 452 2012-08-16 06:30:41 <[Tycho]> No way.
 453 2012-08-16 06:30:47 <[Tycho]> 28nm ?
 454 2012-08-16 06:30:50 <[7]> yes
 455 2012-08-16 06:31:11 <denisx> [Tycho]: http://www.xilinx.com/products/silicon-devices/fpga/kintex-7/index.htm
 456 2012-08-16 06:31:22 <Diablo-D3> and according to my sources you cant beat something that does not and will not exist.
 457 2012-08-16 06:31:35 <[7]> Diablo-D3: well, beat their announced specs then
 458 2012-08-16 06:31:56 <[Tycho]> ASICs are many times cheaper.
 459 2012-08-16 06:32:07 <[7]> kintex is going to be rather expensive, artix will be the next big target
 460 2012-08-16 06:32:16 <Diablo-D3> yes, but how will those beat asicminer?
 461 2012-08-16 06:32:55 <[7]> as long as those guys don't sell boards they aren't an issue
 462 2012-08-16 06:33:08 <[7]> and IIUC they 1. want to mine themselves and 2. sell chips
 463 2012-08-16 06:33:08 <Luke-Jr> jgarzik: it's probably too late to break even on an order anyway
 464 2012-08-16 06:33:12 <[7]> so we can in fact piggyback on that
 465 2012-08-16 06:33:31 <Luke-Jr> jgarzik: the first batch are most likely all claimed, and by the time an order today gets filled the difficulty will have already skyrocketted
 466 2012-08-16 06:34:10 <midnightmagic> Luke-Jr: you mean for singles and jalapenos?
 467 2012-08-16 06:34:28 <Luke-Jr> midnightmagic: well, jgarzik was talking about the SC Rig
 468 2012-08-16 06:34:48 <midnightmagic> Luke-Jr: how many TH do you think are coming?
 469 2012-08-16 06:34:58 <Luke-Jr> midnightmagic: each SC Rig is 1 TH/s
 470 2012-08-16 06:35:08 <midnightmagic> Luke-Jr: All the self-reporting people in the thread equal only 22TH
 471 2012-08-16 06:35:32 <midnightmagic> 15 of which are sc minirig
 472 2012-08-16 06:36:07 <midnightmagic> Luke-Jr: how long do you think until the next order ships?
 473 2012-08-16 06:36:27 <Luke-Jr> midnightmagic: I don't have any inside info on this stuff, I'm just speculating
 474 2012-08-16 06:36:43 <midnightmagic> Luke-Jr: oh, yeah course. same as the rest of us.
 475 2012-08-16 06:37:02 <denisx> [7]: your quad board is still on spartan-6?
 476 2012-08-16 06:37:09 <midnightmagic> Luke-Jr: I'm just wondering why you think it's too late to break even on an sc minirig order? :)
 477 2012-08-16 06:37:11 <Luke-Jr> I'm guessing they're using SASIC or something so their FPGA chips can be used as-is
 478 2012-08-16 06:37:24 <Diablo-D3> who?
 479 2012-08-16 06:37:25 <Diablo-D3> bfl?
 480 2012-08-16 06:37:30 <Diablo-D3> its possible
 481 2012-08-16 06:37:30 <Luke-Jr> in whcih case, this time is just spent waiting on the ASICs
 482 2012-08-16 06:37:40 <Luke-Jr> midnightmagic: because there's a lot of orders in?
 483 2012-08-16 06:37:46 <Luke-Jr> from the sound of it
 484 2012-08-16 06:37:46 <Diablo-D3> 65nm sasics will be fucking raped by 130nm real asics though
 485 2012-08-16 06:39:04 <midnightmagic> Luke-Jr: yeah, by pure order number it looks like there is,
 486 2012-08-16 06:39:55 <jgarzik> midnightmagic: _next_ order ships?  :)  Has any SC order shipped yet?
 487 2012-08-16 06:40:10 <Luke-Jr> I don't think there's any harm in me saying, by BFL staffs' voices too it sounds like there is :P
 488 2012-08-16 06:40:19 <midnightmagic> jgarzik: :)  just wondering why luke thinks it's too late to break-even on 1TH equipment is all
 489 2012-08-16 06:40:22 <Luke-Jr> jgarzik: no, but I'm sure the first order is all preordered
 490 2012-08-16 06:40:49 <Luke-Jr> unless they really have a super-huge quantity coming in or something
 491 2012-08-16 06:41:03 <Luke-Jr> but if they sold all on the first run, they'd have extra and crap
 492 2012-08-16 06:41:19 <jgarzik> Luke-Jr: it's just as possible to go from FPGA code to straight ASIC.  Engineering wise, you just change underlying libs, if your hdl is written correctly.
 493 2012-08-16 06:41:21 <Luke-Jr> and nobody to sell to in the future
 494 2012-08-16 06:41:22 <zveda> if bfl just mine with their asics for 1 day before shipping them.. they could make a billion $
 495 2012-08-16 06:41:32 <Luke-Jr> jgarzik: hmm
 496 2012-08-16 06:41:33 <[7]> denisx: the first generation, yes
 497 2012-08-16 06:41:42 <Luke-Jr> zveda: pretty sure the burn-in period is 1 day
 498 2012-08-16 06:42:02 <jgarzik> Luke-Jr: that's what the big houses do all the time
 499 2012-08-16 06:42:39 <jgarzik> of course, the super big houses spend time hand-tweaking their asic masks for that extra edge
 500 2012-08-16 06:42:45 <jgarzik> (FSVO "hand")
 501 2012-08-16 06:43:28 * jgarzik sees FPGA->ASIC all the time in kernel land
 502 2012-08-16 06:43:50 <midnightmagic> jgarzik: really?  like what?
 503 2012-08-16 06:43:50 <jgarzik> it's more about money and lead time, with SASIC vs. ASIC, IMO
 504 2012-08-16 06:44:25 <Luke-Jr> jgarzik: how do you see that in software? O.o
 505 2012-08-16 06:44:40 <midnightmagic> Luke-Jr: via drivers and driver support maybe..
 506 2012-08-16 06:45:00 <midnightmagic> Luke-Jr: why the heck did you capitalize your nick?
 507 2012-08-16 06:45:20 <Luke-Jr> midnightmagic: got tired of people using "Luke-JR"
 508 2012-08-16 06:45:32 * jgarzik resists the lure of the Luke-Jr TrollQuestion
 509 2012-08-16 06:45:46 <Luke-Jr> …
 510 2012-08-16 06:46:01 <Luke-Jr> I know nothing about making ASICs
 511 2012-08-16 06:46:59 <Karmaon1> Hey Luke-JR, how's it going?
 512 2012-08-16 06:47:21 <CodeLion> Methinks the asic market is both exaggerated and dangerous
 513 2012-08-16 06:47:43 <Luke-Jr> that's silly
 514 2012-08-16 06:47:46 <midnightmagic> CodeLion: why do you think it's dangerous?
 515 2012-08-16 06:48:01 <Karmaon1> midnightmagic: because everyone here is poor
 516 2012-08-16 06:48:22 <midnightmagic> Karmaon1: That doesn't make any sense. Why would that make it dangerous?
 517 2012-08-16 06:48:51 <Karmaon1> because few have the resources to produce asics
 518 2012-08-16 06:48:53 <CodeLion> Because if it works, it will drive prices up and permenantly alter the established method of mining. if it doesn't work, alot of people lost money and that could also destabilize the established market cycle.
 519 2012-08-16 06:49:31 <CodeLion> Also if it works, whoever makes them could just mine with them themselves and be superduperinsanely higher than everyone else in terms of hashrate
 520 2012-08-16 06:49:37 <denisx> the CPU->GPU change went smooth, the GPU->FPGA also
 521 2012-08-16 06:49:47 <denisx> it will be different with asics
 522 2012-08-16 06:49:54 <Karmaon1> until then, its going to be a monopoly until they have compeition
 523 2012-08-16 06:50:07 <Luke-Jr> denisx: unlikely
 524 2012-08-16 06:50:21 <Luke-Jr> Karmaon1: so? AMD had a monopoly for the GPU period
 525 2012-08-16 06:50:47 <[7]> that's a consumer product though
 526 2012-08-16 06:50:53 <[7]> with asic NRE this is a bit different
 527 2012-08-16 06:50:54 <Karmaon1> Luke-Jr: yes, but their market is not focused on bitcoins
 528 2012-08-16 06:51:29 <Karmaon1> what they sell is general-use
 529 2012-08-16 06:51:35 <[7]> and someone who designs an asic could indeed be selling only limited quantities at premium prices, while mining with like 30% of the network hashrate without any real production cost for those boards
 530 2012-08-16 06:51:47 <[7]> just the initial NRE cost, which will prevent many others from competing
 531 2012-08-16 06:54:19 RainbowDashh has quit (Ping timeout: 252 seconds)
 532 2012-08-16 06:54:24 <midnightmagic> well. perhaps it is a careful timed release of asic
 533 2012-08-16 06:56:20 <jgarzik> mining what you produce is always an interesting subject.  BFL claims they do not mine on mainnet.  No way to really prove those claims though.
 534 2012-08-16 06:56:25 <[7]> I really wonder what CodeLion is up to though
 535 2012-08-16 06:56:52 <CodeLion> It's a secret. Bwuahahahahaha! *thunderbolt and organ music*
 536 2012-08-16 06:57:17 <midnightmagic> CodeLion: Why would it drive prices up?
 537 2012-08-16 06:57:22 <[7]> this doesn't really line up
 538 2012-08-16 06:57:32 <[7]> on one hand, you have no clue how mining works
 539 2012-08-16 06:57:38 <Luke-Jr> jgarzik: when did BFL claim that?
 540 2012-08-16 06:57:43 <[7]> on the other hand you talk about 400MH/s on $70 chips
 541 2012-08-16 06:57:58 <CodeLion> [7] I have a great clue about it. but more about SHA than anything.
 542 2012-08-16 06:58:01 <jgarzik> Luke-Jr: read through their posts.  it's practically FAQ.  in fact, check their site's FAQ.  :)
 543 2012-08-16 06:58:04 <CodeLion> unfortunately
 544 2012-08-16 06:58:19 <CodeLion> [7] You read my posts *ghasp*
 545 2012-08-16 06:58:24 <Karmaon1> damn it! why aren't they being evil?
 546 2012-08-16 06:58:58 <[7]> well, how can you efficiently design an FPGA, hardcopy, asic or whatever, without knowing what a midstate is?
 547 2012-08-16 06:58:59 <denisx> CodeLion: and then you don't know what the midstate is?
 548 2012-08-16 06:59:06 <CodeLion> Nope denisx
 549 2012-08-16 06:59:11 <[7]> that's one of the foundations all of this is built on
 550 2012-08-16 06:59:18 <Luke-Jr> jgarzik: forums too messy; not in FAQ. I know I mine on mainnet when I test though <.<
 551 2012-08-16 06:59:38 <CodeLion> But then... I have a reading disorder. I probably know what it is but can't remember the name. Makes it impossible to learn from a bunch of sources :(
 552 2012-08-16 06:59:51 <Karmaon1> denisx: thing goes in, thing comes out, its a black box
 553 2012-08-16 07:00:07 <[7]> well, I assume you iterate nonces over a midstate+data pair on the FPGA or whatever it is?
 554 2012-08-16 07:00:26 <[7]> 400MH/s doesn't quite fit any of the devices I could imagine though
 555 2012-08-16 07:00:50 <midnightmagic> dual lx150 + sneakiness
 556 2012-08-16 07:01:00 <[7]> well he implied that it is one chip
 557 2012-08-16 07:01:19 <CodeLion> Fine i'll tell you. It's a new device. not an fpga at all. and I never implied a "chip" at all, I've been refering to "boards"
 558 2012-08-16 07:01:20 <[7]> 400MH/s is definitely too much for a spartan
 559 2012-08-16 07:01:21 <midnightmagic> And he IS a lion..
 560 2012-08-16 07:01:21 <denisx> [7]: the one from the BFL board?
 561 2012-08-16 07:01:42 <[7]> CodeLion: you've referred to "cores" though
 562 2012-08-16 07:01:44 <CodeLion> well... not really an fpg
 563 2012-08-16 07:01:45 osmosis has quit (Quit: Leaving)
 564 2012-08-16 07:01:45 <CodeLion> *a
 565 2012-08-16 07:01:56 <CodeLion> [7] Yes but I was misusing the term.
 566 2012-08-16 07:02:12 <CodeLion> sorry bout that
 567 2012-08-16 07:02:17 <[7]> well, I still don't get it
 568 2012-08-16 07:02:34 <[7]> you associated $70 with 400MH/s as some kind of "unit"
 569 2012-08-16 07:02:52 <denisx> he is the one making the asic
 570 2012-08-16 07:02:54 <denisx> ;)
 571 2012-08-16 07:02:55 tonikt has joined
 572 2012-08-16 07:03:00 <denisx> Iam not in fear anymore ;)
 573 2012-08-16 07:03:01 <CodeLion> You missed ~2 days of chatting 7
 574 2012-08-16 07:04:55 BCBot has quit (Ping timeout: 265 seconds)
 575 2012-08-16 07:04:56 <denisx> maybe I should prepare a diff-4 port for the day the asics are released
 576 2012-08-16 07:06:00 BCBot has joined
 577 2012-08-16 07:07:17 sirk390 has joined
 578 2012-08-16 07:09:36 <midnightmagic> 1TH of 400MH/s units would cost $175000
 579 2012-08-16 07:10:59 CodeInChaos has quit (Ping timeout: 244 seconds)
 580 2012-08-16 07:11:15 <amiller> how much would the power cost once you had those?
 581 2012-08-16 07:13:29 osxorgate has joined
 582 2012-08-16 07:17:13 ThomasV has joined
 583 2012-08-16 07:17:52 Maccer has quit (Ping timeout: 240 seconds)
 584 2012-08-16 07:19:44 tonikt has quit (Read error: Connection reset by peer)
 585 2012-08-16 07:22:47 galambo__ has quit (Read error: Connection reset by peer)
 586 2012-08-16 07:25:35 Marf has joined
 587 2012-08-16 07:32:49 denisx has quit (Remote host closed the connection)
 588 2012-08-16 07:33:09 denisx has joined
 589 2012-08-16 07:34:31 <[Tycho]> May be he is trying to use some of those useless stream hashers ? Or DSPs.
 590 2012-08-16 07:39:51 sirk390 has quit (Quit: Leaving.)
 591 2012-08-16 07:41:59 Diablo-D3 has quit (Ping timeout: 244 seconds)
 592 2012-08-16 07:43:05 <CodeLion> [Tycho] Stop guessing. Wait a few weeks till I go public with it
 593 2012-08-16 07:43:06 <CodeLion> lol
 594 2012-08-16 07:45:17 Marf has quit (Quit: Marf)
 595 2012-08-16 07:45:28 asa has quit (Remote host closed the connection)
 596 2012-08-16 07:45:57 <[Tycho]> May be you'll never do ?
 597 2012-08-16 07:46:22 <CodeLion> I hope not.
 598 2012-08-16 07:46:35 <CodeLion> I'm litterally pouring myself into learning bitcoin for this
 599 2012-08-16 07:47:16 <weex> and they say you can "just dabble" in Bitcoin...
 600 2012-08-16 07:47:27 <CodeLion> well you can
 601 2012-08-16 07:47:27 <weex> "you can quit anytime you want" they say
 602 2012-08-16 07:47:56 <CodeLion> unless you're trying to push out a new piece of tech with limited resources and limited knowledge and a massive axe called ASIC hanging over your neck...
 603 2012-08-16 07:48:08 <CodeLion> if that isn't the case... sure you can! :D
 604 2012-08-16 07:49:30 PiZZaMaN2K has joined
 605 2012-08-16 07:51:04 TAiS46 has joined
 606 2012-08-16 07:54:26 toffoo has quit ()
 607 2012-08-16 07:56:36 * Luke-Jr grumbles about other devs wanting to break standard best practices to clutter up the git tree with Windows-specific crap
 608 2012-08-16 07:57:03 <CodeLion> lol
 609 2012-08-16 08:00:02 OneFixt has quit (Read error: Connection reset by peer)
 610 2012-08-16 08:00:40 OneFixt has joined
 611 2012-08-16 08:00:41 PiZZaMaN2K has quit (Read error: Connection reset by peer)
 612 2012-08-16 08:01:09 ForceMajeure has quit (Read error: Connection reset by peer)
 613 2012-08-16 08:02:00 jaxtr has quit (Read error: Connection reset by peer)
 614 2012-08-16 08:02:20 jaxtr has joined
 615 2012-08-16 08:02:46 ForceMajeure has joined
 616 2012-08-16 08:05:49 enquirer_ has joined
 617 2012-08-16 08:08:14 enquirer has quit (Ping timeout: 265 seconds)
 618 2012-08-16 08:08:24 enquirer_ is now known as enquirer
 619 2012-08-16 08:12:11 root2 has quit ()
 620 2012-08-16 08:15:36 iocor has joined
 621 2012-08-16 08:15:38 iocor has quit (Changing host)
 622 2012-08-16 08:15:38 iocor has joined
 623 2012-08-16 08:20:51 [\\\] has quit (Remote host closed the connection)
 624 2012-08-16 08:23:02 egecko has quit (Quit: ~ Trillian Astra - www.trillian.im ~)
 625 2012-08-16 08:23:35 wasabi1 has quit (Read error: Connection reset by peer)
 626 2012-08-16 08:30:23 phantomcircuit has quit (Read error: Connection reset by peer)
 627 2012-08-16 08:37:00 Mobius_ has quit (Remote host closed the connection)
 628 2012-08-16 08:38:10 MobiusL has joined
 629 2012-08-16 08:38:32 TAiS46 has left ("Leaving")
 630 2012-08-16 08:42:26 CodeLion has quit (Read error: Connection reset by peer)
 631 2012-08-16 08:43:24 Marf has joined
 632 2012-08-16 08:45:04 iocor has quit (Quit: Computer has gone to sleep.)
 633 2012-08-16 08:45:32 tonikt has joined
 634 2012-08-16 08:47:13 Karmaon1 has quit (Ping timeout: 240 seconds)
 635 2012-08-16 08:51:21 Maged has quit (Read error: Connection reset by peer)
 636 2012-08-16 08:55:09 PiZZaMaN2K has joined
 637 2012-08-16 08:56:34 erska_ is now known as erska
 638 2012-08-16 08:58:29 SomeoneWeird has quit (Excess Flood)
 639 2012-08-16 09:00:27 TD has joined
 640 2012-08-16 09:00:58 Guest12691 has joined
 641 2012-08-16 09:01:03 Guest12691 has quit (Changing host)
 642 2012-08-16 09:01:03 Guest12691 has joined
 643 2012-08-16 09:01:29 t7 has joined
 644 2012-08-16 09:02:50 CodesInChaos has joined
 645 2012-08-16 09:05:08 yellowhat has joined
 646 2012-08-16 09:10:27 zveda has quit (Ping timeout: 245 seconds)
 647 2012-08-16 09:12:05 eb3fu has joined
 648 2012-08-16 09:12:20 eb3fu has quit (Client Quit)
 649 2012-08-16 09:12:52 eb3fu has joined
 650 2012-08-16 09:14:09 RainbowDashh has joined
 651 2012-08-16 09:17:14 iocor has joined
 652 2012-08-16 09:19:59 <cosurg1> ;;bc,halfreward
 653 2012-08-16 09:20:01 <gribble> Estimated time of bitcoin block reward halving: Tue Dec  4 01:24:00 2012 | Time remaining: 15 weeks, 5 days, 0 hours, 10 minutes, and 0 seconds
 654 2012-08-16 09:20:12 cosurg1 is now known as cosurgi
 655 2012-08-16 09:23:10 zveda has joined
 656 2012-08-16 09:23:10 zveda has quit (Changing host)
 657 2012-08-16 09:23:10 zveda has joined
 658 2012-08-16 09:25:52 denisx has quit (Quit: denisx)
 659 2012-08-16 09:26:23 RainbowDashh has quit (Quit: Computer has gone to sleep.)
 660 2012-08-16 09:26:42 Raccoon has joined
 661 2012-08-16 09:27:06 <Raccoon> FYI.  The segmnted progress bar makes it impossible to read progress bar text.
 662 2012-08-16 09:27:16 <Raccoon> I recommend going back to a solid progress bar.
 663 2012-08-16 09:28:06 denisx has joined
 664 2012-08-16 09:38:57 setkeh has quit (Ping timeout: 252 seconds)
 665 2012-08-16 09:39:34 <wump> that's a decision that qt makes, not our code
 666 2012-08-16 09:40:58 <wump> we just ask for a progress bar with text, and on some themes qt is so stupid to make it a segmented progress bar,that's really an upstream issue
 667 2012-08-16 09:43:16 jaxtr has quit (Read error: Connection reset by peer)
 668 2012-08-16 09:43:33 jaxtr has joined
 669 2012-08-16 09:43:54 <weex> no way to force it?
 670 2012-08-16 09:44:27 Guest12691 is now known as SomeoneWeird
 671 2012-08-16 09:44:33 Guest54468 has quit (Ping timeout: 240 seconds)
 672 2012-08-16 09:46:20 phantomcircuit_ has quit (Ping timeout: 260 seconds)
 673 2012-08-16 09:46:23 coingenuity has quit (Ping timeout: 276 seconds)
 674 2012-08-16 09:46:57 RazielZ has quit (Read error: Connection reset by peer)
 675 2012-08-16 09:47:04 phantomcircuit_ has joined
 676 2012-08-16 09:47:15 RazielZ has joined
 677 2012-08-16 09:48:16 coingenuity has joined
 678 2012-08-16 09:48:37 coingenuity is now known as Guest67008
 679 2012-08-16 09:49:08 Guest67008 has quit (Changing host)
 680 2012-08-16 09:49:09 Guest67008 has joined
 681 2012-08-16 09:50:52 Guest67008 is now known as coingenuity
 682 2012-08-16 09:52:27 <wump> it would be possible if someone could enumerate the exact cases in which this happens
 683 2012-08-16 09:52:37 <wump> ie, what the rendering plugin is
 684 2012-08-16 09:53:14 <wump> I'm sure that could be special-cased if we can detect it
 685 2012-08-16 09:54:05 <yellowhat> "The Bitcoin Project will be making a major announcement in September that should contribute some stability, he said. Still, he predicted things will continue to be exciting in the world of Bitcoin with "continued controversy and chaos.""
 686 2012-08-16 09:54:15 <yellowhat> any idea what gavin was hinting here?
 687 2012-08-16 09:58:28 <wump> ecuador will switch to bitcoin as official currency? lol :P let's save the rumors for the non-dev channel
 688 2012-08-16 10:00:20 LuaKT has joined
 689 2012-08-16 10:00:24 LuaKT has quit (Changing host)
 690 2012-08-16 10:00:24 LuaKT has joined
 691 2012-08-16 10:01:55 caedes has joined
 692 2012-08-16 10:01:59 Ken` has quit (Quit: Changing server)
 693 2012-08-16 10:02:05 ken___ has joined
 694 2012-08-16 10:02:05 ken___ is now known as Ken`
 695 2012-08-16 10:02:31 Ken` has quit (Client Quit)
 696 2012-08-16 10:02:38 ken___ has joined
 697 2012-08-16 10:02:38 ken___ is now known as Ken`
 698 2012-08-16 10:04:49 * yellowhat slaps yellowhat around a bit with a large trout
 699 2012-08-16 10:10:34 kokjo has joined
 700 2012-08-16 10:10:57 CodeLion has joined
 701 2012-08-16 10:11:27 <kokjo> hey! i have a problem with the bitcoin dumpprivkey format.
 702 2012-08-16 10:11:51 <kokjo> how do one tell if the publickey should be compressed or not?
 703 2012-08-16 10:11:58 <CodeLion> Hey everyone, good morning. So I've been up most of the night reading up on midstate... Can I ask a few questions to make sure I grasp the concept? Ironically I knew about it, just I didn't know the name.
 704 2012-08-16 10:12:03 <kokjo> it changes the address.
 705 2012-08-16 10:16:01 <kokjo> example the address 1huR1oV8uEE77HqQHCAuCzCzoq9HzXDSh, when i dump it with "bitcoind dumpprivkey 1huR1oV8uEE77HqQHCAuCzCzoq9HzXDSh" it gives L5TG3BkBgABJ1EXqznUSgRNXbdXwUpXEAEd6MDSFPsEnS5v2yX1i but when i regenerate the key in my own code the address is 1FvuiHLcEUhEtWwEe6aMLLaAPVngnqUY4T. but when i compress the publickey(remove one of the EC points) it gives the original address. BUT on the network there are both compressed pub
 706 2012-08-16 10:16:12 <kokjo> how do i know which one is correct?
 707 2012-08-16 10:17:09 <CodeLion> Midstate only appliest to the first hash right? the second time you run sha you don't worry about that?
 708 2012-08-16 10:17:42 <CodeLion> Also, still does anyone have a link to documentation about it? or even some psuedocode or logical analysis of a mining software?
 709 2012-08-16 10:18:03 <CodeLion> Bitcoins are so much more complex than just SHA256 D:
 710 2012-08-16 10:18:05 <kokjo> my assumption is that newer versions of the client uses the compressed format, and old version uses the uncompressed one, any ideas?
 711 2012-08-16 10:18:27 leotreasure has joined
 712 2012-08-16 10:19:11 <kokjo> CodeLion: skip the midstate, you have the whole block header anyway.
 713 2012-08-16 10:20:27 datagutt has joined
 714 2012-08-16 10:20:47 datagutt is now known as Guest92965
 715 2012-08-16 10:20:47 Detritus has joined
 716 2012-08-16 10:21:54 Guest92965 has quit (Client Quit)
 717 2012-08-16 10:22:21 datagutt_ has joined
 718 2012-08-16 10:23:13 <CodeLion> Kokjo: Hmm? Could you explain better? I'm utterly lost when it comes to the bitcoin part of mining. The SHA I have down easily but the bitcoins are... confusing. I keep hearing conflicting things >.<
 719 2012-08-16 10:23:42 <kokjo> CodeLion: sha256 only processes blocks of 64 bytes at one time, which means when that there is needed to sha256 operations to take in a full 80-byte bitcoin block. the midstate is the state after the first sha256 operation in round one.
 720 2012-08-16 10:24:24 <kokjo> by skipping the first sha256 operation(you already know the midstate), you save some time.
 721 2012-08-16 10:25:12 <CodeLion> right but for each block header you have to re-calculate the midstate, right?
 722 2012-08-16 10:25:15 <kokjo> btw. this is only the first round, the socund round is just a standart sha256.
 723 2012-08-16 10:25:45 <CodeLion> mmhm
 724 2012-08-16 10:25:50 <CodeLion> I assumed so for second round
 725 2012-08-16 10:26:14 <CodeLion> But in the first round, you first calculate the midstate and store it, then keep re-using it with different nonces?
 726 2012-08-16 10:26:22 <CodeLion> that would be right, right?
 727 2012-08-16 10:26:53 <kokjo> no, because the nonce is at the end of the bitcoin blocks, the first 64 bytes is static information.
 728 2012-08-16 10:27:03 enquirer has quit (Ping timeout: 240 seconds)
 729 2012-08-16 10:27:13 <CodeLion> I think we are saying the same thing with different words
 730 2012-08-16 10:27:35 <kokjo> CodeLion:yep i think so
 731 2012-08-16 10:27:52 <CodeLion> Once you obtain the midstate hash (We'll call it X) you just keep reloading X into your SHA while incrementing nonce?
 732 2012-08-16 10:28:06 <kokjo> yes!
 733 2012-08-16 10:28:09 <CodeLion> yay
 734 2012-08-16 10:28:12 * CodeLion wins
 735 2012-08-16 10:28:19 <CodeLion> Kokjo you recieve one internet
 736 2012-08-16 10:28:26 <CodeLion> thanks so much man
 737 2012-08-16 10:29:28 d4ve has quit (Quit: Nettalk6 - www.ntalk.de)
 738 2012-08-16 10:29:45 <kokjo> with hashlib in python(SLOW) you could:
 739 2012-08-16 10:30:12 <kokjo> hasher = sha256()
 740 2012-08-16 10:30:28 <kokjo> hasher.update("first 64 bytes")
 741 2012-08-16 10:30:39 <kokjo> midstate = hasher.copy()
 742 2012-08-16 10:31:04 <kokjo> hasher.update("rest")
 743 2012-08-16 10:31:19 <kokjo> sha256(hasher.digest()).digest()
 744 2012-08-16 10:31:29 <kokjo> then inc the nonce
 745 2012-08-16 10:31:33 <CodeLion> Ah I see
 746 2012-08-16 10:31:38 <kokjo> and:
 747 2012-08-16 10:31:39 <CodeLion> thanks for the python code
 748 2012-08-16 10:31:45 <kokjo> hasher = midstate.copy()
 749 2012-08-16 10:31:51 <CodeLion> oh cool
 750 2012-08-16 10:31:52 <CodeLion> hehe
 751 2012-08-16 10:32:10 <kokjo> but its python code so its slow...
 752 2012-08-16 10:32:12 <CodeLion> Well I'm certainly not using python
 753 2012-08-16 10:32:15 <CodeLion> xD
 754 2012-08-16 10:32:19 <CodeLion> I've got my own plans
 755 2012-08-16 10:32:22 <CodeLion> secret ones
 756 2012-08-16 10:32:34 <kokjo> 50% attack?
 757 2012-08-16 10:32:36 <CodeLion> But I've gotta duck out, might be back in here later
 758 2012-08-16 10:32:38 <CodeLion> and no lol
 759 2012-08-16 10:32:48 <CodeLion> Plans for a mining rig to sell.
 760 2012-08-16 10:32:50 <CodeLion> looool
 761 2012-08-16 10:32:54 <CodeLion> Me? do a 50%?
 762 2012-08-16 10:32:59 <CodeLion> no chance bro
 763 2012-08-16 10:33:08 * CodeLion is flat broke at the moment
 764 2012-08-16 10:33:17 CodeLion has left ("Donate bitcents: 1DQrnzjkA7svJ34qmx792BWnsS5THbTos8")
 765 2012-08-16 10:34:45 cdecker has quit (Remote host closed the connection)
 766 2012-08-16 10:34:45 BCBot has quit (Ping timeout: 246 seconds)
 767 2012-08-16 10:36:29 fronti has joined
 768 2012-08-16 10:37:57 BCBot has joined
 769 2012-08-16 10:42:37 CodesInChaos has quit (Ping timeout: 240 seconds)
 770 2012-08-16 10:56:14 setkeh has joined
 771 2012-08-16 10:56:20 setkeh has quit (Client Quit)
 772 2012-08-16 10:56:51 setkeh has joined
 773 2012-08-16 10:57:04 setkeh has quit (Client Quit)
 774 2012-08-16 10:57:19 setkeh has joined
 775 2012-08-16 10:57:59 tonikt has quit (Read error: Connection reset by peer)
 776 2012-08-16 11:02:26 caedes has quit (Ping timeout: 244 seconds)
 777 2012-08-16 11:15:00 caedes has joined
 778 2012-08-16 11:15:01 yellowhat has quit (Ping timeout: 252 seconds)
 779 2012-08-16 11:15:27 datagutt_ is now known as jsnazi
 780 2012-08-16 11:15:37 jsnazi is now known as datagutt
 781 2012-08-16 11:37:24 Turingi has quit (Read error: Connection reset by peer)
 782 2012-08-16 11:37:59 Clipse has joined
 783 2012-08-16 11:42:05 ThomasV has quit (Quit: Leaving)
 784 2012-08-16 11:44:32 vegard has joined
 785 2012-08-16 11:44:56 vegard is now known as Guest15374
 786 2012-08-16 11:47:00 <sipa> kokjo: the dump format for a compressed pubkey is 0x80 + 32 bytes secret + 0x01
 787 2012-08-16 11:49:24 <sipa> Luke-Jr: i don't see why including leveldb means including windows-specific crap; we do plan on switching to leveldb on all platforms, and since there are no packages for leveldb in any distribution/platform that i know off, it's not like we can link against a platform-provided version on those
 788 2012-08-16 11:49:41 <sipa> Luke-Jr: that you don't like it in the same repository is something else
 789 2012-08-16 11:52:48 caedes has quit (Read error: Connection reset by peer)
 790 2012-08-16 11:52:57 wereHams1er is now known as wereHamster
 791 2012-08-16 11:53:09 <kokjo> sipa: how does uncompressed look like?
 792 2012-08-16 11:53:35 <sipa> same, without the 0x01 at the end
 793 2012-08-16 11:53:40 caedes has joined
 794 2012-08-16 11:53:51 <kokjo> thanks! :)
 795 2012-08-16 11:54:41 <kokjo> sipa: does leveldb have database transactions?
 796 2012-08-16 11:55:19 <kokjo> i could find something about writebatches, but is it the same?
 797 2012-08-16 11:55:59 <sipa> only that
 798 2012-08-16 11:56:09 <sipa> db transactions requires a layer on top
 799 2012-08-16 11:56:13 <sipa> http://sourceforge.net/mailarchive/forum.php?thread_name=CAPg%2BsBhDFCjAn1tRRQhaudtqwsh4vcVbxzm%2BAA2OuFxN71fwUA%40mail.gmail.com&forum_name=bitcoin-development
 800 2012-08-16 11:56:19 <sipa> kokjo: see that link
 801 2012-08-16 11:59:01 <kokjo> awesome!
 802 2012-08-16 11:59:11 <kokjo> thank you.
 803 2012-08-16 12:01:12 Maccer has joined
 804 2012-08-16 12:04:01 [\\\] has joined
 805 2012-08-16 12:06:03 <justmoon> is there any actual reason why coinbases are pay-to-pubkey? would it be difficult to patch bitcoin to generate pay-to-address or P2SH coinbases instead?
 806 2012-08-16 12:06:18 <sipa> justmoon: not hard at all
 807 2012-08-16 12:06:30 <sipa> reason is historic, i guess
 808 2012-08-16 12:06:43 <justmoon> I'm thinking about this in the context of the awkward handstands that BlueMatt and TD are doing to make the bloomfilter support pay-to-pubkey
 809 2012-08-16 12:07:05 iocor has quit (Quit: Computer has gone to sleep.)
 810 2012-08-16 12:07:14 <sipa> i don't see the need for any special casing
 811 2012-08-16 12:07:44 <justmoon> you mean you don't understand why they added a special case for pay-to-pubkey?
 812 2012-08-16 12:07:46 <TD> justmoon: i think pay to pubkey is reasonable and should be fully supported
 813 2012-08-16 12:07:57 <TD> if anything pay to address is the historical anachronism
 814 2012-08-16 12:08:18 <TD> it's less efficient than pay-to-address in the absence of pruning. the only reason it exists is satoshi wanted something writable/typable
 815 2012-08-16 12:08:27 <TD> but most users move addresses around via links, qrcodes or copy/paste
 816 2012-08-16 12:08:46 <TD> at any rate, there are use cases for it in the contracts world (where you want to read the key out of the output so it can be used in other contexts)
 817 2012-08-16 12:08:58 <sipa> pay to address is also more secure, given use-once addresses
 818 2012-08-16 12:09:29 <justmoon> the problem with pay-to-pubkey is that you don't know what an input is spending from an indexing perspective
 819 2012-08-16 12:09:42 <sipa> ?
 820 2012-08-16 12:10:20 <justmoon> if you give me a P2SH input or a pay-to address or pay to multisig input, I can generate a hash and put that transaction in my index for that hash
 821 2012-08-16 12:10:37 <justmoon> without looking up the corresponding output in my database
 822 2012-08-16 12:10:57 <t7> ah
 823 2012-08-16 12:11:00 <sipa> not following
 824 2012-08-16 12:11:26 <justmoon> let me concoct a examples
 825 2012-08-16 12:11:46 <justmoon> an* example*
 826 2012-08-16 12:12:16 <justmoon> so let's say I'm a stateless node watching the block chain
 827 2012-08-16 12:12:21 <justmoon> I have no database
 828 2012-08-16 12:12:27 <justmoon> a transaction comes in
 829 2012-08-16 12:12:44 <justmoon> if it's a transaction spending a pay-to-address output, then it has the public key in the input
 830 2012-08-16 12:12:55 <justmoon> I can calculate the address from the pubkey and index it, great
 831 2012-08-16 12:13:03 caedes has quit (Read error: Connection reset by peer)
 832 2012-08-16 12:13:16 <sipa> but what do you index, without a database?
 833 2012-08-16 12:13:42 caedes has joined
 834 2012-08-16 12:13:44 <justmoon> the reason I personally care about it is because a DHT has a sharded database
 835 2012-08-16 12:13:58 <justmoon> so I may want to index all transactions affecting addresses starting with 0A
 836 2012-08-16 12:14:19 <justmoon> but I don't have all outputs outside of my shard to do it
 837 2012-08-16 12:14:56 <sipa> ah, you're building an address-to-txid map, but don't have fast access to it yourself, basically
 838 2012-08-16 12:15:17 <justmoon> yes, basically
 839 2012-08-16 12:15:34 <sipa> but can't you say: i index by hash(txoutscript)
 840 2012-08-16 12:16:10 <sipa> and you store any transaction that has outputs matching this, or all transactions that have an input that is already in your local shard?
 841 2012-08-16 12:19:03 [\\\] has quit (Ping timeout: 246 seconds)
 842 2012-08-16 12:19:59 <justmoon> hmm, then you'd have to look at your local shard, also you'd be including other transactions - it might be plausible, but I'm not sure how it interacts with the filtered routing stuff etc.
 843 2012-08-16 12:20:19 <sipa> including other transactions?
 844 2012-08-16 12:20:22 <justmoon> i.e. there are some cases where you want to relay a transaction to the nodes that would index it, even though you yourself aren't indexing it
 845 2012-08-16 12:20:40 <justmoon> yes, you said in addition to storing the transactions in your shard you would be storing transactions that refer to it
 846 2012-08-16 12:20:59 MC-Eeepc has joined
 847 2012-08-16 12:21:02 <sipa> that is what you want, no?
 848 2012-08-16 12:21:45 MC-Eeepc has quit (Read error: Connection reset by peer)
 849 2012-08-16 12:21:55 <sipa> instead of trying to figure oit what the txout was from the txin, you just check whether it refers to a txout that you know already interests you
 850 2012-08-16 12:24:03 <sipa> you can still add the special cases for P2SH and pubkeyhash for performance, if you must
 851 2012-08-16 12:24:14 <TD> justmoon: that's true, but i don't think we should design bitcoin around the needs of a DHT (at least not yet)
 852 2012-08-16 12:24:16 <justmoon> yes, but you are indexing along several differnt lines - all txs whose hashes start with AA, all address indexes (NOT the corresponding transactions) where the address starts with AA, all blocks (NOT the full transactions it contains) that start with AA
 853 2012-08-16 12:24:40 <justmoon> if you start adding cross dependencies you are adding lots more stuff - it might be ok since you can just make your shard slimmer
 854 2012-08-16 12:24:42 <TD> the block chain is a storage/bandwidth sensitive structure. so repeating things (which is what pay-to-address and p2sh do) isn't great, to start with ...
 855 2012-08-16 12:24:52 <justmoon> but intuitively it seems you get very uneven sharding
 856 2012-08-16 12:25:05 <justmoon> like transactions that everyone knows because they are references from tons of shards etc.
 857 2012-08-16 12:26:04 <justmoon> TD: what matters from a scalability perspective is only the size of the output, not the total size
 858 2012-08-16 12:26:11 <sipa> justmoon: i mean you can always know whether an input is from an address you are selecting, by checking whether you already have its prevout
 859 2012-08-16 12:27:05 <sipa> TD: i'm not convinced storage of the full blockchain will remain the most resource-intensive part; the size of the txout set seems more essential
 860 2012-08-16 12:27:43 <sipa> justmoon: (and that txout interests you)
 861 2012-08-16 12:28:16 <justmoon> sipa: yes, I get your idea, it just requires way too many changes to my model to be able to tell you if it's better or not
 862 2012-08-16 12:28:49 caedes has quit (Read error: Connection reset by peer)
 863 2012-08-16 12:28:49 <sipa> afaics, it does exactly what you're planning to do, but i am probably missing soke details
 864 2012-08-16 12:28:52 <sipa> some
 865 2012-08-16 12:29:10 * justmoon is thinking
 866 2012-08-16 12:29:20 caedes has joined
 867 2012-08-16 12:29:26 <sipa> bit with hash(txoitscript) instead of address, as selecting variable
 868 2012-08-16 12:29:42 <justmoon> basically what you'd need to do is keep a set of outpoints in your index, hmm
 869 2012-08-16 12:29:43 <sipa> but you can calculate hash(txoutscript) from any kind of address
 870 2012-08-16 12:30:18 <justmoon> in order to look up - hey this input references an output in my index, I need to know what outputs are in my index
 871 2012-08-16 12:30:31 <sipa> right, indeed
 872 2012-08-16 12:30:52 <sipa> i was assuming you already had that, in order to serve things from it
 873 2012-08-16 12:31:11 <sipa> but obviously you're looking up by address, and not by outpoomt
 874 2012-08-16 12:31:14 <sipa> oitpoint
 875 2012-08-16 12:31:26 <sipa> oUtpoint
 876 2012-08-16 12:31:33 MC-Eeepc has joined
 877 2012-08-16 12:31:44 * sipa has some typing deficiency when using a tablet
 878 2012-08-16 12:32:17 <justmoon> basically as a querying node I would find a node that has the *index* for the address i'm interested, but the actual transactions are retrieved from a whole number of different nodes
 879 2012-08-16 12:32:48 <justmoon> there is no performance benefit in getting everything from one node - if anything you want to paralellize and spread load as much as possible
 880 2012-08-16 12:33:58 egecko has joined
 881 2012-08-16 12:35:53 <justmoon> as far as the DHT is concerned - thanks to ultraprune I see very few roadblocks to its implementation: you can initialize ultraprune at some checkpoint and download full blocks from there
 882 2012-08-16 12:36:01 <justmoon> for the rest of the chain you download only filtered blocks
 883 2012-08-16 12:36:44 <justmoon> the only new attack against this compared to a node that downloads the world is that when download the filtered blocks we don't know if the other node is leaving something out
 884 2012-08-16 12:37:19 <sipa> but then you lose any trustability
 885 2012-08-16 12:37:35 <sipa> if you don't bootstrap from scratch with full blocks
 886 2012-08-16 12:37:55 <justmoon> well, the only thing you don't know for sure is whether your index is complete or not
 887 2012-08-16 12:38:03 <sipa> or correct
 888 2012-08-16 12:38:10 <justmoon> you do know it's correct
 889 2012-08-16 12:38:18 <sipa> if you start from a faulty ultrapruned set
 890 2012-08-16 12:38:30 <sipa> you have no way of validating it
 891 2012-08-16 12:38:33 <justmoon> the ultrapruned set would be hashed at checkpoints
 892 2012-08-16 12:38:47 <sipa> right, you can do that
 893 2012-08-16 12:39:02 <justmoon> someday maybe some mtuo hash in blocks or whatever, but for now just a verified set at certain points is fine
 894 2012-08-16 12:39:21 <justmoon> when I mean you don't know whether it's complete I'm talking about the main index, not ultraprune
 895 2012-08-16 12:39:28 <sipa> sure
 896 2012-08-16 12:39:51 <sipa> but anything that starts from not the full blocks, cannot have 100% trust in
 897 2012-08-16 12:39:56 <sipa> +you
 898 2012-08-16 12:40:20 <justmoon> correct, I don't think there are any nodes in operation anymore which don't already make that tradeoff
 899 2012-08-16 12:40:38 <sipa> for signatures checks, true
 900 2012-08-16 12:40:53 <sipa> but for completeness of the index, no
 901 2012-08-16 12:41:20 <justmoon> well, as I said: justmoon> the only new attack against this compared to a node that downloads the world is that when download the filtered blocks we don't know if the other node is leaving something out
 902 2012-08-16 12:41:31 slush1 has quit (Ping timeout: 248 seconds)
 903 2012-08-16 12:42:06 <justmoon> fortunately completeness of the index is not particularly security sensitive - I haven't thought about it too much yet, but I don't think it enables any profitable attacks, just DoS
 904 2012-08-16 12:42:18 <sipa> you're downloading filtered blocks?
 905 2012-08-16 12:42:32 <justmoon> yes
 906 2012-08-16 12:42:39 <sipa> who filters them?
 907 2012-08-16 12:42:45 <justmoon> any node out there
 908 2012-08-16 12:42:50 <sipa> how?
 909 2012-08-16 12:43:10 <justmoon> what are you getting at?
 910 2012-08-16 12:43:25 <sipa> i'm just trying to understand
 911 2012-08-16 12:43:44 <justmoon> I would download 1. whatever shard of the index I want to server and 2. the stuff I'm personally interested in like my addresses and transactions
 912 2012-08-16 12:44:08 <sipa> yes but how do you filter?
 913 2012-08-16 12:44:34 <sipa> how do you say "i want all transactions whose hash starts with aa"
 914 2012-08-16 12:45:34 rdponticelli has joined
 915 2012-08-16 12:45:43 <justmoon> for the purpose of the discussion I'll assume this algorithm: I find a node (via kademlia find_node) that is close to the transaction I'm interested in and then query it
 916 2012-08-16 12:45:56 <justmoon> for mass download there would be a separate API to find a node that has the shard you want
 917 2012-08-16 12:46:08 <justmoon> and just sends it to you
 918 2012-08-16 12:46:32 <sipa> hmm
 919 2012-08-16 12:46:37 <justmoon> the shard does include some verification information like merkle branches so you can verify the transactions actually exist
 920 2012-08-16 12:49:04 Marf has quit (Read error: Connection reset by peer)
 921 2012-08-16 12:49:31 Marf has joined
 922 2012-08-16 12:51:02 caedes has quit (Read error: Connection reset by peer)
 923 2012-08-16 12:51:28 <justmoon> I gotta go to zurich for a bit, feel free to email me if you have any new insights on what I'm trying to do :)
 924 2012-08-16 12:51:35 caedes has joined
 925 2012-08-16 12:51:37 <justmoon> ciao
 926 2012-08-16 12:51:41 justmoon has quit (Quit: Leaving)
 927 2012-08-16 12:55:35 minimoose has joined
 928 2012-08-16 13:05:26 iocor has joined
 929 2012-08-16 13:08:00 caedes has quit (Read error: Connection reset by peer)
 930 2012-08-16 13:08:38 caedes has joined
 931 2012-08-16 13:09:27 zveda has quit (Quit: Ex-Chat)
 932 2012-08-16 13:09:44 zveda has joined
 933 2012-08-16 13:09:44 zveda has quit (Changing host)
 934 2012-08-16 13:09:45 zveda has joined
 935 2012-08-16 13:10:17 pecket has quit (Ping timeout: 240 seconds)
 936 2012-08-16 13:11:24 <sipa> anyone know of a way to profile/analyse where a program spends most time *waiting* ?
 937 2012-08-16 13:12:45 <sipa> importing blocks in ultraprune sometimes, for up to 10-20 seconds, works very slowly, with almost no cpu usage
 938 2012-08-16 13:15:27 dvide has quit ()
 939 2012-08-16 13:18:25 <gmaxwell> strace in time diff mode. sort the output. That will show the time it spent in blocking syscalls.
 940 2012-08-16 13:20:23 smiddi has joined
 941 2012-08-16 13:20:37 <Eliel> :/ It's a big problem that the most visible bitcoin software has one of the worst user experiences.
 942 2012-08-16 13:20:59 <Eliel> it happens relatively often that you can no longer get bitcoin-qt to start.
 943 2012-08-16 13:21:09 <Eliel> customer support nightmare. :)
 944 2012-08-16 13:22:04 <gmaxwell> Eliel: erp.
 945 2012-08-16 13:22:17 <gmaxwell> Eliel: if this is often why are there no issues open on it?
 946 2012-08-16 13:23:16 slush1 has joined
 947 2012-08-16 13:24:01 <Eliel> because there's not enough data to usefully open issues.
 948 2012-08-16 13:24:35 <Eliel> this is all I got from this time EXCEPTION: 22DbRunRecoveryExveption            DbEnv::open: DB_RUNRECOVERY: Fatal error, run database  recovery C:\users\omistaja\paska\bitcoin..... in Runaway exception
 949 2012-08-16 13:24:42 RazielZ has quit (Read error: Connection reset by peer)
 950 2012-08-16 13:25:21 <gmaxwell> You absolutely have enough data to open an issue! I mean, you can reproduce it so all data can be gathered.
 951 2012-08-16 13:25:37 <gmaxwell> What is in the debug log before that happens?
 952 2012-08-16 13:25:38 <Eliel> I can't, the customer who contacted us possibly could.
 953 2012-08-16 13:25:55 <Eliel> but I can't contact him until tomorrow I expect.
 954 2012-08-16 13:26:24 <Eliel> but he won't have interest as he's mostly interested in how to export the keys and switch to multibit.
 955 2012-08-16 13:26:34 RazielZ has joined
 956 2012-08-16 13:28:01 one_zero has quit ()
 957 2012-08-16 13:29:32 <gmaxwell> Eliel: So when multibit has some error in the future will you actually report it usefully or will the user just keep switching clients until they ultimately end up with some malware that takes all their coins and thus solves the problem?
 958 2012-08-16 13:30:32 <Eliel> If I can get enough data to actually report it from the user, then sure.
 959 2012-08-16 13:32:17 <Eliel> this db-corruption exception that completely prevents bitcoin-qt from running at all is rather bad though.
 960 2012-08-16 13:32:31 <Eliel> Is there no way to try to automatically recover from it?
 961 2012-08-16 13:32:46 caedes has quit (Read error: Connection reset by peer)
 962 2012-08-16 13:32:52 benjamindees has joined
 963 2012-08-16 13:33:08 abracadabopoulos is now known as abracadabra
 964 2012-08-16 13:33:11 <gmaxwell> Eliel: You're asking an impossible question. Without a way to reproduce it that question can't be answered.
 965 2012-08-16 13:33:12 caedes has joined
 966 2012-08-16 13:33:31 <gmaxwell> I don't even know _which_ database is corrupted.
 967 2012-08-16 13:33:56 <Eliel> well, that says something about the error message bitcoind/bitcoin-qt gives when it fails to start :P
 968 2012-08-16 13:34:30 <Eliel> I'll file an issue on this. About the error message.
 969 2012-08-16 13:37:04 danbri has joined
 970 2012-08-16 13:37:53 <benjamindees> I notice that this organization, the "Complementary Currency Research Center" has a database of alternative currencies, http://www.complementarycurrency.org/
 971 2012-08-16 13:38:35 <benjamindees> And that there is an entry for Bitcoin already, added by Gavin or by someone using his e-mail address
 972 2012-08-16 13:39:30 <benjamindees> In it, Bitcoin is described as being valued by "Non-Contract Promise to Guarantee Currency"
 973 2012-08-16 13:39:45 <benjamindees> Which is one of several options
 974 2012-08-16 13:40:44 <benjamindees> Another of which is "Free-floating against Guaranteed Scarcity"
 975 2012-08-16 13:42:25 <benjamindees> It seems to me that the latter option is a better description of Bitcoin than the one which was chosen.
 976 2012-08-16 13:42:59 <benjamindees> Unless I'm missing something.
 977 2012-08-16 13:43:31 <gmaxwell> Well, I don't know what the former means; but yes, the latter sounds correct.
 978 2012-08-16 13:44:51 <benjamindees> I imagine the former is meant to describe something like an informal peg.
 979 2012-08-16 13:46:30 caedes has quit (Read error: Connection reset by peer)
 980 2012-08-16 13:47:27 ThomasV has joined
 981 2012-08-16 13:49:30 wasabi1 has joined
 982 2012-08-16 13:49:40 <benjamindees> The organization also apparently held a conference on energy-based currencies.
 983 2012-08-16 13:51:45 Cory has quit (Read error: Connection reset by peer)
 984 2012-08-16 13:52:38 Cory has joined
 985 2012-08-16 13:53:31 Turingi has joined
 986 2012-08-16 14:00:34 <wump> I really think that in case of a fatal db exception we should nuke all the files (except the wallet, obviously) automatically and re-start the block chain sync
 987 2012-08-16 14:01:01 <helo> Eliel: i had a problem like that when i ran out of disk space a few months back
 988 2012-08-16 14:01:03 <wump> it happens all the time that people somehow manage to corrupt the db (ie due to unclean shutdown) and don't know what to do and panic
 989 2012-08-16 14:01:26 <gmaxwell> wump: this would be specacularly unhelpful if the problem is actually with the wallet.
 990 2012-08-16 14:01:36 <wump> but usually it isn't
 991 2012-08-16 14:01:45 <gmaxwell> and, again, if this is 'all the time' why can't we get a reproducable test case?
 992 2012-08-16 14:01:51 <wump> obviously it should be detected in what db it happens
 993 2012-08-16 14:02:00 <wump> if it's the wallet it should display a different error
 994 2012-08-16 14:02:22 danbri has quit (Remote host closed the connection)
 995 2012-08-16 14:02:31 <wump> well, I suppose that running bitcoin while power-cycling your pc all the time  will work
 996 2012-08-16 14:02:44 <gmaxwell> helo: hm. We shut down when there is <50mb free; but I'm not sure what happens if you start with 0 free.
 997 2012-08-16 14:03:00 wumpus is now known as wumpus
 998 2012-08-16 14:03:02 caedes has joined
 999 2012-08-16 14:03:09 [\\\] has joined
1000 2012-08-16 14:03:22 * sipa hopes leveldb proves to be a silver bullet
1001 2012-08-16 14:03:55 benjamindees has left ("Leaving")
1002 2012-08-16 14:04:18 <sipa> actually, i believe many of our problems come from the fact that we detach databases (previously all of them, now only the wallet) at shutdown, which is exactly when most pc's lose power
1003 2012-08-16 14:04:34 <sipa> bdb is intended to protect against failing hardware
1004 2012-08-16 14:04:49 <sipa> but not when detaching a file from the environment, maybe
1005 2012-08-16 14:06:01 Cusipzzz has joined
1006 2012-08-16 14:08:10 <MC-Eeepc> why doesnt bitcoin auto submit crash dumps or whatever to the mothership
1007 2012-08-16 14:08:12 <wumpus> the block chain should really be considered temporary state, and being unable to read it should not be a fatal error
1008 2012-08-16 14:08:41 <wumpus> because no one is paying for a mothership MC-Eeepc
1009 2012-08-16 14:08:51 <gmaxwell> MC-Eeepc: Because sending out the users private keys is a fantastic idea.
1010 2012-08-16 14:09:01 <sipa> gmaxwell: hey! let's do that!
1011 2012-08-16 14:09:02 <MC-Eeepc> yeah not they keys
1012 2012-08-16 14:09:07 <wumpus> hey, good idea
1013 2012-08-16 14:09:17 danbri has joined
1014 2012-08-16 14:09:17 <wumpus> with the keys we could pay servers and development and stuff
1015 2012-08-16 14:09:29 <gmaxwell> MC-Eeepc: it's really hard to avoid doign that if it were sending actual crash dumps.
1016 2012-08-16 14:09:36 kokjo has quit (Ping timeout: 245 seconds)
1017 2012-08-16 14:10:21 <wumpus> well if you only send a backtrace and error message it should be fine, but of limited usablility
1018 2012-08-16 14:10:24 <gmaxwell> sipa: so, in the past when I've helped people with startup errors like that, it was often the addr.dat that was busted.
1019 2012-08-16 14:10:24 <MC-Eeepc> well i probqbly dont actually know what a crash dump rally is
1020 2012-08-16 14:10:34 <MC-Eeepc> so lets just say useful data
1021 2012-08-16 14:10:44 <gmaxwell> MC-Eeepc: a snapshot of the process memory at the time of crash. Which is indeed very useful.
1022 2012-08-16 14:10:55 enquirer has joined
1023 2012-08-16 14:10:56 <gmaxwell> Anything less is nowhere near as useful. :)
1024 2012-08-16 14:11:16 <MC-Eeepc> what is debuglog then
1025 2012-08-16 14:11:19 p0s has joined
1026 2012-08-16 14:11:20 <wumpus> it can still be useful for statistics etc
1027 2012-08-16 14:11:38 <sipa> MC-Eeepc: just a log file with events
1028 2012-08-16 14:12:40 <gmaxwell> I really believe that someone made a block device for linux that basically stored a log of all write operations... so you could tear a FS at every point and see that it recovered.
1029 2012-08-16 14:12:55 <gmaxwell> But I don't remember what it was called, and can't find anything like it.
1030 2012-08-16 14:13:07 caedes has quit (Read error: Connection reset by peer)
1031 2012-08-16 14:13:51 caedes has joined
1032 2012-08-16 14:15:06 <sipa> gmaxwell: hmm, but that doesn't tell me whether it was actual sleeping in between those calls, or intensive userspace cpu usage in between
1033 2012-08-16 14:15:22 <sipa> still, i see many close() calls that take close to 40ms
1034 2012-08-16 14:16:00 CodesInChaos has joined
1035 2012-08-16 14:16:51 <gmaxwell> sipa: if the call itself takes time thats all you need to know, I think. e.g. a process sleeping or failling into a blocking select or whatever, will show the time on the respective syscall.
1036 2012-08-16 14:17:50 <sipa> gmaxwell: oh, -T, not -r
1037 2012-08-16 14:18:00 <sipa> -r is the difference between the beginning of syscalls
1038 2012-08-16 14:18:30 <Cusipzzz> Are there any known bugs with 6.3 bitcoind randomly crashing after trying to execute multiple rpc commands?
1039 2012-08-16 14:18:46 <sipa> Cusipzzz: no, please report if you can reproduce such behavior
1040 2012-08-16 14:19:36 <Cusipzzz> sipa: ok, thanks, will try to replicate. one of my nodes keeps going down, another is fine running the same application
1041 2012-08-16 14:20:37 darkskiez has joined
1042 2012-08-16 14:25:08 copumpkin has quit (Quit: Computer has gone to sleep.)
1043 2012-08-16 14:26:48 abracadabra has quit (Changing host)
1044 2012-08-16 14:26:48 abracadabra has joined
1045 2012-08-16 14:32:53 caedes has quit (Read error: Connection reset by peer)
1046 2012-08-16 14:34:06 caedes has joined
1047 2012-08-16 14:38:23 <sipa> gmaxwell: got it!
1048 2012-08-16 14:38:44 <sipa> if (!IsInitialBlockDownload() || (nBestHeight+1) % 500 == 0) FileCommit(fileout);
1049 2012-08-16 14:39:02 <sipa> now, when doing batch processing of blocks, nBestHeight is only updated at batches
1050 2012-08-16 14:39:24 <sipa> which means that if you're unlucky, it's called for many blocks in a row
1051 2012-08-16 14:40:32 <gmaxwell> obviously you need to use %499 there. ;)
1052 2012-08-16 14:40:53 <wumpus> either that or it's never called at all
1053 2012-08-16 14:45:07 <sipa> using fdatasync instead of fsync in FileCommit also seems to help a lot
1054 2012-08-16 14:45:34 caedes has quit (Read error: Connection reset by peer)
1055 2012-08-16 14:46:19 caedes has joined
1056 2012-08-16 14:46:34 copumpkin has joined
1057 2012-08-16 14:46:39 D34TH has joined
1058 2012-08-16 14:55:41 localhost has joined
1059 2012-08-16 14:55:41 localhost has quit (Excess Flood)
1060 2012-08-16 14:55:44 topace has quit (Quit: https://www.canadianbitcoins.com)
1061 2012-08-16 14:55:55 topace has joined
1062 2012-08-16 14:56:22 <TD> BlueMatt: hey, are you going to bring your DNS seed back up at some point?
1063 2012-08-16 14:57:33 MC-Eeepc has quit (Ping timeout: 244 seconds)
1064 2012-08-16 14:58:13 MC-Eeepc has joined
1065 2012-08-16 14:59:51 localhost has joined
1066 2012-08-16 15:04:12 caedes has quit (Read error: Connection reset by peer)
1067 2012-08-16 15:04:43 caedes has joined
1068 2012-08-16 15:10:10 <BlueMatt> TD: yea, I was planning on it today
1069 2012-08-16 15:10:28 <TD> cool
1070 2012-08-16 15:10:29 <TD> thanks
1071 2012-08-16 15:11:01 <BlueMatt> TD: re: ml, yes I am going to add tx merkle branch support, but the reason to do it also as a v<hash> structure for regular nodes is to make block propagation go faster as well
1072 2012-08-16 15:11:23 <sipa> BlueMatt: my dns seed has been running quite stably for the past few days, with latest git code of bitcoin-seeder
1073 2012-08-16 15:11:35 <BlueMatt> sipa: nice, Ill just use that then
1074 2012-08-16 15:11:47 <sipa> if you're interested; i suspect it'll take a while before i start rewriting parts of it again
1075 2012-08-16 15:12:10 caedes has quit (Read error: Connection reset by peer)
1076 2012-08-16 15:12:26 <TD> BlueMatt: yes, right. that's also a good optimization. but for SPV clients a CMerkleBlock structure still makes sense as we want to optimize roundtripping
1077 2012-08-16 15:12:28 <TD> not just bandwidth
1078 2012-08-16 15:12:49 PhantomSpark has joined
1079 2012-08-16 15:13:23 caedes has joined
1080 2012-08-16 15:13:56 <BlueMatt> TD: yep
1081 2012-08-16 15:15:45 <BlueMatt> TD: in terms of having to update the filter after a tx match is found...I suppose you could do a rule to automatically add outputs and txes which match the filter to the filter, though Im not sure how deterministic that is for payment types that are not (currently) standard?
1082 2012-08-16 15:16:46 danbri_ has joined
1083 2012-08-16 15:18:15 PiZZaMaN2K has quit (Ping timeout: 246 seconds)
1084 2012-08-16 15:18:45 <jgarzik> BlueMatt, TD: it sounds like bloom filter stuff is still under discussion, so I will make a small, separate BIP for 'mempool'+'getdata'
1085 2012-08-16 15:18:45 <TD> well the point of matching on data elements is weird scripts still get matched against correctly, and if you add the tx to the filter when it's matched, the spending transactions will also be matched regardless of what the inputs look like
1086 2012-08-16 15:19:31 <BlueMatt> jgarzik: sounds good
1087 2012-08-16 15:19:52 danbri has quit (Ping timeout: 265 seconds)
1088 2012-08-16 15:19:53 <BlueMatt> TD: ?
1089 2012-08-16 15:20:49 <BlueMatt> TD: Im thinking of a p2pubkey that matches the filter, you care when it gets spent, so to download a block after match, you need to add something to the filter afterwards with that tx
1090 2012-08-16 15:21:24 <TD> yes, exactly. if we say that a matching tx has its hash added to the filter (or list of interesting transactions, doesn't have to actually modify the bloom filter), then when that interesting output is spent you are guaranteed to see it
1091 2012-08-16 15:22:30 <sipa> sounds like a good idea
1092 2012-08-16 15:22:55 <BlueMatt> yea, we are saying the same thing...Id say thats probably a pretty good solution, I only asked if you know of a case where it doesnt make sense in some odd script type?
1093 2012-08-16 15:22:56 <sipa> but don't we already match on prevout scripts for txins?
1094 2012-08-16 15:23:19 Fanquake has joined
1095 2012-08-16 15:23:23 <BlueMatt> for regular txn, yes, for in blocks, I decided no, but thats up for discussion
1096 2012-08-16 15:23:45 <BlueMatt> (for regular txn, we always have the inputs, in blocks, we dont)
1097 2012-08-16 15:23:49 <sipa> somehow i prefer this solution, and have the ability to also send an update "from now on, i'm also interested in transactions which spend outputs of <txid>"
1098 2012-08-16 15:24:12 <BlueMatt> you mean and remove all prevout matching?
1099 2012-08-16 15:24:33 <sipa> if that suffices, i'd prefer that, i think
1100 2012-08-16 15:24:41 <sipa> but TD probably understands the use cases better
1101 2012-08-16 15:25:13 denisx_ has joined
1102 2012-08-16 15:25:19 <TD> hmm
1103 2012-08-16 15:25:27 <TD> i think the prevout matching is still needed actually
1104 2012-08-16 15:25:36 <TD> actually, maybe not
1105 2012-08-16 15:25:53 <TD> basically it means you can't find relevant transactions in a segment of chain unless you fully synced all the way up to the start of that segment
1106 2012-08-16 15:26:09 <TD> but i can't think of a use case for wanting to do that, given that the data you extract may not be usable if you don't have a properly synced wallet
1107 2012-08-16 15:26:15 ThomasV has quit (Quit: Quitte)
1108 2012-08-16 15:26:45 <BlueMatt> yes, but even then, that would only help if you match prevouts in blocks
1109 2012-08-16 15:26:51 <BlueMatt> (which I think could get too expensive)
1110 2012-08-16 15:27:25 <sipa> indeed
1111 2012-08-16 15:28:25 denisx has quit (Ping timeout: 246 seconds)
1112 2012-08-16 15:28:25 denisx_ is now known as denisx
1113 2012-08-16 15:28:31 <TD> hmm
1114 2012-08-16 15:28:33 <TD> ok then
1115 2012-08-16 15:32:10 <BlueMatt> alright, Ill look into implementation later today too
1116 2012-08-16 15:37:31 Fanquake has quit (Quit: Fanquake)
1117 2012-08-16 15:38:55 MC-Eeepc has quit (Ping timeout: 246 seconds)
1118 2012-08-16 15:42:48 jurov is now known as away!xzbnxup@84.245.71.31|jurov
1119 2012-08-16 15:50:34 MC-Eeepc has joined
1120 2012-08-16 15:57:06 user__ has joined
1121 2012-08-16 15:57:53 t7 has quit (Quit: ChatZilla 0.9.88-rdmsoft [XULRunner 1.9.0.17/2009122204])
1122 2012-08-16 15:59:27 Fanquake has joined
1123 2012-08-16 16:00:59 Fanquake has left ()
1124 2012-08-16 16:01:07 iocor has quit (Quit: Computer has gone to sleep.)
1125 2012-08-16 16:07:24 <sipa> hmm, 13m15 to load 185k blocks
1126 2012-08-16 16:13:12 MrTiggr has quit (Ping timeout: 246 seconds)
1127 2012-08-16 16:15:01 MrTiggr has joined
1128 2012-08-16 16:15:01 MrTiggr has quit (Excess Flood)
1129 2012-08-16 16:15:30 Guest82021 has joined
1130 2012-08-16 16:19:41 <sipa> 9m19 to load 185k blocks, with -dbcache=200
1131 2012-08-16 16:20:25 toffoo has joined
1132 2012-08-16 16:21:16 d4ve has joined
1133 2012-08-16 16:21:22 iocor has joined
1134 2012-08-16 16:22:22 darkskiez has quit (Remote host closed the connection)
1135 2012-08-16 16:22:47 denisx has quit (Quit: denisx)
1136 2012-08-16 16:24:00 ultrixx has quit (Quit: Verlassend)
1137 2012-08-16 16:26:32 <jgarzik> sipa: I wonder what, precisely, bdb caches.  If it is just purely disk blocks, then there should be no difference between bdb cache and kernel pagecache.  However, if bdb caches some processing as well as data, it makes sense.
1138 2012-08-16 16:27:03 <jgarzik> e.g. pynode only needs to cache blocks because python is so damn slow (relatively speaking) deserializing a block
1139 2012-08-16 16:28:00 <sipa> i suppose there are index lookups to find the data on disk, if it's not in cache
1140 2012-08-16 16:28:11 <sipa> and when the index itself is not in cache either, it gets worse
1141 2012-08-16 16:29:10 <TD> interesting
1142 2012-08-16 16:29:10 <TD> d
1143 2012-08-16 16:29:17 <TD> did you see the allegations about mintchip on slashdot?
1144 2012-08-16 16:29:21 inlikeflynn has quit ()
1145 2012-08-16 16:29:53 <TD> comes from an anon coward, so it's hard to say if it's for real, but it sounds somewhat plausible off hand
1146 2012-08-16 16:29:53 <TD> http://news.slashdot.org/comments.pl?sid=3051283&cid=41008501
1147 2012-08-16 16:29:58 PK has joined
1148 2012-08-16 16:30:04 <sipa> 5m43 with -dbcache=25 but on tmpfs
1149 2012-08-16 16:31:48 pecket has joined
1150 2012-08-16 16:32:01 <gmaxwell> TD: It's the same fud I would produce, if I were in the fud producing business. One signficant weakness of opaque systems like that is that people can invent all sorts of fud that you simply can't disprove.
1151 2012-08-16 16:32:15 <TD> yeah
1152 2012-08-16 16:32:20 usagi has joined
1153 2012-08-16 16:32:27 <TD> it does seem rather difficult to prove one way or another, but i must admit i wondered about the TAC when first reading the docs
1154 2012-08-16 16:32:38 <TD> it simply says it's "additional data that can be used by the mint to check authenticitiy"
1155 2012-08-16 16:32:59 <usagi> Thanks for the info BCBot! Even though I know you are a bot, it's good to be thankful!
1156 2012-08-16 16:32:59 <sipa> that does at least imply it contains some data that normal users can't use?
1157 2012-08-16 16:33:09 <gmaxwell> TD: right, I did too! :)  Really, if its true its probably good news for mintchip. Otherwise any break of the hardware is a class break.
1158 2012-08-16 16:33:58 <TD> even if you could identify (probabilistically) the chip that was broken .... what then? you don't know where it is.
1159 2012-08-16 16:34:00 Diablo-D3 has joined
1160 2012-08-16 16:34:02 <TD> you don't know who owns it.
1161 2012-08-16 16:34:10 <TD> sipa: it's not much data
1162 2012-08-16 16:34:14 <gmaxwell> (I've sort of been waiting to see Mintchip<>btc onion sites; where someone with a honking electron microscope rig as reverse engineered a single mintchip and turned it into a cash fountain you can drink from over the net for modest bitcoin payments)
1163 2012-08-16 16:34:52 osxorgate has quit (Remote host closed the connection)
1164 2012-08-16 16:35:25 <sipa> TD: yes, 24 bytes is not much; but at least he does say probabilistic
1165 2012-08-16 16:35:29 <TD> yeah
1166 2012-08-16 16:36:21 <Diablo-D3> gmaxwell: heh
1167 2012-08-16 16:38:15 <sipa> 5m36s with -dbcache=200 on tmpfs
1168 2012-08-16 16:38:20 <gmaxwell> td: yea, I'm not sure how much that would help, because the onion-cash-fountain would be running in a wallwart computer hidden in a cafe in columbia or something... but at least they'd be able to measure the amount being faked and be able to decide when to do a reissue.
1169 2012-08-16 16:45:29 <jgarzik> sipa: thanks for the tmpfs numbers
1170 2012-08-16 16:45:41 <jgarzik> sipa: definitely what I expected
1171 2012-08-16 16:46:24 Maged has joined
1172 2012-08-16 16:46:26 iocor has quit (Quit: Computer has gone to sleep.)
1173 2012-08-16 16:48:48 <sipa> the working set of the coin database is around 120 MB near the end of the chain, so i assume that setting the dbcache close to that has the highest impact
1174 2012-08-16 16:49:15 <jgarzik> pagedb (my cheap python leveldb-similar database lib) just keeps the index in RAM at all times.  It's pretty cheap, since it's a single, linear sorted list of (key, file id) pairs.  With a pagedb block size of 4MB, you wind up with only 6 index entries (6 data pairs) in RAM.
1175 2012-08-16 16:49:59 <sipa> anyway, seems there is still some room for improvement on the database side; i'll start experimenting with leveld soon
1176 2012-08-16 16:50:33 <sipa> *leveldb
1177 2012-08-16 16:52:32 BTCTrader has joined
1178 2012-08-16 16:53:20 TD has quit (Quit: TD)
1179 2012-08-16 16:54:48 BTCTrader_ has quit (Ping timeout: 244 seconds)
1180 2012-08-16 16:57:51 paul0 has joined
1181 2012-08-16 17:00:08 <usagi> Hi guys, I'm trying to figure out the bitcoind source code, is there a guide somewhere which discusses the general flow of code?
1182 2012-08-16 17:00:58 <sipa> there is some information in https://bitcointalk.org/index.php?topic=41718.0
1183 2012-08-16 17:01:18 <sipa> you're free to ask questions, though :)
1184 2012-08-16 17:01:27 iocor has joined
1185 2012-08-16 17:02:09 <Luke-Jr> sipa: Debian and Ubuntu have packages for LevelDB at least
1186 2012-08-16 17:02:21 LuaKT has quit ()
1187 2012-08-16 17:02:32 LuaKT has joined
1188 2012-08-16 17:02:41 <sipa> Luke-Jr: oh, interesting; i wasn't aware
1189 2012-08-16 17:02:42 <usagi> Oh really
1190 2012-08-16 17:02:53 <gmaxwell> ugh.
1191 2012-08-16 17:03:00 <gmaxwell> Reason to not use leveldb.
1192 2012-08-16 17:03:00 <Luke-Jr> sipa: and any distros that don't currently have a package, would of course prefer to add it than embed
1193 2012-08-16 17:03:25 <sipa> gmaxwell: ?
1194 2012-08-16 17:04:12 <gmaxwell> Flinch reaction to distros swapping around versions of critical interior libraries on us.
1195 2012-08-16 17:04:19 <usagi> sipa; Well I have been working out this plan of attack where I add a 'use me' flag for a particular set of addresses and private keys into the database, and then use cassicus's idea to encrypt and/or disable that private key on command. Then I believe there is a patch that lets you specify addresses, I would modify that to merely exclude addresses and otherwise leave random address choosing alone
1196 2012-08-16 17:04:38 <usagi> And then I would have the ability to encrypt individual accounts vs. the whole wallet. Do you think that is a good idea?
1197 2012-08-16 17:04:53 <sipa> usagi: i think what you want is multiple-wallet support :)
1198 2012-08-16 17:05:04 <sipa> instead of hacking yet more functionality on the accounts stuff
1199 2012-08-16 17:05:13 <usagi> Yeah that would be ideal, but I have heard that isn't supported by BDB
1200 2012-08-16 17:05:19 <sipa> huh?
1201 2012-08-16 17:05:20 <usagi> or BDF or whatever they call it.
1202 2012-08-16 17:05:33 <sipa> it's just not implemented
1203 2012-08-16 17:05:58 <usagi> Okay I'll look in that direction instead.
1204 2012-08-16 17:06:29 <usagi> There is a CWallet class so maybe that is the key :>
1205 2012-08-16 17:06:51 <sipa> yes, you'll need to create multiple CWallet objects, backed by separated files
1206 2012-08-16 17:06:57 <gmaxwell> I do think that regardless of multiwallet support it would be good to be able to be able to remove/disable private keys... just haven't had the time to even look at doing it.
1207 2012-08-16 17:07:14 <sipa> gmaxwell: you mean watch-only keys/wallets?
1208 2012-08-16 17:09:16 <sipa> usagi: and find a way to choose a particular wallet through RPC
1209 2012-08-16 17:09:19 <Luke-Jr> "Count toward balances?" "Automatically use for spending?" "Immediately move funds to ________?"
1210 2012-08-16 17:09:32 <usagi> Well, I'm looking at the registerwallet and wallet dispatch functions in main.cpp now
1211 2012-08-16 17:09:35 <gmaxwell> sipa: What I care more about theat is something of a subset: I want to be able to quarantine addresses so we won't spend from them; for contract or privacy reasons. Even with multiple wallet support it would be useful— e.g. if you only want a temporary quarantine.
1212 2012-08-16 17:09:40 <jgarzik> so, on "mempool" P2P command
1213 2012-08-16 17:09:43 * jgarzik is writing the BIP now
1214 2012-08-16 17:09:56 <jgarzik> what to do about protover > X, yet no mempool?
1215 2012-08-16 17:10:08 <jgarzik> a transaction memory pool is not a requirement of a node
1216 2012-08-16 17:10:16 <usagi> sipa: do you know if all wallet action has been encapsulated into CWallet now?
1217 2012-08-16 17:10:28 <jgarzik> permit the command to be ignored... or require no-mempool nodes to return empty inv?
1218 2012-08-16 17:10:32 <sipa> usagi: i hope so
1219 2012-08-16 17:10:33 <jgarzik> or other?
1220 2012-08-16 17:10:49 <jgarzik> (this problem is one reason why I liked an nServices bit for mempool)
1221 2012-08-16 17:11:01 nsh has quit (Remote host closed the connection)
1222 2012-08-16 17:11:04 <sipa> jgarzik: i'd say that the NODE_NETWORK service flag implies the presence of a mempool
1223 2012-08-16 17:11:17 <sipa> and the protover implies exposure of the mempool
1224 2012-08-16 17:11:34 p0s has quit (Remote host closed the connection)
1225 2012-08-16 17:11:47 <jgarzik> seems reasonable
1226 2012-08-16 17:12:06 <sipa> on the other hand, if you want to bootstrap, you want to do so from a node that is known in advance to expose its mempool
1227 2012-08-16 17:12:12 <sipa> and you cannot know that before connecting
1228 2012-08-16 17:12:33 <sipa> so just a service flag for that would be ideal, but we must be careful with assigning those bits
1229 2012-08-16 17:13:06 <jgarzik> agreed
1230 2012-08-16 17:13:11 <jgarzik> nServices is a scarce resource
1231 2012-08-16 17:13:30 <sipa> mempool exposure does seem to be a very well-defined service to me, though
1232 2012-08-16 17:13:46 <sipa> i'll reply to your mail
1233 2012-08-16 17:14:02 <jgarzik> cool
1234 2012-08-16 17:14:36 <jgarzik> sipa: the BIP will match the current implementation (#1641), and we can discuss revisions on list
1235 2012-08-16 17:21:54 localhost has quit (Read error: Connection reset by peer)
1236 2012-08-16 17:27:51 cande has joined
1237 2012-08-16 17:29:57 <sipa> gmaxwell: i'm mostly interested in leveldb's backward (and forward) compatibility between versions
1238 2012-08-16 17:30:43 <Marf> heallo
1239 2012-08-16 17:30:52 <Marf> whats de big surprise in september??
1240 2012-08-16 17:31:10 <Marf> it should be a secret :(
1241 2012-08-16 17:31:26 <sipa> gmaxwell: if that is good, we can use platform-provided packages just fine
1242 2012-08-16 17:32:49 localhost has joined
1243 2012-08-16 17:33:38 <gmaxwell> sipa: I couldn't find anything on it in their docs; but I don't know that it matters.
1244 2012-08-16 17:33:44 osmosis has joined
1245 2012-08-16 17:34:00 <gmaxwell> In theory bdb has good backwards compatiblity, but without forwards compatiblity upgrades are irreversable and thats no good.
1246 2012-08-16 17:35:06 <sipa> bdb has no backward compatibility at all between major releases, only forward compatibility
1247 2012-08-16 17:35:13 <sipa> and the log files don't even have that
1248 2012-08-16 17:36:34 root2 has joined
1249 2012-08-16 17:38:04 <gmaxwell> I was reversing backwards and forwards there relative to your usage.
1250 2012-08-16 17:38:19 <jgarzik> BIP 35 created & emailed
1251 2012-08-16 17:38:38 <jgarzik> made a minor change:  always return "inv", even if empty
1252 2012-08-16 17:40:08 <sipa> gmaxwell: yours matches wikipedia; apparently i assumed it was the other way around
1253 2012-08-16 17:40:46 Karmaon1 has joined
1254 2012-08-16 17:42:33 Guest52544 has quit (Remote host closed the connection)
1255 2012-08-16 17:43:11 darkee has quit (!~darkee@gateway/tor-sasl/darkee|Remote host closed the connection)
1256 2012-08-16 17:43:38 da2ce7_d has joined
1257 2012-08-16 17:44:13 darkee has joined
1258 2012-08-16 17:44:22 <sipa> so: bdb databases: backward compatible between major releases, forward compatible only between minor releases
1259 2012-08-16 17:44:38 <sipa>     bdb logfiles: not compatible at all between releases
1260 2012-08-16 17:45:42 <jgarzik> correct
1261 2012-08-16 17:45:53 da2ce7 has quit (Ping timeout: 252 seconds)
1262 2012-08-16 17:46:30 <jgarzik> the logfile bit is documented, I remember seeing that.  Buried in strict instructions about how to move a database to another machine.
1263 2012-08-16 17:46:31 PK has quit (Quit: Leaving)
1264 2012-08-16 17:47:20 d4ve has left ()
1265 2012-08-16 17:47:51 <sipa> gmaxwell: i changed ultraprune's getrawtransaction to use the coin database, and rescan the entire block the tx is in to provide it; however, that doesn't work for completely-spent transactions
1266 2012-08-16 17:48:10 <sipa> gmaxwell: one solution would be to add the block hash as well, so it knows where to look
1267 2012-08-16 17:48:29 <sipa> (or have an optional index...)
1268 2012-08-16 17:56:28 B0g4r7__ has joined
1269 2012-08-16 17:58:04 <jgarzik> sipa: disk space is cheap.  pynode is going to have an all-tx index as well as a smaller spent-tx index
1270 2012-08-16 17:58:47 TD has joined
1271 2012-08-16 17:58:49 <jgarzik> all-tx is written for each new block. read traffic is RPC-dependent.
1272 2012-08-16 17:59:05 B0g4r7 has quit (Ping timeout: 276 seconds)
1273 2012-08-16 17:59:06 B0g4r7__ is now known as B0g4r7
1274 2012-08-16 17:59:23 <sipa> jgarzik: i am not going to add a mandatory index that is *only* useful for an RPC call that mostly serves debugging
1275 2012-08-16 18:00:07 <jgarzik> sipa: I disagree that it only serves debugging.  Seems reasonable for an RPC wallet to verify its transactions
1276 2012-08-16 18:00:18 <jgarzik> sipa: but as you said, it can be optional
1277 2012-08-16 18:00:51 <sipa> it's certainly useful for some, so adding an optional index: sure, but i don't feel like implementing that right now :)
1278 2012-08-16 18:02:19 <jgarzik> <shrug> I think its a regression from current behavior, if the branch is merged upstream without the optional index
1279 2012-08-16 18:03:23 <sipa> well, we're not merging it yet, are we? :)
1280 2012-08-16 18:05:21 Zarutian has joined
1281 2012-08-16 18:06:11 <jgarzik> har
1282 2012-08-16 18:06:28 <jgarzik> it would be interesting to make pynode send ping(nonce) after each and every command it sends :)
1283 2012-08-16 18:09:03 phantomcircuit_ is now known as phantomcircuit
1284 2012-08-16 18:09:05 cande has quit (Quit: Lämnar)
1285 2012-08-16 18:09:35 inlikeflynn has joined
1286 2012-08-16 18:11:42 inlikeflynn has quit (Changing host)
1287 2012-08-16 18:11:42 inlikeflynn has joined
1288 2012-08-16 18:11:46 ThomasV has joined
1289 2012-08-16 18:16:10 <jgarzik> gmaxwell: didn't you already do a pull request to sweep up dust when building new transactions (RE SatoshiDICE lost-bet outputs)?  Or am I completely crazy?
1290 2012-08-16 18:16:51 <sipa> i find the trend of trying to discard 1-satoshi txouts alarming
1291 2012-08-16 18:17:15 molecular has quit (Ping timeout: 265 seconds)
1292 2012-08-16 18:17:26 <gmaxwell> hm?
1293 2012-08-16 18:17:50 <sipa> i read reports on the forums of people patching their bitcoind so that it actively ignores too small inputs
1294 2012-08-16 18:17:54 <gmaxwell> jgarzik: I think I posted a patch at some point— but my patch had the undesirable result of screwing up privacy by interlinking everything.
1295 2012-08-16 18:18:07 <gmaxwell> sipa: yes, well, they can make coin selection fail.
1296 2012-08-16 18:18:16 molecular has joined
1297 2012-08-16 18:18:24 <sipa> which is of course expected, given our relaying/mining policy
1298 2012-08-16 18:18:41 pjorrit has joined
1299 2012-08-16 18:18:45 <sipa> but the policy should award spending coins, not punush it, imho
1300 2012-08-16 18:18:50 <sipa> (to a certain extent)
1301 2012-08-16 18:20:04 <gmaxwell> This is why I was saying priority should incentivize net txout set reduction... but we couldn't come to an agreement on the metric.
1302 2012-08-16 18:20:27 <gmaxwell> but I think the reason people are patching has nothing to do with priority..
1303 2012-08-16 18:20:42 <gmaxwell> too many tiny inputs makes it actually fail (or become intolerably slow)
1304 2012-08-16 18:20:43 <sipa> no, it is to avoid heavy fees for spending their dust
1305 2012-08-16 18:20:49 <gmaxwell> oh hm.
1306 2012-08-16 18:21:00 <sipa> (i assume)
1307 2012-08-16 18:21:45 <gmaxwell> you can also get in to cases where you have 100 BTC with 2 confirms... then 1 BTC in dust with 8 confirms... and you send .5 BTC .. and it's telling you that you need a .05 BTC fee.
1308 2012-08-16 18:24:27 <sipa> how large is blkindex.dat these days, typically?
1309 2012-08-16 18:24:49 <sipa> ah, my vps still has one running; 880 MiB...
1310 2012-08-16 18:24:49 BCBot has quit (Ping timeout: 246 seconds)
1311 2012-08-16 18:25:56 BCBot has joined
1312 2012-08-16 18:26:08 t7 has joined
1313 2012-08-16 18:30:47 usagi has quit (Remote host closed the connection)
1314 2012-08-16 18:30:56 usagi has joined
1315 2012-08-16 18:38:30 <DBordello> sipa, I see 845M
1316 2012-08-16 18:38:34 <usagi> hmm
1317 2012-08-16 18:39:49 LuaKT has quit (Ping timeout: 260 seconds)
1318 2012-08-16 18:42:19 rdponticelli has quit (Ping timeout: 246 seconds)
1319 2012-08-16 18:52:38 leotreasure has quit (Ping timeout: 244 seconds)
1320 2012-08-16 18:55:33 RazielZ has quit (Ping timeout: 240 seconds)
1321 2012-08-16 18:55:45 asa has joined
1322 2012-08-16 18:57:48 RazielZ has joined
1323 2012-08-16 19:04:24 ultrixx has joined
1324 2012-08-16 19:07:41 danbri_ has quit (Remote host closed the connection)
1325 2012-08-16 19:10:24 <amiller> hm, so stefan thomas is building a DHT of some sort
1326 2012-08-16 19:14:25 ThomasV has quit (Quit: Quitte)
1327 2012-08-16 19:15:50 dooglus has quit (Remote host closed the connection)
1328 2012-08-16 19:15:51 paul0 has quit (Ping timeout: 244 seconds)
1329 2012-08-16 19:17:17 <amiller> i can't find his bip
1330 2012-08-16 19:17:44 <sipa> there is none yet
1331 2012-08-16 19:17:47 <sipa> afaik
1332 2012-08-16 19:20:44 sgornick has quit (Quit: Ex-Chat)
1333 2012-08-16 19:20:54 LuaKT has joined
1334 2012-08-16 19:20:54 LuaKT has quit (Changing host)
1335 2012-08-16 19:20:54 LuaKT has joined
1336 2012-08-16 19:22:28 phantomcircuit has quit (Quit: Leaving)
1337 2012-08-16 19:22:29 paul0 has joined
1338 2012-08-16 19:24:56 ultrixx has quit (Quit: Verlassend)
1339 2012-08-16 19:26:32 dooglus has joined
1340 2012-08-16 19:27:22 TD has quit (Quit: TD)
1341 2012-08-16 19:28:40 justmoon has joined
1342 2012-08-16 19:29:22 danbri has joined
1343 2012-08-16 19:38:25 chrisb__ has joined
1344 2012-08-16 19:38:40 toffoo has quit ()
1345 2012-08-16 19:43:06 <BlueMatt> gmaxwell: tweak the lib and add some random useless call to bitcoin's level db, then put a big comment that says "if you remove this or switch to a distro supplied version of leveldb, you will break bitcoin, and we will sue you"
1346 2012-08-16 19:44:16 * gmaxwell rolls eyes.
1347 2012-08-16 19:44:40 <BlueMatt> heh
1348 2012-08-16 19:46:08 <sipa> just change some magic byte sequence, or swap some opcodes' values
1349 2012-08-16 19:46:20 <amiller> ;seen justmoon
1350 2012-08-16 19:46:39 <sipa> ;;seen justmoon
1351 2012-08-16 19:46:39 <gribble> justmoon was last seen in #bitcoin-dev 6 hours, 55 minutes, and 1 second ago: <justmoon> ciao
1352 2012-08-16 19:47:17 sirk390 has joined
1353 2012-08-16 19:48:46 <BlueMatt> sipa: that and people using 1-satoshi (and even 0) outputs to publish data
1354 2012-08-16 19:49:08 * amiller greps devlogs for justmoon
1355 2012-08-16 19:49:24 <amiller> awesome, he seems to have a lot of thought into sharding and organization of the DHT, but isn't aware of the merkle tree proposals
1356 2012-08-16 19:49:32 Turingi has quit (Read error: Connection reset by peer)
1357 2012-08-16 19:49:35 <amiller> i shall make his day
1358 2012-08-16 19:49:44 <justmoon> I'll well aware of them
1359 2012-08-16 19:49:46 <justmoon> :)
1360 2012-08-16 19:49:50 leotreasure has joined
1361 2012-08-16 19:49:51 <justmoon> I'm counting on you! :D
1362 2012-08-16 19:50:12 <justmoon> but I do not yet fully understand how to integrate them with what I'm trying to do
1363 2012-08-16 19:50:22 * amiller tumbles into an armchair a la dick van dyke
1364 2012-08-16 19:50:37 * justmoon leans in with big open eyes :)
1365 2012-08-16 19:50:40 <amiller> afaict you've talked about two kinds of data queries to a dht
1366 2012-08-16 19:50:53 <amiller> one for blocks, for example you mention itereating from a checkpoint through to the head
1367 2012-08-16 19:51:33 <amiller> also for fetching indexes e.g. ultraprune, where you would have to rely on some other mechanism to get keys(hashes) for the index, but presumably the security loss there isn't severe
1368 2012-08-16 19:52:41 <amiller> if there are coinbase commitments in each block, a root of a merkle tree containing all the unspent outputs, then the only thing needed to validate any transaction is a short (log N) sequence of lookups-by-hash for nodes in that tree
1369 2012-08-16 19:54:19 <justmoon> I'm with you so far
1370 2012-08-16 19:54:56 <amiller> well, that's about all!
1371 2012-08-16 19:55:04 <amiller> query-by-hash is awesome and essential
1372 2012-08-16 19:55:15 <amiller> whether it's for blocks or other kinds of data chunks
1373 2012-08-16 19:55:34 <amiller> are you planning on using kademlia or something?
1374 2012-08-16 19:55:49 <justmoon> yes a modified version
1375 2012-08-16 19:55:56 <justmoon> 256 bit hashes instead of 160 bit for example
1376 2012-08-16 19:56:16 <gmaxwell> amiller: by 'awesome and essential' you mean irrelevant and unnecessary, right?
1377 2012-08-16 19:56:17 <justmoon> also it won't allow arbitrary keys, but only data that has some meaning to bitcoin
1378 2012-08-16 19:56:29 <gmaxwell> (at least for MTOT stuff)
1379 2012-08-16 19:56:55 <justmoon> amiller: you would send this extra verification information along with every tx as it's being relayed?
1380 2012-08-16 19:57:13 CodesInChaos has quit (Ping timeout: 246 seconds)
1381 2012-08-16 19:57:14 <amiller> gmaxwell, this stuff is all non-normative at least, so i don't feel quite as much pressure to justify it ahead of time
1382 2012-08-16 19:57:45 <amiller> justmoon, yeah roughly, there are all sorts of optimizations that avoid the need for a new lookup each time
1383 2012-08-16 19:57:53 <amiller> really everyone will be processing the same short sequences of hashes
1384 2012-08-16 19:57:54 <gmaxwell> ^ because of that— even if you don't send it by default, you always know that someone who gave you a item (txn, block) can also provide the required validation info.
1385 2012-08-16 19:58:07 <gmaxwell> (^ being justmoon's comment)
1386 2012-08-16 19:58:20 <justmoon> aye
1387 2012-08-16 19:58:41 <amiller> justmoon, so for not including arbitrary keys, i suppose that's what the custom validation would be, is it would only include nodes that appear to be valid
1388 2012-08-16 19:58:51 <gmaxwell> amiller: oh, I'm sure you can invent some cases where it's useful; but I do not see how it's even useful much less essential for doing validation. Thats the only reason I spoke up.
1389 2012-08-16 19:59:11 <amiller> gmaxwell, it's pretty useful for lite clients that don't want to store anything at all
1390 2012-08-16 19:59:35 <amiller> you could fit a full node onto a smart card chip this way
1391 2012-08-16 19:59:54 <gmaxwell> amiller: no! it really isn't— again, if someone gives you a thing you need to validate you can also ask them for the required info— no query by hash required.
1392 2012-08-16 20:01:31 <justmoon> can someone fill me in what precisely you mean by "query by hash"?
1393 2012-08-16 20:01:33 <amiller> okay fair enough, i suppose i mean to use DHT in a very inclusive sense, there are at least a few ways it could turn out (or all of the following) a) a big public DHT where lots of nodes join b) a private distributed network (basically just a library/tool for sharding your full blockchain history) c) the degenerate case is just a hard drive with all the data sorted by hash
1394 2012-08-16 20:01:48 <justmoon> just looking up txs in the block chain by hash?
1395 2012-08-16 20:01:57 <justmoon> or addresses also?
1396 2012-08-16 20:03:26 <amiller> justmoon, given a merkle tree coinbase commitment, the task of looking up all the spendable coins associated with an address reduces to just traversing the merkle tree node by node, beginning with the root hash, where each step in the traversal is a "query-by-hash"
1397 2012-08-16 20:04:01 <sipa> justmoon: it means doing a query for some data element by hash (typically a address-to-txids map, or a txid-to-unspent-outputs map) from a data structure whose merkle root is already known and trusted by you, and get a.reply that either returns the data and a merkle path to prove it, or a merkle root that proves absence of the requested element
1398 2012-08-16 20:04:34 <sipa> amiller: amiright?
1399 2012-08-16 20:04:59 <amiller> sipa, yup
1400 2012-08-16 20:05:30 <justmoon> amiller: where can I find the best description of what you're currently favoring?
1401 2012-08-16 20:05:46 <justmoon> I've read the older comments in ethotheipi's thread and some of the irc logs, but that's about it
1402 2012-08-16 20:05:48 <user__> bit-pay company man is here?
1403 2012-08-16 20:05:51 <amiller> i assume that justmoon mentioned addresses (which are hashes of something, but not something useful) to see if i was misunderstanding a DHT as a distributed key-value store, which is a different (and much harder) problem
1404 2012-08-16 20:06:13 <justmoon> amiller gives justmoon too much credit :P
1405 2012-08-16 20:06:55 <amiller> there have been four generations of this proposal: https://en.bitcoin.it/wiki/Hardfork_Wishlist#Major_structural_changes
1406 2012-08-16 20:07:50 <amiller> i think etotheipi's is the best so far
1407 2012-08-16 20:08:57 <justmoon> I'm - perhaps foolishly - already thinking about miners who distribute verification work and etotheipi's proposal seems to require you to be aware of the entire contents of the merkle tree in order to update it
1408 2012-08-16 20:09:27 <justmoon> so at least on a gut level I'm not superenthusiastic about it although my concern may well turn out to be irrelevant at least in the mid term
1409 2012-08-16 20:09:36 <amiller> updating a merkle tree also requires only (log N) storage, it's certainly not necessary to be aware of the entire contents
1410 2012-08-16 20:09:48 danbri has quit (Remote host closed the connection)
1411 2012-08-16 20:09:49 <amiller> just like a lookup, you only need to ask for the relevant path of data
1412 2012-08-16 20:09:55 rdponticelli has joined
1413 2012-08-16 20:11:03 yorick has quit (Ping timeout: 272 seconds)
1414 2012-08-16 20:11:06 <gmaxwell> justmoon: Sounds like you're suffering from a serious misunderstanding there.
1415 2012-08-16 20:11:13 <justmoon> quite possible
1416 2012-08-16 20:11:33 <gmaxwell> justmoon: in any hash validated data structure, you only need to have access to the data that you access in the coarse of performing your query/update.
1417 2012-08-16 20:11:34 <justmoon> I remember reading that his idea was to regenerate the tree from scratch on every block
1418 2012-08-16 20:11:47 <sipa> hell no
1419 2012-08-16 20:11:49 <gmaxwell> Thats obviously unworkable.
1420 2012-08-16 20:12:02 <justmoon> that's what I said when I read it :P
1421 2012-08-16 20:12:24 <justmoon> ok, perhaps I need to look at it again, I can't find the quote either at the moment
1422 2012-08-16 20:12:27 <gmaxwell> Either you misunderstood that or etotheipi was having a blonde moment.
1423 2012-08-16 20:12:52 yorick has joined
1424 2012-08-16 20:12:57 <amiller> (etotheipi has in fact had several blonde moments in that thread :/)
1425 2012-08-16 20:13:44 ArchMint has joined
1426 2012-08-16 20:13:45 <amiller> but i don't think that was one
1427 2012-08-16 20:13:49 <ArchMint> Hello, how to fix: DbEnv::txn_checkpoint: DB_RUNRECOVERY: Fatal error, run database recovery
1428 2012-08-16 20:13:57 <gmaxwell> All this is possible perfectly efficiently. As a miller pointed out before— you can take any efficient datastructure and add hashes along it; then make queries and updates to it provable... and it just adds a constant factor cost to it (the computation/transmission of the hashes)
1429 2012-08-16 20:14:01 <ArchMint> I had made a backup of the wallet, then changed computer.
1430 2012-08-16 20:14:08 <ArchMint> I put this in new computer, and epic fail. D:
1431 2012-08-16 20:14:26 wasabi2 has quit (Read error: Connection reset by peer)
1432 2012-08-16 20:14:28 <sipa> ArchMint: how did you make the backup?
1433 2012-08-16 20:14:46 <ArchMint> Copy the wallet file to a usb stick
1434 2012-08-16 20:14:56 <sipa> was bitcoin running at the time?
1435 2012-08-16 20:15:06 <ArchMint> idk it was a long time ago
1436 2012-08-16 20:15:29 <sipa> well, backups made while the program is running will not work, unfortunately
1437 2012-08-16 20:15:59 <gmaxwell> It may be possible to recover most or all of the private keys using special extraction utilities. Was the wallet encrypted?
1438 2012-08-16 20:16:04 <ArchMint> No
1439 2012-08-16 20:16:13 <ArchMint> It was made before encryption was supported
1440 2012-08-16 20:16:30 <gmaxwell> ArchMint: see this thread: https://bitcointalk.org/index.php?topic=25091.0
1441 2012-08-16 20:18:08 <ArchMint> I haven't deleted the wallet.
1442 2012-08-16 20:18:12 <ArchMint> I still have it ._.
1443 2012-08-16 20:18:21 <ArchMint> It just seems to be corrupt.
1444 2012-08-16 20:18:30 <helo> what OS?
1445 2012-08-16 20:19:17 <ArchMint> I have installed Mint 11, and Mint 13
1446 2012-08-16 20:19:21 <helo> it may be convenient to try using pywallet to access it: https://github.com/joric/pywallet
1447 2012-08-16 20:19:25 <gmaxwell> ArchMint: I know you haven't, but the tools discussed there should also be able to find private keys in a corrupted wallet.
1448 2012-08-16 20:20:20 <ArchMint> Why is the wallet even able to be corrupted?
1449 2012-08-16 20:20:29 <sipa> ArchMint: it is not
1450 2012-08-16 20:20:31 <justmoon> amiller: I re-read etotheipi's post - he says: "Vanilla merkle-trees will not work, because adding or removing single branches is likely to cause complete recomputation of the tree.  But it should be possible to create an alternative with the following properties:"
1451 2012-08-16 20:20:39 <ArchMint> Should it use sync and etc to ensure it isn't corrupted?
1452 2012-08-16 20:20:40 <justmoon> but he doesn't actually say what the datastructure would be
1453 2012-08-16 20:20:43 ultrixx has joined
1454 2012-08-16 20:20:57 <amiller> justmoon, i implemented a self balancing merkle tree prototype https://github.com/amiller/redblackmerkle/blob/master/redblack.py
1455 2012-08-16 20:21:02 <sipa> ArchMint: it just refers to other files (the db transaction log) that is used to prevent corruption
1456 2012-08-16 20:21:03 <amiller> etotheipi wants to do the same thing with tries
1457 2012-08-16 20:21:11 <ArchMint> ._.
1458 2012-08-16 20:21:16 <ArchMint> That's stupid.
1459 2012-08-16 20:21:25 <amiller> his reasons for preferring them are a little bit confused IMO
1460 2012-08-16 20:21:29 <sipa> amiller: if you made a copy while it was running, it still needs those files, and you didn't copy them
1461 2012-08-16 20:21:33 <sipa> eh, ArchMint
1462 2012-08-16 20:21:57 <justmoon> amiller: ok so this code would have the commutative and O(log n) updates properties?
1463 2012-08-16 20:22:07 <gmaxwell> amiller: I prefer tries too, though for entirely different reasons.
1464 2012-08-16 20:22:24 <amiller> it wouldn't have the commutative property, but that's not important since if you have the hash of a block, you also have the hash of the previous block and therefore the previous merkle tree
1465 2012-08-16 20:22:26 <helo> sipa: if you copy a wallet.dat of a running bitcoind, it won't contain the private keys?
1466 2012-08-16 20:22:30 <amiller> but it would have the log N updates property certainly
1467 2012-08-16 20:22:42 <sipa> helo: surely it will, but the file may have incomplete updates
1468 2012-08-16 20:22:52 <justmoon> amiller: ok makes sense
1469 2012-08-16 20:23:03 <sipa> helo: which is not a problem, data does typically not get lost that way
1470 2012-08-16 20:23:16 <sipa> helo: but bdb cannot deal with databases that lost their logs
1471 2012-08-16 20:23:26 <gmaxwell> helo: but bdb will refuse to work with a potentially incomplete file because it could be inconsistent in strange ways.
1472 2012-08-16 20:23:26 <sipa> at least not in a fully automated way
1473 2012-08-16 20:23:53 <helo> so wallet-recover will probably handle it in that case?
1474 2012-08-16 20:23:59 <sipa> yes
1475 2012-08-16 20:24:16 <justmoon> amiller: I'm thinking about what else I need to ask - from my perspective it's just a magic black box that les me verify txs, really
1476 2012-08-16 20:24:50 <justmoon> my DHT proposal only depends on that there be some efficient way of verifying transactions without having to trust the main "big" index
1477 2012-08-16 20:25:03 <justmoon> ultraprune does that fine for now, this would do it even better
1478 2012-08-16 20:25:27 <gmaxwell> huh?
1479 2012-08-16 20:25:29 <amiller> mostly i wanted to ask you about your DHT proposal, first of all is it recorded anywhere? i looked but coudln't find it (forum post or wiki or bip)
1480 2012-08-16 20:25:41 <sipa> justmoon: ultraprune does not give you any trust
1481 2012-08-16 20:25:42 <gmaxwell> justmoon: ultraprune doesn't change that. you still need the index.
1482 2012-08-16 20:25:57 [\\\] has quit (Ping timeout: 240 seconds)
1483 2012-08-16 20:26:12 <gmaxwell> (though ultraprune avoids leaving old junk in the index, but thats just an artifact of the reference implementation)
1484 2012-08-16 20:26:33 <justmoon> sipa, gmaxwell: if I have a ultraprune snapshot I trust and have seen every block after that I know that my ultraprune set is correct
1485 2012-08-16 20:26:46 <sipa> justmoon: of course
1486 2012-08-16 20:26:47 <gmaxwell> If you were to try to use ultraprune with DHTed storage an attacker could exploit you so hard your head would spin.
1487 2012-08-16 20:26:48 <justmoon> I do not have to store a block chian index for that
1488 2012-08-16 20:27:17 <sipa> justmoon: withoit block chain index you can't know which block to reorg to
1489 2012-08-16 20:27:17 <gmaxwell> justmoon: sure, but you store the ultraprune index.  No node ever needed to store more than that.
1490 2012-08-16 20:27:26 <gmaxwell> (well index wise)
1491 2012-08-16 20:27:38 <justmoon> sipa: I don't follow?
1492 2012-08-16 20:27:52 <gmaxwell> justmoon: ultraprune by itself can't reorg.
1493 2012-08-16 20:27:56 <sipa> justmoon: i send you a block, what do you do?
1494 2012-08-16 20:28:04 <justmoon> gmaxwell: I don't plan on trusting the DHT to give me a valid ultraprune index, I plan on shipping the client with a known valid hash of an ultraprune index at a checkpoint
1495 2012-08-16 20:28:27 <gmaxwell> ugh.
1496 2012-08-16 20:28:38 wasabi2 has joined
1497 2012-08-16 20:29:01 <justmoon> well obviously you need some recent blocks to be able to reorg
1498 2012-08-16 20:29:15 <sipa> more than that
1499 2012-08-16 20:29:27 Joric has joined
1500 2012-08-16 20:29:27 Joric has quit (Changing host)
1501 2012-08-16 20:29:27 Joric has joined
1502 2012-08-16 20:29:30 <sipa> blocks update the ultraprune structure destructively
1503 2012-08-16 20:29:42 <justmoon> well I have the undo information don't I?
1504 2012-08-16 20:29:49 <sipa> if you store it
1505 2012-08-16 20:29:51 <sipa> and trust it
1506 2012-08-16 20:30:02 <justmoon> I created it why wouldn't I trust it?
1507 2012-08-16 20:30:15 <sipa> it's not in the DHT?
1508 2012-08-16 20:30:17 <justmoon> gmaxwell: you do realize the reference client uses checkpoints, right? :P
1509 2012-08-16 20:30:37 <sipa> only for signature checking, not for database consistency
1510 2012-08-16 20:30:51 <Joric> hey justmoon!
1511 2012-08-16 20:30:55 <gmaxwell> justmoon: Thanks for establishing that you don't have a clue about the bitcoin security model.
1512 2012-08-16 20:31:00 <sipa> and the reference client expects its database to be stored in a safe way
1513 2012-08-16 20:31:06 <justmoon> sipa: the checkpoints are so deep in the chain there isn't going to be a reorg that goes that far back and if there is the network wouldn't accept it anyway since most nodes use checkpoints
1514 2012-08-16 20:31:16 <justmoon> and for anything newer I have my own undo information
1515 2012-08-16 20:31:28 <sipa> ok, but what does the DHT store?
1516 2012-08-16 20:31:37 <Joric> justmoon, check out sign/verify http://brainwallet.org/js/bitcoinsig.js also covered in coinbase 005 podcast
1517 2012-08-16 20:31:49 <sipa> if you still keep the ultraprune set and undo information locally?
1518 2012-08-16 20:32:06 <gmaxwell> Ah, perhaps I wasn't clear then: The 'ugh' was that it will be nodes that blindly trust the validation prior to the checkpoint.
1519 2012-08-16 20:32:11 <justmoon> sipa: not much, it's used to do rescans, block explorers, that type of thing
1520 2012-08-16 20:32:29 <justmoon> (a rescan in bitcoinjs isn't really a rescan, more a lookup by address)
1521 2012-08-16 20:33:03 <sipa> ok, it stores an address-to-txids map?
1522 2012-08-16 20:33:55 <justmoon> yes, and a lookup by transaction hash
1523 2012-08-16 20:34:11 <justmoon> transactions are stored with the merkle branches that connect them to the headers
1524 2012-08-16 20:34:25 <amiller> address-to-txid's map isn't something a DHT can do directly (unless you have merkle trees everywhere)
1525 2012-08-16 20:34:26 <justmoon> gmaxwell: I don't follow what information my nodes would blindly trust?
1526 2012-08-16 20:34:35 <gmaxwell> The only way we can answer charges like "I heard the creator of bitcoin secretly gave himself a billion bitcoin before making bitcoin public" is the fact that nodes don't just trust an opaque hash-of-a-state-snapshot. So I'm not not eager to hear about clients promoting modes that abandon that security property.
1527 2012-08-16 20:34:56 <justmoon> gmaxwell: well then you should argue to remove it from the reference clients
1528 2012-08-16 20:35:01 slush1 has quit (Ping timeout: 246 seconds)
1529 2012-08-16 20:35:07 <gmaxwell> justmoon: THAT ISNT WHAT THE CHECKPOINTS DO
1530 2012-08-16 20:35:11 maqr has joined
1531 2012-08-16 20:35:22 <justmoon> the checkpoints disable signature verification
1532 2012-08-16 20:35:30 <gmaxwell> justmoon: and if I wanted it removed, I'd remove it myself— I have commit access, you know.
1533 2012-08-16 20:36:01 <sipa> even if there are checkpoints, they do not allow anyone to bootstrap from anything but the full block history
1534 2012-08-16 20:36:13 <gmaxwell> justmoon: Yes, they do, but they do not defer any of the other validation. And the accuracy of the signature validation is evidence by the enormous amount of computing power confirming the chain after it.
1535 2012-08-16 20:36:30 <gmaxwell> State snapshots do not have the same property.
1536 2012-08-16 20:36:46 <Joric> http://library.agoristradio.com/library/coinbase/coinbase-EP005.mp3 <- 08:26 :D
1537 2012-08-16 20:38:03 <jgarzik> I think genjix makes a good point, regarding mempool (BIP 35):  making it [as] mandatory [as possible] certainly aligns with the desired use cases
1538 2012-08-16 20:38:09 <justmoon> gmaxwell: I'm hoping ultraprune hashes or something like amiller's merkle trees will become part of the block chain or at least a merged mining chain
1539 2012-08-16 20:38:14 <gmaxwell> If I wanted to maliciously introduce a checkpoint in order to give robbed coins I couldn't do so, if I wanted to introduce a checkpoint that gave me fake inflated coins, I really really couln't do so.
1540 2012-08-16 20:38:44 <gmaxwell> justmoon: Hey, when did they become amiller's merkle trees? :)
1541 2012-08-16 20:38:47 <sipa> justmoon: that will allow SPV level security bootstrapping from an ultraprune snapshot, indeed
1542 2012-08-16 20:39:08 <justmoon> gmaxwell: technically you could simply put an exception in the transaction verification code, as you rightly point out you have commit access
1543 2012-08-16 20:39:13 <gmaxwell> justmoon: sure, and while tree committments are not part of the blockchain you have a weak security model.
1544 2012-08-16 20:39:24 <sipa> but that is still srictly weaker than the security we have today with the reference client
1545 2012-08-16 20:39:27 <gmaxwell> justmoon: but thats completely tractable. E.g. anyone auditing the code can trivially see it.
1546 2012-08-16 20:39:35 <justmoon> gmaxwell: tree commitments are PART of my security model
1547 2012-08-16 20:39:37 <gmaxwell> justmoon: vs a hash output which is opaque.
1548 2012-08-16 20:40:10 <justmoon> gmaxwell: well that's rather arguable, there have been rather ingenious hidden backdoors added to open source projects
1549 2012-08-16 20:40:13 <gmaxwell> justmoon: they can't be part of your security model unless they're part of bitcoin's security model: having just you and your friends commit to them doesn't prove much.
1550 2012-08-16 20:40:46 <justmoon> gmaxwell: I don't follow your argument
1551 2012-08-16 20:40:47 <gmaxwell> justmoon: No it really isn't— I mean yes, it could be subtle. But in no case is a hash not less transparent.
1552 2012-08-16 20:41:45 <amiller> afk have to run to a meeting, you two finish splaining' things to justmoon and i'll resume asking about DHT implementations later
1553 2012-08-16 20:41:48 <justmoon> gmaxwell: to verify a hash I can query any client that had the previous hash and a full block chain - anyone can do this -heck it can even be automated, to notice a source code change I have to be a developer and a good one at that
1554 2012-08-16 20:42:05 <justmoon> previous hash or* a full block chain
1555 2012-08-16 20:42:05 <gmaxwell> justmoon: Putting a commitment to a ultraprune index of some height in your code means that anyone using your software has to trust that you haven't opaquely written in fake/stolen coins into that index.
1556 2012-08-16 20:42:25 <justmoon> not necessarily, they can also verify it
1557 2012-08-16 20:42:43 <justmoon> if they like or watch what other people verifying it are saying
1558 2012-08-16 20:42:46 <Joric> justmoon, any plans on getting rid out of window.* globals in bitcoinjs? :^) figured out nodejs is awesome for debugging scripts in console but bitcoinjs still needs some monkey patching
1559 2012-08-16 20:42:51 denisx has joined
1560 2012-08-16 20:43:13 <gmaxwell> justmoon: are you going to write the code to do that? And do you have a determinstic process for generating that will allow all participants to reach the same result?
1561 2012-08-16 20:43:13 <justmoon> Joric: yes I'm planning to switch to boilerplate wrapper that make the modules AMD *and* node compatible
1562 2012-08-16 20:43:39 <justmoon> gmaxwell: I'll do you one better, I'll make sure the hashes are the same as the ones by sipa's COINHASH
1563 2012-08-16 20:43:46 <gmaxwell> justmoon: will you have a way for nodes to communicate that they found cheating in that index, so that if only a few check all can be expected to learn?
1564 2012-08-16 20:44:06 <sipa> justmoon: SPV level security means: trust the data committed to by the block chain with most work; full security is what you have when you yourself validated the entire history. the moment you boot your node using an ultraprune snapshot, you effectively lose both, unless the ultraprune snapshot is committed to by the coinbase (in which case you are limited to SPV security).
1565 2012-08-16 20:44:42 <gmaxwell> sipa: thats for articulating that.
1566 2012-08-16 20:45:42 <sipa> justmoon: the reference client today has full security for everything except signature validation, for which it has SPV security (as the checkpoints are tied to the blockchain and there is work on top of it)
1567 2012-08-16 20:46:05 <gmaxwell> justmoon: in any case, sorry, I realize I'm being really argumentative; that isn't what I intended to do there. :)  I'm concenred about people advertising full node security where not even SPV security is provided; the general userbase doesn't spend the mental cycles to actually understand this stuff usually.
1568 2012-08-16 20:46:21 <justmoon> I don't guys, it seems you are always cherry-picking your attack vectors - you are worried that I will publish a client that has an invalid coinhash baked into it, at the same time you're not worried that somebody will commit something to boost that allows a remote takeover of bitcoin nodes
1569 2012-08-16 20:46:22 <gmaxwell> And of course you can make clients that are enormously faster if you compromise on security; so there is a big incentive to be sloppy. :)
1570 2012-08-16 20:46:27 <justmoon> i don't know* guys
1571 2012-08-16 20:47:14 <gmaxwell> justmoon: Of course we worry about that too— but thats why our binaries are static and built from a mostly static baslined system snapshot.
1572 2012-08-16 20:47:48 <justmoon> and I'm supposed to trust your static binaries? do other nodes warn me if there is a bug somewhere in those binaries?
1573 2012-08-16 20:47:56 <gmaxwell> And widespread deployment of sub-spv-security nodes in place of full nodes would basically ruin the security model of the whole protocol.
1574 2012-08-16 20:48:28 <gmaxwell> justmoon: they're built via a painstakingly created determinstic process, signed by multiple parties who independantly produced the same output; and you can too.
1575 2012-08-16 20:49:03 <justmoon> SO WHY CAN'T A COINHASH BE VERIFIED WITH A PAINSTAKING PROCESS - ESPECIALLY GIVEN THAT IT'S TRIVIAL TO VERIFY IT FOR ANYONE WHO IS ALREADY RUNNING A NODE
1576 2012-08-16 20:49:17 <justmoon> you're are measuring with two totally different meters
1577 2012-08-16 20:49:33 <gmaxwell> We (well, not me) created the process to all this.
1578 2012-08-16 20:49:38 <justmoon> all so you can say "oh this methos is less secure"
1579 2012-08-16 20:49:41 <justmoon> method*
1580 2012-08-16 20:49:58 <gmaxwell> You need to build the same infrastructure work for your client to be trustworthy and not a model violating risky trap for the network.
1581 2012-08-16 20:50:15 <justmoon> ok, so let's get down to earth for a second
1582 2012-08-16 20:50:47 <justmoon> when you update your node obviously we can have a feature that tells you if the new coinhash checkpoint is wrong
1583 2012-08-16 20:50:56 <justmoon> so it's only installation that we're worried about
1584 2012-08-16 20:51:07 <justmoon> we need to verify the coinhash that is included with the code as being correct
1585 2012-08-16 20:51:21 <justmoon> every currently running node can generate the correct coinhash for me to compare
1586 2012-08-16 20:51:57 <justmoon> for example the node could display it on boot and I can compare it to what's shown on blockchain.info, makesurejustmoondoesntcheat.com etc.
1587 2012-08-16 20:52:00 <gmaxwell> You can get to SPV level security if there were enfoced committments to the tree in the blockchain, I believe I mentioned this before.
1588 2012-08-16 20:52:30 <justmoon> gmaxwell: no you can't because you still cannot know with absolute certainty that your code is perfect
1589 2012-08-16 20:52:34 <gmaxwell> justmoon: I think displaying it is 99% security theater. It doesn't actually prevent users from exploitation.
1590 2012-08-16 20:52:44 <gmaxwell> s/prevent/protect/
1591 2012-08-16 20:52:48 <justmoon> you can only ever get to "I trust the developers to send reviewed trusted binaries" security
1592 2012-08-16 20:53:11 <gmaxwell> justmoon: No, that isn't correct. You can recieve the source code, audit it, and produce identical binaries.
1593 2012-08-16 20:53:24 <gmaxwell> Obviously there is an increased work factor there.
1594 2012-08-16 20:53:26 <gmaxwell> :)
1595 2012-08-16 20:53:31 <justmoon> similarly you can receive a coinhash and audit it
1596 2012-08-16 20:53:35 <justmoon> but that is "security theatre"
1597 2012-08-16 20:53:50 <sipa> showing it is not relevant here
1598 2012-08-16 20:53:59 <sipa> the thing is the ability to verify
1599 2012-08-16 20:54:01 <gmaxwell> justmoon: the program displaying it and then expecting someone will look at some website... doesn't accomplish much of anything.
1600 2012-08-16 20:54:18 <justmoon> ok, so neither does showing your binaries' sha hashes on bitcoin.org
1601 2012-08-16 20:54:38 <sipa> and i have to agree that our security model for the source code is similar to his proposed security model for the coinset hash
1602 2012-08-16 20:54:44 <gmaxwell> Right the imporant part is the ability to verify.
1603 2012-08-16 20:55:51 <sipa> and that security model is ultimately "no verification, but make sure you hear when someone finds a mistake"
1604 2012-08-16 20:56:00 <gmaxwell> justmoon: a related point is that we're doing the best we can currently do for the software; I wish it were better. This doesn't excuse adding additional weaknesses where better can be achieved.
1605 2012-08-16 20:56:19 <justmoon> gmaxwell: it's not an additional weakness it's equivalent
1606 2012-08-16 20:56:32 <justmoon> just treat a commit that updates the coinhash like any other commit
1607 2012-08-16 20:56:36 <gmaxwell> justmoon: It's _additional_ because you also have all the basic software vulnerabilites too.
1608 2012-08-16 20:56:55 <justmoon> just like you would review an if statement by thinking about whether it is correct, you can review a coinhash by checking with your current client whether it is right
1609 2012-08-16 20:57:12 <gmaxwell> justmoon: only if you've authored the infrastructure to make that reasonable possible.
1610 2012-08-16 20:57:21 <gmaxwell> (e.g. as was done to make the binary builds determinstic)
1611 2012-08-16 20:57:24 <justmoon> so we agree then :)
1612 2012-08-16 20:57:57 <sipa> i disagree that automated checking against the longest chain is the same as providing the ability for easy verification, but not doing it
1613 2012-08-16 20:58:05 <gmaxwell> justmoon: Somewhat— it requries the infrastructure, and it's still and additional point when we should be moving _away_ from that and towards greater audiability.
1614 2012-08-16 20:58:38 <sipa> i do agree that our security model for the code is the same as his for the coinhash; but our validation for the blockchain data is undeniably stronger
1615 2012-08-16 20:58:56 <gmaxwell> And this is concerning because you'll end up with a node that claims to be full by doesn't actually validate the state of the network, and being enormously faster to install as a result.
1616 2012-08-16 20:59:10 <gmaxwell> Which is ultimately an existential risk for bitcoin.
1617 2012-08-16 21:00:07 <gmaxwell> e.g. say almost everyone deploys your node, then ninjas hold a gun do your head and have you push updates that commit to a coinhash with extra txouts in them.
1618 2012-08-16 21:00:25 <gmaxwell> And since the validation isn't automatic, it's entirely not done.
1619 2012-08-16 21:00:26 <gmaxwell> :(
1620 2012-08-16 21:00:39 <justmoon> same sentense, replace "coinhash" with "modified transaction verification code"
1621 2012-08-16 21:00:45 <justmoon> sentence*
1622 2012-08-16 21:01:30 <sipa> justmoon: that would still require a majority of mimers to update to the infected code before any harm can be done
1623 2012-08-16 21:01:31 genjix has joined
1624 2012-08-16 21:01:41 <sipa> but yes, there is a similarity
1625 2012-08-16 21:01:46 <gmaxwell> justmoon: But the transaction verification code is much harder to secretly change.
1626 2012-08-16 21:02:01 <justmoon> so let me paint a picture for a second
1627 2012-08-16 21:03:17 <justmoon> let's say the client that implements this is the reference client
1628 2012-08-16 21:03:25 <genjix> hey can i change the url on bitcoin.org for electrum. there is a new website.
1629 2012-08-16 21:03:42 <justmoon> let's also say that hundreds of people run automatic verifiers against the code
1630 2012-08-16 21:03:48 <genjix> ecdsa.org/electrum redirects to electrum-desktop.com anyway
1631 2012-08-16 21:04:02 <justmoon> let's say that each release is vetted many times before it is actually uploaded to users
1632 2012-08-16 21:04:35 <justmoon> are you really, *really* honestly worried that there is any, even the remotest possibility that a bogus coinhash will be overlooked?
1633 2012-08-16 21:04:53 <gmaxwell> Certantly, so long as there exist no automation to test it.
1634 2012-08-16 21:05:02 <justmoon> <justmoon> let's also say that hundreds of people run automatic verifiers against the code
1635 2012-08-16 21:05:11 jurov is now known as jurov|away
1636 2012-08-16 21:06:00 <justmoon> i.e. it was one of my premises that there are automatic verifiers
1637 2012-08-16 21:06:02 pnicholson has joined
1638 2012-08-16 21:06:24 <gmaxwell> justmoon: The obvious and intutive way to handle that (e.g. for the users to know the validation ran) is to include commitsments to this index in the coinbase.
1639 2012-08-16 21:06:56 m00p has joined
1640 2012-08-16 21:07:09 <sipa> the difference between visible automated checking, and feedback to the update process
1641 2012-08-16 21:07:14 <gmaxwell> justmoon: and absolutely if thats done then I don't have any problem with the security model; it's still less good than an independant history bootstrap, but it's fine generally.
1642 2012-08-16 21:07:56 <sipa> and putting it in a coinbase, secured by miners' work who risk income when committing an invalid hasj
1643 2012-08-16 21:08:00 <sipa> hash!
1644 2012-08-16 21:08:08 <justmoon> gmaxwell: can you elaborate? if miners don't require the hash to be valid how does including it in the chain help? or are you assuming they will require it to be valid?
1645 2012-08-16 21:08:15 <gmaxwell> Not just risk income, but it shows that a majority of the applied computing power agreed with it.
1646 2012-08-16 21:08:24 Anduck has joined
1647 2012-08-16 21:08:27 <gmaxwell> justmoon: that they would require it to be valid, exactly.
1648 2012-08-16 21:08:29 <sipa> of course they must validate it
1649 2012-08-16 21:08:46 <sipa> they must have a rule not to allow invalid hashes in the coinbase
1650 2012-08-16 21:08:56 <sipa> and a majority must enforce this rule
1651 2012-08-16 21:10:03 Joric has quit (Ping timeout: 245 seconds)
1652 2012-08-16 21:10:08 Joric_ has joined
1653 2012-08-16 21:10:09 Joric_ has quit (Changing host)
1654 2012-08-16 21:10:09 Joric_ has joined
1655 2012-08-16 21:10:13 <sipa> anyway, both that and a very extensive verification and signing system before release, are SPV level security i'd (trust the majority, basically)
1656 2012-08-16 21:10:14 <gmaxwell> We'd introduce it as staged deployment rule. E.g. nodes can put in hashes, they aren't validated. Once N out of the last M blocks have valid committments, then if the commitment is there it is validated and the block rejected if it fails. Then if Y out of the last Z have commitments then all blocks get forced to have them.
1657 2012-08-16 21:10:19 <justmoon> ok, gmaxwell wins, I won't deploy this feature until it's in the chain
1658 2012-08-16 21:10:33 <gmaxwell> justmoon: Help get it into the chain!
1659 2012-08-16 21:10:52 <gmaxwell> Having an implementation is the most important real step beyond all the dreaming. :)
1660 2012-08-16 21:11:24 <sipa> well, i don't think it's hard to add an authenticated tree structure to ultraprune
1661 2012-08-16 21:11:41 <sipa> an efficient one is something else :)
1662 2012-08-16 21:11:54 <gmaxwell> it absolutely isn't. Though .. right and coming to an agreement on the right one since there can only be one.
1663 2012-08-16 21:12:47 <genjix> ultraprune is such a cool name
1664 2012-08-16 21:13:01 <gmaxwell> (which means that there must be an efficient implementation before making the final decision; simply because you don't want a later one to find that the tree structure kills an important optimization)
1665 2012-08-16 21:13:03 <genjix> although uberpoon is cooler
1666 2012-08-16 21:13:45 <sipa> hehe
1667 2012-08-16 21:13:51 <sipa> haha
1668 2012-08-16 21:14:28 <jgarzik> This is why Bitcoin is fun.  It's like performing maintenance on a car, while driving at 100 m.p.h.
1669 2012-08-16 21:14:36 <justmoon> I'm wondering if it wouldn't make sense to include something now (like the coinhash that ultraprune can already calculate) and replacing it with the fancy merkle stuff later
1670 2012-08-16 21:14:38 <genjix> enough laughing. back to code.
1671 2012-08-16 21:14:57 <jgarzik> Or like perform remote software upgrades to the planet MArs
1672 2012-08-16 21:15:01 <justmoon> yes it means one more hardfork, but realistically how long will it take to get the merkle tree all figured out?
1673 2012-08-16 21:15:07 <jgarzik> gah, grammar
1674 2012-08-16 21:15:15 <justmoon> (and agreed upon)
1675 2012-08-16 21:15:21 <sipa> justmoon: no hard fork necessary!
1676 2012-08-16 21:15:29 <gmaxwell> justmoon: well, it doesn't provide that much value until it's enforced... which requires a majority deployment at least.
1677 2012-08-16 21:15:38 <gmaxwell> and yes, absolutely no hard fork is needed for any of this.
1678 2012-08-16 21:15:45 <genjix> jgarzik: i think you opened a can of worms in my head by mentioning software upgrades on mars
1679 2012-08-16 21:15:53 <justmoon> isn't... requiring... hashes to be valid a hard fork?
1680 2012-08-16 21:15:58 <gmaxwell> No.
1681 2012-08-16 21:16:05 <jgarzik> genjix: grin
1682 2012-08-16 21:16:09 <sipa> justmoon: it's like BIP16
1683 2012-08-16 21:16:20 <gmaxwell> justmoon: you can add restrictions without a hardfork, you just can't take any away.
1684 2012-08-16 21:16:40 <justmoon> oh right, hardfork
1685 2012-08-16 21:16:42 <justmoon> derp
1686 2012-08-16 21:16:57 <justmoon> sorry, it's kinda late is my excuse
1687 2012-08-16 21:17:01 <genjix> jgarzik: i imagine i would approach it like this. make a system clone, send it to earth, do the maintainence there (with the help of the internet), send back a delta.
1688 2012-08-16 21:17:11 <gmaxwell> justmoon: miners can indicate an intent to enforce in the coinbase, when almost all blocks have them.. they enforce. Old clients never notice, the only screwed parties are non-upgraded miners.
1689 2012-08-16 21:17:28 <Joric_> i see nobody pulled off the bounty for ff electrum plugin for 3 months ) thought it's simple
1690 2012-08-16 21:17:38 Joric_ is now known as Joric
1691 2012-08-16 21:17:45 <jgarzik> genjix: yeah, they have a clone on Earth, and a bunch of simulators besides that, for Curiosity
1692 2012-08-16 21:17:45 <gmaxwell> Joric: bitcoin is weird.
1693 2012-08-16 21:17:51 <sipa> jgarzik: performimg maintainance on a car driving at 100 mph, and suddenly realizing that SatoshiDike corp. has removed all signs
1694 2012-08-16 21:17:53 <jgarzik> (they just did a remote software upgrade on Curiosity, on Mars)
1695 2012-08-16 21:17:56 <justmoon> Joric: nothing involving any api that mozilla came up with is "simple"
1696 2012-08-16 21:18:02 <jgarzik> sipa: lol
1697 2012-08-16 21:18:19 <genjix> of course now your system must be small. so i would make a small core system, and then a user system on top of that (kinda like busybox)
1698 2012-08-16 21:18:48 <genjix> http://www.theregister.co.uk/2012/08/07/curiosity_software_upgrade/
1699 2012-08-16 21:19:10 <gmaxwell> sipa: also, the car is being driven by a colection of byzantine agents, some of whom are secretly terrorists who want to kill you and everyone else.
1700 2012-08-16 21:19:38 <jgarzik> we should collect this :)
1701 2012-08-16 21:19:45 <genjix> jgarzik: do they use linux or their own os?
1702 2012-08-16 21:19:52 <gmaxwell> (and some are rational-byzantine-agents who only want to kill you if doing so will fill their homes with pricess religious artifacts*)
1703 2012-08-16 21:20:04 <gmaxwell> s/pricess/priceless/
1704 2012-08-16 21:20:20 <genjix> talking of byzantine agents, this is an old project thomasv was working on: http://savannah.nongnu.org/projects/circle/
1705 2012-08-16 21:20:30 <genjix> it wanted to use BFT and DHT to make money
1706 2012-08-16 21:20:36 <jgarzik> genjix: Curiosity runs on VxWorks embedded OS.  ~133 Mhz PowerPC processor.  Primary and backup computers.  ~128MB of RAM or thereabouts.
1707 2012-08-16 21:20:48 <genjix> which is kind of cool because bitcoin uses similar concepts.
1708 2012-08-16 21:21:06 <genjix> aha ok thanks.
1709 2012-08-16 21:21:36 <jgarzik> genjix: here we go... specs ... http://www.wired.com/wiredenterprise/2012/08/cosmic-rays/
1710 2012-08-16 21:21:48 <jgarzik> 132 Mhz PPC, 120 MB RAM
1711 2012-08-16 21:22:08 <jgarzik> BAE Systems RAD750
1712 2012-08-16 21:22:18 <genjix> interesting.
1713 2012-08-16 21:22:41 <jgarzik> that's a good article about bit flipping
1714 2012-08-16 21:22:54 <jgarzik> fun h/w algorithms, for error correction
1715 2012-08-16 21:22:54 <Joric> bet they have some kind of autonomous emergency rollback in case of unlucky update
1716 2012-08-16 21:22:57 <gmaxwell> I wonder if that PPC is the similar design as whats in a IBM 4765; multple ppc 405 cores that do voting.
1717 2012-08-16 21:22:57 <genjix> isnt that what happened to voyager
1718 2012-08-16 21:22:59 Guest42555 has joined
1719 2012-08-16 21:22:59 Ukto has quit (Disconnected by services)
1720 2012-08-16 21:23:03 Guest42555 is now known as Ukto
1721 2012-08-16 21:23:11 <gmaxwell> ah RAD750.. /me looks up
1722 2012-08-16 21:23:13 Ken` has quit (Read error: Operation timed out)
1723 2012-08-16 21:23:14 Shalom_ has quit (Read error: Operation timed out)
1724 2012-08-16 21:23:22 Maccer has quit (Read error: Operation timed out)
1725 2012-08-16 21:23:32 <jgarzik> Joric: that's what the primary and back comps are for.  upgrade one... see if its fried
1726 2012-08-16 21:23:38 <gmaxwell> hm it's 7xx interesting.
1727 2012-08-16 21:23:46 <genjix> i wonder if it's possible to write software which is redundantly secure from random mutation using byzantine algorithms
1728 2012-08-16 21:23:57 <justmoon> gmaxwell: I want to clarify my "gmaxwell wins" comment for the record - I still think you're wrong, but I do recognize that unfortunately right/wrong doesn't matter too much, you have enough influence to FUD any feature you don't agree with
1729 2012-08-16 21:24:00 <jgarzik> I think you just need a swarm of chips, and vote
1730 2012-08-16 21:24:03 <genjix> but then i suppose you always need a central arbitrer
1731 2012-08-16 21:24:17 <genjix> who counts the vote and takes action?
1732 2012-08-16 21:24:19 hnz has quit (Ping timeout: 256 seconds)
1733 2012-08-16 21:24:31 <gmaxwell> justmoon: Bleh. That makes me sad, perhaps later we can talk some more and perhaps I can try again to convince you?
1734 2012-08-16 21:24:33 <jgarzik> collective consensus behavior :)
1735 2012-08-16 21:24:34 <genjix> i think it would need to be like an octopus with a distributed nervous system
1736 2012-08-16 21:24:48 <gmaxwell> justmoon: I don't want to just convince you via force.
1737 2012-08-16 21:25:01 <gmaxwell> And I certantly don't want to go luke-jr on you.
1738 2012-08-16 21:25:08 <jgarzik> heh
1739 2012-08-16 21:25:26 <Luke-Jr> …
1740 2012-08-16 21:25:43 <Luke-Jr> gmaxwell: y u trolling now? -.-
1741 2012-08-16 21:25:50 wasabi3 has joined
1742 2012-08-16 21:26:01 <gmaxwell> Luke-Jr: you taught me well. :)
1743 2012-08-16 21:26:03 <gmaxwell> <3
1744 2012-08-16 21:26:19 <justmoon> gmaxwell: hmm, our actual disagreement is that you don't want to add something that worsens the security model even if that worsening has no practical application - or are you genuinely worried about somebody exploiting this feature?
1745 2012-08-16 21:26:22 <Luke-Jr> gmaxwell: …
1746 2012-08-16 21:26:25 wasabi2 has quit (Ping timeout: 245 seconds)
1747 2012-08-16 21:26:31 hnz has joined
1748 2012-08-16 21:26:42 Ken` has joined
1749 2012-08-16 21:26:52 <genjix> can i change the electrum link from ecdsa.org/electrum to electrum-desktop.com (new website)?
1750 2012-08-16 21:26:58 Shalom_ has joined
1751 2012-08-16 21:27:02 <justmoon> in the former case I agree - it is theoretically worse, I just don't think it matters, we make security performance tradeoffs all the time and this is a good one
1752 2012-08-16 21:27:12 <justmoon> in the latter case - how can we make it more secure?
1753 2012-08-16 21:27:25 <gmaxwell> justmoon: I'm generally worried about the long term consequences of wide deployment that isn't automatically secure in the absense of trust; because it would invalidate the value proposition of Bitcoin. Also the theoretically worse bit— at least if it can be solved without terrible compromise we should.
1754 2012-08-16 21:27:45 <justmoon> ahh ok, hang on
1755 2012-08-16 21:27:56 <justmoon> I think I just understood better where you are coming from
1756 2012-08-16 21:28:14 <justmoon> so the problem is that it is not *automatically* secure?
1757 2012-08-16 21:29:34 <justmoon> what if I don't include a coinhash, but I let those users who want to supply one?
1758 2012-08-16 21:29:37 BCBot has quit (Ping timeout: 246 seconds)
1759 2012-08-16 21:29:47 <justmoon> similarly to how loadblocks lets you turn off validation for example
1760 2012-08-16 21:30:01 <gmaxwell> Loadblocks doesn't let you turn off validation.
1761 2012-08-16 21:30:16 <genjix> what is a "tie point" in processor design?
1762 2012-08-16 21:30:21 <gmaxwell> justmoon: More abstractly—  Basically, all other monies— digital and otherwise— rest on the security of "you can trust that this party or this group of parties won't cheat, but you can't pratically validate that they won't and no one who can does/will tell you". With bitcoin, everything that is possible to make unbreakable is, everything that can be broken but automatically checked is, and everything that requires trust isn't just about
1763 2012-08-16 21:30:51 BCBot has joined
1764 2012-08-16 21:31:01 <gmaxwell> And what can't be automatically checked or validated is at least auditable (if with great cost, because reviewing code is hard)
1765 2012-08-16 21:31:13 <gmaxwell> So it's important that bitcoin— however the software implementing it is created—  preserves those properties as much as possible.
1766 2012-08-16 21:31:22 <gmaxwell> Becuase they're what make bitcoin worth having in the first place.
1767 2012-08-16 21:31:26 <justmoon> ok I see
1768 2012-08-16 21:31:45 <justmoon> manually entering a coinhash would fulfill that though, no?
1769 2012-08-16 21:31:56 <justmoon> let's say I get it from a trusted friend running a node
1770 2012-08-16 21:32:51 <gmaxwell> I wouldn't worry about some less secure software here and there, of course there already is plenty of that. Except there is a big immediacy pressure to not verify.
1771 2012-08-16 21:32:57 <genjix> bitcoin is far from perfect and compromises in tons of places.
1772 2012-08-16 21:33:13 <genjix> "But it seems to work. Just like Unix, there were countless ways to destroy your data or crash the system, which didn’t exist on more ‘proper’ OSs like OpenVMS, and there were countless lacking features compared to systems like ITS or the Lisp machine OSs. But like the proverbial cockroaches, Unix spread, networked, survived - and the rest did not"
1773 2012-08-16 21:33:17 <gmaxwell> justmoon: if people would actually use it that way, — but they won't, they'll copy the value of blockchain.info or whateve.r
1774 2012-08-16 21:33:52 <gmaxwell> justmoon: Certantly I think thats better than just hard coding it. Makes it a lot harder to attack on the broad scale.
1775 2012-08-16 21:33:57 <justmoon> gmaxwell: I see, so people won't get to make that choice because a small group of people like yourself will not even offer it to them
1776 2012-08-16 21:34:03 <justmoon> sounds a lot like other monies to me
1777 2012-08-16 21:34:11 <justmoon> (sorry to be a bit polemic there)
1778 2012-08-16 21:34:17 <gmaxwell> justmoon: You're too hasty to jump to negative conclusions there, thats unfair to me.
1779 2012-08-16 21:34:27 <justmoon> yeah, I know, I take that last one back
1780 2012-08-16 21:34:50 <justmoon> hmm
1781 2012-08-16 21:34:59 <justmoon> ok, so can we do even better than manually entering it
1782 2012-08-16 21:35:13 <gmaxwell> Well, no... to respond,  I do think there are lines of security where we should choose to not offer footguns, because the self inflicted wounds will be bad for all bitcoin users. Perhaps e.g. taking a coinhash isn't on the bad side of that line though.
1783 2012-08-16 21:36:08 TD has joined
1784 2012-08-16 21:37:11 <gmaxwell> justmoon: and yes, 'enough better' is what I'm looking for there. Obviously the enforced-in-the-coinbase is the gold standard of what we can (and I believe will) eventually get for that.
1785 2012-08-16 21:37:18 DaQatz has quit (Read error: Operation timed out)
1786 2012-08-16 21:37:39 <gmaxwell> That isn't to say that something short of that in the near term isn't okay, so long as it doesn't prevent us from getting to the better end goal.
1787 2012-08-16 21:37:54 root2 has quit ()
1788 2012-08-16 21:38:13 <gmaxwell> Though I really worry that if you just pop a hash into the source that they're will be little motivation to ever do anything harder.
1789 2012-08-16 21:40:02 <justmoon> I understand and agree
1790 2012-08-16 21:40:09 <gmaxwell> justmoon: right now there are a ton of people who are downloading reference client blockchains from http://eu1.bitcoincharts.com/ — you couldn't do worse than that.  I advise people (especially miners!) to not do that, but I'm also not actively going and fudding the downloadable chains out of existance.
1791 2012-08-16 21:40:48 sgornick has joined
1792 2012-08-16 21:41:18 DaQatz has joined
1793 2012-08-16 21:41:24 <justmoon> well, compared to the status quo it would obviously be an improvement - right now I'm the only user of bitcoinjs and I just run it with --noverifysignatures, because my time doesn't grow on trees
1794 2012-08-16 21:41:39 <justmoon> although actually nevermind I only connect it to my local bitcoin node
1795 2012-08-16 21:41:41 <justmoon> so I'm safe
1796 2012-08-16 21:41:59 <gmaxwell> :) Well, I have high hopes for your work.
1797 2012-08-16 21:42:22 <justmoon> that's one of us at least
1798 2012-08-16 21:43:36 Shalom_ has quit (Read error: Operation timed out)
1799 2012-08-16 21:43:53 Shalom_ has joined
1800 2012-08-16 21:44:17 <sipa> gmaxwell: i once started thinking how you could create a blkindex.dat + blk0001.dat file, conjured to do harm
1801 2012-08-16 21:44:44 <justmoon> sipa: hex editor? :P
1802 2012-08-16 21:44:44 <sipa> i can't remember exactly, but except from having people get stuck, it's really quite hard
1803 2012-08-16 21:45:03 <sipa> for example, the PoW of the entire chain is checked at startup
1804 2012-08-16 21:45:06 [\\\] has joined
1805 2012-08-16 21:45:23 <gmaxwell> Depends on if you're byzantine or rational-byzantine. For example, if you just want to crap on p2pool, it's sufficient to cause nodes to get stuck, which just takes a bitflip.
1806 2012-08-16 21:45:41 <Luke-Jr> sipa: what about just cloning a txn index with a hash+1?
1807 2012-08-16 21:45:54 <sipa> Luke-Jr: that will make the PoW fail
1808 2012-08-16 21:46:06 <Luke-Jr> it will? PoW is unchanged.
1809 2012-08-16 21:46:12 <sipa> oh, just transactions?
1810 2012-08-16 21:46:14 <Luke-Jr> yes
1811 2012-08-16 21:46:14 torsthaldo has quit (Ping timeout: 244 seconds)
1812 2012-08-16 21:46:34 <sipa> yes, if they're further back than the 2500 blocks checked at startup
1813 2012-08-16 21:46:59 <Luke-Jr> so clone some early coinbase 1000 times, distribute, spend, win
1814 2012-08-16 21:47:30 <sipa> ???
1815 2012-08-16 21:47:32 <sipa> profit!
1816 2012-08-16 21:47:38 <Luke-Jr> maybe even partition the network with the DoS crap
1817 2012-08-16 21:47:53 <sipa> sure, it's by no means impossible to do harm
1818 2012-08-16 21:48:10 <sipa> it just surprised me at the time that there were more precautions than i expected
1819 2012-08-16 21:48:40 <gmaxwell> I think a lot of that is intended to dampen error propagation... probably a good call; PCs are crap.
1820 2012-08-16 21:48:45 pecket has quit (Ping timeout: 252 seconds)
1821 2012-08-16 21:48:56 <gmaxwell> though it's a bit unfortunate that it can't really recover from any errors it detects.
1822 2012-08-16 21:49:23 <gmaxwell> I guess recovery was less of an issue when blowing away the client and reinstalling wasn't an issue.
1823 2012-08-16 21:49:41 <sipa> some errors should really cause discarding the invalid block instead of only trying to reorg away from it
1824 2012-08-16 21:50:16 <gmaxwell> I mentioned before that one issue with ultraprune is that its much more corruption vulnerable.
1825 2012-08-16 21:50:20 <sipa> any violation of the rules that were checked before it was stored in the block tree, in particular
1826 2012-08-16 21:50:34 <gmaxwell> e.g. if you shutdown uncleanly and slightly corrupt something, you lose. There is no recovery.
1827 2012-08-16 21:51:05 <sipa> yup, we really must rely on sane databases... *crap*
1828 2012-08-16 21:51:08 CodesInChaos has joined
1829 2012-08-16 21:51:09 AlexWaters has joined
1830 2012-08-16 21:51:35 <sipa> an alternative is dumping checksummed snapshots, which can be reverted to
1831 2012-08-16 21:51:40 <gmaxwell> sipa: well, one thing to do is to keep a backup ultraprune snapshot... N block back.. then just play forward from there.
1832 2012-08-16 21:51:50 <gmaxwell> Right, but 2x storage.
1833 2012-08-16 21:51:54 <sipa> indeed
1834 2012-08-16 21:51:56 <gmaxwell> (well 2x the index)
1835 2012-08-16 21:53:15 <gmaxwell> at least having code that knew how to rebuild from 0, if you still have the blocks would be nice, I suppose.
1836 2012-08-16 21:53:54 <Eliel> is there a description of how ultraprune works somewhere?
1837 2012-08-16 21:54:24 <sipa> https://bitcointalk.org/index.php?topic=91954.0
1838 2012-08-16 21:55:34 ovidiusoft has quit (Ping timeout: 240 seconds)
1839 2012-08-16 21:56:11 <sipa> gmaxwell: just delete coins.dat and you're there
1840 2012-08-16 21:56:18 <sipa> gmaxwell: it will reorg automatically at startup even
1841 2012-08-16 21:56:43 Maccer has joined
1842 2012-08-16 21:56:51 <Luke-Jr> or just go back in time and delete Satoshi before he releases Bitcoin. CERN is watching.
1843 2012-08-16 21:57:38 <gmaxwell> sipa: it looks like it would be fairly expensive to update a hash root with ultraprune, you'd basically have to keep our own index in addition to what leveldb does. Bummer.
1844 2012-08-16 21:58:03 <sipa> gmaxwell: ?
1845 2012-08-16 21:58:25 <sipa> oh, yes, i think the merkle tree will not coincide with the database tree
1846 2012-08-16 21:58:31 <sipa> but i'm not sure it should, either
1847 2012-08-16 21:59:48 <gmaxwell> Hm. But if the tree's layout supports efficient queries, you would basically halve your cost of updates. Otherwise an update will have to visit 2x the nodes on average.
1848 2012-08-16 22:00:11 <gmaxwell> E.g. one because it's updating the database index, another different one because it's the sibling in the hash tree.
1849 2012-08-16 22:01:54 <justmoon> hmm... if I implemented one of the merkle tree proposals locally right now could I get rid of ultraprune and of the address -> txids index I'm keeping?
1850 2012-08-16 22:02:10 <justmoon> i.e. still full block chain download, just a matter of the data structure I'm keeping
1851 2012-08-16 22:02:18 <sipa> you need a txid->coins map to do block validation
1852 2012-08-16 22:02:44 <sipa> eto's merkle tree proposal does not have a way to do that, iirc
1853 2012-08-16 22:02:46 <justmoon> hmm yeah that's right, but don't I know the address also from the input?
1854 2012-08-16 22:02:56 <sipa> maybe
1855 2012-08-16 22:03:13 <justmoon> I'd have to special case pay-to-pubkey of course and certain custom scripts
1856 2012-08-16 22:05:20 <justmoon> it'd be much much bigger than ultraprune though, right?
1857 2012-08-16 22:05:31 <gmaxwell> Yea, I don't like ETO's address oriented stuff for that reason, it's a fine additional index to keep, but it seems a bit short sighted to me.
1858 2012-08-16 22:05:37 m00p has quit (Ping timeout: 252 seconds)
1859 2012-08-16 22:05:45 <sipa> not that much more, i think
1860 2012-08-16 22:06:02 <sipa> there's on average two outputs per address in the coinset
1861 2012-08-16 22:07:49 LuaKT has quit ()
1862 2012-08-16 22:09:32 <Eliel> justmoon: you can transform any script to a P2SH address it'd have if it was used in a P2SH transaction.
1863 2012-08-16 22:09:57 <Eliel> even if it has no regular address.
1864 2012-08-16 22:10:29 <justmoon> Eliel: you can transform any output script yes, but for certain transactions like pay-to-pubkey, you can not (easily) guess the output from the input, or the address from the input
1865 2012-08-16 22:10:41 <justmoon> a pay-to-pubkey input is just a signature
1866 2012-08-16 22:10:51 <gmaxwell> Eliel: also, no one should expect people will reconize an address they didn't give out.
1867 2012-08-16 22:11:20 <gmaxwell> In _theory_ you could accept all sorts of wonky things.
1868 2012-08-16 22:11:27 <justmoon> gmaxwell: I think it'd be fine for indexing purposes, the index will only really be queried if it is meaningful
1869 2012-08-16 22:11:39 <Eliel> gmaxwell: well, this would only be needed when there's no natural address for a script.
1870 2012-08-16 22:11:55 <sipa> justmoon: also, you cannot ever *know* a certain input has a presumed output
1871 2012-08-16 22:12:19 <Eliel> but yes... this is a difficult problem.
1872 2012-08-16 22:12:30 <sipa> justmoon: i can have a send to pubkey and consume it with a pubkeyhash input script
1873 2012-08-16 22:12:34 <justmoon> sipa: true, but again, it doesn't matter if your assumptions are wrong
1874 2012-08-16 22:12:35 <Eliel> although, can't you keep two trees? one for addresses and other for txids?
1875 2012-08-16 22:13:08 <justmoon> sipa: if you index a transaction under some key and it doesn't actually mean that - it doesn't hurt anyone, it just means your index doesn't cover certain non-standard transactions
1876 2012-08-16 22:13:29 <sipa> agree
1877 2012-08-16 22:13:50 bobke has quit (Read error: Connection reset by peer)
1878 2012-08-16 22:13:53 <gmaxwell> justmoon: right but not if it's your only index. I think as a secondary index its fine.
1879 2012-08-16 22:14:05 <justmoon> though of course someone's custom script *could* collide with someone else's address hash :P
1880 2012-08-16 22:14:26 <justmoon> gmaxwell: right
1881 2012-08-16 22:16:00 <gmaxwell> sipa: as far as tree ordered ultraprune, in theory I think the whole database could be only ~2x the size of the txout values.
1882 2012-08-16 22:16:39 <gmaxwell> sipa: if the hash values are stored in a compressed trie the index+data is basically the same size as the data.
1883 2012-08-16 22:16:50 <sipa> gmaxwell: indeed
1884 2012-08-16 22:19:11 pecket has joined
1885 2012-08-16 22:19:17 <sipa> wow, i have 400 MiB undo data after the 185k checkpoint
1886 2012-08-16 22:20:22 CodesInChaos has quit (Ping timeout: 265 seconds)
1887 2012-08-16 22:20:51 datagutt has quit (Quit: Computer has gone to sleep.)
1888 2012-08-16 22:21:40 <justmoon> gmaxwell: given a standard output I can be sure that the input that spends it, if executed, will leave a stack that let's me find this output again if I index it by address, correct?
1889 2012-08-16 22:21:46 <justmoon> for special casing to work...
1890 2012-08-16 22:22:06 <justmoon> I have to be able to at least be sure for standard outputs that I'll be able to find them again even if the corresponding input is wonky
1891 2012-08-16 22:22:48 <gmaxwell> Unless I'm misunderstanding you, you certantly cant trust that for unusual transactions.
1892 2012-08-16 22:22:56 <justmoon> no for standard outputs
1893 2012-08-16 22:23:04 <justmoon> can I trust it for standard outputs
1894 2012-08-16 22:23:07 <sipa> yes
1895 2012-08-16 22:23:13 <justmoon> I think so, right?
1896 2012-08-16 22:23:21 <gmaxwell> Yes, for standard ones, I think so.
1897 2012-08-16 22:23:49 Zarutian has quit (Quit: Zarutian)
1898 2012-08-16 22:24:12 leotreasure has quit (Quit: leotreasure)
1899 2012-08-16 22:29:51 RainbowDashh has joined
1900 2012-08-16 22:32:51 MobiusL has quit (Remote host closed the connection)
1901 2012-08-16 22:34:02 MobiusL has joined
1902 2012-08-16 22:35:27 minimoose has quit (Quit: minimoose)
1903 2012-08-16 22:39:28 RazielZ has quit (Ping timeout: 244 seconds)
1904 2012-08-16 22:39:35 OneFixt has quit (Read error: Connection reset by peer)
1905 2012-08-16 22:39:50 OneFixt has joined
1906 2012-08-16 22:41:58 rdponticelli_ has joined
1907 2012-08-16 22:42:11 `2Fast2BCn has quit (Ping timeout: 260 seconds)
1908 2012-08-16 22:42:43 `2Fast2BCn has joined
1909 2012-08-16 22:42:52 [7] has quit (Remote host closed the connection)
1910 2012-08-16 22:43:07 TheSeven has joined
1911 2012-08-16 22:43:21 Guest15374 has quit (Ping timeout: 260 seconds)
1912 2012-08-16 22:43:24 rdponticelli has quit (Read error: Connection reset by peer)
1913 2012-08-16 22:47:23 <amiller> justmoon, do you have any experience building a customized kademlia
1914 2012-08-16 22:47:40 <amiller> i think the right thing to do is to develop a merkle tree prototype and a corresponding DHT at the same time
1915 2012-08-16 22:47:48 <amiller> since both will be more useful with each other
1916 2012-08-16 22:48:57 <justmoon> amiller: merkle tree should almost come first, shouldn't it? not necessary in the chain and everything, but integrated in the client to use as its backend
1917 2012-08-16 22:49:04 <justmoon> necessarily*
1918 2012-08-16 22:49:23 Marf has quit (Ping timeout: 272 seconds)
1919 2012-08-16 22:49:28 <amiller> if it's not in the chain and no one is committing to it i'm not sure how useful it would be but i guess it could still be separate
1920 2012-08-16 22:49:39 <amiller> it could always be an alt chain to begin with
1921 2012-08-16 22:49:52 <justmoon> and no I have no experience building a customized kademlia, but I have a lot of experience successfully doing things I have no experience with
1922 2012-08-16 22:50:06 <amiller> i guess the merkle tree should come first but at least setting up the DHT would be roughly orthogonal effort
1923 2012-08-16 22:51:01 <justmoon> I'm quite interested in rushing this into a demo-able prototype stage
1924 2012-08-16 22:51:17 <amiller> gmaxwell, you mentioned you had another reason for prefering tries, i wonder why that is?
1925 2012-08-16 22:51:43 <amiller> i've been having a long discussion with etotheipi about tries-vs-balancetrees over private message
1926 2012-08-16 22:52:07 <amiller> at least for redblack, all that would need to be done is to settle on a serialization format for each node in the tree
1927 2012-08-16 22:52:10 bobke has joined
1928 2012-08-16 22:52:35 brwyatt is now known as Away!~brwyatt@brwyatt.net|brwyatt
1929 2012-08-16 22:54:13 <justmoon> it would probably be good to make that discussion public as soon as possible - for better or for worse if I want to implement this I have to pick one or the other
1930 2012-08-16 22:54:35 <amiller> well we went back and forth a half dozen times at the end of that thread
1931 2012-08-16 22:55:16 <justmoon> yeah, if you can't agree, present the arguments and other people will be able to form an opinion on which they want to put their chips
1932 2012-08-16 22:55:43 copumpkin has quit (Quit: Computer has gone to sleep.)
1933 2012-08-16 22:55:45 <amiller> a lot of people are having a hard time grokking the concept well enough to form an opinion
1934 2012-08-16 22:56:12 <justmoon> well the alternative is for me to flip a coin
1935 2012-08-16 22:56:22 <justmoon> or what do you suggest?
1936 2012-08-16 22:56:40 <amiller> justmoon, obviously i suggest building on my redblack tree, since i already have a reference implementation for one of them
1937 2012-08-16 22:56:44 <justmoon> :D
1938 2012-08-16 22:56:52 <justmoon> that is a strong argument actually
1939 2012-08-16 22:57:29 <justmoon> do you both agree on the structure though? i.e. indexing by address?
1940 2012-08-16 22:58:07 <amiller> i want to either support a) multiple trees, at least a txid->coin and additionally an address->coin, or b) one tree with an 'index-type' prefix
1941 2012-08-16 22:58:10 <amiller> they're equivalent
1942 2012-08-16 22:59:11 <sipa> justmoon: imho, a committed txid->coins tree is more useful, as it allows bootstrapping validating SPV-level nodes from an ultraprune snapshot
1943 2012-08-16 22:59:26 <amiller> agreed
1944 2012-08-16 22:59:30 <sipa> i see address-to-something indexes as interesting extras
1945 2012-08-16 22:59:34 chrisb__ has quit (Quit: Leaving)
1946 2012-08-16 23:00:03 RainbowDashh has quit (Read error: Connection reset by peer)
1947 2012-08-16 23:00:47 <sipa> oh, and the 'coinhash' thing i had before was a very inefficient hash of the linearly serialized coinset
1948 2012-08-16 23:00:53 <sipa> so that is not something you should use
1949 2012-08-16 23:01:00 <justmoon> sure, I saw what it did
1950 2012-08-16 23:01:22 <justmoon> it has the hard-to-argue-with property of existing
1951 2012-08-16 23:01:40 <justmoon> but ok, let's be ambitious and go for the proper(er) solution
1952 2012-08-16 23:02:11 <sipa> btw, i never managed to get the coinhashes to match between data extracted from an ultraprune node, and from a former blkindex.dat
1953 2012-08-16 23:02:37 <sipa> afterwards i just managed to get identical copies of a serialized coinset out of both, but didn't port the bugfixes back to coinhash
1954 2012-08-16 23:03:22 <amiller> in my scheme, a node in a redblack tree consists of: (c, dL, k, dR) where c is a one bit color, dL and dR are digests of the children, and k is the largest hash in the left subtree
1955 2012-08-16 23:03:48 <amiller> a leaf node could have sentinel values for dL and dR and then k is the actual value of that leaf
1956 2012-08-16 23:04:22 <amiller> the digests, in addition to the hash, should also include a a count/rank of all the nodes in that tree
1957 2012-08-16 23:04:38 <amiller> this allows you to terminate early
1958 2012-08-16 23:04:49 <amiller> (you can't be tricked into checking more than O(log N) nodes)
1959 2012-08-16 23:05:09 <sipa> i wonder, given the fact that most practical implementations will map nodes to database entries, with a non-constant lookup time themselves, whether you shouldn't use a higher fanout than 2
1960 2012-08-16 23:05:19 <amiller> practical implementations would use higher fanout than 2
1961 2012-08-16 23:05:30 <sipa> so you need fewer steps, but at the cost of somewhat larger nodes (net)
1962 2012-08-16 23:05:36 <sipa> s/steps/depth/
1963 2012-08-16 23:05:53 <amiller> hashing time is proportional to the size of the data right
1964 2012-08-16 23:06:06 <amiller> so for the normative part of the format i think fanout-2 is as good as optimal
1965 2012-08-16 23:06:27 <sipa> well, hashing time depends on how many blocks of input you have
1966 2012-08-16 23:06:28 <amiller> how you actually store/retrieve it is up to the implementation
1967 2012-08-16 23:06:35 <sipa> and sha256 has 64 byte inputs
1968 2012-08-16 23:07:06 <sipa> right, you can store serialized clusters of several binary nodes on the storage layer
1969 2012-08-16 23:07:16 rdponticelli_ has quit (Ping timeout: 260 seconds)
1970 2012-08-16 23:07:35 <amiller> fanout2 would be at least two full blocks then right
1971 2012-08-16 23:07:45 <amiller> since each hash is 32 bytes
1972 2012-08-16 23:07:53 <amiller> er at least one full block
1973 2012-08-16 23:08:06 <sipa> you could truncate the hashes, if the security parameters allow that
1974 2012-08-16 23:08:14 <amiller> interesting
1975 2012-08-16 23:08:29 <sipa> since our addresses only have 160-bit security anyway...
1976 2012-08-16 23:08:39 <sipa> but i'm not entirely convinced this is a good idea
1977 2012-08-16 23:09:42 <amiller> the CMerkleTree is already just a binary tree right
1978 2012-08-16 23:09:56 <amiller> maybe we can deviate minimally from that
1979 2012-08-16 23:10:15 <sipa> uhu
1980 2012-08-16 23:12:37 dvide has joined
1981 2012-08-16 23:13:33 copumpkin has joined
1982 2012-08-16 23:14:56 <amiller> so (colorbit, largestkeyinleft, (hashLeft, countLeft),(hashRight, countRight))
1983 2012-08-16 23:15:27 <sipa> count?
1984 2012-08-16 23:15:41 bakh has joined
1985 2012-08-16 23:15:55 <Diablo-D3> was that supposed to be a redblack tree?
1986 2012-08-16 23:15:55 <amiller> it's important for ddos resistance to avoid processing too many nodes
1987 2012-08-16 23:16:26 <amiller> it's a count of the number of leaf nodes in each subtree
1988 2012-08-16 23:16:43 <amiller> and of course the sum of those is the total for the tree
1989 2012-08-16 23:17:13 <amiller> maybe it's better just to have (colorbit, numberofleaves, largestkeyinleft, hashleft, hashright)
1990 2012-08-16 23:18:50 <amiller> hrm maybe it's not necessary to include in each node anyway
1991 2012-08-16 23:19:38 <sipa> not include what?
1992 2012-08-16 23:21:26 <amiller> the count of number of leaves
1993 2012-08-16 23:25:51 <amiller> in my reference implementation i made it flexible so that you could include additional annotations in the digest, such as the total number of leaves or a weighted sum. At the time I was really interested in being able to select a leaf node uniformly at random, but I'm not sure it matters now
1994 2012-08-16 23:26:07 <amiller> the minimal tree format is just (colorbit, largestkeyinleft, hashLeft, hashRight)
1995 2012-08-16 23:26:57 bobke has quit (Read error: Connection reset by peer)
1996 2012-08-16 23:27:28 RainbowDashh has joined
1997 2012-08-16 23:28:58 <amiller> other than that, there are about 8 balancing rules that need to be applied - those are the 'moving parts' that are a bit tedious to implement and verify
1998 2012-08-16 23:29:26 <amiller> each one concerns at most 4 nodes at a time
1999 2012-08-16 23:30:32 <sipa> one thing that comes to mind
2000 2012-08-16 23:30:52 <sipa> when applying the 'patch' to the tree that is a block, you'll be updating many leaf nodes
2001 2012-08-16 23:31:09 <sipa> and you'll have to propagate from those upwards to recalculate hashes
2002 2012-08-16 23:31:34 <sipa> but a significant percentage of nodes to be updated may be shared by multiple leafs
2003 2012-08-16 23:32:26 <sipa> so if your implementation has something like a 'dirty bit' (which may be a separate in-memory set) of nodes to be updated, you can gain some performance
2004 2012-08-16 23:32:42 <sipa> instead of just updating all the way to the root for every update
2005 2012-08-16 23:33:41 <amiller> good point, for example the root node will _always_ be recomputed, so a block with K transactions would have K intermediate roots, but you would never need to store/write the intermediate roots
2006 2012-08-16 23:33:58 <sipa> exactly
2007 2012-08-16 23:34:26 bakh has quit (Quit: Ex-Chat)
2008 2012-08-16 23:34:39 <sipa> the tree structure guarantees N*log(M) complexity for M updates, but that number can in theory exceed M
2009 2012-08-16 23:34:43 diki has joined
2010 2012-08-16 23:34:53 <sipa> sorry, for N updates
2011 2012-08-16 23:35:16 <diki> After wondering why I was getting a memory leak from the vanitygen's source I ripped, I noticed it was missing a EC_KEY_free(pkey); line in vg_thread_loop.
2012 2012-08-16 23:35:34 <Diablo-D3> ohnice
2013 2012-08-16 23:36:56 <amiller> sipa, i bet you can even avoid recomputing the intermediate hashes at all
2014 2012-08-16 23:36:58 <diki> The problem would not be noticed in any way due to the way it's built, but if you decide to use the code as a standalone and call vg_thread_loop multiple times, you need to EC_KEY_free(pkey);
2015 2012-08-16 23:37:10 <sipa> amiller: sure, it's not hard
2016 2012-08-16 23:37:17 <sipa> amiller: it's just something you shouldn't forget
2017 2012-08-16 23:37:48 <sipa> amiller: keep a set of nodes-to-be-recomputed in memory, and iterate that set by replacing two entries in it by their parent
2018 2012-08-16 23:38:10 <amiller> great suggestion
2019 2012-08-16 23:38:46 <amiller> it's like it would start out as a tree with actual pointers, and eventually the pointers get turned into hashes and then forgotton
2020 2012-08-16 23:42:38 <Joric> diki, it most probably reuses it
2021 2012-08-16 23:46:21 MC-Eeepc has quit (Quit: Leaving)
2022 2012-08-16 23:48:03 Motest003 has joined
2023 2012-08-16 23:49:41 bobke has joined
2024 2012-08-16 23:50:27 <diki> Joric:There are only a few references to ec_key_free and they are only in keyconv.c
2025 2012-08-16 23:50:42 <diki> which is a standalone app that comes with vanitygen.
2026 2012-08-16 23:50:48 <Eliel> amiller: that sounds like it increases the memory requirement for the update though.
2027 2012-08-16 23:50:57 <diki> plus, this was fixed in the bitcoin client.
2028 2012-08-16 23:51:25 <Eliel> well, that's probably not too significant.
2029 2012-08-16 23:52:24 <Eliel> ... wait, maybe it is. It sounds like it'd make the memory requirement O(M), where M is the number of updated nodes.
2030 2012-08-16 23:52:41 <amiller> it could easily be a tradeoff where you either use more memory or you have to compute more hashes
2031 2012-08-16 23:53:15 <Eliel> yes, true, implementation decision, can do it either way.
2032 2012-08-16 23:54:19 <sipa> Eliel: there already is a memory requires O(M), as you need to cache the intended changes while processing the block, before knowing they're all valid to perform anyway
2033 2012-08-16 23:54:57 <Eliel> well, you could keep them on disk and do them one at a time if needed :P
2034 2012-08-16 23:55:15 <sipa> Eliel: the same can be done with the to-be-updated-set :)
2035 2012-08-16 23:55:32 <Eliel> yep :)
2036 2012-08-16 23:55:42 <amiller> perhaps even within a block, the transactions could be put in a merkle tree
2037 2012-08-16 23:55:55 <amiller> then it would be easier to validate them out of order i guess.. that doesn't really make any sense
2038 2012-08-16 23:56:09 <Eliel> they're already in a merkle tree, no?
2039 2012-08-16 23:56:43 <amiller> uhh.. yeah i guess they are.. in fact that's the sole reason merkle trees are currently used so i know i knew that
2040 2012-08-16 23:56:44 bobke_ has joined
2041 2012-08-16 23:57:00 <sipa> indeed
2042 2012-08-16 23:57:02 rdponticelli has joined
2043 2012-08-16 23:57:09 <amiller> well so yeah you don't necessarily need to cache them
2044 2012-08-16 23:57:21 bobke has quit (Read error: Connection reset by peer)
2045 2012-08-16 23:57:37 <sipa> ?
2046 2012-08-16 23:57:49 bobke_ is now known as bobke
2047 2012-08-16 23:57:55 <amiller> oh i see what you mean
2048 2012-08-16 23:58:13 <amiller> if the last tx is invalid then you must undo all your changes which means you probably didn't want to write them yet
2049 2012-08-16 23:58:17 <sipa> indeed
2050 2012-08-16 23:58:51 <amiller> the tx-in-block merkle tree right now contains double hashes
2051 2012-08-16 23:58:51 <sipa> the current block validation works in several iterations, doing the cheapest checks first, and signature checks lasts
2052 2012-08-16 23:58:53 <amiller> is there a reason why?
2053 2012-08-16 23:59:07 <sipa> satoshi liked it, probably
2054 2012-08-16 23:59:53 <amiller> sipa, ohhhh, i suddenly understand then why checkpoints are about just avoiding the signature check then