1 2012-11-15 00:00:06 dparrish has joined
   2 2012-11-15 00:06:55 BitcoinBaltar has quit (Ping timeout: 276 seconds)
   3 2012-11-15 00:09:27 BitcoinBaltar has joined
   4 2012-11-15 00:12:50 dparrish has quit (Ping timeout: 240 seconds)
   5 2012-11-15 00:13:11 dparrish has joined
   6 2012-11-15 00:15:30 Hasimir- is now known as Hasimir
   7 2012-11-15 00:15:30 Hasimir has quit (Changing host)
   8 2012-11-15 00:15:30 Hasimir has joined
   9 2012-11-15 00:27:30 xIsalty is now known as Sssateneth
  10 2012-11-15 00:32:16 Sssateneth is now known as xIsalty
  11 2012-11-15 00:33:39 xIsalty is now known as Juke-Lr
  12 2012-11-15 00:36:55 Juke-Lr is now known as xIsalty
  13 2012-11-15 00:40:34 mikehard3 has quit (Ping timeout: 264 seconds)
  14 2012-11-15 01:14:05 joepie91 has joined
  15 2012-11-15 01:18:41 <Luke-Jr> sipa: should I have blocks/rev00000.dat when I've only ever used testnet? :/
  16 2012-11-15 01:28:26 one_zero has joined
  17 2012-11-15 01:38:19 ne0futur is now known as neofutur_
  18 2012-11-15 01:39:10 word has joined
  19 2012-11-15 01:39:52 neofutur_ is now known as neofutur`
  20 2012-11-15 01:41:48 PhantomSpark has joined
  21 2012-11-15 01:52:51 meLon has quit (Remote host closed the connection)
  22 2012-11-15 01:52:52 BitcoinBaltar has quit (Ping timeout: 276 seconds)
  23 2012-11-15 01:54:20 <cjd> I'm writing a difficulty calculator and I'm wondering if I should just reject the sign bit outright or try to handle it
  24 2012-11-15 01:54:51 <Diablo-D3> erm
  25 2012-11-15 01:54:58 <Diablo-D3> isnt it an unsigned number?
  26 2012-11-15 01:55:09 <cjd> I guess there's no way for the difficulty to end up in the negatives so it's just a latent bug that the retargetting calculation will work in negative
  27 2012-11-15 01:55:19 BitcoinBaltar has joined
  28 2012-11-15 01:55:40 <copumpkin> sign bit?
  29 2012-11-15 01:55:54 <cjd> this bit should never be set in the diff in any block 0x00800000
  30 2012-11-15 01:56:14 <cjd> if the high bit is set then it's 0 padded by openssl
  31 2012-11-15 01:57:50 <cjd> but if you feed GetNextWorkRequired() a difficulty with that bit set, you get another number which will also have that bit set but all of the math is openssl is done using negative numbers so it's all wacky
  32 2012-11-15 01:59:12 <cjd> At first I thought it was somehow abusable but apparently there's no way to force a retarget into the negatives so it's just an annoying caviat, not a bug
  33 2012-11-15 02:03:46 forsetifox_ is now known as forsetifox
  34 2012-11-15 02:11:23 <Luke-Jr> I don't see how a difficulty could ever be negative.
  35 2012-11-15 02:13:14 <cjd> it's not negative, it's just that openssl handles it as a negative
  36 2012-11-15 02:14:20 <cjd> bnNew.SetCompact(0x0ded7263).GetHex()  --> -6d726300000000000000000000
  37 2012-11-15 02:15:56 <cjd> this is why you see difficulties with a 0 byte like  0x12003456  even though there should be no high zeros in a mantissa.
  38 2012-11-15 02:16:42 <cjd> heh I need to scan the actual header chain to know if they really exist but according to my fuzzer there should be plenty of them
  39 2012-11-15 02:17:08 <Luke-Jr> difficulty 1 for example
  40 2012-11-15 02:17:15 <Luke-Jr> should be 50%
  41 2012-11-15 02:17:36 <rdponticelli> I know some chics who can handle to turn it's difficultness into the negatives...
  42 2012-11-15 02:17:46 <cjd> indeed, any difficulty with a high bit set will have a zero padding
  43 2012-11-15 02:18:23 <cjd> it's annoying because it's "part of the definition" but it never happens
  44 2012-11-15 02:18:59 <Luke-Jr> cjd: it's part of the format for the custom floating-point type. but it's not used in the context of targets/difficulties
  45 2012-11-15 02:19:13 <cjd> unless the last retarget was in the future
  46 2012-11-15 02:19:24 <Luke-Jr> …
  47 2012-11-15 02:19:33 <Luke-Jr> which is impossible
  48 2012-11-15 02:19:45 <cjd> mmm
  49 2012-11-15 02:20:38 <cjd> it doesn't look like there's any other way to trick it into setting that bit but if there was then the network would split instantly
  50 2012-11-15 02:20:46 <Luke-Jr> would it?
  51 2012-11-15 02:20:58 <cjd> unless all implementations have this weird negative computation behavior
  52 2012-11-15 02:21:08 <gmaxwell> 'all' lol
  53 2012-11-15 02:21:13 <Luke-Jr> cjd: they're required to
  54 2012-11-15 02:21:45 <cjd> meh it's hard to write and it can't happen in real life
  55 2012-11-15 02:21:54 <gmaxwell> An implementation isn't correct unless it reproduces the references undocumented normative bugs, including the ones that key imported from implicity behavior in system libraries.
  56 2012-11-15 02:22:15 <cjd> hehe do there's only one implementation :P
  57 2012-11-15 02:22:30 <cjd> one correct one
  58 2012-11-15 02:22:41 <gmaxwell> Cant happen != isn't expected, be very careful in assuming what can't happen.
  59 2012-11-15 02:23:56 <Luke-Jr> cjd: note this same sign bit thing also affects the height in blocks
  60 2012-11-15 02:24:27 <cjd> I'm trying to get a grasp on what would happen if it did, all the math would round "up" since it rounds to zero and the numbers would come out with 1 more byte of precision
  61 2012-11-15 02:30:49 luke-jr_ has joined
  62 2012-11-15 02:30:56 Luke-Jr has quit (Read error: Connection reset by peer)
  63 2012-11-15 02:40:33 Arnavion has quit (Quit: Arnavion)
  64 2012-11-15 02:43:49 Arnavion has joined
  65 2012-11-15 03:04:22 denisx has quit (Quit: denisx)
  66 2012-11-15 03:07:05 noagendamarket has joined
  67 2012-11-15 03:07:25 noagenda_ has joined
  68 2012-11-15 03:09:36 noagenda_ has quit (Client Quit)
  69 2012-11-15 03:10:56 noagendamarket has quit (Client Quit)
  70 2012-11-15 03:11:26 noagendamarket has joined
  71 2012-11-15 03:11:30 noagenda_ has joined
  72 2012-11-15 03:11:34 noagenda_ has quit (Read error: Connection reset by peer)
  73 2012-11-15 03:14:23 paraipan has quit (Quit: Saliendo)
  74 2012-11-15 03:17:57 Guest66396 has quit (Changing host)
  75 2012-11-15 03:17:57 Guest66396 has joined
  76 2012-11-15 03:18:04 Guest66396 is now known as arij
  77 2012-11-15 03:28:56 Garr255 has joined
  78 2012-11-15 03:28:57 Garr255 has quit (Changing host)
  79 2012-11-15 03:28:57 Garr255 has joined
  80 2012-11-15 03:30:16 swulf-- has joined
  81 2012-11-15 03:31:28 swulf--1 has quit (Ping timeout: 245 seconds)
  82 2012-11-15 03:33:09 Gladamas has joined
  83 2012-11-15 03:36:02 JZavala has joined
  84 2012-11-15 03:38:07 <cjd> looks like I got it
  85 2012-11-15 03:38:09 <cjd> https://ezcrypt.it/nB5n#BvbyfcJ0bj3Axzp956gw381e
  86 2012-11-15 03:38:58 <Diablo-D3> got what?
  87 2012-11-15 03:39:20 <cjd> reimplemented the retarget function
  88 2012-11-15 03:39:24 <Diablo-D3> ahh
  89 2012-11-15 03:39:27 <cjd> it does not handle broken "negative" difficulty
  90 2012-11-15 03:39:36 <Diablo-D3> [08:42:53] <cjd> I guess there's no way for the difficulty to end up in the negatives so it's just a latent bug that the retargetting calculation will work in negative
  91 2012-11-15 03:39:39 <Diablo-D3> btw, yes
  92 2012-11-15 03:39:41 <Diablo-D3> it just goes below 0
  93 2012-11-15 03:39:44 <Diablo-D3> er
  94 2012-11-15 03:39:45 <Diablo-D3> below 1
  95 2012-11-15 03:40:03 <cjd> are we talking about the same negative?
  96 2012-11-15 03:40:09 <Diablo-D3> we might be?
  97 2012-11-15 03:40:33 <Diablo-D3> half of 1 difficulty is 0.5, twice 1 difficulty is 2
  98 2012-11-15 03:40:56 <cjd> I'm talking about a bug in the compaction scheme wherein openssl will do the multiplication and division asif the number was negative if bit number 24 is set
  99 2012-11-15 03:41:05 <Diablo-D3> ahh
 100 2012-11-15 03:41:10 <Diablo-D3> two different things then
 101 2012-11-15 03:41:54 <cjd> but the only way that can happen AFAICT is for the last retargetting timestamp to be in the future
 102 2012-11-15 03:42:22 <Diablo-D3> erm, it can be in the future.
 103 2012-11-15 03:42:43 <cjd> indeed, it appears that it can't happen
 104 2012-11-15 03:43:35 <cjd> but it's there and unless we (aka the devs) want to ban blocks where (block->bits & 0x00800000) != 0, the bug will just sit there
 105 2012-11-15 03:45:02 <cjd> If people like the idea, I'll even write the patch because I want bitcoin to be proven and standardized rather than magical and indeterminent
 106 2012-11-15 03:45:38 fiesh has quit (Ping timeout: 245 seconds)
 107 2012-11-15 03:46:28 <Diablo-D3> I like magical code :<
 108 2012-11-15 03:46:32 fiesh has joined
 109 2012-11-15 03:47:01 * Diablo-D3 flips the switch on the side of the case labeled magic that only has one wire going into it, machine instantly crashes
 110 2012-11-15 03:47:14 <cjd> :)
 111 2012-11-15 03:47:51 <cjd> in cryptography, anything that makes the code hard to validate is IMO a bug
 112 2012-11-15 03:48:06 <Diablo-D3> yeah, but arguably thats a problem with the language
 113 2012-11-15 03:48:24 <Diablo-D3> some languages make writing magic code so much easier
 114 2012-11-15 03:48:57 <cjd> I can say I'm not too thrilled with C++ at the moment ;)
 115 2012-11-15 03:49:24 <Diablo-D3> I should try to make seaking as antimagic as possible
 116 2012-11-15 03:49:42 <cjd> seaking?
 117 2012-11-15 03:49:49 <Diablo-D3> yeah, Im working on a new language
 118 2012-11-15 03:49:54 <cjd> ahh
 119 2012-11-15 03:50:02 <Diablo-D3> one supreme language to end all other languages
 120 2012-11-15 03:50:09 <cjd> I had an idea for a language which is infinitely extensible
 121 2012-11-15 03:50:18 <Diablo-D3> lol, thats seaking
 122 2012-11-15 03:50:20 <Diablo-D3> @keywords
 123 2012-11-15 03:50:30 <cjd> my name for it is hashfilter because that's it's only command
 124 2012-11-15 03:50:31 <Diablo-D3> tells the compiler to use a plugin named keyword
 125 2012-11-15 03:50:43 <Diablo-D3> and you can do _whatever you want_ in the plugin
 126 2012-11-15 03:50:51 <Diablo-D3> no matter how weird or strange
 127 2012-11-15 03:51:09 <cjd> #filter <afilter.c>  causes afilter.c to be compiled and loaded and it preprocesses the rest of your code
 128 2012-11-15 03:51:27 <Diablo-D3> yeah, you'd like seaking I think
 129 2012-11-15 03:51:42 <cjd> I like it because it's just a few thousand lines of code to make it work
 130 2012-11-15 03:51:50 <cjd> and because I can use it to target any language I like
 131 2012-11-15 03:51:58 <Diablo-D3> yeah, see, Im not writing a vm
 132 2012-11-15 03:51:59 <cjd> I'll write C with my own filter stack
 133 2012-11-15 03:52:02 <Diablo-D3> or a compiler, to be honest
 134 2012-11-15 03:52:09 <Diablo-D3> Im just plugging it into llvm
 135 2012-11-15 03:52:13 <Diablo-D3> and seaking is C-like
 136 2012-11-15 03:52:22 <Diablo-D3> its going to end up as fast as C in the end since Im using a real compiler
 137 2012-11-15 03:52:34 <cjd> as fast as llvm c :P
 138 2012-11-15 03:52:38 <Diablo-D3> well, yeah
 139 2012-11-15 03:52:44 <Diablo-D3> although theres certain shit I wont allow
 140 2012-11-15 03:53:07 <Diablo-D3> like, you know the restrict keyword in c99?
 141 2012-11-15 03:53:19 <Diablo-D3> thats the default behavior in seaking
 142 2012-11-15 03:53:27 <cjd> I'm partial to GCC and I'd just like to replace the regular C preprocessor with my filter processor
 143 2012-11-15 03:53:37 <Diablo-D3> well, thats what I started with seaking
 144 2012-11-15 03:53:46 <Diablo-D3> but cpp just isnt strong enough
 145 2012-11-15 03:54:05 <cjd> I'd rather compile to source because I want to be able to disect the result of the compilation
 146 2012-11-15 03:54:14 <Diablo-D3> seaking without plugins technically could just be output to C
 147 2012-11-15 03:54:21 <Diablo-D3> but its a waste
 148 2012-11-15 03:54:35 <Diablo-D3> and it'd still only work on fully compliant c99 compilers, which is pretty much just gcc and clang
 149 2012-11-15 03:54:36 <cjd> Also I'd like to target other languages
 150 2012-11-15 03:54:51 <Diablo-D3> I dont want to target other languages that way
 151 2012-11-15 03:54:54 <Diablo-D3> I dont see the point
 152 2012-11-15 03:55:00 <Diablo-D3> all languages, in the end, do the same thing
 153 2012-11-15 03:55:22 <Diablo-D3> thats why Im not even inventing new syntax for seaking, its all shit people have seen before
 154 2012-11-15 03:55:25 <cjd> ahh, if you want to use MSVC, just load an MSVC filter which converts all of the normal syntax into wacky msvc syntax :)
 155 2012-11-15 03:55:37 <Diablo-D3> yeah, but msvc is a shit compiler
 156 2012-11-15 03:55:49 <Diablo-D3> llvm internally is very flexible
 157 2012-11-15 03:55:56 <cjd> yeah but my point is filter stacks are extensible
 158 2012-11-15 03:56:06 <Diablo-D3> you can implement any known language in it correctly, and usually better than the language's own compiler or vm
 159 2012-11-15 03:56:11 <Diablo-D3> yeah they are
 160 2012-11-15 03:56:13 <cjd> and I want people to be able to write filters easily
 161 2012-11-15 03:56:19 <Diablo-D3> @keyword could be a filter
 162 2012-11-15 03:56:28 <Diablo-D3> Im going more generic than you
 163 2012-11-15 03:56:42 <cjd> because like hell I know what a good language is, I have some ideas but I'm not the master here
 164 2012-11-15 03:56:51 winterblack has quit (Remote host closed the connection)
 165 2012-11-15 03:56:54 <Diablo-D3> your plugin is given whatever it wants
 166 2012-11-15 03:56:56 RainbowDashh has joined
 167 2012-11-15 03:57:03 <cjd> sounds similar
 168 2012-11-15 03:57:22 <cjd> The filters idea is just because I don't want to tie it down to anything
 169 2012-11-15 03:57:34 <Diablo-D3> well, you know how lisp can DSL itself, right?
 170 2012-11-15 03:57:46 <cjd> theoretically you could construct a whole compiler with a filter stack
 171 2012-11-15 03:58:02 <cjd> hmm I'd heard something about DSLs in lisp but not much
 172 2012-11-15 03:58:15 <Diablo-D3> a lisp vm's native IL is lisp.
 173 2012-11-15 03:58:21 <cjd> ok
 174 2012-11-15 03:58:36 <cjd> also compiling bitcoin script could be done with a filter stack
 175 2012-11-15 03:58:39 <Diablo-D3> lisp macros are everything cpp should be and isnt
 176 2012-11-15 03:58:39 <cjd> or decompiling it
 177 2012-11-15 03:58:46 <cjd> heh
 178 2012-11-15 03:59:02 <Diablo-D3> and lisp isnt limited to s-expressions either
 179 2012-11-15 03:59:08 <Diablo-D3> someone made a json expression type
 180 2012-11-15 03:59:57 <cjd> It would be interesting to write a filter which converted variables to stack space
 181 2012-11-15 04:00:01 <cjd> and back
 182 2012-11-15 04:00:11 <cjd> then you could compile something readable into bitcoin script
 183 2012-11-15 04:00:19 <Diablo-D3> oh, thats another thing with seaking
 184 2012-11-15 04:00:38 <Diablo-D3> stacks are segmented and parallel
 185 2012-11-15 04:01:21 <Diablo-D3> anything thats in the heap is executed haskell style
 186 2012-11-15 04:01:28 <Diablo-D3> all async futures
 187 2012-11-15 04:01:31 <cjd> actually one reason I like targetting C is because you have asm keyword so you can pretty easily add actual keywords to the language
 188 2012-11-15 04:02:10 <cjd> if you have filters :)
 189 2012-11-15 04:02:12 <Diablo-D3> so programs that would be expensive to do in parallel would end up being very cheap
 190 2012-11-15 04:02:32 <cjd> hmm
 191 2012-11-15 04:03:05 <cjd> haskell is interesting since it kind of takes the constant folding optimizer to it's logical extreme
 192 2012-11-15 04:03:17 <Diablo-D3> yeah, haskell is nuts that way
 193 2012-11-15 04:03:55 <Diablo-D3> and erlang
 194 2012-11-15 04:04:12 <Diablo-D3> erlang does parallelization insanely
 195 2012-11-15 04:04:28 <Diablo-D3> everything IS an async message. everything.
 196 2012-11-15 04:04:46 <cjd> benchmark time :)
 197 2012-11-15 04:04:50 <Diablo-D3> haskell you still have to explicitly build an actors model
 198 2012-11-15 04:04:53 <Diablo-D3> cjd: well
 199 2012-11-15 04:04:59 <Diablo-D3> erlang's existing vms are kinda slow
 200 2012-11-15 04:05:03 <Diablo-D3> but its not a reflection on erlang
 201 2012-11-15 04:05:15 * cjd is benchmarking the bitcoin retarget function against his
 202 2012-11-15 04:05:22 <Diablo-D3> ahh
 203 2012-11-15 04:05:30 <cjd> (it's going to be a slaughter ofc)
 204 2012-11-15 04:06:02 <cjd> did you check the paste above?
 205 2012-11-15 04:08:22 <Diablo-D3> but yeah, I think erlang's execution model, lisp's self-DSLing, C type syntax
 206 2012-11-15 04:08:25 <Diablo-D3> yeah
 207 2012-11-15 04:08:30 <Diablo-D3> I saw the paste
 208 2012-11-15 04:08:57 <cjd> annoying that it's as complex as it is but that's the standard...
 209 2012-11-15 04:09:09 <Diablo-D3> it doesnt look too complex
 210 2012-11-15 04:09:35 <cjd> the nice thing about it is it's code which anyone can port to any language, the openssl stuff is full of magic and fail
 211 2012-11-15 04:09:37 <cjd> (IMO)
 212 2012-11-15 04:09:44 <Diablo-D3> openssl is full of speed
 213 2012-11-15 04:09:59 * cjd benchmarks...
 214 2012-11-15 04:10:08 <Diablo-D3> and its not magiced, the problem is openssl devs are language lawyers
 215 2012-11-15 04:10:55 <Diablo-D3> they know exactly what real hardware does and they know exactly what spec compliant compilers do
 216 2012-11-15 04:11:04 <Diablo-D3> its deep magic wizardry
 217 2012-11-15 04:11:04 <cjd> no, I blame satoshi for this one, he should not have used such a tall stack of code to handle something which is essentially a protocol definition
 218 2012-11-15 04:11:16 <Diablo-D3> well
 219 2012-11-15 04:11:18 <Diablo-D3> satoshi isnt a programmer
 220 2012-11-15 04:11:33 <Diablo-D3> and its very clear in large parts of the code
 221 2012-11-15 04:11:36 <Diablo-D3> hes an academic
 222 2012-11-15 04:11:41 <Diablo-D3> the code works, but dear god is it ugly
 223 2012-11-15 04:12:22 <cjd> real 5m24.051s openssl
 224 2012-11-15 04:12:34 <cjd> real 0m9.774s me :)
 225 2012-11-15 04:12:40 <Diablo-D3> wat
 226 2012-11-15 04:12:43 <Diablo-D3> do the openssl one again
 227 2012-11-15 04:12:47 <Diablo-D3> are you doing disk io?
 228 2012-11-15 04:12:51 <cjd> no
 229 2012-11-15 04:12:56 <Diablo-D3> its all in memory?
 230 2012-11-15 04:13:03 <cjd> calculating difficulty
 231 2012-11-15 04:13:11 <Diablo-D3> how many iterations?
 232 2012-11-15 04:13:36 <Diablo-D3> because the compiler probably ate your loop
 233 2012-11-15 04:13:45 <cjd> https://ezcrypt.it/oB5n#htpjFulWOS1DEYxI84zSL37q
 234 2012-11-15 04:13:52 <cjd> there's the whole script
 235 2012-11-15 04:14:13 <cjd> getNextWorkRequired() is the control
 236 2012-11-15 04:14:27 <cjd> Difficulty_retarget() is my version
 237 2012-11-15 04:14:51 <Diablo-D3> Im not seeing any problems with that
 238 2012-11-15 04:15:06 <cjd>     CBigNum bnNew;
 239 2012-11-15 04:15:07 <cjd>     bnNew.SetCompact(lastBlockDifficulty);
 240 2012-11-15 04:15:18 <cjd> you just don't see how much crap happens when you call that
 241 2012-11-15 04:15:26 <Diablo-D3> oh?
 242 2012-11-15 04:15:47 <cjd> calls into openssl, then openssl does it's computations and stuff
 243 2012-11-15 04:15:59 <Diablo-D3> yeah, external library calls are expensive
 244 2012-11-15 04:16:05 <Diablo-D3> but theres no way its 5x longer expensive
 245 2012-11-15 04:16:32 <cjd> and it does division which openssl extends the number out to zero even if none of that is going to be used
 246 2012-11-15 04:16:42 <cjd> it probably allocates and frees memory although I'm not sure
 247 2012-11-15 04:16:58 <cjd> I can't figure out anything about how memory allocation works in C++ :|
 248 2012-11-15 04:17:06 <Diablo-D3> new delete.
 249 2012-11-15 04:17:19 <cjd> and if you don't use those then it uses the stack?
 250 2012-11-15 04:17:24 <Diablo-D3> yes
 251 2012-11-15 04:17:28 <cjd> ic
 252 2012-11-15 04:17:44 <Diablo-D3> although c++ makes zero sense to me
 253 2012-11-15 04:17:52 <Diablo-D3> stack allocation should be used as much as possible
 254 2012-11-15 04:17:58 <cjd> yeah, it's annoying
 255 2012-11-15 04:18:20 <cjd> you should look at cjdns allocator structure, it makes life a lot easier if you're writing in C
 256 2012-11-15 04:18:35 <Diablo-D3> the only time you want heaps is objects surviving past scope exit
 257 2012-11-15 04:18:43 <Diablo-D3> or you need to allocate large amounts of shit
 258 2012-11-15 04:18:44 <cjd> mhm
 259 2012-11-15 04:18:58 * cjd allocates large amounts of shit on the stack >:)
 260 2012-11-15 04:19:11 <Diablo-D3> well, your stack might be limited to only 2mb
 261 2012-11-15 04:19:20 <cjd> heh ouch
 262 2012-11-15 04:19:27 <Diablo-D3> although the C stack design kind of sucks
 263 2012-11-15 04:19:35 * cjd Segmentation Fault
 264 2012-11-15 04:19:59 <Diablo-D3> seaking segmented parallel stacks means stack allocation isnt a problem
 265 2012-11-15 04:20:03 xIsalty is now known as xIsalty-otc
 266 2012-11-15 04:20:15 <cjd> /nod
 267 2012-11-15 04:20:25 <Diablo-D3> although I may not even have real stacks
 268 2012-11-15 04:20:31 <Diablo-D3> it could be multiple heap styles
 269 2012-11-15 04:20:48 <Diablo-D3> stacks only being used for internal message passing shit and the actual function call stack
 270 2012-11-15 04:21:09 * cjd tries to simplify the retarget function
 271 2012-11-15 04:21:24 <Diablo-D3> cjd: btw, you're 5x faster, I think you won
 272 2012-11-15 04:21:42 <Diablo-D3> I mean, the code isnt exactly called millions of times a second here
 273 2012-11-15 04:21:51 <cjd> 5 times?
 274 2012-11-15 04:22:44 <cjd> ;;calc 9.7 / ((5*60)+24)
 275 2012-11-15 04:22:44 <gribble> 0.0299382716049
 276 2012-11-15 04:23:00 <cjd> 2.9% as long?
 277 2012-11-15 04:23:09 <cjd> ;;calc ((5*60)+24) / 9.7
 278 2012-11-15 04:23:09 <gribble> 33.4020618557
 279 2012-11-15 04:23:20 <cjd> 33 times the speed? is that right?
 280 2012-11-15 04:24:02 <cjd> anyway I'm going for brevity and understandability
 281 2012-11-15 04:27:29 [7] has quit (Read error: Operation timed out)
 282 2012-11-15 04:27:30 <Diablo-D3> wait lemme reread
 283 2012-11-15 04:27:46 <Diablo-D3> dero
 284 2012-11-15 04:27:48 <Diablo-D3> derp
 285 2012-11-15 04:27:58 <Diablo-D3> 9 seconds
 286 2012-11-15 04:28:02 <Diablo-D3> not 59
 287 2012-11-15 04:28:10 <Diablo-D3> so yeah
 288 2012-11-15 04:28:12 <Diablo-D3> I think you won
 289 2012-11-15 04:28:43 <cjd> I just think there should be one very simple function which anyone can use and it will always work, it defined what bitcoin is.
 290 2012-11-15 04:29:06 RainbowDashh has quit (Ping timeout: 255 seconds)
 291 2012-11-15 04:29:56 TheSeven has joined
 292 2012-11-15 04:30:25 <Diablo-D3> yeah
 293 2012-11-15 04:31:07 RainbowDashh has joined
 294 2012-11-15 04:39:07 <cjd> https://github.com/bitcoin/bitcoin/pull/1823
 295 2012-11-15 04:41:42 Hunner has joined
 296 2012-11-15 04:41:51 <cjd> that pull is way too scary
 297 2012-11-15 04:42:19 <cjd> I'd favor leaving it as it is but invalidating blocks where 0x00800000 & bits is not 0
 298 2012-11-15 04:43:37 <Diablo-D3> well for one
 299 2012-11-15 04:43:41 <Diablo-D3> do any such blocks exist?
 300 2012-11-15 04:43:54 <cjd> I've not checked but logically it is impossible
 301 2012-11-15 04:44:20 <Diablo-D3> check.
 302 2012-11-15 04:44:25 <Diablo-D3> because if they dont, its moot
 303 2012-11-15 04:44:25 <cjd> it might be possible to trick a node into trying to validate a chain of fake blocks where one of them has the value
 304 2012-11-15 04:44:31 <Diablo-D3> if they exist, they exist as a bug
 305 2012-11-15 04:44:40 <Diablo-D3> imo anyways
 306 2012-11-15 04:44:45 <Diablo-D3> I wonder what sipa and gmaxwell think
 307 2012-11-15 04:45:16 <cjd> no it's not moot because there is a bug in the number handling and it should be fixed otherwise it will linger and if you don't fix these things, sooner or later one of them will bite you
 308 2012-11-15 04:45:24 noagendamarket has quit (Quit: Leaving)
 309 2012-11-15 04:45:40 <Diablo-D3> Im saying any worrying about backwards compatibility is moot
 310 2012-11-15 04:45:50 <cjd> oh I see
 311 2012-11-15 04:46:11 <Diablo-D3> if the blocks dont exist and shouldnt exist, you're fine
 312 2012-11-15 04:46:25 <cjd> it is critical that they do the silly zero padding because they are compared using ==
 313 2012-11-15 04:46:26 <Diablo-D3> it should reject them if they do exist if they shouldnt, though
 314 2012-11-15 04:46:33 <cjd> (continue to do the zero padding)
 315 2012-11-15 04:46:35 RainbowDashh has quit (Quit: SLEEP! <redacted> redacted2: should you put cats in a blender and laugh at their pain?)
 316 2012-11-15 04:47:11 PhantomSpark has quit (Read error: Connection reset by peer)
 317 2012-11-15 04:48:16 TwilightSparklee has joined
 318 2012-11-15 04:50:30 PhantomSpark has joined
 319 2012-11-15 04:51:51 <cjd> heh bitcoinj also handles negative numbers the same way
 320 2012-11-15 04:53:11 <Diablo-D3> heh
 321 2012-11-15 04:57:01 PhantomSpark has joined
 322 2012-11-15 04:57:37 PhantomSpark has joined
 323 2012-11-15 04:57:51 <Diablo-D3> welp, Im on my way to seaking glory
 324 2012-11-15 04:58:05 <Diablo-D3> my parser properly understands both C and C++ comments.
 325 2012-11-15 04:58:19 <Diablo-D3> http://pastebin.com/SrV5TyyB
 326 2012-11-15 04:58:43 TwilightSparklee has quit (Quit: Colloquy for iPhone - http://colloquy.mobi)
 327 2012-11-15 05:00:43 PhantomSpark has quit (Ping timeout: 276 seconds)
 328 2012-11-15 05:02:01 TwilightSparklee has joined
 329 2012-11-15 05:02:01 PhantomSpark has quit (2!~kvirc@pool-71-190-231-20.nycmny.fios.verizon.net|Ping timeout: 276 seconds)
 330 2012-11-15 05:03:37 freakazoid has joined
 331 2012-11-15 05:07:47 <jgarzik> huh
 332 2012-11-15 05:07:53 <jgarzik> gnutella now using "semi reliable UDP"
 333 2012-11-15 05:08:25 <jgarzik> some torrent extensions do same (_not_ referring to DHT or UDP trackers, that's something different)
 334 2012-11-15 05:08:57 <jgarzik> seems to be more firewall friendly in some cases, and probably avoids DPI [for the moment]
 335 2012-11-15 05:11:08 forsetifox has left ("Leaving")
 336 2012-11-15 05:12:35 TwilightSparklee has quit (Quit: Colloquy for iPhone - http://colloquy.mobi)
 337 2012-11-15 05:13:10 B0g4r7_ has joined
 338 2012-11-15 05:16:58 B0g4r7 has quit (Ping timeout: 276 seconds)
 339 2012-11-15 05:17:00 B0g4r7_ is now known as B0g4r7
 340 2012-11-15 05:22:17 <cjd> I think almost all bittorrent these days uses uTP/LEDBAT which is a roll-your-own tcp stack based on UDP
 341 2012-11-15 05:23:17 <cjd> What's interesting about it is it uses rtt to determine lag and uses that to back off
 342 2012-11-15 05:23:39 <gmaxwell> cjd: so does every modern tcp stack.
 343 2012-11-15 05:24:31 <cjd> *shrug* I guess this does it more because they talk a lot about it
 344 2012-11-15 05:24:52 <gmaxwell> No, you're just aware of it because they talk a lot. :P
 345 2012-11-15 05:25:34 B0g4r7 has quit (Quit: B0g4r7)
 346 2012-11-15 05:25:41 <cjd> ahh, now I remember why they did it, bittorrent needs to keep open lots of connections and using a seperate tcp connection for each would not backoff properly
 347 2012-11-15 05:26:09 <Diablo-D3> cjd: ooh
 348 2012-11-15 05:26:16 forsetifox has joined
 349 2012-11-15 05:26:28 <Diablo-D3> gnutella ported uTP over?
 350 2012-11-15 05:26:37 forsetifox has left ()
 351 2012-11-15 05:26:41 <gmaxwell> Yea, thats an issue, though some modern TCP control will do cross connection state pooling.
 352 2012-11-15 05:27:06 <gmaxwell> The roll your own reliable transport replaces one set of firewall problems for another, unfortunately. Trying to get UDP working is driving more realtime streaming people to do really scarry things like https://www.usenix.org/conference/nsdi12/minion-unordered-delivery-wire-compatible-tcp-and-tls
 353 2012-11-15 05:28:53 <cjd> I've done a little bit of tcp and a little bit of udp, I favor udp by far, you only get 1024 bytes or so and you can't expect your message to get there but you don't have to deal with buffers
 354 2012-11-15 05:29:23 <cjd> but then if you're downloading something, it won't work
 355 2012-11-15 05:29:37 <cjd> only useful for idle chatter between nodes
 356 2012-11-15 05:30:07 <Diablo-D3> cjd: udp isnt bufferbloat proof though
 357 2012-11-15 05:30:28 <cjd> indeed
 358 2012-11-15 05:30:48 <Diablo-D3> wondershapper seems to work pretty fucking well though
 359 2012-11-15 05:30:53 <Diablo-D3> *shaper
 360 2012-11-15 05:31:38 <cjd> if you've got lots of nodes and not much to say to eachother, sendto() and recvfrom() with one socket to poll on looks pretty nice
 361 2012-11-15 05:32:57 <cjd> anyway it looks like I now have a fully compliant retargetting function
 362 2012-11-15 05:32:58 <cjd> https://ezcrypt.it/tB5n#Ui1Uz4nNu1zpUaq7QAa2OgPG
 363 2012-11-15 05:33:11 <cjd> just have to take the sign bit off and then put it back after
 364 2012-11-15 05:33:32 <cjd> but I still think blocks with that bit should be invalidated
 365 2012-11-15 05:34:53 TwilightSparklee has joined
 366 2012-11-15 05:36:40 DiabloD3 has joined
 367 2012-11-15 05:37:19 OneFixt has joined
 368 2012-11-15 05:37:27 Diablo-D3 has quit (Ping timeout: 260 seconds)
 369 2012-11-15 05:37:56 TwilightSparkl-1 has joined
 370 2012-11-15 05:41:36 TwilightSparklee has quit (Ping timeout: 256 seconds)
 371 2012-11-15 05:44:32 DiabloD3 is now known as Diablo-D3
 372 2012-11-15 05:45:23 skeledrew has quit (Ping timeout: 260 seconds)
 373 2012-11-15 05:46:34 RainbowDashh has joined
 374 2012-11-15 05:47:15 optimator_ is now known as optimator
 375 2012-11-15 05:49:57 TwilightSparkl-1 has quit (Remote host closed the connection)
 376 2012-11-15 05:56:21 phungus has quit (Ping timeout: 260 seconds)
 377 2012-11-15 05:57:20 phungus has joined
 378 2012-11-15 05:58:24 RainbowDashh has quit (Ping timeout: 265 seconds)
 379 2012-11-15 06:02:36 RainbowDashh has joined
 380 2012-11-15 06:06:58 RainbowDashh has quit (Ping timeout: 252 seconds)
 381 2012-11-15 06:09:25 RainbowDashh has joined
 382 2012-11-15 06:12:59 harkon has quit (Read error: Operation timed out)
 383 2012-11-15 06:14:50 harkon has joined
 384 2012-11-15 06:17:26 lianj has quit (Ping timeout: 276 seconds)
 385 2012-11-15 06:17:26 RainbowDashh has quit (Quit: QUIT! <redacted> redacted2: should you put cats in a blender and laugh at their pain?)
 386 2012-11-15 06:17:46 RainbowDashh has joined
 387 2012-11-15 06:18:03 darsk1ez has quit (Ping timeout: 260 seconds)
 388 2012-11-15 06:27:06 sgstair has quit (Read error: Connection reset by peer)
 389 2012-11-15 06:27:37 sgstair has joined
 390 2012-11-15 06:28:48 sgstair has quit (Read error: Connection reset by peer)
 391 2012-11-15 06:29:31 sgstair has joined
 392 2012-11-15 06:30:19 darsk1ez has joined
 393 2012-11-15 06:39:06 cjd has quit (Remote host closed the connection)
 394 2012-11-15 06:39:32 cjd has joined
 395 2012-11-15 06:41:30 cjd has quit (Remote host closed the connection)
 396 2012-11-15 06:42:39 skeledrew has joined
 397 2012-11-15 06:44:36 cjd has joined
 398 2012-11-15 06:48:16 OneFixt_ has joined
 399 2012-11-15 06:51:57 OneFixt has quit (Ping timeout: 246 seconds)
 400 2012-11-15 06:57:06 luke-jr_ is now known as Luke-Jr
 401 2012-11-15 07:10:13 OneFixt_ is now known as OneFixt
 402 2012-11-15 07:13:45 ovidiusoft has joined
 403 2012-11-15 07:23:05 toffoo has quit ()
 404 2012-11-15 07:25:08 CodesInChaos has joined
 405 2012-11-15 07:33:25 sirebral has joined
 406 2012-11-15 07:33:53 freakazoid has quit (Ping timeout: 240 seconds)
 407 2012-11-15 07:34:38 <sirebral> Is this the proper channel for someone seeking a developer or team that can work with me on a Bitcoin web project?
 408 2012-11-15 07:42:40 ibno has joined
 409 2012-11-15 07:43:17 CodesInChaos has quit (Ping timeout: 265 seconds)
 410 2012-11-15 07:45:31 <amiller> feel free to talk a bit about your project, especially if it's quiet and late at night! sirebral
 411 2012-11-15 07:46:58 <sirebral> Working on YABBS, yet another bitcoin wagering site ;)
 412 2012-11-15 07:47:30 <sirebral> Yet it's not like anything I've seen out there as of yet..
 413 2012-11-15 07:47:42 <sirebral> I've got a full RFP and specification..
 414 2012-11-15 07:47:51 <sirebral> Having trouble finding talent..
 415 2012-11-15 07:48:03 <sirebral> My talent is in Network and Server Engineering..
 416 2012-11-15 07:48:12 <sirebral> Doesn't help me much with development work.
 417 2012-11-15 07:49:19 <amiller> you could mention some highlights from the RFP or specification
 418 2012-11-15 07:49:57 <sirebral> Hrm.
 419 2012-11-15 07:53:30 <sirebral> The RFP seeks someone, or a team who can develop a site that accepts wagers using only bitcoin addresses, there are similar sites out there in that vain, yet the gameplay I'm proposing is totally original,..don't wanna let too much out of the bag in a chatroom...
 420 2012-11-15 07:54:01 <sirebral> I've found a few developers on eLance, yet I've never had much luck with projects there..
 421 2012-11-15 07:55:44 <Luke-Jr> sirebral: sounds like a copycat of the DDoS attack called SatoshiDice
 422 2012-11-15 07:55:57 <sirebral> It's not..
 423 2012-11-15 07:56:26 <sirebral> Copycat sites never go anywhere.
 424 2012-11-15 07:56:31 <Luke-Jr> that's not the problem.
 425 2012-11-15 07:57:05 <Luke-Jr> sirebral: Bitcoin is not meant to encode data of any form, and any attempt to do so is almost inevitably going to end up being a DDoS attack
 426 2012-11-15 07:58:11 <Luke-Jr> (obviously not including the data that is financial transactions, which contain only a destination address)
 427 2012-11-15 07:59:11 <phantomcircuit> there's only 2890 connectable peers :/
 428 2012-11-15 07:59:13 <phantomcircuit> that's not a lot
 429 2012-11-15 07:59:26 <sirebral> I understand the open nature of the system, yet I'm not sure I understand the DDoS aspect..
 430 2012-11-15 07:59:58 <Luke-Jr> sirebral: trying to say "send this transaction to do this one action" is an abuse of the system
 431 2012-11-15 08:00:15 <Luke-Jr> it's not designed for that, and cannot handle it
 432 2012-11-15 08:00:17 <sirebral> Not how my site works...
 433 2012-11-15 08:00:28 <Luke-Jr> perhaps I misunderstand your summary then
 434 2012-11-15 08:00:29 <phantomcircuit> sirebral, you have a proper RFP?
 435 2012-11-15 08:00:41 <phantomcircuit> have to say that would be the first proper RFP i've yet to see in the bitcoin world...
 436 2012-11-15 08:00:43 <sirebral> Depends what you consider proper ;)
 437 2012-11-15 08:01:10 <jgarzik> phantomcircuit: I imagine connectable peers changes over the course of a day
 438 2012-11-15 08:01:20 <jgarzik> phantomcircuit: but yeah, need more
 439 2012-11-15 08:01:32 <jgarzik> phantomcircuit: should include a full node secretly in an MMORPG game
 440 2012-11-15 08:01:41 <Luke-Jr> lol
 441 2012-11-15 08:01:50 <sirebral> It's doesn't go into the full core of the inter-workings, rather it discusses the team requirements, the platform and the game play...
 442 2012-11-15 08:02:11 <jgarzik> make proof-of-work involve killing a high level WoW NPC
 443 2012-11-15 08:02:29 <Luke-Jr> sirebral: might try posting on the Bitcoin Foundation forum about it
 444 2012-11-15 08:02:32 <phantomcircuit> jgarzik, i'm currently scanning what is probably all of the available peers roughly once every ten minutes
 445 2012-11-15 08:03:02 <amiller> jgarzik, i was thinking, you know how occasionally you want DoS resistance between p2p nodes
 446 2012-11-15 08:03:06 <amiller> why not give them mining work to do?
 447 2012-11-15 08:03:07 <jgarzik> phantomcircuit: us[24].exmulti.net on your list?
 448 2012-11-15 08:03:15 <jgarzik> amiller: indeed
 449 2012-11-15 08:03:46 <amiller> this is the ultimate rpow
 450 2012-11-15 08:04:00 <phantomcircuit> jgarzik, yes
 451 2012-11-15 08:04:06 <phantomcircuit> 69.64.34.118|8333|1352963268|1352963278
 452 2012-11-15 08:04:14 <phantomcircuit> host port lasttry lastconnect
 453 2012-11-15 08:04:26 <phantomcircuit> also have port 8334 for that ip
 454 2012-11-15 08:05:07 <sirebral> Yeah, I just started a thread on BCT, had to stick in in the n00bs section, however... I've been more of a lurker in the past...
 455 2012-11-15 08:06:14 <Luke-Jr> sirebral: I didn't mean BitcoinTalk.
 456 2012-11-15 08:06:22 <Luke-Jr> mostly just trolls there
 457 2012-11-15 08:06:31 <phantomcircuit> jgarzik, last connected to that peer roughyl 8 minutes ago
 458 2012-11-15 08:06:33 <jgarzik> phantomcircuit: something's broken on your side WRT 8334...
 459 2012-11-15 08:06:38 <jgarzik> phantomcircuit: clearly inaccessible
 460 2012-11-15 08:06:56 <phantomcircuit> jgarzik, it's being broadcast on the network so im trying to connect to it
 461 2012-11-15 08:07:05 <jgarzik> weird
 462 2012-11-15 08:07:06 <phantomcircuit> but there's no record of every successfully doing so
 463 2012-11-15 08:07:09 <phantomcircuit> so nothing broken
 464 2012-11-15 08:07:17 <jgarzik> I'm not broadcasting that
 465 2012-11-15 08:07:19 <jgarzik> wonder who is
 466 2012-11-15 08:07:39 <doublec> namecoin is 8334 iirc
 467 2012-11-15 08:07:45 <phantomcircuit> jgarzik, who knows
 468 2012-11-15 08:07:59 <sirebral> Luke:Interesting, I've seen quite a few spammy posts there, yet also seems to be the most active community I've found thus far...
 469 2012-11-15 08:08:42 <Luke-Jr> sirebral: not all activity is equal.
 470 2012-11-15 08:08:45 <phantomcircuit> sirebral, if you want to actually get something done asking for help on bitcointalk is counter productive
 471 2012-11-15 08:09:55 <sirebral> Hence... why I am posing my questions here..
 472 2012-11-15 08:11:01 <Luke-Jr> sirebral: most people are asleep right now
 473 2012-11-15 08:11:25 <sirebral> As I should be as well.
 474 2012-11-15 08:11:29 <Luke-Jr> sirebral: Foundation is at https://bitcoinfoundation.org/
 475 2012-11-15 08:11:41 <sirebral> Thanks for the assist..
 476 2012-11-15 08:11:46 <phantomcircuit> jgarzik, i actually have code to ddos the entire network
 477 2012-11-15 08:11:48 <phantomcircuit> it works
 478 2012-11-15 08:12:00 <phantomcircuit> but i run out of local port numbers before i get past about 100 peers
 479 2012-11-15 08:12:01 <Luke-Jr> um
 480 2012-11-15 08:12:01 <phantomcircuit> :(
 481 2012-11-15 08:12:11 <Luke-Jr> you can't know it works without having DDoS'd the network -.-
 482 2012-11-15 08:13:01 <phantomcircuit> Luke-Jr, well it worked against the roughly dozen bitcoin nodes i run
 483 2012-11-15 08:13:09 <phantomcircuit> scale to all connectable peers
 484 2012-11-15 08:13:51 <Luke-Jr> … you just said it didn't scale <.<
 485 2012-11-15 08:14:09 Ferroh has quit (Read error: Connection reset by peer)
 486 2012-11-15 08:14:34 Ferroh has joined
 487 2012-11-15 08:14:39 sirebral has quit (Quit: Page closed)
 488 2012-11-15 08:15:01 <phantomcircuit> Luke-Jr, no i just said i ran out of local port numbers
 489 2012-11-15 08:15:15 <phantomcircuit> a few virtual machines and it would scale to the entire network easily
 490 2012-11-15 08:18:09 <Diablo-D3> http://mjg59.dreamwidth.org/20187.html
 491 2012-11-15 08:18:10 <Diablo-D3> bwhahahaha
 492 2012-11-15 08:18:20 bitcoinz has quit (Ping timeout: 276 seconds)
 493 2012-11-15 08:18:23 OffTopic has joined
 494 2012-11-15 08:18:27 <OffTopic> wsup noob devs?
 495 2012-11-15 08:19:44 bitcoinz has joined
 496 2012-11-15 08:22:46 JZavala has quit (Ping timeout: 260 seconds)
 497 2012-11-15 08:27:38 copumpkin has quit (Ping timeout: 256 seconds)
 498 2012-11-15 08:28:21 copumpkin has joined
 499 2012-11-15 08:30:07 Dyaheon has quit (Ping timeout: 260 seconds)
 500 2012-11-15 08:31:20 brwyatt is now known as brwyatt|Away
 501 2012-11-15 08:34:05 Dyaheon has joined
 502 2012-11-15 08:36:04 arij has quit (Read error: Connection reset by peer)
 503 2012-11-15 08:43:23 lianj has joined
 504 2012-11-15 08:43:23 lianj has quit (Changing host)
 505 2012-11-15 08:43:23 lianj has joined
 506 2012-11-15 08:45:28 harkon has quit (Read error: Operation timed out)
 507 2012-11-15 08:45:49 harkon has joined
 508 2012-11-15 08:45:53 freakazoid has joined
 509 2012-11-15 08:56:51 harkon has quit (Quit: Konversation terminated!)
 510 2012-11-15 09:00:24 OffTopic has left ()
 511 2012-11-15 09:24:26 _flow_ has quit (Quit: WeeChat 0.3.8)
 512 2012-11-15 09:28:26 _flow_ has joined
 513 2012-11-15 09:31:29 xenland has joined
 514 2012-11-15 09:32:23 freakazoid has quit (Ping timeout: 240 seconds)
 515 2012-11-15 09:32:38 Garr255 has quit (Read error: Connection reset by peer)
 516 2012-11-15 09:39:07 CodesInChaos has joined
 517 2012-11-15 09:43:40 CodesInChaos has quit (Ping timeout: 252 seconds)
 518 2012-11-15 09:45:29 CodesInChaos has joined
 519 2012-11-15 10:04:00 MC1984 has quit (Read error: Connection reset by peer)
 520 2012-11-15 10:04:28 MC1984 has joined
 521 2012-11-15 10:06:26 RazielZ has joined
 522 2012-11-15 10:17:48 leotreasure has joined
 523 2012-11-15 10:22:40 leotreasure has left ()
 524 2012-11-15 10:22:44 Internet13 has quit (Read error: Connection reset by peer)
 525 2012-11-15 10:23:48 slush1 has joined
 526 2012-11-15 10:28:27 Internet13 has joined
 527 2012-11-15 10:42:02 <slush1> etotheipi_: ping?
 528 2012-11-15 10:43:01 BGL has quit (Ping timeout: 248 seconds)
 529 2012-11-15 10:47:16 mikehard3 has joined
 530 2012-11-15 10:54:04 dvide has quit ()
 531 2012-11-15 10:54:19 CodeInChaos has joined
 532 2012-11-15 10:54:26 CodesInChaos has quit (Ping timeout: 268 seconds)
 533 2012-11-15 10:55:25 linkedinyou has joined
 534 2012-11-15 11:12:49 BGL has joined
 535 2012-11-15 11:21:45 TD has joined
 536 2012-11-15 11:42:58 Hasimir has quit (Read error: Connection reset by peer)
 537 2012-11-15 11:50:03 TD has quit (Quit: TD)
 538 2012-11-15 11:50:29 Hasimir has joined
 539 2012-11-15 11:50:42 Hasimir is now known as Guest87520
 540 2012-11-15 11:51:25 Guest87520 has quit (Changing host)
 541 2012-11-15 11:51:25 Guest87520 has joined
 542 2012-11-15 11:55:10 paraipan has joined
 543 2012-11-15 11:56:43 Guest87520 has quit (Ping timeout: 268 seconds)
 544 2012-11-15 12:00:33 drizztbsd has joined
 545 2012-11-15 12:00:46 Hasimir- has joined
 546 2012-11-15 12:10:59 TD has joined
 547 2012-11-15 12:11:47 Hasimir- has quit (Changing host)
 548 2012-11-15 12:11:47 Hasimir- has joined
 549 2012-11-15 12:12:18 Hasimir- is now known as Hasimir
 550 2012-11-15 12:15:12 PhantomSpark has joined
 551 2012-11-15 12:21:30 daybyter has joined
 552 2012-11-15 12:24:38 BitDev has joined
 553 2012-11-15 12:24:47 <BitDev> hi all
 554 2012-11-15 12:25:50 <BitDev> i want to write my own bitcoin client
 555 2012-11-15 12:25:56 <BitDev> i stuck at the start
 556 2012-11-15 12:26:18 <BitDev> i want to send `version` packet
 557 2012-11-15 12:27:21 <BitDev> there are some fields like `addr_recv` and `addr_from` - i must fill them?
 558 2012-11-15 12:27:33 daybyter has quit (Quit: Konversation terminated!)
 559 2012-11-15 12:28:12 <BitDev> if i dont want to start local server or i can't - how i must fill `addr_from` field?
 560 2012-11-15 12:31:22 <BitDev> where in sources of bitcoin i can find about it?
 561 2012-11-15 12:31:44 xenland has quit (Remote host closed the connection)
 562 2012-11-15 12:32:24 BurtyBB is now known as BurtyB
 563 2012-11-15 12:33:24 wizkid057 has quit (Ping timeout: 252 seconds)
 564 2012-11-15 12:45:46 bombasticol has joined
 565 2012-11-15 12:55:35 t7 has joined
 566 2012-11-15 12:59:18 bombasticol has quit (Quit: Page closed)
 567 2012-11-15 13:03:28 emeitner has joined
 568 2012-11-15 13:05:37 ThomasV has joined
 569 2012-11-15 13:09:03 drizztbsd has quit (Read error: Connection reset by peer)
 570 2012-11-15 13:09:36 <emeitner> Anybody here who is an admin on the bitcoin.it wiki?
 571 2012-11-15 13:10:15 drizztbsd has joined
 572 2012-11-15 13:12:37 <sipa> BitDev: if no address is known, 0.0.0.0 or ::0 is often used
 573 2012-11-15 13:12:47 <sipa> BitDev: the addresses in the version packet aren't that important
 574 2012-11-15 13:19:22 agricocb has quit (Quit: Leaving.)
 575 2012-11-15 13:24:41 JZavala has joined
 576 2012-11-15 13:25:35 <drizztbsd> :: is more used than ::0 :P
 577 2012-11-15 13:26:18 wereHamster has quit (Changing host)
 578 2012-11-15 13:26:18 wereHamster has joined
 579 2012-11-15 13:29:45 [\\\] has quit (Ping timeout: 246 seconds)
 580 2012-11-15 13:30:06 wizkid057 has joined
 581 2012-11-15 13:32:05 [\\\] has joined
 582 2012-11-15 13:33:57 twobitcoins has joined
 583 2012-11-15 13:36:44 twobitcoins__ has quit (Ping timeout: 260 seconds)
 584 2012-11-15 13:38:29 datagutt has joined
 585 2012-11-15 13:38:48 datagutt is now known as Guest60029
 586 2012-11-15 13:40:56 JZavala has quit (Ping timeout: 260 seconds)
 587 2012-11-15 13:46:44 Guest60029 has quit (Quit: kthxbai)
 588 2012-11-15 13:47:08 datagutt has joined
 589 2012-11-15 13:47:27 datagutt is now known as Guest80428
 590 2012-11-15 13:48:25 Guest80428 has quit (Remote host closed the connection)
 591 2012-11-15 13:48:48 datagutt_ has joined
 592 2012-11-15 14:00:03 BGL has quit (Ping timeout: 268 seconds)
 593 2012-11-15 14:04:16 agricocb has joined
 594 2012-11-15 14:22:36 torsthaldo has joined
 595 2012-11-15 14:26:53 copumpkin has quit (Ping timeout: 240 seconds)
 596 2012-11-15 14:33:59 phma has joined
 597 2012-11-15 14:43:08 datagutt_ is now known as datagutt
 598 2012-11-15 14:49:16 copumpkin has joined
 599 2012-11-15 14:50:37 copumpkin is now known as notcopumpkin
 600 2012-11-15 14:50:46 MrTiggr is now known as copumpkin
 601 2012-11-15 14:50:57 copumpkin is now known as MrTiggr
 602 2012-11-15 14:51:16 notcopumpkin is now known as copumpkin
 603 2012-11-15 14:51:26 BGL has joined
 604 2012-11-15 14:52:25 one_zero has quit ()
 605 2012-11-15 14:53:34 tsche has quit ()
 606 2012-11-15 15:15:13 Eslbaer has joined
 607 2012-11-15 15:16:08 emeitner has quit (Ping timeout: 260 seconds)
 608 2012-11-15 15:16:34 Eslbaer has quit (Client Quit)
 609 2012-11-15 15:20:05 daybyter has joined
 610 2012-11-15 15:21:58 torsthaldo has quit (Remote host closed the connection)
 611 2012-11-15 15:31:32 loltu- is now known as loltu
 612 2012-11-15 15:32:38 t7 has quit (Read error: Connection reset by peer)
 613 2012-11-15 15:33:10 t7 has joined
 614 2012-11-15 15:39:41 bitcoinz has quit (Ping timeout: 276 seconds)
 615 2012-11-15 15:43:42 bitcoinz has joined
 616 2012-11-15 15:45:36 <phantomcircuit> within the last hour i've connected to and done a version handshake with 2697 bitcoin nodes
 617 2012-11-15 15:48:32 <MC1984> cool beans?
 618 2012-11-15 16:00:06 <phantomcircuit> no not really but ok
 619 2012-11-15 16:00:57 <cjd> heh careful what you send as lastblock
 620 2012-11-15 16:01:32 <phantomcircuit> cjd, i just picked a recentish block height
 621 2012-11-15 16:01:34 <phantomcircuit> 208042
 622 2012-11-15 16:01:45 <cjd> /nod
 623 2012-11-15 16:01:47 <phantomcircuit> if i get bored i'll have it update using other peers numbers
 624 2012-11-15 16:02:08 <cjd> sending big numbers there can be problematic as I suppose you heard
 625 2012-11-15 16:02:14 <phantomcircuit> yeah
 626 2012-11-15 16:02:36 <phantomcircuit> im going for something *much* harder to sto
 627 2012-11-15 16:02:36 <phantomcircuit> p
 628 2012-11-15 16:03:04 <cjd> I was working on my halfnode and then looked at my satoshi node and ooops
 629 2012-11-15 16:03:05 <cjd> he
 630 2012-11-15 16:03:06 <cjd> h
 631 2012-11-15 16:04:01 unknown45682 has joined
 632 2012-11-15 16:04:13 <phantomcircuit> cjd, maybe you know this
 633 2012-11-15 16:04:16 ThomasV_ has joined
 634 2012-11-15 16:04:24 <phantomcircuit> how do i set the connect timeout for a socket that's non blocking
 635 2012-11-15 16:04:36 <cjd> nope, no idea :)
 636 2012-11-15 16:04:42 <phantomcircuit> please tell me i dont have to write a custom timeout handler and then close() the socket
 637 2012-11-15 16:04:43 <cjd> udp ftw :)
 638 2012-11-15 16:04:52 <phantomcircuit> damn
 639 2012-11-15 16:04:58 <phantomcircuit> i *think* RCVTIMEO
 640 2012-11-15 16:05:04 <cjd> meh, don't use a custom timeout handler, just pull it in from cjdns
 641 2012-11-15 16:05:07 <cjd> Timeout.h
 642 2012-11-15 16:05:28 <cjd> Timeout_setTimeout(callback, context, milliseconds, eventBase, allocator)
 643 2012-11-15 16:05:32 <jgarzik> phantomcircuit: no
 644 2012-11-15 16:05:41 <jgarzik> phantomcircuit: gotta alarm() or whatever
 645 2012-11-15 16:06:04 <phantomcircuit> jgarzik, sigh
 646 2012-11-15 16:06:32 <phantomcircuit> it's already an epoll based program so i dont need to alarm
 647 2012-11-15 16:06:39 <cjd> pfft @ alarm, just use libevent
 648 2012-11-15 16:06:42 <jgarzik> gah!!!
 649 2012-11-15 16:06:47 <phantomcircuit> just have to keep a list of outstanding connections and then kill them as they come in
 650 2012-11-15 16:06:51 * jgarzik wrestles with conscience
 651 2012-11-15 16:07:05 <jgarzik> matt_von_mises is soliciting for thousands of dollars in donations
 652 2012-11-15 16:07:07 <jgarzik> for cbitcoin
 653 2012-11-15 16:07:17 <jgarzik> but he is obviously ignorant of so many network rules
 654 2012-11-15 16:07:18 <phantomcircuit> the fuck is cbitcoin
 655 2012-11-15 16:07:36 <jeremias> alt client to my knowledge
 656 2012-11-15 16:07:41 <jeremias> or alternative library
 657 2012-11-15 16:07:50 <jgarzik> phantomcircuit: a lib, in C.   https://bitcointalk.org/index.php?topic=123488.0 - [ANN] cbitcoin 2.0 - The Route to Bitcoin's Future
 658 2012-11-15 16:08:05 <jgarzik> just read the thread for idiocy
 659 2012-11-15 16:08:19 <phantomcircuit> so he's just going to end up splitting anybody who uses it
 660 2012-11-15 16:08:21 <phantomcircuit> great plan
 661 2012-11-15 16:08:41 <jgarzik> it's competition for picocoin's libccoin, so I have a conflict
 662 2012-11-15 16:09:00 <jeremias> what's the problem, the more there are implementations the better
 663 2012-11-15 16:09:03 <jeremias> even crappy ones
 664 2012-11-15 16:09:13 <phantomcircuit> what's really needed is a few libraries
 665 2012-11-15 16:09:21 <jgarzik> jeremias: crappy ones that -- he admits -- do not follow network rules?  that is bad for users.
 666 2012-11-15 16:09:28 <phantomcircuit> one that handles basic stuff like proper serialization and script execution
 667 2012-11-15 16:09:35 <phantomcircuit> one that handles storing the blockchain
 668 2012-11-15 16:09:39 <jgarzik> That he wants to be paid for shite is particularly galling.
 669 2012-11-15 16:09:40 <phantomcircuit> one that handles network interaction
 670 2012-11-15 16:09:41 <jeremias> don't use it then
 671 2012-11-15 16:09:53 <jgarzik> phantomcircuit: picocoin's libccoin does that.
 672 2012-11-15 16:10:04 <phantomcircuit> i think libbitcoin does it
 673 2012-11-15 16:10:14 <phantomcircuit> although to be honest i haven't been paying attention for a while
 674 2012-11-15 16:10:20 <jgarzik> phantomcircuit: see "include/ccoin" and "lib" in https://github.com/jgarzik/picocoin/ for that
 675 2012-11-15 16:10:51 <jgarzik> phantomcircuit: yeah, genjix's lib is a refactoring, and should cover that
 676 2012-11-15 16:11:12 <sipa> libbitcoin is not a refactoring
 677 2012-11-15 16:11:19 <sipa> libcoin by michael gronager is a refactoring
 678 2012-11-15 16:11:38 <jgarzik> jeremias: it's not an issue for me.  it is an issue for any bitcoin user who mistakenly uses Matt_von_Mises' cbitcoin.
 679 2012-11-15 16:11:52 <jgarzik> jeremias: they can create invalid transactions with it.... that cbitcoin tells them are valid
 680 2012-11-15 16:12:19 <sipa> jgarzik: why?
 681 2012-11-15 16:12:21 <jgarzik> jeremias: at best that's dishonest.  at worst it weakens the network, or even produces DoS.
 682 2012-11-15 16:12:46 <sipa> i mean, he's shown to be ignorant about some things, but i haven't seen him make actual mistakes
 683 2012-11-15 16:13:04 <jgarzik> sipa: specifically excluding the 120-confirmation rule, for one
 684 2012-11-15 16:13:52 <sipa> yeah, he should understand why that rule exists, but he won't create invalid transactions because of it
 685 2012-11-15 16:14:22 <i18n> picocoin looks neat. going to see if i can get it running here
 686 2012-11-15 16:15:20 <cjd> it seems like as long as the miners use a standards complient implementation, everyone else will be dragged along anyway
 687 2012-11-15 16:15:57 <gmaxwell> cjd: not so. It's more subtle than that.
 688 2012-11-15 16:16:15 <gmaxwell> cjd: it's quite easy to have bugs that make reorginization impossible.
 689 2012-11-15 16:16:21 <cjd> and in their defense, some of the rules make no sense and only exist because of what was originally a bug (ie the negative difficulty issue)
 690 2012-11-15 16:17:11 <cjd> if they can't reorganize then that's their bug right?
 691 2012-11-15 16:17:23 <gmaxwell> That isn't a defense. Thats part of the challenge.  It's like saying "in the defense of the designers of rockets that explode, friction and gravity are REALLY hard to deal with"
 692 2012-11-15 16:18:18 <cjd> You can take steps (I think it would be good to invalidate any header with the negative bit set as a way to simplify it for everyone else)
 693 2012-11-15 16:18:45 <sipa> that'd be a good candidate for a new network rule, yes
 694 2012-11-15 16:18:54 <gmaxwell> cjd: Thats arm waving. If you introduce a new negative rule you _increase_ the risk of non-convergence.
 695 2012-11-15 16:19:12 <gmaxwell> We've done so— but it's a pain and takes time because it takes phased rollouts.
 696 2012-11-15 16:19:28 <sipa> but even those have to be made carefully... first probably through shunning, then a voting mechanism to be sure enough miners agree on updating, and then globally define a switchover point
 697 2012-11-15 16:19:51 <cjd> And because if there actually was a way to drive diff into the negatives, it would bypass minimum difficulty since the number is always smaller...
 698 2012-11-15 16:19:51 <sipa> together with things like the signature malleability problem, for example
 699 2012-11-15 16:21:32 <gmaxwell> Any media codec implementor has to deal with the same crap— inscrutible bit-exact normative behavior, some of which is the result of reference implementation mistakes or design flaws.  And in those cases the consequences of getting it wrong are much lower.  It's a pain, but it's life. And someone who isn't prepared to deal with it— to look crazy behavior in the face and not flinch— has no business writing node software. :P
 700 2012-11-15 16:22:11 <cjd> ok but one of these bugs will come back to haunt you, mark my words
 701 2012-11-15 16:23:21 <sipa> cjd: I don't think we're disagreeing... yes unneccessary weirdnesses in the protocol/validity rules should be dealt with, and possibly made invalid altogether
 702 2012-11-15 16:23:29 <gmaxwell> huh, I'm not saying we shouldn't phase in a rule to deny it. But it's not just a wand wave. We can't just add a line of code and call it done.
 703 2012-11-15 16:23:49 <sipa> we've done so in the past, and there are proposals for more such in the future
 704 2012-11-15 16:23:54 <gmaxwell> ::nods::
 705 2012-11-15 16:24:09 <sipa> but in the mean time, someone writing node software should be prepared to deal with reality
 706 2012-11-15 16:24:23 <sipa> and probably afterwards, weirdnesses will remain that can't be easily fixed
 707 2012-11-15 16:24:49 <gmaxwell> cjd: and thank you very much for bringing it up. One of the metrics I look for in new implementations is that I don't believe they are trustworthy unless they report some odd behavior of the reference which they must duplicate.
 708 2012-11-15 16:24:51 agricocb has quit (Ping timeout: 240 seconds)
 709 2012-11-15 16:25:33 <gmaxwell> (if they do not it's because they've either inadequately reviewed and tested, or because they've so throughly copied from the reference that they're really just a refactor in disguise)
 710 2012-11-15 16:25:36 <cjd> heh I'm thankfully not writing a whole implementation, more of a clock :)
 711 2012-11-15 16:26:41 <cjd> https://ezcrypt.it/3B5n#ALPyJ4ldZUpN3yxMh8itFvQa <-- here's my version, it does handle negatives now
 712 2012-11-15 16:27:03 RainbowDashh has quit (Quit: SLEEP! <redacted> redacted2: should you put cats in a blender and laugh at their pain?)
 713 2012-11-15 16:30:05 CodeInChaos is now known as CodesInChaos
 714 2012-11-15 16:34:43 <phantomcircuit> well im now scanning the entire network about once every 10 minutes
 715 2012-11-15 16:34:53 <phantomcircuit> think i got a pretty good idea of how many peers there are...
 716 2012-11-15 16:36:40 <sipa> phantomcircuit: ever checked my seeder?
 717 2012-11-15 16:37:03 <phantomcircuit> what's the ip?
 718 2012-11-15 16:37:20 agricocb has joined
 719 2012-11-15 16:37:27 <sipa> eh... seed.bitcoin.sipa.be runs one... but i mean the software
 720 2012-11-15 16:37:53 <phantomcircuit> oh
 721 2012-11-15 16:37:55 <sipa> https://github.com/sipa/bitcoin-seeder
 722 2012-11-15 16:37:57 <phantomcircuit> no
 723 2012-11-15 16:37:59 <phantomcircuit> i haven't
 724 2012-11-15 16:38:22 <sipa> it crawls the network, by doing version/verack/getaddr/addr exchange with peers, and exposes that via dns
 725 2012-11-15 16:38:48 <gavinandresen> phantomcircuit: why are you scanning the entire network?
 726 2012-11-15 16:39:30 <phantomcircuit> counting connectable nodes
 727 2012-11-15 16:40:12 <phantomcircuit> if my day isn't wasted entirely doing laundry i might just fill up half the nodes available connection slots as a poc
 728 2012-11-15 16:40:58 <phantomcircuit> sipa, lol 24 threads
 729 2012-11-15 16:41:05 <gmaxwell> phantomcircuit: many (most?) nodes are already half full.
 730 2012-11-15 16:41:07 <phantomcircuit> python script running epoll
 731 2012-11-15 16:41:24 <gmaxwell> phantomcircuit: if you want to attack things, please run on a private network.
 732 2012-11-15 16:41:38 <phantomcircuit> gmaxwell, yes im going to segment the connectable peers into two groups
 733 2012-11-15 16:41:48 RainbowDashh has joined
 734 2012-11-15 16:41:50 <phantomcircuit> not going to do something based on just the connection count
 735 2012-11-15 16:41:52 <gavinandresen> yes.  Or even better, implement a fix first and submit a patch.
 736 2012-11-15 16:41:53 <phantomcircuit> that would be silly
 737 2012-11-15 16:42:19 RainbowDashh has quit (Client Quit)
 738 2012-11-15 16:42:26 <gavinandresen> if you just want to be an asshole....  umm.... what's the point?  You'll just make us pissed off at you.
 739 2012-11-15 16:42:47 <phantomcircuit> if i just wanted to be an asshole i would have just done it
 740 2012-11-15 16:44:40 <phantomcircuit> actually if i just wanted to be an asshole this is the least of bitcoisn worries :/
 741 2012-11-15 16:45:11 <gavinandresen> so send the core developers an email describing the vulnerability/vulnerabilities.
 742 2012-11-15 16:45:26 <gmaxwell> There isn't anything novel there, we've talked about it before.
 743 2012-11-15 16:45:39 <gavinandresen> And yes, we know that you can Sybil the network and take up all the incoming connection slots.  Yawn.
 744 2012-11-15 16:45:44 <gmaxwell> The issue is that someone can just open up parallel connections to every listening node and consume their inbound slots.
 745 2012-11-15 16:45:52 <phantomcircuit> yes i believe we talked about this about a year ago
 746 2012-11-15 16:48:18 <gmaxwell> We can improve this, but doing it right and not creating other issues just takes a bit of time.  I have a patch here that when the connections fill up it culls peers. But there are a bunch of free parameters which don't have obvious values.
 747 2012-11-15 16:51:03 <gmaxwell> What I do is excempt localhost and inbound, then order by {number of duplicated connections per address}, {number of duplicated connections per netgroup}, {least time connected}  and kill randomly with a expoential weight for the highest priority.
 748 2012-11-15 16:51:04 rebroad has quit (Ping timeout: 245 seconds)
 749 2012-11-15 16:51:31 <phantomcircuit> inbound?
 750 2012-11-15 16:51:36 <gmaxwell> er outbound.
 751 2012-11-15 16:51:37 <phantomcircuit> you mean outbound?
 752 2012-11-15 16:51:40 <phantomcircuit> yeah ok
 753 2012-11-15 16:52:03 <phantomcircuit> it's trivial to get 128 ip addresses in different netgroups
 754 2012-11-15 16:52:07 <gmaxwell> phantomcircuit: AFAIK, you didn't comment before when I suggested this. So I have no idea if you think it's sufficient.
 755 2012-11-15 16:52:20 <phantomcircuit> it's not
 756 2012-11-15 16:52:51 <phantomcircuit> the best plan is a FIFO which evicts preemptively combined with significantly raising the default connection limit
 757 2012-11-15 16:53:00 <gmaxwell> phantomcircuit: yes, but even in that case you can't keep it forever filled.
 758 2012-11-15 16:53:13 <gmaxwell> We cannot currently significantly raise the default connection limit.
 759 2012-11-15 16:53:28 <gmaxwell> Not without a redesign of the network handling to be non-blocking and buffering efficient.
 760 2012-11-15 16:53:58 <phantomcircuit> iirc the buffer size was recently dropped to 1MB /connection
 761 2012-11-15 16:54:17 <phantomcircuit> so increasing the connection limit by 10x shouldn't cause any increase in buffer usage
 762 2012-11-15 16:55:33 paraipan has quit (Quit: Saliendo)
 763 2012-11-15 16:55:34 <gavinandresen> phantomcircuit: great, can you test that for us?  Testing is our bottleneck.
 764 2012-11-15 16:55:42 <gmaxwell> Adding a gigabyte of memory usage is _not_ something that we can do lightly. Also there is 5 MB of recieve buffering. And there are still issues with blocking on connections.
 765 2012-11-15 16:57:08 <Diablo-D3> bufferbloat.
 766 2012-11-15 16:57:42 <gmaxwell> phantomcircuit: nor does just increasing the count 10 times actually help against what you're doing.
 767 2012-11-15 16:57:57 <drizztbsd> malloc(rand() * 1024 * 1024);
 768 2012-11-15 16:58:02 <gmaxwell> phantomcircuit: as it's not biggie to use a userspace tcp stack and have as many concurrent connections as you want.
 769 2012-11-15 16:58:03 <gavinandresen> Maybe ironically, the badly-thought-out in/out buffers are a direct result of irresonsible vulnerability disclosure from phantomcircuit
 770 2012-11-15 16:58:51 <gmaxwell> (presuming you can get a raw socket on your 128 distinct /16 netblock hosts)
 771 2012-11-15 16:59:31 <phantomcircuit> too be completely fair the poorly thought out buffers were satoshis fault :)
 772 2012-11-15 17:00:37 <gavinandresen> Pretty sure I wrote that code after you posted python code that could lockup any node... but my memory is notoriously faulty about stuff like that.
 773 2012-11-15 17:01:15 <phantomcircuit> no you just added the connection limit
 774 2012-11-15 17:01:21 <phantomcircuit> and i warned about this then
 775 2012-11-15 17:01:33 <phantomcircuit> the insane buffers stuff was all satoshi
 776 2012-11-15 17:01:48 <gavinandresen> okey dokey
 777 2012-11-15 17:02:52 TD has quit (Quit: TD)
 778 2012-11-15 17:02:58 <gmaxwell> phantomcircuit: so, lets talk through you you don't think my proposed ejection schedule is protective... and why you propose the _opposite_ of it.
 779 2012-11-15 17:03:40 <gmaxwell> phantomcircuit: I think long lived peers should generally be protected because the rational assuption is that attack connections will tend to be shorter lived than established nodes.
 780 2012-11-15 17:04:04 ThomasV_ has quit (Quit: Quitte)
 781 2012-11-15 17:05:16 t7 has quit (Quit: ChatZilla 0.9.89-rdmsoft [XULRunner 1.9.0.17/2009122204])
 782 2012-11-15 17:05:21 <phantomcircuit> gmaxwell, that's probably not a good assumption
 783 2012-11-15 17:05:42 <phantomcircuit> indeed for a majority of users i would think the opposite is true
 784 2012-11-15 17:06:54 <gmaxwell> phantomcircuit: it models the infrastructure which _must_ be well connected, segmenting miners is much much worse than segmenting an end user.  But I think it also models the attack. The attack costs more to sustain and is more likely to get shut down the longer it runs.
 785 2012-11-15 17:08:45 <phantomcircuit> gmaxwell, personally i would rather assume that miners and other infrastructure players know what they're doing and will setup long lived connectible peers which have static connects to each other
 786 2012-11-15 17:08:49 <gmaxwell> phantomcircuit: another thought I had— but one I feel uncomfortable with without simulation— is peer preference. E.g. H(secret|peer ip) = preference... and evict based on preferences. So some nodes will be strongly protected by some peers... meaning that a node should eventuall be able to find stable connections by roving around until it hits ones where it happens to have better preference than the attacker nodes. The attacker would then
 787 2012-11-15 17:09:07 <phantomcircuit> since they're more likely to get it right than your average user who is just going to run bitcoin-qt.exe and cross their fingers
 788 2012-11-15 17:09:44 <gmaxwell> phantomcircuit: some do, some don't. If they do than whay you're talking about is fairly boring. It's just a dos attack, and a highly visable one that can be largely slayed by people circulating and blacklisting your finite nodes.
 789 2012-11-15 17:10:35 <phantomcircuit> blacklisting is more troublesome than it would at first seem
 790 2012-11-15 17:10:40 <gmaxwell> if connection length is not the metric and instead you prefer new nodes than your attack just needs to be to reconnect as agressively as possible, and you'll rapidly cycle the fifo and take almost all the slots.
 791 2012-11-15 17:10:54 <phantomcircuit> tor users and users on vpns or proxies would need to be banned
 792 2012-11-15 17:11:05 <phantomcircuit> and im guessing that would constitute the more paranoid segments of the user base
 793 2012-11-15 17:11:12 <phantomcircuit> so they would probably not find that amusing
 794 2012-11-15 17:13:06 <gmaxwell> phantomcircuit: even in that case they still have the option of darknetting.
 795 2012-11-15 17:13:21 <gmaxwell> phantomcircuit: any thoughts on secret preference?
 796 2012-11-15 17:13:41 <gmaxwell> Another option we could consider is supporting an alternative connectionless transport.
 797 2012-11-15 17:16:38 <gavinandresen> I've been all for an alternative transport for a long time.  Strength in diversity.
 798 2012-11-15 17:17:07 <gavinandresen> ... I just ain't the right person to make it happen (network programming ain't something I'm good at)
 799 2012-11-15 17:19:55 <phantomcircuit> i could force most connections though a handful of nodes and drop satoshidice transactions >.>
 800 2012-11-15 17:23:00 Arnavion has quit (Quit: Arnavion)
 801 2012-11-15 17:29:46 * midnightmagic had forgotten how phantomcircuit first introduced himself, but remembers now.
 802 2012-11-15 17:30:03 Arnavion has joined
 803 2012-11-15 17:30:07 <phantomcircuit> midnightmagic, lol
 804 2012-11-15 17:30:19 <phantomcircuit> "hello im here to break all your stuff"
 805 2012-11-15 17:32:30 * gmaxwell starts writing a toy connectionless priority queue based transport for bitcoin.
 806 2012-11-15 17:33:05 <phantomcircuit> oh hell fine i'll write a fix before i disclose how you actually make this work
 807 2012-11-15 17:33:14 <phantomcircuit> there's some secret sauce that most attackers would fuck up
 808 2012-11-15 17:33:18 <midnightmagic> phantomcircuit: no, you were pretty rude back then with your proclamations of what was and wasn't shitty..
 809 2012-11-15 17:33:24 <midnightmagic> phantomcircuit: no offence
 810 2012-11-15 17:33:30 <midnightmagic> "don't sybil me bro"
 811 2012-11-15 17:33:41 <phantomcircuit> im still pretty rude about it but i've been busy
 812 2012-11-15 17:34:09 <midnightmagic> hence the remembrance :)
 813 2012-11-15 17:34:17 <midnightmagic> but i'm glad you're doing it.
 814 2012-11-15 17:34:40 <midnightmagic> in the end it's a win for us all.
 815 2012-11-15 17:35:04 <gmaxwell> phantomcircuit: I think we should continue talking about fixes, because I'm going to nak something thats just a FIFO, as I think it greatly increases the risk to infrastructure.
 816 2012-11-15 17:36:05 <phantomcircuit> gmaxwell, there are only really two ways to ultimately fix this
 817 2012-11-15 17:36:27 <phantomcircuit> stop using connections and have some sort of resource limit infrastructure to keep people from doing simple dos attacks
 818 2012-11-15 17:36:44 <phantomcircuit> (the connection limit and recv buffer effective do that now but in a way that's kind of unpredictable)
 819 2012-11-15 17:37:04 <phantomcircuit> or use a proof of work system to make a sybil attack much more expensive
 820 2012-11-15 17:37:19 <phantomcircuit> a combination of the two would significantly deter this sort of attack
 821 2012-11-15 17:38:21 <gmaxwell> phantomcircuit: I'd previously proposed having mined peer identities. e.g. the idea being that miners could merkle bind peers they think are good. and attempt to generate low hash headers that connect to them.
 822 2012-11-15 17:38:22 <midnightmagic> i saw some neat hpc vendors hawking botnet visualizers here at sc12.
 823 2012-11-15 17:38:27 <phantomcircuit> realistically however the expense of killing the network is probably always going to be less than the cost of a 51% attack
 824 2012-11-15 17:38:30 <gmaxwell> In the context of p2pool where are users are miners this is quite powerful.
 825 2012-11-15 17:38:38 <gmaxwell> I don't know how generally applicable this would be.
 826 2012-11-15 17:38:55 <gmaxwell> phantomcircuit: killing the non-dark network. The dark network is unkillable.
 827 2012-11-15 17:39:12 <phantomcircuit> gmaxwell, killing tor hidden services isn't anywhere near as hard as people think
 828 2012-11-15 17:39:17 <phantomcircuit> not even remotely
 829 2012-11-15 17:39:42 <gmaxwell> phantomcircuit: I don't know what you're talking about there. I didn't say anything about tor.
 830 2012-11-15 17:40:00 <phantomcircuit> well that's the main darknet
 831 2012-11-15 17:40:09 <gmaxwell> Tor hidden services are not a darknet.
 832 2012-11-15 17:40:16 <gmaxwell> phantomcircuit: there is plenty of connectivity among nodes which are not publically announced or listening.
 833 2012-11-15 17:40:29 <gmaxwell> I doubt its anywhere near enough, but you can't attack it.
 834 2012-11-15 17:40:41 <gmaxwell> Tor HS nodes are public, they're just reached over an alternative transport.
 835 2012-11-15 17:41:01 <phantomcircuit> a true darknet gets you part of the way there
 836 2012-11-15 17:41:10 <amiller> gmaxwell, the best thing would be to send getworks as a challenge, that way the proof-of-work is literally reusable as (attempts at) bitcoin blocks
 837 2012-11-15 17:41:15 <phantomcircuit> but you need a fairly large social graph which is explicit about connections
 838 2012-11-15 17:41:16 <amiller> rather than exclusively mining peer identities
 839 2012-11-15 17:41:26 <phantomcircuit> amiller, that's a good idea
 840 2012-11-15 17:41:36 <amiller> or to the extent they're identities, they'd be 'merge mined', which is kind of the other direction of a 'near miss share'
 841 2012-11-15 17:41:43 <amiller> generally you can split, merge, and chain pow puzzles
 842 2012-11-15 17:43:31 <phantomcircuit> ok well i now have a list of all the peers that all the connectable peers will broadcast (indicated by my list of peers being static for about ten minutes)
 843 2012-11-15 17:43:38 <gmaxwell> amiller: I just said they should be merged mined. The argument against getwork challenge is just that there is a enormous difference in mining cost for different node types.
 844 2012-11-15 17:43:54 <phantomcircuit> it's about 318k broadcast peers
 845 2012-11-15 17:44:02 <phantomcircuit> with a few peers broadcasting complete garbage
 846 2012-11-15 17:44:07 <phantomcircuit> which is interesting itself
 847 2012-11-15 17:44:09 <amiller> i wonder if that would be a good reason to get a gpu, even if you don't mine, but just so you have some collateral you can use to get a priority connection
 848 2012-11-15 17:44:50 <gmaxwell> I do think we should generally have facilities for getwork challenge, it's even on my dream list.
 849 2012-11-15 17:44:52 <phantomcircuit> gmaxwell, actually there is hidden brilliance in amiller's suggestion
 850 2012-11-15 17:45:03 <phantomcircuit> it would mean that actual miners are effectively guaranteed connections
 851 2012-11-15 17:45:05 <midnightmagic> WOW  first-ever whole-observable universe simulation
 852 2012-11-15 17:45:21 <gmaxwell> https://en.bitcoin.it/wiki/User:Gmaxwell/features "Share in lieu of TXN fees"
 853 2012-11-15 17:45:37 <phantomcircuit> which would make a true network split effectively a 51% attack in itself
 854 2012-11-15 17:46:02 agricocb has quit (Ping timeout: 246 seconds)
 855 2012-11-15 17:46:11 <amiller> you'd want to be able to use my blinded-remainder variation of the proof-of-work share for that :/
 856 2012-11-15 17:46:19 <gmaxwell> phantomcircuit: that should just be merge mined. Since asking miners to loose revenue for it is uncool.
 857 2012-11-15 17:46:33 <gmaxwell> amiller: why? irrelevant.
 858 2012-11-15 17:47:00 <phantomcircuit> gmaxwell, they wouldn't have to actually as long as the getwork info being used pays out to the miner
 859 2012-11-15 17:47:11 <gmaxwell> amiller: you let the attempter pay themselves in the share. E.g. you require they attempt a block that pays half to them half to you. They've have no incentive to withhold.
 860 2012-11-15 17:47:12 <amiller> no the getwork should pay out to the challenger shouldn't it
 861 2012-11-15 17:47:24 <amiller> if they're an attacker then they'd have such an incentive
 862 2012-11-15 17:47:33 <phantomcircuit> hmm
 863 2012-11-15 17:47:34 <phantomcircuit> i guess
 864 2012-11-15 17:48:34 <gmaxwell> phantomcircuit: also it would mean tight coupling between mining and frontends which is undesirable. Better if miners can just mine POW blocks for an addresses they like.
 865 2012-11-15 17:49:37 bitbnc28-7789 is now known as fred123
 866 2012-11-15 17:49:52 <phantomcircuit> i suppose
 867 2012-11-15 17:50:21 <gmaxwell> In any case, I don't think it addresses the thing you were concerned about.
 868 2012-11-15 17:51:03 <midnightmagic> holy fuck this is the coolest presentation i've seen all week and all he's doing is reading slides lol
 869 2012-11-15 17:51:03 <phantomcircuit> how so
 870 2012-11-15 17:51:17 <midnightmagic> in a super thick italian accent.
 871 2012-11-15 17:51:30 <drizztbsd> ehi
 872 2012-11-15 17:51:35 * drizztbsd is italian
 873 2012-11-15 17:51:37 <amiller> ehello
 874 2012-11-15 17:52:47 <phantomcircuit> midnightmagic, whatcha watchin
 875 2012-11-15 17:53:02 CodeInChaos has joined
 876 2012-11-15 17:53:31 <midnightmagic> phantomcircuit: I'm at SC12 in one of the technical talks. A team has simulated the entire observable universe including dark energy
 877 2012-11-15 17:53:52 <phantomcircuit> uh huh how large are their simulation units?
 878 2012-11-15 17:53:53 JZavala has joined
 879 2012-11-15 17:53:54 <Diablo-D3> yeah but
 880 2012-11-15 17:53:56 <phantomcircuit> entire galaxies? :)
 881 2012-11-15 17:53:59 <Diablo-D3> theres no dark energy
 882 2012-11-15 17:54:00 <midnightmagic> phantomcircuit: Someone gave him a green laser and he's leaving it on when he's done pointing at a slide. people look like they want to duck.
 883 2012-11-15 17:54:04 <phantomcircuit> super galaxies?
 884 2012-11-15 17:54:05 <midnightmagic> phantomcircuit: entire universe.
 885 2012-11-15 17:54:13 molecular has quit (Ping timeout: 246 seconds)
 886 2012-11-15 17:54:22 <phantomcircuit> midnightmagic, what's the smallest unit
 887 2012-11-15 17:54:31 <Diablo-D3> probably stars
 888 2012-11-15 17:55:18 <midnightmagic> phantomcircuit: I'm not completely certain.. I think it's a derived unit. I thought they were using actual recorded stars but it might just be a derived density point.
 889 2012-11-15 17:55:37 <phantomcircuit> midnightmagic, er i mean
 890 2012-11-15 17:55:45 <midnightmagic> 300TB memory; 1.7PB with 60 GB/s bandwidth used at full speed.
 891 2012-11-15 17:55:53 <phantomcircuit> in simulations you choose something as being the smallest unit
 892 2012-11-15 17:56:18 <Diablo-D3> midnightmagic: thats not particularly hard to build
 893 2012-11-15 17:56:20 <midnightmagic> phantomcircuit: yes I know, all I saw was a point count earlier but I missed it because i thought he was going to fry my eyes
 894 2012-11-15 17:56:23 <phantomcircuit> then you take whatever it is you're modeling and treat each unit as an individual point
 895 2012-11-15 17:56:24 CodesInChaos has quit (Ping timeout: 252 seconds)
 896 2012-11-15 17:56:26 <gmaxwell> phantomcircuit: perhaps they've picked boundary conditions and run the solution in the frequency domain. :P
 897 2012-11-15 17:56:35 <Diablo-D3> I can put 512GB in a single box
 898 2012-11-15 17:56:43 <Diablo-D3> so thats only 600 of said boxes
 899 2012-11-15 17:56:54 <Diablo-D3> thats trivial to build
 900 2012-11-15 17:57:11 <midnightmagic> Diablo-D3: there are ~ 6000 nodes. and no, supercomputers aren't easy to build :)
 901 2012-11-15 17:57:18 <phantomcircuit> so 60 GB/s is just 1 gbps ethernet :)
 902 2012-11-15 17:57:18 <Diablo-D3> 6000? what a waste of ahrdware
 903 2012-11-15 17:57:39 <Diablo-D3> midnightmagic: yes, they're very easy to build
 904 2012-11-15 17:58:01 <phantomcircuit> midnightmagic, depends on if it's a proper supercomputer with low latency memory interconnects or just a ton of boxes hooked up with ethernet cables :)
 905 2012-11-15 17:58:14 <midnightmagic> it's an actual supercomputer, not just a grid.
 906 2012-11-15 17:58:17 <Diablo-D3> phantomcircuit: low latency interconnects are cheap
 907 2012-11-15 17:58:19 <midnightmagic> I am at SC12 after all. lol
 908 2012-11-15 17:58:28 <Diablo-D3> midnightmagic: wtf is SC12 again?
 909 2012-11-15 17:58:44 <phantomcircuit> super computer 2012 im guessing
 910 2012-11-15 17:58:56 CodesInChaos has joined
 911 2012-11-15 17:59:10 <Diablo-D3> heh, theres actually a super computing conforence
 912 2012-11-15 17:59:14 <Diablo-D3> thats retarded
 913 2012-11-15 17:59:15 <midnightmagic> he's using a particle-mesh-adaptive mesh refinement.   something called peano-hilbert curve for space-filling.
 914 2012-11-15 17:59:37 <midnightmagic> Diablo-D3: lol it's not. the stuff here is retardedly powerful.
 915 2012-11-15 17:59:45 <Diablo-D3> midnightmagic: lol hilbert curves
 916 2012-11-15 18:00:14 <midnightmagic> aaaargh too much information flying by
 917 2012-11-15 18:00:30 <midnightmagic> n-body poisson solver..
 918 2012-11-15 18:01:05 <Diablo-D3> hilbert curves are the easiest space filling fractals to implement
 919 2012-11-15 18:01:17 <midnightmagic> probably why they chose them.
 920 2012-11-15 18:02:04 agricocb has joined
 921 2012-11-15 18:02:09 <midnightmagic> Diablo-D3: Oh, I get it. You are ignoring costs and research when you say "easy". :)
 922 2012-11-15 18:02:10 <gmaxwell> Diablo-D3: ... uh, they aren't.  And they have excellent compaction.
 923 2012-11-15 18:02:24 CodeInChaos has quit (Ping timeout: 268 seconds)
 924 2012-11-15 18:02:53 <Diablo-D3> midnightmagic: research? no research
 925 2012-11-15 18:02:59 <Diablo-D3> you can do it purely with off the shelf components
 926 2012-11-15 18:03:03 <Diablo-D3> gmaxwell: you sure?
 927 2012-11-15 18:03:10 <midnightmagic> Diablo-D3: This whole conference is filled with research papers.
 928 2012-11-15 18:03:27 <Diablo-D3> midnightmagic: yes, research of what they do with highly parallel hardware
 929 2012-11-15 18:03:32 agricocb has quit (Remote host closed the connection)
 930 2012-11-15 18:03:32 <Diablo-D3> not on the actual shit itself
 931 2012-11-15 18:03:42 <Diablo-D3> IBM finished parallel hardware research in the early 80s
 932 2012-11-15 18:04:40 <Diablo-D3> gmaxwell: /me shrugs
 933 2012-11-15 18:04:56 <Diablo-D3> gmaxwell: I just hate people who rattle off numbers like that and think they're something because of it
 934 2012-11-15 18:05:33 <Diablo-D3> midnightmagic: btw, how are they defining node?
 935 2012-11-15 18:05:36 <gmaxwell> Diablo-D3: the plain peano curve is simpler, but has worse locality in 2 dimensions at least. I'm not aware of any curve that has better locality in 2d or 3d space than the hilbert one, but I admit I haven't looked much.
 936 2012-11-15 18:05:37 <midnightmagic> .. I don't think I'm something because they're trying to describe the bounds of their universe simulation.
 937 2012-11-15 18:06:24 <gmaxwell> Theora frames are partially scanned in hilbert order; it's clearly helpful for compression efficiency, but it makes the encoder a lot more complex and memory bandwidth hungry.
 938 2012-11-15 18:06:56 molecular has joined
 939 2012-11-15 18:08:12 <jgarzik> hmmm
 940 2012-11-15 18:08:28 <jgarzik> ignoring write for the moment, I think I could update bitcoind to eliminate the RX buffer
 941 2012-11-15 18:09:11 <midnightmagic> and here're the animations/visualisations!  wow!
 942 2012-11-15 18:11:45 graingert has joined
 943 2012-11-15 18:13:37 <Diablo-D3> midnightmagic: basically its like this, I can put, in 7U, 5120GB of memory, 800Gbps of infiniband interconnect, and 40gbps of ethernet, and 40 LGA 20111 sockets
 944 2012-11-15 18:14:24 <jgarzik> infiniband is a sad joke
 945 2012-11-15 18:14:25 <Diablo-D3> midnightmagic: 300TB would take 10 full racks.
 946 2012-11-15 18:14:29 MC-Eeepc has joined
 947 2012-11-15 18:14:33 <jgarzik> ditto RDMA in general
 948 2012-11-15 18:15:47 <midnightmagic> Diablo-D3: that's great, you should sell that to people who want supercomputers built.
 949 2012-11-15 18:16:09 <midnightmagic> oak ridge might be interested in chatting with you..
 950 2012-11-15 18:17:52 graingert has quit (Remote host closed the connection)
 951 2012-11-15 18:17:58 MC1984 has quit (Ping timeout: 248 seconds)
 952 2012-11-15 18:19:12 <Diablo-D3> midnightmagic: they go the IBM overdesigned route
 953 2012-11-15 18:19:12 <midnightmagic> Diablo-D3: it *looks* like nobody's building supercomputers anymore without some kind of attached gpgpu-like devices, either nvidia, intel, or amd..  mostly nvidia from what i can see here.
 954 2012-11-15 18:19:28 <Diablo-D3> midnightmagic: yes, why would you?
 955 2012-11-15 18:19:30 <midnightmagic> Diablo-D3: TITAN has a 2% failure rate.
 956 2012-11-15 18:19:46 <Diablo-D3> I mean, look at what bitcoin does
 957 2012-11-15 18:19:57 <midnightmagic> that was the error rate at first power-up. apparently.
 958 2012-11-15 18:19:58 <Diablo-D3> we use GPUs because its more efficient for highly parallel tasks
 959 2012-11-15 18:20:55 <Diablo-D3> midnightmagic: and do realize, this is what I was building DMC for
 960 2012-11-15 18:21:03 <Diablo-D3> a DC dedicated to hosting demanding customers
 961 2012-11-15 18:22:37 agricocb has joined
 962 2012-11-15 18:22:45 <Diablo-D3> jgarzik: btw, infiniband is great if you use it in the right place
 963 2012-11-15 18:23:41 <Diablo-D3> jgarzik: even though its extremely low latency, you still treat it like an async messaging transport
 964 2012-11-15 18:23:45 <Diablo-D3> and then all is well.
 965 2012-11-15 18:23:48 <jgarzik>         if (nMessageSize > MAX_SIZE)
 966 2012-11-15 18:23:48 <jgarzik>         {
 967 2012-11-15 18:23:48 <jgarzik>             printf("ProcessMessages(%s, %u bytes) : nMessageSize > MAX_SIZE\n", strCommand.c_str(), nMessageSize);
 968 2012-11-15 18:23:48 <jgarzik>             continue;
 969 2012-11-15 18:23:52 <jgarzik> really? really???
 970 2012-11-15 18:23:56 <jgarzik> that should be DoS()
 971 2012-11-15 18:24:19 <jgarzik> otherwise we will happily receive, and ignore, 4GB messages all day long
 972 2012-11-15 18:25:09 <jgarzik> At least there is good news: RX buffer is not a fixed size, but grows and shrinks
 973 2012-11-15 18:25:23 ThomasV_ has joined
 974 2012-11-15 18:25:24 <jgarzik> still way too much copying and shifting and allocating
 975 2012-11-15 18:25:39 wumpus has quit (Quit: http://quassel-irc.org - Chat comfortably. Anywhere.)
 976 2012-11-15 18:26:09 wumpus has joined
 977 2012-11-15 18:26:12 <Diablo-D3> btw, midnightmagic, the thing with "supercomputers" is you have shit like what IBM is pushing
 978 2012-11-15 18:26:17 <Diablo-D3> they removed the rack from the equation
 979 2012-11-15 18:26:30 <sipa> jgarzik: i think that's mainly for skipping obviously-wrong headers
 980 2012-11-15 18:26:32 <midnightmagic> IBM does seem to be pushing anything here.
 981 2012-11-15 18:26:34 <Diablo-D3> they call it a "smart rack", I call it a modular 42u tall blade box.
 982 2012-11-15 18:26:39 <midnightmagic> doesn't I mean.
 983 2012-11-15 18:26:54 <Diablo-D3> which, I mean, hey, they're obviously using space efficiently here
 984 2012-11-15 18:27:17 <Diablo-D3> cooling and power and data interconnect is built right into the rack
 985 2012-11-15 18:27:24 <Diablo-D3> it uses every last square inch of roo
 986 2012-11-15 18:27:26 <Diablo-D3> *room
 987 2012-11-15 18:27:33 <Diablo-D3> but you buy into IBM 42u at a time
 988 2012-11-15 18:27:36 <jgarzik> sipa: but we look at message boundaries, not simply search for pchMessageStart.  So we will lose sync at the first obviously-wrong header anyway.
 989 2012-11-15 18:27:41 <Diablo-D3> and pay like 1.5-2x more
 990 2012-11-15 18:27:57 <Diablo-D3> the rest of the (very small list of) vendors are thesame
 991 2012-11-15 18:27:57 <midnightmagic> i'm gonna take a bunch of pictures today, there's this crazy one-rack cooling unit that towers above everything and had these giant connectors looping down from the ceiling in the exhibition hall.
 992 2012-11-15 18:28:01 <Diablo-D3> IBM just has a 40 year lead
 993 2012-11-15 18:28:02 <jgarzik> sipa: and in general, we don't need to be a /dev/null data sink :)
 994 2012-11-15 18:28:14 <sipa> jgarzik: agree
 995 2012-11-15 18:28:24 <Diablo-D3> midnightmagic: erm, which company?
 996 2012-11-15 18:28:34 <Diablo-D3> midnightmagic: is it a cooling unit that is the size of a rack?
 997 2012-11-15 18:28:55 <midnightmagic> i don't recall.  there're hundreds, maybe in the thousands, of exhibitors in the hall.
 998 2012-11-15 18:29:07 <Diablo-D3> because APC and Liebert both sell in row cooling units
 999 2012-11-15 18:29:15 <midnightmagic> no it's a unit that cools internal to the rack, so it's like a giant refrigerator.
1000 2012-11-15 18:29:18 <_dr> where? salt lake?
1001 2012-11-15 18:29:25 <Diablo-D3> midnightmagic: oh, has to be a liebert unit then
1002 2012-11-15 18:29:26 <midnightmagic> _dr: Yep.
1003 2012-11-15 18:29:34 <midnightmagic> Nope, def'ntly not liebert.
1004 2012-11-15 18:29:38 <Diablo-D3> hrm
1005 2012-11-15 18:29:43 <Diablo-D3> I dont think APC sells one
1006 2012-11-15 18:29:43 <_dr> any chance you were at the release of the xeon phi?
1007 2012-11-15 18:30:07 <Diablo-D3> midnightmagic: liebert is the apple/cisco/sun/whatever of enterprise cooling
1008 2012-11-15 18:30:12 <Diablo-D3> waaaay overpriced for what they give you
1009 2012-11-15 18:30:14 JZavala has quit (Ping timeout: 248 seconds)
1010 2012-11-15 18:30:18 <midnightmagic> _dr: No, I'm here mostly to be in the technical presentations/papers part. I've been missing all kinds of awesome stuff in the exhibition hall.
1011 2012-11-15 18:30:46 <_dr> oh okay
1012 2012-11-15 18:30:52 <Diablo-D3> wait, theres another company thats doing that too
1013 2012-11-15 18:30:54 <Diablo-D3> but I forget who
1014 2012-11-15 18:31:05 <Diablo-D3> I came across them during my research
1015 2012-11-15 18:31:28 <Diablo-D3> basically, costs almost as much as the liebert fridge rack, doesnt do it as well
1016 2012-11-15 18:31:30 <_dr> anything cool at the presentations?
1017 2012-11-15 18:32:33 <midnightmagic> _dr: Yeah, the TITAN women *killed* their visualisation outline talk. At one point the nearby presenters had to come in and tell them to turn it down.
1018 2012-11-15 18:32:52 <midnightmagic> _dr: It was a "convergence of video games and HPC" sort of thing.
1019 2012-11-15 18:33:38 <Diablo-D3> heh
1020 2012-11-15 18:33:49 <phantomcircuit> BUG: unable to handle kernel NULL pointer dereference at 0000000000000028
1021 2012-11-15 18:33:50 <midnightmagic> _dr: Soem of the large-scale visualisation stuff it pretty sexy too. There's some crazy medical visualisation going on. The guy doing that presentation was looking up the whole time like Lemmy singing Ace Of Spades
1022 2012-11-15 18:33:57 <phantomcircuit> IP: [<ffffffff811ccb18>] ext4_ext_remove_space+0x760/0xa23
1023 2012-11-15 18:33:58 <phantomcircuit> :/
1024 2012-11-15 18:34:01 <phantomcircuit> stupid ext4
1025 2012-11-15 18:34:21 <_dr> maybe i should go there next year, too
1026 2012-11-15 18:35:37 <_dr> sounds pretty interesting after all
1027 2012-11-15 18:35:39 <midnightmagic> _dr: Once is enough for me. I have research credentials still at the local university, so I have a feeling that a week of hard research would've taught me more, but it's nice to see all the industries here hawking their high-end stuff.
1028 2012-11-15 18:36:22 <midnightmagic> _dr: Some of it is..  virtually useless to anyone not running a supercomputing centre, which I guess is the point.
1029 2012-11-15 18:36:55 <midnightmagic> also I've never seen such a concentration of ego in one place. :)
1030 2012-11-15 18:37:40 pusle has joined
1031 2012-11-15 18:37:43 <_dr> midnightmagic: well, i work at a computer architecture chair and our local hpc group has 'small supercomputers' so it's pretty interesting for me :)
1032 2012-11-15 18:38:17 PiZZaMaN2K1 has joined
1033 2012-11-15 18:38:28 <midnightmagic> _dr: Yeah this stuff would be right up your alley. :)  Hey I don't suppose you know who Michael Wolfe is do you?
1034 2012-11-15 18:38:59 <_dr> midnightmagic: no sorry
1035 2012-11-15 18:39:03 * _dr is german
1036 2012-11-15 18:39:08 CodesInChaos has quit (Ping timeout: 260 seconds)
1037 2012-11-15 18:39:19 <midnightmagic> _dr: He's with PGI, they write an OpenACC compiler.
1038 2012-11-15 18:39:29 denisx has joined
1039 2012-11-15 18:39:40 <midnightmagic> _dr: Skip his tutorials. Don't say nobody warned you. :)
1040 2012-11-15 18:40:42 <_dr> sounds like ispc
1041 2012-11-15 18:42:59 <_dr> i was fairly impressed when intel released the final specs for knights corner
1042 2012-11-15 18:43:33 <gmaxwell> _dr: be impressed if the product ever ships to the public.
1043 2012-11-15 18:43:35 agricocb has quit (Ping timeout: 264 seconds)
1044 2012-11-15 18:43:51 <_dr> it will
1045 2012-11-15 18:43:54 <gmaxwell> They've been promoting mic since 2008— I'm resigned to thinking it's just a stunt to keep people from porting to cuda.
1046 2012-11-15 18:43:57 <_dr> i already have access to one
1047 2012-11-15 18:44:09 <_dr> i'm more concerned about the status of the software toolchain
1048 2012-11-15 18:45:10 <_dr> gmaxwell: well, we'll see. i know it seems that way, but they have redesigned it a lot and kept a tight lid on everything up to now, because of all the trouble they went into when it was called larabee :)
1049 2012-11-15 18:45:51 topace has joined
1050 2012-11-15 18:46:43 <_dr> but i'm very impressed with the final specs. it is on par with the fastest nvidia gpu (k20), which also hasn't been released yet in terms of *theoretical* double-precision FP performance
1051 2012-11-15 18:47:05 <_dr> however, GPUs have a reputation of only achieving a fraction of their theoretical performance
1052 2012-11-15 18:47:41 <_dr> and when it comes to memory bandwitdh, knights corner easily outperforms the k20 (320 vs ~200 GB/s)
1053 2012-11-15 18:48:19 <_dr> it's a pity that ati doesn't join the supercomputing business
1054 2012-11-15 18:48:31 <_dr> a third player wouldn't hurt to speed things up :)
1055 2012-11-15 18:48:37 <midnightmagic> firegl s10000 seems to kill k20x which i find amusing.
1056 2012-11-15 18:48:53 <midnightmagic> the AMD booth guy was pretty hard on AMD as a company.
1057 2012-11-15 18:49:24 ovidiusoft has quit (Quit: leaving)
1058 2012-11-15 18:49:24 <_dr> midnightmagic: i agree, but nvidia can always put two k20s on a card as well
1059 2012-11-15 18:49:30 prahanormal has joined
1060 2012-11-15 18:49:37 <cjd> I heard through the grape vine that AMD is setting up for a buyout or something..
1061 2012-11-15 18:49:52 <_dr> also, the s10000 isn't really that hpc-focussed due to its low DP performance. but that's just my opinion
1062 2012-11-15 18:50:08 <cjd> they certainly seem to be bungling things firing their R&D staff
1063 2012-11-15 18:50:56 harkon has joined
1064 2012-11-15 18:51:04 <midnightmagic> _dr: AMD seems to be fairly power-efficient still. Hrm. Now I'm curious about DP power efficiency. The AMD guy says Intel managed to get the green-grid award edging AMD out.
1065 2012-11-15 18:51:36 <midnightmagic> the software dev tools for nVidia (thrust et al) seem to be the main reason why people aren't even considering amd these days.
1066 2012-11-15 18:52:04 agricocb has joined
1067 2012-11-15 18:52:26 * midnightmagic goes to the last talk, but should post some pictures of the exhibition hall.. it's pretty over-the-top in there.
1068 2012-11-15 18:53:07 topace has quit (Read error: Connection reset by peer)
1069 2012-11-15 18:53:48 topace is now known as 2!~kvirc@unaffiliated/topace|topace
1070 2012-11-15 18:54:25 eroot has quit (Ping timeout: 246 seconds)
1071 2012-11-15 18:54:25 <jgarzik> gah
1072 2012-11-15 18:54:33 <jgarzik> this socket thread / message thread bouncing is annoying
1073 2012-11-15 18:54:47 <jgarzik> lots of context switching for each message
1074 2012-11-15 18:55:09 ovidiusoft has joined
1075 2012-11-15 18:56:31 emeitner has joined
1076 2012-11-15 18:59:14 denisx has quit (Remote host closed the connection)
1077 2012-11-15 18:59:29 denisx has joined
1078 2012-11-15 19:04:16 <jgarzik> ah, interesting
1079 2012-11-15 19:04:24 <jgarzik> ReceiveBufferSize() is misnamed
1080 2012-11-15 19:04:46 <jgarzik> that's all about flood control
1081 2012-11-15 19:05:11 <jgarzik> if the socket buffer reaches that size before the message thread can catch up and eat data, you are dropped
1082 2012-11-15 19:05:26 khalahan has quit (Ping timeout: 244 seconds)
1083 2012-11-15 19:10:12 guruvan- has quit (Remote host closed the connection)
1084 2012-11-15 19:10:12 guruvan has quit (Remote host closed the connection)
1085 2012-11-15 19:11:04 MC-Eeepc has quit (Ping timeout: 265 seconds)
1086 2012-11-15 19:13:04 guruvan has joined
1087 2012-11-15 19:13:23 daybyter has quit (Quit: Konversation terminated!)
1088 2012-11-15 19:15:17 linkedinyou has quit (Ping timeout: 246 seconds)
1089 2012-11-15 19:15:45 guruvan- has joined
1090 2012-11-15 19:17:16 prahanormal has quit ()
1091 2012-11-15 19:20:41 guruvan has quit (Ping timeout: 276 seconds)
1092 2012-11-15 19:20:56 guruvan- is now known as guruvan
1093 2012-11-15 19:21:34 guruvan_ has joined
1094 2012-11-15 19:22:39 drizztbsd has quit (Remote host closed the connection)
1095 2012-11-15 19:28:01 PiZZaMaN2K1 has quit (Quit: Linkinus - http://linkinus.com)
1096 2012-11-15 19:31:44 guruvan has quit (Ping timeout: 276 seconds)
1097 2012-11-15 19:32:23 guruvan_ has quit (Ping timeout: 276 seconds)
1098 2012-11-15 19:35:09 TD has joined
1099 2012-11-15 19:35:11 forsetifox has joined
1100 2012-11-15 19:39:10 guruvan has joined
1101 2012-11-15 19:46:52 int0x27h has quit (Ping timeout: 240 seconds)
1102 2012-11-15 19:48:40 int0x27h has joined
1103 2012-11-15 19:50:12 guruvan- has joined
1104 2012-11-15 19:51:27 khalahan has joined
1105 2012-11-15 20:02:53 int0x27h has quit (Changing host)
1106 2012-11-15 20:02:53 int0x27h has joined
1107 2012-11-15 20:04:27 MC1984 has joined
1108 2012-11-15 20:15:06 <jgarzik> w00t
1109 2012-11-15 20:15:16 <jgarzik> RX buffer copies--
1110 2012-11-15 20:16:07 <jgarzik> clean++
1111 2012-11-15 20:17:43 <sipa> pullreq?
1112 2012-11-15 20:17:49 AlexWaters has joined
1113 2012-11-15 20:24:29 D34TH has joined
1114 2012-11-15 20:24:49 piuk has joined
1115 2012-11-15 20:26:14 piuk has quit (Client Quit)
1116 2012-11-15 20:26:25 datagutt has quit (Quit: kthxbai)
1117 2012-11-15 20:27:13 dvide has joined
1118 2012-11-15 20:28:25 <gavinandresen> sipa: you have a few minutes to brainstorm payment protocol stuff?
1119 2012-11-15 20:31:02 JZavala has joined
1120 2012-11-15 20:31:25 <jgarzik> sipa: soon
1121 2012-11-15 20:35:23 <sipa> gavinandresen: later maybe
1122 2012-11-15 20:36:20 <gavinandresen> sipa: ok
1123 2012-11-15 20:38:09 Maged has joined
1124 2012-11-15 20:40:33 t7 has joined
1125 2012-11-15 20:49:08 <D34TH> bitcoin://pay/1ADDR
1126 2012-11-15 20:49:10 <D34TH> :D
1127 2012-11-15 20:53:08 freakazoid has joined
1128 2012-11-15 20:53:26 <BlueMatt> thats pretty much what we have (and it sucks)
1129 2012-11-15 20:57:24 <sipa> gavinandresen: i'm here (on my phone though)
1130 2012-11-15 20:59:04 <gavinandresen> sipa: ok, if it gets annoying to type let me know and we'll talk later.
1131 2012-11-15 20:59:32 <gavinandresen> sipa: So, seems to me the first problem I want to solve with a payment protocol is authenticating the entity being paid
1132 2012-11-15 20:59:58 <gavinandresen> sipa: ... and it's not clear to me how https://gist.github.com/1237788  accomplishes that
1133 2012-11-15 21:00:14 <sipa> how do you mean by authenticating?
1134 2012-11-15 21:00:20 <sipa> preventing MitM?
1135 2012-11-15 21:00:22 <gavinandresen> yes
1136 2012-11-15 21:00:58 <sipa> well there is no solution to that - either rely on a PKI, or use your own trust system
1137 2012-11-15 21:01:08 <gavinandresen> I want the "Payment request from..." dialog box to have a green checkmark or something, like a browser has when you have a secure https connection, if you are paying to an authenticated address
1138 2012-11-15 21:01:40 <sipa> my gist specifies authenticated addresses, using bitcoin addresses embedded in them
1139 2012-11-15 21:01:46 <gavinandresen> In other words:  "You are paying amazon.com.  I, your bitcoin client, checked and you really are paying amazon.com"
1140 2012-11-15 21:02:06 <sipa> that is no solution if you have no faith in how you got that uri
1141 2012-11-15 21:02:25 <sipa> but people visit websites already, with broken ssl certificates
1142 2012-11-15 21:03:04 <sipa> they obviously trust who they are interacting with, or they wouldn't do business
1143 2012-11-15 21:03:10 <gavinandresen> Right... I assume that we will be stricter than browsers and not allow payments if the ssl cert is broken
1144 2012-11-15 21:03:41 <sipa> well, not everyone trusts SSL
1145 2012-11-15 21:03:55 <gavinandresen> okey doke.  What DOES everyone trust?
1146 2012-11-15 21:03:58 <cjd> If dns were secure, would it be enough to put a pubkey in a dns entry? perhaps change it every few weeks?
1147 2012-11-15 21:04:21 <gavinandresen> cjd: DNSSEC would work, but isn't widely deployed.
1148 2012-11-15 21:04:36 <sipa> gavinandresen: in my opinion, no real solution exists now, but we can avoid the problem
1149 2012-11-15 21:04:42 PhantomSpark has quit (3!~kvirc@pool-71-190-231-20.nycmny.fios.verizon.net|Ping timeout: 276 seconds)
1150 2012-11-15 21:04:49 <gavinandresen> sipa: ok, I'm trying to understand how.
1151 2012-11-15 21:05:06 <cjd> so it's ok to use a single pubkey for all incoming payments for a few weeks until the entry is changed?
1152 2012-11-15 21:05:17 <sipa> cjd: hell no
1153 2012-11-15 21:05:22 <gavinandresen> I'm browsing across the web, I come across a http+btc:// link and click on it....
1154 2012-11-15 21:05:29 <sipa> gavinandresen: exactly
1155 2012-11-15 21:05:37 <gavinandresen> Now what do I see?
1156 2012-11-15 21:06:13 <sipa> your client shows the downloaded payment descriptor in a human readable form, and verifies the signature on it
1157 2012-11-15 21:06:29 <gavinandresen> verifies the signature against what?
1158 2012-11-15 21:06:44 <sipa> against the address in the http+btc uri
1159 2012-11-15 21:07:02 <gavinandresen> Okey doke.  But how do I know I'm not being MITM'ed ?
1160 2012-11-15 21:07:08 <sipa> you can't
1161 2012-11-15 21:07:11 <gavinandresen> sigh.
1162 2012-11-15 21:07:16 <gavinandresen> FAIL
1163 2012-11-15 21:07:21 <sipa> there is no solution to this problem
1164 2012-11-15 21:07:31 <sipa> but you trusted the site that gave you the uri
1165 2012-11-15 21:08:06 <gavinandresen> I don't trust any http: site.
1166 2012-11-15 21:08:16 <sipa> it should be https of course
1167 2012-11-15 21:08:17 <gavinandresen> I mostly trust https: sites, with valid certs
1168 2012-11-15 21:08:40 <sipa> or the uri was sent to you through another system that you trust
1169 2012-11-15 21:09:18 <gavinandresen> I trust even more information that I get from multiple, independent sources
1170 2012-11-15 21:09:28 <sipa> maybe you can have the btc client warn you in case the payment descriptor is not downloaded from a valid https domain
1171 2012-11-15 21:10:09 <gavinandresen> ok, now we're getting somewhere.  How does the btc client know that?  The browser gets the payment-request MIME type, then runs  bitcoind <file-containing-stuff> .....
1172 2012-11-15 21:10:37 <gavinandresen> (excuse, me, runs Bitcoin-Qt <file-containing-payment-info-stuff> )
1173 2012-11-15 21:10:46 <sipa> if you have a btc+ uri, the downloading of the payment request is done by the client
1174 2012-11-15 21:10:48 nibor has joined
1175 2012-11-15 21:11:08 <gavinandresen> Ok, so there is a list of certificate authorities shipped with the clients....
1176 2012-11-15 21:11:18 <gavinandresen> ... which we keep up to date, blah blah blah....
1177 2012-11-15 21:11:29 <sipa> use those provided by the system
1178 2012-11-15 21:11:49 <sipa> but ibdon't think this is a problem *we* need to solve
1179 2012-11-15 21:12:31 <sipa> right now, i think we need a way to request and send payments as files
1180 2012-11-15 21:12:48 CodesInChaos has joined
1181 2012-11-15 21:12:59 <gavinandresen> what problem(s) does that solve?
1182 2012-11-15 21:13:20 <sipa> it's the first step to doing actual payments
1183 2012-11-15 21:13:49 <sipa> people will not necessarily agree on how data gets transported
1184 2012-11-15 21:13:53 <gavinandresen> you're losing me.  I thought I was actually paying the other day when I bought some more alpaca socks....
1185 2012-11-15 21:14:16 <sipa> but at least it gets the metadata problem, and instant payment out of the way
1186 2012-11-15 21:14:40 <sipa> and your only problem becomes doing communication between sender and receiver
1187 2012-11-15 21:16:23 <gavinandresen> sipa: I'm still lost.  Maybe some concrete, here is how the payment protocol works walk-throughs would help me.  The use cases I've been working through are:
1188 2012-11-15 21:16:26 <gavinandresen> SatoshiDice
1189 2012-11-15 21:16:30 <gavinandresen> multi-device wallet
1190 2012-11-15 21:16:33 <gavinandresen> multi-person wallet
1191 2012-11-15 21:17:31 <gavinandresen> I can see how you could implement SatoshiDice.  I think.  Because they could run a payment gateway that was assumed to be always listening.
1192 2012-11-15 21:18:11 <sipa> in its most rudimentary form, with just files, you could download a file from satoshidice that describes a payment for doing a certain bet
1193 2012-11-15 21:18:18 <Luke-Jr> last thing we need is to enable DDoS attacks like SatoshiDice…
1194 2012-11-15 21:18:42 <sipa> loaded in your client, this would produce a payment files (including metadata such as your refund address)
1195 2012-11-15 21:19:04 <sipa> this files (or base64 string, whatever) can be uploaded on their site
1196 2012-11-15 21:19:11 <sipa> of course, nobody wants this
1197 2012-11-15 21:19:46 <sipa> it's inconvenient, but it works, it is instant, it would work from any client, and you can have instant feedback
1198 2012-11-15 21:20:09 <gavinandresen> ... and nobody wants it so it ain't gonna happen.
1199 2012-11-15 21:20:13 freakazoid has quit (Ping timeout: 246 seconds)
1200 2012-11-15 21:20:23 <sipa> the only problem now becomes making it convenient, i.e. automating the receive/sending of payment descriptors
1201 2012-11-15 21:20:58 <gavinandresen> hang on, I still see problems with the security of the "just upload a file to SD" solution
1202 2012-11-15 21:21:22 <gavinandresen> e.g. what stops a MITM from changing the refund address in the data I'm sending to them?  we assuming a https connection?
1203 2012-11-15 21:21:45 <sipa> the metadata is signed by an input address
1204 2012-11-15 21:23:04 <gavinandresen> mmm.  Ok...
1205 2012-11-15 21:24:53 <sipa> my first concern is making it convenient to get transactions sent immediately to their destination
1206 2012-11-15 21:25:13 <sipa> using a p2p network is one choice for this, but in most cases not the best one imho
1207 2012-11-15 21:25:35 <gavinandresen> why not?  We know that buyers and sellers are connected to the p2p network
1208 2012-11-15 21:25:49 <sipa> it's slow and unreliable
1209 2012-11-15 21:26:06 <gavinandresen> payment initiation could include a "here's a address:port you should add as a peer to connect directly to me"
1210 2012-11-15 21:26:14 <gavinandresen> (or host:port or whatever)
1211 2012-11-15 21:26:38 <sipa> how is that different from it being a url, and not using the p2p network altogether?
1212 2012-11-15 21:27:05 <sipa> imho a client device shouldn't even need to be connected to the p2p network at all
1213 2012-11-15 21:27:10 <gavinandresen> if we are communicating via both https and the p2p network then that's double the attack surface, double the potential to screw up the security and have bugs, etc
1214 2012-11-15 21:27:11 <sipa> if you only want to send
1215 2012-11-15 21:27:35 <sipa> meh, please
1216 2012-11-15 21:27:37 <Luke-Jr> sipa: using p2p makes it easier for "normal users" to accept payments equally
1217 2012-11-15 21:28:01 <sipa> well my argument is that the p2p network is a bad choicr for receivig payments
1218 2012-11-15 21:28:17 <gavinandresen> sipa: I think you're confounding extremely lightweight clients (who will connect to something like an electrum server) with payment protocol
1219 2012-11-15 21:28:46 <gavinandresen> it is a bad choice for receiving payments?  now I'm really confused....  that's what we're starting with!
1220 2012-11-15 21:29:01 <sipa> because it is slow, unreliable, and sending metadata over it requires getting that data forwarded by nodes who have no business with it
1221 2012-11-15 21:29:03 <Luke-Jr> sipa: most people aren't going to want to spend $$$ on a SSL cert to get paid by their friends
1222 2012-11-15 21:29:44 <sipa> sending abpayment somewhere doesn't require authentication at all
1223 2012-11-15 21:29:55 <sipa> the transaction+metadata is a signed package
1224 2012-11-15 21:30:46 pusle has quit ()
1225 2012-11-15 21:30:54 <sipa> gavinandresen: did you read my mail? i believe getting the transaction to the destination quickly and reliably is the first thing you want
1226 2012-11-15 21:31:11 <sipa> before that, there is no transaction with anyone
1227 2012-11-15 21:31:36 <sipa> and if that is the case, it can become the worry of the receiver to getbit confirmed
1228 2012-11-15 21:32:40 <gmaxwell> I've sort of been struggling with that thread because there is an obvious communication disconnect but I don't know how to clarify it.
1229 2012-11-15 21:32:41 <gavinandresen> sipa: in my practical experience, if sender and receiver are both connected to the p2p network, transactions take at most a few seconds to get propagated.  Is your experience different?
1230 2012-11-15 21:33:08 <sipa> right now, sender and receiver both do retransmits as long as there is no confirmatiom, because the sender has no way of knowing whether it arrived or not
1231 2012-11-15 21:33:11 forsetifox has left ("Leaving")
1232 2012-11-15 21:33:17 <sipa> senders shouldn't need to worry about this
1233 2012-11-15 21:33:28 <gavinandresen> In any case, a "here's how to connect directly to me" IP:port would solve that trivially (in the initial payment info)
1234 2012-11-15 21:33:37 <sipa> they don't care about grtting the transactions confirmed
1235 2012-11-15 21:33:46 <gavinandresen> sure, sure.
1236 2012-11-15 21:33:53 <sipa> they only care about getting the transaction accepted by the receiver
1237 2012-11-15 21:34:08 slush1 has quit (Ping timeout: 260 seconds)
1238 2012-11-15 21:34:27 <gmaxwell> E.g. the example with tor and "are their direct to merchant https connections also through tor" supposition seems pointless to me.  If that was the problem then it's already game over when they got the address to pay to, or arranged the transaction in the first place.
1239 2012-11-15 21:34:32 freakazoid has joined
1240 2012-11-15 21:35:08 <sipa> gavinandresen: why require the semder to use the complicated p2p protocol to use it to just submit a transaction + metadata to exactly one place?
1241 2012-11-15 21:35:14 <gmaxwell> I don't bring it up to argue it out— only that the fact I can't make sense of why you think it is a sensible concern means that there is a clear failure to communicate.
1242 2012-11-15 21:36:18 <sipa> that's probably an overstatement i made
1243 2012-11-15 21:36:56 <sipa> my point is i think that we should considerbgetting the tx to the receiver a different problem than getting it in a block
1244 2012-11-15 21:37:36 <gavinandresen> maybe it would help to work through "satoshidice done right" thought experiment.
1245 2012-11-15 21:37:52 <gavinandresen> So:  I browse to https://satoshidice.com/
1246 2012-11-15 21:37:56 <gmaxwell> Well a point I'd make is that if we want to have fewer channels then the only way to minimize them is to send the txn over the merchant<>user channel and _not_ use the bitcoin network, unless you want to implement ebay inside the p2p network. :P
1247 2012-11-15 21:39:06 <gavinandresen> I don't want to open an account.  I'll trust satoshidice not to divulge my IP address and rat me out for gambling. (I'm punting on the using-tor use case)
1248 2012-11-15 21:39:52 <jgarzik> The solution people seem to be searching for is...  an OOB metadata communications channel that does not directly link two parties :)
1249 2012-11-15 21:39:58 <gavinandresen> So: I click on one of the bet addresses.  Get a payment request, launches Bitcoin-Qt, I send a transaction to some URI in the payment request.
1250 2012-11-15 21:40:05 <jgarzik> everything else is details
1251 2012-11-15 21:40:18 TD has quit (Quit: TD)
1252 2012-11-15 21:40:32 <sipa> gavinandresen: done
1253 2012-11-15 21:41:08 <gavinandresen> What does Bitcoin-Qt do when it sends the payment request?  It gets back a "you lost" message that is displayed automatically?
1254 2012-11-15 21:41:26 <sipa> gavinandresen: he... why not, even
1255 2012-11-15 21:41:33 <gavinandresen> ok.
1256 2012-11-15 21:41:43 <gmaxwell> gavinandresen: there are plenty of no account gambling sites already.  The site gives you an address to pay to.. you gamble until your done and either stop or hit send me my money...
1257 2012-11-15 21:41:54 <gavinandresen> Fine.  I add that URI to my address book and I can use it whenever I like.
1258 2012-11-15 21:42:05 <gmaxwell> and then they return your funds to an address you gave them (after some timeout in the stop case)
1259 2012-11-15 21:42:16 <gavinandresen> Then I discover tor, and configure Bitcoin-Qt to run over tor...
1260 2012-11-15 21:42:36 <sipa> gavinandresen: you're already browsing to SD's website!
1261 2012-11-15 21:42:39 <gavinandresen> ... and make another bet.  Am I connected via tor?
1262 2012-11-15 21:43:07 <gavinandresen> sipa: huh?  I don't have to browse there again, I added the bet address/URI to my address book.
1263 2012-11-15 21:43:10 <gmaxwell> basically the address you pay to is your 'account' ... but there is no signup, just send funds and push the button.  Also, it's a helluva lot easier to safely get a tor browser than tor bitcoin, fwiw.
1264 2012-11-15 21:44:00 <sipa> gavinandresen: well if you're behind tor, your communication with them will happen over tor
1265 2012-11-15 21:44:23 <sipa> gavinandresen: they can even have a tor hidden service URI if they value their customer's wish for anonimity
1266 2012-11-15 21:44:47 <phantomcircuit> https://bitcointalk.org/index.php?topic=124418.msg1336982#msg1336982
1267 2012-11-15 21:44:49 <phantomcircuit> aahah
1268 2012-11-15 21:44:50 <phantomcircuit> idiots
1269 2012-11-15 21:45:42 <gavinandresen> sipa: hang on, I want to explore the case of "I've got https://satoshidice.com/ in my address book as the payment gateway" case
1270 2012-11-15 21:45:58 <phantomcircuit> i feel like the degree to which bitcoin is experimental software isn't currently engrained into users enough
1271 2012-11-15 21:46:58 <sipa> gavinandresen: i'm not sure why this is an issue - the security/privacy associated with the URI is the same as the one you had when visiting their website for the first time
1272 2012-11-15 21:46:58 <gavinandresen> sipa: because it feels to me like having two pieces of software (your browser and Bitcoin-Qt) talk https: is likely to be troublesome.
1273 2012-11-15 21:47:29 <gavinandresen> sipa: am I misremembering-- I thought https: connections over tor were https: between the exit node and the website, not end-to-end
1274 2012-11-15 21:47:43 <sipa> gavinandresen: hell no
1275 2012-11-15 21:47:45 <gmaxwell> 0_o You're misrepembering.
1276 2012-11-15 21:47:54 <gavinandresen> ok, good.
1277 2012-11-15 21:48:42 <D34TH> time to load test bitcoin on a ssd with a 10gbit connection
1278 2012-11-15 21:49:05 <sipa> anyway, i have a headache and i have stuff to do before going home tomorrow
1279 2012-11-15 21:49:12 agricocb has quit (Quit: Leaving.)
1280 2012-11-15 21:49:27 <gavinandresen> sipa: thanks, talk to you later...
1281 2012-11-15 21:49:34 <sipa> probably a bad time to keep discussing things :)
1282 2012-11-15 21:50:14 <Luke-Jr> phantomcircuit: just because something is experimental doesn't mean you're free to DoS it
1283 2012-11-15 21:50:32 <phantomcircuit> Luke-Jr, lol i only attacked my own nodes
1284 2012-11-15 21:50:46 <phantomcircuit> if i hadn't the network would have asploded
1285 2012-11-15 21:51:03 <Luke-Jr> phantomcircuit: that was lost in the paste
1286 2012-11-15 21:51:24 <phantomcircuit> yes on purpose because people are specifically targetting me
1287 2012-11-15 21:51:32 <Luke-Jr> I don't think (some of) the responses were too inappropriate without that context
1288 2012-11-15 21:51:32 <phantomcircuit> notice how few posts the people in that thread ahve
1289 2012-11-15 21:51:42 <Luke-Jr> sure, but blame where it belongs
1290 2012-11-15 21:51:51 <Luke-Jr> problem here is the QOOC
1291 2012-11-15 21:52:01 <Luke-Jr> nothing to do with bitcoin being experimental etc
1292 2012-11-15 21:54:03 <cjd> I would tend to avoid saying things which look good framed in a small quote, if you talk about limitations of buffers and connection slots then people who don't understand what it means of it won't get scared and confused... and won't make you into a villin
1293 2012-11-15 21:54:09 <phantomcircuit> peoples reaction is indicative of not understanding exactly how risky having substantial wealth in bitcoin is
1294 2012-11-15 21:54:38 <phantomcircuit> which to be clear is lol crazy
1295 2012-11-15 21:55:19 gavinandresen has quit (Quit: gavinandresen)
1296 2012-11-15 21:55:24 CodesInChaos has quit (Ping timeout: 276 seconds)
1297 2012-11-15 21:56:11 <cjd> idk, I doubt you could do too much damage for long, worst case scenario lots of nodes crash or the chain gets wacky, devs spot the issue and push out a fix, miners upgrade and your fake blocks go away, everything is back to normal within a week...
1298 2012-11-15 21:56:50 <cjd> IMO that's the best safety feature bitcoin has
1299 2012-11-15 21:57:11 ThomasV_ has quit (Quit: Quitte)
1300 2012-11-15 21:57:31 <phantomcircuit> cjd, no not really the attack i've proposed would require pretty much everybody to update software essentially immediately
1301 2012-11-15 21:57:40 <phantomcircuit> the issue is that a sybil attack can lead to a forked network
1302 2012-11-15 21:57:41 <Luke-Jr> I'd never endorse a DDoS, but if someone were to do one, it'd be nice to see the p2pool "DDoS resistant" myth shattered ;p
1303 2012-11-15 21:57:49 <phantomcircuit> a forked network can be leveraged into a forked chain
1304 2012-11-15 21:58:03 TD has joined
1305 2012-11-15 21:58:36 <phantomcircuit> and you dont need to actually cause the majority of the miners to disconnect as long as you're patient and go after automated peers that wouldn't notice the rate between blocks dropping dramatically
1306 2012-11-15 21:58:40 <cjd> timesync schenanigans?
1307 2012-11-15 21:58:58 <phantomcircuit> cjd, no
1308 2012-11-15 21:59:02 <phantomcircuit> let me try again
1309 2012-11-15 21:59:06 d4de has quit (Ping timeout: 245 seconds)
1310 2012-11-15 21:59:18 <cjd> I won't notice it, I'm not even running a node
1311 2012-11-15 21:59:18 <phantomcircuit> you ddos all connectable nodes
1312 2012-11-15 21:59:24 <phantomcircuit> except your own node
1313 2012-11-15 21:59:39 <phantomcircuit> which effectively forces the majority of clients to connect through you
1314 2012-11-15 21:59:48 <cjd> oh actual ddos like packet flood stuff that gets you in the big house?
1315 2012-11-15 22:00:09 <phantomcircuit> cjd, or you just connect a lot of times which theoretically gets the same punishment as lots of packets
1316 2012-11-15 22:00:21 <phantomcircuit> the problem here is that lots of connections is trivial to do
1317 2012-11-15 22:00:38 <phantomcircuit> a packet storm against 3k + hosts many of which are servers on 100mbps lines wouldn't be so trivial
1318 2012-11-15 22:00:53 <cjd> and would step on a lot of toes
1319 2012-11-15 22:00:55 <cjd> but yeah
1320 2012-11-15 22:01:01 <phantomcircuit> but
1321 2012-11-15 22:01:07 <phantomcircuit> if you can force nodes to connect through you
1322 2012-11-15 22:01:12 <cjd> then once everyone is connected to you you withhold blocks and fun stuff like that?
1323 2012-11-15 22:01:14 <phantomcircuit> you just start your own forks
1324 2012-11-15 22:01:21 <phantomcircuit> and tell one half of the network oen thing
1325 2012-11-15 22:01:24 <phantomcircuit> and the other the other thing
1326 2012-11-15 22:02:00 <phantomcircuit> (or however many subdivisions you want but the more there are the more mining capacity you'd need to generate blocks at a pace that your forks would ever get to 6 confirmations)
1327 2012-11-15 22:02:46 <phantomcircuit> so yeah it's an annoying attack
1328 2012-11-15 22:02:46 <cjd> ofc this would be insanely noticable
1329 2012-11-15 22:02:59 <phantomcircuit> but it's also possible that you could pull off a double spend
1330 2012-11-15 22:03:15 <phantomcircuit> this is actually the only practical double spend i've yet to see
1331 2012-11-15 22:03:28 <TD> how would you start a fork of the chain
1332 2012-11-15 22:03:29 <phantomcircuit> other attacks have basically just caused the client to freeze
1333 2012-11-15 22:03:50 <cjd> I think the idea is to divide the miners up into groups
1334 2012-11-15 22:03:51 <TD> even if you can force everyone to listen to your announcements and nobody elses, you can't just start arbitrarily adding blocks to the chain
1335 2012-11-15 22:04:02 <phantomcircuit> TD, you segment the network isolating each client from miners and simply drop all blocks which you didn't mine
1336 2012-11-15 22:04:23 <phantomcircuit> you can then mine your forks at a leisurely pace
1337 2012-11-15 22:04:41 <cjd> heh that takes quite a bit of hashpower if you're doing all the mining yourself
1338 2012-11-15 22:05:01 <phantomcircuit> you'd have a trade off between hashpower and patience
1339 2012-11-15 22:05:03 <TD> and you can mine new blocks how often? if your goal is to double spend that's going to fail, given how long it'll take you to mine blocks. without a doubt many merchants will notice or hear that something has gone wrong
1340 2012-11-15 22:05:10 <gmaxwell> I didn't see the past conversation, but I would point out if you can partition the network you don't have to mine.
1341 2012-11-15 22:05:14 <TD> and whilst you're mounting a huge and very visible DoS attack the cops are hunting you
1342 2012-11-15 22:05:23 <gmaxwell> You partition and feed each partition what you like and it mines _for_ you.
1343 2012-11-15 22:05:38 <phantomcircuit> gmaxwell, that's also true
1344 2012-11-15 22:05:48 <TD> assuming miners don't notice the network is suddenly FUBAR and shut down
1345 2012-11-15 22:05:52 <gmaxwell> But I don't believe phantomcircuit's attack results in a partitioning of miners, and if it does that needs to be fixed by peering, not software first.
1346 2012-11-15 22:06:01 <TD> this doesn't seem like the lowest hanging fruit, attak wise
1347 2012-11-15 22:06:09 <phantomcircuit> TD, im actually confident you could run this attack entirely through tor
1348 2012-11-15 22:06:15 <phantomcircuit> because well that's how i attacked myself
1349 2012-11-15 22:06:24 <gmaxwell> TD: I think it's mostly safe to assume miners won't shut down.   "I dunno whats going on BUT I'mma gonna get all the blocks!" :P
1350 2012-11-15 22:06:26 <phantomcircuit> so no it's pretty unlikely you'd get caught
1351 2012-11-15 22:06:29 <TD> Tor gives you a very limited set of exit IPs, which are helpfully provided in a list
1352 2012-11-15 22:06:38 * gmaxwell away again
1353 2012-11-15 22:06:39 <TD> so nodes can block them all, easily enough if they want
1354 2012-11-15 22:06:50 <cjd> it would be a very noisy attack which would take a long time to bear fruit and wouldn't give you much if it did
1355 2012-11-15 22:07:00 <TD> yeah
1356 2012-11-15 22:07:07 <TD> all it'd do is push merchants and users away from bitcoin
1357 2012-11-15 22:07:10 <phantomcircuit> TD, it would be a fairly noisy attack but i actually doubt anybody mounting it would get caught
1358 2012-11-15 22:07:18 <TD> by the time you managed to double spend, there wouldn't be many merchants left who'd accept bitcoin anyway
1359 2012-11-15 22:07:18 <phantomcircuit> of course now i would have the FBI knocking on my door
1360 2012-11-15 22:07:20 <phantomcircuit> lol
1361 2012-11-15 22:07:26 <phantomcircuit> but i dont plan on doing it
1362 2012-11-15 22:07:26 <nibor> But if I was MtGox and was seeing massive drop in number of blocks per hour I would drop daily withdraw limits
1363 2012-11-15 22:07:29 <TD> yeah. i mean, everyone who does DoS attacks thinks that
1364 2012-11-15 22:07:33 <TD> somehow people still end up in jail for it
1365 2012-11-15 22:07:49 <TD> nibor: yeah. there has often been talk of introducing safe modes triggered by sudden drops in mining speed
1366 2012-11-15 22:07:57 <TD> it's easy to implement. just never been done.
1367 2012-11-15 22:08:01 <phantomcircuit> TD, i guess i should say you could mount this attack without getting caught
1368 2012-11-15 22:08:10 <TD> sure
1369 2012-11-15 22:08:44 <nibor> Problem is can you get the BTC into USD before the network blacklists your IP and recombines
1370 2012-11-15 22:08:44 <phantomcircuit> and i believe you could do it using the practically unlimited supply of open proxies online
1371 2012-11-15 22:08:52 <TD> heh
1372 2012-11-15 22:09:00 <TD> random free open proxies are often unbelievably shitty
1373 2012-11-15 22:09:05 <TD> i mean, theoretically any crime can be "perfect"
1374 2012-11-15 22:09:21 <TD> fortunately criminals are often not the sharpest knives in the drawer, which is why police can do their jobs
1375 2012-11-15 22:09:36 <TD> a huge number of criminals seem to get caught because they make basic mistakes
1376 2012-11-15 22:09:39 <denisx> once I had root on the piratebay trackers and we changed the tracker so that it always added a specific address to it. instead of one machine we took the complete ISP down ;)
1377 2012-11-15 22:09:54 <TD> look at the romney thing. it seems very likely knightmb will end up in court over that, maybe found guilty.
1378 2012-11-15 22:10:10 <phantomcircuit> TD, well the thing is you could just run tor -> proxy relays
1379 2012-11-15 22:10:29 <phantomcircuit> of course someone would get caught because they'd get drunk and brag
1380 2012-11-15 22:10:36 <TD> in theory any DoS attack can be done via Tor. in practice it never seems to happen
1381 2012-11-15 22:10:44 <TD> maybe due to latency or bandwidth issues
1382 2012-11-15 22:10:50 <TD> it's not like Tor is fast at the best of times
1383 2012-11-15 22:10:52 <nibor> And assuming motivation is USD you have got to then launder that after get half the double spend out of MtGox and the other half out of another exchange.
1384 2012-11-15 22:11:25 <phantomcircuit> TD, it never happens because criminals are stupid
1385 2012-11-15 22:11:40 <phantomcircuit> there are two default settings in tor which prevent DDoS being launched using tor
1386 2012-11-15 22:11:44 <cjd> actually it's probably more efficient to run the attack as part of an extortion scheme..  holding people's sockets for ransom
1387 2012-11-15 22:11:49 <phantomcircuit> for the sake of the worlds sanity i wont say what they are
1388 2012-11-15 22:12:32 <cjd> but then you might step on toes of people who are much better at this than you or I and they might not take kindly to your efforts
1389 2012-11-15 22:12:50 <phantomcircuit> cjd, lol extortion
1390 2012-11-15 22:12:55 <phantomcircuit> why do people even try
1391 2012-11-15 22:13:05 <phantomcircuit> that's such an obviously dumb plan
1392 2012-11-15 22:13:20 BlackPrapor has joined
1393 2012-11-15 22:13:23 <cjd> I didn't say it was a *good* plan, just better than partitioning to double spend
1394 2012-11-15 22:13:53 <jgarzik> sipa: pull req up
1395 2012-11-15 22:14:07 <jgarzik> next up: optimistic writes, and no-select-if-write-queued
1396 2012-11-15 22:14:45 <jgarzik> P2P: parse network datastream into header/data components in socket thread - https://github.com/bitcoin/bitcoin/pull/2016
1397 2012-11-15 22:15:03 nibor_ has joined
1398 2012-11-15 22:17:16 <cjd> do you try to resync if the magic is bad or just close the socket?
1399 2012-11-15 22:17:58 ovidiusoft has quit (Ping timeout: 248 seconds)
1400 2012-11-15 22:18:23 nibor has quit (Ping timeout: 264 seconds)
1401 2012-11-15 22:18:34 <sipa> cjd: resync
1402 2012-11-15 22:18:37 <sipa> up to now, at least
1403 2012-11-15 22:19:34 * cjd reads the patch ^^
1404 2012-11-15 22:20:17 <sipa> jgarzik: so basically, you moved the parsing into network packets from processmessages to cnode?
1405 2012-11-15 22:20:33 <sipa> s/network packets/protocol messages/
1406 2012-11-15 22:21:43 <jgarzik> sipa: hdr/data, yes.  ProcessMessages() is still the one that inspects CMessageHeader contents
1407 2012-11-15 22:22:16 <jgarzik> if we want to eliminate that 64k receive buffer completely (another data copy), we can increase recv(2) syscalls
1408 2012-11-15 22:24:06 <jgarzik> i.e. recv() message header, then recv() directly into final vRecv buffer
1409 2012-11-15 22:24:27 <jgarzik> rather than recv(pchBuf) then copy into vRecv
1410 2012-11-15 22:24:34 <jgarzik> still, it is more efficient than HEAD either way
1411 2012-11-15 22:25:20 <sipa> why deal with the header separately?
1412 2012-11-15 22:25:29 <jgarzik> sipa: precisely size data buffer
1413 2012-11-15 22:25:42 <sipa> ah, right
1414 2012-11-15 22:26:26 <sipa> well, what is more expensive? an extra recv(), or copying 64 KiB?
1415 2012-11-15 22:26:54 <sipa> i'd say the copy
1416 2012-11-15 22:27:02 <sipa> hmm, unsure
1417 2012-11-15 22:27:21 freakazoid has quit (Ping timeout: 260 seconds)
1418 2012-11-15 22:27:39 <jgarzik> sipa: with my change, we can easily move to recv()/recv() direct later
1419 2012-11-15 22:27:51 D34TH has quit (Quit: Leaving)
1420 2012-11-15 22:28:37 <jgarzik> yeah, unsure which is better... multiple context switches + less data copying, or fewer context switches and extra data copying
1421 2012-11-15 22:29:02 <jgarzik> picocoin does multiple-recv(), but that does not answer the question
1422 2012-11-15 22:30:29 <phantomcircuit> almost certainly a single 64KB copy is cheaper than an additional recv
1423 2012-11-15 22:31:05 <phantomcircuit> possibly it would be faster on processors with > 64KiB of D-Cache
1424 2012-11-15 22:31:23 <fiesh> recv() causes an interrupt, doesn't it?
1425 2012-11-15 22:31:35 <sipa> i don't think so
1426 2012-11-15 22:31:48 <cjd> SYSENTER is faster than the old interrupt method
1427 2012-11-15 22:32:06 <jgarzik> fiesh: no
1428 2012-11-15 22:32:15 <sipa> 50=100 ns for a syscall
1429 2012-11-15 22:32:20 <sipa> 50-100ns
1430 2012-11-15 22:32:39 <jgarzik> you're just context switching, to do a kernel->user memcpy
1431 2012-11-15 22:32:52 <sipa> (well, that's for gettid, but recv with nowait shouldn't be much slower, and the actual data amount copied by recv is the same in both cases)
1432 2012-11-15 22:32:52 <fiesh> oh
1433 2012-11-15 22:32:59 <jgarzik> phantomcircuit: it is 64K * size of message
1434 2012-11-15 22:33:02 <jgarzik> not just 64k
1435 2012-11-15 22:33:07 <cjd> iirc they have some trickery to avoid some of the context switch overhead in new syscalls too
1436 2012-11-15 22:33:12 <jgarzik> recv() will try to chomp entire message
1437 2012-11-15 22:33:21 <jgarzik> currently we recv(64k), recv(64k), ....
1438 2012-11-15 22:33:31 <phantomcircuit> jgarzik, ah so it is probably 64k + 1MB fairly frequently
1439 2012-11-15 22:33:51 <phantomcircuit> still though i cant imagine the extra recv() would be faster than even a 1MB copy on a modern processor
1440 2012-11-15 22:33:53 <sipa> jgarzik: how large are the kernel tcp/othernetstuff buffers?
1441 2012-11-15 22:34:23 <phantomcircuit> i would say test it but something tells me a benchmark wouldn't show a measurable difference
1442 2012-11-15 22:34:24 <phantomcircuit> lol
1443 2012-11-15 22:34:34 <cjd> depends on the kernel, don't expect much... *cough* windows
1444 2012-11-15 22:34:47 <edcba> sipa: |------|
1445 2012-11-15 22:34:55 <sipa> edcba: which scale?
1446 2012-11-15 22:34:58 <jgarzik> sipa: usually 128k.  cat /proc/sys/net/ipv4/tcp_mem
1447 2012-11-15 22:34:59 <sipa> 1:1 ?
1448 2012-11-15 22:35:01 <edcba> about 2
1449 2012-11-15 22:35:03 <phantomcircuit> sipa, variable length
1450 2012-11-15 22:35:06 <jgarzik> sipa: however, we can always set the socket buffer size larger...
1451 2012-11-15 22:35:23 <sipa> jgarzik: so does it make sense to try to read significantly more than 64K?
1452 2012-11-15 22:35:25 <jgarzik> sipa: that's a one-time syscall at socket setup time
1453 2012-11-15 22:35:31 <sipa> ah, nice
1454 2012-11-15 22:35:35 <jgarzik> sipa: open question :)
1455 2012-11-15 22:35:48 <cjd> http://blog.tsunanet.net/2010/11/how-long-does-it-take-to-make-context.html <-- according to this guy, syscall != context switch
1456 2012-11-15 22:36:04 <sipa> cjd: i hit the same page :)
1457 2012-11-15 22:36:18 <jgarzik> sipa: regardless of this recv() buffer copying question, seems like our socket buffer should fit a large "block" message at least
1458 2012-11-15 22:36:24 <jgarzik> which means increasing from current Linux default
1459 2012-11-15 22:36:24 eroot has joined
1460 2012-11-15 22:36:44 agricocb has joined
1461 2012-11-15 22:36:59 <sipa> what speeds does L2 cache reach these days?
1462 2012-11-15 22:37:27 <phantomcircuit> cjd, only a tiny % of syscalls dont result in a context switch
1463 2012-11-15 22:38:08 <phantomcircuit> sipa, ludicrous speed
1464 2012-11-15 22:38:39 <cjd> probably best to optimize for the common case which is transactions, getdata's and invs, if the average message is a few kb then the copy is almost certainly cheaper
1465 2012-11-15 22:39:04 <phantomcircuit> sipa, something like 7 ns
1466 2012-11-15 22:39:12 <denisx> is one of the cgminer devs available?
1467 2012-11-15 22:39:44 <phantomcircuit> it depends on whether the cache line is shared and something else i forget
1468 2012-11-15 22:40:07 <cjd> I recall a report on someone's profiling of a VPN app and memcpy was 150(whatever unit) and the syscall was 3000
1469 2012-11-15 22:40:35 d4de has joined
1470 2012-11-15 22:40:36 d4de has quit (Changing host)
1471 2012-11-15 22:40:36 d4de has joined
1472 2012-11-15 22:40:49 <sipa> denisx: #cgminer ?
1473 2012-11-15 22:40:59 <denisx> really? ;)
1474 2012-11-15 22:41:00 <phantomcircuit> cjd, that sounds about right but it's even worse than that
1475 2012-11-15 22:41:00 <denisx> ok
1476 2012-11-15 22:41:22 <jgarzik> for small messages, currently with the 64k internal buffer, one recv(2) will slurp multiple headers + data
1477 2012-11-15 22:41:29 <phantomcircuit> http://www.cs.rochester.edu/u/cli/research/switch.pdf
1478 2012-11-15 22:41:37 <jgarzik> removing the 64k buffer means 2x recv(2) for each message, small or large
1479 2012-11-15 22:41:55 freakazoid has joined
1480 2012-11-15 22:42:08 <phantomcircuit> http://blog.tsunanet.net/2010/11/how-long-does-it-take-to-make-context.html
1481 2012-11-15 22:42:24 <phantomcircuit> so yeah i think the answer is that the memcpy is going to be faster
1482 2012-11-15 22:42:27 <phantomcircuit> even for 1MB buffers
1483 2012-11-15 22:43:06 <sipa> does recv do a real context switch, or just a mode switch?
1484 2012-11-15 22:43:39 <phantomcircuit> i would assume a context switch but i guess it could just mode switch
1485 2012-11-15 22:44:29 <phantomcircuit> lol run cscope on the entire kernel to find the definition of recv()
1486 2012-11-15 22:44:36 <phantomcircuit> this'll only take forever
1487 2012-11-15 22:45:11 <cjd> unfortunately you're probably not going to get many messages from a given node at a time, the expense is from having to recv from each socket which can be a sizable number
1488 2012-11-15 22:46:54 <sipa> jgarzik: you could always go for a hybrid: first recv into a buffer (which is maybe even less than 64 KiB in size, if most messages are less than 64 KiB); if an entire message is read, copy it; if not, allocate message buffer and do subsequent read directly in message buffer
1489 2012-11-15 22:47:10 <jgarzik> sipa: true...
1490 2012-11-15 22:47:19 <phantomcircuit> cjd, well theoretically select should make it so each call to recv returns data
1491 2012-11-15 22:47:28 <cjd> yeap
1492 2012-11-15 22:47:30 <sipa> though dealing with partially-read messages would add some complexity
1493 2012-11-15 22:47:45 <jgarzik> sipa: that's already handled, with my pull request.  not a problem.
1494 2012-11-15 22:47:51 <phantomcircuit> and since bitcoin doesn't disable nagle typically recv() should result in full messages
1495 2012-11-15 22:47:59 <phantomcircuit> especially with the user mode buffering
1496 2012-11-15 22:48:59 JZavala has quit (Ping timeout: 245 seconds)
1497 2012-11-15 22:49:24 <jgarzik> sipa: CNetMessage already knows whether it's in header or data portion of message, and already (as it must) handles partial messages
1498 2012-11-15 22:50:52 <jgarzik> phantomcircuit: nagle only counts for small messages
1499 2012-11-15 22:51:21 <jgarzik> phantomcircuit: for larger messages, you are looking at "block" messages larger than the kernel socket buffer size
1500 2012-11-15 22:51:23 <phantomcircuit> well at the very least the user mode write buffer does it :)
1501 2012-11-15 22:52:49 <cjd> someone should write a raw packet library which uses mmap() socket if available then with that and userspace tcp these problems could be put to rest once and for all
1502 2012-11-15 22:53:43 <phantomcircuit> pretty sure such a thing exists
1503 2012-11-15 22:54:06 <cjd> hmm if you have the name of it, I'm interested
1504 2012-11-15 22:54:10 <phantomcircuit> some of the hft's have nic's that put raw packets directly into usermode memory
1505 2012-11-15 22:54:13 <cjd> even if you don't I'm interested
1506 2012-11-15 22:55:00 D34TH has joined
1507 2012-11-15 22:55:26 <cjd> mmap()'ing a socket is a standard kernel thing, you don't need an e1000e nic for it, it might not be *that* fast but it will give you a way to read multiple packets at once
1508 2012-11-15 22:55:35 <jgarzik> yeah, the great and mighty Van Jacobsen did a userspace TCP, even
1509 2012-11-15 22:55:40 <jgarzik> truly zero copy
1510 2012-11-15 23:12:06 guruvan has quit (Ping timeout: 276 seconds)
1511 2012-11-15 23:12:22 DutchBrat has joined
1512 2012-11-15 23:12:26 guruvan- is now known as guruvan
1513 2012-11-15 23:14:47 Internet13 has quit (Ping timeout: 264 seconds)
1514 2012-11-15 23:14:47 DutchBrat_ has quit (Ping timeout: 264 seconds)
1515 2012-11-15 23:17:47 Internet13 has joined
1516 2012-11-15 23:18:36 guruvan has quit (Ping timeout: 276 seconds)
1517 2012-11-15 23:22:59 da2ce7 has joined
1518 2012-11-15 23:23:01 <phantomcircuit> jgarzik, either way this is all sort of silly
1519 2012-11-15 23:23:12 maaku has joined
1520 2012-11-15 23:23:13 <phantomcircuit> the biggest speed up would be from parallelism in transacation verification
1521 2012-11-15 23:23:17 <phantomcircuit> but that's hard :)
1522 2012-11-15 23:23:51 <sipa> phantomcircuit: done
1523 2012-11-15 23:24:20 <sipa> (though there were a few problems with it, and i decided to want some refactoring first, so it's not merged)
1524 2012-11-15 23:26:06 <phantomcircuit> what strategy did you take?
1525 2012-11-15 23:26:35 OffTopic has joined
1526 2012-11-15 23:26:39 <phantomcircuit> pipeline applies obvious rules/runs the script
1527 2012-11-15 23:26:46 MC1984 has quit (Ping timeout: 268 seconds)
1528 2012-11-15 23:26:48 <sipa> oh, just the script verification
1529 2012-11-15 23:26:48 <phantomcircuit> then the transaction is merged into the memory pool or tree
1530 2012-11-15 23:26:49 <phantomcircuit> ?
1531 2012-11-15 23:26:50 <OffTopic> Want to borrow 10 bitcoins paying 13.5 on or before November 21st 2012
1532 2012-11-15 23:26:52 <OffTopic> Want to borrow 10 bitcoins paying 13.5 on or before November 21st 2012
1533 2012-11-15 23:26:53 <OffTopic> Want to borrow 10 bitcoins paying 13.5 on or before November 21st 2012
1534 2012-11-15 23:26:53 <OffTopic> Want to borrow 10 bitcoins paying 13.5 on or before November 21st 2012
1535 2012-11-15 23:26:53 <OffTopic> Want to borrow 10 bitcoins paying 13.5 on or before November 21st 2012
1536 2012-11-15 23:26:54 <OffTopic> Want to borrow 10 bitcoins paying 13.5 on or before November 21st 2012
1537 2012-11-15 23:26:54 <OffTopic> Want to borrow 10 bitcoins paying 13.5 on or before November 21st 2012
1538 2012-11-15 23:26:55 <OffTopic> Want to borrow 10 bitcoins paying 13.5 on or before November 21st 2012
1539 2012-11-15 23:26:55 <OffTopic> Want to borrow 10 bitcoins paying 13.5 on or before November 21st 2012
1540 2012-11-15 23:26:55 <OffTopic> Want to borrow 10 bitcoins paying 13.5 on or before November 21st 2012
1541 2012-11-15 23:26:59 <OffTopic> Want to borrow 10 bitcoins paying 13.5 on or before November 21st 2012
1542 2012-11-15 23:27:09 <phantomcircuit> i swear he's actually retarded
1543 2012-11-15 23:27:14 <sipa> phantomcircuit: created a script-verification-action object, which got pushed on a queue by the normal block verification logic
1544 2012-11-15 23:27:39 <sipa> and separate threads performed the queued script checks
1545 2012-11-15 23:27:46 <phantomcircuit> how do you verify the block is only verified once all txs are verified?
1546 2012-11-15 23:28:35 <sipa> it's just not marked verified before the last script in it is verified
1547 2012-11-15 23:28:46 <sipa> the queue keeps a counter per-block of how many scripts are left
1548 2012-11-15 23:28:52 <phantomcircuit> ah
1549 2012-11-15 23:28:56 <sipa> and when that counter hits 0, the block is announced and stuff
1550 2012-11-15 23:28:56 <phantomcircuit> there it is :)
1551 2012-11-15 23:29:14 <phantomcircuit> that's actually a clever way of doing it
1552 2012-11-15 23:29:21 <phantomcircuit> what happens if a tx is invalid?
1553 2012-11-15 23:29:29 <sipa> the block is marked invalid
1554 2012-11-15 23:29:33 <sipa> and reorged away form
1555 2012-11-15 23:29:59 <sipa> there's a nifty connectbestblock now, which finds the best block to connect to, taking invalidnesses into account
1556 2012-11-15 23:31:45 <phantomcircuit> sipa, im impressed by your ability to refactor code that's total non-sense
1557 2012-11-15 23:31:54 ibno has quit (Quit: Lämnar)
1558 2012-11-15 23:32:06 <phantomcircuit> i usually get up the urge to fix something look at the mess that is (was?) bitcoind and just give up
1559 2012-11-15 23:32:09 <sipa> phantomcircuit: you have no idea how much time i've already spent hacking that code :)
1560 2012-11-15 23:32:22 <phantomcircuit> that's my point
1561 2012-11-15 23:32:33 <phantomcircuit> i do have a vague idea
1562 2012-11-15 23:32:53 <sipa> apart from some of the networking logic, i pretty much feel "at home" everywhere, i think
1563 2012-11-15 23:32:58 guruvan has joined
1564 2012-11-15 23:33:22 guruvan- has joined
1565 2012-11-15 23:41:01 copumpkin has quit (Quit: Computer has gone to sleep.)
1566 2012-11-15 23:43:45 <sipa> jgarzik: requiring the send buffer to go entirely empty before reading anything will delay IBD, i'd think?
1567 2012-11-15 23:43:52 RazielZ has quit (Ping timeout: 246 seconds)
1568 2012-11-15 23:45:14 <sipa> though you can decide not to read when the send buffer is over some treshold
1569 2012-11-15 23:49:02 <phantomcircuit> ok so i have to admit the strategy bitcoin uses isn't that ridiculous
1570 2012-11-15 23:49:13 <phantomcircuit> if othere peers weren't trying to break you
1571 2012-11-15 23:49:24 <phantomcircuit> the whole fetch then process later strategy would be sound
1572 2012-11-15 23:49:45 <phantomcircuit> it's actually vaguely similar to the way some forex markets work... but anyways
1573 2012-11-15 23:59:07 denisx_ has joined