1 2017-03-12 04:05:25	0|Chris_Stewart_5|Anyone familiar with this error? crypto/common.h:48:12: runtime error: load of misaligned address 0x7fc1ea516169 for type 'uint32_t'
 2 2017-03-12 04:05:35	0|Chris_Stewart_5|Out of memory error possibly?
 3 2017-03-12 04:06:29	0|gwillen|hmmm, that doesn't sound good
 4 2017-03-12 04:08:20	0|Chris_Stewart_5|gwillen: Seems strange, all tests are passing that I am running but this seems to be happening in the setup/tear down process
 5 2017-03-12 04:09:11	0|gwillen|what is it that you're working on that's erroring -- bitcoin-core? Or a private branch thereof?
 6 2017-03-12 04:09:38	0|gwillen|that pointer makes me nervous, the last 3 bytes are alphabetic characters in ascii, and a pointer should never be odd
 7 2017-03-12 04:09:53	0|gwillen|which can mean the pointer has been partly overwritten with garbage
 8 2017-03-12 04:10:40	0|Chris_Stewart_5|it is branch on my repo, I can link if you want.
 9 2017-03-12 04:11:16	0|Chris_Stewart_5|the gist of it is generating random permutations of datastructures and testing invariants hold true on it
10 2017-03-12 04:12:00	0|gwillen|feel free to link, although I suspect the problem is that you've introduced a bug somewhere in the form of overwriting memory erroneously
11 2017-03-12 04:12:29	0|Chris_Stewart_5|It is in #8469
12 2017-03-12 04:12:31	0|gribble|https://github.com/bitcoin/bitcoin/issues/8469 | [POC] Introducing property based testing to Core by Christewart · Pull Request #8469 · bitcoin/bitcoin · GitHub
13 2017-03-12 04:15:03	0|Chris_Stewart_5|Happening some where in between this generator: https://github.com/Christewart/bitcoin/blob/rapidcheck_mem_leak/src/test/gen/merkleblock_gen.h#L12-L30
14 2017-03-12 04:15:34	0|Chris_Stewart_5|and this test case: https://github.com/Christewart/bitcoin/blob/rapidcheck_mem_leak/src/test/merkleblock_properties.cpp#L26-L34
15 2017-03-12 04:16:21	0|gwillen|well, if the problem is a pointer being overwritten, or a use-after-free or something, the real cause of the error could be anywhere prior
16 2017-03-12 04:16:30	0|gwillen|this is the great sadness of debugging C code
17 2017-03-12 04:16:42	0|gwillen|what would be most useful in finding it, I think, would be finding what commit first introduced the problem
18 2017-03-12 04:17:46	0|Chris_Stewart_5|Yes, it is definitely introduced on this commit, the last commit i have on 8469 has everything working (on my machine). This pull request introduces a new dependency that we haven't set up yet :-(
19 2017-03-12 04:19:35	0|gwillen|Chris_Stewart_5: sorry, which commit introduces it? If it's on github, can you link directly to the specific commit? Or if it's not yet, can you push it somewhere I can see it?
20 2017-03-12 04:20:05	0|Chris_Stewart_5|https://github.com/Christewart/bitcoin/commit/a4052a8835f547e88c32bc6bb214b5bf108e42a7
21 2017-03-12 04:20:32	0|gwillen|thx, looking
22 2017-03-12 04:22:39	0|gmaxwell|Chris_Stewart_5: your error sounds clear enough, "load of misaligned address"   though I don't seeh how that makes sense with the code at common.h:48 on my system.
23 2017-03-12 04:24:24	0|Chris_Stewart_5|return be32toh(*((uint32_t*)ptr));
24 2017-03-12 04:24:30	0|Chris_Stewart_5|is what my line 48 is
25 2017-03-12 04:24:41	0|gmaxwell|okay thats not mine and that code looks potentially problmatic.
26 2017-03-12 04:24:47	0|gmaxwell|In C, type punning between arbritary types is not permitted. E.g. you cannot take a random character pointer, cast it to an int, and access it.  On most CPU architectures which have been invented (x86 and the more recent versions of ARM being the big obvious exceptions) it is forbidden to make a load which isn't naturally aligned, e.g. a load of a 4 byte type has to begin at a 4 byte multiple.
27 2017-03-12 04:25:33	0|gmaxwell|C(++) has special rules that you're allowed to access anything through a character pointer... and so you can use things like memcpy to make unaligned accesses by copying the randomly aligned pointer to a pointer of the right type (thus right alignment) elsewhere (e.g. on the stack.)
28 2017-03-12 04:26:30	0|gmaxwell|_My_ common.h here uses a memcpy, which would have been done specifically to avoid this problem. Are you using an old version of the code?  IIRC we fixed some issues with this not so long ago.
29 2017-03-12 04:27:08	0|Chris_Stewart_5|gmaxwell: Yes, I haven't rebased #8469 since I opened it IIRC
30 2017-03-12 04:27:10	0|gribble|https://github.com/bitcoin/bitcoin/issues/8469 | [POC] Introducing property based testing to Core by Christewart · Pull Request #8469 · bitcoin/bitcoin · GitHub
31 2017-03-12 04:27:12	0|gmaxwell|(Since I mentioned x86 above, even on x86 there are some instructions e.g. many of the fast SIMD loads, that require natural alignment... so you can't even just assume that since you're on x86 everything is going to be okay).
32 2017-03-12 04:27:35	0|gwillen|heh, I guess I was too quick to assume an unaligned load meant the pointer was the problem
33 2017-03-12 04:28:47	0|Chris_Stewart_5|really? You can't cast a char -> int and still access that piece of memory? Doesn't that make weak typing irrelevant then?
34 2017-03-12 04:28:59	0|gmaxwell|(And on older ARM systems, the CPU actually used the least significant bits of the pointer to trigger hardware byteswapping... so if you tried to read 32-bits from address 1, you'd really get a read of address 0 with the bytes swapped.. which was super duper fun to troubleshoot-- especially with all the x86 linux jockies writing alignment unsafe code, which is probably why arm dropped that rathe
35 2017-03-12 04:29:05	0|gmaxwell|r cool feature. :P)
36 2017-03-12 04:31:17	0|gmaxwell|Chris_Stewart_5: It's undefined behavior if the resulting pointer is not correctly aligned for its type.
37 2017-03-12 04:31:34	0|sipa|gmaxwell: not just alignment
38 2017-03-12 04:31:56	0|sipa|in C++ you just can't access any type through a type pointer that isn't compatible with it
39 2017-03-12 04:32:25	0|sipa|Chris_Stewart_5: you can cast an int to a char and back to int
40 2017-03-12 04:32:26	0|gmaxwell|sipa: I thought it was unconditionally, but googling didn't give me a cite so I said the more limited thing. :P
41 2017-03-12 04:33:41	0|gmaxwell|I also didn't mention, but even on x86 unaligned loads are sometimes non-trivially slower. (the main factor being if the load straddles a cache-line or page boundary, which is never the case for an aligned load)
42 2017-03-12 04:33:51	0|sipa|Chris_Stewart_5: and you can take a pointer to int, cast it to pointer to char, and access the data through the char pointer
43 2017-03-12 04:34:03	0|sipa|Chris_Stewart_5: which is a special exception that only exists for char
44 2017-03-12 04:34:37	0|sipa|but you can't create a char array, and then cast it to int pointer and then access it through the int pointer
45 2017-03-12 04:34:57	0|gmaxwell|so even if you were willing to ignore the C(++) standards and hope the compiler didn't screw you, and even if you were willing to only work on x86, and even if you were sure the result wouldn't use any alignment requiring vector loads... the result would still end up being potentially slower code. :P
46 2017-03-12 04:40:06	0|Chris_Stewart_5|So many little intracies in c/c++. Those casting semantics seem a little strange to me but I guess it is just the way it is :/
47 2017-03-12 04:40:39	0|gmaxwell|Funny, I like C because there are relatively few things to know!
48 2017-03-12 04:42:26	0|sipa|Chris_Stewart_5: it's not strange at all... you shouldn't see the program memory as an array of bytes, which you can arbitrarily treat as objects as you like
49 2017-03-12 04:42:49	0|sipa|Chris_Stewart_5: instead, the memory contains *objects* which have a well-defined type, and you're only allowed to access those objects through compatible pointers
50 2017-03-12 04:43:13	0|Chris_Stewart_5|gmaxwell: I think it boils down to 'X is easier because I learned X first! Y is waaaaay harder because I have to change the way I think' :-)
51 2017-03-12 04:43:29	0|gmaxwell|some of these things were more obvious in the past, objects of different kinds might get allocated out of entirely seperate chunks of memory which weren't even connected to the same hardware.
52 2017-03-12 04:43:31	0|sipa|note that many C/C++ codebases (and even bitcoin core before 0.14) violate that rule usually
53 2017-03-12 04:44:29	0|sipa|Chris_Stewart_5: these rules for example allow the C++ compiler to assume that two pointers of different type never refer to the same object, so a write through one can be known to never invalidate perhaps loaded-in-registers information from the other object
54 2017-03-12 04:48:32	0|gmaxwell|Chris_Stewart_5: dunno about that, the C language specification (at least for ansi C), which is more exact than even exists for most other language is very small (I think about 250 pages) compared to that of other language where their behavior is at all reproducable. (e.g. I think common lisp's spec is about 2000 pages).
55 2017-03-12 04:48:41	0|gmaxwell|same can't be said for C++ however.
56 2017-03-12 04:50:55	0|Chris_Stewart_5|gmaxwell: hah, good point. Seems like every new language has turing complete type systems these days
57 2017-03-12 04:53:51	0|gmaxwell|well it turns out that it's difficult to produce any complex non-linear system that isn't-- with some gryrantions and squinting-- turing complete. C macros are turing complete if you're willing to loop the preprocessor. :)  (doubly so because physical computers are not, pedantically, turing complete themselves due to having finite memory/storage.. and many non-turning complete langauges are equi
58 2017-03-12 04:53:57	0|gmaxwell|vilent to a turing machine with bounded-space/time...)
59 2017-03-12 04:55:42	0|Chris_Stewart_5|sipa: I didn't realize that property is preserved. I always get caught up on the whole int and char casting stuff and makes me wondering if there are *more* special cases I don't know about.
60 2017-03-12 04:56:21	0|Chris_Stewart_5|Any way, I suppose my misunderstanding of C/C++ are a little off topic for this channel, rebasing onto master now.
61 2017-03-12 04:56:35	0|Chris_Stewart_5|Hopefully the new defintion fixes my problem
62 2017-03-12 05:12:39	0|Chris_Stewart_5|What is the process for getting a bigger pull request like this merged in any way? Seems like most think it is a good idea, but we need to get some sort of managed dependency set up for rapidcheck
63 2017-03-12 05:13:01	0|Chris_Stewart_5|It is finding stuff, which is encouraging
64 2017-03-12 05:22:23	0|Chris_Stewart_5|Seems like it would be a good idea to encourage this kind of testing for down stream projects such as elements too..
65 2017-03-12 11:10:19	0|nemgun|hello
66 2017-03-12 12:42:17	0|bitcoin-git|[13bitcoin] 15laanwj opened pull request #9979: p2p: Bare minimum to support UNIX sockets (06master...062017_03_unix_socket_p2p) 02https://github.com/bitcoin/bitcoin/pull/9979
67 2017-03-12 15:52:03	0|bitcoin-git|13bitcoin/06master 14c624753 15Cory Fields: depends: fix zlib build on osx...
68 2017-03-12 15:52:03	0|bitcoin-git|[13bitcoin] 15laanwj pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/21833f9456f6...7e58b41bd7ce
69 2017-03-12 15:52:04	0|bitcoin-git|13bitcoin/06master 147e58b41 15Wladimir J. van der Laan: Merge #9973: depends: fix zlib build on osx...
70 2017-03-12 15:52:25	0|bitcoin-git|[13bitcoin] 15laanwj closed pull request #9973: depends: fix zlib build on osx (06master...06fix-zlib-osx) 02https://github.com/bitcoin/bitcoin/pull/9973
71 2017-03-12 16:57:12	0|bitcoin-git|[13bitcoin] 15Christewart opened pull request #9980: Fix mem access violation merkleblock (06master...06fix_mem_access_violation_merkleblock) 02https://github.com/bitcoin/bitcoin/pull/9980