1 2018-02-07 00:08:39	0|BlueMatt|cfields: just have to be enough time for you to hit "close"...I was using 5sec, but ofc any number works
  2 2018-02-07 03:05:47	0|conman|hmm 0.16.0rc2 doesn't like my testnet in a box
  3 2018-02-07 03:05:51	0|conman|validation.cpp:4391: void CChainState::CheckBlockIndex(const Consensus::Params&): Assertion `(pindexFirstNeverProcessed != nullptr) == (pindex->nChainTx == 0)' failed.
  4 2018-02-07 03:06:32	0|conman|works fine with 0.15.x
  5 2018-02-07 03:32:40	0|BlueMatt|conman: most likely not a regression.....new defaults - checkblockindex is on by default on testnet now
  6 2018-02-07 03:38:36	0|conman|k
  7 2018-02-07 03:38:46	0|conman|it's regtest obviously
  8 2018-02-07 03:42:57	0|BlueMatt|will look into it tomorrow, please file issue
  9 2018-02-07 03:43:03	0|BlueMatt|(if there isnt one already)
 10 2018-02-07 03:44:06	0|BlueMatt|nvm, 311782
 11 2018-02-07 03:44:10	0|BlueMatt|nvm, #11782
 12 2018-02-07 03:44:11	0|gribble|https://github.com/bitcoin/bitcoin/issues/11782 | Assertion failure in validation.cpp:4203 (re: pindexFirstNeverProcessed) · Issue #11782 · bitcoin/bitcoin · GitHub
 13 2018-02-07 03:44:46	0|BlueMatt|conman: does that match your issue? (due to an upgrade on regtest without a reindex-chainstate)
 14 2018-02-07 03:44:55	0|BlueMatt|its a change in network parameters on regtest
 15 2018-02-07 03:45:14	0|BlueMatt|also, probably want to release-notes this one!
 16 2018-02-07 03:48:58	0|conman|looks the same
 17 2018-02-07 03:52:29	0|conman|thanks
 18 2018-02-07 04:45:50	0|hanzou|I notice that getblocks (NetMsgType::GETBLOCKS) seems to be defined but never used, just GETBLOCKTXN and others.
 19 2018-02-07 04:49:43	0|hanzou|Well Bitcoin core has code such that a client will respond to getblocks, but it never sends it.
 20 2018-02-07 04:49:54	0|hanzou|When did clients stop sending getblocks?
 21 2018-02-07 04:50:36	0|sipa|BIP152
 22 2018-02-07 04:51:04	0|sipa|between compatible clients, getblocks isn't used anymore as BIP152 (compact blocks) is much more efficient
 23 2018-02-07 04:55:34	0|sipa|getblocks is still used while synchronizing old blocks
 24 2018-02-07 04:58:33	0|jjjkkk|test
 25 2018-02-07 05:28:15	0|bitcoin-git|[13bitcoin] 15murrayn opened pull request #12373: Build: Add build support for profiling. (06master...06profiling) 02https://github.com/bitcoin/bitcoin/pull/12373
 26 2018-02-07 05:54:44	0|hanzou|Thanks sipa
 27 2018-02-07 09:34:53	0|bitcoin-git|[13bitcoin] 15laanwj opened pull request #12374: qt: Make sure splash screen is freed on AppInitMain fail (06master...062017_02_splash_abort) 02https://github.com/bitcoin/bitcoin/pull/12374
 28 2018-02-07 14:06:30	0|bitcoin-git|[13bitcoin] 15MarcoFalke opened pull request #12377: qt: Avoid initialization during shutdown (06master...06Mf1802-qtInitShutdown) 02https://github.com/bitcoin/bitcoin/pull/12377
 29 2018-02-07 15:07:56	0|rooster__|Olaa all :)
 30 2018-02-07 15:09:38	0|rooster__|Anyone home? :)
 31 2018-02-07 15:13:10	0|rooster__|Yo Rex!
 32 2018-02-07 15:13:18	0|rooster__|Whats the word?
 33 2018-02-07 15:15:06	0|rooster__|I am wondering if anyone can lend me a quick hand :?
 34 2018-02-07 15:16:09	0|rooster__|I am implementing the bitcoin protocol in Android/Java. When I send a getaddr message, the peer responds with an alert message?
 35 2018-02-07 15:16:36	0|Lauda|Try #bitcoin or #bitcoin-dev; this is the wrong channel for that.
 36 2018-02-07 15:17:29	0|rooster__|Thanks lauda, ill post on there :0
 37 2018-02-07 15:17:30	0|rooster__|:)
 38 2018-02-07 15:52:44	0|sdaftuar|hanzou: sipa: getblocks is what we used to do before headers-first sync.  it's been replaced by getheaders, since 0.10 i think
 39 2018-02-07 16:03:42	0|bitcoin-git|[13bitcoin] 15jnewbery opened pull request #12379: [WIP] Better stderr testing in functional tests (06master...06test_full_stderr2) 02https://github.com/bitcoin/bitcoin/pull/12379
 40 2018-02-07 18:36:06	0|bitcoin-git|[13bitcoin] 15MarcoFalke opened pull request #12380: 0.16: Check in current release notes draft (060.16...06Mf1802-docRel16) 02https://github.com/bitcoin/bitcoin/pull/12380
 41 2018-02-07 21:18:10	0|bitcoin-git|13bitcoin/06master 149ad6746 15practicalswift: Use static_cast instead of C-style casts for non-fundamental types...
 42 2018-02-07 21:18:10	0|bitcoin-git|[13bitcoin] 15MarcoFalke pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/1462bde767a1...0277173b1def
 43 2018-02-07 21:18:11	0|bitcoin-git|13bitcoin/06master 140277173 15MarcoFalke: Merge #10498: Use static_cast instead of C-style casts for non-fundamental types...
 44 2018-02-07 21:18:22	0|bitcoin-git|[13bitcoin] 15MarcoFalke closed pull request #10498: Use static_cast instead of C-style casts for non-fundamental types (06master...06static_cast) 02https://github.com/bitcoin/bitcoin/pull/10498
 45 2018-02-07 22:41:36	0|cfields|dongcarl: what's the issue with gitian and newer LXC ?
 46 2018-02-07 22:42:47	0|dongcarl|cfields: I'm just writing up something right now actually haha.
 47 2018-02-07 22:43:00	0|cfields|dongcarl: ah, ok
 48 2018-02-07 22:43:05	0|dongcarl|cfields: in a nutshell at the very least the config file has to be changed
 49 2018-02-07 22:43:30	0|dongcarl|cfields: I'm also getting a tty error for all lxc-execute commands
 50 2018-02-07 22:43:36	0|cfields|dongcarl: isn't the config written on-the-fly ?
 51 2018-02-07 22:43:59	0|dongcarl|yes, but they changed the key names in 2.1
 52 2018-02-07 22:44:21	0|dongcarl|kallewoof actually ran into the tty errors back in December
 53 2018-02-07 22:44:43	0|dongcarl|and through GitHub digging it seems like an upstream glib thing, but that could be wrong
 54 2018-02-07 22:46:05	0|dongcarl|these issues are with LXC 2.1.1 right now, but I'm guessing non-rolling distros will move over to 2.1 soon
 55 2018-02-07 22:46:18	0|cfields|dongcarl: I assume you tried passing "-t" to ssh ?
 56 2018-02-07 22:46:39	0|dongcarl|cfields: I'm using LXC so there's no ssh, just lxc-execute
 57 2018-02-07 22:47:19	0|dongcarl|look in libexec/on-target:53
 58 2018-02-07 22:47:48	0|cfields|ah, right
 59 2018-02-07 22:48:23	0|dongcarl|I'm looking through and it seems like some of the stuff hasn't been touched for quite a while
 60 2018-02-07 22:48:33	0|dongcarl|and most "touching" have been just workarounds
 61 2018-02-07 22:48:38	0|dongcarl|correct me if I'm wrong
 62 2018-02-07 22:49:02	0|cfields|gitian, you mean?
 63 2018-02-07 22:49:09	0|dongcarl|cfields: yes
 64 2018-02-07 22:50:51	0|cfields|dongcarl: sounds about right, i haven't looked lately though
 65 2018-02-07 22:51:07	0|dongcarl|cfields: gimme a bit, gunna dump a writeup in a sec
 66 2018-02-07 22:51:08	0|cfields|dongcarl: does passing -c to sh/bash help at all?
 67 2018-02-07 22:51:10	0|cfields|ok
 68 2018-02-07 22:52:25	0|dongcarl|cfields: sipa recommended that I talk to you about this. I've been looking into making
 69 2018-02-07 22:52:29	0|dongcarl|gitian more robust over the last few days and was wondering what you and others would t
 70 2018-02-07 22:52:32	0|dongcarl|hink of these possible changes. I refer to the container actually building the Bitcoin
 71 2018-02-07 22:52:35	0|dongcarl|binary as the "build container" and the host/container running the aforementioned conta
 72 2018-02-07 22:52:38	0|dongcarl|iner the "container host."
 73 2018-02-07 22:53:08	0|dongcarl|1. Using vagrant to spin up a "container host" with a provisioner to spin up a "build container." This is similar to what is done for zcash (https://github.com/zcash/zcash-gitian). This solution has the advantage of not breaking existing builds and being easy to implement, but wastes cycles for containerization/virtualization.
 74 2018-02-07 22:53:13	0|dongcarl|2. Replacing gitian's direct interfacing with LXC and KVM with vagrant. As in, let vagrant handle communicating with its provider (LXC, KVM, VirtualBox, VMWare, etc.), and gitian will interface with vagrant. This solution has the advantage of wasting less cycles, but it might break existing builds, and a slight annoyance is relying on third party repos to code up the integration between vagrant
 75 2018-02-07 22:53:19	0|dongcarl|and the provider of choice, but looking through the ones people would use (e.g. vagrant-lxc) it seems like they're well maintained.
 76 2018-02-07 22:53:22	0|dongcarl|3. Same as #2 but have libvirt instead of vagrant.
 77 2018-02-07 22:53:24	0|dongcarl|sorry about that folks
 78 2018-02-07 22:53:25	0|gribble|https://github.com/bitcoin/bitcoin/issues/2 | Long-term, safe, store-of-value · Issue #2 · bitcoin/bitcoin · GitHub
 79 2018-02-07 22:55:38	0|cfields|dongcarl: it's worth mentioning up front that there are some other build changes in the works that simplify what gitian is responsible for
 80 2018-02-07 22:57:05	0|cfields|regardless, #2 sounds reasonable to me
 81 2018-02-07 22:57:08	0|gribble|https://github.com/bitcoin/bitcoin/issues/2 | Long-term, safe, store-of-value · Issue #2 · bitcoin/bitcoin · GitHub
 82 2018-02-07 22:57:59	0|sipa|from what i hear, it sounds like we may end up relying on a pre-built image somehow
 83 2018-02-07 22:58:46	0|cfields|dongcarl: I'm working on a process that builds deterministic toolchain and rootfs for us to use. In theory, gitian isn't even necessary. But in practice, we'll need a very thin wrapper that just runs a script and gathers the results
 84 2018-02-07 22:59:04	0|sipa|cfields: how far along are you with that?
 85 2018-02-07 22:59:09	0|cfields|sipa: pre-built image?
 86 2018-02-07 22:59:12	0|sipa|i was assuming that may not happen overnight
 87 2018-02-07 23:00:16	0|cfields|sipa: I was on a roll for a while, but then got stuck working on static pie for gcc. I think I could have it up and going relatively quickly if it's holding up progress.
 88 2018-02-07 23:00:41	0|sipa|hmmm, static pie </homer>
 89 2018-02-07 23:00:50	0|cfields|haha
 90 2018-02-07 23:01:20	0|dongcarl|cfields: so sort of like what coreboot does?
 91 2018-02-07 23:01:42	0|sipa|cfields: am i right that this would mean we can build deterministic binaries just from depends make, without any vm?
 92 2018-02-07 23:01:45	0|cfields|I only mentioned the toolchain stuff because Mark mentioned reworking gitian a while back, and his plans would've conflicted
 93 2018-02-07 23:02:16	0|cfields|sipa: correct. The only variables are the running kernel and filesystem
 94 2018-02-07 23:02:20	0|cfields|dongcarl: I'm not familiar
 95 2018-02-07 23:02:34	0|sipa|cfields: what do you mean by variables?
 96 2018-02-07 23:02:46	0|cfields|sipa: sorry, sources of non-determinism
 97 2018-02-07 23:02:56	0|sipa|hmm, how so?
 98 2018-02-07 23:03:04	0|sipa|how does the filesystem matter?
 99 2018-02-07 23:03:12	0|sipa|order it lists files etc?
100 2018-02-07 23:03:16	0|cfields|yup
101 2018-02-07 23:03:32	0|cfields|I'm hoping to just slowly patch those things upstream as they arise
102 2018-02-07 23:04:15	0|dongcarl|so we'll end up needing some sort of container to normalize that anyways? I'm trying to decide whether or not I should spend time on solution no. 2 right now...
103 2018-02-07 23:04:18	0|sipa|well that sounds like a major improvement over what we have
104 2018-02-07 23:04:21	0|cfields|gcc/binutils is in good shape these days. busybox is the one I'm unsure about
105 2018-02-07 23:05:03	0|sipa|it would also let us pin a particular gcc version more easily, without depending on what's available in a particular build env
106 2018-02-07 23:05:06	0|cfields|dongcarl: realistically yes, I think
107 2018-02-07 23:05:57	0|arubi|cfields, you've seen landley's mkroot right?
108 2018-02-07 23:06:28	0|cfields|sipa: yes, very much so. Our own config as well. the stupid mingw-w64 thread issue is a good example of how we're at the mercy of Ubuntu atm
109 2018-02-07 23:07:25	0|sipa|cfields: do you have any idea about realistic timelines on the self-built compiler environment stuff?
110 2018-02-07 23:07:59	0|cfields|sipa: just for using locally or for actually building releases?
111 2018-02-07 23:08:15	0|dongcarl|actually building?
112 2018-02-07 23:08:27	0|dongcarl|does sound cool
113 2018-02-07 23:08:40	0|sipa|cfields: i'd say something in between, as in "actually usable for building releases if we wanted to"
114 2018-02-07 23:08:50	0|sipa|even if we don't immediately decide to switch to it
115 2018-02-07 23:09:01	0|cfields|sipa: .17 seems reasonable...
116 2018-02-07 23:09:45	0|dongcarl|cfields: do you have a repo up somewhere where I can also take a look and contribute?
117 2018-02-07 23:09:53	0|cfields|sipa: I think the biggest issue is that I'm targetting gcc 8.1, which hasn't even been released yet. And I'm sure we want to wait for some hardening
118 2018-02-07 23:10:46	0|cfields|dongcarl: not atm, but I can push something up in a week or two
119 2018-02-07 23:11:20	0|dongcarl|cfields: that would be fantastic, I'd love to help get it across the line in whatever way I can
120 2018-02-07 23:11:56	0|cfields|dongcarl: great :)
121 2018-02-07 23:12:43	0|cfields|dongcarl: thinking about it a bit, an abstraction around lxc/kvm/vbox/whatever is kinda just re-introducing the same non-determinism we're looking to avoid.
122 2018-02-07 23:13:16	0|dongcarl|cfields: how so?
123 2018-02-07 23:15:58	0|cfields|dongcarl: presumably they'd all have different methods for creating the target image, meaning that each may have (for ex) a separate filesystem
124 2018-02-07 23:16:11	0|cfields|s/separate/differing/
125 2018-02-07 23:16:24	0|dongcarl|cfields: well, actually, for vagrant they share the same image!
126 2018-02-07 23:17:33	0|sipa|cfields: gcc... 8?
127 2018-02-07 23:18:01	0|dongcarl|cfields: e.g. https://app.vagrantup.com/debian/boxes/jessie64/versions/8.7.0
128 2018-02-07 23:18:02	0|cfields|dongcarl: sure, but they'll still virtualize things differently. lxc may hit a real disk for files, for ex
129 2018-02-07 23:18:08	0|sipa|what in particular is in 8.1 that makes it attractive?
130 2018-02-07 23:18:12	0|cfields|sipa: I'm upstreaming fixes
131 2018-02-07 23:18:24	0|sipa|gotcha
132 2018-02-07 23:18:40	0|dongcarl|cfields: makes sense
133 2018-02-07 23:18:52	0|cfields|sipa: in particular, deterministic cross-libc bootstrap and static pie
134 2018-02-07 23:19:14	0|cfields|dongcarl: I'm not saying it can't work. Obviously it works now, because gitian can use kvm or lxc...
135 2018-02-07 23:19:16	0|sipa|but gcc 8 isn't even out yet?
136 2018-02-07 23:19:34	0|cfields|dongcarl: I was just making the point that it's kinda a lateral move in that sense
137 2018-02-07 23:19:53	0|cfields|sipa: heh, thank gcc's inane versioning scheme
138 2018-02-07 23:20:09	0|cfields|sipa: 8.0 is beta, 8.1 is final.
139 2018-02-07 23:20:10	0|dongcarl|cfields: right, and if we can finish this in .17, doesn't make sense to spend time on moving to that
140 2018-02-07 23:20:12	0|sipa|i see
141 2018-02-07 23:20:23	0|dongcarl|that = vagrant and no. 2
142 2018-02-07 23:20:46	0|cfields|dongcarl: right
143 2018-02-07 23:22:16	0|cfields|dongcarl: basically, we're defining a reference system. We really can't get around that. So I'm apprehensive about attempts to abstract away the reference.
144 2018-02-07 23:22:35	0|cfields|Again, I realize that's what gitian does now.
145 2018-02-07 23:23:28	0|dongcarl|You're absolutely right. I think for my own building needs I'll just make my own version of no. 1 for now. Basically what's in the guides for virtualbox but with vagrant. Super easy to setup.
146 2018-02-07 23:27:06	0|cfields|sounds good
147 2018-02-07 23:28:10	0|cfields|dongcarl: what I'd be curious to see would be the thinnest possible wrapper on top of some vm. Just setup the cache, run the script, report the results
148 2018-02-07 23:29:02	0|dongcarl|cfields: check out the linked zcash README
149 2018-02-07 23:29:12	0|dongcarl|https://github.com/zcash/zcash-gitian
150 2018-02-07 23:29:19	0|dongcarl|uber-simple
151 2018-02-07 23:29:54	0|dongcarl|(not a big fan of the number of build dependencies tho)
152 2018-02-07 23:30:55	0|cfields|well ours will have no dependencies, so that's easy
153 2018-02-07 23:31:26	0|dongcarl|woot woot
154 2018-02-07 23:31:59	0|cfields|hmm
155 2018-02-07 23:32:52	0|cfields|I guess using vagrant with some static config is no different from rolling our own, in terms of determinism
156 2018-02-07 23:33:32	0|gmaxwell|gitian abstracting away the reference mostly is just introducing significant insecurity and pretending we don't need to worry about it.
157 2018-02-07 23:34:57	0|dongcarl|cfields: true, but vagrant references boxes by name, not hashes, and those boxes can be maliciously built.
158 2018-02-07 23:35:29	0|dongcarl|cfields: what'd be nice is a script that builds the vagrant box (like gitian does with debootstrap) then launches it with a static config
159 2018-02-07 23:35:35	0|dongcarl|gmaxwell: Agreed.
160 2018-02-07 23:36:23	0|dongcarl|cfields: btw what did you mean by "rolling our own"?
161 2018-02-07 23:36:26	0|cfields|dongcarl: can we not just use a generic ubuntu-minimal image ?
162 2018-02-07 23:37:27	0|dongcarl|cfields: from ubuntu or from vagrant boxes? Vagrant has its own repository of pre-built boxes that are referred to by name that everyone uses.
163 2018-02-07 23:37:34	0|cfields|(I've never used vagrant, though I assume it's not hugely different from the other similar vm builders/launchers)
164 2018-02-07 23:38:27	0|dongcarl|so the "vagrant boxes" are supposed to just immediately spin up with no install process
165 2018-02-07 23:38:45	0|cfields|gmaxwell: the insecurity being "trusting ubuntu", you mean?
166 2018-02-07 23:39:39	0|cfields|dongcarl: by "rolling our own", I meant writing something like gitian
167 2018-02-07 23:40:20	0|cfields|dongcarl: I see. Looks like I need to read up.
168 2018-02-07 23:40:39	0|gmaxwell|cfields: right, and their ongoing updates.  For users that already run ubuntu arguably there is not much increase in surface over what they already face, but that doesn't apply to people who run other things than ubuntu.
169 2018-02-07 23:41:18	0|cfields|gmaxwell: sure, completely agree. Just wanted clarification.
170 2018-02-07 23:41:26	0|gmaxwell|AFAIK we don't have any reason to think ubuntu's build process is particularly secure either,  e.g. could be internet connected hosts that a hundred people have remote shell access on.
171 2018-02-07 23:41:56	0|gmaxwell|And certantly whatever it is, it runs a lot of effectively unreviewed potentially malicious code.
172 2018-02-07 23:42:07	0|dongcarl|cfields: yeah if we end up needing something to control the two variables you mentioned earlier, vagrant would probably work well with a fixed provider, static config, and building our own boxes instead of fetching from the repo.
173 2018-02-07 23:42:41	0|dongcarl|gmaxwell: :-/
174 2018-02-07 23:43:16	0|gmaxwell|if or once we get to the point where our build process builds the same binaries when run from different systems at least we'll be able to crosscheck against something else, even if the standard build enviroment remains an ubuntu vm.
175 2018-02-07 23:44:55	0|dongcarl|gmaxwell: "where our build process builds the same binaries when run from different systems" <- as in when cfields finishes the work on unVM'd deterministic builds?
176 2018-02-07 23:45:19	0|gmaxwell|right.
177 2018-02-07 23:46:02	0|gmaxwell|I think it's likely though that the standard process for that will still have people use a standard vm image... just because it'll be a baseline system that we know works.
178 2018-02-07 23:47:13	0|dongcarl|gmaxwell: yeah, we were talking about that earlier, kernel and filesystem are the two variables that are most easily controlled by VMs I guess.
179 2018-02-07 23:50:07	0|cfields|gmaxwell: yes. So far I have a minimal rootfs that's capable of building bitcoin. It's basically just busybox/make/toolchain. Dealing with kernels is just too far out of scope, though. Ideally, we'd just boot some minimal image in a vm, chroot, and make
180 2018-02-07 23:50:37	0|gmaxwell|cfields: are you finding that the kernel and whatnot breaks determinism
181 2018-02-07 23:50:38	0|cfields|(that minimal rootfs build is deterministic and would first be built in gitian itself)
182 2018-02-07 23:51:33	0|cfields|gmaxwell: no, I was just enumerating the issues at play. Almost all issues we've had have come from filesystem.
183 2018-02-07 23:51:44	0|gmaxwell|ah good.
184 2018-02-07 23:53:24	0|cfields|dongcarl: I should think that fbsd deterministic builds would come pretty close to working as-is
185 2018-02-07 23:54:10	0|dongcarl|cfields: with virtualbox you mean?
186 2018-02-07 23:54:39	0|cfields|dongcarl: cross build from linux with gitian
187 2018-02-07 23:55:09	0|dongcarl|cfields: Oh. I mean, DOING deterministic builds from FreeBSD haha
188 2018-02-07 23:55:40	0|cfields|oh. why? :)
189 2018-02-07 23:56:25	0|dongcarl|cfields: Because... I'm a sadist who loves making things work on *BSD.
190 2018-02-07 23:56:45	0|gmaxwell|(1) because some people run it, and (2) hey, distinct build enviroment is better for crosschecking determinism.
191 2018-02-07 23:56:58	0|gmaxwell|But it's okay if people have to suffer to make it work.
192 2018-02-07 23:57:08	0|dongcarl|let's pretend I thought of (2) there
193 2018-02-07 23:57:31	0|dongcarl|next stop: plan 9 deterministic builds
194 2018-02-07 23:57:36	0|dongcarl|then: seL4
195 2018-02-07 23:57:48	0|gmaxwell|nah, build on AT&T unix on SIMH pdp11. :P
196 2018-02-07 23:58:18	0|dongcarl|<3
197 2018-02-07 23:58:29	0|cfields|sure, no complaints if someone wants to try
198 2018-02-07 23:59:16	0|cfields|I plan to attempt the same for osx, but just because I want to see where the differences end up coming from
199 2018-02-07 23:59:33	0|sipa|OS/2, please.
200 2018-02-07 23:59:43	0|gmaxwell|it would be kinda interesting if the project there were at least two distinct project recommended base VMs, and people were told to use one based on hashing their name.