1 2017-11-11 00:12:52	0|bitcoin-git|[13bitcoin] 15sipa pushed 3 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/61fb80660f73...033c78671b91
  2 2017-11-11 00:12:53	0|bitcoin-git|13bitcoin/06master 141141364 15John Newbery: [trivial] (whitespace only) fix getblockchaininfo alignment
  3 2017-11-11 00:12:53	0|bitcoin-git|13bitcoin/06master 14bd9c181 15John Newbery: [rpc] Add initialblockdownload to getblockchaininfo
  4 2017-11-11 00:12:54	0|bitcoin-git|13bitcoin/06master 14033c786 15Pieter Wuille: Merge #11258: [rpc] Add initialblockdownload to getblockchaininfo...
  5 2017-11-11 00:13:11	0|bitcoin-git|[13bitcoin] 15sipa closed pull request #11258: [rpc] Add initialblockdownload to getblockchaininfo (06master...06expose_ibd) 02https://github.com/bitcoin/bitcoin/pull/11258
  6 2017-11-11 01:21:44	0|ZSky|Hi
  7 2017-11-11 01:22:41	0|ZSky|When a block is added, is it written/serialized on disk immediately?
  8 2017-11-11 01:23:05	0|ZSky|If so, using what kind of storing? DB-like? flat files?
  9 2017-11-11 01:23:38	0|sipa|flat files
 10 2017-11-11 01:24:06	0|sipa|after processing (which may happen at the same time as it's received, or later), its effects are applied to the UTXO set
 11 2017-11-11 01:24:12	0|ZSky|sipa: immediately after a block is added, or does it stay in RAM for a while before flushing to disk?
 12 2017-11-11 01:24:31	0|sipa|it's written immediately if it's acceptable
 13 2017-11-11 01:25:03	0|sipa|though we reuse the in-memory version IF processing happens at the same time
 14 2017-11-11 01:25:18	0|ZSky|sipa: what kind of flat file structure? just "fopen append" to a single file?
 15 2017-11-11 01:26:00	0|sipa|we keep track of how many bytes are already used in each file in a database
 16 2017-11-11 01:26:09	0|sipa|and then just write the exact range
 17 2017-11-11 01:26:16	0|ZSky|you mean the offset?
 18 2017-11-11 01:26:19	0|sipa|yes
 19 2017-11-11 01:26:34	0|ZSky|so the raw data is in one single big file (100+GB)
 20 2017-11-11 01:26:40	0|sipa|no, it's in many files
 21 2017-11-11 01:26:42	0|ZSky|and there's a db for offsets
 22 2017-11-11 01:26:49	0|sipa|blk?????.dat
 23 2017-11-11 01:26:54	0|sipa|128 MiB each
 24 2017-11-11 01:27:25	0|sipa|and indeed, there is a LevelDB database with metadata about each block (which includes the file and offset)
 25 2017-11-11 01:27:57	0|ZSky|each time a 128 MiB is full, a new one is created, and the DB stores [block13672: { filenumber = 27,  offsetinfile = 283792 }, ...]
 26 2017-11-11 01:28:03	0|sipa|exactly
 27 2017-11-11 01:28:09	0|ZSky|oh that's nice
 28 2017-11-11 01:28:43	0|ZSky|wow leveldb seems nice
 29 2017-11-11 01:29:02	0|ZSky|it's been a long time I was looking for a serialized key-value storage
 30 2017-11-11 01:29:05	0|ZSky|usable in production
 31 2017-11-11 01:29:21	0|sipa|it's not perfect
 32 2017-11-11 01:29:29	0|sipa|but better than anything else we've tried
 33 2017-11-11 01:30:03	0|ZSky|so you're using this currently: https://github.com/google/leveldb
 34 2017-11-11 01:30:24	0|sipa|we have our own fork with minor changes: https://github.com/bitcoin-core/leveldb
 35 2017-11-11 01:30:32	0|sipa|in particular, MinGW windows support
 36 2017-11-11 01:32:48	0|ZSky|sipa: dumb question: why not using leveldb this way:   block13672:  [BLOCKRAWDATA]
 37 2017-11-11 01:33:03	0|ZSky|and totally rely on leveldb's data management?
 38 2017-11-11 01:33:17	0|sipa|because we don't need leveldb's data management
 39 2017-11-11 01:33:43	0|ZSky|you use it for metadata
 40 2017-11-11 01:33:52	0|ZSky|so you kind of need a small key:value db
 41 2017-11-11 01:33:53	0|sipa|yes, which is a tiny amount of data
 42 2017-11-11 01:33:56	0|ZSky|true
 43 2017-11-11 01:34:14	0|ZSky|so the idea is: you keep a key:value store for the smallest amount of data possible
 44 2017-11-11 01:34:15	0|sipa|but for blocks themselves, we don't need consistency, or the ability to selectively update and delete, or ...
 45 2017-11-11 01:34:24	0|ZSky|true
 46 2017-11-11 01:34:38	0|sipa|it's just a stream of data that comes in, and we can drop it in effectively append only files on disk
 47 2017-11-11 01:34:58	0|sipa|in fact, we hardly use the block data after it's been processed
 48 2017-11-11 01:35:10	0|sipa|it's just for when another peer in the network asks for it
 49 2017-11-11 01:35:25	0|sipa|or to rescan for old wallet activity
 50 2017-11-11 01:36:02	0|ZSky|yes i see
 51 2017-11-11 01:36:14	0|ZSky|historically, in the earliest implementations, what was used before leveldb?
 52 2017-11-11 01:36:35	0|sipa|BDB
 53 2017-11-11 01:36:44	0|sipa|changed in 0.8
 54 2017-11-11 01:36:52	0|sipa|and the wallet still uses BDB
 55 2017-11-11 01:37:43	0|ZSky|why no BDB anymore?
 56 2017-11-11 01:38:15	0|sipa|many many reports of corruption
 57 2017-11-11 01:38:46	0|sipa|incompatibilities between versions (you can't downgrade, and in some cases you can't even upgrade)
 58 2017-11-11 01:39:28	0|sipa|and the fact that BDB was indirectly the cause for one of the largest operational problems the system has seen in years (read BIP50)
 59 2017-11-11 01:39:59	0|ZSky|oh really?
 60 2017-11-11 01:40:01	0|sipa|which was related due to it know how many locks the database will require during its operation
 61 2017-11-11 01:40:03	0|ZSky|tl;dr what was it?
 62 2017-11-11 01:40:13	0|sipa|while we don't need any locks at all
 63 2017-11-11 01:40:52	0|sipa|you need to configure BDB with "i need up to X locks", and if any atomic database operation touches more internal records than you have configured locks, the operations fails
 64 2017-11-11 01:41:37	0|sipa|this is because BDB is a multi-application database system (multiple processes can simultaneously open and modify the database, and there is consistency across them), which is something we don't care about at all
 65 2017-11-11 01:55:21	0|ZSky|sipa: what was used in 0.1.0?
 66 2017-11-11 01:55:46	0|sipa|BDB
 67 2017-11-11 01:56:05	0|ZSky|ok right
 68 2017-11-11 01:56:19	0|ZSky|sipa: do you do python sometimes?
 69 2017-11-11 01:56:24	0|sipa|hardly
 70 2017-11-11 01:57:02	0|ZSky|i was looking for such a keystore in py
 71 2017-11-11 02:02:57	0|ZSky|sipa: thanks for these informations!
 72 2017-11-11 02:04:03	0|sipa|yw
 73 2017-11-11 03:53:25	0|cluelessperson|aj: thank you for your help
 74 2017-11-11 05:49:11	0|cluelessperson|How can I help you all?
 75 2017-11-11 05:49:18	0|cluelessperson|I can do some devops
 76 2017-11-11 05:49:21	0|cluelessperson|put me to use
 77 2017-11-11 06:17:07	0|meshcollider|devops...199
 78 2017-11-11 07:28:35	0|mryandao|meshcollider: lol, funny
 79 2017-11-11 08:03:10	0|meshcollider|CScripts in the wallet don't have birth-times associated with them do they
 80 2017-11-11 09:08:37	0|bitcoin-git|[13bitcoin] 15luke-jr opened pull request #11658: During IBD, when doing pruning, prune 10% extra to avoid pruning again soon after (06master...06ibd_prune_extra) 02https://github.com/bitcoin/bitcoin/pull/11658
 81 2017-11-11 11:25:45	0|bitcoin-git|[13bitcoin] 15luke-jr opened pull request #11660: RPC: Internal named params (06master...06internal_named_params) 02https://github.com/bitcoin/bitcoin/pull/11660
 82 2017-11-11 11:27:46	0|luke-jr|^ hope that satisfies concerns with #11441
 83 2017-11-11 11:27:48	0|gribble|https://github.com/bitcoin/bitcoin/issues/11441 | rpc/server: Support for specifying options as named parameters by luke-jr · Pull Request #11441 · bitcoin/bitcoin · GitHub
 84 2017-11-11 12:12:36	0|meshcollider|MarcoFalke: I've seen a few PRs being closed recently as 'up for grabs', can we have a label for it to keep track of them?
 85 2017-11-11 12:14:14	0|meshcollider|Maybe called "Abandoned" or something
 86 2017-11-11 13:41:42	0|bitcoin-git|13bitcoin/06master 146de3203 15Wladimir J. van der Laan: doc: Add historical release notes for 0.15.1...
 87 2017-11-11 13:41:42	0|bitcoin-git|[13bitcoin] 15laanwj pushed 1 new commit to 06master: 02https://github.com/bitcoin/bitcoin/commit/6de3203cdce2d8532f39f9f9428c33b0dd53f623
 88 2017-11-11 13:43:18	0|F0lks|Hi, is there some devs' available for a little talk ? It won't be long I promise
 89 2017-11-11 14:27:27	0|bitcoin-git|[13bitcoin] 15luke-jr opened pull request #11662: [0.15] Sanity-check script sizes in bitcoin-tx (060.15...06bitcoin-tx-script-sizes-0.14) 02https://github.com/bitcoin/bitcoin/pull/11662
 90 2017-11-11 14:57:48	0|wumpus|release date 2017-11-11
 91 2017-11-11 14:57:49	0|wumpus|nice
 92 2017-11-11 14:59:52	0|wxss|\o/
 93 2017-11-11 15:02:11	0|paveljanik|remaining days to Xmas: *42*.
 94 2017-11-11 15:11:34	0|LumberCartel|paveljanik++
 95 2017-11-11 15:16:59	0|wumpus|:D
 96 2017-11-11 16:15:04	0|MarcoFalke|meshcollider: Sure will do that
 97 2017-11-11 16:23:55	0|LowKey|hi, anyone know how to build mac wallet for bitcoin?
 98 2017-11-11 16:27:37	0|mlz|LowKey, ask in #bitcoin channel
 99 2017-11-11 16:27:54	0|LowKey|ok
100 2017-11-11 17:39:57	0|bitcoin-git|[13bitcoin] 15MarcoFalke pushed 3 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/6de3203cdce2...95e14dc81dd3
101 2017-11-11 17:39:58	0|bitcoin-git|13bitcoin/06master 145e0ba8f 15John Newbery: [wallet] getreceivedbyaddress should return error if address is not mine
102 2017-11-11 17:39:58	0|bitcoin-git|13bitcoin/06master 14ea0cd24 15John Newbery: [tests] Tidy up receivedby.py...
103 2017-11-11 17:39:59	0|bitcoin-git|13bitcoin/06master 1495e14dc 15MarcoFalke: Merge #11055: [wallet] [rpc] getreceivedbyaddress should return error if called with address not owned by the wallet...
104 2017-11-11 17:40:22	0|bitcoin-git|[13bitcoin] 15MarcoFalke closed pull request #11055: [wallet] [rpc] getreceivedbyaddress should return error if called with address not owned by the wallet (06master...06getreceivedbyaddress_error) 02https://github.com/bitcoin/bitcoin/pull/11055
105 2017-11-11 18:02:53	0|bitcoin-git|[13bitcoin] 15MarcoFalke opened pull request #11663: [trivial] doc: Add getreceivedbyaddress release notes (06master...06Mf1711-docReleaseNotes16) 02https://github.com/bitcoin/bitcoin/pull/11663
106 2017-11-11 18:12:26	0|sipa|MarcoFalke: your PGP signatures scare me!
107 2017-11-11 18:12:35	0|MarcoFalke|why is that?
108 2017-11-11 18:12:55	0|sipa|any time i receive a bitcoin related email with PGP headers i assume there is some terrible vulnerability :)
109 2017-11-11 18:15:27	0|MarcoFalke|heh. Makes sense now that you say it. I will try to figure out something else.
110 2017-11-11 18:20:03	0|sipa|oh please no, it was not a complaint :)
111 2017-11-11 18:33:50	0|bitcoin-git|[13bitcoin] 15MarcoFalke pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/95e14dc81dd3...13e352dc53de
112 2017-11-11 18:33:51	0|bitcoin-git|13bitcoin/06master 1413e352d 15MarcoFalke: Merge #3716: GUI: Receive: Remove option to reuse a previous address...
113 2017-11-11 18:33:51	0|bitcoin-git|13bitcoin/06master 14927f4ff 15Luke Dashjr: GUI: Receive: Remove option to reuse a previous address...
114 2017-11-11 19:33:20	0|luke-jr|jnewbery: promag: am I missing something? :/ http://luke.dashjr.org/tmp/screenshots/Screenshot_20171111_192644.png
115 2017-11-11 20:27:07	0|promag|luke-jr: what you mean? the test verifies that initialblockdownload is a key of getblockchaininfo response
116 2017-11-11 20:29:13	0|promag|https://github.com/jnewbery/bitcoin/blob/11413646be2a6d0bf1c857753bfcec0af60c72b8/test/functional/blockchain.py#L54-L73
117 2017-11-11 20:29:54	0|promag|or am I wrong?
118 2017-11-11 20:30:45	0|bitcoin-git|[13bitcoin] 15benma closed pull request #9897: AppInitMain: split initialization of Connman into a new function (06master...06connman) 02https://github.com/bitcoin/bitcoin/pull/9897
119 2017-11-11 21:03:01	0|luke-jr|promag: for some reason, GitHub isn't showing me that part :|
120 2017-11-11 21:04:43	0|luke-jr|oh well
121 2017-11-11 21:17:25	0|jtimon|jl2012: not sure if you still want to do #11398 on top of #11426 since some people don't seem to agree it makes things easier, but rebased
122 2017-11-11 21:17:27	0|gribble|https://github.com/bitcoin/bitcoin/issues/11398 | Hardcode CSV and SEGWIT deployment by jl2012 · Pull Request #11398 · bitcoin/bitcoin · GitHub
123 2017-11-11 21:17:29	0|gribble|https://github.com/bitcoin/bitcoin/issues/11426 | BIP90: Make buried deployments slightly more easily extensible by jtimon · Pull Request #11426 · bitcoin/bitcoin · GitHub
124 2017-11-11 23:11:20	0|bitcoin-git|13bitcoin/06master 141e65f0f 15practicalswift: Use compile-time constants instead of unnamed enumerations (remove "enum hack")
125 2017-11-11 23:11:20	0|bitcoin-git|[13bitcoin] 15MarcoFalke pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/13e352dc53de...2adbddb03840
126 2017-11-11 23:11:21	0|bitcoin-git|13bitcoin/06master 142adbddb 15MarcoFalke: Merge #10749: Use compile-time constants instead of unnamed enumerations (remove "enum hack")...
127 2017-11-11 23:11:40	0|bitcoin-git|[13bitcoin] 15MarcoFalke closed pull request #10749: Use compile-time constants instead of unnamed enumerations (remove "enum hack") (06master...06enum-hack) 02https://github.com/bitcoin/bitcoin/pull/10749