1 2016-05-31 02:19:01	0|kanzure|i do not feel entirely comfortable with his "throw tests away" comment
  2 2016-05-31 03:56:31	0|gmaxwell|kanzure: I thought the same, but on reflection, I dunno, any generic advice is ... generic.  I've certantly seen tests that should be thrown away. Ones that basically could never catch an error but still manage to produce false positives. Those should probably be thrown away.
  3 2016-05-31 06:57:52	0|jonasschnelli|cfields_, wumpus: re: https://github.com/bitcoin/bitcoin/issues/8120, "make deploy" is unused during gitian? right?
  4 2016-05-31 08:27:54	0|sipa|gmaxwell: i think the advice to remove never-failing tests specifically is not wanted for consensus logic
  5 2016-05-31 08:28:16	0|sipa|gmaxwell: though i guess that's a case of code whose behaviour is "fully specified"
  6 2016-05-31 10:23:03	0|kanzure|"delete tests" is advice that probably works on teams where people are encouraged to write lots of useless tests. i could imagine the weeds growing pretty fast :).
  7 2016-05-31 10:39:30	0|sipa|well we do suffer from tests that just test exact implementation equivalence rather than correctness
  8 2016-05-31 10:39:50	0|sipa|like any change to the coin selection code will likely break the uniy tests
  9 2016-05-31 10:39:58	0|sipa|which discourages improving them
 10 2016-05-31 10:40:17	0|sipa|rather than helping making sure that changes don't break them
 11 2016-05-31 10:48:11	0|kanzure|yes i am not sure how to correctly "test" stochastic approximation. "is it really approximating?" "is it really stochastic?" hehe
 12 2016-05-31 12:37:46	0|wumpus|I wouldn't agree with deleting tests
 13 2016-05-31 12:38:53	0|wumpus|I'd still like to add more unit tests and more coverage instead of going on some wacky tangent
 14 2016-05-31 12:41:00	0|wumpus|I don't think we have the mentioned problem at all, that we're trusting unit tests too much
 15 2016-05-31 12:41:13	0|wumpus|code review is most important
 16 2016-05-31 12:43:20	0|wumpus|but removing tests sound like removing safety features from a car because it makes people more careful, yea, nice in theory, in practice it is harmful to have more crippled people even if they 'learned a lesson'
 17 2016-05-31 12:44:13	0|sipa|i agree, but that post does contain some interesting view point
 18 2016-05-31 12:44:14	0|wumpus|similarly, having trivial things fail more often in production is not even if it makes developers (who?) learn a lesson
 19 2016-05-31 12:44:18	0|wumpus|I really don't like it
 20 2016-05-31 12:44:26	0|btcdrak|I think it's ridiculous. Unit tests are invaluable, they catch regressions, make refactoring easier to do etc.
 21 2016-05-31 12:44:35	0|wumpus|yes it's pretty ridiculous imo
 22 2016-05-31 12:45:13	0|btcdrak|if this is where "professional" software development is going, god save us.
 23 2016-05-31 12:46:13	0|sipa|well if people are refactoring their software so that higher line coverage is obtained, with tests that will never fail... there is a problem
 24 2016-05-31 12:46:17	0|wumpus|I doubt it is. And yes some companies overdo testing, having test suites that run for 20 hours every time a change is made. For those I guess it makes sense to be more selective.
 25 2016-05-31 12:46:22	0|sipa|i don't think we have that problem
 26 2016-05-31 12:46:28	0|sipa|rather the opposite
 27 2016-05-31 12:47:10	0|sipa|however, i do think we have a number of tests that are focussed on testing exact behaviour, of which nobody still knows why that test was added
 28 2016-05-31 12:47:25	0|wumpus|yes, there are plenty of specific tests that could be improved
 29 2016-05-31 12:47:28	0|wumpus|no argument there
 30 2016-05-31 12:47:35	0|btcdrak|and properly commented.
 31 2016-05-31 12:47:50	0|sipa|so they break when code is changed, with nobody knowing whether the broken test was testing some required property of the system, or just testing the exact behaviour the old code had
 32 2016-05-31 12:48:13	0|wumpus|testing is absolutely a skill in itself and it doesn't overlap 100% with developing, but saying things like 'business people should make tests not developers'... yeah right
 33 2016-05-31 12:48:49	0|sipa|well i guess we're kinda doing that for wallet rpc tests
 34 2016-05-31 12:49:15	0|sipa|not that they're written by business people, but they do test application behaviour, not individual functioms
 35 2016-05-31 12:49:29	0|wumpus|both levels make sense to test
 36 2016-05-31 12:49:33	0|sipa|of course
 37 2016-05-31 12:50:12	0|wumpus|but yes for the wallet we primarily test application-level behavior also because most functions aren't really testable units
 38 2016-05-31 12:50:35	0|sipa|i don't think anyone is actually suggesting removing tests (?), but i think a mindset of testing for correctness rather than exact replication makes sense
 39 2016-05-31 12:50:54	0|wumpus|lots of behavior that depends on the state of the wallet, which depends on an entire history of actions, there's not really much unit testing that can be done in that case
 40 2016-05-31 12:50:58	0|sipa|or at least separating the two
 41 2016-05-31 12:54:41	0|wumpus|indeed
 42 2016-05-31 12:58:40	0|phantomcircuit|wumpus, the question is ultimately "what is a unit?"
 43 2016-05-31 12:58:57	0|wumpus|phantomcircuit: it is, tho let's not get philosophical :)
 44 2016-05-31 12:59:50	0|sipa|phantomcircuit: i think we should use Planck units
 45 2016-05-31 13:00:00	0|wumpus|+1
 46 2016-05-31 13:00:24	0|phantomcircuit|sipa, :P
 47 2016-05-31 13:00:32	0|phantomcircuit|it's in the name
 48 2016-05-31 13:00:34	0|phantomcircuit|unit tests
 49 2016-05-31 13:01:34	0|wumpus|though not all Planck units describe the minimum physical quantities, e.g. plack mass is pretty large (in the micrograms)
 50 2016-05-31 13:02:58	0|wumpus|I guess that describes some of our unit tests that are actually RPC tests in disguise...
 51 2016-05-31 13:03:32	0|sipa|hah
 52 2016-05-31 13:05:22	0|sipa|planck pressure: 4.63 * 10^113 Pa
 53 2016-05-31 13:06:10	0|wumpus|wow
 54 2016-05-31 13:10:29	0|GitHub165|13bitcoin/06master 1463ff57d 15Gregory Maxwell: Avoid integer division in the benchmark inner-most loop....
 55 2016-05-31 13:10:29	0|GitHub165|[13bitcoin] 15laanwj pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/950be19727a5...0026e0ef3472
 56 2016-05-31 13:10:30	0|GitHub165|13bitcoin/06master 140026e0e 15Wladimir J. van der Laan: Merge #8115: Avoid integer division in the benchmark inner-most loop....
 57 2016-05-31 13:10:41	0|GitHub39|[13bitcoin] 15laanwj closed pull request #8115: Avoid integer division in the benchmark inner-most loop. (06master...06better_bench) 02https://github.com/bitcoin/bitcoin/pull/8115
 58 2016-05-31 13:17:09	0|GitHub24|[13bitcoin] 15laanwj closed pull request #8097: RFC: rpc: Remove dns argument for getaddednodeinfo (06master...06remove-getaddednodeinfo-dns) 02https://github.com/bitcoin/bitcoin/pull/8097
 59 2016-05-31 13:19:24	0|GitHub183|13bitcoin/06master 141a8c4d5 15fanquake: [Doc] Add benchmarking notes
 60 2016-05-31 13:19:24	0|GitHub183|[13bitcoin] 15laanwj pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/0026e0ef3472...e7e25ea5128b
 61 2016-05-31 13:19:25	0|GitHub183|13bitcoin/06master 14e7e25ea 15Wladimir J. van der Laan: Merge #8110: [Doc] Add benchmarking notes...
 62 2016-05-31 13:19:33	0|GitHub35|[13bitcoin] 15laanwj closed pull request #8112: Include signal.h for sig_atomic_t in WIN32 (06master...06winsigatomic) 02https://github.com/bitcoin/bitcoin/pull/8112
 63 2016-05-31 13:19:34	0|GitHub145|[13bitcoin] 15laanwj closed pull request #8110: [Doc] Add benchmarking notes (06master...06mention_bench) 02https://github.com/bitcoin/bitcoin/pull/8110
 64 2016-05-31 13:27:35	0|GitHub198|13bitcoin/06master 14b682960 15Chris Stewart: Adding P2SH(p2pkh) script test case...
 65 2016-05-31 13:27:35	0|GitHub198|[13bitcoin] 15laanwj pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/e7e25ea5128b...a2df11524938
 66 2016-05-31 13:27:36	0|GitHub198|13bitcoin/06master 14a2df115 15Wladimir J. van der Laan: Merge #8090: Adding P2SH(p2pkh) script test case...
 67 2016-05-31 13:27:46	0|GitHub128|[13bitcoin] 15laanwj closed pull request #8090: Adding P2SH(p2pkh) script test case (06master...06add_p2sh_p2pkh_script_test) 02https://github.com/bitcoin/bitcoin/pull/8090
 68 2016-05-31 13:29:10	0|wumpus|jonasschnelli: I do think we use 'make deploy' in gitian builds for MacOSX
 69 2016-05-31 13:29:26	0|wumpus|I know this because I broke it at some point and gitian broke too :)
 70 2016-05-31 13:29:43	0|jonasschnelli|wumpus: hmm.. okay. Maybe it uses a different script when crosscompiling over linux.
 71 2016-05-31 13:31:31	0|wumpus|so, as gitian OSX is not broken (AFAIK) it's possibly something with fanquake's local setup
 72 2016-05-31 13:32:14	0|wumpus|not sure where background.tiff is supposed to be though
 73 2016-05-31 13:32:20	0|wumpus|could this be an out-of-tree build issue?
 74 2016-05-31 13:39:22	0|jonasschnelli|wumpus: Some weeks ago I have also encountered that a local "make deploy" on MAC does no longer work.
 75 2016-05-31 13:39:47	0|jonasschnelli|IMO its because the make deploy calls the macdeployplus script locally (on Mac only).
 76 2016-05-31 13:40:02	0|jonasschnelli|And I guess when you are cross compiling on a linux host, it does something different
 77 2016-05-31 13:40:35	0|jonasschnelli|the background.tiff is a relict. It should have be gone after lukejs auto-renaming PR
 78 2016-05-31 13:41:29	0|sipa|maybe he needs a git clean?
 79 2016-05-31 13:43:27	0|wumpus|trying that would make sense
 80 2016-05-31 13:45:10	0|GitHub98|[13bitcoin] 15laanwj closed pull request #7867: deleted restored sampler Configure.ac restored bits to all networks =(%master%masterCode[{rLi}]) (06master...06patch-1) 02https://github.com/bitcoin/bitcoin/pull/7867
 81 2016-05-31 13:47:54	0|GitHub172|13bitcoin/06master 147e908c7 15Gregory Maxwell: Do not use mempool for GETDATA for tx accepted after the last mempool req....
 82 2016-05-31 13:47:54	0|GitHub172|[13bitcoin] 15laanwj pushed 2 new commits to 06master: 02https://github.com/bitcoin/bitcoin/compare/a2df11524938...862fd24b40b4
 83 2016-05-31 13:47:55	0|GitHub172|13bitcoin/06master 14862fd24 15Wladimir J. van der Laan: Merge #8080: Do not use mempool for GETDATA for tx accepted after the last mempool req....
 84 2016-05-31 13:48:04	0|GitHub70|[13bitcoin] 15laanwj closed pull request #8080: Do not use mempool for GETDATA for tx accepted after the last mempool req. (06master...06nonmempool_getdata) 02https://github.com/bitcoin/bitcoin/pull/8080
 85 2016-05-31 13:51:06	0|GitHub158|[13bitcoin] 15laanwj closed pull request #8025: Increase DEFAULT_KEYPOOL_SIZE to 10000. (06master...062016-05-08-wallet-defaults) 02https://github.com/bitcoin/bitcoin/pull/8025
 86 2016-05-31 13:53:16	0|GitHub85|[13bitcoin] 15laanwj pushed 3 new commits to 060.12: 02https://github.com/bitcoin/bitcoin/compare/c3aedca2df89...e7ec24e336dc
 87 2016-05-31 13:53:17	0|GitHub85|13bitcoin/060.12 1487129b2 15Wladimir J. van der Laan: test: script_error checking in script_invalid tests...
 88 2016-05-31 13:53:17	0|GitHub85|13bitcoin/060.12 14e3a9ce9 15Pieter Wuille: Refactor script tests...
 89 2016-05-31 13:53:18	0|GitHub85|13bitcoin/060.12 14e7ec24e 15Wladimir J. van der Laan: Merge #8001: [0.12.2] backport script_tests improvements...
 90 2016-05-31 13:53:21	0|GitHub172|[13bitcoin] 15laanwj closed pull request #8001: [0.12.2] backport script_tests improvements (060.12...06Mf1605-012testbp) 02https://github.com/bitcoin/bitcoin/pull/8001
 91 2016-05-31 13:53:26	0|GitHub160|[13bitcoin] 15laanwj closed pull request #7854: [0.12 backport] script_tests improvements from master and #7818 (060.12...06refactorscriptests_12) 02https://github.com/bitcoin/bitcoin/pull/7854
 92 2016-05-31 14:59:39	0|luke-jr|sipa: what are we doing on https://github.com/bitcoin/bips/pull/365/files#diff-c0db26883ccab057aaa394db5e50e4b1R123 ?
 93 2016-05-31 17:47:56	0|iniana|Is there something I can do, configuration-wise, to speed up initial synchronization? Right now it is painfully slow, and I cannot find the reason why. Around 1 or 2 blocks per second is processed and download peaks are at 15KB/s
 94 2016-05-31 17:49:52	0|gmaxwell|iniana: depends on what is bottlenecking you.  If disk IO is, then increasing dbcache will help a lot.
 95 2016-05-31 17:50:16	0|iniana|gmaxwell: Will try that. Thanks.
 96 2016-05-31 17:50:39	0|gmaxwell|to do that set -dbcache=<megabytes>  don't set it more than the amount of free ram you have, minus about 600mb or so.
 97 2016-05-31 17:50:51	0|gmaxwell|iniana: What sort of hardware are you on?
 98 2016-05-31 17:54:37	0|iniana|gmaxwell: Pretty standard hardware. Core i7 (2yo), 16GB RAM. Blockchain stored on a slow HDD though, so I hope the dbcache will work
 99 2016-05-31 17:55:04	0|sipa|in that case, dbcache will help a lot
100 2016-05-31 17:55:07	0|sipa|set it to 3000 or more
101 2016-05-31 17:55:31	0|gmaxwell|yes, that will make it much faster. with 16GB ram you should be able to set it to 6000 or so and then the whole sync will basically happen in-memory, with just saving out chain to disk.
102 2016-05-31 17:57:17	0|sipa|std::deque<std::pair<int64_t, std::map<uint256, std::shared_ptr<const CTransaction>>::iterator>>
103 2016-05-31 18:00:22	0|iniana|started up again, trying with dbcache=4000, but synching stalls. log mentions reindexing every single block file. Is that necessary? Is there some way to turn it off?
104 2016-05-31 18:00:40	0|sipa|did you start with -reindex?
105 2016-05-31 18:00:53	0|paveljanik|sipa, it looks like Perl 8)
106 2016-05-31 18:01:07	0|iniana|sipa: no
107 2016-05-31 18:01:45	0|sipa|iniana: can you c/p some typical 3 line snippet from debug.log?
108 2016-05-31 18:01:49	0|iniana|sipa: I copied the entire .bitcoin from another node I had though, if that matters.
109 2016-05-31 18:02:01	0|sipa|a running one?
110 2016-05-31 18:02:47	0|iniana|sipa: no, although maybe it didn't have a normal termination?
111 2016-05-31 18:02:54	0|iniana|2016-05-31 17:57:18 Reindexing block file blk00070.dat... 2016-05-31 17:57:21 Block Import: already had block 000000000000003d64d714747fbfb615e056f533386d162f0bf5e049a9b6e0b2 at height 247000 2016-05-31 17:57:22 Reindexing block file blk00071.dat... 2016-05-31 17:57:23 Block Import: already had block 000000000000004d945017c14b75a3a58a2aa6772cacbfcaf907b3bee6d7f344 at height 248000
112 2016-05-31 18:02:58	0|gmaxwell|sipa: Why does dequeue gain the ctransaction shared_ptr iterator?
113 2016-05-31 18:03:20	0|sipa|gmaxwell: because that's smaller and faster than the hash
114 2016-05-31 18:03:40	0|sipa|the old code was stupid
115 2016-05-31 18:04:02	0|sipa|it's not a shared_ptr iterator btw, but a mapRelay iterator
116 2016-05-31 18:04:10	0|gmaxwell|sipa: isn't the iterator invalidated any time the map is updated?
117 2016-05-31 18:04:14	0|sipa|gmaxwell: nope!
118 2016-05-31 18:04:20	0|sipa|only when that specific item is deleted
119 2016-05-31 18:05:44	0|gmaxwell|okay, I understand why then.
120 2016-05-31 18:06:04	0|sipa|http://en.cppreference.com/w/cpp/container/map/insert <- "No iterators or references are invalidated. "
121 2016-05-31 18:06:32	0|sipa|i'll document that change in the commit message
122 2016-05-31 18:09:21	0|sipa|gmaxwell: oops, no, can't do!
123 2016-05-31 18:09:39	0|sipa|it can cause double deletions, as the expiration entries are not unique
124 2016-05-31 18:10:47	0|gmaxwell|I guess it cares about mutations too?  I was thinking about adding a nodeid set to maprelay later to only answer queries for peers we actually inved to.
125 2016-05-31 18:11:34	0|sipa|the value of map entries can be changed, no problems (without iterator invalidation)
126 2016-05-31 18:11:38	0|sipa|keys can't ever change
127 2016-05-31 18:11:41	0|sipa|i'll leave it be
128 2016-05-31 18:13:37	0|sipa|oh, no, it is unique
129 2016-05-31 18:14:05	0|gmaxwell|I was digging through trying to figure out how it wasn't unique...
130 2016-05-31 18:14:23	0|sipa|i'm confusing with askfor
131 2016-05-31 18:15:15	0|gmaxwell|I would like to remove the expiration entry and reinsert it if the old expiration was 'too soon'... but that doesn't change the situation for the map with the shared pointers.
132 2016-05-31 18:33:51	0|sipa|gmaxwell: since #8080, if pto->timeLastMempoolReq == 0, we don't need to check the mempool at all, right?
133 2016-05-31 18:34:11	0|gmaxwell|Right.
134 2016-05-31 19:33:17	0|phantomcircuit|iniana, it's probably a good idea to reindex if you copied ... but it shouldn't need to do so
135 2016-05-31 19:33:27	0|phantomcircuit|are you sure you exited cleanly on the other node?
136 2016-05-31 19:46:59	0|gmaxwell|phantomcircuit: I would make a bet they copied the blocks directory without the chainstate.
137 2016-05-31 19:50:30	0|sipa|that would not result in a 'reindexing' message
138 2016-05-31 22:51:39	0|GitHub133|[13bitcoin] 15theuni opened pull request #8128: Net: Turn net structures into dumb storage classes (06master...06split-resolve) 02https://github.com/bitcoin/bitcoin/pull/8128
139 2016-05-31 22:57:23	0|phantomcircuit|gmaxwell, i assume they had a node which they manually reindexed with a higher dbcache setting which wasn't cleanly shutdown
140 2016-05-31 23:14:18	0|GitHub56|[13bitcoin] 15UdjinM6 opened pull request #8129: Fix RPC console auto completer (06master...06fixRPCAutoCompleter_bitcoin) 02https://github.com/bitcoin/bitcoin/pull/8129