2 Commits (19824aa27b2cec4ee522b97abf08552ccd234a41)

Author	SHA1	Message	Date
Jules Villard	f8d7c81045	[build] switch to 4.05.0+flambda by default ... Summary: With flambda (`-O3`), compilation time is ~5x slower, but the backend is ~25% faster! To mitigate the atrocious compilation times, introduce a new `opt` build mode in the jbuilder files. - build in "opt" mode by default from the toplevel (so that install scripts and external users get the fastest infer by default), in "default" mode by default from infer/src (since the latter is only called directly by infer devs, for faster builds) - `make byte` is as fast as before in any mode - `make test` will build "opt" by default, which is very slow. Solution for testing (or building the models) locally: `make BUILD_MODE=default test`. - You can even change the default locally with `export BUILD_MODE=default`. The benchmarks are to be taken with a sizable pinch of salt because I ran them only once and other stuff could be running in the background. That said, the perf win is consistent across all projects, with 15-20% win in wallclock time and around 25% win in total CPU time, ~9% win in sys time, and ~25% fewer minor allocations, and ~5-10% fewer overall allocations. This is only for the backend; the capture is by and large unaffected (either the same or a tad faster within noise range). Here are the results running on OpenSSL 1.0.2d on osx (12 cores, 32G RAM) === base infer binary: 26193088 bytes compile time: 40s capture: ```lang=text real 1m7.513s user 3m11.437s sys 0m55.236s ``` analysis: ```lang=text real 5m41.580s user 61m37.855s sys 1m12.870s ``` Memory profile: ```lang=json { ... "minor_gb": 0.1534719169139862, "promoted_gb": 0.0038930922746658325, "major_gb": 0.4546157643198967, "allocated_gb": 0.6041945889592171, "minor_collections": 78, "major_collections": 23, "compactions": 7, "top_heap_gb": 0.07388687133789062, "stack_kb": 0.3984375, "minor_heap_kb": 8192.0, ... } ``` === flambda with stock options (no `-Oclassic`, just the same flags as base) Exactly the same as base. === flambda `-O3` infer binary: 56870376 bytes (2.17x bigger) compile time: 191s (4.78x slower) capture is the same as base: ```lang=text real 1m9.203s user 3m12.242s sys 0m58.905s ``` analysis is ~20% wallclock time faster, ~25% CPU time faster: ```lang=text real 4m32.656s user 46m43.987s sys 1m2.424s ``` memory usage is a bit lower too: ```lang=json { ... "minor_gb": 0.11583046615123749, // 75% of previous "promoted_gb": 0.00363825261592865, // 93% of previous "major_gb": 0.45415670424699783, // about same "allocated_gb": 0.5663489177823067, // 94% of previous "minor_collections": 73, "major_collections": 22, "compactions": 7, "top_heap_gb": 0.07165145874023438, "stack_kb": 0.3359375, "minor_heap_kb": 8192.0, ... } ``` === flambda `-O2` Not nearly as exciting as `-O3`, but the compilation cost is still quite high: infer: 37826856 bytes compilation of infer: 100s Capture and analysis timings are mostly the same as base. Reviewed By: jberdine Differential Revision: D4867979 fbshipit-source-id: 99230b7	7 years ago
Jules Villard	8de2b88783	[make] s/ocamlbuild/jbuilder/g ... Summary: Use jbuilder to build infer instead of ocamlbuild. This is mainly to get faster builds: ``` times in 10ms, ±differences measured in speedups, 4 cores | | ocb total | jb | ±total | ocb user | jb | ±user | ocb cpu | jb | ±cpu | ocb sys | jb | ±sys | |-----------------------------------+-----------+------+--------+----------+------+-------+---------+-----+------+---------+------+------| | byte from scratch | 6428 | 2456 | 2.62 | 7743 | 6662 | 1.16 | 138 | 331 | 2.40 | 1184 | 1477 | 0.80 | | native from scratch | 9841 | 4289 | 2.29 | 9530 | 8834 | 1.08 | 110 | 245 | 2.23 | 1373 | 1712 | 0.80 | | byte after native | 29578 | 1602 | 18.46 | 4514 | 4640 | 0.97 | 170 | 325 | 1.91 | 543 | 576 | 0.94 | | change infer.ml byte | 344 | 282 | 1.22 | 292 | 215 | 1.36 | 96 | 99 | 1.03 | 040 | 066 | 0.61 | | change infer.ml native | 837 | 223 | 3.75 | 789 | 174 | 4.53 | 98 | 99 | 1.01 | 036 | 47 | 0.77 | | change Config.ml byte | 451 | 339 | 1.33 | 382 | 336 | 1.14 | 97 | 122 | 1.26 | 056 | 80 | 0.70 | | change Config.ml native | 4024 | 1760 | 2.29 | 4585 | 4225 | 1.09 | 127 | 276 | 2.17 | 559 | 644 | 0.87 | | change cFrontend_config.ml byte | 348 | 643 | 0.54 | 297 | 330 | 0.90 | 96 | 67 | 0.70 | 038 | 102 | 0.37 | | change cFrontend_config.ml native | 1480 | 584 | 2.53 | 1435 | 906 | 1.58 | 106 | 185 | 1.75 | 136 | 178 | 0.76 | #+TBLFM: $4=$2/$3;f2::$7=$5/$6;f2::$10=$9/$8;f2::$13=$11/$12;f2 50 cores | | ocb total | jb | ±total | ocb user | jb | ±user | ocb cpu | jb | ±cpu | ocb sys | jb | ±sys | |---------------------+-----------+------+--------+----------+------+-------+---------+----+------+---------+------+------| | byte from scratch | 9114 | 2061 | 4.42 | 9334 | 5133 | 1.82 | | | 0/0 | 2566 | 1726 | 1.49 | | native from scratch | 13481 | 3967 | 3.40 | 12291 | 7608 | 1.62 | | | 0/0 | 3003 | 2100 | 1.43 | | byte after native | 3467 | 1476 | 2.35 | 5067 | 3912 | 1.30 | | | 0/0 | 971 | 801 | 1.21 | #+TBLFM: $4=$2/$3;f2::$7=$5/$6;f2::$10=$9/$8;f2::$13=$11/$12;f2 ``` Menu: 1. Write a jbuild file, autogenerated from jbuild.in because we need to fill in some information at build-time (really, at configure time, but TODO), such as whether or not clang is enabled. 2. Nuke lots of stuff from infer/src/Makefile that is now in the jbuild file 3. The jbuild file lives in infer/src/ so it can see all the sources. If we put it somewhere else, eg, infer/, then `jbuilder` scans too many files (all irrelevant) and takes 2.5s to start instead of .8s. Adding irrelevant directories to jbuild-ignore does not help. 4. jbuilder does not support subdirectories, so resort to listing all the source files in the generated jbuild (only source directories need to be manually listed in jbuild.in though). Still, the generated .merlin is wrong and makes merlin find source files in _build, so manually tune it to get good merlin support. We also lose some of merlin for unit tests as it cannot see their build artefacts anymore. 5. checkCopyright gets its own jbuild because it's standalone. Also, remove some deprecation warnings in checkCopyright due to the new version of Core from a while ago. 6. Drop less-used Makefile features (they had regressed anyway) such as building individual modules. Also, building mod_dep.pdf now takes all the source files available so they better build (before, it would only take the source files from the config, eg with or without clang) (that's pretty minor). 7. The toplevel is now built as a custom toplevel because that was easier. It should soon be even easier: https://github.com/janestreet/jbuilder/issues/210 8. Move BUILTINS.mli to BUILTINS.ml because jbuilder is not happy about interface files without implementations. In particular, I did not try to migrate too much of the Makefile logic to jbuilder, more can be done in the future. Reviewed By: jberdine Differential Revision: D5573661 fbshipit-source-id: 4ca6d8f	7 years ago