You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

176 lines
3.7 KiB

[make] s/ocamlbuild/jbuilder/g Summary: Use jbuilder to build infer instead of ocamlbuild. This is mainly to get faster builds: ``` times in 10ms, ±differences measured in speedups, 4 cores | | ocb total | jb | ±total | ocb user | jb | ±user | ocb cpu | jb | ±cpu | ocb sys | jb | ±sys | |-----------------------------------+-----------+------+--------+----------+------+-------+---------+-----+------+---------+------+------| | byte from scratch | 6428 | 2456 | 2.62 | 7743 | 6662 | 1.16 | 138 | 331 | 2.40 | 1184 | 1477 | 0.80 | | native from scratch | 9841 | 4289 | 2.29 | 9530 | 8834 | 1.08 | 110 | 245 | 2.23 | 1373 | 1712 | 0.80 | | byte after native | 29578 | 1602 | 18.46 | 4514 | 4640 | 0.97 | 170 | 325 | 1.91 | 543 | 576 | 0.94 | | change infer.ml byte | 344 | 282 | 1.22 | 292 | 215 | 1.36 | 96 | 99 | 1.03 | 040 | 066 | 0.61 | | change infer.ml native | 837 | 223 | 3.75 | 789 | 174 | 4.53 | 98 | 99 | 1.01 | 036 | 47 | 0.77 | | change Config.ml byte | 451 | 339 | 1.33 | 382 | 336 | 1.14 | 97 | 122 | 1.26 | 056 | 80 | 0.70 | | change Config.ml native | 4024 | 1760 | 2.29 | 4585 | 4225 | 1.09 | 127 | 276 | 2.17 | 559 | 644 | 0.87 | | change cFrontend_config.ml byte | 348 | 643 | 0.54 | 297 | 330 | 0.90 | 96 | 67 | 0.70 | 038 | 102 | 0.37 | | change cFrontend_config.ml native | 1480 | 584 | 2.53 | 1435 | 906 | 1.58 | 106 | 185 | 1.75 | 136 | 178 | 0.76 | #+TBLFM: $4=$2/$3;f2::$7=$5/$6;f2::$10=$9/$8;f2::$13=$11/$12;f2 50 cores | | ocb total | jb | ±total | ocb user | jb | ±user | ocb cpu | jb | ±cpu | ocb sys | jb | ±sys | |---------------------+-----------+------+--------+----------+------+-------+---------+----+------+---------+------+------| | byte from scratch | 9114 | 2061 | 4.42 | 9334 | 5133 | 1.82 | | | 0/0 | 2566 | 1726 | 1.49 | | native from scratch | 13481 | 3967 | 3.40 | 12291 | 7608 | 1.62 | | | 0/0 | 3003 | 2100 | 1.43 | | byte after native | 3467 | 1476 | 2.35 | 5067 | 3912 | 1.30 | | | 0/0 | 971 | 801 | 1.21 | #+TBLFM: $4=$2/$3;f2::$7=$5/$6;f2::$10=$9/$8;f2::$13=$11/$12;f2 ``` Menu: 1. Write a jbuild file, autogenerated from jbuild.in because we need to fill in some information at build-time (really, at configure time, but TODO), such as whether or not clang is enabled. 2. Nuke lots of stuff from infer/src/Makefile that is now in the jbuild file 3. The jbuild file lives in infer/src/ so it can see all the sources. If we put it somewhere else, eg, infer/, then `jbuilder` scans too many files (all irrelevant) and takes 2.5s to start instead of .8s. Adding irrelevant directories to jbuild-ignore does not help. 4. jbuilder does not support subdirectories, so resort to listing all the source files in the generated jbuild (only source directories need to be manually listed in jbuild.in though). Still, the generated .merlin is wrong and makes merlin find source files in _build, so manually tune it to get good merlin support. We also lose some of merlin for unit tests as it cannot see their build artefacts anymore. 5. checkCopyright gets its own jbuild because it's standalone. Also, remove some deprecation warnings in checkCopyright due to the new version of Core from a while ago. 6. Drop less-used Makefile features (they had regressed anyway) such as building individual modules. Also, building mod_dep.pdf now takes all the source files available so they better build (before, it would only take the source files from the config, eg with or without clang) (that's pretty minor). 7. The toplevel is now built as a custom toplevel because that was easier. It should soon be even easier: https://github.com/janestreet/jbuilder/issues/210 8. Move BUILTINS.mli to BUILTINS.ml because jbuilder is not happy about interface files without implementations. In particular, I did not try to migrate too much of the Makefile logic to jbuilder, more can be done in the future. Reviewed By: jberdine Differential Revision: D5573661 fbshipit-source-id: 4ca6d8f
7 years ago
(* -*- tuareg -*- *)
(*
* Copyright (c) Facebook, Inc. and its affiliates.
*
* This source code is licensed under the MIT license found in the
* LICENSE file in the root directory of this source tree.
*)
(* NOTE: prepend dune.common to this file! *)
[make] s/ocamlbuild/jbuilder/g Summary: Use jbuilder to build infer instead of ocamlbuild. This is mainly to get faster builds: ``` times in 10ms, ±differences measured in speedups, 4 cores | | ocb total | jb | ±total | ocb user | jb | ±user | ocb cpu | jb | ±cpu | ocb sys | jb | ±sys | |-----------------------------------+-----------+------+--------+----------+------+-------+---------+-----+------+---------+------+------| | byte from scratch | 6428 | 2456 | 2.62 | 7743 | 6662 | 1.16 | 138 | 331 | 2.40 | 1184 | 1477 | 0.80 | | native from scratch | 9841 | 4289 | 2.29 | 9530 | 8834 | 1.08 | 110 | 245 | 2.23 | 1373 | 1712 | 0.80 | | byte after native | 29578 | 1602 | 18.46 | 4514 | 4640 | 0.97 | 170 | 325 | 1.91 | 543 | 576 | 0.94 | | change infer.ml byte | 344 | 282 | 1.22 | 292 | 215 | 1.36 | 96 | 99 | 1.03 | 040 | 066 | 0.61 | | change infer.ml native | 837 | 223 | 3.75 | 789 | 174 | 4.53 | 98 | 99 | 1.01 | 036 | 47 | 0.77 | | change Config.ml byte | 451 | 339 | 1.33 | 382 | 336 | 1.14 | 97 | 122 | 1.26 | 056 | 80 | 0.70 | | change Config.ml native | 4024 | 1760 | 2.29 | 4585 | 4225 | 1.09 | 127 | 276 | 2.17 | 559 | 644 | 0.87 | | change cFrontend_config.ml byte | 348 | 643 | 0.54 | 297 | 330 | 0.90 | 96 | 67 | 0.70 | 038 | 102 | 0.37 | | change cFrontend_config.ml native | 1480 | 584 | 2.53 | 1435 | 906 | 1.58 | 106 | 185 | 1.75 | 136 | 178 | 0.76 | #+TBLFM: $4=$2/$3;f2::$7=$5/$6;f2::$10=$9/$8;f2::$13=$11/$12;f2 50 cores | | ocb total | jb | ±total | ocb user | jb | ±user | ocb cpu | jb | ±cpu | ocb sys | jb | ±sys | |---------------------+-----------+------+--------+----------+------+-------+---------+----+------+---------+------+------| | byte from scratch | 9114 | 2061 | 4.42 | 9334 | 5133 | 1.82 | | | 0/0 | 2566 | 1726 | 1.49 | | native from scratch | 13481 | 3967 | 3.40 | 12291 | 7608 | 1.62 | | | 0/0 | 3003 | 2100 | 1.43 | | byte after native | 3467 | 1476 | 2.35 | 5067 | 3912 | 1.30 | | | 0/0 | 971 | 801 | 1.21 | #+TBLFM: $4=$2/$3;f2::$7=$5/$6;f2::$10=$9/$8;f2::$13=$11/$12;f2 ``` Menu: 1. Write a jbuild file, autogenerated from jbuild.in because we need to fill in some information at build-time (really, at configure time, but TODO), such as whether or not clang is enabled. 2. Nuke lots of stuff from infer/src/Makefile that is now in the jbuild file 3. The jbuild file lives in infer/src/ so it can see all the sources. If we put it somewhere else, eg, infer/, then `jbuilder` scans too many files (all irrelevant) and takes 2.5s to start instead of .8s. Adding irrelevant directories to jbuild-ignore does not help. 4. jbuilder does not support subdirectories, so resort to listing all the source files in the generated jbuild (only source directories need to be manually listed in jbuild.in though). Still, the generated .merlin is wrong and makes merlin find source files in _build, so manually tune it to get good merlin support. We also lose some of merlin for unit tests as it cannot see their build artefacts anymore. 5. checkCopyright gets its own jbuild because it's standalone. Also, remove some deprecation warnings in checkCopyright due to the new version of Core from a while ago. 6. Drop less-used Makefile features (they had regressed anyway) such as building individual modules. Also, building mod_dep.pdf now takes all the source files available so they better build (before, it would only take the source files from the config, eg with or without clang) (that's pretty minor). 7. The toplevel is now built as a custom toplevel because that was easier. It should soon be even easier: https://github.com/janestreet/jbuilder/issues/210 8. Move BUILTINS.mli to BUILTINS.ml because jbuilder is not happy about interface files without implementations. In particular, I did not try to migrate too much of the Makefile logic to jbuilder, more can be done in the future. Reviewed By: jberdine Differential Revision: D5573661 fbshipit-source-id: 4ca6d8f
7 years ago
let ( ^/ ) = Filename.concat
let source_dirs =
(if clang then ["al"; "clang"; "unit" ^/ "clang"] else ["clang_stubs"; "unit" ^/ "clang_stubs"])
@ [ (if java then "java" else "java_stubs")
; "integration"
; "test_determinator"
; "unit"
; "unit" ^/ "nullsafe" ]
[make] s/ocamlbuild/jbuilder/g Summary: Use jbuilder to build infer instead of ocamlbuild. This is mainly to get faster builds: ``` times in 10ms, ±differences measured in speedups, 4 cores | | ocb total | jb | ±total | ocb user | jb | ±user | ocb cpu | jb | ±cpu | ocb sys | jb | ±sys | |-----------------------------------+-----------+------+--------+----------+------+-------+---------+-----+------+---------+------+------| | byte from scratch | 6428 | 2456 | 2.62 | 7743 | 6662 | 1.16 | 138 | 331 | 2.40 | 1184 | 1477 | 0.80 | | native from scratch | 9841 | 4289 | 2.29 | 9530 | 8834 | 1.08 | 110 | 245 | 2.23 | 1373 | 1712 | 0.80 | | byte after native | 29578 | 1602 | 18.46 | 4514 | 4640 | 0.97 | 170 | 325 | 1.91 | 543 | 576 | 0.94 | | change infer.ml byte | 344 | 282 | 1.22 | 292 | 215 | 1.36 | 96 | 99 | 1.03 | 040 | 066 | 0.61 | | change infer.ml native | 837 | 223 | 3.75 | 789 | 174 | 4.53 | 98 | 99 | 1.01 | 036 | 47 | 0.77 | | change Config.ml byte | 451 | 339 | 1.33 | 382 | 336 | 1.14 | 97 | 122 | 1.26 | 056 | 80 | 0.70 | | change Config.ml native | 4024 | 1760 | 2.29 | 4585 | 4225 | 1.09 | 127 | 276 | 2.17 | 559 | 644 | 0.87 | | change cFrontend_config.ml byte | 348 | 643 | 0.54 | 297 | 330 | 0.90 | 96 | 67 | 0.70 | 038 | 102 | 0.37 | | change cFrontend_config.ml native | 1480 | 584 | 2.53 | 1435 | 906 | 1.58 | 106 | 185 | 1.75 | 136 | 178 | 0.76 | #+TBLFM: $4=$2/$3;f2::$7=$5/$6;f2::$10=$9/$8;f2::$13=$11/$12;f2 50 cores | | ocb total | jb | ±total | ocb user | jb | ±user | ocb cpu | jb | ±cpu | ocb sys | jb | ±sys | |---------------------+-----------+------+--------+----------+------+-------+---------+----+------+---------+------+------| | byte from scratch | 9114 | 2061 | 4.42 | 9334 | 5133 | 1.82 | | | 0/0 | 2566 | 1726 | 1.49 | | native from scratch | 13481 | 3967 | 3.40 | 12291 | 7608 | 1.62 | | | 0/0 | 3003 | 2100 | 1.43 | | byte after native | 3467 | 1476 | 2.35 | 5067 | 3912 | 1.30 | | | 0/0 | 971 | 801 | 1.21 | #+TBLFM: $4=$2/$3;f2::$7=$5/$6;f2::$10=$9/$8;f2::$13=$11/$12;f2 ``` Menu: 1. Write a jbuild file, autogenerated from jbuild.in because we need to fill in some information at build-time (really, at configure time, but TODO), such as whether or not clang is enabled. 2. Nuke lots of stuff from infer/src/Makefile that is now in the jbuild file 3. The jbuild file lives in infer/src/ so it can see all the sources. If we put it somewhere else, eg, infer/, then `jbuilder` scans too many files (all irrelevant) and takes 2.5s to start instead of .8s. Adding irrelevant directories to jbuild-ignore does not help. 4. jbuilder does not support subdirectories, so resort to listing all the source files in the generated jbuild (only source directories need to be manually listed in jbuild.in though). Still, the generated .merlin is wrong and makes merlin find source files in _build, so manually tune it to get good merlin support. We also lose some of merlin for unit tests as it cannot see their build artefacts anymore. 5. checkCopyright gets its own jbuild because it's standalone. Also, remove some deprecation warnings in checkCopyright due to the new version of Core from a while ago. 6. Drop less-used Makefile features (they had regressed anyway) such as building individual modules. Also, building mod_dep.pdf now takes all the source files available so they better build (before, it would only take the source files from the config, eg with or without clang) (that's pretty minor). 7. The toplevel is now built as a custom toplevel because that was easier. It should soon be even easier: https://github.com/janestreet/jbuilder/issues/210 8. Move BUILTINS.mli to BUILTINS.ml because jbuilder is not happy about interface files without implementations. In particular, I did not try to migrate too much of the Makefile logic to jbuilder, more can be done in the future. Reviewed By: jberdine Differential Revision: D5573661 fbshipit-source-id: 4ca6d8f
7 years ago
let infer_binaries = ["infer"; "inferunit"] @ if facebook then ["InferCreateTraceViewLinks"] else []
[RFC][build] Use dune environments and profiles instead of contexts Summary: With profiles and `(env ...)` stanza it's possible to consolidate various ocamlc/ocamlopt/etc setups in a single place. Where previously we needed to append `dune.common` to every dune file and specify `flags` and `ocamlopt_flags` now the flags are specified in `env` and applied accross the board. This allows to 1. simplify build definitions, 2. avoid the need to generate dune files, 3. use plain sexps instead of OCaml and JBuilder plugin in build files. (I'll try to address 2 and 3 in the followup patches). Existing `make` targets should continue working as before. Also, we can use dune CLI like so: ``` infer/src$ dune printenv --profile opt # <- very useful for introspection infer/src$ dune build check infer/src$ dune build check --profile test infer/src$ dune build infer.exe --profile dev infer/src$ dune build infer.exe --profile opt ``` Also, with just 1 context something like `dune runtest` will run unit tests only once instead of N times, where N is the number of contexts. Now, there's one difference compared to the previous setup with contexts: - Previously, each context had its own build folder, and building infer in opt context didn't invalidate any of the build artifacts in default context. Therefore, alternating between `make` and `make opt` had low overhead at the expense of having N copies of all build artifacts (1 for every context). - Now, there's just 1 build folder and switching between profiles does invalidate some artifacts (but not all) and rebuild takes a bit more time. So, if you're alternating like crazy between profiles your experience may get worse (but not necessarily, more on that below). If you want to trigger an opt build occasionally, you shouldn't notice much difference. For those who are concerned about slower build times when alternating between different build profiles, there's a solution: [dune cache](https://dune.readthedocs.io/en/stable/caching.html). You can enable it by creating a file `~/.config/dune/config` with the following contents: ``` (lang dune 2.0) (cache enabled) ``` With cache enabled switching between different build profiles (but also branches and commits) has ~0 overhead. Dune cache works fine on Linux, but unfortunately there are [certain problems with MacOS](https://github.com/ocaml/dune/issues/3233) (hopefully, those will be fixed soon and then there will be no downsides to using profiles compared to contexts for our case). Reviewed By: jvillard Differential Revision: D20247864 fbshipit-source-id: 5f8afa0db
5 years ago
let env_stanza =
Format.sprintf
{|
(env
(dev
(flags %s))
(opt
(flags %s)
(ocamlopt_flags (:standard -O3)))
(test
(flags %s))
)
[RFC][build] Use dune environments and profiles instead of contexts Summary: With profiles and `(env ...)` stanza it's possible to consolidate various ocamlc/ocamlopt/etc setups in a single place. Where previously we needed to append `dune.common` to every dune file and specify `flags` and `ocamlopt_flags` now the flags are specified in `env` and applied accross the board. This allows to 1. simplify build definitions, 2. avoid the need to generate dune files, 3. use plain sexps instead of OCaml and JBuilder plugin in build files. (I'll try to address 2 and 3 in the followup patches). Existing `make` targets should continue working as before. Also, we can use dune CLI like so: ``` infer/src$ dune printenv --profile opt # <- very useful for introspection infer/src$ dune build check infer/src$ dune build check --profile test infer/src$ dune build infer.exe --profile dev infer/src$ dune build infer.exe --profile opt ``` Also, with just 1 context something like `dune runtest` will run unit tests only once instead of N times, where N is the number of contexts. Now, there's one difference compared to the previous setup with contexts: - Previously, each context had its own build folder, and building infer in opt context didn't invalidate any of the build artifacts in default context. Therefore, alternating between `make` and `make opt` had low overhead at the expense of having N copies of all build artifacts (1 for every context). - Now, there's just 1 build folder and switching between profiles does invalidate some artifacts (but not all) and rebuild takes a bit more time. So, if you're alternating like crazy between profiles your experience may get worse (but not necessarily, more on that below). If you want to trigger an opt build occasionally, you shouldn't notice much difference. For those who are concerned about slower build times when alternating between different build profiles, there's a solution: [dune cache](https://dune.readthedocs.io/en/stable/caching.html). You can enable it by creating a file `~/.config/dune/config` with the following contents: ``` (lang dune 2.0) (cache enabled) ``` With cache enabled switching between different build profiles (but also branches and commits) has ~0 overhead. Dune cache works fine on Linux, but unfortunately there are [certain problems with MacOS](https://github.com/ocaml/dune/issues/3233) (hopefully, those will be fixed soon and then there will be no downsides to using profiles compared to contexts for our case). Reviewed By: jvillard Differential Revision: D20247864 fbshipit-source-id: 5f8afa0db
5 years ago
|}
lenient_flags lenient_flags strict_flags
let clang_lexer_stanzas =
if clang then ["(ocamllex types_lexer ctl_lexer)"; "(menhir (modules types_parser ctl_parser))"]
else []
[make] s/ocamlbuild/jbuilder/g Summary: Use jbuilder to build infer instead of ocamlbuild. This is mainly to get faster builds: ``` times in 10ms, ±differences measured in speedups, 4 cores | | ocb total | jb | ±total | ocb user | jb | ±user | ocb cpu | jb | ±cpu | ocb sys | jb | ±sys | |-----------------------------------+-----------+------+--------+----------+------+-------+---------+-----+------+---------+------+------| | byte from scratch | 6428 | 2456 | 2.62 | 7743 | 6662 | 1.16 | 138 | 331 | 2.40 | 1184 | 1477 | 0.80 | | native from scratch | 9841 | 4289 | 2.29 | 9530 | 8834 | 1.08 | 110 | 245 | 2.23 | 1373 | 1712 | 0.80 | | byte after native | 29578 | 1602 | 18.46 | 4514 | 4640 | 0.97 | 170 | 325 | 1.91 | 543 | 576 | 0.94 | | change infer.ml byte | 344 | 282 | 1.22 | 292 | 215 | 1.36 | 96 | 99 | 1.03 | 040 | 066 | 0.61 | | change infer.ml native | 837 | 223 | 3.75 | 789 | 174 | 4.53 | 98 | 99 | 1.01 | 036 | 47 | 0.77 | | change Config.ml byte | 451 | 339 | 1.33 | 382 | 336 | 1.14 | 97 | 122 | 1.26 | 056 | 80 | 0.70 | | change Config.ml native | 4024 | 1760 | 2.29 | 4585 | 4225 | 1.09 | 127 | 276 | 2.17 | 559 | 644 | 0.87 | | change cFrontend_config.ml byte | 348 | 643 | 0.54 | 297 | 330 | 0.90 | 96 | 67 | 0.70 | 038 | 102 | 0.37 | | change cFrontend_config.ml native | 1480 | 584 | 2.53 | 1435 | 906 | 1.58 | 106 | 185 | 1.75 | 136 | 178 | 0.76 | #+TBLFM: $4=$2/$3;f2::$7=$5/$6;f2::$10=$9/$8;f2::$13=$11/$12;f2 50 cores | | ocb total | jb | ±total | ocb user | jb | ±user | ocb cpu | jb | ±cpu | ocb sys | jb | ±sys | |---------------------+-----------+------+--------+----------+------+-------+---------+----+------+---------+------+------| | byte from scratch | 9114 | 2061 | 4.42 | 9334 | 5133 | 1.82 | | | 0/0 | 2566 | 1726 | 1.49 | | native from scratch | 13481 | 3967 | 3.40 | 12291 | 7608 | 1.62 | | | 0/0 | 3003 | 2100 | 1.43 | | byte after native | 3467 | 1476 | 2.35 | 5067 | 3912 | 1.30 | | | 0/0 | 971 | 801 | 1.21 | #+TBLFM: $4=$2/$3;f2::$7=$5/$6;f2::$10=$9/$8;f2::$13=$11/$12;f2 ``` Menu: 1. Write a jbuild file, autogenerated from jbuild.in because we need to fill in some information at build-time (really, at configure time, but TODO), such as whether or not clang is enabled. 2. Nuke lots of stuff from infer/src/Makefile that is now in the jbuild file 3. The jbuild file lives in infer/src/ so it can see all the sources. If we put it somewhere else, eg, infer/, then `jbuilder` scans too many files (all irrelevant) and takes 2.5s to start instead of .8s. Adding irrelevant directories to jbuild-ignore does not help. 4. jbuilder does not support subdirectories, so resort to listing all the source files in the generated jbuild (only source directories need to be manually listed in jbuild.in though). Still, the generated .merlin is wrong and makes merlin find source files in _build, so manually tune it to get good merlin support. We also lose some of merlin for unit tests as it cannot see their build artefacts anymore. 5. checkCopyright gets its own jbuild because it's standalone. Also, remove some deprecation warnings in checkCopyright due to the new version of Core from a while ago. 6. Drop less-used Makefile features (they had regressed anyway) such as building individual modules. Also, building mod_dep.pdf now takes all the source files available so they better build (before, it would only take the source files from the config, eg with or without clang) (that's pretty minor). 7. The toplevel is now built as a custom toplevel because that was easier. It should soon be even easier: https://github.com/janestreet/jbuilder/issues/210 8. Move BUILTINS.mli to BUILTINS.ml because jbuilder is not happy about interface files without implementations. In particular, I did not try to migrate too much of the Makefile logic to jbuilder, more can be done in the future. Reviewed By: jberdine Differential Revision: D5573661 fbshipit-source-id: 4ca6d8f
7 years ago
let infer_cflags =
[RFC][build] Use dune environments and profiles instead of contexts Summary: With profiles and `(env ...)` stanza it's possible to consolidate various ocamlc/ocamlopt/etc setups in a single place. Where previously we needed to append `dune.common` to every dune file and specify `flags` and `ocamlopt_flags` now the flags are specified in `env` and applied accross the board. This allows to 1. simplify build definitions, 2. avoid the need to generate dune files, 3. use plain sexps instead of OCaml and JBuilder plugin in build files. (I'll try to address 2 and 3 in the followup patches). Existing `make` targets should continue working as before. Also, we can use dune CLI like so: ``` infer/src$ dune printenv --profile opt # <- very useful for introspection infer/src$ dune build check infer/src$ dune build check --profile test infer/src$ dune build infer.exe --profile dev infer/src$ dune build infer.exe --profile opt ``` Also, with just 1 context something like `dune runtest` will run unit tests only once instead of N times, where N is the number of contexts. Now, there's one difference compared to the previous setup with contexts: - Previously, each context had its own build folder, and building infer in opt context didn't invalidate any of the build artifacts in default context. Therefore, alternating between `make` and `make opt` had low overhead at the expense of having N copies of all build artifacts (1 for every context). - Now, there's just 1 build folder and switching between profiles does invalidate some artifacts (but not all) and rebuild takes a bit more time. So, if you're alternating like crazy between profiles your experience may get worse (but not necessarily, more on that below). If you want to trigger an opt build occasionally, you shouldn't notice much difference. For those who are concerned about slower build times when alternating between different build profiles, there's a solution: [dune cache](https://dune.readthedocs.io/en/stable/caching.html). You can enable it by creating a file `~/.config/dune/config` with the following contents: ``` (lang dune 2.0) (cache enabled) ``` With cache enabled switching between different build profiles (but also branches and commits) has ~0 overhead. Dune cache works fine on Linux, but unfortunately there are [certain problems with MacOS](https://github.com/ocaml/dune/issues/3233) (hopefully, those will be fixed soon and then there will be no downsides to using profiles compared to contexts for our case). Reviewed By: jvillard Differential Revision: D20247864 fbshipit-source-id: 5f8afa0db
5 years ago
[ "-open"
; "Core"
; "-open"
; "Biabduction"
; "-open"
; "Bo"
; "-open"
; "Nullsafe"
; "-open"
; "Pulselib"
; "-open"
; "Checkers"
; "-open"
; "Costlib"
; "-open"
; "Quandary"
; "-open"
; "TOPLlib"
; "-open"
; "Concurrency"
; "-open"
; "Labs"
; "-open"
; "Absint"
; "-open"
; "OpenSource"
; "-open"
[RFC][build] Use dune environments and profiles instead of contexts Summary: With profiles and `(env ...)` stanza it's possible to consolidate various ocamlc/ocamlopt/etc setups in a single place. Where previously we needed to append `dune.common` to every dune file and specify `flags` and `ocamlopt_flags` now the flags are specified in `env` and applied accross the board. This allows to 1. simplify build definitions, 2. avoid the need to generate dune files, 3. use plain sexps instead of OCaml and JBuilder plugin in build files. (I'll try to address 2 and 3 in the followup patches). Existing `make` targets should continue working as before. Also, we can use dune CLI like so: ``` infer/src$ dune printenv --profile opt # <- very useful for introspection infer/src$ dune build check infer/src$ dune build check --profile test infer/src$ dune build infer.exe --profile dev infer/src$ dune build infer.exe --profile opt ``` Also, with just 1 context something like `dune runtest` will run unit tests only once instead of N times, where N is the number of contexts. Now, there's one difference compared to the previous setup with contexts: - Previously, each context had its own build folder, and building infer in opt context didn't invalidate any of the build artifacts in default context. Therefore, alternating between `make` and `make opt` had low overhead at the expense of having N copies of all build artifacts (1 for every context). - Now, there's just 1 build folder and switching between profiles does invalidate some artifacts (but not all) and rebuild takes a bit more time. So, if you're alternating like crazy between profiles your experience may get worse (but not necessarily, more on that below). If you want to trigger an opt build occasionally, you shouldn't notice much difference. For those who are concerned about slower build times when alternating between different build profiles, there's a solution: [dune cache](https://dune.readthedocs.io/en/stable/caching.html). You can enable it by creating a file `~/.config/dune/config` with the following contents: ``` (lang dune 2.0) (cache enabled) ``` With cache enabled switching between different build profiles (but also branches and commits) has ~0 overhead. Dune cache works fine on Linux, but unfortunately there are [certain problems with MacOS](https://github.com/ocaml/dune/issues/3233) (hopefully, those will be fixed soon and then there will be no downsides to using profiles compared to contexts for our case). Reviewed By: jvillard Differential Revision: D20247864 fbshipit-source-id: 5f8afa0db
5 years ago
; "InferStdlib"
; "-open"
; "IStd"
; "-open"
; "InferGenerated"
; "-open"
; "InferIR"
; "-open"
; "InferBase"
; "-open"
; "InferCStubs"
; "-open"
; "Backend" ]
[RFC][build] Use dune environments and profiles instead of contexts Summary: With profiles and `(env ...)` stanza it's possible to consolidate various ocamlc/ocamlopt/etc setups in a single place. Where previously we needed to append `dune.common` to every dune file and specify `flags` and `ocamlopt_flags` now the flags are specified in `env` and applied accross the board. This allows to 1. simplify build definitions, 2. avoid the need to generate dune files, 3. use plain sexps instead of OCaml and JBuilder plugin in build files. (I'll try to address 2 and 3 in the followup patches). Existing `make` targets should continue working as before. Also, we can use dune CLI like so: ``` infer/src$ dune printenv --profile opt # <- very useful for introspection infer/src$ dune build check infer/src$ dune build check --profile test infer/src$ dune build infer.exe --profile dev infer/src$ dune build infer.exe --profile opt ``` Also, with just 1 context something like `dune runtest` will run unit tests only once instead of N times, where N is the number of contexts. Now, there's one difference compared to the previous setup with contexts: - Previously, each context had its own build folder, and building infer in opt context didn't invalidate any of the build artifacts in default context. Therefore, alternating between `make` and `make opt` had low overhead at the expense of having N copies of all build artifacts (1 for every context). - Now, there's just 1 build folder and switching between profiles does invalidate some artifacts (but not all) and rebuild takes a bit more time. So, if you're alternating like crazy between profiles your experience may get worse (but not necessarily, more on that below). If you want to trigger an opt build occasionally, you shouldn't notice much difference. For those who are concerned about slower build times when alternating between different build profiles, there's a solution: [dune cache](https://dune.readthedocs.io/en/stable/caching.html). You can enable it by creating a file `~/.config/dune/config` with the following contents: ``` (lang dune 2.0) (cache enabled) ``` With cache enabled switching between different build profiles (but also branches and commits) has ~0 overhead. Dune cache works fine on Linux, but unfortunately there are [certain problems with MacOS](https://github.com/ocaml/dune/issues/3233) (hopefully, those will be fixed soon and then there will be no downsides to using profiles compared to contexts for our case). Reviewed By: jvillard Differential Revision: D20247864 fbshipit-source-id: 5f8afa0db
5 years ago
|> String.concat " "
[make] s/ocamlbuild/jbuilder/g Summary: Use jbuilder to build infer instead of ocamlbuild. This is mainly to get faster builds: ``` times in 10ms, ±differences measured in speedups, 4 cores | | ocb total | jb | ±total | ocb user | jb | ±user | ocb cpu | jb | ±cpu | ocb sys | jb | ±sys | |-----------------------------------+-----------+------+--------+----------+------+-------+---------+-----+------+---------+------+------| | byte from scratch | 6428 | 2456 | 2.62 | 7743 | 6662 | 1.16 | 138 | 331 | 2.40 | 1184 | 1477 | 0.80 | | native from scratch | 9841 | 4289 | 2.29 | 9530 | 8834 | 1.08 | 110 | 245 | 2.23 | 1373 | 1712 | 0.80 | | byte after native | 29578 | 1602 | 18.46 | 4514 | 4640 | 0.97 | 170 | 325 | 1.91 | 543 | 576 | 0.94 | | change infer.ml byte | 344 | 282 | 1.22 | 292 | 215 | 1.36 | 96 | 99 | 1.03 | 040 | 066 | 0.61 | | change infer.ml native | 837 | 223 | 3.75 | 789 | 174 | 4.53 | 98 | 99 | 1.01 | 036 | 47 | 0.77 | | change Config.ml byte | 451 | 339 | 1.33 | 382 | 336 | 1.14 | 97 | 122 | 1.26 | 056 | 80 | 0.70 | | change Config.ml native | 4024 | 1760 | 2.29 | 4585 | 4225 | 1.09 | 127 | 276 | 2.17 | 559 | 644 | 0.87 | | change cFrontend_config.ml byte | 348 | 643 | 0.54 | 297 | 330 | 0.90 | 96 | 67 | 0.70 | 038 | 102 | 0.37 | | change cFrontend_config.ml native | 1480 | 584 | 2.53 | 1435 | 906 | 1.58 | 106 | 185 | 1.75 | 136 | 178 | 0.76 | #+TBLFM: $4=$2/$3;f2::$7=$5/$6;f2::$10=$9/$8;f2::$13=$11/$12;f2 50 cores | | ocb total | jb | ±total | ocb user | jb | ±user | ocb cpu | jb | ±cpu | ocb sys | jb | ±sys | |---------------------+-----------+------+--------+----------+------+-------+---------+----+------+---------+------+------| | byte from scratch | 9114 | 2061 | 4.42 | 9334 | 5133 | 1.82 | | | 0/0 | 2566 | 1726 | 1.49 | | native from scratch | 13481 | 3967 | 3.40 | 12291 | 7608 | 1.62 | | | 0/0 | 3003 | 2100 | 1.43 | | byte after native | 3467 | 1476 | 2.35 | 5067 | 3912 | 1.30 | | | 0/0 | 971 | 801 | 1.21 | #+TBLFM: $4=$2/$3;f2::$7=$5/$6;f2::$10=$9/$8;f2::$13=$11/$12;f2 ``` Menu: 1. Write a jbuild file, autogenerated from jbuild.in because we need to fill in some information at build-time (really, at configure time, but TODO), such as whether or not clang is enabled. 2. Nuke lots of stuff from infer/src/Makefile that is now in the jbuild file 3. The jbuild file lives in infer/src/ so it can see all the sources. If we put it somewhere else, eg, infer/, then `jbuilder` scans too many files (all irrelevant) and takes 2.5s to start instead of .8s. Adding irrelevant directories to jbuild-ignore does not help. 4. jbuilder does not support subdirectories, so resort to listing all the source files in the generated jbuild (only source directories need to be manually listed in jbuild.in though). Still, the generated .merlin is wrong and makes merlin find source files in _build, so manually tune it to get good merlin support. We also lose some of merlin for unit tests as it cannot see their build artefacts anymore. 5. checkCopyright gets its own jbuild because it's standalone. Also, remove some deprecation warnings in checkCopyright due to the new version of Core from a while ago. 6. Drop less-used Makefile features (they had regressed anyway) such as building individual modules. Also, building mod_dep.pdf now takes all the source files available so they better build (before, it would only take the source files from the config, eg with or without clang) (that's pretty minor). 7. The toplevel is now built as a custom toplevel because that was easier. It should soon be even easier: https://github.com/janestreet/jbuilder/issues/210 8. Move BUILTINS.mli to BUILTINS.ml because jbuilder is not happy about interface files without implementations. In particular, I did not try to migrate too much of the Makefile logic to jbuilder, more can be done in the future. Reviewed By: jberdine Differential Revision: D5573661 fbshipit-source-id: 4ca6d8f
7 years ago
[RFC][build] Use dune environments and profiles instead of contexts Summary: With profiles and `(env ...)` stanza it's possible to consolidate various ocamlc/ocamlopt/etc setups in a single place. Where previously we needed to append `dune.common` to every dune file and specify `flags` and `ocamlopt_flags` now the flags are specified in `env` and applied accross the board. This allows to 1. simplify build definitions, 2. avoid the need to generate dune files, 3. use plain sexps instead of OCaml and JBuilder plugin in build files. (I'll try to address 2 and 3 in the followup patches). Existing `make` targets should continue working as before. Also, we can use dune CLI like so: ``` infer/src$ dune printenv --profile opt # <- very useful for introspection infer/src$ dune build check infer/src$ dune build check --profile test infer/src$ dune build infer.exe --profile dev infer/src$ dune build infer.exe --profile opt ``` Also, with just 1 context something like `dune runtest` will run unit tests only once instead of N times, where N is the number of contexts. Now, there's one difference compared to the previous setup with contexts: - Previously, each context had its own build folder, and building infer in opt context didn't invalidate any of the build artifacts in default context. Therefore, alternating between `make` and `make opt` had low overhead at the expense of having N copies of all build artifacts (1 for every context). - Now, there's just 1 build folder and switching between profiles does invalidate some artifacts (but not all) and rebuild takes a bit more time. So, if you're alternating like crazy between profiles your experience may get worse (but not necessarily, more on that below). If you want to trigger an opt build occasionally, you shouldn't notice much difference. For those who are concerned about slower build times when alternating between different build profiles, there's a solution: [dune cache](https://dune.readthedocs.io/en/stable/caching.html). You can enable it by creating a file `~/.config/dune/config` with the following contents: ``` (lang dune 2.0) (cache enabled) ``` With cache enabled switching between different build profiles (but also branches and commits) has ~0 overhead. Dune cache works fine on Linux, but unfortunately there are [certain problems with MacOS](https://github.com/ocaml/dune/issues/3233) (hopefully, those will be fixed soon and then there will be no downsides to using profiles compared to contexts for our case). Reviewed By: jvillard Differential Revision: D20247864 fbshipit-source-id: 5f8afa0db
5 years ago
let main_lib_stanza =
Format.sprintf
{|
[make] s/ocamlbuild/jbuilder/g Summary: Use jbuilder to build infer instead of ocamlbuild. This is mainly to get faster builds: ``` times in 10ms, ±differences measured in speedups, 4 cores | | ocb total | jb | ±total | ocb user | jb | ±user | ocb cpu | jb | ±cpu | ocb sys | jb | ±sys | |-----------------------------------+-----------+------+--------+----------+------+-------+---------+-----+------+---------+------+------| | byte from scratch | 6428 | 2456 | 2.62 | 7743 | 6662 | 1.16 | 138 | 331 | 2.40 | 1184 | 1477 | 0.80 | | native from scratch | 9841 | 4289 | 2.29 | 9530 | 8834 | 1.08 | 110 | 245 | 2.23 | 1373 | 1712 | 0.80 | | byte after native | 29578 | 1602 | 18.46 | 4514 | 4640 | 0.97 | 170 | 325 | 1.91 | 543 | 576 | 0.94 | | change infer.ml byte | 344 | 282 | 1.22 | 292 | 215 | 1.36 | 96 | 99 | 1.03 | 040 | 066 | 0.61 | | change infer.ml native | 837 | 223 | 3.75 | 789 | 174 | 4.53 | 98 | 99 | 1.01 | 036 | 47 | 0.77 | | change Config.ml byte | 451 | 339 | 1.33 | 382 | 336 | 1.14 | 97 | 122 | 1.26 | 056 | 80 | 0.70 | | change Config.ml native | 4024 | 1760 | 2.29 | 4585 | 4225 | 1.09 | 127 | 276 | 2.17 | 559 | 644 | 0.87 | | change cFrontend_config.ml byte | 348 | 643 | 0.54 | 297 | 330 | 0.90 | 96 | 67 | 0.70 | 038 | 102 | 0.37 | | change cFrontend_config.ml native | 1480 | 584 | 2.53 | 1435 | 906 | 1.58 | 106 | 185 | 1.75 | 136 | 178 | 0.76 | #+TBLFM: $4=$2/$3;f2::$7=$5/$6;f2::$10=$9/$8;f2::$13=$11/$12;f2 50 cores | | ocb total | jb | ±total | ocb user | jb | ±user | ocb cpu | jb | ±cpu | ocb sys | jb | ±sys | |---------------------+-----------+------+--------+----------+------+-------+---------+----+------+---------+------+------| | byte from scratch | 9114 | 2061 | 4.42 | 9334 | 5133 | 1.82 | | | 0/0 | 2566 | 1726 | 1.49 | | native from scratch | 13481 | 3967 | 3.40 | 12291 | 7608 | 1.62 | | | 0/0 | 3003 | 2100 | 1.43 | | byte after native | 3467 | 1476 | 2.35 | 5067 | 3912 | 1.30 | | | 0/0 | 971 | 801 | 1.21 | #+TBLFM: $4=$2/$3;f2::$7=$5/$6;f2::$10=$9/$8;f2::$13=$11/$12;f2 ``` Menu: 1. Write a jbuild file, autogenerated from jbuild.in because we need to fill in some information at build-time (really, at configure time, but TODO), such as whether or not clang is enabled. 2. Nuke lots of stuff from infer/src/Makefile that is now in the jbuild file 3. The jbuild file lives in infer/src/ so it can see all the sources. If we put it somewhere else, eg, infer/, then `jbuilder` scans too many files (all irrelevant) and takes 2.5s to start instead of .8s. Adding irrelevant directories to jbuild-ignore does not help. 4. jbuilder does not support subdirectories, so resort to listing all the source files in the generated jbuild (only source directories need to be manually listed in jbuild.in though). Still, the generated .merlin is wrong and makes merlin find source files in _build, so manually tune it to get good merlin support. We also lose some of merlin for unit tests as it cannot see their build artefacts anymore. 5. checkCopyright gets its own jbuild because it's standalone. Also, remove some deprecation warnings in checkCopyright due to the new version of Core from a while ago. 6. Drop less-used Makefile features (they had regressed anyway) such as building individual modules. Also, building mod_dep.pdf now takes all the source files available so they better build (before, it would only take the source files from the config, eg with or without clang) (that's pretty minor). 7. The toplevel is now built as a custom toplevel because that was easier. It should soon be even easier: https://github.com/janestreet/jbuilder/issues/210 8. Move BUILTINS.mli to BUILTINS.ml because jbuilder is not happy about interface files without implementations. In particular, I did not try to migrate too much of the Makefile logic to jbuilder, more can be done in the future. Reviewed By: jberdine Differential Revision: D5573661 fbshipit-source-id: 4ca6d8f
7 years ago
(library
(name InferModules)
(public_name infer)
[RFC][build] Use dune environments and profiles instead of contexts Summary: With profiles and `(env ...)` stanza it's possible to consolidate various ocamlc/ocamlopt/etc setups in a single place. Where previously we needed to append `dune.common` to every dune file and specify `flags` and `ocamlopt_flags` now the flags are specified in `env` and applied accross the board. This allows to 1. simplify build definitions, 2. avoid the need to generate dune files, 3. use plain sexps instead of OCaml and JBuilder plugin in build files. (I'll try to address 2 and 3 in the followup patches). Existing `make` targets should continue working as before. Also, we can use dune CLI like so: ``` infer/src$ dune printenv --profile opt # <- very useful for introspection infer/src$ dune build check infer/src$ dune build check --profile test infer/src$ dune build infer.exe --profile dev infer/src$ dune build infer.exe --profile opt ``` Also, with just 1 context something like `dune runtest` will run unit tests only once instead of N times, where N is the number of contexts. Now, there's one difference compared to the previous setup with contexts: - Previously, each context had its own build folder, and building infer in opt context didn't invalidate any of the build artifacts in default context. Therefore, alternating between `make` and `make opt` had low overhead at the expense of having N copies of all build artifacts (1 for every context). - Now, there's just 1 build folder and switching between profiles does invalidate some artifacts (but not all) and rebuild takes a bit more time. So, if you're alternating like crazy between profiles your experience may get worse (but not necessarily, more on that below). If you want to trigger an opt build occasionally, you shouldn't notice much difference. For those who are concerned about slower build times when alternating between different build profiles, there's a solution: [dune cache](https://dune.readthedocs.io/en/stable/caching.html). You can enable it by creating a file `~/.config/dune/config` with the following contents: ``` (lang dune 2.0) (cache enabled) ``` With cache enabled switching between different build profiles (but also branches and commits) has ~0 overhead. Dune cache works fine on Linux, but unfortunately there are [certain problems with MacOS](https://github.com/ocaml/dune/issues/3233) (hopefully, those will be fixed soon and then there will be no downsides to using profiles compared to contexts for our case). Reviewed By: jvillard Differential Revision: D20247864 fbshipit-source-id: 5f8afa0db
5 years ago
(flags (:standard %s))
(libraries %s)
(modules :standard \ %s infertop)
(preprocess (pps ppx_compare ppx_fields_conv ppx_hash ppx_sexp_conv ppx_variants_conv -no-check))
)
[RFC][build] Use dune environments and profiles instead of contexts Summary: With profiles and `(env ...)` stanza it's possible to consolidate various ocamlc/ocamlopt/etc setups in a single place. Where previously we needed to append `dune.common` to every dune file and specify `flags` and `ocamlopt_flags` now the flags are specified in `env` and applied accross the board. This allows to 1. simplify build definitions, 2. avoid the need to generate dune files, 3. use plain sexps instead of OCaml and JBuilder plugin in build files. (I'll try to address 2 and 3 in the followup patches). Existing `make` targets should continue working as before. Also, we can use dune CLI like so: ``` infer/src$ dune printenv --profile opt # <- very useful for introspection infer/src$ dune build check infer/src$ dune build check --profile test infer/src$ dune build infer.exe --profile dev infer/src$ dune build infer.exe --profile opt ``` Also, with just 1 context something like `dune runtest` will run unit tests only once instead of N times, where N is the number of contexts. Now, there's one difference compared to the previous setup with contexts: - Previously, each context had its own build folder, and building infer in opt context didn't invalidate any of the build artifacts in default context. Therefore, alternating between `make` and `make opt` had low overhead at the expense of having N copies of all build artifacts (1 for every context). - Now, there's just 1 build folder and switching between profiles does invalidate some artifacts (but not all) and rebuild takes a bit more time. So, if you're alternating like crazy between profiles your experience may get worse (but not necessarily, more on that below). If you want to trigger an opt build occasionally, you shouldn't notice much difference. For those who are concerned about slower build times when alternating between different build profiles, there's a solution: [dune cache](https://dune.readthedocs.io/en/stable/caching.html). You can enable it by creating a file `~/.config/dune/config` with the following contents: ``` (lang dune 2.0) (cache enabled) ``` With cache enabled switching between different build profiles (but also branches and commits) has ~0 overhead. Dune cache works fine on Linux, but unfortunately there are [certain problems with MacOS](https://github.com/ocaml/dune/issues/3233) (hopefully, those will be fixed soon and then there will be no downsides to using profiles compared to contexts for our case). Reviewed By: jvillard Differential Revision: D20247864 fbshipit-source-id: 5f8afa0db
5 years ago
|}
infer_cflags
(String.concat " "
( common_libraries
@ [ "InferIR"
; "InferCStubs"
; "absint"
; "biabduction"
; "nullsafe"
; "bo"
; "pulselib"
; "checkers"
; "costlib"
; "quandary"
; "TOPLlib"
; "concurrency"
; "labs"
; "backend" ] ))
[RFC][build] Use dune environments and profiles instead of contexts Summary: With profiles and `(env ...)` stanza it's possible to consolidate various ocamlc/ocamlopt/etc setups in a single place. Where previously we needed to append `dune.common` to every dune file and specify `flags` and `ocamlopt_flags` now the flags are specified in `env` and applied accross the board. This allows to 1. simplify build definitions, 2. avoid the need to generate dune files, 3. use plain sexps instead of OCaml and JBuilder plugin in build files. (I'll try to address 2 and 3 in the followup patches). Existing `make` targets should continue working as before. Also, we can use dune CLI like so: ``` infer/src$ dune printenv --profile opt # <- very useful for introspection infer/src$ dune build check infer/src$ dune build check --profile test infer/src$ dune build infer.exe --profile dev infer/src$ dune build infer.exe --profile opt ``` Also, with just 1 context something like `dune runtest` will run unit tests only once instead of N times, where N is the number of contexts. Now, there's one difference compared to the previous setup with contexts: - Previously, each context had its own build folder, and building infer in opt context didn't invalidate any of the build artifacts in default context. Therefore, alternating between `make` and `make opt` had low overhead at the expense of having N copies of all build artifacts (1 for every context). - Now, there's just 1 build folder and switching between profiles does invalidate some artifacts (but not all) and rebuild takes a bit more time. So, if you're alternating like crazy between profiles your experience may get worse (but not necessarily, more on that below). If you want to trigger an opt build occasionally, you shouldn't notice much difference. For those who are concerned about slower build times when alternating between different build profiles, there's a solution: [dune cache](https://dune.readthedocs.io/en/stable/caching.html). You can enable it by creating a file `~/.config/dune/config` with the following contents: ``` (lang dune 2.0) (cache enabled) ``` With cache enabled switching between different build profiles (but also branches and commits) has ~0 overhead. Dune cache works fine on Linux, but unfortunately there are [certain problems with MacOS](https://github.com/ocaml/dune/issues/3233) (hopefully, those will be fixed soon and then there will be no downsides to using profiles compared to contexts for our case). Reviewed By: jvillard Differential Revision: D20247864 fbshipit-source-id: 5f8afa0db
5 years ago
(String.concat " " infer_binaries)
[RFC][build] Use dune environments and profiles instead of contexts Summary: With profiles and `(env ...)` stanza it's possible to consolidate various ocamlc/ocamlopt/etc setups in a single place. Where previously we needed to append `dune.common` to every dune file and specify `flags` and `ocamlopt_flags` now the flags are specified in `env` and applied accross the board. This allows to 1. simplify build definitions, 2. avoid the need to generate dune files, 3. use plain sexps instead of OCaml and JBuilder plugin in build files. (I'll try to address 2 and 3 in the followup patches). Existing `make` targets should continue working as before. Also, we can use dune CLI like so: ``` infer/src$ dune printenv --profile opt # <- very useful for introspection infer/src$ dune build check infer/src$ dune build check --profile test infer/src$ dune build infer.exe --profile dev infer/src$ dune build infer.exe --profile opt ``` Also, with just 1 context something like `dune runtest` will run unit tests only once instead of N times, where N is the number of contexts. Now, there's one difference compared to the previous setup with contexts: - Previously, each context had its own build folder, and building infer in opt context didn't invalidate any of the build artifacts in default context. Therefore, alternating between `make` and `make opt` had low overhead at the expense of having N copies of all build artifacts (1 for every context). - Now, there's just 1 build folder and switching between profiles does invalidate some artifacts (but not all) and rebuild takes a bit more time. So, if you're alternating like crazy between profiles your experience may get worse (but not necessarily, more on that below). If you want to trigger an opt build occasionally, you shouldn't notice much difference. For those who are concerned about slower build times when alternating between different build profiles, there's a solution: [dune cache](https://dune.readthedocs.io/en/stable/caching.html). You can enable it by creating a file `~/.config/dune/config` with the following contents: ``` (lang dune 2.0) (cache enabled) ``` With cache enabled switching between different build profiles (but also branches and commits) has ~0 overhead. Dune cache works fine on Linux, but unfortunately there are [certain problems with MacOS](https://github.com/ocaml/dune/issues/3233) (hopefully, those will be fixed soon and then there will be no downsides to using profiles compared to contexts for our case). Reviewed By: jvillard Differential Revision: D20247864 fbshipit-source-id: 5f8afa0db
5 years ago
let infer_exe_stanza =
Format.sprintf
{|
[make] s/ocamlbuild/jbuilder/g Summary: Use jbuilder to build infer instead of ocamlbuild. This is mainly to get faster builds: ``` times in 10ms, ±differences measured in speedups, 4 cores | | ocb total | jb | ±total | ocb user | jb | ±user | ocb cpu | jb | ±cpu | ocb sys | jb | ±sys | |-----------------------------------+-----------+------+--------+----------+------+-------+---------+-----+------+---------+------+------| | byte from scratch | 6428 | 2456 | 2.62 | 7743 | 6662 | 1.16 | 138 | 331 | 2.40 | 1184 | 1477 | 0.80 | | native from scratch | 9841 | 4289 | 2.29 | 9530 | 8834 | 1.08 | 110 | 245 | 2.23 | 1373 | 1712 | 0.80 | | byte after native | 29578 | 1602 | 18.46 | 4514 | 4640 | 0.97 | 170 | 325 | 1.91 | 543 | 576 | 0.94 | | change infer.ml byte | 344 | 282 | 1.22 | 292 | 215 | 1.36 | 96 | 99 | 1.03 | 040 | 066 | 0.61 | | change infer.ml native | 837 | 223 | 3.75 | 789 | 174 | 4.53 | 98 | 99 | 1.01 | 036 | 47 | 0.77 | | change Config.ml byte | 451 | 339 | 1.33 | 382 | 336 | 1.14 | 97 | 122 | 1.26 | 056 | 80 | 0.70 | | change Config.ml native | 4024 | 1760 | 2.29 | 4585 | 4225 | 1.09 | 127 | 276 | 2.17 | 559 | 644 | 0.87 | | change cFrontend_config.ml byte | 348 | 643 | 0.54 | 297 | 330 | 0.90 | 96 | 67 | 0.70 | 038 | 102 | 0.37 | | change cFrontend_config.ml native | 1480 | 584 | 2.53 | 1435 | 906 | 1.58 | 106 | 185 | 1.75 | 136 | 178 | 0.76 | #+TBLFM: $4=$2/$3;f2::$7=$5/$6;f2::$10=$9/$8;f2::$13=$11/$12;f2 50 cores | | ocb total | jb | ±total | ocb user | jb | ±user | ocb cpu | jb | ±cpu | ocb sys | jb | ±sys | |---------------------+-----------+------+--------+----------+------+-------+---------+----+------+---------+------+------| | byte from scratch | 9114 | 2061 | 4.42 | 9334 | 5133 | 1.82 | | | 0/0 | 2566 | 1726 | 1.49 | | native from scratch | 13481 | 3967 | 3.40 | 12291 | 7608 | 1.62 | | | 0/0 | 3003 | 2100 | 1.43 | | byte after native | 3467 | 1476 | 2.35 | 5067 | 3912 | 1.30 | | | 0/0 | 971 | 801 | 1.21 | #+TBLFM: $4=$2/$3;f2::$7=$5/$6;f2::$10=$9/$8;f2::$13=$11/$12;f2 ``` Menu: 1. Write a jbuild file, autogenerated from jbuild.in because we need to fill in some information at build-time (really, at configure time, but TODO), such as whether or not clang is enabled. 2. Nuke lots of stuff from infer/src/Makefile that is now in the jbuild file 3. The jbuild file lives in infer/src/ so it can see all the sources. If we put it somewhere else, eg, infer/, then `jbuilder` scans too many files (all irrelevant) and takes 2.5s to start instead of .8s. Adding irrelevant directories to jbuild-ignore does not help. 4. jbuilder does not support subdirectories, so resort to listing all the source files in the generated jbuild (only source directories need to be manually listed in jbuild.in though). Still, the generated .merlin is wrong and makes merlin find source files in _build, so manually tune it to get good merlin support. We also lose some of merlin for unit tests as it cannot see their build artefacts anymore. 5. checkCopyright gets its own jbuild because it's standalone. Also, remove some deprecation warnings in checkCopyright due to the new version of Core from a while ago. 6. Drop less-used Makefile features (they had regressed anyway) such as building individual modules. Also, building mod_dep.pdf now takes all the source files available so they better build (before, it would only take the source files from the config, eg with or without clang) (that's pretty minor). 7. The toplevel is now built as a custom toplevel because that was easier. It should soon be even easier: https://github.com/janestreet/jbuilder/issues/210 8. Move BUILTINS.mli to BUILTINS.ml because jbuilder is not happy about interface files without implementations. In particular, I did not try to migrate too much of the Makefile logic to jbuilder, more can be done in the future. Reviewed By: jberdine Differential Revision: D5573661 fbshipit-source-id: 4ca6d8f
7 years ago
(executables
(names %s)
(modes byte_complete exe)
[RFC][build] Use dune environments and profiles instead of contexts Summary: With profiles and `(env ...)` stanza it's possible to consolidate various ocamlc/ocamlopt/etc setups in a single place. Where previously we needed to append `dune.common` to every dune file and specify `flags` and `ocamlopt_flags` now the flags are specified in `env` and applied accross the board. This allows to 1. simplify build definitions, 2. avoid the need to generate dune files, 3. use plain sexps instead of OCaml and JBuilder plugin in build files. (I'll try to address 2 and 3 in the followup patches). Existing `make` targets should continue working as before. Also, we can use dune CLI like so: ``` infer/src$ dune printenv --profile opt # <- very useful for introspection infer/src$ dune build check infer/src$ dune build check --profile test infer/src$ dune build infer.exe --profile dev infer/src$ dune build infer.exe --profile opt ``` Also, with just 1 context something like `dune runtest` will run unit tests only once instead of N times, where N is the number of contexts. Now, there's one difference compared to the previous setup with contexts: - Previously, each context had its own build folder, and building infer in opt context didn't invalidate any of the build artifacts in default context. Therefore, alternating between `make` and `make opt` had low overhead at the expense of having N copies of all build artifacts (1 for every context). - Now, there's just 1 build folder and switching between profiles does invalidate some artifacts (but not all) and rebuild takes a bit more time. So, if you're alternating like crazy between profiles your experience may get worse (but not necessarily, more on that below). If you want to trigger an opt build occasionally, you shouldn't notice much difference. For those who are concerned about slower build times when alternating between different build profiles, there's a solution: [dune cache](https://dune.readthedocs.io/en/stable/caching.html). You can enable it by creating a file `~/.config/dune/config` with the following contents: ``` (lang dune 2.0) (cache enabled) ``` With cache enabled switching between different build profiles (but also branches and commits) has ~0 overhead. Dune cache works fine on Linux, but unfortunately there are [certain problems with MacOS](https://github.com/ocaml/dune/issues/3233) (hopefully, those will be fixed soon and then there will be no downsides to using profiles compared to contexts for our case). Reviewed By: jvillard Differential Revision: D20247864 fbshipit-source-id: 5f8afa0db
5 years ago
(flags (:standard %s -open InferModules))
(libraries InferModules)
(modules %s)
(preprocess (pps ppx_compare ppx_fields_conv ppx_hash ppx_sexp_conv ppx_variants_conv -no-check))
(promote (until-clean) (into ../bin))
)
[make] s/ocamlbuild/jbuilder/g Summary: Use jbuilder to build infer instead of ocamlbuild. This is mainly to get faster builds: ``` times in 10ms, ±differences measured in speedups, 4 cores | | ocb total | jb | ±total | ocb user | jb | ±user | ocb cpu | jb | ±cpu | ocb sys | jb | ±sys | |-----------------------------------+-----------+------+--------+----------+------+-------+---------+-----+------+---------+------+------| | byte from scratch | 6428 | 2456 | 2.62 | 7743 | 6662 | 1.16 | 138 | 331 | 2.40 | 1184 | 1477 | 0.80 | | native from scratch | 9841 | 4289 | 2.29 | 9530 | 8834 | 1.08 | 110 | 245 | 2.23 | 1373 | 1712 | 0.80 | | byte after native | 29578 | 1602 | 18.46 | 4514 | 4640 | 0.97 | 170 | 325 | 1.91 | 543 | 576 | 0.94 | | change infer.ml byte | 344 | 282 | 1.22 | 292 | 215 | 1.36 | 96 | 99 | 1.03 | 040 | 066 | 0.61 | | change infer.ml native | 837 | 223 | 3.75 | 789 | 174 | 4.53 | 98 | 99 | 1.01 | 036 | 47 | 0.77 | | change Config.ml byte | 451 | 339 | 1.33 | 382 | 336 | 1.14 | 97 | 122 | 1.26 | 056 | 80 | 0.70 | | change Config.ml native | 4024 | 1760 | 2.29 | 4585 | 4225 | 1.09 | 127 | 276 | 2.17 | 559 | 644 | 0.87 | | change cFrontend_config.ml byte | 348 | 643 | 0.54 | 297 | 330 | 0.90 | 96 | 67 | 0.70 | 038 | 102 | 0.37 | | change cFrontend_config.ml native | 1480 | 584 | 2.53 | 1435 | 906 | 1.58 | 106 | 185 | 1.75 | 136 | 178 | 0.76 | #+TBLFM: $4=$2/$3;f2::$7=$5/$6;f2::$10=$9/$8;f2::$13=$11/$12;f2 50 cores | | ocb total | jb | ±total | ocb user | jb | ±user | ocb cpu | jb | ±cpu | ocb sys | jb | ±sys | |---------------------+-----------+------+--------+----------+------+-------+---------+----+------+---------+------+------| | byte from scratch | 9114 | 2061 | 4.42 | 9334 | 5133 | 1.82 | | | 0/0 | 2566 | 1726 | 1.49 | | native from scratch | 13481 | 3967 | 3.40 | 12291 | 7608 | 1.62 | | | 0/0 | 3003 | 2100 | 1.43 | | byte after native | 3467 | 1476 | 2.35 | 5067 | 3912 | 1.30 | | | 0/0 | 971 | 801 | 1.21 | #+TBLFM: $4=$2/$3;f2::$7=$5/$6;f2::$10=$9/$8;f2::$13=$11/$12;f2 ``` Menu: 1. Write a jbuild file, autogenerated from jbuild.in because we need to fill in some information at build-time (really, at configure time, but TODO), such as whether or not clang is enabled. 2. Nuke lots of stuff from infer/src/Makefile that is now in the jbuild file 3. The jbuild file lives in infer/src/ so it can see all the sources. If we put it somewhere else, eg, infer/, then `jbuilder` scans too many files (all irrelevant) and takes 2.5s to start instead of .8s. Adding irrelevant directories to jbuild-ignore does not help. 4. jbuilder does not support subdirectories, so resort to listing all the source files in the generated jbuild (only source directories need to be manually listed in jbuild.in though). Still, the generated .merlin is wrong and makes merlin find source files in _build, so manually tune it to get good merlin support. We also lose some of merlin for unit tests as it cannot see their build artefacts anymore. 5. checkCopyright gets its own jbuild because it's standalone. Also, remove some deprecation warnings in checkCopyright due to the new version of Core from a while ago. 6. Drop less-used Makefile features (they had regressed anyway) such as building individual modules. Also, building mod_dep.pdf now takes all the source files available so they better build (before, it would only take the source files from the config, eg with or without clang) (that's pretty minor). 7. The toplevel is now built as a custom toplevel because that was easier. It should soon be even easier: https://github.com/janestreet/jbuilder/issues/210 8. Move BUILTINS.mli to BUILTINS.ml because jbuilder is not happy about interface files without implementations. In particular, I did not try to migrate too much of the Makefile logic to jbuilder, more can be done in the future. Reviewed By: jberdine Differential Revision: D5573661 fbshipit-source-id: 4ca6d8f
7 years ago
|}
[RFC][build] Use dune environments and profiles instead of contexts Summary: With profiles and `(env ...)` stanza it's possible to consolidate various ocamlc/ocamlopt/etc setups in a single place. Where previously we needed to append `dune.common` to every dune file and specify `flags` and `ocamlopt_flags` now the flags are specified in `env` and applied accross the board. This allows to 1. simplify build definitions, 2. avoid the need to generate dune files, 3. use plain sexps instead of OCaml and JBuilder plugin in build files. (I'll try to address 2 and 3 in the followup patches). Existing `make` targets should continue working as before. Also, we can use dune CLI like so: ``` infer/src$ dune printenv --profile opt # <- very useful for introspection infer/src$ dune build check infer/src$ dune build check --profile test infer/src$ dune build infer.exe --profile dev infer/src$ dune build infer.exe --profile opt ``` Also, with just 1 context something like `dune runtest` will run unit tests only once instead of N times, where N is the number of contexts. Now, there's one difference compared to the previous setup with contexts: - Previously, each context had its own build folder, and building infer in opt context didn't invalidate any of the build artifacts in default context. Therefore, alternating between `make` and `make opt` had low overhead at the expense of having N copies of all build artifacts (1 for every context). - Now, there's just 1 build folder and switching between profiles does invalidate some artifacts (but not all) and rebuild takes a bit more time. So, if you're alternating like crazy between profiles your experience may get worse (but not necessarily, more on that below). If you want to trigger an opt build occasionally, you shouldn't notice much difference. For those who are concerned about slower build times when alternating between different build profiles, there's a solution: [dune cache](https://dune.readthedocs.io/en/stable/caching.html). You can enable it by creating a file `~/.config/dune/config` with the following contents: ``` (lang dune 2.0) (cache enabled) ``` With cache enabled switching between different build profiles (but also branches and commits) has ~0 overhead. Dune cache works fine on Linux, but unfortunately there are [certain problems with MacOS](https://github.com/ocaml/dune/issues/3233) (hopefully, those will be fixed soon and then there will be no downsides to using profiles compared to contexts for our case). Reviewed By: jvillard Differential Revision: D20247864 fbshipit-source-id: 5f8afa0db
5 years ago
(String.concat " " infer_binaries)
infer_cflags
(String.concat " " infer_binaries)
let infertop_stanza =
Format.sprintf
{|
[make] s/ocamlbuild/jbuilder/g Summary: Use jbuilder to build infer instead of ocamlbuild. This is mainly to get faster builds: ``` times in 10ms, ±differences measured in speedups, 4 cores | | ocb total | jb | ±total | ocb user | jb | ±user | ocb cpu | jb | ±cpu | ocb sys | jb | ±sys | |-----------------------------------+-----------+------+--------+----------+------+-------+---------+-----+------+---------+------+------| | byte from scratch | 6428 | 2456 | 2.62 | 7743 | 6662 | 1.16 | 138 | 331 | 2.40 | 1184 | 1477 | 0.80 | | native from scratch | 9841 | 4289 | 2.29 | 9530 | 8834 | 1.08 | 110 | 245 | 2.23 | 1373 | 1712 | 0.80 | | byte after native | 29578 | 1602 | 18.46 | 4514 | 4640 | 0.97 | 170 | 325 | 1.91 | 543 | 576 | 0.94 | | change infer.ml byte | 344 | 282 | 1.22 | 292 | 215 | 1.36 | 96 | 99 | 1.03 | 040 | 066 | 0.61 | | change infer.ml native | 837 | 223 | 3.75 | 789 | 174 | 4.53 | 98 | 99 | 1.01 | 036 | 47 | 0.77 | | change Config.ml byte | 451 | 339 | 1.33 | 382 | 336 | 1.14 | 97 | 122 | 1.26 | 056 | 80 | 0.70 | | change Config.ml native | 4024 | 1760 | 2.29 | 4585 | 4225 | 1.09 | 127 | 276 | 2.17 | 559 | 644 | 0.87 | | change cFrontend_config.ml byte | 348 | 643 | 0.54 | 297 | 330 | 0.90 | 96 | 67 | 0.70 | 038 | 102 | 0.37 | | change cFrontend_config.ml native | 1480 | 584 | 2.53 | 1435 | 906 | 1.58 | 106 | 185 | 1.75 | 136 | 178 | 0.76 | #+TBLFM: $4=$2/$3;f2::$7=$5/$6;f2::$10=$9/$8;f2::$13=$11/$12;f2 50 cores | | ocb total | jb | ±total | ocb user | jb | ±user | ocb cpu | jb | ±cpu | ocb sys | jb | ±sys | |---------------------+-----------+------+--------+----------+------+-------+---------+----+------+---------+------+------| | byte from scratch | 9114 | 2061 | 4.42 | 9334 | 5133 | 1.82 | | | 0/0 | 2566 | 1726 | 1.49 | | native from scratch | 13481 | 3967 | 3.40 | 12291 | 7608 | 1.62 | | | 0/0 | 3003 | 2100 | 1.43 | | byte after native | 3467 | 1476 | 2.35 | 5067 | 3912 | 1.30 | | | 0/0 | 971 | 801 | 1.21 | #+TBLFM: $4=$2/$3;f2::$7=$5/$6;f2::$10=$9/$8;f2::$13=$11/$12;f2 ``` Menu: 1. Write a jbuild file, autogenerated from jbuild.in because we need to fill in some information at build-time (really, at configure time, but TODO), such as whether or not clang is enabled. 2. Nuke lots of stuff from infer/src/Makefile that is now in the jbuild file 3. The jbuild file lives in infer/src/ so it can see all the sources. If we put it somewhere else, eg, infer/, then `jbuilder` scans too many files (all irrelevant) and takes 2.5s to start instead of .8s. Adding irrelevant directories to jbuild-ignore does not help. 4. jbuilder does not support subdirectories, so resort to listing all the source files in the generated jbuild (only source directories need to be manually listed in jbuild.in though). Still, the generated .merlin is wrong and makes merlin find source files in _build, so manually tune it to get good merlin support. We also lose some of merlin for unit tests as it cannot see their build artefacts anymore. 5. checkCopyright gets its own jbuild because it's standalone. Also, remove some deprecation warnings in checkCopyright due to the new version of Core from a while ago. 6. Drop less-used Makefile features (they had regressed anyway) such as building individual modules. Also, building mod_dep.pdf now takes all the source files available so they better build (before, it would only take the source files from the config, eg with or without clang) (that's pretty minor). 7. The toplevel is now built as a custom toplevel because that was easier. It should soon be even easier: https://github.com/janestreet/jbuilder/issues/210 8. Move BUILTINS.mli to BUILTINS.ml because jbuilder is not happy about interface files without implementations. In particular, I did not try to migrate too much of the Makefile logic to jbuilder, more can be done in the future. Reviewed By: jberdine Differential Revision: D5573661 fbshipit-source-id: 4ca6d8f
7 years ago
(executable
(name infertop)
(modes byte_complete)
[RFC][build] Use dune environments and profiles instead of contexts Summary: With profiles and `(env ...)` stanza it's possible to consolidate various ocamlc/ocamlopt/etc setups in a single place. Where previously we needed to append `dune.common` to every dune file and specify `flags` and `ocamlopt_flags` now the flags are specified in `env` and applied accross the board. This allows to 1. simplify build definitions, 2. avoid the need to generate dune files, 3. use plain sexps instead of OCaml and JBuilder plugin in build files. (I'll try to address 2 and 3 in the followup patches). Existing `make` targets should continue working as before. Also, we can use dune CLI like so: ``` infer/src$ dune printenv --profile opt # <- very useful for introspection infer/src$ dune build check infer/src$ dune build check --profile test infer/src$ dune build infer.exe --profile dev infer/src$ dune build infer.exe --profile opt ``` Also, with just 1 context something like `dune runtest` will run unit tests only once instead of N times, where N is the number of contexts. Now, there's one difference compared to the previous setup with contexts: - Previously, each context had its own build folder, and building infer in opt context didn't invalidate any of the build artifacts in default context. Therefore, alternating between `make` and `make opt` had low overhead at the expense of having N copies of all build artifacts (1 for every context). - Now, there's just 1 build folder and switching between profiles does invalidate some artifacts (but not all) and rebuild takes a bit more time. So, if you're alternating like crazy between profiles your experience may get worse (but not necessarily, more on that below). If you want to trigger an opt build occasionally, you shouldn't notice much difference. For those who are concerned about slower build times when alternating between different build profiles, there's a solution: [dune cache](https://dune.readthedocs.io/en/stable/caching.html). You can enable it by creating a file `~/.config/dune/config` with the following contents: ``` (lang dune 2.0) (cache enabled) ``` With cache enabled switching between different build profiles (but also branches and commits) has ~0 overhead. Dune cache works fine on Linux, but unfortunately there are [certain problems with MacOS](https://github.com/ocaml/dune/issues/3233) (hopefully, those will be fixed soon and then there will be no downsides to using profiles compared to contexts for our case). Reviewed By: jvillard Differential Revision: D20247864 fbshipit-source-id: 5f8afa0db
5 years ago
(flags (:standard %s))
(libraries utop InferModules)
(link_flags (-linkall -warn-error -31))
(modules Infertop)
(preprocess (pps ppx_compare ppx_fields_conv ppx_hash ppx_sexp_conv ppx_variants_conv -no-check))
(promote (until-clean) (into ../bin))
)
[make] s/ocamlbuild/jbuilder/g Summary: Use jbuilder to build infer instead of ocamlbuild. This is mainly to get faster builds: ``` times in 10ms, ±differences measured in speedups, 4 cores | | ocb total | jb | ±total | ocb user | jb | ±user | ocb cpu | jb | ±cpu | ocb sys | jb | ±sys | |-----------------------------------+-----------+------+--------+----------+------+-------+---------+-----+------+---------+------+------| | byte from scratch | 6428 | 2456 | 2.62 | 7743 | 6662 | 1.16 | 138 | 331 | 2.40 | 1184 | 1477 | 0.80 | | native from scratch | 9841 | 4289 | 2.29 | 9530 | 8834 | 1.08 | 110 | 245 | 2.23 | 1373 | 1712 | 0.80 | | byte after native | 29578 | 1602 | 18.46 | 4514 | 4640 | 0.97 | 170 | 325 | 1.91 | 543 | 576 | 0.94 | | change infer.ml byte | 344 | 282 | 1.22 | 292 | 215 | 1.36 | 96 | 99 | 1.03 | 040 | 066 | 0.61 | | change infer.ml native | 837 | 223 | 3.75 | 789 | 174 | 4.53 | 98 | 99 | 1.01 | 036 | 47 | 0.77 | | change Config.ml byte | 451 | 339 | 1.33 | 382 | 336 | 1.14 | 97 | 122 | 1.26 | 056 | 80 | 0.70 | | change Config.ml native | 4024 | 1760 | 2.29 | 4585 | 4225 | 1.09 | 127 | 276 | 2.17 | 559 | 644 | 0.87 | | change cFrontend_config.ml byte | 348 | 643 | 0.54 | 297 | 330 | 0.90 | 96 | 67 | 0.70 | 038 | 102 | 0.37 | | change cFrontend_config.ml native | 1480 | 584 | 2.53 | 1435 | 906 | 1.58 | 106 | 185 | 1.75 | 136 | 178 | 0.76 | #+TBLFM: $4=$2/$3;f2::$7=$5/$6;f2::$10=$9/$8;f2::$13=$11/$12;f2 50 cores | | ocb total | jb | ±total | ocb user | jb | ±user | ocb cpu | jb | ±cpu | ocb sys | jb | ±sys | |---------------------+-----------+------+--------+----------+------+-------+---------+----+------+---------+------+------| | byte from scratch | 9114 | 2061 | 4.42 | 9334 | 5133 | 1.82 | | | 0/0 | 2566 | 1726 | 1.49 | | native from scratch | 13481 | 3967 | 3.40 | 12291 | 7608 | 1.62 | | | 0/0 | 3003 | 2100 | 1.43 | | byte after native | 3467 | 1476 | 2.35 | 5067 | 3912 | 1.30 | | | 0/0 | 971 | 801 | 1.21 | #+TBLFM: $4=$2/$3;f2::$7=$5/$6;f2::$10=$9/$8;f2::$13=$11/$12;f2 ``` Menu: 1. Write a jbuild file, autogenerated from jbuild.in because we need to fill in some information at build-time (really, at configure time, but TODO), such as whether or not clang is enabled. 2. Nuke lots of stuff from infer/src/Makefile that is now in the jbuild file 3. The jbuild file lives in infer/src/ so it can see all the sources. If we put it somewhere else, eg, infer/, then `jbuilder` scans too many files (all irrelevant) and takes 2.5s to start instead of .8s. Adding irrelevant directories to jbuild-ignore does not help. 4. jbuilder does not support subdirectories, so resort to listing all the source files in the generated jbuild (only source directories need to be manually listed in jbuild.in though). Still, the generated .merlin is wrong and makes merlin find source files in _build, so manually tune it to get good merlin support. We also lose some of merlin for unit tests as it cannot see their build artefacts anymore. 5. checkCopyright gets its own jbuild because it's standalone. Also, remove some deprecation warnings in checkCopyright due to the new version of Core from a while ago. 6. Drop less-used Makefile features (they had regressed anyway) such as building individual modules. Also, building mod_dep.pdf now takes all the source files available so they better build (before, it would only take the source files from the config, eg with or without clang) (that's pretty minor). 7. The toplevel is now built as a custom toplevel because that was easier. It should soon be even easier: https://github.com/janestreet/jbuilder/issues/210 8. Move BUILTINS.mli to BUILTINS.ml because jbuilder is not happy about interface files without implementations. In particular, I did not try to migrate too much of the Makefile logic to jbuilder, more can be done in the future. Reviewed By: jberdine Differential Revision: D5573661 fbshipit-source-id: 4ca6d8f
7 years ago
|}
[RFC][build] Use dune environments and profiles instead of contexts Summary: With profiles and `(env ...)` stanza it's possible to consolidate various ocamlc/ocamlopt/etc setups in a single place. Where previously we needed to append `dune.common` to every dune file and specify `flags` and `ocamlopt_flags` now the flags are specified in `env` and applied accross the board. This allows to 1. simplify build definitions, 2. avoid the need to generate dune files, 3. use plain sexps instead of OCaml and JBuilder plugin in build files. (I'll try to address 2 and 3 in the followup patches). Existing `make` targets should continue working as before. Also, we can use dune CLI like so: ``` infer/src$ dune printenv --profile opt # <- very useful for introspection infer/src$ dune build check infer/src$ dune build check --profile test infer/src$ dune build infer.exe --profile dev infer/src$ dune build infer.exe --profile opt ``` Also, with just 1 context something like `dune runtest` will run unit tests only once instead of N times, where N is the number of contexts. Now, there's one difference compared to the previous setup with contexts: - Previously, each context had its own build folder, and building infer in opt context didn't invalidate any of the build artifacts in default context. Therefore, alternating between `make` and `make opt` had low overhead at the expense of having N copies of all build artifacts (1 for every context). - Now, there's just 1 build folder and switching between profiles does invalidate some artifacts (but not all) and rebuild takes a bit more time. So, if you're alternating like crazy between profiles your experience may get worse (but not necessarily, more on that below). If you want to trigger an opt build occasionally, you shouldn't notice much difference. For those who are concerned about slower build times when alternating between different build profiles, there's a solution: [dune cache](https://dune.readthedocs.io/en/stable/caching.html). You can enable it by creating a file `~/.config/dune/config` with the following contents: ``` (lang dune 2.0) (cache enabled) ``` With cache enabled switching between different build profiles (but also branches and commits) has ~0 overhead. Dune cache works fine on Linux, but unfortunately there are [certain problems with MacOS](https://github.com/ocaml/dune/issues/3233) (hopefully, those will be fixed soon and then there will be no downsides to using profiles compared to contexts for our case). Reviewed By: jvillard Differential Revision: D20247864 fbshipit-source-id: 5f8afa0db
5 years ago
infer_cflags
let java_sources_lexer = if java then ["(ocamllex jSourceFileInfo)"] else []
[RFC][build] Use dune environments and profiles instead of contexts Summary: With profiles and `(env ...)` stanza it's possible to consolidate various ocamlc/ocamlopt/etc setups in a single place. Where previously we needed to append `dune.common` to every dune file and specify `flags` and `ocamlopt_flags` now the flags are specified in `env` and applied accross the board. This allows to 1. simplify build definitions, 2. avoid the need to generate dune files, 3. use plain sexps instead of OCaml and JBuilder plugin in build files. (I'll try to address 2 and 3 in the followup patches). Existing `make` targets should continue working as before. Also, we can use dune CLI like so: ``` infer/src$ dune printenv --profile opt # <- very useful for introspection infer/src$ dune build check infer/src$ dune build check --profile test infer/src$ dune build infer.exe --profile dev infer/src$ dune build infer.exe --profile opt ``` Also, with just 1 context something like `dune runtest` will run unit tests only once instead of N times, where N is the number of contexts. Now, there's one difference compared to the previous setup with contexts: - Previously, each context had its own build folder, and building infer in opt context didn't invalidate any of the build artifacts in default context. Therefore, alternating between `make` and `make opt` had low overhead at the expense of having N copies of all build artifacts (1 for every context). - Now, there's just 1 build folder and switching between profiles does invalidate some artifacts (but not all) and rebuild takes a bit more time. So, if you're alternating like crazy between profiles your experience may get worse (but not necessarily, more on that below). If you want to trigger an opt build occasionally, you shouldn't notice much difference. For those who are concerned about slower build times when alternating between different build profiles, there's a solution: [dune cache](https://dune.readthedocs.io/en/stable/caching.html). You can enable it by creating a file `~/.config/dune/config` with the following contents: ``` (lang dune 2.0) (cache enabled) ``` With cache enabled switching between different build profiles (but also branches and commits) has ~0 overhead. Dune cache works fine on Linux, but unfortunately there are [certain problems with MacOS](https://github.com/ocaml/dune/issues/3233) (hopefully, those will be fixed soon and then there will be no downsides to using profiles compared to contexts for our case). Reviewed By: jvillard Differential Revision: D20247864 fbshipit-source-id: 5f8afa0db
5 years ago
let flatten_sources_stanzas =
List.map
(fun source_dir ->
[ Printf.sprintf "(copy_files# %s/*.ml{,i,l})" source_dir
; (* menhir doesn't support '# 1 "<source file>"' directives at the start of the file inserted by
copy# actions *)
Printf.sprintf "(copy_files %s/*.mly)" source_dir ] )
source_dirs
|> List.concat
(** The build stanzas to be passed to dune *)
let stanzas =
(env_stanza :: main_lib_stanza :: infer_exe_stanza :: infertop_stanza :: clang_lexer_stanzas)
@ java_sources_lexer @ flatten_sources_stanzas
[make] s/ocamlbuild/jbuilder/g Summary: Use jbuilder to build infer instead of ocamlbuild. This is mainly to get faster builds: ``` times in 10ms, ±differences measured in speedups, 4 cores | | ocb total | jb | ±total | ocb user | jb | ±user | ocb cpu | jb | ±cpu | ocb sys | jb | ±sys | |-----------------------------------+-----------+------+--------+----------+------+-------+---------+-----+------+---------+------+------| | byte from scratch | 6428 | 2456 | 2.62 | 7743 | 6662 | 1.16 | 138 | 331 | 2.40 | 1184 | 1477 | 0.80 | | native from scratch | 9841 | 4289 | 2.29 | 9530 | 8834 | 1.08 | 110 | 245 | 2.23 | 1373 | 1712 | 0.80 | | byte after native | 29578 | 1602 | 18.46 | 4514 | 4640 | 0.97 | 170 | 325 | 1.91 | 543 | 576 | 0.94 | | change infer.ml byte | 344 | 282 | 1.22 | 292 | 215 | 1.36 | 96 | 99 | 1.03 | 040 | 066 | 0.61 | | change infer.ml native | 837 | 223 | 3.75 | 789 | 174 | 4.53 | 98 | 99 | 1.01 | 036 | 47 | 0.77 | | change Config.ml byte | 451 | 339 | 1.33 | 382 | 336 | 1.14 | 97 | 122 | 1.26 | 056 | 80 | 0.70 | | change Config.ml native | 4024 | 1760 | 2.29 | 4585 | 4225 | 1.09 | 127 | 276 | 2.17 | 559 | 644 | 0.87 | | change cFrontend_config.ml byte | 348 | 643 | 0.54 | 297 | 330 | 0.90 | 96 | 67 | 0.70 | 038 | 102 | 0.37 | | change cFrontend_config.ml native | 1480 | 584 | 2.53 | 1435 | 906 | 1.58 | 106 | 185 | 1.75 | 136 | 178 | 0.76 | #+TBLFM: $4=$2/$3;f2::$7=$5/$6;f2::$10=$9/$8;f2::$13=$11/$12;f2 50 cores | | ocb total | jb | ±total | ocb user | jb | ±user | ocb cpu | jb | ±cpu | ocb sys | jb | ±sys | |---------------------+-----------+------+--------+----------+------+-------+---------+----+------+---------+------+------| | byte from scratch | 9114 | 2061 | 4.42 | 9334 | 5133 | 1.82 | | | 0/0 | 2566 | 1726 | 1.49 | | native from scratch | 13481 | 3967 | 3.40 | 12291 | 7608 | 1.62 | | | 0/0 | 3003 | 2100 | 1.43 | | byte after native | 3467 | 1476 | 2.35 | 5067 | 3912 | 1.30 | | | 0/0 | 971 | 801 | 1.21 | #+TBLFM: $4=$2/$3;f2::$7=$5/$6;f2::$10=$9/$8;f2::$13=$11/$12;f2 ``` Menu: 1. Write a jbuild file, autogenerated from jbuild.in because we need to fill in some information at build-time (really, at configure time, but TODO), such as whether or not clang is enabled. 2. Nuke lots of stuff from infer/src/Makefile that is now in the jbuild file 3. The jbuild file lives in infer/src/ so it can see all the sources. If we put it somewhere else, eg, infer/, then `jbuilder` scans too many files (all irrelevant) and takes 2.5s to start instead of .8s. Adding irrelevant directories to jbuild-ignore does not help. 4. jbuilder does not support subdirectories, so resort to listing all the source files in the generated jbuild (only source directories need to be manually listed in jbuild.in though). Still, the generated .merlin is wrong and makes merlin find source files in _build, so manually tune it to get good merlin support. We also lose some of merlin for unit tests as it cannot see their build artefacts anymore. 5. checkCopyright gets its own jbuild because it's standalone. Also, remove some deprecation warnings in checkCopyright due to the new version of Core from a while ago. 6. Drop less-used Makefile features (they had regressed anyway) such as building individual modules. Also, building mod_dep.pdf now takes all the source files available so they better build (before, it would only take the source files from the config, eg with or without clang) (that's pretty minor). 7. The toplevel is now built as a custom toplevel because that was easier. It should soon be even easier: https://github.com/janestreet/jbuilder/issues/210 8. Move BUILTINS.mli to BUILTINS.ml because jbuilder is not happy about interface files without implementations. In particular, I did not try to migrate too much of the Makefile logic to jbuilder, more can be done in the future. Reviewed By: jberdine Differential Revision: D5573661 fbshipit-source-id: 4ca6d8f
7 years ago
;;
String.concat "\n" stanzas |> Jbuild_plugin.V1.send