Summary:
Before: 3 modes: (where "lenient" build has warnings not crash the
build, while "strict" build errors on warning):
- opt (default): flambda optimisations + lenient build
- dev (recommand for dev): lenient build, no flambda
- test: strict build (used in tests), no flambda
Now:
- dev (default): *strict* build, no flambda
- opt: lenient build, flambda
- dev-noerror: lenient build, no flambda (use when you want to test
infer but there are build warnings)
The goal is to give faster feedback to infer developers and reduce the
amount of times diffs are sent with build warnings. Also it's now faster
to alternate between changing infer and running unit tests since test
mode is just dev mode.
Reviewed By: ngorogiannis
Differential Revision: D23167416
fbshipit-source-id: d663b6054
Summary:
Newer is better. Also upgrade other dependencies to their latest
version in opam.
allow-large-files
Reviewed By: skcho
Differential Revision: D21548074
fbshipit-source-id: 4061c71ee
Summary:
Not sure what's going on but even though `set -e` is passed,
`install_opam_deps` was seen as succeeding even when the `opam install`
command failed. This prevented `opam_retry` from getting triggered (and
the error was also silently ignored).
Also get rid of a couple of useless subshells in `opam_retry`.
Reviewed By: jberdine
Differential Revision: D18615860
fbshipit-source-id: 360232623
Summary:
Currently only user-specified opam switches that name compilers are
supported. This diff generalizes this so that the switch name and
compiler can be specified separately.
Reviewed By: jvillard
Differential Revision: D18477910
fbshipit-source-id: f17c6363d
Summary:
Previously it was required to provide SDKROOT during configure on Mojave
hosts to `make` the project which in scripts was messing up local clang
and somewhat error-prone. Instead we could use xcrun to find required SDK
paths automatically.
Reviewed By: jvillard
Differential Revision: D16072354
fbshipit-source-id: 93cbf3980
Summary: Turns out we were using it wrong and that's why the lock wasn't working. See https://github.com/ocaml/opam/issues/3694
Reviewed By: ezgicicek
Differential Revision: D13529136
fbshipit-source-id: 66c781e5f
Summary:
Avoid sandboxing issues and mark the incompatibility with javalib 3.0.
Also remove workaround for opam bug that has been fixed and that was
causing portability issues on osx where `realpath` isn't always
available.
Reviewed By: ngorogiannis
Differential Revision: D13450029
fbshipit-source-id: ea33b06d8
Summary:
Switches from opam 1 to opam 2.
Opam2 has some cool new features that simplify some of the scripting.
Notable changes:
1. Use the new `opam lock` *plugin* from https://github.com/AltGr/opam-lock/ instead of https://github.com/rgrinberg/opam-lock. This has a simpler interface for our purposes.
2. Change the way `./build-infer.sh` can be called to use an already existing switch: simply pass `--user-opam-switch` to the script and it won't attempt to create/set the current switch. This can be used to build infer in a local switch for instance.
3. Take advantage of automatic pinning where possible, eg to install infer deps without using opam.locked.
Reviewed By: ngorogiannis
Differential Revision: D13167863
fbshipit-source-id: 1a667c270
Summary:
Change the documentation to refer to (upcoming) binary releases. Update the
scripts to treat .release differently: now we want to build clang and the
plugins even in release mode, as that's just the preparation for the release
tarball containing only binaries.
Reviewed By: mbouaziz
Differential Revision: D8235388
fbshipit-source-id: bfb4ae8
Summary:
Change the license of the source code from BSD + PATENTS to MIT.
Change `checkCopyright` to reflect the new license and learn some new file
types.
Generated with:
```
git grep BSD | xargs -n 1 ./scripts/checkCopyright -i
```
Reviewed By: jeremydubreil, mbouaziz, jberdine
Differential Revision: D8071249
fbshipit-source-id: 97ca23a
Summary:
Users missing the development package for libsqlite3 would see only a failure to install "sqlite3". Adding conf-sqlite3 to the dependencies does not improve things that much but there is a bit more information in the console:
```
[ERROR] The compilation of conf-sqlite3 failed at "pkg-config sqlite3".
[ERROR] The compilation of sqlite3 failed at "jbuilder build -p sqlite3 -j 4".
#=== ERROR while installing conf-sqlite3.1 ====================================#
# opam-version 1.2.2
# os linux
# command pkg-config sqlite3
# path /home/jul/.opam/infer-4.06.1+flambda/build/conf-sqlite3.1
# compiler 4.06.1+flambda
# exit-code 1
# env-file /home/jul/.opam/infer-4.06.1+flambda/build/conf-sqlite3.1/conf-sqlite3-7504-7afd23.env
# stdout-file /home/jul/.opam/infer-4.06.1+flambda/build/conf-sqlite3.1/conf-sqlite3-7504-7afd23.out
# stderr-file /home/jul/.opam/infer-4.06.1+flambda/build/conf-sqlite3.1/conf-sqlite3-7504-7afd23.err
#=== ERROR while installing sqlite3.4.3.2 =====================================#
# opam-version 1.2.2
# os linux
# command jbuilder build -p sqlite3 -j 4
# path /home/jul/.opam/infer-4.06.1+flambda/build/sqlite3.4.3.2
# compiler 4.06.1+flambda
# exit-code 1
# env-file /home/jul/.opam/infer-4.06.1+flambda/build/sqlite3.4.3.2/sqlite3-7504-d2c37b.env
# stdout-file /home/jul/.opam/infer-4.06.1+flambda/build/sqlite3.4.3.2/sqlite3-7504-d2c37b.out
# stderr-file /home/jul/.opam/infer-4.06.1+flambda/build/sqlite3.4.3.2/sqlite3-7504-d2c37b.err
### stderr ###
# -> stdout:
# [...]
# | ast_impl_magic_number: Caml1999M022
# | ast_intf_magic_number: Caml1999N022
# | cmxs_magic_number: Caml1999D022
# | cmt_magic_number: Caml1999T022
# -> stderr:
# Fatal error: exception End_of_file
# Raised at file "src/import0.ml" (inlined), line 351, characters 22-32
# Called from file "src/configurator.ml", line 511, characters 13-22
# Called from file "src/config/discover.ml", line 42, characters 2-1023
Exception:
Reqs_error
(Process_error
("opam install -y ANSITerminal.0.8 atd.1.12.0 atdgen.1.12.0 base.v0.11.0 base64.2.2.0 bin_prot.v0.11.0 biniou.1.2.0 camlp4.4.06+1 camlzip.1.07 cmdliner.1.0.2 conf-aclocal.1.0.0 conf-autoconf.0.1 conf-m4.1 conf-pkg-config.1.0 conf-sqlite3.1 conf-which.1 conf-zlib.1 configurator.v0.11.0 core.v0.11.0 "... (* string length 1469; truncated *),
Unix.WEXITED 4)).
```
Also fix some issues with `build-infer.sh`:
- fix a problem where `SCRIPT_PATH` should be `SCRIPT_DIR`
- add `set -u` and `set -o pipefail` to make sure we don't miss errors in the future
- add quotes everywhere
- make number of `JOBS` user-configurable instead of hardcoding `$NCPU`
Reviewed By: mbouaziz
Differential Revision: D8201849
fbshipit-source-id: 19b7c77
Summary:
No need for our own patched version now that it's available in opam.
You should `opam pin remove --no-action javalib && ./build-infer.sh` to get rid of the previous version.
Reviewed By: mbouaziz
Differential Revision: D6063730
fbshipit-source-id: 8efd598
Summary:
With flambda (`-O3`), compilation time is ~5x slower, but the backend is ~25% faster!
To mitigate the atrocious compilation times, introduce a new `opt` build mode in the jbuilder files.
- build in "opt" mode by default from the toplevel (so that install scripts and external users get the fastest infer by default), in "default" mode by default from infer/src (since the latter is only called directly by infer devs, for faster builds)
- `make byte` is as fast as before in any mode
- `make test` will build "opt" by default, which is very slow. Solution for testing (or building the models) locally: `make BUILD_MODE=default test`.
- You can even change the default locally with `export BUILD_MODE=default`.
The benchmarks are to be taken with a sizable pinch of salt because I ran them only once and other stuff could be running in the background. That said, the perf win is consistent across all projects, with 15-20% win in wallclock time and around 25% win in total CPU time, ~9% win in sys time, and ~25% fewer minor allocations, and ~5-10% fewer overall allocations. This is only for the backend; the capture is by and large unaffected (either the same or a tad faster within noise range).
Here are the results running on OpenSSL 1.0.2d on osx (12 cores, 32G RAM)
=== base
infer binary: 26193088 bytes
compile time: 40s
capture:
```lang=text
real 1m7.513s
user 3m11.437s
sys 0m55.236s
```
analysis:
```lang=text
real 5m41.580s
user 61m37.855s
sys 1m12.870s
```
Memory profile:
```lang=json
{
...
"minor_gb": 0.1534719169139862,
"promoted_gb": 0.0038930922746658325,
"major_gb": 0.4546157643198967,
"allocated_gb": 0.6041945889592171,
"minor_collections": 78,
"major_collections": 23,
"compactions": 7,
"top_heap_gb": 0.07388687133789062,
"stack_kb": 0.3984375,
"minor_heap_kb": 8192.0,
...
}
```
=== flambda with stock options (no `-Oclassic`, just the same flags as base)
Exactly the same as base.
=== flambda `-O3`
infer binary: 56870376 bytes (2.17x bigger)
compile time: 191s (4.78x slower)
capture is the same as base:
```lang=text
real 1m9.203s
user 3m12.242s
sys 0m58.905s
```
analysis is ~20% wallclock time faster, ~25% CPU time faster:
```lang=text
real 4m32.656s
user 46m43.987s
sys 1m2.424s
```
memory usage is a bit lower too:
```lang=json
{
...
"minor_gb": 0.11583046615123749, // 75% of previous
"promoted_gb": 0.00363825261592865, // 93% of previous
"major_gb": 0.45415670424699783, // about same
"allocated_gb": 0.5663489177823067, // 94% of previous
"minor_collections": 73,
"major_collections": 22,
"compactions": 7,
"top_heap_gb": 0.07165145874023438,
"stack_kb": 0.3359375,
"minor_heap_kb": 8192.0,
...
}
```
=== flambda `-O2`
Not nearly as exciting as `-O3`, but the compilation cost is still quite high:
infer: 37826856 bytes
compilation of infer: 100s
Capture and analysis timings are mostly the same as base.
Reviewed By: jberdine
Differential Revision: D4867979
fbshipit-source-id: 99230b7
Summary:
Installing with `opam pin` risks upgrading a lot of package in the current
switch, which is very slow. It's also wasted work since we'll install back
their older versions afterwards with `opam lock`. Moreover, that second step
can now fail if javalib needs to be recompiled (which it seems it does),
because we delete the javalib sources after having pinned them.
Solution: 1. do not install javalib at `opam pin` time, and 2. do not remove its source tree after pinning.
Reviewed By: martinoluca
Differential Revision: D5764153
fbshipit-source-id: 9c9b1c7
Summary:
For instance, `./build-infer.sh --opam-switch "4.04.2+spacetime"` now builds an
actual spacetime switch instead of a clone of the 4.04.2 switch called
"4.04.2+spacetime".
Reviewed By: jberdine
Differential Revision: D5631258
fbshipit-source-id: e6295c5
Summary:
Useful for hacking on infer, and when eg the default opam switch changes and
one has to remember which development packages to install all over again
(merlin, etc.).
Reviewed By: mbouaziz
Differential Revision: D5328701
fbshipit-source-id: 7b7694e
Summary:
Let's try this one more time.
- change build-infer.sh so that it also tries to init opam even if a switch is
specified. This shouldn't matter according to my testing and that means that
it's easier to have `./build-infer.sh --opam-switch 4.04.0` work out of the box
without needing to initialise the `4.04.0` opam switch ourselves beforehand.
- use that in .travis.yml
Reviewed By: jeremydubreil
Differential Revision: D5301320
fbshipit-source-id: 12f91ae
Summary: This uses opam.lock which is less likely to break due to external opam changes.
Reviewed By: cristianoc
Differential Revision: D5208622
fbshipit-source-id: af6f03c
Summary: This makes the build less likely to break because of package upgrades in opam.
Reviewed By: mbouaziz
Differential Revision: D4937908
fbshipit-source-id: 5ce9ec7
Summary:
Opam can give cryptic errors when all you need to do is `opam update`, so try
that for the user.
Reviewed By: mbouaziz
Differential Revision: D4867878
fbshipit-source-id: f740b13
Summary:
This enables the following workflow:
```
$ ./build-infer.sh
[...]
$ opam upgrade # takes infer dependencies into account
```
To do so, we install a temporary `infer-deps-XXXX` package as before (the temp
name is because opam thinks there's nothing to do otherwise), then we install
an `infer-deps` package permanently.
Reviewed By: mbouaziz
Differential Revision: D4867195
fbshipit-source-id: da806f9
Summary:
When invoking `./build-infer.sh java`, there is no need to update the facebook-clang-plugins submodule that is done as part of the `autogen.sh` script. The submodule update takes a very long time and can be skipped when building only for java
Closes https://github.com/facebook/infer/pull/557
Reviewed By: dkgi
Differential Revision: D4387800
Pulled By: sblackshear
fbshipit-source-id: 1662f7b
Summary:
If ocaml is not installed (or if homebrew has sandboxed it out), `opam
init` fails unless passed `--compiler`.
Reviewed By: sblackshear, jvillard
Differential Revision: D3927532
fbshipit-source-id: d2fbec9
Summary:
- Copy the opam file to an empty directory and tell opam to install the deps of that. This way, we save an rsync of the whole infer/ directory.
- Enable configure options coming from the outside via INFER_CONFIGURE_OPTS.
- opam pin reason from a newer version with the correct dependencies for reason
- Also cleanup the code a bit
Reviewed By: jberdine
Differential Revision: D3620205
fbshipit-source-id: 5749039
Summary:
build-infer.sh script will create separate switch infer-4.02.3 and install all infer's deps there (including reason)
With this change users won't have to run opam install commands directly
Reviewed By: jvillard
Differential Revision: D3508897
fbshipit-source-id: 9f8e741
Summary:
Build everything at once all the time. This removes the need for multiple
directories, which were a hassle to begin with.
This removes the `java`, `clang`, and `llvm` targets in various Makefiles as
well.
Reviewed By: jberdine
Differential Revision: D3317230
fbshipit-source-id: 8e86140
Summary:public
Use getconf to compute NCPU as it is uniform across platforms.
Use result in several places to avoid fork bombs. I routinely see
errors about fork running out of memory when building e.g. the clang
plugin.
Reviewed By: jvillard
Differential Revision: D3148970
fb-gh-sync-id: 6d071c9
fbshipit-source-id: 6d071c9
Summary:
fixes#267.
By calling opam with the --shell=bash in the script I don't get the errors I had before.
The new line I added in the docker/README.md was so the list would display correctly when rendered.
Closes https://github.com/facebook/infer/pull/272
Reviewed By: jberdine
Differential Revision: D2916859
Pulled By: jvillard
fb-gh-sync-id: 4243f79
shipit-source-id: 4243f79
Summary:
public
Because we ran `./configure` *after* checking if clang was to be compiled, we
could fail mysteriously if `shasum` was not present. Now, we fail at the
`./configure` step with a clear error message.
closes#275
Reviewed By: akotulski
Differential Revision: D2921841
fb-gh-sync-id: b8edd8b
shipit-source-id: b8edd8b
Jules Villard
Anton Kochkov
Josh Berdine
Sungkeun Cho
Artem Pianykh
Phoebe Nichols
Sam Blackshear
Gautam Korlam
Yunxing Dai
Andrzej Kotulski
Skyfold