Summary:
Not sure what's going on but even though `set -e` is passed,
`install_opam_deps` was seen as succeeding even when the `opam install`
command failed. This prevented `opam_retry` from getting triggered (and
the error was also silently ignored).
Also get rid of a couple of useless subshells in `opam_retry`.
Reviewed By: jberdine
Differential Revision: D18615860
fbshipit-source-id: 360232623
Summary:
Currently only user-specified opam switches that name compilers are
supported. This diff generalizes this so that the switch name and
compiler can be specified separately.
Reviewed By: jvillard
Differential Revision: D18477910
fbshipit-source-id: f17c6363d
Summary:
Previously it was required to provide SDKROOT during configure on Mojave
hosts to `make` the project which in scripts was messing up local clang
and somewhat error-prone. Instead we could use xcrun to find required SDK
paths automatically.
Reviewed By: jvillard
Differential Revision: D16072354
fbshipit-source-id: 93cbf3980
Summary: Turns out we were using it wrong and that's why the lock wasn't working. See https://github.com/ocaml/opam/issues/3694
Reviewed By: ezgicicek
Differential Revision: D13529136
fbshipit-source-id: 66c781e5f
Summary:
Avoid sandboxing issues and mark the incompatibility with javalib 3.0.
Also remove workaround for opam bug that has been fixed and that was
causing portability issues on osx where `realpath` isn't always
available.
Reviewed By: ngorogiannis
Differential Revision: D13450029
fbshipit-source-id: ea33b06d8
Summary:
Switches from opam 1 to opam 2.
Opam2 has some cool new features that simplify some of the scripting.
Notable changes:
1. Use the new `opam lock` *plugin* from https://github.com/AltGr/opam-lock/ instead of https://github.com/rgrinberg/opam-lock. This has a simpler interface for our purposes.
2. Change the way `./build-infer.sh` can be called to use an already existing switch: simply pass `--user-opam-switch` to the script and it won't attempt to create/set the current switch. This can be used to build infer in a local switch for instance.
3. Take advantage of automatic pinning where possible, eg to install infer deps without using opam.locked.
Reviewed By: ngorogiannis
Differential Revision: D13167863
fbshipit-source-id: 1a667c270
Summary:
Change the documentation to refer to (upcoming) binary releases. Update the
scripts to treat .release differently: now we want to build clang and the
plugins even in release mode, as that's just the preparation for the release
tarball containing only binaries.
Reviewed By: mbouaziz
Differential Revision: D8235388
fbshipit-source-id: bfb4ae8
Summary:
Change the license of the source code from BSD + PATENTS to MIT.
Change `checkCopyright` to reflect the new license and learn some new file
types.
Generated with:
```
git grep BSD | xargs -n 1 ./scripts/checkCopyright -i
```
Reviewed By: jeremydubreil, mbouaziz, jberdine
Differential Revision: D8071249
fbshipit-source-id: 97ca23a
Summary:
Users missing the development package for libsqlite3 would see only a failure to install "sqlite3". Adding conf-sqlite3 to the dependencies does not improve things that much but there is a bit more information in the console:
```
[ERROR] The compilation of conf-sqlite3 failed at "pkg-config sqlite3".
[ERROR] The compilation of sqlite3 failed at "jbuilder build -p sqlite3 -j 4".
#=== ERROR while installing conf-sqlite3.1 ====================================#
# opam-version 1.2.2
# os linux
# command pkg-config sqlite3
# path /home/jul/.opam/infer-4.06.1+flambda/build/conf-sqlite3.1
# compiler 4.06.1+flambda
# exit-code 1
# env-file /home/jul/.opam/infer-4.06.1+flambda/build/conf-sqlite3.1/conf-sqlite3-7504-7afd23.env
# stdout-file /home/jul/.opam/infer-4.06.1+flambda/build/conf-sqlite3.1/conf-sqlite3-7504-7afd23.out
# stderr-file /home/jul/.opam/infer-4.06.1+flambda/build/conf-sqlite3.1/conf-sqlite3-7504-7afd23.err
#=== ERROR while installing sqlite3.4.3.2 =====================================#
# opam-version 1.2.2
# os linux
# command jbuilder build -p sqlite3 -j 4
# path /home/jul/.opam/infer-4.06.1+flambda/build/sqlite3.4.3.2
# compiler 4.06.1+flambda
# exit-code 1
# env-file /home/jul/.opam/infer-4.06.1+flambda/build/sqlite3.4.3.2/sqlite3-7504-d2c37b.env
# stdout-file /home/jul/.opam/infer-4.06.1+flambda/build/sqlite3.4.3.2/sqlite3-7504-d2c37b.out
# stderr-file /home/jul/.opam/infer-4.06.1+flambda/build/sqlite3.4.3.2/sqlite3-7504-d2c37b.err
### stderr ###
# -> stdout:
# [...]
# | ast_impl_magic_number: Caml1999M022
# | ast_intf_magic_number: Caml1999N022
# | cmxs_magic_number: Caml1999D022
# | cmt_magic_number: Caml1999T022
# -> stderr:
# Fatal error: exception End_of_file
# Raised at file "src/import0.ml" (inlined), line 351, characters 22-32
# Called from file "src/configurator.ml", line 511, characters 13-22
# Called from file "src/config/discover.ml", line 42, characters 2-1023
Exception:
Reqs_error
(Process_error
("opam install -y ANSITerminal.0.8 atd.1.12.0 atdgen.1.12.0 base.v0.11.0 base64.2.2.0 bin_prot.v0.11.0 biniou.1.2.0 camlp4.4.06+1 camlzip.1.07 cmdliner.1.0.2 conf-aclocal.1.0.0 conf-autoconf.0.1 conf-m4.1 conf-pkg-config.1.0 conf-sqlite3.1 conf-which.1 conf-zlib.1 configurator.v0.11.0 core.v0.11.0 "... (* string length 1469; truncated *),
Unix.WEXITED 4)).
```
Also fix some issues with `build-infer.sh`:
- fix a problem where `SCRIPT_PATH` should be `SCRIPT_DIR`
- add `set -u` and `set -o pipefail` to make sure we don't miss errors in the future
- add quotes everywhere
- make number of `JOBS` user-configurable instead of hardcoding `$NCPU`
Reviewed By: mbouaziz
Differential Revision: D8201849
fbshipit-source-id: 19b7c77
Summary:
No need for our own patched version now that it's available in opam.
You should `opam pin remove --no-action javalib && ./build-infer.sh` to get rid of the previous version.
Reviewed By: mbouaziz
Differential Revision: D6063730
fbshipit-source-id: 8efd598
Summary:
With flambda (`-O3`), compilation time is ~5x slower, but the backend is ~25% faster!
To mitigate the atrocious compilation times, introduce a new `opt` build mode in the jbuilder files.
- build in "opt" mode by default from the toplevel (so that install scripts and external users get the fastest infer by default), in "default" mode by default from infer/src (since the latter is only called directly by infer devs, for faster builds)
- `make byte` is as fast as before in any mode
- `make test` will build "opt" by default, which is very slow. Solution for testing (or building the models) locally: `make BUILD_MODE=default test`.
- You can even change the default locally with `export BUILD_MODE=default`.
The benchmarks are to be taken with a sizable pinch of salt because I ran them only once and other stuff could be running in the background. That said, the perf win is consistent across all projects, with 15-20% win in wallclock time and around 25% win in total CPU time, ~9% win in sys time, and ~25% fewer minor allocations, and ~5-10% fewer overall allocations. This is only for the backend; the capture is by and large unaffected (either the same or a tad faster within noise range).
Here are the results running on OpenSSL 1.0.2d on osx (12 cores, 32G RAM)
=== base
infer binary: 26193088 bytes
compile time: 40s
capture:
```lang=text
real 1m7.513s
user 3m11.437s
sys 0m55.236s
```
analysis:
```lang=text
real 5m41.580s
user 61m37.855s
sys 1m12.870s
```
Memory profile:
```lang=json
{
...
"minor_gb": 0.1534719169139862,
"promoted_gb": 0.0038930922746658325,
"major_gb": 0.4546157643198967,
"allocated_gb": 0.6041945889592171,
"minor_collections": 78,
"major_collections": 23,
"compactions": 7,
"top_heap_gb": 0.07388687133789062,
"stack_kb": 0.3984375,
"minor_heap_kb": 8192.0,
...
}
```
=== flambda with stock options (no `-Oclassic`, just the same flags as base)
Exactly the same as base.
=== flambda `-O3`
infer binary: 56870376 bytes (2.17x bigger)
compile time: 191s (4.78x slower)
capture is the same as base:
```lang=text
real 1m9.203s
user 3m12.242s
sys 0m58.905s
```
analysis is ~20% wallclock time faster, ~25% CPU time faster:
```lang=text
real 4m32.656s
user 46m43.987s
sys 1m2.424s
```
memory usage is a bit lower too:
```lang=json
{
...
"minor_gb": 0.11583046615123749, // 75% of previous
"promoted_gb": 0.00363825261592865, // 93% of previous
"major_gb": 0.45415670424699783, // about same
"allocated_gb": 0.5663489177823067, // 94% of previous
"minor_collections": 73,
"major_collections": 22,
"compactions": 7,
"top_heap_gb": 0.07165145874023438,
"stack_kb": 0.3359375,
"minor_heap_kb": 8192.0,
...
}
```
=== flambda `-O2`
Not nearly as exciting as `-O3`, but the compilation cost is still quite high:
infer: 37826856 bytes
compilation of infer: 100s
Capture and analysis timings are mostly the same as base.
Reviewed By: jberdine
Differential Revision: D4867979
fbshipit-source-id: 99230b7
Summary:
Installing with `opam pin` risks upgrading a lot of package in the current
switch, which is very slow. It's also wasted work since we'll install back
their older versions afterwards with `opam lock`. Moreover, that second step
can now fail if javalib needs to be recompiled (which it seems it does),
because we delete the javalib sources after having pinned them.
Solution: 1. do not install javalib at `opam pin` time, and 2. do not remove its source tree after pinning.
Reviewed By: martinoluca
Differential Revision: D5764153
fbshipit-source-id: 9c9b1c7
Summary:
For instance, `./build-infer.sh --opam-switch "4.04.2+spacetime"` now builds an
actual spacetime switch instead of a clone of the 4.04.2 switch called
"4.04.2+spacetime".
Reviewed By: jberdine
Differential Revision: D5631258
fbshipit-source-id: e6295c5
Summary:
Useful for hacking on infer, and when eg the default opam switch changes and
one has to remember which development packages to install all over again
(merlin, etc.).
Reviewed By: mbouaziz
Differential Revision: D5328701
fbshipit-source-id: 7b7694e
Summary:
Let's try this one more time.
- change build-infer.sh so that it also tries to init opam even if a switch is
specified. This shouldn't matter according to my testing and that means that
it's easier to have `./build-infer.sh --opam-switch 4.04.0` work out of the box
without needing to initialise the `4.04.0` opam switch ourselves beforehand.
- use that in .travis.yml
Reviewed By: jeremydubreil
Differential Revision: D5301320
fbshipit-source-id: 12f91ae
Summary: This uses opam.lock which is less likely to break due to external opam changes.
Reviewed By: cristianoc
Differential Revision: D5208622
fbshipit-source-id: af6f03c
Summary: This makes the build less likely to break because of package upgrades in opam.
Reviewed By: mbouaziz
Differential Revision: D4937908
fbshipit-source-id: 5ce9ec7
Summary:
Opam can give cryptic errors when all you need to do is `opam update`, so try
that for the user.
Reviewed By: mbouaziz
Differential Revision: D4867878
fbshipit-source-id: f740b13
Summary:
This enables the following workflow:
```
$ ./build-infer.sh
[...]
$ opam upgrade # takes infer dependencies into account
```
To do so, we install a temporary `infer-deps-XXXX` package as before (the temp
name is because opam thinks there's nothing to do otherwise), then we install
an `infer-deps` package permanently.
Reviewed By: mbouaziz
Differential Revision: D4867195
fbshipit-source-id: da806f9
Summary:
When invoking `./build-infer.sh java`, there is no need to update the facebook-clang-plugins submodule that is done as part of the `autogen.sh` script. The submodule update takes a very long time and can be skipped when building only for java
Closes https://github.com/facebook/infer/pull/557
Reviewed By: dkgi
Differential Revision: D4387800
Pulled By: sblackshear
fbshipit-source-id: 1662f7b
Summary:
If ocaml is not installed (or if homebrew has sandboxed it out), `opam
init` fails unless passed `--compiler`.
Reviewed By: sblackshear, jvillard
Differential Revision: D3927532
fbshipit-source-id: d2fbec9
Summary:
- Copy the opam file to an empty directory and tell opam to install the deps of that. This way, we save an rsync of the whole infer/ directory.
- Enable configure options coming from the outside via INFER_CONFIGURE_OPTS.
- opam pin reason from a newer version with the correct dependencies for reason
- Also cleanup the code a bit
Reviewed By: jberdine
Differential Revision: D3620205
fbshipit-source-id: 5749039
Summary:
build-infer.sh script will create separate switch infer-4.02.3 and install all infer's deps there (including reason)
With this change users won't have to run opam install commands directly
Reviewed By: jvillard
Differential Revision: D3508897
fbshipit-source-id: 9f8e741
Summary:
Build everything at once all the time. This removes the need for multiple
directories, which were a hassle to begin with.
This removes the `java`, `clang`, and `llvm` targets in various Makefiles as
well.
Reviewed By: jberdine
Differential Revision: D3317230
fbshipit-source-id: 8e86140
Summary:public
Use getconf to compute NCPU as it is uniform across platforms.
Use result in several places to avoid fork bombs. I routinely see
errors about fork running out of memory when building e.g. the clang
plugin.
Reviewed By: jvillard
Differential Revision: D3148970
fb-gh-sync-id: 6d071c9
fbshipit-source-id: 6d071c9
Summary:
fixes#267.
By calling opam with the --shell=bash in the script I don't get the errors I had before.
The new line I added in the docker/README.md was so the list would display correctly when rendered.
Closes https://github.com/facebook/infer/pull/272
Reviewed By: jberdine
Differential Revision: D2916859
Pulled By: jvillard
fb-gh-sync-id: 4243f79
shipit-source-id: 4243f79
Summary:
public
Because we ran `./configure` *after* checking if clang was to be compiled, we
could fail mysteriously if `shasum` was not present. Now, we fail at the
`./configure` step with a clear error message.
closes#275
Reviewed By: akotulski
Differential Revision: D2921841
fb-gh-sync-id: b8edd8b
shipit-source-id: b8edd8b
Summary:
public
This fixes two things:
- `make -j9000` now works without crashing
- `make -C infer/src clang java` followed by `make -C infer/src clang` results
in a cached build the second time. Previously, a bug in ocamlbuild meant that
building infer/java would stomp over some files used by infer/clang, hence
infer/clang would get partially recompiled. More generally, alternating
between targets, or simply repeatedly calling `make` would result in unecessary
rebuilds.
Reviewed By: akotulski
Differential Revision: D2765544
fb-gh-sync-id: ecffdee
Summary:
public
This configure option disables building the java analysis and the associated
checks.
Refactor infer/src/Makefile.in to better isolate the Java-specific ocamlfind
dependencies.
Reviewed By: jeremydubreil
Differential Revision: D2712921
fb-gh-sync-id: 08077c2
Jules Villard
Josh Berdine
Sungkeun Cho
Artem Pianykh
Phoebe Nichols
Sam Blackshear
Gautam Korlam
Yunxing Dai
Andrzej Kotulski
Skyfold