Summary:
This was meant to be part of D7569054 but the wrong version landed.
classicblunder
Reviewed By: mbouaziz
Differential Revision: D7615865
fbshipit-source-id: a29b3af
Summary:
Making the shell stricter revealed a bug in `make devsetup`:
`shell_config_file` can be used without being initialised. Also, the error path
when testing if `BUILD_MODE` was set to default was wrong and would execute all
except the first `echo` command unconditionally. I changed the test so that it
tests only if `BUILD_MODE` is set in the environment.
Reviewed By: martinoluca
Differential Revision: D7615333
fbshipit-source-id: 08c5eb0
Summary:
Lost in D7569054: because `-e` is set, the command to run needs to be wrapped a
bit more carefully to continue the execution in case of failure and get to the
part where we print the error messages on the console.
Reviewed By: mbouaziz
Differential Revision: D7599442
fbshipit-source-id: 65f6bf7
Summary:
Run with `SHELL = bash -e -u -o pipefail` to catch many kinds of failures. We
were silently failing during `make install` because of some missing escaping,
and the failure was hidden because it was happening inside a bash `for` loop.
This fixes the escaping issue and makes sure such issues will result in an
error as of now.
Also removes dangerous `find -exec` instances: `find` will `exit 0` event if
some commands failed.
Fixes#887
Reviewed By: mbouaziz
Differential Revision: D7569054
fbshipit-source-id: 542fe50
Summary:
The dead code checker slows down the test suite notably, and can be annoying
when you're in the middle of a change.
Depends on D7443825
Reviewed By: mbouaziz, ngorogiannis
Differential Revision: D7443881
fbshipit-source-id: 746cf6e
Summary:
Make dead code detection part of `make test` so that dead code stops creeping
in. It's only enabled if all the analysers are enabled and if this is a
facebook build, because the dead code detection will have false positives
otherwise.
Reviewed By: mbouaziz
Differential Revision: D6807395
fbshipit-source-id: ebbd835
Summary: This should allow to report several occurences of the an issue appearing several times within the same method.
Reviewed By: jvillard
Differential Revision: D6783298
fbshipit-source-id: 5555906
Summary:
This avoids rebuilding the man pages at every build. Also add a `byte_infer`
target to build infer in bytecode + the models, and an `opt` target, as I found
myself missing those.
Reviewed By: mbouaziz
Differential Revision: D6761008
fbshipit-source-id: ffedd4b
Summary: This subdirectory was only containing tests related to nullable on Objective C.
Reviewed By: sblackshear
Differential Revision: D6657654
fbshipit-source-id: 11003f2
Summary:
Upgrade ocamlformat to 0.3, and (necessarily) base to v0.10.0.
- Fix accumulated mis-formatting
- Update opam.lock to unbreak clean build
- Update to base v0.10.0
- Update opam.lock for base
- Update offline opam repo
- Everyone should already have removed their ocamlformat pin
- ocamlformat 0.3 supports output to stdout natively
- bump version of ocamlformat
Reviewed By: jeremydubreil
Differential Revision: D6636741
fbshipit-source-id: 41a56a8
Summary: This is to allow the bi-abduction analysis and the nullable checker for Clang languages to run together without stepping on each other toes.
Reviewed By: sblackshear
Differential Revision: D6567934
fbshipit-source-id: a318c33
Summary:
Change ocamlformat installation procedure to use opam instead of
pinning.
Reformat all code with v0.2, which has a few improvements.
Reviewed By: jvillard
Differential Revision: D6292057
fbshipit-source-id: 759967f
Summary:
Install ocamlformat from github as part of `make devsetup`, and use it
for formatting OCaml (and jbuild) code.
Reviewed By: jvillard
Differential Revision: D6092464
fbshipit-source-id: 4ba0845
Summary:
Buck reads the version on stderr or, very recently, from either stdout or stderr.
This makes infer output the version of stderr when called from Buck or invoked as javac, and on stdout otherwise.
Reviewed By: martinoluca
Differential Revision: D6098392
fbshipit-source-id: 23f1d5a
Summary: Tuareg is only useful for emacs, and `opam install tuareg` fails if emacs is not found.
Reviewed By: mbouaziz
Differential Revision: D6098693
fbshipit-source-id: 167628a
Summary: We will then be able to merge the tests for the other checkers without affecting these lab tests
Reviewed By: jvillard
Differential Revision: D6039433
fbshipit-source-id: e575ce9
Summary:
The previous domain for SIOF was duplicating some work with the generic Trace
domain, and basically was a bit confused and confusing. A sink was a set of
global accesses, and a state contains a set of sinks. Then the checker has to
needlessly jump through hoops to normalize this set of sets of accesses into a
set of accesses.
The new domain has one sink = one access, as suggested by sblackshear. This simplifies
a few things, and makes the dedup logic much easier: just grab the first report
of the list of reports for a function.
We only report on the fake procedures generated to initialise a global, and the
filtering means that we keep only one report per global.
Reviewed By: sblackshear
Differential Revision: D5932138
fbshipit-source-id: acb7285
Summary:
Use an SQLite database to store proc attributes, instead of files on disk.
Wrap SQLite operations in two layers:
1. `SqliteUtils` provides helper functions to make sure DB operations succeed
2. `KeyValue` provides a functor to expose a simple and type-safe key/value store backed by the SQLite DB.
Reviewed By: jberdine
Differential Revision: D5640053
fbshipit-source-id: 31050e5
Summary: Example of combination between annotating fields with nullable and the biabduction analysis in Objective C
Reviewed By: dulmarod
Differential Revision: D5906016
fbshipit-source-id: b95c6e0
Summary: Useful to know if your build is going to take 40s or 4 minutes.
Reviewed By: jeremydubreil
Differential Revision: D5872878
fbshipit-source-id: 1d370f3
Summary:
Suggesting to add `_Nullable` on the fields checked for, or assigned to, `nullptr` will allow the biabduction analysis to report null dereferences that are related to the lifetime of objects.
Depends on D5832147
Reviewed By: sblackshear
Differential Revision: D5836538
fbshipit-source-id: c1b8e48
Summary:
With flambda (`-O3`), compilation time is ~5x slower, but the backend is ~25% faster!
To mitigate the atrocious compilation times, introduce a new `opt` build mode in the jbuilder files.
- build in "opt" mode by default from the toplevel (so that install scripts and external users get the fastest infer by default), in "default" mode by default from infer/src (since the latter is only called directly by infer devs, for faster builds)
- `make byte` is as fast as before in any mode
- `make test` will build "opt" by default, which is very slow. Solution for testing (or building the models) locally: `make BUILD_MODE=default test`.
- You can even change the default locally with `export BUILD_MODE=default`.
The benchmarks are to be taken with a sizable pinch of salt because I ran them only once and other stuff could be running in the background. That said, the perf win is consistent across all projects, with 15-20% win in wallclock time and around 25% win in total CPU time, ~9% win in sys time, and ~25% fewer minor allocations, and ~5-10% fewer overall allocations. This is only for the backend; the capture is by and large unaffected (either the same or a tad faster within noise range).
Here are the results running on OpenSSL 1.0.2d on osx (12 cores, 32G RAM)
=== base
infer binary: 26193088 bytes
compile time: 40s
capture:
```lang=text
real 1m7.513s
user 3m11.437s
sys 0m55.236s
```
analysis:
```lang=text
real 5m41.580s
user 61m37.855s
sys 1m12.870s
```
Memory profile:
```lang=json
{
...
"minor_gb": 0.1534719169139862,
"promoted_gb": 0.0038930922746658325,
"major_gb": 0.4546157643198967,
"allocated_gb": 0.6041945889592171,
"minor_collections": 78,
"major_collections": 23,
"compactions": 7,
"top_heap_gb": 0.07388687133789062,
"stack_kb": 0.3984375,
"minor_heap_kb": 8192.0,
...
}
```
=== flambda with stock options (no `-Oclassic`, just the same flags as base)
Exactly the same as base.
=== flambda `-O3`
infer binary: 56870376 bytes (2.17x bigger)
compile time: 191s (4.78x slower)
capture is the same as base:
```lang=text
real 1m9.203s
user 3m12.242s
sys 0m58.905s
```
analysis is ~20% wallclock time faster, ~25% CPU time faster:
```lang=text
real 4m32.656s
user 46m43.987s
sys 1m2.424s
```
memory usage is a bit lower too:
```lang=json
{
...
"minor_gb": 0.11583046615123749, // 75% of previous
"promoted_gb": 0.00363825261592865, // 93% of previous
"major_gb": 0.45415670424699783, // about same
"allocated_gb": 0.5663489177823067, // 94% of previous
"minor_collections": 73,
"major_collections": 22,
"compactions": 7,
"top_heap_gb": 0.07165145874023438,
"stack_kb": 0.3359375,
"minor_heap_kb": 8192.0,
...
}
```
=== flambda `-O2`
Not nearly as exciting as `-O3`, but the compilation cost is still quite high:
infer: 37826856 bytes
compilation of infer: 100s
Capture and analysis timings are mostly the same as base.
Reviewed By: jberdine
Differential Revision: D4867979
fbshipit-source-id: 99230b7
Summary: This will run infer with `--no-keep-going` by default in our tests.
Reviewed By: sblackshear
Differential Revision: D5814237
fbshipit-source-id: c1e1a4e
Summary:
This gets rid of the horrible hack of asking jbuilder to build
_build/{default,test}/.ppx/ppx_compare/ppx.exe prior to starting several
jbuilds in parallel. We now refrain from starting several jbuilds in parallel.
Also, I finally understood that order-only deps do nothing for phony targets,
and not much else for non-phony ones. I hate make even more.
Reviewed By: jberdine
Differential Revision: D5678699
fbshipit-source-id: 52179e8
Summary:
This simplifies the jbuild files: no need to list these files explicitly
anymore, nor to exclude them explicitly from the main `InferModules` library
(due to their different compilation flags).
Isolate common parts into jbuild.common do `cat`-based code inclusion into
jbuild files to factorize code.
Reviewed By: jberdine
Differential Revision: D5678328
fbshipit-source-id: 6d7d925
Summary: This takes several seconds and is better tested in parallel with the other tests instead of at the end.
Reviewed By: mbouaziz
Differential Revision: D5591774
fbshipit-source-id: f474cae
Summary:
Use jbuilder to build infer instead of ocamlbuild. This is mainly to get faster builds:
```
times in 10ms, ±differences measured in speedups, 4 cores
| | ocb total | jb | ±total | ocb user | jb | ±user | ocb cpu | jb | ±cpu | ocb sys | jb | ±sys |
|-----------------------------------+-----------+------+--------+----------+------+-------+---------+-----+------+---------+------+------|
| byte from scratch | 6428 | 2456 | 2.62 | 7743 | 6662 | 1.16 | 138 | 331 | 2.40 | 1184 | 1477 | 0.80 |
| native from scratch | 9841 | 4289 | 2.29 | 9530 | 8834 | 1.08 | 110 | 245 | 2.23 | 1373 | 1712 | 0.80 |
| byte after native | 29578 | 1602 | 18.46 | 4514 | 4640 | 0.97 | 170 | 325 | 1.91 | 543 | 576 | 0.94 |
| change infer.ml byte | 344 | 282 | 1.22 | 292 | 215 | 1.36 | 96 | 99 | 1.03 | 040 | 066 | 0.61 |
| change infer.ml native | 837 | 223 | 3.75 | 789 | 174 | 4.53 | 98 | 99 | 1.01 | 036 | 47 | 0.77 |
| change Config.ml byte | 451 | 339 | 1.33 | 382 | 336 | 1.14 | 97 | 122 | 1.26 | 056 | 80 | 0.70 |
| change Config.ml native | 4024 | 1760 | 2.29 | 4585 | 4225 | 1.09 | 127 | 276 | 2.17 | 559 | 644 | 0.87 |
| change cFrontend_config.ml byte | 348 | 643 | 0.54 | 297 | 330 | 0.90 | 96 | 67 | 0.70 | 038 | 102 | 0.37 |
| change cFrontend_config.ml native | 1480 | 584 | 2.53 | 1435 | 906 | 1.58 | 106 | 185 | 1.75 | 136 | 178 | 0.76 |
#+TBLFM: $4=$2/$3;f2::$7=$5/$6;f2::$10=$9/$8;f2::$13=$11/$12;f2
50 cores
| | ocb total | jb | ±total | ocb user | jb | ±user | ocb cpu | jb | ±cpu | ocb sys | jb | ±sys |
|---------------------+-----------+------+--------+----------+------+-------+---------+----+------+---------+------+------|
| byte from scratch | 9114 | 2061 | 4.42 | 9334 | 5133 | 1.82 | | | 0/0 | 2566 | 1726 | 1.49 |
| native from scratch | 13481 | 3967 | 3.40 | 12291 | 7608 | 1.62 | | | 0/0 | 3003 | 2100 | 1.43 |
| byte after native | 3467 | 1476 | 2.35 | 5067 | 3912 | 1.30 | | | 0/0 | 971 | 801 | 1.21 |
#+TBLFM: $4=$2/$3;f2::$7=$5/$6;f2::$10=$9/$8;f2::$13=$11/$12;f2
```
Menu:
1. Write a jbuild file, autogenerated from jbuild.in because we need to fill in
some information at build-time (really, at configure time, but TODO), such as
whether or not clang is enabled.
2. Nuke lots of stuff from infer/src/Makefile that is now in the jbuild file
3. The jbuild file lives in infer/src/ so it can see all the sources. If we put it somewhere else, eg, infer/, then `jbuilder` scans too many files (all irrelevant) and takes 2.5s to start instead of .8s. Adding irrelevant directories to jbuild-ignore does not help.
4. jbuilder does not support subdirectories, so resort to listing all the
source files in the generated jbuild (only source directories need to be
manually listed in jbuild.in though). Still, the generated .merlin is wrong
and makes merlin find source files in _build, so manually tune it to get
good merlin support. We also lose some of merlin for unit tests as it
cannot see their build artefacts anymore.
5. checkCopyright gets its own jbuild because it's standalone. Also, remove
some deprecation warnings in checkCopyright due to the new version of Core from
a while ago.
6. Drop less-used Makefile features (they had regressed anyway) such as
building individual modules. Also, building mod_dep.pdf now takes all the
source files available so they better build (before, it would only take the
source files from the config, eg with or without clang) (that's pretty minor).
7. The toplevel is now built as a custom toplevel because that was easier. It
should soon be even easier: https://github.com/janestreet/jbuilder/issues/210
8. Move BUILTINS.mli to BUILTINS.ml because jbuilder is not happy about
interface files without implementations.
In particular, I did not try to migrate too much of the Makefile logic to jbuilder,
more can be done in the future.
Reviewed By: jberdine
Differential Revision: D5573661
fbshipit-source-id: 4ca6d8f
Summary:
Replace `inferTraceBugs` with `infer-explore` with a similar CLI. Some options changed:
- --max-level -> --max-nesting, and "max" is the default value instead of a possible value
- --no-source -> --no-source-preview
Reviewed By: mbouaziz
Differential Revision: D5526651
fbshipit-source-id: 8383f37
Summary:
This allows the user to specify a different command to build the current version of the project vs to build the previous version of the project. Here's an example:
```
infer diff --gen-previous-build-command-script "echo clang -c hello.c" ... -- clang -c hello2.c
```
By default the two build commands are the same. If the script is to be used for both the current and the previous versions, then it's on the user to run it once first, eg:
```
infer diff --gen-previous-build-command-script "./myscript.sh previous" ... -- $(./myscript.sh current)
```
Reviewed By: martinoluca
Differential Revision: D5500989
fbshipit-source-id: 7374b44
Summary:
Do not use the deprecated (and slower) `#infer` flavor. Instead, `infer-run`
runs capture with the `#infer-capture-all` flavor, followed by merging targets,
followed by the analysis.
Move the call to `MergeCapture` around to make this change easier.
Reviewed By: mbouaziz
Differential Revision: D5547199
fbshipit-source-id: 53c9996
Summary: Define `default` target at the very top of the Makefile to make sure some other target doesn't replace it, as was done accidentally in D5498555
Reviewed By: martinoluca
Differential Revision: D5536691
fbshipit-source-id: 7398ec0
Summary:
Cosmetic changes to the output of `make clean` and `make test-replace`.
Run the inferTraceBugs tests from tests/build_systems/ like other such e2e
tests, so that they can run in parallel with other tests instead of
sequentially at the end.
Reviewed By: dulmarod
Differential Revision: D5526599
fbshipit-source-id: 57231bd
Summary:
Pretty basic: warn when we see an assignment instruction `x = ...` and `x` is not live in the post of the instruction.
Only enabled for Clang at the moment because linters already warn on this for Java. But we can enable it later if we want to (should be fully generic).
Reviewed By: jeremydubreil
Differential Revision: D5450439
fbshipit-source-id: 693514c
Summary:
Fixes 2 issues with the build:
- The OCaml dependencies (Version.ml, atdgen-generated stuff, ...) would be generated twice when running `make -j test` from a clean tree, because `make` has no inter-Makefile visibility over common dependencies.
- We would run `atdgen` twice: once for the .ml, once for the .mli, even though one invocation is enough for both.
Add `make ocaml_clean` to nuke only the OCaml code (in particular because
rebuilding the C++ code in the plugin is slow when all I want to get rid of the
OCaml stuff).
Reviewed By: mbouaziz
Differential Revision: D5415318
fbshipit-source-id: 16f42c6
Summary:
First steps towards implementing diff analysis functionalities inside infer
itself. What works: run infer, checkout parent, re-run infer, checkout top
revision, compute the reportdiff (but no final surfacing on the console). Lots
of TODO still, inlined in the code.
Reviewed By: jberdine
Differential Revision: D5364226
fbshipit-source-id: 5b7f9a5
Summary:
Conversion and reformat of infer source using ocamlformat
auto-formatting tool.
Current status:
- Because Reason does not handle docstrings, the output of the
conversion is not 'Warning 50'-clean, meaning that there are
docstrings with ambiguous placement. I'll need to manually fix
them just before landing.
Reviewed By: jvillard
Differential Revision: D5225546
fbshipit-source-id: 3bd2786
Summary:
Add support for converting without actually converting. To be used as
a landing pad for rebasing over the conversion.
Reviewed By: jvillard
Differential Revision: D5379967
fbshipit-source-id: 1bd932d
Summary:
Useful for hacking on infer, and when eg the default opam switch changes and
one has to remember which development packages to install all over again
(merlin, etc.).
Reviewed By: mbouaziz
Differential Revision: D5328701
fbshipit-source-id: 7b7694e
Summary:
Forgot to substitute "infer" for "infer-lock-deps" and so `make opam.lock` was
failing if `infer` has not been manually pinned (which I didn't notice
previously because I had it git-pinned for some reason...).
Reviewed By: martinoluca
Differential Revision: D5310206
fbshipit-source-id: c023659
Summary: This change introduces the a new argument that lets you restrict the results of a differential report to only certain files.
Reviewed By: mbouaziz
Differential Revision: D5236626
fbshipit-source-id: 52711e9
Summary:
This messes up with the Buck cache: if infer recaptures a file with only trivial changes that shouldn't affect the capture Buck will still believe it has to reanalyze everything that depends on that file if there's non-deterministic data in infer's output.
Re-use the `--buck` flag used by the Java Buck integration for mostly the same purposes. Add a few special cases for the flavours integration (eg: keep capture data).
Change perf stats registration to take `Config.buck_cache_mode` into account instead of relying on each call site to handle that peculiarity correctly. Also, there's no need to create the perf stats directories before calling the registration function since it will do that too.
Reviewed By: jeremydubreil
Differential Revision: D5192311
fbshipit-source-id: 334ea6e
Summary:
This is a minimal change to (poorly) recognize and model std::mutex
lock and unlock methods, and to surface all thread safety issues for
C++ based on the computed summaries with no filtering.
This ignores much of the Java analysis, including everything about the
Threads domain. The S/N is comically low at this point.
Reviewed By: sblackshear
Differential Revision: D5120485
fbshipit-source-id: 0f08caa
Summary:
Introduce `infer-<command>` for each command, except for internal commands
(only `infer-clang` for now) which are not exported. Install these executables
(which are just symlinks to `infer`) on `make install`. The main executable
looks at the name it was invoked with to figure out if it should behave as a
particular command.
Get rid of `InferClang`, `InferAnalyze`, and `InferPrint`. As a bonus, we now
only need to build one executable: `infer`, which should be a few seconds
faster (less link time).
`InferAnalyze` is now `infer-analyze` and `InferPrint` is `infer-print`. To run
`InferClang`, use a symlink named `clang`, `clang++`, etc. to `infer`. There
are such symlinks available in "infer/lib/wrappers/" already.
I also noticed that the scripts in xcodebuild_wrappers/ don't seem useful
anymore, so use wrappers/ instead, as for `make`.
Reviewed By: mbouaziz
Differential Revision: D5036495
fbshipit-source-id: 4a90030
Summary: Sometimes reports need traces to be fully understood, but sometimes reporting where the exception takes place can save time to developers.
Reviewed By: jvillard
Differential Revision: D4914037
fbshipit-source-id: 039ab63
Summary:
There are 2 possible places where to find toplevel.cmo:
- infer/_build/infer/toplevel.cmo, built by `make toplevel`
- infer/_build/test/infer/toplevel.cmo, built by `make test`
The test loading and running toplevel.cmo would give priority to the non-test
version of toplevel.cmo, which might be stale because `make test` doesn't
rebuild it.
This diff makes sure we use the test version of toplevel.cmo in `make test`.
Reviewed By: akotulski
Differential Revision: D4960550
fbshipit-source-id: 9c817ba
Summary: This will be needed to generate man pages with an accurate date.
Reviewed By: jberdine, mbouaziz
Differential Revision: D4937498
fbshipit-source-id: b5ebd31
Summary:
As an interprocedural checker, SIOF should not run unless explicitly required.
Make it a new type of analyzer like other similar checkers.
Reviewed By: mbouaziz
Differential Revision: D4937820
fbshipit-source-id: a9e2d38
Summary: `bufferoverrun` is not part of `checkers` yet so bufferoverrun tests require their own Makefile
Reviewed By: jvillard
Differential Revision: D4937558
fbshipit-source-id: 20380f1
Summary:
Try to read .inferconfig in the current directory, then in .., then in ../..,
etc. This can be overriden with the `INFERCONFIG` environment variable.
This removes the need for two-phase parsing, so clean up that code too.
Paths in .inferconfig are interpreted relative to where .inferconfig is located.
This does not apply to other path-sensitive things like regexpes... this is not
a show stopper because regexpes can account for the fact that infer may be
called from different project roots.
Make sure we fail when .inferconfig exists but cannot be read.
Reviewed By: jberdine
Differential Revision: D4843142
fbshipit-source-id: 340a63f
Summary:
We shouldn't encourage contributors to run only a subset of the tests with
`make test`, but it's more helpful to complain clearly should they try to do
so.
Now `make test` will ensure that both the clang and the Java analyzers are
enabled, otherwise it will refuse to run and suggest to run `make config_tests`
instead.
It can still be useful to be able to run only the Java or Clang tests. The diff
also fixes the cases where that previously failed.
Fixes#634
Reviewed By: jberdine
Differential Revision: D4826515
fbshipit-source-id: 4b76029
Summary:
Nuking the specs then building the models was not a great idea: the models do
not look at specs but only at some dummy marker files, eg
infer/lib/specs/c_models, so they don't necessarily realize that they need to
be rebuilt when the specs have been nuked!
One easy workaround would be to also delete the marker files, but then we would
*always* rebuild the models when building infer. Not good.
The solution here is to nuke the specs and marker files only when the clang
dependencies change, then rebuild all the models.
Reviewed By: jberdine, jeremydubreil
Differential Revision: D4781424
fbshipit-source-id: 2d2606e
Summary:
We shouldn't install byte-executables unless the user types `make byte`.
Othewise everything gets very slow and the cause is not obvious.
Reviewed By: jeremydubreil
Differential Revision: D4779135
fbshipit-source-id: 757bfa2
Summary:
It can be useful when debugging infer or the Makefiles themselves to see what
`make` is doing. Instead of editing Makefiles to remove `@` now you can `make
VERBOSE=1`.
This is just `git ls-files | grep -e Makefile -e '.*\.make' | xargs sed -e 's/^\t@/\t$(QUIET)/' -i`, and adding the definition of `QUIET` to Makefile.config.
Reviewed By: jeremydubreil
Differential Revision: D4779115
fbshipit-source-id: e6e4642
Summary:
Most of our tests work by comparing the output of some command to a baseline
expected output. It's often needed to update that baseline.
Previously, that was crudely done by attempting to move every foo.exp.test file
to foo.exp. This does not work terribly well, in particular because
foo.exp.test might be stale.
Instead, add a `replace` target to every test that knows how to update the
baseline. This allows custom behaviours too, eg in the differential tests.
Most of the tests include base.make or differential.make, so add a replace target
there. A few tests are completely custom, add a replace target to them too.
Reviewed By: jeremydubreil
Differential Revision: D4754279
fbshipit-source-id: ec34273
Summary:
Sometimes building the models segfaults mysteriously. I've observed in the
latest instance that `make -C infer/models clean` solved the issue so this is
likely a good idea.
Reviewed By: mbouaziz
Differential Revision: D4745924
fbshipit-source-id: cdb50d8
Summary:
This removes the dependency of libffi, and gives better guarantees at
compile-time. A bit overkill to retrieve the width of the terminal but what do
you know...
Reviewed By: jberdine
Differential Revision: D4700003
fbshipit-source-id: 036989b
Summary:
Two issues in the previous code:
1. We'd just tack on `#compilation-database` so if there already are `#flavors`
in a Buck target then we get a malformed `#flavors#compilation-database`
instead of `#flavors,compilation-database`. There's already code to deal
with that for other flavors (`#infer-capture-all`), so extend and reuse that.
2. The code didn't work if there are spaces in target names (sigh). Harden the
code a bit so that it works. Unfortunately Buck doesn't escape spaces at all
in its `buck targets --show-output` output, so the parsing still relies on
there being no spaces in flavors to work correctly. Sample output:
```
$ buck targets --show-full-output //clang_compilation_database:Hel\ lo#compilation-database
Not using buckd because watchman isn't installed.
[+] PROCESSING BUCK FILES...0.1s [100%] 🐳 New buck daemon
//clang_compilation_database:Hel lo#compilation-database /home/jul/infer/infer/tests/build_systems/codetoanalyze/buck-out/gen/clang_compilation_database/__Hel lo#compilation-database.json
```
Also, some code in codetoanalyze/clang-compilation-database/ didn't compile, so
fix that.
Reviewed By: dulmarod
Differential Revision: D4714338
fbshipit-source-id: b8ae324
Summary:
This makes sure that one can run `./build-infer.sh` then `make`. Otherwise it's
not always clear what one should do to recompile infer, eg when `make` will
work and when `./build-infer.sh` should be used instead, in particular when the
user doesn't have opam configured for her terminal.
Reviewed By: jberdine
Differential Revision: D4698159
fbshipit-source-id: 5df8059
Summary:
There is no point in attaching plugin to `clang -E` calls since they don't produce any AST.
Same applies to `-M`, `-MM`, etc. (but not `-MD` nor `-MMD`).
Reviewed By: jvillard
Differential Revision: D4681618
fbshipit-source-id: 7a76add
Summary: We had our own mechanism for saving `./configure` flags, which does not work if flags contain whitespace. It turns out that `./config.stats --config` is a thing though.
Reviewed By: mbouaziz
Differential Revision: D4642422
fbshipit-source-id: ae075ca
Summary:
Given two analysis results, it's now possible to compare them with the following command:
infer --diff --report-current reportA.json --report-previous reportB.json --file-renamings file_renamings.json
this command will then generate 3 files in `infer-out/differential/{introduced, fixed, preexisting}.json`, whose meaning is the following:
- `introduced.json` has all issues in `current` that are not in `previous`
- `fixed.json` has all issues in `previous` that are not in `current`
- `preexisting.json` has all issues that are in both `current` and `previous`
The json files generated can then be used to categorise results coming from incremental analyses of a codebase.
Reviewed By: jvillard
Differential Revision: D4482517
fbshipit-source-id: 1f7df3e
Summary: Not priting it results in staring terminal not priting anything for a long time. Just to realize `make` just started to build clang.
Reviewed By: martinoluca, jvillard
Differential Revision: D4642791
fbshipit-source-id: 177a228
Summary:
This is useful to make sure `./build-infer.sh clang` and `./build-infer.sh
java` work correctly.
I had to move one unit test inside a new unit/clang/ directory used only if clang is enabled, and create a stub for it in unit/clang_stubs/ for when clang is disabled.
Renamed `OCAML_SOURCES` to the more consistent and descriptive `OCAML_CONFIG_SOURCES`.
Reviewed By: jeremydubreil
Differential Revision: D4571997
fbshipit-source-id: 9502114
Summary:
It's annoying that changing the Makefile or running `./configure` with
different options does not always retrigger compilation of some targets, eg
infer.
Reviewed By: akotulski
Differential Revision: D4571988
fbshipit-source-id: dda080f
Summary:
Xcode's compilation databases follows a different convention than cmake's and
escape the `"file"` and `"dir"` fields of each unit to make them shell-ready.
We need to treat them differently when reading them.
This adds a new `--clang-compilation-db-files-escaped` option and makes the
code related to reading compilation databases deal correctly with both
conventions.
Reviewed By: akotulski
Differential Revision: D4559239
fbshipit-source-id: 51120ae
Summary: Some compilation databases give relatives paths for the `"file"` field. This is not ambiguous as there is also a `"dir"` field, so use that to make the path absolute when needed.
Reviewed By: dulmarod
Differential Revision: D4559145
fbshipit-source-id: be36a16
Summary: One step on the way to being able to test java/clang separately.
Reviewed By: akotulski
Differential Revision: D4558948
fbshipit-source-id: c0c7556
Summary:
This replaces the previous integration written in Python, which consisted in 1)
run the mvn command and parse its output to locate "directories containing
source files", 2) run on files named "*.java" in these directories. This meant
we had to run javac twice on each source file, and more importantly this
mechanism of finding source files was very fragile. In fact, I could not make
it work on several mvn projects I tried.
The new integration is based on parsing "pom.xml" to add an "infer-capture"
profile which instructs mvn to run `/path/to/infer` instead of `javac`. We also
add this profile to each maven submodule.
Users can specify an "infer-capture" profile themselves if the default one
doesn't work; in that case we don't inject our own "infer-capture" profile.
Reviewed By: jeremydubreil
Differential Revision: D4409613
fbshipit-source-id: d664274
Summary:
`make byte` will populate infer/bin/ with bytecode version of each executable,
plus infer/bin/infer.byte (used to remember which of the native or byte
executables have been built most recently). `make infer` now also creates
infer/bin/infer.native, so that we're sure to replace the executables with
native/byte versions as appropriate.
This is to make debugging a tad easier:
make byte
ledit ocamldebug $(which infer) <infer args>
Whereas previously one had to:
make -C infer/src byte
ledit ocamldebug infer/_build/infer/backend/infer.byte <infer args>
Reviewed By: jberdine
Differential Revision: D4409476
fbshipit-source-id: ab5f57d
Summary:
This diff allows to use the linters written in DSL to check for bugs.
Now new checkers can be written directly in the DSL.
The diff also remove some weirdness and simplify the CTL semantics.
For example no need to unwrap a node when evaluating the IN operator.
Also no need to distinguish anymore between stmt and decl in the
semantics of EX and EF.
Moreover, the diff de-couple hard-coded checkers (eg checks on component kit)
from those checkers parsed in the .al files.
Reviewed By: martinoluca
Differential Revision: D4375207
fbshipit-source-id: 9ac2d47
Summary:
Without this it's not always obvious which test fails. It also makes it easier
to mass-patch test failures from the CI jobs to replace expected outputs with
actual outputs (eg, when debugging osx frontend tests from linux).
Reviewed By: jberdine
Differential Revision: D4352205
fbshipit-source-id: 8887d7b
Summary:
Seems like we cannot run 2 instances of Buck in parallel even when one uses
buck-out/ and the other buck-out/foo/.
Reviewed By: sblackshear
Differential Revision: D4347090
fbshipit-source-id: 7e65d2f
Summary:
This will simplify the InferPrint logic of checking what should/should-not be reported.
I will remove the issue names in Localise in a next diff.
Reviewed By: ddino
Differential Revision: D4334327
fbshipit-source-id: ebcfd6c
Summary:
Most of the diff adds a way to run an existing test with different infer
options.
Also, do not run the Python script when capturing "analyze".
fixes https://github.com/facebook/infer/issues/518
Reviewed By: jberdine
Differential Revision: D4333762
fbshipit-source-id: 642acff
Summary: This makes it a bit easier to read the results of `make test`.
Reviewed By: jberdine
Differential Revision: D4326467
fbshipit-source-id: dd69114
Summary:
Turns out that swapping stdout and stderr using a temporary fd 3 was screwing
up with make's jobserver, who also uses fd 3!
Also, infer is partly to blame as it also calls `make`. Unsetting `MAKEFLAGS`
in infer tells `make` that the way infer calls `make` is independent from
parent `make` invocations.
Also, simplify the rules for direct tests and build system tests.
Reviewed By: jberdine
Differential Revision: D4328979
fbshipit-source-id: 96818e8
Summary: The command `make -k run-test` does not stop in case of failure but still exits with an error code, preventing the actual the .exp file to be replaced by the new analysis results.
Reviewed By: jvillard
Differential Revision: D4326028
fbshipit-source-id: 3354458
Summary:
Run `make -k run-tests` before replacing .exp files with their .exp.test
counterparts. This is the intuitive behaviour according to popular opinions.
Reviewed By: akotulski
Differential Revision: D4320050
fbshipit-source-id: 7c93a76
Summary: we no longer use buck for tests and the script has no reason to exist
Reviewed By: dulmarod
Differential Revision: D4212713
fbshipit-source-id: 7bd1cca
Summary: `make install` was creating symbolic link with absolute path of `InferClang`. Fix it
Reviewed By: martinoluca, dulmarod
Differential Revision: D4220519
fbshipit-source-id: 166714b
Summary: clang has very complicated logic what to translate based on `project_root` and filename. Add tests for different situations in regard of symbolic links in path/project_root
Reviewed By: jvillard
Differential Revision: D4168551
fbshipit-source-id: 586b364
Summary: Only run the buck (and ant) tests if the tools can be found at ./configure-time.
Reviewed By: jberdine
Differential Revision: D4167586
fbshipit-source-id: e77b736
Summary:
- rename java.make -> javac.make, config.make -> java.make, and move to infer/tests/ so it's easier to use from infer/tests/build_systems/
- use these from ant's test Makefile, much code reuse!
- factor out common functionality between java and clang
A wrinkle: sorting is now done the same way for --issues-tests and
--issues-txt, which produces bogus (but still as deterministic) sorting for
--issues-txt. This is more of a cosmetic issue, but I hope to fix it in a later
diff that gets rid of calls to `sort` in favour of sorting directly from
`InferPrint`.
Reviewed By: jberdine
Differential Revision: D4166841
fbshipit-source-id: ed6f232
Summary: The thread safety checker is run independently of other analyses, using the command "infer -a threadsafety -- <build-command>".
Reviewed By: sblackshear
Differential Revision: D4148553
fbshipit-source-id: bc7b3f9
Summary:
This diff ports checkCopyright to Core, builds it separately from other
executables, and fixes the build command in the linter.
Reviewed By: jvillard
Differential Revision: D4148217
fbshipit-source-id: 8aefc98
Summary: `make test` was always exiting with exit code 0, even in the case of test failures. This is definitely not what we want.
Reviewed By: sblackshear
Differential Revision: D4154912
fbshipit-source-id: 87b4b2b
Summary: This message should be printed in the standard output once all the tests have passed which makes it easier to dinstinguish the cases when some tests fail from the cases where all tests are fine.
Reviewed By: sblackshear
Differential Revision: D4150022
fbshipit-source-id: 6be39c2
Summary:
The way to replace failing tests coming from build_integration_tests.py was not
very discoverable, and now that we have a target for precisely this action we
might as well use it.
The downside is that it's much slower to run `make test-replace` that way but
that's all the more incentive to migrate tests from python to Makefiles. Anyway
most of that time is spend in the buck test, which is going to be migrated
soon.
Reviewed By: jeremydubreil
Differential Revision: D4147121
fbshipit-source-id: 53689e9
Summary:
This makes our python code work (instead of crashing) when the source file
should be found not from the current directory (or absolute path), eg with
`infer --project-root .. -- clang -c hello.c`.
Reviewed By: jeremydubreil
Differential Revision: D4130802
fbshipit-source-id: 001f72d
Summary:
Time for this framework to die. This converts the ant test to a Makefile. The
design is that each test or family of tests will have its own directory.
Do the necessary plumbing from the toplevel Makefile so that `make test` runs
the migrated tests.
Reviewed By: jberdine
Differential Revision: D4106298
fbshipit-source-id: 7bd694d
Summary:
InferClang knows what to do if its name ends in ++. So it is not
necessary to pass whether or not the original clang executable ended in
++ to InferClang using the the INFER_XX environment variable. Instead,
create an InferClang++ symbolic link and make the clang wrapper call
either InferClang or InferClang++ as needed.
Reviewed By: jvillard
Differential Revision: D4078416
fbshipit-source-id: 3b5d5d0
Summary:
This diff revises the makefiles for java tests so that they are based on
the files actually produced and depended on, instead of the existing
imperative style. This is, I think, clearer and easier to modify, and
enables a little more parallelism.
Reviewed By: jvillard
Differential Revision: D4072560
fbshipit-source-id: c16d4bd
Summary: The direct test rules are very regular and depend only on their names, except for some *_infer_* rules that I had to change to *_errors_*. We can rename these directories if we want (to make them like Java).
Reviewed By: jeremydubreil
Differential Revision: D4081111
fbshipit-source-id: 3295acb
Summary: Doing like this makes it easier to keep the phony declarations in sync with the target definitions
Reviewed By: jvillard
Differential Revision: D4067679
fbshipit-source-id: 723bc0e
Summary:
In several places the tests were using whatever 'infer' executable was
found in PATH, instead of the one build from the source to be tested.
Reviewed By: jeremydubreil
Differential Revision: D4065019
fbshipit-source-id: 9b65099
Summary:
This also adds `-a compile` support to `InferClang`. This is needed for the
`xcodebuild` integration, which is hard to fold into the same binary as the
rest.
Reviewed By: jberdine
Differential Revision: D4008262
fbshipit-source-id: 0bbd53f
Summary: The dependency to the main Infer build was missing when building the toplvel. Running the the build with the option `-j` was failing then.
Reviewed By: akotulski
Differential Revision: D4034540
fbshipit-source-id: d6e2e2e
Summary:
There's no reason for infer to be in lib/ anymore, move it to the same place as
the other binaries. Thus all binaries are in the same directory and Config.ml
can better know where things are.
Reviewed By: jberdine
Differential Revision: D4015958
fbshipit-source-id: c5e851f
Summary:
This diff adds a make target to generate interface files from
implementation files. These generated interface files can then be used
as a starting point for documenting and restricting the exposed module
interface. For example, to generate an interface for JavaTaintAnalysis.ml,
execute:
```
make -C infer/src M=quandary/JavaTaintAnalysis mli
```
Note that this relies on `ocamlc -i`, which for reason currently
produces syntactically ill-formed files.
Reviewed By: sblackshear, jvillard
Differential Revision: D3998175
fbshipit-source-id: f653737
Summary:
Color modules in dependency graph based on directory, and cluster
modules together into a subgraph if their directory is listed in the
`clusters` variable of infer/src/Makefile.
Reviewed By: akotulski
Differential Revision: D3979253
fbshipit-source-id: dffd76b
Summary:
Convert the last remaining tests to the new direct format: java harness and crashcontext.
Remove what is left of the old testing infrastructure.
Reviewed By: sblackshear
Differential Revision: D3886355
fbshipit-source-id: 5117868
Summary:
This diff converts the Eradicate and Checkers tests to the new direct test format, which does not rely on buck or junit.
A self-contained Makefile is used to compile and analyze the test files, including all the dependencies, and a special option in InferPrint is used to produce a file of expected results `issues.exp`, which is checked into the repository.
Having an explicit Makefile makes it easy to edit and compile one set of test files in isolation, to investigate test failures, do debugging, etc.
A bunch of boilerplate code is removed. For example, the single file of expected results `issues.exp` replaces the 1.5K LOC in `endtoend/java/eradicate`.
Reviewed By: jvillard
Differential Revision: D3764632
fbshipit-source-id: 6c68ab8
Summary: Make it possible to run infer code from within `ocaml`/`utop`. Integration is really basic, but we can extend it if we find it useful.
Reviewed By: jberdine
Differential Revision: D3736029
fbshipit-source-id: 4cebb7c
Summary:
`make` calls `buck`, who calls `make` again. The last `make` then gets confused
because it wasn't called with `make -n` as it should be for a call of
make-within-make. Unsetting this variable makes `make` forget that it's running
inside `make`.
The main benefit is that it removes annoying spamming of stderr by make/buck
about disabling jobserver.
The submake should also be more parallel now, although if other things are
running in parallel they will ignore any limit set with `-j`.
Reviewed By: jberdine
Differential Revision: D3522156
fbshipit-source-id: 45db21a
Summary:
This diff changes the toplevel 'infer' executable from the current
python script to an OCaml binary. Currently this executable only parses
command line arguments, sets up environment variables, and invokes the
existing python script. This improves infer's command-line and
configuration interface, since passing arguments to the frontends or
backend no longer requires manually setting environment variables, and
arguments for the toplevel can now also be specified in .inferconfig.
Simplification and migration of functionality from the python script is
left for the future.
Reviewed By: martinoluca, jvillard
Differential Revision: D3450662
fbshipit-source-id: 1b52302
Summary:
C++ header models were not included into make install target. This made
C++ analysis much worse for infer released using 'make install'
Reviewed By: jvillard
Differential Revision: D3463000
fbshipit-source-id: 6f7e9b9
Summary:
The conf-clean target removes Makefiles that used to be generated by
autoconf, but aren't any longer.
Differential Revision: D3431186
fbshipit-source-id: c453f00
Jules Villard
Sam Blackshear
Nikos Gorogiannis
Mehdi Bouaziz
Jeremy Dubreil
Josh Berdine
Dino Distefano
Dulma Churchill
Jia Chen
Martino Luca
Andrzej Kotulski
Kihong Heo
Peter O'Hearn
Yunxing Dai
Cristiano Calcagno