Summary:
Saw these two types of errors before (but they're hard to reproduce locally) when building the models:
- `ERROR: Zip.Error("/mnt/btrfs/trunk-git-infer-739-1503054473/infer/bin/../lib/java/models.jar", "", "end of central directory not found, not a ZIP file")`. I think this means infer reads a partially-written models jar. We shouldn't try to load this in models mode.
- `install` would complain that the destination already exists. I think this can only happen if there's a race and the file gets created between when install first checks and when it tries to write to it.
This made me realise that the some of the models are computed in C and C++ mode
and we pick one computed spec arbitrarily. That sounds a bit dodgy but at least
now we do so in a non-racy way.
Reviewed By: jeremydubreil
Differential Revision: D5658389
fbshipit-source-id: 8077279
Summary: Also, stop trying to delete directories that do not exist: "sources" and "filelists".
Reviewed By: mbouaziz
Differential Revision: D5658089
fbshipit-source-id: e1cdb13
Summary:
Instead of a whitelist and blacklist and default issue types and default
blacklist and filtering, consider a simpler semantics where
1. checkers can be individually turned on or off on the command line
2. most checkers are on by default
3. `--no-filtering` turns all issue types on, but they can then be turned off again by further arguments
This provides a more flexible CLI and is similar to other options in the infer
CLI, where "global" behaviour is generally avoided.
Dynamically created checkers (eg, AL linters) cause some complications in the
implementation but I think the semantics is still clear.
Also change the name of the option to mention "issue types" instead of
"checks", since the latter can be confused with "checkers".
Reviewed By: jberdine
Differential Revision: D5583238
fbshipit-source-id: 21de476
Summary:
This gives additional information to users. For instance:
```
--biabduction
Activates: the separation logic based bi-abduction analysis using
the checkers framework (Conversely: --no-biabduction)
This option is relevant to infer-analyze(1).
```
Reviewed By: sblackshear
Differential Revision: D5583197
fbshipit-source-id: 2960b90
Summary:
This makes it easier to test a single checker.
Also refactor the code to make it harder to mess up the list of default/all checkers.
Reviewed By: sblackshear
Differential Revision: D5583209
fbshipit-source-id: 7c919b2
Summary:
This makes the CLI more complete: before, it was often impossible to "go back"
once some options were passed, namely options produced by `mk_*_{list,opt}`.
Now these automatically create an accompanying `--<long>-reset` option that
resets the config variable to its default value.
Also unify our naming of `~meta` arguments:
- no more spaces in them (except one instance where it's a whole sentence)
- use `+foo` if `foo` can be specified multiple times
Reviewed By: dulmarod
Differential Revision: D5583187
fbshipit-source-id: a8c2567
Summary:
Replace `inferTraceBugs` with `infer-explore` with a similar CLI. Some options changed:
- --max-level -> --max-nesting, and "max" is the default value instead of a possible value
- --no-source -> --no-source-preview
Reviewed By: mbouaziz
Differential Revision: D5526651
fbshipit-source-id: 8383f37
Summary:
This allows the user to specify a different command to build the current version of the project vs to build the previous version of the project. Here's an example:
```
infer diff --gen-previous-build-command-script "echo clang -c hello.c" ... -- clang -c hello2.c
```
By default the two build commands are the same. If the script is to be used for both the current and the previous versions, then it's on the user to run it once first, eg:
```
infer diff --gen-previous-build-command-script "./myscript.sh previous" ... -- $(./myscript.sh current)
```
Reviewed By: martinoluca
Differential Revision: D5500989
fbshipit-source-id: 7374b44
Summary:
The Eradicate `Nullable` checker should now be run using:
infer -a checkers --eradicate ...
Reviewed By: mbouaziz
Differential Revision: D5529226
fbshipit-source-id: 0de2956
Summary: D5526683 assumed that this was the case, and broke the report hook.
Reviewed By: martinoluca
Differential Revision: D5527353
fbshipit-source-id: 2e169b9
Summary: Automatically set --reactive because the diff analysis will be faster that way.
Reviewed By: jeremydubreil
Differential Revision: D5499443
fbshipit-source-id: 1a35cee
Summary: The `--failures-allowed` was doing for the Clang frontend what `--keep-doing` was doing for the backend. This revision merges the two options to simplify the Infer CLI and our tests.
Reviewed By: jvillard
Differential Revision: D5474347
fbshipit-source-id: 09bcea4
Summary: Using a dedicated abstract domain, like Quandary does, is more suitable for taint analysis.
Reviewed By: sblackshear
Differential Revision: D5473794
fbshipit-source-id: c917417
Summary: Because making a diff which breaks the tests because it silently fails to create the right posts for the models is notoriously hard to debug
Reviewed By: sblackshear
Differential Revision: D5471611
fbshipit-source-id: ef04539
Summary:
Pretty basic: warn when we see an assignment instruction `x = ...` and `x` is not live in the post of the instruction.
Only enabled for Clang at the moment because linters already warn on this for Java. But we can enable it later if we want to (should be fully generic).
Reviewed By: jeremydubreil
Differential Revision: D5450439
fbshipit-source-id: 693514c
Summary:
oops_house
This was introduced when upgrading Core, as `Unix.readdir` became deprecated in
favour of `Unix.readdir_opt` but the exception-based logic was left unchanged.
Reviewed By: mbouaziz
Differential Revision: D5461353
fbshipit-source-id: c0994c3
Summary: Long lists will be broken into lines rather than stretch out on a single line.
Reviewed By: jvillard
Differential Revision: D5424323
fbshipit-source-id: 115f16a
Summary: Changing .inferconfig can change the analysis results, so we want the cache to get invalidated when .inferconfig changes.
Reviewed By: jeremydubreil
Differential Revision: D5419734
fbshipit-source-id: 01ad874
Summary: This way, if infer crashes then the log files will be available for inspection, but if it succeeds we still get rid of the (non-deterministic) logs before Buck caches the results.
Reviewed By: sblackshear
Differential Revision: D5416032
fbshipit-source-id: 22e58be
Summary:
First steps towards implementing diff analysis functionalities inside infer
itself. What works: run infer, checkout parent, re-run infer, checkout top
revision, compute the reportdiff (but no final surfacing on the console). Lots
of TODO still, inlined in the code.
Reviewed By: jberdine
Differential Revision: D5364226
fbshipit-source-id: 5b7f9a5
Summary:
Allowing the user to configure where to store the JSON report is asking for
trouble. In fact, some places in the code hardcoded "results.json" anyway.
Someone wanting to have results.json somewhere else can still copy report.json
once infer has run.
Reviewed By: jberdine
Differential Revision: D5415079
fbshipit-source-id: 9439cb6
Summary: Introduce `Logging.die` to try and exit with consistent error codes depending on what failed.
Reviewed By: mbouaziz
Differential Revision: D5406642
fbshipit-source-id: 25d98fc
Summary: This will allow us to gradually get rid of the exceptions thrown during the analysis while detecting the regressions earlier
Reviewed By: jberdine, jvillard
Differential Revision: D5385154
fbshipit-source-id: 605e3f5
Summary:
Conversion and reformat of infer source using ocamlformat
auto-formatting tool.
Current status:
- Because Reason does not handle docstrings, the output of the
conversion is not 'Warning 50'-clean, meaning that there are
docstrings with ambiguous placement. I'll need to manually fix
them just before landing.
Reviewed By: jvillard
Differential Revision: D5225546
fbshipit-source-id: 3bd2786
Summary:
Error'ing when getting several subcommands on the command line only makes sense
when the user typed these options herself. This was causing a bug in the Gradle
integration, and possibly others (see GitHub issue).
Fixes#668
Reviewed By: jeremydubreil
Differential Revision: D5381850
fbshipit-source-id: f332377
Summary: This is useful when we only want to examine the html for some but not all source files.
Reviewed By: jvillard
Differential Revision: D5334786
fbshipit-source-id: 774c23c
Summary: Rename historical option to its new form, since the old form was no longer accepted by infer.
Reviewed By: martinoluca
Differential Revision: D5329033
fbshipit-source-id: 4fa9402
Summary:
This will be needed to re-use the functions now in Driver.ml in other contexts
without always adding to infer.ml. For instance, this is used in a later diff
to do a diff-analysis orchestrator that needs to run the capture and analysis
several times.
Reviewed By: jberdine
Differential Revision: D5319862
fbshipit-source-id: caf9551
Summary: Needed in a later diff to be able to compute the set of changed files *during* an infer execution.
Reviewed By: jberdine
Differential Revision: D5319667
fbshipit-source-id: 226ec91
Summary: This change introduces the a new argument that lets you restrict the results of a differential report to only certain files.
Reviewed By: mbouaziz
Differential Revision: D5236626
fbshipit-source-id: 52711e9
Summary:
This avoids race conditions when two processes or more try to lock a file for
writing. It could be that the process losing the race writes less than the
winner, then we get rubbish at the end of the file. Calling `ftruncate(2)` inside the critical section makes sure the
contents of the file are erased first. The harmful race was observed in
xcodebuild sometimes, as it can call infer on the same file several times in
parallel (!).
Reviewed By: jberdine
Differential Revision: D5209177
fbshipit-source-id: 744169c
Summary:
When dealing with differential's comparisons, SourceFiles in reports may not exist anymore because of changes between revisions.
Disable user visible warnings if desired, through the labeled argument `~warn_on_error`
Reviewed By: jvillard
Differential Revision: D5217097
fbshipit-source-id: 7a1542b
Summary: Can be useful in case there is compilation issue with the models
Reviewed By: jvillard
Differential Revision: D5208830
fbshipit-source-id: f31d84f
Summary:
Mutating a formatter in `dup_formatter` had unintended consequences of
double-printing newlines. There's actually no need not to make a fresh copy,
which avoids this problem.
Also, don't warn if the log file is still a buffer at the end of execution as
that can happen in some error cases, eg:
```
$ infer -- build
ERROR: (Invalid_argument "Unsupported build command build")
```
Reviewed By: jberdine
Differential Revision: D5199664
fbshipit-source-id: fc2731d
Summary: Also makes the error message for clang commands a bit nicer to the eye.
Reviewed By: mbouaziz
Differential Revision: D5191664
fbshipit-source-id: 801ec72
Summary:
Change the API of `Logging` wrt to writing to files and to the console (see
changes in logging.mli).
Write only to one log file: infer-out/log. Prefix each line with the kind of
warning and the PID of the process emitting it. Writing with `O_APPEND` is
atomic so the file should not get garbled by concurrent writes. To get the
output of a single process, find out which one interests you by looking at
infer-out/log, then `grep ^[<PID>] infer-out/log`.
Introduce 3 log levels for debug output and command-line options to set them
for various categories individually.
Change tons of `"\n"` to `"@\n"` so the `Format` module is aware of newlines
without us having to look through every character of every logged string for
`\n` characters.
Reviewed By: mbouaziz
Differential Revision: D5165317
fbshipit-source-id: 93c922f
Summary:
This makes it clearer that something went wrong. Most `failwith` did not set
this prefix already, so I opted to append it automatically and remove it from
the few instances that added it manually.
Also add quotes around bad user arguments to lessen possible confusion.
Reviewed By: jberdine
Differential Revision: D5182272
fbshipit-source-id: 20e4769
Summary:
Introduce a new option `--no-report` (conversely `--report`) to stop reporting
after the analysis. This is useful to call sub-`infer-analyze` processes with
as they shouldn't compute "result.json" or report bugs to stderr. This bug
would only manifest itself when per-procedure parallelism disabled, which
explains why it was noticed only on Java.
Reviewed By: jberdine
Differential Revision: D5182110
fbshipit-source-id: a892470
Summary: This also allows us to better test that the new commands will keep working.
Reviewed By: jeremydubreil
Differential Revision: D5172891
fbshipit-source-id: 169bd6f
Summary: Needed higher-up the stack, useful to have in the API in general.
Reviewed By: jberdine
Differential Revision: D5165153
fbshipit-source-id: 714aeea
Summary:
This will be needed higher up in the stack because the new `ProcessPool` module
will need to call into `Logging` to refresh the logging formatters to get the
right PID when writing to the log file.
+remove dead code `iter_parallel`
Reviewed By: jberdine
Differential Revision: D5165130
fbshipit-source-id: 95c949b
Summary:
I should not push debugging code to the repo
I should not push debugging code to the repo
I should not push debugging code to the repo
I should not push debugging code to the repo
I should not push debugging code to the repo
I should not push debugging code to the repo
...
Reviewed By: martinoluca
Differential Revision: D5164723
fbshipit-source-id: d944916
Summary:
For now, we just support clearing the taint on a return value.
Ideally, we would associate a kind with the sanitizer and only clear taint that matches that kind.
However, it's fairly complicated to make that work properly with footprint sources.
I have some ideas about how to do it with passthroughs instead, but let's just do the simple thing for now.
Reviewed By: jeremydubreil
Differential Revision: D5141906
fbshipit-source-id: a5b8b5e
Summary:
All files that match the regular-expressions passed via `--skip-analysis-in-path` are just compiled.
With this change, you can also opt for not compiling those files at all, by passing the `--skip-analysis-in-path-skips-compilation` argument
Reviewed By: mbouaziz
Differential Revision: D5121583
fbshipit-source-id: 4e1325a
Summary:
A recent diff tried to replace `L.out "error message"; assert false` with
`failwith "error message"` but infer relies on the type of raised exceptions to
sometimes keep going. A more careful change will be needed but in the meantime
restore the old behaviour.
Reviewed By: jberdine
Differential Revision: D5112969
fbshipit-source-id: 713fe20
Summary: There were some leftover uses of the `Tracing` analyzer option. While I was at it, I also rename the `Config` option name.
Reviewed By: jberdine
Differential Revision: D5113489
fbshipit-source-id: 68d5cc8
Summary:
Try and enforce the following rules:
- stderr is for updating the user about progress or errors
- Introduce Logging.progress that outputs to stderr, but honours --quiet
- Logging.stderr is as before
- Logging.out now prints to stderr (or to log files as before if set up) and
not stdout. If some information should go on stdout then the user should be
able to rely on it (ie, it's not just some progress message). For now only
the summary of the errors is printed on stdout by default.
- Logging.err* functions are gone. If the error is user-visible, it should be
Logging.stderr, or `failwith`. If not, go to the same log file as other
output, which personally I find much more convenient than having to dig through
2 log files every time I'm looking for some output.
Reviewed By: jberdine
Differential Revision: D5095720
fbshipit-source-id: 68999c9
Summary: There was no option to trigger this checker so it was not possible to enable it when not running the default list of checkers
Reviewed By: jberdine
Differential Revision: D5057088
fbshipit-source-id: 7af36f5
Summary: The issues that are not reported by default are all experimental issues from the biabduction analysis. In that case, it is easier to use a blacklist of errors to filter out so that the issues found by the checkers based on the AI framework can be reported by default without having to add them to the whitelist.
Reviewed By: jvillard
Differential Revision: D5051327
fbshipit-source-id: 2a93b11
Summary:
Introduce `infer-<command>` for each command, except for internal commands
(only `infer-clang` for now) which are not exported. Install these executables
(which are just symlinks to `infer`) on `make install`. The main executable
looks at the name it was invoked with to figure out if it should behave as a
particular command.
Get rid of `InferClang`, `InferAnalyze`, and `InferPrint`. As a bonus, we now
only need to build one executable: `infer`, which should be a few seconds
faster (less link time).
`InferAnalyze` is now `infer-analyze` and `InferPrint` is `infer-print`. To run
`InferClang`, use a symlink named `clang`, `clang++`, etc. to `infer`. There
are such symlinks available in "infer/lib/wrappers/" already.
I also noticed that the scripts in xcodebuild_wrappers/ don't seem useful
anymore, so use wrappers/ instead, as for `make`.
Reviewed By: mbouaziz
Differential Revision: D5036495
fbshipit-source-id: 4a90030
Summary:
`infer clang ...` was not being handled correctly and would error incorrectly.
Small fix before bigger change.
Reviewed By: mbouaziz
Differential Revision: D5036447
fbshipit-source-id: 87fcc40
Summary: It's useful to have all the options in a single man page for discoverability.
Reviewed By: mbouaziz
Differential Revision: D5028007
fbshipit-source-id: 37f8d2e
Summary:
Allows one to dump the manuals in various formats:
- plain: plain text
- pager: default, display the man page in all its glory inside a pager
- groff: the source code of the man page
- auto: pager or plain
Reviewed By: mbouaziz
Differential Revision: D5027636
fbshipit-source-id: 3b97edc
Summary:
Glorious, glorious man pages.
This changes a bunch of things that were hard to break up from the diff, so the
resulting diff is big.
Use `Cmdliner.Manpage` to format man pages (and also format a bit ourselves
since it is so stubborn).
As a bonus, introduce the following subcommands:
```
infer run ... # same as default mode with -- before
infer capture ... # -a capture
infer compile ... # -a compile
infer analyze ... # InferAnalyze
infer report ... # InferPrint
infer diff ... # this one is not new
infer clang ... # InferClang, not that you should use it
```
The man pages can still be improved a lot. Notable missing sections:
`ENVIRONMENT`, stuff about .inferconfig, some example usage, `DESCRIPTION`, ...
Reviewed By: jberdine
Differential Revision: D4921083
fbshipit-source-id: 9602230
Summary:
Ran the build with -w,-32 , delete code, repeat, until a fixpoint of no more warnings is reach.
Unfortunately we cannot fatal on w32 because ppx_compare can generate dead code (eg `compare_t` and only `compare` is used).
Reviewed By: mbouaziz
Differential Revision: D4945800
fbshipit-source-id: c95afb6
Summary:
The bufferoverrun checkers can now be run with:
infer -a checkers --bufferoverrun -- ...
Reviewed By: jeremydubreil
Differential Revision: D5010689
fbshipit-source-id: 2eaa396
Summary:
The Siof checkers can now be run with:
infer -a checkers --siof -- ...
and also runs by default using:
infer -a checkers -- ...
Reviewed By: jberdine
Differential Revision: D5009731
fbshipit-source-id: e0e2168
Summary: Pass the classpath infromation from the javac command line option file to allow the dection of errors accros Buck targets
Reviewed By: sblackshear
Differential Revision: D5018329
fbshipit-source-id: cf13198
Summary:
First step to be able to enable and disable the checkers to run in the following form:
> infer -a checkers --checker1 --checker2 --checker3 -- ...
and have a predefined list of checkers that are run by default with:
> infer -a checkers -- ...
Reviewed By: sblackshear
Differential Revision: D5007377
fbshipit-source-id: d7339ef
Summary:
This gives the option to run the biabduction analysis together with the other Clang-based checkers with the command:
infer -a checkers --biabduction -- ...
The filtering does not work yet because the filtering for the biabduction analysis matches the analyzer `Infer`, and does not filter much when the analyzer is `Checkers`, which is the case here.
Reviewed By: sblackshear
Differential Revision: D4773834
fbshipit-source-id: 16300cc
Summary: Sometimes reports need traces to be fully understood, but sometimes reporting where the exception takes place can save time to developers.
Reviewed By: jvillard
Differential Revision: D4914037
fbshipit-source-id: 039ab63
Summary: This will be needed to generate man pages with an accurate date.
Reviewed By: jberdine, mbouaziz
Differential Revision: D4937498
fbshipit-source-id: b5ebd31
Summary:
We won't be needing to know the terminal size when we move the help to
manpages. Hopefully if we need C stubs again in the future we can refer back to
this diff to revive them.
Reviewed By: jberdine
Differential Revision: D4937491
fbshipit-source-id: 99139fa
Summary: This is necessary to create a `report` subcommand (see later diffs).
Reviewed By: jberdine, dulmarod
Differential Revision: D4937488
fbshipit-source-id: 3fec0b5
Summary:
As an interprocedural checker, SIOF should not run unless explicitly required.
Make it a new type of analyzer like other similar checkers.
Reviewed By: mbouaziz
Differential Revision: D4937820
fbshipit-source-id: a9e2d38
Summary:
Running `infer --help` with spacetime showed that `Str` was a hog. And indeed perf is 74% better now:
```
before:
bash -c 'for i in $(seq 100); do infer --help > /dev/null 2>/dev/null; done' 28.02s user 0.62s system 98% cpu 29.205 total
after:
bash -c 'for i in $(seq 100); do infer --help > /dev/null 2>/dev/null; done' 6.34s user 0.64s system 92% cpu 7.557 total
```
Running on many files one after the other is now 64% faster too:
```
$ cd infer/tests/codetoanalyze/c/errors
$ # before
$ rm -fr infer-out && time bash -c 'for src in */*.c; do infer -a capture --continue -- clang -c $src; done'
bash -c 7.77s user 0.66s system 97% cpu 8.647 total
$ # after
$ rm -fr infer-out && time bash -c 'for src in */*.c; do infer -a capture --continue -- clang -c $src; done'
bash -c 2.35s user 0.56s system 93% cpu 3.119 total
$ time infer -a capture -- clang -c */*.c
infer -a capture -- clang -c */*.c 0.54s user 0.20s system 99% cpu 0.737 total
```
Reviewed By: mbouaziz
Differential Revision: D4875803
fbshipit-source-id: cfcfa69
Summary:
OCaml 4.04.0 new warnings raised a few valid points!
Fixing warning 57 in two ways:
- best way: introduce an auxiliary function to avoid code duplication
- not-so-best way: introduce code duplication. I did that when the branches body are small. Typically the number of bound variables in the pattern is high, so an auxiliary function would need to take many arguments and the whole thing will not be readable (we'd still duplicate the arguments we pass to the function for instance).
Reviewed By: jberdine
Differential Revision: D4851006
fbshipit-source-id: fbf1867
Summary:
Limit the use of `SourceFile.invalid` (renamed from `SourceFile.empty`) as much
as possible. In particular, do not generate bogus procnames for external global
variables: their translation unit was set to the invalid source file, now we
distinguish between extern/non-extern global variables more explicitly.
`SourceFile.invalid` is still used in too many places to actually remove it, often as a dummy initial value that never gets used, but sometimes as an actual value... Worse, we cannot fail on all operations on `SourceFile.Invalid` yet: the `SourceFile.to_string` method is used in too many places where it could get `SourceFile.Invalid` as argument. It's easy to see where it's used by making it raise in the code, then running the tests. This results in spaghetti backtraces that are hard to trace back to a root cause.
Reviewed By: akotulski, jeremydubreil
Differential Revision: D4860019
fbshipit-source-id: 45be040
Summary:
Try to read .inferconfig in the current directory, then in .., then in ../..,
etc. This can be overriden with the `INFERCONFIG` environment variable.
This removes the need for two-phase parsing, so clean up that code too.
Paths in .inferconfig are interpreted relative to where .inferconfig is located.
This does not apply to other path-sensitive things like regexpes... this is not
a show stopper because regexpes can account for the fact that infer may be
called from different project roots.
Make sure we fail when .inferconfig exists but cannot be read.
Reviewed By: jberdine
Differential Revision: D4843142
fbshipit-source-id: 340a63f
Summary: This is required to upgrade OCaml as our ancient Reason is not available on 4.04.0.
Reviewed By: yunxing
Differential Revision: D4851582
fbshipit-source-id: 994a9a8
Summary:
- Do not print OCaml backtrace unless in developer mode.
- Print javac output when it fails.
- Refactor Javac.ml so that rerunning the command doesn't duplicate code.
The output of the clang and Javac integration is good, the output for mvn is
ok, the output for Buck is still fairly "meh".
Reviewed By: jberdine
Differential Revision: D4818722
fbshipit-source-id: 1973b93
Summary:
Currently --per-procedure-parallelism defaults to a chunk size of 1
procedure, which has a high overhead. Add a command line option to
control it, and raise the default value.
Reviewed By: jvillard
Differential Revision: D4794692
fbshipit-source-id: 7715a40
Summary:
This checker was always running by default but was apparently never reporting.
This checkers can always be run using:
infer -a checkers --checkers-repeated-calls -- ...
Reviewed By: sblackshear
Differential Revision: D4782472
fbshipit-source-id: 5ec77f4
Summary:
All tests were redirecting `stderr` into duplicates.txt which made it much harder to see other error messages in stderr (such as uncaught exceptions).
To mitigate it, write duplicates to separate file and don't redirect `stderr` to another file.
Reviewed By: jvillard
Differential Revision: D4728938
fbshipit-source-id: 8ad2fc8
Summary:
This removes the dependency of libffi, and gives better guarantees at
compile-time. A bit overkill to retrieve the width of the terminal but what do
you know...
Reviewed By: jberdine
Differential Revision: D4700003
fbshipit-source-id: 036989b
Summary:
Two issues in the previous code:
1. We'd just tack on `#compilation-database` so if there already are `#flavors`
in a Buck target then we get a malformed `#flavors#compilation-database`
instead of `#flavors,compilation-database`. There's already code to deal
with that for other flavors (`#infer-capture-all`), so extend and reuse that.
2. The code didn't work if there are spaces in target names (sigh). Harden the
code a bit so that it works. Unfortunately Buck doesn't escape spaces at all
in its `buck targets --show-output` output, so the parsing still relies on
there being no spaces in flavors to work correctly. Sample output:
```
$ buck targets --show-full-output //clang_compilation_database:Hel\ lo#compilation-database
Not using buckd because watchman isn't installed.
[+] PROCESSING BUCK FILES...0.1s [100%] 🐳 New buck daemon
//clang_compilation_database:Hel lo#compilation-database /home/jul/infer/infer/tests/build_systems/codetoanalyze/buck-out/gen/clang_compilation_database/__Hel lo#compilation-database.json
```
Also, some code in codetoanalyze/clang-compilation-database/ didn't compile, so
fix that.
Reviewed By: dulmarod
Differential Revision: D4714338
fbshipit-source-id: b8ae324
Summary:
These are mentioned on fbinfer.com so IMHO should be displayed when running infer -h
Closes https://github.com/facebook/infer/pull/617
Differential Revision: D4697682
Pulled By: jvillard
fbshipit-source-id: 3d7dd7b
Summary:
It is definitely useful to collect information about how long the analysis of every procedure takes. It allows to detect and focus on outliers when trying to improve performance. However, this kind of information could be collected using a standard logging mechanism and does not need to be stored within the analysis artifacts.
I intend to add some form of similar logging in the context of #16348004 once we can get every analysis procedure analyzed through the `Ondemand` module. In this case, it would be easy to have a single place to log how does the analysis of a procedure take.
Reviewed By: jberdine
Differential Revision: D4636755
fbshipit-source-id: 01f3bca
Summary:
This option is not needed anymore as it was introduced to counter an uncovered
perf issue with creating human readable reports. The perf issue has been
addressed.
Instead of this option, one can use `infer --report-hook /bin/true ...` to
disable reporting. However, right now the Buck integration doesn't honor it so
this would need to be fixed to be a true equivalent of `--disable-bug-list`.
Reviewed By: jberdine
Differential Revision: D4712877
fbshipit-source-id: a09304f
Summary: This is dead code, and depends on libffi which we could get rid of otherwise.
Reviewed By: jberdine
Differential Revision: D4705645
fbshipit-source-id: efd28ef
Summary:
We want to stop supporting '-multiletter' options altogether in the future as
they are not standard.
For boolean switches, we used to have '-short' for true and '-nshort' for
false, but that second form is necessarily > 1 letters, which doesn't work in
this scheme. Replace these by '-s' for true and '-S' for false.
Reviewed By: jberdine
Differential Revision: D4674079
fbshipit-source-id: 95bacfe
Summary:
Add a new command-line option `--per-procedure-parallelism`, to change the granularity of parallelism of the analysis from file to procedure.
This is intended for `--reactive` mode where e.g. a single file is changed and the analysis currently uses just one core.
When the option is used, the Makefile mechanism is replaced by using forking instead.
The parent process does as little allocation as possible, to avoid taxing the kernel.
Caveats:
- Not active in Java, (issues with camlzip).
- Not active in checkers, yet.
Example use:
```
infer --reactive --changed-files-index index.txt --per-procedure-parallelism -- analyze
```
Reviewed By: jberdine
Differential Revision: D4634884
fbshipit-source-id: e358c18
Summary:
There was a race in serialization code where a reader could try to deserialize a partially overwritten value.
Here we use a `file.lock` to synchronize the writers, while the readers are not synchronized.
The atomic `rename` is used to move `file.lock` to `file` and make it available to readers.
Since `rename` is atomic, readers should see possibly stale, but not corrupted, data.
Reviewed By: sblackshear
Differential Revision: D4689664
fbshipit-source-id: dc5b546
Summary: I noticed we don't have `T` or `S` to denote timeouts in debug mode anymore. Today I saw it's still in `--stats` mode. Bring this feature back to `--debug` as well.
Reviewed By: cristianoc
Differential Revision: D4681669
fbshipit-source-id: 16ef19b
Summary:
When parsing .inferconfig and the env var, allow all modes of infer. This
allows people to provide default values for options in .inferconfig. This
requires subcommands' options to be all distinct, but that's already the case.
Reviewed By: jberdine
Differential Revision: D4674672
fbshipit-source-id: 992b454
Summary:
Turns out that we were special-casing the `Infer Driver` case for no particular
reason and that was preventing `infer --diff --help` from working as expected.
Reviewed By: martinoluca
Differential Revision: D4666988
fbshipit-source-id: 0868d4b
Summary:
Given two analysis results, it's now possible to compare them with the following command:
infer --diff --report-current reportA.json --report-previous reportB.json --file-renamings file_renamings.json
this command will then generate 3 files in `infer-out/differential/{introduced, fixed, preexisting}.json`, whose meaning is the following:
- `introduced.json` has all issues in `current` that are not in `previous`
- `fixed.json` has all issues in `previous` that are not in `current`
- `preexisting.json` has all issues that are in both `current` and `previous`
The json files generated can then be used to categorise results coming from incremental analyses of a codebase.
Reviewed By: jvillard
Differential Revision: D4482517
fbshipit-source-id: 1f7df3e
Summary:
All intermediate `.exp` files used for tests can be generated with custom info, based on what is needed for the tests purposes.
This customisation happens via command-line argument `--issues-fields`.
Reviewed By: cristianoc, jvillard
Differential Revision: D4628062
fbshipit-source-id: feaa382
Summary:
For writes of serialized data, write directly to the file instead of using a temporary one, and lock the file before writing.
Also added an `update` function to the API, to update an existing version of the data file instead of just replacing it with a new value.
Reviewed By: jberdine
Differential Revision: D4619958
fbshipit-source-id: 9642408
Summary:
With the ondemand analysis framework, the concept of timestamp was only being use to check if a procedure has already been analyzed. There was already a concept of "active" procedure for the procedure that were already being analyzed. This revision removes the concept of timestamp and merge it with the concept of analysis status.
This can be simplified further once the analysis always goes through `Ondemand.analyze`.
Reviewed By: cristianoc
Differential Revision: D4610371
fbshipit-source-id: 0fc516b
Summary:
This gives a way for users to flag safe methods regarding SIOF, for instance if
the problematic paths in the method cannot happen before `main()` has started.
Reviewed By: akotulski
Differential Revision: D4578700
fbshipit-source-id: 6542dcf
Summary: Shorter is better. `--compilation-database` was taken, renaming it to `--buck-compilation-database`.
Reviewed By: dulmarod
Differential Revision: D4567028
fbshipit-source-id: 011cd6f
Summary:
Two options to do the same thing, one for Java and and one for clang, become
one option to do the same thing.
Reviewed By: akotulski
Differential Revision: D4578472
fbshipit-source-id: fb0f21b
Summary:
One gets very obscure errors when trying to run infer for clang when it was
compiled for Java, or vice-versa. This diff makes sure we crash early with the
appropriate error message. For instance:
```
$ ./build-infer java
$ infer -- clang -c hello.c
Uncaught exception:
(Failure
"Unsupported build mode: make/cc\
\nInfer was built with clang analyzers disabled.\
\nPlease rebuild infer with clang enabled.\
\n")
Raised at file "pervasives.ml", line 30, characters 22-33
Called from file "backend/infer.ml", line 398, characters 6-48
Called from file "backend/infer.ml", line 449, characters 20-38
$ infer --clang-compilation-db-files foo.json
Uncaught exception:
(Failure
"Unsupported build mode: clang compilation database\
\nInfer was built with clang analyzers disabled.\
\nPlease rebuild infer with clang enabled.\
\n")
Raised at file "pervasives.ml", line 30, characters 22-33
Called from file "backend/infer.ml", line 392, characters 8-65
Called from file "backend/infer.ml", line 449, characters 20-38
```
Reviewed By: sblackshear
Differential Revision: D4566641
fbshipit-source-id: d9a118f
Summary:
Sevel auxiliary files made it to the output directory of the analysis of individual targets when analyzing Java projects build with Buck. However, these files are then taken into account= to compute the target rule key and then to decide whether to analyze the dependent targets. Since these auxiliary files were containing time sentive information, every cach miss on a given target would then invalitate the cache entries for all the dependent targets.
This diff cleans up the output directory to only keep the specs files, the `global.tenv` and the `report.json` files which are the only artifacts needed to analyze the dependent targets
This diff makes a minimal number of changes to see how it behaves in prod, but I intend to refoctor this more when continuing to add support for running Infer with genrules
Reviewed By: sblackshear
Differential Revision: D4562615
fbshipit-source-id: 4628420
Summary:
Xcode's compilation databases follows a different convention than cmake's and
escape the `"file"` and `"dir"` fields of each unit to make them shell-ready.
We need to treat them differently when reading them.
This adds a new `--clang-compilation-db-files-escaped` option and makes the
code related to reading compilation databases deal correctly with both
conventions.
Reviewed By: akotulski
Differential Revision: D4559239
fbshipit-source-id: 51120ae
Summary:
This reorganises the contents of `infer --help`:
- Headings are more prominent (start with `**`)
- New "Java" section
- Delete "Analysis" section, distribute contents over other sections
- New "Quandary" section
- Under the hood, new "Buffer Overruns" and "Crashcontext" sections, but do not show them as we don't expect external use yet, although that may be a bit arbitrary
- typo: `--bufferoverrn` -> `--bufferoverrun`
- move some options from one section to another
Reviewed By: jberdine
Differential Revision: D4537500
fbshipit-source-id: a789375
Summary: Being forced to separately define `pp_element`/`pp_key` is uneccessary and makes it more cumbersome to create a set/map from an existing module that already defines `pp`.
Reviewed By: jeremydubreil
Differential Revision: D4517308
fbshipit-source-id: 9b17c9c
Summary:
Propagating the arguments read in .inferconfig shouldn't be necessary as they
are parsed by each executable. There are corner cases where I think this diff
could change the behaviour of infer (eg, .inferconfig redefines project-root
and subsequent exes read a different .inferconfig thanks to the project-root in
INFER_ARGS), but I don't think there are good use cases like that.
Reviewed By: jberdine
Differential Revision: D4475092
fbshipit-source-id: 5c020d0
Summary:
Clients should use `Config.parse_action` instead to figure in what mode they
are operating.
In particular, the biggest change is in logging. Take the `parse_action` into
account instead of the exe, and change the log/ subdirectories to be "capture",
"driver", "analyze", and "print", corresponding to the various phases of an
infer run.
Reviewed By: jberdine
Differential Revision: D4474943
fbshipit-source-id: 6d33ad3
Summary:
Support several parsing modes: Infer, Javac, NoParse
The "Infer" mode specifies a list of sections, ie the parts of infer that are affected by an option (corresponds to the old notion of "exes"):
analysis, clang frontend, print, ...
- .inferconfig and INFER_ARGS always parsed
- outside .inferconfig and INFER_ARGS, do not parse subcommand arguments before the subcommand has been activated
- command-line is parsed or not based on the subcommand/executable selected
- executable dictates subcommand, so almost nothing depends on the executable outside of Config. Another diff will restrict the API around exes to reflect this.
Reviewed By: jberdine
Differential Revision: D4474886
fbshipit-source-id: 442dfef
Summary:
Remove the remaining uses of polymorphic equality `=`.
In case of basic types, this is replaced by String.equal or Int.equal.
In case of `= []`, this is replaced by `List.is_empty`.
In case of `= None`, this is replaced by `is_none`.
In case of a datatype definition such as `type a = A | B`,
a `compare_a` function is defined by adding `type a = A | B [@deriving compare]`
and a `equal_a` function is defined as `let equal_a = [%compare.equal : a]`.
In case of comparison with a polymorphic variant `= `Yes`, the equality
defined in `PVariant.(=)` is used. Typically, `open! Pvariant` is added
at the beginning of the file to cover all the uses.
Reviewed By: jberdine
Differential Revision: D4456129
fbshipit-source-id: f31c433
Summary: Also updated the doc comment of `parse`, it had gone a bit out of sync.
Reviewed By: jberdine
Differential Revision: D4435580
fbshipit-source-id: bb1a6f2
Summary:
Epilogue tasks such as closing logs or putting files back were we found them
run automatically at the end of our executables by registering them with
`at_exit`. They do not run if the program is interrupted by a signal. This diff
makes sure they are run when the user stops infer with Ctrl-C (SIGINT).
Reviewed By: cristianoc
Differential Revision: D4435575
fbshipit-source-id: c3ab702
Summary:
This replaces the previous integration written in Python, which consisted in 1)
run the mvn command and parse its output to locate "directories containing
source files", 2) run on files named "*.java" in these directories. This meant
we had to run javac twice on each source file, and more importantly this
mechanism of finding source files was very fragile. In fact, I could not make
it work on several mvn projects I tried.
The new integration is based on parsing "pom.xml" to add an "infer-capture"
profile which instructs mvn to run `/path/to/infer` instead of `javac`. We also
add this profile to each maven submodule.
Users can specify an "infer-capture" profile themselves if the default one
doesn't work; in that case we don't inject our own "infer-capture" profile.
Reviewed By: jeremydubreil
Differential Revision: D4409613
fbshipit-source-id: d664274
Summary:
Also make sure we don't introduce deprecated options in our repo, eg when
calling infer from infer.
Reviewed By: jeremydubreil
Differential Revision: D4430379
fbshipit-source-id: 77ea7fd
Summary:
We only ever use very few of the possible `Arg.spec` constructors and,
crucially, all of them declare a function to pass argument values to. This is
needed for the next diff, which adds deprecation messages.
Reviewed By: jeremydubreil
Differential Revision: D4430217
fbshipit-source-id: c5ffe5f
Summary:
Make the html output available to checkers when -g is used on the command-line.
A checker needs to call a function to start and finish the processing of each node,
and add prints during the processing.
This diff illustrates the case for Eradicate, by adding printing of the pre-state
and post-states.
Reviewed By: sblackshear
Differential Revision: D4421379
fbshipit-source-id: 67501ba
Summary: `Toplevel` name is confusing - in ocaml world it means interactive ocaml shell (we call that "interactive"). In infer it meant "Toplevel infer binary". We already call it "driver" to avoid confusion, let's rename the code as well.
Reviewed By: jvillard
Differential Revision: D4415111
fbshipit-source-id: 1002f27
Summary:
This makes it more obvious why infer would force a path to be absolute since we
base that decision on the resolved path. For instance:
```
$ mkdir foo
$ cd foo
$ ln -s ../examples goo
$ infer -- clang -c goo/hello.c
[...]
/home/jul/infer/examples/hello.c:14: error: NULL_DEREFERENCE
```
We see that the path is outside of the current directory clearly, whereas
before infer would report on "goo/hello.c".
Reviewed By: akotulski
Differential Revision: D4409579
fbshipit-source-id: 7172005
Summary:
Currently, if we don't find `-d` or `-classes_out` on the command line then we
tell javac to redirect the compiled classes in some other directory, by default
the initial working directory. But we don't detect when these arguments are
hidden inside files (`foo` arguments on the javac command line) so the
heuristic was incomplete. Look inside these files to better tell whether we need
to make up an output directory or not.
Reviewed By: jeremydubreil
Differential Revision: D4397716
fbshipit-source-id: 30c5e4f
Summary:
Change Utils.filename_to_relative to return None in case the filename
is not under root, rather than returning the filename unchanged.
Reviewed By: akotulski
Differential Revision: D4391075
fbshipit-source-id: bf753af
Summary:
This diff allows to use the linters written in DSL to check for bugs.
Now new checkers can be written directly in the DSL.
The diff also remove some weirdness and simplify the CTL semantics.
For example no need to unwrap a node when evaluating the IN operator.
Also no need to distinguish anymore between stmt and decl in the
semantics of EX and EF.
Moreover, the diff de-couple hard-coded checkers (eg checks on component kit)
from those checkers parsed in the .al files.
Reviewed By: martinoluca
Differential Revision: D4375207
fbshipit-source-id: 9ac2d47
Summary: The logic for filtering reports based on their buckets lives in InferPrint, so this code isn't doing anything.
Reviewed By: jvillard
Differential Revision: D4379966
fbshipit-source-id: 5a69304
Summary:
Force clients to specify the path relative to which relative paths
should be made absolute.
Reviewed By: akotulski
Differential Revision: D4370262
fbshipit-source-id: 36a2807
Summary:
Now that the toplevel driver's cwd is passed through realpath, the
dance to preserve symlinks is redundant.
Reviewed By: jeremydubreil
Differential Revision: D4371055
fbshipit-source-id: c8aebaf
Summary:
For example: `infer --print-logs --stats -- clang -c hello.c`.
The option is not on by default.
This forwards all the output to log files to stdout or stderr as appropriate.
The multiplexing is very crude and can be improved later if needed if
stdout/err is too garbled by concurrent partial writes.
Reviewed By: jberdine
Differential Revision: D4365996
fbshipit-source-id: 7f2ab98
Summary:
Remove the need for a dummy initialization of log files.
The fact that we were not setting log files in some cases doesn't seem to be
relevant so I killed it. I observed no difference in output on simple clang and
javac examples. It will be easy to restore a better version of it in the next
diff if needed.
Also fix an fd leak: when opening new log files, previous ones were not being
flushed and closed (except at exit).
Reviewed By: jberdine
Differential Revision: D4365992
fbshipit-source-id: 940bc16
Summary:
Instead of opening new log files each time with non-deterministic names, keep
appending to the same log files. This only removes the randomized part of the
names in the files. In particular, it keeps the name prefixes for, eg, clang
source files.
Also changed most "<executable>/<executable>-out.log" to simply "<executable>/out.log".
Reviewed By: jberdine
Differential Revision: D4365983
fbshipit-source-id: 46792dc
Summary: This more easily allow to switch between the different modes for handeling dynamic dispatch
Reviewed By: sblackshear
Differential Revision: D4367556
fbshipit-source-id: 795d2c4
Summary: Use the lazy dynamic dispatch by default in prod for the Java analysis
Reviewed By: sblackshear
Differential Revision: D4356872
fbshipit-source-id: 491e92e
Summary: Different analyses need different preanalyses to run. It doesn't make sense for all of the pre-analyses to be bundled together into one package.
Reviewed By: jvillard
Differential Revision: D4348243
fbshipit-source-id: 46a8ebd
Summary:
Adding #infer-capture-all et al. by hand is annoying and I always forget to do
it. Let infer figure that out.
Reviewed By: dulmarod
Differential Revision: D4339799
fbshipit-source-id: 55e52dc
Summary: Access to std::vector shouldn't be treated as SKIP. Implementation is simple enough to use one from std:: headers
Reviewed By: jvillard
Differential Revision: D4339577
fbshipit-source-id: d1fbbee
Summary: pattern matching we had before allowed many unintended functions to pass (such as `max_element`). Make matching much more strict
Reviewed By: jvillard
Differential Revision: D4313428
fbshipit-source-id: 189c522
Summary:
This diff parses the build command args to directly handle the -version
option passed to java and javac, to make the integration with buck more
robust by ensuring that the version and no additional debug logging is
generated for `infer --debug -- javac -version`.
Reviewed By: jeremydubreil
Differential Revision: D4158011
fbshipit-source-id: e7d6b4d
Summary:
SuppressWarnings annotations are hardly used and add considerable
complexity due to requiring recompilation with an annotation processor.
Reviewed By: jvillard
Differential Revision: D4312193
fbshipit-source-id: c4fc07e
Summary:
This option is only useful if you want to treat casts angelically, but nothing else.
Since angelic is on by default and this option is off by default, it's basically useless.
Reviewed By: jeremydubreil
Differential Revision: D4334030
fbshipit-source-id: 3c0b0ed
Summary:
The list of argument specs is a global ref inside `CommandLineOptions`, which
need to be reset to the empty list every time `parse` is called. Otherwise, we
get duplicated sections:
```
$ infer --help
Infer version v0.9.4-84d61cb
Copyright 2009 - present Facebook. All Rights Reserved.
Toplevel options
--inferconfig-home <dir> Path to the .inferconfig file
--project-root | -pr <dir> Specify the root directory of the project (default: /home/jul/infer)
Analysis (backend) options
Clang frontend options
Java frontend options
Toplevel options
[... the rest of the options -- without --inferconfig-home or --project-root,
with all the section headers again ...]
```
Reviewed By: jberdine
Differential Revision: D4333448
fbshipit-source-id: f91ea66
Summary:
The javac -classes_out option is used to set the results directory for
the buck build system integration.
Reviewed By: jeremydubreil
Differential Revision: D4162907
fbshipit-source-id: 75d0a6d
Summary: Turns out I forgot to close the fd returned by dup(2) so we were leaking a lot.
Reviewed By: jeremydubreil
Differential Revision: D4327389
fbshipit-source-id: 74574ac
Summary:
C stubs were causing issues with building:
- need OCaml stubs of the C stubs for byte compilation!
- brittle (to remain polite) support in ocamlbuild
Ctypes is awesome, use it instead.
Slight wrinkle on the previous statement:
- ioctl(2) is variable arguments, which is not officially supported by Ctypes but should work in our use-case
- I'm hardcoding the value of a C macro found in system headers
Reviewed By: jberdine
Differential Revision: D4319507
fbshipit-source-id: 352804a
Summary:
<ugly shameful hack>
Temporarily redirect stderr to /dev/null before calling `Javalib.get_class` so
as to avoid getting spammed with "Warning: unexpected attribute: Code" messages
when parsing Java files.
</ugly shameful hack>
I suspect that now that Javalib handles Java 8 this issue is more prevalent. An
issue/PR should be sent to Javalib too so that it's fixed upstream and we can
eventually remove the hack (t15039096).
Reviewed By: jberdine
Differential Revision: D4319466
fbshipit-source-id: af855ba
Summary:
Substituting modules into Core.Std breaks the linking behavior of
module aliases, and so does including the module type of Core.Std in a
signature. So, for now, this diff exposes the broken functions in
Core's Gc, Signal, and Sys modules.
Reviewed By: cristianoc
Differential Revision: D4313140
fbshipit-source-id: d8fea00
Summary:
This diff adds basic support for parsing the arguments passed to the
build command directly from Config.
Reviewed By: dulmarod
Differential Revision: D4201480
fbshipit-source-id: bba6056
Summary:
Several Core functions silently wrap argument functions with catch-all
exception handlers that exit. This diff protects against these from
ever being used by deprecating them, which causes compilation failure if
they are used.
Reviewed By: jvillard
Differential Revision: D4271781
fbshipit-source-id: a096171
Summary:
Use In_channel and Out_channel operations instead of those in Pervasives. Don't
use physical equality on values that aren't heap-allocated since it doesn't help
the compiler generate faster code and the semantics is unspecified. Also use
phys_equal for physical equality.
Reviewed By: sblackshear
Differential Revision: D4232459
fbshipit-source-id: 36fcfa8
Summary:
Utils contains definitions intended to be in the global namespace for
all of the infer code-base, as well as pretty-printing functions, and
assorted utility functions mostly for dealing with files and processes.
This diff changes the module opened into the global namespace to
IStd (Std conflict with extlib), and moves the pretty-printing
definitions from Utils to Pp.
Reviewed By: jvillard
Differential Revision: D4232457
fbshipit-source-id: 1e070e0
Jules Villard
Dulma Churchill
Sam Blackshear
Jeremy Dubreil
Jia Chen
Mehdi Bouaziz
Josh Berdine
Radu Grigore
Martino Luca
Andrzej Kotulski
Matthew Haughton
Cristiano Calcagno
Kihong Heo
Dino Distefano