Summary:
In looking at summaries that Quandary took a long time to compute, one thing I notice frequently is redundancy in the footprint sources (e.g., I might see `Footprint(x), Footprint(x.f), Footprint(x*)`).
`sudo perf top` indicates that joining big sets of sources is a major performance bottleneck, and a large number of footprint sources is surely a big part of this (since we expect the number of non-footprint sources to be small).
This diff addresses the redundancy issue by using a more complex representation for a set of sources. The "known" sources are still in a set, but the footprint sources are now represented as a set of access paths (via an access trie).
The access path trie is a minimal representation of a set of access paths, so it would represent the example above as a simple `x*`.
This should make join/widen/<= faster and improve performance
Reviewed By: jberdine
Differential Revision: D5663980
fbshipit-source-id: 9fb66f8
Summary:
The previous widening operator added stars to the *end* of paths that existed in `next` but not `prev`. This is not enough to ensure termination in the case where the trie is growing both deeper and wider at the same time.
The newly added test demonstrates this issue. In the code, there's an ever-growing path of the form `tmp.prev.next.prev.next...` that wasn't summarized by the previous widening operator. The new widening is much more aggressive: it replaces *any* node present in `next` but not `prev` with a `*` (rather than trying to tack a star onto the end). This fixes the issue.
This issue was causing divergence on tricky doubly-linked list code in prod.
Reviewed By: jeremydubreil
Differential Revision: D5665719
fbshipit-source-id: 1310a92
Summary:
Saw these two types of errors before (but they're hard to reproduce locally) when building the models:
- `ERROR: Zip.Error("/mnt/btrfs/trunk-git-infer-739-1503054473/infer/bin/../lib/java/models.jar", "", "end of central directory not found, not a ZIP file")`. I think this means infer reads a partially-written models jar. We shouldn't try to load this in models mode.
- `install` would complain that the destination already exists. I think this can only happen if there's a race and the file gets created between when install first checks and when it tries to write to it.
This made me realise that the some of the models are computed in C and C++ mode
and we pick one computed spec arbitrarily. That sounds a bit dodgy but at least
now we do so in a non-racy way.
Reviewed By: jeremydubreil
Differential Revision: D5658389
fbshipit-source-id: 8077279
Summary: Also, stop trying to delete directories that do not exist: "sources" and "filelists".
Reviewed By: mbouaziz
Differential Revision: D5658089
fbshipit-source-id: e1cdb13
Summary:
This is a check for when an unavailable class is being allocated.
This diff also adds a check for the context to remove false positives: If the class is not available but the method calls are wrapped in a check whether the class is available, then don't report.
Reviewed By: jvillard
Differential Revision: D5631191
fbshipit-source-id: 2082dfe
Summary: Other parts of the code where using the checks in the AndroidFramework module. It is better to have those things in one place.
Reviewed By: sblackshear
Differential Revision: D5654247
fbshipit-source-id: 2a783e7
Summary:
Calling exit at the end of a proc does not create unreachable code, prior to this commit inferbo reports that it does.
We extend collect_instrs to detect when we're at the end of a procedure in C and not report on unreachable code if we call a procedure there.
Reviewed By: mbouaziz
Differential Revision: D5623637
fbshipit-source-id: 0d5f326
Summary: This check is not possible in Java as it natirally happens in the totally legit case of the `try ... finally`.
Reviewed By: sblackshear
Differential Revision: D5568802
fbshipit-source-id: 24ca074
Summary:
Instead of a whitelist and blacklist and default issue types and default
blacklist and filtering, consider a simpler semantics where
1. checkers can be individually turned on or off on the command line
2. most checkers are on by default
3. `--no-filtering` turns all issue types on, but they can then be turned off again by further arguments
This provides a more flexible CLI and is similar to other options in the infer
CLI, where "global" behaviour is generally avoided.
Dynamically created checkers (eg, AL linters) cause some complications in the
implementation but I think the semantics is still clear.
Also change the name of the option to mention "issue types" instead of
"checks", since the latter can be confused with "checkers".
Reviewed By: jberdine
Differential Revision: D5583238
fbshipit-source-id: 21de476
Summary:
Every module declared but not used in the same source file is warned about
currently. Disable the noisy warning.
For instance, before this diff (and after "M-x merlin-restar-process"):
1. open AbstractInterpreter.ml
2. merlin shows a warning for `module Make` inside emacs
Reviewed By: jberdine
Differential Revision: D5621383
fbshipit-source-id: c175e5d
Summary:
Seeing rrors in "/foo/bar/src/base/infer.ml" is less distracting than seeing
errors in "/foo/bar/src/./base/infer.ml".
Reviewed By: jberdine
Differential Revision: D5621060
fbshipit-source-id: 55ee069
Summary:
The issue was as follows:
- ppx.exe depended on all the ocaml source files
- ppx.exe is an implicit prerequisit of all the (non-automatically-generated) build objects in jbuilder
- as a result, when modifying a source file, we would recompile ppx.exe, thus all the source files
ppx.exe only needs to depend on all the generated source files, so that
jbuilder can find them all.
Reviewed By: jberdine
Differential Revision: D5621007
fbshipit-source-id: 92c5b9c
Summary:
The `-index-store-path` argument does not exist in Clang 5.0 and it gets discarded without any error message. The problem is that its argument, a folder, is not discarded, and Clang considers it as a source file. This leads to the following errors:
- `cannot specify -o when generating multiple output files` (this happens if a `-o` argument is passed)
- `error reading '<PATH>'` (this can be observed when running the "normalized" version of the clang command, generated via the -### flag)
This weird case can be observed when `-index-store-path` is passed in a sequence like the following: `-x c -index-store-path <PATH> -c`.
With this change, we remove the `index-store-path` option, and its argument, from the original clang command.
Reviewed By: jvillard
Differential Revision: D5601808
fbshipit-source-id: 4200308
Summary:
This gives additional information to users. For instance:
```
--biabduction
Activates: the separation logic based bi-abduction analysis using
the checkers framework (Conversely: --no-biabduction)
This option is relevant to infer-analyze(1).
```
Reviewed By: sblackshear
Differential Revision: D5583197
fbshipit-source-id: 2960b90
Summary:
We previously lumped ownership predicates in with all other predicates. That limited us to a flat ownership domain.
This diff separates out the ownership predicates so we can have a richer lattice of predicates with each access path.
This lets us be more precise; for example, we can now show that
```
needToOwnBothParams(Obj o1, Obj o2) {
Obj alias;
if (*) { alias = o1; } else { alias = o2; }
alias.f = ... // both o1 and o2 need to be owned for this to be safe
}
void ownBothParamsOk() {
needToOwnBothParams(new Obj(), new Obj()); // ok, would have complained before
}
```
is safe.
Reviewed By: jberdine
Differential Revision: D5589898
fbshipit-source-id: 9606a46
Summary:
This makes it easier to test a single checker.
Also refactor the code to make it harder to mess up the list of default/all checkers.
Reviewed By: sblackshear
Differential Revision: D5583209
fbshipit-source-id: 7c919b2
Summary:
This makes the CLI more complete: before, it was often impossible to "go back"
once some options were passed, namely options produced by `mk_*_{list,opt}`.
Now these automatically create an accompanying `--<long>-reset` option that
resets the config variable to its default value.
Also unify our naming of `~meta` arguments:
- no more spaces in them (except one instance where it's a whole sentence)
- use `+foo` if `foo` can be specified multiple times
Reviewed By: dulmarod
Differential Revision: D5583187
fbshipit-source-id: a8c2567
Summary:
Use jbuilder to build infer instead of ocamlbuild. This is mainly to get faster builds:
```
times in 10ms, ±differences measured in speedups, 4 cores
| | ocb total | jb | ±total | ocb user | jb | ±user | ocb cpu | jb | ±cpu | ocb sys | jb | ±sys |
|-----------------------------------+-----------+------+--------+----------+------+-------+---------+-----+------+---------+------+------|
| byte from scratch | 6428 | 2456 | 2.62 | 7743 | 6662 | 1.16 | 138 | 331 | 2.40 | 1184 | 1477 | 0.80 |
| native from scratch | 9841 | 4289 | 2.29 | 9530 | 8834 | 1.08 | 110 | 245 | 2.23 | 1373 | 1712 | 0.80 |
| byte after native | 29578 | 1602 | 18.46 | 4514 | 4640 | 0.97 | 170 | 325 | 1.91 | 543 | 576 | 0.94 |
| change infer.ml byte | 344 | 282 | 1.22 | 292 | 215 | 1.36 | 96 | 99 | 1.03 | 040 | 066 | 0.61 |
| change infer.ml native | 837 | 223 | 3.75 | 789 | 174 | 4.53 | 98 | 99 | 1.01 | 036 | 47 | 0.77 |
| change Config.ml byte | 451 | 339 | 1.33 | 382 | 336 | 1.14 | 97 | 122 | 1.26 | 056 | 80 | 0.70 |
| change Config.ml native | 4024 | 1760 | 2.29 | 4585 | 4225 | 1.09 | 127 | 276 | 2.17 | 559 | 644 | 0.87 |
| change cFrontend_config.ml byte | 348 | 643 | 0.54 | 297 | 330 | 0.90 | 96 | 67 | 0.70 | 038 | 102 | 0.37 |
| change cFrontend_config.ml native | 1480 | 584 | 2.53 | 1435 | 906 | 1.58 | 106 | 185 | 1.75 | 136 | 178 | 0.76 |
#+TBLFM: $4=$2/$3;f2::$7=$5/$6;f2::$10=$9/$8;f2::$13=$11/$12;f2
50 cores
| | ocb total | jb | ±total | ocb user | jb | ±user | ocb cpu | jb | ±cpu | ocb sys | jb | ±sys |
|---------------------+-----------+------+--------+----------+------+-------+---------+----+------+---------+------+------|
| byte from scratch | 9114 | 2061 | 4.42 | 9334 | 5133 | 1.82 | | | 0/0 | 2566 | 1726 | 1.49 |
| native from scratch | 13481 | 3967 | 3.40 | 12291 | 7608 | 1.62 | | | 0/0 | 3003 | 2100 | 1.43 |
| byte after native | 3467 | 1476 | 2.35 | 5067 | 3912 | 1.30 | | | 0/0 | 971 | 801 | 1.21 |
#+TBLFM: $4=$2/$3;f2::$7=$5/$6;f2::$10=$9/$8;f2::$13=$11/$12;f2
```
Menu:
1. Write a jbuild file, autogenerated from jbuild.in because we need to fill in
some information at build-time (really, at configure time, but TODO), such as
whether or not clang is enabled.
2. Nuke lots of stuff from infer/src/Makefile that is now in the jbuild file
3. The jbuild file lives in infer/src/ so it can see all the sources. If we put it somewhere else, eg, infer/, then `jbuilder` scans too many files (all irrelevant) and takes 2.5s to start instead of .8s. Adding irrelevant directories to jbuild-ignore does not help.
4. jbuilder does not support subdirectories, so resort to listing all the
source files in the generated jbuild (only source directories need to be
manually listed in jbuild.in though). Still, the generated .merlin is wrong
and makes merlin find source files in _build, so manually tune it to get
good merlin support. We also lose some of merlin for unit tests as it
cannot see their build artefacts anymore.
5. checkCopyright gets its own jbuild because it's standalone. Also, remove
some deprecation warnings in checkCopyright due to the new version of Core from
a while ago.
6. Drop less-used Makefile features (they had regressed anyway) such as
building individual modules. Also, building mod_dep.pdf now takes all the
source files available so they better build (before, it would only take the
source files from the config, eg with or without clang) (that's pretty minor).
7. The toplevel is now built as a custom toplevel because that was easier. It
should soon be even easier: https://github.com/janestreet/jbuilder/issues/210
8. Move BUILTINS.mli to BUILTINS.ml because jbuilder is not happy about
interface files without implementations.
In particular, I did not try to migrate too much of the Makefile logic to jbuilder,
more can be done in the future.
Reviewed By: jberdine
Differential Revision: D5573661
fbshipit-source-id: 4ca6d8f
Summary:
The only path to the `MergeCaptured.slink` function is when we should merge, so
it doesn't make sense to check `Config.merge` again. In the case of `infer run
--flavors -- buck ...`, this would create regular symlinks instead of the much
faster multilinks.
Reviewed By: jberdine
Differential Revision: D5574020
fbshipit-source-id: df710ca
Summary:
Fix for the #669. Recursion problem has been reproduced (see the new doctest in the latest commit) and eliminated. I guess the new version will work faster too (no more unnecessary slicing).
Closes https://github.com/facebook/infer/pull/697
Reviewed By: sblackshear
Differential Revision: D5507925
Pulled By: jvillard
fbshipit-source-id: 21fa07b
Summary:
This is a needed step in the direction of making prenalysis functional: it will return a view of the CFG rather than mutating the CFG.
ProcCfg already works by providing a view on the underyling CFG, but the bi-abduction can't leverage this because it uses the "raw" CFG.
This diff does a partial swap of the raw CFG for an exceptional ProcCfg. The goal is to make sure the bi-abduction never calls `Procdesc.get_instrs`; it should use the `ProcCfg` wrapper instead.
That way, preanalyses that add instructions (like the liveness prenalysis) will work.
There's still some calls to `Procdesc.get_succs` etc., but we can remove those in a future diff.
They're not on the critical path because the current preanalyses only add instructions, not nodes or edges.
Reviewed By: jeremydubreil
Differential Revision: D5556387
fbshipit-source-id: 4ffda00
Summary:
Previous version was hard to understand because it was doing many things within same code. New version has different code for Arrays, Structs and others.
There is some copy-paste, but it's easier to follow code (open to suggestions though)
Reviewed By: dulmarod
Differential Revision: D5547999
fbshipit-source-id: 77ecb24
Summary: It wasn't using code from `std::vector::empty` which recently was improved. Instead of inlining `std::vector::empty`, call it to know whether vector is empty or not.
Reviewed By: jvillard
Differential Revision: D5573379
fbshipit-source-id: e024a42
Summary: Useful for identifying user-controlled array accesses that could lead to buffer overflows
Reviewed By: mbouaziz
Differential Revision: D5520985
fbshipit-source-id: 92984f6
Summary:
When a sink name is specified in `.inferconfig` or in OCaml, it might conflict with a function of the same name that has a different number of args.
We shouldn't try to create a sink in this case, and we definitely shouldn't crash.
Reviewed By: jeremydubreil
Differential Revision: D5561216
fbshipit-source-id: fa1859b
Summary: This is unused, as far as I can tell. If we want to revive it, we can do it in AL or as a simple checker; it certainly doesn't require the full might of bi-abduction.
Reviewed By: jeremydubreil
Differential Revision: D5556325
fbshipit-source-id: e3895c2
Summary:
Replace `inferTraceBugs` with `infer-explore` with a similar CLI. Some options changed:
- --max-level -> --max-nesting, and "max" is the default value instead of a possible value
- --no-source -> --no-source-preview
Reviewed By: mbouaziz
Differential Revision: D5526651
fbshipit-source-id: 8383f37
Summary:
In some cases we normalize expressions to check some facts about them. In these
cases, trying to keep as much information as possible in the expression, such
as the fact it comes from a `sizeof()` expression, is not needed. Doing
destructive normalization allows us to replace `sizeof()` by its
statically-known value.
closes#706
Reviewed By: mbouaziz
Differential Revision: D5536685
fbshipit-source-id: cc3d731
Summary: A temporary workaround until we can understand why this happens and fix it.
Reviewed By: jeremydubreil
Differential Revision: D5559838
fbshipit-source-id: dc86eb9
Summary: That was too noisy. Propagate `--quiet` to the Python reporting hook so as to still emit bugs.txt and so on.
Reviewed By: martinoluca
Differential Revision: D5501106
fbshipit-source-id: 63b6451
Summary:
This allows the user to specify a different command to build the current version of the project vs to build the previous version of the project. Here's an example:
```
infer diff --gen-previous-build-command-script "echo clang -c hello.c" ... -- clang -c hello2.c
```
By default the two build commands are the same. If the script is to be used for both the current and the previous versions, then it's on the user to run it once first, eg:
```
infer diff --gen-previous-build-command-script "./myscript.sh previous" ... -- $(./myscript.sh current)
```
Reviewed By: martinoluca
Differential Revision: D5500989
fbshipit-source-id: 7374b44
Summary: Working on making CFG immutable by killing refs to functions that mutate them. This is an easy one because it doesn't do anything :).
Reviewed By: jeremydubreil, mbouaziz
Differential Revision: D5551521
fbshipit-source-id: bec76a9
Summary:
Record the list of access paths (if any) used in the index expression for each array access.
This will make it possible to use array accesses as sinks in Quandary
Reviewed By: jeremydubreil
Differential Revision: D5531356
fbshipit-source-id: 8204909
Summary:
Do not use the deprecated (and slower) `#infer` flavor. Instead, `infer-run`
runs capture with the `#infer-capture-all` flavor, followed by merging targets,
followed by the analysis.
Move the call to `MergeCapture` around to make this change easier.
Reviewed By: mbouaziz
Differential Revision: D5547199
fbshipit-source-id: 53c9996
Summary:
With current model, there are issues with cxx range loop. It looks like
it comes from std::vector::size model.
example of such FP:
```
int t = vec.size();
for(auto& elem : vec) {
auto x = elem
}
```
Reviewed By: jvillard
Differential Revision: D5545914
fbshipit-source-id: fbe55b3
Summary: Those are not particularly relevant for the biabduction analysis. It would be easy to have a dedicated checker for this if we happen to need one day.
Reviewed By: sblackshear
Differential Revision: D5530834
fbshipit-source-id: 316e60f
Summary:
The Eradicate `Nullable` checker should now be run using:
infer -a checkers --eradicate ...
Reviewed By: mbouaziz
Differential Revision: D5529226
fbshipit-source-id: 0de2956
Summary:
It's nice to have "raw" as the default kind of access path, since it's used much more often than the abstraction.
This is also a prereq for supporting index expressions in access paths, since we'll need mutual recursion between accesses and access paths.
Reviewed By: jeremydubreil
Differential Revision: D5529807
fbshipit-source-id: cb3f521
Summary:
This is unsound but will help the analysis to report less false alarms with the common pattern:
if (a.get() != null) {
a.get().foo();
}
Reviewed By: sblackshear
Differential Revision: D5528227
fbshipit-source-id: 750db4a
Summary:
Previously, only the bug type + file name (up to renaming) were taken into
account, which was too coarse. The key is file-independent and provides
additional signal.
Reviewed By: martinoluca
Differential Revision: D5536858
fbshipit-source-id: 70b732b
Summary: This was reusing the side effects of the `add_constraints_on_retval` for the final purpose of being angelic and just assigning a fresh value to the lhs of the load.
Reviewed By: sblackshear
Differential Revision: D5507037
fbshipit-source-id: ec1c89c
Summary: D5526683 assumed that this was the case, and broke the report hook.
Reviewed By: martinoluca
Differential Revision: D5527353
fbshipit-source-id: 2e169b9
Summary:
Bumps facebook-clang-plugins to a version that outputs sizeof() info in bytes and not bits.
update-submodule: facebook-clang-plugins
Reviewed By: akotulski
Differential Revision: D5526747
fbshipit-source-id: 6019542
Summary:
Cosmetic changes to the output of `make clean` and `make test-replace`.
Run the inferTraceBugs tests from tests/build_systems/ like other such e2e
tests, so that they can run in parallel with other tests instead of
sequentially at the end.
Reviewed By: dulmarod
Differential Revision: D5526599
fbshipit-source-id: 57231bd
Summary:
Works the same way as read/write races on fields, except that are more relaxed (er, unsound) in deciding whether two containers may alias.
This is needed to avoid reporting a ton of FP's; full explanation in comments.
Reviewed By: da319
Differential Revision: D5493404
fbshipit-source-id: 0a5d8b1
Summary:
This was hardcoded but it's passed as an option to the script. It can be useful to generate text reports from arbitrary json reports, eg:
./infer/lib/python/report.py --issues-json infer-out/report.json --project-root . --results-dir infer-out --issues-txt foo.txt
Reviewed By: martinoluca
Differential Revision: D5526683
fbshipit-source-id: 3f9c3ee
Summary: This seems more in line with the expectations of the JSON format.
Reviewed By: mbouaziz
Differential Revision: D5500939
fbshipit-source-id: 76dcc47
Summary: This used to be in a different module but now that `driver_mode` is in `Driver` it should really be called `Driver.mode`.
Reviewed By: mbouaziz
Differential Revision: D5499575
fbshipit-source-id: ab96473
Summary: Automatically set --reactive because the diff analysis will be faster that way.
Reviewed By: jeremydubreil
Differential Revision: D5499443
fbshipit-source-id: 1a35cee
Summary:
This was an oversight. The prologue has to be run before capture and analysis
(see eg Infer.run). This sets up a bunch of things that are useful for the
diff analysis as well.
Reviewed By: jeremydubreil
Differential Revision: D5499405
fbshipit-source-id: 8f339cf
Summary:
If -fembed-bitcode is passed, it leads to multiple cc1 commands, which
try to read .bc files that don't get generated, and fail. So pass
-fembed-bitcode=off to disable.
Reviewed By: martinoluca
Differential Revision: D5516879
fbshipit-source-id: 478057a
Summary:
If you run `infer report --issues-txt ilovecats.txt ...` then bugs may mysteriously miss from `ilovecats.txt`, unless you flush. (See rules of thumb for `Format` module.)
Closes https://github.com/facebook/infer/pull/694
Reviewed By: mbouaziz
Differential Revision: D5442335
Pulled By: jvillard
fbshipit-source-id: 73272a0
Summary:
The python scripts support this option but don't do anything with it. Note:
This doesn't remove support for android harnesses.
Reviewed By: sblackshear
Differential Revision: D5501019
fbshipit-source-id: 14c4e3f
Summary: The `--failures-allowed` was doing for the Clang frontend what `--keep-doing` was doing for the backend. This revision merges the two options to simplify the Infer CLI and our tests.
Reviewed By: jvillard
Differential Revision: D5474347
fbshipit-source-id: 09bcea4
Summary:
update-submodule: facebook-clang-plugins
Moving to a newer version of clang, see ffb5dd0114
Reviewed By: jvillard
Differential Revision: D5452529
fbshipit-source-id: 28bc215
Summary:
Better to bail early than have mysterious failures. These used to have the wrong error messages:
```
$ cd examples/android_hello/
$ infer -- ./gradlew rubbish
Capturing in gradle mode...
Running and capturing gradle compilation...
10:46:56.730 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]
10:46:56.730 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] FAILURE: Build failed with an exception.
10:46:56.730 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]
10:46:56.730 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] * What went wrong:
10:46:56.730 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] Task 'rubbish' not found in root project 'android_hello'.
10:46:56.730 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]
10:46:56.730 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] * Try:
10:46:56.730 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] Run gradlew tasks to get a list of available tasks. Run with --stacktrace option to get the stack trace.
Nothing to compile. Try running `./gradlew clean` first.
ERROR: (Unix.Exit_or_signal (Exit_non_zero 66))
$ cd -
$ infer --flavors --Xbuck --keep-going -- buck build //toto:toto
Capturing in buck mode...
moving files in buck-out to buck-out/.trash/buck-outwiiL6N
Not using buckd because watchman isn't installed.
BUILD FAILED: No build file at toto/BUCK when resolving target //toto:toto#infer.
Buck failed, but continuing the analysis because --keep-going was passed
Not using buckd because watchman isn't installed.
No build file at toto/BUCK when resolving target //toto:toto#infer.
Traceback (most recent call last):
File "/home/jul/infer/infer/bin/../lib/python/infer.py", line 186, in <module>
main()
File "/home/jul/infer/infer/bin/../lib/python/infer.py", line 168, in main
capture_exitcode = imported_module.gen_instance(args, cmd).capture()
File "/home/jul/infer/infer/lib/python/inferlib/capture/buck.py", line 89, in capture
return self.capture_with_flavors()
File "/home/jul/infer/infer/lib/python/inferlib/capture/buck.py", line 242, in capture_with_flavors
result_paths = self._get_analysis_result_paths()
File "/home/jul/infer/infer/lib/python/inferlib/capture/buck.py", line 148, in _get_analysis_result_paths
out = [x.split(None, 1)[1] for x in buck_output.strip().split('\n')]
IndexError: list index out of range
ERROR: (Unix.Exit_or_signal (Exit_non_zero 1))
$ cd infer/build_systems/ant/ && infer -- ant rubbish
Capturing in ant mode...
BUILD FAILED
Target "rubbish" does not exist in the project "null".
at org.apache.tools.ant.Project.tsort(Project.java:1929)
at org.apache.tools.ant.Project.topoSort(Project.java:1837)
at org.apache.tools.ant.Project.topoSort(Project.java:1800)
at org.apache.tools.ant.Project.executeTarget(Project.java:1376)
at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
at org.apache.tools.ant.Project.executeTargets(Project.java:1260)
at org.apache.tools.ant.Main.runBuild(Main.java:857)
at org.apache.tools.ant.Main.startAnt(Main.java:236)
at org.apache.tools.ant.launch.Launcher.run(Launcher.java:287)
at org.apache.tools.ant.launch.Launcher.main(Launcher.java:113)
Total time: 0 seconds
Nothing to compile. Try running `ant clean` first.
ERROR: (Unix.Exit_or_signal (Exit_non_zero 66))
```
Now we fail better, for instance:
```
$ infer --flavors --Xbuck --keep-going -- buck build //toto:toto
Capturing in buck mode...
moving files in buck-out to buck-out/.trash/buck-outHag8Ji
Not using buckd because watchman isn't installed.
BUILD FAILED: No build file at toto/BUCK when resolving target //toto:toto#infer.
Buck failed, but continuing the analysis because --keep-going was passed
Not using buckd because watchman isn't installed.
No build file at toto/BUCK when resolving target //toto:toto#infer.
ERROR: (Unix.Exit_or_signal (Exit_non_zero 70))
$ cd infer/build_systems/ant/ && infer -- ant rubbish
Capturing in ant mode...
BUILD FAILED
Target "rubbish" does not exist in the project "null".
at org.apache.tools.ant.Project.tsort(Project.java:1929)
at org.apache.tools.ant.Project.topoSort(Project.java:1837)
at org.apache.tools.ant.Project.topoSort(Project.java:1800)
at org.apache.tools.ant.Project.executeTarget(Project.java:1376)
at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
at org.apache.tools.ant.Project.executeTargets(Project.java:1260)
at org.apache.tools.ant.Main.runBuild(Main.java:857)
at org.apache.tools.ant.Main.startAnt(Main.java:236)
at org.apache.tools.ant.launch.Launcher.run(Launcher.java:287)
at org.apache.tools.ant.launch.Launcher.main(Launcher.java:113)
Total time: 0 seconds
ERROR: couldn't run compilation command `['ant', '-verbose', u'rubbish']`
ERROR: (Unix.Exit_or_signal (Exit_non_zero 1))
```
Reviewed By: jeremydubreil
Differential Revision: D5469607
fbshipit-source-id: a3eb05c
Summary:
The way we represented container writes before was pretty hacky: just use a dummy field for the name of the method that performs the container write.
This diff introduces a new access kind for container writes that is much more structured.
This will make it easier to soundly handle aliasing between containers and support container reads in the near future.
Reviewed By: da319
Differential Revision: D5465747
fbshipit-source-id: e021ec2
Summary: Using a dedicated abstract domain, like Quandary does, is more suitable for taint analysis.
Reviewed By: sblackshear
Differential Revision: D5473794
fbshipit-source-id: c917417
Summary: Because making a diff which breaks the tests because it silently fails to create the right posts for the models is notoriously hard to debug
Reviewed By: sblackshear
Differential Revision: D5471611
fbshipit-source-id: ef04539
Summary:
Both `stringWithUTF8String` and `stringWithString` implements copy semantics that copies the content of their parameter into a newly allocated buffer. We modeled this as pointer assignment in the past, which means that once we write
```
NSString* foo() {
char buf[...];
...
return [NSString stringWithUTF8String:buf];
}
```
We are going to get a spurious stack variable address escape report because local pointer `buf` is assigned to the newly created string and the string gets returned.
This diff tries to address the issue by heap-allocating a buffer and `memcpy` the contents in `stringWithUTF8String` and `stringWithString`. But this change will create another problem: the allocated buffer will be reported as leaked by the backend, while in reality those buffers won't actually be leaked as they are allocated in a region that will be periodically autoreleased. To suppress spurious memory leak FPs, I added another attribute `Awont_leak` that will suppress the leakage report on any expressions that get tagged with it.
Reviewed By: jeremydubreil
Differential Revision: D5403084
fbshipit-source-id: df6de7f
Summary: This gives us more expressive power when defining sources--we can use heuristics like "`foo(o)` only returns a source when `o` is not a constant".
Reviewed By: jvillard
Differential Revision: D5467935
fbshipit-source-id: f3d581d
Summary: This should make it faster to use embarassingly big regexps in AL.
Reviewed By: mbouaziz
Differential Revision: D5462938
fbshipit-source-id: 0431730
Summary:
Pretty basic: warn when we see an assignment instruction `x = ...` and `x` is not live in the post of the instruction.
Only enabled for Clang at the moment because linters already warn on this for Java. But we can enable it later if we want to (should be fully generic).
Reviewed By: jeremydubreil
Differential Revision: D5450439
fbshipit-source-id: 693514c
Summary:
oops_house
This was introduced when upgrading Core, as `Unix.readdir` became deprecated in
favour of `Unix.readdir_opt` but the exception-based logic was left unchanged.
Reviewed By: mbouaziz
Differential Revision: D5461353
fbshipit-source-id: c0994c3
Summary:
Phony dependencies are not good for incrementality. Retain the nice
factorization of code introduced by D5415318 but depend directly on the
relevant files instead of the phony target.
Reviewed By: mbouaziz
Differential Revision: D5461188
fbshipit-source-id: 069519b
Summary: Making it simple to add a new access type for "un-annotated interface call" in an upcoming diff.
Reviewed By: da319
Differential Revision: D5445914
fbshipit-source-id: f29e342
Summary: We get the wrong answer on most of them for now, but that is expected
Reviewed By: ngorogiannis
Differential Revision: D5429242
fbshipit-source-id: 4899079
Summary:
The previous version
- assumed that `infer` was in `PATH`
- would pick any old `infer` in `PATH` and not necessarily the right version
Reviewed By: mbouaziz
Differential Revision: D5433624
fbshipit-source-id: a59be3d
Summary: Long lists will be broken into lines rather than stretch out on a single line.
Reviewed By: jvillard
Differential Revision: D5424323
fbshipit-source-id: 115f16a
Summary:
This commit avoids precision loss on pruning.
// x -> [s$1, s$2]
if(x) { ... }
// x -> ?
before: x -> [min(0, s$1), max(0, s$2)]
because two x values, [0, 0] (true case) and [s$1, s$2] (false case), were joined after the if branch.
after: x -> [s$1, s$2]
Reviewed By: mbouaziz
Differential Revision: D5431009
fbshipit-source-id: 14a9efe
Summary:
Infer is only using clang's AST which means it can ignore flags passed to LLVM backend via `-mllvm` flag.
It will make it easier to maintain compatibility between different versions of clang/LLVM (ie. when flag is not available in LLVM used by infer)
Reviewed By: jvillard
Differential Revision: D5423685
fbshipit-source-id: a9de681
Summary: Changing .inferconfig can change the analysis results, so we want the cache to get invalidated when .inferconfig changes.
Reviewed By: jeremydubreil
Differential Revision: D5419734
fbshipit-source-id: 01ad874
Summary:
This just makes the warnings silent for now. We may improve the analysis to check if the null check on the captured fields are consistent with the annotation on the corresponding parameters.
Eradicate also has the same issue. I added a test to outline this. The biabduction analysis will also probably fail on the same of annotation lookup. We may want implement the proper fix at the level of `Annotation.field_has_annot`.
Reviewed By: sblackshear
Differential Revision: D5419243
fbshipit-source-id: 6460de8
Summary: This way, if infer crashes then the log files will be available for inspection, but if it succeeds we still get rid of the (non-deterministic) logs before Buck caches the results.
Reviewed By: sblackshear
Differential Revision: D5416032
fbshipit-source-id: 22e58be
Summary:
Fixes 2 issues with the build:
- The OCaml dependencies (Version.ml, atdgen-generated stuff, ...) would be generated twice when running `make -j test` from a clean tree, because `make` has no inter-Makefile visibility over common dependencies.
- We would run `atdgen` twice: once for the .ml, once for the .mli, even though one invocation is enough for both.
Add `make ocaml_clean` to nuke only the OCaml code (in particular because
rebuilding the C++ code in the plugin is slow when all I want to get rid of the
OCaml stuff).
Reviewed By: mbouaziz
Differential Revision: D5415318
fbshipit-source-id: 16f42c6
Summary:
First steps towards implementing diff analysis functionalities inside infer
itself. What works: run infer, checkout parent, re-run infer, checkout top
revision, compute the reportdiff (but no final surfacing on the console). Lots
of TODO still, inlined in the code.
Reviewed By: jberdine
Differential Revision: D5364226
fbshipit-source-id: 5b7f9a5
Summary:
Allowing the user to configure where to store the JSON report is asking for
trouble. In fact, some places in the code hardcoded "results.json" anyway.
Someone wanting to have results.json somewhere else can still copy report.json
once infer has run.
Reviewed By: jberdine
Differential Revision: D5415079
fbshipit-source-id: 9439cb6
Summary:
This reverts commit 7177bb6b8af1404fde408ead4750ef48ed9bdaff.
In practice, `extract_filepath` exhausts the call stack, leading to "RuntimeError: maximum recursion depth exceeded" exceptions. Perhaps a loop needs to be used instead.
Reviewed By: jvillard
Differential Revision: D5409892
fbshipit-source-id: 09ccccd
Summary: CFG nodes were not connected and some instructions ended up in wrong place. Fix those issues
Reviewed By: dulmarod
Differential Revision: D5406720
fbshipit-source-id: 2a70e1a
Summary: Introduce `Logging.die` to try and exit with consistent error codes depending on what failed.
Reviewed By: mbouaziz
Differential Revision: D5406642
fbshipit-source-id: 25d98fc
Summary: This will allow us to gradually get rid of the exceptions thrown during the analysis while detecting the regressions earlier
Reviewed By: jberdine, jvillard
Differential Revision: D5385154
fbshipit-source-id: 605e3f5
Summary:
Problem: The analyzer did not know that the value of `v.size()` is an alias of `v.infer_size`, so `v.infer_size` is not pruned by the if condition. As a result it raises a false alarm.
void safe_access(std::vector<int> v) {
if (v.size() >= 10) {
v[9] = 1; // error: BUFFER_OVERRUN Offset: [9, 9] Size: [5, 5]
}
}
void call_safe_access_Good() {
std::vector<int> v(5, 0);
safe_access(v);
}
Solution: Adding alias for return value to the abstract domain.
Now Inferbo can prune `v.infer_size` because it knows that the value of `v.size()` is an alias of `v.infer_size`. There is already an alias domain in Inferbo, so we added a specific room for the retrun value.
Reviewed By: jvillard, mbouaziz
Differential Revision: D5396988
fbshipit-source-id: 4a4702c
Summary:
1. Allow it to forward -version calls to javac (so that if the build system wants
to check the compiler version it doesn't end up with Infer's version)
2. Preserve empty arguments:
javac -bootclasspath "" -classpath a.jar ...
currently gives:
javac: invalid flag: a.jar
Closes https://github.com/facebook/infer/pull/688
Differential Revision: D5406058
Pulled By: jvillard
fbshipit-source-id: 7fa1bcf
Summary:
Conversion and reformat of infer source using ocamlformat
auto-formatting tool.
Current status:
- Because Reason does not handle docstrings, the output of the
conversion is not 'Warning 50'-clean, meaning that there are
docstrings with ambiguous placement. I'll need to manually fix
them just before landing.
Reviewed By: jvillard
Differential Revision: D5225546
fbshipit-source-id: 3bd2786
Summary:
Add support for converting without actually converting. To be used as
a landing pad for rebasing over the conversion.
Reviewed By: jvillard
Differential Revision: D5379967
fbshipit-source-id: 1bd932d
Summary: Adding to the Quandary tests, the list of tests that are already working for the bi-abduction based taint analysis.
Reviewed By: sblackshear
Differential Revision: D5395734
fbshipit-source-id: c4f2e79
Summary:
Error'ing when getting several subcommands on the command line only makes sense
when the user typed these options herself. This was causing a bug in the Gradle
integration, and possibly others (see GitHub issue).
Fixes#668
Reviewed By: jeremydubreil
Differential Revision: D5381850
fbshipit-source-id: f332377
Summary:
Fixes#683
Disclaimer: I don't know what XML namespaces/namespace prefixes are, but this seems to do the trick.
Reviewed By: mbouaziz
Differential Revision: D5381480
fbshipit-source-id: 3da16e7
Summary: This will allow to replace type vars into concrete types in expressions.
Reviewed By: jvillard, mbouaziz
Differential Revision: D5209276
fbshipit-source-id: c1650f8
Summary:
The thread safety domain manipulates access paths that are a variable
followed by a sequence of field or index accesses. Some expressions
from C++ code do not fit that form, such as cases where subtraction of
an offset from a pointer is used to obtain another pointer, whose
fields are then accessed. Previously the analyzer would crash on such
expressions. This diff partially treats them by introducing dummy
variables.
Reviewed By: da319
Differential Revision: D5343567
fbshipit-source-id: f73b520
Summary:
:
because otherwise people would believe they can use the internal representation of these std lib but it fails for our models.
Reviewed By: jvillard
Differential Revision: D5368671
fbshipit-source-id: 4e53d5a
Summary: So that it get rebuilt. Test is clang_compilation_db_relpath
Reviewed By: mbouaziz
Differential Revision: D5364561
fbshipit-source-id: 8cae984
Summary:
:
Get rid of model location in reports.
The goal is to avoid changing `issues.exp` whenever a model is updated.
Reviewed By: jvillard
Differential Revision: D5356608
fbshipit-source-id: 88ecaba
Summary: This could generate enormous amounts of logs on medium-sized projects (eg rocksdb) when running with `--stats`. Review other debug levels in inferbo as well.
Reviewed By: mbouaziz
Differential Revision: D5364139
fbshipit-source-id: 5c8d206
Summary:
In C++ it may happen that a procedure return 'variable' is an access
path that cannot be translated to Hil. For now just skip these instead
of crashing.
Reviewed By: mbouaziz
Differential Revision: D5356134
fbshipit-source-id: 977dfba
Summary:
They are expected to occur in C++ code, so don't fail on them. For now
just skip them, although a better treatment of dynamic dispatch may be
needed later.
Reviewed By: da319, mbouaziz
Differential Revision: D5292462
fbshipit-source-id: 4285514
Summary:
Indexing into a string literal expression would generate a fresh
variable on every application of a transformer. This violated
finiteness of the domain, and caused divergence.
Reviewed By: da319
Differential Revision: D5342951
fbshipit-source-id: e95e84e
Summary: It was passing the argument multiple times rather than once with all the accepted values
Reviewed By: jvillard
Differential Revision: D5338509
fbshipit-source-id: 22b86a5
Summary: This is useful when we only want to examine the html for some but not all source files.
Reviewed By: jvillard
Differential Revision: D5334786
fbshipit-source-id: 774c23c
Summary:
It instantiates fields of structures when a pointer to which is given
as a function parameter, e.g., `foo(&s);`.
Reviewed By: mbouaziz, jvillard
Differential Revision: D5337645
fbshipit-source-id: c06da29
Summary: Rename historical option to its new form, since the old form was no longer accepted by infer.
Reviewed By: martinoluca
Differential Revision: D5329033
fbshipit-source-id: 4fa9402
Summary:
We keep track of both `beginPtr` and `endPtr` but the modelling was mostly
about `beginPtr` as some sort of approximation I guess. This shouldn't change
much but will be useful later when doing more iterator stuff.
Reviewed By: mbouaziz
Differential Revision: D5255772
fbshipit-source-id: 0f6e3e8
Summary:
This will be needed to re-use the functions now in Driver.ml in other contexts
without always adding to infer.ml. For instance, this is used in a later diff
to do a diff-analysis orchestrator that needs to run the capture and analysis
several times.
Reviewed By: jberdine
Differential Revision: D5319862
fbshipit-source-id: caf9551
Summary: Needed in a later diff to be able to compute the set of changed files *during* an infer execution.
Reviewed By: jberdine
Differential Revision: D5319667
fbshipit-source-id: 226ec91
Summary: This seems to move in the right direction. Also, `const operator[]` did not do an `access_at`, which I fixed.
Reviewed By: mbouaziz
Differential Revision: D5320427
fbshipit-source-id: c31c5ea
Summary: Unknown library returns the unknown pointer as well as the top interval.
Reviewed By: mbouaziz, jvillard
Differential Revision: D5282669
fbshipit-source-id: 34c7e18
Summary:
Thanks to the logging introduced in D5293334 (or because of, depending on your liking of spam), I noticed that there lots of errors being logged of the form
Couldn't read multilink file '/home/jul/infer/infer/tests/codetoanalyze/cpp/errors/infer-out/attributes/d0/multilink.txt': /home/jul/infer/infer/tests/codetoanalyze/cpp/errors/infer-out/attributes/d0/multilink.txt: No such file or directory
I don't think it makes sense to care about multilink files except when `--merge` is specified.
Also introduced a .mli and self-documented a function.
Reviewed By: akotulski
Differential Revision: D5310129
fbshipit-source-id: c3a6276
Summary:
Once the fixed/preexisting/introduced sets have been computed, they endure
further filtering which may decide that more of them are equal. These bugs just
get dropped on the floor. Put these into preexisting as well instead, at least
in the case of the "skip_duplicated_types_on_filenames" filter.
Reviewed By: martinoluca
Differential Revision: D5274248
fbshipit-source-id: 99b3f3d
Summary:
This diff tries to achieve the followings: if we have the following C++ codes:
```
bool foo(int x, int y) {
return &x == &y;
}
```
We want the C++ frontend to emit Sil as if the input is written as
```
bool foo(int x, int y) {
if (&x == &y) return 1; else return 0;
}
```
This matches the behavior of our Java frontend.
The reason why we prefer an explicit branch is that it will force the backend to eagerly produce two different specs for `foo`. Without the explicit branch, for the above example the backend would produce one spec with `return = (&x == &y)` as the post condition, which is not ideal because (1) we don't want local variables to escape to the function summary, and (2) with the knowledge that no two local variables may alias each other, the backend could actually determines that `&x == &y` is always false, emitting a more precise postcondition `return = 0`. This is not possible if we do not eagerly resolve the comparison expression.
Reviewed By: akotulski
Differential Revision: D5260745
fbshipit-source-id: 6bbbf99
Summary:
:
There are throw wrapper functions like `std::__throw_bad_alloc()` defined in both libstdc++ (https://github.com/gcc-mirror/gcc/blob/master/libstdc%2B%2B-v3/include/bits/functexcept.h) and libc++ (e.g. 907c1196a7/include/new (L145)). Folly actually exports some of them as well (diffusion/FBS/browse/master/fbcode/folly/portability/BitsFunctexcept.h). The function body of those wrappers merely throws the corresponding exception. My understanding is that the primary purpose of the wrappers is to throw the exception if everything goes well and to fall back to something reasonable when exception is disabled (e.g. when `-fno-exceptions` is passed to the compiler).
The problem is that infer doesn't really understand what those functions do, and I've seen some false positives get reported as a result of it. So to remove those FPs we need to either model them or handle them specially. Modeling those wrappers by either whitelisting them or overriding the include files turns out to be difficult, as those wrappers are only declared but not defined in the STL headers. Their implementations are not available to Infer so whitelisting them does nothing, and if I provide custom implementations in the headers then normal compilation process will be disrupted because the linker would complain about duplicated implementation.
What I did here is to replace functions whose name matches one of the throw wrapper's name with a `BuiltinDecls.exit`. I have to admit that this is a bit hacky: initially I was trying to do something more general: replacing functions with `noreturn` attribute with `BuitinDecls.exit`. That did not work because, CMIIW, the current frontend only exports function attributes for functions with actual bodies, not declaration-only functions. I'd love to be informed if there are better ways to handle those wrappers.
Reviewed By: jeremydubreil
Differential Revision: D5266030
fbshipit-source-id: 4580227
Sam Blackshear
Dulma Churchill
Jules Villard
Daiva Naudziuniene
Jeremy Dubreil
Greg Nisbet
Martino Luca
Jia Chen
Meng Zhang
Andrzej Kotulski
Alexander Slesarev
Mehdi Bouaziz
Josh Berdine
Radu Grigore
Dino Distefano
Sungkeun Cho
Kyriakos Nikolaos Gkorogiannis
Pavel Grafov
azhdar
Kihong Heo