Summary: Fixing the support for single core analysis in 8ce79a0613 was improving the performance of the genrule-based integration by more that 2X. Running Infer in single core more with the external compiler integration for Buck also improves the performance significantly.
Reviewed By: mbouaziz
Differential Revision: D7989255
fbshipit-source-id: 71fb842
Summary:
Previously, the command parsing code filtered out any elements that were `False`y in the Python sense (starting with D5507925).
This meant that a command like `javac -classpath '' -Xmaxerrs 1000 MyFile.java` would be parsed as `javac -classpath -Xmaxerrs 1000 MyFile.java`, which will fail with "unrecognized option 1000".
This diff removes the filtering and fixes that issue.
Reviewed By: jeremydubreil
Differential Revision: D7797560
fbshipit-source-id: 63df06b
Summary:
It's already turned of systematically for the Java integration, this just
generalises it. The Buck daemon seems to cause issues with infer from time to
time that are hard to debug.
Reviewed By: jeremydubreil
Differential Revision: D7400068
fbshipit-source-id: f05ee07
Summary:
This is to fix the conflicts between Eradicate and the Biabduction when reporting the same kind of errors: when Eradicate is on, the Eradicate warnings will have priority over the null deference reported by the biabduction.
If this approach proved to be successful in prod, I will refactor the reporting mechanism in the analysis itself to simply not report the null dereference in this case at all. For the codebases that aren't yet fully consistently using `Nullable`, this combined approach looks like a good way to deploy Infer toward full null safety.
Reviewed By: mbouaziz
Differential Revision: D7102119
fbshipit-source-id: 35d3add
Summary:
Instead of storing the type environment in infer-out/captured/foo.c/foo.c.tenv,
store it in the `source_files` table of the SQLite db. This limits the number
of files we create on disk.
The "file local" type environemnts are specific to the clang integration. For
Java, there is a "global tenv" file. Instead of matching on string names, this
diff also makes the API of `Tenv` reflect this situation.
The global tenv is serialized as a separate file in "infer-out/.global.tenv"
instead of "infer-out/captured/global.tenv", because "infer-out/captured/" will
soon be removed as it now only contains the global tenv (except in debug mode,
where it will still be created).
In the DB, we either store the local tenv for the file, or "global" to indicate
that the global tenv should be consulted.
This diff also moves `Cfg.store` to `SourceFiles.add` because that function
deals with more than just `Cfg.t`.
Reviewed By: jeremydubreil
Differential Revision: D6937945
fbshipit-source-id: 001c10a
Summary:
In buck queries, the `regex` in `kind(regex, ...)` is open, it can match superstrings, like `prebuilt_cxx_library` when we want `cxx_library` instead.
This adds `^` and `$` to our existing kind filtering.
Also optimizes the query in `buck_target_determinator.py`.
Reviewed By: dulmarod
Differential Revision: D6722665
fbshipit-source-id: 22d839f
Summary: A source can belong to more than one target. In this case, we should keep only one of the report.
Reviewed By: sblackshear
Differential Revision: D6200058
fbshipit-source-id: 4eced42
Summary: Saving the list of bugs in a set removes the ordering. Also, there should be no need to remove the duplicated warnings at this level.
Reviewed By: sblackshear
Differential Revision: D6060554
fbshipit-source-id: a78d35d
Summary:
Fix for the #669. Recursion problem has been reproduced (see the new doctest in the latest commit) and eliminated. I guess the new version will work faster too (no more unnecessary slicing).
Closes https://github.com/facebook/infer/pull/697
Reviewed By: sblackshear
Differential Revision: D5507925
Pulled By: jvillard
fbshipit-source-id: 21fa07b
Summary:
Replace `inferTraceBugs` with `infer-explore` with a similar CLI. Some options changed:
- --max-level -> --max-nesting, and "max" is the default value instead of a possible value
- --no-source -> --no-source-preview
Reviewed By: mbouaziz
Differential Revision: D5526651
fbshipit-source-id: 8383f37
Summary: That was too noisy. Propagate `--quiet` to the Python reporting hook so as to still emit bugs.txt and so on.
Reviewed By: martinoluca
Differential Revision: D5501106
fbshipit-source-id: 63b6451
Summary:
This was hardcoded but it's passed as an option to the script. It can be useful to generate text reports from arbitrary json reports, eg:
./infer/lib/python/report.py --issues-json infer-out/report.json --project-root . --results-dir infer-out --issues-txt foo.txt
Reviewed By: martinoluca
Differential Revision: D5526683
fbshipit-source-id: 3f9c3ee
Summary:
The python scripts support this option but don't do anything with it. Note:
This doesn't remove support for android harnesses.
Reviewed By: sblackshear
Differential Revision: D5501019
fbshipit-source-id: 14c4e3f
Summary:
Better to bail early than have mysterious failures. These used to have the wrong error messages:
```
$ cd examples/android_hello/
$ infer -- ./gradlew rubbish
Capturing in gradle mode...
Running and capturing gradle compilation...
10:46:56.730 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]
10:46:56.730 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] FAILURE: Build failed with an exception.
10:46:56.730 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]
10:46:56.730 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] * What went wrong:
10:46:56.730 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] Task 'rubbish' not found in root project 'android_hello'.
10:46:56.730 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter]
10:46:56.730 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] * Try:
10:46:56.730 [ERROR] [org.gradle.internal.buildevents.BuildExceptionReporter] Run gradlew tasks to get a list of available tasks. Run with --stacktrace option to get the stack trace.
Nothing to compile. Try running `./gradlew clean` first.
ERROR: (Unix.Exit_or_signal (Exit_non_zero 66))
$ cd -
$ infer --flavors --Xbuck --keep-going -- buck build //toto:toto
Capturing in buck mode...
moving files in buck-out to buck-out/.trash/buck-outwiiL6N
Not using buckd because watchman isn't installed.
BUILD FAILED: No build file at toto/BUCK when resolving target //toto:toto#infer.
Buck failed, but continuing the analysis because --keep-going was passed
Not using buckd because watchman isn't installed.
No build file at toto/BUCK when resolving target //toto:toto#infer.
Traceback (most recent call last):
File "/home/jul/infer/infer/bin/../lib/python/infer.py", line 186, in <module>
main()
File "/home/jul/infer/infer/bin/../lib/python/infer.py", line 168, in main
capture_exitcode = imported_module.gen_instance(args, cmd).capture()
File "/home/jul/infer/infer/lib/python/inferlib/capture/buck.py", line 89, in capture
return self.capture_with_flavors()
File "/home/jul/infer/infer/lib/python/inferlib/capture/buck.py", line 242, in capture_with_flavors
result_paths = self._get_analysis_result_paths()
File "/home/jul/infer/infer/lib/python/inferlib/capture/buck.py", line 148, in _get_analysis_result_paths
out = [x.split(None, 1)[1] for x in buck_output.strip().split('\n')]
IndexError: list index out of range
ERROR: (Unix.Exit_or_signal (Exit_non_zero 1))
$ cd infer/build_systems/ant/ && infer -- ant rubbish
Capturing in ant mode...
BUILD FAILED
Target "rubbish" does not exist in the project "null".
at org.apache.tools.ant.Project.tsort(Project.java:1929)
at org.apache.tools.ant.Project.topoSort(Project.java:1837)
at org.apache.tools.ant.Project.topoSort(Project.java:1800)
at org.apache.tools.ant.Project.executeTarget(Project.java:1376)
at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
at org.apache.tools.ant.Project.executeTargets(Project.java:1260)
at org.apache.tools.ant.Main.runBuild(Main.java:857)
at org.apache.tools.ant.Main.startAnt(Main.java:236)
at org.apache.tools.ant.launch.Launcher.run(Launcher.java:287)
at org.apache.tools.ant.launch.Launcher.main(Launcher.java:113)
Total time: 0 seconds
Nothing to compile. Try running `ant clean` first.
ERROR: (Unix.Exit_or_signal (Exit_non_zero 66))
```
Now we fail better, for instance:
```
$ infer --flavors --Xbuck --keep-going -- buck build //toto:toto
Capturing in buck mode...
moving files in buck-out to buck-out/.trash/buck-outHag8Ji
Not using buckd because watchman isn't installed.
BUILD FAILED: No build file at toto/BUCK when resolving target //toto:toto#infer.
Buck failed, but continuing the analysis because --keep-going was passed
Not using buckd because watchman isn't installed.
No build file at toto/BUCK when resolving target //toto:toto#infer.
ERROR: (Unix.Exit_or_signal (Exit_non_zero 70))
$ cd infer/build_systems/ant/ && infer -- ant rubbish
Capturing in ant mode...
BUILD FAILED
Target "rubbish" does not exist in the project "null".
at org.apache.tools.ant.Project.tsort(Project.java:1929)
at org.apache.tools.ant.Project.topoSort(Project.java:1837)
at org.apache.tools.ant.Project.topoSort(Project.java:1800)
at org.apache.tools.ant.Project.executeTarget(Project.java:1376)
at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
at org.apache.tools.ant.Project.executeTargets(Project.java:1260)
at org.apache.tools.ant.Main.runBuild(Main.java:857)
at org.apache.tools.ant.Main.startAnt(Main.java:236)
at org.apache.tools.ant.launch.Launcher.run(Launcher.java:287)
at org.apache.tools.ant.launch.Launcher.main(Launcher.java:113)
Total time: 0 seconds
ERROR: couldn't run compilation command `['ant', '-verbose', u'rubbish']`
ERROR: (Unix.Exit_or_signal (Exit_non_zero 1))
```
Reviewed By: jeremydubreil
Differential Revision: D5469607
fbshipit-source-id: a3eb05c
Summary:
This reverts commit 7177bb6b8af1404fde408ead4750ef48ed9bdaff.
In practice, `extract_filepath` exhausts the call stack, leading to "RuntimeError: maximum recursion depth exceeded" exceptions. Perhaps a loop needs to be used instead.
Reviewed By: jvillard
Differential Revision: D5409892
fbshipit-source-id: 09ccccd
Summary:
It is not used, so one fewer thing to track inside Python.
Note that Python does pass `--buck` to OCaml infer to make it aware that it is
running from within Buck, and the OCaml code uses that flag to trim infer-out
to only what's strictly necessary (and especially to remove non-deterministic
output that would mess with Buck's cache).
Reviewed By: jberdine
Differential Revision: D5200043
fbshipit-source-id: 1c84442
Summary: This also allows us to better test that the new commands will keep working.
Reviewed By: jeremydubreil
Differential Revision: D5172891
fbshipit-source-id: 169bd6f
Summary: There were some leftover uses of the `Tracing` analyzer option. While I was at it, I also rename the `Config` option name.
Reviewed By: jberdine
Differential Revision: D5113489
fbshipit-source-id: 68d5cc8
Summary:
Try and enforce the following rules:
- stderr is for updating the user about progress or errors
- Introduce Logging.progress that outputs to stderr, but honours --quiet
- Logging.stderr is as before
- Logging.out now prints to stderr (or to log files as before if set up) and
not stdout. If some information should go on stdout then the user should be
able to rely on it (ie, it's not just some progress message). For now only
the summary of the errors is printed on stdout by default.
- Logging.err* functions are gone. If the error is user-visible, it should be
Logging.stderr, or `failwith`. If not, go to the same log file as other
output, which personally I find much more convenient than having to dig through
2 log files every time I'm looking for some output.
Reviewed By: jberdine
Differential Revision: D5095720
fbshipit-source-id: 68999c9
Summary: The Buck cache was not correctly invalidated when switching between the different analysis mode. This was causing the analysis results to be mixed up. This revision should fix this.
Reviewed By: sblackshear
Differential Revision: D5073606
fbshipit-source-id: eb14418
Summary:
The code was pretty fragile, it's less fragile now. We should probably just
delete that and start outputting proper class/package info directly in the
report.json instead of reverse-engineering them.
Fixes#640
Reviewed By: jeremydubreil
Differential Revision: D4905973
fbshipit-source-id: 1590067
Summary:
Introduce `infer-<command>` for each command, except for internal commands
(only `infer-clang` for now) which are not exported. Install these executables
(which are just symlinks to `infer`) on `make install`. The main executable
looks at the name it was invoked with to figure out if it should behave as a
particular command.
Get rid of `InferClang`, `InferAnalyze`, and `InferPrint`. As a bonus, we now
only need to build one executable: `infer`, which should be a few seconds
faster (less link time).
`InferAnalyze` is now `infer-analyze` and `InferPrint` is `infer-print`. To run
`InferClang`, use a symlink named `clang`, `clang++`, etc. to `infer`. There
are such symlinks available in "infer/lib/wrappers/" already.
I also noticed that the scripts in xcodebuild_wrappers/ don't seem useful
anymore, so use wrappers/ instead, as for `make`.
Reviewed By: mbouaziz
Differential Revision: D5036495
fbshipit-source-id: 4a90030
Summary:
The error was complaining about a missing `etree` package but that's actually
part of the `lxml` package and that wasn't clear from the output.
See for instance https://github.com/facebook/infer/issues/635
Reviewed By: martinoluca
Differential Revision: D4905953
fbshipit-source-id: 28b26af
Summary:
This option is not needed anymore as it was introduced to counter an uncovered
perf issue with creating human readable reports. The perf issue has been
addressed.
Instead of this option, one can use `infer --report-hook /bin/true ...` to
disable reporting. However, right now the Buck integration doesn't honor it so
this would need to be fixed to be a true equivalent of `--disable-bug-list`.
Reviewed By: jberdine
Differential Revision: D4712877
fbshipit-source-id: a09304f
Jeremy Dubreil
Sam Blackshear
Jules Villard
Tony Hu
Mehdi Bouaziz
Alexander Slesarev
Josh Berdine
Martino Luca
Don Stewart
Dulma Churchill
htxs