Summary:
This changes executions of the former InferClang into a function call. In
particular, it can be called several times per execution.
The new InferClang must be called as if it was clang, and knows how to run
clang with our plugin to get the AST of the source file.
Reviewed By: akotulski
Differential Revision: D3981017
fbshipit-source-id: 7af6490
Summary: Infer should always include the version of the Java compiler as part of the target key used by Buck
Reviewed By: jvillard
Differential Revision: D3989649
fbshipit-source-id: 605ab2f
Summary:
Adds support for running the analysis using Java compilation commands of the form:
java -jar compiler.jar ...
Reviewed By: sblackshear
Differential Revision: D3971517
fbshipit-source-id: 3659216
Summary:
During the incremental analysis using the Buck distributed cache, if there is a cache hit for a given module, the output jars for the intermediate targets are not necessarily dowloaded. We therefore need to filter the jar files that are present on disk before loading the analysis artifacts from it.
This will also be neccesary when combined with the --keep-going option of Buck
Reviewed By: sblackshear
Differential Revision: D3941853
fbshipit-source-id: befda63
Summary:
Before this diff, Infer was simply going through the list of jar files found in `buck-out` and was loading all the `report.json` files found in those jar files in order to merge them into a final report. The main drawback of this was that removing `buck-out` was mandatory to get accurate results when switching between targets to analyze.
With this diff, we now use the `buck audit classpath` option to get from a list of targets, the list of jar files to load the `report.json` files from. This allows to more easily use Infer from the command line when switch branches from the repositories or when switching between targets to analyze.
Reviewed By: martinoluca
Differential Revision: D3922548
fbshipit-source-id: ec550fa
Summary: The reactive mode is not currently supported for Java Buck projects. Print a clear error message to avoid confusion. Using `--reactive` with Buck projects is crashing. When using reactive, the backend crashes because it tries to find the `.start` file in `infer-out` whereas Infer saves the analysis artifacts next to the generated classes following the `-d` options with Buck so that they can be cached. Pointing the backend to the right Infer out would not help as this directory will be blown up as soon as the files are modified.
Reviewed By: sblackshear
Differential Revision: D3901005
fbshipit-source-id: d8dcbce
Summary: We no longer need to disable the Buck daemon since we not pass the path to the compiler via the option `--config` instead of modifying the .buckconfig.local file
Reviewed By: sblackshear
Differential Revision: D3854109
fbshipit-source-id: 1126159
Summary:
This was causing headaches as catching error code 2 may hide real issues.
While there, move crashcontext finalizer code to crashcontext.ml, and create a
.mli file for that module.
Reviewed By: jberdine
Differential Revision: D3742785
fbshipit-source-id: 3032451
Summary:
Infer's mvn integration would collect *all* files in `src_roots` and pass them
to `javac`. But these folders may contain non-Java files.
Filter out any file that doesn't end in `.java`.
closes#401closes#418closes#442
Reviewed By: martinoluca
Differential Revision: D3741893
fbshipit-source-id: 5c489f9
Summary:
On wrong arguments (or on no arguments at all), `infer` would spew the error
message of `infer.py`, which makes no sense. Make the python code swallow error
messages and exit with a special code on errors coming from command line
parsing so that the OCaml side is in charge of printing usage messages.
Reviewed By: cristianoc
Differential Revision: D3731594
fbshipit-source-id: fe49cda
Summary:
Adding a new mode linters. Now if the analyzer is linters, we do the linters and don't translate,
then, if the analyzer is Infer, we do the translation and the backend and not the linters checks, and the
default is that we do capture, backend and lint checks.
Made the tests separated, which saves time and also shows that the linters mode works.
Reviewed By: jvillard
Differential Revision: D3723472
fbshipit-source-id: 9d828d8
Summary: Emit a simple info message on terminal whenever aggregation of stats fails for whatever reason, details about the failure will still be available in `toplevel.log`
Reviewed By: jvillard
Differential Revision: D3516339
fbshipit-source-id: fe24d64
Summary:
This diff changes the toplevel 'infer' executable from the current
python script to an OCaml binary. Currently this executable only parses
command line arguments, sets up environment variables, and invokes the
existing python script. This improves infer's command-line and
configuration interface, since passing arguments to the frontends or
backend no longer requires manually setting environment variables, and
arguments for the toplevel can now also be specified in .inferconfig.
Simplification and migration of functionality from the python script is
left for the future.
Reviewed By: martinoluca, jvillard
Differential Revision: D3450662
fbshipit-source-id: 1b52302
Summary: Infer should be responsible for ensuring that `-j` is respected.
Reviewed By: jberdine
Differential Revision: D3450064
fbshipit-source-id: 5286a6a
Summary:
buck #infer flavor doesn't rely on .o files so we don't need to generate them.
It will save memory/IO and possibly time
Reviewed By: jvillard, martinoluca
Differential Revision: D3379463
fbshipit-source-id: 1d48f7a
Summary:
Run `InferStatsAggregator` at the end of the execution of `infer` top-level, and store
results on `infer-out/(frontend|backend|reporting)_stats/aggregated_stats.json`
Not ready yet for buck targets analyzed without the use of an `#infer*` flavor.
Reviewed By: jvillard
Differential Revision: D3365504
fbshipit-source-id: 98b2eb3
Summary: It is preferable to always create error reports as a proof of succesful termination, even when the Buck target contains no source file to analyze
Reviewed By: sblackshear, jvillard
Differential Revision: D3362581
fbshipit-source-id: 4f27666
Summary:
When syntax highlighting the source excerpts that Infer prints on stdout, we
would crash if `pygments.lexers` did not find a suitable class given the name
of the source file. Instead, do not colorize when that's the case.
Reviewed By: martinoluca
Differential Revision: D3358115
fbshipit-source-id: ccb9b41
Summary:
Make analyzer find out when null dereference comes from std::vector method.
If it does, it means that it's really empty vector access (due to the
way infer models std::vector)
Reviewed By: sblackshear
Differential Revision: D3327933
fbshipit-source-id: b9e11d6
Summary: This way we don't need to make the file be valid json later on.
Reviewed By: jeremydubreil
Differential Revision: D3304998
fbshipit-source-id: 25deb5c
Summary:
The checkers check was causing perf issues because it kept loading the json of
inferconfig. To prevent this from happening again, load json files inside
config.ml, and only export `Yojson.Basic.json Lazy.t` values to other modules.
Also move the list of checks disabled by default into config.ml for better
discoverability.
Reviewed By: jberdine
Differential Revision: D3293041
fbshipit-source-id: 4a38b26
Summary:
Infer prepends the directory containing the ananotation processor and
the current working directory to the classpath javac option. This diff
enables prepending these to the classpath when it is passing in an args
file (as the classpath can get too long to pass on the command line).
Reviewed By: jvillard
Differential Revision: D3270348
fbshipit-source-id: 208077f
Summary:
- [python] decode strings coming from `os.*` commands
- [python] decode strings coming from the command-line
- [python] encode a few remaining unicodes into strings
- [java] replace lex/yacc parser for javac verbose output by regex-based matching to handle unicode in paths
- [make] random fix of `make test` to have `make clean test` work
- [integration tests] add e2e build integration tests for utf8 in the PWD
Closes#76
Reviewed By: martinoluca
Differential Revision: D3240809
fb-gh-sync-id: 8c2e1ed
fbshipit-source-id: 8c2e1ed
Summary:This enables controlling the encoding chosen by infer via the usual environment
variables. For instance:
```
LC_ALL="C" infer ... # sets LOCALE to "ascii"
LC_ALL="en_US.UTF-8" infer ... # sets LOCALE to "UTF-8"
```
This gives an easy solution to #320: run `LC_ALL="en_US.UTF-8" infer ...`.
Right now the only solution is to edit the Python scripts by hand instead!
Reviewed By: jberdine
Differential Revision: D3207573
fb-gh-sync-id: 62d5b98
fbshipit-source-id: 62d5b98
Summary:`exc.output` can be (always is?) `None`. Other places in the code only print
using `traceback.print_exc()` and this was the only place trying to print this
extra info (`git grep CalledProcessError`). This caused `utils.stdout()` to
raise an exception, which was further confusing.
closes#330
Reviewed By: jberdine
Differential Revision: D3203896
fb-gh-sync-id: d2988d8
fbshipit-source-id: d2988d8
Summary:public
Models in headers are purely C++ concept and it's
useless if C++ mode is turned off. Since those
models can lead to compilation errors, turn them
off for non-C++ analysis
Reviewed By: martinoluca
Differential Revision: D3126294
fb-gh-sync-id: 0912e7b
fbshipit-source-id: 0912e7b
Summary:public
Simplifies the way to configure cmake to run infer's fake compiler commands.
Instead of `CC=/path/to/infer/infer/lib/capture/clang cmake .`, which is what
is advised on #25 and is now outdated because the paths to infer's clang has
changed since, simply run `infer -- cmake .`. The only caveat is that infer
tries to analyze the end result, and prints "No issues found". This could be
fixed later.
Reviewed By: jeremydubreil
Differential Revision: D3093162
fb-gh-sync-id: 99df50a
fbshipit-source-id: 99df50a
Summary:public
Allow the use of `infer -- ./configure`. This can be useful if the full path to
the compiler is recorded by the `./configure` command. That is the case for the
samba source tree for instance (because `./configure` calls `waf configure`
under the hood).
Reviewed By: jeremydubreil
Differential Revision: D3093065
fb-gh-sync-id: 2663418
fbshipit-source-id: 2663418
Summary:public
Eradicate need the procedure attributes for callees.
It relies on the java front-end to create proc descs for callees that are declared but not defined.
This diff remove that needs, and when a callee without prodedure attributes is found, it creates one on the fly. The attribute created is similar to what the Java front-end would do, except
that the number and types of arguments are part of the call instruction, so they can
be used to create the formal parameters.
Reviewed By: jeremydubreil
Differential Revision: D3073904
fb-gh-sync-id: 381ff67
fbshipit-source-id: 381ff67
Summary:public
In Python 2, `shutil.rmtree()`, `os.walk()`, `os.path.join()`, etc. are not
happy when the locale cannot decode the filenames they have to deal with.
Decrease the likelihood of this happening by making the file names generated by
infer ascii-only.
Also ignore character decoding errors optimistically when reading the json
report file.
Add tests that we are able to run the analysis and report the bug on a function
with a utf8 name, and that we are able to remove the previous results
directory.
closes#287
Reviewed By: cristianoc
Differential Revision: D3058858
fb-gh-sync-id: b88cd35
shipit-source-id: b88cd35
Summary:public
It seems that creating the procedure description of the callees this should no longer be useful to run the checkers
Reviewed By: sblackshear
Differential Revision: D3083523
fb-gh-sync-id: 040b104
shipit-source-id: 040b104
Summary:public
The "fake" procedure description of the callees, containing info about the formal parameters, is current used by Eradicate. This diff forces the creation of the callee procedure descriptions when running Infer with Buck.
Reviewed By: sblackshear
Differential Revision: D3083452
fb-gh-sync-id: 24a70e6
shipit-source-id: 24a70e6
Summary:public
It seems that the support for using the Buck cache with Infer has been brocken for a while.
Reviewed By: sblackshear
Differential Revision: D3079493
fb-gh-sync-id: fd92d1c
shipit-source-id: fd92d1c
Summary:public
Whenever infer-deps.txt and report.json files are encountered after the analysis with Buck
they will be automatically merged and deduplicated with all the other files of the same kind.
This change also emits the results of the analysis to stdout.
Reviewed By: jvillard
Differential Revision: D3064487
fb-gh-sync-id: 3599fba
shipit-source-id: 3599fba
Summary:We currently only output the report as JSON or plain text, however other analyzers use XML and there are tools and plugins to process them, for instance TeamCity XML Report Processing plugin.
Author: Deniz Türkoglu <deniz@spotify.com>
Author: Jules Villard <jul@fb.com>
Closes https://github.com/facebook/infer/pull/300
Reviewed By: cristianoc
Differential Revision: D3047181
Pulled By: jvillard
fb-gh-sync-id: 9026ca2
shipit-source-id: 9026ca2
Summary:public
The reactive analysis starts from the set of changed files/procedures, and proceeds
reactively to analyze their dependencies.
This means that after every command, the set of changed files/procedures is reset.
With the --continue option, the capture is continued: all the files/procedures marked
as changed stay changed, plus any additional changes are recorded.
In addition to allowing to spread capture over several commands, the option also allows to separate capture and analysis in reactive mode, or to repeat the analysis.
Reviewed By: sblackshear
Differential Revision: D3046361
fb-gh-sync-id: b6e3797
shipit-source-id: b6e3797
Summary:public
Use the configuration file .inferconfig to model the library method that are considered expensive
Reviewed By: cristianoc
Differential Revision: D3045288
fb-gh-sync-id: e58d85c
shipit-source-id: e58d85c
Summary:public
Simplifies the code to collect the `SuppressWarnings` annotations and makes the code more robust in the sense that not finding the output of the annotation processor will result in an error directly at the top-level instead of later on when trying to load the output file in the Java frontend.
Reviewed By: sblackshear
Differential Revision: D3034690
fb-gh-sync-id: 60caa0c
shipit-source-id: 60caa0c
Summary:public
bugs.txt only contains the summary of each report. The terminal output contains
a bit more information, such as source excerpts. If one wants to save the
terminal output into a file, they can always use shell redirection anyway.
closes#294
Reviewed By: martinoluca
Differential Revision: D3023796
fb-gh-sync-id: 9a21d17
shipit-source-id: 9a21d17
Summary:public
Instead of using the collection of suppress warnings annotations to filter out the errors while generating the error reports, we just add this SuppressWarnings at translation time, like any other annotations, and the reporting functions in the Reporting module will just skip the errors when the method is annotated with SuppressWarnings.
This allows us to have a suppress warnings mechanism that is independant from the integration with the build system.
Reviewed By: sblackshear
Differential Revision: D3012395
fb-gh-sync-id: 35f5f9b
shipit-source-id: 35f5f9b
Summary:public
With this change, all the `infer-deps.txt` files generated by buck for those targets
running with the `#infer` flavor, will be merged into one `infer-deps.txt` located in the
designated output folder.
Reviewed By: jvillard
Differential Revision: D2994397
fb-gh-sync-id: 14d8109
shipit-source-id: 14d8109
Summary:public
Lazy dynamic dispatch handling works as follows:
Assuming a call of the form:
foo(a);
where the static type of `a` is `A`. If during the symbolic execution, the dynamic type of the variable `a` is `B` where `B <: A`, then we create on-demand a copy `foo(B)` of `foo(A)` where all the uses of the typed parameter `a` are replaced with a parameter of type `B`. Especially, if `foo` contains virtual call, say `get` where `a` is the receiver, then the call gets redirected to the overridden method in `B`, which simulates the runtime behavior of Java.
This lazy dynamic dispatch mode is only turn on for the tracing mode for now in order to avoid conflicts with sblackshear's approach for sound dynamic dispatch.
Reviewed By: sblackshear
Differential Revision: D2888922
fb-gh-sync-id: 3250c9e
shipit-source-id: 3250c9e
Summary:public
Deprecate the incremental mode.
Several parts of the back-end can be removed.
The options for incremental analysis -i at the python level are now deprecated, and re-routed to --reactive.
The main difference with --reactive is that it does not produce an analysis of the whole project, but is limited to what is reachable via reactive propagation starting from the changed files.
Reviewed By: sblackshear
Differential Revision: D2960078
fb-gh-sync-id: 6e8b46b
shipit-source-id: 6e8b46b
Summary:public
An observer object that registered to a notification center needs to be
unregistered before it is deallocated.
If not, the notification center may send a notification to a gost object.
This diff introduce a checker for this problem.
Reviewed By: dulmarod
Differential Revision: D2949692
fb-gh-sync-id: 1653cec
shipit-source-id: 1653cec
Summary:public
`print(tracer)` has an automatic conversion to string that was missed when
migrating to `utils.stdout(tracer)`.
Reviewed By: peterogithub
Differential Revision: D2982104
fb-gh-sync-id: 4c26727
shipit-source-id: 4c26727
Summary:public
- s/"/'/ in python strings
- kill `utils.error()` in favour of the new, identical `utils.stderr()`
- one more `print(utils.encode())` to `utils.stderr()` conversion
Reviewed By: jeremydubreil
Differential Revision: D2976710
fb-gh-sync-id: 6c0fdfa
shipit-source-id: 6c0fdfa
Summary:public
This class expects a working `jwlib.CompilerCommand` even when we're not doing
anything Java-related. Split the java-specific functionality into a new child
class in jwlib.py.
Reviewed By: jeremydubreil
Differential Revision: D2965832
fb-gh-sync-id: e895b33
shipit-source-id: e895b33
Summary:public
java-specific code such as this belongs in jwlib.py. It will also help the
refactoring in the next diff.
Reviewed By: sblackshear
Differential Revision: D2965814
fb-gh-sync-id: c3adc03
shipit-source-id: c3adc03
Summary:public
This will avoid a circular dependency between analyze.py and jwlib.py in an
upcoming refactoring.
Reviewed By: martinoluca
Differential Revision: D2965734
fb-gh-sync-id: 1cb69d4
shipit-source-id: 1cb69d4
Summary:public
This attempts to properly sanitise text input/output in the Python parts of
infer. Do three things:
- encode user input (coming from the command-line or reading files)
- decode infer output
- in both cases, we may be using the wrong encoding, eg: locale says we're in
ascii, but the source code contains utf-8. In many cases, like error
messages, it's safe to ignore these encoding mismatches.
Also, since we `import __future__.unicode_literals`, it's safe to remove `u'`
prefixes on many unicode literals.
Reviewed By: martinoluca
Differential Revision: D2960493
fb-gh-sync-id: 9812d7d
shipit-source-id: 9812d7d
Summary:public
Add command-line argument --reactive to enable reactive propagation mode.
When the mode is active, the files changed during compilation are detected, and the analysis propagates reactively starting from the modified files.
The reactive mode allows to analyze a subset of the files in a project and follow their dependencies, without storing the results of previous analyses (specs files). Captured files are preserved from previous runs of the analysis (for example, when the previous analysis was the initial capture), so the mode can be used repeatedly while changing code.
Reviewed By: jvillard
Differential Revision: D2931697
fb-gh-sync-id: 9d6dda0
shipit-source-id: 9d6dda0
Summary:
public
The analysis of the Buck project was failing because this script converts the aliases into their expanded target names. It turns out that for Buck, the name of the command is `buck` and the name of the alias is also `buck`, which led to conflicts. This is now fixed by only replacing the targets in the part of the command line that is after `build` in `buck build ...`
Reviewed By: sblackshear
Differential Revision: D2914446
fb-gh-sync-id: ddce4ed
shipit-source-id: ddce4ed
Summary:
public
The PerformanceCritical Checker was not running on Buck project due to a conflict between the incremental mode and the ondemand mode.
Reviewed By: cristianoc
Differential Revision: D2891649
fb-gh-sync-id: 0a503d8
Summary:
public
- add to json reports the location in the ocaml code of assertions raised
during the analysis
- only add when in debug mode. For this, add a flag to `InferPrint` that is
passed by the toplevel `infer` whenever `infer -g` is used.
Reviewed By: jeremydubreil
Differential Revision: D2891286
fb-gh-sync-id: ad4577c
Summary:
public
The env variables used by the Infer backend were only passed in multicore mode. They are now passed before the generation of clusters, whether in single core mode or in multicore mode.
Reviewed By: cristianoc
Differential Revision: D2891816
fb-gh-sync-id: 25074af
Summary:
public
When no classpath was specified, Infer were considering it to be None instead of the current directory. The consequence is that we were replacing the current directory with the annotation processor for suppress warnings, leading to cases where `javac` was compiling fine but `infer -- javac` was failing with classes not found compilation issues. This diff fixes by always having at least "." in the classpath.
Reviewed By: sblackshear
Differential Revision: D2853035
fb-gh-sync-id: e69db7c
Summary:
public
I previously forgot to keep this feature when deleting BuckAnalyze.
Reviewed By: sblackshear
Differential Revision: D2850869
fb-gh-sync-id: d23f1ca
Summary:
public
The script `BuckAnalyze` has been deprecated for a while already. Time to remove it.
Reviewed By: sblackshear
Differential Revision: D2844414
fb-gh-sync-id: b5e1195
Summary:
public
The code to configure to configure the logging was duplicated, but was not not doing exactly the same thing. This diff makes all the code to call the same configuration function.
Reviewed By: sblackshear
Differential Revision: D2844361
fb-gh-sync-id: 9887cad
Summary:
This will stop Infer from creating an `infer-out` directory when just running `infer`.
(I run `infer` without any arguments somewhat often to look at documenting.)
One downside to this is that there will be no logs when running Infer without any commands - is this bad?
Closes https://github.com/facebook/infer/pull/240
Reviewed By: cristianoc
Differential Revision: D2829752
Pulled By: jeremydubreil
fb-gh-sync-id: 6827ef9
Summary:
public
In the case of Buck, the integration works by using the out directory of javac in order to store the analysis artifacts in the jar file. Making buck call:
infer -o infer_out -- javac -d classes_out ...
instead of every javac command is having no effect and is the same as Buck running:
infer -- javac -d classes_out ...
Reviewed By: sblackshear
Differential Revision: D2822642
fb-gh-sync-id: b13e067
Summary:
public
In the case of Java, the name of the method is enough to lookup the summary of the callees and run the symbolic execution. This revision separates the case of method call in Java and in C, C++ and Objective C. Most of the code for executing method calls was Clang specific and this is an intermediate step to be able to run the capture and analysis on demand.
Reviewed By: sblackshear
Differential Revision: D2809171
fb-gh-sync-id: da62dce
Summary:
public
`json.dump` does not have the same number of arguments in 2.7.3 and 2.7.9. Fix
this by naming the optional parameters explicitly.
Reviewed By: akotulski
Differential Revision: D2760579
fb-gh-sync-id: 7ebc69e
Summary: public
Since we are only running the analysis on the Buck targets that are of type java_library or android_library, it may happen that the filtering gives an empty list of targets.
This diff does two things:
1) it filters out the empty strings `''` that can be returned when parsing the `buck query` command, and
2) skip the analysis when the list of targets to analyze is empty.
It also fixes the debug output for the list of targets to analyze, which was previously always printing the list of targets passed in the command line argument, which led for this issue to be unoticed even though it may have been there for a while already.
Reviewed By: sblackshear
Differential Revision: D2704949
fb-gh-sync-id: 83c62d8
Summary: public
With this change it will be possible to instruct Buck to not analyze certain files whose names match
with the regex passed in input.
So for example if you want to skip the analysis of files containing `skip_me` on their names, then
the command:
infer --use-flavors --blacklist-regex ".*skip_me.*" -- buck build <target-name>
will do it, because the `.*skip_me.*` regex will match the desired rule.
Reviewed By: jvillard
Differential Revision: D2702872
fb-gh-sync-id: 498b17b
Summary: public
When merging json reports from different buck targets, the same bug may be
reported several times.
Clean up some bug sorting functions while I'm at it.
Reviewed By: martinoluca
Differential Revision: D2690665
fb-gh-sync-id: 4a12072
Summary: public
Refactor reporting of errors to enhance readability and reuse.
Reviewed By: jeremydubreil
Differential Revision: D2679676
fb-gh-sync-id: f4c1cba
Summary: public
With this change, it will be possible to pass to InferAnalyze a list of folders to be
searched for spec files, through a file containing those paths, separated by the newline
character.
For example, if there's a `PathList.txt` file containing
/path/to/specs/folder1
/path/to/specs/folder2
/path/to/specs/folder3
Then invoking `infer --specs-dir-list-file PathList.txt [--other_args, ...] -- <build_cmd>`
will instruct the analyzer to lookup to those three folders whenever it searches for specs.
It's important to note that since the analyzer runs in parallel from different locations, it's necessary to pass **absolute** paths, or the analyzer will fail with an error.
Reviewed By: jvillard
Differential Revision: D2668700
fb-gh-sync-id: b407a57
Summary: public
This function is useful to understand how infer prints reports, it's worth
cleaning it up a notch.
Reviewed By: jeremydubreil
Differential Revision: D2646878
fb-gh-sync-id: 2ecd894
Summary: public so that we don't get thousands of memory leaks in debug mode.
We will still get them if we pass ml_bucket internal.
Reviewed By: jvillard
Differential Revision: D2648200
fb-gh-sync-id: 8dce66f
Martino Luca
Jeremy Dubreil
Jules Villard
Ryan Rhee
Dulma Churchill
Sam Blackshear
Lázaro Clapp Jiménez Labora
Carlos Sierra
Andrzej Kotulski
Peter O'Hearn
Josh Berdine
Daiva Naudziuniene
Dino Distefano
Cristiano Calcagno
Deniz Türkoglu
kstreee
Tony Cosentini