Summary:
The checkers check was causing perf issues because it kept loading the json of
inferconfig. To prevent this from happening again, load json files inside
config.ml, and only export `Yojson.Basic.json Lazy.t` values to other modules.
Also move the list of checks disabled by default into config.ml for better
discoverability.
Reviewed By: jberdine
Differential Revision: D3293041
fbshipit-source-id: 4a38b26
Summary:
Infer prepends the directory containing the ananotation processor and
the current working directory to the classpath javac option. This diff
enables prepending these to the classpath when it is passing in an args
file (as the classpath can get too long to pass on the command line).
Reviewed By: jvillard
Differential Revision: D3270348
fbshipit-source-id: 208077f
Summary:
- [python] decode strings coming from `os.*` commands
- [python] decode strings coming from the command-line
- [python] encode a few remaining unicodes into strings
- [java] replace lex/yacc parser for javac verbose output by regex-based matching to handle unicode in paths
- [make] random fix of `make test` to have `make clean test` work
- [integration tests] add e2e build integration tests for utf8 in the PWD
Closes#76
Reviewed By: martinoluca
Differential Revision: D3240809
fb-gh-sync-id: 8c2e1ed
fbshipit-source-id: 8c2e1ed
Summary:This enables controlling the encoding chosen by infer via the usual environment
variables. For instance:
```
LC_ALL="C" infer ... # sets LOCALE to "ascii"
LC_ALL="en_US.UTF-8" infer ... # sets LOCALE to "UTF-8"
```
This gives an easy solution to #320: run `LC_ALL="en_US.UTF-8" infer ...`.
Right now the only solution is to edit the Python scripts by hand instead!
Reviewed By: jberdine
Differential Revision: D3207573
fb-gh-sync-id: 62d5b98
fbshipit-source-id: 62d5b98
Summary:`exc.output` can be (always is?) `None`. Other places in the code only print
using `traceback.print_exc()` and this was the only place trying to print this
extra info (`git grep CalledProcessError`). This caused `utils.stdout()` to
raise an exception, which was further confusing.
closes#330
Reviewed By: jberdine
Differential Revision: D3203896
fb-gh-sync-id: d2988d8
fbshipit-source-id: d2988d8
Summary:public
Models in headers are purely C++ concept and it's
useless if C++ mode is turned off. Since those
models can lead to compilation errors, turn them
off for non-C++ analysis
Reviewed By: martinoluca
Differential Revision: D3126294
fb-gh-sync-id: 0912e7b
fbshipit-source-id: 0912e7b
Summary:public
Simplifies the way to configure cmake to run infer's fake compiler commands.
Instead of `CC=/path/to/infer/infer/lib/capture/clang cmake .`, which is what
is advised on #25 and is now outdated because the paths to infer's clang has
changed since, simply run `infer -- cmake .`. The only caveat is that infer
tries to analyze the end result, and prints "No issues found". This could be
fixed later.
Reviewed By: jeremydubreil
Differential Revision: D3093162
fb-gh-sync-id: 99df50a
fbshipit-source-id: 99df50a
Summary:public
Allow the use of `infer -- ./configure`. This can be useful if the full path to
the compiler is recorded by the `./configure` command. That is the case for the
samba source tree for instance (because `./configure` calls `waf configure`
under the hood).
Reviewed By: jeremydubreil
Differential Revision: D3093065
fb-gh-sync-id: 2663418
fbshipit-source-id: 2663418
Summary:public
Eradicate need the procedure attributes for callees.
It relies on the java front-end to create proc descs for callees that are declared but not defined.
This diff remove that needs, and when a callee without prodedure attributes is found, it creates one on the fly. The attribute created is similar to what the Java front-end would do, except
that the number and types of arguments are part of the call instruction, so they can
be used to create the formal parameters.
Reviewed By: jeremydubreil
Differential Revision: D3073904
fb-gh-sync-id: 381ff67
fbshipit-source-id: 381ff67
Summary:public
In Python 2, `shutil.rmtree()`, `os.walk()`, `os.path.join()`, etc. are not
happy when the locale cannot decode the filenames they have to deal with.
Decrease the likelihood of this happening by making the file names generated by
infer ascii-only.
Also ignore character decoding errors optimistically when reading the json
report file.
Add tests that we are able to run the analysis and report the bug on a function
with a utf8 name, and that we are able to remove the previous results
directory.
closes#287
Reviewed By: cristianoc
Differential Revision: D3058858
fb-gh-sync-id: b88cd35
shipit-source-id: b88cd35
Summary:public
It seems that creating the procedure description of the callees this should no longer be useful to run the checkers
Reviewed By: sblackshear
Differential Revision: D3083523
fb-gh-sync-id: 040b104
shipit-source-id: 040b104
Summary:public
The "fake" procedure description of the callees, containing info about the formal parameters, is current used by Eradicate. This diff forces the creation of the callee procedure descriptions when running Infer with Buck.
Reviewed By: sblackshear
Differential Revision: D3083452
fb-gh-sync-id: 24a70e6
shipit-source-id: 24a70e6
Summary:public
It seems that the support for using the Buck cache with Infer has been brocken for a while.
Reviewed By: sblackshear
Differential Revision: D3079493
fb-gh-sync-id: fd92d1c
shipit-source-id: fd92d1c
Summary:public
Whenever infer-deps.txt and report.json files are encountered after the analysis with Buck
they will be automatically merged and deduplicated with all the other files of the same kind.
This change also emits the results of the analysis to stdout.
Reviewed By: jvillard
Differential Revision: D3064487
fb-gh-sync-id: 3599fba
shipit-source-id: 3599fba
Summary:We currently only output the report as JSON or plain text, however other analyzers use XML and there are tools and plugins to process them, for instance TeamCity XML Report Processing plugin.
Author: Deniz Türkoglu <deniz@spotify.com>
Author: Jules Villard <jul@fb.com>
Closes https://github.com/facebook/infer/pull/300
Reviewed By: cristianoc
Differential Revision: D3047181
Pulled By: jvillard
fb-gh-sync-id: 9026ca2
shipit-source-id: 9026ca2
Summary:public
The reactive analysis starts from the set of changed files/procedures, and proceeds
reactively to analyze their dependencies.
This means that after every command, the set of changed files/procedures is reset.
With the --continue option, the capture is continued: all the files/procedures marked
as changed stay changed, plus any additional changes are recorded.
In addition to allowing to spread capture over several commands, the option also allows to separate capture and analysis in reactive mode, or to repeat the analysis.
Reviewed By: sblackshear
Differential Revision: D3046361
fb-gh-sync-id: b6e3797
shipit-source-id: b6e3797
Summary:public
Use the configuration file .inferconfig to model the library method that are considered expensive
Reviewed By: cristianoc
Differential Revision: D3045288
fb-gh-sync-id: e58d85c
shipit-source-id: e58d85c
Summary:public
Simplifies the code to collect the `SuppressWarnings` annotations and makes the code more robust in the sense that not finding the output of the annotation processor will result in an error directly at the top-level instead of later on when trying to load the output file in the Java frontend.
Reviewed By: sblackshear
Differential Revision: D3034690
fb-gh-sync-id: 60caa0c
shipit-source-id: 60caa0c
Summary:public
bugs.txt only contains the summary of each report. The terminal output contains
a bit more information, such as source excerpts. If one wants to save the
terminal output into a file, they can always use shell redirection anyway.
closes#294
Reviewed By: martinoluca
Differential Revision: D3023796
fb-gh-sync-id: 9a21d17
shipit-source-id: 9a21d17
Summary:public
Instead of using the collection of suppress warnings annotations to filter out the errors while generating the error reports, we just add this SuppressWarnings at translation time, like any other annotations, and the reporting functions in the Reporting module will just skip the errors when the method is annotated with SuppressWarnings.
This allows us to have a suppress warnings mechanism that is independant from the integration with the build system.
Reviewed By: sblackshear
Differential Revision: D3012395
fb-gh-sync-id: 35f5f9b
shipit-source-id: 35f5f9b
Summary:public
With this change, all the `infer-deps.txt` files generated by buck for those targets
running with the `#infer` flavor, will be merged into one `infer-deps.txt` located in the
designated output folder.
Reviewed By: jvillard
Differential Revision: D2994397
fb-gh-sync-id: 14d8109
shipit-source-id: 14d8109
Summary:public
Lazy dynamic dispatch handling works as follows:
Assuming a call of the form:
foo(a);
where the static type of `a` is `A`. If during the symbolic execution, the dynamic type of the variable `a` is `B` where `B <: A`, then we create on-demand a copy `foo(B)` of `foo(A)` where all the uses of the typed parameter `a` are replaced with a parameter of type `B`. Especially, if `foo` contains virtual call, say `get` where `a` is the receiver, then the call gets redirected to the overridden method in `B`, which simulates the runtime behavior of Java.
This lazy dynamic dispatch mode is only turn on for the tracing mode for now in order to avoid conflicts with sblackshear's approach for sound dynamic dispatch.
Reviewed By: sblackshear
Differential Revision: D2888922
fb-gh-sync-id: 3250c9e
shipit-source-id: 3250c9e
Summary:public
Deprecate the incremental mode.
Several parts of the back-end can be removed.
The options for incremental analysis -i at the python level are now deprecated, and re-routed to --reactive.
The main difference with --reactive is that it does not produce an analysis of the whole project, but is limited to what is reachable via reactive propagation starting from the changed files.
Reviewed By: sblackshear
Differential Revision: D2960078
fb-gh-sync-id: 6e8b46b
shipit-source-id: 6e8b46b
Summary:public
An observer object that registered to a notification center needs to be
unregistered before it is deallocated.
If not, the notification center may send a notification to a gost object.
This diff introduce a checker for this problem.
Reviewed By: dulmarod
Differential Revision: D2949692
fb-gh-sync-id: 1653cec
shipit-source-id: 1653cec
Summary:public
`print(tracer)` has an automatic conversion to string that was missed when
migrating to `utils.stdout(tracer)`.
Reviewed By: peterogithub
Differential Revision: D2982104
fb-gh-sync-id: 4c26727
shipit-source-id: 4c26727
Summary:public
- s/"/'/ in python strings
- kill `utils.error()` in favour of the new, identical `utils.stderr()`
- one more `print(utils.encode())` to `utils.stderr()` conversion
Reviewed By: jeremydubreil
Differential Revision: D2976710
fb-gh-sync-id: 6c0fdfa
shipit-source-id: 6c0fdfa
Summary:public
This class expects a working `jwlib.CompilerCommand` even when we're not doing
anything Java-related. Split the java-specific functionality into a new child
class in jwlib.py.
Reviewed By: jeremydubreil
Differential Revision: D2965832
fb-gh-sync-id: e895b33
shipit-source-id: e895b33
Summary:public
java-specific code such as this belongs in jwlib.py. It will also help the
refactoring in the next diff.
Reviewed By: sblackshear
Differential Revision: D2965814
fb-gh-sync-id: c3adc03
shipit-source-id: c3adc03
Summary:public
This will avoid a circular dependency between analyze.py and jwlib.py in an
upcoming refactoring.
Reviewed By: martinoluca
Differential Revision: D2965734
fb-gh-sync-id: 1cb69d4
shipit-source-id: 1cb69d4
Summary:public
This attempts to properly sanitise text input/output in the Python parts of
infer. Do three things:
- encode user input (coming from the command-line or reading files)
- decode infer output
- in both cases, we may be using the wrong encoding, eg: locale says we're in
ascii, but the source code contains utf-8. In many cases, like error
messages, it's safe to ignore these encoding mismatches.
Also, since we `import __future__.unicode_literals`, it's safe to remove `u'`
prefixes on many unicode literals.
Reviewed By: martinoluca
Differential Revision: D2960493
fb-gh-sync-id: 9812d7d
shipit-source-id: 9812d7d
Summary:public
Add command-line argument --reactive to enable reactive propagation mode.
When the mode is active, the files changed during compilation are detected, and the analysis propagates reactively starting from the modified files.
The reactive mode allows to analyze a subset of the files in a project and follow their dependencies, without storing the results of previous analyses (specs files). Captured files are preserved from previous runs of the analysis (for example, when the previous analysis was the initial capture), so the mode can be used repeatedly while changing code.
Reviewed By: jvillard
Differential Revision: D2931697
fb-gh-sync-id: 9d6dda0
shipit-source-id: 9d6dda0
Summary:
public
The analysis of the Buck project was failing because this script converts the aliases into their expanded target names. It turns out that for Buck, the name of the command is `buck` and the name of the alias is also `buck`, which led to conflicts. This is now fixed by only replacing the targets in the part of the command line that is after `build` in `buck build ...`
Reviewed By: sblackshear
Differential Revision: D2914446
fb-gh-sync-id: ddce4ed
shipit-source-id: ddce4ed
Summary:
public
The PerformanceCritical Checker was not running on Buck project due to a conflict between the incremental mode and the ondemand mode.
Reviewed By: cristianoc
Differential Revision: D2891649
fb-gh-sync-id: 0a503d8
Summary:
public
- add to json reports the location in the ocaml code of assertions raised
during the analysis
- only add when in debug mode. For this, add a flag to `InferPrint` that is
passed by the toplevel `infer` whenever `infer -g` is used.
Reviewed By: jeremydubreil
Differential Revision: D2891286
fb-gh-sync-id: ad4577c
Summary:
public
The env variables used by the Infer backend were only passed in multicore mode. They are now passed before the generation of clusters, whether in single core mode or in multicore mode.
Reviewed By: cristianoc
Differential Revision: D2891816
fb-gh-sync-id: 25074af
Summary:
public
When no classpath was specified, Infer were considering it to be None instead of the current directory. The consequence is that we were replacing the current directory with the annotation processor for suppress warnings, leading to cases where `javac` was compiling fine but `infer -- javac` was failing with classes not found compilation issues. This diff fixes by always having at least "." in the classpath.
Reviewed By: sblackshear
Differential Revision: D2853035
fb-gh-sync-id: e69db7c
Summary:
public
I previously forgot to keep this feature when deleting BuckAnalyze.
Reviewed By: sblackshear
Differential Revision: D2850869
fb-gh-sync-id: d23f1ca
Summary:
public
The script `BuckAnalyze` has been deprecated for a while already. Time to remove it.
Reviewed By: sblackshear
Differential Revision: D2844414
fb-gh-sync-id: b5e1195
Summary:
public
The code to configure to configure the logging was duplicated, but was not not doing exactly the same thing. This diff makes all the code to call the same configuration function.
Reviewed By: sblackshear
Differential Revision: D2844361
fb-gh-sync-id: 9887cad
Summary:
This will stop Infer from creating an `infer-out` directory when just running `infer`.
(I run `infer` without any arguments somewhat often to look at documenting.)
One downside to this is that there will be no logs when running Infer without any commands - is this bad?
Closes https://github.com/facebook/infer/pull/240
Reviewed By: cristianoc
Differential Revision: D2829752
Pulled By: jeremydubreil
fb-gh-sync-id: 6827ef9
Summary:
public
In the case of Buck, the integration works by using the out directory of javac in order to store the analysis artifacts in the jar file. Making buck call:
infer -o infer_out -- javac -d classes_out ...
instead of every javac command is having no effect and is the same as Buck running:
infer -- javac -d classes_out ...
Reviewed By: sblackshear
Differential Revision: D2822642
fb-gh-sync-id: b13e067
Summary:
public
In the case of Java, the name of the method is enough to lookup the summary of the callees and run the symbolic execution. This revision separates the case of method call in Java and in C, C++ and Objective C. Most of the code for executing method calls was Clang specific and this is an intermediate step to be able to run the capture and analysis on demand.
Reviewed By: sblackshear
Differential Revision: D2809171
fb-gh-sync-id: da62dce
Summary:
public
`json.dump` does not have the same number of arguments in 2.7.3 and 2.7.9. Fix
this by naming the optional parameters explicitly.
Reviewed By: akotulski
Differential Revision: D2760579
fb-gh-sync-id: 7ebc69e
Summary: public
Since we are only running the analysis on the Buck targets that are of type java_library or android_library, it may happen that the filtering gives an empty list of targets.
This diff does two things:
1) it filters out the empty strings `''` that can be returned when parsing the `buck query` command, and
2) skip the analysis when the list of targets to analyze is empty.
It also fixes the debug output for the list of targets to analyze, which was previously always printing the list of targets passed in the command line argument, which led for this issue to be unoticed even though it may have been there for a while already.
Reviewed By: sblackshear
Differential Revision: D2704949
fb-gh-sync-id: 83c62d8
Summary: public
With this change it will be possible to instruct Buck to not analyze certain files whose names match
with the regex passed in input.
So for example if you want to skip the analysis of files containing `skip_me` on their names, then
the command:
infer --use-flavors --blacklist-regex ".*skip_me.*" -- buck build <target-name>
will do it, because the `.*skip_me.*` regex will match the desired rule.
Reviewed By: jvillard
Differential Revision: D2702872
fb-gh-sync-id: 498b17b
Summary: public
When merging json reports from different buck targets, the same bug may be
reported several times.
Clean up some bug sorting functions while I'm at it.
Reviewed By: martinoluca
Differential Revision: D2690665
fb-gh-sync-id: 4a12072
Summary: public
Refactor reporting of errors to enhance readability and reuse.
Reviewed By: jeremydubreil
Differential Revision: D2679676
fb-gh-sync-id: f4c1cba
Summary: public
With this change, it will be possible to pass to InferAnalyze a list of folders to be
searched for spec files, through a file containing those paths, separated by the newline
character.
For example, if there's a `PathList.txt` file containing
/path/to/specs/folder1
/path/to/specs/folder2
/path/to/specs/folder3
Then invoking `infer --specs-dir-list-file PathList.txt [--other_args, ...] -- <build_cmd>`
will instruct the analyzer to lookup to those three folders whenever it searches for specs.
It's important to note that since the analyzer runs in parallel from different locations, it's necessary to pass **absolute** paths, or the analyzer will fail with an error.
Reviewed By: jvillard
Differential Revision: D2668700
fb-gh-sync-id: b407a57
Summary: public
This function is useful to understand how infer prints reports, it's worth
cleaning it up a notch.
Reviewed By: jeremydubreil
Differential Revision: D2646878
fb-gh-sync-id: 2ecd894
Summary: public so that we don't get thousands of memory leaks in debug mode.
We will still get them if we pass ml_bucket internal.
Reviewed By: jvillard
Differential Revision: D2648200
fb-gh-sync-id: 8dce66f
Summary: public
This is the first of a series of patches that reorganise the inferlib package
into a bunch of smaller modules.
Reviewed By: martinoluca
Differential Revision: D2641917
fb-gh-sync-id: 19788b5
Summary: public
but keep the copy-pastability by wrapping the list into a small standalone
script that can be copy-pasted on the command line.
Reviewed By: jeremydubreil
Differential Revision: D2636536
fb-gh-sync-id: 7c689e2
Summary: public
These were only present in a few files. Could be added back on all files if
someone uses them.
Reviewed By: martinoluca
Differential Revision: D2646803
fb-gh-sync-id: edd6c43
Summary: public
This also fixes an issue with utf-8 in source code, as the csv report filters
out non-ascii characters.
Reviewed By: akotulski
Differential Revision: D2641727
fb-gh-sync-id: 3ca6dc6
Summary: public
This should avoid confusion between the toplevel "infer" python script and
inferlib/infer.py.
Reviewed By: jeremydubreil
Differential Revision: D2637087
fb-gh-sync-id: 82ce2a4
Summary: public
Print "Capturing..." message, and some info in case it fails, eg:
$ cd examples/android_hello/
$ infer -- ./gradlew build
Running and capturing gradle compilation...
[... stuff ...]
$ infer -- ./gradlew build
Running and capturing gradle compilation...
Nothing to compile. Try running `./gradlew clean` first.
Reviewed By: jeremydubreil
Differential Revision: D2637024
fb-gh-sync-id: 0e8867d
Summary: public
When analysing projects we would not always display the numbers of procs and
files analysed. This fixes it, and also prints the stats before the reports for
readability. This way we also don't need to print "Analysis done" anymore.
Before:
$ infer -- gcc -c hello.c
Starting analysis (Infer version git-436690cf022a16313dda8447121a5934529e6e5c)
Analysis done
hello.c:5: error: NULL_DEREFERENCE
pointer s last assigned on line 4 could be null and is dereferenced at line 5, column 3
3. void test() {
4. int *s = NULL;
5. > *s = 42;
6. }
After:
$ infer -- gcc -c hello.c
Starting analysis (Infer version git-5b7ff3ac10d58e7ffd17c44574ab330ed07cb188)
Computing dependencies... 100%
Analyzing 1 cluster. 100%
Analyzed 1 procedure in 1 file
Found 1 issue
hello.c:5: error: NULL_DEREFERENCE
pointer s last assigned on line 4 could be null and is dereferenced at line 5, column 3
3. void test() {
4. int *s = NULL;
5. > *s = 42;
6. }
Also looked at the results on buck and gradle
Reviewed By: cristianoc
Differential Revision: D2636991
fb-gh-sync-id: 32b9a7c
Summary: public
This cleans up the output of infer. Before:
$ touch empty.c
$ infer -- gcc -c empty.c
Starting analysis (Infer version git-436690cf022a16313dda8447121a5934529e6e5c)
Analysis done
No issues found
$ infer -- javac Hello.java
Starting analysis (Infer version git-436690cf022a16313dda8447121a5934529e6e5c)
Analysis done
Hello.java:4: error: NULL_DEREFERENCE
object s last assigned on line 3 could be null and is dereferenced at line 4
2. int test() {
3. String s = null;
4. > return s.length();
5. }
6. }
Analyzed 2 procedures in 1 file
$
After:
$ infer -- gcc -c empty.c
Starting analysis (Infer version git-434faa7f70f6b9498615d3ead8c12bcfec6fc553)
Analyzing 0 clusters
Analysis done
No issues found
$ infer -- javac Hello.java
Starting analysis (Infer version git-434faa7f70f6b9498615d3ead8c12bcfec6fc553)
Computing dependencies... 100%
Analyzing 1 cluster. 100%
Analysis done
Found 1 issue
Hello.java:4: error: NULL_DEREFERENCE
object s last assigned on line 3 could be null and is dereferenced at line 4
2. int test() {
3. String s = null;
4. > return s.length();
5. }
6. }
Analyzed 2 procedures in 1 file
$
Also tested with buck, gradle.
Reviewed By: cristianoc
Differential Revision: D2636969
fb-gh-sync-id: 52f06f0
Summary: public
Use InferPrint to generate infer-out/report.json instead of converting
infer-out/report.csv. The json may soon contain more info than the CSV.
Reviewed By: jeremydubreil
Differential Revision: D2603775
fb-gh-sync-id: f141dfe
Summary: public
paths look a bit silly prior to this diff, as they go up from infer/ to go back
into infer/ right away, eg:
/home/jul/infer/infer/lib/python/inferlib/../../../../infer/lib/java/processor.jar
This changes this path to the less silly:
/home/jul/infer/infer/lib/python/inferlib/../../../lib/java/processor.jar"
Reviewed By: martinoluca
Differential Revision: D2631775
fb-gh-sync-id: 6df5d4d
Summary: public
This makes it a bit easier to diagnose errors. This is how it looks:
infer -- javac IDontExist.java
"javac" "-g" "-cp" "/home/jul/infer/infer/lib/python/inferlib/../../../../infer/lib/java/processor.jar" "-d" "/home/jul/infer/examples" "IDontExist.java" "-J-Duser.language=en"
This is how it used to look:
['javac', '-g', '-cp', u'/home/jul/code/infer/infer/bin/../lib/java/processor.jar', '-d', '/home/jul/infer/examples', 'IDontExist.java', '-J-Duser.language=en']
Reviewed By: jberdine
Differential Revision: D2631698
fb-gh-sync-id: 09903df
Summary: public
Add colored output if pygments is installed on the system.
Reviewed By: cristianoc
Differential Revision: D2601509
fb-gh-sync-id: 0fa8e9f
Summary: public
Now that we have modules, we can gather string constants in one place only.
Reviewed By: jeremydubreil
Differential Revision: D2615791
fb-gh-sync-id: 7cc21e4
Summary: public
This unclutters infer/bin/ and gives more structure to infer/lib/
Reviewed By: jeremydubreil
Differential Revision: D2605809
fb-gh-sync-id: 508fc2c
Jules Villard
Josh Berdine
Jeremy Dubreil
Dino Distefano
Sam Blackshear
Andrzej Kotulski
Cristiano Calcagno
martinoluca
Deniz Türkoglu
Dulma Rodriguez
kstreee
Tony Cosentini