Summary:
The checkers check was causing perf issues because it kept loading the json of
inferconfig. To prevent this from happening again, load json files inside
config.ml, and only export `Yojson.Basic.json Lazy.t` values to other modules.
Also move the list of checks disabled by default into config.ml for better
discoverability.
Reviewed By: jberdine
Differential Revision: D3293041
fbshipit-source-id: 4a38b26
Summary:
Reimplement command line options in preparation for uniformly passing
options from the top-level infer driver that invokes a build command
through the build system to the descendant infer processes.
All command line options of all executables are collected into Config,
and declared using a new CommandLineOption module that supports
maintining backward compatibility with the current command line
interface. Very few values representing command line options are
mutable now, as they are set once during parsing but are constant
thereafter. All ordering dependencies are contained within the
implementation of Config, and the implementation of Config is careful to
avoid unintended interactions and ordering dependencies between options.
Reviewed By: jvillard
Differential Revision: D3273345
fbshipit-source-id: 8e8c6fa
Summary:
- [python] decode strings coming from `os.*` commands
- [python] decode strings coming from the command-line
- [python] encode a few remaining unicodes into strings
- [java] replace lex/yacc parser for javac verbose output by regex-based matching to handle unicode in paths
- [make] random fix of `make test` to have `make clean test` work
- [integration tests] add e2e build integration tests for utf8 in the PWD
Closes#76
Reviewed By: martinoluca
Differential Revision: D3240809
fb-gh-sync-id: 8c2e1ed
fbshipit-source-id: 8c2e1ed
Summary:public
When a conditional is the last instruction, there will be a join node leading directly to the exit node.
Some instructions, such as nullification of dead variables, and abstraction, are added to the control flow graph automatically. But, join nodes cannot contain instructions. So when a procedure ends with a conditional, there might be no place to store these instructions.
This diff adds one extra node between the join and the exit node in that situation.
Reviewed By: jvillard
Differential Revision: D3179056
fb-gh-sync-id: 2b9cd7e
fbshipit-source-id: 2b9cd7e
Summary:public
Report statistics on consumed time and memory in results_dir/perf_stats.json.
Reviewed By: martinoluca
Differential Revision: D3162381
fb-gh-sync-id: e802faa
fbshipit-source-id: e802faa
Summary:public
Eliminate the use of the -open Utils command line option passed to the compiler in favor of `open! Utils` in each source file. While slightly convenient, this option causes more headaches than it is worth with other tools e.g. merlin.
Reviewed By: jvillard
Differential Revision: D3168193
fb-gh-sync-id: 4285ef6
fbshipit-source-id: 4285ef6
Summary:public
This will allow SymExec to depend on Inferconfig with introducing
circular dependencies, as Inferconfig calls Builtin.is_registered.
Reviewed By: jeremydubreil
Differential Revision: D3100614
fb-gh-sync-id: 786cf62
fbshipit-source-id: 786cf62
Summary:public
Eradicate need the procedure attributes for callees.
It relies on the java front-end to create proc descs for callees that are declared but not defined.
This diff remove that needs, and when a callee without prodedure attributes is found, it creates one on the fly. The attribute created is similar to what the Java front-end would do, except
that the number and types of arguments are part of the call instruction, so they can
be used to create the formal parameters.
Reviewed By: jeremydubreil
Differential Revision: D3073904
fb-gh-sync-id: 381ff67
fbshipit-source-id: 381ff67
Summary:public
Before this diff, the Java frontend was not adding the definition of the inherited interfaces to the type environment, thus failing to answer questions like "does type X implements Closeable". Infer was therefore missing to detect resource leaks when the resource was indirectly implementing Closeable via an intermediate interface.
Reviewed By: sblackshear
Differential Revision: D3067555
fb-gh-sync-id: 86d0760
shipit-source-id: 86d0760
Summary: public Like the pre-analysis, these should be called for every CFG. This is a stepping stone toward getting rid of remove_tmps and making it part of the liveness analysis.
Reviewed By: jvillard
Differential Revision: D3059021
fb-gh-sync-id: 3a8d818
shipit-source-id: 3a8d818
Summary:public
Generalize command line option parsing to query an environment variable
for args, and then parse the environment and command line arguments.
Each executable uses a distinct environment variable:
- InferAnalyze: INFER_ARGS
- InferJava: INFERJAVA_ARGS
- InferClang: INFERCLANG_ARGS
- InferLLVM: INFERLLVM_ARGS
- checkCopyright: CHECKCOPYRIGHT_ARGS
For now these variables need to be set manually. So the usability is
still pretty bad, but is a step in the direction of enabling debugging
the analyzer on code built with buck.
Reviewed By: cristianoc, martinoluca
Differential Revision: D2987288
fb-gh-sync-id: f477611
shipit-source-id: f477611
Summary:public
Assert false have been observed in Procname when analyzing some C projects.
This diff changes the Procname API to make it safe for Java: the java functions in the module don't assert false now. This takes care of the errors observed in C projects.
The new API forces changes throughout the codebase. In particular, the constant propagation module was making assumptions that it would only be executed on Java code, triggering assert false on C. Now it is safe.
For the remaining functions in the Procname module, those for other languages, a special assert false in Utils is used to print stack traces. This is for future debugging.
Reviewed By: sblackshear
Differential Revision: D3054077
fb-gh-sync-id: a77f1d7
shipit-source-id: a77f1d7
Summary:public
Use the configuration file .inferconfig to model the library method that are considered expensive
Reviewed By: cristianoc
Differential Revision: D3045288
fb-gh-sync-id: e58d85c
shipit-source-id: e58d85c
Summary:public
Revert 6fa9b995e5 (D2843010).
It's slightly less worse to silently ignore Java8 rather than crashing. We'll
wait for an upstream fix to Javalib to resolve the issue of leaked file
descriptors that the original diff was trying to address.
Reviewed By: sblackshear
Differential Revision: D3040978
fb-gh-sync-id: 4020221
shipit-source-id: 4020221
Summary:public
Simplifies the code to collect the `SuppressWarnings` annotations and makes the code more robust in the sense that not finding the output of the annotation processor will result in an error directly at the top-level instead of later on when trying to load the output file in the Java frontend.
Reviewed By: sblackshear
Differential Revision: D3034690
fb-gh-sync-id: 60caa0c
shipit-source-id: 60caa0c
Summary:public
Instead of using the collection of suppress warnings annotations to filter out the errors while generating the error reports, we just add this SuppressWarnings at translation time, like any other annotations, and the reporting functions in the Reporting module will just skip the errors when the method is annotated with SuppressWarnings.
This allows us to have a suppress warnings mechanism that is independant from the integration with the build system.
Reviewed By: sblackshear
Differential Revision: D3012395
fb-gh-sync-id: 35f5f9b
shipit-source-id: 35f5f9b
Summary:public
Revamped Timeout module by storing elapsed wallclock seconds, and the status of symops, in case of recursive calls.
Extended the API with suspend() and resume() to pause and resume the current timeout.
These are used before and after an on-demand call to the analysis functions.
This achieves the effect that each procedure, even though is interrupted, has its own time and symop counters, which are suspended and resumed as required.
Reviewed By: jeremydubreil
Differential Revision: D2976918
fb-gh-sync-id: 0ed1079
shipit-source-id: 0ed1079
Summary:public
I have seen enough comments in this space by people during code review to switch on the analyses the compiler can already do. This diff is an automated renaming of unused identifiers to _, with a few additional changes made when reading the diff of the results for things that stood out as particularly strange. This base-lines all of the existing warnings. I'm not sure this is a good idea, since it might be better for those familiar with each part of the code to look at these warnings and use them as pointers to suspicious code.
Reviewed By: jeremydubreil
Differential Revision: D2938376
fb-gh-sync-id: 6e67817
shipit-source-id: 6e67817
Summary:
public
Remove the need to create a record with two elements to create a procedure description
Reviewed By: cristianoc
Differential Revision: D2872744
fb-gh-sync-id: d26bbdc
Summary:
public
In the case of Java, the name of the method is enough to lookup the summary of the callees and run the symbolic execution. This revision separates the case of method call in Java and in C, C++ and Objective C. Most of the code for executing method calls was Clang specific and this is an intermediate step to be able to run the capture and analysis on demand.
Reviewed By: sblackshear
Differential Revision: D2809171
fb-gh-sync-id: da62dce
Summary:
public
When reading from static fields, the translation was overwriting the list of class members with the list of static ones. The backend was only looking up fields from the list of non static fields.
Reviewed By: sblackshear
Differential Revision: D2801759
fb-gh-sync-id: fe8ed80
Summary:
public
While playing with the type environment for Java, I realised that the types in models.jar where not re-generated when modifying Infer. As a consequence, some changes in Infer where surprisingly having no effect. This diff forces the type environment to be absent when analyzing the models.
Reviewed By: sblackshear
Differential Revision: D2802517
fb-gh-sync-id: 1c2673a
Summary:
public
Using Typename.t in the list of superclasses to match the type for the key of the type environment. This avoids to make back and forth convertions from typename to type (csu, mangled name).
Depends on D2786574
Reviewed By: jberdine
Differential Revision: D2792116
fb-gh-sync-id: 6100f1a
Summary:
public
The paramtere where defined as simple strings in the procedure description. This diff force the use of the Mangled module to avoid possible conflict when converting variable back and forth from string to pvar. The code is now more consistent as the local variable were already named using mangled names.
Reviewed By: jberdine
Differential Revision: D2782863
fb-gh-sync-id: 1867574
Summary:
public
Move the naming of types to it own module, so that it can be used by modules `Sil` depends from like `Procname`
Reviewed By: jberdine
Differential Revision: D2773148
fb-gh-sync-id: a89f595
Summary:
public
Move the representation of data-structure into it own module, so that it can be used by modules `Sil` depends from like `Procname`.
Reviewed By: jberdine
Differential Revision: D2772791
fb-gh-sync-id: cda4e3a
Summary:
public
Infer would previously give confusing reports in the following case: two classes `foo.MyClass` defined in `MyClass.java` under directory `foo/` and `bar.MyClass` defined in file `MyClass.java` under `bar/` are compiled together in a single call to the Java compiler. Then the errors in `foo/MyClass.java` could potentially be reported in `bar/MyClass.java`, or the other way around.
The reason is: Infer starts the translation from the bytecode which only contains information about the base filename in the metadata. For example, both `foo.MyClass` and `bar.MyClass` will contains the information that the source file is `MyClass.java` but not the full path to the actual source file (hopefully).
In order to cope with this issue, this diff adds the possibility to read the package declaration from the source file so that we can map classes to the source files these classes are defined without ambiguity. In order to avoid having to open and read the source files when not necessary, the code will behave as before as long as no name conflict is found. Otherwise, it will only load and search for the package declaration when two or more sources files have the same basename but are defined in different subdirectories.
Closes t9395275
Reviewed By: jberdine
Differential Revision: D2763775
fb-gh-sync-id: 0adc1ac
Summary:
public
Lines other than the first of multi-line comments in non-ocaml files
were flush right instead of aligned.
Reviewed By: jvillard
Differential Revision: D2739752
fb-gh-sync-id: c85f56e
Summary: public
The jar file of the models was opened twice before. This should reduce a little bit the IO of the capture.
Reviewed By: cristianoc
Differential Revision: D2626251
fb-gh-sync-id: 310bef4
Summary: public
This is a non-functional refactoring to add a flag to trigger the translation of the procedure description of callees. This allows to check in which cases we still need to procedure description of callees to run the analysis.
Reviewed By: cristianoc
Differential Revision: D2620402
fb-gh-sync-id: 7ef5b5f
Summary: public
The harness was created all the time but we do not report issues comming from the harness right now. The option:
infer --android-harness -- <build command>
Allows to create the harness
Reviewed By: sblackshear
Differential Revision: D2595211
fb-gh-sync-id: c7a6dc5
Summary: public
This allow to tell Infer to skip the translation of some files. This is especially useful to skip the translation of some generated files following the syntax:
> cat .inferconfig
{
"skip_translation": [
{
"language": "Java",
"source_contains": "_SHOULD_BE_SKIPPED_"
}
]
}
Reviewed By: cristianoc
Differential Revision: D2588095
fb-gh-sync-id: 3fda816
Summary: public
modules are better for namespacing.
How I made this diff:
1. moved list_* functions from utils.ml{,i} to iList.ml{,i}
2. shell commands:
grep '^val ' infer/src/backend/iList.mli | cut -f 2 -d ' ' | tr '\n' ' '
# gives a list of former list_ functions that IList implements, fed into the loops below:
LISTNAMES=" compare equal append combine exists filter flatten flatten_options find fold_left fold_left2 for_all for_all2 hd iter iter2 length fold_right map mem nth partition rev rev_append rev_map sort split stable_sort tl drop_first drop_last rev_with_acc remove_duplicates remove_irrelevant_duplicates merge_sorted_nodup intersect mem_assoc assoc map2 to_string"
# replace " list_*" function calls with IList.* ones
for i in $LISTNAMES; do find . -name '*.ml' -exec sed -i -e "s/ list_$i\b/ IList.$i/g" \{\} \; ; done
# replace (list_* functions with (IList.* ones
for i in $LISTNAMES; do find . -name '*.ml' -exec sed -i -e "s/(list_$i\b/(IList.$i/g" \{\} \; ; done
# ditto with [
for i in $LISTNAMES; do find . -name '*.ml' -exec sed -i -e "s/\[list_$i\b/[IList.$i/g" \{\} \; ; done
3. Then fix up the rest by hand. In particular, stuff that called Utils.list_*
explicitely, and stuff that used the "Fail" exception that has moved to
IList. (may revisit this in the future)
Reviewed By: jeremydubreil, cristianoc
Differential Revision: D2550241
fb-gh-sync-id: cd64b10
Summary:
each procedure has a different scope, so we can restart the fresh name generator and have more stable instructions in the cfg, that don't change when other procedures are changed
Summary:
Move proc_attributes to a separate module.
Field err_log, in common between proc desc and summary, can now be moved to ProcAttributes without creating cycles of dependencies.
Summary:
There's a lot of overlap between the representation of a proc desc and a spec summary. This diff moves all the data in common to the single record proc_attributes defined in Sil.
This gives a unified way of accessing most of the data carried by a procedure, whether it is contained in a proc desc or a spec. Also, it ensures that there is a single flow of information from proc desc to spec in the back-end, making sure that the information represented stays consistent.
Summary:
This commit is the result of
`find infer/src -name '*.ml' -or -name '*.mli' -exec ocp-indent -i \{\} \;`
and
`INFER_CHECK_COPYRIGHT=1 InferPrint`
Summary:
Bumping up the version of the libraries javalib and sawja to 2.3.1 and 1.5.1 respectively in order to use the upstream fix about classnames with $$.
Summary:
the name of the return variable of a procedure only depends on the name of that procedure. This simplifies the need for the procedure description in a couple of places
Summary:
@public
Previously, if the close() method was throwing an exception, then code overriding the file attribute with a mem attribute would be skipped, resulting in reporting a wrong resource leak. This diff fixes this.
Test Plan: Added new end-to-end tests which would previously have been failing
Summary:
@public
Using InferBuiltins.assume previously caused an assertion failure in the analyzer. Fixed this, and fixed the implementation of the assume builtin to block when the assumed condition cannot hold.
Test Plan: Added several new tests.
Summary:
@public
The empty string '' is a valid classpath information. This diff parses it and skips it when detecting the parts of the classpath. This case is happening when analyzing Buck. This one be one possibility why Infer does not load any bytecode in some cases, leading the
TODO: print error message
failure happening with the release (now `Failed to load any Java source code`).
Test Plan: Infer CI.
Summary:
@public
This adds basic support for function attributes in Sil, and for translating
attributes from the clang frontend to these new Sil attributes. For now only
the sentinel attribute is translated.
Note that attributes normally have parameters, but they are currently missing
from the clang plugin.
Test Plan:
Add
(match Sil.get_sentinel_func_attribute_value (Cfg.Procdesc.get_attributes callee_pdesc).Sil.func_attributes with
| Some _ -> L.out "found sentinel attribute!\n"
| _ -> ());
between lines 947 and 948 of symbExec.ml, then analyze a file containing:
int add_all_ints(int a, ...) __attribute__ ((sentinel));
int foo(void) { return add_all_ints(1, 2, 3, (void *)0); }
then `grep 'found sentinel' infer-out/log/analyzer_out`
-> the sentinel attribute is correctly passed from the frontend to the backend.
Sam Blackshear
Jules Villard
Josh Berdine
Jeremy Dubreil
Peter O'Hearn
Cristiano Calcagno
Dulma Rodriguez
Rohan Jacob-Rao
Andrzej Kotulski
Open Source Synchronization