Summary:
`DB.source_file_to_string` is very easy to misuse and it shouldn't even exist.
In preparation for that day, replace most of `source_file_to_string` with `source_file_pp`
Reviewed By: jvillard
Differential Revision: D4258390
fbshipit-source-id: 447cf5a
Summary: Originially, there was a missing package declaration meaning that the generated class was ending in a different place. I also added a test for equality of Integer to complement the test of no equality, which could be always true.
Reviewed By: sblackshear
Differential Revision: D4263676
fbshipit-source-id: 86ab0d3
Summary:
We only ought to report a source-sink flow at the call site where the sink is introduced.
Otherwise, we will report silly false positives.
Reviewed By: jeremydubreil
Differential Revision: D4234766
fbshipit-source-id: 118051f
Summary: This should make it easier to understand complex error reports.
Reviewed By: peterogithub
Differential Revision: D4254341
fbshipit-source-id: fb32d73
Summary: We'll eventually want fancy interprocedural traces. This diff adds the required boilerplate for this and adds the line number of each access to the error message. Real traces will come in a follow-up
Reviewed By: peterogithub
Differential Revision: D4251985
fbshipit-source-id: c9d9823
Summary: we no longer use buck for tests and the script has no reason to exist
Reviewed By: dulmarod
Differential Revision: D4212713
fbshipit-source-id: 7bd1cca
Summary: Noticed this when I was writing the documentation for the abstract interpretation framework and was curious about why `Ondemand.analyze_proc` needs the type environment. It turns out that the type environment is only used to transform/normalize Infer bi-abduction specs before storing them to disk, but this can be done elsewhere. Doing this normalization elsewhere simplifies the on-demand API, which is a win for all of its clients.
Reviewed By: cristianoc
Differential Revision: D4241279
fbshipit-source-id: 957b243
Summary: Adding this so we can test interprocedural trace-based reporting in a subsequent diff.
Reviewed By: peterogithub
Differential Revision: D4243046
fbshipit-source-id: 7d07f20
Summary: We're at risk for some silly false positives without these models.
Reviewed By: peterogithub
Differential Revision: D4244795
fbshipit-source-id: b0367e6
Summary:
Currently cfg nodes are written into dot files in whatever order they
appear in a hash table. This seems unnecessarily sensitive, so this
diff sorts the nodes.
Reviewed By: dulmarod
Differential Revision: D4232377
fbshipit-source-id: a907cc6
Summary: Add some basic command line API to run Infer using Buck genrules. Remains to fix issues with absolute vs relative paths and to see how to create these genrules on the fly for a given java or android library.
Reviewed By: sblackshear
Differential Revision: D4245622
fbshipit-source-id: 1cda4ee
Summary:
Clean up code related to --changed-files-index option:
1. Store DB.SourceFileSet.t in DB.changed_source_files_set
2. Refactor rest of the code to use it
3. Bunch of minor changes to make code more consise
Reviewed By: jberdine
Differential Revision: D4238736
fbshipit-source-id: 51e5684
Summary:
Implement heuristic to capture more of the user code:
In C++ there is a lot of interesting code in header files. On the other hand,
that code gets included in multiple places and we don't want to capture it by default (for performance reasons).
Right now we capture everything from source file + all symbols from headers that source file needs.
New heuristic will extend "capturing everything" to matching header files (ie. capture everything in X.h if source file is X.cpp)
Reviewed By: jberdine
Differential Revision: D4238008
fbshipit-source-id: 0528250
Summary:
Dealing with symbolic links in project root is tricky. To avoid it, always normalize all paths to sources with `realpath`.
Changes to tests are expected - infer started to resolve symbolic links which screws up with our testing mechanism.
Reviewed By: jberdine
Differential Revision: D4237587
fbshipit-source-id: fe1cb01
Summary:
Before, we were using a set domain of strings to model a boolean domain.
An explicit boolean domain makes it a bit clear what's going on.
There are two things to note here:
(1) This actually changed the semantics from the old set domain. The set domain wouldn't warn if the lock is held on only one side of a branch, which isn't what we want.
(2) We can't actually test this because the modeling for `Lock.lock()` etc doesn't work :(.
The reason is that the models (which do things like adding attributes for `Lock.lock`) are analyzed for Infer, but not for the checkers.
We'll have to add separate models for thread safety.
Reviewed By: peterogithub
Differential Revision: D4242487
fbshipit-source-id: 9fc599d
Summary: Add new integration test for compilation databse integration. Because new test needs another flags in infer invocation, I created one directory per test.
Reviewed By: jberdine
Differential Revision: D4231659
fbshipit-source-id: 81bb355
Summary:
In Java, we handle unknown code by propagating behavior from the parameters of the unknown function call to the return value (or constructed object, in the case of a constructor). But we do this in a somewhat silly way--generating a new summary with these semantics at each unknown call site. Instead, this diff introduces these two options as predefined behaviors and adds specialized code for them.
As a side effect of this approach, unknown functions are no longer counted as passthroughs. This is ok; the original behavior was less of a reasoned decision and more of an unintended consequence of the way we decided to handle unknown code.
This new approach ought to be more efficient than the old one, and as a virtuous side effect it will be easier to specify how to handle unknown code in other languages like C++.
Reviewed By: jeremydubreil
Differential Revision: D4205624
fbshipit-source-id: bf97445
Summary: Run all java tests with project-root at `infer/tests`. Do it to keep things consistent between clang and java tests
Reviewed By: sblackshear
Differential Revision: D4233236
fbshipit-source-id: c3f24fd
Summary:
Run all clang tests with project-root at `infer/tests`. I need it because we'll start resolving symbolic links
soon and some tests would lead outside of project root which means we'd start seeing absolute paths in recorded tests.
Diff that does same thing for java tests: D4233236
Reviewed By: jberdine
Differential Revision: D4233194
fbshipit-source-id: c261a2b
Summary:
Let's introduce some concepts. A "known unknown" function is one for which no Java code exists (e.g., `native`, `abstract`, and `interface methods`). An "unknown unknown" function is one for which Java code may or may not exist, but we don't have the code or we choose not to analyze it (e.g., non-modeled methods from the core Java or Android libraries).
Previously, Quandary handled both known unknowns and unknown unknowns by propagating taint from the parameters of the unknown function to its return value. It turns out that it is really expensive to do this for known unknown functions. D4142697 was the diff that starting handling known unknown functions in this way, and bisecting shows that it was the start of the recent performance problems for Quandary.
This diff essentially reverts D4142697 by handling known unknowns as skips instead. Pragmatically, doing the propagation trick for Java/Android library functions (e.g., `String` functions!) matters much more, so i'm not too worried about the missed behaviors from this. Ideally, we will go back to the old handling once performance has improved (have lots of ideas there). But I need this to unblock me in the meantime.
Reviewed By: jeremydubreil
Differential Revision: D4205507
fbshipit-source-id: 79cb9c8
Summary:
Useful for refactoring purposes, to provide a list of modules in
dependency order.
Reviewed By: jeremydubreil
Differential Revision: D4232363
fbshipit-source-id: 2adaaf5
Summary:
Implement heuristics to get from corresponding source files for header files.
We already had initial implementation for CaptureCompilationDatabase - moved it and extended it
to handle C/C++/objC/objC++
Reviewed By: jberdine
Differential Revision: D4231864
fbshipit-source-id: 4516287
Summary:
introduce `AttributesTable.load_defined_attributes` which will return proc attributes only if the procedure is defined. In order to not mess up
with existing caching, create another hashmap to store those procdescs.
We need to do that because with reactive capture we no longer can assume that all proc attributes are final before analysis starts
Reviewed By: jberdine
Differential Revision: D4231575
fbshipit-source-id: e795bcb
Summary: The default `android.jar` used to compile Android code is just a list of empty method stubs that is used for the type checking. In order to improve the type information, we used to redirect the bootclasspath to another `android.jar` that was containing the method bodies and all the fields including the private ones. Since we no longer need the types in the models and the types in the libraries to match, this is now dead code.
Reviewed By: jberdine
Differential Revision: D4230510
fbshipit-source-id: 93417f3
Summary:
This will help during the creation of new checkers, and will prevent errors like misspelling of AST node names.
It will also make it possible to fail immediately during the parsing of CTL inputs.
Reviewed By: ddino
Differential Revision: D4205434
fbshipit-source-id: ed8631a
Summary: Make backend know filenames of compilation database. It will allow it to compile extra files when needed
Reviewed By: cristianoc
Differential Revision: D4231521
fbshipit-source-id: c462448
Summary: Pure refactoring simplifying the code doing the case analysis for execturing the cast instruction.
Reviewed By: dulmarod
Differential Revision: D4215238
fbshipit-source-id: 9f0f163
Summary: Currently the thread safety checker neglects to analyze and files methods we don't want to report on. Like constructors and private methods, and classes where no superclass is marked ThreadSafe. For interprocedural analysis we want to analyze all these to get summaries, even if we don't report on them.
Reviewed By: jberdine
Differential Revision: D4226515
fbshipit-source-id: 7571573
Summary:
`abs_source_file_from_path` is dangerous because
1. It's not trying to make path relative to `project_root` - you can't compare two files created with two different methods
2. It's making relative paths relative to `getcwd` instead of `project_root` which is different from `from_string` function.
Also remove unused `DB.inode_equal`
Reviewed By: cristianoc
Differential Revision: D4204891
fbshipit-source-id: c4c2f99
Summary:
1. Always store cpp model source_file with relative path. This will make them cache friendly independent of infer location
2. Distinguish between "relative to project root" and "relative to infer models src"
3. Unify `source_file_from_path` used by java and C frontends into one function. There are no improvements to that logic yet
4. Move `is_cpp_model_file` to use `source_file` instead of `filename`
Reviewed By: jberdine
Differential Revision: D4204548
fbshipit-source-id: 6e21771
Summary:
This will help during the creation of new checkers, and will prevent errors like misspelling of operators' kinds.
It will also make it possible to fail immediately during the parsing of CTL inputs.
Reviewed By: ddino
Differential Revision: D4212956
fbshipit-source-id: c3c7fe7
Summary: This option is unused and it's making DB.source_file_* API harder to simplify
Reviewed By: cristianoc
Differential Revision: D4219803
fbshipit-source-id: 23ce697
Summary:
Sometimes, we'll need to have extra information in compilation database.
To keep the code simple just keep information about all files and filter
data when scheduling jobs. Perf shouldn't be much worse since we only start one
process for infer invocation.
Reviewed By: jvillard
Differential Revision: D4130884
fbshipit-source-id: 9920c11
Summary:
Using DB.source_file_to_string may return relative path which may or may not be relative to current location.
I went through all calls to `DB.source_file_to_string` and fixed ones that used that output to open files
Reviewed By: jeremydubreil
Differential Revision: D4205407
fbshipit-source-id: b285b7e
Summary:
Add test for compilation database and --changed-files-index option
Fix one bug that the test uncovered
Reviewed By: dulmarod
Differential Revision: D4198502
fbshipit-source-id: 9039c65
Summary:
This diff adds a dependency on ctypes.foreign and uses the example
binding of fts to implement remove_directory_tree.
Reviewed By: yunxing
Differential Revision: D4115569
fbshipit-source-id: 3509955
Summary: D4189956 killed the phantom space printed after types, but the Context leak message was relying on it :).
Reviewed By: jeremydubreil
Differential Revision: D4208591
fbshipit-source-id: 9f0d709
Summary:
Developers will sometimes write GuardedBy("T.f") with the intended semantics: "guarded by the field f of the object with type T in the current state".
We want to support this to avoid false positives.
Reviewed By: peterogithub
Differential Revision: D4197476
fbshipit-source-id: acd00d9
Summary:
- fix python calling function with wrong number of arguments (sic)
- print legend of analysis output, this was lost in translation ('F', '.', ...)
- add "Capturing in <mode> mode..." message before capture
- remove version from "Analyzing..." message (users don't even paste the full output, so this is not as useful as initially hoped)
Reviewed By: akotulski
Differential Revision: D4205072
fbshipit-source-id: 2b6505c
Summary:
The way interfaces are dealt with led to a false positive,
where tryLock() works OK for a Lock but not for a ReentrantLock.
The solution is just to provide the model.
While I am at it I am adding some more standard tests for Lock and ReentrantLock, which were not present.
Reviewed By: sblackshear
Differential Revision: D4204551
fbshipit-source-id: 9b6de28
Summary: These are dangerous if you are trying to compare a type to a string, and they're also unsightly.
Reviewed By: jvillard
Differential Revision: D4189956
fbshipit-source-id: 14ce127
Summary:
This diff implements enough of the functionality in the python code in
the OCaml toplevel driver that executing `infer -- analyze` is done
with direct procedure calls instead of forking the python interpreter.
Except for some reporting code that remains in report.py.
Reviewed By: jvillard
Differential Revision: D4074718
fbshipit-source-id: 56a794d
Summary: Using address equality check to short-circuit comparison of equal lists faster + kill use of `next`.
Reviewed By: jeremydubreil
Differential Revision: D4189581
fbshipit-source-id: bdf5d1e
Summary:
This will help porting more Python code over to OCaml. Since the reporting step
uses a lot of Python libraries that would be a pain to rewrite in OCaml (eg,
syntax highlighting), keep this functionality in Python and make it possible to
call it from OCaml as a script.
Reviewed By: jberdine
Differential Revision: D4182832
fbshipit-source-id: fc83220
Summary:
SIOF is only for interactions between objects of non-POD types. Previously the
checker was also reporting for POD types.
Reviewed By: akotulski
Differential Revision: D4197620
fbshipit-source-id: 7c56571
Summary:
Record an abstraction of the bug traces in the tests. The abstraction of a
trace is the sequence of descriptions. In practice, descriptions are either
empty, or of the form "start/end/return from/call to procedure X". They seem
pretty stable.
Motivation: there is nothing testing the traces reported by Infer right now,
even though they are surfaced to developers. For instance, Quandary uses
--issues-txt instead of --issues-tests to make sure the traces do not regress.
This change would make this approach more widespread.
Reviewed By: sblackshear
Differential Revision: D4159597
fbshipit-source-id: 9c83952
Summary:
clang errors would get diverted to log files instead of being printed on the
console. Also log other types of clang errors and warnings more often.
Before:
$ infer -- clang -c nosuchfile.c
$ # nothing gets printed, exit code set to 1
After:
$ infer -- clang -c nosuchfile.c
clang-4.0: error: no such file or directory: 'nosuchfile.c'
Also print more messages in case of compilation errors:
$ echo ')' >> a.c
$ infer -- clang -c a.c
a.c:1:1: error: expected identifier or '('
)
^
1 error generated.
Error: the following clang command did not run successfully:
'/home/jul/infer/facebook-clang-plugins/clang/install/bin/clang-4.0' "@/home/jul/infer/examples/infer-out/clang/clang_command_2a0a84.txt"
(only the last 2 lines are new)
Reviewed By: akotulski
Differential Revision: D4183039
fbshipit-source-id: b9a9065
Summary: Turns out there was also a problem with how this function deals with final newlines...
Reviewed By: jeremydubreil
Differential Revision: D4191154
fbshipit-source-id: fc10517
Summary: clang has very complicated logic what to translate based on `project_root` and filename. Add tests for different situations in regard of symbolic links in path/project_root
Reviewed By: jvillard
Differential Revision: D4168551
fbshipit-source-id: 586b364
Summary: Generalizing jvillard's awesome work to include passthroughs in traces, then calling it from Quandary.
Reviewed By: jvillard
Differential Revision: D4172108
fbshipit-source-id: 0296c59
Summary:
Outputting too many reports brings little value and spams the console. Don't
print more than 10 issues to stdout.
Reviewed By: jeremydubreil
Differential Revision: D4189814
fbshipit-source-id: 8985559
Summary:
Translate local variable names using the bytecode directly instead of JBir. The bytecode has more precise type information.
We still need to declare the temporary variables intruduced by Sawja until we can base the translation directly on the bytecode.
Reviewed By: sblackshear
Differential Revision: D4185309
fbshipit-source-id: 81904a0
Summary:
`make` doesn't delay variable evaluation in targets' dependencies, so
`$(OBJECTS)` was always empty. Including clang.make after having defined
`OBJECTS` fixes it.
Reviewed By: jberdine
Differential Revision: D4159522
fbshipit-source-id: 6925f8a
Summary: Refactoring to make thread safety checker interpocedural. This should not change funcitonality, and will only set things up for making the interprocedural part more serious.
Reviewed By: sblackshear
Differential Revision: D4124316
fbshipit-source-id: 6721953
Summary:
When `LC_CTYPE` has an invalid value, `locale.getdefaultlocale()` raises
`ValueError` and infer crashes, which is embarrassing. Catch these errors and
report appropriately.
fixes#483
Reviewed By: akotulski
Differential Revision: D4182880
fbshipit-source-id: 31edcf7
Summary: Only run the buck (and ant) tests if the tools can be found at ./configure-time.
Reviewed By: jberdine
Differential Revision: D4167586
fbshipit-source-id: e77b736
Summary:
When loading results from a json file, sort them. This prints results in some
sane order for both --issues-test and --issues-txt, removing the need for
post-processing of the result.
Reviewed By: cristianoc
Differential Revision: D4167029
fbshipit-source-id: 37e9f1c
Summary:
- rename java.make -> javac.make, config.make -> java.make, and move to infer/tests/ so it's easier to use from infer/tests/build_systems/
- use these from ant's test Makefile, much code reuse!
- factor out common functionality between java and clang
A wrinkle: sorting is now done the same way for --issues-tests and
--issues-txt, which produces bogus (but still as deterministic) sorting for
--issues-txt. This is more of a cosmetic issue, but I hope to fix it in a later
diff that gets rid of calls to `sort` in favour of sorting directly from
`InferPrint`.
Reviewed By: jberdine
Differential Revision: D4166841
fbshipit-source-id: ed6f232
Summary: The thread safety checker is run independently of other analyses, using the command "infer -a threadsafety -- <build-command>".
Reviewed By: sblackshear
Differential Revision: D4148553
fbshipit-source-id: bc7b3f9
Summary: Infer starts from bytecode generated by the compilation commands, so there isn't much need to check if the bytecode is compliant or not.
Reviewed By: sblackshear
Differential Revision: D4179218
fbshipit-source-id: 2f27148
Andrzej Kotulski
Jeremy Dubreil
Sam Blackshear
Cristiano Calcagno
Josh Berdine
Dulma Churchill
Dino Distefano
Martino Luca
Peter O'Hearn
Ryan Rhee
Jules Villard