Summary:
Infer reads the arguments passed to clang, in particular to filter out some
incompatible clang command-line options. But, infer only understands arguments
in arg files if they are presented one per line. That's usually the case, but
infer itself stores clang arguments from compilation databases all on one line.
This breaks filtering.
This changes the type of compilation database items to a list of arguments, so
that they can be stored one per line when possible.
Also does some cleanup/renamings, and remove trailing `_` from temp file names, eg:
clang_command_.tmp.2b2602.txt -> clang_command.tmp.2b2602.txt
Also, do not escape arguments from arg files when printing them in logs (that
was useless and ugly).
Reviewed By: mbouaziz
Differential Revision: D7365907
fbshipit-source-id: 5a3fe70
Summary:
The goal is not to end up with a deep copy when nothing has changed, as this
puts lots of pressure on the memory.
Reviewed By: mbouaziz
Differential Revision: D7113976
fbshipit-source-id: 1b85ecd
Summary:
There's no real reason not to use `Core` lists in this module. Changed the
interface to be more `Core`-like. Changed the `*_changed` functions to use a
ref to track changes instead of passing the changed state around.
Reviewed By: mbouaziz
Differential Revision: D7123211
fbshipit-source-id: b27791a
Summary: This makes the code cleaner, and also makes it easier to look at the type of a var (needed in a successor).
Reviewed By: jeremydubreil
Differential Revision: D7359466
fbshipit-source-id: 5acdb7a
Summary: The "advice" section of the error description does not seem to be used in practice.
Reviewed By: mbouaziz
Differential Revision: D7348815
fbshipit-source-id: 1d7c8c7
Summary:
- InferPrint reports time stats only, logged in `reporting_stats/` as well as a `PerformanceStats` event with `stats_type: reporting`
- Perf reporting can now be registered at one point (with desired start real and cpu times) and triggered later
Reviewed By: dulmarod
Differential Revision: D7332435
fbshipit-source-id: 2e17c7f
Summary:
Perf reporting can now be registered at one point (with desired start real and cpu times) and triggered later
- This is accomplished by storing perf file names in a hashtable mapping to the reporting function
- New `stats_kind` parameter that is either Time, Memory, or TimeAndMemory, where Time requires a Unix.process_times and an Mtime_clock.counter to be passed in, but TimeAndMemory just uses the time from the start of the process.
Reviewed By: dulmarod
Differential Revision: D7337158
fbshipit-source-id: 94699cd
Summary:
Perf reporting can now be registered at one point (with desired start real and cpu times) and triggered later
- This is accomplished by storing perf file names in a hashtable mapping to the reporting function
- New `stats_kind` parameter that is either Time, Memory, or TimeAndMemory, where Time requires a Unix.process_times and an Mtime_clock.counter to be passed in, but TimeAndMemory just uses the time from the start of the process.
Reviewed By: dulmarod
Differential Revision: D7337158
fbshipit-source-id: 3890cc7
Summary:
- PerfStats can now report either time and/or memory stats, by default reporting both
- This is in preparation for adding just time or memory related logging in the future
Reviewed By: mbouaziz, dulmarod
Differential Revision: D7324556
fbshipit-source-id: e265b12
Summary:
Show where the invalidation occurred in the trace.
Should make things easier to understand.
Reviewed By: jeremydubreil
Differential Revision: D7312182
fbshipit-source-id: 44ba9cc
Summary: This was causing false positives when returning the constant integer 0.
Reviewed By: sblackshear
Differential Revision: D7330143
fbshipit-source-id: 45d19dd
Summary: Should be no semantic change, just trying to avoid code duplication.
Reviewed By: jeremydubreil
Differential Revision: D7268588
fbshipit-source-id: 8b00125
Summary: This will make it easier to detect regressions, until we can completely solve the issue with the classpath and change the `No_class_found` exception into a hard failure.
Reviewed By: sblackshear
Differential Revision: D7307464
fbshipit-source-id: eab67fb
Summary: It adds an issue type, `BUFFER_OVERRUN_U5`, for alarms involving unknown values, i.e., when the trace set includes an unknown function call.
Reviewed By: mbouaziz
Differential Revision: D7178841
fbshipit-source-id: bfe857b
Summary:
At the moment, Java and Clang sources/sinks live in the same inferconfig entry.
If we try to parse a Java procedure that happens to be an invalid Clang qualified name (e.g., `MyClass.<init>`),
parsing will crash.
As a temporary fix, throw an exception and catch it instead.
In the future, we can avoid this by requiring that JSON source/sink specifications to indicate the language.
Reviewed By: mbouaziz
Differential Revision: D7291880
fbshipit-source-id: f8f4502
Summary: Add new clang_method_kind field to AnalysisIssue, logged similarly to the existing one in AnalysisStats
Reviewed By: dulmarod
Differential Revision: D7273660
fbshipit-source-id: d1ca79b
Summary:
- Rather than passing a directory name to PerfStats reporting api to write the stats to, and then determining the EventLogger `stats_type` from that dirname, there is now a `stats_type` type in PerfStats, and the directory name and EventLogger tag is automatically determined from that type after it is passed to the API
- This allows us to determine whether we're in capture mode or linters mode dynamically from PerfStats itself, and use the appropriate tag when creating an event
Reviewed By: dulmarod
Differential Revision: D7251580
fbshipit-source-id: 0dec071
Summary:
Aggregate initialization (e.g., `S s{1, 2}`) doesn't invoke a contructor.
Our frontend translates aggregation initialization as assigning to each field in the struct.
To avoid the appearance of the struct being uninitialized, count any assignment to a field of an aggregate struct as initializing the struct.
Reviewed By: jeremydubreil
Differential Revision: D7189671
fbshipit-source-id: ace02fc
Summary: Backend perf stats were previously being logged as both `driver_stats` and `backend_stats`. Now they are correctly logged only as `backend_stats`, and a single `driver_stats` exists for the top-level driver process.
Reviewed By: dulmarod
Differential Revision: D7195879
fbshipit-source-id: 0f2ddc0
Summary:
Previously, `backend_stats` were getting logged correctly only when `infer analyze` was directly called, not `infer run`. Now, we report `backend_stats` directly, as part of the `iterate_callbacks` function in the task passed to the `ProcessPool`.
As a side benefit, `aggregated_stats` are also logged correctly now.
Reviewed By: dulmarod
Differential Revision: D7195525
fbshipit-source-id: fb2a400
Summary: It corrects a precision bug in the interval domain, with adding some test cases.
Reviewed By: mbouaziz
Differential Revision: D7230918
fbshipit-source-id: 3ec641a
Summary:
:
Previously, we did not have information about type of `exp` in `sizeof exp` from clang plugin which led to `Bad_footprint` errors. Infer did not understand `sizeof *p` in `struct Person* p = malloc(sizeof *p);` and used some default type.
This resulted in `Bad_footprint` error when trying to assign to a field `age` in `p->age=42;`.
This diff uses the version of clang plugin which exports the appropriate type information.
update-submodule: facebook-clang-plugins
Reviewed By: dulmarod
Differential Revision: D7179870
fbshipit-source-id: 4104f10
Summary:
Show some `SymAssign`s (corresponding to parameters) in the trace.
Depends on D7194448
Reviewed By: skcho
Differential Revision: D7194479
fbshipit-source-id: 0deff6c
Summary:
- Noticed that there were two different type aliases for the same type, representing the return value of `__POS__`
- Combined them under `ocaml_pos` name which more closely matches the pervasive
- Moved to Logging module
Reviewed By: dulmarod
Differential Revision: D7194034
fbshipit-source-id: 22cb949
Summary:
- PerformanceStats rows will be logged to EventLogger regardless of whether Infer is in developer mode
- PerformanceStats files will still not be created unless Infer is in developer mode
Reviewed By: dulmarod
Differential Revision: D7169403
fbshipit-source-id: 85bd7de
Summary:
- New Event type in EventLogger: PerformanceStats
- Contains some of the information logged to disk by PerfStats, as well as a `stats_type` tag to indicate which type of process the stats are from
- PerfStats.register_report_at_exit now takes 3 string arguments and constructs the file path and determines stats_type based off of them
- Changed all clients of PerfStats to use new api
Reviewed By: dulmarod
Differential Revision: D7131904
fbshipit-source-id: 9226b5d
Summary: It corrects a bug that `&(x.f[n])` was evaluated to `&(x.f[0])`.
Reviewed By: mbouaziz
Differential Revision: D7179620
fbshipit-source-id: 04cbaa7
Summary: It simply resizes the target structure instead of allocating new heap memories and copying values.
Reviewed By: mbouaziz
Differential Revision: D7179353
fbshipit-source-id: 9c20f64
Summary: If a `Closure` expression `e` captures variable `x`, consider `e` as borrowing from `x`. When the closure is invoked via `operator()`, check that the borrow is still valid.
Reviewed By: jeremydubreil
Differential Revision: D7071839
fbshipit-source-id: d923a6a
Summary: It avoids that locations of array fields are evaluated to the `unknown` location incorrectly by addressing the case in the `eval_lindex` function.
Reviewed By: mbouaziz
Differential Revision: D7152736
fbshipit-source-id: 2dc825e
Summary: It collects array accesses from all sub expressions in commands.
Reviewed By: mbouaziz
Differential Revision: D7165098
fbshipit-source-id: 584dc80
Summary: It does not only malloc a new heap memory, but also copy its contents.
Reviewed By: mbouaziz
Differential Revision: D7152194
fbshipit-source-id: 58cba5e
Summary:
- Add support for floating point values to JsonBuilder
- This allows EventLogger to log fields of type float
Reviewed By: dulmarod
Differential Revision: D7138542
fbshipit-source-id: 8a4e353
Summary: This is to make sure than the analysis produces the same results independently from the order in which the members of a call cycle are analyzed.
Reviewed By: sblackshear, mbouaziz
Differential Revision: D6881971
fbshipit-source-id: 23872e1
Summary: Add a new command-line option `--external-java-packages` which allows the user to specify a list of Java package prefixes for external packages. Then the analysis will not report non-actionable warnings on those packages (e.g., inconsistent `Nullable` annotations in external packages).
Reviewed By: jeremydubreil
Differential Revision: D7126960
fbshipit-source-id: c4f3c7c
Summary: When a property was defined in a protocol, we were not translating its attributes which leads to retain cycles false positives. This diff fixes it. It also refactors the code for translating fields a bit.
Reviewed By: mbouaziz
Differential Revision: D7136355
fbshipit-source-id: b5e7445
Summary:
A matcher that matches procnames only.
To be used in bi-abduction.
Depends on D7124877
Reviewed By: skcho
Differential Revision: D7125113
fbshipit-source-id: e44f3a8
Summary:
Because it matches calls (procnames and argument list) and because I will create a `Procname`-only one.
Depends on D7124847
Reviewed By: skcho
Differential Revision: D7124877
fbshipit-source-id: eca7c21
Summary:
Fairly simple approach here:
- If the RHS of an assignment is a frontend-generated temporary variable, assume it transfers ownership to the LHS variable
- If the RHS of an assignment is a program variable, assume that the LHS variable is borrowing from it.
- If we try to access a variable that has borrowed from a variable that is now invalid, complain.
Reviewed By: jeremydubreil
Differential Revision: D7069947
fbshipit-source-id: 99b8ee2
Summary:
Just a bunch of minor changes.
- The mutable field doesn't need to be mutable: it's only mutated once in the code somewhere that doesn't even need to mutate
- Hence the type can be public
- All the `ms_get_*` functions are replaced by field accesses
- `CMethod_signature` -> `CMethodSignature` while I'm at it, although maybe that's bad since other files in clang/ follow the former convention
- `type method_signature` -> `type t`
- `pp` function instead of `to_string`, since it's used with `fprintf`. This gets rid of the only caller of `IList.to_string`.
Reviewed By: dulmarod
Differential Revision: D7123795
fbshipit-source-id: fdfae42
Summary:
`Sequence` API to walk over free variables in expressions, instead of computing lists with uniqueness constraints that make them have linear complexity for insertion.
Switch to a Set representation when we don't care about the order of elements,
otherwise to a `Hash_queue`:
https://ocaml.janestreet.com/ocaml-core/113.33/doc/core/Std/Hash_queue.mod/S.modt/
Often, we don't even need to compute the sequence of free variables, as we are
just testing membership/emptiness/...
Reviewed By: mbouaziz
Differential Revision: D7099294
fbshipit-source-id: e96f84b
Summary:
At function calls, it copies a subset of heap memory that is newly
allocated by callees and is reachable from the return value.
Reviewed By: mbouaziz
Differential Revision: D7081425
fbshipit-source-id: 1ce777a
Summary:
This is to fix the conflicts between Eradicate and the Biabduction when reporting the same kind of errors: when Eradicate is on, the Eradicate warnings will have priority over the null deference reported by the biabduction.
If this approach proved to be successful in prod, I will refactor the reporting mechanism in the analysis itself to simply not report the null dereference in this case at all. For the codebases that aren't yet fully consistently using `Nullable`, this combined approach looks like a good way to deploy Infer toward full null safety.
Reviewed By: mbouaziz
Differential Revision: D7102119
fbshipit-source-id: 35d3add
Summary:
Enrich capability domain with borrowing info. Inline comments explain what the domain is doing.
The analyzer isn't actually using the borrowing functionality yet--that comes in the successor.
Reviewed By: jeremydubreil
Differential Revision: D7067942
fbshipit-source-id: 4c03c69
Summary:
Before D7100561, the frontend translated capture-by-ref and capture-by-value in the same way.
Now we can tell the difference and report bugs in the capture-by-value case.
Reviewed By: jeremydubreil
Differential Revision: D7102214
fbshipit-source-id: e9d3ac7
Summary:
The goal is to allocate less and generally be more efficient than handling
lists with uniqueness constraints.
Reviewed By: mbouaziz, jberdine
Differential Revision: D7098904
fbshipit-source-id: 7111f07
Summary:
The `may_last_field` boolean value in the `decl_sym_val` function presents that the location *may* (not *must*) be a flexible array member.
By the modular analysis nature, it is impossible to determine whether a given argument is a flexible array member or not---because of lack of calling context. For example, there are two function calls of `foo` below: (2) passes a flexible array member as an argument and (1) passes a non-flexible array, however it is hard to notice when analyzing the `foo` function.
```
struct T {
int c[1];
};
struct S {
struct T a;
struct T b;
};
void foo(struct T x) { ... }
void goo () {
struct S* x = (struct S*)malloc(sizeof(struct S) + 10 * sizeof(int));
foo(&(x->a)); // (1)
foo(&(x->b)); // (2)
}
```
We assume that any given arguments may stem from the last field of struct, i.e., flexible array member. (This is why `decl_sym_val` is called with `may_last_field:true` at the first time.) With some tests, we noticed that the assumption does not harm the analysis precision, because whether regarding a parameter as a flexible array member or not is about using a symbolic array size instead of a constant array size written in the type during the analysis of callee. Therefore still it can raise correct alarms if the actual parameter is given in its caller.
Reviewed By: mbouaziz
Differential Revision: D7081295
fbshipit-source-id: a4d57a0
Summary:
Instead of storing the type environment in infer-out/captured/foo.c/foo.c.tenv,
store it in the `source_files` table of the SQLite db. This limits the number
of files we create on disk.
The "file local" type environemnts are specific to the clang integration. For
Java, there is a "global tenv" file. Instead of matching on string names, this
diff also makes the API of `Tenv` reflect this situation.
The global tenv is serialized as a separate file in "infer-out/.global.tenv"
instead of "infer-out/captured/global.tenv", because "infer-out/captured/" will
soon be removed as it now only contains the global tenv (except in debug mode,
where it will still be created).
In the DB, we either store the local tenv for the file, or "global" to indicate
that the global tenv should be consulted.
This diff also moves `Cfg.store` to `SourceFiles.add` because that function
deals with more than just `Cfg.t`.
Reviewed By: jeremydubreil
Differential Revision: D6937945
fbshipit-source-id: 001c10a
Summary:
A list of free variables was computed and passed around, but never used. The
reason OCaml wasn't complaining is that the list is eventually passed to a
recursive function, which passes it to its recursive call, so it looks like
it's used there.
Reviewed By: mbouaziz
Differential Revision: D7098556
fbshipit-source-id: b22a591
Summary:
Switch to the current stable branch for clang.
update-submodule: facebook-clang-plugins
Reviewed By: mbouaziz
Differential Revision: D7067890
fbshipit-source-id: aedff90
Summary:
You can capture a variable by reference in a lambda, assign to it, and then invoke the lambda.
This looks like a dead store from the perspective of the current analysis.
This diff mitigates the problem by computing an additional analysis that tracks variables captured by ref at each program point.
It refuses to report a dead store on a variable that has already been captured by reference.
Later, we might want to incorporate the results of this analysis directly into the liveness analysis instead of just using it to gate reporting.
Reviewed By: jeremydubreil
Differential Revision: D7090291
fbshipit-source-id: 25eeffa
Summary:
We frequently want to treat such vars differently than true program variables.
Used in only one place now, but will be used in a successor.
Reviewed By: mbouaziz, jvillard
Differential Revision: D7067882
fbshipit-source-id: 90e0348
Summary:
As far as I can tell nothing uses it and it is ignored by the matching engine:
the "parameters" arguments are collected from config options but never actually
used for matching.
Reviewed By: sblackshear
Differential Revision: D6976273
fbshipit-source-id: 6fed5ff
Summary:
Ran into this issue on Debian Testing, in which assert.h is probably different
due to a more recent toolchain. Without this change I get the following CFG
for `assert(e)`:
```
start
|-> prune (e) -> join
|-> prune (!e) -> __infer_fail("ASSERTION_FAILURE") -> exit
```
Notice that the first branch does not get to the exit, so infer must think that
the assertion is *always* violated, and reports `error: ASSERTION_FAILURE`.
This is broken.
Reviewed By: dulmarod
Differential Revision: D7067822
fbshipit-source-id: a2bf5ac
Summary:
- EventLogger no longer depends on ProcAttributes
- This allows for other backend classes which are used by ProcAttributes to log in the future without creating a cyclic dependency structure
Reviewed By: dulmarod
Differential Revision: D7082444
fbshipit-source-id: f32a6e2
Summary: - When a CallTrace result is CR_skip, then a reason for skipping the call will be included in the record
Reviewed By: dulmarod
Differential Revision: D7082572
fbshipit-source-id: 20ed191
Summary:
It supports flexible array member using the following heuristic:
- a memory for a class is allocated by `malloc(sizeof(C) + n * sizeof(T))` format
- the last field of the class is an array
- the static size of the last field is one, i.e., `T field_name[1]`
When allocating and initializing members of classes, it sets the size of flexible array to `n+1` if the above conditions are met.
Reviewed By: mbouaziz
Differential Revision: D7056291
fbshipit-source-id: 31c5868
Summary:
The semantics of "placement new" is defined simply as an assignment.
For example, `C* x = new (y) C();` is analyzed as if `C* x = y;`.
Reviewed By: mbouaziz
Differential Revision: D7054007
fbshipit-source-id: 1c6754f
Summary:
It abstracts the environments required in modeling functions and type
declarations as a record type.
Reviewed By: mbouaziz
Differential Revision: D7065996
fbshipit-source-id: b60cd3c
Summary:
The struct fields in Cil have been sorted for long time, however the
checkers do not seem to depend on the sortedness.
Reviewed By: sblackshear
Differential Revision: D7027858
fbshipit-source-id: 9e7ab96
Summary:
- During the symbolic execution stage of the backend, Infer will log detailed stats about procedure calls
- Logging is accomplished directly within symExec/Tabulation
- call_result type is moved to tabulation.ml
CallStats was a broken module that allocated a lot of useless memory. Now, Specs.CallStats and InferPrint.CallsCsv as well as the Calls report kind in InferPrint no longer exist.
Reviewed By: dulmarod
Differential Revision: D7016439
fbshipit-source-id: 40911ee
Summary:
This commit improves precision of symbol instantiations.
When a return value of a callee is `[s1 + s2, _]` and if we want to
instantiate `s1` to `c3 + max(c4, s5)`, the lower bound was
substituted to `-oo` because our domain cannot express `c3 + max(c4,
s5) + s2`.
However, we can have instantiations that are preciser than `-oo`:
(1) `c3 + c4 + s2`
(2) or `c3 + s5 + s2`
because they are smaller than the ideal instantiation, `c3 + max(c4,
s5) + s2` and it is on the lower bound position.
For now, the implementation instantiates to (1) between the two ones,
because constant values introduced by `assert` or `assume`(`if`)
command are often used as safety conditions, e.g., `assert(index >=
0);` can place before array accesses. (We can change the stratege
later if we find that it doesn't work on some other cases.)
Reviewed By: mbouaziz
Differential Revision: D7020063
fbshipit-source-id: 62fb390
Summary:
- StatsLogs is now its own functional module in InferPrint
- No longer stores events to be logged in a list to be later logged, just immediately logs them
Reviewed By: dulmarod
Differential Revision: D7010559
fbshipit-source-id: 1837ca9
Summary: Just some minor renaming to be more consistent with other modules. I was about to use these modules and was too lazy to type `Ident.IdentSet`.
Reviewed By: da319, avarun42
Differential Revision: D6999808
fbshipit-source-id: c24edef
Summary:
- change a use of clang_method_kind to match directly on the type rather than a brittle check for equality
- make the clang_method_kind field in AnalysisStats an option
- if procedure lang is not clang, then clang_method_kind field gets skipped while logging
Reviewed By: dulmarod
Differential Revision: D7010060
fbshipit-source-id: 077094d
Summary:
- JsonBuilder now has an add_string_opt function that takes a string option instead of a string
- Further alphabetized EventLogger
Reviewed By: dulmarod
Differential Revision: D7011763
fbshipit-source-id: 73f1a4b
Summary:
A simple intraprocedural analysis that tracks when a storage location is read or deleted.
For now, this works only with local variable storage locations; field and array accesses are ignored.
In order to test this, I added a new "use-after-lifetime" warning. It complains when a variable is read or deleted after it has already been deleted.
Reviewed By: jeremydubreil
Differential Revision: D6961314
fbshipit-source-id: 75e95a2
Summary: We do not inject a destructor call if the destructor declaration does not contain a body in AST. We miss all the cases where the destructor is declared in `.h` file and defined in `.cpp` file as other files include `.h` file and do not contain the body of the destructor when destructor calls are being injected based on AST information. After this diff we inject destructor calls even if we do not have body for the destructor in AST.
Reviewed By: sblackshear
Differential Revision: D6796567
fbshipit-source-id: 1c187ec
Summary: More preparation for extending HIL with dereference and address of. We need left hand side of the assignment to also include dereference and address of.
Reviewed By: sblackshear
Differential Revision: D6976150
fbshipit-source-id: 47d1d76
Summary:
It prunes abstract memories on `assert` commands.
Problem: Since the assert command is sometimes translated to two
sequential `if` statments, it was not able to prune the memory
precisely at `assert` commands in Inferbo---the pruned memory at the
first branch was joined before the second branch.
Solution: To avoid losing the pruning information at the first branch,
now, it records which locations are pruned at the first branch and
applies the same pruning at the next branch if they have
semantically the same condition.
Reviewed By: mbouaziz
Differential Revision: D6895919
fbshipit-source-id: 15ac1cb
Summary:
See comment--`Prop(resType = blah) myProp` will generate `.myProp`, `.myPropRes`, and `.myPropAttr`, and any of them can be used to set the prop.
Because our annotation parameter parsing is a bit primitive, handle this by simply checking the `Res` and `Attr` suffixes for every `Prop`.
This shouldn't lead to false negatives because these methods will only exist if the `resType` annotation is specified anyway.
Reviewed By: jeremydubreil
Differential Revision: D6955362
fbshipit-source-id: ec59b21
Summary: InferPrint should not generate StatsLogs format at all unless Config.log_events is set to true
Reviewed By: mbouaziz, dulmarod
Differential Revision: D6976352
fbshipit-source-id: d12005a
Summary: In Obj-C blocks, we explicitly insert reads of the captured vars. This does the same thing for C++. For example, `foo() { int x = 1; [x]() { return x; } }` would previously not contain a read of `x` in `foo`. Now, we'll create a temporary that reads from `x` and pass it to the closure value.
Reviewed By: dulmarod
Differential Revision: D6939997
fbshipit-source-id: f218afc
Summary:
1) Fixes some false negatives when a method annotated with `nullable` in the header is not annotated in the implementation and the attribute lookup returns the implementation. In that case, we should follow the information given in the header.
2) Fixes some false positives when annotations are in the other way around, i.e. annotated in the implementation but not in the headers. For now, there should be no report in this case, but the analysis should be extended to report the inconsistency between the header and the implementation
3) Fixes some cases of weird reports caused by name conflicts where the method in the include has the same name has another method annotated differently.
Reviewed By: jvillard
Differential Revision: D6935379
fbshipit-source-id: 3577eb0
Summary: Preparing to extend HIL with Dereference and AddressOf expressions. Next steps: (1) change SIL -> HIL translation to preserve address of and dereference; (2) adapt analyses based on HIL to make use access expressions.
Reviewed By: jeremydubreil
Differential Revision: D6961928
fbshipit-source-id: 51da919
Summary:
Added a check for recursive calls not to add abduced reference parameters constraints. Abduced reference parameters constraints were causing assertion failure when renaming variables in specs, in particular, when transforming variables into callee variables.
A similar check is already in place for abduced retvals constraints.
Reviewed By: jeremydubreil
Differential Revision: D6856919
fbshipit-source-id: acfe840
Summary:
This allows Eradicate to lookup the annotations from the classpath and without requiring the code in the classpath to have been previously analyzed. The benefit is that source files can be analyzed independently of each other as long as the classpath is known.
The main goal is to run be able to run Eradicate as a linter without losing warnings.
We may have to add some more models of the standard libraries as no `Nullable` on a parameter does not necessarily mean that the method does not accept `null`.
Reviewed By: sblackshear
Differential Revision: D6921720
fbshipit-source-id: f525269
Summary: Some tags like `Bucket` are used, but a lot are just added to the list of tags and never read.
Reviewed By: mbouaziz
Differential Revision: D6886980
fbshipit-source-id: 4474d7f
Summary:
There's a lot of code for building up and moving around `Tags`.
When working on cleaning up some of the `Errlog` code, I noticed that `Tags` were included in the JSON and wondered why.
The answer is suprisingly just one thing: only the line tags get used, and even then they are only used to decide what frame to select as the start frame for the trace (i.e., the one that is highlighted first).
That seems like overkill; starting on trace on the actual line where the error occurs, starting at the beginning of the procedure where the error occurs, or starting at the first line of the trace all seem equally reasonable.
If we are happy with any of these alternatives, we can kill `Tags` altogether and potentially save a decent amount space in our JSON artifacts.
Reviewed By: mbouaziz
Differential Revision: D6876752
fbshipit-source-id: 1580127
Summary:
Useful to debug without having to rerun commands.
Looks like this:
```
Usage Error: Failed to execute compilation command:
'/home/jul/infer.fb/infer/bin/../../facebook-clang-plugins/clang/install/bin/clang'
'args.txt'
++Contents of 'args.txt':
'arg1.txt' 'arghorashy_hello_c.txt' 'args.txt'
++Contents of 'arg1.txt':
'arghorashy_c.txt'
++Contents of 'arg_c.txt':
'-c'
++Contents of 'arg_hello_c.txt':
'examples/hello.c'
Error message:
clang-5.0: error: no such file or directory: 'examples/hello.c'
*** Infer needs a working compilation command to run.
```
Reviewed By: mbouaziz
Differential Revision: D6872550
fbshipit-source-id: 65cd026
Summary:
- Combine two fields from ProcAttributes.t into a single field `method_kind` with more information
- New field details whether the procedure is an `OBJC_INSTANCE`, `CPP_INSTANCE`, `OBJ_CLASS`, `CPP_CLASS`, `BLOCK`, or `C_FUNCTION`
- `is_objc_instance_method` and `is_cpp_instance_method` fields no longer necessary
- Changed `is_instance` field in CMethod_signature to `method_kind` field of type ProcAttributes.method_kind
Reviewed By: dulmarod
Differential Revision: D6884402
fbshipit-source-id: 4b916c3
Summary: If the procedure is defined, then the attributes should be the same on the specs files or on the attributes table.
Reviewed By: sblackshear
Differential Revision: D6910086
fbshipit-source-id: 709b290
Summary:
- small optimization by starting deconstructing procnames/types in the dispatcher rather than the matchers
- as a consequence, returns fast for unhandled constructs like Java procnames or types
- Java is still not handled but at least does not crash
- re-enable Inferbo for Java
Reviewed By: jberdine
Differential Revision: D6912304
fbshipit-source-id: 76e95a8
Summary:
In some cases infer runs clang commands where we do not attach the plugin,
notably for pre-processor-only commands like `clang -E foo.c`. Usually these
commands are used by build systems to test their output, so infer should not
swallow it.
This makes infer echo the stdout of clang whenever we do not attach the plugin
to clang. When the plugin is attached, stdout is captured because that's where
the plugin outputs its results, so we cannot do the same in this case (not that
we would want to yet).
Fixes#696.
Reviewed By: mbouaziz
Differential Revision: D6912101
fbshipit-source-id: c4ad2e4
Summary:
Record "capture phases" in the runstate and in the source files table of the
database. Use this instead of filesystem timestamps to decide which files need
re-analyzing in the reactive analysis.
Reviewed By: jeremydubreil
Differential Revision: D6760833
fbshipit-source-id: 7955621
Summary:
The boolean lock domain is simple and surprisingly effective.
But it's starting to cause false positives in the case where locks are nested.
Releasing the inner lock also releases the outer lock.
This diff introduces a new locks domain: a map of locks (access paths) to a bounded count representing an underapproximation of the number of times the lock has been acquired.
For now, we just use a single dummy access path to represent all locks (and thus a count actually would have been sufficiently expressive; we don't need the map yet).
But I'm planning to remove this limitation in a follow-up by refactoring the lock models to give us an access path.
Knowing the names of locks could be useful for error messages and suggesting fixes.
Reviewed By: jberdine
Differential Revision: D6182006
fbshipit-source-id: 6624971
Summary: There's a new `ocaml_pos` type that the other clang frontend exceptions use, but Self.SelfClassException still used the raw tuple. Now, SelfClassException also uses this type.
Reviewed By: dulmarod
Differential Revision: D6900258
fbshipit-source-id: 94c7042
Summary:
- During backend execution, infer will log detailed stats about procedure analysis
- Logging is integrated with EventLogger
- `events_to_log` field added to Stats.t record in InferPrint
- New format in InferPrint - Logs
- `format_list` type changed to have a Utils.Outfile option to support Logs format
Reviewed By: dulmarod
Differential Revision: D6834538
fbshipit-source-id: 8c847f5
Summary:
Previously, we could understand than an access was safe either because it was possibly owned or protected by a thread/lock, but not both. If an access was both protected by a lock and rooted in a paramer (i.e., possibly owned), we would forget the ownership part of the precondition and remember only the lock bit. This leads to false positives in cases where an access protected by a lock is owned, but another unowned access to the same memory is not protected by a lock (see the new `unownedLockedWriteOk` E2E test for an example).
This diff makes access safety conditions disjunctive so we can simultaneously track whether an access is owned and whether an access is protected by a thread/lock. This will fix false positives like the one explained in T24015160.
Reviewed By: jberdine
Differential Revision: D6671489
fbshipit-source-id: d17715f
Summary:
- ast_node argument is now optional in functions `unimplemented` and `incorrect_assumption` in cFrontend_config
- The argument type was already an option, and the majority of the calls were with 'None'. This makes the function more intuitive to use
Reviewed By: sblackshear
Differential Revision: D6846141
fbshipit-source-id: 13deb8f
Summary:
We already knew not to warn when non-resource `Closeable`'s like `ByteArrayOutputStream` weren't closed, but we still warned on their subtypes.
This diff fixes that problem by checking subclasses in the frontend.
This also removes the need for Java source code models of non-resource types, so I removed them.
Reviewed By: jeremydubreil
Differential Revision: D6843413
fbshipit-source-id: 60fe7fb
Summary: The heuristics to determine the end of a block/procedure was too brittle, the new one ignores non significant instructions.
Reviewed By: jvillard
Differential Revision: D6845380
fbshipit-source-id: feab557
Summary:
The infer results directories in buck-out/ are "cleaned up" to avoid polluting
the Buck cache with too much data or non-deterministic data. In particular, the
runstate is deleted, which confused subsequent infer processes trying to read
the pre-existing results directory.
Add a special case in infer to delete pre-existing results directories in
buck-out instead of trying to load their state.
Reviewed By: mbouaziz
Differential Revision: D6845128
fbshipit-source-id: 5c716aa
Summary:
- `NonZeroInt` for added guarantees on the invariants of `SymLinear` coefficients
- some simplifications
- some optimizations
Reviewed By: jvillard
Differential Revision: D6833968
fbshipit-source-id: 39e28a0
Summary:
Make dead code detection part of `make test` so that dead code stops creeping
in. It's only enabled if all the analysers are enabled and if this is a
facebook build, because the dead code detection will have false positives
otherwise.
Reviewed By: mbouaziz
Differential Revision: D6807395
fbshipit-source-id: ebbd835
Summary:
This diff fixes the translation of `new` and `placement new` with one argument. If `placement new` has more than one argument it means that it is user-defined (this will be addressed in another diff).
update-submodule: facebook-clang-plugins
Reviewed By: sblackshear, mbouaziz
Differential Revision: D6807751
fbshipit-source-id: 7cf0290
Summary:
Before this diff, the `Ondemand` module would not cache the results of the function `analyze_proc_desc`, which is used by the toplvel iteration.
This should not have any effect on the performances at this point as the summaries were already cached in the `Specs` module. Now, we can start remove the use of the cache in the `Specs` module to avoid the duplication. Caching at the level of `Ondemand` is better as we can safely cache the information that the outcome of the analysis is `None`, which avoids scanning the filesystem or the DB multiple times.
Reviewed By: mbouaziz, jvillard
Differential Revision: D6713546
fbshipit-source-id: 309701b
Summary: This should allow to report several occurences of the an issue appearing several times within the same method.
Reviewed By: jvillard
Differential Revision: D6783298
fbshipit-source-id: 5555906
Summary: It was getting a bit difficult to tell which functions belonged where.
Reviewed By: jvillard
Differential Revision: D6764764
fbshipit-source-id: f9faada
Summary:
This lets us fix the limitation of reporting false positives when a `private` function calls `build()` on a parameter without passing all of the required props.
We will now report such issues in the caller only if it fails to pass the required props.
An unfortunate consequence of this change is that we lose track of where the actual call to `build` occurs--we now report on the declaration of the caller function rather than on the call site of `build`.
I'll work on addressing that in a follow-up.
Reviewed By: jeremydubreil
Differential Revision: D6764153
fbshipit-source-id: 3b173e5
Summary:
The captured variables of a closure are tuples (id, var, typ) with the implicit assumption
that &var -> id holds in the heap. This is true when the closure is created, but is not enforce otherwise.
This becomes a problem when the closure is stored in the heap, goes passed a bi-abduction, and then it's executed
(see new test). This was failing before this diff and now succeeds.
We add the verification of this constraint to the normalization of sigma.
At the moment I expect Precondition_not_met to be removed, but also later, we will be able to compute retain cycles
over the closures, as the correct captured variable info is kept through the execution.
Reviewed By: jvillard
Differential Revision: D6796525
fbshipit-source-id: a8a7655
Summary:
Not sure what an "iCFG" is but the dotty is only about CFGs anyway.
Diff obtained by mass-`sed`.
Reviewed By: sblackshear
Differential Revision: D6324280
fbshipit-source-id: b7603bb
Summary:
Also make it optional, since it's only used for debug messages. Name a couple
more of these for other similar functions.
Reviewed By: sblackshear
Differential Revision: D6797385
fbshipit-source-id: e6e9b2e
Summary:
I needed to do this for something, now I don't know if I want to do the thing
anymore but this seems generally useful to decrease a little bit the size of
Config.ml.
Reviewed By: sblackshear, mbouaziz
Differential Revision: D6796427
fbshipit-source-id: d9c009d
Summary:
Also, always log failures.
This also shows that the dead code detection does not detect dead exceptions :/
Reviewed By: jeremydubreil
Differential Revision: D6796843
fbshipit-source-id: 3d0ff5c
Summary:
Also, make it explicit when we load the global tenv instead of the per-file tenv.
This allows for some nice simplifications in some places, notably:
- `tenv_file` is gone from `Exe_env.file_data`
- `DB.global_tenv_fname` is no more
This will help moving the tenv from the capture/source_file/ directories on the
filesystem to the database, as keys for the relevant table are `SourceFile.t`.
Reviewed By: mbouaziz
Differential Revision: D6796594
fbshipit-source-id: 1ffd5b0
Summary:
The custom exception does not appear to serve any purpose. Also refactor the
complaining code to avoid duplication.
Reviewed By: mbouaziz
Differential Revision: D6784945
fbshipit-source-id: a2d969b
Summary:
They were constructed for each source file, and then joined into a global call
graph, only to get per-file lists of procedures. A tad wasteful.
Get this list from cfgs instead. Still record them in `exe_env` for now as
changing that code is a whole other beast.
One test falls victim of the flakiness of the analysis of recursive functions.
Reviewed By: jeremydubreil, mbouaziz
Differential Revision: D6324268
fbshipit-source-id: d5ff58f
Summary:
Last piece of significant logic relying on the call graph: the execution
environment. Mostly carve out the logic to detect duplicate symbols to not rely
on the call graph.
Also make the keys of the `file_map` be source files, because not having the
"cg filename" makes it harder otherwise.
Reviewed By: sblackshear
Differential Revision: D6621196
fbshipit-source-id: cab50ba
Summary:
In preparation for getting rid of call graphs, we need to find another way to
get the list of defined procedures (which is the only place where we use the
globally-computed call graph for now).
The natural way to get the list of procedures defined in a file is to load the
cfg for that file and look at the proc names that are the keys of the cfg. This
is way too expensive, as the CFG is big. Thus, we cache this list of proc names
as another column in the SQLite database of cfgs. This gives good performance
in benchmarks.
Reviewed By: jeremydubreil
Differential Revision: D6621142
fbshipit-source-id: ed265fe
Summary: At each call to `Component$Builder.build()`, checks that the required props for `Component` have been set via prior calls to the `Builder`. Does not yet handle `Prop(optional = true)`, but will address that in a follow-up.
Reviewed By: jeremydubreil
Differential Revision: D6735524
fbshipit-source-id: 0c812fd
Summary:
Record the db schema, infer version, and run dates into
infer-out/.infer_runstate.json. This allows us to check on startup whether the
results directory was generated using a compatible version of infer or not, and
give a better error message in the latter case than some SQLite error about
mismatching tables.
This will be used in a follow-up diff to record capture phases too, and avoid
relying on filesystem timestamps of the infer-out/capture/foo/ directories for
reactive analysis.
Had to change some tests Makefiles to make sure they do not attempt to re-use
stale infer-out directories, which would now fail the run.
The stale infer-out directory gets deleted if `--force-delete-results-dir` is
passed (but a warning still gets printed).
Reviewed By: mbouaziz
Differential Revision: D6760787
fbshipit-source-id: f36f7df
Summary:
This declutters `CommandLineOption` a little bit, and will be useful in a
follow-up diff where `InferCommand.t` will be used from an atd-generated file.
Reviewed By: mbouaziz
Differential Revision: D6772990
fbshipit-source-id: 3d32d00
Summary:
Previously imports with relative filenames would not get resolved so the result
would depend on where infer had been run from. Usually this was the project
root. Now, resolve path names of imports relative to the file doing the
`#IMPORT`. This changes behaviour most of the time.
Reviewed By: dulmarod
Differential Revision: D6784740
fbshipit-source-id: 4ccb7bf
Summary:
This changes the syntax for AL imports from `#IMPORT <file>` to `#IMPORT
"file"`. As a side-effect, the `file` part is now lex'd more permissively too.
Reviewed By: dulmarod
Differential Revision: D6784669
fbshipit-source-id: cc1bb73
Summary:
- During program translation, infer logs details about SelfClassException exceptions that are caught
- Logging is integrated with EventLogger library, uses existing FrontendException event type
- ast_node field in FrontEndException record used to store SelfClassException class_name field
- SelfClassException exception type extended to add support for storing exception details
- All instances where SelfClassException exception is raised modified to pass these details
Reviewed By: mbouaziz
Differential Revision: D6760513
fbshipit-source-id: a8efa9d
Summary:
- During program translation, infer logs details about IncorrectAssumption exceptions that are caught
- Logging is integrated with EventLogger library, uses existing FrontendException event type
- IncorrectAssumption exception type extended to add support for storing exception details
- All instances where IncorrectAssumption exception is raised modified to pass these details
Reviewed By: mbouaziz
Differential Revision: D6759287
fbshipit-source-id: 64f520e
Summary:
Reading a different .inferconfig makes little sense and is in contradiction
with the fact that we try hard to make all the infer processes agree on the set
of options.
Reviewed By: sblackshear
Differential Revision: D6761146
fbshipit-source-id: 67a0c54
Summary:
- During program translation, infer logs details about Unimplemented exceptions that are caught
- Exception type, source location, exception triggering location, and the ast_item it occurred in
- Logging is integrated with EventLogger library
- New type of event in EventLogger
- Unimplemented exception type extended to add support for storing these extra details
- All instances where unimplemented exception is raised modified to pass these details
Reviewed By: dulmarod
Differential Revision: D6748734
fbshipit-source-id: 725c7f3
Summary:
and add mli. We already had the logic for iterating over call chains, but it was overfitted to the should-update analysis.
Will use the generalized version in a follow-up.
Reviewed By: jvillard
Differential Revision: D6740692
fbshipit-source-id: 8c0d89f
Summary:
Getting double-logged messages when logs have not yet been setup is confusing,
so just don't log these messages to a log file.
Reviewed By: mbouaziz
Differential Revision: D6739173
fbshipit-source-id: 14db6b0
Summary:
Was trying to decide where to add a new Java utility function and realized that things are a bit disorganized.
Some operations on `Typ.Name.t`'s live in `Typ.Procname`, and some live inside an inner `Java` module whereas some are outside of the module with a `java_` prefix.
Let's move toward putting all Java/C/Objc/C++-specific functions in dedicated modules.
This diff does some of the work for Java.
There are Java-specific functions that operate on `Typ.Procname.t`'s that will have to be converted to work on `Typ.Procname.Java.t`'s, but changing those clients will be more involved.
Will also move C/Objc/C++ functions in a follow-up.
Reviewed By: jeremydubreil
Differential Revision: D6737724
fbshipit-source-id: cdd6e68
Summary: Use the Hashtbl functions directly as `Cfg` knows that a cfg is a hashtbl.
Reviewed By: sblackshear, jeremydubreil
Differential Revision: D6727732
fbshipit-source-id: 2cdda91
Summary:
`&::.*-->` allows to match any path end.
Used for models of `std::array` to force unmodelled functions (and types) to have a Skip summary
Depends on D6408415
Reviewed By: jvillard
Differential Revision: D6611203
fbshipit-source-id: 6663b2c
Summary:
In buck queries, the `regex` in `kind(regex, ...)` is open, it can match superstrings, like `prebuilt_cxx_library` when we want `cxx_library` instead.
This adds `^` and `$` to our existing kind filtering.
Also optimizes the query in `buck_target_determinator.py`.
Reviewed By: dulmarod
Differential Revision: D6722665
fbshipit-source-id: 22d839f
Summary:
Horrible but somewhat-documented hack to get inter-file dead code analysis for
the OCaml code.
Reviewed By: mbouaziz
Differential Revision: D6724234
fbshipit-source-id: 3a5b9cd
Summary:
Found the dead code with the script in the next commit, iteratively until no
warnings remained.
Methodology:
1. I kept pretty-printers for values, which can be useful to use from infer's REPL (or
when printf-debugging infer in general)
2. I kept functions that formed some consistent API (but not often, so YMMV), for instance if it looked like `Set.S`, or if it provides utility functions for stuff in development (mostly the procname dispatcher functions)
3. I tried not to lose comments associated with values no longer exported: if the value is commented in the .mli and not the .ml, I moved the comment
4. Some comments needed updating (not claiming I caught all of those)
5. Sometimes I rewrote the comments a bit when I noticed mis-attached comments
Reviewed By: mbouaziz
Differential Revision: D6723482
fbshipit-source-id: eabaafd
Summary:
Almost all the files are supposed to do this. When they don't, the build adds
it automatically anyway, but it's better to always include it to avoid
confusion (and help other automated tools).
Reviewed By: mbouaziz
Differential Revision: D6723285
fbshipit-source-id: 0fe8a16
Summary:
- After completing program translation, infer logs # of attempted procedure translations and # of completed procedure translations via EventLogger
- New possible type of event in EventLogger
Reviewed By: dulmarod
Differential Revision: D6711662
fbshipit-source-id: 5e31332
Summary:
- After completing program translation, infer logs # of attempted procedure translations and # of completed procedure translations
- New global state in cFrontend_config.ml to represent # of attempted procedure translations and # of failed procedure translations
- Will be integrated with logging framework
Reviewed By: dulmarod
Differential Revision: D6703248
fbshipit-source-id: 10c916a
Summary:
In Java, static variables are distinguished by package/class:
the file where they are defined doesn't matter.
Fixes#831.
Closes https://github.com/facebook/infer/pull/833
Reviewed By: jeremydubreil
Differential Revision: D6661240
Pulled By: sblackshear
fbshipit-source-id: beeb2f9
Summary: This should make no difference as the `Ondemand` would already only run the analysis when the procedure description is found, and naturally skip the analysis otherwise.
Reviewed By: sblackshear, jvillard
Differential Revision: D6705813
fbshipit-source-id: 44bffee
Summary:
This makes it easier to add more .atd files in the future: just add the
filename to `INFER_ATDGEN_STUB_BASES`.
Reviewed By: mbouaziz
Differential Revision: D6711695
fbshipit-source-id: 0d88daa
Summary:
Some commands (mostly `infer report`) would attempt to run the initialisation
code of infer from the default results directory instead of the one used by the
test. This is mostly harmless because we do not actually use anything from the
directory (typically, we pass `--from-json-results foo.json` and only foo.json
matters). However, this can trip the initialisation code, eg on db schema
changes.
Reviewed By: mbouaziz
Differential Revision: D6711641
fbshipit-source-id: f04b4c7
Summary: The models tenv is loaded inside a global variable that is never used.
Reviewed By: jeremydubreil
Differential Revision: D6692656
fbshipit-source-id: 4166a22
Summary:
This commit augments codes to clean up temporary files generated by the clang frontend.
Currently clang frontend leaves large number of temporary files int the tmp directory, and it could be seen for example by running these command:
```
git clean -xdf infer/models/
mkdir /tmp/infer
env TMPDIR=/tmp/infer make infer_models
ls -l /tmp/infer
```
P.S. Analyzing real project could easily cause each infer capture to leave hundreds of files in /tmp
P.P.S. There are 11 total references to Filename.temp_file however, other 9 seems don't leak temporary files in such large scale (at least not when using the clang frontend).
Closes https://github.com/facebook/infer/pull/816
Reviewed By: jvillard
Differential Revision: D6385311
Pulled By: dulmarod
fbshipit-source-id: f7956b0
Summary:
This is a more accurate type since an execution environment is always a single
file, and allows us to make one of the fields of `Exe_env.t` immutable.
Reviewed By: mbouaziz
Differential Revision: D6620477
fbshipit-source-id: 9553516
Summary:
This avoids relying on the directories in infer-out/captured/ being created,
and instead gets the list of captured source files from the DB. This gives a
better type to clusters: `SourceFile.t` instead of `DB.source_dir`, which makes
the code a bit nicer too.
Reviewed By: jeremydubreil
Differential Revision: D6620460
fbshipit-source-id: c0edbf6
Summary: Get the error message from the database when there's an error, together with the error type.
Reviewed By: mbouaziz
Differential Revision: D6621695
fbshipit-source-id: 6bc706d
Summary: Somehow sqlite allows this simpler statement when using `exec` but not `prepare`.
Reviewed By: mbouaziz
Differential Revision: D6407732
fbshipit-source-id: 01f029f
Summary:
This makes sure that sqlite doesn't hold read locks for longer than necessary,
which could starve the process of cleaning up the WAL file. This ensures that
the statement is reset as soon as we're done reading.
I haven't observed a difference with this change, and could not find evidence
that it should change something in the docs. Internet wisdom pointed at this as
a potential issue and I was observing it in another change, so it's good to
rule it out.
Reviewed By: mbouaziz
Differential Revision: D6404353
fbshipit-source-id: a123cd6
Summary: Previously we had a single sanitizer kind for escaping, but this isn't quite right. A function that escapes a URL doesn't necessarily make a string safe to execute in SQL, for example.
Reviewed By: the-st0rm
Differential Revision: D6656376
fbshipit-source-id: 572944e
Summary: This should avoid making copies of procedure descriptions which are mutable data-stuctures.
Reviewed By: sblackshear
Differential Revision: D6658527
fbshipit-source-id: 688a142
Ezgi Çiçek
Jeremy Dubreil
Josh Berdine
Sam Blackshear
Jules Villard
Dino Distefano
Varun Arora
Dulma Churchill
Tony Hu
Mehdi Bouaziz
Sungkeun Cho
Nikos Gorogiannis
Daiva Naudziuniene
Jérémie Marguerie
Ilya Sergey
Martino Luca
Radu Grigore
Vladimir Silyaev