Summary: I noticed we don't have `T` or `S` to denote timeouts in debug mode anymore. Today I saw it's still in `--stats` mode. Bring this feature back to `--debug` as well.
Reviewed By: cristianoc
Differential Revision: D4681669
fbshipit-source-id: 16ef19b
Summary:
Provide proper constructor functions for all Typenames following `Typename.Java` module.
Always use those constructor functions.
Reviewed By: jeremydubreil
Differential Revision: D4673943
fbshipit-source-id: 81625c2
Summary: This allows to run the analysis of every procedure on-demand separately from the cluster callbacks
Reviewed By: sblackshear
Differential Revision: D4664936
fbshipit-source-id: d218328
Summary: On Java Buck projects, InferPrint was loading all the specs files from all the jars in the classpath. This was affecting the performance a lot when the analysis was reporting a lot of issues.
Reviewed By: cristianoc
Differential Revision: D4673226
fbshipit-source-id: 6927836
Summary:
It used to be string which:
1. Doesn't have enough information for parametric models
2. Doesn't have good type
Changing this blows up in clang frontend, but I think it's for the better
Reviewed By: jberdine
Differential Revision: D4667633
fbshipit-source-id: 9f61bf1
Summary: I encountered cases where the class name part of the method name was passed as `(None, "package.Class")` instead of `("package", "Class")` and therefore incorrectly failing some inequality checks
Reviewed By: sblackshear
Differential Revision: D4662617
fbshipit-source-id: 98ee3e3
Summary:
Given two analysis results, it's now possible to compare them with the following command:
infer --diff --report-current reportA.json --report-previous reportB.json --file-renamings file_renamings.json
this command will then generate 3 files in `infer-out/differential/{introduced, fixed, preexisting}.json`, whose meaning is the following:
- `introduced.json` has all issues in `current` that are not in `previous`
- `fixed.json` has all issues in `previous` that are not in `current`
- `preexisting.json` has all issues that are in both `current` and `previous`
The json files generated can then be used to categorise results coming from incremental analyses of a codebase.
Reviewed By: jvillard
Differential Revision: D4482517
fbshipit-source-id: 1f7df3e
Summary: The implementation of `touch_start_file` was not updating the timestamp when the file exists.
Reviewed By: jvillard
Differential Revision: D4657708
fbshipit-source-id: 0a88ebc
Summary: I accidentally save a summary with the wrong procedure name, which was affecting the analysis in some weird way. This makes this case no longer possible
Reviewed By: cristianoc
Differential Revision: D4654002
fbshipit-source-id: 9fcbe4e
Summary:
This is part of the plan to have every checker take a summary as input, and return the updated sumamry as output. Doing so, we can run all the registered checkers in sequence for every method
This diff change the type of `Ondemand.analyze_ondemand` to return the analysis summary.
Reviewed By: sblackshear
Differential Revision: D4626918
fbshipit-source-id: f8ad928
Summary:
All intermediate `.exp` files used for tests can be generated with custom info, based on what is needed for the tests purposes.
This customisation happens via command-line argument `--issues-fields`.
Reviewed By: cristianoc, jvillard
Differential Revision: D4628062
fbshipit-source-id: feaa382
Summary:
For writes of serialized data, write directly to the file instead of using a temporary one, and lock the file before writing.
Also added an `update` function to the API, to update an existing version of the data file instead of just replacing it with a new value.
Reviewed By: jberdine
Differential Revision: D4619958
fbshipit-source-id: 9642408
Summary: This seems to only be used for stats and for the concept of call rank that is not used right now
Reviewed By: cristianoc
Differential Revision: D4624681
fbshipit-source-id: 7406496
Summary:
With the ondemand analysis framework, the concept of timestamp was only being use to check if a procedure has already been analyzed. There was already a concept of "active" procedure for the procedure that were already being analyzed. This revision removes the concept of timestamp and merge it with the concept of analysis status.
This can be simplified further once the analysis always goes through `Ondemand.analyze`.
Reviewed By: cristianoc
Differential Revision: D4610371
fbshipit-source-id: 0fc516b
Summary:
Polymorphic models, and type environment refinements, need mutual
references between general types and struct types.
Reviewed By: cristianoc
Differential Revision: D4620076
fbshipit-source-id: f9d01e6
Summary: The summary was stored to disk at the end of the on-demand analysis, unless an exception was raised in which case it was only updated in the in-memory cache.
Reviewed By: sblackshear
Differential Revision: D4612369
fbshipit-source-id: 1c8d75b
Summary: The function `Checkers.ST.store_summary` was only used in one place. This revision moves the functionality to the only place where this function was used, except the part swallowing `Sys_error` which may have the bad side-effect of making issues like race-conditions silent.
Reviewed By: cristianoc
Differential Revision: D4608790
fbshipit-source-id: b84c8ce
Summary: This check is redundant and already happens in `Ondemand`.
Reviewed By: sblackshear
Differential Revision: D4605613
fbshipit-source-id: d249212
Summary:
It seems that what `Checkers.ST.store_summary` was called witin on-demand was actually redundant with what `Ondemand` is doing before storing the summaries.
This also makes the `Ondemand` module no longer depend on `Checkers` as the dependency is expected to be the other way around.
Reviewed By: sblackshear
Differential Revision: D4595006
fbshipit-source-id: d62187e
Summary: Shorter is better. `--compilation-database` was taken, renaming it to `--buck-compilation-database`.
Reviewed By: dulmarod
Differential Revision: D4567028
fbshipit-source-id: 011cd6f
Summary:
A good first step in order to run multiple checkers together is to prevent the analysis the analysis to side effect on the summaries of the method being analyzed from disk, or the shared specs summary. The idea is that `Ondemand` creates a summary for the procedure being analyzed and only saves the summary once all the checkers have been run. The summary for the caller (i.e. the procedure being analyzed) should never be looked up from disk during the analysis. In other words, the analysis should only ever lookup the summaries of the callees and the proposed solution to enforce this is to have `Ondemand.analyze_proc_name` be the only way to lookup the summary of a procedure.
Another objective is to make sure that the summaries are never saved to disk more than once.
Reviewed By: sblackshear
Differential Revision: D4549764
fbshipit-source-id: f0a6e21
Summary:
This simplifies a bit the code to run the analysis on all the prcedures in the cluster. Before, the functions procedure_should_be_analyzed, which loads the attributes, and get_proc_desc were called twice for the analysis of every procedure.
The objective is to remove the calls to procedure_should_be_analyzed and hide it from the ondemand API since it is already called before the analysis of every procedure.
Reviewed By: sblackshear
Differential Revision: D4553397
fbshipit-source-id: 02cffaf
Summary:
One gets very obscure errors when trying to run infer for clang when it was
compiled for Java, or vice-versa. This diff makes sure we crash early with the
appropriate error message. For instance:
```
$ ./build-infer java
$ infer -- clang -c hello.c
Uncaught exception:
(Failure
"Unsupported build mode: make/cc\
\nInfer was built with clang analyzers disabled.\
\nPlease rebuild infer with clang enabled.\
\n")
Raised at file "pervasives.ml", line 30, characters 22-33
Called from file "backend/infer.ml", line 398, characters 6-48
Called from file "backend/infer.ml", line 449, characters 20-38
$ infer --clang-compilation-db-files foo.json
Uncaught exception:
(Failure
"Unsupported build mode: clang compilation database\
\nInfer was built with clang analyzers disabled.\
\nPlease rebuild infer with clang enabled.\
\n")
Raised at file "pervasives.ml", line 30, characters 22-33
Called from file "backend/infer.ml", line 392, characters 8-65
Called from file "backend/infer.ml", line 449, characters 20-38
```
Reviewed By: sblackshear
Differential Revision: D4566641
fbshipit-source-id: d9a118f
Summary:
- inferbo introduced a dependency to extlib. When building Java analyzers, this
is implicitly pulled in by javalib, but it's missing when building only the
clang analyzers. Add `extlib` to the packages we build against.
- infer.ml and Javac.ml depend on Javalib, but it's easy to push down the code
that needs it to `jMain.ml` so that we can build without javalib for the
clang-only case.
- jMain.mli had 2 copies: one in java/ and one in java_stubs/. Make one a symlink to the other.
Reviewed By: jeremydubreil
Differential Revision: D4566581
fbshipit-source-id: 214a4eb
Summary:
Sevel auxiliary files made it to the output directory of the analysis of individual targets when analyzing Java projects build with Buck. However, these files are then taken into account= to compute the target rule key and then to decide whether to analyze the dependent targets. Since these auxiliary files were containing time sentive information, every cach miss on a given target would then invalitate the cache entries for all the dependent targets.
This diff cleans up the output directory to only keep the specs files, the `global.tenv` and the `report.json` files which are the only artifacts needed to analyze the dependent targets
This diff makes a minimal number of changes to see how it behaves in prod, but I intend to refoctor this more when continuing to add support for running Infer with genrules
Reviewed By: sblackshear
Differential Revision: D4562615
fbshipit-source-id: 4628420
Summary:
Xcode's compilation databases follows a different convention than cmake's and
escape the `"file"` and `"dir"` fields of each unit to make them shell-ready.
We need to treat them differently when reading them.
This adds a new `--clang-compilation-db-files-escaped` option and makes the
code related to reading compilation databases deal correctly with both
conventions.
Reviewed By: akotulski
Differential Revision: D4559239
fbshipit-source-id: 51120ae
Summary:
At one point I thought we'd want to have lots of different schedulers for things like exploring loops in different orders, but that hasn't materialized.
Let's make the common use-case simpler by hiding the `Scheduler` parameter inside the `AbstractInterpreter` module.
We can always expose `MakeWithScheduler` later if we want to.
Reviewed By: jberdine
Differential Revision: D4508095
fbshipit-source-id: 726e051
Summary:
Clients should use `Config.parse_action` instead to figure in what mode they
are operating.
In particular, the biggest change is in logging. Take the `parse_action` into
account instead of the exe, and change the log/ subdirectories to be "capture",
"driver", "analyze", and "print", corresponding to the various phases of an
infer run.
Reviewed By: jberdine
Differential Revision: D4474943
fbshipit-source-id: 6d33ad3
Summary:
Infer used to report null dereference when field was accessed later:
```
vector<int> v;
int& a = v[0]; // should be EMPTY_VECTOR_ACCESS here, but it wasn't reported
int b = a; // was NULL_DEREFERENCE here
```
To avoid this problem, model all accesses to vector as dereference of its internal `beginPtr` field.
Reviewed By: jberdine
Differential Revision: D4481942
fbshipit-source-id: 2142894
Summary:
One of the things that confuses me about the current annotations API is that there's a lot of ways to do the same thing.
Some of the concepts like `annotated_signature` are only really needed by Eradicate.
This diff removes usages of `annotated_signature` outside of Eradicate (everyone else was just using `get_annotated_signature` as a roundabout way to get the return annotation of a procedure).
In the future, I'll move `get_annotated_signature` and other Eradicate-specific functionality into its own module inside the Eradicate directory.
Reviewed By: jberdine
Differential Revision: D4472058
fbshipit-source-id: 5bb0846
Summary:
`pdesc_has_annot` checks the annotations of both the return values and the parameters, which seems like a bad idea in general.
The client should have to specify which annotations they actually care about.
Converting existing uses of `pdesc_has_annot` to what I read as the intended behavior (checking the return annotation).
Will make better use of the other new functions in a follow-up.
Reviewed By: jeremydubreil
Differential Revision: D4469885
fbshipit-source-id: de5531e
Summary:
Remove the remaining uses of polymorphic equality `=`.
In case of basic types, this is replaced by String.equal or Int.equal.
In case of `= []`, this is replaced by `List.is_empty`.
In case of `= None`, this is replaced by `is_none`.
In case of a datatype definition such as `type a = A | B`,
a `compare_a` function is defined by adding `type a = A | B [@deriving compare]`
and a `equal_a` function is defined as `let equal_a = [%compare.equal : a]`.
In case of comparison with a polymorphic variant `= `Yes`, the equality
defined in `PVariant.(=)` is used. Typically, `open! Pvariant` is added
at the beginning of the file to cover all the uses.
Reviewed By: jberdine
Differential Revision: D4456129
fbshipit-source-id: f31c433
Summary:
Epilogue tasks such as closing logs or putting files back were we found them
run automatically at the end of our executables by registering them with
`at_exit`. They do not run if the program is interrupted by a signal. This diff
makes sure they are run when the user stops infer with Ctrl-C (SIGINT).
Reviewed By: cristianoc
Differential Revision: D4435575
fbshipit-source-id: c3ab702
Summary:
This replaces the previous integration written in Python, which consisted in 1)
run the mvn command and parse its output to locate "directories containing
source files", 2) run on files named "*.java" in these directories. This meant
we had to run javac twice on each source file, and more importantly this
mechanism of finding source files was very fragile. In fact, I could not make
it work on several mvn projects I tried.
The new integration is based on parsing "pom.xml" to add an "infer-capture"
profile which instructs mvn to run `/path/to/infer` instead of `javac`. We also
add this profile to each maven submodule.
Users can specify an "infer-capture" profile themselves if the default one
doesn't work; in that case we don't inject our own "infer-capture" profile.
Reviewed By: jeremydubreil
Differential Revision: D4409613
fbshipit-source-id: d664274
Summary:
Also make sure we don't introduce deprecated options in our repo, eg when
calling infer from infer.
Reviewed By: jeremydubreil
Differential Revision: D4430379
fbshipit-source-id: 77ea7fd
Summary:
Make the html output available to checkers when -g is used on the command-line.
A checker needs to call a function to start and finish the processing of each node,
and add prints during the processing.
This diff illustrates the case for Eradicate, by adding printing of the pre-state
and post-states.
Reviewed By: sblackshear
Differential Revision: D4421379
fbshipit-source-id: 67501ba
Summary:
Previously, we would first compute which build command is at hand, based on the
first argument after "infer --", then do everything depending on that piece of
information. However, the build command alone is not enough to know in which
"build mode" we are operating. For instance, there are several build modes
corresponding to "buck" build commands.
This led to duplication of the logic (to retrieve which build mode we are in in
the various phases of an infer run), and some invariants that had to be
re-asserted at various points in the code, eg that the arguments are not empty.
This diff adds a `build_mode` type (renaming the previous `build_mode` to
`build_system`) that identifies the various integrations we support. We compute
the build mode at the start of infer, then pass the build mode around.
Also, move `run_javac` to a new `integration/Javac.ml` file given that it's a
bit large.
Reviewed By: jberdine
Differential Revision: D4415074
fbshipit-source-id: db854a0
Summary:
Currently, if we don't find `-d` or `-classes_out` on the command line then we
tell javac to redirect the compiled classes in some other directory, by default
the initial working directory. But we don't detect when these arguments are
hidden inside files (`foo` arguments on the javac command line) so the
heuristic was incomplete. Look inside these files to better tell whether we need
to make up an output directory or not.
Reviewed By: jeremydubreil
Differential Revision: D4397716
fbshipit-source-id: 30c5e4f
Summary:
Sometimes we don't want to analyze but a message gets printed that there was
nothing to analyze and we exit with error, which is confusing.
Reviewed By: jberdine
Differential Revision: D4398120
fbshipit-source-id: 43ce3ab
Summary:
Add more debug output to be able to trace the calls to javac more easily
when --stats or --debug is passed to infer.
Reviewed By: sblackshear
Differential Revision: D4398100
fbshipit-source-id: 3012900
Summary:
This would fail before and works as expected now:
```
$ infer -- clang -c hello.c
$ cd infer-out/ && ln -s ../foo && cd ..
$ infer -- clang -c hello.c # crashes because it fails to delete infer-out/foo
```
Reviewed By: jberdine
Differential Revision: D4398763
fbshipit-source-id: 38465f8
Summary:
One of the tests was failing without `make clean` because infer-out didn't get
deleted when rerunning the clang db test. This was because infer thinks it's in
`Analyze` mode when capturing clang db files.
Reviewed By: akotulski
Differential Revision: D4397731
fbshipit-source-id: 26f423a
Summary:
This error message is confusing when the user is not actually running
InferPrint, eg `infer foo`: `Load Error: file foo: arguments must be .specs
files`.
With this diff, we don't get any error for `infer foo`, which is not great
either and will need to be addressed (do we support all the python arguments in
OCaml now too and are able to turn on argument parsing errors in OCaml land?).
Reviewed By: jberdine
Differential Revision: D4397765
fbshipit-source-id: e7ca48f
Summary:
A domain should not definite its initial state, since distinct users of the domain may want to choose different initial values.
For example, one user might want to bind all of the formals to some special values, and one user might want the initial domain to be an empty map
This diff makes this distinction clear in the types by (a) requiring the initial state to be passed to the abstract interpreter and (b) lifting the requirement that abstract domains define `initial`.
Reviewed By: jberdine
Differential Revision: D4359629
fbshipit-source-id: cbcee28
Summary:
Force clients to specify the path relative to which relative paths
should be made absolute.
Reviewed By: akotulski
Differential Revision: D4370262
fbshipit-source-id: 36a2807
Summary:
There is not much to redirect except for an uninformative line before proper
logging files are set up. This is from before the current logging system, which
has builtin support for logging into custom files.
Reviewed By: jberdine
Differential Revision: D4365988
fbshipit-source-id: 044290a
Summary:
Instead of opening new log files each time with non-deterministic names, keep
appending to the same log files. This only removes the randomized part of the
names in the files. In particular, it keeps the name prefixes for, eg, clang
source files.
Also changed most "<executable>/<executable>-out.log" to simply "<executable>/out.log".
Reviewed By: jberdine
Differential Revision: D4365983
fbshipit-source-id: 46792dc
Summary: This more easily allow to switch between the different modes for handeling dynamic dispatch
Reviewed By: sblackshear
Differential Revision: D4367556
fbshipit-source-id: 795d2c4
Summary: 957b243 removed the last use of `Exe_env.get_tenv ~create:true`
Reviewed By: jeremydubreil
Differential Revision: D4364521
fbshipit-source-id: 819efee
Summary: Use the lazy dynamic dispatch by default in prod for the Java analysis
Reviewed By: sblackshear
Differential Revision: D4356872
fbshipit-source-id: 491e92e
Summary: Adding the information that a procedure has been modelled as part of the attributes, during the translation, instead of getting this information from where is the summary loaded from. This is more consistent with the use of the attributes in other parts of the analysis, but is also useful in the context of the lazy dynamic dispatch algorithm where the procedures, including the models, are cloned and reanalyzed with more specialized parameters. The information about whether a procedure is a model must persist when cloning the procedures.
Reviewed By: sblackshear
Differential Revision: D4356892
fbshipit-source-id: 40ff5ca
Summary:
When you try to log an error on a procedure P and a summary for P doesn't exist, the error gets quietly dropped on the floor.
But we should fail loudly instead, because this should only happen in the case of a user error.
Got burned by this today; I was trying to log an error on the *caller* of `Integer.parseInt`, but was accidentally logging it to `Integer.parseInt` itself instead.
Since no summary for that method exists, my error wasn't appearing.
Reviewed By: jvillard, jeremydubreil
Differential Revision: D4355546
fbshipit-source-id: db2a0e6
Summary:
The two concepts are not negation of each other. The type environment created by the different frontends is not guaranteed to contain a full view of the type hierarchy. In this case, there can be holes preventing Infer to prove that `t <: t'` if the type definition between `t` and `t'` is missing. There are now two functions:
# `is_known_subtype` when the subtyping relation can be proven
# `is_known_not_subtype` when it can be proven that there is no subtyping relation between two types
This diff is intended to make no functional changes but to add functionality to detect cast error angelically, i.e. assuming that the program is probably fine where there is not enough information to prove the cast error.
Reviewed By: jberdine
Differential Revision: D4345803
fbshipit-source-id: 39b79bc
Summary: Different analyses need different preanalyses to run. It doesn't make sense for all of the pre-analyses to be bundled together into one package.
Reviewed By: jvillard
Differential Revision: D4348243
fbshipit-source-id: 46a8ebd
Summary:
Adding #infer-capture-all et al. by hand is annoying and I always forget to do
it. Let infer figure that out.
Reviewed By: dulmarod
Differential Revision: D4339799
fbshipit-source-id: 55e52dc
Summary: reactive capture spawns clang from within analysis. Time it takes to compile source code shouldn't be counted towards timout
Reviewed By: jvillard, cristianoc
Differential Revision: D4334037
fbshipit-source-id: 64f417d
Summary:
Most of the time code outside of project root is not interesting to the user - it's either system library or infer C++ model. Skip all of them.
Previous logic was doing something similar, but in more selective way.
I also need this change for D4313428
Reviewed By: jvillard
Differential Revision: D4339298
fbshipit-source-id: c7b5544
Summary:
This will simplify the InferPrint logic of checking what should/should-not be reported.
I will remove the issue names in Localise in a next diff.
Reviewed By: ddino
Differential Revision: D4334327
fbshipit-source-id: ebcfd6c
Summary:
SuppressWarnings annotations are hardly used and add considerable
complexity due to requiring recompilation with an annotation processor.
Reviewed By: jvillard
Differential Revision: D4312193
fbshipit-source-id: c4fc07e
Summary:
This option is only useful if you want to treat casts angelically, but nothing else.
Since angelic is on by default and this option is off by default, it's basically useless.
Reviewed By: jeremydubreil
Differential Revision: D4334030
fbshipit-source-id: 3c0b0ed
Summary:
Most of the diff adds a way to run an existing test with different infer
options.
Also, do not run the Python script when capturing "analyze".
fixes https://github.com/facebook/infer/issues/518
Reviewed By: jberdine
Differential Revision: D4333762
fbshipit-source-id: 642acff
Summary:
Change the domain of SIOF to be based on sets of pvar * location instead of
single pvars. This allows us to group several accesses together. However, we
still get different trace elems for different instructions in a proc. We do two
things to get around this limitation and get a trace where all accesses within
the same proc are grouped together, instead of one trace for each access:
1. A post-processing phase at the end of the analysis of one proc collects all
the globals directly accessed in the proc into a single trace elem.
2. When creating the error trace, unpack this set into several trace elements
to see each access (at its correct location) separately in the trace.
This is a bit hacky and another way would be to extend the API of traces to
handle in-procedure accesses natively instead of shoe-horning them. However
since SIOF is the only one to use this, it introduces less boilerplate to do it
that way for now.
Also, a few .mlis for good measure.
Reviewed By: sblackshear
Differential Revision: D4299070
fbshipit-source-id: 3bbb5c2
Summary:
Turns out that swapping stdout and stderr using a temporary fd 3 was screwing
up with make's jobserver, who also uses fd 3!
Also, infer is partly to blame as it also calls `make`. Unsetting `MAKEFLAGS`
in infer tells `make` that the way infer calls `make` is independent from
parent `make` invocations.
Also, simplify the rules for direct tests and build system tests.
Reviewed By: jberdine
Differential Revision: D4328979
fbshipit-source-id: 96818e8
Summary:
We don't need to have separate `--` integration for compilation database. Instead use:
infer --compilation-db-files db.json <other_infer_options> // no -- anywhere!
Reviewed By: jberdine
Differential Revision: D4327570
fbshipit-source-id: caf0dc9
Summary: This is required to maintain a set of owned access paths in a subsequent diff.
Reviewed By: jberdine
Differential Revision: D4318859
fbshipit-source-id: bd1a9fa
Summary: Allow backend to trigger compilation of extra files when it needs them. This will allow infer to capture less files initally and possibly speed up compilation
Reviewed By: cristianoc, jberdine
Differential Revision: D4231581
fbshipit-source-id: 181abea
Summary:
This diff adds basic support for parsing the arguments passed to the
build command directly from Config.
Reviewed By: dulmarod
Differential Revision: D4201480
fbshipit-source-id: bba6056
Summary:
Use In_channel and Out_channel operations instead of those in Pervasives. Don't
use physical equality on values that aren't heap-allocated since it doesn't help
the compiler generate faster code and the semantics is unspecified. Also use
phys_equal for physical equality.
Reviewed By: sblackshear
Differential Revision: D4232459
fbshipit-source-id: 36fcfa8
Summary:
Utils contains definitions intended to be in the global namespace for
all of the infer code-base, as well as pretty-printing functions, and
assorted utility functions mostly for dealing with files and processes.
This diff changes the module opened into the global namespace to
IStd (Std conflict with extlib), and moves the pretty-printing
definitions from Utils to Pp.
Reviewed By: jvillard
Differential Revision: D4232457
fbshipit-source-id: 1e070e0
Summary:
Functions related to source files were already namespaced by `source_file_` prefix. Make separate module for them.
In high level it replaces all `source_file_` with `SourceFile.` and then fixes all remaining compilation errors
Reviewed By: jvillard
Differential Revision: D4299053
fbshipit-source-id: 20b1d39
Summary:
When calling function g_realloc(gpointer mem,gsize n_bytes) one of the spec considers the case
whereby n_bytes is zero. In that case g_realloc would return null.
If we call with sizeof(int), infer would compare sizeof(int) with zero. But the prover would fail to
understand that sizeof(int) != 0.
This diff fix this. We try to convert expression to constant when they can be converted (eg in case of sizeof).
The method currently make a partial set of conversion. This could be extended.
Reviewed By: jberdine
Differential Revision: D4166944
fbshipit-source-id: 3ec4fd7
Summary: When infer runs on preprocessed source, original files may not be around anymore. Don't crash infer when that happens.
Reviewed By: jvillard, jberdine
Differential Revision: D4258285
fbshipit-source-id: a19569c
Summary:
source_file_[to|from]_string were dangerous. While removing source_file_to_string is hard/impossible, source file should never be created from string.
Also include many random changes related to `source_file`:
- improve comments in DB.mli
- define behavior of changed-files-index and improve its description
- move some of the "dangerous" code inline to discourage its reuse
This mostly concludes cleanup of DB.source_file, the last bit is to unify filtering by filename (we have duplicated logic in `InferConfig`, `CLocation` and `JMain`)
Reviewed By: jvillard
Differential Revision: D4258795
fbshipit-source-id: 36735a8
Summary:
`DB.source_file_to_string` is very easy to misuse and it shouldn't even exist.
In preparation for that day, replace most of `source_file_to_string` with `source_file_pp`
Reviewed By: jvillard
Differential Revision: D4258390
fbshipit-source-id: 447cf5a
Summary: Noticed this when I was writing the documentation for the abstract interpretation framework and was curious about why `Ondemand.analyze_proc` needs the type environment. It turns out that the type environment is only used to transform/normalize Infer bi-abduction specs before storing them to disk, but this can be done elsewhere. Doing this normalization elsewhere simplifies the on-demand API, which is a win for all of its clients.
Reviewed By: cristianoc
Differential Revision: D4241279
fbshipit-source-id: 957b243
Summary:
Currently cfg nodes are written into dot files in whatever order they
appear in a hash table. This seems unnecessarily sensitive, so this
diff sorts the nodes.
Reviewed By: dulmarod
Differential Revision: D4232377
fbshipit-source-id: a907cc6
Summary: Add some basic command line API to run Infer using Buck genrules. Remains to fix issues with absolute vs relative paths and to see how to create these genrules on the fly for a given java or android library.
Reviewed By: sblackshear
Differential Revision: D4245622
fbshipit-source-id: 1cda4ee
Summary:
Clean up code related to --changed-files-index option:
1. Store DB.SourceFileSet.t in DB.changed_source_files_set
2. Refactor rest of the code to use it
3. Bunch of minor changes to make code more consise
Reviewed By: jberdine
Differential Revision: D4238736
fbshipit-source-id: 51e5684
Summary:
Dealing with symbolic links in project root is tricky. To avoid it, always normalize all paths to sources with `realpath`.
Changes to tests are expected - infer started to resolve symbolic links which screws up with our testing mechanism.
Reviewed By: jberdine
Differential Revision: D4237587
fbshipit-source-id: fe1cb01
Summary: Make backend know filenames of compilation database. It will allow it to compile extra files when needed
Reviewed By: cristianoc
Differential Revision: D4231521
fbshipit-source-id: c462448
Summary: Pure refactoring simplifying the code doing the case analysis for execturing the cast instruction.
Reviewed By: dulmarod
Differential Revision: D4215238
fbshipit-source-id: 9f0f163
Summary:
1. Always store cpp model source_file with relative path. This will make them cache friendly independent of infer location
2. Distinguish between "relative to project root" and "relative to infer models src"
3. Unify `source_file_from_path` used by java and C frontends into one function. There are no improvements to that logic yet
4. Move `is_cpp_model_file` to use `source_file` instead of `filename`
Reviewed By: jberdine
Differential Revision: D4204548
fbshipit-source-id: 6e21771
Summary: This option is unused and it's making DB.source_file_* API harder to simplify
Reviewed By: cristianoc
Differential Revision: D4219803
fbshipit-source-id: 23ce697
Summary:
Using DB.source_file_to_string may return relative path which may or may not be relative to current location.
I went through all calls to `DB.source_file_to_string` and fixed ones that used that output to open files
Reviewed By: jeremydubreil
Differential Revision: D4205407
fbshipit-source-id: b285b7e
Summary:
Developers will sometimes write GuardedBy("T.f") with the intended semantics: "guarded by the field f of the object with type T in the current state".
We want to support this to avoid false positives.
Reviewed By: peterogithub
Differential Revision: D4197476
fbshipit-source-id: acd00d9
Summary:
- fix python calling function with wrong number of arguments (sic)
- print legend of analysis output, this was lost in translation ('F', '.', ...)
- add "Capturing in <mode> mode..." message before capture
- remove version from "Analyzing..." message (users don't even paste the full output, so this is not as useful as initially hoped)
Reviewed By: akotulski
Differential Revision: D4205072
fbshipit-source-id: 2b6505c
Summary: These are dangerous if you are trying to compare a type to a string, and they're also unsightly.
Reviewed By: jvillard
Differential Revision: D4189956
fbshipit-source-id: 14ce127
Summary:
This diff implements enough of the functionality in the python code in
the OCaml toplevel driver that executing `infer -- analyze` is done
with direct procedure calls instead of forking the python interpreter.
Except for some reporting code that remains in report.py.
Reviewed By: jvillard
Differential Revision: D4074718
fbshipit-source-id: 56a794d
Summary:
Record an abstraction of the bug traces in the tests. The abstraction of a
trace is the sequence of descriptions. In practice, descriptions are either
empty, or of the form "start/end/return from/call to procedure X". They seem
pretty stable.
Motivation: there is nothing testing the traces reported by Infer right now,
even though they are surfaced to developers. For instance, Quandary uses
--issues-txt instead of --issues-tests to make sure the traces do not regress.
This change would make this approach more widespread.
Reviewed By: sblackshear
Differential Revision: D4159597
fbshipit-source-id: 9c83952
Summary: clang has very complicated logic what to translate based on `project_root` and filename. Add tests for different situations in regard of symbolic links in path/project_root
Reviewed By: jvillard
Differential Revision: D4168551
fbshipit-source-id: 586b364
Summary: Refactoring to make thread safety checker interpocedural. This should not change funcitonality, and will only set things up for making the interprocedural part more serious.
Reviewed By: sblackshear
Differential Revision: D4124316
fbshipit-source-id: 6721953
Summary:
When loading results from a json file, sort them. This prints results in some
sane order for both --issues-test and --issues-txt, removing the need for
post-processing of the result.
Reviewed By: cristianoc
Differential Revision: D4167029
fbshipit-source-id: 37e9f1c
Summary: The thread safety checker is run independently of other analyses, using the command "infer -a threadsafety -- <build-command>".
Reviewed By: sblackshear
Differential Revision: D4148553
fbshipit-source-id: bc7b3f9
Summary: When searching for cast errors, types that were not Java objects, e.g. arrays of primitive types were not taken into account, leading to incorrect class cast excpetion reports.
Reviewed By: sblackshear
Differential Revision: D4166184
fbshipit-source-id: 7157c95
Summary:
It was defined in two places and I'm about to add a third, so let's share
instead.
Reviewed By: sblackshear
Differential Revision: D4153420
fbshipit-source-id: 3d2c519
Summary:
Move code that initializes the InferAnalyze executable from
InferAnalyze.main to InferAnalyzeExe. This enables InferAnalyze.main to
be called from other executables without conflicts due to
initialization.
Reviewed By: jvillard
Differential Revision: D4137280
fbshipit-source-id: 3dd76db
Summary: Also use the executable as a default name prefix.
Reviewed By: akotulski, jvillard
Differential Revision: D4135539
fbshipit-source-id: 84ba011
Summary:
Location.nLOC was introducing a lot of complexity for little benefit (and edge cases were wrong anyway).
We can restore it in some simplified way if we find that we need it
Reviewed By: jeremydubreil
Differential Revision: D4139868
fbshipit-source-id: 4f8e033
Summary:
Summaries are modified before saving from disk, for example the attributes of the postcondition can change.
I have observed flaky reports of the internal error NULL_TEST_AFTER_DEREFERENCE. Some attributes (e.g. assigned) are changed before saving, but the spec table in memory is not changed.
So in case:
1) the procedure is analyzed on-demand, then subsequent uses in the same process use the summary in memory with the unchanged attribute, and the issue is not reported.
2) the procedure is already on disk and loaded, then the loaded summary has the changed attributes, and the issue is reported.
Flakiness happens as because of parallelism, whether a procedure is analyzed already or whether it is analyzed on-demand, can change.
The normalization function can change the instrumentation of a symbolic heap because it uses the existing comparison functions, which ignore instrumentations.
So normalization can replace part of a symbolic heap with an identical one but where the instrumentation is different — this is what I have observed.
The diff uses a different comparison function where instrumentations are taken into account.
Reviewed By: jberdine
Differential Revision: D4140031
fbshipit-source-id: f4f119a
Summary:
Instead of the custom filtering done by `InferPrint --issues-tests`, use the
filtering done by `infer` and run without filtering for our e2e tests. We still
test the filtering for our build systems integration tests, and this diff
restores that behaviour for the ant test (hence the bugs removed from
ant/issues.exp).
Also add internal exceptions to most tests to get more signal out of them (eg,
knowing when we add assertion failures and the like).
Retire the old `--issues-tests` to limit the number of ways we do filtering.
Reviewed By: jeremydubreil
Differential Revision: D4131308
fbshipit-source-id: 35805cc
Summary:
This will be useful to migrate the existing tests to using report.json to
output the list of bugs found by Infer. This will make the tests reflect what
happens in prod more faithfully: right now running with --issues-tests does its
own filtering starting from the specs.
Moreover, this will allow --issues-tests to support the Buck integration, where
the specs/ directory is not populated after a run (although I suppose we could
also copy them from buck-out/ for InferPrint's benefit).
Reviewed By: jeremydubreil
Differential Revision: D4130851
fbshipit-source-id: 0457fba
Summary:
This makes our python code work (instead of crashing) when the source file
should be found not from the current directory (or absolute path), eg with
`infer --project-root .. -- clang -c hello.c`.
Reviewed By: jeremydubreil
Differential Revision: D4130802
fbshipit-source-id: 001f72d
Summary:
If the project root contains ".." then it doesn't work as expected, eg
infer --project-root .. -- clang hello.c
doesn't report at all. Now it works.
Reviewed By: jeremydubreil
Differential Revision: D4125489
fbshipit-source-id: 06b10ad
Summary: These functions are also called when the summary is guaranteed to exist. Enforcing this within the API
Reviewed By: cristianoc
Differential Revision: D4126839
fbshipit-source-id: 305b484
Summary: For some reason, `Specs.is_active` was re-loading from the specs table the summary that should already be in scope.
Reviewed By: cristianoc
Differential Revision: D4124693
fbshipit-source-id: c0e9113
Summary: Right now there is no test for compilation database integration. Add one
Reviewed By: jvillard
Differential Revision: D4118769
fbshipit-source-id: 5591de7
Summary:
This diff adds a skeleton implementation of the capture and analysis
driver to infer.ml, and removes some unnecessary code from infer.py.
With this, individual capture and analysis modules can be added, or
moved from python.
Reviewed By: jvillard
Differential Revision: D4109547
fbshipit-source-id: 0dce2bf
Summary: Don't use a hardcoded string, and enable reports in --issues-tests.
Reviewed By: jvillard
Differential Revision: D4110731
fbshipit-source-id: 9922557
Summary:
ClusterMakefile need not depend on Sys.executable_name referring to
InferAnalyze, use Config.bin_dir instead.
Reviewed By: jvillard
Differential Revision: D4110730
fbshipit-source-id: c330bb3
Summary:
Child processes invoked in multicore mode get arguments using the usual
INFER_ARGS mechanism already, no need for a special case.
Reviewed By: jvillard
Differential Revision: D4110728
fbshipit-source-id: 0987216
Summary: We want to skip readwrite locks for now, maybe report on their misuses later.
Reviewed By: sblackshear
Differential Revision: D4110998
fbshipit-source-id: 986f77e
Summary:
this makes frontends no longer depend on SymExec.ml. `ModelBuiltins` was split into two modules:
- `BuiltinDecl` with procnames for builtins (used to determine whether some function is a builtin)
- `BuiltinDefn` with implementations used by `SymExec`
- they both have similar type defined in `BUILTINS.S` which makes sure that new builtin gets added into both modules.
During the refactor I ran some scripts:
`BuiltinDecl.ml`:
let X = create_procname "X"
cat BuiltinDecl.ml | grep "create_procname" | tail -70 | awk ' { print $1,$2,$3,$4,"\42"$2"\42"} '
then manually confirm string match. Exceptions:
"__exit" -> "_exit"
"objc_cpp_throw" -> "__infer_objc_cpp_throw"
__objc_dictionary_literal
nsArray_arrayWithObjects
nsArray_arrayWithObjectsCount
`BuiltinDefn.ml`:
let X = Builtin.register BuiltinDecl.X execute_X
cat BuiltinDecl.ml | grep "create_procname" | tail -70 | awk ' { print $1,$2,$3,"Builtin.register BuiltinDecl."$2,"execute_"$2} '
then, fix all compilation problems
Reviewed By: jberdine
Differential Revision: D3951035
fbshipit-source-id: f059602
Summary: Doing `sychronized(A.class)` where `A` is an inner class was not previously recognized by the `GuardedBy` checker.
Reviewed By: peterogithub
Differential Revision: D4095094
fbshipit-source-id: c832f9e
Summary:
Now that it's possible to run clang wrapper as a function from another process,
Logging module cannot rely on `Config.current_exe` to determine which directory
it should write to.
Reviewed By: jeremydubreil
Differential Revision: D4095455
fbshipit-source-id: d989b06
Summary:
Change command line options for dynamic dispatch to capture that the
alternatives are mutually exclusive.
Reviewed By: jeremydubreil
Differential Revision: D4074540
fbshipit-source-id: c329717
Summary: The lazy dynamic dispatch algorithm works by re-analyzing the generic methods with the more specialized types encountered during the symbolic execution. In order to do that, the analysis must access the procedure description of the method to reanalyze in order to run the analysis of the specialized procedure description on demand. This diff adds the procedure description on the summary as the summary are stored in the Buck cache and can easily be retrieved by procname.
Reviewed By: sblackshear
Differential Revision: D4077415
fbshipit-source-id: c2f1cc8
Summary:
- do a semantic analysis of each variable initializer to figure out if they need initialization
- add a flag to globals that is true when they are `constexpr`. In that case, no analysis is needed as the user + compile guarantee that it is a compile-time constant.
Reviewed By: sblackshear
Differential Revision: D4081273
fbshipit-source-id: 44dbe29
Summary:
Merging the results directories of targets on buck projects involved creating symbolic links into buck-out.
The bulk of files are .attr files: one per procedure. Creating these links can be a bottleneck, and the merge phase can be slower than the analysis phases on projects with many procedures.
This diff introduces multilinks to speed up merge.
A multilink is a file `multilink.txt` containing a sequence of paths
```
path/to/file1.ext
path/to/file2.ext
...
```
A multilink file is a compact way to represent a link for each entry.
This diff creates a multilink file for each `attributes/dir` directory, instead of one symbolic link for each file.
Reviewed By: jberdine
Differential Revision: D4067428
fbshipit-source-id: 911f8a9
Summary:
Config.analyze_models, set by the INFER_ANALYZE_MODELS environment
variable, is redundant with Config.models_mode.
Reviewed By: jvillard
Differential Revision: D4047338
fbshipit-source-id: 4522d65
Summary: Some arguments passed from infer.ml to infer.py were only used to pass further to infer.ml invocations. Those args should be passed by env variable anyway (???)
Reviewed By: jberdine
Differential Revision: D4048003
fbshipit-source-id: 6f5fbeb
Summary:
Python isn't needed anymore to pass options between `infer` and `InferClang`.
However, it is still needed to set up `PATH` so that we pick up compilation
commands.
Reviewed By: jberdine
Differential Revision: D4008469
fbshipit-source-id: 05c5716
Summary:
Checker for the Static Initialization Order Fiasco pattern:
https://isocpp.org/wiki/faq/ctors#static-init-order
1. Collect all globals (transitively) accessed in any given procedure.
2. Once the interprocedural analysis has finished, look at globals accessed in
initializers that do not belong to the current translation unit.
Reviewed By: sblackshear
Differential Revision: D3780266
fbshipit-source-id: 1d07161
Summary:
There's no reason for infer to be in lib/ anymore, move it to the same place as
the other binaries. Thus all binaries are in the same directory and Config.ml
can better know where things are.
Reviewed By: jberdine
Differential Revision: D4015958
fbshipit-source-id: c5e851f
Summary:
This fixes a perf issue on large files, where a copy of the type environment and control flow graph were loaded for each procedure analyzed in the file.
If the type environment or the control flow graph are big, and the file contains many procedures, this can cause a big memory overhead.
Reviewed By: jvillard
Differential Revision: D4008655
fbshipit-source-id: 11d07c1
Summary:
This changes executions of the former InferClang into a function call. In
particular, it can be called several times per execution.
The new InferClang must be called as if it was clang, and knows how to run
clang with our plugin to get the AST of the source file.
Reviewed By: akotulski
Differential Revision: D3981017
fbshipit-source-id: 7af6490
Summary:
This diff removes the unused support for reporting props, which enables
refactoring so that the 'base' directory has no dependencies, and the
'IR' directory depends only on 'base'.
Reviewed By: jvillard
Differential Revision: D3981352
fbshipit-source-id: 3700a23
Summary:
This changes the algorithm for pure join to keep the constraints that,
after normalization, occur in both arguments. Previously pure join
would normalize, filter, and then union the constraints of the
arguments.
Reviewed By: sblackshear
Differential Revision: D3970394
fbshipit-source-id: 3dc1672
Summary:
This is needed for later: InferClang will no longer be started once for each
source file to be analysed. Instead, it will be called to analyse several files
at once, and will analyse them one by one. Thus, `clang_lang` and `source_file`
are moved to `cFrontend_config` as references.
The biggest change this entailed was the new logging infrastructure, which was
depending on `Config.source_file`. This diff moves the logic entirely to
`Logging`, and changes the API so that executables wishing to log into files
have to set it up using `Logging.set_log_file_identifier`. This can be called
several times during the execution, allowing to dynamically change the log file
(eg, when analysing several source files one by one!).
Reviewed By: jberdine
Differential Revision: D3944148
fbshipit-source-id: 6129090
Summary:
Let's start migrating some of our bash script to OCaml to make them easier to
maintain and extend.
For now replace just one script and put it in lib/clang_wrappers/ at compile
time, where the former script used to be. Further simplifications will come
later.
Reviewed By: jberdine
Differential Revision: D3929988
fbshipit-source-id: b2d8b37
Jeremy Dubreil
Andrzej Kotulski
Martino Luca
Ben Hudson
Cristiano Calcagno
Josh Berdine
Dulma Churchill
Jules Villard
Sam Blackshear
Kihong Heo
Peter O'Hearn
Dino Distefano
Ryan Rhee