Summary: It seems that we don't need to rely on `make` anymore as the internal process manager is working fine on Buck Java projects.
Reviewed By: jvillard
Differential Revision: D7903639
fbshipit-source-id: 9b32f05
Summary:
This is an attempt to make things more consistent, and maybe save some work
from the `Format` module in case flambda doesn't have our backs.
Reviewed By: jberdine
Differential Revision: D7775496
fbshipit-source-id: 59a6314
Summary: Make errors stand out with colours. Also improve error messages around save states.
Reviewed By: mbouaziz
Differential Revision: D7928794
fbshipit-source-id: c81cfe2
Summary:
Make the starvation checker enabled by default.
Add a deadlock issue type, distinct to starvation, which will be kept for UI thread starvation.
Add checks so that checker will do nothing on non-Java code.
Reviewed By: mbouaziz, ddino
Differential Revision: D7908381
fbshipit-source-id: 889f373
Summary: Historically, this option was only used to parallelize the biabduction analysis, which is now run using the checkers framework.
Reviewed By: jvillard
Differential Revision: D7895118
fbshipit-source-id: 2a54bca
Summary:
One source of non-determinism is racing on procedure summaries when reporting. In particular, the summary of a method may be computed and stored by one thread, but another may be trying to report on it (eg, in cluster checkers).
One solution (at least until everything is in sqlite) is to have separate files just for the reports, a la linters. This diff improves the interface of LintIssues and generalises it ahead of using it in other analysers.
Reviewed By: jeremydubreil
Differential Revision: D7859973
fbshipit-source-id: 8672d3b
Summary:
Add warning 60 (unused module) to the list of fatal warnings. Whitelisting
modules at toplevel is tricky (see inline comments) but doable.
Reviewed By: mbouaziz
Differential Revision: D7790073
fbshipit-source-id: 6f591c4
Summary:
Add a `--source-files` option to `infer explore` to print information about the source files captured by infer.
More precisely, `infer explore --source-files` will print each row of the "source_files" table in the results database.
Option `--source-files-filter` can be used to filter output to file names matching an SQLite "LIKE" pattern.
Flags `--source-files-cfgs`, `--source-files-type-environment`, `--source-files-procedure-names` and `--source-files-freshly-captured` control which columns to print.
The printers for some existing types have been tweaked to improve the output.
Reviewed By: jvillard
Differential Revision: D7735535
fbshipit-source-id: 572389a
Summary: Random edits made while reading ZipLib.ml. They're really not interesting, sorry...
Reviewed By: sblackshear
Differential Revision: D7686159
fbshipit-source-id: 19ef1ce
Summary:
Given a difference between two files, return the relevant lines in the new file; a line is relevant when a change took place in it, or nearby. To generate a valid input for this parser, use unix-diff command with the following formatter arguments:
```
diff --unchanged-line-format="U" --old-line-format="O" --new-line-format="N" File1 File2
```
Reviewed By: ezgicicek
Differential Revision: D7385267
fbshipit-source-id: 0cc3143
Summary:
Upgrade ocamlformat, and base which needs to be done in sync in order to build
ocamlformat, and the other deps can come for the ride.
Reviewed By: jvillard
Differential Revision: D7663537
fbshipit-source-id: 3e90970
Summary:
We were using the "filename" as the key because it's (kinda) unique *and* human
readable, but with the `infer explore --procedures` interface we don't really
need the human readable part anymore, so we can just use the OCaml marshalling
of the pname as the key. The human-readable version (sans unique-fying hash) is
now another column in the table, used to match procedure names in
`--procedures-filter`.
Reviewed By: sblackshear
Differential Revision: D7639158
fbshipit-source-id: e714605
Summary:
Add a `--procedures` option to `infer explore` to print information about the
procedures captured by infer. More precisely, `infer explore --procedures` will
print each row of the "procedures" table in the results database. A new
`--procedures-filter` controls which procedures to print information about, and
there is one flag per column in the db too to print more or less options about
each procedure (in particular, we can now print attributes), with some defaults.
Reviewed By: sblackshear
Differential Revision: D7639062
fbshipit-source-id: 034a2b8
Summary:
It feels risky to me to swallow the `Sys_error` here since this could hide deeper issues with the deserialization of summaries.
This is only used with the Buck Integration for Java when extracting the summaries from the jar files.
Reviewed By: sblackshear
Differential Revision: D7490123
fbshipit-source-id: 68cd556
Summary:
This is unused, but the logic for keeping its value up to date was still alive
and kicking.
I don't know that making use of this flag would be easy: we could just use it
in `Ondemand.should_analyze`, but it would be unsound because a procedure might
need to be re-analysed becauese its dependencies have changed. Since there's
not code to deal with that currently I think it's best to remove it and
re-introduce it when we have some idea how to use it.
Reviewed By: sblackshear, jeremydubreil
Differential Revision: D7444179
fbshipit-source-id: 99a1ec5
Summary:
Utility function that's basically `Logging.progress` but with a more obvious
name for debugging.
The `[@deprecated ...]` ensures no uses gets landed, and the `[@warning "-32"]` ensures that it's not a compile error that there are no uses of this function.
Inspired by #716.
Reviewed By: mbouaziz
Differential Revision: D7443224
fbshipit-source-id: e94798f
Summary:
There's actually a nice separation between IR/, base/, istd/, and the rest of
infer, so they can be made into separate jbuilder libraries so that the
separation remains. This helps make sense of the infer codebase.
Also:
- move everything biabduction-related out of backend/ and into a new
biabduction/ directory. This clarifies the current situation where backend/
contains a mix of analysis-independent code (still there now), and
biabduction-specific code (moved to biabduction/).
- move everything from base/ that is not infer-specific into istd/, e.g. IList.ml
- kill unused `FbTraceCalls`
- A couple of files needed to move around to complete the separation of base/ and IR/
Reviewed By: mbouaziz
Differential Revision: D7381842
fbshipit-source-id: cd86dea
Summary:
New command line option job-id meant to contain the Sandcastle job id
itslog
- New command line option job-id meant to contain the ID of the job that Infer is running in
- If the id is passed to infer, then log it with every row in EventLogger
Reviewed By: dulmarod
Differential Revision: D7398537
fbshipit-source-id: c5186e9
Summary:
Limit the scope of what gets included into IStd.ml to only values that we want
to shadow. New values go into other files.
Also, build istd/ with `Core` open.
Reviewed By: mbouaziz
Differential Revision: D7382111
fbshipit-source-id: 969f0e8
Summary:
Infer reads the arguments passed to clang, in particular to filter out some
incompatible clang command-line options. But, infer only understands arguments
in arg files if they are presented one per line. That's usually the case, but
infer itself stores clang arguments from compilation databases all on one line.
This breaks filtering.
This changes the type of compilation database items to a list of arguments, so
that they can be stored one per line when possible.
Also does some cleanup/renamings, and remove trailing `_` from temp file names, eg:
clang_command_.tmp.2b2602.txt -> clang_command.tmp.2b2602.txt
Also, do not escape arguments from arg files when printing them in logs (that
was useless and ugly).
Reviewed By: mbouaziz
Differential Revision: D7365907
fbshipit-source-id: 5a3fe70
Summary:
There's no real reason not to use `Core` lists in this module. Changed the
interface to be more `Core`-like. Changed the `*_changed` functions to use a
ref to track changes instead of passing the changed state around.
Reviewed By: mbouaziz
Differential Revision: D7123211
fbshipit-source-id: b27791a
Summary:
- PerfStats can now report either time and/or memory stats, by default reporting both
- This is in preparation for adding just time or memory related logging in the future
Reviewed By: mbouaziz, dulmarod
Differential Revision: D7324556
fbshipit-source-id: e265b12
Summary: It adds an issue type, `BUFFER_OVERRUN_U5`, for alarms involving unknown values, i.e., when the trace set includes an unknown function call.
Reviewed By: mbouaziz
Differential Revision: D7178841
fbshipit-source-id: bfe857b
Summary: Add new clang_method_kind field to AnalysisIssue, logged similarly to the existing one in AnalysisStats
Reviewed By: dulmarod
Differential Revision: D7273660
fbshipit-source-id: d1ca79b
Summary:
- Noticed that there were two different type aliases for the same type, representing the return value of `__POS__`
- Combined them under `ocaml_pos` name which more closely matches the pervasive
- Moved to Logging module
Reviewed By: dulmarod
Differential Revision: D7194034
fbshipit-source-id: 22cb949
Summary:
- New Event type in EventLogger: PerformanceStats
- Contains some of the information logged to disk by PerfStats, as well as a `stats_type` tag to indicate which type of process the stats are from
- PerfStats.register_report_at_exit now takes 3 string arguments and constructs the file path and determines stats_type based off of them
- Changed all clients of PerfStats to use new api
Reviewed By: dulmarod
Differential Revision: D7131904
fbshipit-source-id: 9226b5d
Summary: Add a new command-line option `--external-java-packages` which allows the user to specify a list of Java package prefixes for external packages. Then the analysis will not report non-actionable warnings on those packages (e.g., inconsistent `Nullable` annotations in external packages).
Reviewed By: jeremydubreil
Differential Revision: D7126960
fbshipit-source-id: c4f3c7c
Summary:
`Sequence` API to walk over free variables in expressions, instead of computing lists with uniqueness constraints that make them have linear complexity for insertion.
Switch to a Set representation when we don't care about the order of elements,
otherwise to a `Hash_queue`:
https://ocaml.janestreet.com/ocaml-core/113.33/doc/core/Std/Hash_queue.mod/S.modt/
Often, we don't even need to compute the sequence of free variables, as we are
just testing membership/emptiness/...
Reviewed By: mbouaziz
Differential Revision: D7099294
fbshipit-source-id: e96f84b
Summary:
Instead of storing the type environment in infer-out/captured/foo.c/foo.c.tenv,
store it in the `source_files` table of the SQLite db. This limits the number
of files we create on disk.
The "file local" type environemnts are specific to the clang integration. For
Java, there is a "global tenv" file. Instead of matching on string names, this
diff also makes the API of `Tenv` reflect this situation.
The global tenv is serialized as a separate file in "infer-out/.global.tenv"
instead of "infer-out/captured/global.tenv", because "infer-out/captured/" will
soon be removed as it now only contains the global tenv (except in debug mode,
where it will still be created).
In the DB, we either store the local tenv for the file, or "global" to indicate
that the global tenv should be consulted.
This diff also moves `Cfg.store` to `SourceFiles.add` because that function
deals with more than just `Cfg.t`.
Reviewed By: jeremydubreil
Differential Revision: D6937945
fbshipit-source-id: 001c10a
Summary:
- EventLogger no longer depends on ProcAttributes
- This allows for other backend classes which are used by ProcAttributes to log in the future without creating a cyclic dependency structure
Reviewed By: dulmarod
Differential Revision: D7082444
fbshipit-source-id: f32a6e2
Summary: - When a CallTrace result is CR_skip, then a reason for skipping the call will be included in the record
Reviewed By: dulmarod
Differential Revision: D7082572
fbshipit-source-id: 20ed191
Summary:
- During the symbolic execution stage of the backend, Infer will log detailed stats about procedure calls
- Logging is accomplished directly within symExec/Tabulation
- call_result type is moved to tabulation.ml
CallStats was a broken module that allocated a lot of useless memory. Now, Specs.CallStats and InferPrint.CallsCsv as well as the Calls report kind in InferPrint no longer exist.
Reviewed By: dulmarod
Differential Revision: D7016439
fbshipit-source-id: 40911ee
Summary:
- change a use of clang_method_kind to match directly on the type rather than a brittle check for equality
- make the clang_method_kind field in AnalysisStats an option
- if procedure lang is not clang, then clang_method_kind field gets skipped while logging
Reviewed By: dulmarod
Differential Revision: D7010060
fbshipit-source-id: 077094d
Summary:
- JsonBuilder now has an add_string_opt function that takes a string option instead of a string
- Further alphabetized EventLogger
Reviewed By: dulmarod
Differential Revision: D7011763
fbshipit-source-id: 73f1a4b
Summary:
A simple intraprocedural analysis that tracks when a storage location is read or deleted.
For now, this works only with local variable storage locations; field and array accesses are ignored.
In order to test this, I added a new "use-after-lifetime" warning. It complains when a variable is read or deleted after it has already been deleted.
Reviewed By: jeremydubreil
Differential Revision: D6961314
fbshipit-source-id: 75e95a2
Summary:
Useful to debug without having to rerun commands.
Looks like this:
```
Usage Error: Failed to execute compilation command:
'/home/jul/infer.fb/infer/bin/../../facebook-clang-plugins/clang/install/bin/clang'
'args.txt'
++Contents of 'args.txt':
'arg1.txt' 'arghorashy_hello_c.txt' 'args.txt'
++Contents of 'arg1.txt':
'arghorashy_c.txt'
++Contents of 'arg_c.txt':
'-c'
++Contents of 'arg_hello_c.txt':
'examples/hello.c'
Error message:
clang-5.0: error: no such file or directory: 'examples/hello.c'
*** Infer needs a working compilation command to run.
```
Reviewed By: mbouaziz
Differential Revision: D6872550
fbshipit-source-id: 65cd026
Summary:
In some cases infer runs clang commands where we do not attach the plugin,
notably for pre-processor-only commands like `clang -E foo.c`. Usually these
commands are used by build systems to test their output, so infer should not
swallow it.
This makes infer echo the stdout of clang whenever we do not attach the plugin
to clang. When the plugin is attached, stdout is captured because that's where
the plugin outputs its results, so we cannot do the same in this case (not that
we would want to yet).
Fixes#696.
Reviewed By: mbouaziz
Differential Revision: D6912101
fbshipit-source-id: c4ad2e4
Summary:
Record "capture phases" in the runstate and in the source files table of the
database. Use this instead of filesystem timestamps to decide which files need
re-analyzing in the reactive analysis.
Reviewed By: jeremydubreil
Differential Revision: D6760833
fbshipit-source-id: 7955621
Summary:
- During backend execution, infer will log detailed stats about procedure analysis
- Logging is integrated with EventLogger
- `events_to_log` field added to Stats.t record in InferPrint
- New format in InferPrint - Logs
- `format_list` type changed to have a Utils.Outfile option to support Logs format
Reviewed By: dulmarod
Differential Revision: D6834538
fbshipit-source-id: 8c847f5
Summary:
Make dead code detection part of `make test` so that dead code stops creeping
in. It's only enabled if all the analysers are enabled and if this is a
facebook build, because the dead code detection will have false positives
otherwise.
Reviewed By: mbouaziz
Differential Revision: D6807395
fbshipit-source-id: ebbd835
Summary:
I needed to do this for something, now I don't know if I want to do the thing
anymore but this seems generally useful to decrease a little bit the size of
Config.ml.
Reviewed By: sblackshear, mbouaziz
Differential Revision: D6796427
fbshipit-source-id: d9c009d
Summary:
Also, always log failures.
This also shows that the dead code detection does not detect dead exceptions :/
Reviewed By: jeremydubreil
Differential Revision: D6796843
fbshipit-source-id: 3d0ff5c
Summary:
Also, make it explicit when we load the global tenv instead of the per-file tenv.
This allows for some nice simplifications in some places, notably:
- `tenv_file` is gone from `Exe_env.file_data`
- `DB.global_tenv_fname` is no more
This will help moving the tenv from the capture/source_file/ directories on the
filesystem to the database, as keys for the relevant table are `SourceFile.t`.
Reviewed By: mbouaziz
Differential Revision: D6796594
fbshipit-source-id: 1ffd5b0
Summary:
They were constructed for each source file, and then joined into a global call
graph, only to get per-file lists of procedures. A tad wasteful.
Get this list from cfgs instead. Still record them in `exe_env` for now as
changing that code is a whole other beast.
One test falls victim of the flakiness of the analysis of recursive functions.
Reviewed By: jeremydubreil, mbouaziz
Differential Revision: D6324268
fbshipit-source-id: d5ff58f
Summary:
Last piece of significant logic relying on the call graph: the execution
environment. Mostly carve out the logic to detect duplicate symbols to not rely
on the call graph.
Also make the keys of the `file_map` be source files, because not having the
"cg filename" makes it harder otherwise.
Reviewed By: sblackshear
Differential Revision: D6621196
fbshipit-source-id: cab50ba
Summary:
In preparation for getting rid of call graphs, we need to find another way to
get the list of defined procedures (which is the only place where we use the
globally-computed call graph for now).
The natural way to get the list of procedures defined in a file is to load the
cfg for that file and look at the proc names that are the keys of the cfg. This
is way too expensive, as the CFG is big. Thus, we cache this list of proc names
as another column in the SQLite database of cfgs. This gives good performance
in benchmarks.
Reviewed By: jeremydubreil
Differential Revision: D6621142
fbshipit-source-id: ed265fe
Summary: At each call to `Component$Builder.build()`, checks that the required props for `Component` have been set via prior calls to the `Builder`. Does not yet handle `Prop(optional = true)`, but will address that in a follow-up.
Reviewed By: jeremydubreil
Differential Revision: D6735524
fbshipit-source-id: 0c812fd
Summary:
Record the db schema, infer version, and run dates into
infer-out/.infer_runstate.json. This allows us to check on startup whether the
results directory was generated using a compatible version of infer or not, and
give a better error message in the latter case than some SQLite error about
mismatching tables.
This will be used in a follow-up diff to record capture phases too, and avoid
relying on filesystem timestamps of the infer-out/capture/foo/ directories for
reactive analysis.
Had to change some tests Makefiles to make sure they do not attempt to re-use
stale infer-out directories, which would now fail the run.
The stale infer-out directory gets deleted if `--force-delete-results-dir` is
passed (but a warning still gets printed).
Reviewed By: mbouaziz
Differential Revision: D6760787
fbshipit-source-id: f36f7df
Summary:
This declutters `CommandLineOption` a little bit, and will be useful in a
follow-up diff where `InferCommand.t` will be used from an atd-generated file.
Reviewed By: mbouaziz
Differential Revision: D6772990
fbshipit-source-id: 3d32d00
Summary:
Reading a different .inferconfig makes little sense and is in contradiction
with the fact that we try hard to make all the infer processes agree on the set
of options.
Reviewed By: sblackshear
Differential Revision: D6761146
fbshipit-source-id: 67a0c54
Summary:
- During program translation, infer logs details about Unimplemented exceptions that are caught
- Exception type, source location, exception triggering location, and the ast_item it occurred in
- Logging is integrated with EventLogger library
- New type of event in EventLogger
- Unimplemented exception type extended to add support for storing these extra details
- All instances where unimplemented exception is raised modified to pass these details
Reviewed By: dulmarod
Differential Revision: D6748734
fbshipit-source-id: 725c7f3
Summary:
Getting double-logged messages when logs have not yet been setup is confusing,
so just don't log these messages to a log file.
Reviewed By: mbouaziz
Differential Revision: D6739173
fbshipit-source-id: 14db6b0
Summary: Use the Hashtbl functions directly as `Cfg` knows that a cfg is a hashtbl.
Reviewed By: sblackshear, jeremydubreil
Differential Revision: D6727732
fbshipit-source-id: 2cdda91
Summary:
Found the dead code with the script in the next commit, iteratively until no
warnings remained.
Methodology:
1. I kept pretty-printers for values, which can be useful to use from infer's REPL (or
when printf-debugging infer in general)
2. I kept functions that formed some consistent API (but not often, so YMMV), for instance if it looked like `Set.S`, or if it provides utility functions for stuff in development (mostly the procname dispatcher functions)
3. I tried not to lose comments associated with values no longer exported: if the value is commented in the .mli and not the .ml, I moved the comment
4. Some comments needed updating (not claiming I caught all of those)
5. Sometimes I rewrote the comments a bit when I noticed mis-attached comments
Reviewed By: mbouaziz
Differential Revision: D6723482
fbshipit-source-id: eabaafd
Summary:
Almost all the files are supposed to do this. When they don't, the build adds
it automatically anyway, but it's better to always include it to avoid
confusion (and help other automated tools).
Reviewed By: mbouaziz
Differential Revision: D6723285
fbshipit-source-id: 0fe8a16
Summary:
- After completing program translation, infer logs # of attempted procedure translations and # of completed procedure translations via EventLogger
- New possible type of event in EventLogger
Reviewed By: dulmarod
Differential Revision: D6711662
fbshipit-source-id: 5e31332
Summary:
This commit augments codes to clean up temporary files generated by the clang frontend.
Currently clang frontend leaves large number of temporary files int the tmp directory, and it could be seen for example by running these command:
```
git clean -xdf infer/models/
mkdir /tmp/infer
env TMPDIR=/tmp/infer make infer_models
ls -l /tmp/infer
```
P.S. Analyzing real project could easily cause each infer capture to leave hundreds of files in /tmp
P.P.S. There are 11 total references to Filename.temp_file however, other 9 seems don't leak temporary files in such large scale (at least not when using the clang frontend).
Closes https://github.com/facebook/infer/pull/816
Reviewed By: jvillard
Differential Revision: D6385311
Pulled By: dulmarod
fbshipit-source-id: f7956b0
Summary: Get the error message from the database when there's an error, together with the error type.
Reviewed By: mbouaziz
Differential Revision: D6621695
fbshipit-source-id: 6bc706d
Summary: Somehow sqlite allows this simpler statement when using `exec` but not `prepare`.
Reviewed By: mbouaziz
Differential Revision: D6407732
fbshipit-source-id: 01f029f
Summary:
This makes sure that sqlite doesn't hold read locks for longer than necessary,
which could starve the process of cleaning up the WAL file. This ensures that
the statement is reset as soon as we're done reading.
I haven't observed a difference with this change, and could not find evidence
that it should change something in the docs. Internet wisdom pointed at this as
a potential issue and I was observing it in another change, so it's good to
rule it out.
Reviewed By: mbouaziz
Differential Revision: D6404353
fbshipit-source-id: a123cd6
Summary: Previously we had a single sanitizer kind for escaping, but this isn't quite right. A function that escapes a URL doesn't necessarily make a string safe to execute in SQL, for example.
Reviewed By: the-st0rm
Differential Revision: D6656376
fbshipit-source-id: 572944e
Summary:
Upgrade ocamlformat to 0.3, and (necessarily) base to v0.10.0.
- Fix accumulated mis-formatting
- Update opam.lock to unbreak clean build
- Update to base v0.10.0
- Update opam.lock for base
- Update offline opam repo
- Everyone should already have removed their ocamlformat pin
- ocamlformat 0.3 supports output to stdout natively
- bump version of ocamlformat
Reviewed By: jeremydubreil
Differential Revision: D6636741
fbshipit-source-id: 41a56a8
Summary: This factors out some duplicated code for {,de}serializing source files.
Reviewed By: mbouaziz
Differential Revision: D6324234
fbshipit-source-id: 1741657
Summary:
Instead of storing the cfgs of source files inside their own individual files,
put them in results.db, in their own table. (that table may change in the
future to map source files to more than just their cfgs, eg their tenv as well)
Reviewed By: jberdine
Differential Revision: D6297201
fbshipit-source-id: 7fa891d
Summary: There was a back and forth conversion between `string` and `IssueType.t` which was not necessary.
Reviewed By: sblackshear
Differential Revision: D6562747
fbshipit-source-id: 70b57a2
Summary: In every place this was used except one, `debug_mode` is also used as a gate.
Reviewed By: jeremydubreil
Differential Revision: D6450913
fbshipit-source-id: 6a5716d
Summary:
On spinning disks the performance of commits are worse when `synchronous=NORMAL`.
Reading the documentation of SQLite, when `synchronous=OFF` there's a risk of DB corruption when the operating system crashes or the computer loses power before that data has been written to the disk surface; on the other hand, a crash in Infer should keep data in the DB in a sound state.
Buck reached the same conclusions too: 4680162279
Reviewed By: mbouaziz
Differential Revision: D6413384
fbshipit-source-id: 99e4650
Summary:
This is a good moment to close Sqlite's DB handles, and in general can be used to postpone some actions right before infer terminates.
Since exiting is done via uncaught exception handling, the `late_epilogue` callback will run at the very end, even after all the `at_exit` callbacks have been invoked. The only exception is made in case of signalling, in which case the `late_epilogue` is still invoked, but before any of the `at_exit` callbacks.
Reviewed By: jvillard
Differential Revision: D6404961
fbshipit-source-id: 8ff7a05
Summary:
- Plug model checkers
- Add alloc size safety condition on alloc of negative, zero or big size
Reviewed By: sblackshear
Differential Revision: D6375144
fbshipit-source-id: bbea6f3
Summary: There is a lot of code to create LaTeX output of the Infer datastructures, but this does not seem to be used anymore.
Reviewed By: jvillard
Differential Revision: D6355686
fbshipit-source-id: 55de8e9
Summary: This option was for compatibility with the command line options of the previous, but is no longer used. This diff removes the option and the deprecated code.
Reviewed By: sblackshear, mbouaziz
Differential Revision: D6351097
fbshipit-source-id: 0e4cfc5
Summary: This will avoid confusions when running `-a infer --racerd` which would silently not running RacerD before this diff.
Reviewed By: sblackshear
Differential Revision: D6374139
fbshipit-source-id: 2cb5004
Summary:
Naming a variable `_foo` makes the compiler not warn about them if they are
unused, but there are lots of instances of such variables in the code where
they are in fact used, defeating the warning and introducing confusion for
those used to this naming convention.
Basically `sed -i -e "s/ _\([a-zA-Z][a-zA-Z0-9_']*\)/ \1_/g" **/*.ml` followed
by manual fixing of compilation errors (lots of `compare__foo` ->
`compare_foo_`).
Reviewed By: mbouaziz
Differential Revision: D6358837
fbshipit-source-id: 7ffb4ac
Summary:
To resole #797 this adds runtime option to select VFS for SQLite,
When infer runs on WSL this defaults to "unix-excl" (https://sqlite.org/vfs.html) and if VFS is specified, then WAL is not enabled (since WAL is non compatible with custom VFS - https://www.sqlite.org/wal.html).
Closes https://github.com/facebook/infer/pull/798
Reviewed By: jvillard
Differential Revision: D6335037
Pulled By: dulmarod
fbshipit-source-id: d9b9a58
Summary:
Change ocamlformat installation procedure to use opam instead of
pinning.
Reformat all code with v0.2, which has a few improvements.
Reviewed By: jvillard
Differential Revision: D6292057
fbshipit-source-id: 759967f
Summary:
This diff takes the first step toward a more general filtering
system. This step is concerned only with filtering at the reporting
stage, filtering for the capture and analysis stages is left for
later.
This diff adds a new command line / config option
```
--filter-report +string
Specify a filter for issues to report. If multiple filters are
specified, they are applied in the order in which they are
specified. Each filter is applied to each issue detected, and only
issues which are accepted by all filters are reported. Each filter
is of the form:
`<issue_type_regex>:<filename_regex>:<reason_string>`. The first
two components are OCaml Str regular expressions, with an optional
`!` character prefix. If a regex has a `!` prefix, the polarity is
inverted, and the filter becomes a "blacklist" instead of a
"whitelist". Each filter is interpreted as an implication: an issue
matches if it does not match the `issue_type_regex` or if it does
match the `filename_regex`. The filenames that are tested by the
regex are relative to the `--project-root` directory. The
`<reason_string>` is a non-empty string used to explain why the
issue was filtered.
See also infer-report(1) and infer-run(1).
```
Reviewed By: jvillard
Differential Revision: D6182486
fbshipit-source-id: 9d3922b
Summary:
Use a distinct issue type for the Java and C++ concurrency analyses,
as the properties they are checking are significantly different.
Reviewed By: sblackshear
Differential Revision: D6151682
fbshipit-source-id: 00e00eb
Summary:
:
Make both buck capture and compilation database handle buck command line arguments and invoke buck query the same way.
Plus allow:
- target patterns `//some/dir:` and `//some/dir/...`. However since `//some/dir:#flavor` and `//some/dir/...#flavor` are not supported, they need to be expanded before adding the infer flavor.
- target aliases (defined in `.buckconfig`)
- shortcuts `//some/dir` rewritten to `//some/dir:dir`
- relative path `some/dir:name` rewritten to `//some/dir:name`
Reviewed By: jvillard
Differential Revision: D5321087
fbshipit-source-id: 48876d4
Summary:
Linters are now considered a "checker", like backend checkers. This makes, eg,
`--racerd-only` disable the linters, which is more intuitive.
We can now express `-a linters` and `--clang-frontend-action` in terms of these
two new options. For instance, `-a linters --clang-frontend-action lint` is the
same as `--linters-only --no-capture`.
This is another step in the direction of getting rid of `--analyzers`.
Reviewed By: dulmarod
Differential Revision: D6147387
fbshipit-source-id: 53622b2
Summary: A stepping stone to have descriptive issue types for each kind of flow rather that lumping everything into `QUANDARY_TAINT_ERROR`.
Reviewed By: mbouaziz
Differential Revision: D6126690
fbshipit-source-id: a7230c0
Summary: This check is deprecated and will be replaced by a dedicated checker to detect unitialized values.
Reviewed By: mbouaziz
Differential Revision: D6133108
fbshipit-source-id: 1c0e9ac
Summary:
Install ocamlformat from github as part of `make devsetup`, and use it
for formatting OCaml (and jbuild) code.
Reviewed By: jvillard
Differential Revision: D6092464
fbshipit-source-id: 4ba0845
Summary:
Buck reads the version on stderr or, very recently, from either stdout or stderr.
This makes infer output the version of stderr when called from Buck or invoked as javac, and on stdout otherwise.
Reviewed By: martinoluca
Differential Revision: D6098392
fbshipit-source-id: 23f1d5a
Summary: This makes `--biabduction-blacklist-path-regex` and others work as expected.
Reviewed By: mbouaziz
Differential Revision: D6088625
fbshipit-source-id: 8f1daa3
Summary:
This is a better default than running the biabduction analysis only, now that
we have several mature checkers.
Reviewed By: jeremydubreil
Differential Revision: D6051186
fbshipit-source-id: 04ac0c6
Summary:
Refactor `RegisterCheckers` to give a record type to checkers instead of a tuple type.
Print active checkers with their per-language information.
Improve the manual entries slightly.
Reviewed By: sblackshear
Differential Revision: D6051167
fbshipit-source-id: 90bcb61
Summary:
This is useful if some command behaves like one infer knows how to integrate with. For instance:
```
infer --force-integration clang -- clang-3.8 -c examples/hello.c
```
Reviewed By: jeremydubreil, mbouaziz
Differential Revision: D6051589
fbshipit-source-id: dd693b0
Summary: The order of the elements in the list maters since the function `string_to_analyzer` will return the fist element found in the list. Inverting the `"biabduction"` and `"infer"` entries in the list allows D6051146 to have no functional implications.
Reviewed By: sblackshear
Differential Revision: D6055840
fbshipit-source-id: 6cf5ac2
Summary:
One day `-a infer` will alias `-a checkers` so for now create another, more
explicit analyzer name that can be used to migrate progressively. For instance,
after this commit what should continue to use `-a biabduction` can change to
that, so that when `-a infer` becomes an alias to `-a checkers` it can keep
working.
Reviewed By: jeremydubreil
Differential Revision: D6051146
fbshipit-source-id: 1ef4c34
Summary:
This generates `--resource-leak-only` automatically, and make the other
checkers' `-only` option work as expected with respect to `--resource-leak` too
(eg, `--resource-leak --biabduction-only` disables resource leak).
Reviewed By: jeremydubreil
Differential Revision: D6051134
fbshipit-source-id: 2d4a2ba
Summary:
1. Mark some Makefile targets as depending on `MAKEFILE_LIST` so they get rebuilt on Makefile changes
2. Do not show boolean options with no documentation in the man pages (like we do for other option types).
3. Default to Lazy dynamic dispatch for the checkers.
4. In the tests, use `--<checker>-only` instead of relying on `--no-default-checkers`
5. `--no-filtering` is redundant if `--debug-exceptions` is passed
Reviewed By: jeremydubreil
Differential Revision: D6030578
fbshipit-source-id: 3320f0a
Summary:
Another step toward running the biabduction analysis as a checker.
Depends on D6038210
Reviewed By: jvillard
Differential Revision: D6038682
fbshipit-source-id: fed45bf
Summary:
This is to avoid getting BUSY from sqlite when the machine is busy (not
necessarily busy because of infer).
Reviewed By: jberdine
Differential Revision: D6020022
fbshipit-source-id: ca0f913
Summary:
Attempting to translate these will not go well as the declaration still depends
on some template arguments. Added a test that was previously crashing the
frontend.
Also extend the catching of "Unimplemented" and other errors to `translate_one_decl` as it was useful to debug this issue. In particular, reraise all exceptions and log some additional context when doing so.
update-submodule: facebook-clang-plugins
Reviewed By: mbouaziz
Differential Revision: D5976357
fbshipit-source-id: fca8e38
Summary:
Running `infer report foo.specs` would overwrite report.json with an empty json.
Only recompute report.json when running `infer-analyze` or `infer-run`.
Reviewed By: mbouaziz
Differential Revision: D5963500
fbshipit-source-id: e579c6f
Summary:
Use a monotonic time source instead.
Also, sleep between retries in the Serialization code.
Reviewed By: jberdine
Differential Revision: D5941697
fbshipit-source-id: 05efbe1
Summary:
Despite what the sqlite manual says, it looks like it's possible for sqlite not
to remove the -shm and -wal files after a successful termination.
Close the database and clean them up in Buck mode so that they do not perturb
the cache.
Reviewed By: martinoluca
Differential Revision: D5953967
fbshipit-source-id: 9068b01
Summary: If we know for sure we won't need to store an attribute in the DB, there's no need to compute its marshalled value.
Reviewed By: jberdine
Differential Revision: D5891050
fbshipit-source-id: cf4534e
Summary:
This adds more structure to the SQL schema backing attributes. With that, we
can transfer the logic for updating attributes in SQLite, instead of doing
optimistic concurrency in the client.
Reviewed By: jberdine
Differential Revision: D5891038
fbshipit-source-id: 6577ba2
Summary:
- use a similar key as for specs in the attributes table
- cache blob computations
- this improves memory usage a lot
Reviewed By: mbouaziz
Differential Revision: D5824177
fbshipit-source-id: c318577
Summary:
Preparing statements allocates memory and is generally best done once and for all.
We need to re-prepare statements when opening a new DB connexion (after each fork).
Reviewed By: mbouaziz
Differential Revision: D5824157
fbshipit-source-id: 4d239ac
Summary: Not sure if useful but seems sensible. It disappears at the top of the stack when we do merging in SQL.
Reviewed By: mbouaziz
Differential Revision: D5824131
fbshipit-source-id: fd64752
Summary:
Use an SQLite database to store proc attributes, instead of files on disk.
Wrap SQLite operations in two layers:
1. `SqliteUtils` provides helper functions to make sure DB operations succeed
2. `KeyValue` provides a functor to expose a simple and type-safe key/value store backed by the SQLite DB.
Reviewed By: jberdine
Differential Revision: D5640053
fbshipit-source-id: 31050e5
Summary:
`reraise` was error-prone when one forgot to save the backtrace between where the exception is caught and where it is reraised.
If any exception was raised (even caught) in between, the printed backtrace would be the one of the last exception thrown and it would be very confusing.
This diff kills `reraise` and introduces `reraise_after exn ~f` and `reraise_if exn ~f` to be used right after catching the exception.
Also turned some of them to the common pattern `try_finally ~f ~finally`.
Reviewed By: jvillard
Differential Revision: D5911244
fbshipit-source-id: 9883d1e
Summary:
Since D5381239, infer is careful not to delete directories that do not "look
like" results directories on startup, in case the user passed, eg, `-o /`.
In our repo, lots of results dir are created by build/test of infer, and when
the version of infer changes and the expected contents of results directories
change then it might start refusing to delete the results directories created
with another version of infer.
Add an option to force infer to delete the results directory no matter how
dodgy it looks, and use it in our repo by adding the option in every
.inferconfig.
Reviewed By: mbouaziz
Differential Revision: D5870984
fbshipit-source-id: 09412de
Summary: Parmap delivers a better scheduling, which works as a pipeline, as opposed to what existed before, which schedules processes in batches.
Reviewed By: jvillard
Differential Revision: D5678661
fbshipit-source-id: a632c71
Summary:
We need to make sure that destructors of virtual base classes are called only once. Similarly to what clang does, we have two destructors for a class: a destructor wrapper and an inner destructor.
Destructor wrapper is called from outside, i.e., when variables go out of scope or when destructors of fields are being called.
Destructor wrappers have calls to inner destructors of all virtual base classes in the inheritance as their bodies.
Inner destructors have destructor bodies and calls to destructor wrappers of fields and inner destructors of non-virtual base classes.
Reviewed By: dulmarod
Differential Revision: D5834555
fbshipit-source-id: 51db238
Summary:
The clang frontend uses `assert false` for unimplemented features that should
abort method translations, as well as for genuine internal errors.
Distinguishing between the two, we can fail hard on the latter and not the
former.
1. This introduces a new exception `Unimplemented` that is used instead of `assert false` where appropriate.
2. Changed some other cases into `die ...` (when there's an error message to display)
3. Wherever a path in the code that we assumed to be unreachable was observed reachable, we now raise `IncorrectAssumption`. These should be fixed, but the fixes are not obvious.
Reviewed By: sblackshear
Differential Revision: D5784384
fbshipit-source-id: 61b55af
Summary: This will run infer with `--no-keep-going` by default in our tests.
Reviewed By: sblackshear
Differential Revision: D5814237
fbshipit-source-id: c1e1a4e
Summary:
"Running as clang" was its own infer subcommand. That's not terribly good
because it makes it hard to specify another subcommand, and in particular it
broke the `compile` subcommand in some integrations because "running as clang"
would always do capture. The cmake tests required to run with `--keep-going`
because of this.
Instead of having its own fake subcommand, simply add a new boolean in the
config for "infer runs as clang", as we do for javac already (used in the mvn
integration).
Also make logging of the environment better.
Reviewed By: jberdine
Differential Revision: D5813986
fbshipit-source-id: 72b96cd
Summary: I often find myself needing a generic `Pp.seq` where I can specify the separator.
Reviewed By: jeremydubreil
Differential Revision: D5803915
fbshipit-source-id: fb8d30d
Summary: Try to preserve the original backtrace. Introduce `reraise` in the global namespace.
Reviewed By: jberdine
Differential Revision: D5804121
fbshipit-source-id: 0947a47
Summary:
This makes it much easier to read infer logs coming from these subprocesses
that use a sub-results dir.
Reviewed By: jberdine
Differential Revision: D5777783
fbshipit-source-id: f074536
Summary: With Logging.exit you have more control of the code that invokes exit, for example when forking and running certain functions that may in turn invoke exit, and you want to handle the execution flow differently - like invoking certain callbacks before exiting, or not exiting at all.
Reviewed By: jvillard
Differential Revision: D5746914
fbshipit-source-id: 596fba1
Summary:
Calling `Exn.backtrace` doesn't give us the current backtrace, only the one of
the latest exception raised. Change the logic of toplevel exception catching to
print the right backtrace.
Also, do not use `Format` to print from `Config` as `Logging` may have messed
with it already.
Finally, throw in some colours!
Reviewed By: jberdine
Differential Revision: D5764825
fbshipit-source-id: cd51688
Jeremy Dubreil
Jules Villard
Nikos Gorogiannis
Dino Distefano
Frédéric Bour
Martino Luca
Josh Berdine
Ezgi Çiçek
Sam Blackshear
Varun Arora
Sungkeun Cho
Dulma Churchill
Tony Hu
Vladimir Silyaev
Mehdi Bouaziz
Daiva Naudziuniene