Summary:
This generates a lot of code (the datatype is not trivial) and
transitively requires other datatypes to have compare functions, just so
we can then ask whether the mode is "Analyze".
Reviewed By: ngorogiannis
Differential Revision: D19164589
fbshipit-source-id: be6a12b41
Summary:
This diff enables parsing and auto-formatting documentation
comments (aka docstrings).
I have looked at this entire diff and manually made some changes to
improve the formatting. In some cases it looked like it would take too
much time, or benefit from someone more familiar with the code doing
it, and I instead disabled auto-formatting docstrings in those files.
Also, there are some source files where the docstrings are invalid,
and some where the structure detected by the parser appears not to
match what was intended. Auto-formatting has been disabled for these
files.
Reviewed By: ezgicicek
Differential Revision: D18755888
fbshipit-source-id: 68d72465d
Summary:
First step towards a global analysis. A new command line flag activates the step in `Driver`.
The whole-program analysis is a simple, quadratic (inefficient-as-yet), iteration over all domain elements. However, it is restricted to those elements that are explicitly scheduled to run.
Reviewed By: skcho
Differential Revision: D17787441
fbshipit-source-id: 9fecd766c
Summary:
- Convert `task_generator` into a module of `ProcessPool` and collect inside the two combinators which were in semi-random places.
- Make `SyntacticCallGraph` export a `task_generator` as opposed to a call-graph builder.
- Separate `target` type and put it in its own module to avoid dependency cycles.
Reviewed By: skcho
Differential Revision: D18425718
fbshipit-source-id: 7957edac8
Summary:
- Putting test determinator in own directory
- Putting Java procname creation stuff in its own module
Reviewed By: skcho
Differential Revision: D17929885
fbshipit-source-id: 4f2578566
Summary: This makes more explicit what we are talking about here. Also, in extending test determinator to clang, the name is incorrect, but the set is generic procnames which is fine to use for clang, just the name is wrong.
Reviewed By: ngorogiannis
Differential Revision: D17855338
fbshipit-source-id: e93bae083
Summary:
The documentation and uses of filtering disagree. One typical usage is deduplication.
Split that where obvious, add comments where not obvious, and leave alone when obviously unrelated to deduplication.
Reviewed By: mityal
Differential Revision: D17715329
fbshipit-source-id: ec757927b
Summary:
I observed a bug in incremental analysis for thread safety analysis, where a thread safety violation was not being reported because the folder `racerd` was not being cleaned. This meant that the violation was determined to be a preexisting issue when it was actually an introduced issue.
This method can be used to fix this problem by cleaning the `racerd` folder. It also cleans the `captured` folder, I've done this following the original version of the method (see D16602417).
I'm not sure if the `captured` folder is used; it wasn't used in the tests I did. Thoughts about this?
Reviewed By: ngorogiannis
Differential Revision: D17261504
fbshipit-source-id: 8fea23e98
Summary:
An exception thrown during capture/analysis may leave the daemon
running. Kill it even when one is thrown.
Reviewed By: martintrojer
Differential Revision: D17181090
fbshipit-source-id: a7b002f23
Summary:
Implementation of write-serializer for Sqlite. Points of note:
- A Unix socket is used for communication. This avoids buffer-size limitations, as the objects we send for writing may exceed said limits.
- No daemon is used if running under buck or in genrule mode, as this usually means a single-threaded job capturing into the DB.
- When the daemon is running, read-only access is *not* enforced for other processes. This makes starting and stopping the daemon during Infer execution easier and more robust. In WAL mode this should not have any effect on performance.
- This version is not economical with connections, it uses one per query, todo.
Reviewed By: jvillard
Differential Revision: D17077183
fbshipit-source-id: fa9877d6c
Summary:
Write contention is becoming a problem in parallel capture (eg when make runs with high parallelism) or when analysis writes CFGs to the DB in parallel (eg when analysing blocks in ObC). This is believed to lead to BUSY errors in Sqlite.
This is step 1 of a process where all writes are cordoned-off in one module, and fixing the interface for that module.
Reviewed By: skcho
Differential Revision: D16985034
fbshipit-source-id: 3d7ce381b
Summary:
The clang frontend has bugs. When a bug we know about happens some
exception is raised and, most of the time, logged away so as not to
crash the whole process. This catching of exceptions wasn't done from
testDeterminator so it could crash where capture didn't. This diff wraps
the crashy function in test determinator to avoid that.
Reviewed By: ngorogiannis
Differential Revision: D16963178
fbshipit-source-id: 87a4ff70b
Summary:
It's not being worked on and is not in a state where it works.
It would probably better to write this as a script of some kind or else
resurrect this subcommand in a form where it behaves more like a script,
ie fork/execs infer analyses instead of having them be function calls
(but then it might as well *be* a script as it would likely be more
flexible).
In any case...
youarealreadydead
Reviewed By: ezgicicek
Differential Revision: D16602417
fbshipit-source-id: d0d129539
Summary:
- make most behaviours independent of the java version so that either works fine without user intervention
- modify regexp used to parse `javac` output to work for all versions
- no need to be sure we are in Java 11 to match java 11-only method name in quandary
- for the rest, provide a command-line flag to specify the java version manually in case it differs from the version that infer was built against
- this only affects the Maven integration for now
To do all that, also change the configure script to record the version of java instead of just a boolean for whether it's >= 10.
Reviewed By: ezgicicek
Differential Revision: D16493988
fbshipit-source-id: 622e91b25
Summary:
Modify the scheduler to collect results from children at the end of the
parallel execution. Use this to collect backend stats and log their
aggregated sum.
Reviewed By: ezgicicek
Differential Revision: D16358867
fbshipit-source-id: 775792ef7
Summary:
The genrule-capture integration with Java relies on a buck config flag `infer.infer_bin=<path to infer>` (see test changes in `DEFS` below).
In a CI environment where the infer binary is checked out under a random directory, this means that the buck genrule is keyed by a random string (the path to infer), and this defeats caching.
Switch to the following contract: the genrule target does not expect a config flag at all. Instead it runs whichever `infer` binary is in the path. To make sure the binary is the same one with the originator, the capture integration runs buck under a modified `PATH` where the originator `infer` is sure to be the first matching entry.
NB cache invalidation is still OK because we rely on `infer.version` buck config flag, which will be hashed into the rulekey.
Reviewed By: jvillard
Differential Revision: D16332696
fbshipit-source-id: 2975d5c26
Summary:
newer is better, right?
All the code changes in infer are because of core being bumped to v0.12.
Reviewed By: jberdine
Differential Revision: D16223183
fbshipit-source-id: f3c339966
Summary:
Passing an absolute project path as buck config flag makes buck caching almost impossible for infer artefacts, since on every host/run that directory can be different.
Eliminate that and rely on shell commands to find the project root, executed within the genrule.
Reviewed By: jvillard
Differential Revision: D15963807
fbshipit-source-id: b6e590029
Summary:
Move genrule capture integration logic from shell to OCaml.
Also, stop relying on side-effects of buck compilation for constructing the infer-deps.txt file used for merging. Now this is obtained by passing `--show-output` to buck, which spits out the `buck-out` output paths to the targets we asked to build.
Reviewed By: ezgicicek
Differential Revision: D15715608
fbshipit-source-id: 8fa896ba6
Summary:
API and stub implementation for real-time logging capabilities.
Low-level implementation requires interaction with FB-specific deployment of Scribe, hence it is stubbed out.
Reviewed By: jberdine
Differential Revision: D15259559
fbshipit-source-id: 712cb99e1
Summary:
A more dynamic scheduling scheme will potentially run into the situation where no new work packets can be scheduled, but more work will be possible to schedule in the future, perhaps when some dependent work packet finishes being analysed.
The current implementation prevents that, as it expects that if a worker goes idle, it stays idle.
The changes here address this in two parts:
- the `select` call is always given a finite timeout. If given an infinite timeout, we will not be able to poll the task generator for more work, where none were previously possible.
- when the `select` call times out without updates, check if there is an idle child, and if so if the task generator has more work right now.
See also ProcessPool.mli for comments.
Reviewed By: mbouaziz
Differential Revision: D15197749
fbshipit-source-id: babe5da8e
Summary:
Before moving to any kind of non-trivial scheduling, we need to change the Tasks interface.
In particular, it's too restrictive to expect that the tasks to be scheduled are provided as a list before starting execution. For example, dynamic scheduling does not fit the bill here. Also, the list expectation means all scheduling work has to be done up front.
The solution here is to move to a `Sequence`-like interface with one difference:
- The function returning the next task expects a task option argument. That argument is the task that was just finished (if any) by the worker expecting new work. This will be useful for things like task dependencies (for instance, a procedure has been analysed, and can be marked so).
Reviewed By: mbouaziz
Differential Revision: D15181613
fbshipit-source-id: 21f3ba825
Summary: Experimental feature: Use memcached for summaries as a look-aside cache during analysis.
Reviewed By: jvillard
Differential Revision: D12939311
fbshipit-source-id: 9f78994e2
Summary: Use the value of other options instead since we're trying to get rid of it. This should be equivalent.
Reviewed By: jeremydubreil
Differential Revision: D9943274
fbshipit-source-id: 055e1bdd2
Summary:
This adds an option `--trace-events` that generates a Chrome trace event[1] to
quickly visualise the performance of infer.
Reviewed By: mbouaziz
Differential Revision: D9831599
fbshipit-source-id: 96a33c627
Summary: Buck is allowing compiler commands with no source files and skipping them when using the in-memory complier mode. However, those commands are not skipped when using an external compiler. Simulating this behavior at the level of Infer.
Reviewed By: mbouaziz, ngorogiannis
Differential Revision: D9795043
fbshipit-source-id: e80cfa453
Summary:
For some unexplained reason, some of the functions registered in the Epilogues would sometimes be executed several times. I could not figure out why.
This diff fixes that, but also has more explainable benefits:
- Do not run epilogues registered in the parent in the children. Previously it
would do so, but probably only if the children registered some epilogue given
that `at_exit` must be called again once on the child (but the value of the ref
in `Pervasives` would not have been reset).
- Unified behaviour for early and late epilogues given that we now handle both of these directly
We already have all the control needed to run epilogues when needed: we know
when infer exits, and we know when children processes exit.
Reviewed By: mbouaziz
Differential Revision: D9752046
fbshipit-source-id: 13af40081
Summary: This integration rolls out its own parallelisation using `Parmap` and its own error handling. Make it use `Tasks` instead.
Reviewed By: mbouaziz
Differential Revision: D8807109
fbshipit-source-id: ac09b3791
Summary: Otherwise the dead code checker sometimes crashes with a not-totally-related error.
Reviewed By: mbouaziz
Differential Revision: D8732546
fbshipit-source-id: 65caabd
Summary: It's been deprecated for a while now. Use `infer analyze` instead.
Reviewed By: dulmarod
Differential Revision: D8442682
fbshipit-source-id: 0b8c590
Summary:
Forcing integration with `--force-integration` would only work for some
integrations that stayed in OCaml-land. This propagetes the forcing to
infer.py.
Also remove some unused options on the Python side, and add more debug
information.
Fixes#927
Reviewed By: mbouaziz
Differential Revision: D8235504
fbshipit-source-id: 1d98543
Summary:
Change the license of the source code from BSD + PATENTS to MIT.
Change `checkCopyright` to reflect the new license and learn some new file
types.
Generated with:
```
git grep BSD | xargs -n 1 ./scripts/checkCopyright -i
```
Reviewed By: jeremydubreil, mbouaziz, jberdine
Differential Revision: D8071249
fbshipit-source-id: 97ca23a
Summary:
The deadcode checker is poorly written and as a result is more useful than
intended!
Reviewed By: mbouaziz
Differential Revision: D8088654
fbshipit-source-id: 19a94b8
Summary: It seems that we don't need to rely on `make` anymore as the internal process manager is working fine on Buck Java projects.
Reviewed By: jvillard
Differential Revision: D7903639
fbshipit-source-id: 9b32f05
Summary:
This is an attempt to make things more consistent, and maybe save some work
from the `Format` module in case flambda doesn't have our backs.
Reviewed By: jberdine
Differential Revision: D7775496
fbshipit-source-id: 59a6314
Summary: Make errors stand out with colours. Also improve error messages around save states.
Reviewed By: mbouaziz
Differential Revision: D7928794
fbshipit-source-id: c81cfe2
Summary:
Add warning 60 (unused module) to the list of fatal warnings. Whitelisting
modules at toplevel is tricky (see inline comments) but doable.
Reviewed By: mbouaziz
Differential Revision: D7790073
fbshipit-source-id: 6f591c4
Summary: It makes sense to run the same prelude and epilogue for `infer analyze` as for `infer run`.
Reviewed By: da319
Differential Revision: D7639363
fbshipit-source-id: 26170c1
Summary:
Upgrade ocamlformat, and base which needs to be done in sync in order to build
ocamlformat, and the other deps can come for the ride.
Reviewed By: jvillard
Differential Revision: D7663537
fbshipit-source-id: 3e90970
Summary:
It's already turned of systematically for the Java integration, this just
generalises it. The Buck daemon seems to cause issues with infer from time to
time that are hard to debug.
Reviewed By: jeremydubreil
Differential Revision: D7400068
fbshipit-source-id: f05ee07
Summary:
Limit the scope of what gets included into IStd.ml to only values that we want
to shadow. New values go into other files.
Also, build istd/ with `Core` open.
Reviewed By: mbouaziz
Differential Revision: D7382111
fbshipit-source-id: 969f0e8
Summary:
- Time only stats are now reported from the driver for
- TotalFrontend
- TotalBackend
Reviewed By: dulmarod
Differential Revision: D7382161
fbshipit-source-id: 6e34ca2
Summary:
- PerfStats can now directly compute the filename / relative path of the stats to be logged
- There is no longer a need to pass in the filename as a parameter, simplifying the API to just be a one-liner
Reviewed By: dulmarod
Differential Revision: D7381886
fbshipit-source-id: e6623c3
Summary:
Infer reads the arguments passed to clang, in particular to filter out some
incompatible clang command-line options. But, infer only understands arguments
in arg files if they are presented one per line. That's usually the case, but
infer itself stores clang arguments from compilation databases all on one line.
This breaks filtering.
This changes the type of compilation database items to a list of arguments, so
that they can be stored one per line when possible.
Also does some cleanup/renamings, and remove trailing `_` from temp file names, eg:
clang_command_.tmp.2b2602.txt -> clang_command.tmp.2b2602.txt
Also, do not escape arguments from arg files when printing them in logs (that
was useless and ugly).
Reviewed By: mbouaziz
Differential Revision: D7365907
fbshipit-source-id: 5a3fe70
Summary:
There's no real reason not to use `Core` lists in this module. Changed the
interface to be more `Core`-like. Changed the `*_changed` functions to use a
ref to track changes instead of passing the changed state around.
Reviewed By: mbouaziz
Differential Revision: D7123211
fbshipit-source-id: b27791a
Summary:
- Rather than passing a directory name to PerfStats reporting api to write the stats to, and then determining the EventLogger `stats_type` from that dirname, there is now a `stats_type` type in PerfStats, and the directory name and EventLogger tag is automatically determined from that type after it is passed to the API
- This allows us to determine whether we're in capture mode or linters mode dynamically from PerfStats itself, and use the appropriate tag when creating an event
Reviewed By: dulmarod
Differential Revision: D7251580
fbshipit-source-id: 0dec071
Summary: Backend perf stats were previously being logged as both `driver_stats` and `backend_stats`. Now they are correctly logged only as `backend_stats`, and a single `driver_stats` exists for the top-level driver process.
Reviewed By: dulmarod
Differential Revision: D7195879
fbshipit-source-id: 0f2ddc0
Summary:
- PerformanceStats rows will be logged to EventLogger regardless of whether Infer is in developer mode
- PerformanceStats files will still not be created unless Infer is in developer mode
Reviewed By: dulmarod
Differential Revision: D7169403
fbshipit-source-id: 85bd7de
Summary:
- New Event type in EventLogger: PerformanceStats
- Contains some of the information logged to disk by PerfStats, as well as a `stats_type` tag to indicate which type of process the stats are from
- PerfStats.register_report_at_exit now takes 3 string arguments and constructs the file path and determines stats_type based off of them
- Changed all clients of PerfStats to use new api
Reviewed By: dulmarod
Differential Revision: D7131904
fbshipit-source-id: 9226b5d
Summary:
Useful to debug without having to rerun commands.
Looks like this:
```
Usage Error: Failed to execute compilation command:
'/home/jul/infer.fb/infer/bin/../../facebook-clang-plugins/clang/install/bin/clang'
'args.txt'
++Contents of 'args.txt':
'arg1.txt' 'arghorashy_hello_c.txt' 'args.txt'
++Contents of 'arg1.txt':
'arghorashy_c.txt'
++Contents of 'arg_c.txt':
'-c'
++Contents of 'arg_hello_c.txt':
'examples/hello.c'
Error message:
clang-5.0: error: no such file or directory: 'examples/hello.c'
*** Infer needs a working compilation command to run.
```
Reviewed By: mbouaziz
Differential Revision: D6872550
fbshipit-source-id: 65cd026
Summary:
Record "capture phases" in the runstate and in the source files table of the
database. Use this instead of filesystem timestamps to decide which files need
re-analyzing in the reactive analysis.
Reviewed By: jeremydubreil
Differential Revision: D6760833
fbshipit-source-id: 7955621
Summary:
The custom exception does not appear to serve any purpose. Also refactor the
complaining code to avoid duplication.
Reviewed By: mbouaziz
Differential Revision: D6784945
fbshipit-source-id: a2d969b
Jules Villard
Josh Berdine
Nikos Gorogiannis
Dulma Churchill
Martin Trojer
Phoebe Nichols
Artem Pianykh
Mitya Lyubarskiy
David Lively
Ezgi Çiçek
Nikos Gorogiannis
Mehdi Bouaziz
Vincent Siles
dWiGhT
Andrey Epin
Martino Luca
Katie Ots
Dino Distefano
Jeremy Dubreil
Varun Arora