In order to use the same logger from the notebook app we
need to pass it through to the log_request function. To
do that we need to use a partial function so the `log` kwarg
is available. If not provided the default logger is the same
as before, the tornado web access logger.
This is necessary for the `log_json` settings to be used for
both the notebook app logs and the web app request logging.
There is still a todo needed here so that the individual
request fields get formatted separately rather than dumped
into the `msg` field.
This adds the `NotebookApp.log_json` CLI option which defaults to
False. The default value comes from the `JUPYTER_ENABLE_JSON_LOGGING`
environment variable. A validation hook is added such that if json
logging is enabled but the `json-logging` package is not installed,
an error is logged and the effective `log_json` value is False.
If enabled and the `json-logging` package is installed, the `init_logging`
method will initialize the non-web [1] json logging formatter.
[1] https://github.com/bobbui/json-logging-python#21-non-web-application-log
This gets us closer to logging parts of the request
as separate json fields but not all the way there.
For example:
```json
{"written_at": "2020-10-07T21:52:16.975Z", "written_ts": 1602107536975497000, "msg": "{'status': 404, 'method': 'GET', 'ip': '127.0.0.1', 'uri': '/nbextensions/widgets/notebook/js/extension.js', 'request_time': 1.9807815551757812, 'referer': 'http://localhost:8888/notebooks/test.ipynb'}", "type": "log", "logger": "NotebookApp", "thread": "MainThread", "level": "WARNING", "module": "log", "line_no": 54}
```
This adds a new extra package dependency on `json-logging`
and an environment variable, which when set to "true" regardless
of case, will try to use the json-logging non-web app log formatter.
If the `json-logging` package is not installed but the environment
variable is True, something like this will be logged but it will not
crash the application:
```
$ ENABLE_JSON_LOGGING=true jupyter notebook
Unable to use json logging due to missing packages. Run "pip install notebook[json-logging]" to fix.
Traceback (most recent call last):
File "/home/osboxes/jupyter/notebook/notebook/notebookapp.py", line 144, in <module>
import json_logging
ModuleNotFoundError: No module named 'json_logging'
```
Initially I tried to add a new Bool config option to toggle this but
the problem is (from my limited understanding of traitlets and tornado)
is that `_log_formatter_cls` needs to be set early and trying to se the
log formatter later in `init_logging` is too late - or at least I couldn't
figure out a way to reset the log formatter dynamically (I tried calling
`_log_format_changed` [1] but that didn't work).
With this working you get logs like this:
```
{"written_at": "2020-10-07T21:10:51.606Z", "written_ts": 1602105051606265000, "msg": "404 GET /nbextensions/widgets/notebook/js/extension.js (127.0.0.1) 9.26ms referer=http://localhost:8888/notebooks/Untitled.ipynb", "type": "log", "logger": "NotebookApp", "thread": "MainThread", "level": "WARNING", "module": "log", "line_no": 49}
{"written_at": "2020-10-07T21:10:54.443Z", "written_ts": 1602105054443309000, "msg": "Starting buffering for f260ddd7-938c-42d0-ac3b-455bea76694f:49f30b53fc4b4ec6a8f2fb748a171613", "type": "log", "logger": "NotebookApp", "thread": "MainThread", "level": "INFO", "module": "kernelmanager", "line_no": 222}
{"written_at": "2020-10-07T21:10:54.446Z", "written_ts": 1602105054446264000, "msg": "Kernel shutdown: f260ddd7-938c-42d0-ac3b-455bea76694f", "type": "log", "logger": "NotebookApp", "thread": "MainThread", "level": "INFO", "module": "multikernelmanager", "line_no": 201}
```
An obvious improvement here would to be able to split the `msg` fields apart
so that we can log things like response status code, request method, request
URL, response_time_ms etc. That should be possible with the `JSONLogWebFormatter`
from `json-logging` but when I tried using that I was getting errors from
the library about a correlation id (which is based on a request header we
don't use). The `json-logging` library supports several web frameworks like
Flask but unfortunately does not have built in support for Tornado, but it does
support custom formatters so that might be a follow up option to improve on this.
[1] https://github.com/ipython/traitlets/blob/4.3.3/traitlets/config/application.py#L195Closes#5798
* Workaround for wrong errno on socket bind permission errors on Cygwin.
* [ci skip] Add period to comment
Co-authored-by: Steven Silvester <stslve@amazon.com>
* Restore detection of missing terminado package
* Properly handle terminals_enabled config
* Disambiguate terminado availability from web settings
* Further clarity on terminal availability
* Rename terminals_in_use back to terminals_available
* added `--autoreload` flag
When passed, the webapp will watch for any changes to its Python source. On
change, all changed packages will be reimported and the webapp will restart.
Also works on Python source files in Jupyter server extensions. Implemented
using the built in `autoreload` parameter in the constructor of
`tornado.web.Application`.
* updated .gitignore
* Add env variable support for port options
In order to better support use cases relating to containerized environments,
this change adds environment variable support for the `--port` option. Since
anyone setting a specific port probably doesn't want port retry logic enabled,
the `--port-retries` option has also been backed by an env.
Option `--port` will be backed by env `JUPYTER_PORT` and still defaults to `8888`.
Option `--port-retries` will be backed by env `JUPYTER_PORT_RETRIES` and still
defaults to `50`.
The CLI options will override those set via the envrionment, but environment
values override those set via configuration files.
Closes#5212
* Fixup after merge
This adds functionality to track a terminal's last activity and
optionally cull terminals that have been inactive for some specified
duration.
Resolves: #3868
This commit uses the approach used in jupyter_server #191 first proposed
by David Brochart. This reduces code duplication and alleviates redundancy
relative to configurable options.
Also, the startup message now includes the version information.
Co-authored-by: David Brochart <david.brochart@gmail.com>
Matt Riedemann
Kevin Bates
Felix Mönckemeyer
David Lukes
Ram Rachum
Laura Merlo
E. M. Bray
Max Klein
Zachary Sailer
Steven Silvester
Kris Wilson
Rohit Sanjay
Bruno P. Kinoshita
Pallavi Bharadwaj