Merge branch 'master' into master

Thomas Kluyver 6 years ago committed by GitHub
commit fd4275cdf6
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23

@ -80,9 +80,9 @@ If you do not see that your Jupyter Notebook is not running on dev mode, it's po
running other instances of Jupyter Notebook. You can try the following steps:
1. Uninstall all instances of the notebook package. These include any installations you made using
pip or conda
2. Run ``python3 -m pip install -e .`` in the notebook repository to install the notebook from there
3. Run ``npm run build`` to make sure the Javascript and CSS are updated and compiled
pip or conda.
2. Run ``python3 -m pip install -e .`` in the notebook repository to install the notebook from there.
3. Run ``npm run build`` to make sure the Javascript and CSS are updated and compiled.
4. Launch with ``python3 -m notebook --port 8989``, and check that the browser is pointing to ``localhost:8989``
(rather than the default 8888). You don't necessarily have to launch with port 8989, as long as you use
a port that is neither the default nor in use, then it should be fine.
@ -98,16 +98,16 @@ this command whenever there are changes to JavaScript or LESS sources::
npm run build
**IMPORTANT:** Don't forget to run ``npm run build`` after switching branches.
When switching between branches of different versions (e.g. ``4.x`` and
``master``), run ``pip install -e .``. If you have tried the above and still
**IMPORTANT:** Don't forget to run ``npm run build`` after switching branches.
When switching between branches of different versions (e.g. ``4.x`` and
``master``), run ``pip install -e .``. If you have tried the above and still
find that the notebook is not reflecting the current source code, try cleaning
the repo with ``git clean -xfd`` and reinstalling with ``pip install -e .``.
Development Tip
"""""""""""""""
When doing development, you can use this command to automatically rebuild
When doing development, you can use this command to automatically rebuild
JavaScript and LESS sources as they are modified::
npm run build:watch
@ -116,7 +116,7 @@ Git Hooks
"""""""""
If you want to automatically update dependencies and recompile JavaScript and
CSS after checking out a new commit, you can install post-checkout and
CSS after checking out a new commit, you can install post-checkout and
post-merge hooks which will do it for you::
git-hooks/install-hooks.sh
@ -132,7 +132,7 @@ Python Tests
Install dependencies::
pip install -e .[test]
pip install -e '.[test]'
To run the Python tests, use::

@ -1,5 +1,3 @@
# Licensing terms
This project is licensed under the terms of the Modified BSD License
(also known as New or Revised or 3-Clause BSD), as follows:

@ -1,4 +1,4 @@
include COPYING.md
include LICENSE
include CONTRIBUTING.rst
include README.md
include package.json

@ -5,13 +5,13 @@
"backbone": "components/backbone#~1.2",
"bootstrap": "bootstrap#~3.4",
"bootstrap-tour": "0.9.0",
"codemirror": "components/codemirror#~5.37",
"codemirror": "components/codemirror#~5.48.4",
"create-react-class": "https://cdn.jsdelivr.net/npm/create-react-class@15.6.3/create-react-class.min.js",
"es6-promise": "~1.0",
"font-awesome": "components/font-awesome#~4.7.0",
"google-caja": "5669",
"jed": "~1.1.1",
"jquery": "components/jquery#~3.3",
"jquery": "components/jquery#~3.4.1",
"jquery-typeahead": "~2.0.0",
"jquery-ui": "components/jqueryui#~1.12",
"marked": "~0.5",

@ -13,8 +13,8 @@ Jupyter notebook 은 상호 교환을 위한 웹 기반 환경입니다.
### Jupyter notebook, 사용자의 언어에 독립적인 IPython notebook의 진화
Jupyter notebook은 Jupyter 프로젝트를 위한 사용자 언어에 독립적인 HTML 응용 프로그램입니다.
2015년에 Jupyter notebook은 IPython 코드 기반의 The Big Split™ 의 일부분으로 시작되었습니다.
IPython 3는 *IPython notebook* 과 같은 사용자 언어에 독립적인 코드와 *IPython kernel for Python* 과 같은 특정언어 기반의 코드의 기능을 가지고 출시되었습니다.
컴퓨터에는 많은 언어가 사용되기 때문에, Jupyter 프로젝트는 사용자 언어에 독립적인 **Jupyter notebook** 을 이 저장소와 개인의 독립적인 저장소에 있는 특정언어 중심의 커널의 도움으로 지속적으로 개발할 것입니다.
IPython 3는 *IPython notebook* 과 같은 사용자 언어에 독립적인 코드와 *IPython kernel for Python* 과 같은 특정 언어 기반의 코드의 기능을 가지고 출시되었습니다.
컴퓨터에는 많은 언어가 사용되기 때문에, Jupyter 프로젝트는 사용자 언어에 독립적인 **Jupyter notebook** 을 이 저장소와 개인의 독립적인 저장소에 있는 특정 언어 중심의 커널의 도움으로 지속적으로 개발할 것입니다.
[[The Big Split™ announcement](https://blog.jupyter.org/2015/04/15/the-big-split/)]
[[Jupyter Ascending blog post](http://blog.jupyter.org/2015/08/12/first-release-of-jupyter/)]

@ -21,6 +21,44 @@ We strongly recommend that you upgrade pip to version 9+ of pip before upgrading
Use ``pip install pip --upgrade`` to upgrade pip. Check pip version with
``pip --version``.
.. _release-6.0.2:
6.0.2
-----
- Update JQuery dependency to version 3.4.1 to fix security vulnerability (CVE-2019-11358)
- Update CodeMirror to version 5.48.4 to fix Python formatting issues
- Continue removing obsolete Python 2.x code/dependencies
- Multiple documentation updates
Thanks for all the contributors:
- David Robles
- Jason Grout
- Kerwin Sun
- Kevin Bates
- Kyle Kelley
- Luciano Resende
- Marcus D Sherman
- Sasaki Takeru
- Tom Jarosz
- Vidar Tonaas Fauske
- Wes Turner
- Zachary Sailer
.. _release-6.0.1:
6.0.1
-----
- Attempt to re-establish websocket connection to Gateway (:ghpull:`4777`)
- Add missing react-dom js to package data (:ghpull:`4772`)
Thanks for all the contributors:
- Eunsoo Park
- Min RK
.. _release-6.0:
6.0

@ -4,7 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"The Markdown parser included in the Jupyter Notebook is MathJax-aware. This means that you can freely mix in mathematical expressions using the [MathJax subset of Tex and LaTeX](http://docs.mathjax.org/en/latest/tex.html#tex-support). [Some examples from the MathJax site](https://www.mathjax.org/demos/tex-samples/) are reproduced below, as well as the Markdown+TeX source."
"The Markdown parser included in the Jupyter Notebook is MathJax-aware. This means that you can freely mix in mathematical expressions using the [MathJax subset of Tex and LaTeX](https://docs.mathjax.org/en/latest/input/tex/). [Some examples from the MathJax demos site](https://mathjax.github.io/MathJax-demos-web/) are reproduced below, as well as the Markdown+TeX source."
]
},
{
@ -272,9 +272,9 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.5.1"
"version": "3.7.3"
}
},
"nbformat": 4,
"nbformat_minor": 0
"nbformat_minor": 1
}

@ -155,7 +155,7 @@
"\n",
"or other languages:\n",
"\n",
" if (i=0; i<n; i++) {\n",
" for (i=0; i<n; i++) {\n",
" printf(\"hello %d\\n\", i);\n",
" x += 4;\n",
" }"

@ -79,8 +79,11 @@ model. There are three model types: **notebook**, **file**, and **directory**.
- ``file`` models
- The ``format`` field is either ``"text"`` or ``"base64"``.
- The ``mimetype`` field is ``text/plain`` for text-format models and
``application/octet-stream`` for base64-format models.
- The ``mimetype`` field can be any mimetype string, but defaults to
``text/plain`` for text-format models and
``application/octet-stream`` for base64-format models. For files with
unknown mime types (e.g. unknown file extensions), this field may be
`None`.
- The ``content`` field is always of type ``unicode``. For text-format
file models, ``content`` simply contains the file's bytes after decoding
as UTF-8. Non-text (``base64``) files are read as bytes, base64 encoded,
@ -97,11 +100,13 @@ model. There are three model types: **notebook**, **file**, and **directory**.
.. _contentfree:
In certain circumstances, we don't need the full content of an entity to
complete a Contents API request. In such cases, we omit the ``mimetype``,
``content``, and ``format`` keys from the model. This most commonly occurs
when listing a directory, in which circumstance we represent files within
the directory as content-less models to avoid having to recursively traverse
and serialize the entire filesystem.
complete a Contents API request. In such cases, we omit the ``content``, and
``format`` keys from the model. The default values for the ``mimetype``
field will might also not be evaluated, in which case it will be set as `None`.
This reduced reply most commonly occurs when listing a directory, in
which circumstance we represent files within the directory as content-less
models to avoid having to recursively traverse and serialize the entire
filesystem.
**Sample Models**

@ -6,10 +6,8 @@
import re
import os
try:
from urllib.parse import urlparse # Py 3
except ImportError:
from urlparse import urlparse # Py 2
from urllib.parse import urlparse
import uuid
from tornado.escape import url_escape

@ -120,6 +120,8 @@ def persist_config(config_file=None, mode=0o600):
if config_file is None:
config_file = os.path.join(jupyter_config_dir(), 'jupyter_notebook_config.json')
os.makedirs(os.path.dirname(config_file), exist_ok=True)
loader = JSONFileConfigLoader(os.path.basename(config_file), os.path.dirname(config_file))
try:
config = loader.load_config()

@ -14,18 +14,10 @@ import sys
import traceback
import types
import warnings
try:
# py3
from http.client import responses
from http.cookies import Morsel
except ImportError:
from httplib import responses
from Cookie import Morsel
try:
from urllib.parse import urlparse # Py 3
except ImportError:
from urlparse import urlparse # Py 2
from http.client import responses
from http.cookies import Morsel
from urllib.parse import urlparse
from jinja2 import TemplateNotFound
from tornado import web, gen, escape, httputil
from tornado.log import app_log
@ -35,7 +27,7 @@ from notebook._sysinfo import get_sys_info
from traitlets.config import Application
from ipython_genutils.path import filefind
from ipython_genutils.py3compat import string_types, PY3
from ipython_genutils.py3compat import string_types
import notebook
from notebook._tz import utcnow
@ -479,10 +471,6 @@ class IPythonHandler(AuthenticatedHandler):
if host.startswith('[') and host.endswith(']'):
host = host[1:-1]
if not PY3:
# ip_address only accepts unicode on Python 2
host = host.decode('utf8', 'replace')
try:
addr = ipaddress.ip_address(host)
except ValueError:

@ -12,10 +12,8 @@ from nbformat.v4 import (
new_notebook, new_markdown_cell, new_code_cell, new_output,
)
try:
from unittest.mock import patch
except ImportError:
from mock import patch # py3
from unittest.mock import patch
def bundle(handler, model):
"""Bundler test stub. Echo the notebook path."""

@ -7,11 +7,7 @@ import os
import shutil
import unittest
try:
from unittest.mock import patch
except ImportError:
from mock import patch # py2
from unittest.mock import patch
from ipython_genutils.tempdir import TemporaryDirectory
from ipython_genutils import py3compat

@ -11,7 +11,6 @@ import json
import os
import copy
from six import PY3
from traitlets.config import LoggingConfigurable
from traitlets.traitlets import Unicode, Bool
@ -119,10 +118,7 @@ class BaseJSONConfigManager(LoggingConfigurable):
# in order to avoid writing half-finished corrupted data to disk.
json_content = json.dumps(data, indent=2)
if PY3:
f = io.open(filename, 'w', encoding='utf-8')
else:
f = open(filename, 'wb')
f = io.open(filename, 'w', encoding='utf-8')
with f:
f.write(json_content)

@ -7,7 +7,7 @@ import mimetypes
import json
from base64 import decodebytes
from tornado import web
from tornado import gen, web
from notebook.base.handlers import IPythonHandler
from notebook.utils import maybe_future
@ -35,6 +35,7 @@ class FilesHandler(IPythonHandler):
return self.get(path, include_body=False)
@web.authenticated
@gen.coroutine
def get(self, path, include_body=True):
# /files/ requests must originate from the same site
self.check_xsrf_cookie()

@ -130,10 +130,12 @@ class GatewayWebSocketClient(LoggingConfigurable):
self.kernel_id = None
self.ws = None
self.ws_future = Future()
self.ws_future_cancelled = False
self.disconnected = False
@gen.coroutine
def _connect(self, kernel_id):
# websocket is initialized before connection
self.ws = None
self.kernel_id = kernel_id
ws_url = url_path_join(
GatewayClient.instance().ws_url,
@ -148,40 +150,48 @@ class GatewayWebSocketClient(LoggingConfigurable):
self.ws_future.add_done_callback(self._connection_done)
def _connection_done(self, fut):
if not self.ws_future_cancelled: # prevent concurrent.futures._base.CancelledError
if not self.disconnected and fut.exception() is None: # prevent concurrent.futures._base.CancelledError
self.ws = fut.result()
self.log.debug("Connection is ready: ws: {}".format(self.ws))
else:
self.log.warning("Websocket connection has been cancelled via client disconnect before its establishment. "
self.log.warning("Websocket connection has been closed via client disconnect or due to error. "
"Kernel with ID '{}' may not be terminated on GatewayClient: {}".
format(self.kernel_id, GatewayClient.instance().url))
def _disconnect(self):
self.disconnected = True
if self.ws is not None:
# Close connection
self.ws.close()
elif not self.ws_future.done():
# Cancel pending connection. Since future.cancel() is a noop on tornado, we'll track cancellation locally
self.ws_future.cancel()
self.ws_future_cancelled = True
self.log.debug("_disconnect: ws_future_cancelled: {}".format(self.ws_future_cancelled))
self.log.debug("_disconnect: future cancelled, disconnected: {}".format(self.disconnected))
@gen.coroutine
def _read_messages(self, callback):
"""Read messages from gateway server."""
while True:
while self.ws is not None:
message = None
if not self.ws_future_cancelled:
if not self.disconnected:
try:
message = yield self.ws.read_message()
except Exception as e:
self.log.error("Exception reading message from websocket: {}".format(e)) # , exc_info=True)
if message is None:
if not self.disconnected:
self.log.warning("Lost connection to Gateway: {}".format(self.kernel_id))
break
callback(message) # pass back to notebook client (see self.on_open and WebSocketChannelsHandler.open)
else: # ws cancelled - stop reading
break
if not self.disconnected: # if websocket is not disconnected by client, attept to reconnect to Gateway
self.log.info("Attempting to re-establish the connection to Gateway: {}".format(self.kernel_id))
self._connect(self.kernel_id)
loop = IOLoop.current()
loop.add_future(self.ws_future, lambda future: self._read_messages(callback))
def on_open(self, kernel_id, message_callback, **kwargs):
"""Web socket connection open against gateway server."""
self._connect(kernel_id)
@ -205,7 +215,7 @@ class GatewayWebSocketClient(LoggingConfigurable):
def _write_message(self, message):
"""Send message to gateway server."""
try:
if not self.ws_future_cancelled:
if not self.disconnected and self.ws is not None:
self.ws.write_message(message)
except Exception as e:
self.log.error("Exception writing message to websocket: {}".format(e)) # , exc_info=True)

@ -283,12 +283,12 @@ def gateway_request(endpoint, **kwargs):
except ConnectionRefusedError:
raise web.HTTPError(503, "Connection refused from Gateway server url '{}'. "
"Check to be sure the Gateway instance is running.".format(GatewayClient.instance().url))
except HTTPError:
except HTTPError as e:
# This can occur if the host is valid (e.g., foo.com) but there's nothing there.
raise web.HTTPError(504, "Error attempting to connect to Gateway server url '{}'. "
raise web.HTTPError(e.code, "Error attempting to connect to Gateway server url '{}'. "
"Ensure gateway url is valid and the Gateway instance is running.".
format(GatewayClient.instance().url))
except gaierror as e:
except gaierror:
raise web.HTTPError(404, "The Gateway server specified in the gateway_url '{}' doesn't appear to be valid. "
"Ensure gateway url is valid and the Gateway instance is running.".
format(GatewayClient.instance().url))
@ -390,8 +390,8 @@ class GatewayKernelManager(MappingKernelManager):
self.log.debug("Request kernel at: %s" % kernel_url)
try:
response = yield gateway_request(kernel_url, method='GET')
except HTTPError as error:
if error.code == 404:
except web.HTTPError as error:
if error.status_code == 404:
self.log.warn("Kernel not found at: %s" % kernel_url)
self.remove_kernel(kernel_id)
kernel = None
@ -559,8 +559,8 @@ class GatewayKernelSpecManager(KernelSpecManager):
self.log.debug("Request kernel spec at: %s" % kernel_spec_url)
try:
response = yield gateway_request(kernel_spec_url, method='GET')
except HTTPError as error:
if error.code == 404:
except web.HTTPError as error:
if error.status_code == 404:
# Convert not found to KeyError since that's what the Notebook handler expects
# message is not used, but might as well make it useful for troubleshooting
raise KeyError('kernelspec {kernel_name} not found on Gateway server at: {gateway_url}'.
@ -587,8 +587,8 @@ class GatewayKernelSpecManager(KernelSpecManager):
self.log.debug("Request kernel spec resource '{}' at: {}".format(path, kernel_spec_resource_url))
try:
response = yield gateway_request(kernel_spec_resource_url, method='GET')
except HTTPError as error:
if error.code == 404:
except web.HTTPError as error:
if error.status_code == 404:
kernel_spec_resource = None
else:
raise

@ -23,32 +23,16 @@ import time
from io import BytesIO
from threading import Thread, Lock, Event
try:
from unittest.mock import patch
except ImportError:
from mock import patch # py3
from unittest.mock import patch
from jupyter_core.paths import jupyter_runtime_dir
from ipython_genutils.py3compat import bytes_to_str, which
from notebook._sysinfo import get_sys_info
from ipython_genutils.tempdir import TemporaryDirectory
try:
# Python >= 3.3
from subprocess import TimeoutExpired
def popen_wait(p, timeout):
return p.wait(timeout)
except ImportError:
class TimeoutExpired(Exception):
pass
def popen_wait(p, timeout):
"""backport of Popen.wait from Python 3"""
for i in range(int(10 * timeout)):
if p.poll() is not None:
return
time.sleep(0.1)
if p.poll() is None:
raise TimeoutExpired
from subprocess import TimeoutExpired
def popen_wait(p, timeout):
return p.wait(timeout)
NOTEBOOK_SHUTDOWN_TIMEOUT = 10

@ -2,7 +2,7 @@
# Copyright (c) Jupyter Development Team
#
# Distributed under the terms of the BSD License. The full license is in
# the file COPYING, distributed as part of this software.
# the file LICENSE, distributed as part of this software.
#-----------------------------------------------------------------------------
import json

@ -7,13 +7,14 @@ import io
import os
import zipfile
from tornado import web, escape
from tornado import gen, web, escape
from tornado.log import app_log
from ..base.handlers import (
IPythonHandler, FilesRedirectHandler,
path_regex,
)
from ..utils import maybe_future
from nbformat import from_dict
from ipython_genutils.py3compat import cast_bytes
@ -86,6 +87,7 @@ class NbconvertFileHandler(IPythonHandler):
"; sandbox allow-scripts"
@web.authenticated
@gen.coroutine
def get(self, format, path):
exporter = get_exporter(format, config=self.config, log=self.log)
@ -99,7 +101,7 @@ class NbconvertFileHandler(IPythonHandler):
else:
ext_resources_dir = None
model = self.contents_manager.get(path=path)
model = yield maybe_future(self.contents_manager.get(path=path))
name = model['name']
if model['type'] != 'notebook':
# not a notebook, redirect to files

@ -16,12 +16,7 @@ from nbformat.v4 import (
from ipython_genutils.testing.decorators import onlyif_cmds_exist
try: #PY3
from base64 import encodebytes
except ImportError: #PY2
from base64 import encodestring as encodebytes
from base64 import encodebytes
class NbconvertAPI(object):

@ -13,13 +13,8 @@ import tarfile
import zipfile
from os.path import basename, join as pjoin, normpath
try:
from urllib.parse import urlparse # Py3
from urllib.request import urlretrieve
except ImportError:
from urlparse import urlparse
from urllib import urlretrieve
from urllib.parse import urlparse
from urllib.request import urlretrieve
from jupyter_core.paths import (
jupyter_data_dir, jupyter_config_path, jupyter_path,
SYSTEM_JUPYTER_PATH, ENV_JUPYTER_PATH,

@ -5,13 +5,17 @@
from collections import namedtuple
import os
from tornado import web
from tornado import (
gen, web,
)
HTTPError = web.HTTPError
from ..base.handlers import (
IPythonHandler, FilesRedirectHandler, path_regex,
)
from ..utils import url_escape
from ..utils import (
maybe_future, url_escape,
)
from ..transutils import _
@ -68,6 +72,7 @@ def get_frontend_exporters():
class NotebookHandler(IPythonHandler):
@web.authenticated
@gen.coroutine
def get(self, path):
"""get renders the notebook template if a name is given, or
redirects to the '/files/' handler if the name is not given."""
@ -76,7 +81,7 @@ class NotebookHandler(IPythonHandler):
# will raise 404 on not found
try:
model = cm.get(path, content=False)
model = yield maybe_future(cm.get(path, content=False))
except web.HTTPError as e:
if e.status_code == 404 and 'files' in path.split('/'):
# 404, but '/files/' in URL, let FilesRedirect take care of it

@ -32,11 +32,13 @@ import time
import warnings
import webbrowser
try: #PY3
from base64 import encodebytes
except ImportError: #PY2
from base64 import encodestring as encodebytes
try:
import resource
except ImportError:
# Windows
resource = None
from base64 import encodebytes
from jinja2 import Environment, FileSystemLoader
@ -70,11 +72,6 @@ from notebook import (
__version__,
)
# py23 compatibility
try:
raw_input = raw_input
except NameError:
raw_input = input
from .base.handlers import Template404, RedirectWithParams
from .log import log_request
@ -648,6 +645,24 @@ class NotebookApp(JupyterApp):
help=_("Whether to allow the user to run the notebook as root.")
)
use_redirect_file = Bool(True, config=True,
help="""Disable launching browser by redirect file
For versions of notebook > 5.7.2, a security feature measure was added that
prevented the authentication token used to launch the browser from being visible.
This feature makes it difficult for other users on a multi-user system from
running code in your Jupyter session as you.
However, some environments (like Windows Subsystem for Linux (WSL) and Chromebooks),
launching a browser using a redirect file can lead the browser failing to load.
This is because of the difference in file structures/paths between the runtime and
the browser.
Disabling this setting to False will disable this behavior, allowing the browser
to launch by using a URL and visible token (as before).
"""
)
default_url = Unicode('/tree', config=True,
help=_("The default URL to redirect to from `/`")
)
@ -802,6 +817,30 @@ class NotebookApp(JupyterApp):
"""
)
min_open_files_limit = Integer(config=True,
help="""
Gets or sets a lower bound on the open file handles process resource
limit. This may need to be increased if you run into an
OSError: [Errno 24] Too many open files.
This is not applicable when running on Windows.
""")
@default('min_open_files_limit')
def _default_min_open_files_limit(self):
if resource is None:
# Ignoring min_open_files_limit because the limit cannot be adjusted (for example, on Windows)
return None
soft, hard = resource.getrlimit(resource.RLIMIT_NOFILE)
DEFAULT_SOFT = 4096
if hard >= DEFAULT_SOFT:
return DEFAULT_SOFT
self.log.debug("Default value for min_open_files_limit is ignored (hard=%r, soft=%r)", hard, soft)
return soft
@observe('token')
def _token_changed(self, change):
self._token_generated = False
@ -1247,14 +1286,6 @@ class NotebookApp(JupyterApp):
raise TraitError(trans.gettext("No such notebook dir: '%r'") % value)
return value
@observe('notebook_dir')
def _update_notebook_dir(self, change):
"""Do a bit of validation of the notebook dir."""
# setting App.notebook_dir implies setting notebook and kernel dirs as well
new = change['new']
self.config.FileContentsManager.root_dir = new
self.config.MappingKernelManager.root_dir = new
# TODO: Remove me in notebook 5.0
server_extensions = List(Unicode(), config=True,
help=(_("DEPRECATED use the nbserver_extensions dict instead"))
@ -1379,6 +1410,23 @@ class NotebookApp(JupyterApp):
logger.parent = self.log
logger.setLevel(self.log.level)
def init_resources(self):
"""initialize system resources"""
if resource is None:
self.log.debug('Ignoring min_open_files_limit because the limit cannot be adjusted (for example, on Windows)')
return
old_soft, old_hard = resource.getrlimit(resource.RLIMIT_NOFILE)
soft = self.min_open_files_limit
hard = old_hard
if old_soft < soft:
if hard < soft:
hard = soft
self.log.debug(
'Raising open file limit: soft {}->{}; hard {}->{}'.format(old_soft, soft, old_hard, hard)
)
resource.setrlimit(resource.RLIMIT_NOFILE, (soft, hard))
def init_webapp(self):
"""initialize tornado webapp and httpserver"""
self.tornado_settings['allow_origin'] = self.allow_origin
@ -1667,12 +1715,47 @@ class NotebookApp(JupyterApp):
pc = ioloop.PeriodicCallback(self.shutdown_no_activity, 60000)
pc.start()
def _init_asyncio_patch(self):
"""set default asyncio policy to be compatible with tornado
Tornado 6 (at least) is not compatible with the default
asyncio implementation on Windows
Pick the older SelectorEventLoopPolicy on Windows
if the known-incompatible default policy is in use.
do this as early as possible to make it a low priority and overrideable
ref: https://github.com/tornadoweb/tornado/issues/2608
FIXME: if/when tornado supports the defaults in asyncio,
remove and bump tornado requirement for py38
"""
if sys.platform.startswith("win") and sys.version_info >= (3, 8):
import asyncio
try:
from asyncio import (
WindowsProactorEventLoopPolicy,
WindowsSelectorEventLoopPolicy,
)
except ImportError:
pass
# not affected
else:
if type(asyncio.get_event_loop_policy()) is WindowsProactorEventLoopPolicy:
# WindowsProactorEventLoopPolicy is not compatible with tornado 6
# fallback to the pre-3.8 default of Selector
asyncio.set_event_loop_policy(WindowsSelectorEventLoopPolicy())
@catch_config_error
def initialize(self, argv=None):
self._init_asyncio_patch()
super(NotebookApp, self).initialize(argv)
self.init_logging()
if self._dispatching:
return
self.init_resources()
self.init_configurables()
self.init_server_extension_config()
self.init_components()
@ -1782,6 +1865,12 @@ class NotebookApp(JupyterApp):
if not browser:
return
if not self.use_redirect_file:
uri = self.default_url[len(self.base_url):]
if self.token:
uri = url_concat(uri, {'token': self.token})
if self.file_to_run:
if not os.path.exists(self.file_to_run):
self.log.critical(_("%s does not exist") % self.file_to_run)
@ -1797,9 +1886,12 @@ class NotebookApp(JupyterApp):
else:
open_file = self.browser_open_file
b = lambda: browser.open(
urljoin('file:', pathname2url(open_file)),
new=self.webbrowser_open_new)
if self.use_redirect_file:
assembled_url = urljoin('file:', pathname2url(open_file))
else:
assembled_url = url_path_join(self.connection_url, uri)
b = lambda: browser.open(assembled_url, new=self.webbrowser_open_new)
threading.Thread(target=b).start()
def start(self):

@ -8,7 +8,6 @@ import io
import errno
from tornado import web
from ipython_genutils.py3compat import PY3
from ...base.handlers import APIHandler
class ConfigHandler(APIHandler):

@ -24,10 +24,7 @@ from ipython_genutils.py3compat import str_to_unicode
from traitlets.config import Configurable
from traitlets import Bool
try: #PY3
from base64 import encodebytes, decodebytes
except ImportError: #PY2
from base64 import encodestring as encodebytes, decodestring as decodebytes
from base64 import encodebytes, decodebytes
def replace_file(src, dst):

@ -34,11 +34,7 @@ from notebook.utils import (
from notebook.base.handlers import AuthenticatedFileHandler
from notebook.transutils import _
try:
from os.path import samefile
except ImportError:
# windows + py2
from notebook.utils import samefile_simple as samefile
from os.path import samefile
_script_exporter = None

@ -200,7 +200,7 @@ class MappingKernelManager(MultiKernelManager):
Parameters
----------
kernel_id : str
The id of the kernel to stop buffering.
The id of the kernel to start buffering.
session_key: str
The session_key, if any, that should get the buffer.
If the session_key matches the current buffered session_key,

@ -85,7 +85,14 @@ define([
.addClass('btn btn-warning btn-xs')
.text(i18n._('Shutdown'))
.click(function() {
var path = $(this).parent().parent().parent().data('path');
var parent = $(this).parent().parent().parent();
var path = parent.data('path');
if(!path) {
path = parent.parent().data('path');
}
if(!path) {
throw new Error("Shutdown path not present");
}
that.shutdown_notebook(path);
})
.appendTo(running_indicator);

@ -255,8 +255,8 @@ data-notebook-path="{{notebook_path | urlencode}}"
</li>
</ul>
</li>
<li class="dropdown"><a href="#" class="dropdown-toggle" data-toggle="dropdown">{% trans %}Kernel{% endtrans %}</a>
<ul id="kernel_menu" class="dropdown-menu">
<li class="dropdown"><a href="#" class="dropdown-toggle" data-toggle="dropdown" id="kernellink">{% trans %}Kernel{% endtrans %}</a>
<ul id="kernel_menu" class="dropdown-menu" aria-labelledby="kernellink">
<li id="int_kernel"
title="{% trans %}Send Keyboard Interrupt (CTRL-C) to the Kernel{% endtrans %}">
<a href="#">{% trans %}Interrupt{% endtrans %}</a>

@ -13,10 +13,7 @@ from unittest import TestCase
pjoin = os.path.join
try:
from unittest.mock import patch
except ImportError:
from mock import patch #py2
from unittest.mock import patch
import requests
from tornado.ioloop import IOLoop
@ -143,9 +140,6 @@ class NotebookTestBase(TestCase):
started = Event()
def start_thread():
if 'asyncio' in sys.modules:
import asyncio
asyncio.set_event_loop(asyncio.new_event_loop())
try:
app = cls.notebook = NotebookApp(
port=cls.port,
@ -160,6 +154,10 @@ class NotebookTestBase(TestCase):
allow_root=True,
token=cls.token,
)
if 'asyncio' in sys.modules:
app._init_asyncio_patch()
import asyncio
asyncio.set_event_loop(asyncio.new_event_loop())
# don't register signal handler during tests
app.init_signal = lambda : None
# clear log handlers and propagate to root for nose to capture it

@ -1,115 +0,0 @@
//
// Test code cell execution.
//
casper.notebook_test(function () {
this.evaluate(function () {
var cell = IPython.notebook.get_cell(0);
cell.set_text('a=10; print(a)');
cell.execute();
});
this.wait_for_output(0);
// refactor this into just a get_output(0)
this.then(function () {
var result = this.get_output_cell(0);
this.test.assertEquals(result.text, '10\n', 'cell execute (using js)');
});
// do it again with the keyboard shortcut
this.thenEvaluate(function () {
var cell = IPython.notebook.get_cell(0);
cell.set_text('a=11; print(a)');
cell.clear_output();
});
this.then(function(){
this.trigger_keydown('shift-enter');
});
this.wait_for_output(0);
this.then(function () {
var result = this.get_output_cell(0);
var num_cells = this.get_cells_length();
this.test.assertEquals(result.text, '11\n', 'cell execute (using ctrl-enter)');
this.test.assertEquals(num_cells, 2, 'shift-enter adds a new cell at the bottom')
});
// do it again with the keyboard shortcut
this.thenEvaluate(function () {
IPython.notebook.select(1);
IPython.notebook.delete_cell();
var cell = IPython.notebook.get_cell(0);
cell.set_text('a=12; print(a)');
cell.clear_output();
});
this.then(function(){
this.trigger_keydown('ctrl-enter');
});
this.wait_for_output(0);
this.then(function () {
var result = this.get_output_cell(0);
var num_cells = this.get_cells_length();
this.test.assertEquals(result.text, '12\n', 'cell execute (using shift-enter)');
this.test.assertEquals(num_cells, 1, 'ctrl-enter adds no new cell at the bottom')
});
// press the "play" triangle button in the toolbar
this.thenEvaluate(function () {
var cell = IPython.notebook.get_cell(0);
IPython.notebook.select(0);
cell.clear_output();
cell.set_text('a=13; print(a)');
$("button[data-jupyter-action='jupyter-notebook:run-cell-and-select-next']")[0].click()
});
this.wait_for_output(0);
this.then(function () {
var result = this.get_output_cell(0);
this.test.assertEquals(result.text, '13\n', 'cell execute (using "play" toolbar button)')
});
// run code with skip_exception
this.thenEvaluate(function () {
var cell0 = IPython.notebook.get_cell(0);
cell0.set_text('raise IOError');
IPython.notebook.insert_cell_below('code',0);
var cell1 = IPython.notebook.get_cell(1);
cell1.set_text('a=14; print(a)');
cell0.execute(false);
cell1.execute();
});
this.wait_for_output(1);
this.then(function () {
var result = this.get_output_cell(1);
this.test.assertEquals(result.text, '14\n', "cell execute, don't stop on error");
});
this.thenEvaluate(function () {
var cell0 = IPython.notebook.get_cell(0);
cell0.set_text('raise IOError');
IPython.notebook.insert_cell_below('code',0);
var cell1 = IPython.notebook.get_cell(1);
cell1.set_text('a=14; print(a)');
cell0.execute();
cell1.execute();
});
this.wait_for_output(0);
this.then(function () {
var outputs = this.evaluate(function() {
return IPython.notebook.get_cell(1).output_area.outputs;
})
this.test.assertEquals(outputs.length, 0, 'cell execute, stop on error (default)');
});
});

@ -1,45 +0,0 @@
//
// Test kernel interrupt
//
casper.notebook_test(function () {
this.evaluate(function () {
var cell = IPython.notebook.get_cell(0);
cell.set_text(
'import time'+
'\nfor x in range(3):'+
'\n time.sleep(1)'
);
cell.execute();
});
this.wait_for_busy();
// interrupt using menu item (Kernel -> Interrupt)
this.thenClick('li#int_kernel');
this.wait_for_output(0);
this.then(function () {
var result = this.get_output_cell(0);
this.test.assertEquals(result.ename, 'KeyboardInterrupt', 'keyboard interrupt (mouseclick)');
});
// run cell 0 again, now interrupting using keyboard shortcut
this.thenEvaluate(function () {
var cell = IPython.notebook.get_cell(0);
cell.clear_output();
cell.execute();
});
// interrupt using ii keyboard shortcut
this.then(function(){
this.trigger_keydown('esc', 'i', 'i');
});
this.wait_for_output(0);
this.then(function () {
var result = this.get_output_cell(0);
this.test.assertEquals(result.ename, 'KeyboardInterrupt', 'keyboard interrupt (shortcut)');
});
});

@ -1,44 +0,0 @@
casper.notebook_test(function () {
var that = this;
var menuItems = ['#restart_kernel', '#restart_clear_output', '#restart_run_all', '#shutdown_kernel']
var cancelSelector = ".modal-footer button:first-of-type"
menuItems.forEach( function(selector) {
that.thenClick(selector);
that.waitForSelector(cancelSelector);
that.thenClick(cancelSelector);
that.waitWhileSelector(".modal-content", function() {
that.test.assert(true, selector + " confirmation modal pops up and is cancelable");
});
});
var shutdownSelector = menuItems.pop();
var confirmSelector = ".modal-footer .btn-danger"
menuItems.forEach( function(selector) {
that.thenClick(shutdownSelector);
that.waitForSelector(confirmSelector);
that.thenClick(confirmSelector);
// wait for shutdown to go through
that.waitFor(function() { return this.evaluate(function() {
return IPython.notebook.kernel.is_connected() === false;
})});
// Click on one of the restarts
that.thenClick(selector);
// Kernel should get connected, no need for confirmation.
that.waitFor(function() { return this.evaluate(function() {
return IPython.notebook.kernel.is_connected() === true;
})});
that.then(function() {
that.test.assert(true, "no confirmation for " + selector + " after session shutdown")
})
});
});

@ -118,4 +118,6 @@ def authenticated_browser(selenium_driver, notebook_server):
@pytest.fixture
def notebook(authenticated_browser):
return Notebook.new_notebook(authenticated_browser)
tree_wh = authenticated_browser.current_window_handle
yield Notebook.new_notebook(authenticated_browser)
authenticated_browser.switch_to.window(tree_wh)

@ -0,0 +1,66 @@
from selenium.webdriver.common.keys import Keys
from .utils import shift, cmdtrl
def test_execute_code(notebook):
browser = notebook.browser
def clear_outputs():
return notebook.browser.execute_script(
"Jupyter.notebook.clear_all_output();")
# Execute cell with Javascript API
notebook.edit_cell(index=0, content='a=10; print(a)')
browser.execute_script("Jupyter.notebook.get_cell(0).execute();")
outputs = notebook.wait_for_cell_output(0)
assert outputs[0].text == '10'
# Execute cell with Shift-Enter
notebook.edit_cell(index=0, content='a=11; print(a)')
clear_outputs()
shift(notebook.browser, Keys.ENTER)
outputs = notebook.wait_for_cell_output(0)
assert outputs[0].text == '11'
notebook.delete_cell(index=1)
# Execute cell with Ctrl-Enter
notebook.edit_cell(index=0, content='a=12; print(a)')
clear_outputs()
cmdtrl(notebook.browser, Keys.ENTER)
outputs = notebook.wait_for_cell_output(0)
assert outputs[0].text == '12'
# Execute cell with toolbar button
notebook.edit_cell(index=0, content='a=13; print(a)')
clear_outputs()
notebook.browser.find_element_by_css_selector(
"button[data-jupyter-action='jupyter-notebook:run-cell-and-select-next']").click()
outputs = notebook.wait_for_cell_output(0)
assert outputs[0].text == '13'
# Set up two cells to test stopping on error
notebook.edit_cell(index=0, content='raise IOError')
notebook.edit_cell(index=1, content='a=14; print(a)')
# Default behaviour: stop on error
clear_outputs()
browser.execute_script("""
var cell0 = Jupyter.notebook.get_cell(0);
var cell1 = Jupyter.notebook.get_cell(1);
cell0.execute();
cell1.execute();
""")
outputs = notebook.wait_for_cell_output(0)
assert notebook.get_cell_output(1) == []
# Execute a cell with stop_on_error=false
clear_outputs()
browser.execute_script("""
var cell0 = Jupyter.notebook.get_cell(0);
var cell1 = Jupyter.notebook.get_cell(1);
cell0.execute(false);
cell1.execute();
""")
outputs = notebook.wait_for_cell_output(1)
assert outputs[0].text == '14'

@ -0,0 +1,36 @@
from .utils import wait_for_selector
def interrupt_from_menu(notebook):
# Click interrupt button in kernel menu
notebook.browser.find_element_by_id('kernellink').click()
wait_for_selector(notebook.browser, '#int_kernel', single=True).click()
def interrupt_from_keyboard(notebook):
notebook.body.send_keys("ii")
def test_interrupt(notebook):
""" Test the interrupt function using both the button in the Kernel menu and the keyboard shortcut "ii"
Having trouble accessing the Interrupt message when execution is halted. I am assuming that the
message does not lie in the "outputs" field of the cell's JSON object. Using a timeout work-around for
test with an infinite loop. We know the interrupt function is working if this test passes.
Hope this is a good start.
"""
text = ('import time\n'
'for x in range(3):\n'
' time.sleep(1)')
notebook.edit_cell(index=0, content=text)
for interrupt_method in (interrupt_from_menu, interrupt_from_keyboard):
notebook.clear_cell_output(0)
notebook.to_command_mode()
notebook.execute_cell(0)
interrupt_method(notebook)
# Wait for an output to appear
output = wait_for_selector(notebook.browser, '.output_subarea', single=True)
assert 'KeyboardInterrupt' in output.text

@ -0,0 +1,60 @@
from selenium.common.exceptions import TimeoutException
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.support.ui import WebDriverWait
from notebook.tests.selenium.utils import wait_for_selector
restart_selectors = [
'#restart_kernel', '#restart_clear_output', '#restart_run_all'
]
notify_interaction = '#notification_kernel > span'
shutdown_selector = '#shutdown_kernel'
confirm_selector = '.btn-danger'
cancel_selector = ".modal-footer button:first-of-type"
def test_cancel_restart_or_shutdown(notebook):
"""Click each of the restart options, then cancel the confirmation dialog"""
browser = notebook.browser
kernel_menu = browser.find_element_by_id('kernellink')
for menu_item in restart_selectors + [shutdown_selector]:
kernel_menu.click()
wait_for_selector(browser, menu_item, visible=True, single=True).click()
wait_for_selector(browser, cancel_selector, visible=True, single=True).click()
WebDriverWait(browser, 3).until(
EC.invisibility_of_element((By.CSS_SELECTOR, '.modal-backdrop'))
)
assert notebook.is_kernel_running()
def test_menu_items(notebook):
browser = notebook.browser
kernel_menu = browser.find_element_by_id('kernellink')
for menu_item in restart_selectors:
# Shutdown
kernel_menu.click()
wait_for_selector(browser, shutdown_selector, visible=True, single=True).click()
# Confirm shutdown
wait_for_selector(browser, confirm_selector, visible=True, single=True).click()
WebDriverWait(browser, 3).until(
lambda b: not notebook.is_kernel_running(),
message="Kernel did not shut down as expected"
)
# Restart
# Selenium can't click the menu while a modal dialog is fading out
WebDriverWait(browser, 3).until(
EC.invisibility_of_element((By.CSS_SELECTOR, '.modal-backdrop'))
)
kernel_menu.click()
wait_for_selector(browser, menu_item, visible=True, single=True).click()
WebDriverWait(browser, 10).until(
lambda b: notebook.is_kernel_running(),
message="Restart (%r) after shutdown did not start kernel" % menu_item
)

@ -218,6 +218,11 @@ class Notebook:
def get_cell_output(self, index=0, output='output_subarea'):
return self.cells[index].find_elements_by_class_name(output)
def wait_for_cell_output(self, index=0, timeout=10):
return WebDriverWait(self.browser, timeout).until(
lambda b: self.get_cell_output(index)
)
def set_cell_metadata(self, index, key, value):
JS = 'Jupyter.notebook.get_cell({}).metadata.{} = {}'.format(index, key, value)
return self.browser.execute_script(JS)
@ -300,7 +305,13 @@ class Notebook:
trigger_keystrokes(self.body, keys)
def is_kernel_running(self):
return self.browser.execute_script("return Jupyter.notebook.kernel.is_connected()")
return self.browser.execute_script(
"return Jupyter.notebook.kernel && Jupyter.notebook.kernel.is_connected()"
)
def clear_cell_output(self, index):
JS = 'Jupyter.notebook.clear_output({})'.format(index)
self.browser.execute_script(JS)
@classmethod
def new_notebook(cls, browser, kernel_name='kernel-python3'):
@ -455,4 +466,3 @@ def validate_dualmode_state(notebook, mode, index):
assert is_focused_on(index) #The specified cell is focused
assert is_only_cell_edit(index) #The specified cell is the only one in edit mode

@ -8,7 +8,8 @@ from unittest.mock import patch
import nose.tools as nt
from tornado import gen
from tornado.httpclient import HTTPRequest, HTTPResponse, HTTPError
from tornado.web import HTTPError
from tornado.httpclient import HTTPRequest, HTTPResponse
from notebook.gateway.managers import GatewayClient
from notebook.utils import maybe_future

@ -14,10 +14,7 @@ from os.path import basename, join as pjoin
from traitlets.tests.utils import check_help_all_output
from unittest import TestCase
try:
from unittest.mock import patch
except ImportError:
from mock import patch # py2
from unittest.mock import patch
import ipython_genutils.testing.decorators as dec
from ipython_genutils import py3compat

@ -9,10 +9,7 @@ from subprocess import Popen, PIPE, STDOUT
import sys
from tempfile import NamedTemporaryFile
try:
from unittest.mock import patch
except ImportError:
from mock import patch # py2
from unittest.mock import patch
import nose.tools as nt

@ -1,17 +1,9 @@
import re
import nose.tools as nt
from nose.tools import assert_regex, assert_not_regex
from notebook.base.handlers import path_regex
try: # py3
assert_regex = nt.assert_regex
assert_not_regex = nt.assert_not_regex
except AttributeError: # py2
assert_regex = nt.assert_regexp_matches
assert_not_regex = nt.assert_not_regexp_matches
# build regexps that tornado uses:
path_pat = re.compile('^' + '/x%s' % path_regex + '$')

@ -2,10 +2,7 @@ import imp
import os
import sys
from unittest import TestCase
try:
from unittest.mock import patch
except ImportError:
from mock import patch # py2
from unittest.mock import patch
from ipython_genutils.tempdir import TemporaryDirectory
from ipython_genutils import py3compat

@ -4,10 +4,7 @@ import io
from notebook.utils import url_path_join
from nbformat import write
from nbformat.v4 import new_notebook
try:
from urllib.parse import urlparse
except ImportError:
from urlparse import urlparse
from urllib.parse import urlparse
import requests

@ -16,13 +16,8 @@ import sys
from distutils.version import LooseVersion
try:
from urllib.parse import quote, unquote, urlparse, urljoin
from urllib.request import pathname2url
except ImportError:
from urllib import quote, unquote, pathname2url
from urlparse import urlparse, urljoin
from urllib.parse import quote, unquote, urlparse, urljoin
from urllib.request import pathname2url
# tornado.concurrent.Future is asyncio.Future
# in tornado >=5 with Python 3
from tornado.concurrent import Future as TornadoFuture

@ -2,7 +2,7 @@
universal=0
[metadata]
license_file = COPYING.md
license_file = LICENSE
[nosetests]
warningfilters=module |.* |DeprecationWarning |notebook.*

@ -8,17 +8,15 @@
#
# Distributed under the terms of the Modified BSD License.
#
# The full license is in the file COPYING.md, distributed with this software.
# The full license is in the file LICENSE, distributed with this software.
#-----------------------------------------------------------------------------
from __future__ import print_function
import os
import sys
name = "notebook"
if sys.version_info < (3, 4):
if sys.version_info < (3, 5):
pip_message = 'This may be due to an out of date pip. Make sure you have pip >= 9.0.1.'
try:
import pip
@ -34,8 +32,8 @@ if sys.version_info < (3, 4):
error = """
Notebook 6.0+ supports Python 3.4 and above.
When using Python 2.7, please install Notebook 5.x.
Notebook 6.0+ supports Python 3.5 and above.
When using Python 3.4 or earlier (including 2.7), please install Notebook 5.x.
Python {py} detected.
{pip}
@ -92,8 +90,10 @@ for more information.
'Intended Audience :: Science/Research',
'License :: OSI Approved :: BSD License',
'Programming Language :: Python',
'Programming Language :: Python :: 2.7',
'Programming Language :: Python :: 3',
'Programming Language :: Python :: 3.5',
'Programming Language :: Python :: 3.6',
'Programming Language :: Python :: 3.7'
],
zip_safe = False,
install_requires = [
@ -104,18 +104,16 @@ for more information.
'pyzmq>=17',
'ipython_genutils',
'traitlets>=4.2.1',
'jupyter_core>=4.4.0',
'jupyter_client>=5.3.1',
'jupyter_core>=4.6.0',
'jupyter_client>=5.3.4',
'nbformat',
'nbconvert',
'nbconvert<6',
'ipykernel', # bless IPython kernel for now
'Send2Trash',
'terminado>=0.8.1',
'prometheus_client'
],
extras_require = {
':python_version == "2.7"': ['ipaddress'],
'test:python_version == "2.7"': ['mock'],
'test': ['nose', 'coverage', 'requests', 'nose_warnings_filters',
'nbval', 'nose-exclude', 'selenium', 'pytest', 'pytest-cov'],
'test:sys_platform == "win32"': ['nose-exclude'],

@ -150,6 +150,7 @@ def find_package_data():
pjoin(components, "jquery-ui", "themes", "smoothness", "images", "*"),
pjoin(components, "marked", "lib", "marked.js"),
pjoin(components, "react", "react.production.min.js"),
pjoin(components, "react", "react-dom.production.min.js"),
pjoin(components, "requirejs", "require.js"),
pjoin(components, "requirejs-plugins", "src", "json.js"),
pjoin(components, "requirejs-text", "text.js"),

Loading…
Cancel
Save