" - Get the notebook data and other required files. (you are responsible for that)\n",
" - Feed them to the exporter that will\n",
" - sequentially feed the data to a number of `Transformers`. Transformer only act on the **structure**\n",
" - sequentially feed the data to a number of `Preprocessors`. Preprocessor only act on the **structure**\n",
" of the notebook, and have access to it all. \n",
" - feed the notebook through the jinja templating engine\n",
" - the use templates are configurable.\n",
@ -1000,20 +982,20 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Here we'll be interested in the `Transformers`. Each `Transformer` is applied successively and in order on the notebook before going through the conversion process.\n",
"Here we'll be interested in the `Preprocessors`. Each `Preprocessor` is applied successively and in order on the notebook before going through the conversion process.\n",
"\n",
"We provide some transformer that do some modification on the notebook structure by default.\n",
"One of them, the `ExtractOutputTransformer` is responsible for crawling notebook,\n",
"We provide some preprocessor that do some modification on the notebook structure by default.\n",
"One of them, the `ExtractOutputPreprocessor` is responsible for crawling notebook,\n",
"finding all the figures, and put them into the resources directory, as well as choosing the key\n",
"(`filename_xx_y.extension`) that can replace the figure in the template.\n",
"\n",
"\n",
"The `ExtractOutputTransformer` is special in the fact that it **should** be availlable on all `Exporter`s, but is just inactive by default on some exporter."
"The `ExtractOutputPreprocessor` is special in the fact that it **should** be availlable on all `Exporter`s, but is just inactive by default on some exporter."
" <IPython.nbconvert.preprocessors.svg2pdf.SVG2PDFPreprocessor at 0x107d1a630>,\n",
" <IPython.nbconvert.preprocessors.extractoutput.ExtractOutputPreprocessor at 0x107d1a748>,\n",
" <IPython.nbconvert.preprocessors.csshtmlheader.CSSHTMLHeaderPreprocessor at 0x107d1aba8>,\n",
" <IPython.nbconvert.preprocessors.revealhelp.RevealHelpPreprocessor at 0x107d1a710>,\n",
" <IPython.nbconvert.preprocessors.latex.LatexPreprocessor at 0x107daa860>,\n",
" <IPython.nbconvert.preprocessors.clearoutput.ClearOutputPreprocessor at 0x107db7080>,\n",
" <IPython.nbconvert.preprocessors.execute.ExecutePreprocessor at 0x107db7160>,\n",
" <IPython.nbconvert.preprocessors.highlightmagics.HighlightMagicsPreprocessor at 0x107db7048>]"
]
},
"execution_count": 12,
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# second transformer shoudl be Instance of ExtractFigureTransformer\n",
"exportHtml._transformers # 3rd one shouel be <ExtractOutputTransformer>"
"# 3rd one should be <ExtractOutputPreprocessor>\n",
"html_exporter._preprocessors"
]
},
{
@ -1047,14 +1031,14 @@
"To enable it we will use IPython configuration/Traitlets system. If you are have already set some IPython configuration options, \n",
"this will look pretty familiar to you. Configuration option are always of the form:\n",
"\n",
" ClassName.attribute_name = value\n",
" ClassName.attribute_name = value\n",
" \n",
"A few ways exist to create such config, like reading a config file in your profile, but you can also do it programatically usign a dictionary. Let's create such a config object, and see the difference if we pass it to our `HtmlExporter`"
"A few ways exist to create such config, like reading a config file in your profile, but you can also do it programatically usign a dictionary. Let's create such a config object, and see the difference if we pass it to our `HTMLExporter`"
"print'resources without the \"figures\" key :'\n",
"print resources.keys()\n",
"print('resources without the \"figures\" key :')\n",
"print(list(resources))\n",
"\n",
"print ''\n",
"print 'Here we have one more field '\n",
"print resources_with_fig.keys()\n",
"resources_with_fig['outputs'].keys()"
"print('')\n",
"print('Here we have one more field')\n",
"print(list(resources_with_fig))\n",
"list(resources_with_fig['outputs'])"
]
},
{
@ -1126,27 +1102,27 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Custom transformer"
"#### Custom Preprocessor"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Of course you can imagine many transformation that you would like to apply to a notebook. This is one of the reason we provide a way to register your own transformers that will be applied to the notebook after the default ones.\n",
"Of course you can imagine many transformation that you would like to apply to a notebook. This is one of the reason we provide a way to register your own preprocessors that will be applied to the notebook after the default ones.\n",
"\n",
"To do so you'll have to pass an ordered list of `Transformer`s to the Exporter constructor. \n",
"To do so you'll have to pass an ordered list of `Preprocessor`s to the Exporter constructor. \n",
"\n",
"But what is an transformer ? Transformer can be either *decorated function* for dead-simple `Transformer`s that apply\n",
"But what is an preprocessor ? Preprocessor can be either *decorated function* for dead-simple `Preprocessor`s that apply\n",
"independently to each cell, for more advance transformation that support configurability You have to inherit from\n",
"`Transformer` and define a `call` method as we'll see below.\n",
"`Preprocessor` and define a `call` method as we'll see below.\n",
"\n",
"All transforers have a magic attribute that allows it to be activated/disactivate from the config dict."
]
},
{
"cell_type": "code",
"execution_count": 14,
"execution_count": 13,
"metadata": {
"collapsed": false
},
@ -1157,27 +1133,29 @@
"text": [
"Four relevant docstring\n",
"=============================\n",
" A configurable transformer\n",
" A configurable preprocessor\n",
"\n",
" Inherit from this class if you wish to have configurability for your\n",
" transformer.\n",
" preprocessor.\n",
"\n",
" Any configurable traitlets this class exposed will be configurable in profiles\n",
" using c.SubClassName.atribute=value\n",
" Any configurable traitlets this class exposed will be configurable in\n",
" profiles using c.SubClassName.attribute=value\n",
"\n",
" you can overwrite transform_cell to apply a transformation independently on each cell\n",
" or __call__ if you prefer your own logic. See corresponding docstring for informations.\n",
" you can overwrite :meth:`preprocess_cell` to apply a transformation\n",
" independently on each cell or :meth:`preprocess` if you prefer your own\n",
" logic. See corresponding docstring for informations.\n",
"\n",
" Disabled by default and can be enabled via the config by\n",
" 'c.YourTransformerName.enabled = True'\n",
" 'c.YourPreprocessorName.enabled = True'\n",
" \n",
"=============================\n",
"\n",
" Transformation to apply on each notebook.\n",
" Preprocessing to apply on each notebook.\n",
" \n",
" You should return modified nb, resources.\n",
" If you wish to apply your transform on each cell, you might want to \n",
" overwrite transform_cell method instead.\n",
" Must return modified nb, resources.\n",
" \n",
" If you wish to apply your preprocessing to each cell, you might want\n",
" to override preprocess_cell method instead.\n",
" \n",
" Parameters\n",
" ----------\n",
@ -1185,12 +1163,12 @@
" Notebook being converted\n",
" resources : dictionary\n",
" Additional resources used in the conversion process. Allows\n",
" transformers to pass variables into the Jinja engine.\n",
" preprocessors to pass variables into the Jinja engine.\n",
" \n",
"=============================\n",
"\n",
" Overwrite if you want to apply a transformation on each cell. You \n",
" should return modified cell and resource dictionary.\n",
" Override if you want to apply some preprocessing to each cell.\n",
" Must return modified cell and resource dictionary.\n",
" \n",
" Parameters\n",
" ----------\n",
@ -1198,7 +1176,7 @@
" Notebook cell being processed\n",
" resources : dictionary\n",
" Additional resources used in the conversion process. Allows\n",
" transformers to pass variables into the Jinja engine.\n",
" preprocessors to pass variables into the Jinja engine.\n",
"We don't provide convenient method to be aplied on each worksheet as the **data structure** for worksheet will be removed. (not the worksheet functionnality, which is still on it's way)\n",
"We don't provide convenient method to be aplied on each worksheet as the **data structure** for worksheet will be removed. (not the worksheet functionality, which is still on it's way)\n",
"***"
]
},
@ -1239,14 +1217,14 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"I'll now demonstrate a specific example [requested](https://github.com/ipython/nbconvert/pull/137#issuecomment-18658235) while nbconvert 2 was beeing developped. The ability to exclude cell from the conversion process based on their index. \n",
"I'll now demonstrate a specific example [requested](https://github.com/ipython/nbconvert/pull/137#issuecomment-18658235) while nbconvert 2 was being developed. The ability to exclude cell from the conversion process based on their index. \n",
"\n",
"I'll let you imagin how to inject cell, if what you just want is to happend static content at the beginning/end of a notebook, plese refer to templating section, it will be much easier and cleaner."
]
},
{
"cell_type": "code",
"execution_count": 15,
"execution_count": 14,
"metadata": {
"collapsed": false
},
@ -1257,71 +1235,66 @@
},
{
"cell_type": "code",
"execution_count": 16,
"execution_count": 15,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"class PelicanSubCell(Transformer):\n",
" \"\"\"A Pelican specific transformer to remove somme of the cells of a notebook\"\"\"\n",
"class PelicanSubCell(Preprocessor):\n",
" \"\"\"A Pelican specific preprocessor to remove somme of the cells of a notebook\"\"\"\n",
" \n",
" # I could also read the cells from nbc.metadata.pelican is someone wrote a JS extension\n",
" # But I'll stay with configurable value. \n",
" start = Integer(0, config=True, help=\"first cell of notebook to be converted\")\n",
" end = Integer(-1, config=True, help=\"last cell of notebook to be converted\")\n",
" \n",
" def call(self, nb, resources):\n",
" def preprocess(self, nb, resources):\n",
"\n",
" #nbc = deepcopy(nb)\n",
" nbc = nb\n",
" # don't print in real transformer !!!\n",
" print \"I'll keep only cells from \", self.start, \"to \", self.end, \"\\n\\n\"\n",
" print(\"I'll keep only cells from \", self.start, \"to \", self.end, \"\\n\\n\")\n",
" nbc.cells = nb.cells[self.start:self.end] \n",
" return nbc, resources"
]
},
{
"cell_type": "code",
"execution_count": 17,
"execution_count": 16,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"# I create this on the fly, but this could be loaded from a DB, and config object support merging...\n",
"c = Config({\n",
" 'PelicanSubCell':{\n",
" 'enabled':True,\n",
" 'start':4,\n",
" 'end':6,\n",
" }\n",
" })"
"c = Config()\n",
"c.PelicanSubCell.enabled = True\n",
"c.PelicanSubCell.start = 4\n",
"c.PelicanSubCell.end = 6"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"I'm creating a pelican exporter that take `PelicanSubCell` extra transformers and a `config` object as parameter. This might seem redundant, but with configuration system you'll see that one can register an inactive transformer on all exporters and activate it at will form its config files and command line. "
"I'm creating a pelican exporter that take `PelicanSubCell` extra preprocessors and a `config` object as parameter. This might seem redundant, but with configuration system you'll see that one can register an inactive preprocessor on all exporters and activate it at will form its config files and command line. "
"from IPython.nbformat import current as nbformat"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Here we define a dustom 'highlight' filter that apply a custom class to code in css. We register this filter with a already existing name, so it will replace the default one."
"<p>This post was written entirely in an IPython Notebook: the notebook file is available for download <a href=\"http://jakevdp.github.com/downloads/notebooks/XKCD_plots.ipynb\">here</a>. For more information on blogging with notebooks in octopress, see my <a href=\"http://jakevdp.github.com/blog/2012/10/04/blogging-with-ipython/\">previous post</a> on the subject.</p>\n",
"<blockquote class=\"twitter-tweet\"><p>As <a href=\"https://twitter.com/Mbussonn\">@Mbussonn</a> requested... easieeeeer! Deploy your Nikola site with just a click in the IPython notebook! <a href=\"http://t.co/860sJunZvj\">http://t.co/860sJunZvj</a> cc <a href=\"https://twitter.com/ralsina\">@ralsina</a></p>— Dami\u00e1n Avila (@damian_avila) <a href=\"https://twitter.com/damian_avila/statuses/370306057828335616\">August 21, 2013</a></blockquote>\n",
"<blockquote class=\"twitter-tweet\"><p>As <a href=\"https://twitter.com/Mbussonn\">@Mbussonn</a> requested... easieeeeer! Deploy your Nikola site with just a click in the IPython notebook! <a href=\"http://t.co/860sJunZvj\">http://t.co/860sJunZvj</a> cc <a href=\"https://twitter.com/ralsina\">@ralsina</a></p>— Damián Avila (@damian_avila) <a href=\"https://twitter.com/damian_avila/statuses/370306057828335616\">August 21, 2013</a></blockquote>\n",
"</center>"
]
},
@ -1525,18 +1423,26 @@
"source": [
"Jinja blocks use `{% %}`by default which does not play nicely with $\\LaTeX$, hence thoses are replaced by `((* *))` in latex templates."
Min RK
11 years ago