From dca0581782940851e3bb880a00afc9466c00c58d Mon Sep 17 00:00:00 2001 From: zeptodoctor <44736852+zeptodoctor@users.noreply.github.com> Date: Thu, 21 Nov 2019 22:40:01 +0000 Subject: [PATCH] build based on 233b374 --- dev/customprocessing/index.html | 2 +- dev/documenter/index.html | 2 +- dev/fileformat/index.html | 2 +- dev/generated/example.ipynb | 168 +++++++++++++++---------------- dev/generated/example/index.html | 86 ++++++++-------- dev/generated/name/index.html | 2 +- dev/index.html | 2 +- dev/outputformats/index.html | 4 +- dev/pipeline/index.html | 2 +- dev/search/index.html | 2 +- 10 files changed, 136 insertions(+), 136 deletions(-) diff --git a/dev/customprocessing/index.html b/dev/customprocessing/index.html index d1f43a8..900807a 100644 --- a/dev/customprocessing/index.html +++ b/dev/customprocessing/index.html @@ -26,4 +26,4 @@ include("file2.jl")

Let's say we have saved this fi end return str end

(of course replace included with your respective files)

Finally, you simply pass this function to e.g. Literate.markdown as

Literate.markdown("examples.jl", "path/to/save/markdown";
-                  name = "markdown_file_name", preprocess = replace_includes)

and you will see that in the final output file (here markdown_file_name.md) the include statements are replaced with the actual code to be included!

This approach is used for generating the examples in the documentation of the TimeseriesPrediction.jl package. The example files, included together in the stexamples.jl file, are processed by literate via this make.jl file to generate the markdown and code cells of the documentation.

+ name = "markdown_file_name", preprocess = replace_includes)

and you will see that in the final output file (here markdown_file_name.md) the include statements are replaced with the actual code to be included!

This approach is used for generating the examples in the documentation of the TimeseriesPrediction.jl package. The example files, included together in the stexamples.jl file, are processed by literate via this make.jl file to generate the markdown and code cells of the documentation.

diff --git a/dev/documenter/index.html b/dev/documenter/index.html index 838d99b..b933f79 100644 --- a/dev/documenter/index.html +++ b/dev/documenter/index.html @@ -9,4 +9,4 @@ EditURL = "$(relpath(inputfile, outputdir))" \int f dx ```is replaced with notebook compatible
$$
 \int f dx
-$$

Literate.script:

+$$

Literate.script:

diff --git a/dev/fileformat/index.html b/dev/fileformat/index.html index ca19ea2..e3c9556 100644 --- a/dev/fileformat/index.html +++ b/dev/fileformat/index.html @@ -15,4 +15,4 @@ z = x + y

In the lines starting with # we can use re #md # Literate.notebook #md # Literate.markdown #md # ```

The lines in the example above would be filtered out in the preprocessing step, unless we are generating a markdown file. When generating a markdown file we would simple remove the leading #md from the lines. Beware that the space after the tag is also removed.

The #src token can also be placed at the end of a line. This is to make it possible to have code lines exclusive to the source code, and not just comment lines. For example, if the source file is included in the test suite we might want to add a @test at the end without this showing up in the outputs:

using Test                      #src
-@test result == expected_result #src

2.3. Default Replacements

The following convenience "macros" are always expanded:

Note

@__REPO_ROOT_URL__ and @__NBVIEWER_ROOT_URL__ works for documentation built with DocumentationGenerator.jl but @__BINDER_ROOT_URL__ does not, since mybinder.org requires the files to be located inside a git repository.

+@test result == expected_result #src

2.3. Default Replacements

The following convenience "macros" are always expanded:

Note

@__REPO_ROOT_URL__ and @__NBVIEWER_ROOT_URL__ works for documentation built with DocumentationGenerator.jl but @__BINDER_ROOT_URL__ does not, since mybinder.org requires the files to be located inside a git repository.

diff --git a/dev/generated/example.ipynb b/dev/generated/example.ipynb index 581826a..38bf890 100644 --- a/dev/generated/example.ipynb +++ b/dev/generated/example.ipynb @@ -205,114 +205,114 @@ "\n", "\n", "\n", - " \n", + " \n", " \n", " \n", "\n", - "\n", "\n", - " \n", + " \n", " \n", " \n", "\n", - "\n", "\n", - " \n", + " \n", " \n", " \n", "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", + "\n", "0\n", "\n", - "\n", + "\n", "5\n", "\n", - "\n", + "\n", "10\n", "\n", - "\n", + "\n", "15\n", "\n", - "\n", + "\n", "-1.0\n", "\n", - "\n", + "\n", "-0.5\n", "\n", - "\n", + "\n", "0.0\n", "\n", - "\n", + "\n", "0.5\n", "\n", - "\n", + "\n", "1.0\n", "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", + "\n", "y1\n", "\n", - "\n", - "\n", + "\n", "y2\n", "\n", "\n" @@ -542,114 +542,114 @@ "\n", "\n", "\n", - " \n", + " \n", " \n", " \n", "\n", - "\n", "\n", - " \n", + " \n", " \n", " \n", "\n", - "\n", "\n", - " \n", + " \n", " \n", " \n", "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", + "\n", "0\n", "\n", - "\n", + "\n", "5\n", "\n", - "\n", + "\n", "10\n", "\n", - "\n", + "\n", "15\n", "\n", - "\n", + "\n", "-1.0\n", "\n", - "\n", + "\n", "-0.5\n", "\n", - "\n", + "\n", "0.0\n", "\n", - "\n", + "\n", "0.5\n", "\n", - "\n", + "\n", "1.0\n", "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", + "\n", "y1\n", "\n", - "\n", - "\n", + "\n", "y2\n", "\n", "\n" diff --git a/dev/generated/example/index.html b/dev/generated/example/index.html index 897767b..734954f 100644 --- a/dev/generated/example/index.html +++ b/dev/generated/example/index.html @@ -16,114 +16,114 @@ y2 = cos.(x) plot(x, [y1, y2]) - + - - + - - + - - - - - - - - - - - - - - - - - - - - - + 0 - + 5 - + 10 - + 15 - + -1.0 - + -0.5 - + 0.0 - + 0.5 - + 1.0 - - - - - - + y1 - - + y2 -

Custom processing

It is possible to give Literate custom pre- and post-processing functions. For example, here we insert two placeholders, which we will replace with something else at time of generation. We have here replaced our placeholders with z and 1.0 + 2.0im:

z = 1.0 + 2.0im
1.0 + 2.0im

Documenter.jl interaction

In the source file it is possible to use Documenter.jl style references, such as @ref and @id. These will be filtered out in the notebook output. For example, here is a link, but it is only visible as a link if you are reading the markdown output. We can also use equations:

\[\int_\Omega \nabla v \cdot \nabla u\ \mathrm{d}\Omega = \int_\Omega v f\ \mathrm{d}\Omega\]

using Documenters math syntax. Documenters syntax is automatically changed to \begin{equation} ... \end{equation} in the notebook output to display correctly.

This page was generated using Literate.jl.

+

Custom processing

It is possible to give Literate custom pre- and post-processing functions. For example, here we insert two placeholders, which we will replace with something else at time of generation. We have here replaced our placeholders with z and 1.0 + 2.0im:

z = 1.0 + 2.0im
1.0 + 2.0im

Documenter.jl interaction

In the source file it is possible to use Documenter.jl style references, such as @ref and @id. These will be filtered out in the notebook output. For example, here is a link, but it is only visible as a link if you are reading the markdown output. We can also use equations:

\[\int_\Omega \nabla v \cdot \nabla u\ \mathrm{d}\Omega = \int_\Omega v f\ \mathrm{d}\Omega\]

using Documenters math syntax. Documenters syntax is automatically changed to \begin{equation} ... \end{equation} in the notebook output to display correctly.

This page was generated using Literate.jl.

diff --git a/dev/generated/name/index.html b/dev/generated/name/index.html index fd177af..4a9de7b 100644 --- a/dev/generated/name/index.html +++ b/dev/generated/name/index.html @@ -1,2 +1,2 @@ -Rational numbers · Literate.jl
Edit on GitHub

Rational numbers

In julia rational numbers can be constructed with the // operator. Lets define two rational numbers, x and y:

x = 1//3
1//3
y = 2//5
2//5

When adding x and y together we obtain a new rational number:

z = x + y
11//15
+Rational numbers · Literate.jl
Edit on GitHub

Rational numbers

In julia rational numbers can be constructed with the // operator. Lets define two rational numbers, x and y:

x = 1//3
1//3
y = 2//5
2//5

When adding x and y together we obtain a new rational number:

z = x + y
11//15
diff --git a/dev/index.html b/dev/index.html index 916ded7..3302187 100644 --- a/dev/index.html +++ b/dev/index.html @@ -1,2 +1,2 @@ -1. Introduction · Literate.jl
Edit on GitHub

1. Introduction

Welcome to the documentation for Literate – a simplistic package for Literate Programming.

What?

Literate is a package that generates markdown pages (for e.g. Documenter.jl), and Jupyter notebooks, from the same source file. There is also an option to "clean" the source from all metadata, and produce a pure Julia script.

The main design goal is simplicity. It should be simple to use, and the syntax should be simple. In short, all you have to do is to write a commented julia script!

The public interface consists mainly of three functions, all of which take the same script file as input, but generate different output:

Why?

Examples are (probably) the best way to showcase your awesome package, and examples are often the best way for a new user to learn how to use it. It is therefore important that the documentation of your package contains examples for users to read and study. However, people are different, and we all prefer different ways of trying out a new package. Some people wants to RTFM, others want to explore the package interactively in, for example, a notebook, and some people wants to study the source code. The aim of Literate is to make it easy to give the user all of these options, while still keeping maintenance to a minimum.

It is quite common that packages have "example notebooks" to showcase the package. Notebooks are great for showcasing a package, but they are not so great with version control, like git. The reason being that a notebook is a very "rich" format since it contains output and other metadata. Changes to the notebook thus result in large diffs, which makes it harder to review the actual changes.

It is also common that packages include examples in the documentation, for example by using Documenter.jl @example-blocks. This is also great, but it is not quite as interactive as a notebook, for the users who prefer that.

Literate tries to solve the problems above by creating the output as a part of the doc build. Literate generates the output based on a single source file which makes it easier to maintain, test, and keep the manual and your example notebooks in sync.

+1. Introduction · Literate.jl
Edit on GitHub

1. Introduction

Welcome to the documentation for Literate – a simplistic package for Literate Programming.

What?

Literate is a package that generates markdown pages (for e.g. Documenter.jl), and Jupyter notebooks, from the same source file. There is also an option to "clean" the source from all metadata, and produce a pure Julia script.

The main design goal is simplicity. It should be simple to use, and the syntax should be simple. In short, all you have to do is to write a commented julia script!

The public interface consists mainly of three functions, all of which take the same script file as input, but generate different output:

Why?

Examples are (probably) the best way to showcase your awesome package, and examples are often the best way for a new user to learn how to use it. It is therefore important that the documentation of your package contains examples for users to read and study. However, people are different, and we all prefer different ways of trying out a new package. Some people wants to RTFM, others want to explore the package interactively in, for example, a notebook, and some people wants to study the source code. The aim of Literate is to make it easy to give the user all of these options, while still keeping maintenance to a minimum.

It is quite common that packages have "example notebooks" to showcase the package. Notebooks are great for showcasing a package, but they are not so great with version control, like git. The reason being that a notebook is a very "rich" format since it contains output and other metadata. Changes to the notebook thus result in large diffs, which makes it harder to review the actual changes.

It is also common that packages include examples in the documentation, for example by using Documenter.jl @example-blocks. This is also great, but it is not quite as interactive as a notebook, for the users who prefer that.

Literate tries to solve the problems above by creating the output as a part of the doc build. Literate generates the output based on a single source file which makes it easier to maintain, test, and keep the manual and your example notebooks in sync.

diff --git a/dev/outputformats/index.html b/dev/outputformats/index.html index abb9abd..08546a3 100644 --- a/dev/outputformats/index.html +++ b/dev/outputformats/index.html @@ -31,8 +31,8 @@ When adding `x` and `y` together we obtain a new rational number: ```@example name z = x + y -```

We note that lines starting with # are printed as regular markdown, and the code lines have been wrapped in @example blocks. We also note that an @meta block have been added, that sets the EditURL variable. This is used by Documenter to redirect the "Edit on GitHub" link for the page, see Interaction with Documenter.

Some of the output rendering can be controlled with keyword arguments to Literate.markdown:

Literate.markdown — Function
Literate.markdown(inputfile, outputdir; kwargs...)

Generate a markdown file from inputfile and write the result to the directory outputdir.

Keyword arguments:

  • name: name of the output file, excluding .md; name is also used to name all the @example blocks, and to replace @__NAME__. Defaults to the filename of inputfile.
  • preprocess, postprocess: custom pre- and post-processing functions, see the Custom pre- and post-processing section of the manual. Defaults to identity.
  • documenter: boolean that tells if the output is intended to use with Documenter.jl. Defaults to true. See the manual section on Interaction with Documenter.
  • codefence: A Pair of opening and closing code fence. Defaults to
    "```@example $(name)" => "```"
    if documenter = true and
    "```julia" => "```"
    if documenter = false.
  • credit: boolean that controls the addition of This file was generated with Literate.jl ... to the bottom of the page. If you find Literate.jl useful then feel free to keep this to the default, which is true.
source

4.2. Notebook Output

The (default) notebook output of the source snippet can be seen here: notebook.ipynb.

We note that lines starting with # are placed in markdown cells, and the code lines have been placed in code cells. By default the notebook is also executed and output cells populated. The current working directory is set to the specified output directory the notebook is executed. Some of the output rendering can be controlled with keyword arguments to Literate.notebook:

Literate.notebook — Function
Literate.notebook(inputfile, outputdir; kwargs...)

Generate a notebook from inputfile and write the result to outputdir.

Keyword arguments:

  • name: name of the output file, excluding .ipynb. name is also used to replace @__NAME__. Defaults to the filename of inputfile.
  • preprocess, postprocess: custom pre- and post-processing functions, see the Custom pre- and post-processing section of the manual. Defaults to identity.
  • execute: a boolean deciding if the generated notebook should also be executed or not. Defaults to true. The current working directory is set to outputdir when executing the notebook.
  • documenter: boolean that says if the source contains Documenter.jl specific things to filter out during notebook generation. Defaults to true. See the the manual section on Interaction with Documenter.
  • credit: boolean that controls the addition of This file was generated with Literate.jl ... to the bottom of the page. If you find Literate.jl useful then feel free to keep this to the default, which is true.
source

Notebook metadata

Jupyter notebook cells (both code cells and markdown cells) can contain metadata. This is enabled in Literate by the %% token, similar to Jupytext. The format is as follows

%% optional ignored text [type] {optional metadata JSON}

Cell metadata can, for example, be used for nbgrader and the reveal.js notebook extension RISE.

4.3. Script Output

The (default) script output of the source snippet above is as follows

x = 1//3
+```

We note that lines starting with # are printed as regular markdown, and the code lines have been wrapped in @example blocks. We also note that an @meta block have been added, that sets the EditURL variable. This is used by Documenter to redirect the "Edit on GitHub" link for the page, see Interaction with Documenter.

Some of the output rendering can be controlled with keyword arguments to Literate.markdown:

Literate.markdown — Function
Literate.markdown(inputfile, outputdir; kwargs...)

Generate a markdown file from inputfile and write the result to the directory outputdir.

Keyword arguments:

  • name: name of the output file, excluding .md; name is also used to name all the @example blocks, and to replace @__NAME__. Defaults to the filename of inputfile.
  • preprocess, postprocess: custom pre- and post-processing functions, see the Custom pre- and post-processing section of the manual. Defaults to identity.
  • documenter: boolean that tells if the output is intended to use with Documenter.jl. Defaults to true. See the manual section on Interaction with Documenter.
  • codefence: A Pair of opening and closing code fence. Defaults to
    "```@example $(name)" => "```"
    if documenter = true and
    "```julia" => "```"
    if documenter = false.
  • credit: boolean that controls the addition of This file was generated with Literate.jl ... to the bottom of the page. If you find Literate.jl useful then feel free to keep this to the default, which is true.
source

4.2. Notebook Output

The (default) notebook output of the source snippet can be seen here: notebook.ipynb.

We note that lines starting with # are placed in markdown cells, and the code lines have been placed in code cells. By default the notebook is also executed and output cells populated. The current working directory is set to the specified output directory the notebook is executed. Some of the output rendering can be controlled with keyword arguments to Literate.notebook:

Literate.notebook — Function
Literate.notebook(inputfile, outputdir; kwargs...)

Generate a notebook from inputfile and write the result to outputdir.

Keyword arguments:

  • name: name of the output file, excluding .ipynb. name is also used to replace @__NAME__. Defaults to the filename of inputfile.
  • preprocess, postprocess: custom pre- and post-processing functions, see the Custom pre- and post-processing section of the manual. Defaults to identity.
  • execute: a boolean deciding if the generated notebook should also be executed or not. Defaults to true. The current working directory is set to outputdir when executing the notebook.
  • documenter: boolean that says if the source contains Documenter.jl specific things to filter out during notebook generation. Defaults to true. See the the manual section on Interaction with Documenter.
  • credit: boolean that controls the addition of This file was generated with Literate.jl ... to the bottom of the page. If you find Literate.jl useful then feel free to keep this to the default, which is true.
source

Notebook metadata

Jupyter notebook cells (both code cells and markdown cells) can contain metadata. This is enabled in Literate by the %% token, similar to Jupytext. The format is as follows

%% optional ignored text [type] {optional metadata JSON}

Cell metadata can, for example, be used for nbgrader and the reveal.js notebook extension RISE.

4.3. Script Output

The (default) script output of the source snippet above is as follows

x = 1//3
 
 y = 2//5
 
-z = x + y

We note that lines starting with # are removed and only the code lines have been kept. Some of the output rendering can be controlled with keyword arguments to Literate.script:

Literate.script — Function
Literate.script(inputfile, outputdir; kwargs...)

Generate a plain script file from inputfile and write the result to outputdir.

Keyword arguments:

  • name: name of the output file, excluding .jl. name is also used to replace @__NAME__. Defaults to the filename of inputfile.
  • preprocess, postprocess: custom pre- and post-processing functions, see the Custom pre- and post-processing section of the manual. Defaults to identity.
  • documenter: boolean that says if the source contains Documenter.jl specific things to filter out during script generation. Defaults to true. See the the manual section on Interaction with Documenter.
  • keep_comments: boolean that, if set to true, keeps markdown lines as comments in the output script. Defaults to false.
  • credit: boolean that controls the addition of This file was generated with Literate.jl ... to the bottom of the page. If you find Literate.jl useful then feel free to keep this to the default, which is true.
source
+z = x + y

We note that lines starting with # are removed and only the code lines have been kept. Some of the output rendering can be controlled with keyword arguments to Literate.script:

Literate.script — Function
Literate.script(inputfile, outputdir; kwargs...)

Generate a plain script file from inputfile and write the result to outputdir.

Keyword arguments:

  • name: name of the output file, excluding .jl. name is also used to replace @__NAME__. Defaults to the filename of inputfile.
  • preprocess, postprocess: custom pre- and post-processing functions, see the Custom pre- and post-processing section of the manual. Defaults to identity.
  • documenter: boolean that says if the source contains Documenter.jl specific things to filter out during script generation. Defaults to true. See the the manual section on Interaction with Documenter.
  • keep_comments: boolean that, if set to true, keeps markdown lines as comments in the output script. Defaults to false.
  • credit: boolean that controls the addition of This file was generated with Literate.jl ... to the bottom of the page. If you find Literate.jl useful then feel free to keep this to the default, which is true.
source
diff --git a/dev/pipeline/index.html b/dev/pipeline/index.html index 5e8c569..895abf4 100644 --- a/dev/pipeline/index.html +++ b/dev/pipeline/index.html @@ -29,4 +29,4 @@ x = 1 // 3 y = 2 // 5

Chunk #3:

When adding `x` and `y` together we obtain a new rational number:

Chunk #4:

z = x + y

It is then up to the Document generation step to decide how these chunks should be treated.

Custom control over chunk splits

Sometimes it is convenient to be able to manually control how the chunks are split. For example, if you want to split a block of code into two, such that they end up in two different @example blocks or notebook cells. The #- token can be used for this purpose. All lines starting with #- are used as "chunk-splitters":

x = 1 // 3
 y = 2 // 5
 #-
-z = x + y

The example above would result in two consecutive code-chunks.

Tip

The rest of the line, after #-, is discarded, so it is possible to use e.g. #------------- as a chunk splitter, which may make the source code more readable.

It is also possible to use #+ as a chunk splitter. The difference between #+ and #- is that #+ enables Documenter's "continued"-blocks, see the Documenter manual.

3.3. Document generation

After the parsing it is time to generate the output. What is done in this step is very different depending on the output target, and it is describe in more detail in the Output format sections: Markdown Output, Notebook Output and Script Output. Using the default settings, the following is happening:

3.4. Post-processing

When the document is generated the user, again, has the option to hook-into the generation with a custom post-processing function. The reason is that one might want to change things that are only visible in the rendered document. See Custom pre- and post-processing.

3.5. Writing to file

The last step of the generation is writing to file. The result is written to $(outputdir)/$(name)(.md|.ipynb|.jl) where outputdir is the output directory supplied by the user (for example docs/generated), and name is a user supplied filename. It is recommended to add the output directory to .gitignore since the idea is that the generated documents will be generated as part of the build process rather than beeing files in the repo.

+z = x + y

The example above would result in two consecutive code-chunks.

Tip

The rest of the line, after #-, is discarded, so it is possible to use e.g. #------------- as a chunk splitter, which may make the source code more readable.

It is also possible to use #+ as a chunk splitter. The difference between #+ and #- is that #+ enables Documenter's "continued"-blocks, see the Documenter manual.

3.3. Document generation

After the parsing it is time to generate the output. What is done in this step is very different depending on the output target, and it is describe in more detail in the Output format sections: Markdown Output, Notebook Output and Script Output. Using the default settings, the following is happening:

3.4. Post-processing

When the document is generated the user, again, has the option to hook-into the generation with a custom post-processing function. The reason is that one might want to change things that are only visible in the rendered document. See Custom pre- and post-processing.

3.5. Writing to file

The last step of the generation is writing to file. The result is written to $(outputdir)/$(name)(.md|.ipynb|.jl) where outputdir is the output directory supplied by the user (for example docs/generated), and name is a user supplied filename. It is recommended to add the output directory to .gitignore since the idea is that the generated documents will be generated as part of the build process rather than beeing files in the repo.

diff --git a/dev/search/index.html b/dev/search/index.html index fbbfa0d..7252778 100644 --- a/dev/search/index.html +++ b/dev/search/index.html @@ -1,2 +1,2 @@ -Search · Literate.jl

Loading search...

    +Search · Literate.jl

    Loading search...