From 14ee8719a53a8277014309ff745277435b1f8588 Mon Sep 17 00:00:00 2001 From: zeptodoctor <44736852+zeptodoctor@users.noreply.github.com> Date: Tue, 26 Nov 2019 14:10:58 +0000 Subject: [PATCH] build based on f1ca792 --- dev/customprocessing/index.html | 2 +- dev/documenter/index.html | 2 +- dev/fileformat/index.html | 2 +- dev/generated/example.ipynb | 168 +++++++++++++++---------------- dev/generated/example/index.html | 86 ++++++++-------- dev/generated/name/index.html | 2 +- dev/index.html | 2 +- dev/outputformats/index.html | 4 +- dev/pipeline/index.html | 2 +- dev/search/index.html | 2 +- 10 files changed, 136 insertions(+), 136 deletions(-) diff --git a/dev/customprocessing/index.html b/dev/customprocessing/index.html index cdd3176..3b1e0df 100644 --- a/dev/customprocessing/index.html +++ b/dev/customprocessing/index.html @@ -26,4 +26,4 @@ include("file2.jl")

Let's say we have saved this fi end return str end

(of course replace included with your respective files)

Finally, you simply pass this function to e.g. Literate.markdown as

Literate.markdown("examples.jl", "path/to/save/markdown";
-                  name = "markdown_file_name", preprocess = replace_includes)

and you will see that in the final output file (here markdown_file_name.md) the include statements are replaced with the actual code to be included!

This approach is used for generating the examples in the documentation of the TimeseriesPrediction.jl package. The example files, included together in the stexamples.jl file, are processed by literate via this make.jl file to generate the markdown and code cells of the documentation.

+ name = "markdown_file_name", preprocess = replace_includes)

and you will see that in the final output file (here markdown_file_name.md) the include statements are replaced with the actual code to be included!

This approach is used for generating the examples in the documentation of the TimeseriesPrediction.jl package. The example files, included together in the stexamples.jl file, are processed by literate via this make.jl file to generate the markdown and code cells of the documentation.

diff --git a/dev/documenter/index.html b/dev/documenter/index.html index f111f32..712662b 100644 --- a/dev/documenter/index.html +++ b/dev/documenter/index.html @@ -9,4 +9,4 @@ EditURL = "$(relpath(inputfile, outputdir))" \int f dx ```is replaced with notebook compatible
$$
 \int f dx
-$$

Literate.script:

+$$

Literate.script:

diff --git a/dev/fileformat/index.html b/dev/fileformat/index.html index 4906163..e76de30 100644 --- a/dev/fileformat/index.html +++ b/dev/fileformat/index.html @@ -15,4 +15,4 @@ z = x + y

In the lines starting with # we can use re #md # Literate.notebook #md # Literate.markdown #md # ```

The lines in the example above would be filtered out in the preprocessing step, unless we are generating a markdown file. When generating a markdown file we would simple remove the leading #md from the lines. Beware that the space after the tag is also removed.

The #src token can also be placed at the end of a line. This is to make it possible to have code lines exclusive to the source code, and not just comment lines. For example, if the source file is included in the test suite we might want to add a @test at the end without this showing up in the outputs:

using Test                      #src
-@test result == expected_result #src

2.3. Default Replacements

The following convenience "macros"/source placeholders are always expanded:

+@test result == expected_result #src

2.3. Default Replacements

The following convenience "macros"/source placeholders are always expanded:

diff --git a/dev/generated/example.ipynb b/dev/generated/example.ipynb index bcafe59..48cf538 100644 --- a/dev/generated/example.ipynb +++ b/dev/generated/example.ipynb @@ -205,114 +205,114 @@ "\n", "\n", "\n", - " \n", + " \n", " \n", " \n", "\n", - "\n", "\n", - " \n", + " \n", " \n", " \n", "\n", - "\n", "\n", - " \n", + " \n", " \n", " \n", "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", + "\n", "0\n", "\n", - "\n", + "\n", "5\n", "\n", - "\n", + "\n", "10\n", "\n", - "\n", + "\n", "15\n", "\n", - "\n", + "\n", "-1.0\n", "\n", - "\n", + "\n", "-0.5\n", "\n", - "\n", + "\n", "0.0\n", "\n", - "\n", + "\n", "0.5\n", "\n", - "\n", + "\n", "1.0\n", "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", + "\n", "y1\n", "\n", - "\n", - "\n", + "\n", "y2\n", "\n", "\n" @@ -542,114 +542,114 @@ "\n", "\n", "\n", - " \n", + " \n", " \n", " \n", "\n", - "\n", "\n", - " \n", + " \n", " \n", " \n", "\n", - "\n", "\n", - " \n", + " \n", " \n", " \n", "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", + "\n", "0\n", "\n", - "\n", + "\n", "5\n", "\n", - "\n", + "\n", "10\n", "\n", - "\n", + "\n", "15\n", "\n", - "\n", + "\n", "-1.0\n", "\n", - "\n", + "\n", "-0.5\n", "\n", - "\n", + "\n", "0.0\n", "\n", - "\n", + "\n", "0.5\n", "\n", - "\n", + "\n", "1.0\n", "\n", - "\n", - "\n", - "\n", - "\n", - "\n", - "\n", + "\n", "y1\n", "\n", - "\n", - "\n", + "\n", "y2\n", "\n", "\n" diff --git a/dev/generated/example/index.html b/dev/generated/example/index.html index 780f863..c97b759 100644 --- a/dev/generated/example/index.html +++ b/dev/generated/example/index.html @@ -16,114 +16,114 @@ y2 = cos.(x) plot(x, [y1, y2]) - + - - + - - + - - - - - - - - - - - - - - - - - - - - - + 0 - + 5 - + 10 - + 15 - + -1.0 - + -0.5 - + 0.0 - + 0.5 - + 1.0 - - - - - - + y1 - - + y2 -

Custom processing

It is possible to give Literate custom pre- and post-processing functions. For example, here we insert two placeholders, which we will replace with something else at time of generation. We have here replaced our placeholders with z and 1.0 + 2.0im:

z = 1.0 + 2.0im
1.0 + 2.0im

Documenter.jl interaction

In the source file it is possible to use Documenter.jl style references, such as @ref and @id. These will be filtered out in the notebook output. For example, here is a link, but it is only visible as a link if you are reading the markdown output. We can also use equations:

\[\int_\Omega \nabla v \cdot \nabla u\ \mathrm{d}\Omega = \int_\Omega v f\ \mathrm{d}\Omega\]

using Documenters math syntax. Documenters syntax is automatically changed to \begin{equation} ... \end{equation} in the notebook output to display correctly.

This page was generated using Literate.jl.

+

Custom processing

It is possible to give Literate custom pre- and post-processing functions. For example, here we insert two placeholders, which we will replace with something else at time of generation. We have here replaced our placeholders with z and 1.0 + 2.0im:

z = 1.0 + 2.0im
1.0 + 2.0im

Documenter.jl interaction

In the source file it is possible to use Documenter.jl style references, such as @ref and @id. These will be filtered out in the notebook output. For example, here is a link, but it is only visible as a link if you are reading the markdown output. We can also use equations:

\[\int_\Omega \nabla v \cdot \nabla u\ \mathrm{d}\Omega = \int_\Omega v f\ \mathrm{d}\Omega\]

using Documenters math syntax. Documenters syntax is automatically changed to \begin{equation} ... \end{equation} in the notebook output to display correctly.

This page was generated using Literate.jl.

diff --git a/dev/generated/name/index.html b/dev/generated/name/index.html index 0fb1bf8..b9a242e 100644 --- a/dev/generated/name/index.html +++ b/dev/generated/name/index.html @@ -1,2 +1,2 @@ -Rational numbers · Literate.jl
Edit on GitHub

Rational numbers

In julia rational numbers can be constructed with the // operator. Lets define two rational numbers, x and y:

x = 1//3
1//3
y = 2//5
2//5

When adding x and y together we obtain a new rational number:

z = x + y
11//15
+Rational numbers · Literate.jl
Edit on GitHub

Rational numbers

In julia rational numbers can be constructed with the // operator. Lets define two rational numbers, x and y:

x = 1//3
1//3
y = 2//5
2//5

When adding x and y together we obtain a new rational number:

z = x + y
11//15
diff --git a/dev/index.html b/dev/index.html index 378e9b8..3c99fc1 100644 --- a/dev/index.html +++ b/dev/index.html @@ -1,2 +1,2 @@ -1. Introduction · Literate.jl
Edit on GitHub

1. Introduction

Welcome to the documentation for Literate – a simplistic package for Literate Programming.

What?

Literate is a package that generates markdown pages (for e.g. Documenter.jl), and Jupyter notebooks, from the same source file. There is also an option to "clean" the source from all metadata, and produce a pure Julia script.

The main design goal is simplicity. It should be simple to use, and the syntax should be simple. In short, all you have to do is to write a commented julia script!

The public interface consists mainly of three functions, all of which take the same script file as input, but generate different output:

Why?

Examples are (probably) the best way to showcase your awesome package, and examples are often the best way for a new user to learn how to use it. It is therefore important that the documentation of your package contains examples for users to read and study. However, people are different, and we all prefer different ways of trying out a new package. Some people wants to RTFM, others want to explore the package interactively in, for example, a notebook, and some people wants to study the source code. The aim of Literate is to make it easy to give the user all of these options, while still keeping maintenance to a minimum.

It is quite common that packages have "example notebooks" to showcase the package. Notebooks are great for showcasing a package, but they are not so great with version control, like git. The reason being that a notebook is a very "rich" format since it contains output and other metadata. Changes to the notebook thus result in large diffs, which makes it harder to review the actual changes.

It is also common that packages include examples in the documentation, for example by using Documenter.jl @example-blocks. This is also great, but it is not quite as interactive as a notebook, for the users who prefer that.

Literate tries to solve the problems above by creating the output as a part of the doc build. Literate generates the output based on a single source file which makes it easier to maintain, test, and keep the manual and your example notebooks in sync.

+1. Introduction · Literate.jl
Edit on GitHub

1. Introduction

Welcome to the documentation for Literate – a simplistic package for Literate Programming.

What?

Literate is a package that generates markdown pages (for e.g. Documenter.jl), and Jupyter notebooks, from the same source file. There is also an option to "clean" the source from all metadata, and produce a pure Julia script.

The main design goal is simplicity. It should be simple to use, and the syntax should be simple. In short, all you have to do is to write a commented julia script!

The public interface consists mainly of three functions, all of which take the same script file as input, but generate different output:

Why?

Examples are (probably) the best way to showcase your awesome package, and examples are often the best way for a new user to learn how to use it. It is therefore important that the documentation of your package contains examples for users to read and study. However, people are different, and we all prefer different ways of trying out a new package. Some people wants to RTFM, others want to explore the package interactively in, for example, a notebook, and some people wants to study the source code. The aim of Literate is to make it easy to give the user all of these options, while still keeping maintenance to a minimum.

It is quite common that packages have "example notebooks" to showcase the package. Notebooks are great for showcasing a package, but they are not so great with version control, like git. The reason being that a notebook is a very "rich" format since it contains output and other metadata. Changes to the notebook thus result in large diffs, which makes it harder to review the actual changes.

It is also common that packages include examples in the documentation, for example by using Documenter.jl @example-blocks. This is also great, but it is not quite as interactive as a notebook, for the users who prefer that.

Literate tries to solve the problems above by creating the output as a part of the doc build. Literate generates the output based on a single source file which makes it easier to maintain, test, and keep the manual and your example notebooks in sync.

diff --git a/dev/outputformats/index.html b/dev/outputformats/index.html index cb3bc67..f52aa57 100644 --- a/dev/outputformats/index.html +++ b/dev/outputformats/index.html @@ -31,8 +31,8 @@ When adding `x` and `y` together we obtain a new rational number: ```@example name z = x + y -```

We note that lines starting with # are printed as regular markdown, and the code lines have been wrapped in @example blocks. We also note that an @meta block have been added, that sets the EditURL variable. This is used by Documenter to redirect the "Edit on GitHub" link for the page, see Interaction with Documenter.

See the section about Configuration for how to configure the behavior and resulting output of Literate.markdown.

Literate.markdown — Function
Literate.markdown(inputfile, outputdir; config::Dict=Dict(), kwargs...)

Generate a markdown file from inputfile and write the result to the directory outputdir.

See the manual section on Configuration for documentation of possible configuration with config and other keyword arguments.

source

4.2. Notebook Output

Notebook output is generated by Literate.notebook. The (default) notebook output of the source snippet can be seen here: notebook.ipynb.

We note that lines starting with # are placed in markdown cells, and the code lines have been placed in code cells. By default the notebook is also executed and output cells populated. The current working directory is set to the specified output directory the notebook is executed.

See the section about Configuration for how to configure the behavior and resulting output of Literate.notebook.

Literate.notebook — Function
Literate.notebook(inputfile, outputdir; config::Dict=Dict(), kwargs...)

Generate a notebook from inputfile and write the result to outputdir.

See the manual section on Configuration for documentation of possible configuration with config and other keyword arguments.

source

Notebook metadata

Jupyter notebook cells (both code cells and markdown cells) can contain metadata. This is enabled in Literate by the %% token, similar to Jupytext. The format is as follows

%% optional ignored text [type] {optional metadata JSON}

Cell metadata can, for example, be used for nbgrader and the reveal.js notebook extension RISE.

4.3. Script Output

Script output is generated by Literate.script. The (default) script output of the source snippet above is as follows:

x = 1//3
+```

We note that lines starting with # are printed as regular markdown, and the code lines have been wrapped in @example blocks. We also note that an @meta block have been added, that sets the EditURL variable. This is used by Documenter to redirect the "Edit on GitHub" link for the page, see Interaction with Documenter.

See the section about Configuration for how to configure the behavior and resulting output of Literate.markdown.

Literate.markdown — Function
Literate.markdown(inputfile, outputdir; config::Dict=Dict(), kwargs...)

Generate a markdown file from inputfile and write the result to the directory outputdir.

See the manual section on Configuration for documentation of possible configuration with config and other keyword arguments.

source

4.2. Notebook Output

Notebook output is generated by Literate.notebook. The (default) notebook output of the source snippet can be seen here: notebook.ipynb.

We note that lines starting with # are placed in markdown cells, and the code lines have been placed in code cells. By default the notebook is also executed and output cells populated. The current working directory is set to the specified output directory the notebook is executed.

See the section about Configuration for how to configure the behavior and resulting output of Literate.notebook.

Literate.notebook — Function
Literate.notebook(inputfile, outputdir; config::Dict=Dict(), kwargs...)

Generate a notebook from inputfile and write the result to outputdir.

See the manual section on Configuration for documentation of possible configuration with config and other keyword arguments.

source

Notebook metadata

Jupyter notebook cells (both code cells and markdown cells) can contain metadata. This is enabled in Literate by the %% token, similar to Jupytext. The format is as follows

%% optional ignored text [type] {optional metadata JSON}

Cell metadata can, for example, be used for nbgrader and the reveal.js notebook extension RISE.

4.3. Script Output

Script output is generated by Literate.script. The (default) script output of the source snippet above is as follows:

x = 1//3
 
 y = 2//5
 
-z = x + y

We note that lines starting with # are removed and only the code lines have been kept.

See the section about Configuration for how to configure the behavior and resulting output of Literate.script.

Literate.script — Function
Literate.script(inputfile, outputdir; config::Dict=Dict(), kwargs...)

Generate a plain script file from inputfile and write the result to outputdir.

See the manual section on Configuration for documentation of possible configuration with config and other keyword arguments.

source

4.4. Configuration

The behavior of Literate.markdown, Literate.notebook and Literate.script can be configured by keyword arguments. There are two ways to do this; pass config::Dict as a keyword argument, or pass individual keyword arguments.

Configuration precedence

Individual keyword arguments takes precedence over the config dictionary, so for e.g. Literate.markdown(...; config = Dict("name" => "hello"), name = "world") the resulting configuration for name will be "world". Both individual keyword arguments and the config dictionary takes precedence over the default.

Available configurations with description and default values are given in the reference for Literate.DEFAULT_CONFIGURATION just below.

Literate.DEFAULT_CONFIGURATION — Constant
DEFAULT_CONFIGURATION

Default configuration for Literate.markdown, Literate.notebook and [Literate.script] which is used for everything not specified by the user. See the manual section about Configuration for more information.

Configuration keyDescriptionDefault valueComment
nameName of the output file (excluding file extension).filename(inputfile)
preprocessCustom preprocessing function mapping String to String.identitySee Custom pre- and post-processing.
postprocessCustom preprocessing function mapping String to String.identitySee Custom pre- and post-processing.
documenterBoolean signaling that the source contains Documenter.jl elements.trueSee Interaction with Documenter.
creditBoolean for controlling the addition of This file was generated with Literate.jl ... to the bottom of the page. If you find Literate.jl useful then feel free to keep this.true
keep_commentsWhen true, keeps markdown lines as comments in the output script.falseOnly applicable for Literate.script.
codefencePair containing opening and closing fence for wrapping code blocks."```julia" => "```"If documenter is true the default is "```@example"=>"```".
executeWhether to execute and capture the output.trueOnly applicable for Literate.notebook.
devurlURL for "in-development" docs."dev"See Documenter docs. Unused if repo_root_url/nbviewer_root_url/binder_root_url are set.
repo_root_urlURL to the root of the repository.-Determined automatically on Travis CI, GitHub Actions and GitLab CI. Used for @__REPO_ROOT_URL__.
nbviewer_root_urlURL to the root of the repository as seen on nbviewer.-Determined automatically on Travis CI, GitHub Actions and GitLab CI. Used for @__NBVIEWER_ROOT_URL__.
binder_root_urlURL to the root of the repository as seen on mybinder.-Determined automatically on Travis CI, GitHub Actions and GitLab CI. Used for @__BINDER_ROOT_URL__.
repo_root_pathFilepath to the root of the repository.-Determined automatically on Travis CI, GitHub Actions and GitLab CI. Used for computing Documenters EditURL.
source
+z = x + y

We note that lines starting with # are removed and only the code lines have been kept.

See the section about Configuration for how to configure the behavior and resulting output of Literate.script.

Literate.script — Function
Literate.script(inputfile, outputdir; config::Dict=Dict(), kwargs...)

Generate a plain script file from inputfile and write the result to outputdir.

See the manual section on Configuration for documentation of possible configuration with config and other keyword arguments.

source

4.4. Configuration

The behavior of Literate.markdown, Literate.notebook and Literate.script can be configured by keyword arguments. There are two ways to do this; pass config::Dict as a keyword argument, or pass individual keyword arguments.

Configuration precedence

Individual keyword arguments takes precedence over the config dictionary, so for e.g. Literate.markdown(...; config = Dict("name" => "hello"), name = "world") the resulting configuration for name will be "world". Both individual keyword arguments and the config dictionary takes precedence over the default.

Available configurations with description and default values are given in the reference for Literate.DEFAULT_CONFIGURATION just below.

Literate.DEFAULT_CONFIGURATION — Constant
DEFAULT_CONFIGURATION

Default configuration for Literate.markdown, Literate.notebook and [Literate.script] which is used for everything not specified by the user. See the manual section about Configuration for more information.

Configuration keyDescriptionDefault valueComment
nameName of the output file (excluding file extension).filename(inputfile)
preprocessCustom preprocessing function mapping String to String.identitySee Custom pre- and post-processing.
postprocessCustom preprocessing function mapping String to String.identitySee Custom pre- and post-processing.
documenterBoolean signaling that the source contains Documenter.jl elements.trueSee Interaction with Documenter.
creditBoolean for controlling the addition of This file was generated with Literate.jl ... to the bottom of the page. If you find Literate.jl useful then feel free to keep this.true
keep_commentsWhen true, keeps markdown lines as comments in the output script.falseOnly applicable for Literate.script.
codefencePair containing opening and closing fence for wrapping code blocks."```julia" => "```"If documenter is true the default is "```@example"=>"```".
executeWhether to execute and capture the output.trueOnly applicable for Literate.notebook.
devurlURL for "in-development" docs."dev"See Documenter docs. Unused if repo_root_url/nbviewer_root_url/binder_root_url are set.
repo_root_urlURL to the root of the repository.-Determined automatically on Travis CI, GitHub Actions and GitLab CI. Used for @__REPO_ROOT_URL__.
nbviewer_root_urlURL to the root of the repository as seen on nbviewer.-Determined automatically on Travis CI, GitHub Actions and GitLab CI. Used for @__NBVIEWER_ROOT_URL__.
binder_root_urlURL to the root of the repository as seen on mybinder.-Determined automatically on Travis CI, GitHub Actions and GitLab CI. Used for @__BINDER_ROOT_URL__.
repo_root_pathFilepath to the root of the repository.-Determined automatically on Travis CI, GitHub Actions and GitLab CI. Used for computing Documenters EditURL.
source
diff --git a/dev/pipeline/index.html b/dev/pipeline/index.html index d0773e9..e8c0676 100644 --- a/dev/pipeline/index.html +++ b/dev/pipeline/index.html @@ -29,4 +29,4 @@ x = 1 // 3 y = 2 // 5

Chunk #3:

When adding `x` and `y` together we obtain a new rational number:

Chunk #4:

z = x + y

It is then up to the Document generation step to decide how these chunks should be treated.

Custom control over chunk splits

Sometimes it is convenient to be able to manually control how the chunks are split. For example, if you want to split a block of code into two, such that they end up in two different @example blocks or notebook cells. The #- token can be used for this purpose. All lines starting with #- are used as "chunk-splitters":

x = 1 // 3
 y = 2 // 5
 #-
-z = x + y

The example above would result in two consecutive code-chunks.

Tip

The rest of the line, after #-, is discarded, so it is possible to use e.g. #------------- as a chunk splitter, which may make the source code more readable.

It is also possible to use #+ as a chunk splitter. The difference between #+ and #- is that #+ enables Documenter's "continued"-blocks, see the Documenter manual.

3.3. Document generation

After the parsing it is time to generate the output. What is done in this step is very different depending on the output target, and it is describe in more detail in the Output format sections: Markdown Output, Notebook Output and Script Output. Using the default settings, the following is happening:

3.4. Post-processing

When the document is generated the user, again, has the option to hook-into the generation with a custom post-processing function. The reason is that one might want to change things that are only visible in the rendered document. See Custom pre- and post-processing.

3.5. Writing to file

The last step of the generation is writing to file. The result is written to $(outputdir)/$(name)(.md|.ipynb|.jl) where outputdir is the output directory supplied by the user (for example docs/generated), and name is a user supplied filename. It is recommended to add the output directory to .gitignore since the idea is that the generated documents will be generated as part of the build process rather than beeing files in the repo.

+z = x + y

The example above would result in two consecutive code-chunks.

Tip

The rest of the line, after #-, is discarded, so it is possible to use e.g. #------------- as a chunk splitter, which may make the source code more readable.

It is also possible to use #+ as a chunk splitter. The difference between #+ and #- is that #+ enables Documenter's "continued"-blocks, see the Documenter manual.

3.3. Document generation

After the parsing it is time to generate the output. What is done in this step is very different depending on the output target, and it is describe in more detail in the Output format sections: Markdown Output, Notebook Output and Script Output. Using the default settings, the following is happening:

3.4. Post-processing

When the document is generated the user, again, has the option to hook-into the generation with a custom post-processing function. The reason is that one might want to change things that are only visible in the rendered document. See Custom pre- and post-processing.

3.5. Writing to file

The last step of the generation is writing to file. The result is written to $(outputdir)/$(name)(.md|.ipynb|.jl) where outputdir is the output directory supplied by the user (for example docs/generated), and name is a user supplied filename. It is recommended to add the output directory to .gitignore since the idea is that the generated documents will be generated as part of the build process rather than beeing files in the repo.

diff --git a/dev/search/index.html b/dev/search/index.html index b521f0f..b1fbed4 100644 --- a/dev/search/index.html +++ b/dev/search/index.html @@ -1,2 +1,2 @@ -Search · Literate.jl

Loading search...

    +Search · Literate.jl

    Loading search...