Update and use Weave

[ChrisRackauckas]

No description provided.

[ChrisRackauckas]

@YingboMa can you check the outer solar system one?

[ChrisRackauckas]

The Unitful notebook breaks Weave.jl: JunoLab/Weave.jl#175

[ChrisRackauckas]

minmax segfaults

[ChrisRackauckas]

minmax fixed

[ChrisRackauckas]

Weave's notebook output is all kinds of wonky right now: JunoLab/Weave.jl#173 . I'll leave one in this PR as a reference.

[ChrisRackauckas]

The Measurements notebook runs into a missing method for latex for Weave.jl.

[ChrisRackauckas]

## Viewing the Notebooks Locally

To view the notebooks locally and interact with the contents, use the following
commands (requires [IJulia](https://github.com/JuliaLang/IJulia.jl)):

```julia
using Pkg
#Pkg.add("IJulia") # Need to do this the first time to install IJulia!
]add "https://github.com/JuliaDiffEq/DiffEqTutorials.jl"
using IJulia, DiffEqTutorials
notebook(dir = joinpath(dirname(pathof(DiffEqTutorials)),".."))

is removed. When Weave fixes the notebook generation we can make a simpler way to run it.

[ChrisRackauckas]

All other issues will get their own issue.

[HenriLaurie]

Way cool. Tested the Introduction subdirectory, herewith some repair notes (none that I can fix, I think):

• in the ode_introduction notebook:
  --- it says the interpolation reports a fourth order, in the cell immediately prior to the one with sol(0.45), but the message says third-order Hermite. I guess this might mean cubic interpolation, with fourth order errors, but it reads a little oddly.
  --- there is no difference for the saveat=[0.2, 0.7, 0.9] without and then with save_end = false, it appears the default is now = false
  --- LaTeX isn't rendered in the presentation of the Lorenz equations, appears with \frac etc all visible, this also happens later, I guess somehow markdown isn't working properly, I guess this is already evident at the top of the page where the title doesn't render correctly, and in the overall style, which is decidedly clunky
  --- at the end, the Appendix doesn't compile, says WEAVE_ARGS not defined.
• in the selecting algorithms notebook:
  --- same problem with Appendix and WEAVE_ARGS as before
• in the optimising code notebook:
  --- I suppose differences are to be expected, but after the very first benchmark, notebook says it took about 11.11 MiB, whereas on my laptop it says 10.63 MiB, it's also closer to 4 ms than to 5 ms for the min time
  --- something odd about broadcast fusion, on my machine at least: after benchmarking plain A+B+C with test(), the notebook goes to test2(), which uses map(), and it is actually slower than the test() as far as the median and the mean are concerned, and the differences are relative large, nearly 25%. In fact only test6!(), test7!() and test8!() are markedly faster than test()
  --- similarly, the comparison of A*B with mul!(D, A, B) is underwhelming
  --- and as before, LaTeX doesn't render, now visible in the typesetting of the PDE
  --- fast_gm!() benchmarks only to 5x faster on my machine, not the 8x claimed in the notebook
  --- same WEAVE_ARGS issue, of course
• callbacks notebook
  --- same rendering problem, of course, and WEAVE_ARGS
• formatting plots notebook
  --- same rendering problem, of course, and WEAVE_ARGS
  --- I notice you took out the \mu m in one of the titles, using instead mm. Am I right that LaTeX inside title strings in Plots is not working anymore?

Anyway, that's all, nothing serious, a markdown issue, a new default behaviour for save_end when there's a saveat, and some perplexing non-gains in performance.

Thanks tremendously!

[HenriLaurie]

Oh, two more things

Cutting and pasting the lines
]add "https://github.com/JuliaDiffEq/DiffEqTutorials.jl"
using DiffEqTutorials
DiffEqTutorials.open_notebooks()
is misleading, doesn't work as is. The first line should be entered without the square bracket but at the pkg> prompt, the other two then back at the julia> prompt

Not all the notebooks on the Tutorials page, when viewed directly, have actually been expanded. For example, the introduction at
http://juliadiffeq.org/DiffEqTutorials.jl/html/introduction/ode_introduction.html
shows up to the reltol=1e-8 plot with the analytical solution dotted over, but then when the saveats start it shows no more output.

[ChrisRackauckas]

Not all the notebooks on the Tutorials page, when viewed directly, have actually been expanded. For example, the introduction at
http://juliadiffeq.org/DiffEqTutorials.jl/html/introduction/ode_introduction.html
shows up to the reltol=1e-8 plot with the analytical solution dotted over, but then when the saveats start it shows no more output.

Interesting. @mpastell do types with show overloads not print out in Weave?

[ChrisRackauckas]

Cutting and pasting the lines is misleading, doesn't work as is. The first line should be entered without the square bracket but at the pkg> prompt, the other two then back at the julia> prompt

These were changed to pkg commands which should be easier to copy/paste.

Thanks for all of the other points. We'll work on them. The WEAVE_ARGS part is just for the other 3 formats... I wish we could delete that cell from the notebook.

fast_gm!() benchmarks only to 5x faster on my machine, not the 8x claimed in the notebook

It will be different on different machines.

• something odd about broadcast fusion, on my machine at least: after benchmarking plain A+B+C with test(), the notebook goes to test2(), which uses map(), and it is actually slower than the test() as far as the median and the mean are concerned, and the differences are relative large, nearly 25%. In fact only test6!(), test7!() and test8!() are markedly faster than test()

JuliaLang/julia#28126 🤷‍♂️ . I think we just keep documenting it correctly and let the bug get fixed.

and as before, LaTeX doesn't render, now visible in the typesetting of the PDE

Yes, we have a fix for that. I'll try and do that now.

* --- it says the interpolation reports a fourth order, in the cell immediately prior to the one with sol(0.45), but the message says third-order Hermite. I guess this might mean cubic interpolation, with fourth order errors, but it reads a little oddly.
  --- there is no difference for the saveat=[0.2, 0.7, 0.9] without and then with save_end = false, it appears the default is now  = false

yes, there were some default changes that happened in the v0.6 -> v1.0 time frame. saveat only saving at the specified time points is one of them (and hopefully much more intuitive!). I'll update this file.

[ChrisRackauckas]

@HenriLaurie for the math blocks see JunoLab/Weave.jl#182

[ChrisRackauckas]

@HenriLaurie for the WEAVE_ARGS part see JunoLab/Weave.jl#183 .

Overall, we are close! In the end, it'll make the tutorials much more maintainable to have them in this format, and we should get good renders pretty soon I think. Maybe for you class it might take a little hand-tweaking if we aren't quick enough.

[mpastell]

Interesting. @mpastell do types with show overloads not print out in Weave?

They should, I'll need to check what the issue is.

[mpastell]

@ChrisRackauckas I changed the display behavior to match jupyter with JunoLab/Weave.jl@e52083c . The objects should now display as expected.

Note that Weave doesn't handle ANSI colors like jupyter does, so e.g. ODEProblem displays as:

[36mODEProblem[0m with uType [36mFloat64 [0m and tType [36mFloat64 [0m.

[ChrisRackauckas]

Should I be setting up a different MIMEType to use with Weave then?

[mpastell]

Defining show for "text/markdown" would fix this.

[mpastell]

I think I can also fix this in Weave by stripping ANSI color codes.

[ChrisRackauckas]

Just a show on that HTML and returning a string makes it not format well. How do you make that output a Markdown object?

[ChrisRackauckas]

Is there a way to make the skipped portion be the entire bottom Markdown block? I wanna skip this whole chunk: https://github.com/JuliaDiffEq/DiffEqTutorials.jl/blob/master/tutorials/introduction/ode_introduction.jmd#L384-L389 . I'm not sure how to output the Markdown to make https://github.com/JuliaDiffEq/DiffEqTutorials.jl/blob/master/src/DiffEqTutorials.jl#L60-L79 do the right thing. Other than that, I think that's the last major Weave question!

[mpastell]

You can use display("text/markdown", *) instead of print and use hold=true chunk option to show all output after the chunk.

[mpastell]

I just added support for this 2 hours ago, so make sure you use latest master

[ChrisRackauckas]

It looks like all of the formatting issues have been fixed @HenriLaurie . Thanks @mpastell ! Looks great and works great. @HenriLaurie , please open issues on any inconsistencies you see in the tutorials themselves.

[mpastell]

Great that everything is working and the tutorials are excellent! Also thanks for reporting all the issues. I’ll tag a new release soon.

[HenriLaurie]

OK, found a few issues that remain:

--- UnitfulPlots doesn't exist as a package as far as my Pkg is concerned
--- Formatting is still an issue: the very first line in each notebook is still something like
title: An Intro to DifferentialEquations.jl, and the LaTeX presentation of Lorenz system etc. still shows marked up, not the formatted text
--- in ode_introduction, still mistakenly claims that saveat saves the start and end values by default
--- Appendix still doesn't render properly

... hm. maybe I should run a full update first?

Ah. There seems to be some sort of Pkg problem:

v1.1) pkg> up
  Updating registry at `~/.julia/registries/General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`
  Updating git-repo `https://github.com/JuliaDiffEq/DiffEqBenchmarks.jl`
  Updating git-repo `https://github.com/JuliaDiffEq/DiffEqTutorials.jl`
 Resolving package versions...
ERROR: path /home/henri/.julia/packages/DiffEqTutorials/SLil6 for package DiffEqTutorials no longer exists. Remove the package or `develop` it at a new path

(v1.1) pkg> remove DiffEqTutorials
  Updating `~/.julia/environments/v1.1/Project.toml`
  [225cb15b] - DiffEqTutorials v0.0.0 #master (https://github.com/JuliaDiffEq/DiffEqTutorials.jl)
  Updating `~/.julia/environments/v1.1/Manifest.toml`
  [225cb15b] - DiffEqTutorials v0.0.0 #master (https://github.com/JuliaDiffEq/DiffEqTutorials.jl)

(v1.1) pkg> add DiffEqTutorials
ERROR: The following package names could not be resolved:
 * DiffEqTutorials (not found in project, manifest or registry)
Please specify by known `name=uuid`.

and after I did this, I simply had trouble with the code as given in the Tutorials Readme, i.e. with

julia> using Pkg

julia> pkg"add https://github.com/JuliaDiffEq/DiffEqTutorials.jl"
  Updating git-repo `https://github.com/JuliaDiffEq/DiffEqTutorials.jl`
 Resolving package versions...
ERROR: Unsatisfiable requirements detected for package UnitfulPlots [2a06ce6d]:
 UnitfulPlots [2a06ce6d] log:
 ├─UnitfulPlots [2a06ce6d] has no known versions!
 └─restricted to versions * by DiffEqTutorials [6d1b261a] — no versions left
   └─DiffEqTutorials [6d1b261a] log:
     ├─possible versions are: 0.1.0 or uninstalled
     └─DiffEqTutorials [6d1b261a] is fixed to version 0.1.0
Stacktrace:
 [1] deps_graph(::Pkg.Types.Context, ::Dict{Base.UUID,String}, ::Dict{Base.UUID,Pkg.Types.VersionSpec}, ::Dict{Base.UUID,Pkg.Types.Fixed}) at /build/julia/src/j

... so now the tutorials are inaccessible

[ChrisRackauckas]

Let's just get rid of UnicodePlots.

[ChrisRackauckas]

New update. UnicodePlots is gone. Everything should install from fresh now, but throwing some errors if you don't have Cuda (but it should still work).

The only things not fixed are the formatting. It does look like the formatting in the notebooks is odd, with the header going after the first block. Also, the appendix isn't being skipped in the notebooks even though it's using {echo=false,skip="notebook"}

[mpastell]

I've fixed the problem with the header on Weave master, I made a stupid coding mistake and only tested output with header block with no content after it...

@ChrisRackauckas chunk options don't work because the syntax is incorrect. Use:

```{julia; echo=false; skip="notebook"}

or

```julia; echo=false; skip="notebook"

[ChrisRackauckas]

Okay cool. The syntax fix worked. The header we can wait for a release (we are using Project/Manifest now so it's easier to just wait on releases).

[HenriLaurie]

Fantastic. Intro section is all good. Formatting seems fine everywhere, noted a few problems (some may be local to my installation):

In type_handling
--- unitful.ipynb: the final plot recipe seems to be off, as the claim about units being reflected in the label doesn't seem to be reflected by the plot
--- number_types.ipynb: DecFP causes StackOverflowError on my machine, and as noted Decimals is not appropriate at the moment

In ode_extras
--- monte_carlo_parameter_estim fails right up front, because DiffEqParamEstim doesn't precompile for me (this replicates in the REPL)

In advanced
--- beeler_reuter gets a long way, but the third-last code cell, where the test for CUDA version is to be defined, fails, with cryptic error message "UndefVarError: BeelerReuterGpu not defined" (I haven't ever done GPU programming, nor has anybody else on this machine, so I cannot guarantee that the error isn't purely local to me and my machine)

[ChrisRackauckas]

--- number_types.ipynb: DecFP causes StackOverflowError on my machine, and as noted Decimals is not appropriate at the moment

Yes, and that's why DecFP is in the "Incompatible Number Types" category right now.

--- beeler_reuter gets a long way, but the third-last code cell, where the test for CUDA version is to be defined, fails, with cryptic error message "UndefVarError: BeelerReuterGpu not defined" (I haven't ever done GPU programming, nor has anybody else on this machine, so I cannot guarantee that the error isn't purely local to me and my machine)

Yes, if you didn't install CUDA then GPU programming will not work.

--- unitful.ipynb: the final plot recipe seems to be off, as the claim about units being reflected in the label doesn't seem to be reflected by the plot

Fixed! I just removed the line about the labels. Using ustrip to avoid UnitfulPlots.jl won't add any labels automatically.

--- monte_carlo_parameter_estim fails right up front, because DiffEqParamEstim doesn't precompile for me (this replicates in the REPL)

Are you using the provided Project/Manifest? If not, are you on the latest version of DiffEqParamEstim? If so, what is your precompilation error?