Julia Programming Language

Tuxedo.jl: Documentation-driven Development

Tuxedo.jl

Dressing up your algorithms with documentation-driven development

Implementing technical and scientific algorithms in code starts with a thorough understanding of the problem that is being solved and how a particular algorithm solves that problem. Tuxedo provides a framework for transforming theory found in publications and technical papers into documented, testable, and usable code.

A proper and well-dressed algorithm implementation starts with naming it and documenting its purpose. Tuxedo provides a macro in the Julia environment that scaffolds documentation, function definition and unit tests.

julia> @tux MyPackage mymethod

Download Details:

Author: Milktrader
Source Code: https://github.com/milktrader/Tuxedo.jl 
License: MIT license

#julia #documentation 

Remark.jl: Create Markdown Presentations From Julia

Remark 

A simple Julia package to create presentations from markdown using Remark.

Example use

To install type the following command in the Julia Pkg REPL

(v1.0) pkg> add Remark

Check out the Remark docs on how to write the markdown for a Remark slideshow. The most important thing is to use --- to separate slides; an example markdown file can be found here.

import Remark

# Generate a presentation (html+markdown) from the markdown template
# and save it in the folder "presentation", which should not exist when this is called.
Remark.generate("presentation")
slideshowdir = Remark.slideshow("presentation",
                                options = Dict("ratio" => "16:9"),
                                title = "My beautiful slides")

# Open presentation in default browser.
Remark.open(slideshowdir)

The presentation folder will also include a make.jl file that can be used to automatically rebuild the presentation (it is enough to include it). Of course, the make.jl file can be customized to fit your needs, e.g., setting keyword arguments of Remark.slideshow or activating a Project.toml file for reproducibility.

It is also possible to use a julia file as a starting point (i.e., index.jl), thanks to the Literate package. As explained in the Literate documentation, add a comment to slides corresponding to markdown. A slide separator is now # --- for example.

To get started, simple do:

import Remark

# Generate a presentation (html+markdown) from the julia template
# and save it in the folder "julia_presentation".
julia_template = joinpath(dirname(dirname(pathof(Remark))), "examples", "julia")
cp(julia_template, "julia_presentation")
slideshowdir = Remark.slideshow("julia_presentation",
                                options = Dict("ratio" => "16:9"),
                                title = "My beautiful slides")

# Open presentation in default browser.
Remark.open(slideshowdir)

External assets, styling, and customization

If you need extra assets, simply add a assets folder inside src. It will be copied in the build folder automatically.

If you want to use a custom stylesheet rather than the default, simply add a style.css file in your src folder. Your overall folder structure would be:

presentation/
├── src/
│   └── index.md (or .jl)
│   └── style.css
│   └── assets/
├── build/ (generated by the package)
│   └── index.html
│   └── style.css
│   └── assets/

The title of the presentation can be customized via the title keyword. Remark.js options can be set using the options keyword. Documenter is run on the markdown by default: to keep the markdown as is, use documenter=false.

Features

The slideshow command creates a slideshow that uses local javascript libraries: the resulting presentation folder can be opened offline.

All features of Documenter are automatically available here, for example use # hide at the end of a line of code for it to not appear in the slideshow (useful for saving plots, see example).

Download Details:

Author: Piever
Source Code: https://github.com/piever/Remark.jl 
License: View license

#julia #markdown 

Roxygen.jl: A Roxygen-like Documentation Package

Roxygen.jl

A draft implementation of a Roxygen-like package for automatically generating documentation from Julia source files.

Basic Format

The following file shows how a Julia file can use Roxygen through specially formatted comments:

#' @@name count
#'
#' @@description
#'
#' Count the number of missing values in every column of an
#' AbstractDataFrame.
#'
#' @@arg adf::AbstractDataFrame An AbstractDataFrame.
#' @@arg skip::Int The index of a column to skip.
#'
#' @@return missing::Vector{Int} The number of missing values in each column.
#' @@return success::Bool Did the operation complete successfully?
#'
#' @@examples
#'
#' df = DataFrame(A = 1:3, B = ["x", "y", "z"])
#' count(df)

function count(adf::AbstractDataFrame, skip::Int)
    return Int[], false
end

#' @@name sum
#'
#' @@description
#'
#' Sum the elements of each column of an AbstractDataFrame.
#'
#' @@arg adf::AbstractDataFrame An AbstractDataFrame.
#'
#' @@return sums::Vector{Float64} The sums of each column's entries.
#'
#' @@examples
#'
#' df = DataFrame(A = 1:3, B = ["x", "y", "z"])
#' sum(df)

function sum(adf::AbstractDataFrame)
    return Float64[]
end

This file contains Julia code as well as comments that use a set of directives, like @@arg and `@@return`` to describe properties of the function being documented. The valid directives and requirements for their use are described below:

• @@name: The name of the function. A name directive usage must look like @@name NAME with whitespace after the directive and a single name after the initial whitespace.
• @@exported: Is the function exported by the package being documented?
• @@description: A short summary of the use and purpose of the function.
• @@arg: A summary of each argument of a function in the order required by the function. Must contain argument name, argument type and a short verbal description.
• @@field: A summary of each field of a composite type in the order required by the type's constructor. Must contain field name, field type and a short verbal description.
• @@return: A summary of each return value of a function in the order returned by the function. Must contain value name, value type and a short verbal description.
• @@examples: A free-form set of examples of the function or type's use.

Usage Example

To generate documentation for all .jl files in the src directory and place this documentation in the doc directory, run the following command:

using Roxygen
roxygenize("src", "doc")

Only files from the source directory that contained Roxygen documentation will give rise to files in the output directory.

Download Details:

Author: johnmyleswhite
Source Code: https://github.com/johnmyleswhite/Roxygen.jl 
License: View license

#julia #documentation 

Juliavm: Julia Version Manager

juliavm

A Julia version manager

JuliaVM is a command-line tool which allows you to easily install, manage, and work with Julia environments and switch between them easily. It's inspired in rvm and nvm.

Install

Clone the repo:

git clone https://github.com/pmargreff/juliavm

Inside the repo provide the right permissions to install:

cd juliavm && chmod u+x install.sh

Run the script for install:

./install.sh

Commands

• juliavm ls-remote - list all remote versions
• juliavm ls - list all locale versions
• juliavm install x.y.z [-ARCHITECTURE] - install x.y.x version, ARCHITECTURE is an optional param
• juliavm use x.y.z [-ARCHITECTURE] - use x.y.x version, ARCHITECTURE is an optional param
• juliavm update - update juliavm with latest resources
• juliavm uninstall [--hard] - uninstall juliavm and all julia versions downloaded inside juliavm, with hard parameter it uninstall all Julia packages, if not pass hard param soft uninstall (doesn't delete Julia major packages) will be used.
• juliavm help - list all available commands

Architectures

• -x64 - unix 64 bits
• -x86 - unix 32 bits
• If you don't pass the architecture unix 64 bits will be used.

Unix (32 and 64 bits) version is supported right now, feel free to add OSX compatibility or use asdf instead.

Download Details:

Author: Pmargreff
Source Code: https://github.com/pmargreff/juliavm 
License: MIT license

#julia #version 

Educational Material and Tutorials for The Turing Language

Turing Tutorials 

This repository contains tutorials on the universal probabilistic programming language Turing.

The tutorials are defined in the tutorials folder. All the outputs are generated automatically from that.

Additional educational materials can be found at StatisticalRethinkingJulia/TuringModels.jl, which contains Turing adaptations of models from Richard McElreath's Statistical Rethinking. It is a highly recommended resource if you are looking for a greater breadth of examples.

Interactive Notebooks

To run the tutorials interactively via Jupyter notebooks, install the package and open the tutorials like:

# Install TuringTutorials
using Pkg
pkg"add https://github.com/TuringLang/TuringTutorials"

# Generate notebooks in subdirectory "notebook"
using TuringTutorials
TuringTutorials.weave(; build=(:notebook,))

# Start Jupyter in "notebook" subdirectory
using IJulia
IJulia.notebook(; dir="notebook")

You can weave the notebooks to a different folder with

TuringTutorials.weave(; build=(:notebook,), out_path_root="my/custom/directory")

Then the notebooks will be generated in the folder my/custom/directory/notebook and you can start Jupyter with

IJulia.notebook(; dir="my/custom/directory/notebook")

Contributing

First of all, make sure that your current directory is TuringTutorials. All of the files are generated from the jmd files in the tutorials folder. So to change a tutorial, change one of the .jmd file in the tutorials folder.

To run the generation process, do for example:

using TuringTutorials
TuringTutorials.weave("00-introduction", "00_introduction.jmd")

To generate all files do:

TuringTutorials.weave()

If you add new tutorials which require new packages, simply updating your local environment will change the project and manifest files. When this occurs, the updated environment files should be included in the PR.

Credits

The structure of this repository is mainly based on SciMLTutorials.jl.

Download Details:

Author: TuringLang
Source Code: https://github.com/TuringLang/TuringTutorials 
License: MIT license

#julia #machinelearning #language 

Weave.jl: Scientific Reports/literate Programming for Julia

Weave  

Weave is a scientific report generator/literate programming tool for the Julia programming language. It resembles Pweave, knitr, R Markdown, and Sweave.

You can write your documentation and code in input document using Markdown, Noweb or ordinal Julia script syntax, and then use weave function to execute code and generate an output document while capturing results and figures.

Current features

• Publish markdown directly to HTML and PDF using Julia or Pandoc
• Execute code as in terminal or in a unit of code chunk
• Capture Plots.jl or Gadfly.jl figures
• Supports various input format: Markdown, Noweb, Jupyter Notebook, and ordinal Julia script
• Conversions between those input formats
• Supports various output document formats: HTML, PDF, GitHub markdown, Jupyter Notebook, MultiMarkdown, Asciidoc and reStructuredText
• Simple caching of results

Citing Weave: Pastell, Matti. 2017. Weave.jl: Scientific Reports Using Julia. The Journal of Open Source Software. http://dx.doi.org/10.21105/joss.00204

Installation

You can install the latest release using Julia package manager:

using Pkg
Pkg.add("Weave")

Usage

using Weave

# add depencies for the example
using Pkg; Pkg.add(["Plots", "DSP"])

filename = normpath(Weave.EXAMPLE_FOLDER, "FIR_design.jmd")
weave(filename, out_path = :pwd)

If you have LaTeX installed you can also weave directly to pdf.

filename = normpath(Weave.EXAMPLE_FOLDER, "FIR_design.jmd")
weave(filename, out_path = :pwd, doctype = "md2pdf")

NOTE: Weave.EXAMPLE_FOLDER just points to examples directory.

Documentation

Documenter.jl with MKDocs generated documentation:

Editor support

Install language-weave to add Weave support to Juno. It allows running code from Weave documents with usual keybindings and allows preview of html and pdf output.

The Julia extension for Visual Studio Code adds Weave support to Visual Studio Code.

Contributing

You can contribute to this package by opening issues on GitHub or implementing things yourself and making a pull request. We'd also appreciate more example documents written using Weave.

Contributors

You can see the list of contributors on GitHub: https://github.com/JunoLab/Weave.jl/graphs/contributors . Thanks for the important additions, fixes and comments.

Example projects using Weave

• DiffEqTutorials.jl uses Weave to output tutorials (.jmd documents) to html, pdf and Jupyter notebooks.
• TuringTutorials uses Weave to convert notebooks to html.

Download Details:

Author: JunoLab
Source Code: https://github.com/JunoLab/Weave.jl 
License: MIT license

#julia #programming #workflow 

CommonMark.jl: A CommonMark-compliant Parser for Julia

CommonMark

A CommonMark-compliant parser for Julia. 

Interface

using CommonMark

Create a markdown parser with the default CommonMark settings and then add footnote syntax to our parser.

parser = Parser()
enable!(parser, FootnoteRule())

Parse some text to an abstract syntax tree from a String:

ast = parser("Hello *world*")

Parse the contents of a source file:

ast = open(parser, "document.md")

Write ast to a string.

body = html(ast)
content = "<head></head><body>$body</body>"

Write to a file.

open("file.tex", "w") do file
    latex(file, ast)
    println(file, "rest of document...")
end

Or write to a buffer, such as stdout.

term(stdout, ast)

Output Formats

Supported output formats are currently:

• html
• latex
• term: colourised and Unicode-formatted for display in a terminal.
• markdown
• notebook: Jupyter notebooks.

Extensions

Extensions can be enabled using the enable! function and disabled using disable!.

Typography

Convert ASCII dashes, ellipses, and quotes to their Unicode equivalents.

enable!(parser, TypographyRule())

Keyword arguments available for TypographyRule are

• double_quotes
• single_quotes
• ellipses
• dashes

which all default to true.

Admonitions

enable!(parser, AdmonitionRule())

Front matter

Fenced blocks at the start of a file containing structured data.

+++
[heading]
content = "..."
+++

The rest of the file...

The block must start on the first line of the file. Supported blocks are:

• ;;; for JSON
• +++ for TOML
• --- for YAML

To enable provide the FrontMatterRule with your choice of parsers for the formats:

using JSON
enable!(parser, FrontMatterRule(json=JSON.Parser.parse))

You can access front matter from a parsed file using frontmatter. As follows.

ast = open(parser, "document.md")
meta = frontmatter(ast)

Footnotes

enable!(parser, FootnoteRule())

Math

Julia-style inline and display maths:

Some ``\LaTeX`` math:

```math
f(a) = \frac{1}{2\pi}\int_{0}^{2\pi} (\alpha+R\cos(\theta))d\theta
```

Enabled with:

enable!(parser, MathRule())

Dollar-style inline and display math is also available using

enable!(parser, DollarMathRule())

Supported syntax:

• single dollar signs surrounding inline math,
• double dollars surrounding a single line paragraph for display math.

For more complex math, such as multiline display math, use the literal block syntax available with MathRule().

Tables

Pipe-style tables, similar to GitHub's tables. Literal | characters that are not wrapped in other syntax such as * must be escaped with a backslash. The number of columns in the table is specified by the second line.

| Column One | Column Two | Column Three |
|:---------- | ---------- |:------------:|
| Row `1`    | Column `2` |              |
| *Row* 2    | **Row** 2  | Column ``|`` |

Rows with more cells than specified have the trailing cells discarded, and rows with less cells are topped up with empty cells.

Enabled with:

enable!(parser, TableRule())

Raw Content

Overload literal syntax to support passing through any type of raw content.

enable!(parser, RawContentRule())

By default RawContentRule will handle inline and block content in HTML and LaTeX formats.

This is raw HTML: `<img src="myimage.jpg">`{=html}.

And here's an HTML block:

```{=html}
<div id="main">
 <div class="article">
```

```{=latex}
\begin{tikzpicture}
\draw[gray, thick] (-1,2) -- (2,-4);
\draw[gray, thick] (-1,-1) -- (2,2);
\filldraw[black] (0,0) circle (2pt) node[anchor=west] {Intersection point};
\end{tikzpicture}
```

This can be used to pass through different complex content that can't be easily handled by CommonMark natively without any loss of expressiveness.

Custom raw content handlers can also be passed through when enabling the rule. The naming scheme is <format>_inline or <format>_block.

enable!(p, RawContentRule(rst_inline=RstInline))

The last example would require the definition of a custom RstInline struct and associated display methods for all supported output types, namely: html, latex, and term. When passing your own keywords to RawContentRule the defaults are not included and must be enabled individually.

Attributes

Block and inline nodes can be tagged with arbitrary metadata in the form of key/value pairs using the AttributeRule extension.

enable!(p, AttributeRule())

Block attributes appear directly above the node that they target:

{#my_id color="red"}
# Heading

This will attach the metadata id="my_id" and color="red" to # Heading.

Inline attributes appear directly after the node that they target:

*Some styled text*{background="green"}.

Which will attach metadata background="green" to the emphasised text Some styled text.

CSS-style shorthand syntax #<name> and .<name> are available to use in place of id="<name>" and class="name". Multiple classes may be specified sequentially.

AttributeRule does not handle writing metadata to particular formats such as HTML or LaTeX. It is up to the implementation of a particular writer format to make use of available metadata itself. The built-in html and latex outputs make use of included attributes. html will include all provided attributes in the output, while latex makes use of only the #<id> attribute.

Citations

Use the following to enable in-text citations and reference list generation:

enable!(p, CitationRule())

Syntax for citations is similar to what is offered by Pandoc. Citations start with @.

Citations can either appear in square brackets [@id], or they can be written as
part of the text like @id. Bracketed citations can contain more than one
citation; separated by semi-colons [@one; @two; and @three].

{#refs}
# References

A reference section that will be populated with a list of all references can be marked using a {#refs} attribute from AttributeRule at the toplevel of the document. The list will be inserted after the node, in this case # References.

Citations and reference lists are formatted following the Chicago Manual of Style. Styling will, in future versions, be customisable using Citation Style Language styles.

The reference data used for citations must be provided in a format matching CSL JSON. Pass this data to CommonMark.jl when writing an AST to a output format.

html(ast, Dict{String,Any}("references" => JSON.parsefile("references.json")))

CSL JSON can be exported easily from reference management software such as Zotero or generated via pandoc-citeproc --bib2json or similar. The references data can be provided by the front matter section of a document so long as the FrontMatterRule has been enabled, though this does require writing your CSL data manually.

Note that the text format of the reference list is not important, and does not have to be JSON data. So long as the shape of the data matches CSL JSON it is valid. Below we use YAML references embedded in the document's front matter:

---
references:
- id: abelson1996
  author:
    - family: Abelson
      given: Harold
    - family: Sussman
      given: Gerald Jay
  edition: 2nd Editon
  event-place: Cambridge
  ISBN: 0-262-01153-0
  issued:
    date-parts:
      - - 1996
  publisher: MIT Press/McGraw-Hill
  publisher-place: Cambridge
  title: Structure and interpretation of computer programs
  type: book
---

Here's a citation [@abelson1996].

{#refs}
# References

Auto Identifiers

Headings within a document can be assigned ids automatically using

enable!(p, AutoIdentifierRule())

Identifiers are determined with CommonMark.slugify, which is based on the algorithm used by Pandoc. Non-unique identifiers are suffixed with a numeric counter and so cannot be considered stable. If you need stable identifiers then you should use AttributeRule to assign stable ids manually.

CommonMark Defaults

Block rules enabled by default in Parser objects:

• AtxHeadingRule()
• BlockQuoteRule()
• FencedCodeBlockRule()
• HtmlBlockRule()
• IndentedCodeBlockRule()
• ListItemRule()
• SetextHeadingRule()
• ThematicBreakRule()

Inline rules enabled by default in Parser objects:

• AsteriskEmphasisRule()
• AutolinkRule()
• HtmlEntityRule()
• HtmlInlineRule()
• ImageRule()
• InlineCodeRule()
• LinkRule()
• UnderscoreEmphasisRule()

These can all be disabled using disable!. Note that disabling some parser rules may result in unexpected results. It is recommended to be conservative in what is disabled.

Note

Until version 1.0.0 the rules listed above are subject to change and should be considered unstable regardless of whether they are exported or not.

Writer Configuration

When writing to an output format configuration data can be provided by:

• passing a Dict{String,Any} to the writer method,
• front matter in the source document using the FrontMatterRule extension.

Front matter takes precedence over the passed Dict.

Notable Variables

Values used to determine template behaviour:

template-engine::Function Used to render standalone document templates.

No default is provided by this package. The template-engine function should follow the interface provided by Mustache.render. It is recommended to use Mustache.jl to provide this functionalilty.

Syntax for opening and closing tags used by CommonMark.jl is ${...}. See the templates in src/writers/templates for usage examples.

<format>.template.file::String Custom template file to use for standalone <format>.

<format>.template.string::String Custom template string to use for standalone <format>.

Generic variables that can be included in templates to customise documents:

abstract::String Summary of the document.

authors::Vector{String} Vector of author names.

date::String Date of file generation.

keywords::Vector{String} Vector of keywords to be included in the document metadata.

lang::String Language of the document.

title::String Title of the document.

subtitle::String Subtitle of the document.

Format-specific variables that should be used only in a particular format's template. They are namespaced to avoid collision with other variables.

html

html.css::Vector{String} Vector of CSS files to include in document.

html.js::Vector{String} Vector of JavaScript files to include in document.

html.header::String String content to add at end of <head>.

html.footer::String String content to add at end of <body>.

latex

latex.documentclass::String Class file to use for document. Default is article.

latex.preamble::String String content to add directly before \begin{document}.

The following are automatically available in document templates.

body::String Main content of the page.

curdir::String Current directory.

outputfile::String Name of file that is being written to. When writing to an in-memory buffer this variable is not defined.

Download Details:

Author: MichaelHatherly
Source Code: https://github.com/MichaelHatherly/CommonMark.jl 
License: View license

#julia #markdown #language 

Publish.jl: A Universal Document Authoring Package for Julia

Publish.jl

A universal document authoring package for Julia.  

Disclaimer

Currently this package should be regarded as experimental --- a proving ground for new features for the Julia documentation ecosystem rather than a mature and proven piece of software. If you need a solution that will definitely work, look at Documenter.jl instead.

This is a package for Julia that provides a general framework for writing prose --- technical documentation is its focus, though it is general enough to be applied to any kind of written document.

Some standout features:

• built-in live server to view your changes in real-time,
• uses a fully-compliant commonmark parser, CommonMark.jl,
• produces HTML and PDF files natively, no LaTeX dependencies to manage yourself,
• publication-quality PDF generation uses tectonic for self-contained, reproducible builds,
• combine markdown files, Jupyter notebooks, and Julia files for your content,
• supports themes to customise both HTML and PDF output,
• and declarative configuration built on top of Julia's Pkg.jl package manager.

Publish can scale from single pages all the way to large cross-referenced multi-project documents.

To jump straight in and begin using Publish run the following in your Julia REPL:

pkg> add Publish

julia> using Publish

julia> serve(Publish)
✓ LiveServer listening on http://localhost:8000/ ...
  (use CTRL+C to shut down)

The above will install Publish, import it, and then start up a local web-server for Publish's own documentation --- the content you're reading right now. Open the link in your web browser and then continue on to the next section, Getting Started.

Download Details:

Author: MichaelHatherly
Source Code: https://github.com/MichaelHatherly/Publish.jl 
License: View license

#julia #markdown #publish 

MicroscopyLabels.jl: Easily Embed annotations in Your Microscopy Image

MicroscopyLabels.jl

Easily embed sane annotations in your microscopy images.

The reality is that you don't need this library, but it can make your life easier if you have microscopy images and want to quickly annotate things. It aims to be sane over being customizable because really it's just a wrapper around FreeTypeAbstraction.jl.

Installation

This package is available through the Julia registry and can be installed via the Julia REPL:

] add MicroscopyLabels

Download Details:

Author: tlnagy
Source Code: https://github.com/tlnagy/MicroscopyLabels.jl 
License: MIT license

#julia #images 

QuizQuestions.jl: Simple Means to Make Basic Web Pages using Markdown

QuizQuestions

A simple means to make basic web pages using Markdown with self-grading quiz questions. Question types are for numeric response, text response (graded with a regular expression), matching, a selection of one from many, or one or more from many. Can be used with Weave, Documenter, quarto, or Pluto.

The package creates show methods for mime type text/html for a few objects that produce HTML showing an input widget with attached javascript code to grade the input once the widget loses focus.

Download Details:

Author: jverzani
Source Code: https://github.com/jverzani/QuizQuestions.jl 
License: MIT license

#julia #questions #markdown 

Simple Development Server with Live-reload Capability for Julia

Live Server for Julia  

This is a simple and lightweight development web-server written in Julia, based on HTTP.jl. It has live-reload capability, i.e. when modifying a file, every browser (tab) currently displaying the corresponding page is automatically refreshed.

LiveServer is inspired from Python's http.server and Node's browsersync.

Installation

To install it in Julia ≥ 1.3, use the package manager with

pkg> add LiveServer

For Julia [1.0, 1.3), you can use LiveServer's version 0.7.4:

pkg> add LiveServer@0.7.4

Make it a shell command

LiveServer is a small package and fast to load with one main functionality (serve), it can be convenient to make it a shell command: (I'm using the name lss here but you could use something else):

alias lss='julia -e "import LiveServer as LS; LS.serve(launch_browser=true)"'

you can then use lss in any directory to show a directory listing in your browser, and if the directory has an index.html then that will be rendered in your browser.

Usage

The main function LiveServer exports is serve which starts listening to the current folder and makes its content available to a browser. The following code creates an example directory and serves it:

julia> using LiveServer
julia> LiveServer.example() # creates an "example/" folder with some files
julia> cd("example")
julia> serve() # starts the local server & the file watching
✓ LiveServer listening on http://localhost:8000/ ...
  (use CTRL+C to shut down)

Open a Browser and go to http://localhost:8000/ to see the content being rendered; try modifying files (e.g. index.html) and watch the changes being rendered immediately in the browser.

In the REPL:

julia> using LiveServer
julia> serve(host="0.0.0.0", port=8001, dir=".") # starts the remote server & the file watching
✓ LiveServer listening on http://0.0.0.0:8001...
  (use CTRL+C to shut down)

In the terminal:

julia -e 'using LiveServer; serve(host="0.0.0.0", port=8001, dir=".")'

Open a browser and go to https://localhost:8001/ to see the rendered content of index.html or, if it doesn't exist, the content of the directory. You can set the port to a custom number. This is similar to the http.server in Python.

Serve docs

servedocs is a convenience function that runs Documenter along with LiveServer to watch your doc files for any changes and render them in your browser when modifications are detected.

Assuming you are in directory/to/YourPackage.jl, that you have a docs/ folder as prescribed by Documenter.jl and LiveServer installed in your global environment, you can run:

$ julia

pkg> activate docs

julia> using YourPackage, LiveServer

julia> servedocs()
[ Info: SetupBuildDirectory: setting up build directory.
[ Info: ExpandTemplates: expanding markdown templates.
...
└ Deploying: ✘
✓ LiveServer listening on http://localhost:8000/ ...
  (use CTRL+C to shut down)

Open a browser and go to http://localhost:8000/ to see your docs being rendered; try modifying files (e.g. docs/index.md) and watch the changes being rendered in the browser.

To run the server with one line of code, run:

$ julia --project=docs -ie 'using YourPackage, LiveServer; servedocs()'

Note: this works with Literate.jl as well. See the docs.

Download Details:

Author: tlienart
Source Code: https://github.com/tlienart/LiveServer.jl 
License: View license

#julia #webserver #websocket 

CellFishing.jl: Fast and Scalable Cell Search tool

CellFishing.jl 🎣

CellFishing.jl (cell finder via hashing) is a tool to find similar cells of query cells based on their transcriptome expression profiles.

Kenta Sato, Koki Tsuyuzaki, Kentaro Shimizu, and Itoshi Nikaido. "CellFishing.jl: an ultrafast and scalable cell search method for single-cell RNA sequencing." Genome Biology, 2019 20:31. https://doi.org/10.1186/s13059-019-1639-x

# Import packages.
using CellFishing
using TableReader

# Load expression profiles of database cells.
# Note: We highly recommend using the Loom format (http://loompy.org/) to
# load expression data, because loading a large matrix in plain text takes
# extremely long time.
data = readtsv("database.txt")  # use readcsv if your file is comma-separated
cellnames = string.(names(data))
featurenames = string.(data[:,1])
counts = Matrix{Int}(data[:,2:end])

# Select features and create an index (or a database).
features = CellFishing.selectfeatures(counts, featurenames)
database = CellFishing.CellIndex(counts, features, metadata=cellnames)

# Save/load the database to/from a file (optional).
# CellFishing.save("database.cf", database)
# database = CellFishing.load("database.cf")

# Load expression profiles of query cells.
data = readtsv("query.txt")
cellnames = string.(names(data))
featurenames = string.(data[:,1])
counts = Matrix{Int}(data[:,2:end])

# Search the database for similar cells; k cells will be returned per query.
k = 10
neighbors = CellFishing.findneighbors(k, counts, featurenames, database)

# Write the neighboring cells to a file.
open("neighbors.tsv", "w") do file
    println(file, join(["cell"; string.("n", 1:k)], '\t'))
    for j in 1:length(cellnames)
        print(file, cellnames[j])
        for i in 1:k
            print(file, '\t', database.metadata[neighbors.indexes[i,j]])
        end
        println(file)
    end
end

Installation

First of all, you need to install a Julia compiler. A recommended way is to download a pre-built binary of Julia. The pre-built binaries for several major platforms are distributed at https://julialang.org/downloads/. Currently, CellFishing.jl supports Julia 1.0 or later.

Then, install CellFishing.jl with the following command:

$ julia -e 'using Pkg; Pkg.add(PackageSpec(url="git://github.com/bicycle1885/CellFishing.jl.git"))'

Alternatively, you can use the add command in the package management mode of Julia:

(v1.0) pkg> add git@github.com:bicycle1885/CellFishing.jl.git

To check the installation, you can try using CellFishing in your REPL:

$ julia
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.0.0 (2018-08-08)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> using CellFishing  # load the package
[ Info: Precompiling CellFishing [5ab3512e-c64d-48f6-b1c0-509c1121fdda]

julia>

No error messages mean you have successfully installed CellFishing.jl.

To run unit tests, execute the following command:

$ julia -e 'using Pkg; Pkg.test("CellFishing")'

Command-line interface (WIP)

The bin/cellfishing script is a command-line interface to CellFishing.jl.

$ ./bin/cellfishing build Plass2018.dge.loom
Build a search database from Plass2018.dge.loom.
  Loading data ―――――――――――― 13 seconds, 173 milliseconds
  Selecting features ―――――― 1 second, 376 milliseconds
  Creating a database ――――― 16 seconds, 418 milliseconds
  Writing the database ―――― 659 milliseconds
The serialized database is in Plass2018.dge.loom.cf.
$ ./bin/cellfishing search Plass2018.dge.loom.cf Plass2018.dge.loom >neighbors.tsv
Search Plass2018.dge.loom.cf for 10 neighbors.
  Loading the database ―――― 512 milliseconds
  Loading query data ―――――― 12 seconds, 960 milliseconds
  Searching the database ―― 31 seconds, 821 milliseconds
  Writing neighbors ――――――― 64 milliseconds
$ head -5 neighbors.tsv | cut -f1-3
plan1_GACTTTCTCTTC      plan1_GACTTTCTCTTC      h2b_TTTTGCTACGGG
plan1_GTAAGGCGACAN      plan1_GTAAGGCGACAN      gfp_ATTCCTAGCGAT
plan1_TGGCCCAGCTGC      plan1_TGGCCCAGCTGC      plan1_GACTTTCTCTTC
plan1_CTCCTGTAATTT      plan1_CTCCTGTAATTT      plan1_ATCCTCCATTAA
plan1_ATGACGCATAAT      plan1_ATGACGCATAAT      plan1_TACTTGACGGTA

Download Details:

Author: Bicycle1885
Source Code: https://github.com/bicycle1885/CellFishing.jl 
License: MIT license

#julia #search #tool 

StaticWebPages.jl: A Black-box Generator for Static Websites

StaticWebPages.jl

A user-friendly static website generator written in Julia. No programming nor webdev skills required!!! Entirely Open-Source (GPLv2 licence)!

Although this generator originally targets academicians, it works out well for personal webpage and any static usage (front-end only) A more advanced static framework in Julia is available (cf Franklin.jl). For a full-stack web framework, please look at the great Genie.jl.

This generator is for you if

• you want something simple to update and manage
• responsive design (looks good on most devices)
• want light load on the server and/or low bandwidth use
• have no time/interest in making your own website by "hand"
• new item/gadget access with a simple update in the julia command line

The beta only uses text files and command lines, but the first stable release will also integrate a graphical user interface (GUI).

I deeply apologize for those that tested the versions v0.1.x, as, from v0.2.0, the syntax has changed a lot. The reason is to add options to most components. It should not change drastically from now on.

How does it work

The user provides the content (text, images, files) and select which Items will display it. As simple as that. A full working example is provided here (or in the example folder). The result is available as http://baffier.fr

Installation

The only requirement is to install Julia and this package (preferably through the package interface of Julia). Please install the last stable release of Julia on the official download page.

In the Julia REPL (that you can launch as a standalone or call within a console), please enter Pkg REPL. To quote the package manager documentation:

Pkg comes with a REPL. Enter the Pkg REPL by pressing ] from the Julia REPL. To get back to the Julia REPL, press backspace or ^C.

The following code snippet update the general registry of Julia's packages, then install the StaticWebPages.jl package.

(@v1.5) pkg> up
(@v1.5) pkg> add StaticWebPages

You can check that the installation is complete and trigger a precompilation of the package (usually take a few minutes) by using the following command.

import StaticWebPages

Please note that precompilation occurs at first use after installation and updates.

The package can be use from the REPL, but we recommend it to be used through a script file as the run.jl presented below. Running the script should be as simple as writing the following command line.

julia run.jl

or in the REPL

include("run.jl")

Files organization

A compressed template can be found in the example/ folder as either template.tar or template.zip.

The two first following files are the only requirement. However, most users will prefer to provide some images/pictures and additional files such as a cv, a bibliography, and some downloadable content.

Local script file: run.jl

# import the website generator functions
import StaticWebPages
import StaticWebPages: local_info

## private information (local folders, connection infos, etc.)
# content and site paths are always required
local_info["content"] = "path/to/content_folder"
local_info["site"] = "path/to/site_folder"

# necessary only if using the upload_site function
local_info["protocol"] = "ftp"
local_info["user"] = "user"
local_info["password"] = "password"
local_info["server"] = "server_address"

# `rm_dir = true` will clean up the site folder before generating it again. Default to false.
# `opt_in = true` will add a link to this generator website in the side menu. Default to false.
StaticWebPages.export_site(d=local_info, rm_dir = true, opt_in = true)

## upload website (comment/delete if not needed)
# unfortunately does not work yet on windows system, please sync manually for the moment
StaticWebPages.upload_site(local_info)

Content folder

It must contain a content.jl file, and both an img and a files folders. Other content files, such as BibTeX bibliographic file should be put at the root of the folder (alongside content.jl). Please check the provided example if necessary.

## content.jl
######################################
# General information
######################################

# The pic.jpg file needs to be put in the img folder
info["avatar"] = "pic.jpg"

# The cv.pdf file needs to be put in the files folder
info["cv"] = "cv.pdf"

info["lang"] = "en"
info["name"] = "Jean-François BAFFIER"
info["title"] = "Baffier"

# The email is obfuscated using a reverse email writing. The email appear normally (re-reverse) through CSS.
# Although this is an effective technique against bots, it probably won't eventually.
# The user is free to add additional security such as replacing '@' by 'at'.
info["email"] = "jf@baffier.fr"

## icons to social networks in the side menu
# comment/delete the unwanted entries
info["researchgate"] = "https://www.researchgate.net/profile/Jean_Francois_Baffier"
info["googlescholar"] = "https://scholar.google.fr/citations?user=zo7FgSIAAAAJ&hl=fr"
info["orcid"] = "https://orcid.org/0000-0002-8800-6356"
info["dblp"] = "https://dblp.org/pid/139/8142"
info["linkedin"] = "https://www.linkedin.com/in/jeanfrancoisbaffier/"
info["github"] = "https://github.com/Azzaare"
info["twitter"] = "https://twitter.com/bioazzaare"

# The navigation side-bard default to 250px. If the content above doesn't fit, it be set to another length (in px)
info["nav_width"] = "300"

######################################
# publications.html
######################################
page(
    title="publications",
    sections=[
        Section(
            title="Publications",
            items=Publications("publications.bib")
        )
    ]
)

Themes

The only available theme for the moment is using Zurb foundation responsive front-end. Customization of this theme and addition of other themes are expected in a near future (contributions/suggestions/requests are welcome).

Items

Items are guaranteed to be compatible with the main theme (and hopefully new ones will be too).

Publications : a bibliography based on a bibliography file (uses Bibliography.jl)

# Simply provide a BibTeX file (usually .bib)
# Path is based on the root of the content folder
Publications("publications.bib")

To add labels to the publications entries, please add a swp-labels field to your entries. A labels field is also accepted, but will also appear in the bibTeX citation generated for that article. For instance,

@inproceedings{parmentier2019introducing,
    title={Introducing multilayer stream graphs and layer centralities},
    author={Parmentier, Pimprenelle and Viard, Tiphaine and Renoust, Benjamin and Baffier, Jean-Francois},
    booktitle={International Conference on Complex Networks and Their Applications},
    pages={684--696},
    year={2019},
    organization={Springer},
    doi = {10.1007/978-3-030-36683-4_55},
    swp-labels = {conference, preprint, software}
}

The attribution of colors is done automatically (within the limit of 22 labels, please issue a request if you need more ...)

Deck of Cards : a list of ordered and clearly separate cards

Deck( # Start of the list
    Card( # generic
        "Left",
        "Right",
        "Title",
        "Content"
    ),
    Card( # example
        "2019",
        "current",
        "Postdoctoral Researcher",
        "RIKEN Center for Advanced Intelligence (AIP)"
    )
 ) # End of the list

GitRepo : a list of GitHub repository displayed similarly to BibTeX entries

gitrepo = GitRepo( # currently work only with GitHub
    "Azzaare/CompressedStacks.cpp",
    "Humans-of-Julia/StaticWebPages.jl",
    "Humans-of-Julia/Bibliography.jl",
    "Humans-of-Julia/BibParser.jl",
    "Humans-of-Julia/BibInternal.jl",
    "JuliaGraphs/LightGraphs.jl",
    "JuliaGraphs/LightGraphsExtras.jl",
    "JuliaGraphs/SNAPDatasets.jl",
    "Azzaare/PackageStream.jl"
)

Please note that GitHub will restrict unidentified requests to a certain amount per IP within a time limit (that I don't know the value). If it happens, a message error from GitHub API will be returned.

To circumvent this issue, you can uncomment the line local_info["auth_tokens"] = "PATH/TO/token.jl' in run.jl (or add it if necessary) and edit the token.jl by updating your Personnal Access Token.

token.jl aims to store all authentication tokens (for now, just GitHub). It content must be as follow (it is a simple variable containing your PAT):

# WARNING!
# YOUR TOKENS SHOULD NEVER BE SHARED! IF YOU USE GIT, DON'T FORGET TO ADD `token.jl` TO YOUR `.gitignore` FILE!
# YOU CAN ALSO STORE `token.jl` OUTSIDE OF THE STATICWEBPAGES FOLDER!

github_pat = "YOUR_PERSONAL_ACCESS_TOKEN" 

If the token is not valid, a 401: Bad Creditential error from GitHUb API will be returned.

As the comment states, take care not to push your token to your git repo! A good practice could be to store token.jl outside of your git project.

Block : Block of paragraphs with optional images on the side

# Examples with and without images
biography = Block(
    paragraphs(
    """
   Jean-François Baffier is an academic researcher at the RIKEN Center for Advanced Intelligence Project (AIP), and a consultant in Artificial Intelligence, Big Data Science, Data Structures, and Algorithms. As an academic, he gives back to society through fundamental research in computer science supplemented by open source libraries and softwares.
    """,
    """
    paragraph 2
    """,
    """
    paragraph 3
    """,
    """
    Jean-François implemented the StaticWebPages.jl package that was used to generate this website using a simple content file. This is a dummy email: $(email("dummy@example.purpose"))
    """
    ),
    images(
        Image("cs.png", "Compressed Stack"),
        Image("knowledge.png", "Flow of Knowledge")
    )
)

research = Block(
    paragraphs(
        """
        Principal Research Projects: Network Interdiction, Compressed Data Structures, Modern Academics, Explainable AI. Other research interest includes Graph Theory, Geometry, Optimization, and Games.
        """,
        """
        All of this research is supported by Open-Source Softwares and published as peer-review academic papers. 
        """
    ),
    images()
)

TimeLines : a list of continuous items

grants = TimeLine(
    Dot(
        "Top",
        "Title",
        "Content"
    ),
    Dot(
        "2012-2015",
        "MEXT Scholarship",
        "The Monbukagakusho Scholarship is an academic scholarship offered by the Japanese Ministry of Education, Culture, Sports, Science and Technology (MEXT)."
    )
)

Nest! A container to list several items within the same section

# Nest will take a tuple of any Items (except itself)
positions_grants = Double(
    Section(
        title="Positions and grants",
        items=Nest(work_cards, grants)
    ),
    Section(title="Research Topics", items=research)
)

Pages

A page is composed of a name and a list of sections which can each be either single or double column. Each section is either a single Item or a pair of Items.

A publication page with only one single column section (by default, publications spread over two columns on large screens)

######################################
# publications.html
#   option 1: background for the page is set to start with white to emphasize the bibliographic items
######################################
page(
    title="publications",
    background=bg_white,
    sections=[
        Section(
            title="Publications",
            items=Publications("publications.bib")
        )
    ]
)

An index page with a TextSection and a double column (Card, TimeLine)

######################################
# index.html
# biography
# academic positions | honors, awards, and grants
######################################

# using previously defined items, we can define sections
section_biography = Section(title="Biography", items=biography)
positions_grants = Double(
    Section(title="Positions", items=work_cards),
    Section(title="Grants", items=grants)
)

# the next line will add the index page to the generated content
page(
    title="index",
    sections=[section_biography, positions_grants]
)

The hide option for page and Section

# Both page() and Section() can take the hide option
# If a page is hidden, it will not be generated, but it will still appear in the side menu
# If a section is hidden, it will just not appear (and the content will not be generated, including request to external API, such as GitHub)

page(
    title="software",
    background=bg_white,
    sections=[
        Section(
            title="Software",
            hide=true, # default to false
            items=github,
        )
    ]
)

Inline components

Some components can be inserted within the content of usual Items, such as Block, Card, Dot. Currently, only link and email are supported.

Email can obfuscated (default), or not ...

email("dummy@example.purpose") # obfuscated
email(
    "dummy@example.purpose";
    content = "content that appears as the email link", # ignored if obfuscated
    obfuscated = false
    )

Link can be an internal or an external link

link("research project", "research.hmtl") # inner link
link("StaticWebPages.jl", "https://github.com/Humans-of-Julia/StaticWebPages.jl")

The main theme, and some items (BibTeX, Card, TimeLine) are inspired by the Word Press Faculty template of owwwlab, also under GPLv2. All elements inspired from that template can be found in the HTML/CSS code associated with those items.

Download Details:

Author: Humans-of-Julia
Source Code: https://github.com/Humans-of-Julia/StaticWebPages.jl 
License: GPL-2.0 license

#julia #generator #websites 

Parser for Bibliographic Formats, including BibTeX, in Pure Julia

BibParser.jl

BibParser is a Julia package for parsing different bibliographic formats. The output are entries following the BibParser.jl structures. Instead of rewriting from scratch existing (and efficient) parsers in Julia, it is preferable to import them.

This package is not meant to be used on its own. Please check Bibliography.jl for a package handling both import/export from various bibliographic formats.

The output of an example parsing a BibTeX file can be found at baffier.fr/publications.html.

BibTeX

A new parser is in used since v0.1.12. It is almost complete. Currently missing features follow:

• Applying the LaTeX commands from @preambles entries to other entries
• Storing @preamble, @string, @comment, and free text to enable the reconstruction of the original .bib file
• Optional transformation of Unicode <-> LaTeX characters

Citation Style Language (CSL-JSON)

Ongoing work.

CFF

This implementation follows similar parsers: cff-converter-python and ruby-cff.

Download Details:

Author: Humans-of-Julia
Source Code: https://github.com/Humans-of-Julia/BibParser.jl 
License: MIT license

#julia #includeing 

A Julia Package to Fold All Local & Long-range RNA Duplexes

EvoDuplexes.jl 

RNA suffix array traversal through chromosomal multiple alignment files (MAF format) to fold and score (with both free-energy models and phylogenetic likelihood) all local and long-range RNA duplexes while allowing bulges and mismatches.

Full documentation is coming soon (Requires Julia v0.6.4), however comprehensive examples of API usage for EvoDuplexes package can be found in test/runtests.jl.

Executable can be run using evo-duplex.jl in /bin:

$ julia evo-duplex.jl -h
usage: evo-duplex.jl --tree TREE [--cons-regions CONS-REGIONS]
                     --gene-regions GENE-REGIONS --maf MAF
                     [--model-load MODEL-LOAD]
                     [--model-data MODEL-DATA] [--model-train]
                     [-o OUTPUT] [--cons-branch CONS-BRANCH]
                     [--sig-ratio SIG-RATIO] [--output-all]
                     [--max-distance MAX-DISTANCE]
                     [--max-duplex-length MAX-DUPLEX-LENGTH]
                     [--max-bulges MAX-BULGES]
                     [--max-mismatches MAX-MISMATCHES]
                     [--max-deltag MAX-DELTAG] [-h]

optional arguments:
  --tree TREE           Phylogenetic tree with neutral branch lengths
                        (in newick format)
  --cons-regions CONS-REGIONS
                        BED file containing conserved regions
  --gene-regions GENE-REGIONS
                        BED file containing gene units to allow
                        long-range folding within
  --maf MAF             Directory with MAF files named by chromosome
                        (chr1.maf.gz...) (default: "../maf")
  --model-load MODEL-LOAD
                        Load pre-trained IsolationForest models,
                        .evt.jls
  --model-data MODEL-DATA
                        Load training data from `.jlt` file, output
                        `.evt.jls` file
  --model-train         Train the IsolationForest models from scratch
                        using input regions, output `.evt.jls` file
  -o, --output OUTPUT   Output prefix (default: "evoduplex")
  --cons-branch CONS-BRANCH
                        When conserved regions are given, use this
                        branch length multiplier (type: Float64,
                        default: 0.33)
  --sig-ratio SIG-RATIO
                        Output the top fraction of outliers (type:
                        Float64, default: 0.05)
  --output-all          Output all folds regardless of prediction
                        status
  --max-distance MAX-DISTANCE
                        Set a limit on the maximum distance between a
                        left/right arm of a duplex (type: Int64,
                        default: 2000)
  --max-duplex-length MAX-DUPLEX-LENGTH
                        Set the maximum duplex length that will be
                        accessible from the suffix array (type: Int64,
                        default: 50)
  --max-bulges MAX-BULGES
                        Set the maximum number of bulges to allow for
                        a duplex (type: Int64, default: 3)
  --max-mismatches MAX-MISMATCHES
                        Set the maximum number of mismatches to allow
                        for a duplex (type: Int64, default: 3)
  --max-deltag MAX-DELTAG
                        Set the maximum allowable deltaG value (type:
                        Float64, default: -8.0)
  -h, --help            show this help message and exit

Download Details:

Author: Timbitz
Source Code: https://github.com/timbitz/EvoDuplexes.jl 
License: MIT license

#julia #local #array