Create HTML page containing all package examples using knitr and markdown

Yesterday, I was updating my website to include more information about some R packages I am working on. I wanted to create a page showing examples of the functionality that is available in each package. Now, I usually write a quite extensive amount of examples in the help pages, showing most of the functionality. A single page with the examples of each function in the package including the output would be a, although very simply, quite nice way to quickly demonstrate the functionality of the package.

This is not hard to do by hand. Simply copy the examples to an R markdown file, add knitr chunks and a title here and there and it should look good. But this seemed like something that would be nice to do every update, so I wanted to automate it. The results (which can be seen here and here) were quite nice, so I figured I maybe others could use these codes as well. Do keep in mind that these codes are hardly tested!

Below is the code for the function examplePage. It scans all of a packages .Rd files for an example section and extracts them. The function creates and compiles a markdown document with the following title levels:

1. The package name (only at the top)
2. The help-file name
3. Any line in the example section that contain word characters and either starts with three or more #s or starts with an # and ends with an # or -.
4. Any line that starts with (after spaces) exactly ##, followed by word characters and does not end on # or -.

These titles are chosen specifically such that the control-shift-R command in RStudio or something like #### BIG SECTION ####results in a large title.

Usage:

examplePage has the following arguments:

• pkg: Path to the package folder, should include a directory man with the Rd files.
• openChunk: code to open chunks. Defaults to "{r, message=FALSE, warning = FALSE, error = FALSE}". This can be used to enter more knitr options.
• includeDontshow: Logical stating if don't show environments should be included in the codes. Defaults to FALSE.
• includeDontrun: Logical stating if don't run environments should be included in the codes. Defaults to TRUE.

The function:

examplePage <- function(pkg, openChunk = "```{r, message=FALSE, warning = FALSE, error = FALSE}", 
    includeDontshow = FALSE, includeDontrun = TRUE, exclude) {
    if (!require("knitr")) 
        stop("'knitr must be intalled.")
    if (!require("markdown")) 
        stop("'knitr must be intalled.")

    # Inner function to find closing brackets:
    findClose <- function(x, openLoc, open = "\\{", close = "\\}") {
        # Find close:
        nest <- 1
        i <- openLoc + 1
        repeat {
            # If open bracket in line:
            if (grepl(open, x[i])) {
                nest <- nest + length(gregexpr(open, x[i])[[1]])
            }
            if (grepl(close, x[i])) {
                nest <- nest - length(gregexpr(close, x[i])[[1]])
            }
            if (nest == 0) 
                break
            i <- i + 1
        }
        return(i)
    }

    files <- list.files(paste0(pkg, "/man"), pattern = "\\.Rd$", ignore.case = TRUE, 
        full.names = TRUE)

    # Exclude:
    if (!missing(exclude)) 
        files <- files[!grepl(exclude, files)]

    # Preparation:
    n <- length(files)
    subs <- character(n)

    # For each rd file:
    for (i in seq_along(files)) {
        # Read file:
        txt <- readLines(files[i])

        # Only include if there is only one example section:
        if (sum(grepl("\\\\examples\\{", txt)) == 1) {
            # Extract examples section:
            start <- grep("\\\\examples\\{", txt)
            end <- findClose(txt, start)
            txt <- txt[(start + 1):(end - 1)]

            # Don't show fields:
            dontshows <- grep("\\\\dontshow\\{", txt)
            if (length(dontshows) > 0) {
                ends <- numeric(length(dontshows))
                for (k in seq_along(dontshows)) {
                  ends[k] <- findClose(txt, dontshows[k])
                }

                # Remove:
                if (includeDontshow) {
                  txt <- txt[-c(dontshows, ends)]
                } else txt <- txt[-do.call(c, mapply(dontshows, ends, FUN = ":", 
                  SIMPLIFY = FALSE))]
            }

            # Don't run fields:
            dontruns <- grep("\\\\dontrun\\{", txt)
            if (length(dontruns) > 0) {
                ends <- numeric(length(dontruns))
                for (k in seq_along(dontruns)) {
                  ends[k] <- findClose(txt, dontruns[k])
                }

                # Remove:
                if (includeDontrun) {
                  txt <- txt[-c(dontruns, ends)]
                } else txt <- txt[-do.call(c, mapply(dontruns, ends, FUN = ":", 
                  SIMPLIFY = FALSE))]
            }

            # Enter main title and first R chunk:
            txt <- c(paste("##", gsub("\\.rd$", "", basename(files[i]), ignore.case = TRUE)), 
                openChunk, txt, "```")

            # Crawl over lines. If a title is encountered, close chunk and replace
            # title with markdown:
            j <- 3
            repeat {
                # Small section (start with exactly two hashes, does not end with nonword:
                if (grepl("^\\s*##\\s*(\\w|\\s)+$", txt[j])) {
                  txt[j] <- gsub("^\\s*##\\s*", "#### ", txt[j])
                  txt <- c(txt[1:(j - 1)], "```", txt[j], openChunk, txt[(j + 
                    1):length(txt)])
                  j <- j + 2

                  # Else large section, starts with #, ends with nonchar, or starts with
                  # more than 2 #'s
                } else if (grepl("\\w", txt[j]) & (grepl("^\\s*###", txt[j]) | 
                  grepl("^\\s*#.*[#-]\\s*$", txt[j]))) {
                  txt[j] <- gsub("^\\W*(?=\\w)", "### ", txt[j], perl = TRUE)
                  txt[j] <- gsub("(?<=\\w)\\W*$", "", txt[j], perl = TRUE)

                  txt <- c(txt[1:(j - 1)], "```", txt[j], openChunk, txt[(j + 
                    1):length(txt)])
                  j <- j + 2
                } else if (grepl("^\\s*#\\W*$", txt[j])) {
                  # If start is comment and no words, remove:
                  txt <- txt[-j]
                  j <- j - 1
                }

                j <- j + 1
                if (j > length(txt)) 
                  break
            }

            emptySections <- which(txt[-length(txt)] == openChunk & txt[-1] == 
                "```")
            if (length(emptySections) > 0) 
                txt <- txt[-c(emptySections, emptySections + 1)]

            txt <- gsub("\\\\%", "%", txt)
            subs[i] <- paste(txt, collapse = "\n")
        }
    }

    subs <- subs[order(nchar(subs), decreasing = TRUE)]
    subs <- c(paste0("# ", basename(pkg), "\n\n```{r,echo=FALSE,message=FALSE}\nlibrary(\"", 
        basename(pkg), "\")\n```"), subs)

    # Write Rmd:
    RmdFile <- paste0(basename(pkg), ".Rmd")
    write(paste(subs, collapse = "\n\n"), RmdFile)

    # Knit:
    mdFile <- gsub("Rmd", "md", RmdFile)
    knit(RmdFile, mdFile)

    # Markdown:
    htmlFile <- gsub("Rmd", "html", RmdFile)
    markdownToHTML(mdFile, htmlFile)

    browseURL(htmlFile)

    return(htmlFile)
}

data(big5)
data(big5groups)
qgraph(cor(big5),groups=big5groups,gui=TRUE)

Will open a plotting window and the GUI that allows the user to set several parameters before plotting:

Graphical User Interface for qgraph

There is also a simplified version of this GUI for regular graphs (not correlation matrices).

What I particularly like about this GUI is that it took me only one day to implement. Making a GUI for qgraph has been on my to do list for a long time, but I never really got to learning how one of the packages that allow this work. When I found out about the ‘rpanel‘ package (which is build on ‘tcltk’) I was pleasantly amazed with how simple and intuitive it was to use.

In short, the way ‘rpanel’ works is by making a ‘panel’ using ‘rp.control()’ which is basically a list and also opens an empty GUI frame. We can use functions such as ‘rp.checkbox’ and ‘rp.slider’ to add elements to the GUI. Using these elements then does two things: an element of the list is changed (e.g., the element ‘xlim’), and a function is called on the list. This way, we can easily make an appropriate GUI element for each argument we need in a plotting function and make buttons that plot, save to PDF, etcetera.

To illustrate this I wrote a small function with comments that uses ‘plot.default’ to plot a scatter plot, and adds a small GUI frame allowing a user to zoom into the scatter plot. The codes can be downloaded here. A small example:

x <- rnorm(100)
y <- x + rnorm(100)
ScatterZoom(x,y)

More information on 'rpanel' can be found in its Journal of Statistical Software article.

Code

It is very basic and abit buggy (mainly in displaying unwanted warning messages) but does seem to work. The code above (requires tcltk package) gives the reference class “playlist”. First we create a new empty playlist:

pl <- playlist$new()

Next we fill the playlist by manually adding music files:

pl$add()

or by importing all music files in a folder (recursively):

pl$dir()

Both open a gui window by default, but can be supplied a character with the locations as well.

Now we can shuffle the playlist if we want:

pl$shuffle()

Finally, start playing:

pl$play()

And if we don't want to listen anymore:

pl$stop()

Furthermore 'pause' can be used to pause playing, 'skip' to skip a song, 'step' to skip multiple songs and 'previous' to go back a song.

I have only tested this on a Linux computer (Ubuntu 11.10).

10 thousand questions tagged! I started using Stackoverflow almost exactly a year ago and remember that the counter was about 3 thousand at the time. It is a beautiful example of how fast the R community is growing.

For example, consider a small toy dataset of lengths with 100 females have a normally distributed length with mean 170cm and standard deviation of 10, and 100 males with mean length of 180cm and standard deviation of 10:

R> foo <- data.frame(
length = c(rnorm(100,170,10), rnorm(100,180,10)),
sex = rep(c("female","male"),each=100))

A t-test shows that the means are different:

R> t.test(length~sex,data=foo,var.equal=TRUE)

	Two Sample t-test

data:  length by sex
t = -6.8396, df = 198, p-value = 9.653e-11
alternative hypothesis: true difference in means is not equal to 0
95 percent confidence interval:
 -12.715980 -7.024375
sample estimates:
mean in group female   mean in group male
            170.2455           180.1157

The t.test() function returns a "htest" class object which is commonly used in R and allows us to easily extract the statistic, degrees of freedom and p-value:

R> res <- t.test(length~sex,data=foo,var.equal=TRUE)
R> res[['statistic']]
        t
-6.839605
R> res[['parameter']]
 df
198
R> res[['p.value']]
[1] 9.653065e-11

Great, now we can reference the statistic in our Sweave document:

Men were significantly taller than women
($t(\Sexpr{res[['parameter']]})=\Sexpr{res[['statistic']]}$,
$p=\Sexpr{res[['p.value']]}$)

This returns: "Men were significantly taller than women (t(198) = −6.83960491494726,
p = 9.65306549553569e − 11)". Obviously we need to round the values:

Men were significantly taller than women
($t(\Sexpr{res[['parameter']]})=\Sexpr{round(res[['statistic']],3)}$,
$p=\Sexpr{round(res[['p.value']],3)}$)

Which returns "Men were significantly taller than women (t(198) = −6.84, p = 0)". Better, but the p value should not be rounded to zero but rather be reported as being smaller than 0.001 or something similar if it is very small. To do this and make sure it stays dynamic an ifelse statement is needed:

Men were significantly taller than women
($t(\Sexpr{res[['parameter']]})=\Sexpr{round(res[['statistic']],3)}$,
$p \Sexpr{ifelse(res[['p.value']]<0.001,'< 0.001',
   paste('=',round(res[['p.value']],3)))}$)

Which returns "Men were significantly taller than women (t(198) = −6.84, p < 0.001)".

Good, but this sure was a lot of code to make this simple reference, and for some other classes extracting the statistics from the output object is also a lot harder than the "htest" class. For this reason I wrote the 'swst' package, which stands for SWeave STatistics.

'swst' has two main functions. The 'swp()' function can be used to generate proper LaTeX code with rounded numbers and inequality signs if needed given the name of a statistic, its value, optional degrees of freedom and the p-value. The 'swst()' function is an S3 generic with methods for a few commonly used object classes that extract the statistic, df, and p-value and send the results to 'swp()'.

This reduces the code to:

Men were significantly taller than women \Sexpr{swst(res)}

Which returns "Men were significantly taller than women (t(198) = −6.84, p < 0.001)". That's much less code!

I have written this package in a fairly short time and it is now very short and only supports a few objects. Help is greatly appreciated! If you know an object that needs to be implemented let me know or write your own method on:

http://github.com/SachaEpskamp/swst

The CRAN link is:

http://cran.r-project.org/web/packages/swst/index.html

• how to obtain a LaTeX distribution
• How to use LaTeX to write professional scientific reports
• How to use BibTeX and apacite to easily add references and completely automate the reference list
• How to use the apa document class to produce articles in APA-style
• How to use Beamer to produce presentation slides
• How to use Sweave to incorperate R codes in your LaTeX document for reporting reproducible research

The slides and examples as well as a lot of additional links can be found at:

http://sachaepskamp.com/latex-course