The hardware and bandwidth for this mirror is donated by METANET, the Webhosting and Full Service-Cloud Provider.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]metanet.ch.

tRnslate: translate chunks or inline R code in source files

Mario A. Martínez Araya

2021-07-07

Author: Mario A. Martínez Araya
Date: 2021-07-07
Url: https://marioma.me/?i=soft
CRAN: https://cran.r-project.org/package=tRnslate

This R package is intended to translate chunks or inline R code in source files in general, replacing with its output if desired. It was first created to translate scripts for R package generation (to generate R code from R code) and then I used it for something similar within R and Bash scripts necessary for parallel computation using MPI. Then I packaged it independently since it was useful for other purposes as well.

Requirements

In principle any version of R should be useful but I have not tested them all.

Installation

The same than for any other R package. You can download the tar file from CRAN (tRnslate) and then install it using

R CMD INSTALL /path/to/tRnslate_0.0.3.tar.gz

or from R console

install.packages("tRnslate", lib = "path/to/R/library")

How to use it?

1. Templates with R code in chunks and inline

Imagine you have a template file of any kind called template.txt. Let us assume we will write a Bash script for submitting a parallel job using SLURM. Let us read the content from the template which has R code in chunks and inline:

# template with R code
T <- readLines(system.file("examples/template.txt", package = "tRnslate"))

In R we can write the content of the template to console:

cat(T, sep = "\n")

#!/bin/bash
@r # This is a chunk (only assignation)
@r if(.Platform$OS.type=="unix"){
@r     is_q <- system("clu=$(sinfo --version 2>&1) || clu=`echo -1`; echo $clu",intern = TRUE)
@r } else {
@r     is_q <- "-1"
@r }
@r s$intro <- ifelse(is_q=="-1", "<:NULL:>", s$intro)

<r@ s$intro @> --partition=<r@ s$partition @>
<r@ s$intro @> --nodes=<r@ s$nodes @>
<r@ s$intro @> --tasks-per-node=<r@ s$tasks @>
<r@ s$intro @> --mem=<r@ s$memory @>
<r@ s$intro @> --time=<r@ s$time @>
<r@ s$intro @> --nodes=<r@ s$nodes @>

@r # This is a chunk (only assignation)
@r # NOTE: remember, separate chunks with empty lines
@r array <- ifelse(s$array, paste(s$intro," --array=",s$array,sep=""), "")
<r@ array @>

@r # This is another chunk (only assignation)
@r if(.Platform$OS.type=="unix"){
@r     is_mod <- system("mod=$(module --dumpversion 2>&1) || mod=`echo -1`; echo $mod",intern = TRUE)
@r } else {
@r     is_mod <- "-1"
@r }

@r # And this a printing chunk
@r ifelse(is_mod=="-1", "# module environment not found", paste(s$modules))

@r # And this other printing chunk
@r if(is_q=="-1"){
@r     "# no slurm machine"
@r } else {
@r     s$workdir
@r }

@r # And the last chunk (printing)
@r # NOTE: that it also includes inline elements
<r@ system("which mpirun",intern = TRUE)@>/mpirun --mca mpi_warn_on_fork 0 -n <r@ s$nodes * s$tasks @> <r@ R.home("bin") @>/Rscript r-code-script.R

echo "Job submitted on $(date +%F) at $(date +%T)."

2. R code explanation

Lines starting with @r or @R followed by one space or tabular, define chunks of R code that is also interpreted and translated. The chunks of R code can be assignation or output chunks. Assignation chunks are those including <- for assigning an object, while output chunks print R output to the template. Thus several assignation chunks can be placed in adjacent lines, however assignation and output chunks must be separated by one empty line (the same for consecutive output chunks). Alternatively, inline R code can be entered using <r@ code @> or <R@ code @>. Inline R code with assignation does not produce output so is replaced by blank, while inline R code producing output will modify the resulting template.

KEEP IN MIND A FEW RULES
  • R code in chunks and inline is evaluated from top to bottom and from left to right in each line.
  • Code chunks allow R comments. In the example above I used them to explain a bit the code in the template.
  • Code chunks can also be marked only using the opening @r, for example:

      @r # And this other printing chunk
      @  if(is_q=="-1"){
      @      "# no slurm machine"
      @  } else {
      @      s$workdir
      @  }
  • Do not mix (actions of) assignation with printing in the same chunk. Note, that for an if clause, it is valid that if the condition is true then the chunk action prints and if the condition is false its action assigns.
  • Each printing chunk must produce only one output object.
  • Loops can be used in chunks but only with assignation (no printing).
  • Separate chunks with an empty line.
  • Chunks cannot contain inline R code (so far).
  • Inline code does not allow comments in it.
  • Tend to use inline code mainly for printing (although assignation is allowed).
  • In inline code print objects explicitly using paste, print or similar commands. For instance, use <r@ paste("some random string") @> instead of <r@ "some random string" @> which will not be processed similarly as <r@ some random string @>.

3. Environment used to evaluate the R code

The R code in the template needs to be evaluated in an environment which can be specified by the user. This environment can contain objects which are called from the chunks or inline R code in the template. For example, in the previous template the R code calls an object s with several elements (partition, nodes, tasks, etc.) which are being used to replace the content in the template. For this template, to create the environment and object s we can do:

# NOTE: this is the environment that will be used later (see below)
renv <- new.env(parent = parent.frame())
# list with input arguments
renv$s <- list(
    intro = "#SBATCH",
    partition = "hpc01",
    nodes = 4,
    tasks = 10,
    memory = "2gb",
    time = "01:00:00",
    array = FALSE,
    modules = 'module load openmpi/chosen/module R/chosen/module',
    workdir = 'cd ${SLURM_SUBMIT_DIR}'
)

4. The translate_r_code command

Given the template above then we can “translate” its R code using the function translate_r_code as follows:

## Evaluate the R code
TT <- translate_r_code(T, envir = renv)

## See the output
cat(TT, sep="\n")

Resulting source file

Depending on the system where you execute the previous code, the resulting output will vary. For example, for a multicore PC with OpenMPI but without a dynamic environment modules manager such as environment-modules or Lmod and without a job scheduler such as SLURM then the output of cat(TT, sep="\n") will be something like this:

#!/bin/bash

# module environment not found

# no slurm machine

/usr/bin/mpirun/mpirun --mca mpi_warn_on_fork 0 -n 40 /usr/lib/R/bin/Rscript r-code-script.R

echo "Job submitted on $(date +%F) at $(date +%T)." 

While for an HPC cluster having OpenMPI, environment-modules and SLURM the “translated” output file will be similar to:

#!/bin/bash

#SBATCH --partition=hpc01
#SBATCH --nodes=4
#SBATCH --tasks-per-node=10
#SBATCH --mem=2gb
#SBATCH --time=01:00:00
#SBATCH --nodes=4

module load openmpi/chosen/module R/chosen/module

cd ${SLURM_SUBMIT_DIR}

/usr/bin/mpirun/mpirun --mca mpi_warn_on_fork 0 -n 40 /usr/lib/R/bin/Rscript r-code-script.R

echo "Job submitted on $(date +%F) at $(date +%T)."

Additional rules could be added to control the lenght of the mpirun line, however as it is it works fine. Other source code can be generated following the same principles described before.

Limitations

As it is, translate_r_code has some limitations such as:

Therefore, use this function carefully.

RECOMMENDATIONS
  1. Never replace the content of a template writing the output to the same file.

  2. Always check the content of the “translated” output before using it for other tasks.

  3. Be cautious.

References

These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.