Apparent package conflicts

When there are apparent package conflicts, Require uses, in this order:

• version requirement;
• CRAN priority;
• order requested

See these examples:

# No version specifications — CRAN version installed, or nothing if already installed
Require::Install(c("PredictiveEcology/reproducible@development", "reproducible"))

# `HEAD` after the GitHub ref forces the tip of the development branch
Require::Install(c("PredictiveEcology/reproducible@development (HEAD)", "reproducible"))

# Same: `HEAD` after the package name (of either form) forces the tip
Require::Install(c("PredictiveEcology/reproducible@development", "reproducible (HEAD)"))

# No conflict: version requirement is satisfiable by the named branch
Require::Install(c("PredictiveEcology/reproducible@modsForLargeArchives (>= 2.0.10.9010)",
                   "PredictiveEcology/reproducible (>= 2.0.10)"))

# Even if a branch doesn't exist, no error if a later requirement names a different branch
Require::Install(c("PredictiveEcology/reproducible@modsForLargeArchives (>= 2.0.10.9010)",
                   "PredictiveEcology/reproducible@validityTest (>= 2.0.9)"))

Key features (when usePak = TRUE)

Features include:

1. Fast, parallel installs and downloads (delegated to pak).
2. Installs CRAN and CRAN-alike packages even if they have been archived.
3. Installs GitHub packages.
4. Can loads packages after installing, if using Require::Require.
5. User can specify which version to install using the standard R-version approach (e.g., ==3.5.0 or >=3.5.0).
6. Local package caching (see below) for fast (re-)installs.
7. Manages (several types of) conflicting package requests, i.e., different GitHub branches.
8. Finds specific versions of packages from an incomplete CRAN-like repository (such as r-universe.dev), even when the version is not available, but it is available on the main CRAN mirrors.

Rerun-tolerance

To be functionally reproducible, code must be regularly run and tested on many operating systems and computers. When this does not happen, a user/developer does not know that certain code chunks no longer work until they try to run it later. In other words, code gets stale because underlying algorithms and data change. To be rerun-tolerant, a function must:

1. return the same result or outcome every time it is run (first, second or more times later);
2. be very fast after the first time; when it is not fast, users will skip running it “because we don’t need to run it again and it is slow”

Require does both of these. See below “why is it fast”.

Why these features help teams

It is common during code development to work in teams, and to be updating package code. This is beneficial whether the team is very tight, all working on exactly the same project, or looser where they only share certain components across diverse projects.

Require::Install(
  c("PredictiveEcology/reproducible@development (HEAD)",
    "PredictiveEcology/SpaDES.core@development (>=2.0.5.9004)"))

How Require differs from pak in philosophy

By default, as of version 2.0.0, Require uses pak to calculate package dependency trees and installations. However, Require applies a different philosophy to package management. The two tools answer the same question — “what should be installed?” — in different ways.

Require is therefore not an alternative to pak. It is a complementary wrapper that applies a different policy on top of pak. The differences described in this vignette are differences in policy, not in installation machinery. For example: when you call pak::pkg_install("data.table"), pak will offer to upgrade data.table if a newer version is on CRAN. When you call Require::Install("data.table"), Require first checks whether the installed version already satisfies your request; if it does, nothing happens at all. The actual install, when one is needed, is done by pak either way.

Practically speaking, this means that a user can write their list of packages they need in their code, and leave it there, without concern that their packages may unexpectedly be updated – using time and possibly changing functionality at an inopportune moment.

Stability vs. Most-recent

The biggest difference is what each tool does when a package is already installed.

• pak is current-first. If you ask pak to install a package that is already there, it will check for a newer version and offer to upgrade.
• Require is stability-first. If the installed version satisfies your request, Require does not install. It will only install or upgrade when the version constraint you wrote actually requires it.

This is what makes Require “set-and-forget”. You can put a Require::Install(...) line near the top of a script, run that script every day for a year, and your packages will not silently change underneath you. They only change when you change your version requirement.

Require exposes the upgrade policy through the version constraints in your code. If you want the latest, ask for it (e.g. data.table (>= 1.16) or data.table (HEAD)); if you want stability, leave the constraint off.

GitHub branches: exact pin vs. version minimum

The same stability-first policy shows up clearly when you install from a GitHub branch. pak reads the DESCRIPTION on that branch and enforces every dependency exactly as written — even with upgrade = FALSE, it will downgrade (or upgrade) installed dependencies to match the pin. Require reads the same DESCRIPTION but treats each line as a minimum: if an installed dependency already satisfies the constraint, it is left alone.

Here LandR@development lists reproducible (>= 3.0.0.9001) in its DESCRIPTION, while the user already has reproducible 3.0.0.9083 installed:

> pak::pak("PredictiveEcology/LandR@development", upgrade = FALSE)

→ Will install 1 package.
→ Will update 1 package.
→ All 2 packages (0 B) are cached.
+ LandR                     1.1.5.9101 [bld][cmp] (GitHub: c5e771d)
+ reproducible 3.0.0.9083 → 3.0.0.9001 [bld][cmp] (GitHub: ffffec4)
✔ All system requirements are already installed.

? Do you want to continue (Y/n) n
Error: Aborted.

> Require::Install("PredictiveEcology/LandR@development", upgrade = FALSE)
Require/pak skipping new package dependency identification: using cache (103 packages, 0.6h old)
All requested packages are in the pak download cache; installing from cache (no metadata refresh, no network)
offline mode: installing 1 package(s) from pak cache: LandR

→ Will install 1 package.
→ The package (0 B) is cached.
+ LandR   1.1.5.9101 [bld][cmp] (GitHub: c5e771d)

ℹ No downloads are needed, 1 pkg is cached
ℹ Building LandR 1.1.5.9101
✔ Built LandR 1.1.5.9101 (24.8s)
✔ Installed LandR 1.1.5.9101 (github::PredictiveEcology/LandR@c5e771d) (37ms)
✔ 1 pkg: added 1 [25.9s]
Installed 1 packages in 26.3 secs

pak insists on replacing reproducible 3.0.0.9083 with the exact pin from the branch (3.0.0.9001), even though the installed version is newer. Require keeps the newer copy because it still satisfies LandR’s constraint. The two behaviours have different use cases: pak’s exact-pin enforcement is what you want when you need to reproduce the dependency graph the branch author committed to; Require’s version-minimum policy is what you want for “set-and-forget” scripts where any version meeting the minimum is acceptable.

Installs and loads in one line

pak installs packages. To use them, you still need a separate library() call.

Require::Require() does both: it installs (if needed) and then loads. The whole package-management story for a script can fit on one line:

Require(c("data.table (>= 1.16)", "lme4", "PredictiveEcology/SpaDES.core@development"))

Version constraints in the package name

pak accepts exact version pins via pkg@1.2.3. It does not accept ranges like >= or <= directly — you would have to either pin a specific version yourself or put the constraint in a DESCRIPTION file:

# Won't work — pak does not parse this
try(pak::pak("data.table (>= 1.8.0)"))

# What you have to write instead — pick an exact version yourself
pak::pak("data.table@1.8.0")

Consistent with the version requirements that can be specified in a package DESCRIPTION file, Require accepts the full set of R-style constraints right in the call, mixed freely:

Require::Install(c("data.table (>= 1.16)",
                   "stringfish (<= 0.15.8)",
                   "qs (== 0.27.3)"))

This matters because the constraint is what tells Require “stop, don’t install” or “yes, please upgrade”. The constraint is the policy.

Conflicts: resolved vs. raised as errors

When two of your dependencies (or sub-dependencies) point to different sources or different branches of the same package, pak reports a conflict and stops. The user is expected to fix it — usually by adding any:: prefixes or removing one of the requests.

Require resolves the conflict for you, using the priority documented above (version requirement, then CRAN, then order requested).

# pak: errors out — both branches of LandR are requested
try(pak::pak(c("PredictiveEcology/LandR@development",
               "PredictiveEcology/LandR@main")))

# Require: takes them in order — main wins
Require::Install(c("PredictiveEcology/LandR@main",
                   "PredictiveEcology/LandR@development"))

# Require: takes by version requirement — development wins because it satisfies the constraint
Require::Install(c("PredictiveEcology/LandR@main",
                   "PredictiveEcology/LandR@development (>= 1.1.5)"))

The same conflict-resolution applies to mismatches between a CRAN package and a GitHub Remotes field deep inside someone else’s package: Require picks something and explains why, rather than asking you to untangle it.

Archived packages: automatic vs. manual

When a package is removed from CRAN (“archived”), pak cannot install it from a plain name — you need to give it the explicit URL of the archive tarball (url::https://...). And if the archived package is a sub-dependency of something else, even that workaround doesn’t always help.

Require retrieves the most recent archived copy automatically and continues. This means a workflow that worked yesterday continues to work today, even if a CRAN package has been archived overnight.

# pak: fails — `knn` is archived
try(pak::pkg_install("knn"))

# Require: succeeds — fetches the most recent archived copy
Require::Install("knn")

Installing from a snapshot

A snapshot is a flat list of exact pins (CRAN versions and GitHub SHAs). On paper, that’s the easiest possible install — every version is already chosen. In practice, handing the same list to pak::pkg_install() runs into trouble that doesn’t apply to a “just install the latest” workflow:

• All-or-nothing solving. pak’s resolver evaluates every pin together. If one pin is unsolvable (an archived version, a sub-dep that contradicts another pin), it refuses to install anything. Require’s snapshot installer goes pin-by-pin with install.packages(dependencies = NA) against a synthesized local repo, so a bad row removes one package, not all of them.
• Archived / disappeared versions. Snapshots routinely pin versions that have since left CRAN. pak 404s. Require substitutes the nearest available archived version and reports the substitution.
• Non-CRAN homes. Rows that came from r-universe, RSPM, or another CRAN-alike carry a Repository URL. pak’s resolver only consults options(repos), so those rows fail to resolve. Require honours each row’s Repository column.
• Incomplete snapshots. A snapshot built from a session that already had a transitive dep loaded from another libPath can be missing that dep. pak errors with dependency 'X' is not available. Require auto-fills the missing dep from CRAN/PPM and flags it so the user can add it to the snapshot for full reproducibility.
• Opaque failures. When pak does fail, the user sees ! error in pak subprocess. Require keeps per-package install logs and prints a structured report: status (download-failed / version-conflict / missing-dep / compile-failed / cascade / substituted / auto-filled), the reason, and a concrete fix.
• Speed on Linux/macOS. Tarballs are fetched in parallel via libcurl multi, with PPM binaries preferred (and the right User-Agent set so PPM actually serves binaries). The cache is pkgcache — the same cache pak uses — so anything downloaded here is reusable by pak next time, and vice versa.

Working offline

Require can install and load packages with no internet, as long as they (or compatible builds of them) were downloaded once before. This is useful in some settings, including e.g., a high performance computer cluster that has no internet access on the compute nodes. Set:

options(Require.offlineMode = TRUE)
Require::Require("dplyr")

Require looks for each package in the local pak cache and lets pak install from there. With a warm cache, installs are near-instant — no rebuild, no download.

Two things make this work that calling pak directly does not:

• Network probes are suppressed. pak normally fetches bioconductor.org/config.yaml and refreshes its repo metadata at startup, even when nothing remote is needed. Require sets the right environment variables so those calls are skipped for the duration of the install.
• Refs are translated. Require’s internal pkg (>= X.Y.Z) constraint form is rewritten to the bare package name before pak sees it (pak rejects the parenthetical form).

You don’t have to set Require.offlineMode yourself. If Require tries an online install and any package fails because the network is unreachable, it probes for connectivity (~2 seconds) and, if there really is no internet, automatically retries from the cache. On the happy path you pay nothing extra; the probe only runs when an install fails.

When a package isn’t in the cache at all, Require warns “not in pak cache” — a separate message from “tarball was in cache but install failed”, so the cause is unambiguous.

Summary of differences

The “installation engine” rows that used to appear here (parallel downloads, parallel installs, local cache) are no longer differences: Require uses pak for those.

Set it and forget it speed

Because a major objective for Require is to be set it and forget it, it cannot use meaningful human time. Thus, when all packages are installed, rerunning Require lines is 2x-10x faster than the equivalent pak::pak line:

> system.time(pak::pak(c("devtools", "testthat", "roxygen2")))
                                                                             
ℹ No downloads are needed
✔ 3 pkgs + 90 deps: kept 92 [1.4s]
   user  system elapsed 
   0.00    0.00    1.47 
   
> system.time(Require::Install(c("devtools", "testthat", "roxygen2")))
Require/pak skipping new package dependency identification: using memory cache (93 packages)
No packages to install/update
   user  system elapsed 
   0.04    0.00    0.20 

Extra from Require

If the packages supplied to a Require/Install call are identical as a previous one (commonly the case for ongoing projects), the package dependency tree is not re-calculated as it is stored on disk and in memory (so in-session re-runs are very fast). Since this is a slow process for >200 packages, users will see near instant package assessments.

Managing projects during development

renv has a concept of a lockfile. This lockfile records a specific version of a package. If the current installed version of a package is different from the lockfile (e.g., I am the developer and I increment the local version), renv will attempt to revert the local changes (with prompt to confirm) unless the local package is installed from a cloud repository (e.g., GitHub), and a snapshot is taken. This sequence is largely incompatible with pkgload::load_all() or devtools::install(), as these do not record “where” to get the current version from. Thus, the renv sequence can be quite time consuming (1-2 minutes, instead of 1 second with pkgload::load_all()).

Require does not attempt to update anything unless required by a package. Thus, this issue never comes up. If and when it is important to “snapshot”, then pkgSnapshot or pkgSnapshot2 can be used.