Overview

rixpress focuses on “micropipelines”: pipelines executed on a single machine for small-to-medium projects, with reproducible environments via Nix and a simple, pragmatic user experience from R. This document clarifies what rixpress is and is not, and lays out a short roadmap so users and contributors can align expectations and proposals.

Key goals:

• Single-machine, small-to-medium pipelines (“micropipelines”).
• Reproducible builds pinned by Nix.
• Simple DAG wiring.
• Polyglot steps where R is first-class, with pragmatic Python/Julia interop.
• CI-friendly (e.g., GitHub Actions) without requiring distributed stacks.

What rixpress is

• A thin, opinionated bridge between R and Nix for building pipelines locally.
• A way to define steps (R / Python / Julia / Quarto/Rmd) as Nix derivations.
• A reproducible mechanism to:
  • parse a DAG from the steps,
  • generate and build a Nix pipeline,
  • store artifacts in the Nix store, and
  • read/load/copy outputs back into R (or interoperate with Python/Julia).
• A “just enough” visualisation layer to inspect the DAG and understand dependencies, geared toward small-to-medium graphs.

What rixpress is not

• A distributed workflow engine (no cluster schedulers, no multi-node orchestration).
• A long-running workflow daemon, task scheduler, or job server.
• A general-purpose ETL/ELT framework for very large datasets or streaming workloads.
• A feature-rich dashboarding / progress-tracking UI.
• A pluggable storage abstraction beyond the Nix store.
• An alternative DSL with complex branching semantics or dynamic targets (beyond what Nix + the current design support).

Primary audience

• R users (and teams) who want strong reproducibility via Nix without leaving R.
• Python users that are not satisfied with current solutions for micropipelines, and who are ok with defining the pipeline as an R script.
• Projects that fit on a developer’s workstation or CI runner:
  • Research, analysis, reporting.
  • Model training/evaluation at modest scales.
  • Lightweight ETL on bounded datasets.
• Teams that occasionally mix R steps with Python/Julia, without adopting a heavyweight orchestration platform.

In-scope features (current)

• Define R/Python/Julia/Quarto steps as derivations (rxp_*()).
• Generate and build a Nix pipeline (rxp_populate(), rxp_make()).
• Serialise/deserialise and basic interop (R↔︎Python/Julia) via common formats/APIs.
• Read/load/copy outputs from the Nix store (rxp_read(), rxp_load(), rxp_copy()).
• DAG generation and simple visual inspection (local and CI-friendly).
• CI support (GitHub Actions) with cache import/export helpers.

Out-of-scope features (not planned)

• Cluster execution, distributed scheduling, or remote executors.
• Complex DSLs for branching, runtime expansion, or heavy dynamic targets.
• Pluggable storage backends other than the Nix store.
• High-frequency streaming pipelines, message brokers, or long-running services.
• Large-scale progress servers/dashboards.

But depending on {rixpress}’s success and outside contributions, these features might be implemented sometime in the future.

Roadmap

This roadmap lists “near-term”, “maybe later”, and “not planned” items to clarify priorities. Timelines are indicative and may change.

How to propose new features

Before filing a feature request, please:

1. Read this scope and roadmap.
2. Check existing issues and discussions.
3. If your request is out-of-scope, consider whether it belongs in:
   • a separate package,
   • an optional companion tool, or
   • a PR to documentation (e.g., “how-to” under current scope).

When you file an issue, please: - Explain your use case and scale (single machine, data sizes). - Clarify why the feature belongs in rixpress vs. alternatives. - Suggest a minimal interface that preserves simplicity and reproducibility.

Related projects and inspiration

• {targets}, whose user-experience {rixpress} takes the most inspiration from.
• Micropipelines inspirations like Ploomber’s local-first approach (Python).
• Nix and nixpkgs for declarative, reproducible environments.
• {rix} for defining Nix environments from R.

Links

• README: https://github.com/ropensci/rixpress#readme
• Website: https://docs.ropensci.org/rixpress/
• Demos: https://github.com/b-rodrigues/rixpress_demos