The hardware and bandwidth for this mirror is donated by METANET, the Webhosting and Full Service-Cloud Provider.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]metanet.ch.
$start_workers(),
$start_local_workers(), start_worker(), and
RushWorker$new() now append a random suffix to generated
worker ids so that ids are unique by construction and can no longer
collide.$start_workers() and RushWorker$new()
now reject fractional heartbeat_period and
heartbeat_expire values and coerce integer-valued
doubles."local" and
"remote".AppenderRedis now strips custom fields from log
events in its own layout instead of mutating the shared log event.
Previously, the custom fields were also removed from the output log and
console output when output_log was set. The
filter_custom_fields() function is removed.$fail_tasks(), $finish_tasks(),
$pop_task(), and $push_running_tasks() are
moved from Rush to RushWorker so that a task
is only marked as failed or finished by the worker that processes
it.$fail_tasks(), $finish_tasks(),
$pop_task(), and $detect_lost_workers() now
change task states with guarded first-writer-wins transitions
implemented as Lua scripts. A task can no longer be recorded as finished
and failed at the same time when a live worker is wrongly declared lost.
The losing transition is discarded with a warning.$stop_workers(type = "kill") now marks the running
and pending tasks of killed workers as failed with the condition message
"Worker was killed". Previously these tasks remained in the
running state indefinitely.$push_failed() and
$push_results() are removed. Use $fail_tasks()
and $finish_tasks() instead.$empty_queue() no longer creates an orphaned hash
in the Redis database when the queue is already empty.$empty_queue() method now empties
the entire queue instead of a specified set of tasks. The
keys and conditions arguments are
removed.$detect_lost_workers() no longer marks a task as
both finished and failed, or loses a task.$detect_lost_workers() now returns only the worker
ids it actually detected as lost during the call, instead of also
including workers that terminated cleanly while the method was
running.$detect_lost_workers() no longer reports a
heartbeat worker as lost and fails its tasks when the worker terminates
cleanly while the method is running.$pop_task() no longer loses a task when a worker
crashes between popping the task from the queue and marking it as
running. The task is now moved atomically into a per-worker pending list
and recovered as a failed task by
$detect_lost_workers().$push_tasks(),
$push_finished_tasks(), $push_failed_tasks(),
and $push_running_tasks() now return early when called with
an empty list of tasks.$fetch_tasks(),
$fetch_finished_tasks(), and
$fetch_new_tasks() no longer fail when task hashes have
been removed from the database. Affected tasks are dropped with a
warning.$fetch_new_tasks() and
$fetch_finished_tasks() no longer fail after more than
100,000 tasks have been fetched. The internal counter of consumed tasks
is now stored as an integer so it can no longer be formatted in
scientific notation in Redis commands.$fetch_failed_tasks() and related methods now
return the documented condition column holding the whole
condition object.extra argument of $push_tasks(),
$push_running_tasks(), and $finish_tasks()
methods is deprecated in favor of the consistently named
xss_extra and yss_extra arguments.$start_local_workers() now redirects the standard
error stream of workers to a file instead of a pipe. Previously, a
worker writing more than the operating system’s pipe buffer to standard
error blocked forever because the pipe was only drained after the worker
terminated.$start_local_workers() no longer generates
unparseable worker startup code on Windows or when the temporary
directory path contains quotes. The temporary arguments file is now
deleted after the worker reads it.$stop_workers() no longer errors when a requested
worker id is not running. Such ids are now skipped with a warning.$wait_for_tasks() no longer reads the full
finished list and failed task set on every poll.$worker_script() now embeds values as R string
literals and shell-quotes the whole Rscript -e payload
exactly once for POSIX shells.$worker_script() no longer logs the Redis
password. The logged script shows <redacted> instead.
The method now returns the script visibly.$worker_script() now validates the
heartbeat_period, heartbeat_expire,
message_log, and output_log arguments instead
of interpolating them into the shell command unchecked.RushWorker$new() now errors when the heartbeat
process fails to set the heartbeat key within the startup timeout
instead of silently registering a dead heartbeat.
heartbeat_period and heartbeat_expire must now
be at least 1 second, and heartbeat_expire must be at least
heartbeat_period seconds.$reset() now also resets the internal log counter,
so $print_log() no longer skips messages when the same
network id is reused after a reset.$reset() now deletes all keys in a single
MULTI/EXEC transaction so a concurrent reader
can no longer observe a half-reset network.rush_plan() gains the
start_worker_timeout argument, which sets the default
timeout used by $wait_for_workers(). An explicit
timeout passed to $wait_for_workers() is no
longer overridden by the configuration.$wait_for_workers(timeout = 0) now checks once and
errors immediately if the workers are not yet registered, instead of
never checking.start_worker() no longer errors on exit when
message_log or output_log is set. The sinks
are now reverted before the log connections are closed.start_worker() now checks that
message_log and output_log are existing
directories, so a wrong path raises a clear error instead of a cryptic
“cannot open the connection”.$worker_info reads all workers in one pipelined
round trip.$detect_lost_workers() no longer creates phantom
failed tasks when a worker crashes between task evaluations (#89).$fetch_new_tasks() now correctly tracks seen
results by using the actual cache size instead of the Redis counter,
fixing cases where new results could be missed or duplicated.$write_hashes() now requires all value lists to
have the same length or length 1 instead of recycling. Length mismatches
raise an error (#87).rsh() no longer accepts ... (which
was ignored).$start_workers() now strips the enclosing
environment from worker_loop before serialization, avoiding
bloated Redis payloads when the function is a closure.$fetch_tasks() and related methods no longer fail
when task parameters contain vector values instead of scalars.$push_finished_tasks() method.$wait_for_new_tasks() and
$fetch_new_tasks() into $fetch_new_tasks()
with timeout argument.$wait_for_finished_tasks() method is
removed.data.table() instead of a list.
data_format argument is removed.with_rng_state(), is_lecyer_cmrg_seed(),
get_random_seed(), set_random_seed(),
make_rng_seeds() helper functions.$push_tasks() and $push_running_tasks()
methods.worker_extra field is renamed to
worker_id.$push_results() is renamed to
$finish_tasks().$push_results() is deprecated. Use
$finish_tasks() instead.$push_failed() is deprecated. Use
$fail_tasks() instead.$terminated_on_idle is removed.$push_priority_tasks(),
$n_queued_priority_tasks, $priority_info, and
$fetch_priority_tasks().$restart_workers() method."terminated", "killed"
and "lost" worker states into
"terminated".push_failed_tasks() for creating failed
tasks.reset_cache arguments in favor of
$reset_cache() method.network_id, config and
connector are now active bindings with validation.$snapshot_schedule() and
$redis_info() methods.$all_workers_terminated and
all_workers_lost active bindings.$worker_states to
$worker_info."rush.max_object_size" option to limit the
size of objects stored in Redis.$start_remote_workers() to
$start_workers().$reset_data() method to reset task data
without resetting the network.rush_plan() now allows script workers.$reset() instead of once per worker.$wait_for_workers() method can now wait for a
specific number of workers or a specific set of worker ids. The workers
are checked for registration in the network now.$empty_queue() method.n_workers.message_log and output_log arguments.mlr3. The mlr3/rush logger is a child of
the mlr3 logger and is used for logging messages from the
rush package.mirai package
now.$reconnect() method.Rush class was not exported.R6 package was not imported.These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.