The hardware and bandwidth for this mirror is donated by METANET, the Webhosting and Full Service-Cloud Provider.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]metanet.ch.
A track expression of "NULL" now errors instead of
crashing a worker process.
NA in a character column of an intervals set is no
longer turned into the string "NA" by
gintervals.rbind() and gintervals.neighbors(),
which made is.na() report FALSE.
misha’s C routines are now registered with R, so a call made with the wrong number of arguments is reported instead of silently running.
An error while gextract() or gscreen()
recovers from a failed shared-memory allocation no longer breaks the
rest of the session.
gtrack.modify() now stages its writes, so an
interrupted or failed modification leaves the track as it was instead of
half-old and half-new.
gtrack.array.import() no longer aborts the R session
when the disk fills up.
gtrack.2d.import() and
gtrack.import_contacts() now report a failed write instead
of creating a track that reads as empty.
Partial track directories left behind by a killed writer are removed when misha next opens the database.
Ctrl-C now stops indexed-format track conversions and
gseq.pwm(), gseq.kmer() and
gseq.pwm_edits(), which used to run to completion
regardless.
An interrupted track expression now reports “Command interrupted!” instead of a blank error.
An unusable gbuf.size option - too large,
NA, or not a number - is now reported. It used to break the
session’s Ctrl-C.
gbuf.size may now be given as an integer
(options(gbuf.size = 5000L)), which was previously ignored
in favour of the default.
gtrack.lookup() errors raised before the first
chromosome is written no longer carry a garbled file name.
Writing an intervals set with intervals.set.out no
longer fails with “protection stack overflow” on genomes with more than
~7,000 contigs.
gintervals.mapply(enable.gapply.intervals = TRUE) no
longer fails with “protection stack overflow” past ~8,300
intervals.
gtrack.attr.import(remove.others = TRUE) no longer
fails partway - leaving attributes applied to only the first track -
when read-only attributes are set and several tracks are listed.gseq.pwm() no longer leaves the session uncleaned when
an error occurs on its non-parallel fallback path.gpermissions.umask (default
"0007"): misha writes files as 660 and directories as 770.
Set "0002" for the previous permissions, or
NULL to follow your own umask.gseq.pwm() no longer leaks an R protection-stack slot
on every call, and when it runs on several processes it reports a
worker’s error instead of returning whatever the worker left
behind.gtrack.mv(), gtrack.copy() and
gintervals.update() as well as
gsetroot().gdataset.save() errors instead of reporting success
when a copy fails, and no longer mishandles namespaced names such as
"subdir.mytrack".gtrack.dataset() and gintervals.dataset()
no longer return NA for tracks and interval sets that
already existed.intervals.set.out arguments reject a non-string set
name with a message naming the argument order.gmax.processes now
auto-configures to at most 32 workers. Raise it by hand for very large
scans; hosts with fewer than 46 cores are unaffected.gintervals.save() call and both cross-database copy
recipes. Use gtrack.copy(src, db = target) to copy tracks
between databases.gtrack.export() names an installable converter when
bedGraphToBigWig is missing.NaN bin; gextract()’s extra column is
intervalID, not columnID; and
gsample()/gquantiles() point at
set.seed() rather than the long-removed
grnd.seed option.gquantiles(), gintervals.quantiles(),
gbins.quantiles(), or from loading an intervals set that
contains zero-length intervals - with tryCatch(warning = ),
options(warn = 2), or
testthat::expect_warning() - left the session broken: later
multitasking calls failed, and the overlapping-iterator warning fell
silent. The warnings themselves are unchanged.gtrack.liftover() of a 2D
track (rects/points) into a database with more
than getOption("gmulticontig.2d.threshold", 100) contigs -
which includes hg38 with scaffolds - wrote past the end of an internal
per-chromosome-pair buffer, crashing R or corrupting the track it was
writing. Any 2D track produced by gtrack.liftover() into
such a database before this release should be regenerated.gquantiles() returned wrong
values - zeros, and every percentile above them shifted - or aborted the
session, when it ran across processes on a scope holding more values
than gmax.data.size but fewer than
gquantile.edge.data.size (100000 by default).NULL on a database with more than
getOption("gmulticontig.2d.threshold", 100) contigs while
gmultitasking was on, and the correct result with it off.
Every 2D entry point that multitasks was affected, including
gextract(), gsummary(),
gquantiles(), gdist(), gcor(),
gscreen() and gintervals.mapply().gcis_decay() returned an
all-zero decay curve on a database with more than
getOption("gmulticontig.2d.threshold", 100) contigs - and
NULL (surfacing as “attempt to set an attribute on NULL”)
while gmultitasking was on - because it dropped every
chromosome pair from its scope. Any cis-decay curve computed on such a
database should be recomputed.score.thresh is now
mandatory for pwm.count virtual tracks and for
gseq.pwm(mode = "count"), which also rejects a threshold
vector instead of silently keeping its first element. The old default of
0 sits above the score range of any PSSM scored with a non-zero
prior, so those calls counted nothing at all; pick a
threshold from your own PSSM’s range, which pwm.max /
gseq.pwm(mode = "max") reports. With prior = 0
the default was a real exact-match threshold - pass
score.thresh = 0 explicitly and your results are unchanged.
pwm, pwm.max and pwm.max.pos
neither require nor use it.pwm.edit_distance
family, pwm.n_mutations and gseq.pwm_edits()
take the same score.thresh values as
pwm.count: a number, an integer, or the character or factor
spelling of a number that a config file or a read.csv()
column hands you. A missing (NA) threshold now errors: it
used to return exactly what an out-of-reach threshold returns - an
all-NaN column, or no rows at all from
gseq.pwm_edits() - so the two were indistinguishable.gextract()
with the same intervals as its scope and map the rows back with the
intervalID column.gextract() killed a worker
process - failing the call and deleting the session
tempdir(), with any database kept there - when it produced
more than 64 rows per iterator interval, which is what a merged block of
overlapping peaks typically does.gtrack.import() errors
when none of the chromosome names in a WIG / bedGraph / tab file exist
in the genome database, instead of silently creating an all-NaN track. A
partial match still imports the rest and now reports the names it
skipped - as a message for scaffolds/contigs, or as a warning that
aborts and leaves no track behind under options(warn = 2)
when a skipped name looks like a primary chromosome.gvtrack.iterator() sshift/eshift
pushes the iterator interval off the chromosome now returns
NaN, whatever the function. pwm*,
kmer* and masked* returned 0 or
-Inf, or a value that depended on the database format, and
errored out (“start coordinate must be lesser than end coordinate”) when
four or more sequence virtual tracks were extracted in one call;
coverage and neighbor.count returned
0, indistinguishable from a real answer. An interval that
only partly leaves the chromosome is still clamped to it, as
before.gintervals.import_bed(),
gintervals.import_gff() and
gintervals.import_vcf() now import gzipped files, and error
on a truncated or corrupt .gz instead of importing whatever
happened to decompress.gvtrack.create() rejects
parameter names that func does not accept, whether given in
params or as named arguments, and rejects mixing the two in
one call. A misspelled key was silently dropped and defaulted before,
and gvtrack.create("v", "trk", "avg", iterator = 10) now
errors instead of doing nothing.
masked.count/masked.frac keep their existing
warning.gbins.quantiles() and
gbins.summary() no longer drop the last
bin_expr/breaks pair when expr is
passed by name, which silently turned a 2-dimensional binning into a
1-dimensional one.gdist(),
gbins.quantiles() and gbins.summary() error
when a scope (or expr, for the gbins.*
functions) is given both positionally and by name; the named argument
used to be discarded silently.gtrack.ls() accepts pattern = like its
gintervals.ls()/gvtrack.ls() siblings; as with
db/perl/fixed, a track attribute
of that name can no longer be filtered by name.<groot>/.db.cache without checking which database
that listing came from. If GROOT and the listing had
drifted apart - gsetroot() interrupted with Ctrl-C
mid-reload, or misha.ext::gset_genome(force = FALSE)
restoring a memoized session - one database’s inventory was published
into another’s cache, and every subsequent session on that database saw
the wrong tracks until someone ran
gdb.reload(rescan = TRUE). The cache is now only written
when the listing provably came from that database; otherwise it is
marked dirty and rescanned.gsetroot() is no longer swallowed: it
unloads the session instead of returning as if it had succeeded.gtrack.create_pwm_energy() (and, for sequence-based virtual
tracks, gtrack.smooth(), glookup() and
gcis_decay()) opened the genome sequence before forking.
Forked workers share the file offset of an inherited descriptor, so on
an indexed database they read each other’s bytes - producing a small
fraction of silently wrong values, varying run to run. Measured at 0.2%
of bins on a 120 Mb scope.gseq.extract() distributes large interval sets (500+)
across processes when the sequence files are not already cached, which
is where it spent nearly all of its time (19,000 intervals on an NFS
database: ~200s -> ~23s). Cached extractions and densely tiled
interval sets stay in one process. Note it now forks like the other
multitasking functions, and peak memory roughly doubles while the result
is assembled; gmultitasking = FALSE restores the old
behaviour.gtrack.import() no longer fails on non-Linux platforms:
the bundled bigWigToWig is a Linux x86-64 binary, so
macOS/Windows now use one found on PATH (or
options(misha.bigWigToWig = "...")), with an actionable
error if none is installed.gtrack.create_sparse() can take the values from a
value column of intervals when
values is omitted, which rules out a values/intervals order
mismatch. Documented that gintervals() returns its rows
sorted, so a separately held value vector must be reordered too.N/*
are now scored the same on both strands in the PWM energy routines
(reverse used log(0.25), forward used the column average;
both now use the average). No behavior change for genomic scoring, where
N-windows are masked to -Inf; keeps
DnaPSSM in sync with the prego package.gintervals.quantiles()
(options(gmultitasking = FALSE)) no longer truncates the
result to the first ~1000 intervals on a large scope; the streaming
(intervals.set.out=) and in-memory results now match the
multitasking output (#149).Fixes from a full source audit (see commit messages for details):
gintervals.liftover() /
gtrack.liftover() no longer drop a mapping reachable only
through a wider earlier chain when source chains overlap
(src_overlap_policy = "keep").gtrack.export_bedgraph()
/ gtrack.export_bigwig() write coordinates as plain
integers; large round positions were previously emitted in scientific
notation, corrupting the output.gextract() (many tracks over an interval iterator) now
returns value columns in the requested track order; worker scheduling
could reorder them (column names were already correct).gintervals.quantiles()
with multiple percentiles and intervals.set.out= no longer
writes stale values for in-scope chromosomes the iterator does not cover
(now correctly NaN).pwm.edit_distance* virtual tracks no longer read out of
bounds for motifs longer than 64 bp.gextract() errors whose R message contains a
% are reported cleanly instead of crashing the
session.gtrack.create() / gtrack.rm() run from a
database subdirectory (gdir.cd()) no longer corrupt the
root track listing cache for the next session.gextract(..., file=) with very many expressions no
longer overflows its line buffer.gintervals.canonic() /
gintervals.mark_overlaps() return the correct
mapping for 2D intervals (it was inverted, breaking the
documented tapply(..., mapping, ...) pattern and 2D
mark_overlaps).gintervals.annotate()
fills na_value for query intervals that have no neighbor in
range (previously left as NA).gtrack.convert() with an explicit target track now
registers the new track (previously it was unusable, or trashed on an
indexed database, until a reload).gtrack.import_set() derives the track name from the
file name only; a dot in a parent directory no longer truncates it
(which silently collided imports).gtrack.ls(db=...) returns the tracks when only one
database is loaded (previously NULL).gtrack.var.ls() accepts an expression
(e.g. tracks[i]) as the track argument, like
gtrack.var.get/set/rm.gsetroot() leaves the current database loaded when the
target’s chrom_sizes.txt is missing or malformed
(previously it unloaded the session first).gintervals.liftover() aggregation, strand-autocorrelation
reads near contig ends, and indexed fixed-bin track validation; clearer
“data size exceeded” message.gtrack.array.extract() rejects NA column
indices instead of forwarding them.options(gmulticontig.indexed_format = TRUE).created.by provenance strings for
gtrack.2d.create(),
gtrack.2d.import_contacts(), and
gtrack.modify() are now well-formed.gmax.processes is floored at 1 (was 0 on single-core /
undetectable-core hosts).ggenome.implant() no longer leaks
buffers on error paths; corrupt-file checks in the on-disk 2D
quadtree.gtrack.liftover() now
aggregates overlapping mapped rectangles instead of inserting them as
overlapping objects (which corrupted read-back and double-counted).
multi_target_agg, na.rm, and
min_n now apply to 2D tracks too, matching the 1D path.
Overlaps arise when a chain maps disjoint source rectangles onto
overlapping target rectangles; lifted 2D track values change
accordingly.gintervals.liftover() no
longer silently drops an interval whose start coincides with its
leftmost overlapping chain when it is lifted in the same call after a
wider interval (a map_interval slow-path miss; the carried
hint had advanced past the chain). Affected multi-interval lifts under
any target policy.gtrack.liftover() now rejects the cluster
target-overlap policies (best_source_cluster,
best_cluster_union, best_cluster_sum,
best_cluster_max) with a clear error - they are implemented
only for gintervals.liftover(); previously they failed with
a confusing internal message.gtrack.liftover()
aggregation (multi_target_agg) now combines all distinct
source bins that map to a target bin, instead of keeping only the first
when several come from the same chain. Previously, with a chain whose
blocks are not aligned to the bin grid (i.e. most real liftovers),
max/sum/mean/count
reflected only the first contributing source bin.
(gintervals.liftover was already correct.) Lifted track
values change accordingly.gtrack.liftover() /
gintervals.liftover() with a minus-strand chain that the
target-overlap sweep truncates or splits (e.g. under
tgt_overlap_policy = "auto") now keep the correct reversed
source coordinates for the surviving slice instead of the coordinates of
the discarded half.gtrack.liftover() /
gintervals.liftover() no longer drop a chain’s finite
contribution to a target locus when the same chain also maps a
NaN source bin there with na.rm = TRUE; NaN
pieces are now removed before the per-chain merge.gtrack.liftover() no
longer drops rectangles when the chain has multiple blocks (the
quadtree’s spatial iteration order defeated a carried mapping
cursor).gintervals.liftover() /
gtrack.liftover() no longer drop a mapping through a wider
earlier source chain when source chains overlap
(src_overlap_policy = "keep"). The
map_interval fast path trusted a carried cursor as the
leftmost overlap after checking only its immediate predecessor; it now
confirms leftmost-ness via the per-chromosome prefix-max, falling back
to the full search otherwise.gtrack.liftover(tgt_overlap_policy = "agg") no longer drops
a lifted contribution that spans a target bin boundary when an
overlapping chain shares its interval. The per-bin sweep over-advanced
its cursor, losing the spanning contribution for the next bin and
producing spurious NaNs (or wrong aggregates) in bins fed
by multiple overlapping chains.gintervals.load_chain(..., src_groot=) no longer leaves the
session pointing at the source database after validation. It now fully
restores the active database - the working directory (GWD),
loaded datasets and virtual tracks included, not just the genome
root.gquantiles() /
gintervals.quantiles() name percentile columns with the
shortest decimal that round-trips each percentile, so nearby percentiles
no longer collapse to the same name - e.g. 0.123456789 and
0.1234567891 previously both became
"0.123457", and 0.9999999 became
"1" (colliding with the 1.0 quantile). Common
percentiles (0.5, 0.95, …) are unchanged.gextract() no longer
truncates auto-generated column names to 40 characters - columns are
named after the full expression (the complete track name) instead of
<first 37 chars>.... Long names that previously
collapsed to the same truncated name, hiding a column, are now distinct.
Pass colnames= to set names explicitly.max.pos.*/min.pos.*,
mixed-function, and avg+nearest virtual tracks
could read stale values for the empty contig instead of NA.
Single-function avg/sum/lse
tracks were unaffected.pwm.edit_distance /
pwm.edit_distance.pos / pwm.max.edit_distance
with direction = "below" and max_edits set no
longer skip N-containing windows. An N is not a mandatory edit when
going below threshold, so an N-heavy window already scoring below
threshold needs 0 edits; the N-count skip (added in 5.6.11) wrongly
pruned such windows, returning NA or missing the true
minimum. The skip is now restricted to direction = "above"
substitutions-only, where it is exact.lse
(or sum/exists/size) virtual
track scanned over a sliding window
(gvtrack.iterator(sshift=, eshift=)) genome-wide is fast
again. Since 5.6.7 a single-function “fast path” recomputed the windowed
reduction from scratch on every step, bypassing the incremental
sliding-window path; the common motif-energy quantile workload (windowed
lse + gquantiles/gscreen) was
~2.5x slower (worse with wider windows). Such single-function reducers
now keep the sliding-window path. Output is unchanged.gtrack.copy() when the destination DB has a leading chrom
that was not in the source’s per-chrom files) no longer report
bin.size = 0 from gtrack.info() and no longer
SIGFPE on subsequent reads. GenomeTrackFixedBin::init_read
now back-fills m_bin_size from the first non-empty index
entry on a length-0 lookup, instead of leaving it at the constructor
default.gtrack.info() on a
per-chromosome dense track whose first genome chrom has no per-chrom
file (an empty scaffold, or a partial track) now reports the correct
bin.size instead of erroring with “No such file or
directory”. This also unblocks gtrack.copy() of such
tracks, which calls gtrack.info() on the source.gdb.install_intervals() /
gdb.build_genome() now attach name (transcript
accession) and geneName (gene symbol) columns to the
installed
tss/exons/utr3/utr5
sets, taken from the source annotation. Sources without symbols build
cleanly with a blank geneName.distance,
distance.edge, and distance.center
virtual-track functions now always return the true nearest source
interval. With overlapping or nested sources
(e.g. intervs.global.rmsk) the old scan could return a
non-nearest interval - even a nonzero distance for a query that overlaps
the source. distance.edge again matches
gintervals.neighbors exactly.distance.center now accepts overlapping source
intervals (a query center inside several resolves to the nearest center)
instead of erroring at vtrack creation.gintervals.2d.intersect() and
gintervals.2d.union() for general 2D (rectangle) interval
set operations, complementing the existing
gintervals.2d.band_intersect(). Intersection clips
overlapping rectangles pairwise per chromosome pair; union concatenates
without merging (the union of two rectangles is not a rectangle).gintervals.from_strings() builds 1D intervals from
UCSC-style coordinate strings such as "chr1:1000-2000"
(optional :+/:- strand; a bare
"chr1" is the whole chromosome).gvtrack.clear() removes all virtual tracks of the
current working directory in one call.gdb.unload() unloads the active genome database and
resets the session to an uninitialized state.gtrack.import_mappedseq() and the
strand-autocorrelation importer now print a correct message (instead of
a garbage value or a possible crash) when given a non-positive
cols.order entry; the latter also reports the offending
maxread rather than binsize.ggenome.implant() validates the output trackdb
destination before writing the FASTA, so a pre-existing trackdb with
overwrite = FALSE no longer leaves an orphaned output file
behind.gintervals.force_range() is markedly faster when
writing to an output set (or with big-set inputs) on genomes with many
contigs: the per-chromosome chromosome-end lookup is computed once
instead of being re-derived from the full genome on every
chromosome.gtrack.import_mappedseq() now accepts BAM files
directly: bgzip magic bytes are auto-detected and the file is streamed
through samtools view into the existing parser. Requires
samtools on PATH (declared in
SystemRequirements); a clear error names the missing tool
otherwise. For BAM input the function treats the default
cols.order = c(9, 11, 13, 14) as SAM mode
(i.e. NULL); explicitly passing a non-NULL
cols.order with a BAM file is rejected with an error.gtrack.import_mappedseq() also auto-detects plain
gzipped SAM / TSV input (.sam.gz, .tsv.gz) via
zlib. No user-facing API change.gintervals.load() of a
1D/2D big intervals set whose meta-stats has exactly one row was
returning the wrong slice (typo meta$stat ->
meta$stats).gtracksummary() /
gintervals.summary() standard deviation now returns 0
instead of NaN on nearly-constant input; same catastrophic-cancellation
clamp applied in Spearman correlation.gtrack.rm() /
gintervals.rm() / gdir.rm() in a
non-interactive session without force = TRUE now stop with
a clear message instead of crashing on readLines(n = 1)
returning length-zero.gsynth.random() rejects
nuc_probs with duplicated or missing names (previously
silently produced NaN probabilities).gtrack.import() accepts bedGraph from genomes whose
contig names don’t start with chr (Ensembl-style,
non-mammalian, scaffolded contigs).RSaneSerialize/RSaneUnserialize, and
Computer2D::unserializeComputer2D.sprintf args, broken format specifiers) and defensive
emptiness handling (1:length() / 1:nrow()
-> seq_along() / seq_len())..gintervals.apply and the gintervals FUN closures
(gintervals.diff / .intersect /
.union / .canonic / .normalize /
.rbind / .neighbors /
gintervals.mapply). ~70× speedup at 5000+ chromosomes;
previously catastrophic on million-contig scHi-C-scaffolded
genomes.gseq.pwm_edits() chromosome-boundary clamp vectorized:
O(N) instead of O(N · scaffolds).gdb.reload() now also clears the C++ track-index cache,
so a track replaced out-of-process (sibling R session, manual rebuild
from another script) is picked up by the running session without
restarting R. Previously the cached track type could shadow the new
on-disk track and surface as
e.g. function global.percentile.max is not supported by sparse tracks
after the track was already rebuilt as dense.gtrack.import() of a malformed WIG/CSV file now
surfaces the underlying parser error (file name, line number, what was
wrong) instead of the opaque
Unrecognized format of file .... Makes typical failure
modes (e.g. a value line glued to a fixedStep header from a botched
cat-style concatenation) diagnosable at a glance.gextract() gains an intervals_join
argument with values "id" (default; current behavior),
"intervals" (drop intervalID, attach the input
intervals data frame’s columns to each output row), or
"none" (drop intervalID). Built-in replacement
for misha.ext::gextract.left_join, around 2x faster on
wide-window workloads (e.g. TSS +/- 5kb at 50bp iterator).gtrack.import() of BED files with
binsize failing with
func argument is not a string. The BED-to-dense path was
not updated when gtrack.create_dense() gained the
func argument in 5.6.31; it now passes the historical
"weighted.mean" default.gintervals.to_mat() and
gintervals.from_mat() round-trip an intervals + values
data.frame to a numeric matrix indexed by intervals. Pass
labels = FALSE to skip rowname construction.gtrack.create_sparse(),
gtrack.create_dense(), gtrack.create(),
gtrack.smooth(), gtrack.modify(),
gtrack.convert_to_indexed()) coalesce into far fewer
syscalls on networked filesystems by bumping the stdio buffer to 1 MiB
on every misha-managed write file. Noticeable on NFS-mounted track
DBs.gtrack.create_sparse() on per-chromosome (non-indexed)
DBs now creates the empty-chrom signature files in parallel via raw
POSIX syscalls. Worker count is configurable via the
gtrack.create.threads option (default 4, matching NFSv3
CREATE concurrency).gtrack.rm("X") followed
by gtrack.create_*("X", ...) (and the analogous
gintervals.rm + gintervals.save cycle) on the
same indexed DB no longer breaks subsequent reads. The process-static
index caches keyed by track/intervals directory path were never
invalidated, so after rm-then-recreate readers could route to a stale
layout and error with
Cannot open track.dat: No such file or directory or
silently return data from the prior lifecycle. Latent since v5.1.1 (1D
tracks and interval sets) and v5.5.0 (2D tracks).gtrack.create_dense() returning wrong values (and
func = "coverage" exploding to huge numbers) when
overlapping intervals of mixed lengths fed into the same bin. Older
callers using the default func = "weighted.mean" got
plausible-looking but algebraically wrong means in affected bins.gtrack.create_dense() gains a func
argument selecting the per-bin aggregation over overlapping intervals:
"weighted.mean" (default, unchanged from before),
"weighted.sum", "max", "min",
"median", "count", and "coverage"
(= sum(v_i * ov_i) / binsize, i.e. average per-base signal;
with values = rep(1, n) this is a ChIP-seq-style pileup
track in one call).gdb.init_examples() on some R installations.gdb.install_intervals(..., sets = "genes") from NCBI no
longer aborts on a handful of unconvertible GFF records (typically the
immunoglobulin loci in RefSeq human/mouse): gff3ToGenePred
is now invoked with -warnAndContinue, so the offending
transcripts are skipped with a warning and the rest of the genome is
installed.gtrack.rm() (and gintervals.rm()) now
returns immediately even when the target directory contains millions of
files; the actual filesystem cleanup runs in the background.gtrack.create() and friends) is now
atomic on databases with many contigs: an interrupted create no longer
leaves a partial track directory that blocks re-creation. Cleanup of
failed creates runs in the background.gtrack.create() for dense
(fixed-bin) and sparse tracks now writes the indexed format
(track.dat + track.idx) directly instead of
writing N per-chromosome files and then deleting them. Substantially
faster on databases with >1M contigs.O(num_contigs^2) per-pair entries.
gtrack.create_meta() for 2D tracks uses
track.idx (indexed) or a directory scan (legacy) to
enumerate populated pairs instead of probing all N*N. Enables 2D tracks
on databases with >1000 contigs without OOM or multi-day metadata
scans.gdataset.info() and
gdataset.ls(dataframe=TRUE) now use a single C++ filesystem
traversal (with FTS_SKIP) instead of two recursive R-level
list.files() calls. Faster on databases with many
tracks/intervals when .db.cache is missing or stale.gtrack.liftover() now writes the indexed format
directly when the target database is indexed, avoiding ~1M
per-chromosome empty-placeholder file creations when lifting onto a
many-contig genome.gdb.convert_to_indexed(convert_tracks = TRUE) and
convert_intervals = TRUE accept a new threads
argument (default: min(detectCores(), 8)) and run per-track
conversions in parallel via parallel::mclapply. Per-track
failures are reported as warnings without aborting the batch.gtrack.create_meta() for sparse/array tracks on indexed
databases now consults track.idx to find populated
chromosomes instead of stat’ing per-chromosome files that don’t exist.
Drops a one-time O(N_contigs) syscall cost from the first use of a
sparse/array track.gextract() / gscreen() and other
track-expression iterators now validate indexed 1D tracks via
track.idx instead of stat’ing it once per chromosome. On a
1.28M-contig database, gextract of 100 random 5 bp
intervals from a dense indexed track drops from ~64 s to ~8 s (system
time from ~52 s to ~2 s).gintervals.random() rejecting chromosomes whose
length exactly equals size + 2*dist_from_edge (where a
single valid interval is available) and biasing sampling away from the
right edge. The no-filter path now uses the same full-range semantics as
the filter path.gtrack.info() drops from ~1.4 s to
<1 ms and gseq.extract() from ~14 s to ~0.4 s; smaller
genomes are unaffected.gsetroot() is ~3x faster on million-contig databases
(~13 s -> ~5 s on a 1.28M-contig DB).gextract() and other scan operations are faster on
indexed tracks of databases with many chromosome aliases (skips a
per-contig alias lookup that has no effect on indexed tracks).gdb.install_intervals() from ncbi now
translates chrom names to the groot’s convention, fetches
genes and rmsk from the NCBI FTP when the
Datasets API doesn’t ship them (older / suppressed accessions like
GRCm38.p6), and no longer downloads the FASTA.min_coverage is now checked after the alias rescue
passes, so hybrid naming (e.g. UCSC-style chroms mixed with bare GenBank
accessions) passes at the strict default.gdb.install_intervals() now errors if any requested set
is unavailable from the source (previously skipped silently with a
warning). Pass force = TRUE to demote the error to a
summary warning and install the available sets.gtrack.copy() gained a db argument to copy
tracks across databases, and an overwrite argument to
replace existing destinations. Format conversion (per-chromosome
<-> indexed) and chromosome-order remap are handled automatically.
Multi-track input is also supported.gdb.build_genome(name) for build-from-source
genome creation. Backends: ucsc (golden path),
ucsc-hub (224/241 Zoonomia mammals), ncbi
(Datasets API), manual, local,
s3. name resolves through a registry chain
(misha.genome_registry option, project-local
misha.yaml, built-in inst/genomes.yaml), with
a fallback to ucsc-hub for GC[FA]_*
accessions. Companions: gdb.list_genomes(),
gdb.genome_info().gdb.install_intervals() to layer gene /
rmsk / cgi / cytoband sets onto an existing
groot - useful for adding annotations to a private FASTA build or
resuming a failed install without re-fetching. rmsk now
also produces per-class subsets rmsk_<class>
(e.g. rmsk_sine).target_chroms / target_lengths /
match_by_length (default TRUE) /
chrom_naming / min_coverage arguments: misha
picks the chromAlias column that best matches the target and falls back
to per-row length pairing when a single column doesn’t cover everything.
UCSC-hub builds also merge <acc>_assembly_report.txt
when present to expose extra alias columns. Coverage is bp-weighted on
the groot side, so small unmapped contigs (e.g. a 16 kb mitochondrion)
don’t trip thresholds.dataset_report.
Missing-annotation builds drop genes with an actionable
warning (suggesting a RefSeq companion if one exists). Post-pre-flight
failures clean up the partial output directory.gdb.install_gff3_converter() /
gdb.install_gtf_converter() to pre-install the UCSC
gff3ToGenePred / gtfToGenePred binaries;
override paths via MISHA_GFF3_TO_GENEPRED /
MISHA_GTF_TO_GENEPRED. UCSC’s prebuilt binaries need glibc
>= 2.34; older systems can
conda install -c bioconda ucsc-gff3togenepred ucsc-gtftogenepred.gdb.build_genome():
argument annotations -> sets; track
cpgIsland -> cgi.gsetroot is much faster on fragmented assemblies
(e.g. ~93 s -> ~11 s on a 2.4M-contig genome).
gdb.build_genome() and gdb.install_intervals()
are also substantially faster on large assemblies (vectorized chromAlias
translation and RepeatMasker .out parsing, optional
data.table for genePred I/O).gpartition, gquantiles and other
consumers of BinFinder silently routing every value into
the highest bin when the breaks vector contained both -Inf
and +Inf (e.g. c(-Inf, 0, Inf)). The
uniform-binsize fast path computed Inf/Inf = NaN whose cast
to int is undefined behaviour. BinFinder::init now falls
back to binary search when binsize is non-finite.gsynth.sample() and gsynth.random() now
preserve N (and lowercase n) positions from
the original reference by default. Previously, every position was filled
with a sampled/random ACGT base, so reference gaps and centromeres came
out as fabricated nucleotides. Pass preserve_n = FALSE to
restore the legacy behavior.gquantiles (gmultitasking = FALSE)
hanging for hours on the first global.percentile.max lookup
of a fresh dense track (regression in the 5.6.20 single-process fast
path). Above ~2*log_2(N) target ranks the fast path now sorts the
reservoir once and indexes; below it the suffix walk is preserved.
Single-process and multitask results remain bit-identical.gscreen regression introduced in
5.6.7 by sub-chromosome range splitting: with iterator=1 on
a ~5 Mb scope, range-split forks were too small to amortise fork
overhead. Added a 500 kbp-per-kid floor; full-genome scans still cap at
gmax.processes kids. Measured chr19:0-5 Mb gscreen: 1.47 s
-> 0.31 s.CHROM_ALIAS.gintervals.import_bed(),
gintervals.import_gff(),
gintervals.import_vcf() for direct import from common
interval file formats. All three normalize chromosome names via
CHROM_ALIAS, apply misha’s 0-based half-open convention
(subtracting 1 from start for 1-based GFF/GTF/VCF inputs), and preserve
common metadata columns
(name/score/strand for BED;
type/source/score/attrs
for GFF;
id/ref/alt/qual/filter/info
for VCF).strand column now accepts character
("+", "-", ".", "*",
"") or factor input in addition to numeric
1/-1/0. Strings are normalized to
the numeric convention at the R->C++ boundary; output stays
numeric.prior argument to gsynth.train()
(default "marginal"). Per-bin Dirichlet priors are now
learned from the trainer’s own counts by default, so unobserved (cell,
k-mer-context) entries fall back to the cell’s empirical base
composition instead of uniform 1/4. Other modes: "global",
NULL/"uniform", length-4 numeric, and
n_bins x 4 matrix.(N + alpha) / (sum_a N + 4*alpha) to
(N + alpha * pi_a(b)) / (sum_a N + alpha) (pi sums to 1).
To reproduce the pre-5.6.21 Laplace-add-one behavior, pass
prior = NULL, pseudocount = 4..gsm metadata gains optional
prior/prior_mode fields; older files load with
prior = uniform (their on-disk CDF is the truth).gquantiles hanging for many minutes on dense
binsize=1 whole-genome scans. Kids now sort their samples
buffer before packing and the parent does a heap-based k-way merge
instead of a global single-thread sort; the single-process path uses
std::nth_element per percentile rank. mm10 dense
full-genome gquantiles: was hung, now ~75 s (multitask) /
~270 s (single-process).gintervals.load failing with “invalid columns
definition” after gintervals.save of a bigset whose input
had character chrom (e.g. a tibble from
dplyr). On-disk per-chromosome files and the
.meta zeroline now both store
chrom/chrom1/chrom2 as factor
with full ALLGENOME levels, and the on-disk frame is normalized to plain
data.frame.gsynth.score(): writes a misha fixed-bin dense
track of summed natural-log conditional probability under a trained
stratified Markov-k model. Two scored tracks subtracted at any
resolution give a windowed log-Bayes-factor. mask argument
NA-poisons positions inside the mask (e.g. repeats); predicted-base
N is unconditional NA.gsynth.sample() now looks up the stratum bin at
pos - k to match gsynth.train(). Previously it
queried the bin at the predicted-base position, which differed from
training at the first k bp of every iter window. Cached
.gsm models are unchanged; samples generated with earlier
versions had a slight stratum-shift artifact at iter boundaries.getOption("gmultitasking.strategy") for
gextract (default "auto"). For large
many-track workloads (>= 8 tracks, non-streaming iterator)
auto routes to a track-parallel mode (each worker handles a
track subset across all tiles) instead of the legacy tile-parallel mode.
Realistic 3,110 motif tracks x 2.19M peaks: 57.6 min vs ~3.4 h projected
(3.5x per-track speedup). Override with
options(gmultitasking.strategy = "tracks" | "tiles" | "auto").gextract over thousands of tracks failing with
“Too many open files” (EMFILE). BufferedFile handles are
now released once data is loaded (sparse) or mmapped (dense), so the
open-FD count no longer scales with track count. Negligible perf
cost.gextract regression introduced in
v5.6.11 where calls over ~50 motif tracks became 10-20x slower.
MmapFile no longer eagerly pages with
MAP_POPULATE, and track-validation loops use a
metadata-only path (stat for size, read bin_size once)
instead of round-tripping init_read() per chromosome per
track. Tamar’s 51-motif workload: 22 s -> 0.4 s.MISHA_PERF_TESTS=true) covering many-track
gextract setup overhead.gsynth.sample() with
output_format = "fasta" now writes a samtools-compatible
.fai alongside the FASTA, removing the need to call
samtools faidx by hand. Matches the convention already used
by ggenome.implant().cell_merge argument to
gsynth.sample() and a companion exported utility
gsynth.cell_merge(). Unlike bin_merge (which
redirects bins independently along each dimension),
cell_merge redirects specific per-joint-cell CDFs to other
joint cells
(e.g. (GC=0.725, CG=0.05) -> (GC=0.70, CG=0.08)).
Intended for reassigning under-trained joint cells to a
nearest-sufficient neighbor to avoid sequence leakage from tiny training
sets.gsynth.forbid_kmer(model, pattern) returning a
new gsynth.model whose samples are guaranteed pattern-free
downstream of the seeding window
(e.g. gsynth.forbid_kmer(model, "CG") for a CpG-null
background). Pattern length is capped at model$k + 1.gsynth.sample() silently falling back to
uniform-random sampling for most positions inside any interval whose
start was not aligned to model$iterator (#94). Output k-mer
statistics on unaligned intervals were wrong; composition was partially
uniform.gsynth.train(), gsynth.sample(), and
gsynth.random_seqs() silently reading sequences from the
wrong chromosome when intervals omitted one or more earlier
chromosomes in the chromkey - the C++ side opened the wrong sequence
(shifted by the number of earlier missing chromosomes), producing
invalid models and corrupted sampled genomes without any error. Calls
that passed intervals = gintervals.all() (or left it at the
default) were not affected. Users who ran these functions on custom
interval subsets should re-run them with this version.track.dat
for every chromosome during iterator init and chromosome transitions.
Made indexed tracks unusable on genomes with many contigs
(e.g. Pan_troglodytes with 4344 contigs).pwm.max.pos returning wrong strand sign. The
direction (positive/negative) of the returned position was determined by
the last scanned position in the interval rather than the position with
the best score. Bug existed since pwm.max.pos was
introduced (v4.3.0).rexit() due to R’s
SIGTERM handler. All multitasked operations
(gmultitasking = TRUE) were affected. Track-creating
functions (gtrack.create, gtrack.smooth,
gtrack.create_pwm_energy) could corrupt data on indexed
databases because children ran gtrack.convert_to_indexed
concurrently. Non-indexed track data and query functions
(gextract, gscreen, gdist,
gquantiles, …) were unaffected (data was written to shared
memory before the failed exit).ggenome.implant() for replacing intervals in a
reference genome with donor sequences and writing a new FASTA. Supports
literal donor sequences or extraction from a misha database, with
optional trackdb creation.ggenome.transplant() as sugar for cross-genome
sequence swaps - extracts from a source genome and implants into a
target genome in a single call.direction="below" with bidirect=TRUE
taking min across strands instead of max. A genomic substitution changes
both strands, so disrupting a motif requires both strands to fall below
the threshold.score.min = score.thresh when
direction = "below". score.min now defaults to
NULL (no filter) for both directions. Set score.min
explicitly to filter windows.k parameter to gsynth.train() to
configure the Markov order (1-8, default 5).gextract with PWM vtracks, gseq.pwm, PWM edit
distance). Replaced switch-statement DNA base encoding with O(1) lookup
tables in the inner scoring loop.DnaPSSM::integrate_energy
(case 'h' for case 'g') on the
reverse-complement scoring path. The affected function is dead code (all
user-facing callers use integrate_energy_logspat), so no
user-visible results were affected.direction parameter (“above”/“below”) to PWM edit
distance functions for computing minimum edits to bring score below a
threshold (motif disruption).pwm.edit_distance, pwm.edit_distance.pos,
pwm.max.edit_distance, pwm.edit_distance.lse,
and pwm.edit_distance.lse.pos. Compute minimum edit
distance (substitutions and indels) to reach a PWM score threshold, with
per-window (max) and aggregate (LSE) scoring modes.gseq.pwm_edits() for retrieving detailed per-edit
information (positions and replacement bases) from PWM edit distance
computation.gscreen and
gextract, improving parallel efficiency on genomes with
large chromosomes.gscreen returning split intervals at
sub-chromosome parallel boundaries instead of merged contiguous
intervals.gsummary,
gdist, and gcor (Pearson), improving parallel
efficiency with explicit iterators.gscreen and
gextract with dense iterators.pipe.Rd documenting
%>% with a \usage section.Rf_findVar with
R_getVar for R 4.6.0 compatibility.gintervals.load() failing with “Intervals set
does not exist” for intervals from databases loaded via
gdataset.load().Fixed gdataset.load() performance: loading a large
database as a dataset is now fast regardless of call order. Previously,
gsetroot(small_db); gdataset.load(large_db) was extremely
slow because the dataset scan used R’s
list.files(recursive=TRUE), which lists every file in the
tree. Replaced with a fast path that reads .db.cache or
falls back to C++ fts-based scanning. Also changed
gdb.reload() calls in
gdataset.load()/gdataset.unload() to use
rescan=FALSE, so existing databases use their cache instead
of being rescanned.
Fixed collision detection for hierarchical tracks in
gdataset.load(): the R-level scan returned
/-separated names while the dataset map uses
.-separated names, causing collision detection to silently
fail for tracks in subdirectories.
Added motif format import functions:
gseq.read_meme(), gseq.read_jaspar(), and
gseq.read_homer() for reading MEME, JASPAR PFM, and HOMER
motif formats. Returns named lists of position probability matrices
directly usable with gseq.pwm(). All parsers are native
with no new dependencies.
Added track export functions:
gtrack.export_bedgraph() and
gtrack.export_bigwig() for exporting tracks and track
expressions to standard bedGraph and BigWig formats. Supports gzip
compression, virtual tracks, track expressions, and custom
iterators.
gsynth.save() and gsynth.load() now use
the cross-platform .gsm format (YAML metadata + binary
arrays) instead of R-specific RDS. Models saved with pymisha can now be
loaded in R and vice versa. Legacy RDS files are still supported for
backward compatibility.
Added compress parameter to
gsynth.save() to optionally save as a ZIP archive.
Added gsynth.convert() to convert legacy RDS model
files to the new .gsm format.
Added gintervals.attr.get(),
gintervals.attr.set(),
gintervals.attr.export(), and
gintervals.attr.import() for managing interval set
attributes. Attributes are stored as .iattr binary files
(null-separated key/value pairs) next to .interv files for
small interval sets, or inside the directory for big interval
sets.
gintervals.rm() now cleans up companion
.iattr attribute files when deleting interval
sets.
Replaced the naive variance formula (E[X²]-E[X]²) with Welford’s numerically stable online algorithm for standard deviation computation in all track types (GenomeTrackFixedBin, GenomeTrackSparse, GenomeTrackInMemory, GenomeTrackArrays). The naive formula is prone to catastrophic cancellation when values are large or have small variance relative to their mean.
Improved sum accumulation precision by using double-precision
intermediate accumulators in all non-sliding sum paths. The
m_last_sum member remains float for API compatibility, but
per-interval accumulation now happens in double, eliminating
float->float rounding errors for large sums.
Fixed potential out-of-bounds access in SAMPLE/SAMPLE_POS functions when the random number generator returns exactly 1.0. Added bounds clamping (ported from pymisha).
Fixed umask leak in GenomeTrack::write_type() and
GenomeTrackFixedBin::init_write() - the old umask is now
saved and restored immediately after open(), preventing
permanent process-wide umask changes.
Improved header validation in
GenomeTrackFixedBin::init_read() to use integer modulo
instead of floating-point division, with an additional underflow
guard.
Added reusable scratch buffers
(m_scratch_all_values,
m_scratch_all_positions) to GenomeTrackFixedBin, avoiding
per-call heap allocation for SAMPLE/SAMPLE_POS operations (ported from
pymisha).
Added indexed format support for 2D tracks (rectangles and
points). Per-chromosome-pair files can now be consolidated into a single
track.dat + track.idx, matching the indexed
format already available for 1D tracks.
New function gtrack.2d.convert_to_indexed() converts
existing 2D tracks to indexed format with optional removal of old
per-pair files.
gtrack.convert_to_indexed() now dispatches to 2D
conversion automatically when given a rectangles or points
track.
gdb.convert_to_indexed() with
convert_tracks = TRUE now includes 2D tracks in batch
conversion.
2D tracks created via gtrack.2d.create(),
gtrack.2d.import(), and
gtrack.2d.import_contacts() are automatically converted to
indexed format when the database is in indexed mode.
Fixed NaN comparison bug in distance.center virtual
track with iterator modifiers - used != instead of
std::isnan(), causing the guard condition to always be true
under IEEE 754 semantics.
Fixed distance virtual track returning NaN instead
of a valid distance when one side of the sorted interval array has no
intervals on the current chromosome (NaN-unsafe min in
sequential path).
Fixed gtrack.smooth MEAN algorithm producing
spurious NaN values during periodic recalculation when NaN values are
present in the smoothing window. The recalculation checked
isnan on the accumulator instead of the incoming value,
permanently poisoning the sum.
Fixed incorrect standard deviation/variance computation in virtual tracks with filter/mask - the parallel variance formula used the already-incremented weight instead of the pre-increment value.
Fixed min.pos/max.pos virtual track
aggregation comparing genomic positions instead of signal values when
combining sub-intervals, returning the leftmost/rightmost sub-interval
position instead of the position of the actual min/max value.
Fixed missing out_of_range check in
distance.center virtual track with iterator modifiers,
causing undefined behavior when all source intervals lie upstream of the
query coordinate.
Fixed copy-paste error in quantile track computation where
highest_vals buffer was resized using
kid_lowest_vals_buf_size instead of
kid_highest_vals_buf_size.
Fixed int32 truncation of int64_t
genomic coordinates in gintervals.normalize - could
silently produce wrong results for chromosomes longer than ~2.1
Gbp.
Fixed p-value transformation (pv function) being
skipped when virtual track has both a filter and a p-value function
active.
Fixed lse_accumulate double overload producing NaN
when both inputs are -infinity (missing isinf
guards that the float overload had).
Fixed GInterval::dist2coord treating
coord == end as inside the interval, inconsistent with the
half-open [start, end) convention used throughout the
codebase.
Fixed distance.center virtual track function
returning incorrect values when gextract is called with
overlapping input intervals. The sequential scanner could miss
containing source intervals when processing bins that go backward in
position between overlapping regions.
Fixed neighbor.count virtual track function
undercounting neighbors when gextract is called with
overlapping input intervals. Same root cause as the
distance/distance.edge fix in 5.4.6.
distance and distance.edge virtual
track functions returning incorrect values when gextract is
called with overlapping input intervals. The sequential scanner could
miss closer source intervals when processing bins that go backward in
position between overlapping regions.GenomeTrackBinnedTransform.cpp by avoiding arithmetic
between distinct anonymous enum types.gdb.export_fasta() to export all contigs from a
database to a multi-FASTA file, defaulting to the current gdb with
optional groot override.gscreen and
gextract with dense iterators.lse virtual track function that computes the
log-sum-exp of the values in the iterator interval.gsetroot() for your primary writable database, then
gdataset.load() to add read-only datasetsforce=TRUE
to override (working db always wins)gdataset.load(): Load a dataset into the namespace
(tracks and intervals become available)gdataset.unload(): Remove a dataset from the
namespacegdataset.save(): Create a new dataset from selected
tracks/intervalsgdataset.ls(): List working database and all loaded
datasetsgdataset.info(): Show metadata and contents of a
datasetgtrack.dataset(): Get the source path for a track
(working db or dataset)gtrack.dbs(): Get all paths where a track exists (for
debugging shadowed tracks)gintervals.dataset(): Get the source path for an
interval setgintervals.dbs(): Get all paths where an interval set
existsdb parameter for filtering by source:
gtrack.ls(db = "/path/to/dataset"): List tracks from a
specific sourcegintervals.ls(db = "/path/to/dataset"): List intervals
from a specific sourcechrom_sizes.txt files
(same genome assembly)gsetroot() works unchangedgtrack.mv(): Rename or move a track within the same
databasegtrack.copy(): Copy a track (can copy between databases
when multiple are loaded)gcor function that computes correlation between
two tracks, or between multiple pairs of tracks.gextract and gscreen almost always
did not enable multitasking mode due to incorrect gating.dataframe and names parameters to
gdist function that return a data frame instead of an
N-dimensional vector.gsynth.train, gsynth.sample and
gsynth.save functions that train a Markov model from a
genome sequence and sample a synthetic genome from the modelgseq.kmer.dist function that counts the number of
occurrences of k-mers in genomic intervals.gtrack.liftover that created overlapping
intervals when lifting sparse tracks.gintervals.normalize.gintervals.normalize now
returns +1bp for intervals with odd sizes:
interval_relative parameter to
giterator.intervals() for interval-aligned bin
iteration.genome.seq +
genome.idx files instead of per-chromosome filesgdb.info(),
gdb.convert_to_indexed(),
gtrack.convert_to_indexed(),
gintervals.convert_to_indexed(),
gintervals.2d.convert_to_indexed()options(gmulticontig.indexed_format = FALSE) to
create databases in legacy format for compatibility with older misha
versionsvignette("Database-Formats") for more details.gmax.processes automatically set to 70% of available
CPU coresgmax.data.size coordinated with process limits to
ensure total memory usage <= 70% of RAM (capped at 10GB per
process)gmax.data.size = min((RAM * 0.7) / gmax.processes, 10GB)
ensures safe memory usage across all parallel processesgmax.processes * 1000 records (e.g., 2K on laptops, 89K on
128-core servers)options()vignette("Manual")
for detailsgvtrack.create with src parameter). These
tracks behave exactly like regular sparse tracks, but are stored in
memory and can be used in track expressions.sshift, eshift and
filter parameters to gvtrack.create.gintervals.path() and gtrack.path()
functions that return the actual file system paths for interval sets and
tracks.masked.count and masked.frac virtual
track functions that count and fraction masked base pairs (lowercase
letters) in the current iterator interval.distance.edge virtual track function that
computes edge-to-edge distance from the iterator interval to the closest
source interval, using the same calculation as
gintervals.neighbors.gtrack.liftover did not fill chromosomes missing
the chain with NA values. This caused errors when trying to access the
tracks afterwards.gintervals.as_chain function that converts a data
frame to a chain object.gintervals.liftover via value_col and
multi_target_agg parameters.src_overlap_policy and
tgt_overlap_policy parameters to
gintervals.liftover, gintervals.load_chain,
and gtrack.liftover functions.gtrack.liftover via
multi_target_agg parameter.gintervals.load_chain now returns valid misha intervals
instead of a chain object.gintervals.load_chain now includes score
and chain_id columns for all loaded chainsmin_score parameter in
gintervals.load_chain, gintervals.liftover,
and gtrack.liftover filters out low-quality chainstgt_overlap_policy = "auto_score" (or
"auto") selects the best chain mapping based on alignment
score (highest score -> longest span -> lowest chain_id)include_metadata parameter in
gintervals.liftover optionally returns score and chain_id
for each mapping BREAKING: “auto” is now an alias for
“auto_score”. For the old behavior, use
tgt_overlap_policy = "auto_first".canonic parameter to
gintervals.liftover (default FALSE) to merge
adjacent target intervals resulting from the same source interval and
chain.tgt_overlap_policy = "best_cluster_union" (default,
aliased as "best_source_cluster"): Uses source union
coveragetgt_overlap_policy = "best_cluster_sum": Uses sum of
target lengthstgt_overlap_policy = "best_cluster_max": Uses longest
single membermax.pos.abs, max.pos.relative,
min.pos.abs, min.pos.relative: Returns the
position of the maximum/minimum value in the iterator intervalexists: Returns 1 if any value exists (or specific vals
if provided), 0 otherwisesize: Returns the number of non-NaN values in the
iterator intervalsample: Returns a uniformly sampled source value from
the iterator intervalsample.pos.abs and sample.pos.relative:
Returns the position of a uniformly sampled valuefirst and last: Returns the first/last
value in the iterator intervalfirst.pos.abs, first.pos.relative,
last.pos.abs, last.pos.relative: Returns the
position of the first/last valuegintervals.neighbors when using
mindist=0, maxdist=0: the function would miss zero-distance
(touching) intervals when using mindist=0, maxdist=0.pwm.count with spatial sliding windows
double-counting bidirectional hits (forward + reverse) at the same
genomic position; the sliding path now matches the baseline per-position
union semantics.gintervals.load_chain now
returns a data frame with 8 columns instead of 7. Columns are:
chrom, start, end,
strand, chromsrc, startsrc,
endsrc, strandsrc.src_overlap_policy and
tgt_overlap_policy parameters to
gintervals.load_chain, gtrack.liftover and
gintervals.liftover functions.neighbor.count virtual track.gintervals.mark_overlaps function that marks
overlapping intervals with a group ID.pssm parameter of
gvtrack.create and gseq.pwm functions.gseq.pwm and added
neutral_chars_policy parameter.pwm, pwm.max and pwm.count) for
dense iterators when spatial weighting is disabled, providing
significant performance improvements for consecutive genomic
intervals.pwm.count(bidirect=TRUE) now
counts per-position union of strands (via log-sum-exp),
aligning with pwm/pwm.max. Each position
contributes at most 1 to the count. To reproduce the old per-strand-sum
behavior, add the two strand-specific counts:
pwm.count(bidirect=FALSE, strand=1) + pwm.count(bidirect=FALSE, strand=-1).gseq.pwm and gseq.kmer functions
that compute pwm and kmer scores on sequences without the need for a
genome database.gseq.rev and gseq.comp functions
that reverse and complement DNA sequences without the need for a genome
database.gseq.revcomp alias for grevcomp
function.gintervals.random function that generates random
genome intervals.gintervals.covered_bp and
gintervals.coverage_fraction functions that calculate the
number of base pairs and the fraction of base pairs covered by a set of
intervals.gvtrack.filters: per vtrack genomic masks.pwm.count virtual track function that counts the
number of occurrences of a PWM in the current iterator interval.gintervals.neighbors.upstream() - Find upstream
neighbors relative to query strandgintervals.neighbors.downstream() - Find downstream
neighbors relative to query strandgintervals.neighbors.directional() - Find both upstream
and downstream neighborsuse_intervals1_strand parameter to
gintervals.neighbors() to use query intervals’ strand for
distance directionality.warn.ignored.strand parameter to
gintervals.neighbors() to control warnings when query
strand is ignored.gintervals.neighbors: a stack imbalance
in the C++ code in very rare cases of 2D intervals.gintervals.neighbors due to unbalanced
rprotect calls.gintervals.normalize and
gintervals.annotate functions.m1-asan build.pwm and kmer virtual track
functions: iterator shifts were not applied.colnames parameter to
gintervals.mapply function.attrs parameter to gtrack.import
function.created.user default attribute in track creation
functions.gtrack.import function.gtrack.create_dense function - creates a dense
track from an intervals and values.clock_gettime is missing).gtrack.import_bigwig:
intern argument was not passed to system
calls.grevcomp function (reverse complement of a DNA
sequence).gvtrack.create.gdb.create_genome function.R_curErrorBuf,
SET_TYPEOFRf_
prefix in the c++ code.gtrack.import from bigwig.ALLGENOME is now only soft deprecated in order to
support old misha scripts.gtrack.create_dirs function.gcluster.run.gintervals.neighbors..misha. Variables such as
ALLGENOME can now be accessed as
.misha$ALLGENOME. This change is not backwards compatible,
please update your code accordingly.gintervals.neighbors (same as
gintervals.neighbors1 from misha.ext). This
means that instead of having two columns of ‘chrom’, ‘start’ and ‘end’,
the resulting data frame would have ‘chrom1’, ‘start1’ and ‘end1’.gwget now uses curl in order to work on
systems that do not have ftp installed.markdown format.Genomes vignette that demonstrates how to
create a new genome database.These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.