The hardware and bandwidth for this mirror is donated by METANET, the Webhosting and Full Service-Cloud Provider.
If you wish to report a bug, or if you are interested in having us mirror your free-software or open-source project, please feel free to contact us at mirror[@]metanet.ch.
read_mzml(parser = "mzR") now returns a list of streams
like the RaMS parser, adding MS1, MS2, TIC and BPC to DAD. The DAD
chromatogram, now $DAD, has retention times in minutes
rather than seconds, and takes its wavelengths from the spectra rather
than the scan window, so DAD files without one, such as those
write_mzml writes, can now be read.chroms from mzML and ‘Thermo’ RAW files
is now a list of 2D chromatograms, one per stored trace, with
precursor_mz and product_mz attributes. Use
data_format = "long" for the previous table..d directories by default, and from ‘OpenLab’
.dx files with what = c("MS1", "TIC", "BPC").
See ?read_agilent_d.call_rainbow gains centroid and
hrms arguments to read the centroid and profile mass
spectra of ‘MassHunter’ .d directories, which ‘rainbow’
otherwise skips. See ?call_rainbow.read_mzml now returns extracted ion chromatograms with
what = "EIC" or "EIC_MS2", for the masses
given by mz and ppm. See
?read_mzml..dx files and
.rslt/.sirslt directories are now in mAU. To
convert DAD data read by earlier versions, multiply it by its
intensity_multiplier attribute..lcd files are no longer about one sampling interval early
(thanks to @JustCoderWorld, #55).read_chroms now recognizes ‘Agilent’ .d,
‘OpenLab’ .rslt/.sirslt and ‘Waters’
.raw directories without format_in..dx files whose injection volume unit is
written with the Greek letter mu, rather than the micro sign, now report
their injection volume..lcd and .gcd files and
Allotrope (ASM) files now report the acquisition software,
which write_mzml records in the mzML. ASM files also report
software_version..qgd files now report the software release
(e.g. 4.45) as software_version. The
file-format version they reported before is now
file_version.instrument,
polarity, and the ion source, analyzer and detector (as
ms_params) recorded in the file.data_format attribute of "wide" rather than
"long".call_rainbow(by = "name") no longer errors when
‘rainbow’ finds a trace with no detector type, and names files in their
original case rather than upper case. Asking what for an
absent detector now gives a clear warning.read_peaklist now reads peak tables from ‘Agilent
OpenLab CDS’ .rx files with
format_in = "openlab", named by signal from the
.acaml file beside them.get_element to extract one element, such as
"dad" or a pump pressure trace, from each sample of a
chrom_list read with several what values.read_peaklist now warns when two samples get the same
name, as read_chroms does.c() now returns a chrom_list that is also
a list when combining chrom_list objects, such
as the output of two read_chroms calls. Previously the
result was not a list, so chromatographR’s
preprocess rejected it.tools::R_user_dir("chromConverter", "config") and survive
package updates. You may be asked for them once more after
upgrading.path_out contains spaces.read_chroms and read_peaklist now return
files found in a directory in acquisition order, if every file records
its run time, rather than alphabetically. sort_by = "none"
restores the previous order.rt,
start, end, area,
height and FWHM (NA where the
file records none), in both read_chroms and
read_peaklist. ‘ChemStation’ width is now
FWHM. Use peaktable_format = "original" to
return all the columns read by the parser.read_peaklist tables from every format now have a
numeric lambda column after sample, giving the
signal’s wavelength (NA where none is recorded)."254") wherever the file records one, both in
read_peaklist and in ‘ChemStation’ .D
directories. Tables without a wavelength keep their names.?read_cdf.time_unit is "Minutes". Retention
volumes (mL) are unchanged.U2, U5 and U6 as
Start, End and
SubstanceIndex..dx files now apply the recorded
time-zone offset, so they are true UTC rather than local time mislabeled
as UTC.detector_range now holds only a numeric wavelength
range, as for ‘ChemStation’ since 0.10.0. Channel descriptions and
single wavelengths that ‘Waters’ .arw, ANDI, ‘Lumex’ MDF
and ASM files put there are now signal_descriptor or
wavelength.time_range, are now numbers,
both on chromatograms read from text formats and in
extract_metadata output. So are the wavelengths of
‘Shimadzu’ .lcd traces, previously text such as
"260nm", including the lambda column of
long-format chromatograms. An injection volume written with its unit,
e.g. 17 µl, is now split into the number and
sample_injection_volume_unit.extract_metadata now returns one row per sample,
retaining the values its chromatograms agree on. The
by = "chromatogram" argument restores one row per
chromatogram (in nested lists).what = "chroms" from
read_varian_sms, which returned an internal table rather
than a chromatogram. Use what = "TIC" or
"BPC".extdata/ladder.txt is now
extdata/alkane_ladder.txt, documented at
?alkane_ladder. Update the path in system.file
calls.read_agilent_amx returns the DAD’s peak width setting
as peakwidth_code, replacing peakwidth_nm. The
value is a stored code, not a width in nm.read_agilent_amx now names solvent channels
A–D rather than Channel_A,
matching the gradient’s pct_A columns, and DAD signals
A, B, etc. rather than SigA.vignette("chromConverter"), which works through a
calibration series of benzoxazinoid standards now included as example
data. See ?benzoxazinoid_standards.read_chemstation_method to read the gradient,
solvents and other instrument settings from ‘Agilent ChemStation’
methods, including the copy a .D directory may hold,
whether stored as register files or as ‘RapidControl’ XML.read_shimadzu_method to read the oven program,
gradient and other instrument settings stored in ‘Shimadzu’ GC
(.gcd), LC (.lcd) and GC-MS
(.qgd) files.read_chemstation_logs to read the instrument
errors, aborted runs and pump pressures recorded in ‘Agilent
ChemStation’ sequence logs, including every log in a folder..D directories now return pump
pressure, flow, solvent composition and temperature traces from
LCDIAG.REG with what = "instrument". See
?read_agilent_d.read_agilent_d and read_peaklist now read
‘ChemStation’ Result.xml files where present, which unlike
Report.TXT record where each peak starts and ends.read_asm now reads mass spectra from ASM GC-MS files,
and peak lists and instrument traces (e.g. pump pressure, flow rate,
temperature) from any ASM chromatography file, chosen with a new
what argument. read_peaklist accepts
format_in = "asm".read_peaklist now reads peak tables from ‘ANDI chrom’
files with format_in = "cdf".read_thermoraw now accepts
format_out = "data.table", like every other parser.rainbow parser now report most of
the metadata the parser supplies, including instrument,
operator, detector_model,
wavelength and instrument_modules..dx files read with the
rainbow parser are now returned as POSIXct in
UTC, applying the time-zone offset the file records..dx files now report their injection
volume..ch, .uv and .it
traces now report the channel they were recorded on as
channel_id (e.g. DAD1A or PMP1A),
and its module as detector (e.g. DAD)..ch traces and ‘ChemStation’ peak tables now
report wavelength and bandwidth from their
signal descriptor (e.g. Sig=254,8); peak tables also report
detector, channel_id and the descriptor itself
as signal_descriptor.chrom_listssubset() now selects the chromatograms of a
chrom_list by their metadata,
e.g. subset(chroms, sample_name == "S1"). See
?subset.chrom_list.add_metadata to attach sample metadata from a
table to a chrom_list, matched by name, for use with
subset() and extract_metadata.extract_metadata now accepts an element of a nested
field in what, such as "SampleLabel" or
"acaml_metadata.SampleLabel", and returns that column
alone.print() of a chrom_list now shows numeric
metadata to getOption("digits") significant digits, set
with a new digits argument that summary() and
extract_metadata() also take.write_mzml now writes MS2 spectra, interleaved with MS1
in scan order, each with its precursor m/z and a reference to the MS1
scan before it. MS2 was previously skipped.centroided argument lets profile data be marked as profile.
All spectra were previously assumed to be centroided.instrument_info argument still overrides the
instrument.write_andi_chrom.read_cdf(peaktable_format = "original") now keeps the
text columns of ‘ANDI chrom’ peak tables, such as
peak_name, which were previously dropped.write_andi_chrom now record the
correct time unit, run length and detector range, and the exported
signal’s wavelength, which read_cdf reads back.NA and no TIC is
returned.detector_y_unit, which was always empty, and files from
other software also report their mass unit in
detector_x_unit.write_mzml
to keep both.write_andi_ms now follow the ANDI MS
specification more closely, recording each scan’s actual mass and time
range, the intensity unit, polarity, instrument, software and
acquisition mode. See ?write_andi_ms.ms_params in
write_andi_ms no longer drops the defaults of the
others.call_rainbow now works when format_in is
not supplied, which failed on R 4.2 and later.rainbow parser no longer
lose their retention times when it returns fewer times than rows, as for
compressed 181 .ch files. The time axis is rebuilt, with a
warning.Andi-MS format).id
(sNA) or characters an ID cannot hold, and
sample_name now also names the <sample>
element. Metadata values are now XML-escaped.run_datetime no
longer fail schema validation.write_mzml now errors on a table without
mz (or lambda) and intensity
columns, instead of writing a corrupt file..lcd chromatograms
(scale = FALSE) now report detector_y_unit as
NA where the channel’s calibration factor is not 1, since
those values are not in a defined unit..lcd files now carry the
scaled attribute..qgd scans whose intensities are more than 4
bytes wide can now be read, with a warning that this decoding is
unverified..lcd peak tables stored as
Peak Table-100 and similar now read correctly; every peak
after the first was misread..lcd peak tables no longer include mass
spectrometry tables (Mass Peak Table,
Compound Peak Table), which were misread.read_peaklist no longer returns the peak table of a PDA
channel twice for ‘Shimadzu’ .lcd files..lcd and .gcd peak
tables now carry the file’s metadata, such as sample_name,
instrument and run_datetime, and their
channel’s wavelength (and bandwidth, for
channels extracted from PDA data), as the chromatograms do..gcd, .lcd and
.qgd files now report their method’s settings as
method_params, and the mass spectrometer settings of GC-MS
and quadrupole LC-MS runs, such as ion source temperatures, as
ms_params. See ?read_shimadzu_method..ch files written by ‘Asterix
ChemStation’ are now decoded as 8-byte doubles. They were incorrectly
read as 4-byte floats, which compressed the signal into tiny values near
0.001.read_agilent_amx now warns when a method lacks a
requested module, such as a DAD, and returns the others; asking for a
missing module alone gives a clear error.solvents table from read_agilent_amx
now returns the class that format_out asks for.read_agilent_amx signal tables now include each
signal’s reference wavelength and bandwidth, as
reference_nm and reference_bandwidth_nm.slitwidth_nm from read_agilent_amx is now
the slit width in nm (e.g. 4). It previously held the
stored code (2).read_agilent_dx(what = "instrument") now gives a clear
error for an archive without instrument data.run_datetime, and
read sample_name, method and the injection
volume correctly.extract_metadata no longer splits the
source_file of ‘ChemStation’ peak lists into one column per
sample.sample_name and run_datetime, as the
chromatograms do..ch and .uv traces now report
the volume actually injected, which can differ from the volume the
sequence requested..D directory now get
distinct names, e.g. RUTIN_2.dad1A, instead of all sharing
the directory or sample name." or :, with _.
Sample names containing them previously failed to export on
Windows.read_chroms adds to its list of samples."UV 1_280") where their detection types
don’t tell them apart, in the list and in the detector
column.detector_reference, detector_y_unit and
time_unit now describe each chromatogram, rather than being
copied from the file’s other traces. LC-MS files no longer warn that row
names were discarded.detector_model is now reported, as are the
method and injection volume of GC files and the sample name
and run time of files from the 2022 releases.wavelength setting.'names' attribute error.metadata_format = "raw") now keep the
file’s own field names, e.g. injection time and
analyst, and describe each chromatogram rather than the
whole file.read_chromeleon now honors
decimal_mark = "," when it is given explicitly. Previously,
the commas were stripped, inflating every value.read_peaklist now warns about files it cannot parse and
drops them, rather than returning the error message as a peak
table.read_mzml now returns DAD data in the correct class as
requested by format_out..lcd files are now
scaled by the calibration factor as well as the value factor, so the
intensities match those reported by ‘LabSolutions’. The calibration
factor converts the encoded integers into the base unit of the detector,
and is stored alongside the raw data in the
Chromatogram Status stream. The parser was instead taking
it from the copy of the 2D Data Item under
LSS Data Processing, where it is always 1.
Channels where it is not 1 were off by a constant factor,
such as ~42x for an SPD-20A UV detector and ~310x for an RID-10A
refractive index detector, so any factor applied by hand to match
‘LabSolutions’ should now be removed. Older files, written by
‘LCsolution’ rather than ‘LabSolutions’, have no
2D Data Item at all, so neither factor reached them; both
are now read from the status record, changing the scale of those
chromatograms by up to ~5000x. scale = FALSE still returns
the unscaled integers.apply(x, 2, as.numeric). Because retention times entered
that table as character (from the rownames), the coercion routed every
column through a character matrix, formatting each intensity with
getOption("digits") and so rounding it to 7 significant
figures. Long-format intensities now match the wide-format values
exactly. Exported files were affected too, since write_mzml
and write_andi_ms reshape to long format before
encoding.format_out = "matrix", which is the default,
read_shimadzu_qgd, read_chemstation_ms and
read_cdf returned MS1 as a matrix, and
read_varian_sms returned a data.frame but recorded
format_out as matrix. Long spectral data has
no useful matrix representation: the coercion promoted scan
and rt to double alongside the intensities and gave up
$. All four now return a data.table for this
value of format_out, and record it. data.frame
and data.table are returned as requested, and the
two-dimensional streams (TIC, BPC) still honor
matrix..lcd or
.gcd file now reports its detector as
DAD rather than PDA, matching the vocabulary
used by every other parser. read_shimadzu_lcd also returns
the stream under the name DAD rather than pda,
so a result indexed as x$pda needs updating — though only
where more than one stream was requested, or
collapse = FALSE, since a single-stream read returns the
object itself, unnamed. what takes either spelling, so the
two are interchangeable throughout.parser is not specified, reading a ‘Waters’
.raw directory now requires the rainbow Python
module, and reports how to install it when it is missing. Auto-detection
previously fell back on the internal parser, which reads only the analog
_CHRO traces and reported nothing about the MS and PDA data
it had skipped, so a file with three detectors could come back with one.
The fallback also depended on a test that started Python for every file,
including the formats that need no Python at all.
rainbow-api is requested automatically, so only
hand-managed environments are affected;
parser = "chromconverter" still selects the internal parser
for files that hold analog traces alone.read_agilent_rslt function to read whole sequence
of files from OpenLab and automatically attach corresponding metadata
from the acaml file.read_shimadzu_lcd can now read mass spectra from
.lcd files, with what = "MS1",
"MS2", or "MS" (to retrieve both levels). Two
types of containers are supported: QTFL RawData (centroided
quadrupole time-of-flight) and TLM Raw Data (triple
quadrupole full scan, product-ion scan, MRM and SIM).
what = "TIC" reads the total ion current from either
container. QTOF m/z are computed from the TOF calibration stored in the
file, with the mass correction ‘LabSolutions’ cached for the run folded
in, which reproduces its mass axis exactly. Where a file carries no
cached correction, the calibration is refit against the lock mass
reference ions found in the data instead, which is good to a few tenths
of a ppm. A new sparse argument controls whether zeros are
dropped from triple quadrupole profile spectra.sort_by argument to read_chroms to
control chromatogram order. Options are “none” (default),
“acquisition_time” (using run_datetime from metadata), and
“file_time” (using file modification time). The default will change to
“acquisition_time” in a future release.bin_width argument to call_rainbow
as an alternative to precision, for m/z grids that are not
a power of ten (e.g. bin_width = 0.5).
precision is unchanged and remains the default.summary method for chrom_list
objects, which returns what print displays as a table: one
row per chromatogram, with the sample it belongs to, its dimensions and
relevant metadata fields, such as wavelength or
detector_range for an optical detector,
scan_type, polarity,
precursor_mz, product_mz and
mz_range for a mass spectrometer.expand argument to
extract_metadata for the nested metadata fields, whose
value is itself a list or table rather than a single value per
chromatogram: the ms_params instrument settings, the
acaml_metadata injection record
read_agilent_rslt reads from the .acaml file,
or the whole vendor list when metadata_format = "raw".collapse argument to
extract_metadata, which renders a field holding more than
one value as a single comma-separated string instead of spreading it
over numbered columns (time_range1,
time_range2).detector argument to
extract_metadata to select which detectors to include
(e.g. detector = "UV" or
detector = c("UV", "MS")), matched case-insensitively
against each chromatogram’s detector attribute. This is
useful for lists containing more than one detector per sample, such as
those returned by the rainbow parser.write_andi_ms is now exported, like
write_andi_chrom and write_mzml. It was
previously reachable only through
write_chroms(what = "MS1") or
read_chroms(export_format = "cdf"), which write a whole
list of chromatograms and offer no control over the file name or the
instrument settings.write_chroms(export_format = "cdf") now forwards
... to the underlying writer, as the mzml
exporter already did. This makes the ms_params argument of
write_andi_ms and the lambda argument of
write_andi_chrom usable when writing a batch of files.rainbow,
olefile or Aston) is actually called, so the
formats read by the internal parsers no longer require an internet
connection. When Python is needed and the package index can’t be
reached, chromConverter now falls back on a previously cached
environment instead of failing.rainbow or olefile parsers
no longer installs the ‘Aston’ requirements or constrains which version
of scipy you can have.configure_python_environment so it accepts the
parser argument it is called with, and removed its
interactive prompts, which failed in non-interactive sessions..uv file went from ~9 s to ~0.57 s..qgd file went from ~56 s to ~1 s, and reading a PDA stream
from an .lcd file went from ~7 s to ~1 s.read_varian_sms for increased speed (~8x)
through vectorization. Reading STRD15.SMS (2.4 MB, 935k MS1
rows) drops from ~13 s to ~1.7 s. The stream is also bounded by the end
of the MSData section rather than the end of the file,
which reduced peak memory requirements for files carrying a large tail
of peak tables and results.write_mzml for increased speed (~1.4x) and
lower memory use. The spectra are now sliced out of the long-format
table in place instead of being copied into a list of per-scan tables,
and the byte offsets for the index are accumulated as the file is
written rather than probed with seek() once per scan.
Writing 3432 scans (935k points) drops from ~1.24 s to ~0.87 s, with
peak memory falling from ~384 MB to ~339 MB.data_format = "long" (or
format_out = "data.table", which implies long format), as
well as the mzML and ANDI MS writers, which reshape to long format
internally.source_sha1 metadata attribute, is now computed once per
file instead of once per chromatogram. Formats that return several
chromatograms from one file hashed it again for each of them, which
dominated the read for large files: a 40 MB .qgd drops from
~2.2 s to ~1.3 s.extract_metadata reports
cannot drift apart. Five fields had drifted and are renamed:
software_name is now software (‘Shimadzu’,
‘Varian’ SMS and read_agilent_rslt); run_date
is now run_datetime and injection_volume is
now sample_injection_volume (‘Thermo’ RAW);
time_start and time_end (or
end_time) are now the two ends of time_range
(ANDI, ‘Lumex’ MDF and ‘Varian’ SMS); and the two formats that record a
scan count, ‘Varian’ SMS and ANDI MS, now both report it as
n_scans, matching the n_ prefix used for
counts throughout the package, rather than no_scans in one
and ms_params$n_scans in the other.
extract_metadata accepts the old names and maps them to the
new ones.parser attribute is now always spelled
chromconverter. With metadata_format = "raw",
some readers reported chromConverter instead.sample_amount is no longer copied from the injection
volume (‘Shimadzu’ ASCII, ‘ChemStation’ .ch,
.uv and .ms files, ‘ChemStation’ report files,
and ‘MassHunter’). None of these records a sample amount, so it is now
NA. ‘Lumex’ MDF likewise no longer reports an injection
volume and amount of 1, which the file does not
record.detector_id is renamed as detector_model.
Every format that fills the field supplies a module or a model number,
and says as much internally: detector_model for
‘ChemStation’, ‘OpenLab’ and ‘Chromatotec’ (G1315B,
HP G1530A), detector_model_number for ASM,
detector_name for ANDI (9065 UV-DAD) and
Detector Name for the ‘Shimadzu’ ascii exports. None
supplies a serial number or any other identifier of a particular unit,
so the old name was misleading. extract_metadata accepts
detector_id and maps it to the new name.detector_range is now reserved for the numeric
wavelength range recorded by .uv files. For ‘ChemStation’
versions 30 and 130 the signal descriptor was previously reported in
this field, and is now reported as signal_descriptor.detector field is now NA for
‘ChemStation’ .ch files. These files do not record a
detector type; the field previously reported the detector module,
duplicating detector_id.read_acaml now also returns the injection volume
(InjectionVolume, InjectionVolume_unit) and
the acquisition software name and version (Software,
SoftwareVersion).wavelength attribute of a ‘Shimadzu’ 2D
chromatogram is now NA rather than an empty string when the
channel records none, as a refractive index or FID trace does. An empty
string printed as a blank cell instead of as a missing value..lcd and .gcd files now report
the instrument the file was acquired on, rather than the detector module
of whichever trace you are looking at: one run on one HPLC previously
came back as SPD-20A on two channels and
RID-10A on a third. instrument is taken from
the SystemInformation stream (Instrument2,
HPLC RID, GC-2014), the same string the ascii
exports report as Instrument Name, so a run exported both
ways now agrees. The module is now reported as
detector_model. A mass spectrometry trace has no module of
its own, so it reports whatever unit SystemInformation
lists for the mass spectrometer instead. That is a model number on newer
software (LCMS-9030), but older versions list the generic
platform name (e.g., LCMS-3030 for every triple
quadrupole)..lcd or .gcd
trace was read from (LC.1.1.DET.1.CH#1,
PDA.1.1.PDA.1.3D) is now reported as
channel_id, (replacing detector_id). This
field is used to name the peak table belonging to a trace
(PT-LC.1.1.DET.1.CH#1), so the two can still be matched
up..lcd, .gcd and
.qgd files now report file_version, the
version of the container format (5.01 for files written by
‘LabSolutions’; absent in the older files, which report only a
software_version of 1.x).segment_start_time and
segment_end_time within ms_params.
time_range gives the span of the whole run, as it does for
every other format.aston parser is deprecated and will be removed in a
future release. ‘Aston’ has been unmaintained since 2020. It is now used
only by sp_converter to read ‘Agilent MassHunter’
.sp files (format_in = "masshunter_dad"), and
read_chroms selects it automatically only as a last resort,
when no other parser can read the file. Please use the internal
chromConverter parsers or the entab parser (by the same
author as ‘Aston’) instead.uv_converter is now defunct; use
read_chemstation_uv or the entab parser
instead. The aston binding for
format_in = "other" has also been removed; this format is
still handled by the entab parser. Both relied on an
‘Aston’ reader that requires scipy < 1.14, which would
otherwise constrain the Python environment for every user.dat argument in read_chroms.
Instead, chrom_lists can be combined with c().data_format argument of
read_peaklist and read_chemstation_reports to
peaktable_format. This argument selects
chromatographr or original peak table layout,
so it had nothing to do with the data_format argument of
the chromatogram readers, which selects wide or
long format. peaktable_format is the name
already used for this option by read_shimadzu. The old name
still works but warns, and will be removed in a future release.read_sz_lcd_2d and read_sz_lcd_3d are no
longer exported. Each reads a single stream of an .lcd file
and neither has to be called directly:
read_shimadzu_lcd(what = ...) picks the reader a stream
needs. Their documentation remains, since each records the layout of the
stream it reads, but is marked internal and no longer appears in the
reference index.format_out = "data.frame", which returned a
data.table for any parser that assembles its result as one.
The conversion tested the object with inherits, and a
data.table inherits from data.frame, so the conversion was skipped.rainbow parser, which raised
read() no longer takes precision on every call once
rainbow-api v1.5.0 was released. v1.5.0 split
precision into bin_width (the m/z grid, in
daltons) and display_precision (label rounding, in
decimals); chromConverter now derives both from precision,
so the argument and the data it returns are unchanged. v1.5.0 is now the
minimum required version.nchar() and
toupper() failed on them and grepl() could not
match them. Accented characters in a path or sample name are now
preserved rather than mangled.input string 1 is invalid UTF-8) when
printing a chrom_list read from a ‘Shimadzu’
.lcd written in a non-Latin locale. The hex-encoded
@StoX@ fields, which hold the method and
batch paths, skipped the Latin-1 decoding applied to the
rest of the file’s strings. Undecodable bytes are now replaced with
?, and print.chrom_list repairs whatever it is
handed, so one mangled path cannot take down the summary.thermoraw,
openchrom, agilent_dx and
agilent_amx parsers deleted the whole session temporary
directory on exit, instead of just the files they created. This behavior
could potentially create conflicts with other packages. Each call now
gets its own directory inside the session temp directory which is
cleaned up on exit.reticulate::py_install("rainbow"), which installs an
unrelated package; the ‘rainbow’ module comes from
rainbow-api.detector_id for ‘ChemStation’ version 130
files.sample_position metadata field for ‘ChemStation’
179 files (.ch and .it).POSIXct instead of being attached as an unparsed string,
which extract_metadata reported as NA.read_agilent_dx,
read_agilent_amx and read_agilent_rslt to fail
when path_out was supplied.read_chemstation_reports, which listed the accepted values
as chromatographr or chemstation. The second
value has always been original, so following the
documentation raised an error.NA for every export not written by a machine
using a 12-hour month-first date format. Note that the times in an ASCII
export are local to that machine, which does not record its time zone,
whereas .lcd files record the acquisition instant in
UTC.detector and no detector_range; it now reports
DAD and the wavelength range the detector covered.read_shimadzu_lcd for .lcd files
that do not contain a 2D Data Item, which failed with
'names' attribute [4] must be the same length as the vector [2].
This bug seems to affect older files, which store their chromatograms
under LC Raw Data rather than
LSS Raw Data.read_shimadzu_lcd so it can return PDA data in
long format.
read_shimadzu_lcd(what = "PDA", data_format = "long")
previously failed with an error about a missing lambda
column, because the reshaping step was called with the wrong target
format.format_out now reaches the peak tables of ‘Shimadzu’
.lcd and .gcd files.
read_shimadzu_lcd(what = "peak_table") and
read_shimadzu_gcd(what = "peak_table") accepted the
argument and then dropped it, so a table always came back as a
data.frame. data.table is now returned when
asked for — matrix, which has no useful representation for
a peak table, resolves to data.table as it does for mass
spectra.~,
which is not expanded by Python..lcd data
streams, which was read as a signed 2-byte field rather than the 4-byte
field it is. A 2D chromatogram with 32,768 to 65,535 points failed with
invalid 'length' argument, and one with more than 65,535
points was truncated to the remainder. Files below that threshold, such
as the 30,000-point chromatograms in the test suite, were read
correctly.run_datetime for ‘Varian SMS’ files is now the
acquisition start time, as a single value rather than a start/end
pair.read_varian_sms for
format_out = "data.table", which failed previously with an
error. The TIC and BPC returned by this parser
also had their intensity column named tic/bpc
instead of intensity for this value of
format_out.instrument and
method metadata from Varian SMS files (read from the
InjectionLog section).read_cdf for ‘ANDI MS’
files in which every scan holds the same number of points (common when
the instrument scans a fixed mass range). The retention times arrived as
a matrix and were split into one column per scan, so a 20-scan file
returned a table with rt.1, rt.2, …
rt.20 columns instead of a single rt column.
With ms_format = "list" the same files returned a list of
individual numbers rather than a list of spectra. Files with a varying
number of points per scan were unaffected, and their output is
unchanged.read_cdf no longer opens the netCDF file twice, and the
peak table returned for ‘ANDI chrom’ files is no longer transposed when
it holds a single peak.read_cdf(what = "peak_table") on an ‘ANDI chrom’ file
that holds no peak table now warns and returns the other streams that
were asked for, rather than failing with
value for 'peak_table' not found.write_mzml now warns that MS2 spectra are skipped
rather than counting them in the spectrumList header and
writing none of them. Writing MS2 is not supported yet.write_chroms(export_format = "mzml"), which
failed for every file unless what was given explicitly. The
streams to write are now inferred from the data, as they are when
write_mzml is called directly.write_mzml no longer writes a one-dimensional
chromatogram as DAD spectra, and points to write_andi_chrom
instead. mzML stores scans of (m/z or wavelength, intensity), so a
single trace has no axis to put in one: it was written as a single-point
spectrum per retention time, which for a 66,000-point trace meant 66,000
scans, as many warnings about empty ranges, and a 128 MB file. The trace
is skipped with a warning if other streams were requested, and reported
as an error if it was the only one, since skipping it would leave an
empty file. TIC and BPC are unaffected, since
the format has terms for those MS-derived summaries and they are written
to the chromatogram list rather than as spectra.write_mzml now throws a more informative error when
handed a single chromatogram whose detector attribute is
missing, NA, or names a detector it has no mzML stream for,
rather than failing with EXPR must be a length 1 vector or
quietly writing an unnamed stream.fileChecksum written into indexed mzML files,
which was the SHA-1 of the first line of the file
(<?xml version="1.0" encoding="UTF-8"?>) rather than
of the file itself, because the digest was taken over a multi-element
character vector. Files are now checksummed as required by the mzML
specification, over the bytes up to and including the opening
<fileChecksum> tag, and the file no longer has to be
read back into memory to do it.indexList of mzML files.
<indexListOffset> pointed one byte before
<indexList>, and every offset in the DAD spectrum
index pointed at the newline preceding its <spectrum>
element rather than at the element. Offsets are now counted as the file
is written instead of being probed with seek(), which is
unreliable on a connection opened in text mode and ignores the write
buffer.<offset> pointed four bytes before its
<chromatogram> element, and its idRef
named the element’s index rather than its id,
so no entry in the index resolved to the chromatogram it was meant to
locate.MS:1000580 (“MSn spectrum”) even though
ms level was 1.write_mzml(compress = FALSE), which was ignored
for spectra (though not for chromatograms), since the argument was never
passed on.count attribute of
<spectrumList>, which was always written as
1 for data read as a data.table.sample_name, source_file or
source_sha1 attributes: a missing field collapsed the
sprintf that builds the header, dropping the
<mzML> element itself. The header read its metadata
from MS1, or from DAD when MS1
was not requested, without checking the stream was still there — a
DAD trace skipped earlier leaves nothing to read from. It
now reads from a stream that is actually being written, and a missing
sample_name is an error naming the argument that supplies
one.count attribute of
<spectrumList> when the TIC starts before the first
MS1 scan. Those leading retention times are written as empty spectra, so
the file holds one spectrum per TIC point, but the count was taken from
the MS1 table and fell short by the number of padded scans.index attribute of the DAD spectra, which
with indexed = FALSE restarted at zero and repeated the
numbers the MS1 spectra already used. The starting value was parsed from
the id of the last spectrum-index entry, and those entries carry an id
only when the file is indexed; it is now the count of spectra already
written.LF on all platforms.switch, and attributes
like source_file, source_sha1 and
parser are no longer repeated in each one. The change is
internal, but it resolves several problems listed below.metadata_format, which several readers
mishandled. metadata_format = "raw" errored for
‘Chromeleon’ files and returned NULL instead of a
chromatogram for the rainbow parser; the entab
parser and read_shimadzu ignored the argument altogether;
and parser = "entab" with format_in = "other"
returned NULL. The argument is now resolved in one place
instead of separately by each reader.read_peaklist returns, so
extract_metadata can see it.detector_range for ‘Chromeleon’ 3D files, which
reported only the lower end of the scan range. For 2D files, which
record no scan range, it was a zero-length value that disappeared from
extract_metadata instead of reading NA.sample_info.xml.read_mzml recovers are
now attached as attributes, so extract_metadata and
print.chrom_list can see them, and the
metadata element carrying them is no longer counted as a
chromatogram. The run_datetime, time_range,
time_unit and detector_range fields previously
came back as NA for mzML files even though ‘RaMS’ had
parsed them, which also meant
read_chroms(sort_by = "acquisition_time") could not order
them. The metadata element is still returned in full, since
it carries several fields with no attribute equivalent.extract_metadata now reports them as empty
instead of dropping them altogether. scaled was accepted by
every reader that takes a scale argument and then dropped
by thirteen of the twenty field maps, so it was missing for most
formats; metadata_format = "raw" recorded
source_file and source_sha1 but not
format_out or source_file_format; and ‘Thermo’
RAW files lost any field that the mzML conversion recorded but the
converter’s own metadata output did not.source_file_format attribute is now set correctly
for every format. It was previously missing for some formats, and named
the parser or an intermediate file for others.extract_metadata now reports the injection volume for
every format that records one, and the acquisition software for
‘Shimadzu’ and ‘Varian’ files. These were previously ignored, because
the formats did not agree on what to call them.extract_metadata now returns a row for every
chromatogram, however deeply nested, and reads sample-level attributes
from the list enclosing a sample’s traces as well as from the traces
themselves. Previously only the top level of the list was examined, so
nested traces, and any metadata held on the list grouping them, were
left out of the table. A field that varies from trace to trace, such as
detector in a multichannel file, stays with the trace;
where the traces agree, the value on the enclosing list is used, since
it describes the sample as a whole.extract_metadata no longer reports a multi-valued field
as missing. The check tested the requested names against the columns of
the assembled table, where a field holding several values appears as
product_mz1, product_mz2, … and so never under
the name that was asked for.extract_metadata now matches attribute names exactly.
Previously a requested element could be filled in from a different
attribute that merely started with the same characters, so a
chromatogram with no detector attribute could report its
detector_y_unit as its detector.extract_metadata now returns NA instead of
a metadata frame with only a name column when none of the
requested metadata elements are found.print.chrom_list now handles lists holding more than
one trace per sample, such as a multichannel ‘Shimadzu’ file or an
‘Agilent’ .dx read with
what = c("chroms", "dad"). Traces are grouped under the
sample they belong to, however deeply nested, and attributes shared by
all of a sample’s traces are shown in that sample’s block header instead
of being repeated on every row. Previously only the top-level elements
were counted, so the chromatogram count was wrong and only the first
trace of each sample was shown.print.chrom_list formatting: datetimes print
as timestamps rather than raw epoch seconds; the header wraps to the
width of the console, breaking between fields; long values, such as a
‘Windows’ method path, are shortened from the middle; and a
field that is empty for every chromatogram is dropped.
print no longer errors when none of the requested
cols are present or when n is negative, and
n now defaults to 10, as documented.read_chromsread_chroms. A single
internal table now records which formats it can read and, for each one,
the file extension, and the reader each parser uses. These facts were
previously spread across the argument list, an
if/else chain, and several utility functions
(check_parser and format_to_extension). The
change is internal, but it resolves several problems listed below.... are now matched against
the arguments the selected parser actually accepts. An unrecognized
argument previously made every file fail, surfacing from inside
try() as a warning with an unreadable message; such
arguments are now ignored with a warning naming them.format_in is not supplied and the type of a file
cannot be recognized, read_chroms now says so and asks for
a format, rather than failing with
argument is of length zero.read_chroms now gives an informative error when no
parser is available for a format, instead of failing with
missing value where TRUE/FALSE needed.read_chroms now
names it instead of reporting its position in the list.suppressWarnings and was
invisible to callers handling the warning.format_in accepts an alias as readily as the format’s own
name, and the two behave identically: rslt and
sirslt for agilent_rslt,
openlab_dx for agilent_dx,
chemstation_fid for chemstation_ch,
andi for cdf, and allotrope for
asm.shimadzu_ascii, csv, asm,
openlab_131 and chemstation formats are now
matched to the correct file extension, instead of falling through to a
pattern matching any file containing a ..metadata_format argument now reaches the
agilent_dx, agilent_rslt,
shimadzu_lcd, shimadzu_qgd, cdf
and entab parsers. Previously,
metadata_format = "raw" had no effect for these
formats.sample_names = "sample_name" named every sample with the
literal string "NULL" and then warned about duplicate
names. This was because the sample_name attribute is
attached to the individual chromatograms rather than to the list
grouping them, so the lookup came up empty and the resulting
NULL was coerced to a string. Samples with no recorded
sample name now fall back to the file name, with a warning naming them.
A name that is recorded but empty counts as no name, since a parser that
finds the field but reads nothing out of it leaves an empty string
behind. The traces making up a sample are now also checked against each
other: if they disagree about the sample name there is no basis for
preferring one over another, so the file name is used instead, again
with a warning.[.chrom_list method so that subsetting a
chrom_list preserves its class instead of dropping it to a
plain list.c.chrom_list method so that combining
chrom_list objects with c() preserves the
class instead of dropping it to a plain list.read_shimadzu_lcd: (bytes are now read explicitly as
ISO-8859-1 rather than relying on system locale via
readLines()).prec argument to precision; chromConverter now
requires rainbow >= 1.3.0.varian_sms so the example
file is downloaded in binary mode (mode = "wb"), preventing
file corruption on Windows.sample_position field to
extract_metadata.sample_id and vial metadata
fields into new sample_position field.sparse argument for rainbow parser
(now enabled by default) to remove zeros form long-format MS data.what arguments in
read_agilent_d to prioritize DAD data (instead of 2D
chromatograms).chrom_list class and
print.chrom_list method. Instead of dumping the full
contents of every chromatogram, prints a compact metadata summary with
configurable columns (cols) and row limit
(n)..amx)..Chrom files through the read_chromatotec
function.csv
files.agilent_dx files
so that the original source file is stored in metadata instead of a temp
file.read_chroms for duplicated names as
they may silently interfere with downstream analyses.write_chroms. This
format seems to be the simplest way to get DAD data into
OpenChrom.csv
files and fixed a bug causing column names to be prepended with
“X”.write_chroms so it invisibly returns file
names of the exported chromatograms.write_chroms..cdf (thanks to @pbulsink for PR #37)..cdf files.write_mzml to correctly handle DAD spectra.write_mzml..ch files to wide
format when specified.read_agilent_d when subsetting data with
the what argument.rainbow parser are in minutes.data.table object when specified by user.read_agilent_dx and read_themoraw) are
actually deleted on completion.extract_metadata function for
simplicity..dx files:
extraction of DAD and auxiliary instrumental data (stored in
.IT files).read_shimadzu_qgd for a 1.4x speedup in the
parsing of Shimadzu .qgd files, cutting execution time by
30%.read_shimadzu_lcd for a 2.4x speedup in the
parsing of Shimadzu .lcd files, cutting execution time by
60%.write_mzml for massive speed-up when writing
mzML files, especially for large MS data.read_agilent_d that were
causing possible bug.read_chemstation_uv causing error for long
format data.read_agilent_d.data_format attribute.data_format and read_metadata
arguments for read_chemstation_csv.data_format attributes for MS data to
reflect that they are always returned in long format.data.table output
is selected since data.tables do not have rownames.write_mzml causing retention time shifts
for BPC and TIC.configure_python_environment function to
facilitate configuration of a chromConverter virtual environment or
conda environment, though a dedicated environment is no longer required
(as of chromConverter v0.7.4).collapse argument causing functions to
return vector when format_out is
data.frame.format_out is data.table.data.table format in
read_shimadzu_ascii..dx
file by read_chroms.read_shimadzu (‘Shimadzu’
ASCII files).sample_names argument in
read_chroms: This argument can no longer be
supplied with a vector of names. Instead, sample_names can
be provided with one of two arguments: basename or
sample_name. The default setting (basename)
will use the basename of the file, while sample_name will
instead use the sample name encoded in the file’s metadata.extract_metadata when sub-setting metadata
elements.extract_metadata.data.table format option to
extract_metadata.call_openchrom.call_openchrom
manual concerning OpenChrom installation.read_shimadzu_lcd to infer retention times in
Shimadzu 3D Data from Max Plot stream since it is always
(?) present.read_shimadzu_lcd to skip parsing of metadata
from 3D Data Item when it is not present.read_shimadzu_lcd to include
Max Plot stream when parsing 2D chromatograms.read_chromeleon related to inference of
decimal separators.decimal_mark argument to
read_chromeleon to manually set decimal separator..lcd files..lcd files lacking Data Item streams..lcd files..raw and Agilent .D)Waters chromatograms
with lowercase filenames..raw header files..sms) format through read_varian_sms
function.read_shimadzu_qgd function..D directories through read_agilent_d
function.read_agilent_ms.read_chroms so that
format_in comes second after path.export argument from
read_chroms. To export files, you now only need to provide
an argument to export_format.read_shimadzu_lcd. The function now returns a list of named
chromatograms if data_format == "wide" and returns multiple
chromatograms in a single data.frame if
data_format == "long".scale argument to
read_chemstation_uv and read_shimadzu_ascii to
toggle scaling of chromatograms.file arguments to path.read_cdf UI: what now
defaults to NULL and defaults are coded into downstream
read_andi_chrom and read_andi_ms
functions.read_chroms when
verbose is TRUE.reticulate::configure_environment).data.table as an option for
format_out.read_shimadzu_lcd by dealing with
twos-complements more sensibly.path_out when using
‘ThermoRawFileParser’ and ‘OpenChrom’ parsers.read_chroms if
path_out does not exist.mdf files lacking null bytes
after the file header.read_chroms.export_csv function to label first column for
wide-format chromatograms..ch)
files through read_chemstation_ch.Agilent Chemstation files due to typo.ChemStation CSV files
on (some) mac machines by specifying little-endian format in call to
read.csv.read_shimadzu_lcd_2d to more
accurately reflect file structure.scale argument to
read_chemstation_ch and read_shimadzu_lcd to
toggle scaling of chromatograms resolving (#30).read_shimadzu_lcd function to correctly
determine the number of blocks in the “Shimadzu” LCD PDA stream (thanks
to kco-hereon).export_cdfs function to permit
conversion of files lacking metadata..cdf) files.read_chemstation_ch parser to correctly read
“Mustang Chemstation” 179 files with 8-byte encoding.read_shimadzu function and added support
for new types of chromatograms (e.g. status, uv and total ion
chromatograms). Added support for reading multiple types of
chromatograms at once.read_shimadzu.write_cdf and added additional arguments
(lambda and force) for greater control by
users.read_waters_raw).collapse argument to call_rainbow
and to collapse superfluous lists.... argument to read_chroms for
supplying additional arguments to parsers.read_chroms for reading
mzxml files with RaMS.precision argument to call_rainbow
to control number of digits “mz” values are rounded to. (Also changed
default behavior so values are rounded to one decimal by default).read_shimadzu_lcd on Windows due to issue
with passing escaped paths to Python.pbapply
package. (Note: The pbapply package must
be manually installed to enable parallel processing).read_chemstation_uv function)..dx)
files (through read_agilentdx function)..lcd files through the
read_shimadzu_lcd function. Only the PDA stream (not MS) is
currently supported.read_peaklist function for reading peak lists.
Currently ‘Agilent ChemStation’ and ‘Shimadzu ASCII’ formats are
supported.verbose argument to control console output for
external parsers (‘OpenChrom’ and ‘ThermoRawFileParser’).read_chroms.read_thermoraw function to simplify
paths.thermoraw and openchrom parsers now
use a proper temp directory if an export directory is not specified
through the path_out argument.reshape_chroms, speeding up conversion from
wide to long format.openchrom export format to
mzml.what == "peak_table" and
read_metadata == TRUE).format_out == "matrix".data_format attributes in ‘Waters ARW’
and ‘Chromeleon’ parsers.chemstation_ch parser (version 130) (#17).arw) PDA
files..ch) files (version
30).read_chromeleon to better deal with comma
decimal separators in metadata.read_chromeleon to deal with more datetime
formats.read_chromeleon to deal with unicode
microliters.read_chemstation_ch.cdf files through the
read_cdf function..mdf files through the
read_mdf function.chemstation_csv (utf-16) and ANDI chrom cdf
files through read_chroms.read_chroms when providing direct paths to files (i.e. when
find_files == FALSE).read_varian_peaklist function for reading peak
lists from ‘Varian MS Workstation’.wide and long
data_format options for 2D data, such that the
wide format option writes retention times as rownames of
the matrix or data.frame. while the long format writes
retention times as the first column of the object.configure_openchrom for better discovery of
‘OpenChrom’ path and added path argument for directly
specifying the path to ‘OpenChrom’.source_file field to track data origin.fs package for parsing paths, eliminating buggy
check_paths function..uv) files
(version 131) through the read_chemstation_uv
function.extract_metadata function for extracting metadata
from a list of chromatograms and returning it as a
data.frame or tibble.progress_bar option in
read_chroms.reshape_chroms and reshape_chrom
to allow switching between “wide” and “long” formats.read_mzml.read_chroms.read_chroms.POSIXct
format..ch) files (version
130).read_chemstation_fid function to
read_chemstation_ch.read_chroms.format_data argument to
data_format to select wide or long format.mzML files with
RaMS..D) and “Waters”
(.raw) files with rainbow.data_format option available consistently for
choosing wide or long format..ch) data
(versions 81, 179 & 181).call_openchrom so that it can actually
find ‘OpenChrom’ path.call_openchrom to allow ‘animl’ as valid
export_format.configure_aston function for configuration of
Aston parsers and fixed issues with configuration of Aston.read_shimadzu function for parsing ‘Shimadzu’
ascii files.read_chromeleon function for parsing ‘Chromeleon’
ascii files.read_thermoraw function to convert ‘Thermo Raw’
files by calling the ‘ThermoRawFileParser’.read_mzml function to extract UV data from mzML
files using mzR.call_entab function for calling Entab
parsers.call_openchrom to call OpenChrom parsers through
the command-line interface.(All of the new functions described above can be called from the
read_chroms function by setting the format_in
and parser arguments).
read_metadata = TRUE in
read_chroms.read_chroms syntaxread_chroms will now automatically assign a parser if
the parser isn’t specified.read_chroms will throw more informative errors for
mismatch between format_in and parser
arguments.read_chroms will try to automatically determine if
files or directories are being provided.read_chroms syntax:Dots were replaced with underscores in all arguments to
read_chroms for internal syntactical consistency across the
package. Thus:
R.format to format_out.format.out to export_format.NEWS.md file to track changes to the
package.These binaries (installable software) and packages are in development.
They may not be fully stable and should be used with caution. We make no claims about them.