Introduction

How it works:

• For each point, it calculates the mean of surrounding points within a defined window
  
• Larger windows provide more smoothing but may blur important features
  
• Smaller windows preserve detail but provide less noise reduction

Parameters:

• wn_col: Column name for wavelength data (default: “Wn”)
  
• window: Window size for the moving average (default: 15)
  
• degree: Polynomial degree for smoothing (default: 2)

CoHAspec_filt %>% 
  spec_smooth_avg() %>%
  spec_smartplot(geom = "line")

How it works:

• For each data point, it fits a polynomial of specified degree to nearby points within a window
  
• The smoothed value is calculated from this local polynomial fit
  
• This preserves the underlying spectral features better than simple averaging
  
• The method can also calculate derivatives directly during the smoothing process

Parameters:

• wn_col: Column name for wavelength data (default: “Wn”)
  
• window: Window size for smoothing (default: 15, should be odd)
  
• forder: Polynomial order for fitting (default: 4)
  
• degree: Degree of differentiation (default: 0 = smoothing only)

CoHAspec_filt %>% 
  spec_smooth_sga() %>%
  spec_smartplot(geom = "line")

Function Parameters

The spec_blc_rollingBall() function provides comprehensive control over the baseline correction process:

• .data: A data.frame or tibble containing spectral data
• wn_col: Column name for wavelength data (default: “Wn”)
• wn_min: Minimum wavelength for baseline correction range
• wn_max: Maximum wavelength for baseline correction range
• wm: Window size of the rolling ball algorithm (controls ball radius)
• ws: Smoothing factor of the rolling ball algorithm
• is_abs: Logical indicating if data is in absorbance format

Baseline Correction Application

The following example demonstrates baseline correction applied to a specific wavelength range:

CoHAspec_filt %>% 
    spec_smooth_sga() %>%
    spec_blc_rollingBall(wn_min = 1030,
                         wn_max = 1285,
                         ws = 10,
                         wm = 50) %>%
    tidyspec::spec_smartplot(geom = "line")

In this pipeline:

1. Smoothing is applied first to reduce noise
   
2. Rolling ball correction is performed in the 1030-1285 cm⁻¹ range
   
3. Window size (wm = 50) and smoothing factor (ws = 10) are optimized for the data

Comparing Original and Baseline

For a comprehensive view of the correction process, we can overlay the original spectrum with the estimated baseline:

bl <- CoHAspec_filt %>% 
    spec_smooth_sga() %>%
    spec_bl_rollingBall(wn_col = "Wavenumber",
                         wn_min = 1030,
                         wn_max = 1285,
                         ws = 10, wm = 50)

CoHAspec_filt %>%
  spec_smooth_sga() %>%
  spec_filter(wn_min = 1030, wn_max = 1285) %>% 
  left_join(bl) %>% 
  spec_smartplot(geom = "line")
#> Joining with `by = join_by(Wavenumber, CoHA100)`

This comparison allows you to:

• Evaluate the baseline estimation quality
• Identify regions where the algorithm may need parameter adjustment
• Verify that the baseline follows the expected spectral trend

Mathematical Foundation

For each spectrum, the transformation is:

• Normalized value = (Value - Minimum) / (Maximum - Minimum)
• Results in values ranging from 0 to 1
• Preserves the original spectral shape

CoHAspec_region %>%
    spec_norm_01() %>% 
    spec_smartplot(geom = "line")

The spec_norm_01() function:

• Finds the minimum and maximum values for each spectrum
• Applies the min-max transformation
• Maintains spectral shape while standardizing intensity range
• Is particularly useful when absolute intensity differences are not analytically relevant

When to Use Min-Max Normalization

Min-max scaling is ideal when:

• You want to preserve spectral shape
• Absolute intensities vary due to concentration differences
• Visual comparison of spectral patterns is important
• The dynamic range needs standardization

The user can set any min and max value with the spec_norm_minmax() function:

CoHAspec_region %>%
    spec_norm_minmax(min = 1, max = 2) %>% 
    spec_smartplot(geom = "line")

Mathematical Foundation

For each spectrum, the transformation is:

• Standardized value = (Value - Mean) / Standard Deviation
• Results in mean = 0 and variance = 1
• Emphasizes relative variations around the mean

Application

CoHAspec_region %>%
    spec_norm_var() %>% 
    spec_smartplot(geom = "line")

The spec_norm_var() function:

• Calculates the mean and standard deviation for each spectrum
• Applies z-score transformation
• Centers the data around zero
• Standardizes the variance across all spectra

When to Use Standardization

Standardization is appropriate when:

• You want to emphasize spectral variations
• Baseline shifts need to be removed
• Multivariate analysis requires standardized inputs
• Focus is on relative changes rather than absolute values

Setting up the parameter grid

First, we prepare our spectral data by focusing on a specific region and applying initial smoothing:

dados_1030_1285 <- CoHAspec_filt %>% 
    tidyspec::spec_smooth_sga() %>%
    dplyr::filter(Wavenumber <= 1285, Wavenumber >= 1030)

Next, we create a comprehensive parameter grid using tidyr::crossing(), which generates all possible combinations of the specified parameter values:

params <- tidyr::crossing(ws_val = c(2,4,6,8,10,12),
                          wm_val = c(10, 25, 40, 50))
params 
#> # A tibble: 24 x 2
#>    ws_val wm_val
#>     <dbl>  <dbl>
#>  1      2     10
#>  2      2     25
#>  3      2     40
#>  4      2     50
#>  5      4     10
#>  6      4     25
#>  7      4     40
#>  8      4     50
#>  9      6     10
#> 10      6     25
#> # i 14 more rows

This approach creates a systematic grid of 24 parameter combinations (6 × 4), ensuring thorough exploration of the parameter space.

Applying baseline correction across parameters

The power of the tidyverse becomes evident when we need to apply the same operation across multiple parameter sets. Using dplyr::mutate() with purrr::pmap(), we can efficiently apply baseline correction to each parameter combination:

df <- params %>%
    dplyr::mutate(spectra = list(dados_1030_1285)) %>%
    dplyr::mutate(spectra_blc = purrr::pmap(list(spectra, ws_val, wm_val),
                             \(x, y, z ) spec_blc_rollingBall(.data = x, ws = y, wm = z))) %>%
    dplyr::mutate(title = paste0("ws = ", ws_val," , wm = ", wm_val )) %>%
    dplyr::mutate(plot = purrr::map2(spectra_blc, title, ~ spec_smartplot(
            .data = .x,
            wn_col = "Wavenumber",
            geom = "line") + labs(title = .y)
            ))

This pipeline demonstrates several key tidyverse concepts:

• Nested data structures: Each row contains the complete spectral dataset
• Functional programming: purrr::pmap() applies the baseline correction function across multiple arguments
• Pipeline operations: Each step builds upon the previous one using the pipe operator
• Automated visualization: purrr::map2() creates customized plots for each parameter combination

Visualizing parameter optimization results

The final step involves creating comparative visualizations to assess the effect of different parameter combinations:

library(gridExtra)
#> 
#> Attaching package: 'gridExtra'
#> The following object is masked from 'package:dplyr':
#> 
#>     combine

grid.arrange(grobs = df$plot[1:4], nrow = 2, ncol = 2)

grid.arrange(grobs = df$plot[5:8], nrow = 2, ncol = 2)

grid.arrange(grobs = df$plot[9:12], nrow = 2, ncol = 2)

grid.arrange(grobs = df$plot[13:16], nrow = 2, ncol = 2)

grid.arrange(grobs = df$plot[17:20], nrow = 2, ncol = 2)

grid.arrange(grobs = df$plot[21:24], nrow = 2, ncol = 2)

Benefits of the tidyverse approach

This workflow demonstrates several advantages of integrating tidyspec with the tidyverse:

• Scalability: Easy to add more parameters or parameter values
• Reproducibility: All steps are clearly documented and can be easily modified
• Efficiency: Vectorized operations handle multiple datasets simultaneously
• Flexibility: Each step can be customized or extended as needed
• Readability: The pipeline structure makes the analysis workflow transparent

Extending the approach

This methodology can be extended to optimize other spectral processing parameters:

• Smoothing parameters (window size, polynomial order)
• Normalization methods and ranges
• Derivative calculation parameters Spectral filtering ranges

The combination of tidyspec functions with tidyverse tools creates a powerful framework for systematic spectral analysis, enabling researchers to make data-driven decisions about preprocessing parameters while maintaining clean, reproducible code.