# Data Analysis Part of 'IOHprofiler'

## Documentation for package ‘IOHanalyzer’ version 0.1.8.17

- DESCRIPTION file.

## Help Pages

A B C D E F G I L M P R S misc

IOHanalyzer-package | IOHanalyzer: Data Analysis Part of IOHprofiler

## -- A --

arrange | S3 sort function for DataSetList
arrange.DataSetList | S3 sort function for DataSetList
as.character.DataSet | S3 generic as.character operator for DataSet
AUC | Area Under Curve (Empirical Cumulative Dsitribution Function)
AUC.ECDF | Area Under Curve (Empirical Cumulative Dsitribution Function)

## -- B --

bootstrap_RT | Bootstrapping for running time samples

## -- C --

c.DataSet | S3 concatenation function for DataSet
c.DataSetList | S3 concatenation function for DataSetList
cat.DataSet | S3 generic cat operator for DataSet
change_id | Add unique identifiers to each DataSet in the provided DataSetList based on static attributes
check_dsc_configured | Verify that the credentials for DSCtool have been set
check_format | Check the format of data
clean_DataSetList | Clean DataSetList object by concatenating DataSets

## -- D --

DataSet | Constructor of S3 class 'DataSet'
DataSetList | S3 constructor of the 'DataSetList'
dsl | Example DataSetList used in tests / examples
dsl_large | Larger example DataSetList used in tests / examples

## -- E --

ECDF | Empirical Cumulative Dsitribution Function of Runtime of a single data set
ECDF.DataSet | Empirical Cumulative Dsitribution Function of Runtime of a single data set
ECDF.DataSetList | Empirical Cumulative Dsitribution Function of Runtime of a single data set

## -- F --

fast_RT_samples | Function to get just the RT samples needed, without any formatting to improve speed

## -- G --

generate_data.Aggr | Generate dataframe of a single function/dimension pair
generate_data.AUC | Generate dataframe containing the AUC for any ECDF-curves
generate_data.CDP | Generate data for the cumulative difference plot.
generate_data.EAF | Generate dataframe consisting of the levelsets of the EAF
generate_data.EAF_Difference | Generate differences between two EAFs
generate_data.EAF_diff_Approximate | Generate EAF-differences between each function and the remaining portfolio
generate_data.ECDF | Generate dataframe of a single function/dimension pair
generate_data.ECDF_From_EAF | Generate dataframe consisting of the ECDF-equivalent based on the EAF
generate_data.ECDF_raw | Generate dataframe of a the unaggregated values of individual algorithms. Stripped-down version of
generate_data.Heatmaps | Nevergrad-dashboard based algorithm comparison
generate_data.hist | Generate dataframe of a single function/dimension pair
generate_data.Parameters | Generate dataframe of a single function/dimension pair
generate_data.Parameter_correlation | Generate dataframe of exactly 2 parameters, matched by running time
generate_data.PMF | Generate dataframe of a single function/dimension pair for creating PDF or PMF plots
generate_data.Single_Function | Generate dataframe of a single function/dimension pair
get_algId | Get all algorithm ids present in a DataSetList
get_color_scheme | Get colors according to the current colorScheme of the IOHanalyzer
get_color_scheme_dt | Get datatable of current color (and linestyle) scheme to file
get_default_ECDF_targets | Generate ECDF targets for a DataSetList
get_dim | Get all dimensions present in a DataSetList
get_dsc_omnibus | Perform omnibus statistical tests on the matrix of rankings from the DSCtool api
get_dsc_posthoc | Perform post-hoc processing on data from DSCtool
get_dsc_rank | Get the matrix of rankings using the DSCtool api for a DataSetList
get_ECDF_targets | Generation of default ECDF-targets
get_ERT | Get Expected RunTime
get_ERT.DataSet | Get Expected RunTime
get_ERT.DataSetList | Get Expected RunTime
get_funcId | Get all function ids present in a DataSetList
get_funcName | Get all function names present in a DataSetList
get_funvals | Get all function values present in a DataSetList
get_FV | Get function value matrix of the used dataset.
get_FV.DataSet | Get function value matrix of the used dataset.
get_FV_overview | Get Function Value condensed overview
get_FV_overview.DataSet | Get Function Value condensed overview
get_FV_overview.DataSetList | Get Function Value condensed overview
get_FV_sample | Get Funtion Value Samples
get_FV_sample.DataSet | Get Funtion Value Samples
get_FV_sample.DataSetList | Get Funtion Value Samples
get_FV_summary | Get Function Value Summary
get_FV_summary.DataSet | Get Function Value Summary
get_FV_summary.DataSetList | Get Function Value Summary
get_id | Get condensed overview of datasets
get_id.DataSet | Get condensed overview of datasets
get_id.DataSetList | Get condensed overview of datasets
get_line_style | Get line styles according to the current styleScheme of the IOHanalyzer
get_marg_contrib_ecdf | Get the marginal contribution of an algorithm to a portfolio
get_maxRT | Get the maximal running time
get_maxRT.DataSet | Get the maximal running time
get_maxRT.DataSetList | Get the maximal running time
get_ontology_data | Get the list of available options for data from the OPTION ontology
get_ontology_var | Get the list of available options for data from the OPTION ontology
get_overview | Get condensed overview of datasets
get_overview.DataSet | Get condensed overview of datasets
get_overview.DataSetList | Get condensed overview of datasets
get_parId | Get all parameter ids present in a DataSetList
get_PAR_name | Get the parameter names of the algorithm
get_PAR_name.DataSet | Get the parameter names of the algorithm
get_PAR_sample | Get Parameter Value Samples
get_PAR_sample.DataSet | Get Parameter Value Samples
get_PAR_sample.DataSetList | Get Parameter Value Samples
get_PAR_summary | Get Parameter Value Summary
get_PAR_summary.DataSet | Get Parameter Value Summary
get_PAR_summary.DataSetList | Get Parameter Value Summary
get_position_dsl | Extract the position information from a datasetlist object
get_RT | Get runtime matrix of the used dataset.
get_RT.DataSet | Get runtime matrix of the used dataset.
get_RT_overview | Get Runtime Value condensed overview
get_RT_overview.DataSet | Get Runtime Value condensed overview
get_RT_overview.DataSetList | Get Runtime Value condensed overview
get_RT_sample | Get RunTime Sample
get_RT_sample.DataSet | Get RunTime Sample
get_RT_sample.DataSetList | Get RunTime Sample
get_RT_summary | Get RunTime Summary
get_RT_summary.DataSet | Get RunTime Summary
get_RT_summary.DataSetList | Get RunTime Summary
get_runtimes | Get all runtime values present in a DataSetList
get_shapley_values | Get the shapley-values of a portfolio of algorithms
get_static_attributes | Get all attributes which can be used to subset a DataSetList
get_static_attribute_values | Get all options for a specific attribute which can be used to subset a DataSetList
get_target_dt | Generate datatables of runtime or function value targets for a DataSetList
glicko2_ranking | Glicko2 raning of algorithms

## -- I --

IOHanalyzer | IOHanalyzer: Data Analysis Part of IOHprofiler
IOH_plot_ly_default | Template for creating plots in the IOHanalyzer-style

## -- L --

limit.data | Reduce the size of the data set by evenly subsampling the records

## -- M --

max_ERTs | Get the ERT-values for all DataSets in a DataSetList at certain targets
max_ERTs.DataSetList | Get the ERT-values for all DataSets in a DataSetList at certain targets
mean_FVs | Get the expected function-values for all DataSets in a DataSetList at certain runtimes
mean_FVs.DataSetList | Get the expected function-values for all DataSets in a DataSetList at certain runtimes

## -- P --

pairwise.test | Performs a pairwise Kolmogorov-Smirnov test on the bootstrapped running times among a data set
pairwise.test.DataSetList | Performs a pairwise Kolmogorov-Smirnov test on the bootstrapped running times among a data set
pairwise.test.list | Performs a pairwise Kolmogorov-Smirnov test on the bootstrapped running times among a data set
Plot.Comparison.Heatmap | Plot a heatmap according to the specifications from the Nevergrad dashboard
Plot.Comparison.Heatmap.DataSetList | Plot a heatmap according to the specifications from the Nevergrad dashboard
Plot.cumulative_difference_plot | Plot the cumulative difference plot given a DataSetList.
Plot.FV.Aggregated | Plot expected function value-based comparison over multiple functions or dimensions
Plot.FV.Aggregated.DataSetList | Plot expected function value-based comparison over multiple functions or dimensions
Plot.FV.ECDF_AUC | Radarplot of the area under the aggregated ECDF-curve of a DataSetList.
Plot.FV.ECDF_AUC.DataSetList | Radarplot of the area under the aggregated ECDF-curve of a DataSetList.
Plot.FV.ECDF_Per_Target | Plot the empirical cumulative distriburtion as a function of the target values of a DataSetList at certain target runtimes
Plot.FV.ECDF_Per_Target.DataSetList | Plot the empirical cumulative distriburtion as a function of the target values of a DataSetList at certain target runtimes
Plot.FV.ECDF_Single_Func | Plot the aggregated empirical cumulative distriburtion as a function of the function values of a DataSetList.
Plot.FV.ECDF_Single_Func.DataSetList | Plot the aggregated empirical cumulative distriburtion as a function of the function values of a DataSetList.
Plot.FV.Histogram | Plot histograms of the function values of a DataSetList at a certain target runtime
Plot.FV.Histogram.DataSetList | Plot histograms of the function values of a DataSetList at a certain target runtime
Plot.FV.Multi_Func | Plot FV-plots for multiple functions or dimensions
Plot.FV.Multi_Func.DataSetList | Plot FV-plots for multiple functions or dimensions
Plot.FV.Parameters | Plot the parameter values recorded in a DataSetList (aligned by budget)
Plot.FV.Parameters.DataSetList | Plot the parameter values recorded in a DataSetList (aligned by budget)
Plot.FV.PDF | Plot probablity density function of the function values of a DataSetList at a certain target runtime
Plot.FV.PDF.DataSetList | Plot probablity density function of the function values of a DataSetList at a certain target runtime
Plot.FV.Single_Func | Plot lineplot of the expected function values of a DataSetList
Plot.FV.Single_Func.DataSetList | Plot lineplot of the expected function values of a DataSetList
Plot.Performviz | Create the PerformViz plot
Plot.RT.Aggregated | Plot ERT-based comparison over multiple functions or dimensions
Plot.RT.Aggregated.DataSetList | Plot ERT-based comparison over multiple functions or dimensions
Plot.RT.ECDF_AUC | Radarplot of the area under the aggregated ECDF-curve of a DataSetList.
Plot.RT.ECDF_AUC.DataSetList | Radarplot of the area under the aggregated ECDF-curve of a DataSetList.
Plot.RT.ECDF_Multi_Func | Plot the aggregated empirical cumulative distriburtion as a function of the running times of a DataSetList. Aggregated over multiple functions or dimensions.
Plot.RT.ECDF_Multi_Func.DataSetList | Plot the aggregated empirical cumulative distriburtion as a function of the running times of a DataSetList. Aggregated over multiple functions or dimensions.
Plot.RT.ECDF_Per_Target | Plot the empirical cumulative distriburtion as a function of the running times of a DataSetList at certain target function values
Plot.RT.ECDF_Per_Target.DataSetList | Plot the empirical cumulative distriburtion as a function of the running times of a DataSetList at certain target function values
Plot.RT.ECDF_Single_Func | Plot the aggregated empirical cumulative distriburtion as a function of the running times of a DataSetList.
Plot.RT.ECDF_Single_Func.DataSetList | Plot the aggregated empirical cumulative distriburtion as a function of the running times of a DataSetList.
Plot.RT.Histogram | Plot histograms of the runtimes of a DataSetList at a certain target function value
Plot.RT.Histogram.DataSetList | Plot histograms of the runtimes of a DataSetList at a certain target function value
Plot.RT.Multi_Func | Plot ERT-plots for multiple functions or dimensions
Plot.RT.Multi_Func.DataSetList | Plot ERT-plots for multiple functions or dimensions
Plot.RT.Parameters | Plot the parameter values recorded in a DataSetList (aligned by funcion value)
Plot.RT.Parameters.DataSetList | Plot the parameter values recorded in a DataSetList (aligned by funcion value)
Plot.RT.PMF | Plot probablity mass function of the runtimes of a DataSetList at a certain target function value
Plot.RT.PMF.DataSetList | Plot probablity mass function of the runtimes of a DataSetList at a certain target function value
Plot.RT.Single_Func | Plot lineplot of the ERTs of a DataSetList
Plot.RT.Single_Func.DataSetList | Plot lineplot of the ERTs of a DataSetList
Plot.Stats.Glicko2_Candlestick | Create a candlestick plot of Glicko2-rankings
Plot.Stats.Glicko2_Candlestick.DataSetList | Create a candlestick plot of Glicko2-rankings
Plot.Stats.Significance_Graph | Plot a network graph showing the statistically different algorithms
Plot.Stats.Significance_Graph.DataSetList | Plot a network graph showing the statistically different algorithms
Plot.Stats.Significance_Heatmap | Plot a heatmap showing the statistically different algorithms
Plot.Stats.Significance_Heatmap.DataSetList | Plot a heatmap showing the statistically different algorithms
plot_eaf_data | Create EAF-based polygon plots
plot_eaf_differences | Create EAF-difference contour plots
plot_general_data | General function for plotting within IOHanalyzer
print.DataSet | S3 generic print operator for DataSet
print.DataSetList | S3 print function for DataSetList

## -- R --

read_index_file | Read .info files and extract information
read_IOH_v1plus | Read Nevergrad data
read_pure_csv | Read Nevergrad data
register_DSC | Register an account to the DSCtool API
runServer | Create a shiny-server GUI to interactively use the IOHanalyzer

## -- S --

save_plotly | Save plotly figure in multiple format
save_table | Save DataTable in multiple formats
scan_index_file | Scan *.info files for IOHProfiler or COCO
seq_FV | Function for generating sequences of function values
seq_RT | Function for generating sequences of runtime values
set_color_scheme | Set the colorScheme of the IOHanalyzer plots
set_DSC_credentials | Register an account to the DSCtool API
SP | Estimator 'SP' for the Expected Running Time (ERT)
subset.DataSet | S3 subset function for DataSet
subset.DataSetList | Filter a DataSetList by some criteria
summary.DataSet | S3 generic summary operator for DataSet
summary.DataSetList | S3 summary function for DataSetList

## -- misc --

==.DataSet | S3 generic == operator for DataSets
[.DataSetList | S3 extraction function for DataSetList