syncdr: Facilitate File Handling, Directory Comparison & Synchronization

Description

Compare directories flexibly (by date, content, or both) and synchronize files efficiently, with asymmetric and symmetric modes, helper tools, and visualization support for file management.

Author(s)

Maintainer: Rossana Tatulli rtatulli@worldbank.org

Authors:

• R.Andres Castaneda acastanedaa@worldbank.org

Other contributors:

• Global Poverty and Inequality Data Team World Bank [copyright holder]

See Also

Useful links:

• https://rossanatat.github.io/syncdr/

• https://github.com/RossanaTat/syncdr

• Report bugs at https://github.com/RossanaTat/syncdr/issues

Partial asymmetric synchronization to right (update common files)

Description

Partially synchronize right directory based on left one -i.e., the function will:

• for common_files:

  • if by date only: copy files that are newer in left to right

  • if by date and content: copy files that are newer and different in left to right

  • if by content only: copy files that are different in left to right

• for non common files, nothing changes: i.e.,

  • disregard those files that are only in left

  • keep in right those files that are only in right (i.e., files 'missing in left')

Usage

common_files_asym_sync_to_right(
  left_path = NULL,
  right_path = NULL,
  sync_status = NULL,
  by_date = TRUE,
  by_content = FALSE,
  recurse = TRUE,
  force = TRUE,
  backup = FALSE,
  backup_dir = "temp_dir",
  verbose = getOption("syncdr.verbose")
)

Arguments

Value

Invisible TRUE indicating successful synchronization.

Examples

# Asymmetric synchronization of common files


e <- toy_dirs()
left  <- e$left
right <- e$right
# Synchronize common files by content only
common_files_asym_sync_to_right(
  left_path  = left,
  right_path = right,
  by_date    = FALSE,
  by_content = TRUE
)

Compare Two Directories for Synchronization Status

Description

This function compares two directories, typically referred to as 'left' and 'right', to determine their synchronization status at the file level. The primary goal is to identify the synchronization status of files present in both directories and those exclusive to either directory.

Usage

compare_directories(
  left_path,
  right_path,
  recurse = TRUE,
  by_date = TRUE,
  by_content = FALSE,
  verbose = getOption("syncdr.verbose")
)

Arguments

Value

A list of class "syncdr_status" containing the following elements:

• Non-common files: Paths and synchronization status of files exclusive to either directory.

• Common files: Paths and synchronization status of files present in both directories.

• Path of the left directory.

• Path of the right directory.

Sync Status Types

The synchronization status is determined for files present in both directories, as well as for files exclusive to either directory. It can be computed based on modification date only, content only, or both.

For Common Files:

• When comparing by date: 'new', 'old', or 'same'.

• When comparing by date and content: 'new and different', 'new and same', 'old and different', 'old and same', 'same and different', or 'same and same'.

• When comparing by content only: 'different' or 'same'.

For Non-Common Files:

• When comparing by date (or by date and content, or by content only): 'only in left' or 'only in right'.

Examples

e <- toy_dirs()
left  <- e$left
right <- e$right
compare_directories(left, right)
compare_directories(left, right, by_content = TRUE)
compare_directories(left, right, by_content = TRUE, by_date = FALSE)

Compare contents of two files and determine synchronization status

Description

This function compares the contents of two files located at specified paths and determines their synchronization status based on their content

Usage

compare_file_contents(
  path_left,
  path_right,
  verbose = getOption("syncdr.verbose")
)

Arguments

Value

A list containing the following components:

Compare modification times of two files and determine synchronization status

Description

This function compares the date of last modification of two files and determines their synchronization status

Usage

compare_modification_times(modification_time_left, modification_time_right)

Arguments

Value

A list containing the following components:

Copy files from right directory to left directory

Description

This function copies files from a right (source) directory to a left (destination) directory based on a provided data frame containing file paths and synchronization status.

Usage

copy_files_to_left(left_dir, right_dir, files_to_copy, recurse = TRUE)

Arguments

Details

The function performs the following steps:

1. Checks if the source (right) and destination (left) directories exist and creates the destination directory if it doesn't already exist.

2. Copies files from the right directory to the corresponding subdirectory in the left directory based on the provided file paths.

Value

Invisible TRUE upon successful completion of the file copying process.

Copy files from left to right directory

Description

This function copies files from a source directory (left_dir) to a destination directory (right_dir) based on a provided data frame containing file paths and synchronization status.

Usage

copy_files_to_right(left_dir, right_dir, files_to_copy, recurse = TRUE)

Arguments

Details

The function performs the following steps:

1. Checks if the source and destination directories exist and creates the destination directory if it doesn't already exist.

2. Copies files from the source directory to the corresponding subdirectory in the destination directory based on the provided file paths.

Value

Invisible TRUE upon successful completion of the file copying process.

Create a temporary copy of .syncdrenv to test functions

Description

This function creates a copy of the original environment, allowing tests to be executed without modifying the original environment.

Usage

copy_temp_environment()

Value

A list of temporary paths left and right.

Retrieve information about files in a directory

Description

This function retrieves information about files in a specified directory

Usage

directory_info(dir, recurse = TRUE)

Arguments

Value

A data frame containing detailed information about the files in the directory, including all information produced by fs::file_info().

Display tree structure of one (or two) directory

Description

Display tree structure of one (or two) directory

Usage

display_dir_tree(path_left = NULL, path_right = NULL, recurse = TRUE)

Arguments

Value

directories tree

Examples

# Create a temporary directory structure


e <- toy_dirs()
left  <- e$left
right <- e$right

display_dir_tree(
  path_left  = left,
  path_right = right
)

display_dir_tree(path_right = right)

Display file actions in table

Description

This function displays actions (either "copy" or "delete") to be performed on a list of files.

Usage

display_file_actions(path_to_files, directory, action = c("copy", "delete"))

Arguments

Value

console-friendly table with files and actions

Display status of synchronization/comparison info between two directories in DT table

Description

Display status of synchronization/comparison info between two directories in DT table

Usage

display_sync_status(sync_status_files, left_path, right_path)

Arguments

Value

DT table showing the comparison between the two directories together with their synchronization status

Filter common files in a syncdr_status object based on specified criteria

Description

This function filters common files within a "syncdr_status" object, which is the result of 'compare_directories()', according to the specified filtering criteria: Filtering is dependent on the 'dir' argument, determining the primary directory for comparison

Usage

filter_common_files(
  sync_status,
  by_date = TRUE,
  by_content = FALSE,
  dir = "left"
)

Arguments

Details

Filtering Options:

• by_date: Filters files that are new in the specified primary directory ('left', 'right', or both).

• by_date_and_content: Filters files that are either new or different in the specified primary directory ('left', 'right', or both).

• by_content_only: Filters files that are different between the two directories.

Value

A 'syncdr_status' object filtered according to the specified criteria.

See Also

compare_directories for directory comparison and sync status creation.

Filter files in a syncdr_status object that are NOT common between two directories compared

Description

This function filters files that are not common between the directories compared in the 'sync_status' object resulting from 'compare_directories()'.

Usage

filter_non_common_files(sync_status, dir = "left")

Arguments

Value

An updated 'syncdr_status' object with filtered files according to the specified criteria.

Full asymmetric synchronization to right directory

Description

This function performs a full asymmetric synchronization of the right directory based on the left directory. It includes the following synchronization steps (see Details below):

Usage

full_asym_sync_to_right(
  left_path = NULL,
  right_path = NULL,
  sync_status = NULL,
  by_date = TRUE,
  by_content = FALSE,
  recurse = TRUE,
  force = TRUE,
  delete_in_right = TRUE,
  backup = FALSE,
  backup_dir = "temp_dir",
  verbose = getOption("syncdr.verbose")
)

Arguments

Details

• For common files:

  • If comparing by date only (by_date = TRUE): Copy files that are newer in the left directory to the right directory.

  • If comparing by date and content (by_date = TRUE and by_content = TRUE): Copy files that are newer and different in the left directory to the right directory.

  • If comparing by content only (by_content = TRUE): Copy files that are different in the left directory to the right directory.

• Copy to the right directory those files that exist only in the left directory.

• Delete from the right directory those files that are exclusive in the right directory (i.e., missing in the left directory)

Value

Invisible TRUE indicating successful synchronization.

Examples

e <- toy_dirs(fast = TRUE)
left  <- e$left
right <- e$right
full_asym_sync_to_right(
  left_path  = left,
  right_path = right,
  by_date    = FALSE,
  by_content = TRUE
)
sync_status <- compare_directories(left_path = left, right_path = right)
full_asym_sync_to_right(sync_status = sync_status)

Full symmetric synchronization

Description

This function updates directories in the following way:

• For common files:

  • if by date: If the file in one directory is newer than the corresponding file in the other directory, it will be copied over to update the older version. If modification dates are the same, no action is taken

  • if by date and content: If the file in one directory is newer AND different than the corresponding file in the other directory, it will be copied over to update the older version. If modification dates/contents are the same, no action is taken

  • if by content only: this option is not active

• For non common files:

  • if a file exists in one but not in the other it is copied to the other directory

Usage

full_symmetric_sync(
  left_path = NULL,
  right_path = NULL,
  sync_status = NULL,
  by_date = TRUE,
  by_content = FALSE,
  recurse = TRUE,
  force = TRUE,
  backup = FALSE,
  backup_dir = "temp_dir",
  verbose = getOption("syncdr.verbose")
)

Arguments

Value

Invisible TRUE indicating successful synchronization.

Examples

# Create a temporary synchronization environment


e <- toy_dirs()
left  <- e$left
right <- e$right
# Symmetric synchronization by date and content
# Option 1: provide left and right paths
full_symmetric_sync(
  left_path  = left,
  right_path = right,
  by_date    = TRUE,
  by_content = TRUE
)

# Option 2: provide a precomputed sync_status object
sync_status <- compare_directories(
  left_path  = left,
  right_path = right
)
full_symmetric_sync(sync_status = sync_status)

Hash files by content

Description

Hash files by content

Usage

hash_files(files_path, verbose = getOption("syncdr.verbose"))

Arguments

Value

hashes of files

Hash files in a directory based on content

Description

This function calculates hashes for files in a specified directory based on their content.

Usage

hash_files_in_dir(dir_path)

Arguments

Value

A data frame containing file paths and their corresponding SHA-256 hashes.

Partial symmetric synchronization -common files only

Description

This function updates directories in the following way:

• For common files:

  • if by date: If the file in one directory is newer than the corresponding file in the other directory, it will be copied over to update the older version. If modification dates are the same, nothing is done

  • if by date and content: If the file in one directory is newer AND different than the corresponding file in the other directory, it will be copied over to update the older version. If modification dates/contents are the same, nothing is done

  • if by content only: this option is not active

• For non common files: unchanged, i.e.,

  • keep in right those that are only in right

  • keep in left those that are only in left

Usage

partial_symmetric_sync_common_files(
  left_path = NULL,
  right_path = NULL,
  sync_status = NULL,
  by_date = TRUE,
  by_content = FALSE,
  recurse = TRUE,
  force = TRUE,
  backup = FALSE,
  backup_dir = "temp_dir",
  verbose = getOption("syncdr.verbose")
)

Arguments

Value

Invisible TRUE indicating successful synchronization.

Examples

# Create a temporary synchronization environment


e <- toy_dirs()
left  <- e$left
right <- e$right

# Partial symmetric synchronization of common files
# Option 1: provide left and right paths
partial_symmetric_sync_common_files(
  left_path  = left,
  right_path = right,
  by_date    = TRUE
)

# Option 2: provide a precomputed sync_status object
sync_status <- compare_directories(
  left_path  = left,
  right_path = right
)
partial_symmetric_sync_common_files(sync_status = sync_status)

Partial asymmetric asymmetric synchronization of non common files

Description

update non common files in right directory based on left one -i.e., the function will:

• for common_files:

  • do nothing, left unchanged

• for non common files,

  • copy those files that are only in left to right

  • keep in right those files that are only in right (i.e., files 'missing in left')

Usage

partial_update_missing_files_asym_to_right(
  left_path = NULL,
  right_path = NULL,
  sync_status = NULL,
  recurse = TRUE,
  force = TRUE,
  backup = FALSE,
  backup_dir = "temp_dir",
  verbose = getOption("syncdr.verbose")
)

Arguments

Value

Invisible TRUE indicating successful synchronization.

Examples

# Create a temporary synchronization environment

e <- toy_dirs()
left  <- e$left
right <- e$right

# Partially update missing files asymmetrically (left → right)
# Option 1: provide left and right paths
partial_update_missing_files_asym_to_right(
  left_path  = left,
  right_path = right
)

# Option 2: provide a precomputed sync_status object
sync_status <- compare_directories(
  left_path  = left,
  right_path = right
)
partial_update_missing_files_asym_to_right(sync_status = sync_status)

Print Synchronization Status

Description

Print Synchronization Status

Usage

## S3 method for class 'syncdr_status'
print(x, ...)

Arguments

Value

prints syncdr_status object

Set theme for colorDF

Description

Set theme for colorDF

Usage

rs_theme()

Value

invisible RStudio theme

Save sync_status file

Description

Save sync_status file

Usage

save_sync_status(dir_path)

Arguments

Value

the file is saved in a ⁠_syncdr⁠ subdirectory within the specified directory

Examples

# Set the directory path
e = toy_dirs()
left <- e$left

save_sync_status(dir_path = left)

Search for duplicate files in a directory

Description

This function searches for duplicate files within a directory based on their content. Duplicate files are identified by having either the same filename and same content or different filenames but same content.

Usage

search_duplicates(dir_path, verbose = TRUE)

Arguments

Value

A data frame containing information about duplicate files (invisible by default)

Examples

# Search for duplicate files in a directory


e <- toy_dirs()
search_duplicates(dir_path = e$left)

Apply Custom Style to Text

Description

This function applies a custom color and bold style to a given text string.

Usage

style_msgs(color_name, text)

Arguments

Value

The styled text is printed to the console.

Create toy directories to test syncdr functions

Description

create directories in syncdr environment. Directories are deleted when a new R session is started

Usage

toy_dirs(verbose = FALSE, fast = FALSE)

Arguments

Details

This function is a little slow because it must use Sys.sleep() to save files with the same name but different time stamp.

Value

syncdr environment with toy directory paths, i.e., left and right paths

Examples

# Create toy directories for testing / examples

toy_dirs(verbose = TRUE)

Full asymmetric synchronization of non common files

Description

update non common files in right directory based on left one -i.e., the function will:

• for common_files:

  • do nothing, left unchanged

• for non common files,

  • copy those files that are only in left to right

  • delete in right those files that are only in right (i.e., files 'missing in left')

Usage

update_missing_files_asym_to_right(
  left_path = NULL,
  right_path = NULL,
  sync_status = NULL,
  recurse = TRUE,
  force = TRUE,
  backup = FALSE,
  backup_dir = "temp_dir",
  copy_to_right = TRUE,
  delete_in_right = TRUE,
  exclude_delete = NULL,
  verbose = getOption("syncdr.verbose")
)

Arguments

Value

Invisible TRUE indicating successful synchronization.

Examples

# Create a temporary synchronization environment

e <- toy_dirs()
left  <- e$left
right <- e$right

# Update missing files asymmetrically (left → right)
# Option 1: provide left and right paths
update_missing_files_asym_to_right(
  left_path  = left,
  right_path = right
)

# Option 2: provide a precomputed sync_status object
sync_status <- compare_directories(
  left_path  = left,
  right_path = right
)
update_missing_files_asym_to_right(sync_status = sync_status)