Option 1: Installing the stable version from CRAN

install.packages(pkgs="RAQSAPI", dependencies = TRUE )

Option 2: Installing the development version of RAQSAPI

To install the development version of RAQSAPI, first if not already installed, install the remotes package and its dependencies. Then run the following in a R environment.

remotes::install_github(repo = "USEPA/raqsapi",
                        dependencies = TRUE,
                        upgrade = "always",
                        build = TRUE,
                        #optional, set TRUE if the manual is desired,
                        #requires pandoc
                        build_manual = FALSE,
                        build_vignettes = TRUE 
                        )

Load RAQSAPI

after successfully installing the RAQSAPI package, load the RAQSAPI library:

library(RAQSAPI)

Sign up and setting up user credentials with the RAQSAPI library

If you have not already done so you will need to sign up with AQS Data Mart using aqs_sign_up function, ² this function takes one input, “email”, which is a R character object, that represents the email address that you want to use as a user credential to the AQS Data Mart service. After a successful call to aqs_sign_up an email message will be sent to the email address provided with a new Data Mart key which will be used as a credential key to access the Data Mart API. The aqs_sign_up function can also be used to regenerate a new key for an existing user, to generate a new key simply call the aqs_sign_up function with the parameter “email” set to an existing account. A new key will be e-mailed to the account given.

The credentials used to access the Data Mart API service are stored in a R environment variable that needs to be set every time the RAQSAPI library is attached or the key is changed. Without valid credentials, the Data Mart server will reject any request sent to it. The key used with Data Mart is a key and is not a password, so the RAQSAPI library does not treat the key as a password; this means that the key is stored in plain text and there are no attempts to encrypt Data Mart credentials as would be done for a username and password combination. The key that is supplied to use with Data Mart is not intended for authentication but only account monitoring. Each time RAQSAPI is loaded and before using any of it’s functions use the aqs_credentials ³ function to enter in the user credentials so that RAQSAPI can access the AQS Data Mart server.

(suggested) Use the keyring package to manage credentials

It is highly suggested that users use a keyring manager to store and retrieve their credentials while using RAQSAPI. One such credential manager is provided by the keyring package ⁴. The Keyring package uses the credential manager available for most popular operating systems to store and manage user credentials. This will help avoid hard coding credential information into R scripts.

To use the keyring package with RAQSAPI first install keyring:

install.package("keyring")

Ensure that your system is supported by the keyring package before proceeding.

  keyring::has_keyring_support()

then set the keyring used to access AQS Data Mart (make sure to replace the text in the angled brackets with your specific user information):

  library("keyring")  
  keyring::key_set(service = "AQSDatamart",
                   username = "\<user email account\>")

a popup window will appear for the user to input their keyring information. Enter the AQS Data mart credential key associated with the AQS user name provided, then hit enter. Now the AQS Data Mart user credential is set using keyring.

To retrieve the keyring to use with RAQSAPI load the keyring package and use the function key_get to return the user credential to RAQSAPI:

  library(RAQSAPI)  
  library(keyring)  
  datamartAPI_user <- \<user email account\>  
  server <- "AQSDatamart"

then pass these variables to the aqs_credentials function when using RAQSAPI:

  aqs_credentials(username = datamartAPI_user,
                  key = key_get(service = server,
                                username = datamartAPI_user
                                )
                  )

To change the keyring stored with the keyring package repeat the steps above to call the keyring::key_set function again with the new credential information.

To retrieve a list of all keyrings managed with the keyring package use the function: > keyring::key_list()

Refer the thekeyring package documentation for an in depth explanation on using the keyring package.

Usage tips and precautions

This section contains suggestions for completing certain data related tasks.

• Determine if or how much data exists for a time-parameter-geography combination:
  • Retrieve data using the annualdata service.
  • If no records are returned, we do not have the data.
  • If records are returned, use the observation count to determine the temporal and geographic distribution of the data.
• Monthly averages:
  • AQS does not routinely calculate monthly aggregate statistics.
  • If you need these, you must calculate them yourself.
  • These can be calculated from the sample data or the daily data without loss of fidelity.
• Determine a single value for a site with collocated monitors:
  • Many sites will have collocated monitors - monitors collecting the same parameter at the same time.
  • The API currently provides only monitor level values. (site-level values will be added in the future.)
  • For some criteria pollutants (PM2.5, ozone, lead, and NO2), the regulations define procedures for defining a single site-level value.
  • For other pollutants, determining a single site-level value is left to the investigator.
• Please adhere to the following when using the AQS Data Mart API:
  • Limit the size of queries. The AQS Data Mart contains billions of values and you may request more than you intend. If you are unsure of the amount of data, start small and work your way up. Please limit queries to 1,000,000 rows of data each. You can use the “observation count” field on the annualdata service to determine how much data exists for a time-parameter-geography combination.
    
  • Limit the frequency of queries. The AQS Data Mart can process a limited load. Please wait for one request to complete before submitting another and do not make more than 10 requests per minute.
  • Be advised that RAQSAPI is capable of retrieving results for multiple pollutants, this can result in the amount of data being returned being multiplied by the number of pollutants being requested.
  • Be advised that the AQS Data Mart API limits certain data requests to one year of data at a time with the exception of the Monitor service. In order to retrieve multiple years of data for these functions the RAQSAPI library conveniently sends multiple API requests to the Data Mart API server, one request for each year, this can result in the amount of data being returned being multiplied by the number of years of data being requested.

The AQS Data Mart administrators may disable accounts without notice for failure to adhere to these terms (Though they will contact the offending user via the email address provided)

Sign up and credentials

The functions included in this family of functions are:

aqs_credentials  
aqs_sign_up

These functions are used to sign up with Data Mart and to store credential information to use with RAQSAPI. The RAQSAPI::aqs_signup function takes one parameter:

• email:

The RAQSAPI::aqs_credentials function takes two parameters:

• username:
• key:

Data Mart API metadata functions

aqs_fields_by_service  
aqs_isavailable  
aqs_knownissues

These functions return the status of Data Mart API or metadata associated with it.

The RAQSAPI::aqs_isavailable function takes no parameters and returns a table which details the status of the AQS API.

The RAQSAPI::aqs_fields_by_service function takes one parameter, service, which is a R character object which represents the services provided by the AQS API. For a list of available services see Air Quality System (AQS) API - Services Overview

The RAQSAPI::aqs_knownissues function takes no parameters and Returns a table of any known issues with system functionality or the data. These are usually issues that have been identified internally and will require some time to correct in Data Mart or the API. This function implements a direct API call to Data Mart and returns data directly from the API. Issues returned via this function do not include any issues from the RAQSAPI R package.

The RAQSAPI::aqs_revisionhistory function is used to query Data Mart for the change history to the API.

Data Mart API list functions

aqs_cbsas  
aqs_classes  
aqs_counties_by_state  
aqs_mas  
aqs_pqaos  
aqs_sites_by_county  
aqs_states

List functions return the API options or groupings that can be used in conjunction with other API calls. By default each function in this category returns results as a tibble. If return_header parameter is set to TRUE a AQSAPI_v2 object is returned instead.

RAQSAPI::aqs_cbsas returns a table of all available Core Based Statistical Areas (cbsas) and their respective cbsa codes.

RAQSAPI::aqs_states takes no arguments and returns a table of the available states and their respective state FIPS codes.

RAQSAPI::aqs_sampledurations takes no arguments and returns a table of the available sample duration code used to construct other requests.

RAQSAPI::aqs_classes takes no arguments and returns a table of parameter classes (groups of parameters, i.e. “criteria” or “all”).

RAQSAPI::aqs_counties_by_state takes one parameter, stateFIPS, which is a two digit state FIPS code for the state being requested represented as a R character object and returns a table of counties and their respective FIPS code for the state requested. Use RAQSAPI::aqs_states to receive a table of valid state FIPS codes.

RAQSAPI::aqs_sites_by_county takes two parameters, stateFIPS, which is a two digit state FIPS code for the state being requested and county_code which is a three digit county FIPS code for the county being requested, both stateFIPS and county_code should be encoded as a R character object This function returns a table of all air monitoring sites with the requested state and county FIPS code combination.

RAQSAPI::aqs_pqaos takes no parameters and returns an AQS_DATAMART_APIv2 S3 object containing a table of primary quality assurance organizations (pqaos).

RAQSAPI::aqs_mas takes no parameters and returns an AQS_DATAMART_APIv2 S3 object containing a table of monitoring agencies (MA).

Data Mart aggregate functions

These functions retrieve aggregated data from the Data Mart API and are grouped by how each function aggregates the data. There are 5 different families of related aggregate functions. These families are arranged by how the Data Mart API groups the returned data, _by_site, _by_county, _by_state, by (_by_box) and by (_by_cbsa). Within each family of aggregated data functions there are functions that call on the 10 different services that the Data Mart API provides. All Aggregate functions return a tibble by default. If the return_Header parameter is set to TRUE an AQS_DATAMART_APIv2 S3 object is returned instead.

• These fourteen services are:

1. Monitors: Returns operational information about the samplers (monitors) used to collect the data. Includes identifying information, operational dates, operating organizations, etc. Functions using this service contain *monitors_by_* in the function name.

2. Sample Data: Returns sample data - the most fine grain data reported to EPA. Usually hourly, sometimes 5-minute, 12-hour, etc. This service is available in several geographic selections based on geography: site, county, state, cbsa (core based statistical area, a grouping of counties), or by latitude/longitude bounding box. Functions using this service contain *sampledata_by_* in the function name. All Sample Data functions accept two additional, optional parameters; cbdate and cedate:

   • cbdate: a R date object which represents a “beginning date of last change” that indicates when the data was last updated. cbdate is used to filter data based on the change date. Only data that changed on or after this date will be returned. This is an optional variable which defaults to NA_Date_.
   • cedate: a R date object which represents an “end date of last change” that indicates when the data was last updated. cedate is used to filter data based on the change date. Only data that changed on or before this date will be returned. This is an optional variable which defaults to NA_Date_.
   • duration: an optional R character string that represents the parameter duration code that limits returned data to a specific sample duration. The default value of NA_character_ results in no filtering based on duration code. Valid durations include actual sample durations and not calculated durations such as 8 hour CO or \(O_3\) rolling averages, 3/6 day PM averages or Pb 3 month rolling averages. Refer to [aqs_sampledurations()] for a list of all available duration codes.

3. Daily Summary Data: Returns data summarized at the daily level. All daily summaries are calculated on midnight to midnight basis in local time. Variables returned include date, mean value, maximum value, etc. Functions using this service contain *dailysummary_by_* in the function name. All Daily Summary Data functions accept two additional parameters; cbdate and cedate:

   • cbdate: a R date object which represents a “beginning date of last change” that indicates when the data was last updated. cbdate is used to filter data based on the change date. Only data that changed on or after this date will be returned. This is an optional variable which defaults to NA_Date_.
   • cedate: a R date object which represents an “end date of last change” that indicates when the data was last updated. cedate is used to filter data based on the change date. Only data that changed on or before this date will be returned. This is an optional variable which defaults to NA_Date_.

4. Annual Summary Data: Returns data summarized at the yearly level. Variables include mean value, maxima, percentiles, etc. Functions using this service contain *annualdata_by_* in the function name. All Annual Summary Data functions accept two additional parameters; cbdate and cedate:

   • cbdate: a R date object which represents a “beginning date of last change” that indicates when the data was last updated. cbdate is used to filter data based on the change date. Only data that changed on or after this date will be returned. This is an optional variable which defaults to NA_Date_.
   • cedate: a R date object which represents an “end date of last change” that indicates when the data was last updated. cedate is used to filter data based on the change date. Only data that changed on or before this date will be returned. This is an optional variable which defaults to NA_Date_.

5. Quarterly Summary Data: Returns data summarized at the quarterly level. Variables include mean value, maxima, percentiles, etc. Functions using this service contain *quarterlydata_by_* in the function name. All Annual Summary Data functions accept two additional parameters; cbdate and cedate:

   • cbdate: a R date object which represents a “beginning date of last change” that indicates when the data was last updated. cbdate is used to filter data based on the change date. Only data that changed on or after this date will be returned. This is an optional variable which defaults to NA_Date_.
   • cedate: a R date object which represents an “end date of last change” that indicates when the data was last updated. cedate is used to filter data based on the change date. Only data that changed on or before this date will be returned. This is an optional variable which defaults to NA_Date_.

6. Quality Assurance - Blanks Data: Quality assurance data - blanks samples. Blanks are unexposed sample collection devices (e.g., filters) that are transported with the exposed sample devices to assess if contamination is occurring during the transport or handling of the samples. Functions using this service contain *qa_blanks_by_* in the function name.
   

7. Quality Assurance - Collocated Assessments: Quality assurance data - collocated assessments. Collocated assessments are pairs of samples collected by different samplers at the same time and place. (These are “operational” samplers, assessments with independently calibrated samplers are called “audits”.). Functions using this service contain *qa_collocated_assessments_by_* in the function name.

8. Quality Assurance - Flow Rate Verifications: Quality assurance data - flow rate verifications. Several times per year, each PM monitor must have it’s (fixed) flow rate verified by an operator taking a measurement of the flow rate. Functions using this service contain *qa_flowrateverification_by_* in the function name.

9. Quality Assurance - Flow Rate Audits: Quality assurance data - flow rate audits. At least twice year, each PM monitor must have it’s flow rate measurement audited by an expert using a different method than is used for flow rate verifications. Functions using this service contain *qa_flowrateaudit_by_* in the function name.

10. Quality Assurance - One Point Quality Control Raw Data: Quality assurance data - one point quality control check raw data. At least every two weeks, certain gaseous monitors must be challenged with a known concentration to determine monitor performance. Functions using this service contain *qa_one_point_qc_by_* in the function name.
    

11. Quality Assurance - pep Audits: Quality assurance data - performance evaluation program (pep) audits. pep audits are independent assessments used to estimate total measurement system bias with a primary quality assurance organization. Functions using this service contain *qa_pep_audit_by_* in the function name.

12. Transaction Sample - AQS Submission data in transaction format (RD): Transaction sample data - The raw transaction sample data uploaded to AQS by the agency responsible for data submissions in RD format. Functions using this service contain *transactionsample_by_* in the function name. Transaction sample data is only available aggregated by site, county, state or monitoring agency.

13. Quality Assurance - Annual Performance Evaluations: Quality assurance data - Annual performance evaluations. A performance evaluation must be conducted on each primary monitor once per year. The percent differences between known and measured concentrations at several levels are used to assess the quality of the monitoring data. Functions using this service contain *aqs_qa_annualperformanceeval_by_* in the function name. Annual performance in transaction format are only available aggregated by site, county, state, monitoring agency, and primary quality assurance organization. Annual performance evaluations are only available aggregated by site, county, state, monitoring agency, and primary quality assurance organization.

14. Quality Assurance - Annual performance Evaluations in transaction
    format (RD): Quality assurance data - The raw transaction annual performance evaluations data in RD format. Functions using this service contain *aqs_qa_annualperformanceevaltransaction_by_* in the function name. Annual performance evaluations in transaction format are only available aggregated by site, county, state, monitoring agency, and primary quality assurance organization.

aqs_annualsummary_by_site  
aqs_dailysummary_by_site  
aqs_monitors_by_site  
aqs_qa_annualperformanceeval_by_site  
aqs_qa_annualperformanceevaltransaction_by_site  
aqs_qa_blanks_by_site  
aqs_qa_collocated_assessments_by_site  
aqs_qa_flowrateaudit_by_site  
aqs_qa_flowrateverification_by_site  
aqs_qa_one_point_qc_by_site  
aqs_qa_pep_audit_by_site  
aqs_quarterlysummary_by_site  
aqs_sampledata_by_site  
aqs_transactionsample_by_site

aqs_annualsummary_by_county  
aqs_dailysummary_by_county  
aqs_monitors_by_county  
aqs_qa_annualperformanceeeval_by_county  
aqs_qa_annualperformanceevaltransaction_by_county  
aqs_qa_blanks_by_county  
aqs_qa_collocated_assessments_by_county  
aqs_qa_flowrateaudit_by_county  
aqs_qa_flowrateverification_by_county  
aqs_qa_one_point_qc_by_county  
aqs_qa_pep_audit_by_county  
aqs_quarterlysummary_by_county  
aqs_sampledata_by_county  
aqs_sites_by_county  
aqs_transactionsample_by_county

aqs_annualsummary_by_state  
aqs_counties_by_state  
aqs_dailysummary_by_state  
aqs_monitors_by_state  
aqs_qa_annualperformanceeval_by_state  
aqs_qa_annualperformanceevaltransaction_by_state  
aqs_qa_blanks_by_state  
aqs_qa_collocated_assessments_by_state  
aqs_qa_flowrateaudit_by_state  
aqs_qa_flowrateverification_by_state  
aqs_qa_one_point_qc_by_state  
aqs_qa_pep_audit_by_state  
aqs_quarterlysummary_by_state  
aqs_sampledata_by_state  
aqs_transactionsample_by_state

aqs_qa_annualperformanceeval_by_MA  
aqs_qa_annualperformanceevaltransaction_by_MA  
aqs_qa_blanks_by_MA  
aqs_qa_collocated_assessments_by_MA  
aqs_qa_flowrateaudit_by_MA  
aqs_qa_flowrateverification_by_MA  
aqs_qa_one_point_qc_by_MA  
aqs_qa_pep_audit_by_MA  
aqs_transactionsample_by_MA

aqs_annualsummary_by_cbsa  
aqs_dailysummary_by_cbsa  
aqs_monitors_by_cbsa  
aqs_sampledata_by_cbsa

aqs_qa_annualperformanceeval_by_pqao  
aqs_qa_annualperformanceevaltransaction_by_pqao  
aqs_qa_blanks_by_pqao  
aqs_qa_collocated_assessments_by_pqao  
aqs_qa_flowrateaudit_by_pqao  
aqs_qa_flowrateverification_by_pqao  
aqs_qa_one_point_qc_by_pqao  
aqs_qa_pep_audit_by_pqao  
aqs_quarterlysummary_by_pqao

aqs_annualsummary_by_box  
aqs_dailysummary_by_box  
aqs_monitors_by_box  
aqs_quarterlysummary_by_box  
aqs_sampledata_by_box

aqs_removeheader

pyaqsapi - a port of RAQSAPI to the python 3 programming environment

For users that feel more comfortable working within a python 3 environment, pyaqsapi⁴, a port of RAQSAPI to the python 3 language has been released. Both projects aim to maintain feature parity with the other and there are no inherent advantages to using either project over the other, except for the ability of working within the programming language environment of choice. The API of both packages are very structured similarly, both packages export the same data, use the same credentials and data source to retrieve data.