Time Zones, Missing Hours and Daily Data

1. Timestamps are UTC and mark the end of the hour

The historical files record time in UTC. inmet_read() keeps it that way by default, so datetime is unambiguous for every station in the country:

x$datetime[1:3]
#> [1] "2023-01-01 00:00:00 UTC" "2023-01-01 01:00:00 UTC"
#> [3] "2023-01-01 02:00:00 UTC"
attr(x$datetime, "tzone")
#> [1] "UTC"

Many columns describe the hour that ends at the timestamp: precipitation is the total of the previous hour, and temp_max_prev_c and temp_min_prev_c are the highest and lowest air temperature during that hour. The record stamped 03:00 UTC therefore covers 02:00 to 03:00 UTC, which is 23:00 to midnight in Porto Alegre.

Displaying the same instants in local time only changes how they print:

local <- x$datetime[1:3]
attr(local, "tzone") <- "America/Sao_Paulo"
local
#> [1] "2022-12-31 21:00:00 -03" "2022-12-31 22:00:00 -03"
#> [3] "2022-12-31 23:00:00 -03"

2. Which day does an hour belong to?

A daily maximum in Brazil should be the maximum of a local day. Grouping by as.Date(datetime) uses UTC days instead, which in Porto Alegre run from 21:00 to 21:00 local time (22:00 to 22:00 during daylight saving time, which Brazil observed until 2019). An evening shower or a hot late afternoon can then land on the wrong day.

inmet_daily() assigns each record to the local day containing the middle of its hour (datetime minus 30 minutes). The two choices give different days:

utc_day <- as.Date(x$datetime)
local_day <- as.Date(x$datetime - 1800, tz = "America/Sao_Paulo")
mean(utc_day != local_day)
#> [1] 0.1624149

One hour in six (the records stamped 00:00 to 03:00 UTC) is placed on a different day, and the differences matter for daily totals and extremes:

wrong <- tapply(x$precip_mm, utc_day, sum)
right <- inmet_daily(x, min_hours = 1)
right <- setNames(right$precip_mm, right$date)
common <- intersect(names(wrong), names(right))
summary(abs(wrong[common] - right[common]))
#>    Min. 1st Qu.  Median    Mean 3rd Qu.    Max. 
#>   0.000   0.000   0.000   1.315   0.200  33.200

Brazil spans four time zones, and some states observed daylight saving time in some years and others not. By default (tz = "auto"), inmet_daily() uses each station’s own zone, derived from its state and longitude by inmet_timezone(), and reports it in the time_zone column:

inmet_timezone(c("RS", "MT", "AC", "AM"), c(-51.2, -56.1, -67.8, -69.9))
#> [1] "America/Sao_Paulo"  "America/Cuiaba"     "America/Rio_Branco"
#> [4] "America/Eirunepe"

Pass a fixed zone, such as tz = "America/Sao_Paulo", to put all stations on the same clock, or tz = "UTC" for UTC days.

3. Missing hours

INMET files contain rows for every hour of the year, including hours when a station was not reporting. By default inmet_read() drops rows in which every measurement is missing (drop_empty = FALSE keeps them). Individual variables can still be missing in the rows that remain.

A daily total over a day with half its hours missing is not a daily total. inmet_daily() therefore computes a variable only when it has at least min_hours valid hourly values that day (20 by default), and reports the number of hourly records in n_hours:

daily <- inmet_daily(x)
table(cut(daily$n_hours, c(0, 12, 19, 23, 25), right = TRUE))
#> 
#>  (0,12] (12,19] (19,23] (23,25] 
#>       2       2      46     316
daily[daily$n_hours < 20, c("date", "n_hours", "temp_max_c", "precip_mm")]
#>           date n_hours temp_max_c precip_mm
#> 1   2022-12-31       4         NA        NA
#> 124 2023-05-03      17         NA        NA
#> 148 2023-05-27       1         NA        NA
#> 149 2023-05-28      13         NA        NA

To see completeness before computing anything, inmet_completeness() gives the percentage of valid hours per station and month (or year):

head(inmet_completeness(x, variables = c("temp_dry_c", "precip_mm", "wind_speed_ms")))
#>   station_code  period hours_expected hours_present temp_dry_c precip_mm
#> 1         A801 2023-01            744           734       98.7      98.7
#> 2         A801 2023-02            672           666       99.1      99.1
#> 3         A801 2023-03            744           734       98.7      98.7
#> 4         A801 2023-04            720           720      100.0     100.0
#> 5         A801 2023-05            744           702       94.4      94.4
#> 6         A801 2023-06            720           720      100.0     100.0
#>   wind_speed_ms
#> 1          98.5
#> 2          99.1
#> 3          98.7
#> 4         100.0
#> 5          93.7
#> 6          99.7

Choose min_hours to suit the variable and the analysis: precipitation totals are sensitive to every missing hour, while a daily maximum of temperature is usually reached in the afternoon. Radiation is an exception: it is missing at night at most stations, so radiation_mj_m2 sums whatever daytime values are present (see ?inmet_daily).

4. Which temperature columns?

The hourly files contain both the air temperature at the top of the hour (temp_dry_c) and its extremes during the previous hour (temp_max_prev_c, temp_min_prev_c). The daily maximum and minimum should come from the extremes; the instantaneous values miss the peaks between readings:

from_extremes <- inmet_daily(x)$temp_max_c
from_hourly <- inmet_daily(x[, setdiff(names(x), "temp_max_prev_c")])$temp_max_c
summary(from_extremes - from_hourly)
#>    Min. 1st Qu.  Median    Mean 3rd Qu.    Max.    NA's 
#>  0.0000  0.3000  0.5000  0.5707  0.8000  2.3000       4

At Porto Alegre in 2023 the hourly readings underestimate the daily maximum by about half a degree on average, and by more than two degrees on some days.

inmet_daily() uses the extremes when they are present and falls back to temp_dry_c otherwise.

5. Quality control

INMET labels automatic-station observations as raw data. inmet_qc() flags the most common problems: values outside physical limits, internally inconsistent records (such as an hourly maximum below the minimum), isolated spikes, and sensors stuck on the same value. It adds a <variable>_flag column naming the failed checks, and inmet_daily() and inmet_completeness() then ignore the flagged values:

checked <- inmet_qc(x)
#> Flagged 0 value(s) in 0 variable(s). See attr(x, "qc_summary").

Station A801 passed every check in 2023. To see what a flag looks like, plant a sensor spike of 15 degC in one hour:

y <- x
y$temp_dry_c[1000] <- y$temp_dry_c[1000] + 15
flagged <- inmet_qc(y, quiet = TRUE)
flagged[998:1002, c("datetime", "temp_dry_c", "temp_max_prev_c", "temp_dry_c_flag")]
#>                 datetime temp_dry_c temp_max_prev_c   temp_dry_c_flag
#> 998  2023-02-12 02:00:00       24.2            24.6              <NA>
#> 999  2023-02-12 03:00:00       23.6            24.3              <NA>
#> 1000 2023-02-12 04:00:00       38.4            23.8 consistency,spike
#> 1001 2023-02-12 05:00:00       22.7            23.5              <NA>
#> 1002 2023-02-12 06:00:00       21.9            22.6              <NA>

The reading is both a spike and inconsistent with the hour’s maximum. inmet_daily(flagged) leaves it out of the daily statistics, and attr(flagged, "qc_summary") counts the flags per variable and check. Across all 545 stations in the 2024 archive the same checks flag, for example, a stuck wind vane that reported 236 degrees for months and temperature readings that drop by 9 degC for one hour and return. Use action = "remove" to set flagged values to NA directly.

These checks catch gross errors only. Comparing with neighbouring stations (inmet_nearest()) remains the most effective way to find subtler problems.