sapfluxnetr package
offers a very flexible but powerful API based on the
tidyverse packages to aggregate and summarise the site/s
data in the form of the sfn_metrics function. All the
metrics family of functions (?metrics) make use of the
sfn_metrics function under the hood. If you want full
control to the statistics returned and aggregation periods, we recommend
you to use this API. This vignette will show you how.
daily_metricsmonthly_metricspredawn_metricsmidday_metricsnightly_metricsdaylight_metricsSee each function help for a detailed description and examples of use.
daily_metrics and related functions return a complete
set of metrics ready for use, but if you want different metrics you can
supply your own summarising functions using the .funs
argument.
The correct way of specifying the functions to use is described in the
summarise_all help (?dplyr::summarise_all).
The recommended way is a list of formulas with the function call:
# libraries
library(sapfluxnetr)
library(dplyr)
#>
#> Attaching package: 'dplyr'
#> The following objects are masked from 'package:stats':
#>
#> filter, lag
#> The following objects are masked from 'package:base':
#>
#> intersect, setdiff, setequal, union
### only mean and sd at a daily scale
# data
data('ARG_TRE', package = 'sapfluxnetr')
# summarising funs (as a list of formulas)
custom_funs <- list(mean = ~ mean(., na.rm = TRUE), std_dev = ~ sd(., na.rm = TRUE))
# metrics
foo_simpler_metrics <- sfn_metrics(
ARG_TRE,
period = '1 day',
.funs = custom_funs,
solar = TRUE,
interval = 'general'
)
#> [1] "Crunching data for ARG_TRE. In large datasets this could take a while"
#> [1] "General data for ARG_TRE"
foo_simpler_metrics[['sapf']]
#> # A tibble: 14 × 9
#> TIMESTAMP ARG_TRE_Nan_Jt_1_mean ARG_TRE_Nan_Jt_2_mean
#> <dttm> <dbl> <dbl>
#> 1 2009-11-17 00:00:00 308. 173.
#> 2 2009-11-18 00:00:00 507. 376.
#> 3 2009-11-19 00:00:00 541. 380.
#> 4 2009-11-20 00:00:00 330. 218.
#> 5 2009-11-21 00:00:00 338. 219.
#> 6 2009-11-22 00:00:00 384. 243.
#> 7 2009-11-23 00:00:00 492. 300.
#> 8 2009-11-24 00:00:00 573. 389.
#> 9 2009-11-25 00:00:00 601. 400.
#> 10 2009-11-26 00:00:00 502. 360.
#> 11 2009-11-27 00:00:00 544. 411.
#> 12 2009-11-28 00:00:00 573. 451.
#> 13 2009-11-29 00:00:00 371. 285.
#> 14 2009-11-30 00:00:00 386. 293.
#> # ℹ 6 more variables: ARG_TRE_Nan_Jt_3_mean <dbl>, ARG_TRE_Nan_Jt_4_mean <dbl>,
#> # ARG_TRE_Nan_Jt_1_std_dev <dbl>, ARG_TRE_Nan_Jt_2_std_dev <dbl>,
#> # ARG_TRE_Nan_Jt_3_std_dev <dbl>, ARG_TRE_Nan_Jt_4_std_dev <dbl>When supplying only one function to .funs, names of variables are not changed to contain the metric name at the end, as the summary function returns the same columns as the original data
You can also choose if the “special interest” intervals (predawn, midday, nighttime or daylight) are calculated or not. For example, if you are only interested in the midday interval you can use:
foo_simpler_metrics_midday <- sfn_metrics(
ARG_TRE,
period = '1 day',
.funs = custom_funs,
solar = TRUE,
interval = 'midday', int_start = 11, int_end = 13
)
#> [1] "Crunching data for ARG_TRE. In large datasets this could take a while"
#> [1] "midday data for ARG_TRE"
foo_simpler_metrics_midday[['sapf']]
#> # A tibble: 13 × 9
#> TIMESTAMP_md ARG_TRE_Nan_Jt_1_mean_md ARG_TRE_Nan_Jt_2_mean_md
#> <dttm> <dbl> <dbl>
#> 1 2009-11-18 00:00:00 685. 665.
#> 2 2009-11-19 00:00:00 879. 594.
#> 3 2009-11-20 00:00:00 438. 272.
#> 4 2009-11-21 00:00:00 631. 379.
#> 5 2009-11-22 00:00:00 783. 535.
#> 6 2009-11-23 00:00:00 841. 478.
#> 7 2009-11-24 00:00:00 951. 636.
#> 8 2009-11-25 00:00:00 907. 602.
#> 9 2009-11-26 00:00:00 861. 697.
#> 10 2009-11-27 00:00:00 806. 594.
#> 11 2009-11-28 00:00:00 837. 730.
#> 12 2009-11-29 00:00:00 638. 605.
#> 13 2009-11-30 00:00:00 548. 371.
#> # ℹ 6 more variables: ARG_TRE_Nan_Jt_3_mean_md <dbl>,
#> # ARG_TRE_Nan_Jt_4_mean_md <dbl>, ARG_TRE_Nan_Jt_1_std_dev_md <dbl>,
#> # ARG_TRE_Nan_Jt_2_std_dev_md <dbl>, ARG_TRE_Nan_Jt_3_std_dev_md <dbl>,
#> # ARG_TRE_Nan_Jt_4_std_dev_md <dbl>period argument in sfn_metrics is passed to
.collapse_timestamp function, and so, it can use the same
input:
# weekly
foo_weekly <- sfn_metrics(
ARG_TRE,
period = '7 days',
.funs = custom_funs,
solar = TRUE,
interval = 'general'
)
#> [1] "Crunching data for ARG_TRE. In large datasets this could take a while"
#> [1] "General data for ARG_TRE"
foo_weekly[['env']]
#> # A tibble: 3 × 19
#> TIMESTAMP ta_mean rh_mean vpd_mean sw_in_mean ws_mean precip_mean
#> <dttm> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
#> 1 2009-11-15 00:00:00 4.81 35.3 0.598 280. 15.5 0.00612
#> 2 2009-11-22 00:00:00 6.15 35.3 0.656 327. 24.5 0.192
#> 3 2009-11-29 00:00:00 2.55 40.9 0.453 261. 23.1 0.122
#> # ℹ 12 more variables: swc_shallow_mean <dbl>, ppfd_in_mean <dbl>,
#> # ext_rad_mean <dbl>, ta_std_dev <dbl>, rh_std_dev <dbl>, vpd_std_dev <dbl>,
#> # sw_in_std_dev <dbl>, ws_std_dev <dbl>, precip_std_dev <dbl>,
#> # swc_shallow_std_dev <dbl>, ppfd_in_std_dev <dbl>, ext_rad_std_dev <dbl>...) argument of
sfn_metrics. Also, this function always must return a
vector of timestamps of the same length as the original timestamp.quarter function from the lubridate package:foo_custom <- sfn_metrics(
AUS_CAN_ST2_MIX,
period = lubridate::quarter,
.funs = custom_funs,
solar = TRUE,
interval = 'general',
with_year = TRUE # argument for lubridate::quarter
)
#> [1] "Crunching data for AUS_CAN_ST2_MIX. In large datasets this could take a while"
#> Warning: There was 1 warning in `dplyr::mutate()`.
#> ℹ In argument: `period_minutes = .period_to_minutes(period, .data$TIMESTAMP,
#> unique(.data$timestep))`.
#> Caused by warning in `.period_to_minutes()`:
#> ! when using a custom function as period, coverage calculation
#> can be less accurate
#> There was 1 warning in `dplyr::mutate()`.
#> ℹ In argument: `period_minutes = .period_to_minutes(period, .data$TIMESTAMP,
#> unique(.data$timestep))`.
#> Caused by warning in `.period_to_minutes()`:
#> ! when using a custom function as period, coverage calculation
#> can be less accurate
#> [1] "General data for AUS_CAN_ST2_MIX"
foo_custom['env']
#> $env
#> # A tibble: 5 × 17
#> TIMESTAMP ta_mean vpd_mean sw_in_mean ws_mean precip_mean ppfd_in_mean rh_mean
#> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl> <dbl>
#> 1 2006. 8.48 0.106 25.4 0.161 0.0158 53.7 93.1
#> 2 2006. 10.5 0.278 82.0 0.318 0.0399 173. 86.9
#> 3 2006. 16.6 0.826 219. 0.581 0.0200 463. 72.8
#> 4 2007. 20.9 0.985 197. 0.439 0.0333 416. 75.7
#> 5 2007. 15.8 0.386 110. 0.200 0.0181 231. 86.7
#> # ℹ 9 more variables: ext_rad_mean <dbl>, ta_std_dev <dbl>, vpd_std_dev <dbl>,
#> # sw_in_std_dev <dbl>, ws_std_dev <dbl>, precip_std_dev <dbl>,
#> # ppfd_in_std_dev <dbl>, rh_std_dev <dbl>, ext_rad_std_dev <dbl>sfn_metrics has a ... parameter intended to
supply additional parameters to the internal functions used:
.collapse_timestamp accepts the following extra
arguments:
sidedplyr::summarise_all accepts extra arguments
intended to be applied to the summarising functions provided (to
all, so they all must have the argument provided or an
error will be raised). That’s the reason because we recommend to use the
list way, as the arguments are specified for the individual
functions.
For example, if we want the TIMESTAMPs after aggregation to show the end of the period instead the beginning (default) we can do the following:
foo_simpler_metrics_end <- sfn_metrics(
ARG_TRE,
period = '1 day',
.funs = custom_funs,
solar = TRUE,
interval = 'general',
side = "end"
)
#> [1] "Crunching data for ARG_TRE. In large datasets this could take a while"
#> [1] "General data for ARG_TRE"
foo_simpler_metrics_end[['sapf']]
#> # A tibble: 14 × 9
#> TIMESTAMP ARG_TRE_Nan_Jt_1_mean ARG_TRE_Nan_Jt_2_mean
#> <dttm> <dbl> <dbl>
#> 1 2009-11-18 00:00:00 308. 173.
#> 2 2009-11-19 00:00:00 507. 376.
#> 3 2009-11-20 00:00:00 541. 380.
#> 4 2009-11-21 00:00:00 330. 218.
#> 5 2009-11-22 00:00:00 338. 219.
#> 6 2009-11-23 00:00:00 384. 243.
#> 7 2009-11-24 00:00:00 492. 300.
#> 8 2009-11-25 00:00:00 573. 389.
#> 9 2009-11-26 00:00:00 601. 400.
#> 10 2009-11-27 00:00:00 502. 360.
#> 11 2009-11-28 00:00:00 544. 411.
#> 12 2009-11-29 00:00:00 573. 451.
#> 13 2009-11-30 00:00:00 371. 285.
#> 14 2009-12-01 00:00:00 386. 293.
#> # ℹ 6 more variables: ARG_TRE_Nan_Jt_3_mean <dbl>, ARG_TRE_Nan_Jt_4_mean <dbl>,
#> # ARG_TRE_Nan_Jt_1_std_dev <dbl>, ARG_TRE_Nan_Jt_2_std_dev <dbl>,
#> # ARG_TRE_Nan_Jt_3_std_dev <dbl>, ARG_TRE_Nan_Jt_4_std_dev <dbl>If it is compared with the foo_simpler_metrics
calculated before, now the period is identified in the TIMESTAMP by the
ending of the period (daily in this case).
When supplying custom functions as “period” argument, the default coverage statistic is not reliable as there is no way of knowing beforehand the period/s in minutes.
The internal aggregation process in sfn_metrics
generates some transitory columns which can be used in the summarising
functions:
TIMESTAMP_collWhen aggregating by the declared period (i.e. "daily"),
the TIMESTAMP column collapses to the period start/end value (meaning
thet all the TIMESTAMP values for the same day becomes identical).
This makes impossible to use any summarise functions thet obtain the
time of the day at which one event happens (i.e. time of the day at
which the maximum sap flow occurs) because all TIMESTAMP values are
identical. For thet kind of summarising functions, a transitory column
called TIMESTAMP_coll is created. So in this case we can
create a function thet takes de variable values for the day, the
TIMESTAMP_coll values for the day and return the TIMESTAMP at which the
max sap flow occurs and use it with sfn_metrics:
max_time <- function(x, time) {
# x: vector of values for a day
# time: TIMESTAMP for the day
# if all the values in x are NAs (a daily summmarise of no measures day for
# example) this will return a length 0 POSIXct vector, which will crash
# dplyr summarise step. So, check if all NA and if true return NA as POSIXct
if(all(is.na(x))) {
return(as.POSIXct(NA, tz = attr(time, 'tz'), origin = lubridate::origin))
} else {
time[which.max(x)]
}
}
custom_funs <- list(max = ~ max(., na.rm = TRUE), ~ max_time(., TIMESTAMP_coll))
max_time_metrics <- sfn_metrics(
ARG_TRE,
period = '1 day',
.funs = custom_funs,
solar = TRUE,
interval = 'general'
)
#> [1] "Crunching data for ARG_TRE. In large datasets this could take a while"
#> [1] "General data for ARG_TRE"
max_time_metrics[['sapf']]
#> # A tibble: 14 × 9
#> TIMESTAMP ARG_TRE_Nan_Jt_1_max ARG_TRE_Nan_Jt_2_max
#> <dttm> <dbl> <dbl>
#> 1 2009-11-17 00:00:00 322. 190.
#> 2 2009-11-18 00:00:00 778. 715.
#> 3 2009-11-19 00:00:00 1015. 694.
#> 4 2009-11-20 00:00:00 648. 401.
#> 5 2009-11-21 00:00:00 664. 406.
#> 6 2009-11-22 00:00:00 812. 564.
#> 7 2009-11-23 00:00:00 1085. 676.
#> 8 2009-11-24 00:00:00 992. 736.
#> 9 2009-11-25 00:00:00 976. 646.
#> 10 2009-11-26 00:00:00 932. 766.
#> 11 2009-11-27 00:00:00 862. 704.
#> 12 2009-11-28 00:00:00 845. 763.
#> 13 2009-11-29 00:00:00 714. 747.
#> 14 2009-11-30 00:00:00 875. 646.
#> # ℹ 6 more variables: ARG_TRE_Nan_Jt_3_max <dbl>, ARG_TRE_Nan_Jt_4_max <dbl>,
#> # ARG_TRE_Nan_Jt_1_max_time <dttm>, ARG_TRE_Nan_Jt_2_max_time <dttm>,
#> # ARG_TRE_Nan_Jt_3_max_time <dttm>, ARG_TRE_Nan_Jt_4_max_time <dttm>sfn_metrics allows to perform sub-daily aggregations, by
means of the period parameter. Sapfluxnet datasets have
sub-daily data usually in the range of 30 minutes to 2 hours. This means
thet data can be aggregated in periods above 2 hours. We can aggregate
to a 3 hours period easily:
custom_funs <- list(max = ~ max(., na.rm = TRUE))
three_hours_agg <- sfn_metrics(
ARG_TRE,
period = '3 hours',
.funs = custom_funs,
solar = TRUE,
interval = 'general'
)
#> [1] "Crunching data for ARG_TRE. In large datasets this could take a while"
#> [1] "General data for ARG_TRE"
three_hours_agg[['sapf']]
#> # A tibble: 105 × 5
#> TIMESTAMP ARG_TRE_Nan_Jt_1_max ARG_TRE_Nan_Jt_2_max
#> <dttm> <dbl> <dbl>
#> 1 2009-11-17 21:00:00 322. 190.
#> 2 2009-11-18 00:00:00 301. 178.
#> 3 2009-11-18 03:00:00 343. 198.
#> 4 2009-11-18 06:00:00 504. 386.
#> 5 2009-11-18 09:00:00 698. 715.
#> 6 2009-11-18 12:00:00 778. 617.
#> 7 2009-11-18 15:00:00 724. 531.
#> 8 2009-11-18 18:00:00 660. 514.
#> 9 2009-11-18 21:00:00 384. 261.
#> 10 2009-11-19 00:00:00 403. 339.
#> # ℹ 95 more rows
#> # ℹ 2 more variables: ARG_TRE_Nan_Jt_3_max <dbl>, ARG_TRE_Nan_Jt_4_max <dbl>