powerAD.RmdIn this vignette, we present a short introduction on how to conduct power analyses for affect time series. Based on the empirical power estimations and results described in Pirla et al. (2021), powerAD provides a set of user-ready tools for those interested in running studies in the field of affect dynamics.
Vignette Sections:
powerAD
powerAD
powerAD is maintained as a Github repository. You will need to use the function install_github() from the devtools package to download the package.
To install and load powerAD, introduce the following commands in your console:
install.packages("devtools")
devtools::install_github("sergiopirla/powerAD")
library(powerAD)The main functions of powerAD are designed to guide practitioners and researchers in conducting well-powered studies of affect dynamics. Our functions estimate power or sample size to detect a Pearson correlation of a given size r, using a two-tailed t-test and a given alpha level.
To identify a set of sampling approaches that yield a minimum statistical power, we use samplingAD(). This function takes the following inputs:
Given these inputs, samplingAD() provides the user with a set of sampling approaches that yield sufficient statistical power.
For instance, let’s assume that a user is interested in running an experience sampling study. Let’s further assume that the user is interested in detecting a Pearson correlation of expected size r=0.3 between the standard deviation in affect and an exogenous variable. To define statistical significance, the user will use an alpha level of 0.05. The user wants to have a priory probability of 80% to find the correlation of interest.
library(powerAD)
samplingAD(power=0.8, metric = "SD", r=0.3, p.value = 0.05)##
## Number of individuals and affect reports per individual (samples) needed to achieve a power of 80 % or more to detect a Pearson correlation of size r = 0.3 between the Standard Deviation (SD) in affect. and a given variable using a two-tailed t-test and an alpha of 0.05 .
##
## Power Individuals Samples
## 0.81 230 5
## 0.82 150 10
## 0.81 130 15
## 0.81 120 20
## 0.81 110 30
## 0.81 100 50
##
## -----------------------
## Power is estimated through a linear interpolation using the sample combinations included in our main analyses.We refrain from making power extrapolations and therefore, only consider sampling approaches that range between 10 and 5120 participants and from 5 to 50 affect reports per participant (samples). The following table presents the minimal sampling combinations included in our main analyses that yielded the specified power:
##
## Power Individuals Samples
## 0.94 320 5
## 0.86 160 10
##
## -----------------------
## How to report:
##
## Power analysis for affect dynamic studies (Pirla et al., 2021) suggests that our sampling strategy achieved a statistical power of 80 % to detect a Pearson correlation of size r = 0.3 using a two-tailed t-test with an alpha of 0.05 .
## -----------------------
## Reference:
##
## Pirla, S., Taquet, M., & Quoidbach, J. (2021). Measuring Affect Dynamics: An Empirical Framework. https://doi.org/10.31219/osf.io/x2ywaThe output presents 1) the sampling approaches needed to achieve sufficient power given the study characteristics, 2) the points in our empirical strategy used to estimate these sampling approaches, 3) a section on how to report the sample size calculations, and 4) a reference to Pirla et al. (2021).
Conversely, a user might be interested in calculating statistical power for a given sampling approach. The function powerAD() is designed to do so. powerAD() takes the following arguments:
Let’s assume that a user is interested in calculating statistical power for a study analyzing a Pearson’s correlation of expected size r=0.2 between the relative standard deviation in affect and an exogenous variable (using a two-tailed t-test). Let’s further assume that the sampling and subject pool is fixed at 200 individuals and 22 affect reports per individual. The user can estimate power using the following command:
powerAD(individuals=200, samples = 22, metric = "Rel.SD", r=0.2, p.value = 0.05)##
## Power to detect a Pearson correlation of size r= 0.2 between the Relative Standard Deviation (Rel. SD) in affect and a given variable using a two-tailed t-test and an alpha of 0.05 .
##
## Power is estimated through a linear interpolation using the closest combinations of number of subjects and number of observations per subject included in our main analyses. The following table presents the sampling approaches used in the interpolation:
##
## Power Individuals Samples
## 0.61 160 20
## 0.89 320 20
## 0.65 160 25
## 0.91 320 25
##
## -----------------------
## Linear aproximation of Power under the sampling approach specified ( 200 individuals and 22 observations per individual):
##
## Aprox. Power= 69.23 %
## -----------------------
## How to report:
##
## Power analysis for affect dynamic studies (Pirla et al., 2021) suggests that our sampling strategy achieved a statistical power of 69.23 % to detect a Pearson correlation of size r = 0.2 using a two-tailed t-test with an alpha of 0.05 .
## -----------------------
## Reference:
##
## Pirla, S., Taquet, M., & Quoidbach, J. (2021). Measuring Affect Dynamics: An Empirical Framework. https://doi.org/10.31219/osf.io/x2ywaIn this setting, statistical power approximates 70%.
So far, we have been concerned with power and sampling approaches, taking the other inputs as given. While most researchers will rely on previous literature to define an expected effect size, we provide a companion table with effect sizes to be used as reference. To access this data, type effect.sizes in your R console. If you want to see the benchmark effect sizes for a specific affect dynamic measure (e.g. TKEO), you can enter the following command:
effect.sizes[, c("Outcome", "TKEO")]## Outcome TKEO
## 1 Age -0.24695305
## 2 Gender -0.08099905
## 3 Average Meaning -0.19435538
## 4 SD Meaning 0.45159116
## 5 Rel.SD Meaning 0.37263128
## 6 RMSSD Meaning 0.44826577
## 7 TKEO Meaning 0.28387417
## 8 PAC Meaning 0.38268047
## 9 AUTO Meaning -0.05753214
## 10 Average Satisfaction -0.14425015
## 11 SD Satisfaction 0.42777312
## 12 Rel.SD Satisfaction 0.32854741
## 13 RMSSD Satisfaction 0.43058491
## 14 TKEO Satisfaction 0.24140383
## 15 PAC Satisfaction 0.36371948
## 16 AUTO Satisfaction -0.06120916
## 17 Sleep 0.02261081
## 18 Alone 0.01545217
## 19 Family 0.00433623
## 20 Friends 0.09124939
## 21 Work -0.03097502
## 22 Sports -0.01705431This table presents the effect size (Pearson’s r) between the affect TKEO and demographic variables (rows 1 and 2), static and dynamic measures of meaning in life (3-9), static and dynamic measures of life satisfaction (10-16) and proportion of time spent with different groups of people or engaging in some activities (17-22).
The main functions in powerAD rely on the methods and data presented in Pirla et al. (2021). More specifically, the main functions in this package assume that momentary affect is measured as a continuous scale (from 0 to 100). Yet, powerAD includes a function to estimate power when using a different affect elicitation scale - powerADgen().
This function uses pre-exiting affective data to conduct power simulations. To run these simulations, you will need a dataset containing a relatively large number of participants and affect reports per participant. Given this data and the study characteristics, powerADgen() calculates statistical power using a simulation and resampling technique.
powerADgen() takes the following arguments:
Before proceeding with an example, a number of comments regarding these arguments is in order. First, the data used in the simulation must have a variable identifying the participant, the affect report, and the relative time of such affect report (whether that is the first, second, third… affect report collected from that specific individual). Second, power can only be estimated for sampling combinations strictly contained in the dataset used for power simulations. For example, if our dataset includes 25 affect reports from 200 individuals, we cannot estimate power when sampling 300 individuals or 30 affect reports. Third although the “perm” argument defaults to 100 simulations, we recommend setting this parameter to a minimum of 1000 simulations.
Let data.ESM be a dataset containing 50 affect reports from 200 different individuals.This data is not contained in our package, but represents a dataset available to the researcher from previous work. The affect reports in data.ESM are measured on a 1 to 4 scale. Let’s assume that we are interested in calculating the statistical power to detect a correlation of size r=0.1 between the Probability of Acute Change (PAC) in affect and a given variable. Let’s further assume that we plan on recruiting 100 individuals and sample them 25 times.
First, check the structure of data.ESM:
head(data.ESM)## Participant Affect.Report Report.Number
## 1 1 2 1
## 2 1 2 2
## 3 1 1 3
## 4 1 3 4
## 5 1 3 5
## 6 1 3 6Given the names of the variables in data.ESM, we use the following code to compute power:
powerADgen(individuals = 100, samples = 25, metric = "PAC", r=0.1, p.value = 0.05,
min = 1, max = 4, data = data.ESM, id="Participant",
affect="Affect.Report", time = "Report.Number", perm = 1000)##
## Power to detect a Pearson correlation of size r= 0.1 between the Probability of Acute Change (PAC) in affect and a given variable using a two-tailed t-test and an alpha of 0.05 .
##
## Power is estimated using the resampling and simulation approaches detailed in Pirla et al. (2021):
##
## -----------------------
## Empirical power simulation under the sampling approach specified ( 100 individuals and 25 observations per individual):
##
## Aprox. Power= 7.7 %
## -----------------------
## How to report:
##
## Power analysis for affect dynamic studies (Pirla et al., 2021) suggests that our sampling strategy achieved a statistical power of 7.7 % to detect a Pearson correlation of size r = 0.1 using a two-tailed t-test with an alpha of 0.05 .
## -----------------------
## Reference:
##
## Pirla, S., Taquet, M., & Quoidbach, J. (2021). Measuring Affect Dynamics: An Empirical Framework. https://doi.org/10.31219/osf.io/x2ywaIn this setting, the study would be seriously underpowered. Note that for a large number of simulations, number of individuals, and samples per individual, the simulations can become computationally costly - causing large computation times.