Purpose
You can use computer simulations to:
- Test your statistical intuition or demonstrate mathematical properties you cannot easily anticipate.
- Example: Check whether there are more than 5% significant effects for a variable in a model when supposedly random data are generated.
- Example: Check whether there are more than 5% significant effects for a variable in a model when supposedly random data are generated.
- Understand sampling theory and probability distributions or test whether you understand the underlying processes of your system.
- Example: See whether simulated data drawn from specific distributions is comparable to real data.
- Example: See whether simulated data drawn from specific distributions is comparable to real data.
- Perform power analyses.
- Example: Assess whether the sample size (within a simulation repetition) is high enough to detect a simulated effect in more than 80% of the cases.
- Example: Assess whether the sample size (within a simulation repetition) is high enough to detect a simulated effect in more than 80% of the cases.
- Perform bootstrapping to get a confidence interval around a parameter estimate.
- Bootstrapping means to sample with replacement (i.e. all the original options to draw from are available at each draw) in an observed dataset. Doing this generates new ‘simulated’ datasets. With each of them, one can run the statistical analysis made on the observed dataset, saving each time the parameter estimate of interest. After doing this multiple times, one can obtain a confidence interval for the parameter of interest.
- Bootstrapping means to sample with replacement (i.e. all the original options to draw from are available at each draw) in an observed dataset. Doing this generates new ‘simulated’ datasets. With each of them, one can run the statistical analysis made on the observed dataset, saving each time the parameter estimate of interest. After doing this multiple times, one can obtain a confidence interval for the parameter of interest.
- Prepare a pre-analysis plan.
- To be confident about the (confirmatory) statistical analyses you may wish to commit to before data collection (e.g. through a preregistration or registered report), practising the analyses on a simulated dataset is very helpful! If you are still unsure about the most appropriate statistical test to apply to your data, providing a simulated dataset to a statistician or mentor will allow them to provide concrete suggestions! The code containing the analyses of simulated data can be submitted along with your preregistration or registered report for reviewers to exactly understand what analyses you intend to perform. Once you get your real data, you may simply plug them into this code and get the results of your confirmatory analyses immediately!