Purpose
You can use computer simulations to:
- Test your statistical intuition or demonstrate mathematical properties you cannot easily anticipate
- e.g. test whether when supposedly random data are generated, there is no more than 5% of significant effects for a variable in a model
- e.g. test whether when supposedly random data are generated, there is no more than 5% of significant effects for a variable in a model
- Understand sampling theory, probability distributions or test whether you understand the underlying processes of your system
- e.g. see whether simulated data drawn from specific distribution are comparable to real data
- e.g. see whether simulated data drawn from specific distribution are comparable to real data
- Perform power analyses
- e.g. assess whether the sample size (within a replicate) is high enough to detect an effect simulated, in more than 80% of the cases
- e.g. assess whether the sample size (within a replicate) is high enough to detect an effect simulated, in more than 80% of the cases
- Perform bootstrapping to get a confidence interval around a parameter estimate
- i.e. bootstrapping means to sample with replacement (i.e. all the original options to draw from are available at each draw) in an observed dataset. Doing this generates new ‘simulated’ datasets. With each of them, one can run the statistical analysis made on the observed dataset, saving each time the parameter estimate of interest. After doing this multiple time, you will obtain a confidence interval for the parameter of interest
- i.e. bootstrapping means to sample with replacement (i.e. all the original options to draw from are available at each draw) in an observed dataset. Doing this generates new ‘simulated’ datasets. With each of them, one can run the statistical analysis made on the observed dataset, saving each time the parameter estimate of interest. After doing this multiple time, you will obtain a confidence interval for the parameter of interest
- Prepare a pre-analysis plan
- e.g. in order to be confident about the (confirmatory) statistical analyses you may wish to commit to before data collection, through a preregistration or registered report, practicing the analyses on a simulated dataset is very helpful! If you are still unsure about the most appropriate statistical test to apply to your data, providing a simulated dataset to a statistician or mentor will allow them to provide concrete suggestions! The simulation code containing the analyses of simulated data can be submitted along your preregistration or registered report for reviewers to exactly understand what analyses you intend on performing. Once you get your real data, you may simply plug them in this code and get the results of your confirmatory analyses immediatly!