---
title: "Add Data and Data Dictionary"
engine: knitr
---
## Add Data File
```{r}
#| include: false
invisible(loadNamespace("palmerpenguins")) # Tell renv that we need this package
stopifnot(packageDescription("palmerpenguins", fields = "License") == "CC0")
dat <- palmerpenguins::penguins
# Convert factors to integers
idx <- sapply(dat, is.factor)
dat[idx] <- lapply(dat[idx], as.integer)
dat[is.na(dat)] <- -99L
# Add extra column for training purposes
set.seed(1056517553L)
dat$cuteness <- sample.int(10, nrow(dat), replace = TRUE)
write.csv(
dat,
file = "data.csv",
row.names = FALSE
)
rm(idx, dat)
```
You can now download the data set we have prepared for you
and put it into your project folder:
[data.csv](data.csv){.btn .btn-lg .btn-info download="data.csv"}
::: {.column-margin}
{width=250px}
:::
The data set is from the package [`palmerpenguins`][palmerpenguins]
(v`r packageVersion("palmerpenguins")`)
and contains the recorded bill lengths and sex of penguins
living on three islands in the Palmer Archipelago, Antarctica.
It was made available by Allison Horst, Alison Hill, and Kristen Gorman
under the license [CC0\ 1.0][cc0].
For educational purposes, we added the fictitious column `cuteness`.
[palmerpenguins]: https://allisonhorst.github.io/palmerpenguins/
[cc0]: https://creativecommons.org/publicdomain/zero/1.0/
::: {#imp-legal-restrictions .callout-important}
### Consider Legal Restrictions Before Sharing
Everything you put into the project folder will be shared publicly.
For reasons of reproducibility, this should include the data you analyze.
Of course, you should only share them to the extent you are allowed to.
Besides copyright and similar rights, you need to take into account
applicable privacy laws (e.g., the GDPR for European citizens)
and contractual obligations (e.g., with your data provider).
Privacy laws and contractual obligations may require you
to create an anonymized or synthetic data set[^anon-synth-tools],
in which case you should provide a reference
to a repository where the originally measured data can be obtained from.
For further information, you can watch the talk "[Data anonymity](https://osf.io/j6fy8)"
by Felix Schönbrodt recorded during the LMU Open Science Center Summer School 2023
and have a look at [the accompanying slides](https://osf.io/z6gcu).
Another resource to look at is the presentation
"Creating reproducible packages when data are confidential" by @Vilhuber2024.
:::
[^anon-synth-tools]: For example, using [Amnesia](https://amnesia.openaire.eu/),
[ARX](https://arx.deidentifier.org/), [sdcTools](https://github.com/sdcTools),
[Synthpop](https://www.synthpop.org.uk/), [OpenDP](https://opendp.org/),
or [Greenmask](https://www.greenmask.io/).
Let's take a first look at our data:
```{r}
#| class-output: "txt code-overflow-scroll"
#| attr-output: 'filename="R Console"'
dat <- read.csv("data.csv")
str(dat)
```
We can see that the variables `species`, `island`, and `sex` are stored as integers:
- species
- 1 = Adelie
- 2 = Chinstrap
- 3 = Gentoo
- island
- 1 = Biscoe
- 2 = Dream
- 3 = Torgersen
- sex
- 1 = female
- 2 = male
Further, missing values are stored as `-99`. We need to document this information somewhere!
## Add a Data Dictionary
Whether or not distributing the data set,
it is important to document the meaning (e.g., units)
and values of its variables.
This is typically done with a _data dictionary_ (also called a _codebook_).
The recommendation for data dictionaries vary between fields --
both in terms of the recommended content (i.e., what exactly should be documented)
and the technical implementation (i.e., which file formats should be used).
For the purpose of this exercise, we keep it easy and propose that you
manually create a file with the data dictionary
(e.g., as a table in .xlsx, .docx, .ods, or as a Markdown table),
documenting only the bare minimum.
<!-- TODO: Add a link to the forthcoming data dictionary OSC tutorial -->
::: {#tip-in-depth .callout-tip collapse="false"}
### Creating data dictionaries using R
Our in-depth supplementary material "[Generation of Data Dictionaries Using R](in-depth-material/data-dic-generation.qmd)"
explains how you can create a data dictionary with R.
This advanced chapter also contains a section on how to create machine-readable data dictionaries.
:::
### A bare minimum data dictionary
Most standards for data dictionaries require at least
this information for each variable in your data set:
- `name`: Column name of the variable in the data set
- `label`: Short title for the variable
- `description`: Brief description of what the variable measures or represents
- `item_text`: Exact wording of a question in a survey
- `values`: A mapping of stored answers to their meanings (especially for categorical variables). For example: 1 = male, 2 = female, 9 = missing
- `units`: The units of measurement (only for numeric variables). For example: `kg`, `USD`, `years`
Here's an example from a different data set, with one row for each variable:
| name | label | description | item_text | values | units |
| ----------------- | ------------------------- | ----------------------- | ------------------------------------- | ----------------------------------------------------------------------------------------------------------- | ----- |
| gender | Gender | self‑identified gender | | 1 = male, 2 = female, 3 = other, 9 = missing | |
| age | Age | age in years | | -99 = Not answered | years |
| blood_pressure | Blood Pressure (systolic) | systolic blood pressure | | -99 = Not answered | mmHg |
| life_satisfaction | Life Satisfaction | 5-point scale | How satisfied are you with your life? | 1 = very dissatisfied, 2 = dissatisfied, 3 = neutral, 4 = satisfied, 5 = very satisfied, -99 = not answered | |
You can download this example data dictionary as an ODS file [here](data-dictionary-solution.ods){download="data-dictionary-solution.ods"}.
Depending on the type of data, it may also be necessary
to describe sampling procedures (e.g., selection criteria),
measurement instruments (e.g., established questionnaires),
appropriate weighting,
already applied preprocessing steps, or contact information.
### Practical Exercise: Add your own data dictionary
Now it's your turn. Fill in the missing information (denoted by a bold question mark "**?**") in the following data dictionary.
<details>
<summary>Show data dictionary</summary>
| name | label | description | item_text | values | units |
| ----------------- | -------------- | ---------------------------------------------------------------------- | -------------------------------------- | ---------------------------------------------------- | ----- |
| species | Species | Penguin species | | 1 = Adelie, 2 = Chinstrap, 3 = Gentoo, -99 = missing | |
| island | Island | Island in Palmer Archipelago, Antarctica where individual was observed | | 1 = Biscoe, 2 = Dream, 3 = Torgersen, -99 = missing | |
| bill_length_mm | Bill length | Length of the bill (beak) | | -99 = missing | mm |
| bill_depth_mm | Bill depth | **?** | | -99 = missing | mm |
| flipper_length_mm | Flipper length | Length of the flipper | | -99 = missing | mm |
| body_mass_g | **?** | Body mass | | -99 = missing | **?** |
| sex | Sex | Sex of the penguin | | 1 = female, 2 = male, -99 = missing | |
| **?** | Year | Study year | | | |
| cuteness | Cuteness | Subjective cuteness rated on 10-point scale | How cute do you think this penguin is? | 1 = not at all, **?** = extremely | |
</details>
[data-dictionary.ods](data-dictionary.ods){.btn .btn-lg .btn-info download="data-dictionary.ods"}
Then save the data dictionary in the same folder as the actual data set file.
::: {#tip-data_dic .callout-tip collapse="true"}
#### Data dictionary for penguins data set (Solution)
| name | label | description | item_text | values | units |
| ----------------- | -------------- | ---------------------------------------------------------------------- | -------------------------------------- | ---------------------------------------------------- | ----- |
| species | Species | Penguin species | | 1 = Adelie, 2 = Chinstrap, 3 = Gentoo, -99 = missing | |
| island | Island | Island in Palmer Archipelago, Antarctica where individual was observed | | 1 = Biscoe, 2 = Dream, 3 = Torgersen, -99 = missing | |
| bill_length_mm | Bill length | Length of the bill (beak) | | -99 = missing | mm |
| bill_depth_mm | Bill depth | Depth of the bill (beak) | | -99 = missing | mm |
| flipper_length_mm | Flipper length | Length of the flipper | | -99 = missing | mm |
| body_mass_g | Mass | Body mass | | -99 = missing | g |
| sex | Sex | Sex of the penguin | | 1 = female, 2 = male, -99 = missing | |
| year | Year | Study year | | | |
| cuteness | Cuteness | Subjective cuteness rated on 10-point scale | How cute do you think this penguin is? | 1 = not at all, 10 = extremely | |
You can download the solution [here](data-dictionary-solution.ods){.btn .btn-lg .btn-info download="data-dictionary-solution.ods"}.
:::
## Practical Exercise: Add Data Citation and Attribution
All data relied upon should be cited in the manuscript
to allow for precise identification and access.
Please add an appropriate citation for the data set to the manuscript
where it says "cite data here".
::: {#tip-cite-palmerpenguins-hint .callout-tip collapse="true"}
#### Citing the Data Set (Hint)
You can find an appropriate BibTeX entry on [the `palmerpenguins` package website](https://allisonhorst.github.io/palmerpenguins/#citation).
:::
::: {#tip-cite-palmerpenguins .callout-tip collapse="true"}
#### Citing the Data Set (Solution)
Copy the following BibTeX entry to the file `Bibliography.bib`:
```{r}
#| echo: false
#| class.output: "bib code-overflow-scroll"
#| attr-output: 'filename="Bibliography.bib"'
citation("palmerpenguins", auto = TRUE) |>
transform(key = "horst2020") |>
toBibtex()
```
Then, find the line in the manuscript that says "cite data here"
and replace it with a sentence such as the following:
```{.md filename="Manuscript.qmd"}
The analyzed data are by @horst2020.
```
Render the document to check that the citation is displayed properly.
```{.bash filename="Terminal"}
quarto render Manuscript.qmd
```
:::
While citation happens in the manuscript for reasons of academic integrity and reproducibility,
to comply with any licenses you also may need to provide attribution within your project folder.
Even though the data file we use here does not require attribution,
we recommend adding a short paragraph to `LICENSE.txt`:
```{r}
#| echo: false
#| class-output: "txt code-overflow-wrap"
#| attr-output: 'filename="LICENSE.txt"'
cat(paste0(
"The penguins data stored in \"data.csv\" by Allison Horst, Alison Hill, and Kristen Gorman available from <",
citation("palmerpenguins", auto = TRUE)$url,
"> are licensed under CC0 1.0: <https://creativecommons.org/publicdomain/zero/1.0/>"
))
```
## Wrap up
Congrats! You documented your data set and cited it correctly.
To finalize this step, you can go through the commit routine:
```{.bash filename="Terminal"}
git status
git add .
git commit -m "Add data"
```