SDTM with R

Every drug and diagnostic that gets submitted to the FDA (and most other regulators) arrives with its clinical trial data in a very specific shape. That shape is SDTM — the Study Data Tabulation Model, a CDISC standard. If you work in clinical data science, biostatistics, or statistical programming, SDTM is the lingua franca. This page is a from-scratch introduction: what it is, why it exists, and how to build SDTM-style datasets in R using a handful of small, reusable functions.

You don’t need any clinical background. If you’ve done a little tidyverse (see the Learn R course), you can follow everything here.

What is SDTM?

Imagine every pharma company invented its own column names, table layouts, and coding conventions for their trial data. A reviewer at the FDA would have to relearn the data from scratch for every single submission. SDTM exists to stop that.

SDTM standardizes the content and structure of the raw, collected trial data so that any reviewer, any validation tool, and any downstream program knows exactly where to look. It defines:

A fixed set of domains — one per type of data collected. Each has a two-letter code.
Standard variable names within each domain (again, mostly two-to-eight letter codes).
Rules for how rows, subjects, visits, and timing are represented.

💡 The mental model

SDTM is “one tidy table per kind of thing that happened in the trial.” Demographics is one table. Adverse events is another. Vital signs is another. Every table shares a common backbone (subject ID, study ID) so they can be joined back together.

The domains you’ll see most often

Code	Domain	One row per…
`DM`	Demographics	subject
`AE`	Adverse Events	adverse event a subject had
`VS`	Vital Signs	vital-sign measurement
`LB`	Laboratory	lab test result
`CM`	Concomitant Meds	medication a subject took
`EX`	Exposure	dose of study drug administered

There are dozens more, but if you understand these you understand the model.

The naming conventions

SDTM variable names follow a pattern. Once you see it, the columns stop looking like alphabet soup:

--SEQ — a sequence number, unique within a subject and domain (AESEQ, VSSEQ).
--TESTCD / --TEST — the short code and long name of what was measured (VSTESTCD = "SYSBP", VSTEST = "Systolic Blood Pressure").
--ORRES / --ORRESU — the original result as collected, and its unit (VSORRES = "120", VSORRESU = "mmHg").
--STRESC / --STRESN — the result standardized to a character and a numeric form.
--DTC — a date/time in ISO 8601 text (AESTDTC = "2023-04-12").
STUDYID / USUBJID — the study ID and the unique subject ID, the keys that tie every domain together.

The leading -- is a placeholder for the domain code. So the “test code” variable is VSTESTCD in Vital Signs and LBTESTCD in Labs.

Some raw data to start from

In real life the raw data comes out of an electronic data capture (EDC) system and looks nothing like SDTM. Our job is to reshape it. Let’s fabricate a tiny, messy “raw” export so we have something concrete to transform.

library(dplyr)
library(tidyr)

set.seed(42)

raw_demographics <- tibble::tribble(
  ~subject, ~site, ~birth_date,  ~sex,     ~race,    ~arm,
  "001",    "01",  "1968-03-11", "Male",   "White",  "Active",
  "002",    "01",  "1975-11-30", "Female", "Asian",  "Placebo",
  "003",    "02",  "1954-07-02", "Female", "White",  "Active",
  "004",    "02",  "1981-01-19", "Male",   "Black",  "Placebo",
  "005",    "03",  "1990-09-25", "Male",   "White",  "Active"
)

raw_demographics

#> # A tibble: 5 × 6
#>   subject site  birth_date sex    race  arm    
#>   <chr>   <chr> <chr>      <chr>  <chr> <chr>  
#> 1 001     01    1968-03-11 Male   White Active 
#> 2 002     01    1975-11-30 Female Asian Placebo
#> 3 003     02    1954-07-02 Female White Active 
#> 4 004     02    1981-01-19 Male   Black Placebo
#> 5 005     03    1990-09-25 Male   White Active

A raw vital-signs export, in “long” form, one measurement per row:

raw_vitals <- tibble::tribble(
  ~subject, ~visit,      ~measure, ~value, ~units, ~date,
  "001",    "Screening", "SYSBP",  128,    "mmHg", "2023-01-05",
  "001",    "Screening", "DIABP",  82,     "mmHg", "2023-01-05",
  "001",    "Week 4",    "SYSBP",  122,    "mmHg", "2023-02-02",
  "001",    "Week 4",    "DIABP",  79,     "mmHg", "2023-02-02",
  "002",    "Screening", "SYSBP",  141,    "mmHg", "2023-01-06",
  "002",    "Screening", "DIABP",  90,     "mmHg", "2023-01-06",
  "003",    "Screening", "SYSBP",  118,    "mmHg", "2023-01-09",
  "003",    "Week 4",    "SYSBP",  115,    "mmHg", "2023-02-07"
)

raw_vitals

#> # A tibble: 8 × 6
#>   subject visit     measure value units date      
#>   <chr>   <chr>     <chr>   <dbl> <chr> <chr>     
#> 1 001     Screening SYSBP     128 mmHg  2023-01-05
#> 2 001     Screening DIABP      82 mmHg  2023-01-05
#> 3 001     Week 4    SYSBP     122 mmHg  2023-02-02
#> 4 001     Week 4    DIABP      79 mmHg  2023-02-02
#> 5 002     Screening SYSBP     141 mmHg  2023-01-06
#> 6 002     Screening DIABP      90 mmHg  2023-01-06
#> 7 003     Screening SYSBP     118 mmHg  2023-01-09
#> 8 003     Week 4    SYSBP     115 mmHg  2023-02-07

And a raw adverse-event log:

raw_ae <- tibble::tribble(
  ~subject, ~event,        ~severity,  ~start,       ~end,
  "001",    "Headache",    "Mild",     "2023-01-20", "2023-01-22",
  "001",    "Nausea",      "Moderate", "2023-02-05", NA,
  "003",    "Fatigue",     "Mild",     "2023-01-15", "2023-01-30",
  "004",    "Dizziness",   "Severe",   "2023-01-25", "2023-01-26"
)

raw_ae

#> # A tibble: 4 × 5
#>   subject event     severity start      end       
#>   <chr>   <chr>     <chr>    <chr>      <chr>     
#> 1 001     Headache  Mild     2023-01-20 2023-01-22
#> 2 001     Nausea    Moderate 2023-02-05 <NA>      
#> 3 003     Fatigue   Mild     2023-01-15 2023-01-30
#> 4 004     Dizziness Severe   2023-01-25 2023-01-26

Building the DM (Demographics) domain

DM is the anchor domain: exactly one row per subject. Everything else joins back to it. Building it means renaming to SDTM variables, constructing the USUBJID key, and deriving AGE from the birth date.

Here’s a reusable function:

make_dm <- function(raw, studyid) {
  raw |>
    mutate(
      STUDYID = studyid,
      DOMAIN = "DM",
      SITEID = site,
      SUBJID = subject,
      USUBJID = paste(studyid, site, subject, sep = "-"),
      RFSTDTC = birth_date,                       # reference start (placeholder)
      BRTHDTC = birth_date,
      AGE = as.integer(
        floor(as.numeric(Sys.Date() - as.Date(birth_date)) / 365.25)
      ),
      AGEU = "YEARS",
      SEX = dplyr::recode(sex, "Male" = "M", "Female" = "F"),
      RACE = toupper(race),
      ARM = arm,
      ACTARM = arm
    ) |>
    select(
      STUDYID, DOMAIN, USUBJID, SUBJID, SITEID,
      BRTHDTC, AGE, AGEU, SEX, RACE, ARM, ACTARM
    )
}

dm <- make_dm(raw_demographics, studyid = "DEMO01")
dm

#> # A tibble: 5 × 12
#>   STUDYID DOMAIN USUBJID     SUBJID SITEID BRTHDTC   AGE AGEU  SEX   RACE  ARM  
#>   <chr>   <chr>  <chr>       <chr>  <chr>  <chr>   <int> <chr> <chr> <chr> <chr>
#> 1 DEMO01  DM     DEMO01-01-… 001    01     1968-0…    58 YEARS M     WHITE Acti…
#> 2 DEMO01  DM     DEMO01-01-… 002    01     1975-1…    50 YEARS F     ASIAN Plac…
#> 3 DEMO01  DM     DEMO01-02-… 003    02     1954-0…    72 YEARS F     WHITE Acti…
#> 4 DEMO01  DM     DEMO01-02-… 004    02     1981-0…    45 YEARS M     BLACK Plac…
#> 5 DEMO01  DM     DEMO01-03-… 005    03     1990-0…    35 YEARS M     WHITE Acti…
#> # ℹ 1 more variable: ACTARM <chr>

A few SDTM conventions on display here:

USUBJID must be unique across the whole study, so we build it from study + site + subject. Two sites could both have a subject “001”; the USUBJID keeps them distinct.
SEX is coded to controlled terminology (M/F), not free text.
AGE is derived, not collected. SDTM is full of derived variables like this.

⚠️ Controlled terminology

Real SDTM uses CDISC controlled terminology — fixed, published code lists for values like SEX, RACE, and units. You can’t put “Male” in SEX; it has to be M. Here we’re doing a simplified recode(), but in a real study you’d validate every coded column against the official code list.

Building the VS (Vital Signs) domain

Vital signs are a findings domain: one row per measurement, using the --TESTCD / --ORRES pattern. Our raw data is already long, which is the right shape — we mostly need to rename and add the standard result columns.

vs_test_labels <- c(
  SYSBP = "Systolic Blood Pressure",
  DIABP = "Diastolic Blood Pressure",
  PULSE = "Pulse Rate",
  TEMP = "Temperature",
  RESP = "Respiratory Rate"
)

make_vs <- function(raw, dm, studyid) {
  subject_key <- dm |>
    select(subject = SUBJID, USUBJID)
  raw |>
    left_join(subject_key, by = "subject") |>
    mutate(
      STUDYID = studyid,
      DOMAIN = "VS",
      VSTESTCD = measure,
      VSTEST = unname(vs_test_labels[measure]),
      VSORRES = as.character(value),
      VSORRESU = units,
      VSSTRESC = as.character(value),
      VSSTRESN = as.numeric(value),
      VSSTRESU = units,
      VISIT = visit,
      VSDTC = date
    ) |>
    arrange(USUBJID, VSDTC, VSTESTCD) |>
    group_by(USUBJID) |>
    mutate(VSSEQ = row_number()) |>
    ungroup() |>
    select(
      STUDYID, DOMAIN, USUBJID, VSSEQ, VSTESTCD, VSTEST,
      VSORRES, VSORRESU, VSSTRESC, VSSTRESN, VSSTRESU, VISIT, VSDTC
    )
}

vs <- make_vs(raw_vitals, dm, studyid = "DEMO01")
vs

#> # A tibble: 8 × 13
#>   STUDYID DOMAIN USUBJID       VSSEQ VSTESTCD VSTEST   VSORRES VSORRESU VSSTRESC
#>   <chr>   <chr>  <chr>         <int> <chr>    <chr>    <chr>   <chr>    <chr>   
#> 1 DEMO01  VS     DEMO01-01-001     1 DIABP    Diastol… 82      mmHg     82      
#> 2 DEMO01  VS     DEMO01-01-001     2 SYSBP    Systoli… 128     mmHg     128     
#> 3 DEMO01  VS     DEMO01-01-001     3 DIABP    Diastol… 79      mmHg     79      
#> 4 DEMO01  VS     DEMO01-01-001     4 SYSBP    Systoli… 122     mmHg     122     
#> 5 DEMO01  VS     DEMO01-01-002     1 DIABP    Diastol… 90      mmHg     90      
#> 6 DEMO01  VS     DEMO01-01-002     2 SYSBP    Systoli… 141     mmHg     141     
#> 7 DEMO01  VS     DEMO01-02-003     1 SYSBP    Systoli… 118     mmHg     118     
#> 8 DEMO01  VS     DEMO01-02-003     2 SYSBP    Systoli… 115     mmHg     115     
#> # ℹ 4 more variables: VSSTRESN <dbl>, VSSTRESU <chr>, VISIT <chr>, VSDTC <chr>

The key moves:

We joined to dm to attach the proper USUBJID — findings domains never re-derive the subject key, they inherit it.
VSSEQ is generated per subject with group_by() + row_number(), giving each row a unique sequence number within the subject.
We split the result into original (VSORRES, text) and standardized (VSSTRESN, numeric) forms. Downstream analysis uses the numeric one.

Building the AE (Adverse Events) domain

Adverse events are an events domain: one row per event, with start/end dates and a severity. AE also carries a couple of the most-scrutinized variables in all of clinical data — seriousness and severity.

make_ae <- function(raw, dm, studyid) {
  subject_key <- dm |>
    select(subject = SUBJID, USUBJID)
  raw |>
    left_join(subject_key, by = "subject") |>
    mutate(
      STUDYID = studyid,
      DOMAIN = "AE",
      AETERM = event,
      AESEV = toupper(severity),
      AESTDTC = start,
      AEENDTC = end,
      AEONGO = if_else(is.na(end), "Y", "N")   # ongoing at data cutoff?
    ) |>
    arrange(USUBJID, AESTDTC) |>
    group_by(USUBJID) |>
    mutate(AESEQ = row_number()) |>
    ungroup() |>
    select(
      STUDYID, DOMAIN, USUBJID, AESEQ,
      AETERM, AESEV, AESTDTC, AEENDTC, AEONGO
    )
}

ae <- make_ae(raw_ae, dm, studyid = "DEMO01")
ae

#> # A tibble: 4 × 9
#>   STUDYID DOMAIN USUBJID       AESEQ AETERM    AESEV    AESTDTC   AEENDTC AEONGO
#>   <chr>   <chr>  <chr>         <int> <chr>     <chr>    <chr>     <chr>   <chr> 
#> 1 DEMO01  AE     DEMO01-01-001     1 Headache  MILD     2023-01-… 2023-0… N     
#> 2 DEMO01  AE     DEMO01-01-001     2 Nausea    MODERATE 2023-02-… <NA>    Y     
#> 3 DEMO01  AE     DEMO01-02-003     1 Fatigue   MILD     2023-01-… 2023-0… N     
#> 4 DEMO01  AE     DEMO01-02-004     1 Dizziness SEVERE   2023-01-… 2023-0… N

Note how a missing end date becomes a derived “ongoing” flag rather than just an NA — deriving meaningful flags from raw gaps is a huge part of the real work.

Why the common structure pays off

Because every domain shares STUDYID and USUBJID, answering cross-domain questions is just a join. For example: “for each subject, how many adverse events did they have, and what arm were they in?”

dm_arm <- dm |>
  select(USUBJID, ARM)

dm_arm |>
  left_join(
    ae |> count(USUBJID, name = "n_ae"),
    by = "USUBJID"
  ) |>
  mutate(n_ae = tidyr::replace_na(n_ae, 0)) |>
  arrange(desc(n_ae))

#> # A tibble: 5 × 3
#>   USUBJID       ARM      n_ae
#>   <chr>         <chr>   <int>
#> 1 DEMO01-01-001 Active      2
#> 2 DEMO01-02-003 Active      1
#> 3 DEMO01-02-004 Placebo     1
#> 4 DEMO01-01-002 Placebo     0
#> 5 DEMO01-03-005 Active      0

Or a quick clinical summary — mean systolic blood pressure at screening by treatment arm:

vs |>
  filter(VSTESTCD == "SYSBP", VISIT == "Screening") |>
  left_join(dm_arm, by = "USUBJID") |>
  group_by(ARM) |>
  summarise(
    n = n(),
    mean_sysbp = mean(VSSTRESN),
    .groups = "drop"
  )

#> # A tibble: 2 × 3
#>   ARM         n mean_sysbp
#>   <chr>   <int>      <dbl>
#> 1 Active      2        123
#> 2 Placebo     1        141

This is the whole point of SDTM: once the data is in this shape, any analysis is a predictable combination of filters, joins, and summaries.

A tiny validation function

Regulators run automated conformance checks (Pinnacle 21 is the famous one) against submitted SDTM. You can write lightweight versions of those checks yourself. Here’s one that flags the most common structural problems:

check_domain <- function(df, domain, key_vars = c("STUDYID", "USUBJID")) {
  issues <- character(0)

  missing_keys <- setdiff(key_vars, names(df))
  if (length(missing_keys) > 0) {
    issues <- c(issues, paste("Missing key variable(s):",
                              paste(missing_keys, collapse = ", ")))
  }

  if ("DOMAIN" %in% names(df) && any(df$DOMAIN != domain)) {
    issues <- c(issues, "DOMAIN value does not match expected domain")
  }

  seq_var <- paste0(domain, "SEQ")
  if (seq_var %in% names(df)) {
    dup <- df |>
      count(USUBJID, .data[[seq_var]]) |>
      filter(n > 1)
    if (nrow(dup) > 0) {
      issues <- c(issues, paste0(seq_var, " is not unique within USUBJID"))
    }
  }

  if (length(issues) == 0) {
    cat("PASS:", domain, "looks structurally valid\n")
  } else {
    cat("ISSUES in", domain, ":\n")
    cat(paste0("  - ", issues, collapse = "\n"), "\n")
  }
  invisible(issues)
}

check_domain(dm, "DM")

#> PASS: DM looks structurally valid

check_domain(vs, "VS")

#> PASS: VS looks structurally valid

check_domain(ae, "AE")

#> PASS: AE looks structurally valid

Real conformance rules number in the hundreds, but the idea is the same: programmatic, repeatable checks that either pass or point you at the exact problem.

💡 The pharmaverse

Everything here is hand-rolled to show the mechanics, but you rarely start from zero in practice. The open-source pharmaverse — packages like admiral (for building the analysis-ready ADaM datasets that come after SDTM), sdtm.oak (for building SDTM itself), and pharmaverseadam / pharmaversesdtm (realistic example data) — gives you tested, regulator-aware tooling. This page is the “under the hood” version so those packages make sense when you reach for them.

The hard part: edge cases and persistent issues

The functions above work on clean, well-behaved data. Real trial data is never clean. This is the part nobody warns you about — the edge cases that quietly corrupt an analysis and the issues that resurface on every single study. Each one below comes with a demonstration of how it bites and a defensible fix.

1. Partial and impossible dates

This is the single most persistent issue in all of SDTM. Sites collect dates that are incomplete (“the patient started the med sometime in March 2023, they don’t remember the day”), and the ISO 8601 --DTC format is built to represent exactly that: 2023-03 (no day), 2023 (year only), or even --03-15 (no year). Naively parsing these with as.Date() silently turns them into NA, and every downstream calculation that touches them breaks.

messy_dates <- c(
  "2023-04-12",   # complete
  "2023-03",      # missing day
  "2023",         # year only
  "",             # entirely missing
  "2023-02-30"    # IMPOSSIBLE: February has no 30th
)

as.Date(messy_dates)   # naive parse: silent NAs, no signal about *why*

#> [1] "2023-04-12" NA           NA           NA           NA

Notice the trap: the missing-day date, the year-only date, and the impossible Feb 30th all collapse to NA indistinguishably. You can’t tell a legitimately-partial date from genuinely-bad data. The fix is to classify the completeness first, then impute deterministically (the convention is to impute to the first of the period for start dates and the last for end dates), and flag what you did:

classify_dtc <- function(dtc) {
  dplyr::case_when(
    is.na(dtc) | dtc == ""            ~ "missing",
    grepl("^\\d{4}-\\d{2}-\\d{2}$", dtc) ~ "complete",
    grepl("^\\d{4}-\\d{2}$", dtc)     ~ "no_day",
    grepl("^\\d{4}$", dtc)            ~ "year_only",
    TRUE                              ~ "other"
  )
}

impute_start_dtc <- function(dtc) {
  precision <- classify_dtc(dtc)
  imputed <- dplyr::case_when(
    precision == "complete"  ~ dtc,
    precision == "no_day"    ~ paste0(dtc, "-01"),
    precision == "year_only" ~ paste0(dtc, "-01-01"),
    TRUE                     ~ NA_character_
  )
  tibble::tibble(
    dtc_raw = dtc,
    precision = precision,
    dtc_imputed = imputed,
    # as.Date on a well-formed string still catches Feb 30th as NA:
    date = as.Date(imputed),
    imputed_flag = precision %in% c("no_day", "year_only")
  )
}

impute_start_dtc(messy_dates)

#> # A tibble: 5 × 5
#>   dtc_raw      precision dtc_imputed date       imputed_flag
#>   <chr>        <chr>     <chr>       <date>     <lgl>       
#> 1 "2023-04-12" complete  2023-04-12  2023-04-12 FALSE       
#> 2 "2023-03"    no_day    2023-03-01  2023-03-01 TRUE        
#> 3 "2023"       year_only 2023-01-01  2023-01-01 TRUE        
#> 4 ""           missing   <NA>        NA         FALSE       
#> 5 "2023-02-30" complete  2023-02-30  NA         FALSE

Now the year-only and missing-day dates are imputed and flagged (imputed_flag = TRUE), the truly-missing one is honestly NA, and Feb 30th is caught as bad data (date is NA even though precision said “complete”) — a signal you can act on rather than a silent hole.

⚠️ Imputation is a decision, not a default

Every imputed date must carry a flag (in ADaM these become --DTF / --TMF, the date/time imputation flags). If a statistician later computes “days on treatment” from an imputed start date, they need to know it was imputed. Silently guessing is how you end up with a finding in an FDA audit.

2. Duplicate records that aren’t obviously duplicates

Sites re-enter data. A subject’s screening visit gets keyed twice, or an AE gets logged by two coordinators. The --SEQ variable is supposed to be unique within subject — but if you generate it with row_number() after the duplicates are already in, you’ll happily assign 1 and 2 to two rows that are actually the same event, and the duplicate becomes invisible.

raw_vitals_dupes <- dplyr::bind_rows(
  raw_vitals,
  raw_vitals |> dplyr::slice(1)   # subject 001's screening SYSBP, re-entered
)

# Wrong: seq assigned over the dupes, so they look like distinct records
raw_vitals_dupes |>
  dplyr::filter(subject == "001", measure == "SYSBP", visit == "Screening")

#> # A tibble: 2 × 6
#>   subject visit     measure value units date      
#>   <chr>   <chr>     <chr>   <dbl> <chr> <chr>     
#> 1 001     Screening SYSBP     128 mmHg  2023-01-05
#> 2 001     Screening SYSBP     128 mmHg  2023-01-05

The fix is to define what makes a record unique (the “natural key” — here subject + visit + measure + date) and de-duplicate on it before assigning --SEQ:

natural_key_vs <- c("subject", "visit", "measure", "date")

dedup_report <- raw_vitals_dupes |>
  dplyr::add_count(dplyr::across(dplyr::all_of(natural_key_vs)),
                   name = "n_with_key") |>
  dplyr::filter(n_with_key > 1) |>
  dplyr::distinct(dplyr::across(dplyr::all_of(natural_key_vs)))

dedup_report   # exactly which records collide

#> # A tibble: 1 × 4
#>   subject visit     measure date      
#>   <chr>   <chr>     <chr>   <chr>     
#> 1 001     Screening SYSBP   2023-01-05

raw_vitals_clean <- raw_vitals_dupes |>
  dplyr::distinct(dplyr::across(dplyr::all_of(natural_key_vs)),
                  .keep_all = TRUE)

nrow(raw_vitals_dupes)

#> [1] 9

nrow(raw_vitals_clean)   # one fewer: the re-entered row is gone

#> [1] 8

Report the collisions (never delete silently — a data-management query goes back to the site), then de-duplicate on the natural key, and only then assign --SEQ.

3. Unit chaos in findings domains

Findings domains are where labs live, and labs are a unit nightmare. The same test arrives from different sites in different units: glucose in mg/dL from one lab and mmol/L from another. If you compare --ORRES numerically without standardizing, you’re comparing apples to a number that’s ~18x different. This is what --STRESN / --STRESU (the standardized result) exists to solve, and getting the conversion factors right is a perpetual source of bugs.

raw_labs <- tibble::tribble(
  ~subject, ~test,      ~result, ~unit,
  "001",    "GLUC",     99,      "mg/dL",
  "002",    "GLUC",     5.5,     "mmol/L",   # same magnitude, different unit
  "003",    "GLUC",     110,     "mg/dL"
)

# Naive: treat the numbers as comparable -> subject 002 looks impossibly low
raw_labs |>
  dplyr::group_by(test) |>
  dplyr::summarise(mean_result = mean(result), .groups = "drop")

#> # A tibble: 1 × 2
#>   test  mean_result
#>   <chr>       <dbl>
#> 1 GLUC         71.5

A mean of a mg/dL value and a mmol/L value is meaningless. Standardize to a single unit per test with an explicit, auditable conversion table:

# Conversion factors TO the standard unit for each test
lab_conversions <- tibble::tribble(
  ~test,  ~from_unit, ~std_unit, ~factor,
  "GLUC", "mg/dL",    "mg/dL",   1,
  "GLUC", "mmol/L",   "mg/dL",   18.0182   # 1 mmol/L glucose = 18.0182 mg/dL
)

lb <- raw_labs |>
  dplyr::left_join(lab_conversions, by = c("test", "unit" = "from_unit")) |>
  dplyr::mutate(
    LBORRES = as.character(result),
    LBORRESU = unit,
    LBSTRESN = result * factor,
    LBSTRESU = std_unit
  )

lb |>
  dplyr::select(subject, test, LBORRES, LBORRESU, LBSTRESN, LBSTRESU)

#> # A tibble: 3 × 6
#>   subject test  LBORRES LBORRESU LBSTRESN LBSTRESU
#>   <chr>   <chr> <chr>   <chr>       <dbl> <chr>   
#> 1 001     GLUC  99      mg/dL        99   mg/dL   
#> 2 002     GLUC  5.5     mmol/L       99.1 mg/dL   
#> 3 003     GLUC  110     mg/dL       110   mg/dL

# Now the summary is meaningful because everything is in mg/dL
lb |>
  dplyr::group_by(test) |>
  dplyr::summarise(mean_std = mean(LBSTRESN), .groups = "drop")

#> # A tibble: 1 × 2
#>   test  mean_std
#>   <chr>    <dbl>
#> 1 GLUC      103.

⚠️ A missing conversion factor is worse than a wrong one

If a unit shows up that isn’t in your conversion table, the left_join gives factor = NA and LBSTRESN silently becomes NA. Always assert that every --ORRES produced a non-missing --STRESN (or a documented reason why not), or an entire lab’s worth of results can vanish without a warning.

4. Character results in a numeric world

The other half of the labs problem: not every result is a number. Results come back as "<0.5" (below the limit of detection), "POSITIVE", ">1000", or "NEGATIVE". --ORRES is character precisely so it can hold these, but --STRESN is numeric — and as.numeric("<0.5") is NA with a warning. You have to decide, per test, how each non-numeric value maps.

raw_results <- c("42", "<0.5", ">1000", "POSITIVE", "12.7", "NEGATIVE")

suppressWarnings(as.numeric(raw_results))   # everything non-numeric -> NA

#> [1] 42.0   NA   NA   NA 12.7   NA

A defensible approach keeps the character result verbatim in --STRESC, derives --STRESN only where a number is genuinely present, and records why a value is non-numeric so it isn’t mistaken for missing:

standardize_result <- function(orres) {
  is_below <- grepl("^<", orres)
  is_above <- grepl("^>", orres)
  numeric_part <- suppressWarnings(as.numeric(gsub("[<>]", "", orres)))
  tibble::tibble(
    STRESC = orres,                       # always keep the original
    STRESN = numeric_part,                # NA for POSITIVE / NEGATIVE
    result_type = dplyr::case_when(
      is_below                 ~ "below_limit",
      is_above                 ~ "above_limit",
      !is.na(numeric_part)     ~ "numeric",
      TRUE                     ~ "qualitative"
    )
  )
}

standardize_result(raw_results)

#> # A tibble: 6 × 3
#>   STRESC   STRESN result_type
#>   <chr>     <dbl> <chr>      
#> 1 42         42   numeric    
#> 2 <0.5        0.5 below_limit
#> 3 >1000    1000   above_limit
#> 4 POSITIVE   NA   qualitative
#> 5 12.7       12.7 numeric    
#> 6 NEGATIVE   NA   qualitative

Now "<0.5" is flagged below_limit (a stats analysis might substitute half the detection limit), and "POSITIVE" is qualitative rather than a mysterious NA. The information about why the number is absent survives.

5. Subjects who exist in one domain but not another

DM is supposed to be the master list of subjects, and every other domain should only contain subjects who are in DM. In practice, an AE gets logged against a subject ID that was mistyped, or a subject is in DM but has an AE row that references a slightly different ID. These orphan records are a top conformance finding and they silently drop out of any inner join.

ae_with_orphan <- dplyr::bind_rows(
  ae,
  ae |> dplyr::slice(1) |> dplyr::mutate(USUBJID = "DEMO01-99-999")  # typo'd ID
)

# Orphans: AE subjects with no matching DM record
orphans <- dplyr::anti_join(ae_with_orphan, dm, by = "USUBJID")
orphans |> dplyr::select(USUBJID, AETERM)

#> # A tibble: 1 × 2
#>   USUBJID       AETERM  
#>   <chr>         <chr>   
#> 1 DEMO01-99-999 Headache

# And the reverse: DM subjects with zero AEs (often fine, but worth knowing)
dm_no_ae <- dplyr::anti_join(dm, ae_with_orphan, by = "USUBJID")
dm_no_ae |> dplyr::select(USUBJID)

#> # A tibble: 2 × 1
#>   USUBJID      
#>   <chr>        
#> 1 DEMO01-01-002
#> 2 DEMO01-03-005

The anti_join in both directions is the workhorse referential-integrity check. An orphan AE is never dropped silently — it’s surfaced as a query. The reverse (subjects with no AEs) is usually legitimate but is exactly the kind of thing you confirm rather than assume.

6. The `left_join` vs `inner_join` trap

Following directly from the orphan problem: the choice between left_join and inner_join when combining domains is one of the most consequential and most-overlooked decisions in the whole pipeline. An inner_join silently drops subjects, which can bias a result — and because nothing errors, it can survive all the way to a submitted table.

# inner_join drops the orphan AE and any DM subject without an AE
dm |>
  dplyr::inner_join(ae_with_orphan, by = "USUBJID") |>
  dplyr::distinct(USUBJID) |>
  nrow()

#> [1] 3

# left_join from DM keeps every enrolled subject (correct denominator)
dm |>
  dplyr::left_join(ae_with_orphan, by = "USUBJID") |>
  dplyr::distinct(USUBJID) |>
  nrow()

#> [1] 5

The safety habit: decide the denominator explicitly. For a “% of subjects with an AE” table, the denominator is enrolled subjects (DM), so you left_join from DM and count non-matches as “no event,” never as “absent.” An inner_join here would quietly shrink your denominator and inflate the rate.

7. Controlled terminology drift

Every coded variable (SEX, RACE, severity, units, …) must match a published CDISC code list exactly — including case and spelling. Sites and labs supply free-ish text that drifts: "Male", "MALE", "m", "Man". Recoding only the values you happened to see is a recipe for a value slipping through unmapped on the next data cut.

incoming_sex <- c("Male", "MALE", "m", "Female", "F", "Man", "")

# A closed allow-list, not an open recode: anything unmapped is caught
sex_map <- c(
  "male" = "M", "m" = "M", "man" = "M",
  "female" = "F", "f" = "F", "woman" = "F"
)

map_controlled <- function(x, map) {
  key <- tolower(trimws(x))
  mapped <- unname(map[key])
  tibble::tibble(
    raw = x,
    mapped = mapped,
    unmapped = is.na(mapped) & !(key %in% c("", NA))
  )
}

map_controlled(incoming_sex, sex_map)

#> # A tibble: 7 × 3
#>   raw      mapped unmapped
#>   <chr>    <chr>  <lgl>   
#> 1 "Male"   M      FALSE   
#> 2 "MALE"   M      FALSE   
#> 3 "m"      M      FALSE   
#> 4 "Female" F      FALSE   
#> 5 "F"      F      FALSE   
#> 6 "Man"    M      FALSE   
#> 7 ""       <NA>   FALSE

"Man" maps fine (it’s in the allow-list), the empty string is honestly unmapped-but-not-flagged (genuinely missing), and if "Intersex" or a typo like "Femal" showed up it would surface as unmapped = TRUE instead of being silently dropped or, worse, passed through as invalid terminology. The principle: validate against a closed code list and flag the misses, rather than an open-ended recode() that only handles today’s values.

The through-line

Every one of these has the same shape: the naive operation succeeds silently and produces plausible-looking garbage — an NA that means three different things, a mean across incompatible units, a denominator that quietly shrank. The discipline that makes SDTM trustworthy isn’t clever code; it’s refusing to let anything happen silently: classify before you convert, flag every imputation, assert referential integrity, and validate coded values against closed lists. That mindset is most of what separates a production clinical programmer from someone who merely knows the tidyverse.

SDTM vs. ADaM (the next step)

One more thing worth knowing: SDTM is not the end of the pipeline. It’s the tabulation layer — a faithful, standardized record of what was collected. For the actual statistical analysis, that data gets transformed again into ADaM (Analysis Data Model) datasets, which are built to be one-row-per-analysis-record and carry all the derived flags and baselines a statistician needs.

The typical flow:

Raw EDC data  ->  SDTM (standardized tabulation)  ->  ADaM (analysis-ready)  ->  Tables, Figures, Listings

Each arrow is a set of documented, validated R (or SAS) programs. If you can build the SDTM step, the ADaM step is the same skills applied one layer up.

Recap

SDTM is a CDISC standard that puts clinical trial data into a fixed set of domains (DM, AE, VS, LB, …) with standardized variable names.
Every domain shares STUDYID and USUBJID, so cross-domain analysis is just a join.
Building a domain in R is: rename to SDTM variables, derive the keys and flags, add sequence numbers, and validate.
Findings domains (VS, LB) use the --TESTCD / --ORRES / --STRESN pattern; events domains (AE) use start/end dates; DM is one row per subject.
In production you’d lean on the pharmaverse, but the transformations are ordinary tidyverse under the hood.
The hard part isn’t the happy path — it’s partial dates, duplicates, unit conversions, non-numeric results, orphan records, join semantics, and terminology drift. The unifying discipline is refusing to let bad data pass silently: classify before converting, flag every imputation, and validate against closed code lists.

If you found this useful or spotted something to fix, email me at cavandonohoe@gmail.com or open an issue on GitHub.

SDTM with R: Clinical Trial Data Standards

Cavan Donohoe

SDTM with R

What is SDTM?

The domains you’ll see most often

The naming conventions

Some raw data to start from

Building the DM (Demographics) domain

Building the VS (Vital Signs) domain

Building the AE (Adverse Events) domain

Why the common structure pays off

A tiny validation function

The hard part: edge cases and persistent issues

1. Partial and impossible dates

2. Duplicate records that aren’t obviously duplicates

3. Unit chaos in findings domains

4. Character results in a numeric world

5. Subjects who exist in one domain but not another

6. The `left_join` vs `inner_join` trap

7. Controlled terminology drift

The through-line

SDTM vs. ADaM (the next step)

Recap

SDTM with R: Clinical Trial Data Standards

Cavan Donohoe

SDTM with R

What is SDTM?

The domains you’ll see most often

The naming conventions

Some raw data to start from

Building the DM (Demographics) domain

Building the VS (Vital Signs) domain

Building the AE (Adverse Events) domain

Why the common structure pays off

A tiny validation function

The hard part: edge cases and persistent issues

1. Partial and impossible dates

2. Duplicate records that aren’t obviously duplicates

3. Unit chaos in findings domains

4. Character results in a numeric world

5. Subjects who exist in one domain but not another

6. The left_join vs inner_join trap

7. Controlled terminology drift

The through-line

SDTM vs. ADaM (the next step)

Recap

6. The `left_join` vs `inner_join` trap