Skip to main content

Recipe

Pivoting coursecustomfields from Long to Wide

Convert the EAV-shaped coursecustomfields table to one row per course with each custom field as a column.

intermediateVerified 2026-05-19
All recipes
intermediateVerified 2026-05-19

Tables

coursecustomfields, courses

Tags

analytics
redshift
custom-fields

TL;DR

coursecustomfields is EAV-shaped (one row per custom field per course). Pivot to wide with MAX(CASE WHEN label = '<label>' THEN value END) grouped by courseid.

Problem

The coursecustomfields table is stored in **long (entity-attribute-value) format** — one row per (courseid, label) pair, with the actual value held in the value column. This shape is flexible (any number of custom fields, no schema change required to add a new one) but it is awkward to query:

  • Filtering on "courses where Department = Sales AND Difficulty = Advanced" requires multiple self-joins or subqueries.
  • BI tools generally expect one row per course with custom fields as columns.

Solution

Replace these labels with your own instance's labels

The label names below (Department, Difficulty, Duration (hrs), Product Line) are illustrative examples — they are specific to one instance. Substitute the actual custom-field labels from your own Thought Industries instance. If you run the query unchanged, it WILL execute and return one row per course, but every column except `courseid` will come back NULL, which can look like the query is broken when it isn't.

-- Pivot coursecustomfields: one row per course, one column per custom field
SELECT
  ccf.courseid,
  MAX(CASE WHEN ccf.label = 'Department'     THEN ccf.value END) AS department,
  MAX(CASE WHEN ccf.label = 'Difficulty'     THEN ccf.value END) AS difficulty,
  MAX(CASE WHEN ccf.label = 'Duration (hrs)' THEN ccf.value END) AS duration_hours,
  MAX(CASE WHEN ccf.label = 'Product Line'   THEN ccf.value END) AS product_line
FROM coursecustomfields ccf
WHERE ccf.statusgroup = 'a'
GROUP BY ccf.courseid;

Null statusgroup records can undercount

A meaningful share of records still have a null statusgroup even after the recent backfill, because the backfill skipped records that were missing a source update timestamp. Until a future full re-backfill is completed, filtering on statusgroup excludes those records and can undercount. Treat statusgroup-filtered results as a lower bound for now.

-- Enrich the courses table with pivoted custom fields
WITH ccf_pivot AS (
  SELECT
    courseid,
    MAX(CASE WHEN label = 'Department' THEN value END) AS department,
    MAX(CASE WHEN label = 'Difficulty' THEN value END) AS difficulty
  FROM coursecustomfields
  WHERE statusgroup = 'a'
  GROUP BY courseid
)
SELECT
  c.id          AS course_id,
  c.title       AS course_title,
  ccf.department,
  ccf.difficulty
FROM courses c
LEFT JOIN ccf_pivot ccf ON ccf.courseid = c.id
WHERE c.statusgroup = 'a';

How it works

  • MAX(CASE WHEN label = '<X>' THEN value END) collapses the per-label rows into one column per label. MAX is used because aggregation is required after GROUP BYMIN/ANY_VALUE work equally well since there's only one value per (courseid, label).
  • GROUP BY courseid produces the one-row-per-course shape.
  • statusgroup = 'a' on both the pivot and the outer courses filter ensures we ignore archived/deleted custom field rows and archived courses.

Variations

  • **Numeric custom fields:** wrap with CAST(... AS DECIMAL) if you need to do math on a custom field stored as a string (value is varchar).
  • **Dynamic column set:** the MAX(CASE WHEN ...) pattern requires you to enumerate labels statically. For a truly dynamic column set, generate the pivot SQL in your BI tool or use a long-format export.
  • **Per-instance labels:** custom field labels vary per tenant. Run SELECT DISTINCT label FROM coursecustomfields WHERE companyid = '<tenant>' to discover them before writing the pivot.

Label values are case-sensitive

The CASE WHEN label = '...' comparisons match the label column exactly, including case and whitespace. Use LOWER(TRIM(label)) on both sides if your instance has inconsistent label casing.

Related recipes