feat: comet data release

.spelling

@@ -597,6 +597,7 @@ deconvoluting
 DMG-H3
 gemcitabine
 Hematopoiesis
+iDAT
 Illumina
 in-vitro
 in-vivo

content/3.genomics-platform/1.getting-started/3.making-a-data-request.md

@@ -124,3 +124,15 @@ If you receive an email from us that your DAA is incomplete, you may edit your D
 ## Managing your Data Request
 
 Go to our [Managing Data Overview](/genomics-platform/managing-data/overview) documentation page to learn how to check the status of your data request, complete an EDAA draft, upload a revised DAA, and ultimately access your data from your [My Dashboard](https://platform.stjude.cloud/requests/manage) page.
+
+## Unrestricted Data
+
+Certain Data within Genomics Platform is unrestricted, meaning that access is available to all requestors and does not require a data access agreement.
+
+To access this data, please complete the following steps:
+
+1. Create an account on or log in to Genomics Platform.
+2. Narrow your selection by filtering to Feature Count Files only and/or iDAT files only and then selecting Request Data at the bottom right of the screen.
+3. Choose to vend the data to a new or existing project
+4. Submit the request.
+5. The data will be vended to your selected project in a folder labeled with the date the data was requested.

content/3.genomics-platform/2.about-our-data/1.data-sets-and-data-access-units.md

@@ -9,6 +9,7 @@ title: Data Sets and Data Access Units
   - [Data Set](#data-set)
   - [Data Access Committee (DAC)](#data-access-committee-dac)
   - [Embargo Date](#embargo-date)
+  - [Unrestricted Data](#unrestricted-data)
 - [List of DAUs](#list-of-daus)
 - [List of Data Sets](#list-of-data-sets)
 
@@ -62,6 +63,16 @@ Publishing using any of the files _before_ the embargo date has passed is strict
 Some Data, including Data funded by the NIH, are not subject to embargo.
 Applicable Embargo Dates can be found in [Genomics Platform Metadata](https://platform.stjude.cloud/api/v1/manifest.tsv){target="_blank"} in the `SJ_Embargo_Date` column.
 
+### Unrestricted Data
+
+Certain data within the Genomics Platform is unrestricted. Unrestricted data is not subject to DAC-reviewed approval before a user can obtain it.
+The unrestricted dataset on St. Jude Cloud currently includes:
+
+- Feature count files
+- [COMET](https://comet.stjude.org/) iDAT files
+
+Steps to access unrestricted data can be found [here](http://docs.stjude.cloud/genomics-platform/getting-started/making-a-data-request#unrestricted-data).
+
 ---
 
 ## List of DAUs
@@ -158,7 +169,7 @@ The following data set(s) are included within SJLIFE:
 
 ## List of Data Sets
 
-We currently have 21 [Data Sets](#data-set) listed below.
+We currently have 22 [Data Sets](#data-set) listed below.
 Additional information can also be seen including which [Data Access Units (DAU)](#data-access-unit-dau) the Data Set belongs to, tissue type, sequencing type, number of samples, additional links, and a brief description.
 
 | Data Set                                                                                         | DAU(s)                  | Tissue Type         | Sequencing                | Samples |
@@ -167,6 +178,7 @@ Additional information can also be seen including which [Data Access Units (DAU)
 | [CCSS](#childhood-cancer-survivor-study)                                                         | CCSS                    | Germline Only       | WGS                       |   2,912 |
 | [CICERO Benchmark](#cicero-benchmark)                                                            | PCGP, Clinical Genomics | Paired Tumor-Normal | RNA-Seq                   |     124 |
 | [Clinical Pilot](#clinical-pilot)                                                                | PCGP, Clinical Genomics | Paired Tumor-Normal | WGS, WES, RNA-Seq         |     155 |
+| [COMET](#comet)                                                                                  | Unrestricted            | iDAT file           | —                         |    4269 |
 | [CReATe](#clinical-research-in-als-and-related-disorders-for-therapeutic-development-consortium) | CReATe                  | PBMC Germline DNA   | WGS                       |     705 |
 | [CSTN](#childhood-solid-tumor-network)                                                           | PCGP, Clinical Genomics | Paired Tumor-Normal | WGS, WES, RNA-Seq         |     143 |
 | [G4K](#genome-4-kids)                                                                            | PCGP, Clinical Genomics | Paired Tumor-Normal | WGS, WES, RNA-Seq         |     565 |
@@ -183,7 +195,7 @@ Additional information can also be seen including which [Data Access Units (DAU)
 | [RTCG](#real-time-clinical-genomics)                                                             | PCGP, Clinical Genomics | Paired Tumor-Normal | WGS, WES, RNA-Seq         |   7,767 |
 | [SGP](#sickle-cell-genome-project)                                                               | SGP                     | Germline Only       | WGS                       |     807 |
 | [SJLIFE](#st-jude-life)                                                                          | SJLIFE                  | Germline Only       | WGS, WES                  |   4,838 |
-| [SJLIFE_ClonalHematopoiesis](#st-jude-life-clonal-hematopoiesis)                                 | SJLIFE                  | —                   | Single Cell-WGS, Targeted |   3,192 |
+| [SJLIFE_ClonalHematopoiesis](#st-jude-life-clonal-hematopoiesis)                                 | PCGP                    | —                   | Single Cell-WGS, Targeted |   3,192 |
 | [tMN](#pediatric-therapy-related-myeloid-neoplasms-tmn)                                          | PCGP                    | Paired Tumor-Normal | WGS, WES, RNA-Seq         |     206 |
 
 ### Atypical Teratoid / Rhabdoid Tumor-derived Tumoroid Models
@@ -254,6 +266,14 @@ In addition to patients enrolled in the PGB1 Cohort (primary participants), the
 This dataset includes WGS data from N=705 in PGB1, including N=472 ALS/ALS-FTD, N=20 PMA, N=47 PLS, N=162 HSP, and N=4 with other related disorders.
 The findings of the project were published in [Translational Neurodegeneration](https://translationalneurodegeneration.biomedcentral.com/articles/10.1186/s40035-025-00516-2).
 
+### COMET
+
+**DAU**: - | **Tissue Type**: - | **Assay Type**: Illumina Infinium 850K array | **Samples**: 4,629| **[Additional Information About COMET](https://www.stjude.org/research/departments/computational-biology/comet.html)**
+
+The solid tumor COmprehensive METhylation (COMET) database is a searchable repository of pediatric solid tumor DNA methylation and copy number variant (CNV) profiles, generated using the Illumina Infinium 850K array, paired with matched whole slide histology images (WSI).
+It is the largest and most comprehensive extracranial pediatric solid tumor epigenetic reference dataset in the world, offering DNA methylation profiles across 20 different types of pediatric solid tumors along with a comparative collection of patient-derived orthotopic xenografts, cell lines, adult sarcomas, and normal tissues.
+See [Unrestricted Data](#unrestricted-data) for more details on requesting access to this data set.
+
 ### DMG-H3K27a Clonal Evolution
 
 **DAU**: PCGP | **Tissue Type**: — | **Sequencing Type**: WGS, WES | **Samples**: 70

content/3.genomics-platform/2.about-our-data/2.file-formats-and-sequencing-information.md

@@ -13,6 +13,7 @@ St. Jude Cloud hosts both raw genomic data files and processed results files:
 | Somatic VCF    | Curated list of somatic variants produced by the St. Jude somatic variant analysis pipeline.                                                                                                                                                               | [Click here](#somatic-vcf-files)    |
 | CNV            | List of somatic copy number alterations produced by St. Jude CONSERTING pipeline.                                                                                                                                                                          | [Click here](#cnv-files)            |
 | Feature Counts | Curated list of read counts mapped to each gene produced by [HTSeq](https://htseq.readthedocs.io/en/master/)                                                                                                                                               | [Click here](#feature-counts-files) |
+| iDAT           | Raw, paired intensity files output by an Illumina microarray scanner for a single sample — one per fluorescence channel — before normalization or genotype/methylation calling.                                                                            | [Click here](#idat-files)           |
 
 ### BAM files
 
@@ -195,6 +196,11 @@ The files are tab-delimited text and contain the feature key and read count for
 [rnaseq-rfc]: https://stjudecloud.github.io/rfcs/0001-rnaseq-workflow-v2.0.0.html#specification
 [gencode]: https://www.gencodegenes.org/human/release_31.html
 
+### iDAT files
+
+These are the raw, paired intensity files output by an Illumina microarray for a single sample.
+Each sample includes two IDAT files — one per fluorescence channel (Green and Red) containing the raw, unprocessed probe intensity signal from the array before any normalization or genotype/methylation calling.
+
 ## Sequencing Information
 
 ### Whole Genome and Whole Exome

content/4.pecan/1.overview/1.getting-started.md

@@ -98,6 +98,16 @@ Data Facets represent a distinct type of post-processed genomic data for collect
   </div>
 </div>
 
+<div class="flex flex-row">
+  <div class="pr-8 pt-0 flex-none">
+    <a href="/pecan/data-facets/epigenetics"><img src="/img/pecan/overview/getting-started/histology.svg" alt="Epigenetics" ></a>
+  </div>
+  <div>
+    <h5 class="font-bold"><a href="/pecan/data-facets/epigenetics" class="text-blue-primary">Epigenetics</a></h5>
+    <p style="margin-top: 0;">Methylation landscape of over 4,400 pediatric cancer samples in PeCan.</p>
+  </div>
+</div>
+
 ## Tools
 
 <div class="flex flex-row">

content/4.pecan/2.data-facets/5.epigenetics.md

@@ -0,0 +1,87 @@
+---
+title: Epigenetics
+---
+
+![Epigenetics](/img/pecan/overview/getting-started/histology.svg)
+Explore the methylation landscape of over 4,400 pediatric cancer samples in PeCan.
+
+## Overview
+
+The Epigenetics facet lets you explore methylation data across 4,400+ pediatric cancer samples.
+Navigate the data using a UMAP plot with gene- or probe-level overlays of β-values, or view it collectively in a data table.
+
+## UMAP Overlay Features
+
+On page load, each sample on the UMAP is colored by its cancer subtype.
+When you select a gene or probe to overlay, the sample color changes to a gradient representative of the β-value for the sample.
+β-value is the range in which the target is methylated, with 0 being fully unmethylated and 1 being fully methylated; values can range from 0 to 1.
+You can overlay either the mean or median β-value; mean is selected by default (see β-value Toggle below).
+
+### Gene Overlay
+
+Select one or more genes to overlay their associated methylation on the UMAP.
+The overlay shows the mean β-value across all probes mapped to the selected gene by default.
+You can toggle to the median β-value or switch between selected genes to compare.
+To narrow results, apply filters to display only probes in the promoter region, specifically TSS1500 and/or TSS200.
+
+### Probe Overlay
+
+Select one or more probe IDs to overlay their methylation values on the UMAP.
+The overlay shows the mean β-value for the selected probe.
+The probe overlay is only available for CpG probes that start with `cg`.
+
+### Overlay Considerations
+
+The UMAP and its overlays are drawn from the same underlying dataset, but different filters are applied depending on the view.
+Understanding these filtering rules helps you interpret what you see.
+
+### Probe Filtering by Analysis Level
+
+| Level               | Low Quality Filter | Common SNP Probes | Cross Reactive Probes | Sex Chromosome Probes |
+|---------------------|--------------------|-------------------|-----------------------|-----------------------|
+| UMAP                | Yes                | Yes               | No                    | Yes                   |
+| Gene level overlay  | Yes                | Yes               | No                    | No                    |
+| Probe level overlay | Yes                | No                | No                    | No                    |
+
+### Filter Descriptions
+
+- **Low Quality Filter:** Removes probes that fail quality control thresholds.
+- **Common SNP Probes:** Excludes probes located at common single nucleotide polymorphisms to reduce genotype-driven variation.
+- **Cross Reactive Probes:** Excludes probes known to hybridize to multiple genomic locations.
+Currently not applied at any level.
+Please refer to the following lists of cross-reactive probes:
+  - Pidsley Cross-Reactive Probes
+  - McCartney Supplement Probes
+- **Sex Chromosome Probes:** Excludes probes on the X and Y chromosomes.
+Applied only at the UMAP level to prevent sex-driven clustering.
+
+## Data Table Features
+
+Select the Data tab to view a table of samples with corresponding metadata.
+Columns include Sample ID, Diagnosis Subtype Code, and Diagnosis Subtype Name.
+β-value columns are sortable, allowing you to rank samples by methylation level for a given gene or probe.
+
+## How to Narrow and Refine the Data
+
+- **Filters:** Filter samples by Sample ID(s), Subtype Root, Subtype, Subtype Biomarkers, Patient Phenotype (sex, age at diagnosis, race, ethnicity), or Sample Preparation parameters.
+- **Lasso and Pan/Zoom:** Use the Lasso tool to select a region of samples on the UMAP, or Pan/Zoom to focus on a specific area.
+- **β-value Toggle:** In the UMAP tab header, toggle between mean and median β-values for the gene overlay.
+Mean is selected by default.
+- **Hover and Drawer:**
+  - **Sample Hover:** Hover over or click a sample on the UMAP to view its metadata (diagnosis, demographics, etc.).
+  - **Gene Overlay Hover:** Hover over the selected gene to view probe-level data for that gene.
+
+## Menu Options (⋯)
+
+Access additional features from the three-dot menu in the header:
+
+- Show or hide diagnosis category labels on the UMAP
+- Copy the URL for the current UMAP view
+- Copy Sample IDs for the current view
+- Copy data as TSV
+- Export the current UMAP as SVG
+
+## UMAP Generation
+
+Methylation β-values were generated using the Infinium MethylationEPIC BeadChip v1.0 array and normalized with subset-quantile within-array normalization (SWAN) to correct for probe-type bias between Type I and Type II probes. After normalization, the standard deviation of β-values was calculated per probe, and the 10,000 probes with the highest standard deviation were kept for UMAP generation.
+Probes associated with SNPs at CpG sites, cross-reactive probes, and probes on sex chromosomes were excluded.

deployment/preview/pr184/app.yaml

@@ -0,0 +1,87 @@
+apiVersion: helm.toolkit.fluxcd.io/v2
+kind: HelmRelease
+metadata:
+  name: docs
+  namespace: docs-pr184
+spec:
+  interval: 30m
+  chart:
+    spec:
+      chart: generic
+      version: 1.1.x
+      sourceRef:
+        kind: HelmRepository
+        name: stjudecloud
+        namespace: flux-system
+      interval: 1h
+  values:
+    nameOverride: docs
+    extraDeploy:
+      - |
+        apiVersion: v1
+        kind: Service
+        metadata:
+          name: {{ template "common.names.fullname" . }}-oauth-bridge
+          labels: {{- include "common.labels.standard" . | nindent 4 }}
+            {{- if .Values.commonLabels }}
+            {{- include "common.tplvalues.render" ( dict "value" .Values.commonLabels "context" $ ) | nindent 4 }}
+            {{- end }}
+          {{- if .Values.commonAnnotations }}
+          annotations: {{- include "common.tplvalues.render" ( dict "value" .Values.commonAnnotations "context" $ ) | nindent 4 }}
+          {{- end }}
+        spec:
+          type: ExternalName
+          externalName: oauth2-proxy.oauth2-proxy
+      - |
+        ---
+        apiVersion: networking.k8s.io/v1
+        kind: Ingress
+        metadata:
+          name: {{ .Release.Name }}-oauth
+        spec:
+          ingressClassName: nginx
+          rules:
+          - host: {{ .Values.ingress.hostname }}
+            http:
+              paths:
+                - backend:
+                    service:
+                      name: {{ template "common.names.fullname" . }}-oauth-bridge
+                      port:
+                        number: 80
+                  path: /oauth2
+                  pathType: ImplementationSpecific
+          tls:
+          - hosts:
+            - {{ .Values.ingress.hostname }}
+            secretName: {{ .Values.ingress.hostname }}-tls
+    image:
+      repository: stjudecloud/docs
+      tag: pr184-318878b-406 # {"$imagepolicy": "flux-system:docs-pr184:tag"}
+    podAnnotations:
+      linkerd.io/inject: enabled
+      config.linkerd.io/proxy-cpu-request: 20m
+    containerPorts:
+      http: 3000
+    service:
+      ports:
+        - name: http
+          protocol: TCP
+          port: 3000
+          targetPort: http
+    ingress:
+      enabled: true
+      hostname: docs-pr184.staging.stjude.cloud
+      path: /
+      annotations:
+        cert-manager.io/cluster-issuer: letsencrypt-prod
+        linkerd.io/inject: ingress
+        nginx.ingress.kubernetes.io/auth-signin: https://$host/oauth2/start?rd=$escaped_request_uri
+        nginx.ingress.kubernetes.io/auth-url: https://$host/oauth2/auth
+        nginx.ingress.kubernetes.io/service-upstream: "true"
+        nginx.ingress.kubernetes.io/enable-modsecurity: "true"
+        nginx.ingress.kubernetes.io/enable-owasp-core-rules: "true"
+      tls:
+        enabled: true
+    datadog:
+      enabled: false