COSD: The Architecture of Cancer Data in England

Every year, around 375,000 people in England receive a cancer diagnosis. Each of those diagnoses generates a cascade of clinical events — referrals, staging investigations, MDT decisions, treatments, follow-ups — all of which must be captured, standardised, and reported to the National Disease Registration Service. The mechanism through which all of this flows is COSD: the Cancer Outcomes and Services Dataset.

COSD is not simply a data collection form. It is a precisely-specified, schema-governed, coded XML standard that constitutes the backbone of cancer intelligence in England. It defines not just what is collected, but in which vocabulary (ICD-O-3, SNOMED CT, OPCS-4, TNM), at which point in the pathway, and in what hierarchical structure. It is, in a very real sense, the language in which cancer data speaks nationally.

For engineering teams building cancer data platforms, COSD is both a gift and a challenge. The schemas are rigorous and well-documented. The content is clinically rich. But the complexity is real: 13 site-specific record types, a modular XSD hierarchy spanning over 20 schema files, coded values drawn from multiple clinical ontologies, and a standard that evolves with each TRUD release.

What is COSD — and Who Owns It?

COSD is the national standard for cancer data collection in England, mandated by NHS England and operated by the National Disease Registration Service (NDRS). NDRS replaced the previous network of regional cancer registries and now acts as the single national collector and custodian of cancer data in England.

The standard defines:

• What data providers must submit for cancer patients — diagnoses, staging, treatments, pathway milestones, outcomes
• In what structure and codes — XML schemas, value sets, SNOMED CT, ICD-O-3, OPCS-4, TNM
• At which points in the care pathway — referral, diagnosis, staging, treatment, follow-up

In practice, NHS Trusts and cancer services generate COSD XML submissions from their clinical systems and submit them to NDRS via secure channels. Downstream platforms ingest that XML, validate it against the schemas, transform it into an internal data model, and may further map it to standardised analytics models such as OMOP CDM.

The current production version is COSD v10.2.0, distributed via NHS TRUD.

Getting COSD Schemas from NHS TRUD

COSD schemas are freely available — but only through TRUD (Technology Reference data Update Distribution), NHS England’s official distribution platform for reference data and technical standards. TRUD distributes coding schemes, messaging specifications, and XML schemas including COSD.

Access requires a free account and acceptance of per-product licence terms.

Step-by-step

1. Register at isd.digital.nhs.uk/trud3/ Create an account using your organisational email. NHS or organisational domain email is strongly recommended over personal addresses to simplify licence compliance paperwork downstream.

2. Search and subscribe to the COSD product Once logged in, search for “Cancer Outcomes and Services Dataset”. Locate the core COSD product and review the NHS data standards licence terms before subscribing. Subscription is free.

3. Download the release zip From the product’s Releases page, select the version matching your target (e.g. v10.2.0). The zip contains: all XSDs, sample XML submissions, and Excel/PDF field-level documentation — the latter being essential reference material for your implementation team, as it details permitted value sets, mandatory/optional flags, and clinical definitions for every element.

4. Version-control and align Extract schemas into a versioned path in your repository (e.g. documents/nhs_schemas/cosd/xml/). When upgrading versions, run a structured diff against your mapper script and XSD hierarchy document, then execute regression tests using the official TRUD sample XML before promoting to production.

Critical: Your XSD validation pipeline, mapper script, internal data model, and the XML submissions coming from Trusts must all reference the same COSD version. A mismatch — even a minor version difference — can result in silent data loss when new fields are unrecognised or deprecated fields continue to be parsed.

The 13 Cancer Record Types

COSD organises cancer data by tumour site group. Each record type corresponds to a distinct XSD, a set of site-specific data extensions, and different business rules for validation, staging, and treatment details.

From an implementation perspective, the valid record types are:

def validRecordTypes = [
    'BreastRecord', 'CNSRecord', 'ColorectalRecord', 'CTYARecord',
    'GynaecologicalRecord', 'HaematologicalRecord', 'LiverRecord', 'LungRecord',
    'OtherRecord', 'SarcomaRecord', 'SkinRecord', 'UpperGIRecord', 'UrologicalRecord'
] as Set

BreastRecord

Breast cancers. Includes breast-specific risk factors, hormone receptor status (ER, PR, HER2), BRCA/germline variant status, and surgical detail (mastectomy vs. breast-conserving surgery). High submission volume. Schema: COSD-v10-2-0_BREAST.xsd.

CNSRecord

Central nervous system tumours. Adds CNS-specific staging (e.g. Chang staging for medulloblastoma), WHO CNS grade, and neurological performance status fields not present in core COSD. Schema: COSD-v10-2-0_CNS.xsd.

ColorectalRecord

Colorectal cancers. Site-specific fields for peritoneal involvement, extramural vascular invasion (EMVI), Dukes staging, and circumferential resection margin (CRM) status. High submission volume. Schema: COSD-v10-2-0_COLORECTAL.xsd.

CTYARecord

Children, Teenagers, and Young Adults. Adds paediatric/young-adult specific fields including CCLG tumour classification, principal treatment centre designation, clinical trial participation, and parental consent status. Schema: COSD-v10-2-0_CTYA.xsd.

GynaecologicalRecord

Ovarian, cervical, uterine, and vulval cancers. FIGO staging, HPV status, peritoneal involvement, and cytoreductive surgery detail. Schema: COSD-v10-2-0_GYNAECOLOGICAL.xsd.

HaematologicalRecord

Leukaemias, lymphomas, and myeloma. Disease-specific staging systems (Ann Arbor, Durie-Salmon, ISS), cytogenetic risk group, bone marrow trephine status, and transplant eligibility. Schema: COSD-v10-2-0_HAEMATOLOGICAL.xsd.

LiverRecord

Liver cancers including HCC and cholangiocarcinoma. Cirrhosis type and cause, surveillance scan history, Barcelona Clinic Liver Cancer (BCLC) stage, cholangiocarcinoma anatomical category, and biliary risk factors. Schema: COSD-v10-2-0_LIVER.xsd.

// Site-specific liver fields — LiverRecord only
liverSurveillanceScans        : [email protected](),
liverCirrhosisType            : [email protected](),
causeOfLiverCirrhosis         : [email protected](),
cholangiocarcinomaCategory    : [email protected](),
cholangiocarcinomaRiskFactorsType : diag.Cholangiocarcinoma.CholangiocarcinomaRiskFactorsType.@code.text(),

LungRecord

Lung cancers. Lung health check indicator, bronchoscopy findings, mediastinal sampling status, EGFR/ALK mutation status, performance status, and surgical fitness assessment. Schema: COSD-v10-2-0_LUNG.xsd.

OtherRecord

Cancers not covered by a dedicated site schema — a generic pathway using core COSD modules without site-specific extensions. Provides data coverage for rare tumour types. No site-specific XSD extension.

SarcomaRecord

Soft tissue and bone sarcomas. Rare tumour-specific staging, specialist sarcoma centre designation, surgical margin status (R0/R1/R2), and limb salvage vs. amputation detail. Schema: COSD-v10-2-0_SARCOMA.xsd.

SkinRecord

Melanoma and non-melanoma skin cancers. For melanoma: Breslow thickness, Clark level, ulceration status, and sentinel lymph node biopsy result. For non-melanoma: excision margin status and reconstruction approach. Schema: COSD-v10-2-0_SKIN.xsd.

UpperGIRecord

Oesophageal, gastric, and pancreatic cancers. Barrett’s surveillance history, Siewert classification for junctional tumours, neoadjuvant therapy response grading, and resection type (Ivor Lewis, total gastrectomy, Whipple’s). Schema: COSD-v10-2-0_UPPERGI.xsd.

UrologicalRecord

Prostate, bladder, kidney, and testicular cancers. PSA values, Gleason/ISUP grade group, NMIBC vs. MIBC bladder classification, nephrectomy type, and serum tumour markers (AFP, beta-HCG, LDH) for testicular. Schema: COSD-v10-2-0_UROLOGICAL.xsd.

The XSD Schema Hierarchy

COSD’s schema is intentionally modular. The root XSD (COSD-v10-2-0.xsd) imports a constellation of domain-specific and utility schemas. Understanding this hierarchy is essential for both validation pipeline construction and for knowing which XSD to consult when debugging a field-level issue.

COSD-v10-2-0.xsd                          ← root, validation entry point
├── COSD-v10-2-0_DEMOGRAPHICS.xsd
├── COSD-v10-2-0_PATIENTIDENTITYDETAILS.xsd
├── COSD-v10-2-0_PRIMARYPATHWAY.xsd
├── COSD-v10-2-0_NONPRIMARYPATHWAY.xsd
├── COSD-v10-2-0_DIAGNOSTICPROCEDURES.xsd
├── COSD-v10-2-0_IMAGING.xsd
├── COSD-v10-2-0_TREATMENT.xsd
├── COSD-v10-2-0_LABORATORYRESULTS.xsd
├── COSD-v10-2-0_MULTIDISCIPLINARYTEAMMEETINGS.xsd
├── COSD-v10-2-0_HOLISTICNEEDSASSESSMENT...xsd
├── Site-specific extensions:
│   ├── COSD-v10-2-0_BREAST.xsd
│   ├── COSD-v10-2-0_LUNG.xsd
│   ├── COSD-v10-2-0_CNS.xsd
│   ├── COSD-v10-2-0_LIVER.xsd
│   ├── COSD-v10-2-0_HAEMATOLOGICAL.xsd
│   ├── COSD-v10-2-0_GYNAECOLOGICAL.xsd
│   ├── COSD-v10-2-0_RENALTUMOURS.xsd
│   └── ... (all 13 site groups)
└── COSD-v10-2-0_DATA_TYPES.xsd           ← shared type definitions, load first

The dependency ordering matters for schema validators. Load DATA_TYPES.xsd first, then the domain module schemas, then site-specific extensions, and finally the root. A COSD_XSD_HIERARCHY.md in your schema directory should document this load order explicitly and be updated on every version upgrade.

The Ingestion Pipeline

A typical COSD ingestion architecture moves through four stages:

Stage 1 — Trust XML Generation. NHS Trusts generate COSD XML from their clinical systems (PAS, cancer management systems). The root element is the record type: <BreastRecord>, <LungRecord>, etc.

Stage 2 — Schema Validation. Incoming XML is validated against the COSD XSD hierarchy. A production-grade implementation surfaces field-level error messages back to the submitting provider — not just a binary pass/fail. Invalid records are quarantined, not silently dropped.

Stage 3 — NiFi Transformation. Apache NiFi flows execute a Groovy mapper script to convert validated COSD XML into a flat JSON/DTO representation. This is where XML GPath extraction, coded value normalisation, and nested array handling (treatments, imaging episodes, MDT meetings) all occur.

Stage 4 — ETL to Data Vault and OMOP. A .NET ETL engine persists the transformed data to a Data Vault model (raw vault → business vault), and optionally maps to OMOP CDM for standardised analytics.

The mapper script is not just a transform — it is your organisation’s canonical interpretation of what COSD means in your domain. Every field extraction choice, every null-handling decision, every code path is an implicit design decision about your data model.

Field-Level Mapping: Inside mapper.groovy

The NiFi Groovy mapper is the connective tissue between NHS standards and your internal model. The following sections cover the most architecturally significant field groups.

Envelope and Record Identity

recordType      : xml.name().toString(),
recordId        : [email protected](),
uploadBatchId   : flowFile.getAttribute('uploadbatch.id'),
organisationId  : flowFile.getAttribute('uploadbatch.organisationId'),

xml.name() retrieves the XML root element name, used to route records to site-specific processing logic. The organisationId from NiFi FlowFile attributes enables multi-tenancy — the same pipeline can ingest COSD from multiple Trusts with full data isolation.

Patient Identity

nhsNumber                            : [email protected](),
localPatientIdentifier               : xml.PatientIdentityDetails.LocalPatientIdentifier.text(),
nhsNumberStatusIndicatorCode         : xml.PatientIdentityDetails.NhsNumberStatusIndicatorCode.@code.text(),
personBirthDate                      : xml.PatientIdentityDetails.PersonBirthDate.text(),
organisationIdentifierCodeOfProvider : xml.PatientIdentityDetails.OrganisationIdentifierCodeOfProvider.@extension.text(),

Note the distinction between [email protected]() for HL7 identifier attributes and .text() for plain text elements. Confusing these is a common source of silent empty fields in implementations.

Demographics

ethnicCategory                         : [email protected](),
personStatedGenderCode                 : [email protected](),
personSexualOrientationCodeAtDiagnosis : xml.Demographics.PersonSexualOrientationCodeAtDiagnosis.@code.text(),
postcodeOfUsualAddressAtDiagnosis      : xml.Demographics.PostcodeOfUsualAddressAtDiagnosis.text(),
streetAddressLine                      : (xml.Demographics.Address.StructuredAddress.StreetAddressLine.text()
                                          ?: xml.Demographics.Address.UnstructuredAddress.StreetAddressLine.text()),

The address fallback pattern (Elvis operator ?:) handles the real-world variability in Trust XML — some submit StructuredAddress, others UnstructuredAddress. Ethnic category uses ONS ethnic group codes; gender and sexual orientation use NHS-specific coded value sets from the COSD documentation.

Diagnosis and Staging

primaryDiagnosisIcd                  : [email protected](),
morphologyIcdO3                      : diag.'MorphologyIcd-o-3'[email protected](),
topographyIcdO3                      : diag.'TopographyIcd-o-3'[email protected](),
diagnosisCodeSnomedCt                : [email protected](),
gradeOfDifferentiationAtDiagnosis    : [email protected](),
performanceStatusAdult               : [email protected](),
ki67                                 : [email protected](),
basisOfDiagnosisCancer               : [email protected](),
familialCancerSyndrome               : [email protected](),

ICD-O-3 provides both morphology (histological type, e.g. 8500/3 for invasive ductal carcinoma) and topography (primary site, C-codes). SNOMED CT codes provide an additional clinical diagnostic layer. Ki-67 is a proliferation index used in breast and other cancers — note it uses @value not @code.

TNM staging captures both final pre-treatment and integrated stage:

tCategoryFinalPretreatment        : stag.TCategoryFinalPretreatment.text(),
nCategoryFinalPretreatment        : stag.NCategoryFinalPretreatment.text(),
mCategoryFinalPretreatment        : stag.MCategoryFinalPretreatment.text(),
tnmStageGroupingFinalPretreatment : stag.TnmStageGroupingFinalPretreatment.text(),
tCategoryIntegratedStage          : stag.TCategoryIntegratedStage.text(),
nCategoryIntegratedStage          : stag.NCategoryIntegratedStage.text(),
mCategoryIntegratedStage          : stag.MCategoryIntegratedStage.text(),
tnmStageGroupingIntegrated        : stag.TnmStageGroupingIntegrated.text(),
tnmCodingEdition                  : [email protected](),

Always capture tnmCodingEdition — whether the 7th or 8th edition of TNM was used materially affects what each T/N/M category means clinically, and mixing editions in aggregate analysis is a known data quality pitfall.

Treatments — Nested Arrays

The treatment block is the most structurally complex section in COSD mapping:

treatments: xml.Treatment.collect { t -> [
    cancerTreatmentIntent        : [email protected](),
    treatmentStartDate           : t.TreatmentStartDateCancer.text(),
    modalityRegistration         : [email protected](),
    organisationSiteIdentifier   : t.OrganisationSiteIdentifierOfProviderCancerTreatmentStartDate.@extension.text(),
    endOfTreatmentSummaryDate    : t.EndOfTreatmentSummaryDate.text(),

    // Surgery sub-block
    surgeryProcedureDate         : t.Surgery.ProcedureDate.text(),
    primaryProcedureOpcs         : [email protected](),
    primaryProcedureSnomedCt     : [email protected](),
    asaScore                     : [email protected](),
    surgicalAccessType           : [email protected](),
    unplannedReturnToTheatre     : [email protected](),

    // Consultant surgeons — further nested array
    consultantSurgeons: t.Surgery.ConsultantSurgeon.collect { cs -> [
        issuerCode      : cs.'ProfessionalRegistrationIssuerCode-ConsultantSurgeon'[email protected](),
        entryIdentifier : cs.'ProfessionalRegistrationEntryIdentifier-ConsultantSurgeon'.text()
    ]}
]},

A patient record may have multiple treatment episodes; each surgical treatment may have multiple consultant surgeons. Groovy’s .collect{} idiom handles both levels cleanly. A key advantage: an empty xml.Treatment node produces an empty list, not null — which simplifies downstream null-handling in ETL.

OPCS-4 procedure codes are the primary surgical classification. SNOMED CT procedure codes are optionally present and should be captured where available for OMOP mapping, since OPCS-4 has limited vocabulary coverage in standard OMOP releases.

Risk Factors

tobaccoSmokingStatus              : xml.ClinicalNurseSpecialistAndRiskFactorAssessment.TobaccoSmokingStatus.@code.text(),
tobaccoSmokingCessation           : xml.ClinicalNurseSpecialistAndRiskFactorAssessment.TobaccoSmokingCessation.@code.text(),
historyOfAlcoholCurrent           : xml.ClinicalNurseSpecialistAndRiskFactorAssessment.HistoryOfAlcoholCurrent.@code.text(),
diabetesMellitusType1AndType2Indicator : xml.ClinicalNurseSpecialistAndRiskFactorAssessment.DiabetesMellitusType1AndType2Indicator.@code.text(),
menopausalStatus                  : xml.ClinicalNurseSpecialistAndRiskFactorAssessment.MenopausalStatus.@code.text(),
physicalActivityCurrent           : xml.ClinicalNurseSpecialistAndRiskFactorAssessment.PhysicalActivityCurrent.@code.text(),

These fields support lifestyle risk factor analysis, stratification of outcomes by modifiable factors, and resource planning for CNS input. Completeness on these fields is often lower than for diagnostic fields — important to track in data quality dashboards.

From COSD to OMOP CDM

OMOP CDM (Observational Medical Outcomes Partnership Common Data Model) is the target analytics standard for federated research networks and many NHS analytics platforms. Converting COSD to OMOP is non-trivial: COSD is pathway-centric and England-specific; OMOP is person-centric and designed for international interoperability.

The core conceptual mappings are:

COSD Field	OMOP Table	Notes
primaryDiagnosisIcd (ICD-10)	CONDITION_OCCURRENCE	Standard ICD-10 mapping via OMOP vocabulary
morphologyIcdO3	CONDITION_OCCURRENCE	Histology; ICD-O-3 to SNOMED mapping required
treatments[].primaryProcedureOpcs	PROCEDURE_OCCURRENCE	OPCS-4 has limited standard OMOP coverage
treatments[].primaryProcedureSnomedCt	PROCEDURE_OCCURRENCE	Direct SNOMED mapping preferred where available
tnmStageGroupingFinalPretreatment	OBSERVATION	No standard OMOP table for staging
performanceStatusAdult (ECOG)	OBSERVATION	ECOG maps to LOINC in OMOP vocabularies
laboratoryResults[].ldh	MEASUREMENT	Lab results map cleanly to MEASUREMENT
personBirthDate	PERSON.birth_datetime	Direct
ethnicCategory (ONS codes)	PERSON.ethnicity_concept_id	ONS → OMOP concept mapping required

The most common implementation challenge is OPCS-4 concept resolution. OPCS-4 is not well-covered in standard OMOP vocabulary releases, which means custom concept ranges (2,000,000,000+) and local concept mappings are typically required. Capturing both primaryProcedureOpcs and primaryProcedureSnomedCt wherever possible mitigates this, since SNOMED CT maps cleanly to standard OMOP concepts.

Information Governance

COSD carries the most sensitive category of personal health data — identifiable, detailed cancer diagnoses, treatments, and outcomes. The IG framework is correspondingly rigorous.

Data classification. COSD data is Special Category Personal Data under UK GDPR (Article 9 health data). Processing requires an Article 9(2) condition — typically (h) for healthcare provision or (j) for research/public health. NHS Number combined with date of birth and postcode creates significant re-identification risk even without a name.

Technical controls. Minimum requirements for a production COSD platform: TLS 1.3 in transit across all NiFi pipelines; AES-256 encryption at rest in the data vault; role-based access control with least-privilege principles; column-level security on NHS Number and date of birth; and full audit logging on all access to identifiable data.

Regulatory framework. The overlapping requirements include: UK GDPR / Data Protection Act 2018; NHS Data Security and Protection Toolkit; DCB0129/DCB0160 clinical safety; Data Sharing Agreements with provider Trusts; TRUD per-product licence terms; and Confidentiality Advisory Group (CAG) approval where applicable for research uses.

Key Implementation Considerations

Version governance. Track which COSD version each environment (dev, test, prod) supports. Subscribe to TRUD release notifications and plan schema upgrade projects proactively rather than reactively. A structured version upgrade runbook — diff, mapper review, regression test, staged rollout — should be a standing procedure.

Validation feedback loop. Surfacing clear, field-level validation errors back to submitting Trusts is as important as the validation itself. A Trust that receives opaque failure messages will resubmit identical broken XML. Actionable error messages reduce resubmission cycles and improve data quality at source.

Completeness monitoring. COSD defines mandatory, required-if, and optional fields. Build completeness dashboards tracking: percentage of records with valid staging data, percentage with full treatment detail, completeness of demographic fields, and gap rates on key prognostic indicators (performance status, Ki-67, TNM edition). These dashboards serve both data quality management and clinical audit purposes.

Mapping as a single source of truth. The mapper script should be the single authoritative document for how your organisation interprets COSD. Any schema version upgrade must trigger a formal mapper review, not just a validation pipeline update. Field semantics can change between versions even when element names stay the same.

Summary

COSD is the definitional standard for cancer intelligence in England — not just a data format but a nationally mandated clinical ontology governing how cancer events are recorded, coded, and reported. Getting the implementation right matters beyond the technical: the accuracy of England’s national cancer register, the reliability of pathway performance metrics, and the validity of research using this data all depend on engineering decisions made in the ingestion pipeline.

The schemas are freely available from NHS TRUD. The record types are well-defined and clinically meaningful. The mapping from XML to a usable data model is tractable, if detailed. And the path from COSD to OMOP CDM, while requiring careful vocabulary work, is well-understood.

Every field you handle correctly is a contribution to the accuracy of cancer data in England.