CA Baseline Maturity Level Definitions Discussion

We finished the last of the six maturity levels on today's governance call as well as made a few additions regarding process for community feedback in earlier levels (additions in red). The table will be incorporated into the Simplifier intro page and the CI Build IGuide shortly.

The updated deck can be found here:
infocentral.infoway-inforoute.ca/en/resources/docs/hl7/canadian-core-profiles/3743-ca-baseline-governance-call-baseline-maturity-level-definition-discussion-slide-deck-2021-08-06

Level	Current Definitions	Proposed CA Baseline Definition
0	FHIR Base Definition: the artifact has been published on the current build. This level is synonymous with Draft. UK Core Definition: The Profile has been published on the current build. This Profile has had no formal review and therefore may have quality issues. It is published only to allow the review process to start	The artifact has been published on the current build. This level is synonymous with Draft. Implementors should be aware that artifacts at this maturity level are subject to community review and refinement.
1	FHIR Base Definition: FMM0 + the artifact produces no warnings during the build process and the responsible WG has indicated that they consider the artifact substantially complete and ready for implementation. For resources, profiles and implementation guides, the FHIR Management Group has approved the underlying resource/profile/IG proposal. UK Core Definition: The Profile produces no warnings during the build process and has had a formal internal review by the UK Core development team	The artifact produces no warnings during the build process and has had a formal internal review by the CA Baseline Working Stream. The artifact is considered substantially complete and ready for trial use. Profiles at this level are available for the community to experiment with, in developing their own profiles, in order to solicit implementor feedback. Implementors should be aware that artifacts at this maturity level are still subject to community review and refinement.
2	FHIR Base Definition: FMM1 + the artifact has been tested and successfully supports interoperability among at least three independently developed systems leveraging most of the scope (e.g. at least 80% of the core data elements) using semi-realistic data and scenarios based on at least one of the declared scopes of the artifact (e.g. at a connectathon). These interoperability results must have been reported to and accepted by the FMG UK Core Definition: The Profile has been released for review to the UK FHIR community, any feedback received has been addressed as far as possible	The artifact has been released for review to the CA FHIR community, and is being used and/or demonstrated by at least three implementations (e.g., pan-Canadian, jurisdictional, vendor, etc.) from at least two different jurisdictions. Implementer feedback has been addressed as far as possible and is socialized through the Canadian Governance Collaborative. Implementors should be aware that artifacts at this maturity level are still subject to community review and refinement.
3	FHIR Base Definition: FMM2 + the artifact has been verified by the work group as meeting the Conformance Resource Quality Guidelines; has been subject to a round of formal balloting; has at least 10 distinct implementer comments recorded in the tracker drawn from at least 3 organizations resulting in at least one substantive change UK Core Definition: The Profile has been presented for inclusion in the Technical and Clinical Assurance process	The artifact has been verified by the work group as meeting the CA Baseline Quality Guidelines; has been subject to a round of formal balloting; Implementation guides at this maturity:have at least 10 distinct implementer comments submitted to [location tbd], provided by at least 3 organizations across three jurisdictions resulting in at least one substantive change Resources at this level of maturity: have at least 3 distinct implementer comments submitted to [location tbd] across at least three jurisdictions Requests for changes from this level onwards are expected to follow an evaluation & consultation process with the Canadian Governance Collaborative.
4	FHIR Base Definition: FMM3 + the artifact has been tested across its scope (see below), published in a formal publication (e.g. STU), and implemented in multiple prototype projects. As well, the responsible work group agrees the artifact is sufficiently stable to require implementer consultation for subsequent non-backward compatible changes. UK Core Definition: The Profile has completed the Technical and Clinical Assurance process and the status has been changed to active	FMM3 + the artifact has been tested across its scope (see below), published in a formal publication (e.g. STU) that FHIR projects have referenced/utilized across multiple prototype projects across at least three jurisdictions As well, the FHIR Implementors work group agrees the artifact is sufficiently stable, and that any future non-backward compatible [breaking] changes will go to the Canadian FHIR community for consultation.
5	FHIR Base Definition: FMM4 + the artifact has been published in two formal publication release cycles at FMM1+ (i.e. STU level) and has been implemented in at least 5 independent production systems in more than one country. UK Core Definition: The Profile has been presented for inclusion in the HL7 UK Ballot Process.	Artifact is ready for inclusion in normative ballot in the HL7 Canada Ballot Process. At least 5 independent production systems, across 3 jurisdictions, have demonstrated* conformance to the CA Baseline Profile as a starting point. “Independent production systems” should be inclusive of the following implementor types: - Health Vendors (e.g., EHR, HIT, Consumer Health) - Jurisdictional Health Assets - Pan-Canadian/Federal Health Assets
6	FHIR Base Definition: FMM5 + the responsible work group and the FMG agree the material is ready to lock down and the artifact has passed HL7 normative ballot. UK Core Definition: The Profile has completed the HL7 UK Ballot process and is now deemed to be Normative.	The FHIR Implementors Group and the Canadian Governance Collaborative agree the material is ready to lock down and the artifact has passed HL7 Canada normative ballot.

On today's governance call we confirmed and completed the level 5 maturity level definitions (see below).

We also discussed what our demonstration expectations are for the purposes of maturing the CA Baseline artifacts.

While some instances may wish to prove conformance for other purposes outside of maturing the CA Baseline (e.g., proving conformance of tool for procurement criteria)... our demonstration requirement to mature the artifact should be focused on what best proves the Baseline's maturity in our Canadian ecosystem.

The group refined the following language for what demonstration means in the context of the artifacts maturity level: "*demonstrated could mean 1) profiles formally derive from the baseline using the baseDefinition and show no conformance errors , OR 2) profiles can be validated against the {BaselineProfileURL] and is conformant (produces no errors)"

We encourage the community to provide feedback prior to our next call August 6th. We will be tackling the last maturity level (level 6) as well as discussing expectations for collecting and addressing feedback/changes from the community (levels 2+) as well as approaches for our terminology capabilities (pointing to external value sets, shell value sets, free/open source servers).

The most up to date slide deck can be found here: infocentral.infoway-inforoute.ca/en/resources/docs/hl7/canadian-core-profiles/3720-ca-baseline-governance-call-baseline-maturity-level-definition-discussion-slide-deck-2021-07-09

Level	Current Definitions	Proposed CA Baseline Definition
5	FHIR Base Definition: FMM4 + the artifact has been published in two formal publication release cycles at FMM1+ (i.e. STU level) and has been implemented in at least 5 independent production systems in more than one country. UK Core Definition: The Profile has been presented for inclusion in the HL7 UK Ballot Process.	Artifact is ready for inclusion in normative ballot in the HL7 Canada Ballot Process. At least 5 independent production systems, across 3 jurisdictions, have demonstrated* conformance to the CA Baseline Profile as a starting point. “Independent production systems” should be inclusive of the following implementor types: - Health Vendors (e.g., EHR, HIT, Consumer Health) - Jurisdictional Health Assets - Pan-Canadian/Federal Health Assets

Apologies folks, I'm unable to paste our current progress in a chart / side-by-side format, so what follows is a comparison between the definitions for Maturity Level 5 from a) the existing sources we're looking at (Base, UK, AU), and b) our current synthesized definition for the CA Baseline + discussion questions and notes:

The updated version of the working document has been uploaded here: infocentral.infoway-inforoute.ca/en/resources/docs/hl7/canadian-core-profiles/3720-ca-baseline-governance-call-baseline-maturity-level-definition-discussion-slide-deck-2021-07-09

Level 5: Definitions from Existing Sources:

FHIR Base Definition: FMM4 + the artifact has been published in two formal publication release cycles at FMM1+ (i.e. STU level) and has been implemented in at least 5 independent production systems in more than one country
UK Core: The Profile has been presented for inclusion in the HL7 UK Ballot Process
AU Core: FMM 5 PLUS the artifact has been published in two formal publication release cycles at FMM1+ (i.e. Trial Use level) and has been implemented in at least 5 independent production systems.

Level 5: Working Definition for CA Baseline (+ discussion questions and notes)

Artifact is ready for inclusion in normative ballot in the HL7 Canada Ballot Process

For Continued Discussion:
-inclusion of some interoperability milestone - e.g., between at least one pairing of implementors
-do we need to define what combinations and numbers would be necessary for meeting this bar (e.g., implemented in at least 5 independent production instances deployed across at least X jurisdictions representing X implementor types (e.g., EHR vendors, jurisdictional health assets, mobile applications, etc.)?

July 9 2021 Discussion:

-Emphasize maturation
-5 production instances seems good to inherit
-Call out the implementor types explicitly; adds to the authority of the maturation process
-EHR Vendors
-Jurisdictional Health Assets
-(to continue to fill this out)

-Given it’s a baseline profile, what does ‘implementation’ look like?
-CA Baseline Profiles are meant to be used as the starting point for Canadian fhir profiles > expectation at Level 5 is that these derived profiles have been ‘implemented’ in whichever project they’re part of
-Perhaps revise wording of “implementation” and focus more on CA Baseline’s role as a “starting point” for other profiles
-“at least 5 independent production systems have used (derived from) the CA Baseline Profile as a starting point”

-Difficult to require every jurisdiction to have ‘derived’ a profile off of it
-3 has been the # jurisdictions used thus far
-Connecting the two metrics: 5 ind prod sys’s, across 3 jurisdictions
-(a project which is implemented across jurisdictions [e.g. RxIT] would cover all 3 jurisdictions with 1 project; thus, a CA Profile derived off of by this project would qualify across those jurisdictions)

Content generated from the July 09 Discussion will be worked into the Working Definition, and next session (July 23) will be an opportunity to further review the proposed wording

Level	Current Definitions	Proposed CA Baseline Definition
0	FHIR Base Definition: the artifact has been published on the current build. This level is synonymous with Draft. UK Core Definition: The Profile has been published on the current build. This Profile has had no formal review and therefore may have quality issues. It is published only to allow the review process to start	The artifact has been published on the current build. This level is synonymous with Draft. Implementors should be aware that artifacts at this maturity level are subject to community review and refinement.
3	FHIR Base Definition: FMM2 + the artifact has been verified by the work group as meeting the Conformance Resource Quality Guidelines; has been subject to a round of formal balloting; has at least 10 distinct implementer comments recorded in the tracker drawn from at least 3 organizations resulting in at least one substantive change UK Core Definition: The Profile has been presented for inclusion in the Technical and Clinical Assurance process	The artifact has been verified by the work group as meeting the CA Baseline Quality Guidelines; has been subject to a round of formal balloting; Implementation guides at this maturity: have at least 10 distinct implementer comments submitted to [location tbd], provided by at least 3 organizations across three jurisdictions resulting in at least one substantive change Resources at this level of maturity: have at least 3 distinct implementer comments submitted to [location tbd] across at least three jurisdictions
4	FHIR Base Definition: FMM3 + the artifact has been tested across its scope (see below), published in a formal publication (e.g. STU), and implemented in multiple prototype projects. As well, the responsible work group agrees the artifact is sufficiently stable to require implementer consultation for subsequent non-backward compatible changes. UK Core Definition: The Profile has completed the Technical and Clinical Assurance process and the status has been changed to active	FMM3 + the artifact has been tested across its scope (see below), published in a formal publication (e.g. STU) that FHIR projects have referenced/utilized across multiple prototype projects across at least three jurisdictions As well, the FHIR Implementors work group agrees the artifact is sufficiently stable, and that any future non-backward compatible [breaking] changes will go to the Canadian FHIR community for consultation.
5	FHIR Base Definition: FMM4 + the artifact has been published in two formal publication release cycles at FMM1+ (i.e. STU level) and has been implemented in at least 5 independent production systems in more than one country UK Core Definition: The Profile has been presented for inclusion in the HL7 UK Ballot Process	FMM4 + the artifact has been published in two formal publication release cycles at FMM1+ (i.e. STU level) and has been implemented across a diverse set of implementors* across at least X jurisdictions in order to be representative of usage across the Canadian Health IT ecosystem Artifact is ready for inclusion in normative ballot in the HL7 Canada Ballot Process For Continued Discussion: -inclusion of some interoperability milestone - e.g., between at least one pairing of implementors *Group is discussing if we need to define what combinations and numbers would be necessary for meeting this bar (e.g., implemented in at least 5 independent production instances deployed across at least X jurisdictionsrepresenting X implementor types (e.g., EHR vendors, jurisdictional health assets, mobile applications, etc.)?

We'll be centralizing any future discussion of the CA Baseline Artifact Maturity Levels in this thread - prior discussion related to the call a month ago can still be found here: infocentral.infoway-inforoute.ca/en/forum/266-fhir-implementations/3891-ca-baseline-governance-stream-meeting-friday-may-28th-2-3pm-est#6874

Last Friday - We bolstered the definitions in levels 0-2 to include what implementers could expect in terms of profile stability and got part of the way through level 3 (see table below).

We looked at the process the UK Core and the Netherlands National Projects employ for validating both their national profiles and derivative profiles. Both groups have developed an internal technical assurance review process for nominated profiles rather than a more conventional balloting process.

We'll be continuing on level 3 & 4 next governance call, starting by confirming what the # of unique jurisdictions providing feedback are reasonable to expect for profiles emerging from level 3 to level 4.

We also will be discussing what our expectations are for tooling regarding demonstrating conformance to terminology (e.g., if a full terminology server is needed, if Simplifier packaging or the FHIR validator is enough for the capabilities we are looking to support.) We anticipate this may have a larger discussion so we're encouraging folks to let us know if they are interested in having some off-Friday discussions/have an opinion in this space.

Level	Current Definitions	Proposed CA Baseline Definition
0	FHIR Base Definition: the artifact has been published on the current build. This level is synonymous with Draft. UK Core Definition: The Profile has been published on the current build. This Profile has had no formal review and therefore may have quality issues. It is published only to allow the review process to start	The artifact has been published on the current build. This level is synonymous with Draft. Implementors should be aware that artifacts at this maturity level are subject to community review and refinement.
1	FHIR Base Definition: FMM0 + the artifact produces no warnings during the build process and the responsible WG has indicated that they consider the artifact substantially complete and ready for implementation. For resources, profiles and implementation guides, the FHIR Management Group has approved the underlying resource/profile/IG proposal. UK Core Definition: The Profile produces no warnings during the build process and has had a formal internal review by the UK Core development team	The artifact produces no warnings during the build process and has had a formal internal review by the CA Baseline Working Stream. The artifact is considered substantially complete and ready for trial use. Profiles at this level are available for the community to experiment with, in developing their own profiles, in order to solicit implementor feedback. Implementors should be aware that artifacts at this maturity level are still subject to community review and refinement.
2	FHIR Base Definition: FMM1 + the artifact has been tested and successfully supports interoperability among at least three independently developed systems leveraging most of the scope (e.g. at least 80% of the core data elements) using semi-realistic data and scenarios based on at least one of the declared scopes of the artifact (e.g. at a connectathon). These interoperability results must have been reported to and accepted by the FMG UK Core Definition: The Profile has been released for review to the UK FHIR community, any feedback received has been addressed as far as possible	The artifact has been released for review to the CA FHIR community, and is being used and/or demonstrated by at least three implementations (e.g., pan-Canadian, jurisdictional, vendor, etc.) from at least two different jurisdictions. Implementer feedback has been addressed as far as possible. Implementors should be aware that artifacts at this maturity level are still subject to community review and refinement.
3	FHIR Base Definition: FMM2 + the artifact has been verified by the work group as meeting the Conformance Resource Quality Guidelines; has been subject to a round of formal balloting; has at least 10 distinct implementer comments recorded in the tracker drawn from at least 3 organizations resulting in at least one substantive change UK Core Definition: The Profile has been presented for inclusion in the Technical and Clinical Assurance process	The artifact has been verified by the work group as meeting the CA Baseline Quality Guidelines; has been subject to a round of formal balloting; Implementation guides at this maturity: has at least 10 distinct implementer comments recorded in the tracker drawn from at least 3 organizations and three jurisdictions resulting in at least one substantive change. Resources at this level of maturity: has at least X distinct implementer comments recorded in the tracker drawn from at least X organizations and three jurisdictions resulting in at least X substantive change .

Edits have been made and are reflected in most updated version of the ppt deck, accessible here: infocentral.infoway-inforoute.ca/en/resources/docs/hl7/canadian-core-profiles/3701-ca-baseline-governance-call-baseline-maturity-level-definition-discussion-slide-deck-2021-06-11