EQUAL CARE® Certification
AI-Supported Diagnostic / Treatement
Evaluation Criteria
Evaluation Items
Evaluation Description
Methodology
Study Representation
Evaluation Description
Sufficient gender representation in comparison to prevalence in the population present in foundational research.
Methodology
Ensure that foundational research includes a gender representation that reflects the prevalence in the population. Companies must provide data demonstrating this balance in their studies, showing that both genders are adequately represented in clinical trials. Deviation up to 25 percentage point will be accepted since exact recruitment can be associated with disproportional efforts in resource and time. In the future we aim to lower this standard.
Training Data Quality
Evaluation Description
Sufficient gender representation in comparison to prevalence in the population. Information on the origin and quality of the training data. Transparency in data preprocessing and applied transformations. Investigation and documentation of possible biases in the data and their impact on the model.
Methodology
Companies must: Ensure gender representation in their training data matches the population prevalence as closely as possible. Provide detailed information on the origin and quality of the training data. Maintain transparency in data preprocessing and any transformations applied. Investigate and document potential biases in the data and their impacts on the model.
Validation Data Quality
Evaluation Description
Sufficient gender representation in comparison to prevalence in the population. Information on the origin and quality of the training data. Transparency in data preprocessing and applied transformations. Investigation and documentation of possible biases in the data and their impact on the model.
Methodology
Companies must: Ensure gender representation in their validation data matches the population prevalence as closely as possible. Provide detailed information on the origin and quality of the validation data. Maintain transparency in data preprocessing and any transformations applied. Investigate and document potential biases in the data and their impacts on the model.
Algorithm Adaptability
Evaluation Description
Assessment of the importance of gender-specific needs as input variables for the model's decisions are included.
Methodology
Assess and document the inclusion of gender-specific needs as input variables for the model's decisions. Companies must demonstrate how these needs influence model outcomes.
Efficacy/ Accuracy
Evaluation Description
Validation of the model’s performance as a whole and segregated by gender was done.
Methodology
Validate the model's performance as a whole and separately by gender. Companies need to provide evidence of this validation process and its results.
Transparency
Evaluation Description
Explainability of the hyperparameters used, the training dataset, and the training process.
Methodology
Ensure the explainability of the hyperparameters used, the training dataset, and the training process. Companies must provide clear documentation of these aspects.
Accessibility
Evaluation Description
Provide explanations in a way that is understandable even to non-experts.
Methodology
Provide explanations of the model and its processes in a manner understandable to non-experts. Companies should use clear and accessible language in their documentation.
Affordability
Evaluation Description
Costs associated with the application for Users, Healthcare Professionals and Organizations
Methodology
Describe the cost model associated with the application for users, healthcare professionals, and organizations. Companies must provide information, contact or reference.
Possible Side Effects
Evaluation Description
Evidence of any possible side effects or absences of them.
Methodology
Provide evidence of any possible side effects or the absence thereof. Companies need to document and present findings related to side effects.
Regulatory Compliance
Evaluation Description
Ensure that the algorithm meets relevant regulatory requirements and ethical guidelines.
Methodology
Ensure that the algorithm complies with relevant regulatory requirements and ethical guidelines. Companies must provide documentation of their compliance measures. References to GDPR, HIIPA, MDR, FDA, ISO 13485 and others.
Level of Evidence
Evaluation Description
Level of evidence for AI development, Level of evidence for AI validation
Methodology
Appropriate documentation and evidence to facilitate the certification process.