Search terms are determined and the protocol is defined
- The process begins with determining the search terms (more details provided in section 2.0). The search protocol is then drafted which lays down the path forward to carry out the literature review process including information such as the search timeline, search database, inclusion/exclusion criteria etc.
Searches are run and the results are compiled for review
- In this phase, all search terms are run across all chosen databases and the results are prepared for review.
The main steps are:
- Run search terms on all databases
- Download results
- Remove (and count) duplicates for each search.
- Prep abstracts, citations, titles, and full texts (if available freely) for review.
- Note* The above steps are completed with the assistance of CiteMed’s Literature Review System. Such efforts can be duplicated manually if the client ever desires
The abstract review takes place
- In this phase (1st Pass Screening), all of the abstracts (minus the duplicates) will be reviewed by the CiteMed Reviewer.
- Sufficient information on the relevance of the article based on the Abstracts and Title review comprises the bulk of the review. In this step, the reviewer will decide to exclude articles that are clearly meeting the exclusion criteria. If the reviewer could not take a decision based on the information available in the abstract and title, the article will be moved to the next pass (2nd pass screening) for full-text review.
- However, some full texts are available during this review for particularly challenging decisions, these articles will be moved to 2nd pass screening.
- For each article reviewed in this step, there are only two possible outcomes; include or exclude
- The included article will move for further review (Long Form Extraction Field Table)
- The article is Excluded with a descriptive reason (reasons defined in Search Protocol).
Full texts are retrieved and organised for review
- In this step, a list is compiled of all Included articles (from the previous step). The CiteMed team will then source freely or purchase the Full Text of each article.
- Below is an illustration of the process of article review:
Data extraction from retained articles occur
- For Basic Project: In this step, each Included article is reviewed at the Full-Text level. Detailed ‘Extraction’ criteria are filled in for each article. Data Extraction is only applied to Retained Articles that are Related to the Device Under Evaluation (DUE).
- For Standard Project: In this step, each Included article is reviewed at the Full-Text level. Detailed ‘Extraction’ criteria are filled in for each article. Data Extraction is applied to Retained Articles that are Related to the Device Under Evaluation (DUE) and Similar devices. The safety and performance criteria will be fixed based on similar device data extracted in the Long Form Extraction Field Table.
- For Comprehensive Project: In this step, each Included article is reviewed at the Full-Text level. Detailed ‘Extraction’ criteria are filled in for each article. Data Extraction is only applied to Retained Articles that are Related to the Device Under Evaluation (DUE) and Similar devices. The safety and performance criteria will be fixed and a complete SOTA section will be developed covering (i) the Disease Background, (ii) the Alternative Therapy/Option and (iii) the Similar Devices sections.
Appraisal of literature is performed and classifications (GRADE etc.) are applied
- The literature is appraised in terms of its suitability for inclusion in the evaluation and data contribution in accordance with MEDDEV 2.7/1, Rev.4
Suitability and Outcomes
These criteria are taken from the MEDDEV 2.7.1 Rev 4 standards of Clinical Evaluation.
Quality and Contribution Questions (Yes/No)
- These are Yes/No Questions Answered for every Included article.
- Was the design of the study appropriate?
- Do the outcome measures reported reflect the intended performance of the device?
- Is the duration of follow-up long enough to assess whether the duration of treatment effects and identify complications?
- Has a statistical analysis of the data been provided and is it appropriate?
- Is the size of the study appropriate for statistical analysis?
- Was the magnitude of the treatment effect observed clinically significant?
- Study Conclusions
Long-Form Extraction Fields
These are where the reviewer will pull out relevant info from the article to support the specific category.
We intentionally keep these fields fluid, as our primary goal is to extract data from articles that support the device’s indications. However, in the case of CMR, indications are not the primary driver for this review, and thus a more data-intensive approach (detailed in the below section) will be used.
- Safety: Any acknowledgment of how the device safety claim(s) was(were) described for patients.
- Performance: Any acknowledgment of how the device performance claim(s) was(were) described for patients (I.e.to improve patient outcome, procedural outcome, long-term outcome)
- Adverse Events: Patient-reported AEs
- State of the Art: Describe any (predefined) SOTA procedures used within the article.
- Guidance: List any guidance used for patient assessment, procedural assessment
- Other (Commentary): General field to describe potential new information and the Devices used in the study
- Study Design: Type of study (e.g., Case Series, Randomized, Placebo-Controlled, Double-Blind) including any follow-up.
- Total Sample Size: Number of patients by group evaluated. Also includes Sex, Age (mean, percent, range as available)
- Objective: Describe the primary objective of the article/ study
- Treatment Modality: List all treatment arms (I.e., what was tested or compared)
- Study Conclusions: Briefly describe study results/outcomes/main conclusions
- Justification (Writeup): This is a general overview of the information gathered that provides support to the device's claims, capabilities, and overall safety.
- Custom Extraction Fields: At the clients request, additional extractions fields can be chosen to highlight specific indications they are attempting to support.
Additional Classifications (GRADE and SoTA) are Applied
Either in parallel or shortly after Extractions are completed on each article, several additional classifications criteria are determined.
SoTA
If an article is determined to be relevant to the State of the Art Determination, a SoTA Classification will be applied.
GRADE
If the article is determined to be relevant to the target device of the search, a GRADE score will be calculated in the following way.
- Note* The basics of GRADE can be found freely online https://bestpractice.bmj.com/info/toolkit/learn-ebm/what-is-grade/
Initial Grade of Study Score
Score Adjustments
Final Grade Score Formula
GRADE SCORE= Grade of Study score + Risk of bias + Observational study imprecision + RCT inconsistency + Observational study indirectness + RCT study limitations