Research on research: systematic reviews for retinal and vitreous conditions are classified as not reliable

Researchers at the Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, have reported that systematic reviews for retinal and vitreous conditions were deemed to be “as not reliable”.  Just under 60% of systematic reviews were judged to be inadequate and a primary concern indicates that there is a lack of a comprehensive literature search to support the required evidence-base.  The research report, published at JAMA Ophthalmology (Oct. 10, 2019), comments that there is a failure to conduct a comprehensive search, leading to an omission of a potentially large proportion of studies and this may impoverish the quality of data for patients.


According to the study, 327 systematic reviews of interventions for retina and vitreous

conditions were assessed. Out of these, 131 systematic reviews were classified “as reliable” but 196 systematic reviews were classified “as not reliable based on our criteria”. The most common unmet criterion was the lack of a comprehensive literature search for eligible studies (149 [76.0%]), followed by the lack of a risk of bias assessment of individual studies included in the review. (98 [50.0%]). In age-related macular degeneration, some researchers found that approximately 1 in 4 systematic reviews were not reliable. By way of example, a systematic review team may have searched multiple databases, but did not use a comprehensive search strategy or inappropriately limited results by date and language.  Criteria for reliable systematic reviews regarding a comprehensive literature search includes: “(1) Described an electronic search of ≥2 bibliographic databases, (2) used a search strategy comprising a mixture of controlled vocabulary and keywords, (3) reported using at least 1 other method of searching such as searching of conference abstracts, identified ongoing trials, complemented electronic searching by hand search methods (eg, checking reference lists), and contacted experts or authors of included studies”.  Lacking these steps will miss the necessary evidence-base and this will impact on healthcare.


The 7 conditions targeted for updating in the American Academy of Ophthalmology Retina/Vitreous PPP (Preferred Practice Pattern) include: (i) age-related macular degeneration; (ii) diabetic retinopathy, including diabetic macular edema; (iii) idiopathic epiretinal membrane and vitreomacular traction; (iv) idiopathic macular hole; (v) posterior vitreous detachment, retinal breaks, and lattice degeneration; (vi) retinal and ophthalmic artery occlusions; and (vii) retinal vein occlusions.  If the systematic review process on these 7 conditions is not corrected, there will be wasted resources, money, time and effort from research across the global community.  Most of all, this unreliable effort will ultimately harm patients.