Objective: The number of mobile applications addressing health topics is increasing. Whether these apps underwent scientific evaluation is unclear. We comprehensively assessed papers investigating the diagnostic value of available diagnostic health applications using in-built smartphone-sensors. Methods: Systematic Review - Medline, Scopus, Web of Science inclusive Medical Informatics and Business Source Premier (by citation of reference) were searched from inception until December 15th, 2016. Checking of reference lists of review articles and of included articles complemented electronic searches. We included all studies investigating a health application that used in-built sensors of a smartphone for diagnosis of disease. The methodological quality of 11 studies used in an exploratory meta-analysis was assessed with the QUADAS-2 tool and the reporting quality with the STARD statement. Sensitivity and specificity of studies reporting two-by-two tables were calculated and summarized. Results We screened 3’296 references for eligibility. Eleven studies, most of them assessing melanoma screening apps, reported 17 two-by-two tables. Quality assessment revealed high risk of bias in all studies. Included papers studied 1’048 subjects (758 with the target conditions and 290 healthy volunteers). Overall, the summary estimate for sensitivity was 0.82 (95 % confidence interval (CI); 0.56 to 0.94) and 0.89 (95 %CI; 0.70 to 0.97) for specificity. Conclusions The diagnostic evidence of available health apps on Apple’s and Google’s app stores is scarce. Consumers and healthcare professionals should be aware of this when using or recommending them.