Objectives: Gain an overview of expected response rates (RRs) to patient-reported outcome measures (PROMs) in clinical quality registry-based studies and long-term cohorts in order to better evaluate the validity of registries and registry-based studies. Examine the trends of RRs over time and how they vary with study type, questionnaire format, and the use of reminders.

Design: Literature review with systematic search.

Data sources: PubMed, MEDLINE, EMBASE, kvalitetsregistre.no, kvalitetsregister.se, and sundhed.dk.

Eligibility criteria: Articles in all areas of medical research using registry-based data or cohort design with at least two follow-up time-points collecting PROMs and reporting RRs. Annual reports of registries including PROMs that report RRs for at least two time-points.

Primary outcome measure: Response rates to PROMs.

Results: A total of 10 articles, 12 registry reports, and 6 registry articles were included in the review. The overall RR at baseline was 75% ±22.1 but decreased over time. Cohort studies had a markedly better RR (baseline 97% ±4.7) compared to registry-based data at all time points (baseline 72% ±21.8). For questionnaire formats, paper had the highest RR at 86% ±19.4, a mix of electronic and paper had the second highest at 71% ±15.1, and the electronic-only format had a substantially lower RR at 42% ±8.7. Sending one reminder (82% ±16.5) or more than one reminder (76% ±20.9) to non-responders resulted in a higher RR than sending no reminders (39% ±6.7).

Conclusions: The large variation and downward trend of RRs to PROMs in cohort and registry-based studies are of concern and should be assessed and addressed when using registry data in both research and clinical practice.

We obtained the data for this study through two systematic searches of peer-reviewed papers, as well as from annual reports from CQRs. The search protocol was adapted from Oglund et al. (21). Search 1 was performed on 11^th Apr, 2017 in PubMed (www.ncbi.nlm.nih.gov), and search 2 was performed on 12^th Dec, 2017 in MEDLINE and EMBASE through Ovid (www.ovid.com). Search terms can be found in Table 1 in the article.

Inclusion criteria

We included journal articles for studies conducted in humans after 01.01.1990 published in English and Scandinavian languages. Articles in all areas of medical research that used PROMs and reported RR for at least two time-points were included in order to understand change in RR over time. We were mainly interested in articles using registry-based data, but also included prospective cohort studies with a follow-up of at least one year, as we believe long-term cohorts have a similar logistical administration as registries. For registry-based articles, we accepted any follow-up time. We excluded randomised controlled trials (RCT) as the nature and logistics of an RCT do not reflect the logistics of a registry. We also excluded articles where children under 16 years were respondents, articles with patients unable to answer the PROMs themselves or where the PROMs were filled in at control appointments as these factors can affect the RR. In the case of studies with multiple articles for follow-up time points, all related articles reporting RR were included.

Registry search

As we did not identify a large number of journal articles meeting inclusion criteria in the systematic searches, we supplemented them with annual reports by CQRs. We searched through all CQRs in Scandinavia listed on each country’s national registry websites kvalitetsregistre.no, kvalitetsregister.se, and sundhed.dk, as well as all national knee ligament or joint registries known to the authors. We also searched any registries mentioned in articles from the previous systematic searches, as well as the list of published articles from each registry to identify articles with RRs that were not identified in our previous searches. If the RRs to PROMs were described in the annual report but an article reported a different RR, both were included.

Review

The articles were assessed by two independent researchers to ensure inclusion of all relevant articles. We first performed a title review, followed by an abstract review, and lastly, a full-text review. In search 1, these phases were performed in EndNote by both primary researchers. The results were assessed after completion and an agreement was made in case of any differences. In search 2, these phases were conducted in Covidence (www.covidence.org). The annual reports from CQRs were assessed by only one of the researchers. Due to the nature of the annual reports as describing the activity of a registry, we considered it unnecessary for two researchers to assess them. Risk of bias was not assessed as this was deemed not relevant for the main outcome measure.

Data extraction

Relevant data from the included articles was extracted by the same two researchers to ensure accuracy. Data from annual reports was extracted by one researcher as they were presented more clearly. Data extracted from the articles and reports include year of publication, lead author, format of questionnaires, number of reminders sent, and response rates and their time-points. We also collected data on which questionnaires were sent at each follow-up and if this changed between follow-ups, but this data was not readily available or specified clearly in all of articles and was therefore not included in this paper. Methods used to calculate RR were not explicitly stated in most of the articles and were therefore not collected. When subgroups within articles or reports had separate RRs reported, these were treated as separate subgroups instead of averaging the RR. For example, some studies reported separate RRs for different types of PROMs or for different patient groups. Where indicated, the RR was calculated by a researcher using the available numbers. The main outcome measure was RRs to PROMs. If the relevant data was not described in the article or report, an email was sent to one of the authors or the registry with a request for the missing information.

Statistics

The results are presented with descriptive statics (average ±SD) and figures using IBM SPSS Statistics 25. Microsoft Excel was used to visualise the data. No statistical tests were performed to compare the values due to the heterogeneity in the reporting of RRs. No meta-analysis was conducted.