Ruiz Díaz et al. BMC Health Services (2020) 20:354RESEARCH ARTICLEOpen AccessPatient satisfaction with remote monitoringof cardiac implantable electronic devices:the VALIOSA questionnaireMiguel A. Ruiz Díaz1*, Marta Egea García2, Roberto Muñoz Aguilera3, Xavier Viñolas Prat4, Jorge Silvestre García5,María Álvarez Orozco2, José Martínez Ferrer6 and On behalf of the VALIOSA study groupAbstractBackground: Remote monitoring of cardiac implantable electronic devices (CIEDs) has demonstrated substantialbenefits. Treatment guidelines have therefore endorsed its use and is being increasingly adopted in the clinicalsetting, but the level of satisfaction they convey remains still unknown. We developed and validated aquestionnaire to measure patient satisfaction with remote monitoring using Medtronic CareLink Network andassessed its internal reliability and dimensional validity.Methods: After a thorough literature review, cognitive debriefing of 18 patients, and an expert panel discussion, a30-item instrument was proposed and grouped into 5 dimensions (items): 1- Information on cardiac condition (3),2- Device convenience (3), 3- Transmission process (6), 4- Satisfaction with medical monitoring (8), and 5- Generalopinions (10). Correlation with the visual analog scale (VAS), overall health related quality of life (HRQoL) measuredby the EuroQoL-5D accompanied by the VAS as well as with the Medical Outcomes Study (MOS) SF-36 wereassessed. Psychometric properties, exploratory factor analysis and a second order confirmatory factor analysis (ahierarchical CFA with a general common factor explaining the relations between the first order common factors,See Figure 1) were estimated. Models were assessed based on item loading size, sign and statistical significance,and goodness-of-fit statistics.Results: A total of 186 patients (77% male) with a mean age of 66.03 (SD 13.94) years were assessed. 48% hadimplantable cardioverter-defibrillators, 24% had pacemakers, and 29% had cardiac resynchronization therapydevices. An overall Cronbach’s α 0.893 was achieved, with acceptable reliabilities for isolated dimensions.Correlations with corresponding VAS scales were meaningful and significant (p 0.01). The second order factorsolution yielded good goodness-of-fit indexes (χ2/df 1.44, CFI 0.96, TLI 0.95, RMSEA 0.05). Satisfaction withremote monitoring was not related to HRQoL (r 0.05), except for the correlation between the SF-36 mentalcomponent and the information on cardiac condition dimension (r 0.263, p 0.001).Conclusions: The 30-item questionnaire showed good reliability and validity to assess satisfaction with remotemonitoring in patients with CIEDs.Keywords: Patient satisfaction, Patient reported outcomes, Remote monitoring, Cardiac implantable electronicdevices, Patient health questionnaire* Correspondence: [email protected] of Methodology, School of Psychology, Universidad Autónomade Madrid, Madrid, SpainFull list of author information is available at the end of the article The Author(s). 2020 Open Access This article is licensed under a Creative Commons Attribution 4.0 International License,which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you giveappropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate ifchanges were made. The images or other third party material in this article are included in the article's Creative Commonslicence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commonslicence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtainpermission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.The Creative Commons Public Domain Dedication waiver ) applies to thedata made available in this article, unless otherwise stated in a credit line to the data.
Ruiz Díaz et al. BMC Health Services Research(2020) 20:354BackgroundTechnology available for managing cardiac implantableelectronic devices (CIED) has advanced considerably. Inthis sense, remote monitoring (RM) represents a complement to routine in-office care that provides access toa tremendous wealth of information recorded and storedby CIEDs such as device performance, including historyof cardiac arrhythmias, battery and lead parameterswithout face-to-face interaction. Remote monitoringenables remote feedback from device to physician andallows for continuous remote monitoring, which improves patient safety and care by allowing any deviceproblems to be detected immediately . In addition toclinical benefits, RM has been shown to allow for longerintervals between in-office visits and a shorter visit duration as querying the device is no longer necessary, thuslightening the burden of in-hospital follow-up of patientswith implanted devices, with the consequent cost reduction [2, 3] Thanks to demonstrated substantial benefits,treatment guidelines have endorsed its use for all eligiblepatients and RM is being increasingly adopted in theclinical setting .Despite the existing clinical and economic evidenceavailable, patient reported outcomes have received littleattention, even though they are expected to play a moreprominent role in assessing performance and determiningthe comparative effectiveness of treatment alternatives, inpart because of a growing emphasis on patient-centeredcare and value-based procurement initiatives.In particular, patient satisfaction represents an important measure of the extent to which a patient is satisfiedwith all aspects of healthcare delivery that are of relevance to health, which could be related with the qualityof care patients receive. It has been shown that patientsatisfaction affects patients’ health-related decisions andtreatment-related behaviors (like appropriate use of services, correct medication use or treatment continuation),and consequently, impacting substantially the success oftreatment outcomes .To date, few publications evaluating patient satisfaction associated with RM of patient with CIEDs havebeen published. Large registries like ALTITUDE andPREDICT-RM did not include patient reported outcomes [6, 7]; besides these registries, certain observational studies have considered measures of health status,satisfaction and/or acceptation, remote patient monitoring experiences, and preferences for follow-up but usinga self-designed questionnaire that did not undergo astrict validation process [8–11]. There is currently aEuropean randomized controlled trial ongoing (REMOTE-CIED) designed to examine the patient perspective of RM but it only covers patients with implantablecardioverter-defibrillators (ICD) and cardiac resynchronization therapy devices (CRT) .Page 2 of 9Despite the increasing interest in patient opinions andconcerns, to date there is no validated questionnaire specifically designed to measure patient satisfaction withCIED RM systems. We believe that patient satisfactionmeasurement is not only a compromise with qualitymanagement for health service providers  but also aneed towards identifying problematic users, and particularly among those who do not make automated transmissions (like pacemaker users).The aim of the present study was to develop a newquestionnaire with proven psychometric properties thatmeasures patient satisfaction with RM in patients withCIEDs, focusing on a specific RM system, CareLink Network (Medtronic, Minneapolis, MN), currently used bymore than 1,300,000 patients from more than 1200 healthcenters worldwide in about 80 different countries .MethodsThe present research was designed as an observational,two-stage, cross-sectional, multicenter study to developa specific instrument for the measurement of patient satisfaction with RM of CIED. Patients with implantedpacemakers (PM), ICD, and CRT devices who had beenusing CareLink Network for at least 2 months beforerecruitment were included. The study was conductedunder actual treatment conditions for their disorder inclinical practice.The Medtronic CareLink Network is the Internetbased remote monitoring service for patients with Medtronic implanted cardiac devices. It allows patients aconvenient timely connection to their clinic using theperson’s monitor/ smartphone/ tablet to collect andtransmit device data. A hand-held smart reader is usedto collect data from the implanted device. The smartreader communicates with the monitor and, through asoftware application, it transfers the data to the CareLink Network. All the information is recorded and reviewed bythe nurses through the online platform. In case there is acritical event, the patient is called by the medical staff tovisit the physician (Figure 6 Additional file 3).The study was performed following the HelsinkiTokyo-Venice guidelines for human research.Development phase: study design and participantsThe aim of the development phase was to create and design the VALIOSA questionnaire incorporating the patient perspective.For that purpose, a thorough literature review and patient focus groups were conducted to identify relevantcontents and topics addressing treatment satisfactionand patient management.Two focus groups were held at two different hospitals.The first one took place at Hospital Universitario deAraba (Vitoria) with 10 patients and the other one at
Ruiz Díaz et al. BMC Health Services Research(2020) 20:354Hospital Infanta Leonor (Madrid) with 8 patients. Patientswere recruited at random from those with a scheduledvisit in the selected week and balancing for gender andage. The topics discussed included symptoms and diseaseburden, daily limitations, device-related discomfort, lifestyle changes, impact on social relationships, family andspecialized support, and appropriate follow-up. Patientcontributions were summarized following a semantic reduction process by two independent content specialists.As a result, a preliminary questionnaire of 52 items wasproposed.This raw version underwent a discussion and semanticrefinement process by an expert panel consisting of fourcardiologists, one psychometrician and one pharmacist.Redundant and inappropriate items were discarded andreplaced by equivalent items and was reduced to 37 items.This initial set of 37 items comprised the followingRM aspects: information on monitor use, training andconvenience of monitor use, problems with the transmission process, satisfaction with medical care and RM,and benefits and impact on activities of daily living.The reviewed version was administered to a pilot sample consisting of 10 patients to check comprehensionand acceptance, and the information gathered was usedto refine the items. The resulting version was composedof 30 items grouped into 5 dimensions (number of items):information (3), monitor convenience (3), monitor handling and transmission process (6), disease follow-up (7),general impression and benefits (11). A panel composedby 9 specialists (cardiologists, nurses and psychometricians) assessed content validity, scoring item adequacy formeasuring each content dimension and the unidimensional  and multidimensional  item-domain congruence indexes were computed.Validation phase: study design and participantsThe aim of the validation phase was to administer the 30item version to a representative sample of patients to checkitem fit to the proposed construct, check psychometricproperties of the preliminary questionnaire designed inthe development phase, and reduce the number of items(if necessary) in accordance with the proposed construct.PatientsThree samples of patients were recruited at 4 hospitalsin 3 regions of Spain. A pilot sample of 10 patients wasrecruited to check item comprehension and to identifypossible response problems. Finally, a sample of 186patients was recruited to measure psychometric properties. All patients had to meet selection criteria: beingolder than 18 years of age, being implanted with a CIED,having more than 2 months of experience using RM,being able to understand and answer questionnaires inSpanish, and having no cognitive impairment.Page 3 of 9Sample size was determined based on the anticipatednumber of dimensions, the number of items per dimension (3:1) [17, 18], the number of patients per item (4:1),[19, 20] and the sample representativeness (n 150) .A sample size of 180 patients with complete informationwas deemed necessary.Psychometric properties of final versionDimension and overall scale reliability (internal consistency)was estimated by computing Cronbach’s alpha and 95%confidence interval intra-class correlation coefficient (ICC)for internal consistency.To test the construct validity of the questionnaire, aconfirmatory factor analysis was performed including allitems, assuming a second order factor structure withitems as indicators of the proposed first order dimensions and with all dimension loading in an overall satisfaction second order dimension. The robust weightedleast squares estimation method was used.The following psychometric properties were studied.Feasibility: response time, floor-ceiling effect, and missing values distribution; Internal Consistency/Reliability:Cronbach’s alpha, item correlations, adjusted item-totalcorrelation, ICC and omega ; Construct Validity:Confirmatory factor analysis; Concurrent Validity: Correlation between the new questionnaire (VALIOSA) and theVAS measures; Convergent and construct Validity: Correlation between similar/dissimilar dimensions of the VALIOSA,MOS SF-36 dimensions and corresponding VAS measures.InstrumentsThe final refined questionnaire, named VALIOSA (seeAdditional files 1 & 2), was included in a clinical recordform along with questions about patient socia