Access to subspecialty services, including pediatric neurology services, is important for the health and well-being of children. However, pediatric specialty services are often not easily available.1 Access to neurology care is particularly problematic for children of vulnerable social groups, or those living farther from tertiary care children’s hospitals.2 These problems in access to specialist services worsened during the COVID-19 pandemic, threatening the health of children.3
Even before the pandemic, telehealth has been considered as a solution to reduce problems in access to neurology services for children.2,4 A systematic review of randomized controlled trials showed telehealth was equal to usual care in clinical effectiveness and patient satisfaction and quality of life.5 Children living in rural areas who used pediatric neurology telehealth visits have lower emergency room (ER) visits and hospitalizations compared to those who received care through in-person visits.6 A recent study showed that pediatric neurology was one of the subspecialties with high telehealth use.3 Another study showed that telehealth use was higher for children compared to adults attending neurology clinics.7 However, there is a concern that telehealth services may not be available for children of vulnerable groups.8
At the beginning of the COVID-19 public health emergency, like many states in the country, North Carolina enacted policies to allow reimbursement of clinicians for telehealth services.9 This exponentially increased adoption of telehealth in North Carolina10 and provided an unprecedented opportunity to test whether use of telehealth had an effect on children’s access to neurology services. Therefore, the objective of our study is to evaluate whether children who used telehealth services had greater access to outpatient pediatric neurology services compared to those who did not use telehealth services. We used appointment outcome—completion, cancellation, and no-show rates—as a proxy for access to neurology services.
Methods
Setting
This study was conducted at Brenner Children’s Hospital (BCH), a tertiary care children’s hospital within the Atrium Health Wake Forest Baptist (AHWFB) system in Winston-Salem, North Carolina. The Institutional Review Board of Wake Forest University School of Medicine approved the study. We employed an observational cohort study design and used existing demographic and clinical encounter data for children available in the institution’s electronic health record (EHR). We followed the STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) checklist for cohort studies to report our study.11
Telehealth at BCH
Prior to March 2020, telehealth was not an option for children attending neurology clinics at BCH. In March 2020, in response to the COVID-19 pandemic, structures and processes to deliver telehealth services were created within the AHWFB system. The pediatric neurology department created a policy about encounters/conditions for which telehealth visits were appropriate for new patients. For return patient visits, appropriateness of telehealth was at the discretion of the neurology provider. Video visits were conducted using Doximity or through WebEx within the institution’s EHR, Epic (Epic Systems, Verona, WI).
Study Cohort
We used a clinical administrative database to identify children who had at least one pediatric neurology outpatient appointment scheduled between March 10, 2019, and March 9, 2020, at BCH. Using children’s medical record numbers, we queried the EHR to identify children eligible for the study. Inclusion criteria were: age < 18 years as of March 10, 2019, residence in North Carolina, and at least one pediatric neurology appointment scheduled between March 10, 2020, and March 9, 2021.
Pediatric Neurology Outpatient Encounters
Outpatient encounters were initially identified using search terms for pediatric neurology services within BCH’s pediatrics and neurology departments. Encounters were then limited to the 37 types of outpatient encounters relevant to pediatric neurology and provided by one of the 18 pediatric neurology providers (nine attending physicians, three advanced practice providers, and six resident physicians) practicing at BCH between March 10, 2019, and March 9, 2021.
Child-Level Data
Child-level data abstracted from the EHR included date of birth, race, ethnicity, health insurance type, and address. Encounter-level data included date of encounter, type of visit (in-person versus telehealth), and health insurance and address at the time of the encounter.
Encounter Data
An outpatient encounter was coded as a telehealth visit if any one of the 13 video or audio appointment codes was applied to the encounter. Data about outcome of appointments (completed, cancelled, or no-show) were abstracted. Duplicate encounters (i.e., same type of encounter in a day) were excluded. When there was more than one type of encounter on the same day (e.g., completed and cancelled), we used the following inclusion rule: completed appointment was considered superior to cancelled appointment, and cancelled appointment superior to no-show appointment.
Variables
Age, race, ethnicity, health insurance type, rural/urban residence, and distance from BCH were the demographic variables. Race was categorized as White or Caucasian, Black or African American, and Other. The Other category included American Indian, Asian, Latin American, Native Hawaiian, and other. Ethnicity was categorized as Hispanic and non-Hispanic. Health insurance was categorized into public and private. Public insurance included Medicaid, Medicare, Tricare, and other government programs. Private insurance included commercial insurance and managed care.
Geocoding. We used child’s address for residence (rural versus urban) and to calculate distance to BCH. We used data from a database developed from the American Community Survey based on 2015 geocoding at census-tract level. Rurality score for each address (0 to 1, with 0 being most rural and 1 being most urban) was calculated. A rurality score of > 0.5 was defined as urban, and a score of ≤ 0.5 as rural. Distance from BCH was calculated using coordinates for BCH and the child’s address, using a custom-built function within the local database developed by the Wake Forest Center for Biomedical Informatics that uses variables from the American Community Survey.12
Telehealth use was the independent variable. “Telehealth user” was defined as a child who had at least one telehealth appointment scheduled between March 10, 2020, and March 9, 2021. Completion, cancellation, and no-show rates were the outcome variables and were calculated as the proportion of scheduled appointments that were completed, cancelled, and no-show, respectively. “Time period” was categorized into pre-pandemic (March 10, 2019, to March 9, 2020) and pandemic (March 10, 2020, to March 9, 2021).
Data Analysis
Race, ethnicity, health insurance, and rural/urban residence were categorical variables. Age and household distance from BCH were continuous variables. For bivariate analysis, we used demographic characteristics of children at the time of their first scheduled appointment and compared these characteristics between telehealth-user and non-user groups. We used chi-square tests for categorical variables and t-tests for continuous variables.
To compare completion, cancellation, and no-show rates between telehealth user and telehealth non-user, and between pre-pandemic and pandemic periods, we used paired t-tests. For multivariate analysis, we tested two models: one comparing cancellation versus completion and another comparing no-show versus completion rates. Both models included demographic variables, the independent variable (telehealth user versus non-user), and the time-period variable (pre-pandemic versus pandemic). Demographic variables in the model were at the encounter level (time-dependent covariates). An interaction term between the time-period and telehealth-user variables was included in the models. In multivariate models, data for age, rural/urban status, distance from BCH, and health insurance at each encounter were used. Odds ratios for cancellation versus completion and no-show versus completion rates, adjusted for all other variables in the model, were calculated. Stratified analysis was performed for the interaction term (time-period and telehealth-user). A p-value of < .05 was considered statistically significant. For multivariate models, Generalized Estimating Equations using R package geepack13 was used.
Results
Of the 5097 children who had a neurology appointment scheduled in the pre-pandemic period (March 10, 2019, to March 9, 2020), 2110 (41%) had an appointment scheduled in the pandemic period (March 10, 2020, to March 9, 2021) as well and were included in the study (Figure 1). Demographic characteristics of children included in the study were as follows: mean age 9.8 years; 65% public insurance; 64% White; 14% Hispanic; 65% lived in an urban area; mean distance to the hospital 30 miles. Of these children, 1251 (59%) had at least one telehealth visit (telehealth user) and 859 (41%) had no telehealth visit (telehealth non-user) scheduled. These constituted 11,194 scheduled encounters.
Characteristics of Telehealth User versus Telehealth Non-User
Demographic characteristics of telehealth user and non-user are compared in Table 1. There was no difference between telehealth user and non-user in age, distance from the children’s hospital, and rural versus urban residence. Telehealth users were more likely to be White and non-Hispanic, and to have private health insurance.
Appointment Outcomes in the Pre-Pandemic and Pandemic Periods
Overall, outcomes of appointments were as follows: 57% completed, 35% cancelled, and 8% no-showed. In the pandemic compared to the pre-pandemic period, there was a statistically significant decrease in completion rate (66% versus 57%, p <.0001), and increase in cancellation rate (27% versus 33%, p < .0001) and no-show rate (7% versus 10%, p < .0001).
Completion rates were significantly higher during the pre-pandemic compared with the pandemic period for both groups: 67% versus 63% (p < .01) for telehealth users, respectively, and 66% versus 48% (p < .0001) for non-users, respectively (Table 2). Cancellation rates between the pre-pandemic and pandemic periods were not different for the telehealth user group (27% versus 28%, p = .6). However, the cancellation rate was significantly higher in the pandemic period for the non-user group: pre-pandemic versus pandemic (26% versus 41%, p < .0001). The no-show rate increased significantly in the pandemic period compared to the pre-pandemic period for both groups: pre-pandemic versus pandemic for the telehealth user group was 6% versus 10% (p < .0001); and for the non-user groups was 9% versus 11% (p = .02)
When comparing the telehealth users and non-users in the pre-pandemic period, there was no statistically significant difference between the groups in completion and cancellation rates (Table 3). However, in the pandemic period, there was a statistically significant increase in completion rate and decrease in cancellation and no-show rates in the telehealth user group compared to the non-user group (Table 3).
Multivariate Analysis
In the statistical model comparing cancellation versus completion rates, age and time period were significant variables, after adjusting for other variables (Table 4). Increasing age was associated with decreased odds of cancellation. The odds of cancellation were three times higher in the pandemic period compared with the pre-pandemic period. In the no-show versus completion rates model, age, race, health insurance, and distance from BCH were significant variables. With increasing age and distance from BCH, no-show rates decreased. Children of Black race compared with those of White race and those with public insurance compared to private insurance were twice as likely to have no-shows rather than completion. Ethnicity and rural/urban residence were not significant variables in either model.
Stratified analysis. The interaction term for telehealth-use and time-period variables was statistically significant for the cancellation versus completion rate model, but not for the no-show versus completion rate model. In Figure 2 we show the results of cancellation and no-show rates for telehealth users and non-users stratified by time period. Cancellation and no-show rates were not significantly different between telehealth users and non-users in the pre-pandemic period. However, in the pandemic period, cancellation rate was lower for telehealth user compared to non-user (34.7% versus 43.5%, p < .0001); no-show rate was lower for telehealth user compared to non-user (7.9% versus 10.6%, p =.01).
Discussion
Our study shows that children of minority race and ethnicity and those with public insurance were less likely to be telehealth users compared to their counterparts. Children who lived farther from the tertiary care children’s hospital were more likely to be telehealth users than those who lived closer. Completion rates for neurology appointments decreased significantly during the pandemic. We found telehealth users to have lower cancellation and no-show rates during the pandemic compared to non-users. This suggests the potential usefulness of telehealth in mitigating access problems for pediatric neurology services resulting from the pandemic.
In our study, telehealth users were more likely to be White and non-Hispanic, have private insurance, and live farther from the children’s hospital. Our findings of lower telehealth use in children of minority race and ethnicity are consistent with prior studies.3,14,15 Differences in telehealth use based on children’s race/ethnicity and insurance status are concerning, since they potentially widen the disparities in access to neurology services for these vulnerable groups. One possible reason for this is problems in access to the technology needed for telehealth, including broadband internet access.16 Disparities in telehealth use warrant further evaluation. Joshi and coauthors noted that telemedicine use was substantially lower for children living in rural communities.17 Interestingly, we did not find differences in telehealth use based on rural versus urban residence. On the other hand, children who lived farther from the children’s hospital were more likely to use telehealth in our study. Telehealth is an alternative to in-person visits when pediatric neurology services may be limited, such as for children living farther from a tertiary care children’s hospital.6
Our study showed that there was a significant drop in the number of children scheduled for a pediatric neurology visit during the pandemic compared to the year prior. Completion rates for pediatric neurology appointments also decreased substantially during the pandemic. Telehealth use helped mitigate these appointment problems. In North Carolina, telehealth use during the pandemic was associated with less need for follow-up clinic visits, supporting the benefit of telehealth visits to mitigate health care access problems.10 Previous studies have also reported lower ER visits, hospitalizations, and mortality in children using telehealth for pediatric neurology outpatient services.6,17 Additionally, parents and providers alike are satisfied with telehealth for pediatric neurology clinic visits.14,18 As health insurance companies are considering scaling back provision of telehealth offered during the pandemic, policymakers should consider the positive effect of telehealth in improving pediatric specialty access and health outcomes of children. It is encouraging that NC Medicaid has made telehealth flexibilities permanent.19 The effect of this policy change in improving access to specialty care for children should be evaluated in the future.
Cancellation and no-show rates increased significantly during the pandemic even after controlling for other factors, including telehealth use. In a prior study, no-show rate increased in subspecialty clinics that adopted telehealth.3 In our study, while telehealth use had a marked effect in reducing cancellation rate during the pandemic, it was not as pronounced for reducing no-show rates. Age was an important factor in pediatric neurology appointment completion. As child’s age increased, both cancellation and no-show rates decreased, suggesting parents of younger children may have been fearful of bringing their children for clinic visits during the pandemic. While race and insurance were not factors in the cancellation versus completion rate multivariate model, Black race and public insurance were significant factors in the no-show versus completion rate model. Prior studies have noted the higher number of missed clinic appointments for vulnerable groups including minority race/ethnicity and those using public insurance.20–22 Economic and logistic factors, mistrust in the health care system, and systemic racism have all been attributed as barriers to disadvantaged populations attempting to access health care services.8,15,21,23 The barriers to accessing pediatric neurology services faced by children in these vulnerable sociodemographic groups need to be investigated further.
There are certain limitations to our study. EHR data were limited to a single institution and could have missed children’s clinical encounters elsewhere. The definition we used for telehealth user has not been validated. Since this study used secondary data, we did not have other sociodemographic variables that could have affected appointment outcomes, such as transportation or parents’ preferred language. Since data were not collected for research purposes, it is possible that there were unmeasured confounders. Although race and ethnicity variables were associated with telehealth use and appointment outcomes, there are challenges in using race/ethnicity variables in EHR data as described before.24 Finally, since this is an observational study, the association between telehealth use and appointment outcomes do not indicate causation.
Conclusion
Completion rates for pediatric neurology appointments decreased substantially during the pandemic. However, telehealth use mitigated the problems with access to neurology services for children. Since children of minority race and ethnicity and those with public insurance are less likely to use telehealth, this puts them at increased risk for worsening access to neurology services when in-person visits are not available.
Acknowledgments
We thank Kwabena Sekyere (Programmer), Brian Ostasiewski (IT Program Director), and Larisa Rodgers (Lead Project Manager) of WF CTSI for their assistance with data abstraction and management.
Funding support
The project described was supported by the Wake Forest Clinical and Translational Science Institute (WF CTSI) Pilot Grant (PI: Nageswaran) as part of the CTSI Pilot Grant Program and the Wake Forest Center for Biomedical Informatics (WFBMI) funded by the National Center for Advancing Translational Sciences (NCATS), National Institutes of Health, through Grant Award Number UL1TR001420. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH.
Disclosure of interests
No interests were disclosed.