Improving the Reach Out and Read Program at a Student-Run Free Clinic for Homeless Women and Children

Background: We evaluated the Reach Out and Read program in a student-run clinic serving homeless women and children. Objectives were to improve documentation of book delivery and provision of anticipatory guidance in electronic health records (EHRs) and determine changes in student managers’ knowledge, attitudes, and practices towards pediatric literacy. Methods: We evaluated eligible pediatric visits (N=201) and compared the number of books distributed to patients documented on paper to those documented in SOAP notes, which provides a clinic visit summary in EHRs. Student managers received didactic trainings on pediatric literacy, documentation skills, and ways to train volunteers. Student managers were trained to document that a book was provided to the patient, identify which book was provided, and specify that anticipatory guidance about reading was given to the parent in the SOAP note. Student managers were advised to train other student volunteers in this skill. Pediatric literacy knowledge and attitudes were evaluated before and three months after didactic training. Practice behaviors were evaluated after training. SOAP notes were evaluated six months later to determine improvements. Results: Documentation of book delivery in SOAP notes increased (12.5-77.8%) after didactic training (p<0.001). Significant improvements in students’ literacy knowledge were found (p=0.0201). Most students (67%) practiced reading aloud to patients and asked parents to demonstrate reading. After training, all SOAP notes included the name of the book and that anticipatory guidance was provided. Conclusions: Our results emphasize that training can be effective at improving student managers’ knowledge, attitudes, practices, and documentation skills in student-run clinics.


Introduction
Lack of exposure to reading at an early age and poverty have been linked with poor school readiness and future academic performance. 1The Reach Out and Read program is an evidencebased strategy for primary care providers designed to promote early literacy while also preventing problems of early childhood development and learning.Founded in 1989, the Reach Out and Read program has partnered with over 5,200 program sites and provides books to more than four million children and families annually. 2The Reach Out and Read model includes the following com-ponents: integration of age-appropriate book delivery into well-child visits for patients ages six months to five years; delivery of anticipatory guidance to parents on how to read out loud to their children; assessment of developmental literacy milestones in well-child visits; and modeling reading out loud by volunteers in clinic waiting rooms. 1 Research on the effectiveness of the Reach Out and Read program showed a two-year improvement of language acquisition in children participating in the program versus children who are not in the program. 2 Studies have evaluated the Reach Out and Read program in community-based clinic settings.For example, a study by Byington and colleagues found that books promote family bonding, physician-family relationships, and introduced more educational opportunities for low-income immigrant children. 3 Reach Out and Read clinics have also implemented quality improvement projects to augment provider execution of book delivery, assessment of literary milestones, and provision of anticipatory guidance to parents about reading through training modules. 4,5Although numerous studies have evaluated the Reach Out and Read program's effectiveness at underrepresented and underserved clinics, no studies have evaluated the efficacy of the Reach Out and Read program among homeless children.Homeless children have decreased literacy skills when they reach school age due to language and speech deficits.Recurrent moves and placement in new schools effects their education and school performance. 6Homeless children are more likely to experience hunger and malnutrition and have higher estimates of chronic disease and acute medical problems than their low-income counterparts with homes. 6Homeless families often receive fragmented care and rely on emergency rooms for their primary source of medical care.The negative effects of homelessness on children's development and health can be addressed by improving access to health services through studentrun free clinics participating in early literacy programs such as Reach Out at Read.Nevertheless, no studies have evaluated the Reach Out and Read program at a student-run free clinic.Student-run free clinics meet dual goals of health care institutions by providing clinical education and direct patient care.Students receive service learning experiences under the supervision of health professional faculty and clinic preceptors while providing direct patient care to underserved populations. 7Studies have shown that students who volunteer at student-run free clinics obtain valuable clinical experiences and reports of patient satisfaction with the care they receive are as good as comparable non-student-run clinics. 8,9ur study aimed to evaluate the effectiveness of an intervention designed to improve students' knowledge, attitudes, and practices towards pediatric literacy concepts and improve their documentation of book delivery and provision of anticipatory guidance in the electronic health records (EHR) at a student-run clinic embedded in a homeless shelter for women and children.Objectives were to: 1) compare the documentation of book delivery and provision of anticipatory guidance in the EHR and 2) determine changes in knowledge, attitudes, and practices towards pediatric literacy promotion before and after didactic trainings for student managers.

Setting
This project was developed using communitybased participatory research principles and previously published research evaluating the Reach Out and Read program in underserved clinic settings. 4,5,10A collaborative and equitable process was followed to establish a quality improvement project aimed at improving the Reach Out and Read program.The project was designed with the medical director of a student-run free clinic which served homeless women and their children.The medical director expressed an interest in transitioning from the paper documentation to the documentation of book delivery and anticipatory guidance in each patient's chart in the EHR.Several efforts have been undertaken to improve documentation among volunteers at this clinic.With an increase in the number of medical student volunteers working at the clinic and the potential for an increase in patients, it may become challenging for volunteers to remember documenting that they gave out a book at each appointment and provided anticipatory guidance about reading at each visit.The documentation of book delivery within the EHR took out the additional step on documenting books on paper since volunteers are already putting other notes from the patient's visit into their chart.Medical care and student supervision at the student-run free clinic was provided by a certified physician assistant (PA) medical director and staffed with a licensed vocational nurse.Clinic operations and programs were managed by eight medical and PA student managers.Student managers coordinated volunteer scheduling, developed patient education efforts and trained other medical volunteers.The clinic provided wellchild exams, vaccinations, school physicals and acute medical care to pediatric patients.The clinic was open two evenings each week and 24 students volunteered every month (8 clinic managers; 16 other volunteers).Student managers typically volunteered at least once per month and other students volunteered once or several times throughout the year based on their interest, schedule and volunteer availability.

Chart Review
The Reach Out and Read program was implemented at the homeless shelter clinic in June 2013.This project evaluated documentation of books delivered and anticipatory guidance provided to patients in the EHR, Practice Fusion.Practice Fusion is a free electronic health record used in community and private office clinic settings available at http://www.practicefusion.com.Books were initially documented on a paper inventory and recent efforts were made to transition to documentation in the EHR.Pediatric patient encounters (ages <5 years) from June 2013 to December 2014 (N=201) were evaluated for indication of a routine infant or child health check in the SOAP text or as a diagnosis code [V20.2].The paper inventory was reviewed to determine whether the books documented on paper were documented in the Plan section of the SOAP note.The total number of books reported in the EHR was compared to the number of delivered books written on a paper inventory.The chart review and review of the paper inventory were repeated six months later to determine any changes in EHR documentation.

Didactic Training
In February 2015, incoming student managers (N=8) received a 30-minute in-person didactic training during their new manager orientation.Student managers received training on: the Reach Out and Read program; general pediatric literacy concepts; EHR documentation skills; and instruction on how to train student volunteers to improve documentation of book delivery and anticipatory guidance in the EHR.Volunteers were instructed to enter "ROR -[Name of book] -ROR education provided" in the patient's medical chart.The didactic training also contained information about pediatric literacy disparities in the United States and provided ways to encourage volunteers to engage parents and children in early reading.Students were given time to practice providing anticipatory guidance and model reading behaviors to parents.An overview of curriculum topic areas and learning objectives is provided in Appendix 1.In April 2015, an informal refresher training was conducted during a clinic manager meeting.The refresher training comprised of a quick (5-10 minute) review of the initial training.
The training followed a train-the-trainer (TTT) model.The TTT model is used to train health care providers in a particular subject with instruction on how to train and supervise others on the same content. 11The TTT model has been proven effective at improving didactic knowledge and patient outcomes among health care professionals 11 and resident physicians 12 ; however, it has been underutilized with medical students.The purpose of our TTT model was to train student managers to train other student volunteers.Due to the large number of student volunteers who volunteered at the clinic once or once per year, we were only able to train student managers.Student managers volunteered at the clinic at least once per month, attended meetings with the medical director on an ongoing basis, and completed most of the documentation for the clinic visits.The student managers were responsible for training volunteers when to hand out Reach Out and Read books, how to provide anticipatory guidance, and how to document book delivery and provision of anticipatory guidance in the patient's chart.

Evaluations
Changes in pediatric literacy knowledge and attitudes were evaluated before and three months after the didactic training.Clinical practices were evaluated after the training only.Student managers completed a 14-item survey developed based on previous studies. 13Knowledge was assessed through a series of six true-false and multiplechoice questions.Questions evaluated the following: how many children the Reach Out and Read program reaches per year (answer: 4 million); the ages books should be given to children (answer: 6 months to 5 years); the type of clinic visit Reach Out and Read books should be distributed during (answer: well-child visits); that television programs such as Sesame Street are less likely to foster reading than seeing parents read or being read to by parents; and that reading books word for word aloud is important to foster language acquisition.The total number of correct responses were compiled and a total knowledge score was determined.Attitudes were measured on a 5-point Likert scale.Questions addressing comfort level assessing literacy among patients and whether the clinic was an appropriate place to encourage literacy were measured from 1=strongly disagree to 5=strongly agree.Questions evaluating the necessity of literacy assessments and anticipatory guidance before school age and whether parents are offended by questions about literacy were measured in reverse from 1=strongly agree to 5=strongly disagree.Responses were reverse coded for consistency of interpretation of all attitude measures.Practices were measured by asking students whether they consistently, usually, sometimes, rarely or never did the following: documented book delivery in the SOAP note for each patient encounter in the EHR; provided anticipatory guidance to parents about reading to their child; and provided a book that was developmentally appropriate to their patient.An overview of survey measures is provided in Appendix 2. The pre-test data was collected on paper during the face-toface student managers' training and a post-test survey was collected after the refresher training via Survey Monkey.Students were emailed a web link to complete the survey online.

Data Analysis
Fisher's exact tests were used to compare documentation results between the EHR and paper inventory before and after the student managers received training.Wilcoxon signed-rank test was used to evaluate significant changes in pediatric literacy knowledge, attitudes after the didactic training.Data were analyzed through the STATA 14.0.

Ethical Approval
This study was classified as a quality improvement project and was exempt from Institutional Review Board approval.

Chart Review
Table 1 reports the comparison of the total number of Reach Out and Read books delivered that were documented in the Plan section of the SOAP note before and after student managers received training.We found a significant increase in the total number of books delivered that were documented in the EHR after the student managers received didactic training (p<0.0001).Prior to their training, only 13% of the total number of books (N=48) documented on the paper inventory were also documented in the EHR.Six months later, 28 out of 36 books (78%) documented on the paper inventory were also documented in the EHR.All documentation included the name of the book provided (i.e.Smile) and that ROR education was provided.

Didactic Training
Table 2 reports pre-and post-knowledge and attitudes before and after the didactic training.Seven out of the eight (87.5%)managers completed the post-test.Significant improvements in total knowledge scores were found after completion of both trainings (p=0.0201).The greatest increases in knowledge were found on measures evaluating children's ability to turn pages in board books by ages 1 ½ years (50%), that books should be provided to children at all well-child visits (32.1%), and the age range (6 months to 5 years) that books should be provided to children at wellchild visits as part of the Reach Out and Read program (16.1%).Slight attitude improvements were found in mean scores when recognizing that parents would not be offended by assessing literacy (0.83), the need to provide anticipatory guidance well before school age (0.60), and the clinic served as an appropriate place for literacy measurement (0.20).No differences were found in student managers' comfort level assessing literacy among their patients.We did not find any statistically significant differences in any attitude measures (all p's>0.05).
After didactic training, most (67%) student managers reported consistently modeling reading aloud or asking the parents to demonstrate reading with his/her child during well-child visits.Furthermore, most (67%) student managers reported that the books they offered were appropriate for their pediatric patients.However, only half (50%) of respondents reported consistently documenting Reach Out and Read books in Practice Fusion and only 17% reported consistently providing anticipatory literacy guidance to mothers.

Discussion
The goal of this project was to improve documentation of book delivery in the EHR and improve medical and PA student managers' knowledge, attitudes, and practices towards pediatric literacy promotion.Our first main finding was a significant improvement in the documentation of books delivered to pediatric patients in their SOAP notes six months after an initial chart review of well-child visits.Other projects developed to evaluate the delivery of Reach Out and Read books have focused on improving rates of book delivery.Khandekar and colleagues aimed to improve the rates of age-appropriate book delivery and anticipatory guidance in six pediatric clinics.Results indicated an increase in the median rate of book giving (97% to 99%) and no changes in anticipatory guidance (89%). 5A similar project was conducted by Thandekar and colleagues in a large inner-city pediatric residency clinic.The percentage of books delivered increased from 30% to 96% and anticipatory guidance reports increased from 26% to 87%. 4 Our project differed from other projects due to its focus on documentation skills.The role of documentation training is important in this setting due to the nature of a student-run free clinic and the high turnover rate of students.Of the 201 eligible medical charts for review, only 93 charts had an ICD-9 code for a well-child visit or some indication in the visit summary that the appointment was for a routine infant or child health check-up.Other visits without an ICD-9 code in the chart may have been for vaccinations or sick visits.In August 2015, the percentage of books delivered to pediatric patients during well-child visits at our student-run free clinic was 90% (n=84).Future evaluations of the Reach Out and Read program at this clinic can now aim to improve the percentage of book delivery and anticipatory guidance at well-child visits.
Our second main finding was that our didactic training improved knowledge and attitudes towards pediatric literacy promotion among student managers.Similar results were reported in studies evaluating trainings designed to improve knowledge, attitudes, and practices among pediatric and family medicine residents. 13,14Although results were not statistically significant, Hazzard and colleagues found improvements in knowledge and attitudes after pediatric residents received either a brief didactic training or enhanced training model which included grand rounds, readings, and modeling of reading behaviors by their supervising physician. 14Rosenthal, Werner, and Dubin conducted a similar training with family medicine residents.Their intervention was delivered to all residents and included didactic sessions, role-playing, and peer feedback during grand rounds and weekly conferences. 13oth studies differ from our project in their assessment of pediatric milestones.Future iterations of our didactic training can incorporate the evaluation and documentation of literacy milestones into our training of medical student providers.
Our project had several strengths and limitations.A strength of this project was the ability to review over 200 electronic pediatric patient charts in Practice Fusion over a two-year time period.The Practice Fusion electronic system had several features for easily extracting patient reports for data collection purposes.A limitation of this project was the small number of students managers (<10) who received the didactic training.Due to the high turnover of volunteers, we were not able to train all students who volunteered at the clinic.The homeless shelter clinic only selects eight student managers per year.This training was replicated in January 2015 with future plans to continue trainings for student managers to learn about the Reach Out and Read program and then train medical and PA students who volunteer at the clinic.
Our project was effective at improving documentation of book delivery and medical and PA student managers' knowledge, attitudes, and practices towards pediatric literacy promotion in a student-run free clinic for homeless women and their children.Anecdotally, one of the authors observed that managers were following the TTT model and faithfully describing the Reach Out and Read program to the 16 volunteers each month who passed through the clinic.Consistent training of student managers in the TTT model and hands-on training for all student volunteers is needed to improve their knowledge and attitudes and practice as future health care providers.Accordingly, we have developed an expanded curriculum which encompasses didactic training and an objective structured clinical exam for residents and medical and PA students.Next steps are to evaluate parents' attitudes and perceptions about the Reach Out and Read program and literacy related behaviors.

Table 1 .
Comparison of Reach Out and Read book delivery documentation in the electronic health record before and after didactic training

Table 2 .
Student managers' knowledge and attitudes before and after didactic training (N=8)