Implementation Evaluation of Pulse: Findings from a Rigorous Evaluation of New or Innovative Approaches to Prevent Teen Pregnancy
July 2021
This implementation report provides details on the recruitment procedures and implementation findings of an evaluation of Pulse, a web–based mobile health application (app)accessible through smartphones and tablets. The goals of Pulse are to increase birth control use and clinic use and ultimately decrease teen pregnancy. Pulse provides comprehensive, medically accurate sexual and reproductive health information to young women through engaging interactive and multimedia features.
Pulse was evaluated through an individual-level randomized controlled trial. In total, we randomly assigned 2,317 women to the Pulse intervention group or to the Pulse comparison group over a two-year period. The comparison group was directed to the Pulse “Comparison” app which was custom-built to function similarly to the Pulse intervention app and had the same name and branding. The primary behavioral outcomes examined through this study included unprotected sex (sex without using any contraceptive method; sex without using hormonal or long-acting contraceptive methods); hormonal or long-acting reversible contraceptive use among those who were sexually active at baseline; current contraceptive use; contraceptive use frequency in last six weeks among those who were sexually active at baseline; and visiting a clinic for sexual and reproductive health services.
This implementation report provides an in-depth explanation of the online recruitment process. The report then provides findings on adherence, dosage, quality, and contextual factors of the implementation. Implementation findings are primarily based on text message data, app usage data, post-test survey data, and a small number of in-depth interviews.
Pulse was developed by Healthy Teen Network and MetaMedia Training International. The evaluation was led by Child Trends and supported by Ewald & Wasserman Research Consultants.
Contributing Staff
Presentations on Social Media Recruitment
This presentation focuses on our lessons learned using social media recruitment for health studies.
Citation: Martinez-Garcia, G., Garrido, Sufrinko, N. (2020, October 6-8). Recruiting Youth? Using social media to recruit participants to sexual health research studies [Virtual presentation], CDC Health Communication, Marketing & Media Forum.
This presentation focuses on our lessons learned using social media recruitment and internet-based survey to measure unprotected sex.
Citation: Martinez-Garcia, G., Garrido, M., Manlove, J., & Finocharo, J. (2020, May 26-28). Instagram them and they will come… but can you survey them right? [Virtual Presentation], Office of Population Affairs “Looking Back, Moving Forward: A Special Series for TPP and PAF Grantees” Grantee Expo.
Social media platforms such as Facebook, Instagram, and Twitter offer the opportunity to reach large numbers of youth. However, program managers’ success in effectively recruiting youth is mixed. Managers and researchers often lack the know-how and skills to effectively reach youth through these channels and resort to outdated recruitment methods or highly ineffective and expensive banner ads. In order to inform youth of program or research opportunities, we must speak their language and reach them where they are. This presentation shows how to design effective ads to maximize advertising budgets.
Citation: Garrido, M. and Sufrinko, N. (2019, July 10-12) Instagram them and they will come…or will they?: The secrets of recruiting youth through social media [Conference session]. National Sexual Health Conference, Chicago, IL.
Are your social media ads flopping? Still using a flyer to recruit youth for research studies? Learn the to-do and to-don’ts of using social media to recruit youth for research studies and programs.
Citation: Garrido, M., & Martinez-Garcia, G, (2018, May 6-7). Inst them and they will come…or will they?: The secrets of recruiting youth through social media platforms [Conference session]. FYSB National Grantee Conference, Fort Lauderdale, FL.
Due to its democratic reach, researchers and program managers alike often use social media platforms such as Facebook, Instagram, and Twitter to recruit participants, but experience mixed results. We will show you our process for selecting the appropriate social media format to reach youth, developing relevant materials that make youth click, and monitoring the campaigns to assess their effectiveness.
Citation: Garrido, M., Sufrinko, N. and Martinez-Garcia, G. (2018, May 6-7). Do’s and Don’ts of social media recruitment [Conference session]. YTH annual conference, San Francisco, CA.
Presentations about Pulse and Research
Learn about our mhealth application and the preliminary research findings of its randomized control trial. Pulse is a comprehensive and media-rich mobile app that promotes healthy sexual outcomes to 18-20 year old Black and Latinx women in the United States.
Citation: Garrido, M., & Whitfield, B. (2020, March 3). About the Pulse Study [Webinar session]. Office of Population Affairs’ Teen Pregnancy Prevention Tier 2 Showcase Webinar, Washington, DC.
Conducting research on sexual behavior is conveniently conducted online. This presentation shares lessons from the Pulse evaluation to equip participants with the tools and knowledge to recruit and enroll participants, implement a program, and evaluate its efficacy in an online environment.
Citation: Martinez-Garcia, G., Garrido, M., Manlove, J., Cook, E., & Whitfield, B. (2019, October 28-30). Sex Research in Cyberspace [Conference session]. Healthy Teen Network’s Annual Conference, New Orleans, LA.
While using technology offers opportunities for innovation in evaluation, it also presents challenges that can hinder effective evaluation. In order to enhance evaluation projects, technology must be selected wisely, employed appropriately, and monitored consistently. This presentation provides concrete examples of the barriers face using technology and tested solutions to overcome common challenges.
Citation: Martínez-García, G., Cook, E., and Reed, R. (2017, November 8-11) Sex Ed Goes Tech: Integrating technology into the evaluation of three sexual health education programs [Conference session]. American Evaluation Association Annual Conference, Washington, DC.
Presentations on MHealth and Technology
This presentation focus on lessons learned on conducting the randomized-controlled trial of Pulse.
Citation: Martínez-García, G. (2020, February 19-21). Two apps, two RCTs, many lessons. Switched on- Sexuality Education in the Digital Space. [Conference session]. International Symposium, Istanbul, Turkey.
This presentation focused on lessons learned along the lifecycle of a digital health product.
Citation: Martínez-García, G. (2020, February 19-21). Scale up. See it grow. Let it go. Switched on- Sexuality Education in the Digital Space. [Conference session]. International Symposium, Istanbul, Turkey.
This presentation focused on how we applied behavior change theories to Pulse.
Citation: Martínez-García, G. (2020, February 19-21). On how health behavior theories are important, but not the whole story in developing effective digital content. Switched on- Sexuality Education in the Digital Space. [Conference session]. International Symposium, Istanbul, Turkey.
Digital media and mHealth interventions offer a unique opportunity to reach a large population with highly engaging and rich content to support healthy sexual behaviors. Moreover, content can easily be customizable, updated, and adapted to people with diverse needs and preferences. Content can reach the end user quickly and with very few barriers. But is it worth it? In a world of fast communication exchange and technological advances, digital media and mHealth interventions may become outdated and irrelevant very quickly. Moreover, testing them for efficacy presents a challenge when your product’s shelf-life is short and your budget limited. This presentations shows evidence on how digital media and mHealth interventions can be used to support behavior change.
Citation: Martinez-Garcia, G. (2017, July 6-7). Is it worth it?: Using digital media and mHealth interventions for sexual health promotion [Conference session]. National Sexual Health Conference, Denver, CO.
Are you trying to reach youth through new media? Are you trying to figure out how to make sexual health information engaging and attractive? This presentation discusses the best practices to align health content with new media selection and lessons learned from our own process developing two mobile apps with a mix of media formats, developing storyboards, and keeping media relevant in a world of ever-changing medical and health information.
Citation: Martinez-Garcia, G. and Garrido, M. (2016, November 14-16). Sassy and Short but with a punch: Translating complex health information for new media [Conference session]. Healthy Teen Network Annual Conference, Las Vegas, NV.
This presentation presents current trends in sexual health outcomes, the state of the field, emerging trends, and how to use innovation to advance the field.
Citation: Martinez-Garcia, G. and Sufrinko, N. (2017, July 6-7). Using innovation to support adolescent sexual health outcomes [Conference session]. National Sexual Health Conference, Denver, CO.
