SPAIN AND USA. SOCIAL WORK AND SOCIAL WELFARE PERSPECTIVES / ESPAÑA Y ESTADOS UNIDOS. PERSPECTIVAS SOBRE EL TRABAJO SOCIAL Y EL BIENESTAR SOCIAL

DIGITAL TECHNOLOGY AND MENTAL HEALTH INTERVENTIONS: OPPORTUNITIES AND CHALLENGES

Adrian Aguilera

University of California, Berkeley. School of Social Welfare

aguila@berkeley.edu

 

ABSTRACT

The growth of the Internet, mobile phones, social media and other digital technologies has changed our world in many ways. It has provided individuals with information that was previously only available to a select few. An example of the reach of technology is data that as of October 2012, there are over 6 billion phones worldwide (BBC, 2012). The availability of data in real time has presented hopes of intervening more efficiently and managing health problems by leveraging limited human resources. It also has an impact in changing the roles of providers and patients and in legal and ethical issues including privacy in digital health interactions. This paper will discuss why digital technology has received recent attention in the area of mental health, present some applications of technology for mental health to date, explore the challenges to full implementation in clinical settings, and present future opportunities for digital technologies.

TECNOLOGÍA DIGITAL E INTERVENCIONES PARA LA SALUD MENTAL: OPORTUNIDADES Y BARRERAS

RESUMEN

El crecimiento del Internet, los teléfonos móviles, las redes sociales y otras tecnologías digitales ha cambiado nuestro mundo de muchas maneras. Ha proporcionado a las personas con la información que antes sólo estaba disponible para un grupo selecto, por ejemplo a partir de octubre de 2012. Un ejemplo del alcance de la tecnología son los datos que dicen que hay más de 6 millones de teléfonos en todo el mundo (BBC, 2012). La disponibilidad de los datos en tiempo real a presentado la esperanza de intervenir de manera más eficiente y manejar los problemas de salud los recursos humanos limitados. También tiene un impacto en el cambio de los roles de los proveedores y los pacientes y en aspectos legales y éticos, incluyendo la privacidad en las interacciones de salud digital. Este artículo discutirá unas razones por cual la tecnología digital ha recibido atención recientemente en el área de salud mental, presentará algunas aplicaciones de la tecnología para mejorar la salud mental hasta la fecha, explorará algunas barreras para la diseminación en la práctica clínica, y presentará algunas oportunidades futuras de las tecnologías digitales.

Received: January 16, 2014; Accepted: July 25, 2014.

Citation/ Cómo citar este artículo: Aguilera, A. (2015). "Digital Technology and Mental Health Interventions: Opportunities and Challenges". Arbor, 191 (771): a210. doi: http://dx.doi.org/10.3989/arbor.2015.771n1012

KEYWORDS: digital health; mobile health; mental health; technology; intervention.

PALABRAS CLAVE: salud digital; salud móvil; salud mental; tecnología; intervención.

Copyright: © 2015 CSIC. This is an open-access article distributed under the terms of the Creative Commons Attribution-Non Commercial (by-nc) Spain 3.0 License.

CONTENTS

ABSTRACT
RESUMEN
WHY TECHNOLOGY?

WHAT HAS BEEN DONE?

WHAT ARE THE BARRIERS?

WHAT’S NEXT?

CONCLUSION

REFERENCES

 

WHY TECHNOLOGY?
 Top

Digital technologies, including the Internet, mobile applications, sensors and others have the potential to improve the delivery of health interventions. These technologies are expected to enhance patient outcomes by increasing the reach of existing interventions and by utilizing new technologies to measure health behaviors. Digital technology can allow for real time data collection and result in immediate clinical interventions based on that data (Morris and Aguilera, 2012Morris, M. E. and Aguilera, A. (2012). "Mobile, social, and wearable computing and the evolution of psychological practice". Professional Psychology: Research and Practice, 43 (6), pp. 622-626, http://dx.doi.org/10.1037/a0029041.). These technologies offer hope to fill gaps in improving clinical interventions in social work and related professions. In particular, the growth of mobile phone applications for health has grown very rapidly and provides opportunities to expand current care beyond the traditional clinic setting. 



While there is a large discrepancy in home Internet access between high and low income households in the US, this is not the case with mobile phone use as more Americans own a mobile phone than a computer (Lenhart, 2010Lenhart, A. (2010). Adults, cell phones and texting, Retrieved 9/28, 2010, from http://pewresearch.org/pubs/1716/adults-cell-phones-text-messages.). The use of digital technology in health applications is experiencing a tremendous boom in large part as a result of ballooning health care costs. There is much excitement but there is still limited data on the impact of technology aside from evidence for antiretroviral medication adherence and smoking cessation (Free et al., 2013Free, C.; Phillips, G.; Galli, L.; Watson, L.; Felix, L.; Edwards, P.; Patel, V. and Haines, A. (2013). "The effectiveness of mobile-health technology-based health behaviour change or disease management interventions for health care consumers: A systematic review". PLoS Medicine, 10 (1), e1001362. http://dx.doi.org/10.1371/journal.pmed.1001362.). Although it is unclear whether technology will plug all of the holes in the American healthcare system, technology is being applied rapidly and thus it is imperative to study the effects of integrating digital technologies into health interventions. There are many factors that limit the effectiveness of traditional health interventions in the US that could be addressed by the implementation of technological solutions. Two of those factors that hinder efficient clinical intervention include limited human resources and slow dissemination of evidence based treatments.

Slow Dissemination


For health and mental health care, digital technologies may allow clinicians to reach difficult to reach populations. For example, using video streaming, these technologies could facilitate access to specialist services in communities with a low supply of trained providers or limited resources, including bilingual bicultural providers. In the area of mental health, traditional psychotherapy relies on a one-to-one treatment model which inherently limits the number of people any individual therapist can help in a lifetime. As a result, there have been calls to develop innovative ways of improving the quality and reach of psychotherapies using innovative digital technologies such as mobile phones and smartphones in particular (Boschen, 2009Boschen, M. J. (2009). "Mobile telephones and psychotherapy: I. capability and applicability". Behavior Therapist, 32 (8), pp. 168-175.; Kazdin and Blase, 2011Kazdin, A. E. and Blase, S. L. (2011). "Rebooting psychotherapy research and practice to reduce the burden of mental illness". Perspectives on Psychological Science, 6 (1), pp. 21, http://dx.doi.org/10.1177/1745691610393527.).



Digital technologies also have the potential to speed the intervention development pipeline and bring evidence-based interventions to populations in need at a faster pace. In the last 10-20 years, there has been a push to develop and implement clinical interventions that are evidence based. However, even after such interventions are developed, they take a long time to get disseminated. The Institute of Medicine reported that on average, 20% of clinical trials result in changes in healthcare and those take 17 years to get to people that could benefit from them (Institute of Medicine, 2001Institute of Medicine (2001). Crossing the quality chasm: A new health care system for the 21st century.). Furthermore, when they are available, they are not always accessible to everyone in need; particularly those from low-income backgrounds who may not be able to afford quality care as well as individuals in rural areas who do not have physical access to trained clinicians. In cases where interventions are disseminated, fidelity to the original evidence based treatment varies when they are applied to complex service settings, which differ from controlled lab settings where treatments are often developed. If interventions morph over time, new questions arise about whether adaptations and changes might lessen efficacy and effectiveness. Finally, patients and clients in community based health and mental health settings tend to have lower attendance and completion rates for behavioral treatments (Miranda et al., 1996Miranda, J.; Azocar, F.; Organista, K. C.; Muñoz, R. F. and Lieberman, A. (1996). "Recruiting and retaining low-income latinos in psychotherapy research". Journal of Consulting and Clinical Psychology, 64 (5), pp. 868-874, http://dx.doi.org/10.1037/0022006X.64.5.868.; Miranda et al., 2003Miranda, J.; Chung, J. Y.; Green, B. L.; Krupnick, J.; Siddique, J.; Revicki, D. A. and Belin, T. (2003). "Treating depression in predominantly low-income young minority women". JAMA: The Journal of the American Medical Association, 290 (1), pp. 57-65, http://dx.doi.org/10.1001/jama.290.1.57.). Digital technologies may help improve the rates of studies that go “to market” and reduce the time that it takes to get there. For example, if an intervention is automated, there is little to change when going from a research trial to a clinical service thereby making it almost immediately available for dissemination and often requires less human resources.

Limited Workforce


Assuming the previously mentioned problems of efficient development and implementation of behavioral interventions did not exist, there are still limitations to the number of people in need that trained (and non-trained) individuals can reach. The reason for this is that the efforts to implement behavioral interventions are inherently consumable; that is the time that people put into providing therapy, giving advice, or managing cases can never be regained (Muñoz, 2010Muñoz, R. F. (2010). "Using evidence-based internet interventions to reduce health disparities worldwide". Journal of Medical Internet Research, 12 (5), e60.). Digital technologies make these interventions more scalable and potentially more sustainable over time. Additionally, even with medication based interventions, those medications, once consumed cannot benefit anyone else. However, technology can act as a force multiplier by helping disseminate digital information to large numbers of people. For example, one could develop an automated online evidence based behavioral treatment for depression and anxiety, which provides access to almost unlimited numbers of people at a time without degrading the quality of the intervention. The next section will provide examples of interventions that have utilized technology to improve the delivery of health interventions.


 

WHAT HAS BEEN DONE?
 Top

Online interventions


Digital technologies have been involved in clinical practice for years but are only recently reaching the mainstream of practice, although at a slow pace. Web-based interventions have the most evidence of efficacy when compared with other technologies. The Internet boom of the early 2000s brought with it development of health interventions online due to the low cost, increase in convenience and stigma reduction (Griffiths et al, 2006Griffiths, F.; Lindenmeyer, A.; Powell, J.; Lowe, P. and Thorogood, M. (2006). "Why are health care interventions delivered over the internet? A systematic review of the published literature". Journal of medical Internet research, 8 (2), http://dx.doi.org/10.2196/jmir.8.2.e12 / http://dx.doi.org/10.2196/jmir.8.2.e10.). Online interventions began as passive websites with information that people would read and complete much like a self help book and have evolved into more interactive sites that are customized to individuals. Online interventions have addressed a wide range of problems including depression (Christensen, Griffiths and Korten, 2002Christensen, H.; Griffiths, K. M. and Korten, A. (2002). "Web-based cognitive behavior therapy: Analysis of site usage and changes in depression and anxiety scores". Journal of Medical Internet Research, 4 (1), e3. http://dx.doi.org/10.2196/jmir.4.1.e3.), smoking (Muñoz et al., 2006Muñoz, R. F.; Lenert, L. L.; Delucchi, K.; Stoddard, J.; Perez, J. E.; Penilla, C. and Pérez-Stable, E. J. (2006). "Toward evidence-based internet interventions: A spanish/english web site for international smoking cessation trials". Nicotine and Tobacco Research, 8 (1), pp. 77-87, http://dx.doi.org/10.1080/14622200500431940.), obesity (Williamson et al., 2006Williamson, D. A.; Walden, H. M.; White, M. A.; York Crowe, E.; Newton, R. L.; Alfonso, A.; Gordon, S. and Ryan, D. (2006). "Two Year Internet Based Randomized Controlled Trial for Weight Loss in African American Girls". Obesity, 14 (7), pp. 1231-1243, http://dx.doi.org/10.1038/oby.2006.140.; Gold et al., 2007Gold, B. C.; Burke, S.; Pintauro, S.; Buzzell, P. and Harvey Berino, J. (2007). "Weight loss on the web: A pilot study comparing a structured behavioral intervention to a commercial program". Obesity, 15 (1), pp. 155-164, http://dx.doi.org/10.1038/oby.2007.520.), diabetes (Glasgow, Boles, Mckay, Feil and Barrera Jr, 2003Glasgow, R. E.; Boles, S. M.; Mckay, H. G.; Feil, E. G. and Barrera Jr, M. (2003). "The D-net diabetes self-management program: Long-term implementation, outcomes, and generalization results". Preventive Medicine, 36 (4), pp. 410-419, http://dx.doi.org/10.1016/S00917435(02)000567.), and alcohol consumption (Murray et al., 2007Murray, E.; McCambridge, J.; Khadjesari, Z.; White, I. R.; Thompson, S. G.; Godfrey, C.; Linke, S. and Wallace, P. (2007). "The DYD-RCT protocol: An on-line randomised controlled trial of an interactive computer-based intervention compared with a standard information website to reduce alcohol consumption among hazardous drinkers". BMC Public Health, 7 (1), pp. 306, http://dx.doi.org/10.1186/147124587306.) among others. These interventions have largely been shown to be efficacious, however there are still major limitations. Primarily, web based interventions suffer from high rates of attrition -- many people who start are not likely to complete the interventions. In an open intervention as many as 85% of participants fail to come back for a second “session” of treatment (Eysenbach, 2005Eysenbach, G. (2005). "The law of attrition". Journal of Medical Internet Research, 7 (1), p. e11. http://dx.doi.org/10.2196/jmir.7.1.e11.).


mHealth


Mobile phones have garnered much attention in recent years spawning the area of mHealth (mobile health) due to the ubiquity of phones worldwide as well as other characteristics inherent to the technology. Standard mobile phones are attractive for behavioral health interventions because they are always on, portable, low cost and ubiquitous while being able to send and receive various forms of data including audio, text, and photos. Smartphones add connections to the Internet, programmable apps, video, as well as location and movement tracking (Boschen and Casey, 2008Boschen, M. J. and Casey, L. M. (2008). "The use of mobile telephones as adjuncts to cognitive behavioral psychotherapy". Professional Psychology: Research and Practice, 39 (5), pp. 546-552, http://dx.doi.org/10.1037/07357028.39.5.546.). These features are desirable for health interventions as they address the challenges of reaching individuals where they are and collecting accurate real time data as opposed to relying on retrospective reports. Instead of asking patients, how much they have exercised in the past week or month, they could be sent that question via text message on a daily basis or better yet, their phones could track their movement throughout the week. 



The interest in mobile devices stems in part from realizing the potential of automated technologies using online interventions paired with the potential for higher participation using mobile devices that people carry with them at all times. As consumers become more accustomed to using mobile technology in their daily lives, they are increasingly interested in gaining access to information via mobile phones. For example, in an Australian sample, 76% of people were interested in using mobile phones for mental health monitoring and self-management. Importantly, people with mental health symptoms were more interested in using such services, suggesting that there is indeed an untapped market of consumers of mental health information that could benefit from psychological interventions outside of the traditional tools of psychotherapy practice (Proudfoot et al., 2010Proudfoot, J.; Parker, G.; Pavlovic, D. H.; Manicavasagar, V.; Adler, E. and Whitton, A. (2010). "Community attitudes to the appropriation of mobile phones for monitoring and managing depression, anxiety, and stress". Journal of Medical Internet Research, 12 (5), e64. http://dx.doi.org/10.2196/jmir.1475.). In another study, 98% of low income mostly unemployed clients in outpatient substance abuse treatment were interested in using interactive text messaging to help them maintain sobriety (Muench et al., 2013Muench, F.; Weiss, R. A.; Kuerbis, A. and Morgenstern, J. (2013). "Developing a theory driven text messaging intervention for addiction care with user driven content". Psychology of Addictive Behaviors, 27 (1), pp. 315-321, http://dx.doi.org/10.1037/a0029963.). Reviews have highlighted that mobile interventions are well accepted by end-users and welcome additions to physical and mental health treatments (Cole-Lewis and Kershaw, 2010Cole-Lewis, H. and Kershaw, T. (2010). "Text messaging as a tool for behavior change in disease prevention and management". Epidemiologic Reviews, 32, pp. 56-69, http://dx.doi.org/10.1093/epirev/mxq004.; Head et al., 2013Head, K. J.; Noar, S. M.; Iannarino, N. T. and Grant Harrington, N. (2013). "Efficacy of text messaging-based interventions for health promotion: A meta-analysis". Social Science & Medicine, 97, pp. 41-48, http://dx.doi.org/10.1016/j.socscimed.2013.08.003.; Heron and Smyth, 2010Heron, K. E. and Smyth, J. M. (2010). "Ecological momentary interventions: Incorporating mobile technology into psychosocial and health behaviour treatments". British Journal of Health Psychology, 15 (1), pp. 1-39, http://dx.doi.org/10.1348/135910709X466063.). These interventions have the benefit of ongoing contact and assessment beyond the traditional therapeutic environment. 



The research base for mobile technology applications is growing but is still limited in terms of specific applications that can be delivered on a broad scale. Although much research has been done using mobile phone based text messaging (SMS) based on cognitive behavioral therapy (CBT) principles, most of these efforts have been limited to feasibility studies. Evidence based smartphone applications using behavioral and cognitive methods tend to be in developmental stages. These include a Dialectical Behavior Therapy (DBT) app for borderline personality disorder, (Rizvi et al., 2011Rizvi, S. L.; Dimeff, L. A.; Skutch, J.; Carroll, D. and Linehan, M. M. (2011). "A pilot study of the DBT coach: An interactive mobile phone application for individuals with borderline personality disorder and substance use disorder". Behavior Therapy, 42 (4), pp. 589-600, http://dx.doi.org/10.1016/j.beth.2011.01.003.) a mobile therapy application using CBT techniques (Morris et al., 2010Morris, M. E.; Kathawala, Q.; Leen, T. K.; Gorenstein, E. E.; Guilak, F.; Labhard, M. and Deleeuw, W. (2010). "Mobile therapy: Case study evaluations of a cell phone application for emotional self-awareness". Journal of Medical Internet Research, 12 (2), e10. http://dx.doi.org/10.2196/jmir.1371.), and an Internet and mobile intervention for depression using context sensing to identify emotional states and intervene appropriately (Burns et al., 2011Burns, M. N.; Begale, M.; Duffecy, J.; Gergle, D.; Karr, C. J.; Giangrande, E. and Mohr, D. C. (2011). "Harnessing context sensing to develop a mobile intervention for depression". Journal of Medical Internet Research, 13 (3), e55. http://dx.doi.org/10.2196/jmir.1838.). Although these applications require a broader evidence base before they are ready for implementation, they foretell future applications for moving psychotherapy beyond the one on one encounter.



Text messaging is a basic feature of mobile phones, but one that can be deployed for messaging around health. Text messaging is perhaps the most widely available mobile tool available on nearly all phones that can be utilized by clinicians and their entire patient population. There is more research on text messaging than any other mobile format with the most promising results being for appointment adherence and smoking cessation (Free et al., 2013Free, C.; Phillips, G.; Galli, L.; Watson, L.; Felix, L.; Edwards, P.; Patel, V. and Haines, A. (2013). "The effectiveness of mobile-health technology-based health behaviour change or disease management interventions for health care consumers: A systematic review". PLoS Medicine, 10 (1), e1001362. http://dx.doi.org/10.1371/journal.pmed.1001362.). Text messages can make change goals more salient in one’s natural environment because they automatically push therapeutic content rather than relying on the client to proactively open an application which reduces the likelihood that clients will be able to ignore change goals in the face of unhealthy environment triggers. There are several simple free messaging services in which clinicians can send one time or repeated reminders to a patient at a specific day and time. There are also more interactive text messaging systems that can be utilized in conjunction with therapeutic goals. For example, the US federal government has released “smokefreetxt” a free SMS program for individuals attempting to quit smoking. Additionally, SMS can be used to trigger mobile web applications and cloud based audio and video files when clients need extra support. Most phones also have MMS capability where pictures of a loved one or a visual goal can be sent to patients at specific times. Not only do these applications help patients make goals more salient and track progress, but they can be used as simple appointment and homework reminders. 



One example of a text messaging application is an adjunct to group cognitive behavioral therapy for depression developed by Aguilera and Muñoz (2011Aguilera, A. and Muñoz, R. F. (2011). "Text messaging as an adjunct to CBT in low-income populations: A usability and feasibility pilot study". Professional Psychology: Research and Practice, 42 (6), pp. 472, http://dx.doi.org/10.1037/a0025499.). This system is designed to send messages to individuals currently in therapy asking them what their mood is at a random time during the day. This mood data is received by clinicians and can be displayed back to provide discussion points for negative and positive mood states throughout the week along with correlates and coping strategies. In addition to sending mood monitoring messages, patients also receive messages to reinforce the group therapy session for that week. For example, messages will ask patients to reflect on helpful and unhelpful thoughts for the day during the focus on thoughts and mood. On other days they receive other queries (e.g. about daily activities, social interactions, and sleep patterns) to reinforce the connection between mood states and thoughts, activities, and social interactions that are central to the therapy. This adjunct is designed to deepen self-awareness and knowledge of the themes that are discussed in therapy, but it has also resulted in increased perceived social support. For example, patients have mentioned that when they receive a message, they feel that “someone cares about my health” even though messages are automated. This text messaging system is an example of technology extending human capacity and strengthening an intervention that is known to be efficacious.



There are also numerous mobile phone applications that attempt to provide overlapping therapeutic services highlighting CBT theory. Some applications have been developed in clinical settings and take principles of CBT and transfer them to a mobile device very specifically. The Veterans Administration, in particular, has embraced the use of technology and has developed applications based on evidence based practices designed for veterans but available to all. One such application is the PTSD Coach available on iPhone and Android platforms. The application targets the management of PTSD with four modules focused on 1) education, 2) self-assessment, 3) symptom management, and 4) social support. The application is targeted towards vets but can likely be used by others dealing with PTSD. While these applications have not been directly tested, they were developed based on empirically supported interventions and principles providing some foundation for their use in clinical practice. 



As noted above other intervention apps have been developed for borderline personality disorder (Rizvi et al., 2011Rizvi, S. L.; Dimeff, L. A.; Skutch, J.; Carroll, D. and Linehan, M. M. (2011). "A pilot study of the DBT coach: An interactive mobile phone application for individuals with borderline personality disorder and substance use disorder". Behavior Therapy, 42 (4), pp. 589-600, http://dx.doi.org/10.1016/j.beth.2011.01.003.), depression using context sensing to identify emotional states and intervene appropriately (Burns et al., 2011Burns, M. N.; Begale, M.; Duffecy, J.; Gergle, D.; Karr, C. J.; Giangrande, E. and Mohr, D. C. (2011). "Harnessing context sensing to develop a mobile intervention for depression". Journal of Medical Internet Research, 13 (3), e55. http://dx.doi.org/10.2196/jmir.1838.), and substance abuse using GPS to trigger reminders when someone may be entering an areas previously associated with substance use (Gustafson et al., 2011Gustafson, D. H.; Shaw, B. R.; Isham, A.; Baker, T.; Boyle, M. G. and Levy, M. (2011). "Explicating an evidence-based, theoretically informed, mobile technology-based system to improve outcomes for people in recovery for alcohol dependence". Substance use & Misuse, 46 (1), pp. 96-111, http://dx.doi.org/10.3109/10826084.2011.521413.). Additionally, there are numerous applications that target specific components of CBT and may compliment practice such as programs that target gratitude, meditation, guided relaxation, therapeutic breathing and increasing positive emotion. However, like all self-guided change programs the question becomes, how many times will a person use an application without therapist support?

Passive Sensing


The most rapidly growing area of mobile health that is being embraced by the general public and can have tremendous utility to clinicians are ambulatory personal physiological, activity and location sensing tools (Intille, 2007Intille, S. S. (2007). "Technological innovations enabling automatic, context-sensitive ecological momentary assessment". In Stone, A.; Shiffman, S.; Atienza, A. and Nebeling, L. (eds.), The Science of Real-Time Data Capture. Self-Reports in Health Research, pp. 308-337. Oxford: Oxford University Press.), which will build on some of the examples presented earlier. Many promising applications are those that passively monitor a range of stimuli from activity level to heart rate or galvanic skin response using external sensors. The most basic level of mobile sensing utilizes technologies built into modern smartphones such as accelerometers, gyroscopes, and GPS. A popular application of these technologies is the detection of movement and location. It may soon be common practice for therapists to monitor the activity level of patients through smart phone accelerometers and GPS. With available applications today you can monitor the activity level of depressed patients and see whether they expanded their behavioral repertoires by engaging in new pleasant activities through GPS and even mapping. Sleep disturbances associated with a range of diagnoses can be monitored with various apps and inexpensive devices that track and graph sleep through activity monitoring features in smart phones. These applications can be combined with self-monitoring to help understand the daily correlates of poor sleep.



A benefit of mobile phones is the ability to capture objective data and nowhere is this more pronounced than with psychophysiological assessment. While biomonitoring typically requires additional hardware, which includes a sensor, we can now review periods when our clients were most aroused or hypervigilant with objective data and help them prepare for these situations when clear patterns are taking place. These applications will increasingly include intervention components such as notifications when an individual is aroused (e.g. through galvanic skin response, heart rate variability or heart rate) to engage in stress management techniques or as noted earlier, alerts based on global positioning or geographic information to avoid high risk situations (e.g. drug use areas) as is currently being implemented through The Comprehensive Health Enhancement Support System (CHESS™) developed through the University of Wisconsin (Gustafson et al., 2011Gustafson, D. H.; Shaw, B. R.; Isham, A.; Baker, T.; Boyle, M. G. and Levy, M. (2011). "Explicating an evidence-based, theoretically informed, mobile technology-based system to improve outcomes for people in recovery for alcohol dependence". Substance use & Misuse, 46 (1), pp. 96-111, http://dx.doi.org/10.3109/10826084.2011.521413.). This system integrates human services and computer technology to help individuals manage health problems by providing accessible and personalized information and support.


 

WHAT ARE THE BARRIERS?
 Top

Privacy and HIPAA


There are many opportunities and immense potential for digital technologies to improve health and mental health service provision but there are many challenges to overcome as well. For example some patients/clients simply may not like using technology mobile phones. Other issues include concerns about intrusiveness and the invasion of privacy (Proudfoot et al., 2010Proudfoot, J.; Parker, G.; Pavlovic, D. H.; Manicavasagar, V.; Adler, E. and Whitton, A. (2010). "Community attitudes to the appropriation of mobile phones for monitoring and managing depression, anxiety, and stress". Journal of Medical Internet Research, 12 (5), e64. http://dx.doi.org/10.2196/jmir.1475.). While these barriers may hinder adoption, understanding them may also help to facilitate adoption. For example, Muench and colleagues ( 2013Muench, F.; Weiss, R. A.; Kuerbis, A. and Morgenstern, J. (2013). "Developing a theory driven text messaging intervention for addiction care with user driven content". Psychology of Addictive Behaviors, 27 (1), pp. 315-321, http://dx.doi.org/10.1037/a0029963.) found that 40% of individuals interested in using text messaging for continuing care around problems of addiction preferred NOT having any messages that specifically reference drug use in the text. These findings sensitize us to the importance of research on how to individualize applications to suit client preferences and needs. By no means are mobile applications a panacea, but they are a potentially powerful tool to increase the impact of interventions that we know work when applied properly.



In addition to challenges from patients and clients, therapists are sometimes reluctant to implement the technology. One of the primary concerns in the use of technology for health is the security of data and potential privacy breaches. In the U.S., technologies must be compliant with HIPAA (Health Insurance Portability and Accountability Act), a law enforced by the Department of Health and Human services to protect identifiable health information. The act is meant to protect consumers, but it has had a negative effect on innovation due to the fears of violating privacy and being at risk for lawsuits. Large health organizations have received substantial monetary penalties for having systems that allowed access to patient data. The risk of lawsuits has made many large healthcare organizations hesitant to implement technological innovations into care due to the fear of exposing data. Fines range from thousands of dollars up to a $4.3 million dollar penalty issued to Cignet Health (U.S. Department of Health and Human Services, 2013U.S. Department of Health and Human Services (2013). HHS imposes a .3 million civil money penalty for HIPAA privacy rule violations, Retrieved 9/30, 2013, from http://www.hhs.gov/ocr/privacy/hipaa/news/cignetnews.html.). These fears are particularly pronounced because most people in healthcare do not have expertise in the data management technologies necessary to ensure privacy. Furthermore, even when steps are taken to ensure privacy, determined hackers may still be able to access such data.



On the private sector or developer side, companies are reluctant to develop digital health tools due to the regulations that come with working with patient data and large healthcare system record systems. As a result the majority of applications and devices that are being developed by private companies are targeted to consumers and not to health care providers or organizations. This is problematic because those who are the most motivated to download the latest app or purchase a monitoring device are likely to be better off and in better health than those who may not afford or do not know about such options. Because of policy barriers, both real and perceived private companies are not addressing larger health problems nor integrating into clinical systems due at least in part to liability concerns. 



There are steps that can be taken to limit privacy breaches such as restricting the transfer of sensitive information, using code words, ensuring the information is secure on a single phone, using passwords and deleting messages. Protocols that are in place to address crises should remain in place and do not need to be supplanted by the use of technology. It is important to inform patients about how to use technology and that the use of mobile technology does not necessarily mean that a therapist will be available or will be monitoring messages at all times. Other safety measures could include integrating a safety protocols such as texting the word “HELP” to receive information about a suicide hotline and instructions on going to the emergency room (Aguilera and Muñoz, 2011Aguilera, A. and Muñoz, R. F. (2011). "Text messaging as an adjunct to CBT in low-income populations: A usability and feasibility pilot study". Professional Psychology: Research and Practice, 42 (6), pp. 472, http://dx.doi.org/10.1037/a0025499.). There is also the concern of increased therapist time commitment if one is constantly connected to clients via technology. However, boundaries can be set up similar to boundaries used in current evidence based treatments regarding phone or email contact.



Clinicians and health professionals are also sometimes concerned that technology is simply duplicating their efforts and will result in reduced need for them or in less treatment seeking by potential clients. That thinking fails to recognize that technological applications can be used to enhance care. Even with technology, personal contact and real-time intervention and feedback will still be required to treat most individuals seeking in-person services. Stepped care models are alredy using these technologies as first line treatments and there are simply not enough individual therapists to address the unmet need for mental health problems in the U.S. and globally. Furthermore, the application of technology is improved when combined with a live, trained support (Mohr, Cuijpers and Lehman, 2011Mohr, D.; Cuijpers, P. and Lehman, K. (2011). "Supportive accountability: A model for providing human support to enhance adherence to eHealth interventions". Journal of Medical Internet Research, 13 (1), e30. http://dx.doi.org/10.2196/jmir.1602.). Although technology holds potential promise, most research indicates that uptake of technological solutions is aided greatly by human relationships.


Research Designs


Gold standard research designs such as randomized controlled trials (RCTs) are slow and not always compatible with ever changing technologies. The process of research and testing is slow compared to the development of new technologies. For example in the U.S., large scale studies that are funded by National Institutes of Health grants typically take 1-2 years in the grant preparation, submission, and approval process, 5 years to conduct the study, and 1-2 years to publish the results. The typical 7-11 year time span from grant submission to publication of results will see a variety of changes in technologies that may make the results much less relevant than they were when a research idea was initially being developed. As an example, the last 7-11 years saw the proliferation of text messaging, development of the iPhone and Android mobile phones and accompanying applications for each of those platforms. If one conceived of a technology based intervention 7-11 years ago, it probably would not have included any of these now ubiquitous technologies. Therefore, funding organizations might consider expedited reviews and shorter timelines for reviews. 



Also, the preferred RCT design may not always be the best choice in testing an intervention due to the length of time they require as well as the rigidity of design. As noted in recent reviews on mobile interventions (Kumar et al., 2013Kumar, S.; Nilsen, W. J.; Abernethy, A.; Atienza, A.; Patrick, K.; Pavel, M.; Riley, W. T.; Shar, A.; Spring, B.; Sprujit-Metz, D.; Hedeker, D.; Honavar, V.; Kravitz, R.; Lefevre, R. C.; Mohr, D. C.; Murphy, S. A.; Quinn, C.; Shusterman, V. and Swendeman, D. (2013). "Mobile health technology evaluation: The mHealth evidence workshop". American Journal of Preventive Medicine, 45 (2), pp. 228-236, http://dx.doi.org/10.1016/j.amepre.2013.03.017.; Mohr et al., 2013Mohr, D. C.; Cheung, K.; Schueller, S. M.; Hendricks Brown, C. and Duan, N. (2013). "Continuous evaluation of evolving behavioral intervention technologies". American Journal of Preventive Medicine, 45 (4), pp. 517-523, http://dx.doi.org/10.1016/j.amepre.2013.06.006.; Riley et al., 2013Riley, W. T.; Glasgow, R. E.; Etheredge, L. and Abernethy, A. P. (2013). "Rapid, responsive, relevant (R3) research: A call for a rapid learning health research enterprise". Clinical and Translational Medicine, 2 (1), pp. 1-6, http://dx.doi.org/10.1186/20011326210.), the technology is advancing so quickly that research cannot keep up with development. Other ways of assessing impact besides RCTs, includes continuously evaluating interventions as they are developed (Mohr et al., 2013Mohr, D. C.; Cheung, K.; Schueller, S. M.; Hendricks Brown, C. and Duan, N. (2013). "Continuous evaluation of evolving behavioral intervention technologies". American Journal of Preventive Medicine, 45 (4), pp. 517-523, http://dx.doi.org/10.1016/j.amepre.2013.06.006.) and incorporating novel research designs that can take advantage of advanced statistical methods.

Digital Divide


Despite the ubiquity of many technologies, there are still barriers to access some of these digital technologies. For example, in the US, Mobile phones are widely available and owned, but there is variability in terms of who has broadband internet access in their home, with lower income and education as well as increased age being related to lower connectivity at home (Zickhur and Smith, 2013Zickhur, K. and Smith, A. (2013). Home Broadband 2013. Pew Research Center, Retrieved from: http://www.pewinternet.org/2013/08/26/home-broadband-2013/.). As of May 2013, 56% of people using mobile phones owned a smartphone, however ownership is also largely related to income, education and age. If exposure to technology is variable, then interventions developed on technology platforms may reach variable audiences as well. For example, an iPhone app could be developed to help treat and track a mental health problem such as depression or PTSD but people who do not own those phones will not use the technology. This is particularly concerning since income and education are also related to higher incidences of many health and mental health problems that these technologies are targeting (Aguilera et al., 2012Aguilera, A.; Leykin, Y.; Adler, N. and Muñoz, R. F. (2012). "Assessing the impact of relative social position and absolute community resources on depression and obesity among smokers". American Journal of Community Psychology, 50 (1-2), pp. 211-216, http://dx.doi.org/10.1007/s104640129492z.). The type of smartphone that one owns is also related to income level with higher earners more likely to own iPhones. These are important considerations when developing an intervention that is meant to reach large numbers of people in high need.



Strategies to counter effects of the digital divide include developing interventions with the most ubiquitous yet limited technologies (e.g. text messaging, mobile voice calls) or providing individuals with no or low cost access to advanced technologies (e.g. smartphones and sensor devices). The strategy of using existing technologies is likely to result in more sustainability but is limited in the potential positive impact compared to more advanced technologies. On the other hand, the use of sophisticated smartphones and sensors have potential for collecting more accurate data and thus resulting in more personalized intervention. As the cost of smartphones and sensors decreases, many more individuals will have access to these technologies and in turn newer interventions. However, there will always be a group of people, mostly from low-income backgrounds that will lag in adoption of technology. Therefore researchers such as Aguilera and Muñoz (2011Aguilera, A. and Muñoz, R. F. (2011). "Text messaging as an adjunct to CBT in low-income populations: A usability and feasibility pilot study". Professional Psychology: Research and Practice, 42 (6), pp. 472, http://dx.doi.org/10.1037/a0025499.) have focused on technologies that are the most ubiquitious, such as mobile phone based text messaging in order to create the most scalable and sustainable intervention possible.

Big data


The availability of large quantities of real time data can be both a blessing and a curse for patients, providers and health care systems. More data can help better assess what is occurring in people’s daily lives and help determine how that relates to illness and wellness. Having access to real time data from multiple applications and/or sensor systems might also allow for targeting interventions when individuals actually need them. For example, if a clinician can determine when a patient is feeling depressed or is at risk, a system that is tracking relevant data might alert the clinician so that a targeted outreach is made. This type of system could be applied to assertive community treatment programs that intensively work with the neediest clients. However, these data could easily be overwhelming to clinicians if they are not relevant or well organized. Another problem with the large amounts of data that can be generated from digital health technologies is the interoperability of various applications that may be collecting data and feeding into electronic health records. Applications are often in different formats and have difficulty “speaking to each other” so that data can be integrated. 



The sheer quantity of data can pose problems for storing large data files as well as making sense of what the data mean. For example, a passive activity or location monitor could be collecting data constantly but not all of that data may be useful for determining health outcomes. Analytic techniques and algorithms are necessary to pull out the “signal from the noise.” This is the type of data overload that health systems and providers are concerned will take up more of their time and efforts than is being saved by increased efficiencies in data collection.


Interdisciplinarity


Solving crucial problems such as determining what data to capture and picking out the most important pieces of information requires experts from multiple disciplines. A major challenge in this area of work is that technologists (computer scientists, engineers, etc.) and clinicians/researchers on the health side have different expertise and often different goals. During the development phases of technologies and interventions, academics from both sides of the field are often speaking to different audiences with different goals. The technology side is tasked with developing new technologies that are often years away from reaching real use while health researchers are concerned with technologies that can be tested and implemented with patients today, not years from now. One example of the different goals is the fact that both sides publish articles in different scholarly journals that have vastly different formats and requirements. These “separate worlds” keep both sides of the digital health field from truly integrating.


 

WHAT’S NEXT?
 Top

Digital health technologies initially experienced a heightened level excitement and expectations. However, development and testing have not occurred as rapidly as was imagined, possibly due to the difficulties of collaboration between the technology and health camps. Insurance companies have been hesitant to fund most of these newer uses of technology, possibly because data on their effectiveness are not well established. We are likely heading down from the “peak of inflated expectations” and hopefully moving soon into a more productive period where people are not extolling the virtues of technology as much as they are attempting to solve the problems posed when technology meets health. The future of digital health technologies includes building a solid evidence base for interventions using current technologies, and the development of interventions utilizing cutting edge technology that is not yet as pervasive.



One area of future development will likely include a variety of small sensors that measure biometric and environmental data, which can then be sent to computing systems that can detect events that need targeted intervention. One might imagine a patient with serious mental illness streaming data to providers and that data is then deciphered for instances that are triggering symptoms or functional impairment. Resources could then be targeted more accurately to address specific problems that are based on real time data and not only self report. Many technology applications and sensors in the future will be based on ideas that have thus far only existed in science fiction novels. One example is an electrochemical tatoo that is in development to sense real time biometric data (Jia et al., 2013Jia, W.; Bandodkar, A. J.; Valdes-Ramirez, G.; Windmiller, J. R.; Yang, Z.; Ramirez, J.; Chang, G. and Wang, J. (2013). "Electrochemical tattoo biosensors for real-time noninvasive lactate monitoring in human perspiration". Analytical Chemistry, 85 (14), pp. 6553-6560, http://dx.doi.org/10.1021/ac401573r.). Data resulting from this sensor could then trigger interventions or notifications to healthcare providers. Another novel sensor is a microchip that is embedded into medication and once ingested, sends a signal to a patch on the user’s arm which transmits data to a mobile phone. An application could provide information regarding next dosage and could send data to healthcare providers to inform them about medication adherence. This pill sensor is a good example of a technology that while useful also presents serious concerns about invasion of privacy and coercive treatments. This battle between effective interventions and privacy concerns is likely to continue and will lead to many ethical debates and dilemmas.



The future of health information technology will be best served by allowing for multiple ways of accessing data and information. These technologies will soon be able to provide intelligent just-in-time interventions based on real-time data capture, which have the potential to help us understand and intervene with clients on an exponentially different level. However, enthusiasm for the potential of digital health should not lead to using untested applications or abandoning the empirical process to improve the application of these technologies in an evidence-based manner.

 

CONCLUSION
 Top

There are calls to develop innovative ways of improving the quality and reach of efficacious clinical interventions using innovative tools such as mobile phones and smartphones (Kazdin and Blase 2011Kazdin, A. E. and Blase, S. L. (2011). "Rebooting psychotherapy research and practice to reduce the burden of mental illness". Perspectives on Psychological Science, 6 (1), pp. 21, http://dx.doi.org/10.1177/1745691610393527.; Boschen, 2009Boschen, M. J. (2009). "Mobile telephones and psychotherapy: I. capability and applicability". Behavior Therapist, 32 (8), pp. 168-175.). The use of digital technology in health applications is experiencing a tremendous boom in large part as a result of ballooning health care costs and the limitations of one on one therapy to meet the mental health needs of the population. In particular, mobile phone applications for health have grown very rapidly which provides opportunities to expand current care beyond the traditional clinic setting. While there is a large discrepancy in home Internet access between high and low income households in the US, this is not the case with mobile phone use as more Americans own a mobile phone than a computer (Lenhart, 2010Lenhart, A. (2010). Adults, cell phones and texting, Retrieved 9/28, 2010, from http://pewresearch.org/pubs/1716/adults-cell-phones-text-messages.). There are many challenges to the continued development of digital health technologies but with flexibility, collaboration and empirical testing we can work to ensure that the public receives the benefit of tools that can help improve health and lower the cost curve.

 

REFERENCESTop

Aguilera, A.; Leykin, Y.; Adler, N. and Muñoz, R. F. (2012). "Assessing the impact of relative social position and absolute community resources on depression and obesity among smokers". American Journal of Community Psychology, 50 (1-2), pp. 211-216. http://dx.doi.org/10.1007/s104640129492z
Aguilera, A. and Muñoz, R. F. (2011). "Text messaging as an adjunct to CBT in low-income populations: A usability and feasibility pilot study". Professional Psychology: Research and Practice, 42 (6), p. 472. http://dx.doi.org/10.1037/a0025499
BBC (2012). UN: Six billion mobile phone subscriptions in the world, Retrieved 9/30/2013, 2013, from http://www.bbc.co.uk/news/technology-19925506
Boschen, M. J. (2009). "Mobile telephones and psychotherapy: I. capability and applicability". Behavior Therapist, 32 (8), pp. 168-175.
Boschen, M. J. and Casey, L. M. (2008). "The use of mobile telephones as adjuncts to cognitive behavioral psychotherapy". Professional Psychology: Research and Practice, 39 (5), pp. 546-552. http://dx.doi.org/10.1037/07357028.39.5.546
Burns, M. N.; Begale, M.; Duffecy, J.; Gergle, D.; Karr, C. J.; Giangrande, E. and Mohr, D. C. (2011). "Harnessing context sensing to develop a mobile intervention for depression". Journal of Medical Internet Research, 13 (3), e55. http://dx.doi.org/10.2196/jmir.1838
Christensen, H.; Griffiths, K. M. and Korten, A. (2002). "Web-based cognitive behavior therapy: Analysis of site usage and changes in depression and anxiety scores". Journal of Medical Internet Research, 4 (1), e3. http://dx.doi.org/10.2196/jmir.4.1.e3
Cole-Lewis, H. and Kershaw, T. (2010). "Text messaging as a tool for behavior change in disease prevention and management". Epidemiologic Reviews, 32, pp. 56-69. http://dx.doi.org/10.1093/epirev/mxq004
Eysenbach, G. (2005). "The law of attrition". Journal of Medical Internet Research, 7 (1), p. e11. http://dx.doi.org/10.2196/jmir.7.1.e11
Free, C.; Phillips, G.; Galli, L.; Watson, L.; Felix, L.; Edwards, P.; Patel, V. and Haines, A. (2013). "The effectiveness of mobile-health technology-based health behaviour change or disease management interventions for health care consumers: A systematic review". PLoS Medicine, 10 (1), e1001362. http://dx.doi.org/10.1371/journal.pmed.1001362
Glasgow, R. E.; Boles, S. M.; Mckay, H. G.; Feil, E. G. and Barrera Jr, M. (2003). "The D-net diabetes self-management program: Long-term implementation, outcomes, and generalization results". Preventive Medicine, 36 (4), pp. 410-419. http://dx.doi.org/10.1016/S00917435(02)000567
Gold, B. C.; Burke, S.; Pintauro, S.; Buzzell, P. and Harvey Berino, J. (2007). "Weight loss on the web: A pilot study comparing a structured behavioral intervention to a commercial program". Obesity, 15 (1), pp. 155-164. http://dx.doi.org/10.1038/oby.2007.520
Griffiths, F.; Lindenmeyer, A.; Powell, J.; Lowe, P. and Thorogood, M. (2006). "Why are health care interventions delivered over the internet? A systematic review of the published literature". Journal of medical Internet research, 8 (2). http://dx.doi.org/10.2196/jmir.8.2.e12 / http://dx.doi.org/10.2196/jmir.8.2.e10
Gustafson, D. H.; Shaw, B. R.; Isham, A.; Baker, T.; Boyle, M. G. and Levy, M. (2011). "Explicating an evidence-based, theoretically informed, mobile technology-based system to improve outcomes for people in recovery for alcohol dependence". Substance use & Misuse, 46 (1), pp. 96-111. http://dx.doi.org/10.3109/10826084.2011.521413
Head, K. J.; Noar, S. M.; Iannarino, N. T. and Grant Harrington, N. (2013). "Efficacy of text messaging-based interventions for health promotion: A meta-analysis". Social Science & Medicine, 97, pp. 41-48. http://dx.doi.org/10.1016/j.socscimed.2013.08.003
Heron, K. E. and Smyth, J. M. (2010). "Ecological momentary interventions: Incorporating mobile technology into psychosocial and health behaviour treatments". British Journal of Health Psychology, 15 (1), pp. 1-39. http://dx.doi.org/10.1348/135910709X466063
Institute of Medicine (2001). Crossing the quality chasm: A new health care system for the 21st century.
Intille, S. S. (2007). "Technological innovations enabling automatic, context-sensitive ecological momentary assessment". In Stone, A.; Shiffman, S.; Atienza, A. and Nebeling, L. (eds.), The Science of Real-Time Data Capture. Self-Reports in Health Research, pp. 308-337. Oxford: Oxford University Press.
Jia, W.; Bandodkar, A. J.; Valdes-Ramirez, G.; Windmiller, J. R.; Yang, Z.; Ramirez, J.; Chang, G. and Wang, J. (2013). "Electrochemical tattoo biosensors for real-time noninvasive lactate monitoring in human perspiration". Analytical Chemistry, 85 (14), pp. 6553-6560. http://dx.doi.org/10.1021/ac401573r
Kazdin, A. E. and Blase, S. L. (2011). "Rebooting psychotherapy research and practice to reduce the burden of mental illness". Perspectives on Psychological Science, 6 (1), pp. 21. http://dx.doi.org/10.1177/1745691610393527
Kumar, S.; Nilsen, W. J.; Abernethy, A.; Atienza, A.; Patrick, K.; Pavel, M.; Riley, W. T.; Shar, A.; Spring, B.; Sprujit-Metz, D.; Hedeker, D.; Honavar, V.; Kravitz, R.; Lefevre, R. C.; Mohr, D. C.; Murphy, S. A.; Quinn, C.; Shusterman, V. and Swendeman, D. (2013). "Mobile health technology evaluation: The mHealth evidence workshop". American Journal of Preventive Medicine, 45 (2), pp. 228-236. http://dx.doi.org/10.1016/j.amepre.2013.03.017
Lenhart, A. (2010). Adults, cell phones and texting, Retrieved 9/28, 2010, from http://pewresearch.org/pubs/1716/adults-cell-phones-text-messages
Miranda, J.; Azocar, F.; Organista, K. C.; Muñoz, R. F. and Lieberman, A. (1996). "Recruiting and retaining low-income latinos in psychotherapy research". Journal of Consulting and Clinical Psychology, 64 (5), pp. 868-874. http://dx.doi.org/10.1037/0022006X.64.5.868
Miranda, J.; Chung, J. Y.; Green, B. L.; Krupnick, J.; Siddique, J.; Revicki, D. A. and Belin, T. (2003). "Treating depression in predominantly low-income young minority women". JAMA: The Journal of the American Medical Association, 290 (1), pp. 57-65. http://dx.doi.org/10.1001/jama.290.1.57
Mohr, D.; Cuijpers, P. and Lehman, K. (2011). "Supportive accountability: A model for providing human support to enhance adherence to eHealth interventions". Journal of Medical Internet Research, 13 (1), e30. http://dx.doi.org/10.2196/jmir.1602
Mohr, D. C.; Cheung, K.; Schueller, S. M.; Hendricks Brown, C. and Duan, N. (2013). "Continuous evaluation of evolving behavioral intervention technologies". American Journal of Preventive Medicine, 45 (4), pp. 517-523. http://dx.doi.org/10.1016/j.amepre.2013.06.006
Morris, M. E.; Kathawala, Q.; Leen, T. K.; Gorenstein, E. E.; Guilak, F.; Labhard, M. and Deleeuw, W. (2010). "Mobile therapy: Case study evaluations of a cell phone application for emotional self-awareness". Journal of Medical Internet Research, 12 (2), e10. http://dx.doi.org/10.2196/jmir.1371
Morris, M. E. and Aguilera, A. (2012). "Mobile, social, and wearable computing and the evolution of psychological practice". Professional Psychology: Research and Practice, 43 (6), pp. 622-626. http://dx.doi.org/10.1037/a0029041
Muench, F.; Weiss, R. A.; Kuerbis, A. and Morgenstern, J. (2013). "Developing a theory driven text messaging intervention for addiction care with user driven content". Psychology of Addictive Behaviors, 27 (1), pp. 315-321. http://dx.doi.org/10.1037/a0029963
Muñoz, R. F. (2010). "Using evidence-based internet interventions to reduce health disparities worldwide". Journal of Medical Internet Research, 12 (5), e60.
Muñoz, R. F.; Lenert, L. L.; Delucchi, K.; Stoddard, J.; Perez, J. E.; Penilla, C. and Pérez-Stable, E. J. (2006). "Toward evidence-based internet interventions: A spanish/english web site for international smoking cessation trials". Nicotine and Tobacco Research, 8 (1), pp. 77-87. http://dx.doi.org/10.1080/14622200500431940
Murray, E.; McCambridge, J.; Khadjesari, Z.; White, I. R.; Thompson, S. G.; Godfrey, C.; Linke, S. and Wallace, P. (2007). "The DYD-RCT protocol: An on-line randomised controlled trial of an interactive computer-based intervention compared with a standard information website to reduce alcohol consumption among hazardous drinkers". BMC Public Health, 7 (1), p. 306. http://dx.doi.org/10.1186/147124587306
Proudfoot, J.; Parker, G.; Pavlovic, D. H.; Manicavasagar, V.; Adler, E. and Whitton, A. (2010). "Community attitudes to the appropriation of mobile phones for monitoring and managing depression, anxiety, and stress". Journal of Medical Internet Research, 12 (5), e64. http://dx.doi.org/10.2196/jmir.1475
Riley, W. T.; Glasgow, R. E.; Etheredge, L. and Abernethy, A. P. (2013). "Rapid, responsive, relevant (R3) research: A call for a rapid learning health research enterprise". Clinical and Translational Medicine, 2 (1), pp. 1-6. http://dx.doi.org/10.1186/20011326210
Rizvi, S. L.; Dimeff, L. A.; Skutch, J.; Carroll, D. and Linehan, M. M. (2011). "A pilot study of the DBT coach: An interactive mobile phone application for individuals with borderline personality disorder and substance use disorder". Behavior Therapy, 42 (4), pp. 589-600. http://dx.doi.org/10.1016/j.beth.2011.01.003
U.S. Department of Health and Human Services (2013). HHS imposes a $4.3 million civil money penalty for HIPAA privacy rule violations, Retrieved 9/30, 2013, from http://www.hhs.gov/ocr/privacy/hipaa/news/cignetnews.html
Williamson, D. A.; Walden, H. M.; White, M. A.; York Crowe, E.; Newton, R. L.; Alfonso, A.; Gordon, S. and Ryan, D. (2006). "Two Year Internet Based Randomized Controlled Trial for Weight Loss in African American Girls". Obesity, 14 (7), pp. 1231-1243. http://dx.doi.org/10.1038/oby.2006.140
Zickhur, K. and Smith, A. (2013). Home Broadband 2013. Pew Research Center. Retrieved from: http://www.pewinternet.org/2013/08/26/home-broadband-2013/