This review was conducted in accordance with PRISMA guidelines [46] and registered on PROSPERO (Prospero Registration Number: CRD42013003904)[47]. Peer-reviewed English citations up to February 27, 2013, on the databases PsycINFO, PubMed, and the Cochrane Library, were searched using search terms representing three concepts: Web-based technology, mental health, and help-seeking (details described in Multimedia Appendix 1). All controlled and uncontrolled studies were eligible for inclusion including qualitative studies. The reference lists of all relevant studies, reviews, and meta-analyses identified in this search were also manually searched for inclusion into the review.

The first author examined all titles and abstracts extracted for relevance and read the full text for any potentially eligible article. Table 1 describes the exclusion criteria. The second author confirmed that all selected articles were eligible for inclusion.

The first author extracted data from the studies using the form described in Table 2, which was confirmed by the second author. Risk of bias within each study was assessed using a version (Multimedia Appendix 2) of the Quality Rating Scale (QRS) [48] adapted to include qualitative and uncontrolled studies. The first two authors used the QRS independently. Interrater reliability was assessed by Cohen’s absolute weighted kappa statistic in Stata Version 12.0. Weighted kappa allows for different levels of agreement in ordered data, and the absolute function allows for all numbers including those unassigned by either rater.

The country, research group, and year published were also examined to determine whether there were biases across studies in terms of the country of research, research group, or years in which the research was conducted. All studies were included irrespective of their design, quality, and biases. As suggested by Linde and colleagues [49], this review included uncontrolled trials and qualitative studies to obtain an overview of the topic and inform future research. No statistical analyses were conducted due to the heterogeneity of study designs permitted in the inclusion criteria. Instead, a broad qualitative overview of the data was conducted and statistical analyses from the studies were reported.

A total of 487 articles were identified through the literature search. All relevant review and research papers [37,41,50-70] were then manually searched, which uncovered a further 121 potential studies. Of these 608 papers, 405 were excluded based on their abstracts and a further 149 excluded after the full article was examined, leaving a total of 18 studies to review. Figure 1 depicts the PRISMA flow diagram for inclusion.

Eighteen publications met the inclusion criteria including two qualitative [51,55], 12 cross-sectional [53,56,60,63,64,71-77], one quasi-experimental [68], and three randomized controlled trials (RCTs) [52,78,79]. The sample sizes ranged from 9 to 2700: median 420, mean 762.3 (SD 838.10). There were a high proportion of females in the studies, ranging from 50% to 80%: median 68%, mean 67.1% (SD 9.73%). The characteristics of each study are presented in Table 3.

The majority of participants were students at a university [55,56,60,63,74,76] or high school [73]. Other studies involved users of online services [53,68,72,75,77]. Within these settings, 10 studies targeted young people with mild to moderate mental health problems [52,53,55,63,64,68,72,75-77], one study focused on young athletes at risk of mental health problems [79], and seven studies involved all young people regardless of mental health status [51,56,60,71,73,74,78].

The types of programs investigated were varied as outlined in Table 3. Most commonly, the use of several programs and websites [56,60,71,73,74,76] were investigated, such as online information sites, chat groups, self-directed online therapy, and forums. Other studies investigated a specific information site [53,64,77-79], a self-directed online therapy [51,52,55], a discussion forum about mental health problems [72,75], a Web-based mental health educational game [68], and a screening tool [63].

Figure 2 shows the age ranges and means for each study. Most studies (11/18) were inclusive of the target age range; 55% (10/18) of studies focused on participants who were 18 years of age or older. The mean age of participants for the studies ranged from 16.5-26.2 years of age.

A range of barriers and facilitators of help-seeking were mentioned in the articles, although these were rarely measured empirically. Barriers to help-seeking included lack of awareness [51,56,76], being male [68,71,73], a preference for face-to-face services [73,74], lack of motivation [51], uncertainty about confidentiality [74], and unfavorable content [64,76]. Facilitators to help-seeking included accessibility to online resources [51,53,55,71,74], the ease of sharing personal information compared to face-to-face services [74-76], anonymity [51,71,74,75], trust and credibility [53], reduction of stigma [55,68,79], high distress [55,73], and an increase in mental health literacy [53,68,74-79]. Table 4 describes these facilitators and barriers in detail.

Young people’s experiences with online services were investigated in 50% (9/18) of the studies. Generally, participants were asked to rate how helpful the service was [56,63,71,73,76,78], how easy it was to use [63,68], whether they would use it again [63,68], whether they would recommend it to others [53,68,71,73], or if they were satisfied with the service [63,71,73]. There was high variability in the measures used, and therefore the results are difficult to compare between studies. All of these studies used from one to six questions, which were constructed by the authors. No standard measures were used. User experience is summarized in Table 5.

Overall, experiences of the online services were positive. Of the nine studies that evaluated these experiences, 90% of participants were satisfied with the service, 86% would continue to use the service or use it again in the future, and 72% would recommend it to a friend. However, approximately half of participants received the information they were looking for, and only 65% found the programs helpful.

The adapted QRS ranged from 6 to 37 for both raters and had similar means (Rater 1 mean 19.5 [SD 7.45]; Rater 2 mean 20.2 [SD 8.66]). The qualitative and cross-sectional studies scored low, between 6 and 24 for both raters. The quasi-experimental study [68] scored 20 by rater 1 and 29 by rater 2, and the RCTs scored high from 31 to 37 by both raters. Interrater agreement was higher (89.8%) than the expected agreement (76.1%) with an acceptable kappa score (κ=.57, CI 0.32-0.82, P<.001). All studies were published between 2002 and 2012 (median 2010), with 15 studies occurring in the last 5 years (2009-2012). Seven studies [52,53,55,63,77-79] developed the intervention that they were evaluating, which may have led to a bias towards more favorable reporting.

There was a bias towards affiliations with four organizations, with nine studies conducted within Australia compared to three in the United States, two in Canada, and one each in Germany, Ireland, Norway, and United Kingdom. Three studies were from the Inspire Foundation, Sydney [53,71,77], two from the National eTherapy Centre at Swinburne University in Melbourne [64,68], two from the Centre for Mental Health Research at Australian National University in Canberra [78,79], and two from the Faculty of Health Sciences at the University of Sydney [56,76]. The remaining nine were not affiliated with the other studies included in this review. Table 6 lists the study, research group, year of publication, country where the research was conducted, and QRS.

The majority of studies included a comprehensive list of limitations: small sample size [51,52,55,60,68,78,79], resulting in insufficient power to detect a change [52,68,78,79]; a self-selected sample, not representative and possibly biased [56,60,63,68,73,77-79]; majority of participants were female [51,56,60,64,68,79]; a lack of longitudinal tracking in the study [53,71,77]; only one online mental health service was investigated [51]; only one behavior change theory was investigated [51]; limited outcome measures [52,60,64,73]; insufficient length of follow-up time points [78]; no quantitative analysis [55]; non-validated and possibly biased measures [56,64,74,77]; and lack of qualitative information to provide depth about attitudes [77].

There is a plethora of online services available with the aim of facilitating help-seeking for young people with mental health problems [80], yet only 18 studies were identified in this review that evaluated whether these services increased help-seeking in young people. Overall, these studies did not indicate that online services facilitate mental health help-seeking in young people. No change in help-seeking was found in the three RCTs [52,78,79]. The quasi-experimental study found a slight but significant increase in help-seeking [52,68,78,79]. The cross-sectional studies found that only 35% of participants indicated the services helped them seek help from a professional. Only 52% of participant also reported that they received the information they wanted, and only 65% found the services helpful.

Despite these unfavorable results, the results show that young people regularly used these services, would generally recommend the services to friends, would use them again, and generally found the services easy to use and satisfactory. Furthermore, young people suggested that these services were accessible and available [51,53,55,71,74], anonymous [51,71,74,75], allowed personal stories to be shared with others [55,73-76], were less stigmatizing than phonelines and face-to-face services [68,79], and trustworthy [53]. This finding suggests that online services fulfil a need, although perhaps do not increase help-seeking. Further exploration into what young people use these services for is warranted.

As with face-to-face services, some barriers to online services remain such as lack of awareness of online resources [51,56,76], young men seeking online help less often than females [68,71,73], some young people’s preference for face-to-face services [73,74], lack of motivation to seek help online [51], the lack of trust of websites [74], and unfavorable content [64,76].

Interestingly, young women were overrepresented in online mental health services in much the same way as in traditional face-to-face care [81]. This could be due to young women using online services, or participating in research, more often than young men. A recent study indicated that young men were generally high use Internet users [82] with 55% seeking help online. Utilizing the advantages of technology by specifically tailoring services for young men may increase rates of young men seeking help online as well as face-to-face [82]. Using the Internet may also assist with young people who prefer face-to-face services by directing young people to appropriate face-to-face services in local areas in the form of online directory services.

Furthermore, online services often include information about mental health literacy [83], providing young people with information about mental illness, where to get help, and what to expect at services. This type of information may increase readiness for care and motivation to seek help.

Another interesting finding was that online services appeared to increase mental health literacy [53,74-77,79], although two studies found no change in mental health literacy [68,78]. Improvements in mental health literacy are likely to assist young people in recognition and management of mental health and may also reduce the self-stigma associated with mental illness [84]. Mental health literacy is associated with seeking help from appropriate treatment and professional services [85]. A recent meta-analysis demonstrated that interventions with a focus on mental health literacy significantly improved help-seeking intentions, although no effect was found for help-seeking behaviors [86]. Ensuring that websites maintain young people’s confidentiality and anonymity appears to be critical to increasing usage of online services.

This systematic review highlights the need for rigorous evaluation methods of online help-seeking programs. The methodology was generally poor across the studies with a high risk of bias. Samples were generally small, the studies often included short-term, if any, follow-up, and non-validated measures of help-seeking behavior and intentions were often used, in some cases with just one question to assess the complex concept of help-seeking. Help-seeking was not the primary outcome for most studies. It is also important to note that help-seeking was not the primary purpose of some of the services, such as discussion forums and self-help programs, where the primary purpose may have been self-help rather than assisting young people to seek help from other services. These methodological issues may account for the lack of change in help-seeking. Also, despite the general satisfaction with online services reported, evaluation of satisfaction was poor. No standard measures of satisfaction were used in the studies limiting the ability to compare user experiences across studies. Short user satisfaction measures are available such as the validated Client Satisfaction Questionnaire (CSQ-3 or 8) [87], which summed together gives an understanding of client’s satisfaction of the service.

Structured online treatment programs, though effective at reducing adolescent depression and anxiety [36-40,88], have poor uptake [89], high dropout rates, and the reduction of mental health symptomatology is not maintained long term [90]. In contrast, unstructured mental health websites have high uptake [77], allowing users to explore the contents, select links that appeal to them, and disregard information that is not relevant or interesting. The primary aim of these services is to give young people information about mental health as well as facilitate help-seeking and pathways to mental health care; however, these aims are rarely evaluated [45]. It is time now to focus on whether these sites facilitate help-seeking and improve well-being to ensure that the online services we provide to young people assist their help-seeking journey and lead to better outcomes and better access to care.

High-quality randomized control trials are needed before the implementation of new services as well as ongoing longitudinal trials to ensure the efficacy of existing services. These trials should include large representative samples, long-term follow-up measures of at least 6-12 months, the use of appropriate, validated measures of help-seeking behavior, help-seeking intentions, beliefs about help-seeking, and client satisfaction. Analyses should also be appropriate and indicate an effect size for future inclusion into meta-analyses.

This review is timely and highlights the need to properly evaluate websites aiming to assist young people with their mental health problems and seek help. Including uncontrolled studies in this review allowed for a broad overview of research in this area to date, and as only three RCTs were found, a meta-analysis was not possible. Some studies included were primarily self-help websites, therefore one would hope that further help-seeking was reduced rather than increased. Nevertheless, as the studies themselves included help-seeking as a primary or secondary aim, it can be assumed that help-seeking was considered a goal of these studies. One of the strengths of this review was the focus on both help-seeking behaviors and intentions, as intentions do not always translate into behavior. Further research is needed to explore the mechanisms that facilitate and hinder this process.

At present, there is a paucity of research exploring the relationship between online services and help-seeking behavior. This is not to say that there is no benefit in online services, rather, that this field has yet to be properly evaluated. Only 35% of young people experiencing mental health problems seek professional face-to-face help [91,92]. Online mental health services may conceivably assist in all elements of the help-seeking process; however, further research into the effectiveness of online services, how they interact with face-to-face services, and whether online services can overcome barriers to mental health care, facilitate readiness for care, and increase help-seeking behavior is needed.