Revista Electrónica de Investigación Educativa
Vol. 26, 2024/e06

Multidimensional Scale to Assess Digital Security in Adolescents

José Néstor Peraza-Balderrama
Instituto Tecnológico de Sonora, México
Ángel Alberto Valdés-Cuervo
Instituto Tecnológico de Sonora, México
Lizeth Guadalupe Parra-Pérez
Instituto Tecnológico de Sonora, México
Maricela Urías Murrieta
Instituto Tecnológico de Sonora, México
Received: March 1, 2021
Accepted: June 29, 2021

How to cite: Peraza-Balderrama, J. N., Valdés-Cuervo, A. A., Parra-Pérez, L. G. y Urías-Murrieta, M. (2024). Multidimensional scale to assess digital security in adolescentes. Revista Electrónica de Investigación Educativa, 26, e06, 1-19.

Licencia Creative Commons


The study examined the psychometric properties of a Digital Security Scale (DSS-A) in a sample of 868 adolescents (460 female, 408 male), made up of 424 early adolescents (M age = 11.84 years, SD = 0.36) and 444 middle adolescents (M age = 13.44, SD = 0.49). The sample was randomly split into two for validation (n = 434) and cross-validation (n = 434) purposes. The results confirmed the fit of a one second-order factor model with four first-order factors (personal data protection, safe relationships, healthy internet use, and self-disclosure). As expected, digital security dimensions were positively associated with parental mediation. Measurement invariance was found in this model by gender and stage of adolescence. Latent mean comparisons showed differences by gender and stage of adolescence. Overall, findings indicate that the measurement model is helpful for measuring adolescents’ digital security behaviors.

Keywords: citizenship, digital citizenship, adolescents

I. Introduction

The internet influences adolescents’ learning, communication, and social participation (Babaoglu & Akman, 2022; Hurwitz & Schmitt, 2020). Adolescents are the largest group of internet users (Organization for Economic Cooperation and Development [OECD], 2022). In Mexico, 90% of adolescents use the internet (National Institute of Statistics and Geography [INEGI], 2022). Even though responsible internet use improves adolescents’ learning, psychosocial development, and communication with peers and families (Padilla-Walker et al., 2012; Pereira et al., 2019; Skryabin et al., 2015), it also poses harmful threats associated with problematic and addictive internet use (Aydın et al., 2020; Pastor et al., 2022). Civil society organizations and governments have proposed addressing these adverse effects by educating adolescents to engage in healthy and responsible online behaviors. In line with this international strategy, the Mexican government has improved the national education curriculum with content aimed at promoting civic behaviors in online environments (Secretaría de Educación Pública [Secretariat of Public Education, SEP], 2022).

Digital citizenship is a suitable framework for fostering healthy and responsible internet behaviors in adolescents (Emejulu & McGregor, 2019; Jones & Mitchell, 2016; Ribble, 2015) and comprises multiple dimensions. One of the most relevant of these dimensions is digital security, which involves practices aimed at keeping internet users safe from any danger caused by internet use (Gallego-Arrufat et al., 2019; Ribble & Bailey, 2011). Unsafe internet use may have harmful and dangerous consequences, such as an increased risk of falling victim to sexual offenses (Gottfried et al., 2020), grooming (Orosco & Pomasunco, 2020), cyberbullying (Madrid López et al., 2020), or cyber dating violence (Chugh & Guggisberg, 2022). Therefore, digital security education remains essential in school settings to support the reduction of risky online behaviors (Curran & Ribble, 2017; Ribble, 2015).

1.1 Measures of digital security

There is a consensus on digital security as a multidimensional construct that involves multiple aspects such as personal data protection, safe relationships, healthy internet use, and self-disclosure (Chen et al., 2022; Curran & Ribble, 2017; Gallego-Arrufat et al., 2019; Hamza et al., 2019; Nowell et al., 2022). Personal data protection involves actions aimed at averting wide-ranging online problems related to the misuse of personal information (Chen et al., 2021). Healthy internet use refers to actions necessary to ensure individuals’ physical and psychological health (Chen et al., 2022; Gallego-Arrufat et al., 2019; Hamza et al., 2019). Safe relationships mean being careful about one’s online relations, which can lead to risky and painful outcomes. Finally, self-disclosure involves voluntarily sharing information about online activities and relationships (Lionetti et al., 2016; Shin & Kang, 2016).

Although digital security is a multidimensional construct (e.g. Lauricella et al., 2020; Martin et al., 2020; Nowell et al., 2022), and measure just a single dimension of digital security in isolation (e.g. Lionetti et al., 2016; Liu et al., 2013). However, digital security dimensions have been understood as interwoven, and instruments must explore the relationship between them. To address this gap, we aimed to develop a theoretically grounded scale that captures the four aspects of digital security (personal data protection, safe relationships, healthy internet use, and self-disclosure with parents or other adults).

1.2 Measurement of invariance

According to several studies (Kapetanovic et al., 2017; Tifferet, 2019), online security varies by gender. Females display higher levels of digital security than males during adolescence. Similarly, middle adolescents report higher levels of digital security than do early adolescents (e.g. Livingstone et al., 2011; Martin et al., 2020). Regardless of their relevance, these previous studies did not examine measurement invariance by gender or stage of adolescence to ensure the validity of findings. Measurement invariance tests the degree to which measurements conducted with different groups exhibit similar psychometric properties. Therefore, it is critical to make a valid and meaningful group comparison to examine the measurement invariance of digital security scales. These scores can then be used to examine differences in variables associated with digital security behaviors for these groups (Widaman & Olivera-Aguilar, 2023).

1.3 Validity evidence based on relations with external variables

Empirical evidence suggests parenting practices should explain adolescents’ digital security behaviors (Wang & Xing, 2018). Parental media mediation refers to parents’ practices that regulate children’s internet use (Nikken & Jansz, 2014). Active mediation occurs when parents speak with their children about media content and its benefits and risks. By contrast, restrictive mediation involves setting rules that limit the time children spend on the internet or the content they access (Nikken & Jansz, 2014). Empirical evidence has confirmed that active parental mediation enhances adolescent digital security practices (Dedkova & Mýlek, 2022; Fu et al., 2020). However, findings for restrictive mediation are inconsistent; some studies report that it helps to improve adolescents’ safety behaviors on the internet (Lee, 2013; Chen & Shi, 2019), whereas others point to continued higher risks (Lukavská et al., 2022; Steinfeld, 2021). Based on these results, we expected that adolescents’ digital security behaviors would be associated with both types of parental mediation.

1.4 The present study

Based on previous work that conceptualizes online security as a multidimensional construct (Chen et al., 2022; Curran & Ribble, 2017), this study develops a new measure, the Digital Security Scale in Adolescents (DSS-A), and examines the validity of the scale scores in a sample of Mexican adolescents. We tested the content validity and dimensionality of a second-order measurement model of digital security with four first-order factors: (a) personal data protection, (b) safe relationships, (c) healthy internet use, and (d) self-disclosure. We measured score reliability with McDonald's omega and average variance extracted. Moreover, the discriminant validity of each first-order factor was examined. Measurement invariance by gender and stage of adolescence (early vs. middle) was tested. If scalar invariance was confirmed, latent means across gender and stage of adolescence were compared. Evidence of validity based on relations with external variables was assessed by calculating the relationship between digital security and styles of parental mediation (active and restrictive). Finally, we tested model replicability using an independent sample (cross-validation).

To accomplish the research goals, we tested the following hypotheses. Hypothesis 1 (internal structure): A second-order model with four first-order factors fits the data. Hypothesis 2 (reliability): The McDonald's omega coefficient and average variance extracted indicate acceptable score validity. Hypothesis 3 (discriminant validity): Each subscale is genuinely distinct. Hypothesis 4 (measurement invariance): The measurement model is psychometrically invariant by gender and stage of adolescence. Hypothesis 5 (means comparisons): Females and middle adolescents display a greater degree of personal data protection behaviors, safe relationships, healthy behaviors, and self-disclosure than do males and early adolescents, respectively. Hypothesis 6 (relations with external variables): Correlations exist between the four dimensions of digital security and the two types of parental mediation examined. Hypothesis 7 (cross-validation): The measurement model replicates in the cross-validation sample.

II. Method

2.1 Participants

The research participants were students from 52 urban middle schools from Sinaloa and Sonora, Mexico. As public schools in Mexico, these schools include students of low and medium socioeconomic status (National Institute for the Evaluation of Education [INEE], 2019). The sample includes 868 adolescents (460 female, 408 male), comprising 424 early adolescents (M age = 11.84 years, SD = 0.36) and 444 middle adolescents (M age = 13.44 years, SD = 0.49). All the students included in the sample have internet access. The sample was randomly split into two for validation (n = 434) and cross-validation (n = 434) purposes.

2.2 Measures

Digital security. The initial version of a pool of 16 items aimed to measure the dimensions of digital security (see Table 1) on a five-point Likert scale (0 = Never to 4 = Always). These items were created from focus group discussion analysis. The focus group included 20 adolescents (10 males and 10 females) from middle and high schools, aged between 12 and 18 years (M = 15.2 years, SD = 2.3). We presented a brief definition of digital security to research participants and asked them to discuss three issues related to this concept: (a) How do you define digital security? (b) What strategies do you use to keep safe online? (c) How have those strategies worked for you?

The focus group work provided critical information that allowed us to propose a set of items to measure digital security and its four dimensions. Subsequently, we used a panel of experts to assess the content validity of the proposed items; eight experts in educational research were asked to evaluate the relevance of all the items to measure four dimensions of digital security on a 4-point Likert scale (1 = not relevant to 4 = very relevant). Next, we calculated the item content validity index (CVI). All the items with a CVI > .79 were kept for further analysis (Almanasreh et al., 2019); the remaining seven items were discarded.

Afterward, we conducted nine cognitive interviews, including five adolescents and five middle school teachers, to judge if the 13 proposed items were appropriate for measuring digital security in adolescence. We emphasized the importance of verifying the clarity of the instructions, grammatical or semantic issues, and cultural or contextual issues in all the items. As a result of cognitive interviews, seven items were reworded based on participants’ suggestions. This work led to a theoretically grounded scale validated by educational researchers and revisited by the ultimate users: adolescent students. The final version of the scale employs three items, in each case, to measure personal data protection, safe relationships, and healthy internet use, and four items to assess self-disclosure.

The final form of the DSS-A comprises 13 items grouped into four dimensions (see Appendix 1): (a) Personal data protection (3 items, e.g. “I avoid posting personal information on social media.”); (b) Safe relationships (3 items, e.g. “I reject friendship requests from strangers on social media.”); (c) Healthy internet use (3 items, e.g. “I limit the time that I spend on the internet in order to avoid physical repercussions such as bad eyesight, backache, and neck pain.”); and (d) Self-disclosure (4 items, e.g. “I talk to my parents about my social media friendships.”).

Parental mediation. We used the Parental Mediation of Young Children’s Internet Use Scale (Nikken & Jansz, 2014) to measure how students perceive parents’ actions to mediate their Internet use. The back-translation method was used to translate the scale from English to Spanish. These items were used to measure active mediation and restrictive mediation. Active mediation explores how often parents talk to children about internet use and digital content consumption (8 items, McDonald's omega ω = .84; e.g. “How often do your parents speak with you about the rules you must follow while surfing the internet?”). Restrictive mediation examines how often parents limit and regulate children’s internet use (8 items, ω = .86; e.g. “How often do your parents tell you how long you are allowed to use the internet?”). Responses were given on a five-point Likert scale (0 = never to 4 = always). Results from confirmatory analysis (CFA) proved the model fit the data (SBX2 = 30.67, df = 13, p = .004; SRMR = .06; CFI = .98; TLI = .97; RMSEA = .05, 90% CI [.025, .071].

2.3 Procedure

The study gained approval from the Ethical Research Committee of the Technological Institute of Sonora (ITSON). The schools agreed to participate voluntarily in the study. A consent letter was sent to parents/guardians to request permission for children to participate in the study. Permission was denied by about 6% of parents. Finally, adolescents were informed about the study’s purpose and the nature of their potential participation. Data collection was carried out in classrooms during school hours.

2.4 Statistical analysis

Item distribution. The percentage of missing data in the sample was 3%. Missing data were treated using multiple imputation (MI) available in SPSS 27. Then descriptive analyses (means, standard deviation, skewness, and kurtosis) were run for each item of the DSS-A. We tested univariate normality based on the skewness and kurtosis values (Ho, 2014). The statistical Z value is calculated as Z Skew = Skew/√se Skew and Z Kurt = Kurt/√se Kurt. If the Z value exceeds +- 2.86, the normality assumption at the .01 critical probability level is rejected. Based on the rules of thumb proposed in the literature, we assumed that values of kurtosis smaller than 7 and skewness smaller than 3 indicated that deviation from univariate normality did not affect the model estimation (Bandalos & Finney, 2019). Furthermore, in the model assessment we used a robust procedure (Satorra-Bentler corrections) that is unaffected by departures from normality (Mueller & Hancock, 2019).

Internal structure validity. To assess the dimensionality of digital security, we examined the fit of a four first-order factor model and calculated all possible covariance between the factors. After confirming the fit of this measurement model, we tested the four factors as indicators of a second-order dimension (see Figure 1). We conducted a confirmatory factor analysis using the diagonally weighted least squares (DWLS) method with LISREL 12 software. Exploratory factor analysis was not carried out due to the soundness of the theory used to develop the scale.

Based on structural modeling literature, we evaluated the goodness-of-fit indices using the Satorra-Bentler chi-square adjustment (SBX2 with p > .001). Due to the relatively large sample size, additionally, we used the standardized root mean square residual (SRMR ≤ .08), Tucker-Lewis index (TLI ≥ .90), comparative fit index (CFI ≥ .95), and root mean square error of approximation (RMSEA ≤ .08) (Brown, 2023). We used differences in SBX2 (ΔSBX2) and the Bayesian information criterion (ΔBIC) for the model comparison. When ΔSBX2 is significant (p < .001), a model with a smaller value of SBX2 provides a better fit. Differences in BIC > 10 suggest a different model fit to the data; a model with lower BIC better fits the data (Muthén & Muthén, 2017; Vrieze, 2012).

Reliability. Reliability was measured by calculating the average variance extracted (AVE) and McDonald’s omega; AVE > .50 and ω > .70 were indicators of acceptable score reliability (Green & Yang, 2015; Hair et al., 2010).

Discriminant validity. Discriminant validity indicates the uniqueness of the measurement construct. Scholars suggest that discriminant validity is confirmed when the AVE of the dimensions is larger than the construct’s squared correlation (R2) (Hair et al., 2017).

Measurement invariance. We used a multigroup approach to test several nested models to examine the level of measurement invariance by gender and educational level (Widaman & Olivera-Aguilar, 2023). We constrained the number of factors and the patterns of factor loadings to be the same across gender and stage of adolescence (configural invariance). Then, we constrained factor loadings to be equal across groups for the two variables (metric invariance). Finally, we constrained the measurement intercepts to be equivalent across groups to assess scalar invariance. Values of ΔSBX2 with p > .05, ΔCFI < .01, and ΔRMSEA < .015 were considered indicators of invariance (Widaman & Olivera-Aguilar, 2023). When the values of these procedures diverged, we relied on ΔCFI and ΔRMSEA.

As scalar invariance was confirmed, we assessed latent mean differences by gender and stage of adolescence (early vs. middle). To test latent mean differences, we fit gender and stage of adolescence to zero. A z statistic was used to calculate the latent mean differences by gender and stage of adolescence.

Validity evidence based on relations with external variables. This study examined the correlation between dimensions of the DSS-A and active and restrictive parental mediation. Concurrent validity evidence is confirmed when scores correlate as expected with other constructs measured simultaneously (Furr, 2018). We adopted guidelines proposed by Funder and Ozer (2019) to assess the effect size: r = .10 small, r = .20 medium, and r = .30 large.

Model cross-validation. As suggested in the literature, we used a multigroup invariance procedure to test measurement model equivalence in the independent sample (see Widaman & Olivera-Aguilar, 2023). This strategy explores whether a model tested in a specific sample can be replicated over a second independent sample from the same population. Configural, metric, and scalar invariance were estimated using multigroup analysis for the nested models. Model sample invariance was confirmed when ΔSBX2 was not statistically significant (p > .001), ΔCFI < .01, and ΔRMSEA < .015 (Widaman & Olivera-Aguilar, 2023). Since SBX2 is sensitive to large sample sizes (Brown, 2023), we relied on CFI and RMSEA differences when the criteria were inconsistent.

III. Results

3.1 Descriptive analysis

Table 1 shows means, standard deviations, skewness, and kurtosis for the items of the DSS-A. The participants’ responses centered on the categories “seldom” and “sometimes,” suggesting that adolescents frequently forgo digital security practices. Items 1, 8, 9, 11, and 13 depart from normality. Furthermore, skew and kurtosis values are less than 2 and 7, respectively, suggesting that it is unlikely there is substantial distortion in the data (Bandalos & Finney, 2019).

Table 1. Descriptive statistics for the digital security scale
Item M SD Skew Zskew Kurt Zkurt CVI
1. I avoid posting personal information on social networks. 2.20 1.67 -0.18 (0.12) -0.52 -1.63 (0.23) -3.39** .87
2. I make a copy of important information in digital files or in the cloud. 1.82 1.49 0.20 (0.12) 0.59 -1.36 (0.23) -2.83 .90
3. I only provide personal information on trusted websites. 2.56 1.37 -0.46 (0.12) -1.35 -1.05 (0.23) -2.18 .88
4. On social networks, I reject friend requests from people I do not know offline. 2.85 1.42 -0.91 (0.12) -2.67 -0.62 (0.23) -1.29 .86
5. I block people who try to harm me from my social networks. 2.74 1.42 -0.71 (0.12) 2.09 -0.89 (0.23) -1.85 .92
6. I block people who make me feel stressed, anxious or uncomfortable from my social networks. 2.13 1.35 -0.08 (0.12) -0.23 -1.07 (0.23) -2.23 .94
7. I limit the time I spend online to avoid causing harm to my physical health. 2.58 1.30 -0.51 (0.12) -1.50 -0.88 (0.23) -1.83 .84
8. I can easily go several hours without connecting to the internet. 2.31 1.51 -0.24 (0.12) -0.70 -1.42 (0.23) 2.95** .93
9. I avoid internet sites that cause me concern, discomfort or stress. 2.14 1.53 -0.09 (0.12) -0.26 -1.43 (0.23) 3.04** .86
10. I ask my parents for help when I have problems with personal information online. 2.07 1.45 -0.05 (0.12) -0.14 -1.32 (0.23) -2.75 .85
11. I talk to my parents about my best online friends. 1.98 1.54 0.03 (0.12) .009 1.48 (0.23) -3.08** .92
12. I ask my parents for help when I feel that using digital media is harming my physical health. 1.89 1.44 0.11 (0.12) 0.32 -1.31 (0.23) -2.73 .89
13. I talk to my parents about the harmful psychological effects of internet use (e.g. stress, worry or unease). 2.20 1.67 -0.18 (0.12) 0.53 -1.63 (0.23) 3.39** .94

3.2 Validity evidence based on internal structure

The CFA results confirmed the goodness-of-fit of the first-order four-dimensional model (Model A) to data. The correlations between the four factors were statistically significant, from moderate (r = .33) to high (r = .51), suggesting that a second-order model is plausible. Then, we tested a second-order model (Model B) with four first-order digital security factors that fit the data well (see Figure 1). The difference in fit between Model A and Model B was statistically significant (ΔSBX2 = 10.96, df = 1, p < .001). Additionally, the change in the BIC was greater than 10, indicating a difference between the fit data for the models. Thus, based on statistical and theoretical considerations, we decided to work with the second-order measurement model (see Table 2).

Table 2. Goodness-of-fit statistics of the hypothesized first-order factor models and second-order factor models
First-order 77.79 58 .042 .03 .96 .98 .97 .03 278.20
Second-order 66.83 59 .226 .03 .96 .98 .97 .02 265.17

Standardized factor loadings ranging between .63 and .91 were significant at p < .001 (see Figure 1). The reliability values of the personal data protection (AVE = .50, ω = .72), safe relationships (AVE = .52, ω = .70), healthy internet use (AVE = .57, ω = .71), and self-disclosure (AVE = .54, ω = .76) dimensions were acceptable.

Figure 1. Confirmatory factor analysis results for the second-order factor model of digital security
Figure 1. Confirmatory factor analysis results for the second-order factor model of digital security
Note. Standardized coefficients are reported. All standardized coefficients are significant p < .001

3.3 Assessing invariance by gender and stage of adolescence

As shown in Table 3, the measurement model is equivalent in both genders. The unconstrained model served as a baseline showing that the data has an acceptable fit (SBX2 = 159.32, df = 118, p = .007; SRMR = .05; CFI = .96; TLI = .95; RMSEA = .03, 90% CI [.01, .03]) (configural invariance). We found no statistically significant differences when constraining the factor loadings to be equal between males and females (ΔSBX2 = 8.83, Δdf = 9, p = .008; ΔCFI < .001 and ΔRMSEA < .015) (metric invariance). In addition, we constrained the item intercepts to be equivalent between males and females (scalar invariance), and no statistically significant differences were found between the two groups (ΔSBX2 = 59.2, Δdf = 35, p = .006; ΔCFI < .001 and ΔRMSEA < .015).

Additionally, the analysis confirms model invariance in early and middle adolescents. The unconstrained baseline model fit the data well (SBX2 = 164.21, df = 118, p = .003; SRMR = .06; AGFI = .92; CFI = .96; TLI = .94; RMSEA = .03; CI 90 [.01, .04]). The results confirm metric (ΔSBX2 = 6.70, Δdf = 9, p = .006; ΔCFI < .001 and ΔRMSEA < .015) and scalar invariance (ΔSBX2 = 60.97, Δdf = 35, p = .004; ΔCFI < .001 and ΔRMSEA < .015).

Table 3. Results of the invariance analysis by gender and stage of adolescence for the second-order measurement model
Invariance SBΧ2 df ΔSBX2 Δdf p ΔCFI ΔRMSEA
Configural 159.32 118
Metric 168.15 127 8.83 9 .008 .001 .001
Scalar 227.35 162 59.2 35 .006 .003 .001
Stage of adolescence (early vs. middle)
Configural 164.21 118
Metric 170.91 127 6.70 9 .006 .003 .002
Scalar 231.88 162 60.97 35 .004 .003 .001

3.4 Assessing between-group latent mean differences

To assess latent mean differences between males and females, we chose males and middle adolescents as the reference groups by constraining their latent means to zero. The results showed no differences in personal data protection between the two genders. However, the latent means of females are greater than those of males in the safe relationships, healthy internet use, and self-disclosure dimensions. To examine latent mean differences by stage of adolescence (early vs. middle), we set middle adolescence to zero. Early adolescents had a higher score for safe relationships and self-disclosure. The differences in personal data protection and healthy internet use were not statistically significant (see Table 4).

Table 4. Differences in means by gender and stage of adolescence for digital security behaviors
Variable Factor Mdiff Z-statistics p Cohen’s d
Gender Personal data protection 0.07 0.64 .522 0.06
Safe relationships 0.17 2.21 .027 0.21
Healthy internet use -0.24 2.53 .011 0.24
Self-disclosure 0.33 3.52 < .001 0.34
Stage of adolescence Personal data protection -0.17 -1.47 .140 0.14
Safe relationships 0.17 2.15 .031 0.20
Healthy internet use 0.13 1.40 .160 0.13
Self-disclosure -0.37 3.79 < .001 0.36

3.5 Discriminant validity

The AVE is greater than the squares of the correlations between dimensions. According to the guidelines proposed in the literature, the results confirm the uniqueness of each dimension of the DSS-A (see Table 5).

3.6 Validity evidence based on relations with external variables

Spearman’s correlation coefficient was used to examine the relations between variables (see Table 5). The results show the expected significant positive correlations between dimensions of the DSS-A and types of parental mediation of children’s internet use.

Table 5. Correlations between digital security and types of parental mediation
Variable 1 2 3
AVE = .50
AVE = .52
AVE = .57
AVE = .54
1. Active parental mediation -
2. Restrictive parental mediation .43** -
3. Personal data protection .23** .17** -
4. Safe relationships .43** .24** .39**(.15) -
5. Healthy internet use .39** .28** .33**(.11) .38**(.14) -
6. Self-disclosure .54** .51** .25**(.06) .33**(.11) .41**(.17) -
Note: Squared correlations are reported in parentheses (R2).
**p < .001

3.7 Cross-validation analysis

The unconstrained model fit the data (SBX2 = 148.73, df = 118, p = .029; SRMR = .04; CFI = .97; TLI = .96; RMSEA = .02, 90% CI [.01, .03]). We examined the measurement and structural model invariance. The results indicate that the measurement model (relations between unobserved and observed variables) and structural model (relations between latent variables) are invariant in both samples (see Table 6). Overall, these results support the equivalence of the measurement model in both samples.

Table 6. Model invariance in calibration and cross-validation samples
Invariance SBΧ2 df ΔSBX2 Δdf p ΔCFI ΔRMSEA
Configural 148.73 118 .029
Metric 153.69 127 4.96 9 .054 .004 .003
Scalar 184.71 162 31.02 35 .066 .001 .001

IV. Discussion

Improving digital security in adolescents is essential for reducing online risks and developing digital citizenship. Unfortunately, researchers need a theoretically grounded and robust psychometric instrument to measure online security behaviors accurately. This study expands on the literature by developing a reliable, valid, and invariant self-report scale to measure early and middle adolescents’ digital security. To our knowledge, this study is the first attempt to offer a scale proving that a second-order online security model fits the data well. Overall, a second-order factor suggests that digital security is a construct that displays four unique dimensions: personal data protection, safe relationships, healthy internet use, and self-disclosure. By accounting for the second-order digital security factor and specific dimensions of adolescents’ online security behaviors, it becomes possible to gain insight into how global and specific online security behaviors are associated with protective and risky outcomes.

4.1 Online security as a multidimensional construct

Consistent with the literature on online security (Curran & Ribble, 2017; Chen et al., 2022; Hamza et al., 2019; Gallego-Arrufat et al., 2019), our findings provide evidence supporting online security as a multidimensional construct. We found online security to be a one second-order factor model that includes four first-order factors, confirming that digital security should be conceptualized and measured as a multidimensional construct. A multidimensional online security measure – as opposed to considering each dimension separately – offers a more profound representation of how adolescents protect personal data, exercise caution in their relationships, use the internet healthily, and share information with their parents or other adults. Approaching online security as a multidimensional construct also allows us to appreciate the influence of these dimensions, together or separately, on psychosocial outcomes.

4.2 Discriminant validity

The second-order factor captures the common variance shared by four dimensions of adolescents’ online security. On the other hand, each dimension of online security represents the uniqueness captured by each factor in the construct’s variance. These findings support the conceptualization that personal data protection, safe relationships, healthy internet use, and self-disclosure, although empirically related, also represent specific factors that capture specific variance in the construct. The researchers examined the relations of each type of security behavior with antecedent and consequent variables. Furthermore, the multidimensional scale can be used in educational settings or psychological interventions to identify the digital security dimensions where adolescents encounter difficulties. Additionally, a global measurement of online security may be relevant for testing educational policy issues related to adolescent online security behaviors.

4.3 Measurement invariance

Confirming measurement invariance by gender and stage of adolescence (early vs. middle adolescence) enables researchers to make meaningful inferences in future research. Our findings support measurement invariance in the scale, confirming that item content is interpreted similarly across the tested groups. An analysis by gender showed that mean scores for safe relationships and self-disclosure were higher for females, whereas males scored more highly in healthy internet use. These results suggest that digital safety behaviors vary by gender and across dimensions. This finding may shed light on the inconsistencies reported throughout the literature regarding safety behavior differences in internet use by gender (see Casaló & Escario, 2019; Jiang et al., 2017; Kapetanovic et al., 2017; Tifferet, 2019). We suggest focusing future research on analyzing the factors that might cause these gender differences in online security practices, specifically in a Mexican context.

Regarding differences by stage of adolescence, we found that early adolescents exhibited higher levels of self-disclosure and safe relationships than middle adolescents; no differences were found in personal data protection and healthy internet use. This finding aligns with previous research showing that middle adolescents tend to diminish their self-disclosure to parents and become more involved in unsafe online relationships (Smetana et al., 2009; Son & Padilla-Walker, 2019; Koutamanis et al., 2015). Therefore, we recommend conducting future research to analyze the variables that affect each dimension of digital security and examine how these dimensions can be undermined by adolescents’ need for autonomy.

4.4 Validity evidence based on relations with external variables

We found the expected relationship between parental mediation and online security behaviors in adolescents. Moreover, the effect size of these correlations suggests that the results have theoretical and practical implications. These findings are consistent with other research (e.g., Mesch, 2018; Wang & Xing, 2018) and thus provide evidence of the concurrent validity of our scale. Overall, results confirm that parental mediation is critical in explaining positive online behaviors in adolescents. We suggest that future research address how forms of parental mediation lead to specific online security practices in adolescents.

V. Conclusions

This research confirmed the value of measuring online security as a multidimensional construct. Findings have shown that a one-dimensional digital security measure reveals no differences between online security practices. Practitioners may examine digital security across four dimensions that vary between adolescents, thus targeting interventions based on these results. Following on from previous research demonstrating links between digital security and adolescents’ online risk behaviors (Gallego-Arrufat et al., 2019; Ribble & Bailey, 2011), school counseling should consider targeted intervention for different dimensions of digital security in order to promote responsible and safe internet use in adolescents. Adolescents with lower scores in particular dimensions or lower overall scores across all dimensions could be identified for counseling support. Finally, differences in means help to identify groups of students at risk for unsafe and problematic internet use.

These findings have methodological implications for researchers and practitioners seeking to enhance digital citizenship in adolescents, who base their efforts partly on the extent to which adolescents demonstrate digital security behaviors, such as personal data protection, safe relationships, healthy internet use, and self-disclosure. In education, interventions to improve digital security in adolescents include digital education for students, teachers, and parents; integrating digital citizenship in the school curriculum; involving parents in the digital citizenship education process; and creating a peer mentorship program.

The results presented in this paper should be treated cautiously since the study has some limitations. First, we used a second-order factor model to examine the scale’s dimensionality. Additional methods of analysis, such as the bifactor model, should ensure the interpretability of the total and subscale information (Reise et al., 2023). Second, we relied on self-report instruments to assess online security, and responses may be biased by social desirability. We recommend that future studies assess the construct based on multiple data sources, such as parents’ or teachers’ reports. Different measurement methods (e.g. observation) should also offer a more accurate construct evaluation. Third, the data pertain to Mexican adolescents, and youth from different countries may experience different digital security issues and may understand the scale differently. Using cross-cultural samples across diverse contexts to extrapolate the results to other populations would offer more precise insight. Lastly, the cross-sectional design used for this study limits any analysis of changes over the stages of adolescence. Longitudinal designs are warranted to analyze differences across ages more accurately.

Contribution of each author:

José Nestor Peraza: Conception and design, data analysis and article writing.

Ángel Alberti Valdés: Funding acquisition, data analysis and article writing.

Lizeth Guadalupe Parra: Methodology and writing review.

Maricela Urías: Data collection, methodology and writing review.

Declaration of no conflict of interest

The authors declare no conflict of interest.

Source of funding:

Programa de Fortalecimiento a la Investigación (Profapi) del Instituto Tecnológico de Sonora.


Almanasreh, E., Moles, R., & Chen, T. F. (2019). Evaluation of methods used for estimating content validity. Research in Social and Administrative Pharmacy, 15(2), 214–221.

Aydın O., Güçlü, M., Ünal-Aydın, P., & Spada, M. M. (2020). Metacognitions and emotion recognition in Internet Gaming Disorder among adolescents. Addictive Behavior Reports, 12, e100296.

Babaoglu C., & Akman, E. (2022). Participation with social media: The case of Turkish metropolitan municipalities in Facebook. In Information Resources Management Association (Ed.), Research anthology on citizen engagement and activism for social change (pp. 104–125). IGI Global.

Bandalos, D. L., & Finney, S. J. (2019). Factor analysis: Exploratory and confirmatory. In G. R. Hancock, L. M. Stapleton, & R. O. Mueller (Eds.), The reviewer’s guide to quantitative methods in the social sciences (pp. 98–122). Routledge.

Brown, T. A. (2023). Confirmatory factorial analysis. In R. H. Hoyle (Ed.), Handbook of structural equation modeling (2nd ed., pp. 261–276). The Guildford Press.

Casaló, L. V., & Escario, J.-J. (2019). Predictors of excessive internet use among adolescents in Spain: The relevance of the relationship between parents and their children. Computers in Human Behavior, 92, 344–351.

Chen, A., Rosli, S. A., Basri, R., & Hoe, C. Y. W. (2022). Investigation of screen time inclination and the accompanying visual and musculoskeletal discomfort in young smartphone users. Trends in Sciences, 19(2), e1753.

Chen, L., & Shi, J. (2019). Reducing harm from media: A meta-analysis of parental mediation. Journalism & Mass Communication Quarterly, 96(1), 173–193.

Chen, Y. T., Shih, W. L., Lee, C. H., Wu, P. L, & Tsai, C. Y. (2021). Relationships among undergraduates’ problematic information security behavior, compulsive internet use, and mindful awareness in Taiwan. Computers & Education, 164, e104131.

Chugh, R., & Guggisberg, M. (2022). Stalking and other forms of dating violence: Lessons learned from You in relation to cyber safety. Journal of Interpersonal Violence, 37(9–10), 6760–6784.

Curran, M. B. F. X., & Ribble M. (2017). P-20 model of digital citizenship. New Directions for Student Leadership, 2017(153), 35–46.

Dedkova, L., & Mýlek, V. (2022). Parental mediation of online interactions and its relation to adolescents’ contacts with new people online: The role of risk perception. Information, Communication & Society, 26(16), 3179-3196.

Emejulu, A., & McGregor, C. (2019). Towards a radical digital citizenship in digital education. Critical Studies in Education, 60(1), 131–147.

Fu, X., Liu, J., Liu, R.-D., Ding, Y., Hong, W., & Jiang, S. (2020). The impact of parental active mediation on adolescent mobile phone dependency: A moderated mediation model. Computers in Human Behavior, 107, e106280.

Funder, D. C., & Ozer, D. J. (2019). Evaluating effect size in psychological research: Sense and nonsense. Advances in Methods and Practices in Psychological Science, 2(2), 156–168.

Furr, R. M. (2018). Psychometrics. An introduction (3rd ed.). Sage.

Gallego-Arrufat, M. J., Torres-Hernández, N., & Pessoa, T. (2019). Competence of future teachers in the digital security area. Comunicar, 61(27), 53–62.

Gottfried, E. D., Shier, E. K., & Mulay, A. L. (2020). Child pornography and online sexual solicitation. Current Psychiatry Reports, 22, e10.

Green, S. B., & Yang, Y. (2015). Evaluation of dimensionality in the assessment of internal consistency reliability: Coefficient alpha and omega coefficients. Educational Measurement: Issues and Practice, 34(4), 14–20.

Hair, J. F., Black, W. C., Babin, B. J., & Anderson, R. E. (2010). Multivariate data analysis (7th ed.). Pearson.

Hair, J. F., Hult, T. M., Ringle, C. M., & Sarstedt, M. (2017). A primer on partial least squares structural equation modeling (PLS-SEM) (2th ed.). Sage.

Hamza, A., Kumar, M., Anand, N., Marimuthu, P., Thamilselvan, P., Chakraborty, P., Suma, N., Baglari, H., & Singh, P. (2019). Urban and rural pattern of Internet use among youth and its association with mood state. Journal of Family Medicine and Primary Care, 8(8), 2602–2606.

Ho, R. (2014). Handbook of univariate and multivariate data analysis and interpretation with SPSS (2nd ed.). CRC Press.

Hurwitz, L. B., & Schmitt, K. L. (2020). Can children benefit from early internet exposure? Short- and long-term links between internet use, digital skill, and academic performance. Computers & Education, 146, e103750.

INEE. (2019). Panorama educativo de México 2018. Indicadores del sistema educativo nacional. Educación básica y media superior [Overview of education in Mexico 2018. National education system indicators. Basic and upper secondary education].

INEGI. (2022). Encuesta Nacional sobre Disponibilidad y Uso de Tecnologías de la Información en los Hogares 2021. [2021 National Survey on Availability and Use of Information Technologies in Households].

Jiang, L. C., Yang, I. M., & Wang, C. (2017). Self-disclosure to parents in emerging adulthood: Examining the roles of perceived parental responsiveness and separation-individuation. Journal of Social and Personal Relationships, 34(4), 425–445.

Jones, L. M., & Mitchell, K. J. (2016). Defining and measuring youth digital citizenship. New Media and Society, 18(9), 2063–2079.

Kapetanovic, S., Bohlin, M., Skoog, T., & Gerdner A. (2017). Structural relations between sources of parental knowledge, feelings of being overly controlled and risk behaviors in early adolescence. Journal of Family Studies, 26(2), 226–242.

Koutamanis, M., Vossen, H. G. M., & Valkenburg, P. M. (2015). Adolescents’ comments in social media: Why do adolescents receive negative feedback and who is most at risk? Computers in Human Behavior, 53, 486–494.

Lauricella, A. R., Herdzina, J., & Robb, M. (2020). Early childhood educators’ teaching of digital citizenship competencies. Computers & Education, 158, e103989.

Lee, S. J. (2013). Parental restrictive mediation of children’s internet use: Effective for what and for whom? New Media & Society,15(4), 466–481.

Lionetti, F., Keijsers, L., Dellagiulia, A., & Pastore, M. (2016). Evidence of factorial validity of parental knowledge, control and solicitation, and adolescent disclosure scales: When the ordered nature of Likert scales matters. Frontiers in Psychology, 7, e941.

Liu, C., Ang, R. P., & Lwin, M. O. (2013). Cognitive, personality, and social factors associated with adolescents’ online personal information disclosure. Journal of Adolescence, 36(4), 629–638.

Livingstone, S., Haddon, L., Görzig, A., & Ólafsson, K. (2011). Risks and safety on the internet: The perspective of European children: Full findings and policy implications from the EU Kids Online survey of 9–16 year olds and their parents in 25 countries. EU Kids Online Network.

Lukavská, K., Hrabec, O., Lukavský, J., Demetrovics, Z., & Király, O. (2022). The associations of adolescent problematic internet use with parenting: A meta-analysis. Addictive Behaviors, 135, e107423.

Madrid, E. J., Valdés, A. A., Urías, M., Torres, G. M., & Parra-Pérez, L. G. (2020). Factores asociados al ciberacoso en adolescentes: Una perspectiva ecológico-social [Factors associated with cyberbullying in adolescents: An ecological social perspective]. Perfiles Educativos, 42(167), 68–83.

Martin, F., Hunt, B., Wang, C., & Brooks, E. (2020). Middle school student perception of technology use and digital citizenship practices. Computers in the Schools, 37(3), 196–215.

Mesch, G. S. (2018). Parent-child connections on social networking sites and cyberbullying. Youth and Society, 50(8), 1145–1162.

Mueller, R. O., & Hancock, G. R. (2019). Structural equation modeling. In G. R. Hancock, L. M. Stapleton, & R. O. Mueller (Eds.), The reviewer’s guide to quantitative methods in the social sciences (2nd ed., pp. 445–456). Routledge.

Muthén, L. K., & Muthén, B. O. (2017). Mplus user’s guide (8th ed.). Muthén & Muthén.

Nikken, P., & Jansz, J. (2014). Developing scales to measure parental mediation of young children’s internet use. Learning, Media and Technology, 39(2), 250–266.

Nowell, C., Pfeifer, J., Enticott, P., Silk, T., & Vijayakumar, N. (2022). Value of self-disclosure to parents and peers during adolescence. Journal of Research on Adolescence, 33(1), 289–301.

Organization for Economic Cooperation and Development (2022). Internet access (indicator).

Orosco, J. R., & Pomasunco, R. (2020). Adolescentes frente a los riesgos en el uso de las TIC [Adolescents and risks of ICT]. Revista Electrónica de Investigación Educativa, 22, e17, 1-13.

Padilla-Walker, L. M., Coyne, S. M., & Fraser, A. M. (2012). Getting a high-speed family connection: Associations between family media use and family connection. Family Relations, 61(3), 426–440.

Pastor, Y., García-Jiménez, A., & López-de-Ayala, M.-C. (2022). Estrategias de regulación de uso del smartphone y uso problemático de internet en la adolescencia [Regulatory strategies for smartphone use and problematic internet use in adolescence]. Anales de Psicología, 38(2), 269–277.

Pereira, S., Fillol, J., & Moura, P. (2019). El aprendizaje de los jóvenes con medios digitales fuera de la escuela: De lo informal a lo formal [Young people learning from digital media outside of school: The informal meets the formal]. Comunicar, 27(58), 41–50.

Reise, S., Mansolf, M., & Haviland, M. (2023). Bifactor measurement models. In R. H. Hoyle (Eds.), Handbook of structural equation modeling (2nd ed., pp. 329–348). Guilford Press.

Ribble, M. (2015). Digital citizenship in schools: Nine elements all students should know (3rd ed.). International Society for Technology in Education.

Ribble, M., & Bailey, G. (2011). Digital citizenship in schools. International Society for Technology in Education.

Secretaría de Educación Pública (2022). Plan de estudios 2022. Educación básica. [2022 curriculum. Basic education].

Shin, W., & Kang, H. (2016). Adolescents’ privacy concerns and information disclosure online: The role of parents and the Internet. Computers in Human Behavior, 54, 114–123.

Skryabin, M., Zhang, J., Liu, L., & Zhang, D. (2015). How the ICT development level and usage influence student achievement in reading, mathematics, and science. Computers & Education, 85, 49–58.

Smetana, J. G., Villalobos, M., Tasopoulus-Chan, M., Gettman, D. C., & Campione-Barr, N. (2009). Early and middle adolescents' disclosure to parents about activities in different domains. Journal of Adolescence, 32(3), 693–713.

Son, D., & Padilla-Walker, L. M. (2019). Whereabouts and secrets: A person-centered approach to emerging adults’ routine and self-disclosure to parents.Emerging Adulthood, 9(2), 145–157.

Steinfeld, N. (2021). Parental mediation of adolescent Internet use: Combining strategies to promote awareness, autonomy and self-regulation in preparing youth for life on the web. Education and Information Technologies, 26, 1897–1920.

Tifferet, S. (2019). Gender differences in privacy tendencies on social network sites: A meta-analysis. Computers in Human Behavior, 93, 1–12.

Vrieze, S. I. (2012). Model selection and psychological theory: A discussion of the differences between the Akaike information criterion (AIC) and the Bayesian information criterion (BIC). Psychological Methods, 17(2), 228–243.

Wang, X., & Xing, W. (2018). Exploring the influence of parental involvement and socioeconomic status on teen digital citizenship: A path modeling approach. Educational Technology & Society, 21(1), 186–199.

Widaman, K. F., & Olivera-Aguilar, M. (2023). Investigating measurement invariance using confirmatory factorial analysis. In R. Hoyle (Ed.), Structural equation modeling (2nd ed., pp. 367–384). Guilford Press.