There is a consensus on digital security as a multidimensional construct that involves multiple aspects such as personal data protection, safe relationships, healthy internet use, and self-disclosure (Chen et al., 2022; Curran & Ribble, 2017; Gallego-Arrufat et al., 2019; Hamza et al., 2019; Nowell et al., 2022). Personal data protection involves actions aimed at averting wide-ranging online problems related to the misuse of personal information (Chen et al., 2021). Healthy internet use refers to actions necessary to ensure individuals’ physical and psychological health (Chen et al., 2022; Gallego-Arrufat et al., 2019; Hamza et al., 2019). Safe relationships mean being careful about one’s online relations, which can lead to risky and painful outcomes. Finally, self-disclosure involves voluntarily sharing information about online activities and relationships (Lionetti et al., 2016; Shin & Kang, 2016).

Although digital security is a multidimensional construct (e.g. Lauricella et al., 2020; Martin et al., 2020; Nowell et al., 2022), and measure just a single dimension of digital security in isolation (e.g. Lionetti et al., 2016; Liu et al., 2013). However, digital security dimensions have been understood as interwoven, and instruments must explore the relationship between them. To address this gap, we aimed to develop a theoretically grounded scale that captures the four aspects of digital security (personal data protection, safe relationships, healthy internet use, and self-disclosure with parents or other adults).

According to several studies (Kapetanovic et al., 2017; Tifferet, 2019), online security varies by gender. Females display higher levels of digital security than males during adolescence. Similarly, middle adolescents report higher levels of digital security than do early adolescents (e.g. Livingstone et al., 2011; Martin et al., 2020). Regardless of their relevance, these previous studies did not examine measurement invariance by gender or stage of adolescence to ensure the validity of findings. Measurement invariance tests the degree to which measurements conducted with different groups exhibit similar psychometric properties. Therefore, it is critical to make a valid and meaningful group comparison to examine the measurement invariance of digital security scales. These scores can then be used to examine differences in variables associated with digital security behaviors for these groups (Widaman & Olivera-Aguilar, 2023).

Empirical evidence suggests parenting practices should explain adolescents’ digital security behaviors (Wang & Xing, 2018). Parental media mediation refers to parents’ practices that regulate children’s internet use (Nikken & Jansz, 2014). Active mediation occurs when parents speak with their children about media content and its benefits and risks. By contrast, restrictive mediation involves setting rules that limit the time children spend on the internet or the content they access (Nikken & Jansz, 2014). Empirical evidence has confirmed that active parental mediation enhances adolescent digital security practices (Dedkova & Mýlek, 2022; Fu et al., 2020). However, findings for restrictive mediation are inconsistent; some studies report that it helps to improve adolescents’ safety behaviors on the internet (Lee, 2013; Chen & Shi, 2019), whereas others point to continued higher risks (Lukavská et al., 2022; Steinfeld, 2021). Based on these results, we expected that adolescents’ digital security behaviors would be associated with both types of parental mediation.

Based on previous work that conceptualizes online security as a multidimensional construct (Chen et al., 2022; Curran & Ribble, 2017), this study develops a new measure, the Digital Security Scale in Adolescents (DSS-A), and examines the validity of the scale scores in a sample of Mexican adolescents. We tested the content validity and dimensionality of a second-order measurement model of digital security with four first-order factors: (a) personal data protection, (b) safe relationships, (c) healthy internet use, and (d) self-disclosure. We measured score reliability with McDonald's omega and average variance extracted. Moreover, the discriminant validity of each first-order factor was examined. Measurement invariance by gender and stage of adolescence (early vs. middle) was tested. If scalar invariance was confirmed, latent means across gender and stage of adolescence were compared. Evidence of validity based on relations with external variables was assessed by calculating the relationship between digital security and styles of parental mediation (active and restrictive). Finally, we tested model replicability using an independent sample (cross-validation).

To accomplish the research goals, we tested the following hypotheses. Hypothesis 1 (internal structure): A second-order model with four first-order factors fits the data. Hypothesis 2 (reliability): The McDonald's omega coefficient and average variance extracted indicate acceptable score validity. Hypothesis 3 (discriminant validity): Each subscale is genuinely distinct. Hypothesis 4 (measurement invariance): The measurement model is psychometrically invariant by gender and stage of adolescence. Hypothesis 5 (means comparisons): Females and middle adolescents display a greater degree of personal data protection behaviors, safe relationships, healthy behaviors, and self-disclosure than do males and early adolescents, respectively. Hypothesis 6 (relations with external variables): Correlations exist between the four dimensions of digital security and the two types of parental mediation examined. Hypothesis 7 (cross-validation): The measurement model replicates in the cross-validation sample.

The research participants were students from 52 urban middle schools from Sinaloa and Sonora, Mexico. As public schools in Mexico, these schools include students of low and medium socioeconomic status (National Institute for the Evaluation of Education [INEE], 2019). The sample includes 868 adolescents (460 female, 408 male), comprising 424 early adolescents (M age = 11.84 years, SD = 0.36) and 444 middle adolescents (M age = 13.44 years, SD = 0.49). All the students included in the sample have internet access. The sample was randomly split into two for validation (n = 434) and cross-validation (n = 434) purposes.

Digital security. The initial version of a pool of 16 items aimed to measure the dimensions of digital security (see Table 1) on a five-point Likert scale (0 = Never to 4 = Always). These items were created from focus group discussion analysis. The focus group included 20 adolescents (10 males and 10 females) from middle and high schools, aged between 12 and 18 years (M= 15.2 years,SD= 2.3). We presented a brief definition of digital security to research participants and asked them to discuss three issues related to this concept: (a) How do you define digital security? (b) What strategies do you use to keep safe online? (c) How have those strategies worked for you?

The focus group work provided critical information that allowed us to propose a set of items to measure digital security and its four dimensions. Subsequently, we used a panel of experts to assess the content validity of the proposed items; eight experts in educational research were asked to evaluate the relevance of all the items to measure four dimensions of digital security on a 4-point Likert scale (1 = not relevant to 4 = very relevant). Next, we calculated the item content validity index (CVI). All the items with a CVI > .79 were kept for further analysis (Almanasreh et al., 2019); the remaining seven items were discarded.

Afterward, we conducted nine cognitive interviews, including five adolescents and five middle school teachers, to judge if the 13 proposed items were appropriate for measuring digital security in adolescence. We emphasized the importance of verifying the clarity of the instructions, grammatical or semantic issues, and cultural or contextual issues in all the items. As a result of cognitive interviews, seven items were reworded based on participants’ suggestions. This work led to a theoretically grounded scale validated by educational researchers and revisited by the ultimate users: adolescent students. The final version of the scale employs three items, in each case, to measure personal data protection, safe relationships, and healthy internet use, and four items to assess self-disclosure.

The final form of the DSS-A comprises 13 items grouped into four dimensions (see Appendix 1): (a)Personal data protection (3 items, e.g. “I avoid posting personal information on social media.”); (b)Safe relationships (3 items, e.g. “I reject friendship requests from strangers on social media.”); (c)Healthy internet use (3 items, e.g. “I limit the time that I spend on the internet in order to avoid physical repercussions such as bad eyesight, backache, and neck pain.”); and (d)Self-disclosure (4 items, e.g. “I talk to my parents about my social media friendships.”).

Parental mediation. We used the Parental Mediation of Young Children’s Internet Use Scale (Nikken & Jansz, 2014) to measure how students perceive parents’ actions to mediate their Internet use. The back-translation method was used to translate the scale from English to Spanish. These items were used to measure active mediation and restrictive mediation. Active mediation explores how often parents talk to children about internet use and digital content consumption (8 items, McDonald's omega ω = .84; e.g. “How often do your parents speak with you about the rules you must follow while surfing the internet?”). Restrictive mediation examines how often parents limit and regulate children’s internet use (8 items, ω = .86; e.g. “How often do your parents tell you how long you are allowed to use the internet?”). Responses were given on a five-point Likert scale (0 = never to 4 = always). Results from confirmatory analysis (CFA) proved the model fit the data (SBX² = 30.67, df = 13, p = .004; SRMR = .06; CFI = .98; TLI = .97; RMSEA = .05, 90% CI [.025, .071].

The study gained approval from the Ethical Research Committee of the Technological Institute of Sonora (ITSON). The schools agreed to participate voluntarily in the study. A consent letter was sent to parents/guardians to request permission for children to participate in the study. Permission was denied by about 6% of parents. Finally, adolescents were informed about the study’s purpose and the nature of their potential participation. Data collection was carried out in classrooms during school hours.

Item distribution. The percentage of missing data in the sample was 3%. Missing data were treated using multiple imputation (MI) available in SPSS 27. Then descriptive analyses (means, standard deviation, skewness, and kurtosis) were run for each item of the DSS-A. We tested univariate normality based on the skewness and kurtosis values (Ho, 2014). The statistical Z value is calculated as Z Skew = Skew/√se Skew and Z Kurt = Kurt/√se Kurt. If the Z value exceeds +- 2.86, the normality assumption at the .01 critical probability level is rejected. Based on the rules of thumb proposed in the literature, we assumed that values of kurtosis smaller than 7 and skewness smaller than 3 indicated that deviation from univariate normality did not affect the model estimation (Bandalos & Finney, 2019). Furthermore, in the model assessment we used a robust procedure (Satorra-Bentler corrections) that is unaffected by departures from normality (Mueller & Hancock, 2019).

Internal structure validity. To assess the dimensionality of digital security, we examined the fit of a four first-order factor model and calculated all possible covariance between the factors. After confirming the fit of this measurement model, we tested the four factors as indicators of a second-order dimension (see Figure 1). We conducted a confirmatory factor analysis using the diagonally weighted least squares (DWLS) method with LISREL 12 software. Exploratory factor analysis was not carried out due to the soundness of the theory used to develop the scale.

Based on structural modeling literature, we evaluated the goodness-of-fit indices using the Satorra-Bentler chi-square adjustment (SBX² with p > .001). Due to the relatively large sample size, additionally, we used the standardized root mean square residual (SRMR ≤ .08), Tucker-Lewis index (TLI ≥ .90), comparative fit index (CFI ≥ .95), and root mean square error of approximation (RMSEA ≤ .08) (Brown, 2023). We used differences in SBX² (ΔSBX²) and the Bayesian information criterion (ΔBIC) for the model comparison. When ΔSBX² is significant (p< .001), a model with a smaller value of SBX² provides a better fit. Differences in BIC > 10 suggest a different model fit to the data; a model with lower BIC better fits the data (Muthén & Muthén, 2017; Vrieze, 2012).

Reliability. Reliability was measured by calculating the average variance extracted (AVE) and McDonald’s omega; AVE > .50 and ω > .70 were indicators of acceptable score reliability (Green & Yang, 2015; Hair et al., 2010).

Discriminant validity. Discriminant validity indicates the uniqueness of the measurement construct. Scholars suggest that discriminant validity is confirmed when the AVE of the dimensions is larger than the construct’s squared correlation (R ²) (Hair et al., 2017).

Measurement invariance. We used a multigroup approach to test several nested models to examine the level of measurement invariance by gender and educational level (Widaman & Olivera-Aguilar, 2023). We constrained the number of factors and the patterns of factor loadings to be the same across gender and stage of adolescence (configural invariance). Then, we constrained factor loadings to be equal across groups for the two variables (metric invariance). Finally, we constrained the measurement intercepts to be equivalent across groups to assess scalar invariance. Values of ΔSBX² with p > .05, ΔCFI < .01, and ΔRMSEA < .015 were considered indicators of invariance (Widaman & Olivera-Aguilar, 2023). When the values of these procedures diverged, we relied on ΔCFI and ΔRMSEA.

As scalar invariance was confirmed, we assessed latent mean differences by gender and stage of adolescence (early vs. middle). To test latent mean differences, we fit gender and stage of adolescence to zero. A z statistic was used to calculate the latent mean differences by gender and stage of adolescence.

Validity evidence based on relations with external variables. This study examined the correlation between dimensions of the DSS-A and active and restrictive parental mediation. Concurrent validity evidence is confirmed when scores correlate as expected with other constructs measured simultaneously (Furr, 2018). We adopted guidelines proposed by Funder and Ozer (2019) to assess the effect size:r= .10 small,r= .20 medium, andr= .30 large.

Model cross-validation. As suggested in the literature, we used a multigroup invariance procedure to test measurement model equivalence in the independent sample (see Widaman & Olivera-Aguilar, 2023). This strategy explores whether a model tested in a specific sample can be replicated over a second independent sample from the same population. Configural, metric, and scalar invariance were estimated using multigroup analysis for the nested models. Model sample invariance was confirmed when ΔSBX² was not statistically significant (p > .001), ΔCFI < .01, and ΔRMSEA < .015 (Widaman & Olivera-Aguilar, 2023). Since SBX² is sensitive to large sample sizes (Brown, 2023), we relied on CFI and RMSEA differences when the criteria were inconsistent.

Table 1 shows means, standard deviations, skewness, and kurtosis for the items of the DSS-A. The participants’ responses centered on the categories “seldom” and “sometimes,” suggesting that adolescents frequently forgo digital security practices. Items 1, 8, 9, 11, and 13 depart from normality. Furthermore, skew and kurtosis values are less than 2 and 7, respectively, suggesting that it is unlikely there is substantial distortion in the data (Bandalos & Finney, 2019).

Table 1

Item	M	SD	Skew	Zskew	Kurt	Zkurt	CVI
1. I avoid posting personal information on social networks.	2.20	1.67	-0.18 (0.12)	-0.52	-1.63 (0.23)	-3.39**	.87
2. I make a copy of important information in digital files or in the cloud.	1.82	1.49	0.20 (0.12)	0.59	-1.36 (0.23)	-2.83	.90
3. I only provide personal information on trusted websites.	2.56	1.37	-0.46 (0.12)	-1.35	-1.05 (0.23)	-2.18	.88
4. On social networks, I reject friend requests from people I do not know offline.	2.85	1.42	-0.91 (0.12)	-2.67	-0.62 (0.23)	-1.29	.86
5. I block people who try to harm me from my social networks.	2.74	1.42	-0.71 (0.12)	2.09	-0.89 (0.23)	-1.85	.92
6. I block people who make me feel stressed, anxious or uncomfortable from my social networks.	2.13	1.35	-0.08 (0.12)	-0.23	-1.07 (0.23)	-2.23	.94
7. I limit the time I spend online to avoid causing harm to my physical health.	2.58	1.30	-0.51 (0.12)	-1.50	-0.88 (0.23)	-1.83	.84
8. I can easily go several hours without connecting to the internet.	2.31	1.51	-0.24 (0.12)	-0.70	-1.42 (0.23)	2.95**	.93
9. I avoid internet sites that cause me concern, discomfort or stress.	2.14	1.53	-0.09 (0.12)	-0.26	-1.43 (0.23)	3.04**	.86
10. I ask my parents for help when I have problems with personal information online.	2.07	1.45	-0.05 (0.12)	-0.14	-1.32 (0.23)	-2.75	.85
11. I talk to my parents about my best online friends.	1.98	1.54	0.03 (0.12)	.009	1.48 (0.23)	-3.08**	.92
12. I ask my parents for help when I feel that using digital media is harming my physical health.	1.89	1.44	0.11 (0.12)	0.32	-1.31 (0.23)	-2.73	.89
13. I talk to my parents about the harmful psychological effects of internet use (e.g. stress, worry or unease).	2.20	1.67	-0.18 (0.12)	0.53	-1.63 (0.23)	3.39**	.94

The CFA results confirmed the goodness-of-fit of the first-order four-dimensional model (Model A) to data. The correlations between the four factors were statistically significant, from moderate (r = .33) to high (r = .51), suggesting that a second-order model is plausible. Then, we tested a second-order model (Model B) with four first-order digital security factors that fit the data well (see Figure 1). The difference in fit between Model A and Model B was statistically significant (ΔSBX² = 10.96, df = 1, p < .001). Additionally, the change in the BIC was greater than 10, indicating a difference between the fit data for the models. Thus, based on statistical and theoretical considerations, we decided to work with the second-order measurement model (see Table 2).

Table 2

Model	SBX2	df	p	SRMR	AGFI	CFI	TLI	RMSEA	BIC
First-order	77.79	58	.042	.03	.96	.98	.97	.03	278.20
Second-order	66.83	59	.226	.03	.96	.98	.97	.02	265.17

Standardized factor loadings ranging between .63 and .91 were significant at p< .001 (see Figure 1). The reliability values of the personal data protection (AVE = .50, ( ω = .72), safe relationships (AVE = .52, ( ω = .70), healthy internet use (AVE = .57, ( ω = .71), and self-disclosure (AVE = .54, ( ω = .76) dimensions were acceptable.

Figure 1 Note: Standardized coefficients are reported. All standardized coefficients are significant p < .001

As shown in Table 3, the measurement model is equivalent in both genders. The unconstrained model served as a baseline showing that the data has an acceptable fit (SBX² = 159.32, df = 118, p = .007; SRMR = .05; CFI = .96; TLI = .95; RMSEA = .03, 90% CI [.01, .03]) (configural invariance). We found no statistically significant differences when constraining the factor loadings to be equal between males and females (ΔSBX² = 8.83, Δdf = 9, p = .008; ΔCFI < .001 and ΔRMSEA < .015) (metric invariance). In addition, we constrained the item intercepts to be equivalent between males and females (scalar invariance), and no statistically significant differences were found between the two groups (ΔSBX² = 59.2, Δdf = 35, p = .006; ΔCFI < .001 and ΔRMSEA < .015).

Additionally, the analysis confirms model invariance in early and middle adolescents. The unconstrained baseline model fit the data well (SBX² = 164.21, df = 118, p = .003; SRMR = .06; AGFI = .92; CFI = .96; TLI = .94; RMSEA = .03; CI 90 [.01, .04]). The results confirm metric (ΔSBX² = 6.70, Δdf = 9, p = .006; ΔCFI < .001 and ΔRMSEA < .015) and scalar invariance (ΔSBX² = 60.97, Δdf = 35, p = .004; ΔCFI < .001 and ΔRMSEA < .015).

Table 3

Invariance	SBΧ 2	df	ΔSBX2	Δdf	p	ΔCFI	ΔRMSEA
Gender
Configural	159.32	118
Metric	168.15	127	8.83	9	.008	.001	.001
Scalar	227.35	162	59.2	35	.006	.003	.001
Stage of adolescence (early vs. middle)
Configural	164.21	118
Metric	170.91	127	6.70	9	.006	.003	.002
Scalar	231.88	162	60.97	35	.004	.003	.001

To assess latent mean differences between males and females, we chose males and middle adolescents as the reference groups by constraining their latent means to zero. The results showed no differences in personal data protection between the two genders. However, the latent means of females are greater than those of males in the safe relationships, healthy internet use, and self-disclosure dimensions. To examine latent mean differences by stage of adolescence (early vs. middle), we set middle adolescence to zero. Early adolescents had a higher score for safe relationships and self-disclosure. The differences in personal data protection and healthy internet use were not statistically significant (see Table 4).

Table 4

Variable	Factor	M diff	Z-statistics	p	Cohen’s d
Gender	Personal data protection	0.07	0.64	.522	0.06
	Safe relationships	0.17	2.21	.027	0.21
	Healthy internet use	-0.24	2.53	.011	0.24
	Self-disclosure	0.33	3.52	< .001	0.34
Stage of adolescence	Personal data protection	-0.17	-1.47	.140	0.14
	Safe relationships	0.17	2.15	.031	0.20
	Healthy internet use	0.13	1.40	.160	0.13
	Self-disclosure	-0.37	3.79	< .001	0.36

The AVE is greater than the squares of the correlations between dimensions. According to the guidelines proposed in the literature, the results confirm the uniqueness of each dimension of the DSS-A (see Table 5).

Spearman’s correlation coefficient was used to examine the relations between variables (see Table 5). The results show the expected significant positive correlations between dimensions of the DSS-A and types of parental mediation of children’s internet use.

Table 5

Variable	1	2	3 AVE = .50	4 AVE = .52	5 AVE = .57	6 AVE = .54
1. Active parental mediation	-
2. Restrictive parental mediation	.43**	-
3. Personal data protection	.23**	.17**	-
4. Safe relationships	.43**	.24**	.39**(.15)	-
5. Healthy internet use	.39**	.28**	.33**(.11)	.38**(.14)	-
6. Self-disclosure	.54**	.51**	.25**(.06)	.33**(.11)	.41**(.17)	-

Note: Squared correlations are reported in parentheses (R ² ).

The unconstrained model fit the data (SBX² = 148.73,df= 118,p= .029; SRMR = .04; CFI = .97; TLI = .96; RMSEA = .02, 90% CI [.01, .03]). We examined the measurement and structural model invariance. The results indicate that the measurement model (relations between unobserved and observed variables) and structural model (relations between latent variables) are invariant in both samples (see Table 6). Overall, these results support the equivalence of the measurement model in both samples.

Table 6

Invariance	SBΧ 2	df	ΔSBX2	Δdf	p	ΔCFI	ΔRMSEA
Configural	148.73	118			.029
Metric	153.69	127	4.96	9	.054	.004	.003
Scalar	184.71	162	31.02	35	.066	.001	.001

Consistent with the literature on online security (Curran & Ribble, 2017; Chen et al., 2022; Hamza et al., 2019; Gallego-Arrufat et al., 2019), our findings provide evidence supporting online security as a multidimensional construct. We found online security to be a one second-order factor model that includes four first-order factors, confirming that digital security should be conceptualized and measured as a multidimensional construct. A multidimensional online security measure - as opposed to considering each dimension separately - offers a more profound representation of how adolescents protect personal data, exercise caution in their relationships, use the internet healthily, and share information with their parents or other adults. Approaching online security as a multidimensional construct also allows us to appreciate the influence of these dimensions, together or separately, on psychosocial outcomes.

The second-order factor captures the common variance shared by four dimensions of adolescents’ online security. On the other hand, each dimension of online security represents the uniqueness captured by each factor in the construct’s variance. These findings support the conceptualization that personal data protection, safe relationships, healthy internet use, and self-disclosure, although empirically related, also represent specific factors that capture specific variance in the construct. The researchers examined the relations of each type of security behavior with antecedent and consequent variables. Furthermore, the multidimensional scale can be used in educational settings or psychological interventions to identify the digital security dimensions where adolescents encounter difficulties. Additionally, a global measurement of online security may be relevant for testing educational policy issues related to adolescent online security behaviors.

Confirming measurement invariance by gender and stage of adolescence (early vs. middle adolescence) enables researchers to make meaningful inferences in future research. Our findings support measurement invariance in the scale, confirming that item content is interpreted similarly across the tested groups. An analysis by gender showed that mean scores for safe relationships and self-disclosure were higher for females, whereas males scored more highly in healthy internet use. These results suggest that digital safety behaviors vary by gender and across dimensions. This finding may shed light on the inconsistencies reported throughout the literature regarding safety behavior differences in internet use by gender (see Casaló & Escario, 2019; Jiang et al., 2017; Kapetanovic et al., 2017; Tifferet, 2019). We suggest focusing future research on analyzing the factors that might cause these gender differences in online security practices, specifically in a Mexican context.

Regarding differences by stage of adolescence, we found that early adolescents exhibited higher levels of self-disclosure and safe relationships than middle adolescents; no differences were found in personal data protection and healthy internet use. This finding aligns with previous research showing that middle adolescents tend to diminish their self-disclosure to parents and become more involved in unsafe online relationships (Smetana et al., 2009; Son & Padilla-Walker, 2019; Koutamanis et al., 2015). Therefore, we recommend conducting future research to analyze the variables that affect each dimension of digital security and examine how these dimensions can be undermined by adolescents’ need for autonomy.

We found the expected relationship between parental mediation and online security behaviors in adolescents. Moreover, the effect size of these correlations suggests that the results have theoretical and practical implications. These findings are consistent with other research (e.g., Mesch, 2018; Wang & Xing, 2018) and thus provide evidence of the concurrent validity of our scale. Overall, results confirm that parental mediation is critical in explaining positive online behaviors in adolescents. We suggest that future research address how forms of parental mediation lead to specific online security practices in adolescents.