Analysis Validity and Reliability of Self -Efficacy and Metacognitive Awareness Instrument Toward Mathematical Reasoning

This study was conducted to analyze the degree of validity and reliability of an instrument to measure self-efficacy and metacognitive awareness of university students toward mathematics reasoning. A total of 184 respondent of a public university in Malaysia has been chosen to answer the instrument. Findings from Exploratory Factor Analysis (EFA) support three dimensions of self-efficacy and six dimensions of metacognitive awareness of students toward mathematics reasoning that has been conceptualized. The overall internal consistency reliability of Cronbach Alpha coefficients was above 0.9. Based on the analysis performed, it can be concluded that developed instrument has sufficient evidence of validity and reliability to measure self-efficacy and metacognitive awareness of university students toward mathematics reasoning.


INTRODUCTION
In the year 2012, result of PISA and TIMSS became major input in the drafting of Malaysia education development plan 2013-2025, which dropped the cognitive process aspect and student reasoning ability in learning.According to the most recent studies, majority of students who performed well in school still have trouble applying process reasoning correctly to solve a mathematical problem.(Napitupulu, 2017; Saleh, Charitas, Prahmana, & Isa, 2018; Zayyadi & Kurniati, 2018).This situation requires efficient and organized self -management as well as schools (Norazmi et  Mathematical reasoning skills are essential for an individual to compare similarities and logically explain mathematical structures.According to Putu, Putra, & Kristanto (2017), students' ability to provide reasons for each interpretation is critical to the abstraction process.The implementation of mathematical ideas can be designed using strong statements.Furthermore, one of the key goals of learning practices is to develop the ability to have rational explanations for inferring mathematical values.Putu et al. (2017) characterized mathematical reasoning as a mental operation involving mathematical reasoning skills.
Previous studies Liu et al., 2020; Morán-Soto & Benson, (2018) stated that high self-efficacy in mathematics is derived from the students' previous experience or awareness.Students with high self-efficacy can affect both their own skill and self-discipline in solving difficult mathematical problems.Self-efficacy is defined as self-assurance and belief in one's ability to cope or action in order to achieve a goal.Self-efficacy is an essential aspect in a student's internal development toward success (Bandura, 1993).
An individual must have appropriate control strategies, particularly in the area of metacognition, in order to solve mathematical problems successfully (Zakaria & Habib, 2006).According to Zakaria & Habib (2006), one of the most crucial components in solving mathematical problems is students' inability to regulate their thinking processes.The importance of mathematical thought and comprehension derives from the topic itself, as well as the understanding of concepts and ideas required to solve problems in everyday life.
A student should be aware of the strategy should use, why that strategy would be implementing it, and the mistakes he is making.Next, student should knowing how the mind operates will enable them to operate and monitor the strategies that are to be implemented optimally (Ozturk, 2017).
Although most previous studies have focused on overseas education involvement (Aminah, Kusumah, Suryadi, & Sumarmo, 2018; Dori, Mevarech, & Baker, 2018; Hammann, Stevens, & Hammann, Stevens, 1998; Yelgec & Dagyar, 2020), this study adapts models from metacognitive awareness and self-efficacy as measurement analysis.In the sense of Malaysian mathematics education, the combination of these two variables still infant or new phenomenon in the context of mathematics education.In the other hand, mathematical questions from previous studies are used in this research (Calvin & Duane, 2002;Mumu & Tanujaya, 2019;Yankelewitz, 2010).

SAMPLE STUDY
The participants of this study consisted of 184 students at a public university located around the Klang Valley in peninsular Malaysia.Researchers have used random sampling technique because it is the best sampling method (Larry, Johnson, & Lisa, 2017).The demographic characteristics studied to represent the profiles of study participants were (a) gender; (b) stream; (c) race; and (d) institutions.In terms of gender, 75 people (40.8%) of the study participants were female students, while 109 people (59.2%) were male students.In terms of stream, 159 people (86.4%) of the study participants were science stream students while 25 people (13.6%) of the study participants were non -science stream students.In terms of race, 52 patients (28.3%) of survey participants are students are Malays, 120 (65.2%) of survey participants are students of Chinese, 3 patients (2.5%) of survey participants are students of Indian, 5 patients (2.7 %) of the study participants were Iban students, 2 people (1.1%) of the study participants were Melanau students and the study participants were Bidayuh students was 2 (1.1%).

INSTRUMENT
Research instrument for mathematical reasoning have been adapted from three previous studies that have been conducted by Calvin & Duane (2002), Yankelewitz (2010) and Mumu & Tanujaya (2019).Mathematical reasoning questions consist of 8 questions covering the topics of critical thinking, sets and whole numbers, fractions and geometry.Mathematics achievement scores will be used to measure students' level of mathematical reasoning skills.In addition, the Metacognitive Awareness Inventory was adapted from Rahman, Yasin, Salamuddin, & Surat (2014) and Schraw & Dennison (1994) as a measure to measure the level of metacognitive awareness of students.The instrument consisted of 30 items involving two components in metacognitive awareness, namely metacognitive knowledge and cognitive regulation.Furthermore, the Self -Efficacy Instrument of this study was adapted from the Self -Efficacy Inventory that was constructed by May (2009), this instrument consists of 13 items.The three components include self-efficacy in terms of course, self-efficacy in terms of assessment and self -efficacy in terms of future.

PROCEDURE
The researcher had to get permission from the faculty involved before administering the instrument to the respondents in the sample.The application letter is then sent to the Dean of the related Faculty, along with the study's intent and research instrument.After receiving permission, the researcher contacted the course lecturer to request permission and schedule an appointment to administer the instrument to the students in their class.Before the respondents were given the instrument, the researcher explained how to fill it out and what the objective of the study was.This instrument was given to respondents for 30 minutes to complete.

DATA ANALYSIS
The study used exploratory factor analysis with Principal Component Analysis and Varimak Rotation to evaluate construct of the instrument that was built and designed.Construct validity is defined as an assessment of the appropriateness of an inference made on an individual based on test scores obtained in a construct (Cohen, Manion, & Morrison, 2017).The usability of a research instrument depends on the aspects of validity that can bring significance to the study.If there are data dropouts, outliers and normality analysis for the study data, then exploratory factor analysis should be performed (Cohen et al., 2017).
The researcher then uses three methods to calculate the number of factors derived as a result of the exploratory factor analysis: i) Kaiser-Guttman criteria (eigen value> 1), ii) screen plot, and iii) parallel analysis.The method's intention is to calculate the number of factors identified in a more authentic way than a single method.In addition, Indicator Kaiser-Meyer-Olkin (KMO) should be carefully evaluated and paid close attention to during the analysis of exploratory factors in deciding the suitability of the data in the analysis.KMO values approaching to value 1 should be seen in exploratory factor analysis that yields accurate and distinct factors from each other.Finally, confirmation of the existence of a factorability relationship between the variables studied can be examined through the results of the test Bartlet sphericity (Hair, Black, Babin, & Anderson, 2019).
The researcher compared the results of the exploration factor analysis by using different loading factor sizes, starting with sizes 0.3, 0.4, 0.5 and 0.6.The action is to determine the appropriate size in producing the best exploratory factor analysis results in terms of empirical and theoretical parallel to the study.The researcher's assessment of whether to retain or discard an item as a result of factor analysis results is made based on several conditions as suggested by Hair et al. (2019) mention that i) items that are heavy on two or more factors (cross-loading), ii) items with a loading factor below the size of a significant loading factor, iii) items with a significant loading factor but having too low communality value, iv) meet the theory underlying the study (Hair et al., 2019).
After the exploratory factor analysis, the next step is to conduct an instrument reliability analysis.Reliability is an assessment of the degree of consistency between several measurements of an attribute (Hair et al., 2019).The researcher conducted instrument reliability analysis in Cronbach Alpha to determine the degree of instrument reliability in the study.The method can help researchers assess whether the measuring items are the same or not as well as methods that are often used by other researchers.According to Hair et al., (2019) for identify the degree of inconsistency in the instrument that has been constructed should meet two conditions which are i) the correlation between items with items exceeding the value of 0.3, ii) the Cronbach Alpha value exceeding 0.7.

FINDING AND DISCUSSION
The skewness and kurtosis values for the items in Table 1 were in the range of -1.00 and +1.00, indicating that the data met the normality assumption (Hair et al., 2019).The researcher then used the Principal Component Analysis method and Varimak Rotation to execute an exploratory factor analysis to determine the validity of the instrument that have constructed.After that, a reliability analysis using the Cronbach Alpha reliability method was used to establish the study instrument's degree of reliability.

VALIDITY ANALYSIS
Following an analysis of the screen plots, the researchers discovered a continuous sloping graph beginning at the tenth factor.As shown in Figure 1, there are nine solution variables that are taken into account.

Figure 1: Screen Plot
Next, the researchers used a parallel analysis method to compare the eigenvalues resulting from the actual data with the size of the eigenvalues extracted from a randomly generated data with the same number of samples and items.A factor will be retained if the size of the eigenvalue of the factor is greater than the size of the eigenvalue that results through random generation (Ledesma & Valero-Mora, 2007).Table 2 is a parallel analysis confirming that 9 factors in the study data.To evaluate the instrument that has been construct in the study, exploratory factor analysis using the Principal Analysis approach with Virimak Rotation was used.Construct validity refers to an assessment of the appropriateness of an inference made on an individual based on scores obtained in a study (Coaley, 2010).The most important aspects of validity instrument, which is the key focus when evaluating the usability of a research instrument.Exploratory factor analysis was performed after outlier analysis, normality analysis and missing data analysis were conducted (Coaley, 2010).
The Kaiser-Meyer-Olkin (KMO) indicator was used to assess the suitability of the data in this analysis, with KMO values near 1.0 reflecting factors that are both accurate and distinct (Tabachnick & Fidell, 2007).An exploratory factor analysis was performed for item reinforcement in the selfefficacy construct.All products have a KMO value of 0.884, which means that they are very good and acceptable.Furthermore, the Barlett test was significant [χ 2 =5125.821],p<.05, rejecting the hypothesis that the correlation matrix was in the identity matrix.According to preliminary findings, communality range from 0.530 to 0.802, and there were 10 indicators with eigenvalues.Table 3 describes in detail each of the variants found in the study.

RELIABILITY ANALYSIS
A Goodness-of-Fit model was built using the statistical value of good-of-fit, χ 2 and the Mean Square Root Error of Approximation in the validation factor analysis of the study results (RMSEA).In the RMSEA, values less than 0.08 indicate model acceptance, while values greater than 0.10 indicate model rejection (Browne & Cudeck, 1992).The Comparative Fit Index (CFI) and Tucker-Lewis Goodness of Fit Indexes are correlated with the analysis (TLI).The value greater than 0.90 is considered to be a reasonable value for both indices.Table 4 lists the indexes that have been examined: The value of the fit model in this study's instrument did not fulfill the defined standard.Any items with a loading factor of less than 0.60 were eliminated by the researcher.The following are the findings of CFA research after modification.The selective items used in the analysis has a loading factor of less than 0.40 is remain, because it has a benefit in this research instrument (Awang-Hashim & Murad Sani, 2008).The Chi Square/df = 1.879,CFI = 0.821, GFI = 0.760, and RMSEA = 0.69 chi-square correspondence index for the model is at the level of significance is strong (Markus, 2012).This demonstrates that the final model is good.
The following is a summary of the validity factor analysis results: .697 Table 6 shows that the interval validity for metacognitive awareness was 0.712 to 0.789 when the metacognitive awareness constructs achieved the criteria conditions, with Cronbach alpha values range from 0.712 to 0.789.All three constructs fulfilled the specified criteria for self-efficacy, which had Cronbach alpha factor loadings from 0.712 to 0.860.The criteria to fullfill the Cronbach Alpha condition suggested by oleh Zainudin Awang (2018)should be a value ≥ 0.70.
Furthermore, where the constructs of metacognitive awareness have achieved the required criteria, the value of Construct validity (CR) for metacognitive awareness is between 0.639 and 0.817.Meanwhile, the Construct Validity (CR) for self-efficacy ranges from 0.647 to 0.785, indicating that the constructs of self-efficacy have fulfilled the requirements.The criteria to fullfill the Construct Validity (CR) requirement must be a value of ≥ 0.60 (Zainudin Awang, 2018).
Finally, the average variance extracted (AVE) value for metacognitive awareness range from 0.594 to 0.972, which met the requirements.Meanwhile, the Average Variance Extracted (AVE) value for self-efficacy ranged from 0.567 to 0.699, indicating that it satisfies the standards.According to Zainudin Awang (2018), the value of Average Variance Extracted (AVE) should be less than 0.50.Overall, the validation factor analysis satisfies the specific requirements.

CONCLUSION
This study was conducted to identify the validity and reliability of the instrument for assessing students self -efficacy and metacognitive awareness of mathematical reasoning.Through the analysis of exploratory factors that have been conducted, the variables of self-efficacy are divided into three dimensions, namely self-efficacy in terms of course, self-efficacy in terms of assessment and selfefficacy in terms of future, while metacognitive awareness variables are divided into six dimensions namely declarative knowledge, procedural knowledge, conditional knowledge, planning, monitoring and assessment.Although there is an exclusion of 4 items in self-efficacy and 7 items in metacognitive awareness, but all factors still retain the characteristics of factors that have been conceptualized by researchers based on research theory and views of experts in the field of education in the country.Cronbach Alpha internal consistency reliability analysis showed that the constructed instrument had a good degree of reliability.The findings of the study have shown that the instrument that has been built has good psychometric characteristics which in turn can be used for researchers to make an assessment of self-efficacy and metacognitive awareness of university students toward mathematical reasoning.

Table 1 :
Analysis of mean, standard deviation, skewness and kurtosis