Measuring Critical Thinking Skills through Writing Performance Assessment: The Profile of EFL Students ’ Critical Thinking Skills

This study tries to find the profiles of EFL students' critical thinking skills in writing problem-solution expository essays within wetland themes. It employs a descriptive design with quantitative approach by utilizing a validated writing test. The subjects of the study were 40 students of the English Language Education Study Program of the University of Lambung Mangkurat, Indonesia, who were enrolled in the Academic Writing courses. The essay produced by each student was measured based on the depth of cause-and-effect analysis, the logic of the argument given, the viability of the solution, and the validity of evidence used to support the argument. The findings reveal that the subjects ’ critical thinking skill is categorized as medium since the mean score on the test of critical thinking is 11.38. Particularly, they have a high level of critical thinking in the aspects of analyzing the issue (3.03) and providing viable solutions (3.18). It indicates that they are able to distinguish the cause of a problem and the consequences as well and consider the causes and consequences when they propose a solution. However, their skill in providing argument is medium (2.85) whereas their skill in providing the evidence is low (2.33). These findings suggest that the subjects need to be trained in providing arguments and sufficient evidence through leading questions, discussion, and reading relevant sources. It implies that the study program needs to develop a teaching model that facilitates the students to practice their critical thinking skills.


INTRODUCTION
The ability to think critically is an ability that is seen as increasingly important for students to have in today's digital era where people can access much information easily (Lun et al., 2010).Moreover, living in the era of multicultural societies, human mobility, the advancement of digital and technology, social networking, innovations and inclusiveness has become more demanding for 21st century citizens, particularly for students (Saleh, 2019), to not to completely believe the available information on the Internet.The changing nature of technology not only provides us with better teaching techniques or methods, but also contributes the need of thinking critically (Turan et al., 2019).Hence, students need to select the information and critically analyze the causes and effects before they make a decision.In addition, students need to prepare themselves to face the increasingly dynamic world of work and society (Braun et al., 2020;Raslan, 2023).
As revealed in the research, having critical thinking skills will help students in their studies and in filtering any information before making decisions and sharing information with others (Changwong et al., 2010;Szabo & Schwartz, 2011).Therefore, explicit instructions on critical thinking in tertiary education is becoming essential (Wang & Seepho, 2017) and that teachers can implement vary of teaching strategies to facilitate students' development.
Previous research shows various teaching strategies implemented in EFL classrooms to facilitate the development of students' critical thinking skills like group discussion, concept mapping, and analytical questioning (Wang & Seepho, 2017).
Defining critical thinking skills has been made by scholars and each has similarity and distinctions as well.Critical thinking is the ability that a person has in thinking rationally and systematically.In the report compiled by the National Council on Measurement in Education (Lai & Viering, 2012), critical thinking skills are high-stage abilities consisting of problem-solving and decisionmaking skills.Skills included in this ability are the ability to select information, analyze a problem, analyze the cause and effect of a problem, propose alternative solutions to a problem, and propose the most appropriate solution.Critical thinking skill is a multifaceted construct defined as the abilities to evaluate evidence, analyze arguments, understand implications and consequences, develop sound arguments, and understand causation and explanation (Liu et al., 2014).This ability is achieved through a thinking process (Braun et al., 2020) that requires sufficient time and practice (Szabo & Schwartz, 2011).
Critical thinking is the process of a person's thinking in analyzing and evaluating information and producing arguments correctly so as to provide better information by outlining the results of observation, experience, and communication (Paul & Elder, 2002).This skill is manifested through three levels of critical thinking namely critical analysis which require one's ability in understanding an issue and deducing meaning, critical reflection which demands one's skills in reasoning and considering all the consequences, and critical alertness which requires one's ability in questioning his or other's thinking by evaluating the activities that have been done and actions needed in the future (Oser & Biedermann, 2019).Hence, it is represented through one's abilities in solving problems in daily life and professional settings in which careful judgment is needed.
There are several indicators that specify a person has the ability to think critically.As described in the National Council on Measurement in Education (Lai & Viering, 2012), critical thinking indicators are as follows: the ability to analyze arguments, the ability to draw conclusions deductively and inductively, the ability to provide judgment, the ability to solve problems, and the ability to make decisions.If it is associated with the field of English learning, especially the skill of writing problem-solution expository essays, critical thinking skills are shown by the ability of students to analyze problems, evaluate the cause and effect of a problem, and propose a solution that is considered most appropriate supported by good arguments and valid evidence.Explaining writer's reason by providing evidence to support it is a way of reasoning in writing arguments which requires one's skills in thinking critically (Lin, 2018).These indicators are important; therefore, teachers should consider when they assess their students' critical thinking skills through essay writing.
Another indicator that can be used in measuring students' critical thinking skills is the indicators proposed by (Facione, 2015).There are six indicators, namely the ability to: (1) interpret an experience and event, (2) analyze a problem in the form of exploring a problem and finding the causal relationship of each factor, (3) draw conclusions, (4) evaluate or assess something, (5) explain an event based on concepts, methods, and strong considerations, and (6) organize themselves in terms of understanding a problem.Essentially, critical thinking encompasses cognitive skills and dispositions that are necessary (Lun et al., 2010).This implies that being able to think critically is the skill that can be trained and developed.
However, in the context of English as a Foreign Language (EFL) classroom, developing critical thinking skills is challenging because of language proficiency issues.Studies reveal that students' language proficiency plays essential role in showing their critical thinking skills like verbal reasoning (Lun et al., 2010) and providing sufficient evidence from resources that support their critical-thinking performance (Manalo & Sheppard, 2016).Nevertheless, research on EFL learners with homogeneous language proficiency show that their critical thinking ability affects the quality of their writing with regard to the coherence, use of grammar, and organization (Golpour, 2014).This finding is in contrast with a study showing that there is no significant correlation between EFL learners' critical thinking skills and their argumentative writing performance (Pei et al., 2017).
The challenges in developing critical thinking skills in a foreign language suggest that EFL teachers work extra hard that is to facilitate their students in becoming proficient in the target language while at the same time developing their critical thinking skills.In fact, studies show that EFL teachers can facilitate their students by using analytical questions and concept mapping (Wang & Seepho, 2017) or using thought provoking questions, different groupings, displaying students' work, and modelling critical thinking (Al-Kindi & Al-Mekhlafi, 2017) that led to critical thinking before asking them to perform verbal reasoning or writing arguments.Similarly, to facilitate the development of critical thinking skills, teachers are suggested to establish a learning atmosphere that is potential for enquiry, question essential concepts, and evaluate reasons through interaction with the teacher and the classmates (Zhao et al., 2016) and through systematic analysis and open-mindedness (Snyder & Snyder, 2008).Furthermore, (Li & Liu, 2021) suggest that it is essential to engage EFL teachers in designing the course that facilitate the development of students' critical thinking skills since teachers know best about their students' thinking activities.
Critical thinking skills can be trained and measured through essay writing activities such as critical writing (Al-Kindi & Al-Mekhlafi, 2017) and problem-solution expository essays (Jumariati et al., 2021) using real-life issues that are unsolved yet.This is due to the characteristics of expository essays of typical problems that allow students to practice and demonstrate the ability to analyze problems, find the cause and effect of a problem, determine alternative solutions to problems, and provide one of the most appropriate solutions.In the same line, (Lin, 2018) postulates that writing decision-making, problem-solving and making arguments involves the writer's critical thinking.She further asserts that writing requires students to think over and develop ideas into a piece of writing.All of these characteristics are relevant to the indicators of critical thinking ability.In the same vein, (Braun et al., 2020) assert that measuring critical thinking skills through performance task will yield in more authentic representations of students' skill.Additionally, utilizing real-life issues to work with in performance test will adequately facilitate learners to think critically (Oser & Biedermann, 2019).
Based on the above explanation, it is crucial to conduct research to measure EFL students' critical thinking level as the starting point for EFL teachers to facilitate the development of their students' skills in thinking critically.Being informed about students' critical thinking skills is essential for both teachers and students to know the learning progress and learning improvement (Braun et al., 2020).Unfortunately, this has not received sufficient attention particularly in the teaching of EFL in Indonesia let alone in Kalimantan Island.In fact, a study to investigate the profile of EFL students' critical thinking skills is essential to develop suitable learning materials and improve the teaching-learning strategies that facilitates the students to be successful learners in higher education where critical thinking skills is fundamental.Hence, this study tries to find the profiles about EFL students' critical thinking skills in writing problem-solution expository essays at the English Department of Universitas Lambung Mangkurat, South Kalimantan.This is essential because the profiles of students' critical thinking skills will serve an important information for the university in depicting the students' abilities in thinking critically and in improving the teaching and learning process that facilitates the development of students' critical thinking skills.

Research Design
This study employs a descriptive design with a quantitative approach to find the profile of students' critical thinking skills in writing expository essays on problems -solutions.The profile is obtained through measurements, namely writing test and analytical scoring rubric that have been developed and tested valid and reliable.

Research Participants
The subjects of this study were 40 students of the English Language Education Study Program of the University of Lambung Mangkurat in Banjarmasin, Indonesia.The subjects were intact classes enrolled in the Academic Writing Course where the critical thinking skills are trained through essay writings as stated in the syllabus of the course.The course consisted of four classes which were then randomly selected due to time constraint.

Research Instruments
This study applies two main instruments, namely writing test and its analytical scoring rubric.As part of validity and reliability, the researchers used content validation techniques by consulting experts regarding the test content and assessment rubrics.The content validity is focused on elements of critical thinking skills that include the depth of cause-and-effect analysis, accuracy of problem solutions offered, arguments given, and sources of evidence/facts cited.Furthermore, this study used inter-rater reliability by involving 2 raters in evaluating the students' writings.The writing test was carried out in a meeting after they learned about how to write problem-solution and practice writing the essay in the middle of the semester.Meanwhile, the Academic Writing Course is held in 14 meetings during one semester in the forms of lecture, discussion, practice writing, and class conference.The procedure of this study is shown in Figure 1.

Figure 1 | Research Process
Meanwhile, Table 1 displays the prompt of the writing test used in the study.

TABLE 1 | Writing Test
The essay produced by each student are measured based on the following indicators: (1) depth of cause-andeffect analysis, (2) logic of the argument given, (3) accuracy of the solution offered, and (4) validity of evidence used as support for the argument.Table 2 shows the scoring rubric used in the study Write a five-paragraph essay by choosing one of the topics below.Remember to cite properly in the body of your essay and write the list of the references that you use to support your argument.You have 3 hours to write the essay.(a) South Kalimantan is surrounded by many small and large rivers, peat, and other wetland forms.Unfortunately, the society has a low awareness on the cleanliness of the environment.They often throw garbage in to the rivers which shallows the rivers and causes flood.Write a five-paragraph essay to discuss about the detailed problems occur due to littering the rivers and propose solutions to solve it.(b) The fire of forest and peat lands in Indonesia, including Banjarmasin, is a serious problem which affect many people not only in Indonesia but also neighboring countries.The fire is mainly due to the people habit of land clearing for palm oil industry and building houses, hotels, or factories.What are actually the problems caused by this fire and what solutions can you propose?Write a five-paragraph essay to discuss the detailed problems that occur due to wetland fires and offer solutions to solve it.(c) South Kalimantan is a developing province.Buildings and industries are being built everywhere including the palm oil plantation.These clearing and functions diversion activities use large area of wetland which eventually endanger the ecosystem of wetland and humans' life.Write a five-paragraph essay to discuss the detailed problems occur due to wetland functions diversion and offer solutions to help restoring wetland functions.

RESULTS AND DISCUSSION
The data collected through writing test was then analyzed in order to find out the level of students' critical thinking skills pertaining to the depth of the analysis, appropriacy of the solution, depth of the argument, and evidence used to support the argument.The scores in each component were summed up into a final score.Table 5 displays the scores obtained by each subject from Rater 1 and Rater 2. Based on the data in Table 5, the mean score is 11.38 which falls into the category of medium referring to the category in Table 1.This score represents that the subjects' level of thinking critically is in average position.
Subsequent to calculating the mean scores, the scores from each component of critical thinking were also analyzed in order to reveal on what component the subjects perform the highest score and the lowest one.The findings of each component are displayed in the Table 6 below.Table 6 indicates that the mean score for analysis is 3.03 which is classified as high while the mean score for solution is 3.18 that is classified as high, too.Then, the mean score for argument is 2.85 which is categorized as medium whereas the mean score for evidence is 2.33 which is categorized as low.
The critical thinking skills of the subjects of the study were measured through a writing test with four indicators derived from (Facione, 2015) namely the depth of analysis of the problem, the viability of the solution proposed, the depth of the argument, and the credibility of the evidence proposed.The performance test on critical thinking skills that has been carried out shows that the subjects of the study perform medium level of critical thinking skills as shown by the mean score of the test that is 11.8.The use of performance test in this study is in line with the theory that using performance task to measure students' critical thinking skills is appropriate as it can produce a reliable depiction of students' skills (Braun et al., 2020;Oser & Biedermann, 2019).This is to say that the subjects' scores in essay writing in this research represent their skills in thinking critically pertaining to their skills in analyzing the cause and effect of a problem, proposing a viable solution, providing logic arguments, and supporting their arguments with sufficient evidence.
Specifically, the subjects showed high level of critical thinking in analyzing the issue indicated by the score 3.03 and providing viable solution indicated by the score 3.18.They were able to distinguish the cause and the effect of the real-life issues in wetland societies.For instance, the subjects considered the challenge in making the society aware of the cleanliness of the rivers in Banjarmasin and thus they proposed the government not only to provide various trash bins but also to remind the society through banners and television commercial break.Some of the subjects wrote that giving punishment for people who litter in the rivers might not be effective.These aspects were taken into their consideration and thus they provided viable solutions such as maximizing the socialization and conducting contests with rewards for clean environment and rivers.This indicates that the subjects consider the cause of the dirty rivers and they consider the consequence also.As stated by (Lai & Viering, 2012) and (Facione, 2015), one indicator of being able to think critically is the ability to analyze a problem and the causal relationship of each factor.The possible reason for this is the use of reallife issues as the topics that they write enable them to explain what causes the problem and the effects that the problem brings to society.As revealed in studies, familiarity on the topic of writing may enhance students' writing performance (Lin, 2018).
However, in the component of providing argument, the subjects showed medium level of ability while in the component of providing evidence the subjects had low level of skill.Their writings showed that the arguments were not supported with sufficient evidence.For example, in an essay that the subject argued on the use of punishment as an effective way to reduce people littering the rivers, the writer did not provide information about the real situations nor expert's judgement about the punishment.The writer did not support his argument by providing the document of the punishment and the real condition in the society.Another example is from an essay that supports the idea of increasing the socialization through setting up banners in public places.The writer of the essay did not provide sufficient evidence showing that socialization can increase society awareness; the writer uses his personal judgment in convincing the readers.This may be due to the lack of skills in using evidence as the support of writer's arguments or the time limit the students had in writing the essay.
As asserted by (Lin, 2018), explaining writer's reason by providing evidence to support it is a way of reasoning in writing arguments which requires one's skills in thinking critically.
Henceforth, the subjects of the study were students of English as a foreign language; therefore, their lack of English proficiency may be the possible reason for their lack of skills in providing arguments and evidence in their essays.As suggested by previous research, students' language proficiency plays essential role in showing their critical thinking skills like verbal reasoning (Lun et al., 2010) and providing sufficient evidence from resources that support their critical-thinking performance (Manalo & Sheppard, 2016).Critical thinking demands working memory which requires adequate language proficiency that enables good use of cognitive skills as critical thinking skills (Manalo & Sheppard, 2016).Therefore, teachers need to facilitate their students in developing their critical thinking skills for instance through training on systematic analytic and open-minded (Snyder & Snyder, 2008) and through interactions that make them enquiry, question essential concepts, and evaluate reasons (Zhao et al., 2016).These, at the same time, are given sufficient attention while they facilitate their students in developing their English language skills.

CONCLUSION
The findings reveal that the subjects' critical thinking skill is categorized as medium.Particularly, they have a high level of critical thinking in the aspects of analyzing the issue and providing viable solutions.It indicates that they are able to distinguish the cause of a problem and the consequences as well and considering the causes and consequences when they propose a solution.However, their skill in providing argument is medium whereas the skill in providing the evidence is low.These findings suggest that the subjects need to be trained in developing their skills in providing arguments and sufficient evidence through questioning, discussion, and reading relevant sources.It also implies that the study program needs to develop a teaching model that facilitates the students to practice their critical thinking skills through enquiry learning, project-based learning, and problem-based learning.Nevertheless, this study involves a limited number of subjects, that is 40 students from two classes of Academic Writing Course.Therefore, further research involving larger samples is recommended in order to gain more comprehensive findings on EFL students' critical thinking skills level.

Table 2
shows that the scores for each indicator range from 1 to 4; the highest is scored 4 while the lowest is 1.Then, the scores from each indicator are summed up into one final score in which the highest score is 16.The category of the students' scores in writing test is depicted in Table3below.

TABLE 3 |
Category of Students' Scores in Writing Test

TABLE 5 |
Students' Scores on Critical Thinking Test