PSYCHOMETRIC ANALYSIS OF WAEC AND NECO PRACTICAL PHYSICS TESTS USING PARTIAL CREDIT MODEL
CHAPTER ONE
INTRODUCTION
1.2 BACKGROUND OF THE STUDY
Societal development and break through in any nation are predicated on education sector of such a nation. New researches prove the long held expectation that human capital formation (the population education) plays a significant role in country’s economic development. Quality education leads not only to higher individual output but is also a necessary precondition for long term economic growth. Rigorous analysis of data provides policy makers with proof that education is a necessary precondition for long term economic development.
It is for the above reason that Nigeria in her National Policy in Education adopted education as an instrument “par excellence” for eff ecting national development and harnessing the potentials of the citizens (Federal Republic of Nigeria, FRN 2008). Akindogu and Bamjoko (2010) pointed out that the country’s vision is for a complete transformation of all aspects of the nations life over time and that education should be able to effect inter and intra generational transmission of our cherished heritages and life inventions and should reposition Nigeria global status in science and technology in all spheres of life.
While commenting on the role of education in national development, Blogspot (2009) noted that education is a milestone for all types of development and provides all knowledge to do any work in a systematic way. According to this author, with education any country can develop her economy and society, develop the personality of youths of the nation; make the citizens more productive by providing large number of skills to make them self reliant.
The major challenge for education in the twenty first century for our country Nigeria according to Maduagwu (2008) is designing an educational system that will be stable and global in outlook, and maintaining high standard of education. A cardinal challenge for Nigeria, if she is to use education to achieve the objective of overall development, is maintaining high standard of education through high quality assessment. And to achieve high quality assessment, in order to realize the goal of overall development, all the dimensions of educational objectives must be adequately measured and assessed.
Educational objectives according to Onwuka (1981) are expressed in terms of knowledge (cognitive domain), attitude (affective domain) and practical/motor skills (psychomotor domain). Hence education is said to be balanced when it satisfies the demands of the three major domains of educational objectives. Behaviour under the three domains of objective should form the bases for teaching and learning process and subsequently assessment. Bandele (2002) noted that the three domains should be taught and assessed critically to mould individual in totality and make the recipient of education to live a fulfilled life and contribute meaningfully to the society in which he lives.
The cognitive objectives refer to the intellectual results of schooling, the improvement in the child’s intellectual structure, his increase in knowledge and his ability to reason rather than just to remember. The affective objectives refer to the emotional education, and the learners’ acquisition of certain desirable attitudes, interest and appreciation; while psychomotor objectives refer to physical and practical manipulative skills learnt at school (Nwana, 1979).
The three domains of educational objectives according to Oyesola (1986) are inter-related. In general psychomotor domain deals with practical activities and some examples of practical and motor activities include writing legibly, drawing maps accurately, ability to manipulate laboratory equipment and use them effectively, maintaining farm tools, weave and make baskets etc (Osunde, 1997). This author posited that practical skill assessment requires some form of performance testing under a controlled condition.
The National Policy on Education (FRN, 2008) considers the acquisition of appropriate skills, abilities and competences as equipment for the individual to live and contribute to the development of the society as one of the cardinal national educational goals (FRN, 2008). The national policy was explicit on developing the manipulative skills of students in the schools and de-emphasizing the memorization and regurgitation of facts while encouraging practical, exploratory and experimental methods of developing motor skills. Also, for secondary and tertiary institutions, the national policy has vividly emphasized the acquisition of manual and practical skills that will enable us to live and keep pace in the modern age of technology. The various policies of national government as stated above can readily be realized by emphasizing qualitative assessment of the aspect of curriculum that teach practical skills. Therefore, for the objectives of national policy on technological advancement to be realized through the school system, a greater emphasis has to be given to the psychomotor assessment of practical aspect of various courses.
Since the instructional objective include psychomotor domains (practical skills), this domain should be assessed and stressed like the cognitive assessment. Generally, in sciences, WAEC and NECO place more emphasis on cognitive domain than on other domains of educational objectives. At secondary school level, most often than not assessment is concentrated on cognitive achievement to the detriment of psychomotor and affective development of the learners. This is not unconnected with Nigeria societal quest for paper qualification. Thus, a child with pass mark in his or her subject receives a certificate at the end of the course no matter how bad his/her manners are or how unskilled he/she may be (Idowu and Esere, 2009). In other words, psychomotor and affective traits do not fully count towards obtaining a certificate. Educational evaluators like Miller, Frank, Frank and Eheltor (1989), have prescribed a departure from excessive emphasis on only cognitive domain to make room for a more comprehensive picture in the development of the learners in the school system.
Test of practical skills in physics is measurement of psychomotor domain of behaviour. Different instruments exist for the assessment of psychomotor domains of our educational enterprise. To evaluate achievement in psychomotor domain, the procedures are the same with that of cognitive domain, although the objectives differ. The procedure for assessing the psychomotor domain includes among others practical work and projects (Harbor Peters, 1999). Test of practical skills are of importance because like every other reliable and valid test performances, they are utilized for selection of candidates, for further studies, for employment etc. Several experts such as Yoloye (2004), Harbor-Peter (1999), Nworgu (1992), and Gronlund (1975) have noted that achievement test in psychomotor domains of educational objectives serve the purpose of evaluating students progress and giving students, parents, family, school and society feedback on the students progress. Also, achievement tests in psychomotor motivate students to learn more; giving feedback on teaching effectiveness; predicting future performances; providing methods of selection, etc.
Achievement test in physics practicals is inevitable because the practice of physics equips us with the knowledge of underlying principle for the majority of our technological products. According to Egbugara (1989), “physics is the most fundamental science subject which act as the basic index to all courses in technological development and myriad of other scientific development necessary to mankind”. WAEC (2009) stated that the objectives of practical physics among others are to inculcate in students the spirit of scientific investigation, to establish some basic principles of physics using experiment, to understand the use of certain equipment, to develop the ability of conducting experiment according to specification while using same for analysis.
Also, Kirschner and Meester (1988) suggested the students’ centered objective for practical work to include (i) to formulate hypotheses (ii) to solve problems (iii) to use knowledge and skill in unfamiliar situation (iv) to design simple experiment for testing hypothesis (v) to use laboratory skills in performing experiment, interpret the data and draw inference. In the same vein, Carduff and Reid (2003) provided many possible reasons for inclusion of practical work in various subjects to include, illustrating key concepts, training in specific practical skills, developing observational skills, deduction and interpretation skills, developing problem solving skills, showing that theory arises from practical and developing the scientific bases for some products etc. Practical physics as a matter of fact is indispensable as they improve our disposition towards the scientific bases of technology. Adequate practical activities in physics correlate with good school results in physics. Practical, project and examinations, test achievement in psychomotor domain.
Test is only one technique of measuring educational outcomes and other techniques include questionnaire, interview, practicals, observations etc. Test connotes a structured situation comprising a set of questions which an individual is expected to respond to, on which bases his behaviour, and/or performance is quantified ( Habor Peters, 1999; Nworgu, 1992; Gronlund, 1976)
Gronlund (1976) hinted that the validity of information provided by the test (of practical skills) however, depends on the care with which the test are planned and developed. Also, measurement of practical skills in education is the quantitative description of pupils change in behavior and measurement instruments are tests, class work, projects, assignment etc (Habor Peters, 1999; Nworgu, 1992;). Nenty (2004) and Kerlinger and Lee, (2000), have suggested that for measurement in education to be meaningful, the objectives to be measured, the number to be assigned and the rules of the assignment of the number must be well defined. Yoloye (2004) stated that responses to tests and other measuring instruments enable the examiner to assign the testees numeral or set of numerals from which inferences could be made about the testees performance on whatever the test is supposed to measure. This means that a good instrument for testing of psychomotor ability should have some psychometric properties. In the first instance psychometric analysis is the science of measuring latent traits or constructs in our subjects of interests. The psychometric analysis of a test would imply analyzing such constituents of psychometrics as (i) Validity- whether a test measures what it is intended to measure (ii) Reliability – t he consistency in measuring what it intends to measure (iii) Difficulty index or conversely easiness index ( iv) Discrimination index –how sharply does the test distinguish between low and high ability students. The psychometric analysis of a psychomotor test in physics therefore implies analysis of practical test in physics to obtain the validity, reliability, difficulty and discrimination indices.
In practice, the relevance of practical test is largely dependent on the levels of reliability, validity, difficulty and discrimination indices. All these constitute psychometric properties of a test. The psychometric analysis of a test is a multi step process that can follow more than one measurement theory frame work. These frameworks are usually classical test theory (CTT) and item response theory (IRT).
The teacher, the school and various assessment agencies such as West African Examination Council (WAEC) and National Examination Council (NECO), etc are saddled with the responsibility of implementing the objectives as stated in national policy on education. In WAEC and NECO the analysis of psychometric qualities of their polytomously scored items are mostly done with classical test theory (Korashy, 1995). Thereafter the qualities of these items are kept as classified information and can be hardly accessed by the public, researchers or other educational agencies. Since the practical aspect of physics curriculum is a sine-qua-non to technological advancement and realization of objectives of national policy on education, it is therefore pertinent that the psychometric properties of practical tests by the examination bodies such as WAEC and NECO should be determined. This will indicate the overall quality of assessment/ test conducted by the examination bodies in practicals. This will go a long way towards enlisting confidence or otherwise in the examining bodies.
The ultimate examining bodies in secondary schools such as WAEC and NECO assess /test the psychomotor aspects of objectives through practical examinations in the sciences. WAEC, NECO and National Business and Technical Education Board (NABTEB) are the three bodies in Nigeria today that have the responsibility of awarding ordinary level certificate.
The origin of WAEC dates back to 1949, when the British Council of states for colonies invited Jeffry to visit West Africa to study and come up with a proposal for West African Examination Council (WAEC, 2002). The report was submitted and adopted in 1950 by four West African governments (Nigeria, Ghana, Sierra Leone and Gambia).
These governments came up with an ordinance that established WAEC as a cooperate body. WAEC in these countries conduct both national and international examination at both ordinary and advanced levels.
Also, in 1999, the Federal government of Nigeria established the National Examination council (NECO). The aim of this is for Nigeria to have an independent and national examination body that has the same standard with WAEC. The cooperate headquarters of NECO is Minna and they conduct national examinations such as examination into unity schools i.e. entrance examination into Federal government secondary schools, entrance examinations into schools for the gifted children and ordinary level school certificate examinations (NECO, 2001).
The WAEC physics O’level examinations are made up of three parts: paper 1 (practical – 50 marks); paper 2 (objective – 50 mar ks); and paper 3 (essay – 60 marks)(WAEC,2009).Exactly the same allocation of marks to various papers apply to NECO. For this study, only the practical questions will be used for analyses. This is because many studies like Obinne (2011, 2008) have dwelt on psychometric properties of objective test items in various subjects. Up to now no study could be assessed in literature that has attempted the analyses of psychometric properties of practical aspects of physics (polytomously scored with varied category) in WAEC and NECO O’level examinations.
The West African Examination Council and the National Examination Council base the analyses of psychometric properties of items on classical test theory framework (Obinne, 2008). On the premise of having weak theoretical assumptions and the item and person circular dependency statistics, the classical test theory has been seen as not precise as item response theory for ensuring objectivity in psychometric analyses (Ndalichako and Rogers, 1997; Smith, 1996; Korashy, 1995).
Despite the importance of practical physics, the various analysis of psychometric properties of questions (in research studies) have not attempted psychometric analysis of practicals (Korashy, 1995). Such studies as Obinne (2011), Obinne (2008), Nworgu (1985), Agwagah (1985), Obioma (1985) went variously into psychometric analyses of questions that are dichotomously scored. Moreover, most psychometric analyses so far used classical test theory (CTT) model. The classical test theory is no longer considered fully or 100% valid enough for ensuring objectivity in measurement (Smith, 1996; Korashy, 1995).
The classical test theory (that is mostly being used for test analysis) has many limitations such as circular dependency, weak theoretical assumptions etc which cast doubts when psychometric properties of tests are obtained using the CTT model. There is therefore, the need to change the method of analysis of psychometric properties of tests from CTT to a theory that will further attenuate the shortcomings of CTT model. In particular, there is the need to study the psychometric properties of physics practical in WAEC and NECO practical physics as many has been done for objective physics and virtually none for practical examinations and both are of equal weight in these exams ie they carry equal marks.
Almost every, if not all instruments used in Nigeria currently for assessment of achievements in our educational processes rely on classical test theory. This CTT model produces scale that yield different results across different population i.e. item and person parameter are sample dependent; there is weak theoretical assumption to meet with test data such as the assumption that error scores in high and low ability students are equal – in other words student error of measurement is consistent across the entire population and the sample size for the item parameter estimation is small. [Embretson and Reise, 2000; Fan, 1998; Hambleton and Jones, 1993; Lord and Norvick, 1968; and Lord, 1952; 1953]. The major limitation of CTT can be summarized as circular dependency (a) the person statistics (i.e. observed score) is item sample dependent and (b) the item statistics (i.e. item difficulty and item discrimination) are examinee sample dependent. This circular dependence poses some theoretical difficulty in CTT application in some measurement situations (Fan, 1998).
Due to the inherent advantages of item response theory, it becomes absolutely compelling that emphases are to shift from classical test theory to item response theory in test analyses. Theoretically IRT overcomes the major weakness of CTT, that is the circular dependency of CTT item/ person statistics. As a result IRT models produce item statistics independent of examinees samples and person statistics which are independent of the particular set of items administered. This invariance property of item and person statistics of IRT has been illustrated theoretically (Hambleton and Swaminathan, 1985); Hambleton, Swaminathan and Rogers, 1991) and these have been widely accepted by the global measurement community: This invariance property of IRT model parameters makes it theoretically possible to solve some measurement problems that have been difficult to handle within the CTT framework such as computerized adaptive testing (Hambleton et al., 1991). The importance of invariance property of IRT model parameter cannot be overstated. Without this crucial property however, the complexity of IRT models can hardly be justified on either theoretical or practical grounds (Fan, 1998).
Item Response Theory (IRT) is an attempt to model the relationship between an unobserved variable - the examinees ability and the probability of the examinee correctly responding to any particular test item. IRT models are therefore mathematical functions which relate the probability of success on a task to the underlying proficiency measured by the task. IRT avails us of the opportunity of attaining invariant item parameters such as difficulty index (b-parameter), discrimination index (a-parameter), guessing index, (c-parameter) in the cases of dichotomously scored responses. IRT on the whole is a statistical framework for addressing measurement problems such as test development, test score equating and identification of biased test items (Hambleton and Jones, 1991). With IRT, it is possible to construct trait line for exact measurement of a particular trait possessed by an individual. The foregoing merits of IRT made possible by invariant property of Item Response Theory (IRT) makes IRT a plausible alternative to classical test theory in an attempt to enthrone better objectivity in measurement.
Item response theory can be divided into two families – uni-dimensional and multi-dimensional models. While uni-dimensional model require a single trait or ability dimension, multi dimensional IRT models response data (instrument data) arising from multiple traits. Most item response theory researchers and applications make use of uni-dimensional IRT models. IRT models are also categorized on the bases of scored responses. The typical multiple choice items are dichotomously scored. Even if there are four or five options they are still being scored as correct or incorrect, right or wrong. A different class of models apply to polytomous outcomes where each response has different score values. An example of polytomously scored items are those rated on a scale of 1-5 or a situation where some number of steps are required to complete a particular assignment.
The relationship between examinees performance and the set of traits underlying the item performance can be explained by a monotonically increasing function known as Item Characteristic Curve (ICC) or Item Characteristic Function (ICF) (Hambleton, Swaminathan and Rogers, 1991). For items that are dichotomously scored, the ICF can be verified using the one parameter, two parameter and three parameter logistic models. Using these models the item statistics,- the item difficulty (b-parameter), item discrimination (a-parameter) and pseudo guessing, (c- parameter) can be verified for items that are dichotomously scored. The one parameter model (Rasch model) can only verify b, the two parameter model or Birnbaun model can verify b and a; while the three parameter model or Lords models can verify b, a and c.
For items that are polytomously scored various models for studying the item statistics exist. Some of these models are: Graded Response model, Nominal Model, Partial Credit model and Rating Scale Model. The various models provide mathematical equations for the relationship that exists between the probability of correct response (q) to the ability level of the student. Each of the models has one or more parameter –(b, a or c defined above) that describes a particular item characteristic curve along with other technical properties of the item. In each IRT model, a mathematical function is used to estimate the probability of correct response at several ability levels (-3 to +3).
In IRT, the Item Characteristics Curve (ICC) is described by (i) the difficulty parameter b which is the location on the ability axis at which the probability of correct response P(q) = 0.5 (ii) the discrimination parameter (a) which is the slope of ICC at a particular ability level; the higher the value of a, the steeper the slope (iii) the guessing parameter c is the vulnerability to guessing which makes ICC asymptotic and always positive along vertical axis.
The assessment of psychomotor/practical skills in any area will be better studied if the examinees responses are polytomously scored rather than dichotomously scored. Cognitive outcomes may be simply studied using dichotomous scoring but for psychomotor outcomes they are better done with polytomous scoring. This makes partial credit scoring indispensable in many assessment situations. The usual motive for partial credit scoring as stated in Masters (1982) is the hope that it will lead to a more precise estimate of person’s ability than a simple pass/fail score. This author noted that certain type of data should come from an observation format which requires the prior identification of several other levels of performance on each item and thereby award partial credit for partial success on each item.
The polytomous Rasch model which is a generalization of the dichotomous model can be applied in contexts in which successive integer scores represent categories of increasing level of magnitude of a latent trait such as increasing ability, motor function and so forth. The PCM is an appealing model for many applications because unlike the Graded Response model, Generalized partial credit model, etc it does not contain a discrimination parameter and thus can be used with sample sizes that are smaller than those required for models containing a discrimination parameter. Furthermore, because the PCM belongs to Rasch family of models it brings with it the advantageous properties known to exist for all Rasch models including separation of person and item parameters (Andrich, 1988; Michell, 1990; Fischer, 1995).
The polytomous Rasch model which is a generalization of the dichotomous model can be applied in contexts in which successive integer scores represent categories of increasing level of magnitude of a latent trait such as increasing ability, motor function and so forth. The PCM is an appealing model for many applications because unlike the Graded Response model, Generalized partial credit model, etc it does not contain a discrimination parameter and thus can be used with sample sizes that are smaller than those required for models containing a discrimination parameter. Furthermore, because the PCM belongs to Rasch family of models it brings with it the advantageous properties known to exist for all Rasch models including separation of person and item parameters (Andrich, 1988; Michell, 1990; Fischer, 1995).
Hypothetically, an examinee could take the test a great many times and obtain a variety of test scores. One would anticipate that these scores would cluster themselves around some average value. In measurement theory, this value is known as true score and its definition depends upon the particular measurement theory. In item response theory, the definition of true score according to Lawley, in Baker (2001) is given by
Ts j
=
N
∑
Pi (q j )
1=1
where
Tsj is the true score for examinee with ability level qj i denotes an item and Piq j depends on the item characteristics curve model employed
Just like other IRT models, the PCM is characterized by specific objectivity and uni-dimensionality. Mellember (1994) stated that specific objectivity means that comparison of two items difficulty are assumed to be independent of any group of subjects being studied and does not depend on any subset of item being administered. And uni-dimensionality means that a single latent variable fully explains task performance (Carlson, 1993).
Latent characteristics of examinees cannot be measured with physical implements or instruments like measurements in the physical sciences. But objective measurements of latent traits can be achieved using the item response approach. The sample free nature of the results provided by the IRT models are technically known as Invariance of item parameters. With this invariance of item parameters a uniform scale of measurement is provided for use in different populations. Latent traits are behaviours that can be indirectly observed. To measure these traits we need to provoke the examinees to act while trying to capture the intensity of such a trait in the individual by putting up a related graded task (known as item) for the examinees. Through this we can elicit the behaviour that describes the trait under study (Nenty, 2005).
All examinations conducted in Nigeria have been based on classical Test Theory framework CTT for years (Obinne, 2008). Examination bodies have as well relied on CTT for testing their candidates. Psychometric analyses done on instruments by Nworgu (1985), Agwagah (1985), Obioma (1985) all relied on classical test theory. There is nearly no psychometric analysis of practical physics yet as revealed by literature. Even though practical physics in WAEC and NECO have the same weight with their objective tests, studies that have so far been done have mostly dwelt in objective aspect. There is therefore the need to study the psychometric qualities of our practical questions in these two examinations using item response theory Format-Partial Credit, model. This will help attenuate the shortcomings of CTT model that is presently used for analysis by the exam bodies. There is also the need for more concise, objective and pragmatic method of constructing, scoring and analyzing psychometric properties of practical physics. This will go a long way to convince the public that the standard of the two examinations WAEC and NECO are about the same thereby removing bias and doubt against any of their standard as is sometimes the case.
1.2 STATEMENT OF THE PROBLEM
West African Examination Council Ordinary level examinations in Physics are made up of three parts, namely paper 1(practical -50 marks); Paper 2 (objective-50 marks) and paper 3 (essay-60 marks) (WAEC 2009). Exactly the same mark allocation is applicable to the respective patterns of assessment in National Examination Council O’level physics examination. Studies on psychometric properties of WAEC and NECO examination have dwelt on the objective component of these examinations (i.e. paper 2) to the exclusion of the practical, component (i.e. paper 1) and the essay component (i.e. paper 3). The polytomously scored components i.e. practical (paper 1) and essay (paper3) of these examination (practical and essay) contribute more than two thirds of the total score on both WAEC and NECO O’level physics. Yet there is no study that has gone into psychometric analyses of polytomously scored components of WAEC and NECO physics examinations. This situation creates an obvious gap, part of which this study was able to address.
Furthermore, these studies on psychometric properties of WAEC and NECO mostly utilized the classical test theory (CTT) approach in their analyses. The modern measurement theory – item response theory (IRT) has not been fully explored with respect to analyses of psychometric properties of our tests in WAEC and NECO. Considering the obvious advantages in the underlying assumption and basic tenets of the framework- IRT, would the scenario be different if we utilize IRT for analyses of psychometric properties of WAEC and NECO examinations in Nigeria?
1.3 PURPOSE OF THE STUDY
The purpose of the study was to investigate some psychometric properties of WAEC and NECO practical physics questions using the partial credit model of item response theory.
Specifically, the study did:
1. Estimate the standard error of measurement of the practical physics test items set by the National Examination Council (NECO) of Nigeria.
2. Estimate the standard error of measurement of the practical physics test items set by the West African Examination Council WAEC.
3. Investigate the validity of the practical physics test items produced by NECO.
4. Investigate the validity of the test items of practical physics test items produced by WAEC.
5. Estimate the item parameter (item difficulty) of NECO practical physics test items using the Partial Credit Model.
6. Estimate the item parameter (item difficulty) of WAEC practical physics test items using Partial Credit Model.
7. Determine the proportion of fit of NECO practical physics questions using Partial Credit Model of IRT.





