Training Modules
Displaying 9 Training Modules.
Quantitative Research Methods in Educational Planning
Research is the orderly investigation of a subject matter for the purpose of adding to knowledge. Research can mean ‘re- search’ implying that the subject matter is already known but, for one reason or another, needs to be studied again. Alternatively, the expression can be used without a hyphen and in this case it typically means investigating a new problem or phenomenon.
Within the realm of educational planning, many things are always changing: the structure of the education system, curriculum and textbooks, modes of teaching , methods of teacher training, the amount and type of provisions to schools such as science laboratories, textbooks, furniture, classroom supplies, and so on. These changes may lead to an improvement, or a worsening , in the quality of an educational system. Sometimes they may result in no impact upon quality – in which case major government expenditures on such changes have been wasted. The educational planner working within this kind of environment must be able to undertake assessments of the effects of major changes and then provide policy advice that will consolidate and extend the post productive courses of action, and also intercept and terminate existing practices that are shown to be damaging and wasteful.
In an ideal world, educational research h as a vital role to play in the improvement of education, whether this be in the development of theory to better explain why things occur the way they do in particular learning situations, or stimulating ideas for innovative practices, or developing new procedures and materials to enhance the efficiency and effectiveness of instruction. Educational research also has the role of providing attested information to improve the quality of decision-making for education al policy. It is this last objective which forms the object of this module.
At the outset it should be admitted that this happens all too rarely, for reasons which will be explained in the next section.
Policy decisions are often ta ken in t he absence of good research, and sometimes in spite of the finding s of available research. Furthermore, creating a well-researched policy does not mean taking any action on that policy! But at least it is a beginning. It is the objective of this modu le to assist researchers to interact with policy makers in fruitful ways, so that gaps are bridged and research results made available in forms which are helpful to all.
Numerous definitions of policy research exist. A simple and useful one is ‘research under taken by qualified researchers in order to produce evidence on issues of importance to policy-makers’. The hope is that such evidence ca n then be used to help in formulation or revision of laws or educational policy guidelines. The intention always is that decision-oriented research should provide results which are useful for resolving current problems in education.
Paris: International Institute for Educational Planning (UNESCO).
Sampling in educational research is generally conducted in order to permit the detailed study of part, rather than the whole, of a population. The information derived from the resulting sample is customarily employed to develop useful generalizations about the population. These generalizations may be in the form of estimates of one or more characteristics associated with the population, or they may be concerned with estimates of the strength of relation ships between characteristics within the population.
Paris: International Institute for Educational Planning (UNESCO).
Educational planners are continually being asked to participate in, and to provide information that can be used to guide administrative decisions. These decisions may range from developing a set of detailed procedures for some aspect of an educational enterprise, to a major reorganization of an entire education system. The quality of such decisions is the major determinant of successful administrative practice and, eventually, these decisions define the long-term nature of educational organizations.
Paris:International Institute for Educational Planning (UNESCO).
Within the field of test development, the tasks and /or questions that are used to construct tests and examinations are refer red to as ‘items’, and the range techniques involved in preparing t hose items are collectively refer red to as ‘item writing ’.
Is item writing an art or a science? T he best item development techniques combine elements of both these intellectual activities. On t he one hand, there is a fair amount of experimental method, which we might recognize as scientific, incorporated within the whole set of procedures for developing a good item, or t he sets of such things we call ‘tests’. However, as this document will make clear, writing a good item is also a highly creative act. By the end of the process something new, powerful, and useful has emerged – a test instrument which has used words, symbols or other materials from a curriculum or a syllabus in a new way, often to serve a variety of educational purposes. In doing so, the item developer needs imagination and ingenuity as well as knowledge: form, structure and balance become important, as they are to a sculptor or a musician.
Paris: International Institute for Educational Planning (UNESCO).
Assessment of student learning provides evidence so that educational decisions can be made. We m ay use the evidence to help us evaluate (or judge the merit of) a teaching programme or we may use the evidence to make statements about student competence or to make decisions about the next aspect of teaching for particular students.
The choice of what to evaluate, the strategies of assessment, and the modes of reporting depend upon the intent ions of the curriculum, the importance of different parts of the curriculum, and the audiences needing the information that assessment provides. For example, national audiences for this information may include both those who will be making decisions and those who wish or need to know t hat appropriate decisions have been taken.
Paris: International Institute for Educational Planning (UNESCO).
Assessment involves selecting evidence from which inferences can be made about current status in a learning sequence. The tasks that are chosen to provide that evidence have to be effective in distinguishing between those who have the required knowledge and those who do not. Trial testing (sometimes called pi lot testing) involves giving a test under specified conditions to a group of candidates similar to those who will use the final test. Subsequent analysis of the data from the trials ex amines the extent to which the assessment tasks per formed as expected under practical conditions.The shading i n Figure 1 below, i ndicates t he position of trial testing and item analysis in the overall test construction process.
Paris: International Institute for Educational Planning (UNESCO).
T his module provides guidance for the design of standardized questionnaires that are to be administered in school systems to students, teachers, and school heads. The module is divided into four sections that cover initial planning, the design of questions, examples of question types, and moving from a draft to a final questionnaire.
After reading this module, the reader should be able to design a quality survey questionnaire that is suitable for addressing the research issues at hand. He or she w ill know how to:
- Decide on the target population for the questionnaire.
- Identify the variables and indicators that w ill address the research issues and hypotheses on which data are to be collected.
- Develop demographic, knowledge, attitude, and practice questions.
- ‘Close’ open ended quantitative and qualitative questions and design skip, filter, and contingency questions, where appropriate.
- Decrease response bias and maximize response rates.
- Design probe questions and interviewer or respondent instructions on the questionnaire.
Paris: International Institute for Educational Planning (UNESCO).
Whenever data are collected in educational survey research studies, two problems are often found at the data preparation phase. First, errors can be introduced in the entry of data into computers and as a result some data collections provide inaccurate and faulty results. Second, the computer entry and cleaning of data prior to the main data analyses can be extremely time consuming and therefore this information can rapidly become “out of date” and consequently lose its value to policy-makers.
The root causes of these two problems of “accuracy” and “timeliness” are sometimes associated with the selection of inappropriate research designs or the use of research designs that are not manageable within prevailing economic, administrative and socio-cultural constraints. In other cases, these two problems arise from the lack of a systematic analysis of decision-making requirements that can cause too many data of limited use to be collected.