Dissertations / Theses on the topic 'Educational tests and measurements Victoria'

Current research indicates there is a correlation between participating in an early childhood program and a student’s performance on future standardized measures, including the challenge of using early learning standards (Feldman, 2010). This research study focused on state initiatives, and student participation in an early childhood preschool model centered on the use of universal curriculum and assessment designed to measure student outcomes aligned to learning targets, outlined in state preschool curriculum standards. Research shows learning decreases for students who have not participated in an early childhood program, while those who have participated in some kind of early childhood program show progress (Heckman, 2011). Young children come to school with varying degrees of experiences, which may or may not enhance their learning. Educators are responsible for providing positive experiences and provide academic activities to develop academic awareness, social/emotional skills, in addition to displaying appropriate behavioral skills. Participation in preschool should also build a student’s level of independence and competency skills. This research study examined state initiatives and curriculum materials, and assessment tools related to the importance of early childhood education programming and teacher practices, and the impact of universal curriculum and assessment implemented in the classroom during the school year. In addition, it further explored teacher perspectives on educational programming, Louisiana’s early childhood initiatives, and the use of universal curriculum and assessment in their classroom.

The college admission systems of the United States require the Scholastic Aptitude Test (SAT) and American College Testing (ACT) examinations. Although, some resources suggest that SAT and ACT scores give some meaningful information about academic success, others disagree. The objective of this study was to determine whether there is significant predictive validity of SAT and ACT exams for college success. This study examined the effectiveness of SAT and ACT scores for predicting college students’ first year GPA scores with a meta-analytic approach. Most of the studies were retrieved from Academic Search Complete and ERIC databases, published between 1990 and 2016. In total, 60 effect sizes were obtained from 48 studies. The average correlation between test score and college GPA was 0.36 (95% confidence interval: .32, .39) using a random effects model. There was a significant positive relationship between exam score and college success. Moderators examined were publication status and exam type with no effect found for publication status. A significant effect of exam type was found, with a slightly higher average correlation for SAT compared to ACT score and college GPA. No publication bias was found in the study.

Test anxiety is a phenomenon which has been researched for decades. Student performance, goal attainment, and personal lives are all negatively affected by the multiple factors of test anxiety. This quantitative study was designed to determine if a particular relaxation technique, called sensory activation, could mitigate the symptoms and effects of test anxiety. The Test and Anxiety Examination Measure, developed by Brooks, Alshafei, and Taylor (2015), was used to measure test anxiety levels before and after implementation of the sensory activation relaxation technique. Two research questions guided the study using not only the overall test anxiety score from the Test and Anxiety Examination Measure, but also using the five subscale scores provided within the instrument. After collection and analysis of data, the results for research question one indicated a statistically significant positive difference in mean levels of overall test anxiety. Not only were overall mean test anxiety levels lowered, but findings for research question two showed significant decreases in worry and state anxiety subscale scores. Considering the sensory activation relaxation technique was used during the examination period, it is reasonable to assume its effectiveness would be limited to lowering state anxiety levels rather than trait anxiety levels. Also, results from prompt 10 of the Test and Examination Anxiety Measure (Brooks et al., 2015) indicated the sensory activation relaxation technique could serve as a possible deterrent to the “going blank” problem as described anecdotally by students. Instructors could introduce the sensory activation relaxation technique to their students prior to the first testing event in the course, thus producing the desired outcomes of better test performance and less anxiety.

Relieving test anxiety actions range from relaxation exercises to prescription medication. Humor can be a simple method of test anxiety relief. The current study was used to determine if humor, in the form of a cartoon, placed on the splash page of an online exam improved the test scores of students who have high test anxiety. In the current study, 2 theories were used to guide the research. The interference theory by Ralf Schwarzer and Matthias Jerusalem indicated students have difficulty separating competing thoughts during an exam. In the adult learning theory by Malcolm Knowles, the learning of children and adults was differentiated, while explaining how adults learn. A quasi-experimental quantitative design was used to find a possible correlation between humor and test anxiety relief. The study sample comprised an equal number of students with high test anxiety and students with low test anxiety. The low test anxiety group comprised the control group. A 2-sample t test was used to search for a correlation between the cartoon and the exam scores. Intended benefits of the study included: (a) students with test anxiety find relief from test anxiety, (b) instructors achieve reliable assessments of students with test anxiety, and (c) confident, well-educated graduates. The current study results showed the opposite of expected results. The high test anxiety group did worse on the exam with the cartoon. The 2-sample t test showed a negative improvement of –6.222 between midterm and final exams for the high test anxiety group.

In general, newer educational assessments are deemed more demanding challenges than students are currently prepared to face. Two types of factors may contribute to the test scores: (1) factors or dimensions that are of primary interest to the construct or test domain; and, (2) factors or dimensions that are irrelevant to the construct, causing residual covariance that may impede the assessment of psychometric characteristics and jeopardize the validity of the test scores, their interpretations, and intended uses. To date, researchers performing item response theory (IRT)-based model simulation research in educational measurement have not been able to generate data, which mirrors the complexity of real testing data due to difficulty in separating different types of errors from multiple sources and due to comparability issues across different psychometric models, estimators, and scaling choices.

Using the context of the next generation K-12 assessments, I employed a computer simulation to generate test data under six test configurations. Specifically, I generated tests that varied based on the sample size of examinees, the degree of correlation between four primary dimensions, the number of items per dimension, and the discrimination levels of the primary dimensions. I also explicitly modeled the potential nuisance dimensions in addition to the four primary dimensions of interest, for which (when two nuisance dimensions were modeled) I also used varying degrees of correlation. I used this approach for two purposes. First, I aimed to explore the effects that two calibration strategies have on the structure of residuals of such complex assessments when the nuisance dimensions are not explicitly modeled during the calibration processes and when tests differ in testing configurations. The two calibration models I used included a unidimensional IRT (UIRT) model and a multidimensional IRT (MIRT) model. For this test, both models only considered the four primary dimensions of interest. Second, I also wanted to examine the residual covariance structures when the six test configurations vary. The residual covariance in this case would indicate statistical dependencies due to unintended dimensionality.

I employed Luecht and Ackerman’s (2017) expected response function (ERF)-based residuals approach to evaluate the performance of the two calibration models and to prune the bias-induced residuals from the other measurement errors. Their approach provides four types of residuals that are comparable across different psychometric models and estimation methods, hence are ‘metric-neutral’. The four residuals are: (1) e0, which comprises the total residuals or total errors; (2) e1, the bias-induced residuals; (3) e2, the parameter-estimation residuals; and, (4) e3, the estimated model-data fit residuals.

With regard to my first purpose, I found that the MIRT model tends to produce less estimation error than the UIRT model on average (e2MIRT is less than e2UIRT) and tends to fit the data better than the UIRT model on average (e3MIRT is less than e3UIRT). With regard to my second research purpose, my analyses of the correlations of the bias-induced residuals provide evidence of the large impact of the presence of nuisance dimension regardless of its amount. On average, I found that the residual correlations increase with the presence of at least one nuisance dimension but tend to decrease with high item discriminations.

My findings shed light on the need to consider the choice of calibration model, especially when there are some intended and unintended indications of multidimensionality in the assessment. Essentially, I applied a cutting-edge technique based on the ERF-based residuals approach (Luecht & Ackerman, 2017) that permits measurement errors (systematic or random) to be cleanly partitioned, understood, examined, and interpreted—in-context and in relative to difference-that-matters criteria—regardless of the choice of scaling, calibration models, and estimation methods. For that purpose, I conducted my work based on the context of the complex reality of the next generation K-12 assessments and based on my effort to maintain adherence to the established educational measurement standards (American Educational Research Association (AERA), the American Psychological Association (APA), and the National Council on Measurement in Education (NCME), 1999, 2014); International Test Commission (ITC) (ITC, 2005a, 2005b, 2013a, 2013b, 2014, 2015)).

The educational system in the United States continues to pose many challenges for law and policy makers. Many of these challenges can be traced back to two landmark cases, Plessy vs. Ferguson and Brown vs. Board of Education of Topeka. And, while the U.S. Department of Education developed programs to address many of these issues, the cost versus the benefits must be considered. This research study examined the impact of federally-funded School Improvement Grants (SIGs) for elementary, middle, and high schools across the state of Missouri from 2010 to 2015 on retention rates, graduation rates, and test scores. The state of Missouri identified 56 schools as low-performing, and therefore, eligible to receive the grants. Specifically, this study examined whether the amount of SIG funds allocated per student was associated with increases in achievement scores (mathematics and English), graduation rates, and dropout rates. Using bivariate regression, the findings showed a statistically significant relationship only between the amount of SIG funds allocated per student and English scores. Surprisingly, the relationship showed that as the amount of funds allocated per student increased, English scores decreased. However, after a multivariate regression, findings indicated mathematics scores significantly increased as the amount of SIG funds per student increased, while English scores remained significant in the same direction. This research study also analyzed the relationship between the amount of SIG funds allocated per student and median household income during the first year the funds were disseminated. Because special attention was given to the educational achievement gap and race/ethnicity, this research study also compared Black and White student populations. The results showed that as the population of Black students increased, mathematics and English scores decreased. Furthermore, the findings showed that as the population of Black students increased, the amount of SIG funds allocated per student decreased. This suggested that there may be a need to examine how funds were allocated and what other issues may have confounded the relationships between SIG funds and the major variables presented in this research.

The objective of this exploratory quantitative study was to investigate the relationships between grit, self-control, and the first academic semester of college students, and determine if the relationships differed by gender. Two research questions were examined; (1) What are the relationships between the individual factors of grit, self-control, and first-semester college GPA? And do they differ by gender? and (2) What combinations of factors (grit, self-control, high school GPA, and SAT scores) best predicts first-semester college GPA? And do they differ by gender?

This study investigated 88 first-time, first-year college students and their academic success during their first college semester using three instruments: the 12-Item Grit Scale, the Self-Control Scale, and the Short-Form C of the Marlowe-Crowne Social Desirability Scale. Correlation analysis and stepwise regression methods were used to examine relationships.

Findings from this study reinforce that high school GPA and SAT scores are predictors of college academic performance. However, the relationships between high school GPA, SAT scores, and fall GPA in this study were not as strong as indicated in previous studies. Results indicated that grit and academic performance had no relationship, while a small yet significant relationship was found between self-control and academic performance. Additionally, male and female students had somewhat different results in terms of grit, self-control, and academic performance. Predictors of academic performance for male students were high school GPA, self-control, and SAT scores. Female students’ predictors were high school GPA and SAT scores. During an exploration process in this study, self-control was the only predictor of students’ fall GPA when it was less than 2.67. Gender did not play a role in that particular finding, and the best and only predictor of all students’ fall GPA < 2.67 was self-control. That indicated that levels of earned GPA may be related to levels of self-control. The overall findings of this study contribute to further understanding factors related to college success, graduation, and better options for both life and career.

Conflicting opinions and stances concerning standardization of curriculum in the United States education system are evidenced throughout the nation in the format of debates about the purpose of education and the curriculum. In 1892, nationally recognized American educators met as a Committee of Ten (National Education Association [NEA], 1893) to determine what subject matter should be contained in a formalized system of education, thus establishing the roots of the modern American school system. The results from the meeting were not accepted by all educational entities within the United States and curriculum content continued to be a matter of social and political debate resulting in legislative mandates designed to transform educational policy and practice. Leaders on the local, state, and national levels continue to pass new laws establishing and regulating educational standards and measurements for accountability, while classroom teachers are directed to adhere to many new directives and to become adept at a myriad of strategies and requirements to avoid being judged as inept and ultimately removed from the classroom. This action research study investigated the effectiveness of the use of Learning Targets throughout classroom curriculum by teachers as they promote instructional alignment to ensure student learning. This study explored how teachers develop, deliver, and assess student learning based on the processes and strategies contained within the Learning Target Theory of Action. It determined the perceptions about the processes involved and effectiveness of the Learning Target theory in the classroom. Finally, it placed emphases on ascertaining how students perceive the effectiveness of Learning Targets to their success in learning.

Research has shown that the working environment of physical education influences teacher motivation. Identifying the characteristics of teachers’ working environment may contribute to developing a productive and motivating working environment for physical education teachers.

The first part of this dissertation study was focused on developing and validating an instrument that measures physical education teachers’ job demands/resources perception on five theorized dimensions: organizational resources, physical resources, cognitive demands, physical demands, and emotional demands. The content validity was achieved through expert evaluation of the consistency between the items and the dimensions they represent. The evaluation rendered an average consistency rating of 3.6 on a 5 point scale. The construct validity and reliability were determined with a physical education teacher sample (n=193). Exploratory factor analysis (EFA) established a five-dimension construct structure matching the theoretical construct with factor loadings ranging from .57 to .85. The intraclass correlational coefficients ranged from .75 to .80 for job resources and from .80 to .83 for job demands, respectively. The inter-scale correlational coefficients ranged from .14 to .25, showing both convergent and divergent validity. Confirmatory factor analysis (CFA) confirmed the construct structure found in the EFA with high dimensional factor loadings ranging from .47 to .81 for job resources scale and from .51 to .86 for job demands scale. The model fit tests produced acceptable indices including the RMSEA < .05. It is concluded that the instrument met the required psychometric standards to be useful to measure physical education teachers’ perception of their working environment.

The second part of the study was to determine the extent to which the perceived job demands and resources influence physical education teachers’ motivation regulatory processes and motivation. An a priori model was proposed for testing based on the integration of the Self-Determination Theory (SDT) and Job Demands-Resources Model. Certified physical education teachers in two southeastern states (n=193) provided self-reported data on perceived job demands and resources in the working environment, motivation regulatory processes, and motivation to teach. Structural equation modeling analysis revealed that increasing job demands in working environment enabled the teachers to adopt more autonomous regulatory processes, such as integrated regulation (γ = .20) but not more controlling regulatory processes, such as external regulation (γ = -.16) and introjected regulation (γ = -.22). The findings of the study provide empirical evidence that relate perceptions of working environment to physical education teachers’ motivation to teach.

Research in the area of 21st century learning suggested the American public education system lacked educational preparation for students to compete in a global/connection economy. The United States performed lower than other nations on the Programme for International Student Assessment (PISA) 2012 reading performance. However, the United States had a higher number of 21st century skills and knowledge embedded in the curriculum. The 21st century skills, referred to as the 4Cs (collaboration, communication, creativity, and critical thinking) served as the foundation of this research. Moreover, while the United Sates performance on the PISA was no match to other developed nations; the country ranked above competitors in other international indexes such as the Global Competitiveness Index and the Global Entrepreneurship Monitor.

The researcher analyzed data using an Analysis of Variance (ANOVA), Pearson Product Moment Correlation Coefficient (PPMCC), and Chi-Square tests for independence and goodness of fit, to seek a possible relationship between the number of 21st century skills included within the 2012 reading curriculums in the countries of Finland, Singapore, and the United States and compared to reading scores measured by the 2012 PISA. For the null hypotheses numbers one through five the researcher applied a PPMCC to the data by comparing a single 4C to the score of each researched country for reading PISA results. With exception to null hypothesis three, a significant inverse relationship existed between the number of 21st century skills included within the 2012 reading curriculums and the 2012 PISA reading scores of the researched countries. Although null hypothesis three was not significant, an observable inverse relationship did exist. This study revealed when a country scored higher on the PISA 2012, the total number of 21st century skills included in the reading curriculums were lower. Additionally, students within the American educational system may benefit from increased focus on academic performance and instructional design to harness creativity and develop an entrepreneurial spirit.

This study was a documentation and assessment of Beta Academy Alternative School’s (pseudonym) transition to a newly introduced educational model/ leadership paradigm and examination of student educational outcomes, resulting from the leadership change. As a first year administrator, the researcher undertook the task of transforming an underperforming alternative education program by targeting areas of identified deficiency and/or concern (graduation rates, attendance, and discipline).

In this study, the researcher executed a mixed-method evaluation of the new educational model in an effort to determine contributions to success, potential barriers to change, and the characteristics associated with both, as well as the quantitative analysis that would support or not support the researcher’s hypotheses.

The first goal of the study stated that following the implementation of the new model for alternative education, building discipline referrals would decrease by 10% per semester, as compared to previous referral data. The results indicated a 280% decrease in student referrals, thus illustrating a dramatic and statistically significant decrease.

The second goal indicated that graduation rates would increase or would stay the same, within 2% of previous rates (percentage of total seniors), as compared to the previous year’s results and following implementation of the new educational and leadership paradigms. A z-test for difference in proportion tested a change in graduation rates of less than 1%, thus, supporting the graduation rates goal.

The last goal outlined in the study stated that following the implementation of the new model for alternative education, building attendance would increase by 30% per semester, as determined by ADA hours and compared to previous attendance data. Upon calculation, it was determined that there was an increase in attendance of 36.2% providing statistical support that the increase in attendance was significant, as well as met the outlined goal for attendance improvement.

The qualitative component of the study used responses to a questionnaire analysis to gauge stakeholder involvement and perceptions associated with the new educational model. The feedback was positive and indicated the measured criteria to be impactful and effective in the areas of fidelity, implementation, development, and attainment of desired goals.

The current study seeks to validate a second-order, multifaceted model of engagement that contains a behavioral, an emotional, and a cognitive subtype as proposed by Fredericks, Blumenfeld, and Paris’ (2004), while also incorporating literature on student interactions. The second-order, 12-factor model proposed and tested for its validity partitioned engagement into the second-order constructs of academic and social engagement and examined each of the three engagement subtypes in relation to the interactions that students experience with their course material, with their classmates, and with their instructors/teaching assistants. Since the proposed model did not meet accepted standards of fit, the dataset was randomly split into two approximately equal halves and a follow-up exploratory factor analysis (EFA) was conducted on the first half of the dataset, which yielded a second-order, five-factor solution. The second-order academic engagement constructs that emerged from the EFA consisted of students’ behavioral, emotional, and cognitive engagement with their course material. In addition, two first-order factors emerged from the EFA, consisting of students’ emotional and cognitive engagement with their fellow students or classmates.

These constructs and relationships were consistent with the theory that drove the original proposed model, but differed slightly in their composition and relationship with one another. After establishing this empirical model through EFA procedures, the model was cross-validated on the second-half of the randomly split dataset and examined for invariance across students enrolled in online courses and students enrolled in traditional, in-person college courses, as well students from ethnically and economically diverse backgrounds. Latent mean comparisons revealed differences in levels of academic and social engagement between these three groups of students, suggesting that students enrolled in online courses and students from African-American and Latino/a ethnicities were slightly more academically engaged than their counterparts. However, students enrolled in online courses scored much lower than students enrolled in face-to-face courses on the social engagement measures, while students from African-American and Latino/a ethnic groups scored higher on the social engagement measures than did students from Asian and Caucasian ethnicities. Interestingly, no differences emerged between groups of students from lower and higher economic backgrounds.

While many factors have been identified as influencing student academic performance, previous studies consistently determined effective teaching as the most significant factor, within the control of educators, leading to improved student achievement. Nonetheless, educational experts, statisticians, and policy-makers alike acknowledged the complexity of isolating the contributions of individual teachers on their students’ achievement. Converging with these changing beliefs about teaching and learning, the landscape of education faced an additional challenge—marked by an increased demand for schools and individual teachers to be held accountable for the academic growth of his/her students. Local districts have been empowered to create and implement teacher evaluation systems, with the caveat they maintain student achievement data as one measure of teacher effectiveness.

While there has been research conducted investigating a relationship between performance-based teacher evaluation systems and student achievement, studies have been limited to the most common large-scale models. This study was unique because the research focused on a specific teacher evaluation system, created by and for, a rural Missouri school district during its first two years of implementation. The purpose of this mixed-methods research study was two-fold: (1) to investigate the relationship between teachers’ annual evaluation ratings (as measured by the researched district’s teacher evaluation tool) and their students’ academic performance (as measured by the MAP and i-Ready assessments), and (2) to analyze teacher and administrator perceptions of the impact of the new teacher evaluation system on improving student achievement and the teachers’ instructional performance.

This study’s analysis took both math and reading achievement scores into account, considering two different standardized assessments, the state-mandated Missouri Assessment Program (MAP) and a locally-administered i-Ready Benchmark Assessment. The student achievement data yielded an increase in student achievement over the two years of the study. However, the results of the study did not establish a correlation between the two variables: teacher quality and student achievement. More sensitive evaluation methods are needed to isolate the variable of teacher evaluation ratings on student achievement.

This qualitative, multi-case study was designed to examine the impact high-stakes accountability and data-driven decision making has had on administrators and teacher leaders. Serving as the conceptual framework, instructional leadership theory is defined as a multitude of relationships, behaviors, and responsibilities that directly impact student achievement (O’Donnell & White, 2005; Bottoms & Fry, 2009). The researcher utilized instructional leadership theory as lens to explore the participants’ thoughts, feelings and perceptions with respect to the implementation of these tenets (Mertens, 2005). The focus of this study is to analyze how administrators and educators are directly responsible for students’ performances and with the rigors of accountability from the principles of NCLB, educators are having to turn to new approaches such as data-driven decision making (King, 2002) and quick-paced instruction to meet the needs of students.

A qualitative multi-case study approach allowed the researcher to examine how principals and teachers were affected by tenets of the No Child Left Behind Act and high-stakes accountability (Creswell, 2007). For this research study, four single cases (i.e., individual subjects) as well as four focus groups (containing 5-7 participants) were selected to “capture multiple realities” (Hancock & Algozzine, 2006, p. 72) and open-ended, emerging data (Creswell, 2003). Through data analysis, three themes emerged: 1) Changing Culture, with a subtheme of Collaboration; and 2) Changing Evidence with subthemes of Data-Driven Input and Purposeful Goals; and 3) Increased Rigor with subthemes of Aggressive Pace and Performance and Individualized Instruction. These themes provide an understanding of the impact high-stakes accountability and data-driven decision making has had on public school principals and educators.

The main purpose of this study was to conduct a validation analysis of student surveys of teaching effectiveness implemented at Bangkok University, Thailand. This study included three phases; survey development, a pilot study, and a full implementation study. Four sources of validity evidence were collected to support intended interpretations and uses of survey scores. To this end, this study evaluated the extent to which the content evidence supported the construct definition of the survey (RQ1), the relationships among survey items and survey components corresponded to the construct dimension (RQ2), the survey exhibited gender differential item functioning (RQ3), and student ratings and a similar measure of teaching quality and student achievement (RQ4) were related.

Overall, the student survey demonstrated good psychometric quality and the intended purposes and uses of the survey were supported. Based on expert reviews, the dimensions and survey items were perceived adequate in covering teaching quality, the survey items were perceived to properly assess the associated dimensions, and the response scales were perceived suitable with what was intended to measure. Exploratory factor analysis suggested that the construct of teaching effectiveness as defined in this survey may be unidimensional. Although the results did not support multidimensionality, the dimensions can still be used by individual instructors to evaluate their own teaching. Cronbach’s α coefficients were high and supported the internal consistency of the survey. There was no occurrence of gender DIF in this student survey. Therefore, the validity evidence of survey score interpretations was supported since the meaning of survey categories/scales was shared across male and female students. Finally, the results based on relation to other variables showed a strong positive relationship between the student survey and another currently used survey at Bangkok University which was used to evaluate teaching effectiveness for a decade. This could indicate that the student survey was measuring a similar construct of teaching effectiveness.

This research study compared the three commonly used norm-referenced assessments (Northwest Evaluation Assessment, STAR Enterprise, and AIMSweb) to the Minnesota Comprehensive Assessment. The basic question was which one of the three assessments provided the best predictive validity scores to the Minnesota Comprehensive Assessment. Yearly scores from three years were gathered to evaluate which one of the three assessments had a stronger correlation score to the MCA. The study was confined to using 4^th grade scores from three different schools located on a Native American reservation. Each school used one of the three common standardized reference assessments, and each school administered the MCA in the spring using winter scores. These scores were used to evaluate whether a student is on track to reach proficiency on the MCA. Findings showed that two of the three assessments had strong correlation scores. NWEA-MAP and STAR Enterprise had the strongest correlation. Further findings showed that STAR Enterprise had the strongest correlation score with a caveat that this is a new assessment and needs more research. Findings from this study allow schools to use two of the assessments with confidence that it is giving them quality scores.