Dissertations / Theses on the topic 'Educational assessment and evaluation'

General education assessment practices and data use were examined among regionally accredited, four-year (baccalaureate degree-granting), private, not-for-profit colleges and universities. An online survey instrument was created, piloted, and sent to 1044 institutions resulting in a 45% response rate. Results indicate general education assessment data most frequently influence changes to the assessment process and course content. Challenges preventing data use include lack of time and campus cultures. Assessment data use varies by the regional accreditor in which an institution is a member. Regular class assignments (embedded assessments) and capstone assignments were the most frequently used assessment methods for which collected data are used. Interviews, portfolios, and locally created tests appear to yield more usable data.

Corporations seek Master of Business Administration (MBA) students who are ready to perform upon hiring. Business schools need to align instructional practices and technology with student, accreditation, and marketplace demands. Complex simulation use has increased exponentially to provide MBA students with business experience in the classroom. Methods to assess the effectiveness of complex simulations to achieve learning outcomes is limited to student perceptions of learning, satisfaction, and direct assessment separately. The purpose of this quantitative ex post facto comparative study was to examine MBA students’ perception of learning to real performance in integrative courses with complex simulation. Archival MBA student Peregrine COMP™ pretest, posttest, and SIRII™ scores were analyzed using independent t-test, paired sample t-test, and Pearson r coefficient. MBA students perceived higher levels of learning in courses with complex simulation based on the statistically significant increase in SIRII™ scores over courses without simulation. Another key finding from the quantitative study was the statistically significant negative correlation of students’ perception of learning to actual performance. Positive student perceptions of learning could hide a complex simulation’s inability to meet student learning outcomes, according to the statistically significant decrease between pretest, and posttest Peregrine COMP™ scores. Based on the quantitative correlation analysis of student perceptions of learning to actual performance, MBA administrators and faculty need to evaluate the use of instructional technology from multiple data points to avoid applications that offer minimal value to achieving learning outcomes. Future research opportunities could include a larger MBA population from multiple regions of the United States. Additional studies could investigate undergraduate perceptions of learning to actual performance to assess any benefit from complex simulations.

The Student Growth Objectives are assessments created locally or by commercial educational organizations. The students’ scores from the Student Growth Objectives are included in teacher summative evaluation as one of the measures of teacher’s effectiveness. The Danielson Model for Teaching and Learning supports the idea that assessment literate teachers are highly effective. The purpose of this quantitative causal-comparative study was to identify New Jersey teachers’ competence in student educational assessments. Prizovskaya measured teachers’ assessment literacy level between different groups based on subject taught, years of experience, school assignment and educational degree attained. The data collection occurred via e-mail. Seven hundred ninety eight teachers received an Assessment Literacy Inventory survey developed by Mertler and Campbell. Eighty-two teachers fully completed the survey (N=82). The inferential analysis included an independent-sample t test, One-Way Analyses of Variances test, a post hoc, Tukey test and Welch and Brown-Forsythe tests. The results of this study indicated teachers’ overall score of 51% on entire instrument. The highest overall score of 61% was for Standard 1, Choosing Appropriate Assessment Methods (M = 0.61, SD = 0.23). The lowest overall score of 39% (M = 0.39 and SD = 0.24) was for Standard 2, Developing Appropriate Assessment Methods. The conclusion of this study was that New Jersey teachers demonstrated a low level of competence in student educational assessments. In general, the teacher assessment literacy did not improve during the last two decades. Keywords: assessment literacy, teacher evaluation, student assessment, effective teaching

In order to evaluate the teacher education program for the state of Missouri, the researcher investigated the piloted MoPTA at a private Midwestern university. Through evaluating the piloted MoPTA program, this study aimed to address possible changes needed within the teacher education program itself to better prepare future educators not only for the assessment, but more importantly, the classroom. In order to evaluate the program, the researcher observed scoring sessions for Tasks 1-4 and analyzed feedback from the evaluators of the tasks (university supervisors and faculty) in the fall of 2014 and the spring of 2015. Furthermore, this study examined the scores received from each task during the piloted school year (fall of 2014 and spring of 2015). The researcher analyzed the scores for the following comparisons: Tasks 1-4 (i.e. Task 1 overall scores to Task 2 overall scores); undergraduate students with graduate students’ scores; inter-rater reliability (comparing the scores of multiple raters for one student); and K-12, elementary, middle, and secondary teacher candidates. By completing quantitative analyses of the comparisons through examining approximately 276 teacher candidates’ scores, the researcher ascertained: student performance on Tasks 2 and 3 was significantly lower than Task 4; undergraduate students performed lower on Tasks 1 and 3 in comparison to graduate students; inter-rater reliability had a low correlation for Tasks 1, 3, and 4, but Task 2 reported a high correlation; and there were no differences between elementary teacher candidates and secondary/K-12 teacher candidates. Through analysis of qualitative data the researcher ascertained that the university supervisors and faculty found the scoring sessions for MoPTA helpful and that changes to the university’s curriculum were necessary to better prepare teacher candidates. The researcher suggests adding more data analysis, critical thinking, and writing courses at the university would better prepare teacher candidates; and ongoing scoring sessions and further professional development regarding changes in MoPTA and inter-rater reliability would benefit teacher candidates and consistency among university supervisors and faculty.

Community colleges across the nation are under increasing pressure to find ways to improve the rate of which students, placed in remediation, complete college-level coursework. The attrition of students placed into the lowest levels of developmental mathematics has been a challenge for many colleges to overcome. Research has well recorded the lack of progress of students placed three to four levels below a transfer-level course. Yet, few studies have compared the outcome of similar students in accelerated programs designed to shorten the pathways through remediation. This study focused on students placed in the lowest levels of remediation at two colleges offering consecutive sequences of course-redesign and compression models of acceleration. Using multivariate analyses, the comparative effect on completion rates of students accelerated through two different developmental math acceleration programs from two different colleges within a four year period (2013–2017) were examined. Moreover, this study used student background characteristics, math placement and math acceleration model to predict developmental and college level math course completion using logistic regression analysis.

The results of this study suggest students placed in developmental mathematics who are in an accelerated pathway have decreased time to complete remediation and a transfer-level math course. Findings indicate course-redesign acceleration model yielded more statistically significant improvements in transfer-level math and developmental math completion rates for first-generation students, as well as students placed in both low-level and mid-level remediation. The compression model of acceleration showed significant improvement in completion rates for students placed in mid-level remediation yet results were mixed for students placed in low-level remediation. Students in consecutive acceleration courses were most likely to complete a transfer-level math course, and historically underrepresented minority students were more likely to complete remediation, under certain circumstances, in the compression acceleration model.

These findings inform the college administrators on the potential of sequential accelerated programs. The implications of these results contribute to redesigning academic programs and support current developmental policy reforms. Community colleges are encouraged to consider the recommendations in this study, such as integrating course redesign in California Assembly Bill 705 and California Community College Guided Pathways, to help non-traditional students who are most often placed into the lowest levels of remediation.

The purpose of this study was to determine if there was a correlation between the implementation of standards-based grading (SBG) and a rise in student achievement as indicated by the evaluation of end-of-course exam data from the four core subject areas in secondary schools—English, Algebra, Government, and Biology. This mixed methods study focused on the collected data of 6,000 test scores, split evenly from tests taken prior to the launch of SBG and after the launch of SBG, as well as focusing on varying perceptions of SBG from both teachers and members of the community as a whole. Quantitative data consisted of test scores aggregated by the Missouri Department of Elementary and Secondary Education (MODESE) and distributed to school districts. Qualitative data were secondary in nature and taken from two separate surveys administered by the Cooperating School District to teachers and parents concerning how they felt about the implementation of SBG. These data were aggregated and analyzed by using coding techniques for qualitative data to determine the study outcomes. The quantitative data indicated that there was no statistically significant increase in test scores over the five years researched for this study. The qualitative data indicated that perceptions of SBG were frequently negative in nature, although there were varying degrees of negativity. This result came from both the teacher and parent responses. According to these qualitative data, parents and teachers alike viewed SBG as diminishing the foundation of the education the students were receiving, while at the same time inadequately preparing them for the post-secondary world in that too many chances were given for them to succeed under SBG.

In 2002, the Accrediting Commission of Community and Junior Colleges (ACCJC) revised its accreditation standards and mandated institutions to implement assessment of student learning outcomes (SLO) for all courses and programs. Effective assessment of SLO provides a mechanism for faculty to analyze, discuss, and use data to improve instruction. This process has been integral to meeting and maintaining standards required for accreditation. However, assessment should instead be aimed at improving teaching and learning and providing instructional consistency that results in a better experience for students.

Data from this qualitative study indicated key components to consider in implementing SLO assessment. The study examined faculty perceptions of a single department’s process and provides leaders with a road map to consider for implementation of SLO assessment. The study used a qualitative, single-site case study design to address the research questions through collection of data via in-depth interviews with 11 of 13 full-time faculty members in the target department who had participated in the full assessment cycle, observation of faculty meetings, and document review.

Key factors identified by interviewees were communication, knowledge of SLO, a clear plan, training, expertise, staff to assist faculty, and time to conduct assessment and analysis. Elements that were not evident in the literature emerged and indicated that department culture and faculty characteristics should be considered when creating an implementation plan.

The main obstacle to SLO assessment was the time required for comprehensive and high-quality assessment. There was a clear disconnect between tasks, required time, and institutional deadlines. Other campus-wide barriers cited were a lack of communication from campus leadership, inadequate training, and the perception that the college did not support necessary clerical and professional staff to assist faculty with the effort.

The resources and policies that were reported to assist faculty include a faculty-driven effort, an investment in the process to include compensation for the time spent, clerical and professional staff, technology to simplify the process and an examination of faculty workload.

The purpose of this study was to compare the traditional grading and feedback systems used in most classrooms to a competency-based grading and feedback system. The traditional system used the familiar grading system of A, B, C, D and F applied to assignments. The competency model was based on providing students formative and summative feedback regarding their achievement toward proficiency of specific skills and concepts.

This quasi-experimental action research study had a control group and an intervention group comprised of general education and special education sixth grade students in language arts classes. Quantitative data in the form of student achievement scores and student survey responses was analyzed. Qualitative data in the form of teacher interview responses was analyzed.

Overall there was no statistically significant change in the MAP reading scores between the control and intervention group. However, when the variables of time, group and gender from an ANOVA were analyzed, the males in the intervention group showed a statically significant increase in achievement. This achievement was regardless if the male was an IEP or non-IEP student. Overall, the results do not show that either the control or intervention group sees the feedback as effective. However, the results of the girls’ responses in the control group were statistically significant. The girls in the control group did see the provided feedback as effective. The teachers’ interview responses provided three main themes, which included students applied feedback more in the competency-based classroom than in the traditional classroom. Both teachers used the feedback to adjust curriculum and instruction. Additionally, the intervention teacher pointed out that more time is needed to help students apply the competency-based system.

Based on this study, the first recommendation is to implement competency-based grading and feedback processes. The second recommendation is that formative and summative feedback processes based on proficiencies be implemented to assists students in identifying their understanding of and performance on skills and concepts. Recommendations for future studies include having a larger sample size and continuing the study for a longer period of time.

This dissertation is a mixed-method study that includes research on how school administrators perceive the effectiveness of their district walk-through instrument and their walk-through process as they pertain to helping teachers improve in instruction and prepare for their formal observations. The Louisiana Department of Education's desire for an increase in student achievement and classroom instruction led to the phasing in of the Common Core State Standards and Louisiana's Comprehensive Performance Management System (Compass) in the 2012-13 school year. Louisiana, just like Tennessee, Texas, Florida, and other states, chose to use a value-added assessment system to evaluate teachers and measure student progress. Louisiana's Compass, a value-added model, is developed to use student achievement data to make decisions about student learning, school status, and teacher effectiveness. In order for teachers to meet the expectations of Compass, they need immediate support from school leadership. One support strategy to get teachers prepared for their Compass Evaluations is for principals to frequently conduct walk-through observations. “According to the Center of Comprehensive School Reform and Improvement, a classroom walk-through is defined as a brief, structured, non-evaluative classroom observation by a principal that is followed by a conversation between the principal and the teacher about what was observed” (Moersch, 2013, p. 3). Value-Added Measure (VAM) is accompanied by great accolades as well as harsh criticism. Whether researchers are in favor or against VAM, 234 stronger accountability is here to stay, and school administrators must help teachers to be successful in the classroom, thus be successful on their evaluations.

Current efforts in higher education institutions to increase persistence and success among Hispanic students continue to be ineffective and thus new conceptual frameworks need to be explored. Data from the Hispanic Association of Colleges and Universities asserts that increasing the number of Hispanics that graduate is vital for our country’s future. In turn, Hispanic-Serving Institutions need to nourish and nurture their students to ensure that they graduate and institutional frameworks would benefit from cultural and epistemological congruence with Hispanic students, their families, and their communities.

Educational leaders have urged educators to take on the responsibility and commitment to students’ success and to have a positive impact on the communities they serve. This quasi-experimental study intends to measure the impact of a Hispanic-Serving Institution’s social responsibility on underrepresented students’ institution affiliation, especially Hispanic students in South Texas.

The following research questions guided this study: 1) What types of perceptual and behavioral characteristics (e.g. social integration, academic integration, perceived campus climate, CESL enrollment status, service learning enrollment status, language proficiency, gender, and immigration status) are associated with sense of belonging for college students, especially Hispanic students at a HSI in South Texas? and 2) How do community-engaged scholarship and learning experiences encompassed in CESL courses (the treatment) impact college students’ sense of belonging and academic and social integration, especially Hispanic students at a HSI in South Texas?

In order to answer the two research questions, a quasi-experimental research design was used in this investigation. It involved two forms of analyses: Regression Analysis addressing question one and Multivariate Analysis of Variance (MANOVA) addressing question two. The Multiple Regression Analysis (N = 208) yielded significant findings (p < .05). The full model revealed that 48% of the variance in Sense of Belonging, the dependent variable, was explained by four predictor variables: Peer Group Interaction; Faculty Concern for Student Development and Teaching; Academic and Intellectual Development; and English Proficiency. Although there were no differences (p > .05) detected among the comparison groups, recommendations to improve research design, methodology and treatment fidelity for future studies were provided.

The purpose of the current study was to test the theory of action hypothesized for the Mathematics Formative Assessment System (MFAS) based on results from a large-scale randomized field trial. Using a multilevel structural equation modeling analytic approach with multiple latent response variables decomposed across student, teacher, and school levels of clustering, the current study found evidence of effects of MFAS that were consistent with the MFAS theory of action. First, assignment to the treatment condition was associated with higher mean student mathematics performance and a higher prevalence of small group instruction compared to schools assigned to the control condition—both of which are outcomes hypothesized to result from MFAS use. Also, a positive association between teacher-level mathematics knowledge for teaching and student mathematics performance was found in the current study, which is consistent with the interrelation of constructs specified in the MFAS theory of action. However, evidence of the particular linkages of MFAS use→teacher knowledge→classroom practice→student mathematics performance and the putative cascade of effects that would substantiate the mechanisms of change posited in the MFAS theory of action were not detected in the current study. Thus, positive effects of MFAS on teacher and student outcomes were substantiated; however, as to how the effects of MFAS on teachers transfer to improved outcomes for students remains to be empirically demonstrated. Based on my review of the results from the current study and consideration of the literature on formative assessment as it relates to the design of MFAS tasks and rubrics, I discuss a proposed modification to the theory of action that specifies the addition of a direct path from MFAS use to student mathematics performance, in addition to the indirect path currently specified.

Alternative high schools serve some of the most vulnerable students and their programs present a significant challenge to evaluate. Determining the impact of an alternative high school that serves mostly at-risk students presented a significant research problem. Few studies exist that dig deeper into the characteristics and strategies of successful alternative schooling. Moreover valid program evaluation methods to identify successful alternative school practices are hit and miss. As a result, public policy and systems of accountability have either disregarded information relating to alternative high schools or unjustifiably included them in comparisons with traditional high schools.

This dissertation studied the issue of how best to evaluate alternative high schools and what tools support leaders in planning a thorough and accurate program evaluation. The Alternative High School Program Evaluation Toolkit was developed to support school leaders and evaluation teams made up of internal and external stakeholders as they facilitate the program evaluation process. The features of the Toolkit address the need for alternative school evaluation to be practical, useful, fair and accurate. The Evaluation Toolkit includes training materials, protocols, an evaluation planning worksheet and an evaluation planning matrix that supports the team in conducting the evaluation.

The research represented in this dissertation is theoretically and practically grounded in Bridges and Hallinger's (1995) Problem-Based Learning (PBL) and Borg and Gall's (1989) Research and Development (R&D) Cycle. The product of the R&D Cycle was the Alternative High School Program Evaluation Toolkit and a process for use by evaluation teams assigned the task of planning and carrying out program evaluations.