Статті в журналах з теми "Consistency of expert judgments"
Щоб переглянути інші типи публікацій з цієї теми, перейдіть за посиланням: Consistency of expert judgments.
Оформте джерело за APA, MLA, Chicago, Harvard та іншими стилями
Ознайомтеся з топ-50 статей у журналах для дослідження на тему "Consistency of expert judgments".
Біля кожної праці в переліку літератури доступна кнопка «Додати до бібліографії». Скористайтеся нею – і ми автоматично оформимо бібліографічне посилання на обрану працю в потрібному вам стилі цитування: APA, MLA, «Гарвард», «Чикаго», «Ванкувер» тощо.
Також ви можете завантажити повний текст наукової публікації у форматі «.pdf» та прочитати онлайн анотацію до роботи, якщо відповідні параметри наявні в метаданих.
Переглядайте статті в журналах для різних дисциплін та оформлюйте правильно вашу бібліографію.
1
Aarts, Rembrant, Lennard Van Wanrooij, Evert Bloemen, and Geert Smid. "Expert medico-legal reports: The relationship between levels of consistency and judicial outcomes in asylum seekers in the Netherlands." Torture Journal 29, no. 1 (May 22, 2019): 36–46. http://dx.doi.org/10.7146/torture.v29i1.111205.
Повний текст джерелаАнотація:
Introduction: If asylum applicants need to prove that they have been persecuted in their home country, expert judgment of the psychological and physical consequences of torture may support the judicial process. Expert medico-legal reports can be used to assess whether the medical complaints of the asylum seeker are consistent with their asylum account. It is unclear which factors influence medical expert judgement about the consistency between an asylum seeker’s symptoms and story, and to what extent expert medico-legal reports are associated with judicial outcomes. Methods: We analysed 97 medico-legal reports on traumatised asylum seekers in the Netherlands. First, we evaluated the impact of trauma-related and other variables on experts’ judgments of the consistency of symptoms and story. Second, we evaluated the effect of experts’ judgments of symptom-story consistency on subsequent judicial outcomes. Results: Gender, receipt of mental health care and trauma-related variables were associated with symptomstory consistency. Positive asylum decisions were predicted by expert judgments about the presence of physical signs and symptoms of torture, and ill-treatment and their consistency with the refugee’s story, but not psychological symptoms. Conclusion: These results suggest that standardised procedures for the documenting of medical evidence by independent experts can improve judicial decision quality and the need to improve psychological and psychiatric assessments.
2
Kurennykh, A. E., V. A. Sudakov, and V. P. Osipov. "The Usage of Web Services for Consistency Improvement in Pairwise Comparison Matrixes." Моделирование и анализ данных 09, no. 4 (2019): 80–87. http://dx.doi.org/10.17759/mda.2019090406.
Повний текст джерелаАнотація:
Paired comparisons of criteria and alternatives are widely used in a large number of technical and scientific problems of the present, in which it is necessary to rank a finite set of objects or to evaluate an object. Paired comparisons are understandable and simple for an expert, they are a high-quality and reliable way of rating, however, it is known that the complexity and dimension of the criteria space in many problems leads to a high load on the expert, as a result of which incorrect or erroneous situations may arise when compiling matrixes of paired comparisons leading to a decrease in the coherence of judgments, and, as a consequence, to the adoption of irrational decisions. Algorithmic software to increase the consistency of judgments is in demand among experts and researchers, which, together with a large number of diverse tasks, creates requirements for the development of appropriate software: the ability to access a large number of users and independence from the subject area, which are highly satisfied by the web interface. In this paper, the authors describe an effective method for increasing the consistency of judgments in matrixes of pairwise comparisons. The main objective of the method is to maximize the consistency of judgments with a minimum of changes made to the matrix proposed by the expert as initial estimates. As a quantitative measure of consistency of judgments, a classic indicator is used – the index of consistency. Based on the created algorithm, the authors developed software that is available to researchers in distributed web services to support decision-making ws-dss.com.
3
Ettenson, Richard, James Shanteau, and Jack Krogstad. "Expert Judgment: Is More Information Better?" Psychological Reports 60, no. 1 (February 1987): 227–38. http://dx.doi.org/10.2466/pr0.1987.60.1.227.
Повний текст джерелаАнотація:
Two groups of professional auditors (expert ns = 10 and 11) and one group of 11 accounting students (novices) made judgments for 32 hypothetical auditing cases which were based on 8 dimensions of accounting-related information. Analyses indicated that the experts did not differ significantly from the novices in the number of significant dimensions; both the professionals and the students had roughly three significant factors. When evaluating the information, however, the experts' judgments primarily reflected one source of information, with other cues having secondary impart. In comparison, no single cue was dominant for the students' judgments. These results were interpreted to indicate that the nonuse of information by experts does not necessarily indicate a cognitive limitation. Instead, experts may have better abilities to focus on relevant information. The professional auditors also exhibited greater consistency and consensus than did the students. In contrast to much previous work, the experts here are viewed as being skilled and competent judges.
4
Tutygin, A. G., V. B. Korobov та T. V. Menshikova. "Проблемы согласованности экспертных суждений в методе анализа иерархий". Вестник гражданских инженеров 16, № 5 (2019): 291–97. http://dx.doi.org/10.23968/1999-5571-2019-16-5-291-297.
Повний текст джерела
5
Vicente, E., A. Mateos, and A. Jiménez-Martín. "A Betting- and Lottery-Based Method for Fuzzy Probability Elicitation." International Journal of Uncertainty, Fuzziness and Knowledge-Based Systems 27, no. 01 (February 2019): 121–44. http://dx.doi.org/10.1142/s0218488519500065.
Повний текст джерелаАнотація:
It is very common to use linguistic term scales, whose terms are previously associated with different fuzzy numbers to assign probabilities to events in decision-making processes. However, the rigidity of such scales generates bias in the probability elicitation process and does not allow experts to adequately express their probabilistic judgments. We propose a betting and lottery-based method for eliciting a fuzzy number from the expert that represents his/her probabilistic judgments for a given event, along with a quality measure of the probabilistic judgments based on precision and consistency measures. We also enact a simulation process to analyze possible biases in the proposed fuzzy probability elicitation method.
6
Stepanyuk, O. I., and V. P. Novosad. "Verification of final ranking which received because of the expert evaluation." Scientific Messenger of LNU of Veterinary Medicine and Biotechnologies 21, no. 93 (November 16, 2019): 96–101. http://dx.doi.org/10.32718/nvlvet-e9319.
Повний текст джерелаАнотація:
In the modern dynamic information society, situations that are characterized by complexity, multifactoriality, uncertainty, and non-formalization occur very often. Under such conditions, the adoption of effective management decisions is impossible without interaction with the expert environment. This applies to a wide range of human activities, including economic issues. The article is devoted to one of the key stages of expert evaluation – the analysis of members of expert commissions’ judgments and drawing up final conclusions. The author aims to get closer to solving one of the key problems characteristic to this stage, namely: truth verification problems, reliability confirmation for final conclusions obtained as a result of generalization of expert groups’ work. Most often, the experts are asked to compare certain objects according to certain criteria, to identify the best and the worst among them. It is clear that expert judgments are rarely unanimous. Therefore, it is important to apply scientific approaches to constructing final ranking, that is, the final location of objects in order of increase or decrease of a certain quality or usability. The mathematical modeling of activity of an expert group conducted by the author of the article made it possible to identify cases where, depending on the applied techniques, the final rankings obtained on the basis of the judgments of the same experts may differ (sometimes even substantially). This again emphasizes the need to verify the final rankings. The advantages and disadvantages of applying quantitative and qualitative methods in expert-analytical activity are also considered. The scientific novelty of the conducted research is to improve the methodological approaches that will allow to improve the consistency of expert judgments, and to develop recommendations for the complex application of separate methods for constructing total ranking. The article is intended for scientists and practitioners who are interested in contemporary sociological and management approaches to solving economic problems. Also, the article may be useful for specialists and programmers who work in the field of development of artificial intelligence.
7
Benítez, J., X. Delgado-Galván, J. A. Gutiérrez, and J. Izquierdo. "Balancing consistency and expert judgment in AHP." Mathematical and Computer Modelling 54, no. 7-8 (October 2011): 1785–90. http://dx.doi.org/10.1016/j.mcm.2010.12.023.
Повний текст джерела
8
Kochetova-Kozloski, Natalia, and William F. Messier. "Strategic Analysis and Auditor Risk Judgments." AUDITING: A Journal of Practice & Theory 30, no. 4 (November 1, 2011): 149–71. http://dx.doi.org/10.2308/ajpt-10147.
Повний текст джерелаАнотація:
SUMMARY The study investigates whether and how senior auditors' strategic analysis of a client affects their identification of significant business and financial statement risks, and their risk assessments. Sixty-seven senior auditors participated in an experiment that examined the effect of analyzing two aspects of strategic analysis (strategic positioning and the strategy implementation process) against performing no strategic analysis. An expert panel of senior managers was used to develop a benchmark for comparison purposes. Our results show that (1) auditors who performed guided strategic analysis did not identify more significant business and financial statement risks than auditors who did not perform strategic analysis, (2) senior auditors who performed strategic analysis of strategic positioning or the strategy implementation process assessed risk of material misstatement at the entity level more consistently with an expert panel than auditors who did not perform such an analysis, and (3) senior auditors' analysis of the client's strategy implementation process was associated with assessments of the strength of the control environment that were more consistent with the expert panel than assessments done by auditors who did not perform any strategic analysis or who performed only an analysis of strategic positioning. Data Availability: Contact the first author.
9
Pankratova, N. D., and L. Y. Malafeeva. "Formalizing the consistency of experts’ judgments in the Delphi method." Cybernetics and Systems Analysis 48, no. 5 (September 2012): 711–21. http://dx.doi.org/10.1007/s10559-012-9451-6.
Повний текст джерела
10
Plake, Barbara S., and Gerald J. Melican. "Effects of Item Context on Intrajudge Consistency of Expert Judgments via the Nedelsky Standard Setting Method." Educational and Psychological Measurement 49, no. 1 (March 1989): 45–51. http://dx.doi.org/10.1177/0013164489491005.
Повний текст джерела
11
Ulfah, Aini Azkiyatu, Kartono Kartono, and Endang Susilaningsih. "Validity of Content and Reliability of Inter-Rater Instruments Assessing Ability of Problem Solving." Journal of Educational Research and Evaluation 9, no. 1 (March 31, 2020): 1–7. http://dx.doi.org/10.15294/jere.v9i1.40423.
Повний текст джерелаАнотація:
In general, problems that occur in the actual are the discovery of instruments that have not been tested for problem solving abilities. The aim of this research is to reveal the content validity and interrater reliability of the instrument for evaluating the problem solving abilities that have been prepared. The research method is used a quantitative description by 3 expert judgments, they are experts in research and evaluation, mathematics education experts, and mathematics teachers. The instrument was developed in the form of an expert observation sheet with 3 aspects of assessment, they are the aspect of content eligibility, construction aspects, and language aspects, which of each aspect has 4 categories, which are very relevant, relevant, quite relevant, and highly irrelevant. Data were analyzed using the Aiken's V formula to determine the level of instrument validity and to determine the level of consistency / constancy between assessors using Instraclass Correlation Coefficient (ICC) analysis with the help of SPSS version 23.0. the results of analysis of the content validity of all items valued above 0.3 which means that all aspects assessed by experts are valid. Interrater reliability test using ICC obtained a value of 0.516, which means that all aspects of the instrument for evaluating problem solving abilities that have been rated have a level of consistency. Thus, the instrument of problem-solving ability that has been tested for validity and reliability can be used by educators to determine the level of students' problem solving abilities appropriately.
12
PutrI, Indah Cantika, Damri Damri, Engkizar Engkizar, Zainal Asril, and Efendi Efendi. "The Use of Android Game to Improve Impaired Hearing Student Vocabulary Mastery." Journal of Educational Research and Evaluation 9, no. 2 (August 23, 2020): 85–93. http://dx.doi.org/10.15294/jere.v9i2.44744.
Повний текст джерелаАнотація:
In general, problems that occur in the actual are the discovery of instruments that have not been tested for problem solving abilities. The aim of this research is to reveal the content validity and interrater reliability of the instrument for evaluating the problem solving abilities that have been prepared. The research method is used a quantitative description by 3 expert judgments, they are experts in research and evaluation, mathematics education experts, and mathematics teachers. The instrument was developed in the form of an expert observation sheet with 3 aspects of assessment, they are the aspect of content eligibility, construction aspects, and language aspects, which of each aspect has 4 categories, which are very relevant, relevant, quite relevant, and highly irrelevant. Data were analyzed using the Aiken's V formula to determine the level of instrument validity and to determine the level of consistency / constancy between assessors using Instraclass Correlation Coefficient (ICC) analysis with the help of SPSS version 23.0. the results of analysis of the content validity of all items valued above 0.3 which means that all aspects assessed by experts are valid. Interrater reliability test using ICC obtained a value of 0.516, which means that all aspects of the instrument for evaluating problem solving abilities that have been rated have a level of consistency. Thus, the instrument of problem-solving ability that has been tested for validity and reliability can be used by educators to determine the level of students' problem solving abilities appropriately
13
Unkelbach, Christian, and Daniel Memmert. "Game Management, Context Effects, and Calibration: The Case of Yellow Cards in Soccer." Journal of Sport and Exercise Psychology 30, no. 1 (February 2008): 95–109. http://dx.doi.org/10.1123/jsep.30.1.95.
Повний текст джерелаАнотація:
Referees in German first-league soccer games do not award as many yellow cards in the beginning of a game as should be statistically expected. One explanation for this effect is the concept of game management (Mascarenhas, Collins, & Mortimer, 2002). Alternatively, the consistency model (Haubensak, 1992) explains the effect as a necessity of the judgment situation: Referees need to calibrate a judgment scale, and, to preserve degrees of freedom in that scale, they need to avoid extreme category judgments in the beginning (i.e., yellow cards). Experiment 1 shows that referees who judge scenes in the context of a game award fewer yellow cards than referees who see the same scenes in random order. Experiment 2 shows the combined influence of game management (by explicitly providing information about the game situation) and calibration (early vs. late scenes in the time course of a game). Theoretical implications for expert refereeing and referee training are discussed.
14
Chen, Dingjun, Shaoquan Ni, Chang’an Xu, Hongxia Lv, and Keyun Qin. "A Soft Rough-Fuzzy Preference Set-Based Evaluation Method for High-Speed Train Operation Diagrams." Mathematical Problems in Engineering 2016 (2016): 1–8. http://dx.doi.org/10.1155/2016/5795604.
Повний текст джерелаАнотація:
This paper proposes a method of high-speed railway train operation diagram evaluation based on preferences of locomotive operation, track maintenance, S & C, vehicles and other railway departments, and customer preferences. The application of rough set-based attribute reduction obtains the important relative indicators by eliminating excessive and redundant evaluation indicators. Soft fuzzy set theory is introduced for the overall evaluation of train operation diagrams. Each expert utilizes a set of indicators during evaluation based on personal preference. In addition, soft fuzzy set theory is applied to integrate the information obtained via expert evaluation in order to obtain an overall evaluation. The proposed method was validated by a case study. Results demonstrate that the proposed method flexibly expresses the subjective judgments of experts while effectively and reasonably handling the uncertainty of information, which is consistent with the judgment process of humans. The proposed method is also applicable to the evaluation of train operation schemes which consist of multiple diagrams.
15
Asakir, Ibnu, and Dian Hidayati. "Rasch Model Analysis: Teacher Commitment Indicators by Experts Judgment." International Journal of Educational Management and Innovation 3, no. 1 (January 31, 2022): 59–73. http://dx.doi.org/10.12928/ijemi.v3i1.5501.
Повний текст джерелаАнотація:
Teacher commitment is the key to success in educational institutions. The school needs to measure the teacher's commitment to improving the quality of the school. The purpose of this study was to model expert judgment in the reliabilities and validity tests of Teacher commitment instruments using the Rasch model. The study involving 12 experts was conducted using a survey by assessing 18 items. Experts (respondents) are asked to evaluate the consistency of each item to represent one part of the Teacher commitment instrument. Expert judgment results using Rasch model analysis show that the average value of the logit scale is higher than logit 0.0. It is understood that more answering experts agree across a variety of items. The value Cronbach alpha, measuring reliability, i.e., the interaction between the person and the item as a whole, is seen as a value of 0.99, which means excellent. Person reliability value is estimated at 0.89, and item reliability is estimated at 0.60, indicating that the consistency of respondents' answers is good. However, the quality of items in the infrastructure is poor. It shows 17 items recommended being used as instruments to measureTeacher commitment, while 11 items are advised to be repaired.
16
Tolcheev, Vladimir O. "Expert survey and analysis of the results." Industrial laboratory. Diagnostics of materials 85, no. 7 (August 11, 2019): 73–82. http://dx.doi.org/10.26896/1028-6861-2019-85-7-73-82.
Повний текст джерелаАнотація:
The issues of organizing an expert survey and carrying out statistical processing and analysis of the results are considered. The experts are the fifth-year students undergoing training at the Department of Management and Informatics «Moscow Power Engineering Institute» of the National Research University. The goal of the survey is revealing the disciplines that are most useful for employment in their specialty. We discuss the special features of the survey and a concept of «work in the specialty», with due regard for statistical reliability of the results. Data of written questionnaire gained in 2018 were processed and analyzed using cluster analysis (construction of dendrograms and application of the K-means method) and non-parametric statistical criteria (Friedman and Mann – Whitney – Wilcoxon). Data processing is implemented in the program STATISTICA. The analysis is carried out to reveal significant differences between the educational courses and assess the degree of consistency of the respondents to divide them into clusters that unite the students with similar judgments. Data analysis revealed that experts’ estimates in 2018 are in fairly good agreement with the estimates of previous studies; among the respondents there are three coalitions corresponding to the training modules «Software», «Management Theory», «Data Analysis»; the overall consistency of students in the two groups is very low (and, on the contrary, high in the identified clusters); grades are homogeneous and do not depend on training groups (and employment – unemployment of the respondents). The obtained results allow us to address a number of important questions regarding the ways of improving the educational process, e.g., to optimize yearly course hours for different educational modules.
17
Tannenbaum, Richard J., and Priya Kannan. "Consistency of Angoff-Based Standard-Setting Judgments: Are Item Judgments and Passing Scores Replicable Across Different Panels of Experts?" Educational Assessment 20, no. 1 (January 2, 2015): 66–78. http://dx.doi.org/10.1080/10627197.2015.997619.
Повний текст джерела
18
Fesenko, Tetiana, Igor Ruban, Kateryna Karpenko, Galyna Fesenko, Andriy Kovalenko, Anatolii Yakunin, and Hryhorii Fesenko. "Improving of the decision-making model in the processes of external quality assurance of higher education." Eastern-European Journal of Enterprise Technologies 1, no. 3(115) (February 28, 2022): 74–85. http://dx.doi.org/10.15587/1729-4061.2022.253351.
Повний текст джерелаАнотація:
The peculiarities of external quality assurance processes in the higher education management system are considered. It is noted that the quality of educational programs (EP) of higher education institutions is controlled by quality assurance agencies (QAAs) using an accreditation system. The key features of the accreditation process in terms of peer review are identified. The problem of accreditation process management, namely the subjectivity and lack of consistency of expert decisions, is highlighted. The correlation method was applied to determine the interdependencies in expert assessments (competence, meaningful orientation of judgments, and perception of the linguistic rating scale). The identified types of variables make it possible to explain the existing measure of subjectivity that affects the collective conclusion of experts. A comprehensive methodology for quantitative evaluation of the EP quality under conditions of uncertainty based on the relative importance of the relevant criteria and subcriteria, as well as the levels of expert competence using the apparatus of fuzzy mathematics, is proposed. A basic model for the formation of a collegial expert opinion on the EP quality has been developed using the example of a system of quality criteria approved by the Ukrainian QAA. Variations in the expert values of the weight coefficients and parameters of fuzzy numbers in the context of the linguistic rating scale (“A – B – E – F”) made it possible to use the means of a computational experiment. The application of this model will allow managers to positively influence the existing ambiguity of the assessment method, which requires being guided by standard criteria and at the same time determining the EP innovativeness. In general, the application of the proposed evaluation tools on the quality of EP allows experts and managers to make decisions at a higher level of academic and managerial culture.
19
Jung, Seoyoung, Seulki Lee, and Jungho Yu. "Ontological Approach for Automatic Inference of Concrete Crack Cause." Applied Sciences 11, no. 1 (December 29, 2020): 252. http://dx.doi.org/10.3390/app11010252.
Повний текст джерелаАнотація:
The cause of cracks in concrete is traditionally estimated by analyzing information such as patterns and locations of the cracks and whether other defects are present, followed by aggregating the findings to estimate the cause. This method is highly dependent on the expert’s knowledge and experience in the process of identifying the cause of the cracks by compiling information related to the occurrence of the cracks, and it is likely that each expert will make a different diagnosis or an expert with insufficient knowledge and experience will make an inaccurate diagnosis. Therefore, we propose automated technology using the ontology to improve the consistency and accuracy of crack diagnosis results in this research. The proposed approach uses information on the crack patterns, locations, and penetration status, as well as the occurrence of other defects, to automatically infer the causes of cracks. We developed ontology that can infer the cause of cracks using the information on their appearance and applied actual cases of cracks to verify the ontological operation. In addition, the consistency and accuracy of the ontology were validated using eight actual cases of crack. The approach of this study can support expert decision-making in the crack diagnosis process, thereby reducing the possibility of various errors caused by the intervention of inaccurate judgments in the crack diagnosis process and improving the efficiency of the crack diagnosis tasks.
20
Spence, Mark T., and Merrie Brucks. "The Moderating Effects of Problem Characteristics on Experts’ and Novices’ Judgments." Journal of Marketing Research 34, no. 2 (May 1997): 233–47. http://dx.doi.org/10.1177/002224379703400204.
Повний текст джерелаАнотація:
A growing body of literature suggests that experts are little if at all better than novices in terms of the quality of decision outputs. To explain this counter-intuitive finding, the authors propose a conceptual framework that focuses on initial problem structure as a key moderator of the effect of expertise on performance. Specifically, they argue that the expert–novice performance differential should be greatest at moderate levels of problem structure and weakest at both extremes. To examine this central hypothesis, the authors conduct a controlled experiment that compares experts with novices when solving a complex problem that had characteristics of a moderately ill-structured problem. Relative to novices, the authors find that experts select fewer, but more diagnostic, information inputs and are more consistent when evaluating nonquantified inputs. As a result, they make more accurate and tightly clustered judgments than do novices, and also are more confident in their decisions. To examine the moderating influence of problem characteristics, certain task variables are manipulated to increase or decrease initial problem structure. As hypothesized, the benefits of expertise are less pronounced when solving a problem with increased initial structure.
21
Grenier, Jonathan H., D. Jordan Lowe, Andrew Reffett, and Rick C. Warne. "The Effects of Independent Expert Recommendations on Juror Judgments of Auditor Negligence." AUDITING: A Journal of Practice & Theory 34, no. 4 (February 1, 2015): 157–70. http://dx.doi.org/10.2308/ajpt-51064.
Повний текст джерелаАнотація:
SUMMARY Audit firms claim that they are subject to unreasonable litigation risk and that legal reforms are needed. One frequently proposed reform is to utilize independent (i.e., court-appointed) experts to examine case facts and provide recommendations to the court. This study provides theory and evidence to examine the general effects of such recommendations on jurors' judgments and, also, more specifically, to inform critics' concerns that jurors will merely “rubber stamp” independent experts' recommendations. Results of an experiment indicate that independent experts' recommendations shift jurors' judgments in the direction of the recommendation, but that such effects depend on jurors' perceptions of the experts' credibility. Further, consistent with critics' concerns, independent experts' recommendations reduce jurors' sensitivity to specific case facts in some, but not all, contexts. Specifically, when the experts conclude that the auditors were negligent, jurors' negligence judgments are insensitive to variation in specific case facts (the auditors' use versus nonuse of a specialist), but are sensitive to such variation when the experts conclude that the auditors were not negligent. Implications for theory, practice, and regulation are discussed. Data Availability: Available from the authors upon request.
22
Adelman, Leonard, James Gualtieri, and Sharon L. Riedel. "A multifaceted approach to evaluating expert systems." Artificial Intelligence for Engineering Design, Analysis and Manufacturing 8, no. 4 (1994): 289–306. http://dx.doi.org/10.1017/s0890060400000974.
Повний текст джерелаАнотація:
AbstractA multifaceted approach to evaluating expert systems is overviewed. This approach has three facets: a technical facet, for “looking inside the black box”; an empirical facet, for assessing the system’s impact on performance; and a subjective facet, for obtaining users’ judgments about the system. Such an approach is required to test the system against the different types of criteria of interest to sponsors and users and is consistent with evolving lifecycle paradigms. Moreover, such an approach leads to the application of different evaluation methods to answer different types of evaluation questions. Different evaluation methods for each facet are overviewed.
23
Wang, Li Jiu, and Li Li. "Development of a Method in Evaluation and Adjustment for Consistency of Group Decision-Making Based on Grey Relational Analysis and its Application in Evaluation of New Rural Economic Building Materials." Advanced Materials Research 168-170 (December 2010): 1163–68. http://dx.doi.org/10.4028/www.scientific.net/amr.168-170.1163.
Повний текст джерелаАнотація:
In this paper, grey relational analysis (GRA) was used in consistency test of group judgment matrixes. The evaluation indicator weight was obtained through the judgment matrix calculated with expert evaluating method in analytical hierarchy process (AHP). The judgment method and the adjustment process for consistency of group judgment matrixes were studied. First, the consistency indicators were put forward, and then the definitions and the theorems of consistency indicators in the judgment method were defined based on GRA. Then the theorems were given proof. A method using GRA to judge the consistency of group judgment matrixes was proposed for the first time. Second, adjustment modeling was developed to solve the consistency of group judgment matrixes. Finally, the applications of the judgment method and the adjustment process have been illustrated by given example. It is believed that the proposed methods are applicable to test consistency of evaluation indicators of new rural economic building materials.
24
Hartanto, Hartanto, Ani Rusilowati, and Kartono Kartono. "the Developing Assessment Instrument In Critical Thinking Ability For Fifth Grade Of Elementary School In Thematic Learning." Journal of Educational Research and Evaluation 8, no. 2 (August 23, 2019): 123–32. http://dx.doi.org/10.15294/jere.v8i2.36685.
Повний текст джерелаАнотація:
The background of this study is about the availibility of the test assessment instrument to measure the critical thinking ability of fifth grade elementary school students which is limitted. Therefore, it needs to be developed. This development research used the Borg & Gall development method. The purpose of this study was to develop an assessment instrument in essay. The assessment instrument was validated by 5 Experts Judgment by using the Aiken'V formula. The results of trials using the construct validity of the Confirmatory Factor Analysis (CFA), interater reliability used Two Ways Anova proved by the Ebel formula and internal consistent reliability with the Alpha Cronbach formula. The result of expert judgment validation showed that ≥0.8 which mean that 10 items were in valid category, construct validity results from large-scale test analysis by using LISREL 8.8 namely Confirmatory Factor fulfilled the testing of goodness of fit GFI value 0,93, CFI=0,97 and NFI=0,91. The three assessment criteria have value > 0,90. It can be concluded that the construct validity is met. The construct validity is also evidenced by the loading factor of 10 items. All of which have a price of > 0,3. The interrater reliability assessment instruments based on Expert Judgment coefficient value was 0,62. The expert agreement indicated that the rating given by each rater is reliable or consistent between one another and internal consistency of small scale test results of coefficient value of alpha 0,861, large scale 0.813. Practicality of assessment instruments based on expert judgment had mean 37.2 and included in practical category. The profile of critical thinking skills of fifth grade elementary school students category “Mediumâ€. The Conclusions was the assessment instruments in critical thinking skills of fifth grade students on thematic learning were tested in validity, reliability and suitable to be used.
25
Ruan, Chuanyang, and Jianhui Yang. "Software Quality Evaluation Model Based on Weighted Mutation Rate Correction Incompletion G1 Combination Weights." Mathematical Problems in Engineering 2014 (2014): 1–9. http://dx.doi.org/10.1155/2014/541292.
Повний текст джерелаАнотація:
Aiming at the common problems of quality evaluation method, this paper first establishes a fuzzy software quality evaluation model according to the relationship of software quality subcharacteristics and indicators; furthermore, considering the uncertainty and individual deviations of expert judgment results, this paper corrects and tests the consistency of the incomplete information sorting given by the experts and obtains an integration sorting of gathering different expert opinions through the idea of circling modification; at last, this paper proposes the weighted mutation rate which is used to measure the development balance degree and determines weights of evaluation indicators via weighted mutation rate correction incompletion G1 method, which avoids the problem of integration of subjective and objective weights.
26
Kiseleva, E., V. Stepanets, and S. Valkova. "THE INFLUENCE OF THE CONSISTENCY OF EXPERT JUDGMENTS ON THE DECISION TO CHOOSE THE TERMS OF THE CONTRACT OF CARRIAGE BY SEA." Transport Business of Russia, no. 2 (2021): 151–53. http://dx.doi.org/10.52375/20728689_2021_2_151.
Повний текст джерела
27
Böckenholt, Ulf. "Thresholds and Intransitivities in Pairwise Judgments: A Multilevel Analysis." Journal of Educational and Behavioral Statistics 26, no. 3 (September 2001): 269–82. http://dx.doi.org/10.3102/10769986026003269.
Повний текст джерелаАнотація:
The method of paired comparisons belongs to a small class of models that simultaneously allow for consistency checks of the response behavior of a person and the estimation of item parameters. As a result, the paired comparison technique is of much importance when there are reasons to expect that respondents have difficulties appraising their values or preferences about an issue of interest. This article presents a hierarchical framework for the analysis of paired comparison data with three response categories that allow judges to be indifferent or undecided. The approach can be viewed as a stochastic representation of Luce’s (1956) semiorder. It facilitates statistical analyses of different types of inconsistent responses and yields new tests of the underlying judgmental process. An extensive analysis of a survey study illustrates the usefulness of a multilevel approach for modeling multiple trinary judgments.
28
Orugbo, Ena E., Babakalli M. Alkali, Anjali DeSilva, and David K. Harrison. "RCM and AHP hybrid model for road network maintenance prioritization." Baltic Journal of Road and Bridge Engineering 10, no. 2 (June 25, 2015): 182–90. http://dx.doi.org/10.3846/bjrbe.2015.23.
Повний текст джерелаАнотація:
Category1 defects such as potholes significantly accelerate structural deterioration and pose imminent hazards on trunk road networks. Frequent occurrences of category1 defects have increased service disruptions on trunk road networks. Road maintenance agencies are now required to effectively prioritize trunk road network category1 defects maintenance works. However, existing road maintenance prioritization methods such as value engineering and traditional expert judgment methods have limitations. Value engineering is resource and time intensive thus best suited for project level prioritization and traditional expert judgments are subjective and lack audit trails. In an attempt to address the limitations of above methods, this study presents a Reliability Centered Maintenance and Analytical Hierarchy Process based hybrid model for trunk road network maintenance prioritization. The proposed hybrid model is used to establish failure diagnostic and Multi-criteria Decision Making respectively. As a case study, the hybrid model is implemented on a trunk road network in the United Kingdom. Relevant category1 defect failure information linked to trunk road network subassets are extracted from databases and relevant information elicited from maintenance experts. The criticality analysis results presented show the risk priority numbers of category1 defect related failures and cost effective preventative maintenance tasks are proposed. The Analytical Hierarchy Process results are used to address the complex prioritization process. The proposed hybrid model facilitates a systematic prioritization of a large number of trunk road network category1 defect failure maintenance activities consistently.
29
Fabbri, Alexander R., Wojciech Kryściński, Bryan McCann, Caiming Xiong, Richard Socher, and Dragomir Radev. "SummEval: Re-evaluating Summarization Evaluation." Transactions of the Association for Computational Linguistics 9 (2021): 391–409. http://dx.doi.org/10.1162/tacl_a_00373.
Повний текст джерелаАнотація:
Abstract The scarcity of comprehensive up-to-date studies on evaluation metrics for text summarization and the lack of consensus regarding evaluation protocols continue to inhibit progress. We address the existing shortcomings of summarization evaluation methods along five dimensions: 1) we re-evaluate 14 automatic evaluation metrics in a comprehensive and consistent fashion using neural summarization model outputs along with expert and crowd-sourced human annotations; 2) we consistently benchmark 23 recent summarization models using the aforementioned automatic evaluation metrics; 3) we assemble the largest collection of summaries generated by models trained on the CNN/DailyMail news dataset and share it in a unified format; 4) we implement and share a toolkit that provides an extensible and unified API for evaluating summarization models across a broad range of automatic metrics; and 5) we assemble and share the largest and most diverse, in terms of model types, collection of human judgments of model-generated summaries on the CNN/Daily Mail dataset annotated by both expert judges and crowd-source workers. We hope that this work will help promote a more complete evaluation protocol for text summarization as well as advance research in developing evaluation metrics that better correlate with human judgments.
30
Setiawan, Adib Rifqi. "Instrumen Penilaian untuk Pembelajaran Ekologi Berorientasi Literasi Saintifik." Assimilation: Indonesian Journal of Biology Education 2, no. 2 (September 30, 2019): 42. http://dx.doi.org/10.17509/aijbe.v2i2.19250.
Повний текст джерелаАнотація:
The aims of this cross-sectional survey research was to find the validity and reliability of assessment instruments for ecological learning scientific literacy oriented’s. Determination of the sample used purposive sampling of 4 experts and 122 high school level students. To reveal validity is assessed based on obtain judgment expert and reliability measured by internal consistency. It was gained that the validity is 7 items very feasible and 3 item quite feasible with reliability’s value is 0.763. It showed that all items can be used to analyzing the difficulties of students for designing ecological learning scientific literacy oriented’s lesson plans.
31
Kulagina, I. Y., E. V. Apasova, and V. V. Fyodorov. "A Technique for Assessing Learning Motivation in Primary School Age." Психологическая наука и образование 26, no. 5 (2021): 43–53. http://dx.doi.org/10.17759/pse.2021260504.
Повний текст джерелаАнотація:
The article describes the development of a diagnostic tool for determining the level of learning motivation in primary school children. The questionnaire developed by the authors includes 4 scales: negative attitude towards full-time schooling, demonstration of competence, positive attitude towards school life, and social significance of learning as a value. The study was carried out on a stage-by-stage basis and involved 352 students of 3—4 grades of Moscow schools in the first stage and 364 students in the second. The first stage allowed us to select 15 out of 33 judgments which differentiate the answers of children the most. These judgments made up the final version of the questionnaire which was used in the subsequent stages of the study. Standardization of the questionnaire showed the internal consistency of its scales and the correspondence between the indicators of motivation obtained in the test and the expert assessments of teachers. The results obtained in the study demonstrate construct validity and reliability of the “Learning Motivation Level” questionnaire. The developed technique can be used for monitoring purposes in primary schools in order to study and assess children’s motivation and needs, in counseling and research practice, and for assessing the effectiveness of various educational programs.
32
Demorest, Steven M., and Peter Q. Pfordresher. "Singing Accuracy Development from K-Adult." Music Perception 32, no. 3 (February 1, 2015): 293–302. http://dx.doi.org/10.1525/mp.2015.32.3.293.
Повний текст джерелаАнотація:
The development of singing accuracy, and the relative role of training versus maturation, is a central issue for both music educators and those within music cognition. Although various studies have focused on singing accuracy in different age groups, to date we know of no data sets that maintain the consistency in recruitment, methodology, and measurement that is necessary to make direct comparisons. We report analyses of three data sets that meet these criteria: two groups of children (kindergarten, middle school), and one group of adults (college aged). The data were collected at different times, but used a similar set of tasks and identical scoring procedures. Results indicate considerable improvement in accuracy from kindergarten to late elementary that dramatically reverses such that college students perform at the level of kindergartners. It appears singing accuracy may be related to variables involving singing experience rather than general development, and singing skill could decline over time if not maintained through engagement. A secondary purpose was to explore the efficacy of acoustic scoring for some singing tasks and how well it mimics human judgments of accuracy. The acoustic scoring procedure was highly correlated with expert judgment and could provide a standard approach to scoring that is largely automated. We discuss the potential benefits of a more unified approach to measuring singing accuracy and suggest future research that includes children, adolescents and adults in the sample.
33
Triwibowo, Febri Dhany, Ani Rusilowati, and Dwi Anggani Linggar Bharati. "Content Validity and Reliability of The Inter-Rater Instrument for Android-Based Speaking Performance Assessment." Journal of Educational Research and Evaluation 9, no. 1 (January 8, 2021): 52–57. http://dx.doi.org/10.15294/jere.v9i1.43865.
Повний текст джерелаАнотація:
This research is part of development research based on problems encountered in the field, which is the absence of an android-based speaking assessment instrument and one that supports one of the conservation values of Semarang State University, which is paperless. The aim of this study was to show the results of the validity and reliability test of the developed android-based speaking performance assessment instrument. The research method used is quantitative description of 3 expert judgments. The validation instrument developed in the form of an expert assessment sheet with 4 criteria, there are ease of use, visuals, content, and benefits. Analysis of the content validity of the assessment sheet using the V coefficient by Aiken and the reliability of the instrument content using the Interclass Correlation Coefficient (ICC) analysis with the help of SPSS version 16.0. The results showed valid results with all criteria valued> 0.3, namely with the lowest index 0.8 and the highest 1.0. Inter-rater reliability test using ICC obtained a value of 0.875, which means that all the criteria for the developed android-based speaking performance assessment instrument have a good level of consistency. Based on the results, it can be showed that the android-based speaking performance assessment instrument can be used.
34
Neroba, Vadym. "DEVELOPMENT OF METHODS FOR ASSESSMENT AND SELECTION OF UNMANNED AERIAL VEHICLE FOR MINE RECONNAISSANCE." ScienceRise, no. 5 (November 11, 2020): 44–50. http://dx.doi.org/10.21303/2313-8416.2020.001496.
Повний текст джерелаАнотація:
Object of research: comparative assessment and selection of an unmanned aerial vehicle for mine reconnaissance sample.
Investigated problem: substantiation of the methodological apparatus for comparative assessment and selection of an unmanned aerial vehicle for mine reconnaissance sample, taking into consideration the presence of both quantitative and qualitative indicators.
Main scientific results: the methods of comparative assessment and selection of an unmanned aerial vehicle for mine reconnaissance sample is developed. The technique is based on an expert method, which allows a drone sample to be evaluated and selected objectively, taking into consideration the presence of both quantitative and qualitative indicators. At the same time, group interaction and discussion of experts are realized. When the judgments do not coincide, an artificial consensus is not imposed. The number of experts is not limited. The experts are not linked in any way. The need to ensure transitive consistency (10–12 %) makes it possible to record attempts by an expert (experts) to artificially overestimate the indicators of one of the drone samples (or the one being evaluated), therefore, the indicators of another sample will automatically deteriorate. The principle of impartiality and fairness is maintained. A well-trained objective coordinator is not required, and the reality is that the absence of the disrupting the problem solution possibility is due to a change in the psychological situation among the experts.
Area of practical use of research results: humanitarian demining in the interests of ensuring the detection of mines for various purposes by sappers from a safe distance. At the same time, an increase within the probability of mines detecting is ensured due to special equipment installed onboard the drone.
Innovative technological product: a technique has been developed that allows not only assessing the drone samples for mine reconnaissance objectively, but making an objective choice of a sample for specific requirements also.
Scope of application of the innovative technological product: clearance of the terrain remaining after the end of hostilities. With the help of unmanned aerial vehicles, a significant acceleration of the demining process is possible, especially in those territories where mines are installed and being for a sufficiently long time.
35
Tisocco, Franco, and Mercedes Fernández Liporace. "THE TUCKMAN PROCRASTINATION SCALE: PSYCHOMETRIC FEATURES AMONG BUENOS AIRES UNDERGRADUATES." Psychological Thought 14, no. 2 (October 30, 2021): 444–66. http://dx.doi.org/10.37708/psyct.v14i2.603.
Повний текст джерелаАнотація:
Procrastination is a deleterious and increasingly pervasive phenomenon within the higher-academic domain, and the progressive refinement of its measurement tools proves vital to shed light and undertake this behavior. Thus, the present study examines renewed psychometric quality features of the Tuckman Procrastination Scale within an Argentinian sample. The sample was composed of 923 undergraduates from Buenos Aires City and its environs (80.7% female; 18.7% male; 0.5% non-binary; Mage = 26.60; SDage = 8.25). The Cordoban-Argentinian adaptation of the Tuckman Procrastination Scale was employed. Content validity analysis of the scale’s items was carried out upon consideration of expert judgments. Face validity of the instrument was analyzed via a pilot study with a subsample of undergraduates. Subsequently, a confirmatory factor analysis of the Tuckman Procrastination Scale structure was conducted, and the internal consistency of the resulting factor was examined. Finally, correlations with the Academic Motivation Scale were analyzed to provide evidence of convergent validity. Results of the Confirmatory Factor Analysis supported an adequate fit of the Tuckman Procrastination Scale’s structure in its Cordoban-version 15 items, while internal consistency was acceptable-to-excellent. Finally, convergent validity evidence mostly exhibited positive associations between Procrastination and both Amotivation and less self-determined motivational subscales of the Academic Motivation Scale, while negative associations were observed with regards to Intrinsic Motivation subscales.
36
Edwards, Sarah J. L., Tracey Stone, and Teresa Swift. "Differences between research ethics committees." International Journal of Technology Assessment in Health Care 23, no. 1 (January 2007): 17–23. http://dx.doi.org/10.1017/s0266462307051525.
Повний текст джерелаАнотація:
Objectives:To examine differences in the ethical judgments made by Research Ethics Committees (RECs) or Institutional Review Boards (IRBs).Methods:We did a review of the literature and included any study that attempted to compare the ethical judgments made by different RECs or IRBs when reviewing one or more protocol.Results:There were twenty-six articles reporting such discrepancies across Europe, within the United Kingdom, Spain, and United States. Of these studies, there were only five reports of some RECs approving while others rejecting the same protocol. All studies, however, reported differences in the clarifications and revisions asked of researchers regarding consent, recruitment, risks and benefits, compensation arrangements, and scientific issues.Conclusions:The studies were generally anecdotal reports of researchers trying to do research. New rules requiring a single ethical opinion for multi-site research at least in European Member States may simply conceal problematic issues in REC decision making. In the last analysis, we should expect a certain degree of variation and differences if we are to keep a committee system of review, although there is a pressing need to investigate the way in which RECs make these judgments. In particular, we need to identify the source of any aberrations, distortions, or confusions that could arbitrarily affect these judgments. Furthermore, local conditions remain important ethical considerations and should not be sidelined in pursuit of greater “consistency.”
37
Luna-Krauletz, María Delfina, Luis Gibran Juárez-Hernández, Ricardo Clark-Tapia, Shafía Teresa Súcar-Súccar, and Cecilia Alfonso-Corrado. "Environmental Education for Sustainability in Higher Education Institutions: Design of an Instrument for Its Evaluation." Sustainability 13, no. 13 (June 25, 2021): 7129. http://dx.doi.org/10.3390/su13137129.
Повний текст джерелаАнотація:
Higher Education Institutions (HEI) play a fundamental role in the transition towards Environmental Education for Sustainability (EES). As a consequence, one of the most critical challenges is the need to know their level of incorporation into the environmental agenda. Therefore, an instrument was made and validated to determine the level of incorporation of Environmental Education for Sustainability into the environmental agenda of HEIs. For its construction, the dimensions of Institutional Identity, Teaching, Research, Extension/dissemination, and Linkage were considered, relying on a total of 17 items. Its validation was carried out through an expert review and expert judgment, and a pilot test was carried out to adapt it to the target population. The main result was an instrument that integrates the substantive and procedural functions of HEIs. Following the expert review, the instrument was improved according to their suggestions. The expert judgment showed an adequate content validity (Aiken’s V > 0.80; LL > 0.60). The pilot test also suggested that the understanding of instructions and items was adequate with an optimal value of internal consistency (Cronbach’s alpha of 0.862). An instrument that determines the level of incorporation of the EES in the substantive and procedural functions of HEIs is presented, valid in content, and with adequate levels of clarity and understanding of the target population.
38
Danovitch, Judith H., and Christine K. Shenouda. "Adults’ and Children’s Understanding of How Expertise Influences Learning." Experimental Psychology 65, no. 1 (January 2018): 1–12. http://dx.doi.org/10.1027/1618-3169/a000387.
Повний текст джерелаАнотація:
Abstract. Adults and children use information about expertise to infer what a person is likely to know, but it is unclear whether they realize that expertise also has implications for learning. We explore adults’ and children’s understanding that expertise in a particular category supports learning about a closely related category. In four experiments, 5-year-olds and adults (n = 160) judged which of two people would be better at learning about a new category. When faced with an expert and a nonexpert, adults consistently indicated that expertise supports learning in a closely related category; however, children’s judgments were inconsistent and were strongly influenced by the description of the nonexpert. The results suggest that although children understand what it means to be an expert, they may judge an individual’s learning capacity based on different considerations than adults.
39
Octafia, Dwiana, Supriyadi Supriyadi, and Sulhadi Sulhadi. "Validity and Reliability Content of Physics Problem Solving Test Instrument Based on Local Wisdom." Journal of Educational Research and Evaluation 9, no. 1 (January 5, 2021): 46–51. http://dx.doi.org/10.15294/jere.v9i1.43712.
Повний текст джерелаАнотація:
This research was part of the research development a instrument test for the ability to solve physics problems based on problems that occur in the field, that is an instrument for assessing the ability to solve problems that has not been tested. The purpose of this study was to reveal the validity and reliability of the contents of the physics problem solving ability test instrument that had previously been compiled. The research method used is a quantitative description by 5 expert judgments. The instrument developed in the form of an expert observation sheet with 3 aspects of assessment, namely aspects of content feasibility, aspects of construction and aspects of language. Analysis of the content validity of the observation sheet using the V coefficient by Aiken and the reliability of the instrument content using the Inter-class Correlation Coefficient (ICC) analysis with the help of SPSS version 16.0. Based on the results of the study, it shows valid results with all item items valued> 0.3, those are with the lowest index 0.6 and the highest 1.0. Inter-rater reliability test using ICC obtained a value of 0.630, which means that all aspects of the instrument for assessing the ability of physics problem solving that have been assessed have a level of consistency. Based on the results of these studies, the test instrument for the ability to solve physics problems is feasible to use.
40
DE CLERCQ, ORPHÉE, VÉRONIQUE HOSTE, BART DESMET, PHILIP VAN OOSTEN, MARTINE DE COCK, and LIEVE MACKEN. "Using the crowd for readability prediction." Natural Language Engineering 20, no. 3 (December 14, 2012): 293–325. http://dx.doi.org/10.1017/s1351324912000344.
Повний текст джерелаАнотація:
AbstractWhile human annotation is crucial for many natural language processing tasks, it is often very expensive and time-consuming. Inspired by previous work on crowdsourcing, we investigate the viability of using non-expert labels instead of gold standard annotations from experts for a machine learning approach to automatic readability prediction. In order to do so, we evaluate two different methodologies to assess the readability of a wide variety of text material: A more traditional setup in which expert readers make readability judgments and a crowdsourcing setup for users who are not necessarily experts. To this purpose two assessment tools were implemented: a tool where expert readers can rank a batch of texts based on readability, and a lightweight crowdsourcing tool, which invites users to provide pairwise comparisons. To validate this approach, readability assessments for a corpus of written Dutch generic texts were gathered. By collecting multiple assessments per text, we explicitly wanted to level out readers' background knowledge and attitude. Our findings show that the assessments collected through both methodologies are highly consistent and that crowdsourcing is a viable alternative to expert labeling. This is a good news as crowdsourcing is more lightweight to use and can have access to a much wider audience of potential annotators. By performing a set of basic machine learning experiments using a feature set that mainly encodes basic lexical and morpho-syntactic information, we further illustrate how the collected data can be used to perform text comparisons or to assign an absolute readability score to an individual text. We do not focus on optimising the algorithms to achieve the best possible results for the learning tasks, but carry them out to illustrate the various possibilities of our data sets. The results on different data sets, however, show that our system outperforms the readability formulas and a baseline language modelling approach. We conclude that readability assessment by comparing texts is a polyvalent methodology, which can be adapted to specific domains and target audiences if required.
41
Luft, Joan L., and Michael D. Shields. "Why Does Fixation Persist? Experimental Evidence on the Judgment Performance Effects of Expensing Intangibles." Accounting Review 76, no. 4 (October 1, 2001): 561–87. http://dx.doi.org/10.2308/accr.2001.76.4.561.
Повний текст джерелаАнотація:
This study shows experimentally that when individuals use information on intangibles expenditures to predict future profits, expensing (vs. capitalizing) the expenditures significantly reduces the accuracy, consistency, consensus, and self-insight of individuals' subjective profit predictions. The experimental design allows us to eliminate several competing explanations for this apparent fixation on accounting. Subjects do not base their judgments on a nai¨ve prior belief that expensing precludes effects on future profits; a preexperiment question shows that subjects expect intangibles expenditures will affect future profits even when expensed. Moreover, subjects do not lack, or fail to use, data that would allow them to learn the exact expenditure-profit relation. They receive data on intangibles expenditures and profits as a basis for learning, and in some respects the learning is quite successful even when intangibles are expensed; subjects' profit predictions accurately reflect the mean and standard deviation of actual profits. Nevertheless, consistent with psychological theories of learning, subjects do not learn the exact magnitude of the effect of intangibles on future profits as well when the intangibles are expensed. Although the mean of their predictions is accurate, they do not discriminate well between cases with high and low actual profits. In consequence, their prediction accuracy, consistency, consensus, and self-insight are lower when intangibles are expensed. Thus, in this case, learning does not mitigate fixation on accounting, because accounting affects the learning process itself.
42
Hotchkiss, James. "Polar Opposites: Judgments and Counterfactuals in Sainsburys V. Mastercard and Asda V. Mastercard." World Competition 41, Issue 3 (September 1, 2018): 419–51. http://dx.doi.org/10.54648/woco2018023.
Повний текст джерелаАнотація:
This article explores the recent cases Sainsbury’s v. Mastercard and Asda v. Mastercard and uses them to demonstrate how the decentralization of Article 101 TFEU enforcement is creating legal uncertainty due to national courts being unequipped to apply complex Ex Post counterfactuals consistently. It considers the distinction between restriction of competition by object and restriction of competition by effect to show that EU and national courts now apply the latter. It then considers the requirements for effects-based analysis, focussing on the mandatory use of Ex Post counterfactuals, highlighting their emergence as a legal mechanism in Article 101 application. This article argues that Ex Post counterfactuals’ basis in vague economic theory creates significant difficulties for national courts attempting to enforce Article 101 consistently and evidences these difficulties by considering the courts’ composition, their overreliance on expert economic witnesses, the standard of proof, complex court interplay and referral for preliminary ruling. Ultimately, it argues that despite procedural tools being provided to national courts to ensure consistent application of Article 101 at national and EU levels, the courts are failing to utilize them, resulting in the creation of significant legal uncertainty as evidenced by the polaropposite judgments reached in the Mastercard cases.
43
Chowdhury, Shakhawat. "Decision making with uncertainty: an example of water treatment approach selection." Water Quality Research Journal 47, no. 2 (May 1, 2012): 153–65. http://dx.doi.org/10.2166/wqrjc.2012.107.
Повний текст джерелаАнотація:
Decision makers often encounter uncertainty in selecting the best available option. Fuzzy multicriteria decision making aggregates different basic criteria through a hierarchy structure. This aggregation combines fuzzy assessment and priority matrices. When available data are imprecise, the assessment and priority matrices for different hierarchy level criteria are developed from expert judgments. In fuzzy aggregation, uncertainties in the assessment matrix are captured with fuzzy membership functions. The priority matrix is developed through pairwise comparison, in which the relative importance of the criteria is represented in crisp values; thus uncertainties associated with priority assignments are not incorporated in traditional fuzzy aggregation. This paper presents the application of a methodology to incorporate fuzziness in developing a priority matrix for environmental decision making. Fuzzy α-cut technique for different confidence intervals has been incorporated. The gradient eigenvector method has been followed to obtain consistency in constructing the priority matrix for different hierarchy level criteria. The max-min paired elimination method for hierarchical aggregation has been used to obtain the final fuzzy set. A case study for drinking water treatment technology evaluation is performed to present the potential environmental application of this approach, leading to the identification of the best treatment technology.
44
Zribi, Rania, and Chokri Smaoui. "Rater-task interaction effects on testing EFL learners’ written performances." Language Testing in Focus: An International Journal 4 (December 2021): 23–36. http://dx.doi.org/10.32038/ltf.2021.04.03.
Повний текст джерелаАнотація:
This study examines the interaction effect between rater groups and tasks on evaluating EFL learners’ written performances. Fifty raters took part in this study. The experienced rater group (n=25) and the novice rater group (n=25) judged sixty essays (30 narratives and 30 argumentative writing modes) written by third-year English students. Raters’ decision-making behaviours, in terms of scores assignment and written comments, were diagnosed based on different quantitative and qualitative tools. Scores were analysed based on FACETS to examine the effects of rater-task interaction on raters’ severity and internal consistency and the analytic scale’s functionality. Qualitative data were also analysed to diagnose which aspects of writing were deemed more important than others across rater groups and task types. The analysis revealed that both raters and tasks were substantially influential factors. The majority of expert raters displayed more severity in assessing narrative essays than argumentative essays. Different qualitative judgments are also detected across raters and tasks due to rating experience and task requirements. The findings of this study reflected implications not only for testing learners’ writing proficiency but also for test validation research in the task-based writing performance assessment field.
45
Aly, S., and I. Vrana. "Approaches to assess the group consensus in Yes-or-No type experts’ group decision making." Agricultural Economics (Zemědělská ekonomika) 56, No. 4 (April 22, 2010): 192–99. http://dx.doi.org/10.17221/97/2009-agricecon.
Повний текст джерелаАнотація:
Group consensus indicators provide an important insight and information about how to combine a group of expert judgments. This paper is concerned with the development of a set of indicators to be used in analyzing the group consensus in evaluating Yes-or-No type’s decision problems. The opinions of the experts are in the form of a real number between 0 and 10 expressing the degree of answers Yes or No (0 for sharp No and 10 for sharp Yes). Two methods for obtaining the consensus indicators are developed. The first of them is based on configuring the one previously developed by (Ngwenyama et al. 1996), which is reviewed in this paper. The other one is an improved one that does not rely on the existence of the known or desired similarity significance levels or thresholds. A new measure of consensus is introduced, the standard deviation. An experiment is conducted to get acquainted with the relationship between the standard deviation of group decisions and one of the developed group consensus indicators, which measures the agreement level within the group of decisions. This research is intended to develop more consistent indicators and measures group consensus and position of each individual relative to others for Yes-or-No type group decisions. This is aimed at the exploitation of such important and relevant consensus information for developing a new consensus-based heuristic algorithm to combine the multiple experts’ judgments or to be able to select the adequate combining criteria. Finally, the presented approach could be usefully utilized in critical “Yes – or – No” GDM problems in business and industry.
46
Ariyano, Ariyano, Amay Suherman, and Handiansyah Akhmadi. "Development of Multimedia Based on Autodesk Inventor Software on the Concept of Relative Velocity to Increase Students’ Generic Science Skill." Journal of Vocational Education Studies 3, no. 2 (November 28, 2020): 111. http://dx.doi.org/10.12928/joves.v3i2.2482.
Повний текст джерелаАнотація:
This research aims to develop autodesk inventor-based multimedia that was designed to increase students’ generic science skill on the application of relative velocity at kinematics and dynamics courses. This study used the mini course method developed by Borg and Gall, including the stage of analysing and planning, developing early product, and validating from the expert and revision the early product. Based on the analysis conducted, it was revealed that there were five indicators that students had difficulties with, including illustrating kinematic diagram, illustrating velocity direction, calculating absolute velocity, illustrating velocity polygon, and calculating velocity based on velocity polygon. Those five indicators were related to six generic science aspects, including modelling, symbolic language, laws of causality, logical consistency, scale awareness, and observation. The developed multimedia consists of nine displays of slider-crank mechanism and eight displays of four-bar mechanisms, using .idw, .iam, and .mp4 formats and has been validated by material and media experts. Based on the judgment from the experts, the inventor-based multimedia was worthy to be applied in the course.
47
Weiss, David J., and James Shanteau. "Empirical Assessment of Expertise." Human Factors: The Journal of the Human Factors and Ergonomics Society 45, no. 1 (March 2003): 104–16. http://dx.doi.org/10.1518/hfes.45.1.104.27233.
Повний текст джерелаАнотація:
The assessment of expertise is vital both in practical situations that call for expert judgment and in theoretical research on the psychology of experts. It can be difficult, however, to determine whether a judge is in fact performing expertly. Our goal was to develop an empirical measure of expert judgment. We argue that two necessary characteristics of expertise are discrimination of the various stimuli in the domain and consistent treatment of similar stimuli. We combine measures of these characteristics to form a ratio we call the Cochran-Weiss-Shanteau (CWS) index of expertise. The proposed index was demonstrated using two studies that distinguished experts from nonexperts based on their judgmental performance. The index provides new insights into expertise and offers a partial definition of expertise that may be useful in a variety of theoretical and applied settings. Potential applications of this research include selection, training, and evaluation of experts and of expert-machine systems.
48
Lincoln, Stephen E., Shan Yang, Melissa S. Cline, Yuya Kobayashi, Can Zhang, Scott Topper, David Haussler, Benedict Paten, and Robert L. Nussbaum. "Consistency of BRCA1 and BRCA2 Variant Classifications Among Clinical Diagnostic Laboratories." JCO Precision Oncology, no. 1 (November 2017): 1–10. http://dx.doi.org/10.1200/po.16.00020.
Повний текст джерелаАнотація:
Purpose Genetic tests of cancer predisposition genes, BRCA1 and BRCA2, inform significant clinical decisions for both physicians and patients. Most uncovered variants are benign, and determining which few are pathogenic—disease causing—is sometimes challenging and can potentially be inconsistent among laboratories. The ClinVar database makes deidentified clinical variant classifications from multiple laboratories publicly available for comparison and review, per recommendations by the American Medical Association, the American College of Medical Genetics, the National Society for Genetic Counselors, and other organizations. Methods Classifications of more than 2,000 BRCA1/2 variants in ClinVar that represent approximately 22,000 patients were dichotomized as clinically actionable or not actionable and compared among as many as seven laboratories. The properties of these variants and classification differences were investigated in detail. Results Per-variant concordance was 98.5% (CI, 97.9% to 99.0%). All discordant variants were rare; thus, per-patient concordance was estimated to be higher (99.7%). ClinVar facilitated resolution of many of the discordant variants, and concordance increased to 99.0% per variant and 99.8% per patient when reclassified, but not yet resubmitted, variants and submission errors were addressed. Most of the remaining discordances seemed to involve either legitimate differences in expert judgment regarding particular scientific evidence or were classifications that predated the availability of important scientific evidence. Conclusion Significant classification disagreements among professional clinical laboratories represented in ClinVar are infrequent yet important. Unrestricted sharing of clinical genetic data allows detailed interlaboratory quality control and peer review, as exemplified by this study.
49
Repp, Bruno H., and Robert M. Goehrke. "Music Notation, But Not Action on a Keyboard, Influences Pianists' Judgments of Ambiguous Melodies." Music Perception 28, no. 3 (February 1, 2011): 315–20. http://dx.doi.org/10.1525/mp.2011.28.3.315.
Повний текст джерелаАнотація:
Pitch Increases from Left to Right on Piano keyboards. When pianists press keys on a keyboard to hear two successive octave-ambiguous tones spanning a tritone (half-octave interval), they tend to report hearing the tritone go in the direction consistent with their key presses (Repp & Knoblich, 2009). This finding has been interpreted as an effect of action on perceptual judgment. Using a modified design, the present study separated the effect of the action itself from that of the visual stimuli that prompt the action. Twelve expert pianists reported their perception of octave-ambiguous three-note melodies ending with tritones in two conditions: In the active condition, they saw a notated melody and played it on a keyboard to hear it, while in the passive condition they viewed the notation while the melody was played to them. Participants tended to report hearing the tritone as it appeared in the notation, but action had no additional effect. We discuss whether the "action direction effect" described by Repp and Knoblich may have been caused by the visual action prompts, not by the action itself.
50
Shpakou, A., and A. Shpakau. "Evaluation of the possibility of using Anderson and Dedrick’s Trust in Physician Scale in Belarusian conditions." Progress in Health Sciences 7, no. 2 (December 29, 2017): 50–59. http://dx.doi.org/10.5604/01.3001.0010.7850.
Повний текст джерелаАнотація:
Introduction: A patient that trusts a doctor feels safer and more easily adapts to a doctor's recommendations. Aim of the study: To assess the possibility of using the patient's trust scale by Anderson and Dedrick in Belarusian conditions. Materials and Methods: The study used the Trust in Physician Scale (TPS) by Anderson and Dedrick. Validation was performed on a group of 251 randomly selected individuals. The validation process consisted of two parts: translation and evaluation of the psychometric properties of the newly translated instrument, and its purpose was to compare the results obtained at the intercultural (international) level and apply the test in Belarus. Results: Internal consistency of the Russian TPS was high (Cronbach’s alpha = .891). The highest mean scores were for items “My doctor is a real expert in taking care of medical problems like mine” - 3.95±0.77; “I trust my doctor to put my medical needs above all other considerations when treating my medical problems” - 3.83±0.80; “I trust my doctor’s judgments about my medical care” - 3.66±0.88; and “I trust my doctor so much, I always try to follow his/her advice” - 3.64±0.99. Conclusions: The Russian language scale fulfills all the criteria of psychometric equivalence with the original version of The Trust in Physician Scale.