Dissertationen zum Thema „Human-centered evaluation“

Thesis (S.M.)--Massachusetts Institute of Technology, Dept. of Civil and Environmental Engineering, February 2002.
Includes bibliographical references (p. 69-75).
by Simonetta Andrea Rodriguez.
S.M.

There  are  currently  multiple  usability  evaluation methods  in  use. Some  of  them  involve  users  whereas  others  do  not.  It  is commonly assumed that the usability problems found via user-involved methods are real problems that the end-users may face when using the product in real situations. In this study, one usability evaluation method that does not involve  users,  Heuristic  Evaluation,  was  evaluated  by  comparing the results  it  provided to the results provided by a user-involved method, Think Aloud. Heuristic Evaluation’s performance in detecting usability problems of a system was evaluated in two cases: taking all of the usability problems of the interface into account and only taking the serious usability problems of the interface into account. A system called Nytt, provided by Nytt Ab, was used as the evaluated system. The values of thoroughness, validity and effectiveness were calculated based on the numbers of actual usability problems found via Heuristic Evaluation, actual problems missed by Heuristic Evaluation and false problems found via Heuristic Evaluation. The findings of this study suggest that Heuristic Evaluation does not detect all of the problems that Think Aloud does but instead offers a lot of other findings as problems. Moreover, the performance of Heuristic Evaluation is not drastically affected by  whether detecting only serious usability problems or usability problems in general.

A great deal of effort has been put into developing Embodied Conversational Agent (ECA) systems that provide a human-like assistant in the user interface. However, little is known whether improvements to ECA interfaces made by such efforts can ever be significant from the users point of view. I studied user experiences with ECA interfaces and evaluated the ECA style of interaction with respect to user expectation, perception, behavior and performance. I introduce a conceptual framework that offers a holistic view of the design space of ECA systems. I also have created a middleware toolkit that facilitates rapid development of application content across different speech and animation platforms. A series of user studies has been carried out to investigate the similarities and differences between human-computer interaction and human-ECA interaction and between human-ECA interaction and human-human interaction. Results from these studies provide strong evidence that people are consciously aware of the capabilities and limitations of ECAs. Traditional GUI design heuristics should be carefully followed when designing ECA interfaces. Furthermore, the results soundly suggest that designers of ECA interfaces take extra care to accommodate individual differences and preferences. Social norms that guide human-human interaction greatly affect individuals expectation and perception of ECA characteristics. The findings support the argument that drawing from both human-computer interaction and human-human interaction can be significantly advantageous to the design of both effective and affective human-ECA interaction.

The practice of health self-management offers behavioral and problem-solving strategies that can effectively promote responsibility for one's own wellbeing, improve one's health outcomes, and decrease the cost of health services. Personal informatics applications support health self-management by allowing their users to easily track personal health information, and to review the changes and patterns in this information. Over the course of the past several years, I have pursued a research agenda centered on understanding how personal health informatics applications can further support the strategies of health self-management--specifically those relating to goal-management and behavior change. I began by developing a flexible personal informatics tool, called Salud!, that I could use to observe real-world goal management and behavior change strategies, as well as use to evaluate new interfaces designed to assist in goal management. Unlike existing personal informatics tools, Salud! allows users to self-define the information that they will track, which allows tracking of highly personal and meaningful data that may not be possible to track given other tools. It also enables users to share their account data with facilitators (e.g. fitness grainers, nutritionists, etc.) who can provide input and feedback. Salud! was built on top of an infrastructure consisting of a stack of modular services that make it easier for others to develop and/or evaluate a variety of personal informatics applications. Several research teams used this infrastructure to develop and deploy a variety of custom projects. Informal analysis of their efforts showed an unmet need for data storage and visualization services for home- and health-based sensor data. In order to design a goal management support tool for Salud!, I first, I conducted a meta-analysis of relevant research literature to cull a set of proven goal management strategies. The key outcome of this work was an operationalization of Action Plans--goal management strategies that are effective at supporting behavior change. I then deployed Salud! in two fitness-related contexts to observe and understand the breadth of health-related behavior change and goal management practices. Findings from these deployments showed that personal informatics tools are most helpful to individuals who are able to articulate short-term, actionable goals, and who are able to integrate self-tracking into their daily activities. The literature meta-analysis and the two Salud! deployments provided formative requirements for a goal management interaction that would both incorporate effective goal management strategies and support the breadth of real-world goals. I developed a model of the goal management process as the framework for such an interaction. This model enables goals to be represented, evaluated, and visualized, based on a wide range of user objectives and data collection strategies. Using this model, I was able to develop a set of interactions that allow users of Salud! to manage their personal goals within the application. The generalized goal management model shows the inherent difficulty in supporting open-ended, highly personalized goal management. To function generically, Salud! requires facilitator input to correctly process goals and meaningfully classify their attributes. However, for specific goals represented by specific data collection strategies, it is possible to fully- or semi-automate the goal management process. I ran a large-scale evaluation of Salud! with the goal management interaction to evaluate the effectiveness of a fully-automated goal management interaction. The evaluation consisted of a common health self-management intervention: a simple fitness program to increase participants' daily step count. The results of this evaluation suggest that the goal management interaction may improve the rate of goal realization among users who are initially less active and less confident in their ability to succeed. Additionally, this evaluation showed that, while it can significantly increase participants' step count, a fully automated fitness program is not as effective as traditional, instructor-led fitness programs. However, it is much easier to administer and much less resource intensive, showing that it can be utilized to rapidly evaluate concrete goal management strategies.

When aiming to successfully improve an existing software system, usability evaluation methods (UEMs) and user experience (UX) are key aspects for consideration. The UEMs identify the level of usability of the system through assessing: (1) the extent to which it is easy and pleasant for the user (Cockton, 2012); (2) the specific effects of the system user interface (UI) on the user; and (3) any other problems that the system may have (Dix et al., 2007). On the other hand, considering UX places usability in context through providing a comprehensive understanding of the users' perceptions during and after their interactions with a specific system (Kuniavsky, 2010). Undoubtedly, in most countries, there is a wide range of services, activities, and procedures that are supported by government systems (Buie and Murray, 2012). However, because of the lack of consideration of the usability requirements in addition to the limited attention given to the involvement of UX in system development (Downey and Rosales, 2011), a significant number of these government systems were designed without taking into account human-centred design guidelines (Johnson et al, 2005). Consequently, the success of these systems varies widely in terms of their usability (Downey and Rosales, 2011). In some cases, they fail to provide effective, efficient, and generally positive UX to people who interact with government systems from the outside, such as the citizens, or for those who work for the government on the inside, such as the employees (Buie and Murray, 2012). The research problem in this thesis addresses how UEMs, techniques, and tools can be integrated and developed to support the redesigning and enhancement of current government systems (Legacy Systems) in a developing country. More specifically, the main aim of the research work reported in this thesis is to develop a way of proposing appropriate methods for evaluating the usability of the current internal systems in the Saudi government context. In this regard, three studies were conducted to achieve the aims of the research. As a general approach for the thesis, Human-Centred Design (HCD) was adopted due to the fact that HCD is concerned with the integration of the users’ opinions into the software development process in order to achieve a usable system (Spencer, 2004). In addition, a mixed method of quantitative and qualitative approaches was used in all of the studies. In the development of this project, the first study was aimed at evaluating the usability of a current internal system of a governmental organization in Saudi Arabia, the Visa Issuance (VI) system, from the actual users’ points of view in order to identify the strengths and weaknesses of the system. A usability evaluation query technique was employed for collecting data via a survey method by targeting 135 participants who were the users of the VI system. The survey used both qualitative and quantitative instruments, namely a questionnaire and semi-structured interviews. In the second study, an experimental approach was applied and a comparative usability test was conducted between the current VI system and a suggested prototype design which was developed based on the outcomes of the usability evaluation in the first study. The results of this study showed improvements in the quality of the system (usefulness), the information, and the interface. After analysing these results, the iterative method was used in the third study to redesign the suggested prototype. Therefore, another comparison test was conducted between the two versions of the prototype and the results indicated enhancement in the UX by using the new version. This research developed a methodological framework for the usability evaluation of the current government systems which involved query techniques and user testing methods. It was formulated by combining different methods for guiding the redesign process, and testing was conducted throughout the entire research project. The results indicated that the involvement of query techniques as a preliminary step provides a quick, simple, and cost-effective way of identifying the usability problem areas in the VI government system. Furthermore, the usefulness of this developed framework could be beneficial in raising awareness and acceptance of the established methods among governmental organisations in other contexts in order to enhance their software systems effectively and improve the UX. It is hoped that this awareness of the fundamental usability methods could lead to developments in Information Communication Technology (ICT) for all communities (Holzinger, 2005) so that the advantages of making certain improvements could be shared with others. In addition, the outcomes of the two experiments conducted in this research provide some lessons that are considered valuable in the usability testing domain. In this regard, the results are expected to assist and support the usability practitioners and system developers who are concerned with improving the usability of existing internal software systems, and in planning and conducting usability testing sessions in government organisations through utilising such guidance about UEMs.

Mobile Augmented Reality (MAR) is an emerging technology that, when done right, can offer the users a rich experience. However, as with most emerging technologies, it is primarily technology-driven, often leaving the user needs aside. In this study a sensor-based MAR prototype was developed and evaluated, following a User-Centered Design process. The prototype visualized a future building in the city of Stockholm, Sweden. However, sensors in today’s smartphones are not perfect, which is a problem when used as tracking method to augment 3D-content. Also, developing a MAR experience includes some challenges and issues that need to be addressed in order to develop a good experience for the user. One of those challenges is to test and validate ideas at an early stage of the development. Usually when following a User-Centered Design process, prototyping is a useful tool for testing and validating. Though, even the simplest MAR experience requires an advanced infrastructure, which is expensive and time-consuming to develop. In this study both the inherited problems of smartphone sensors and the challenges regarding MAR development were taken into account during the development. To counteract these problems, cleverly designed functions were implemented. To evaluate the final version of the prototype, a user test was conducted in an urban environment. The most important lessons learned during this study were that today’s smartphones do not provide robust and accurate sensor data in an urban environment. Resulting in incorrect placement and flickering of the augmented 3D-object. Also, the functions that were developed to counteract these problems helped in some degree, indicating the potential of overcoming hardware problems with cleverly designed functions. Last, using existing AR framework to produce an interactive and AR-enabled prototype, to a low cost and relatively fast, proved successful.
Mobil förstärkt verklighet (Mobile Augmented Reality, eller MAR) är en teknik på frammarsch som utvecklas i snabb takt. MAR har möjligheten att leverera en rik upplevelse till användaren. Dock är utvecklingen av MAR, likt de flesta tekniker under snabb utveckling, teknikdriven vilket ofta medför att användarens behov glöms bort. I denna studie står användaren i fokus när en sensorbaserad MAR-prototyp utvecklas och utvärderas. Allt detta sker med hjälp utav en användarcentrerad designprocess och prototypen används för att visualisera en kommande byggnad i Stockholm, Sverige. Vid utveckling av en sensorbaserad MAR-upplevelse står man inför ett flertal utmaningar och problem som alla behöver adresseras för att kunna leverera en bra upplevelse till användaren. Dels finns hårdvaruproblem så som att sensorer i dagens smartphones är långt ifrån perfekta, detta medför problematik i placeringen utav 3D-objekt som baseras på sensordata. En annan utmaning är att testa och validera idéer i ett tidigt skede av utvecklingen. Vanligtvis när man följer en användarcentrerad designprocess är tidiga prototyper ett användbart verktyg för testning och validering. Dock kräver även den enklaste MAR-upplevelsen en avancerad infrastruktur, som är både dyr och tidskrävande att utveckla. I denna studie tas dessa problem och utmaningar i beaktning redan i ett tidigt utvecklingsstadium. Väldesignade funktioner utvecklas för att motverka och minska inverkan utav dessa problem. För att utvärdera den slutgiltiga prototypen utfördes ett användartest i stadsmiljö. En viktig lärdom från denna studie är att dagens smartphones inte erbjuder tillräckligt robust och precis mätning av sensordata i stadsmiljö för att leverera en bra användarupplevelse. Detta resulterade i felaktig och hoppande/skakig placering av 3D-objekt. De funktioner som utvecklades för att motverka problemen med MAR hjälpte i viss utsträckning, dessa påvisade en potential att överkomma hårdvarubrister med smarta och väldesignade funktioner. Att använda AR-ramverk för att producera interaktiva MAR-prototyper, till en låg kostnad och med relativt kort utvecklingstid, visade sig lyckat.

Quantifying personal information is a rapidly emerging lifestyle that has now extended to tracking non-personal information as well. The Quantified Pet (QP) domain enablespet owners to gain insights in their pet’s behavior and wellbeing. This study investigates how to design a QP application using the User Centered Design Method. To gain understanding in the inner drivers that form the human-pet relationship, five dog owners are interviewed. Three inner drivers that trigger interaction are revealed: habits, love and guilt. By surveying 104 users of existing QP applications, this study examined what motivates users to use a pet activity tracker and what sustains this usage. After performing a thematic analysis on this data, it was found that Activities,interest in activity data, and Health, improving and ensuringthe dog’s health, are dominantly portraited in the results. These themes provide the foundation for the establishment of two personas, of which the Health persona is chosen to be the primary design target. Next to user requirements following from the persona design, a heuristic evaluation is performed on one QP application (FitBark) to provide an additional set of design requirements. From these requirements, a design solution is proposed and evaluated amongst ten participants by means of a task list, semi- structured interview and a questionnaire. This revealed high usability for navigation and successful implementation of most requirements. However, design flaws, in specific of data visualization, and some misunderstanding of informational components remain. Future work proposes an improved design and provides additional suggestions for implementation and examination.
Att kvantifiera personlig information är en starkt växande uppkommande livsstil och har nu även utökats till att följa icke-personlig information. Quantified Pet (QP) domänen möjliggör husdjursägare att få insikter i deras husdjurs beteende och välmående. I denna studie undersöks hur en QP-applikation ska designas genom att använda metoden User Centered Design. För att få förståelse för de innersta drivkrafterna till det som formar relationen mellan människa och husdjur blev fem hundägare intervjuade. Tre innre drivkrafter som triggar interactionen klarlades: vanor, kärlek och skuldkänslor. Genom att studera 104 användare av redan existerande QP-applikationer, denna studie undersökte vad som motiverar användare till att använda en pet activity tracker och vad som upperäthåller ett fortsatt användande. Efter att ha genomfört en tematisk analys av denna data visade det sig att Aktiviteter, intresse av aktivitetsdata, och Hälsa, förbättra och säkerställa hundens hälsa är de mest framträdande i resultaten. Dessa teman bidrar till grunden för etableringen av två personas, där personan för Hälsa är vald som primärt mål för designen. Utöver de användarkrav som uppkommit från personadesignen, har en hueristisk evaluering av en QR-applikation (Fitbark) gjorts för att bidra till ett ytterligare antal designkrav. Utifrån dessa krav föreslås en designlösning som evalueras bland tio deltagare genom en uppdragslista, semistrukturerad intervju och frågeformulär. Detta visade på en hög användbarhet för navigering och en lyckad implementation av de flesta av kraven. Dock existerar fortfarande brister i designen, specifikt för datavisualisering och några missförstånd över informationskomponenter. Framtida arbeten föreslår en förbättrad design och bidrar med ytterligare förslag på implementation och examinering.

Les algorithmes d'apprentissage machine déployés dans la société ou la technologie offrent généralement aux utilisateurs aucune prise sur la manière dont les modèles d'apprentissage sont optimisés à partir des données. Seuls les experts conçoivent, analysent et optimisent les algorithmes d'apprentissage automatique. À l'intersection de l'Interaction Humain-Machine (IHM) et de l'apprentissage machine, le domaine de l'apprentissage automatique interactif vise à intégrer l'apprentissage automatique dans des pratiques existantes. L'enseignement machine interactif (Interactive Machine Teaching), en particulier, cherche à impliquer des utilisateurs non experts en tant qu'enseignant de la machine afin de les autonomiser dans le processus de construction de modèles d'apprentissage. Ces utilisateurs pourraient profiter de la construction de modèles d'apprentissage pour traiter et automatiser des tâches sur leurs propres données, conduisant à des modèles plus robustes et moins biaisés pour des problèmes spécialisés. Cette thèse adopte une approche empirique sur l'enseignement machine interactif en se concentrant sur la façon dont les utilisateurs développent des stratégies et comprennent les systèmes d'apprentissage machine interactifs à travers l'acte d'enseigner. Cette recherche fournit deux études utilisateurs impliquant des participants en tant qu'enseignant de classificateurs d'images utilisant des réseaux de neurones artificiels appris par transfert. Ces études se concentrent sur ce que les utilisateurs comprennent du comportement du modèle ML et sur la stratégie qu'ils peuvent utiliser pour le "faire fonctionner". La seconde étude se concentre sur la compréhension et l'utilisation de deux types d'incertitude : l'incertitude aléatorique, qui traduit l'ambiguïté, et l'incertitude épistémique, qui traduit la nouveauté. Je discute de l'utilisation de l'incertitude et de l'apprentissage actif (Active Learning) comme outils pour l'enseignement machine interactif. Enfin, je présente mes collaborations artistiques et adopte une approche réflexive sur les obstacles et les opportunités de développement de l'apprentissage automatique interactif pour l'art. Je soutiens que les utilisateurs novices développent différentes stratégies d'enseignement qui peuvent évoluer en fonction des informations obtenues tout au long de l'interaction. Les stratégies d'enseignement structurent la composition des données d'entraînement et affectent la capacité des utilisateurs à comprendre et à prédire le comportement de l'algorithme. En plus de permettre aux gens de construire des modèles d'apprentissage automatique, l'enseignement machine interactif présente un intérêt pédagogique en favorisant les comportements d'investigation, renforçant les connaissances des novices en apprentissage machine
Machine Learning algorithms in society or interactive technology generally provide users with little or no agency with respect to how models are optimized from data. Only experts design, analyze, and optimize ML algorithms. At the intersection of HCI and ML, the field of Interactive Machine Learning (IML) aims at incorporating ML workflows within existing users' practices. Interactive Machine Teaching (IMT), in particular, focuses on involving non-expert users as "machine teachers" and empowering them in the process of building ML models. Non-experts could take advantage of building ML models to process and automate tasks on their data, leading to more robust and less biased models for specialized problems. This thesis takes an empirical approach to IMT by focusing on how people develop strategies and understand interactive ML systems through the act of teaching. This research provides two user studies involving participants as teachers of image-based classifiers using transfer-learned artificial neural networks. These studies focus on what users understand from the ML model's behavior and what strategy they may use to "make it work." The second study focuses on machine teachers' understanding and use of two types of uncertainty: aleatoric uncertainty, which conveys ambiguity, and epistemic uncertainty, which conveys novelty. I discuss the use of uncertainty and active learning in IMT. Finally, I report artistic collaborations and adopt an auto-ethnographic approach to challenges and opportunities for developing IMT with artists. I argue that people develop different teaching strategies that can evolve with insights obtained throughout the interaction. People's teaching strategies structure the composition of the data they curated and affect their ability to understand and predict the algorithm behavior. Besides empowering people to build ML models, IMT can foster investigative behaviors, leveraging peoples' literacy in ML and artificial intelligence

ABSTRACT Today there are lots of different health care computer systems in use. However, according to recent studies many of them lack necessary usability. Within Nordic pediatric cancer care, analogue treatment protocols on paper are currently used, as a complement to the digital medical records and prescription systems. In these protocols, doctors and nurses note information regarding the patient’s treatment. Comments and changes are noted in the margin, which sometimes leads to making the protocol messy and difficult to grasp. Since several people are involved in the handling of the treatment protocols it occasionally happens that the protocol disappears for periods of time. We had two aims with this project. The first was to examine and map requirements for a usable interactive treatment plan for acute lymphoblastic leukemia, ALL. The second was to investigate if our suggested combination of methods would be sufficient to acquire these requirements in a setting where the users, i.e. physicians, were time-­‐constrained.   Based on large variety of theories and methods from educational science and research in human computer interaction, we have conducted a qualitative study, iterating a combination of user-­‐centered design methods, with a revision of the requirements as well as the design following each iteration. The requirements analysis was performed in close collaboration with the doctors at the Astrid Lindgren Children's Hospital, Karolinska University Hospital, Stockholm, Sweden.   Our results indicate that by using a combination of methods from usability engineering and participatory design, a well-­‐defined list of requirements from the doctors could be identified which might be sufficient to develop an interactive prototype for a digital treatment protocol. In addition we found that our method enabled an exchange of knowledge between the designers and the users. In conclusion, these combined methods were suitable for enhancing the software designer’s understanding of the user needs in this time-­‐constrained environment.

Les personnes en situation de handicap moteur des membres supérieurs ont besoin de système de suppléance pour leur autonomie. Cependant 1/3 de ces aides de compensation sont abandonnées car pas encore suffisamment centrées sur les besoins et les aptitudes de l'utilisateur. La problématique centrale de cette thèse concerne la mise en œuvre d'une méthode de conception centrée utilisateur, adaptée en fonction de la population d'études (personnes en situation de handicap ayant des troubles neuro-moteurs langagiers) pour la conception de trois aides de compensation. La première concerne la commande d'un bras robotique de préhension JACO pour laquelle la conception centrée utilisateur classique a été appliquée. Celle-ci est adaptée pour les personnes pouvant s'exprimer oralement mais qu'en est-il lorsque ces personnes ne peuvent participer directement ou bien ont besoin de leurs aidants pour exprimer leurs besoins ou répondre aux questionnaires? Une alternative est de recueillir les besoins par l'intermédiaire d'experts. Nous montrerons comment la méthode de conception centrée utilisateur a été adaptée pour la conception du système HandiMathKey pour la saisie de formules Mathématiques. Notre dernière adaptation est d'utiliser l'écosystème de la personne comme médiateur pour communiquer et un profil utilisateur. Cette méthode a permis de concevoir l'interface de communication intégrée et de contrôle d'environnement nommée CECI. Nous décrirons également comment la plateforme SoKeyTo permet le prototypage des aides de compensation. Pour chacune des aides de compensation, nous rapporterons les évaluations réalisées et nous discuterons la manière dont la méthode de conception a été adaptée
People with motor impairment of the upper limbs need an assistive technology for their autonomy. However, 1/3 of these assistive technologies are abandoned because they are not yet sufficiently focused on the needs and abilities of the user. The central problem of this thesis concerns the implementation of a user-centered design method adapted to the study population (persons with disabilities with neuro-motor language impairments) for the design of three applications. The first concerns the control of a robot arm JACO for which the classic user centered design has been applied. This is appropriate for people who can speak orally, but what if they cannot participate directly or need their caregivers to express their needs or answer questionnaires? An alternative is to gather needs through experts. We will show how the user centered design method has been adapted for the design of the HandiMathKey system for entering mathematical formulas. Our last adaptation is to use the person's ecosystem as a mediator to communicate and a user profile. This method allowed to design the integrated communication and environmental control interface named CECI. We will also describe how the SoKeyTo platform allows prototyping of assistive technologies. For each of the assistive technologies, we will report the assessments made and discuss how the design method has been adapted

Research and development on vehicle safety has lately started to direct its focus towards how to actively support the driver and make it easier for her to drive safely through letting Advanced Driver Assistance Systems (ADAS) have effect on how the driver interacts with the vehicle and the surrounding traffic. This requires research on both how the driver and vehicle perform in different situations, in terms of psychology, cognition and individual differences. In addition, physical limitations and requirements of the driver and the vehicle must be taken into account. Therefore scenarios for evaluation of these systems are required. In the area of user-centered design a rather new method, Personas, is being adopted. This thesis tries to explore if the Persona method is a viable tool for creating scenarios for such evaluations. Experiences after completing this work imply that personas indeed is a viable way to include aspects and raise issues concerning individual variability and situational context in ADAS scenarios.

Working with two small group of domain experts I evaluate human-centered approaches to application development which are applicable to geovisualization, following an ISO13407 taxonomy that covers context of use, eliciting requirements, and design. These approaches include field studies and contextual analysis of subjects' context; establishing requirements using a template, via a lecture to communicate geovisualization to subjects and by communicating subjects' context to geovisualization experts with a scenario; autoethnography to understand the geovisualization design process; wireframe, paper and digital interactive prototyping with alternative protocols; and a decision making process for prioritising application improvement. I find that the acquisition and use of real user data is key; that a template approach and teaching subjects about visualization tools and interactions both fail to elicit useful requirements for a visualization application. Consulting geovisualization experts with a scenario of user context and samples of user data does yield suggestions for tools and interactions of use to a visualization designer. The complex and composite natures of both visualization and human-centered domains, incorporating learning from both domains, with user context, makes design challenging. Wireframe, paper and digital interactive prototypes mediate between the user and visualization domains successfully, eliciting exploratory behaviour and suggestions to improve prototypes. Paper prototypes are particularly successful at eliciting suggestions and especially novel visualization improvements. Decision-making techniques prove useful for prioritising different possible improvements, although domain subjects select data-related features over more novel alternative and rank these more inconsistently. The research concludes that understanding subject context of use and data is important and occurs throughout the process of engagement with domain experts, and that standard requirements elicitation techniques are unsuccessful for geovisualization. Engagement with subjects at an early stage with simple prototypes incorporating real subject data and moving to successively more complex prototypes holds the best promise for creating successful geovisualization applications.

To date, hundreds of millions of mini-QWERTY keyboard equipped devices (miniaturized versions of a full desktop keyboard) have been sold. Accordingly, a large percentage of text messages originate from fixed-key, mini-QWERTY keyboard enabled mobile phones. Over a series of three longitudinal studies I quantify how quickly and accurately individuals can input text on mini-QWERTY keyboards. I evaluate performance in ideal laboratory conditions as well as in a variety of mobile contexts. My first study establishes baseline performance measures; my second study investigates the impact of limited visibility on text input performance; and my third study investigates the impact of mobility (sitting, standing, and walking) on text input performance. After approximately five hours of practice, participants achieved expertise typing almost 60 words per minute at almost 95% accuracy. Upon completion of these studies, I examine the types of errors that people make when typing on mini-QWERTY keyboards. Having discovered a common pattern in errors, I develop and refine an algorithm to automatically detect and correct errors in mini-QWERTY keyboard enabled text input. I both validate the algorithm through the analysis of pre-recorded typing data and then empirically evaluate the impacts of automatic error correction on live mini-QWERTY keyboard text input. Validating the algorithm over various datasets, I demonstrate the potential to correct approximately 25% of the total errors and correct up to 3% of the total keystrokes. Evaluating automatic error detection and correction on live typing results in successfully correcting 61% of the targeted errors committed by participants while increasing typing rates by almost two words per minute without introducing noticeable distraction.

The expected quality on manufactured components in the automotive industry is high, often with an accuracy of tenths of a millimeter. The conventional methods used to ensure the manufactured components are very accurate, but they are both time consuming and insufficient and only a small part of the produced series are analyzed today. The measurement is performed manually in so-called measurement fixtures. Where each component is fixed and predetermined points of investigation are controlled with a dial indicator. These fixtures are very expensive to manufacture and they are only compatible with one specific kind of component. Nowadays, great volumes of material are scrapped from these procedures in the automotive industry. Hence, there is a great need to increase the amount of controlled components without affecting the production rate negatively. This project was carried out for the relatively new company Viospatia, which is a spin-off company based on research from Luleå University of Technology. They have developed a system that automatically measures each component directly at the production line with the use of photogrammetry technology. This makes it possible to discover erroneous components almost immediately and the manufacturer gets a more distinct view of their production and its capability. The aim of this thesis has been to investigate how a user interface should be developed to be as user-friendly as possible without limiting the system’s functions. The objective has been to design a proposal of a user interface adapted for the intended user, creating value and is easy to use. The progression has been structured around a human-centered approach expedient for interaction design, where the developing phase, containing analyze, design and validate, is performed through iterations with continuous feedback from users and the project’s employer. The context, where the intended solution is supposed to be used, was investigated through interviews and observations at the involved companies. In the project there were three factories involved, Gestamp Hardtech and Scania Ferruform in Luleå and Volvo Cars in Olofström. These factories are using similar production methods, sheet metal stamping, so their prerequisites and needs are similar for this type of quality control system. Creative methods have been applied throughout the project to generate as much ideas as possible while trying to satisfy all the important aspects. Initially analog prototypes were created but they were soon developed to digital interactive prototypes. A larger usability-test was conducted with seven participants by using a weblink to the digital prototype. With support from the feedback these tests generated some adjustments were made and the final user interface was designed, separated in two levels - Supervisor and Operator. Through extensive literature study and user-testing it became clear that the operator needs to get an unmistakable message from the user interface. There should not be any doubts whatsoever and the operator should react immediately. This message is delivered with the use of colors that have an established meaning. By identifying what needs the different actors have, the system’s functions can be separated and made accessible only for the intended user. The functions can then be more specifically developed for the intended user instead of modifying them trying to make a compromise that fits everybody. This separation of functions is not anything the user has to actively do but it is performed automatically by the user interface when the user is signing in.
Den förväntade kvalitén på tillverkade delar inom bilindustrin är väldigt hög, med toleranser på så lite som tiondels millimeter många gånger. De konventionella metoderna som används för att kontrollmäta de tillverkade delarna idag är mycket noggranna, men de är både tidskrävande och otillräckliga och endast en väldigt liten del av en producerad serie blir kontrollmätt idag. Mätningen utförs manuellt i så kallade mätfixturer. Där varje komponent fixeras och förutbestämda undersökningspunkter kontrolleras med en så kallad mätklocka. Dessa fixturer är även väldigt dyra att tillverka och de är bara kompatibla med en specifik komponent. I dagens läge så kasseras otroligt stora mängder material från dessa komponenter inom bilindustrin. Här finns det alltså ett stort behov för att öka mängden komponenter som kontrolleras utan att påverka tillverkningstakten. Det här projektet utfördes åt det relativt nystartade företaget Viospatia, vilket är ett spin-off företag från forskning utförd vid Luleå tekniska universitet. De har utvecklat ett system som med hjälp av fotogrammetri automatiskt mäter av varje komponent direkt i produktionslinan. Detta gör att eventuella fel upptäcks nästan omedelbart samtidigt som tillverkaren får en tydligare bild av sin produktion och dess kapacitet. Syftet med denna masteruppsats har varit att undersöka hur ett gränssnitt bör utvecklas för att det ska bli så användarvänligt som möjligt utan att begränsa systemets viktiga funktioner. Målet har varit att ta fram ett förslag på ett gränssnitt som är anpassat för den tänkta användaren, som skapar ett mervärde och är enkelt att använda. Processen har följt en användarcentrerad struktur fördelaktig för interaktionsdesign, där utvecklingsfasen bestående av analys, design och validering sker i flera iterationer med kontinuerlig återkoppling med användare och uppdragsgivare. Kontexten, där den tänkta lösningen ska användas, undersöktes initialt hos de involverade företagen. I projektet var tre fabriker involverade, Gestamp Hardtech och Scania Ferruform i Luleå och Volvo Cars i Olofström. Dessa fabriker använder mestadels liknande tillverkningsmetoder, metallpressning, vilket gör att de rimligtvis har en del gemensamma förutsättningar och behov. Under arbetets gång har diverse kreativa metoder använts för att generera så mycket idéer som möjligt utan att förbise viktiga aspekter. Till en början utvecklades prototyper analogt för att sedan utvecklas till digitala interaktiva prototyper. Ett större användbarhetstest genomfördes på distans med sju testpersoner via en länk till den digitala prototypen. Med hjälp av responsen från dessa tester gjordes en del ändringar och den slutliga designen på gränssnittet blev uppdelat i två nivåer, Supervisor och Operator. Genom teoristudie och användartester framgick det att operatören behöver få en omisskännlig uppmaning från gränssnittet. Det bör inte uppstå några som helst tveksamheter och operatören skall kunna agera direkt. Denna uppmaning sker genom en tydlig färgkodning som utnyttjar vedertagna uppfattningar om färgers innebörd. Genom att identifiera vilka behov de olika aktörerna har kan man på så sätt också hålla isär de olika funktionerna och göra de tillgängliga endast för den typen av aktör som behöver de. De kan på så sätt också utvecklas mer specifikt för den tänkta aktören istället för att modifieras för att passa alla. Denna separering av funktioner är inget som användaren behöver ställa in själv utan görs automatiskt då den loggar in med sitt användarkonto.

This study aimed to develop a help system for a communication app, identify usability issues regarding that help system and develop redesigns to improve it. The focus was to maximize perceived usefulness and minimize perceived annoyance of the help system. During the study two design proposals were developed and two user tests, which used low-fidelity prototypes. were performed to evaluate the design proposals. The first design proposal was evaluated in the first user test, thereafter an iteration of that design was developed based on the usability issues found in the user test. This iteration of the design was the second design proposal which were evaluated in the second user tests. Both design proposals and data from both user tests were analysed to- gether which resulted in seven recommendation that aimed to maximize perceived usefulness and minimize perceived annoyance when developing a help system for a communication app. Due to a lack of generalizability these recommendations should however be used with caution since they are mainly applicable to the system evaluated in this study. They can however be used as an inspi- ration and a starting point for someone designing a help system for an other communication app.

Thesis (Ph. D.)--Ohio State University, 2004.
Title from first page of PDF file. Document formatted into pages; contains xi, 170 p.; also includes graphics. Includes abstract and vita. Advisor: Brenda Dervin, Communication Graduate Program. Includes bibliographical references (p. 135-144).

Recent advances in mobile Augmented Reality (AR) create promising opportunities for novel, engaging, potentially scalable, interactive products. However, evaluating the usability and UX of mobile AR can be tricky given that the interface is highly adaptable, often ambiguous, and varies depending on context. This project will recount the evaluation of a foot scanner that is guided via mobile AR interaction, in doing so offering one way to approach mobile AR evaluation. Some themes that will be discussed are the importance of iterative in-situ evaluation and the benefits of both in-lab and remote usability testing, as well as how to identify user insights given the perceptual complexity and technical ambiguity of mobile AR interaction. Furthermore, design implications to enhance the usability and experiential aspects for mobile AR will be proposed, including how to guide and instruct users, techniques for movement-based interaction design, and how to utilize the various input types in mobile AR. Lastly, an overview of inserting mobile AR user-evaluation in an Agile product environment, including iterative lo-fi testing, insight communication, team collaboration, and the use of evaluation measures, will be discussed. The hopes of this project is to provide a valuable resource to help push future mobile AR designers and evaluators in the right direction.
Nya framsteg inom mobil Augmented Reality (AR) skapar lovande möjligheter for nya, engagerande, intuitiva och potentiellt skalbara interaktiva produkter. Att utvärdera användbarhet och UX for mobil AR kan dock vara svart eftersom gränssnittet ar mycket anpassningsbart, ofta tvetydigt och varierar beroende på sammanhang. Detta projekt kommer att beskriva utvärderingen av en fotscanner som styrs via mobil AR-interaktion och därmed medföra ett satt att närma sig mobil AR-utvärdering. Några teman som kommer att diskuteras ar vikten av iterativ in-situ-utvärdering, fördelarna med användbarhetstestning i laboratorium och på distans, samt hur man identifierar användar-insikter i relation till den perceptuella komplexiteten och tekniska oklarheten i mobil AR-interaktion. Dessutom föreslås designimplikationer for att förbättra användbarhet och erfarenhetsaspekter for mobil AR, inkluderande hur användare kan vägledas och instrueras, tekniker for rörelsebaserad interaktionsdesign och hur man använder de olika ingångstyperna i mobil AR. Slutligen diskuteras en översikt kring att införa mobil AR-användarutvärdering i en agil produktmiljö, inklusive iterativt lo-fi-testande, insikts-kommunikation, samarbete i teams och användningen av utvärderingsmetoder. Förhoppningarna med detta projekt ar att tillhandahålla en värdefull resurs for att hjälpa till att driva framtida mobila AR-designers och utvärderare i ratt riktning.

Thesis (Ph.D)--Industrial and Systems Engineering, Georgia Institute of Technology, 2010.
Committee Chair: Kemp, Charles; Committee Member: Glass, Jonathan; Committee Member: Griffin, Paul; Committee Member: Howard, Ayanna; Committee Member: Thomaz, Andrea. Part of the SMARTech Electronic Thesis and Dissertation Collection.

The nursing discipline lacks a consensus on a best practice method for debriefing students following simulation-based training. A recognized, standardized method does not exist and various methods are utilized within the domain. The similarities between aviation and healthcare are well documented. Training members of both disciplines require standardization and methods of best practice. The aviation industry through the Federal Aviation Administration has found Learner Centered Grading (LCG) to be a successful educational format. The utilization of the LCG Debriefing method in simulation-based training is the standardized debriefing format for a technologically dynamic industry. The aim of this research was to examine the LCG debriefing approach and determine the added value of the approach using a scenario-specific behavioral checklist as an instrument for the nursing faculty and the learner to assess the learner’s performance. A repeated measures was conducted to evaluate whether there were differences between the control and treatment groups across the pre and post-test. The test statistic demonstrated no statistical significance between the control and treatment groups. Results of Pearson’s correlations showed that self-efficacy was not significantly correlated with change in performance by debriefing method. A number of factors contribute to this finding, one of which is the small sample size. The small sample size led to insufficient power to detect an effect if one did exist. Other factors included time allotted for data gathering, simulation space availability and participants’ prior exposure to the control debriefing method. This study served as a pilot for future research. Implications for the next study include extending the time allotted for gathering data to allow for a larger sample size, utilizing the Certified Healthcare Simulation Educator (CHSE) designees to function as facilitators as well as evaluators and to design the study to evaluate performance immediately after the debriefing session and once again at a different interval of time. A second simulation session conducted one week after the initial participation would be beneficial to evaluate if knowledge acquisition occurred.

Responding to the need within the human-computer interaction field to address ubiquitous and multitasking systems more scientifically, this research extends the usefulness of a new research framework for a particular class of systems. Notification systems are interfaces used in a divided-attention, multitasking situation, attempting to deliver current, valued information through a variety of platforms and modes in an efficient and effective manner. Through review of literature and experiences with empirical dual-task perceptual studies, we recognize a lack of unifying framework for understanding, classifying, analyzing, developing, evaluating, and discussing notification systems--fundamentally inhibiting scientific growth and knowledge reuse that should help designers advance the state-of-the-art. To this end, we developed a framework (referred to as the IRC framework) for notification systems research based on a core taxonomy of critical parameters describing user goals. Next, we extend the framework, focusing on three key aspects: 1) a system description process, allowing articulation of abstract design objectives that focus on critical user requirements; 2) interface usability evaluation tools, enabling comparison of the design and user's models, while supporting generalizability of research and early identification of usability concerns; and 3) design comparison and reuse mechanisms, saving time and effort in requirements analysis and early design stages by enabling design reuse and appreciation of design progress. Results from this research include the development of tools to express IRC design models (IRCspec) and user's models (IRCresults), and the extension of the critical parameters concept. Validation studies with novice designers show sufficient assessment accuracy and consistency. Leveraging these tools that help designers express abstract, yet critical, design intentions and effects as classification and retrieval indices, we develop a repository for reusable design knowledge (a claims library). Responding to challenges of design knowledge access that we observed through initial user testing, we introduce a vision for an integrated design environment (LINK-UP) to operationalize the IRC framework and notification systems claims library in a computer-aided design support system. Proof-of-concept testing results encourage the thought that when valuable design tools embody critical parameters and are coupled with readily accessible reusable design knowledge, interface development will improve as a scientific endeavor.
Ph. D.

L’aéronautique civile commerciale poursuit l’objectif du déplacement de biens ou de personnes, par les airs, en maintenant un niveau optimal de sécurité. Depuis plus de trente ans, en dépit de cadres normatifs de plus en plus stricts et d’automatismes de plus en plus performants, le rapport entre performance visée et risque encouru ne progresse plus.Le facteur humain constitue un levier d’action majeur pour franchir ce plancher de verre. Dans le cadre contraint de l’aéronautique, la conception d’outils visant à assister la cognition des pilotes est ainsi une direction d’avenir. L’anticipation a été identifiée comme un processus central dans la gestion des ressources cognitives. Dans une démarche de cognitique, nous avons ainsi entrepris la conception d’un outil d’aide à l’anticipation en impliquant des pilotes à chaque étape des développements.D’une analyse de l’activité sur la base d’enregistrements en cockpit et d’entretiens, nous avons construit un modèle de l’activité réelle des pilotes lors de la descente et l’approche sur l’aéroport de Rio de Janeiro. L’étude bibliographique mit en lumière des points critiques relevant de l’anticipation et nécessitant une expérimentation préliminaire. Les résultats expérimentaux conciliés à nos hypothèses de compréhension de l’anticipation achevèrent le cahier des charges du cœur fonctionnel de notre outil d’aide à l’anticipation. Un algorithme de planification dynamique exploitant notre modèle de l’activité fut conçu et implémenté au sein d’ASAP (Anticipation Support for Aeronautical Planning) le démonstrateur de concept industriel de Thales Avionics. 36 pilotes civils commerciaux participèrent enfin à son évaluation en simulateur
Civil aviation pursues the objective of moving people or goods through the air with an optimal level of safety. For more than thirty years, despite a stricter and stricter regulatory framework and highly reliable automation, the ratio between performance and acceptable risk is not improving anymore.Human factors are a major action lever to break this glass floor. In the constrained context of aviation, designing tools aiming at assisting pilots’ cognition is thus a promising direction. Anticipation has been identified central in the process of cognitive resources management. In a human factors engineering approach, we undertook the design of an anticipation support tool involving pilots at each step of the development.From an activity analysis performed on the basis of in-cockpit recordings and interviews we constructed a model of the actual pilots’ activity during the descent and approach phases on Rio de Janeiro airport. The state of the art highlighted the key elements related to anticipation which could take benefit of a preliminary experiment. Experimental results brought together with our hypotheses about how anticipation works completed the requirements of the functional core of our anticipation support tool. A dynamic planning algorithm was then designed and implemented within ASAP (Anticipation Support for Aeronautical Planning), Thales Avionics’ proof of concept. 36 commercial pilots took part to its evaluation in a simulated environment

As the technologies evolve, more complex products such as real-time web applications are made. These applications introduce new usability challenges and interaction design opportunities. The purpose of this work was to evaluate and enhance the usability of a complex real-time web user interface using a User-centered design approach. Another aim was to investigate how to adapt the current usability research methods, guidelines and principles to today's modern technology, such as real-time web applications. The research was carried out for the Stockholm based start-up Challengermode AB that is currently developing a competitive gaming platform with a focus in automatic tournaments. The research method consisted of a user and system analysis, usability testing and a creative design workshop. The research results were used in combination with current user interface design guidelines and principles to improve and design new features to the current user interface. The results show that the User-centered design approach in combination with Participatory design was a successful way of evaluating and improving the usability of the Challengermode real-time web user interface. The current usability research and design guidelines are applicable to modern technology, but in order to evaluate the Challengermode real-time web user interface, and other applications with similar functionality in a realistic scenario, traditional usability testing can be limited. Furthermore, the main takeaways from the user interface improvement process is that designing for empty states and not hiding important elements even though they are not active is as important as designing for the optimal case where data is available.
I och med att tekniken förbättras utvecklas allt mer komplexa produkter såsom realtidswebbapplikationer. Dessa applikationer inför nya utmaningar och möjligheter inom användbarhet och interaktionsdesign. Syftet med detta arbete var att utvärdera och förbättra användbarheten av ett komplex realtidswebbgränssnitt med hjälp av en användarcentrerad designstrategi. Ett annat syfte var att undersöka hur man kan anpassa existerande metoder, riktlinjer och principer inom användbarhetsforskning och interaktionsdesign för dagens moderna teknik, såsom realtidswebbapplikationer. Arbetet genomfördes för det Stockholm baserade start-up företaget Challengermode AB som för närvarande utvecklar en tävlingsplattform för eSport med fokus på automatiska turneringar. Forskningsmetoden bestod av en användar- och systemanalys, användbarhetstester och en kreativ designworkshop. Forskningsresultaten användes sedan i kombination med riktlinjer och principer inom interaktionsdesign för att förbättra och utveckla nya funktioner för det aktuella användargränssnittet. Resultaten visar att en användarcentrerad designstrategi i kombination med kooperativ design var ett framgångsrikt sätt att utvärdera och förbättra användbarheten av Challengermodes realtidswebbgränssnitt. Dagens forskning inom användbarhet och designriktlinjer är applicerbara på modern teknik, men för att utvärdera Challengermodes realtidswebbgränssnitt, och andra liknande applikationer i ett realistiskt scenario, kan traditionella användbarhetstester vara begränsade. De viktigaste slutsatserna från designprocessen är att design för element i tomma tillstånd är lika viktigt som design för element i det optimala tillståndet där data är tillgänglig.

Dissertation submitted in partial fulfilment of the requirements for the degree of Master of Science in Geospatial Technologies
Feature info is an interactive map tool that allows viewing background information about a map feature in response to a user action. Mobile devices come with a number of limitations, e.g. small screen real estate and the variety of screen sizes, that can affect map and feature info usability. Existing recommendations for feature info design [1], [2], [3] focus mostly on content, i.e. effective communication of data, leaving the “interaction design” aspect overlooked. This Master’s Thesis attempts at improving the visualization and usability of feature info interfaces in the context of mobile tourist applications and presents nine feature info design guidelines that address specific usability problems. The problems were identified through heuristic evaluation of five tourist applications and a user interview. Literature survey and two usability experts provided ideas regarding how several of these problems can be resolved. Three guidelines were evaluated in a lab-based usability test with twenty participants. On average, assessed guidelines demonstrated a significant positive effect on feature info usability by decreasing task completion time by 33% and increasing task completion rate and System Usability Scale (SUS) score by 26% and 28% respectively. Proposed guidelines are not restricted by any specific use case and can be applied to other application domains. Researchers and business practitioners can use the guidelines as a reference in their daily work.

Indiana University-Purdue University Indianapolis (IUPUI)
Our connections to the digital world are invoked by brands, but the intersection of branding and interaction design is still an under-investigated area. Particularly, current websites are designed not only to support essential user tasks, but also to communicate an institution's intended brand values and traits. What we do not yet know, however, is which design factors affect which aspect of a brand. To demystify this issue, three sub-projects were conducted. The first project developed a systematic approach for evaluating the branding effectiveness of content-intensive websites (BREW). BREW gauges users' brand perceptions on four well-known branding constructs: brand as product, brand as organization, user image, and brand as person. It also provides rich guidelines for eBranding researchers in regard to planning and executing a user study and making improvement recommendations based on the study results. The second project offered a standardized perceived usability questionnaire entitled DEEP (design-oriented evaluation of perceived web usability). DEEP captures the perceived website usability on five design-oriented dimensions: content, information architecture, navigation, layout consistency, and visual guidance. While existing questionnaires assess more holistic concepts, such as ease-of-use and learnability, DEEP can more transparently reveal where the problem actually lies. Moreover, DEEP suggests that the two most critical and reliable usability dimensions are interface consistency and visual guidance. Capitalizing on the BREW approach and the findings from DEEP, a controlled experiment (N=261) was conducted by manipulating interface consistency and visual guidance of an anonymized university website to see how these variables may affect the university's image. Unexpectedly, consistency did not significantly predict brand image, while the effect of visual guidance on brand perception showed a remarkable gender difference. When visual guidance was significantly worsened, females became much less satisfied with the university in terms of brand as product (e.g., teaching and research quality) and user image (e.g., students' characteristics). In contrast, males' perceptions of the university's brand image stayed the same in most circumstances. The reason for this gender difference was revealed through a further path analysis and a follow-up interview, which inspired new research directions to unpack even more the nexus between branding and interaction design.

Indiana University-Purdue University Indianapolis (IUPUI)
Pedestrian and vehicle interaction analysis in a naturalistic driving environment can provide useful information for designing vehicle-pedestrian crash warning/mitigation systems. Many researchers have used crash data to understand and study pedestrian behaviors and interactions between vehicles and pedestrian during crash. However, crash data may not provide detailed pedestrian-vehicle interaction information for us. In this thesis, we designed an automatic pedestrian tracking and status analysis method to process and study pedestrian and vehicle interactions. The proposed pedestrian tracking and status analysis method includes pedestrian detection, pedestrian tracking and pedestrian status analysis modules. The main contributions of this thesis are: we designed a new pedestrian tracking method by learning the pedestrian appearance and also their motion pattern. We designed a pedestrian status estimation method by using our tracking results and thus helped estimate the possibility of collision. Our preliminary experiment results using naturalistic driving data showed promising results.