Academic literature on the topic 'First-person hand activity recognition'
Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles
Consult the lists of relevant articles, books, theses, conference reports, and other scholarly sources on the topic 'First-person hand activity recognition.'
Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.
You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.
Journal articles on the topic "First-person hand activity recognition"
1
Medarevic, Jelena, Marija Novicic, and Marko Markovic. "Feasibility test of activity index summary metric in human hand activity recognition." Serbian Journal of Electrical Engineering 19, no. 2 (2022): 225–38. http://dx.doi.org/10.2298/sjee2202225m.
Full textAbstract:
Activity monitoring is a technique for assessing the physical activity that a person undertakes over some time. Activity Index (AI) is a metric that summarizes the raw measurements from tri-axial accelerometers, often used for measuring physical activity. Our research compared the Activity Index for different activity groups and hand usage [1]. We also tested this metric as a classification feature, and how different data acquisition and segmentation parameter configurations influence classification accuracy. Data acquisition was done with a previously developed system that includes a smartwatch on each wrist and a smartphone placed in the subject?s pocket; raw data from smartwatch accelerometers was used for the analysis. We calculated the Activity Index for labeled data segments and used ANOVA1 statistical test with Bonferroni correction. Significant differences were found between cases of hand usage (left, right, none, both). In the next analysis phase, the Activity Index was used as the classification feature with three supervised machine learning algorithms-Support Vector Machine, k-Nearest Neighbors, and Random Forest. The best accuracy (measured by F1 score) of classifying hand usage was achieved by using the Random Forest algorithm, 50 Hz sampling frequency, and a window of 10 s without overlap for AI calculation, and it was 97%. On the other hand, the classification of activity groups had a low accuracy, which indicated that a specific activity group can?t be identified by using only one simple feature.
2
Senyurek, Volkan, Masudul Imtiaz, Prajakta Belsare, Stephen Tiffany, and Edward Sazonov. "Electromyogram in Cigarette Smoking Activity Recognition." Signals 2, no. 1 (February 9, 2021): 87–97. http://dx.doi.org/10.3390/signals2010008.
Full textAbstract:
In this study, information from surface electromyogram (sEMG) signals was used to recognize cigarette smoking. The sEMG signals collected from lower arm were used in two different ways: (1) as an individual predictor of smoking activity and (2) as an additional sensor/modality along with the inertial measurement unit (IMU) to augment recognition performance. A convolutional and a recurrent neural network were utilized to recognize smoking-related hand gestures. The model was developed and evaluated with leave-one-subject-out (LOSO) cross-validation on a dataset from 16 subjects who performed ten activities of daily living including smoking. The results show that smoking detection using only sEMG signal achieved an F1-score of 75% in person-independent cross-validation. The combination of sEMG and IMU improved reached the F1-score of 84%, while IMU alone sensor modality was 81%. The study showed that using only sEMG signals would not provide superior cigarette smoking detection performance relative to IMU signals. However, sEMG improved smoking detection results when combined with IMU signals without using an additional device.
3
Ramirez, Heilym, Sergio A. Velastin, Paulo Aguayo, Ernesto Fabregas, and Gonzalo Farias. "Human Activity Recognition by Sequences of Skeleton Features." Sensors 22, no. 11 (May 25, 2022): 3991. http://dx.doi.org/10.3390/s22113991.
Full textAbstract:
In recent years, much effort has been devoted to the development of applications capable of detecting different types of human activity. In this field, fall detection is particularly relevant, especially for the elderly. On the one hand, some applications use wearable sensors that are integrated into cell phones, necklaces or smart bracelets to detect sudden movements of the person wearing the device. The main drawback of these types of systems is that these devices must be placed on a person’s body. This is a major drawback because they can be uncomfortable, in addition to the fact that these systems cannot be implemented in open spaces and with unfamiliar people. In contrast, other approaches perform activity recognition from video camera images, which have many advantages over the previous ones since the user is not required to wear the sensors. As a result, these applications can be implemented in open spaces and with unknown people. This paper presents a vision-based algorithm for activity recognition. The main contribution of this work is to use human skeleton pose estimation as a feature extraction method for activity detection in video camera images. The use of this method allows the detection of multiple people’s activities in the same scene. The algorithm is also capable of classifying multi-frame activities, precisely for those that need more than one frame to be detected. The method is evaluated with the public UP-FALL dataset and compared to similar algorithms using the same dataset.
4
Ray, Sujan, Khaldoon Alshouiliy, and Dharma P. Agrawal. "Dimensionality Reduction for Human Activity Recognition Using Google Colab." Information 12, no. 1 (December 23, 2020): 6. http://dx.doi.org/10.3390/info12010006.
Full textAbstract:
Human activity recognition (HAR) is a classification task that involves predicting the movement of a person based on sensor data. As we can see, there has been a huge growth and development of smartphones over the last 10–15 years—they could be used as a medium of mobile sensing to recognize human activity. Nowadays, deep learning methods are in a great demand and we could use those methods to recognize human activity. A great way is to build a convolutional neural network (CNN). HAR using Smartphone dataset has been widely used by researchers to develop machine learning models to recognize human activity. The dataset has two parts: training and testing. In this paper, we propose a hybrid approach to analyze and recognize human activity on the same dataset using deep learning method on cloud-based platform. We have applied principal component analysis on the dataset to get the most important features. Next, we have executed the experiment for all the features as well as the top 48, 92, 138, and 164 features. We have run all the experiments on Google Colab. In the experiment, for the evaluation of our proposed methodology, datasets are split into two different ratios such as 70–10–20% and 80–10–10% for training, validation, and testing, respectively. We have set the performance of CNN (70% training–10% validation–20% testing) with 48 features as a benchmark for our work. In this work, we have achieved maximum accuracy of 98.70% with CNN. On the other hand, we have obtained 96.36% accuracy with the top 92 features of the dataset. We can see from the experimental results that if we could select the features properly then not only could the accuracy be improved but also the training and testing time of the model.
5
Gao, Zhiqiang, Dawei Liu, Kaizhu Huang, and Yi Huang. "Context-Aware Human Activity and Smartphone Position-Mining with Motion Sensors." Remote Sensing 11, no. 21 (October 29, 2019): 2531. http://dx.doi.org/10.3390/rs11212531.
Full textAbstract:
Today’s smartphones are equipped with embedded sensors, such as accelerometers and gyroscopes, which have enabled a variety of measurements and recognition tasks. In this paper, we jointly investigate two types of recognition problems in a joint manner, e.g., human activity recognition and smartphone on-body position recognition, in order to enable more robust context-aware applications. So far, these two problems have been studied separately without considering the interactions between each other. In this study, by first applying a novel data preprocessing technique, we propose a joint recognition framework based on the multi-task learning strategy, which can reduce computational demand, better exploit complementary information between the two recognition tasks, and lead to higher recognition performance. We also extend the joint recognition framework so that additional information, such as user identification with biometric motion analysis, can be offered. We evaluate our work systematically and comprehensively on two datasets with real-world settings. Our joint recognition model achieves the promising performance of 0.9174 in terms of F 1 -score for user identification on the benchmark RealWorld Human Activity Recognition (HAR) dataset. On the other hand, in comparison with the conventional approach, the proposed joint model is shown to be able to improve human activity recognition and position recognition by 5.1 % and 9.6 % respectively.
6
Guo, Jiang, Jun Cheng, Yu Guo, and Jian Xin Pang. "A Real-Time Dynamic Gesture Recognition System." Applied Mechanics and Materials 333-335 (July 2013): 849–55. http://dx.doi.org/10.4028/www.scientific.net/amm.333-335.849.
Full textAbstract:
In this paper, we present a dynamic gesture recognition system. We focus on the visual sensory information to recognize human activity in form of hand movements from a small, predefined vocabulary. A fast and effective method is presented for hand detection and tracking at first for the trajectory extraction. A novel trajectory correction method is applied for simply but effectively trajectory correction. Gesture recognition is achieved by means of a matching technique by determining the distance between the unknown input direction code sequence and a set of previously defined templates. A dynamic time warping (DTW) algorithm is used to perform the time alignment and normalization by computing a temporal transformation allowing the two signals to be matched. Experiment results show our proposed gesture recognition system achieve well result in real time.
7
Bieck, Richard, Reinhard Fuchs, and Thomas Neumuth. "Surface EMG-based Surgical Instrument Classification for Dynamic Activity Recognition in Surgical Workflows." Current Directions in Biomedical Engineering 5, no. 1 (September 1, 2019): 37–40. http://dx.doi.org/10.1515/cdbme-2019-0010.
Full textAbstract:
AbstractWe introduce a wearable-based recognition system for the classification of natural hand gestures during dynamic activities with surgical instruments. An armbandbased circular setup of eight EMG-sensors was used to superficially measure the muscle activation signals over the broadest cross-section of the lower arm. Instrument-specific surface EMG (sEMG) data acquisition was performed for 5 distinct instruments. In a first proof-of-concept study, EMG data were analyzed for unique signal courses and features, and in a subsequent classification, both decision tree (DTR) and shallow artificial neural network (ANN) classifiers were trained. For DTR, an ensemble bagging approach reached precision and recall rates of 0.847 and 0.854, respectively. The ANN network architecture was configured to mimic the ensemble-like structure of the DTR and achieved 0.952 and 0.953 precision and recall rates, respectively. In a subsequent multi-user study, classification achieved 70 % precision. Main errors potentially arise for instruments with similar gripping style and performed actions, interindividual variations in the acquisition procedure as well as muscle tone and activation magnitude. Compared to hand-mounted sensor systems, the lower arm setup does not alter the haptic experience or the instrument gripping, which is critical, especially in an intraoperative environment. Currently, drawbacks of the fixed consumer product setup are the limited data sampling rate and the denial of frequency features into the processing pipeline.
8
Bragin, A. D., and V. G. Spitsyn. "Motor imagery recognition in electroencephalograms using convolutional neural networks." Computer Optics 44, no. 3 (June 2020): 482–87. http://dx.doi.org/10.18287/2412-6179-co-669.
Full textAbstract:
Electroencephalography is a widespread method to record brain signals with the use of electrodes located on the surface of the head. This method of recording the brain activity has become popular because it is relatively cheap, compact, and does not require implanting the electrodes directly into the brain. The article is devoted to a problem of recognition of motor imagery by electroencephalogram signals. The nature of such signals is complex. Characteristics of electroencephalograms are individual for every person, also depending on their age and mental state, as well as the presence of noise and interference. The multitude of these parameters should be taken into account when analyzing encephalograms. Artificial neural networks are a good tool for solving this class of problems. Their application allows combining the tasks of extracting, selecting and classifying features in one signal processing unit. Electroencephalograms are time signals and we note that Gramian Angular Fields and Markov Transition Field transforms are used to represent time series in the form of images. The article shows the possibility of using the Gramian Angular Fields and Markov Transition Field transformations of the electroencephalogram (EEG) signal for motor imagery recognition using examples of imaginary movements with the right and left hand, also studying the effect of the resolution of Gramian Angular Fields and Markov Transition Field images on the classification accuracy. The best classification accuracy of the EEG signal into the motion and state-of-rest classes is about 99%. In future, the research results can be applied in constructing the brain-computer interface.
9
Liu, Dan, Mao Ye, and Jianwei Zhang. "Improving Action Recognition Using Sequence Prediction Learning." International Journal of Pattern Recognition and Artificial Intelligence 34, no. 12 (March 20, 2020): 2050029. http://dx.doi.org/10.1142/s0218001420500299.
Full textAbstract:
Skeleton-based action recognition distinguishes human actions using the trajectories of skeleton joints, which can be a good representation of human behaviors. Conventional methods usually construct classifiers with hand-crafted or the learned features to recognize human actions. Different from constructing a direct action classifier for action recognition task, this paper attempts to identify human actions based on the development trends of behavior sequences. Specifically, we first utilize the memory neural network to construct action predictors for each kind of activity. These action predictors can then output the action trends at the next time step. According to the predictions of these action predictors at each time step and the removal rule, the poor predictors can be eliminated step by step, and the IDentity(ID) number of the last predictor left is considered as the label of the action sequence to be categorized. We compare the proposed action recognition algorithm using sequence prediction learning with other methods on two publicly available datasets. Our experimental results consistently demonstrate the feasibility and effectiveness of the suggested method. It also proves the importance of prediction learning for action recognition.
10
Yin, Guanghao, Shouqian Sun, Dian Yu, Dejian Li, and Kejun Zhang. "A Multimodal Framework for Large-Scale Emotion Recognition by Fusing Music and Electrodermal Activity Signals." ACM Transactions on Multimedia Computing, Communications, and Applications 18, no. 3 (August 31, 2022): 1–23. http://dx.doi.org/10.1145/3490686.
Full textAbstract:
Considerable attention has been paid to physiological signal-based emotion recognition in the field of affective computing. For reliability and user-friendly acquisition, electrodermal activity (EDA) has a great advantage in practical applications. However, EDA-based emotion recognition with large-scale subjects is still a tough problem. The traditional well-designed classifiers with hand-crafted features produce poorer results because of their limited representation abilities. And the deep learning models with auto feature extraction suffer the overfitting drop-off because of large-scale individual differences. Since music has a strong correlation with human emotion, static music can be involved as the external benchmark to constrain various dynamic EDA signals. In this article, we make an attempt by fusing the subject’s individual EDA features and the external evoked music features. And we propose an end-to-end multimodal framework, the one-dimensional residual temporal and channel attention network (RTCAN-1D). For EDA features, the channel-temporal attention mechanism for EDA-based emotion recognition is first involved in mine the temporal and channel-wise dynamic and steady features. The comparisons with single EDA-based SOTA models on DEAP and AMIGOS datasets prove the effectiveness of RTCAN-1D to mine EDA features. For music features, we simply process the music signal with the open-source toolkit openSMILE to obtain external feature vectors. We conducted systematic and extensive evaluations. The experiments on the current largest music emotion dataset PMEmo validate that the fusion of EDA and music is a reliable and efficient solution for large-scale emotion recognition.
Dissertations / Theses on the topic "First-person hand activity recognition"
1
Boutaleb, Mohamed Yasser. "Egocentric Hand Activity Recognition : The principal components of an egocentric hand activity recognition framework, exploitable for augmented reality user assistance." Electronic Thesis or Diss., CentraleSupélec, 2022. http://www.theses.fr/2022CSUP0007.
Full textAbstract:
Les êtres humains utilisent leurs mains pour diverses tâches dans la vie quotidienne et professionnelle, ce qui fait que la recherche dans ce domaine a récemment suscitée un grand intérêt. De plus, l'analyse et l'interprétation du comportement humain à l'aide de signaux visuels est l'un des domaines les plus actifs et les plus explorés de la vision par ordinateur. Avec l'arrivée des nouvelles technologies de réalité augmentée, les chercheurs s'intéressent de plus en plus à la compréhension de l'activité de la main d'un point de vue de la première personne, en explorant la pertinence de son utilisation pour le guidage et l'assistance humaine.L'objectif principal de cette thèse est de proposer un système de reconnaissance de l'activité de l'utilisateur incluant quatre composants essentiels, qui peut être utilisé pour assister les utilisateurs lors d'activités orientées vers des objectifs spécifiques : industrie 4.0 (par exemple, assemblage assisté, maintenance) et enseignement. Ainsi, le système observe les mains de l'utilisateur et les objets manipulés depuis le point de vue de l'utilisateur afin de reconnaître et comprendre ses activités manuelles réalisées. Le système de réalité augmenté souhaité doit reconnaître de manière robuste les activités habituelles de l'utilisateur. Néanmoins, il doit détecter les activités inhabituelles afin d'informer l'utilisateur et l'empêcher d'effectuer de mauvaises manœuvres, une exigence fondamentale pour l'assistance à l'utilisateur. Cette thèse combine donc des techniques issues des domaines de recherche de la vision par ordinateur et de l'apprentissage automatique afin de proposer des composants de reconnaissance de l'activité de l'utilisateur nécessaires à un outil d'assistance completHumans use their hands for various tasks in daily life and industry, making research in this area a recent focus of significant interest. Moreover, analyzing and interpreting human behavior using visual signals is one of the most animated and explored areas of computer vision. With the advent of new augmented reality technologies, researchers are increasingly interested in hand activity understanding from a first-person perspective exploring its suitability for human guidance and assistance. Our work is based on machine learning technology to contribute to this research area. Recently, deep neural networks have proven their outstanding effectiveness in many research areas, allowing researchers to jump significantly in efficiency and robustness.This thesis's main objective is to propose a user's activity recognition framework including four key components, which can be used to assist users during their activities oriented towards specific objectives: industry 4.0 (e.g., assisted assembly, maintenance) and teaching. Thus, the system observes the user's hands and the manipulated objects from the user's viewpoint to recognize his performed hand activity. The desired framework must robustly recognize the user's usual activities. Nevertheless, it must detect unusual ones to feedback and prevent him from performing wrong maneuvers, a fundamental requirement for user assistance. This thesis, therefore, combines techniques from the research fields of computer vision and machine learning to propose comprehensive hand activity recognition components essential for a complete assistance tool
2
Zhan, Kai. "First-Person Activity Recognition." Thesis, The University of Sydney, 2014. http://hdl.handle.net/2123/12948.
Full textAbstract:
With advances in sensing technology, automatic recognition of human activities has become a popular research topic. Miniaturised wearable devices can now collect and process the data during activities of daily living. Such technologies rely on algorithms that can effectively combine and interpret wearable sensor data to identify different activities. There are four contributions in this thesis based on a novel wearable device - `Smart Glasses'. The device is able to recognise the subjects' activities of daily living (ADLs) using their first-person vision and motion data. This system consists of a series of algorithms and models. Firstly, we develop first-person video feature extraction algorithms for egocentric vision. The method utilises the optical flow principle on consequent images and extracts the informative motion features, Secondly, we propose dynamic motion detection algorithms to automatically extract the `motion' from the first-person view. The algorithms, based on a Bayesian regression framework known as Gaussian Processes (GP), extract the dynamic portion and related motion tracks of the images. Thirdly, we present a Multi-Scale Conditional Random Field model, which can be applied on top of the conventional approaches. This allows the system to obtain multi-scale context from the sequence of activities. Furthermore, it also has the capacity to integrate multiple sensors into the same system, and potentially increase the system functions. Finally, all the designed algorithms are validated and tested on a wide range of populations, including healthy adults, elders and a mixed disabled patient population, aged between 22 to 89 years old. Our approaches achieve an overall accuracy of up to 89.59% and 84.45% over 12 ADLs for adults and the elderly, and up to 77.07% accuracy on 14 activities from the disabled patients.
3
Tadesse, Girmaw Abebe. "Human activity recognition using a wearable camera." Doctoral thesis, Universitat Politècnica de Catalunya, 2018. http://hdl.handle.net/10803/668914.
Full textAbstract:
Advances in wearable technologies are facilitating the understanding of human activities using first-person vision (FPV) for a wide range of assistive applications. In this thesis, we propose robust multiple motion features for human activity recognition from first person videos. The proposed features encode discriminant characteristics form magnitude, direction and dynamics of motion estimated using optical flow. M:>reover, we design novel virtual-inertial features from video, without using the actual inertial sensor, from the movement of intensity centroid across frames. Results on multiple datasets demonstrate that centroid-based inertial features improve the recognition performance of grid-based features.
Moreover, we propose a multi-layer modelling framework that encodes hierarchical and temporal relationships among activities. The first layer operates on groups of features that effectively encode motion dynamics and temporal variaitons of intra-frame appearance descriptors of activities with a hierarchical topology. The second layer exploits the temporal context by weighting the outputs of the hierarchy during modelling. In addition, a post-decoding smoothing technique utilises decisions on past samples based on the confidence of the current sample. We validate the proposed framework with several classi fiers, and the temporal modelling is shown to improve recognition performance.
We also investigate the use of deep networks to simplify the feature engineering from first-person videos. We propose a stacking of spectrograms to represent short-term global motions that contains a frequency-time representation of multiplemotion components. This enables us to apply 2D convolutions to extract/learn motion features. We employ long short-term memory recurrent network to encode long-term temporal dependency among activiites. Furthermore, we apply cross-domain knowledge transfer between inertial based and vision-based approaches for egocentric activity recognition. We propose sparsity weightedcombination of information from different motion modalities and/or streams . Results show that the proposed approach performs competitively with existing deep frameworks, moreover, with reduced complexity.Los avances en tecnologías wearables facilitan la comprensión de actividades humanas utilizando cuando se usan videos grabados en primera persona para una amplia gama de aplicaciones. En esta tesis, proponemos características robustas de movimiento para el reconocimiento de actividades humana a partir de videos en primera persona. Las características propuestas codifican características discriminativas estimadas a partir de optical flow como magnitud, dirección y dinámica de movimiento. Además, diseñamos nuevas características de inercia virtual a partir de video, sin usar sensores inerciales, utilizando el movimiento del centroide de intensidad a través de los fotogramas. Los resultados obtenidos en múltiples bases de datos demuestran que las características inerciales basadas en centroides mejoran el rendimiento de reconocimiento en comparación con grid-based características. Además, proponemos un algoritmo multicapa que codifica las relaciones jerárquicas y temporales entre actividades. La primera capa opera en grupos de características que codifican eficazmente las dinámicas del movimiento y las variaciones temporales de características de apariencia entre múltiples fotogramas utilizando una jerarquía. La segunda capa aprovecha el contexto temporal ponderando las salidas de la jerarquía durante el modelado. Además, diseñamos una técnica de postprocesado para filtrar las decisiones utilizando estimaciones pasadas y la confianza de la estimación actual. Validamos el algoritmo propuesto utilizando varios clasificadores. El modelado temporal muestra una mejora del rendimiento en el reconocimiento de actividades. También investigamos el uso de redes profundas (deep networks) para simplificar el diseño manual de características a partir de videos en primera persona. Proponemos apilar espectrogramas para representar movimientos globales a corto plazo. Estos espectrogramas contienen una representación espaciotemporal de múltiples componentes de movimiento. Esto nos permite aplicar convoluciones bidimensionales para aprender funciones de movimiento. Empleamos long short-term memory recurrent networks para codificar la dependencia temporal a largo plazo entre las actividades. Además, aplicamos transferencia de conocimiento entre diferentes dominios (cross-domain knowledge) entre enfoques inerciales y basados en la visión para el reconocimiento de la actividad en primera persona. Proponemos una combinación ponderada de información de diferentes modalidades de movimiento y/o secuencias. Los resultados muestran que el algoritmo propuesto obtiene resultados competitivos en comparación con existentes algoritmos basados en deep learning, a la vez que se reduce la complejidad.
4
Fathi, Alireza. "Learning descriptive models of objects and activities from egocentric video." Diss., Georgia Institute of Technology, 2013. http://hdl.handle.net/1853/48738.
Full textAbstract:
Recent advances in camera technology have made it possible to build a comfortable, wearable system which can capture the scene in front of the user throughout the day. Products based on this technology, such as GoPro and Google Glass, have generated substantial interest. In this thesis, I present my work on egocentric vision, which leverages wearable camera technology and provides a new line of attack on classical computer vision problems such as object categorization and activity recognition.
The dominant paradigm for object and activity recognition over the last decade has been based on using the web. In this paradigm, in order to learn a model for an object category like coffee jar, various images of that object type are fetched from the web (e.g. through Google image search), features are extracted and then classifiers are learned. This paradigm has led to great advances in the field and has produced state-of-the-art results for object recognition. However, it has two main shortcomings: a) objects on the web appear in isolation and they miss the context of daily usage; and b) web data does not represent what we see every day.
In this thesis, I demonstrate that egocentric vision can address these limitations as an alternative paradigm. I will demonstrate that contextual cues and the actions of a user can be exploited in an egocentric vision system to learn models of objects under very weak supervision. In addition, I will show that measurements of a subject's gaze during object manipulation tasks can provide novel feature representations to support activity recognition. Moving beyond surface-level categorization, I will showcase a method for automatically discovering object state changes during actions, and an approach to building descriptive models of social interactions between groups of individuals. These new capabilities for egocentric video analysis will enable new applications in life logging, elder care, human-robot interaction, developmental screening, augmented reality and social media.
5
Liu, Hsuan-Ming, and 劉軒銘. "Activity Recognition in First-Person Camera View Based onTemporal Pyramid." Thesis, 2013. http://ndltd.ncl.edu.tw/handle/92962830683022916719.
Full textAbstract:
碩士國立臺灣大學
資訊網路與多媒體研究所
101
We present a simple but effective online recognition system for detecting interleaved activities of daily life (ADLs) in first-person-view videos. The two major difficulties in detecting ADLs are interleaving and variability in duration. We use temporal pyramid in our system to attack these difficulties, and this means we can use relatively simple models instead of time dependent probability ones such as Hidden semi-Markov model or nested models. The proposed solution includes the combination of conditional random fields (CRF) and an online inference algorithm, which explicitly considers multiple interleaved sequences by inferencing multi-stage activities on temporal pyramid. Although our system only uses linear chain-structured CRF model, which can be easily learned without a large amount of training data, it still recognizes complicated activity sequences. The system is evaluated on a data set provided by the work from state-of-the-art, and the result is comparable to their method. We also provide some experiment result using a customized dataset.
6
Lei, Yan-Jing, and 雷晏菁. "Activity Recognition of First-Person Vision and Sleep Posture Analysis." Thesis, 2017. http://ndltd.ncl.edu.tw/handle/ygh973.
Full textAbstract:
碩士國立臺灣大學
資訊工程學研究所
105
First-person vision camera technology is getting wildly used in our daily life to record every seconds of our activities, exercise, adventures, and so on. We present a succinct and robust 3D Convolutional Neural Network (CNN) architecture for both long-term and short-term activity recognition in first-person-view (FPV) videos. Recognizing activities allow us to categorize the amorphous input videos into meaningful chapters, enable efficient browsing, and find the fragments we need immediately. Previous methods for this task are based on hand-craft features, such as hands of subject, visual objects and optical flow. Our 3D CNN is deeper and use small kernel size with some strides. The network is designed for both long-term and short-term activity as well as trained on low resolution sparse optical flow for classifying the camera wearer activity in videos. Reduce the computational complexity while we train network with sparse optical flow. Next, we train an ensemble-learning meta-classifier to aggregate the predicted result of multiple models. No requirement of numerous time on training model and converging under limited amount of data. We achieve classification accuracy of 90%, which outperforms the current state-of-the-art by 15%. Evaluate on an extended FPV video dataset, which has almost twice amount of subjects than current state-of-the-art and nine classes of daily life activity. Our method finds the balance between long-term and short-term activity. For examples, sleep and watch TV for long-term activity or eat medicine and use phone for short-term activity. No assumptions are made on the scene structure. Different background would operate fine theoretically. In sleep posture classification, we propose a three-stream network to recognize the 10 types of sleep posture. Utilize the depth camera Kinect to capture the sleep image stream. Normalize the depth image and calculate the vertical distance map as network input data. Distinguish the major 10 types of sleep posture under different covering conditions, such as without covering, blanket covering and quilt covering. Allow us to observe the sleep status all night long and recommend the improvement method for better sleep quality. Furthermore, we gather 36 subjects to record the sleep image for 10 types of sleep posture. Evaluate the ability of network that we suggest can complete the tasks well. These days, elderly home care is the popular topic that everyone is discussed. All of us are looking for the solutions. Therefore, we propose a method of daily diary to assist the memory decline situation and hope it will delay the deterioration of disease. Review the whole day life by using the application to browse daily diary. Accomplish the goal of recollecting and recording memories.
7
Xia, Lu active 21st century. "Recognizing human activity using RGBD data." Thesis, 2014. http://hdl.handle.net/2152/24981.
Full textAbstract:
Traditional computer vision algorithms try to understand the world using visible light cameras. However, there are inherent limitations of this type of data source. First, visible light images are sensitive to illumination changes and background clutter. Second, the 3D structural information of the scene is lost when projecting the 3D world to 2D images. Recovering the 3D information from 2D images is a challenging problem. Range sensors have existed for over thirty years, which capture 3D characteristics of the scene. However, earlier range sensors were either too expensive, difficult to use in human environments, slow at acquiring data, or provided a poor estimation of distance. Recently, the easy access to the RGBD data at real-time frame rate is leading to a revolution in perception and inspired many new research using RGBD data. I propose algorithms to detect persons and understand the activities using RGBD data. I demonstrate the solutions to many computer vision problems may be improved with the added depth channel. The 3D structural information may give rise to algorithms with real-time and view-invariant properties in a faster and easier fashion. When both data sources are available, the features extracted from the depth channel may be combined with traditional features computed from RGB channels to generate more robust systems with enhanced recognition abilities, which may be able to deal with more challenging scenarios. As a starting point, the first problem is to find the persons of various poses in the scene, including moving or static persons. Localizing humans from RGB images is limited by the lighting conditions and background clutter. Depth image gives alternative ways to find the humans in the scene. In the past, detection of humans from range data is usually achieved by tracking, which does not work for indoor person detection. In this thesis, I propose a model based approach to detect the persons using the structural information embedded in the depth image. I propose a 2D head contour model and a 3D head surface model to look for the head-shoulder part of the person. Then, a segmentation scheme is proposed to segment the full human body from the background and extract the contour. I also give a tracking algorithm based on the detection result. I further research on recognizing human actions and activities. I propose two features for recognizing human activities. The first feature is drawn from the skeletal joint locations estimated from a depth image. It is a compact representation of the human posture called histograms of 3D joint locations (HOJ3D). This representation is view-invariant and the whole algorithm runs at real-time. This feature may benefit many applications to get a fast estimation of the posture and action of the human subject. The second feature is a spatio-temporal feature for depth video, which is called Depth Cuboid Similarity Feature (DCSF). The interest points are extracted using an algorithm that effectively suppresses the noise and finds salient human motions. DCSF is extracted centered on each interest point, which forms the description of the video contents. This descriptor can be used to recognize the activities with no dependence on skeleton information or pre-processing steps such as motion segmentation, tracking, or even image de-noising or hole-filling. It is more flexible and widely applicable to many scenarios. Finally, all the features herein developed are combined to solve a novel problem: first-person human activity recognition using RGBD data. Traditional activity recognition algorithms focus on recognizing activities from a third-person perspective. I propose to recognize activities from a first-person perspective with RGBD data. This task is very novel and extremely challenging due to the large amount of camera motion either due to self exploration or the response of the interaction. I extracted 3D optical flow features as the motion descriptor, 3D skeletal joints features as posture descriptors, spatio-temporal features as local appearance descriptors to describe the first-person videos. To address the ego-motion of the camera, I propose an attention mask to guide the recognition procedures and separate the features on the ego-motion region and independent-motion region. The 3D features are very useful at summarizing the discerning information of the activities. In addition, the combination of the 3D features with existing 2D features brings more robust recognition results and make the algorithm capable of dealing with more challenging cases.text
Books on the topic "First-person hand activity recognition"
1
Norris, Pippa. Political Activism: New Challenges, New Opportunities. Edited by Carles Boix and Susan C. Stokes. Oxford University Press, 2009. http://dx.doi.org/10.1093/oxfordhb/9780199566020.003.0026.
Full textAbstract:
This article discusses political activism and provides an overview that highlights four key themes that have emerged during the last ten years. The first two themes are the growing recognition of the importance of the institutional context of formal rules for electoral turnout and the widespread erosion of party membership in established democracies and questions about its consequences. The last two themes, on the other hand, are the substantial revival of interest in voluntary associations and social trust spurred by theories of social capital and the expansion of diverse forms of cause-oriented types of activism. After briefly illustrating some of the literature which has developed around these themes, the article concludes by considering the challenges for the future research agenda in comparative politics.
Book chapters on the topic "First-person hand activity recognition"
1
Siddiqi, Faisal. "Paradoxes of Strategic Labour Rights Litigation: Insights from the Baldia Factory Fire Litigation." In Interdisciplinary Studies in Human Rights, 59–96. Cham: Springer International Publishing, 2021. http://dx.doi.org/10.1007/978-3-030-73835-8_4.
Full textAbstract:
AbstractThis chapter focuses on the legal activism that followed the Ali Enterprises factory fire and its aftermath in Pakistan. This chapter has two purposes: firstly, it documents the legal proceedings that were initiated and pursued in the courts of Pakistan as well as its interconnected developments. Secondly, I aim to use this engagement with the legal proceedings of the Baldia factory fire aftermath as an opportunity for an in-depth reflection on the capacity and, finally, suitability of the judicial process to bring about justice in struggles over human and labour rights. Providing a rare and insider account of the legal proceedings in the Pakistani courts and its interconnected developments, I hope to lay the empirical foundation for the theoretical and strategic claims of this study. It is against the background and based on the experience with the litigation and legal advocacy following the Baldia fire that I examine the two what I perceive as “paradoxes” at the heart of the litigation. The first is the inseparability of the “limited justice” that may result from such litigation on one hand, and the “structural injustice” that informs and determines the conditions the litigation seeks to address—and transform—on the other hand. The second paradox concerns the inseparability of both law and lawlessness as regards the legal context of the litigation, advocacy and policy proposal elements that are here in play.My argument is that these apparently contradictory phenomena not only coexist alongside one another but that they guarantee each other’s existence. This analysis leads me to the conclusion that in order to understand and improve such forms of strategic litigation, it is necessary to measure its success and failure in terms of three distinct but interconnected criteria. These are the tactical, strategic and structural impacts of the litigation. Ultimately, I will argue for rejecting what is often perceived by involved stakeholders to be an unavoidable choice between nihilism, euphoria or incremental reform in this context. But, to the contrary, I will argue for a conception of legal struggles as a means of building sustainable and fruitful forms of resistance and of change based on the recognition and exploitation of these irreconcilable paradoxes rather than fruitless attempts to ignore or transcend these irreconcilable contradictions.
2
Verma, Kamal Kant, and Brij Mohan Singh. "A Six-Stream CNN Fusion-Based Human Activity Recognition on RGBD Data." In Challenges and Applications for Hand Gesture Recognition, 124–55. IGI Global, 2022. http://dx.doi.org/10.4018/978-1-7998-9434-6.ch007.
Full textAbstract:
RGBD-based activity recognition is quite an interesting task in computer vision. Inspired by the exemplary results obtained from automatic features learning from RGBD data, in this work a six-stream CNN fusion approach has been addressed, which is developed on 2D-convolution neural network (2DCNN) and spatial-temporal 3D-convolution neural networks (ST3DCNN). The proposed approach has six streams and runs in parallel, where the first and second streams are used to extract space and time features with the help of a ST3DCNN model. Similarly, the remaining four streams have been used to extract the temporal features by means of two motion templates on motion history image (MHI) and motion energy image (MEI) via a 2DCNN. Further, a support vector machine (SVM) is employed to generate the score from each stream. Finally, a decision level fusion scheme particularly a weighted product model (WPM) to fuse the scores is obtained from all the streams. The effectiveness of the proposed approach has been tested on popular benchmark public datasets, namely UTD-MHAD, and gives promising results.
3
Oladapo Adenaiye, Oluwasanmi, Kathleen Marie McPhaul, and Donald K. Milton. "Acute Respiratory Infections: Diagnosis, Epidemiology, Management, and Prevention." In Modern Occupational Diseases Diagnosis, Epidemiology, Management and Prevention, 145–63. BENTHAM SCIENCE PUBLISHERS, 2022. http://dx.doi.org/10.2174/9789815049138122010012.
Full textAbstract:
Acute respiratory infections (ARI) are infectious diseases of the respiratory tract caused by viruses, bacteria, and atypical bacteria. They range in severity and even mild cases may cause a significant reduction in workplace productivity. ARIs commonly occur in outbreaks and disproportionally impact workers in occupations where workers are in close proximity to co-workers, members of the public, or where they reside in densely populated housing. High-risk workers include those in the healthcare sector, protective service, food and meat processing, service, and education industries. A person can become infected by inhaling virus-laden aerosols, having virus-contaminated sprayborne drops impinge on exposed mucous membranes, and touching contaminated surfaces followed by self-inoculation. More than one transfer process may be involved in the transmission, and the dominant route may differ for different causative agents, environments, and activity patterns. Preventing ARI transmission in the workplace must be holistic in approach and begin with anticipation and recognition of potential risks, reinforced by the continuous evaluation and implementation of control strategies. Control measures should be layered and multiple routes of transmission should be addressed. Controls should be adapted to the specific workplace and the ARI to prevent pathogen introduction, rapidly detect cases, and promptly eliminate exposure. Prevention and control can be accomplished by promoting vaccination, improving ventilation and air cleaning, providing paid sick leave, flexible working conditions, and work-from-home options. Promoting hand sanitation and providing appropriate personal protective equipment are important but never sufficient in isolation. Occupational health professionals should partner with workplace engineers and human resource departments to design effective programs.
4
Jeon, Moon-Jin, Sang Wan Lee, and Zeungnam Bien. "Hand Gesture Recognition Using Multivariate Fuzzy Decision Tree and User Adaptation." In Contemporary Theory and Pragmatic Approaches in Fuzzy Computing Utilization, 105–19. IGI Global, 2013. http://dx.doi.org/10.4018/978-1-4666-1870-1.ch008.
Full textAbstract:
As an emerging human-computer interaction (HCI) technology, recognition of human hand gesture is considered a very powerful means for human intention reading. To construct a system with a reliable and robust hand gesture recognition algorithm, it is necessary to resolve several major difficulties of hand gesture recognition, such as inter-person variation, intra-person variation, and false positive error caused by meaningless hand gestures. This paper proposes a learning algorithm and also a classification technique, based on multivariate fuzzy decision tree (MFDT). Efficient control of a fuzzified decision boundary in the MFDT leads to reduction of intra-person variation, while proper selection of a user dependent (UD) recognition model contributes to minimization of inter-person variation. The proposed method is tested first by using two benchmark data sets in UCI Machine Learning Repository and then by a hand gesture data set obtained from 10 people for 15 days. The experimental results show a discernibly enhanced classification performance as well as user adaptation capability of the proposed algorithm.
5
Anderson, Cindy L., and Kevin M. Anderson. "Practical Examples of Using Switch-Adapted and Battery-Powered Technology to Benefit Persons With Disabilities." In Handmade Teaching Materials for Students With Disabilities, 212–30. IGI Global, 2019. http://dx.doi.org/10.4018/978-1-5225-6240-5.ch009.
Full textAbstract:
Handmade switch-adapted toys and LED lights were created by a first grader student as part of a makerspace activity to aid a person with disabilities. Commercial toys and light strings were adapted for ease of use by interrupting the electrical current by use of a handmade battery interrupter and the addition of remote switches. In addition, an illuminated glove was created using conductive thread, LED lights, and an Arduino LilyTiny controller to enable the person with disabilities to signal turns on a disability scooter using hand signs. Basic information on the creation of these materials and their possible use are presented in this chapter.
6
Anderson, Cindy L., and Kevin M. Anderson. "Practical Examples of Using Switch-Adapted and Battery-Powered Technology to Benefit Persons With Disabilities." In Research Anthology on Physical and Intellectual Disabilities in an Inclusive Society, 736–53. IGI Global, 2022. http://dx.doi.org/10.4018/978-1-6684-3542-7.ch040.
Full textAbstract:
Handmade switch-adapted toys and LED lights were created by a first grader student as part of a makerspace activity to aid a person with disabilities. Commercial toys and light strings were adapted for ease of use by interrupting the electrical current by use of a handmade battery interrupter and the addition of remote switches. In addition, an illuminated glove was created using conductive thread, LED lights, and an Arduino LilyTiny controller to enable the person with disabilities to signal turns on a disability scooter using hand signs. Basic information on the creation of these materials and their possible use are presented in this chapter.
7
Kumar Sharma, Avinash, Pratiyaksha Mittal, Ritik Ranjan, and Rishabh Chaturvedi. "Bank Robbery Detection System Using Computer Vision." In Advances in Transdisciplinary Engineering. IOS Press, 2023. http://dx.doi.org/10.3233/atde221322.
Full textAbstract:
We propose a computer-vision-based detection and recognition system which can be used in the banks to detect the anomalous activity of bank robbery. We propose CCTV based robbery detection along with tracking of thieves. We have used computer vision to detect theft and robbers in CCTV footage, without the use of sensors. This system concentrates on object detection. This detection of bank robbery is done based on detecting components, which are prohibited by using in the banks and are common in robbery, like handguns, a person wearing a helmet or a ski mask which comes under the object detection. Apart from these, recognition is done on the human postures like raising hands and kneel down which comes under the posture detection. The security official will be notified about the suspicious event by using Real-time analysis of the movement of any human from CCTV footage and thus gives a chance to avert the same, so that necessary action will be taken by the authority and prevent threat to bank as well as to the human life presents there.
8
Tsatsoulis, P. Daphne, Aaron Jaech, Robert Batie, and Marios Savvides. "Multimodal Biometric Hand-Off for Robust Unobtrusive Continuous Biometric Authentication." In IT Policy and Ethics, 389–409. IGI Global, 2013. http://dx.doi.org/10.4018/978-1-4666-2919-6.ch018.
Full textAbstract:
Conventional access control solutions rely on a single authentication to verify a user’s identity but do nothing to ensure the authenticated user is indeed the same person using the system afterwards. Without continuous monitoring, unauthorized individuals have an opportunity to “hijack” or “tailgate” the original user’s session. Continuous authentication attempts to remedy this security loophole. Biometrics is an attractive solution for continuous authentication as it is unobtrusive yet still highly accurate. This allows the authorized user to continue about his routine but quickly detects and blocks intruders. This chapter outlines the components of a multi-biometric based continuous authentication system. Our application employs a biometric hand-off strategy where in the first authentication step a strong biometric robustly identifies the user and then hands control to a less computationally intensive face recognition and tracking system that continuously monitors the presence of the user. Using multiple biometrics allows the system to benefit from the strengths of each modality. Since face verification accuracy degrades as more time elapses between the training stage and operation time, our proposed hand-off strategy permits continuous robust face verification with relatively simple and computationally efficient classifiers. We provide a detailed evaluation of verification performance using different pattern classification algorithms and show that the final multi-modal biometric hand-off scheme yields high verification performance.
9
Lynch, John Roy. "Democrats in the South: The Race Question." In Reminiscences of an Active Life, edited by John Hope Franklin, 503–12. University Press of Mississippi, 2008. http://dx.doi.org/10.14325/mississippi/9781604731149.003.0050.
Full textAbstract:
This chapter explores how, when John Roy Lynch came to Chicago, whether or not he should take an active part in politics was one of the first questions that occurred to him. He had no intention of actively participating politically in local matters, but it occurred to him that like some other retired army officers, he could, with propriety, take an active part in national matters. But after going over the field very carefully, Lynch found that conditions nationally, as well as locally, were not such as would justify him in doing so. In fact, beginning with the unfortunate administration of President William Howard Taft, the colored American had no standing with either of the two major parties. The Democratic party, nationally, was still a white man's party and, beginning with the Taft administration, the Republican party was no longer a champion of human rights. In fact, the policy inaugurated by President Taft was equivalent to transforming the Republican party, as far as it was in the power of an administration to do so, into a race proscriptive party. In other words, racial identity regardless of merit was made a bar to official recognition.
10
Anisimov, Dmytro, Dmytro Petrushin, and Victor Boguslavsky. "IMPROVEMENT OF PHYSICAL TRAINING OF FIRST-YEAR CADETS OF DNIPROPETROVSK STATE UNIVERSITY OF INTERNAL AFFAIRS." In Scientific space in the conditions of global transformations of the modern world. Publishing House “Baltija Publishing”, 2022. http://dx.doi.org/10.30525/978-9934-26-255-5-1.
Full textAbstract:
In the conditions of a complex criminogenic situation, a high level of preparedness of police officers for effective actions in regular and extreme conditions is extremely important for successfully solving the tasks of ensuring proper law and order and public safety in the state. The professional training of employees is a decisive factor in the quality of law enforcement activities, ensuring legality in maintaining law and order and fighting crime.A high level of physical training as a component of professional training guarantees the effectiveness of operational and service tasks, ensuring the personal safety of the police officer and his surroundings.In modern society, it is no longer a secret for anyone how important it is for every person to be in good physical shape. The need for this is caused by the rhythm of modern life, new opportunities and challenges of today, and even the fashion for a healthy lifestyle. After all, with the development of society, new relationships also develop, the criminal world also actively uses new opportunities and gaps in the legislation, unprofessional actions or insufficient preparation of employees of the Security Service.Thus, the presence of a high level of physical training, perfect mastery of techniques and tactics for the application of measures of physical influence, self-defense skills and techniques of hand-to-hand combat in regular and extreme conditions of operational-service activity is the professional duty of every police officer.We can come to the conclusion that the level of physical training as a component of professional training affects the effectiveness of operational-service tasks, ensures the personal safety of the police officer and his environment, and therefore requires constant improvement and scientific research. The purpose of the research is the theoretical analysis and synthesis of scientific and methodical literature on the researched topic, as well as the expert examination of the preparedness of the first-year cadets of the Dnipropetrovsk State University of Internal Affairs. The following research methods were used to implement the tasks and achieve the goal: analysis of literary sources; synthesis of literary sources; generalization of literary sources; pedagogical observation; control exercises (tests); assessment of testing levels; mathematical statistics.In order to successfully solve the tasks for physical training classes, the motivational levers of the stable evaluation system, namely the “Scale for evaluating the results of physical training tests” were used. Resultsthe analysis of scientific and methodical literature and documents that regulate the educational process in special higher educational institutions of the armed forces gave reasons to assert that proper physical training is an important prerequisite for high-quality professional preparation of graduates of these universities for practical activities. At the same time, the problem of improving the quality of physical training of cadets in higher educational institutions of the Ministry of Internal Affairs of Ukraine, despite a significant number of developments, remains insufficiently studied.Taking into account the need for police officers to engage in confrontations with aggressive and, as a rule, well-trained violators, the problem of special physical training of cadets of higher educational institutions of the National Police of Ukraine has become extremely urgent.Also, based on our research, we can conclude that due to high workload, cadets do not have the opportunity to regularly engage in physical training and improve their physical level, and classes according to the educational and methodological plan cannot fully develop physical qualities cadets, that is why we see an insufficient development of strength in the cadets of the first year of the DDUVS. This is evidenced by the fact that, according to our research, only 32% (n = 8) of the subjects have an excellent level. Good was noted in 40% (n = 10), and a satisfactory level of strength development was found in 28% (n = 7) of the cadets. Practical implications. The scientific novelty of the obtained results lies in the fact that a complex theoretical and applied monodisciplinary study was conducted, dedicated to solving scientific and practical problems of the state of physical training of first-year cadets of the Dnipropetrovsk State University of Internal Affairs. Value/originality. The obtained research results can be used to improve the level of physical training of cadets of the National Police of Ukraine.
Conference papers on the topic "First-person hand activity recognition"
1
Grewe, Lynne L., Chengzhi Hu, Krishna Tank, Aditya Jaiswal, Thomas Martin, Sahil Sutaria, Tran Huynh, and Francis David Bustos. "First person perspective video activity recognition." In Signal Processing, Sensor/Information Fusion, and Target Recognition XXIX, edited by Lynne L. Grewe, Erik P. Blasch, and Ivan Kadar. SPIE, 2020. http://dx.doi.org/10.1117/12.2557922.
Full text
2
Ma, Minghuang, Haoqi Fan, and Kris M. Kitani. "Going Deeper into First-Person Activity Recognition." In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2016. http://dx.doi.org/10.1109/cvpr.2016.209.
Full text
3
Iwashita, Yumi, Asamichi Takamine, Ryo Kurazume, and M. S. Ryoo. "First-Person Animal Activity Recognition from Egocentric Videos." In 2014 22nd International Conference on Pattern Recognition (ICPR). IEEE, 2014. http://dx.doi.org/10.1109/icpr.2014.739.
Full text
4
Zhan, Kai, Vitor Guizilini, and Fabio Ramos. "Dense motion segmentation for first-person activity recognition." In 2014 13th International Conference on Control Automation Robotics & Vision (ICARCV). IEEE, 2014. http://dx.doi.org/10.1109/icarcv.2014.7064291.
Full text
5
Demachi, Kazuyuki, and Shi Chen. "Development of Malicious Hand Behaviors Detection Method by Movie Analysis." In 2018 26th International Conference on Nuclear Engineering. American Society of Mechanical Engineers, 2018. http://dx.doi.org/10.1115/icone26-81643.
Full textAbstract:
An urgent lesson learned from Fukushima Daiichi accident is what can happen by natural disaster can also occur by human design. The accident raised a fear that terrorists could cause a similar accident by acts of sabotage against nuclear power plant (NPP) and it is noticeable that threats of terrorism for nuclear security are increased after the accident. When considering sabotage, the prime threat to nuclear power plants, due attention should be paid to sabotage by insiders. Generally, insiders are the individuals with authorized access to nuclear facilities in transport who could attempt unauthorized sabotage. They could take advantage of their access authority and knowledge, to bypass dedicated physical protection elements or other provisions [1]. Thus, we should value the catastrophic consequences of the attack or act of insider sabotage which may lead to loss of safety functions of NPP. International Atomic Energy Agency (IAEA) indicated that the physical protection system (PPS) of a nuclear facility should be integrated and effective against both sabotage and unauthorized removal. The primary PPS functions are deterrence, detection, delay and response. It is noticeable that if detection failed, delay and response would become invalid. Thus, detection of insiders’ sabotage should be enhanced. Considering current countermeasures of PPS to insiders’ sabotage, the most significant challenge is how to distinguish ordinary maintenance behaviors and malicious behaviors since some malicious behaviors may hidden in ordinary maintenance behaviors. It appears that hand behavior has high contribution to human activity and a significant portion of maintenance behaviors and malicious behaviors. In this study, we proposed a hand behavior detection algorithm for insiders’ malicious behaviors for nuclear security [2]. We focused on the fact that the hand shape is uniquely determined by the fingertip coordinates. First, the depth image of the hand was captured with Kinect v2, and after removing the five fingers were remained by removing the palm and wrist parts, and the five fingers were identified using the K-means clustering [3], and the farthest point of each finger from wrist pixel was taken as the fingertip coordinates. The fingertip coordinates of the five fingers were combined for 60 frames to be time-series data, and this was used as the training data of the neural network. Time-series data obtained from five kinds of behaviors of five hands was used for training data. For the machine learning method, the Stacked-Auto Encoder (SAE) [4–5] which is one of popular methods was used. It extracts the feature of input data at intermediate layer of the first stage. In the second layer, the extracted feature is input and its feature is extracted to be used as the input of the softmax layer for pattern classification. Meanwhile, a real-time fingertip tracking system was developed and time-series data of each fingertip was successfully obtained with 29.8fps using MATLAB whose CPU was Intel Xeon Processor E5-2630v4 (25M Cache, 2.20 GHz). Moreover, a time-series data analysis based behavior recognition method was developed and all assumed malicious behaviors were detected with high accuracy (82.555% in overall) and speed (0.0023 seconds per frame) in the same computing environment. Also, robustness of the behavior recognition method was verified.
6
Ozkan, Fatih, Mehmet Ali Arabaci, Elif Surer, and Alptekin Temizel. "Boosted multiple kernel learning for first-person activity recognition." In 2017 25th European Signal Processing Conference (EUSIPCO). IEEE, 2017. http://dx.doi.org/10.23919/eusipco.2017.8081368.
Full text
7
Prabhakar, Manav, and Snehasis Mukherjee. "First-person Activity Recognition by Modelling Subject - Action Relevance." In 2022 International Joint Conference on Neural Networks (IJCNN). IEEE, 2022. http://dx.doi.org/10.1109/ijcnn55064.2022.9892547.
Full text
8
Spriggs, Ekaterina H., Fernando De La Torre, and Martial Hebert. "Temporal segmentation and activity classification from first-person sensing." In 2009 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2009. http://dx.doi.org/10.1109/cvprw.2009.5204354.
Full text
9
Garcia-Hernando, Guillermo, Shanxin Yuan, Seungryul Baek, and Tae-Kyun Kim. "First-Person Hand Action Benchmark with RGB-D Videos and 3D Hand Pose Annotations." In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2018. http://dx.doi.org/10.1109/cvpr.2018.00050.
Full text
10
Baydoun, Mohamad, Alejandro Betancourt, Pietro Morerio, Lucio Marcenaro, Matthias Rauterberg, and Carlo Regazzoni. "Hand pose recognition in First Person Vision through graph spectral analysis." In 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2017. http://dx.doi.org/10.1109/icassp.2017.7952481.
Full text