Letteratura scientifica selezionata sul tema "Music auto-tagging"

In the realm of music information retrieval, similarity-based retrieval and auto-tagging serve as essential components. Similarity-based retrieval involves automatically analyzing a music track and fetching analogous tracks from a database. Auto-tagging, on the other hand, assesses a music track to deduce associated tags, such as genre and mood. Given the limitations and non-scalability of human supervision signals, it becomes crucial for models to learn from alternative sources to enhance their performance. Contrastive learning-based self-supervised learning, which exclusively relies on learning signals derived from music audio data, has demonstrated its efficacy in the context of auto-tagging. In this work, we propose a model that builds on the self-supervised learning approach to address the similarity-based retrieval challenge by introducing our method of metric learning with a self-supervised auxiliary loss. Furthermore, diverging from conventional self-supervised learning methodologies, we discovered the advantages of concurrently training the model with both self-supervision and supervision signals, without freezing pre-trained models. We also found that refraining from employing augmentation during the fine-tuning phase yields better results. Our experimental results confirm that the proposed methodology enhances retrieval and tagging performance metrics in two distinct scenarios: one where human-annotated tags are consistently available for all music tracks, and another where such tags are accessible only for a subset of music tracks.

Audio auto-tagging is the process of assigning labels to audio clips for better categorization and management of audio file databases. With the advent of advanced artificial intelligence technologies, there has been increasing interest in directly using raw audio data as input for deep learning models in order to perform tagging and eliminate the need for preprocessing. Unfortunately, most current studies of audio auto-tagging cannot effectively reflect the semantic relationships between tags—for instance, the connection between “classical music” and “cello”. In this paper, we propose a novel method that can enhance audio auto-tagging performance via joint embedding. Our model has been carefully designed and architected to recognize the semantic information within the tag domains. In our experiments using the MagnaTagATune (MTAT) dataset, which has high inter-tag correlations, and the Speech Commands dataset, which has no inter-tag correlations, we showed that our approach improves the performance of existing models when there are strong inter-tag correlations.

La croissance exponentielle des services en ligne et des données des utilisateurs a changé la façon dont nous interagissons avec divers services, et la façon dont nous explorons et sélectionnons de nouveaux produits. Par conséquent, il existe un besoin croissant de méthodes permettant de recommander les articles appropriés pour chaque utilisateur. Dans le cas de la musique, il est plus important de recommander les bons éléments au bon moment. Il est bien connu que le contexte, c'est-à-dire la situation d'écoute des utilisateurs, influence fortement leurs préférences d'écoute. C'est pourquoi le développement de systèmes de recommandation fait l'objet d'une attention croissante. Les approches les plus récentes sont des modèles basés sur les séquences qui visent à prédire les pistes de la prochaine session en utilisant les informations contextuelles disponibles. Cependant, ces approches ne sont pas faciles à interpréter et ne permettent pas à l'utilisateur de s'impliquer. De plus, peu d'approches précédentes se sont concentrées sur l'étude de la manière dont le contenu audio est lié à ces influences situationnelles et, dans une moindre mesure, sur l'utilisation du contenu audio pour fournir des recommandations contextuelles. Par conséquent, ces approches souffrent à la fois d'un manque d'interprétabilité. Dans cette thèse, nous étudions le potentiel de l'utilisation du contenu audio principalement pour désambiguïser les situations d'écoute, fournissant une voie pour des recommandations interprétables basées sur la situation.Tout d'abord, nous étudions les situations d'écoute potentielles qui influencent ou modifient les préférences d'écoute des utilisateurs. Nous avons développé une approche semi-automatique pour faire le lien entre les pistes écoutées et la situation d'écoute en utilisant les titres des listes de lecture comme proxy. Grâce à cette approche, nous avons pu collecter des ensembles de données de pistes musicales étiquetées en fonction de leur utilisation situationnelle. Nous avons ensuite étudié l'utilisation de marqueurs automatiques de musique pour identifier les situations d'écoute potentielles à partir du contenu audio. Ces études ont permis de conclure que l'utilisation situationnelle d'un morceau dépend fortement de l'utilisateur. Nous avons donc étendu l'utilisation des marqueurs automatiques de musique à un modèle tenant compte de l'utilisateur afin de faire des prédictions personnalisées. Nos études ont montré que l'inclusion de l'utilisateur dans la boucle améliore considérablement les performances de prédiction des situations. Cet auto-tagueur de musique adapté à l'utilisateur nous a permis de marquer une piste donnée à travers le contenu audio avec une utilisation situationnelle potentielle, en fonction d'un utilisateur donné en tirant parti de son historique d'écoute.Enfin, pour réussir à utiliser cette approche pour une tâche de recommandation, nous avions besoin d'une méthode différente pour prédire les situations actuelles potentielles d'un utilisateur donné. À cette fin, nous avons développé un modèle pour prédire la situation à partir des données transmises par l'appareil de l'utilisateur au service, et des informations démographiques de l'utilisateur donné. Nos évaluations montrent que les modèles peuvent apprendre avec succès à discriminer les situations potentielles et à les classer en conséquence. En combinant les deux modèles, l'auto-tagueur et le prédicteur de situation, nous avons développé un cadre pour générer des sessions situationnelles en temps réel et les proposer à l'utilisateur. Ce cadre fournit une voie alternative pour recommander des sessions situationnelles, en dehors du système de recommandation séquentiel primaire déployé par le service, qui est à la fois interprétable et aborde le problème du démarrage à froid en termes de recommandation de morceaux basés sur leur contenu
The exponential growth of online services and user data changed how we interact with various services, and how we explore and select new products. Hence, there is a growing need for methods to recommend the appropriate items for each user. In the case of music, it is more important to recommend the right items at the right moment. It has been well documented that the context, i.e. the listening situation of the users, strongly influences their listening preferences. Hence, there has been an increasing attention towards developing recommendation systems. State-of-the-art approaches are sequence-based models aiming at predicting the tracks in the next session using available contextual information. However, these approaches lack interpretability and serve as a hit-or-miss with no room for user involvement. Additionally, few previous approaches focused on studying how the audio content relates to these situational influences, and even to a less extent making use of the audio content in providing contextual recommendations. Hence, these approaches suffer from both lack of interpretability.In this dissertation, we study the potential of using the audio content primarily to disambiguate the listening situations, providing a pathway for interpretable recommendations based on the situation.First, we study the potential listening situations that influence/change the listening preferences of the users. We developed a semi-automated approach to link between the listened tracks and the listening situation using playlist titles as a proxy. Through this approach, we were able to collect datasets of music tracks labelled with their situational use. We proceeded with studying the use of music auto-taggers to identify potential listening situations using the audio content. These studies led to the conclusion that the situational use of a track is highly user-dependent. Hence, we proceeded with extending the music-autotaggers to a user-aware model to make personalized predictions. Our studies showed that including the user in the loop significantly improves the performance of predicting the situations. This user-aware music auto-tagger enabled us to tag a given track through the audio content with potential situational use, according to a given user by leveraging their listening history.Finally, to successfully employ this approach for a recommendation task, we needed a different method to predict the potential current situations of a given user. To this end, we developed a model to predict the situation given the data transmitted from the user's device to the service, and the demographic information of the given user. Our evaluations show that the models can successfully learn to discriminate the potential situations and rank them accordingly. By combining the two model; the auto-tagger and situation predictor, we developed a framework to generate situational sessions in real-time and propose them to the user. This framework provides an alternative pathway to recommending situational sessions, aside from the primary sequential recommendation system deployed by the service, which is both interpretable and addressing the cold-start problem in terms of recommending tracks based on their content

One of the many challenges of machine learning are systems for automatic tagging of music, the complexity of this issue in particular. These systems can be practically used in the content analysis of music or the sorting of music libraries. This thesis deals with the design, training, testing, and evaluation of artificial neural network architectures for automatic tagging of music. In the beginning, attention is paid to the setting of the theoretical foundation of this field. In the practical part of this thesis, 8 architectures of neural networks are designed (4 fully convolutional and 4 convolutional recurrent). These architectures are then trained using the MagnaTagATune Dataset and mel spectrogram. After training, these architectures are tested and evaluated. The best results are achieved by the four-layer convolutional recurrent neural network (CRNN4) with the ROC-AUC = 0.9046 ± 0.0016. As the next step of the practical part of this thesis, a completely new Last.fm Dataset 2020 is created. This dataset uses Last.fm and Spotify API for data acquisition and contains 100 tags and 122877 tracks. The most successful architectures are then trained, tested, and evaluated on this new dataset. The best results on this dataset are achieved by the six-layer fully convolutional neural network (FCNN6) with the ROC-AUC = 0.8590 ± 0.0011. Finally, a simple application is introduced as a concluding point of this thesis. This application is designed for testing individual neural network architectures on a user-inserted audio file. Overall results of this thesis are similar to other papers on the same topic, but this thesis brings several new findings and innovations. In terms of innovations, a significant reduction in the complexity of individual neural network architectures is achieved while maintaining similar results.

碩士
國立清華大學
資訊工程學系所
105
We proposed a novel and intuitive music retrieval interface that allows users to precisely search music containing multiple localized social tags with merely simple sketches. For example, one may search for a “classical” music clip that also includes a segment with “violin”, followed by another segment which simultaneously includes “slow” and “guitar”, while such complex conditions can be simply and correctly expressed in the query. We also proposed a segment-level database with thousands of songs and its preprocessing algorithms for our music retrieval method, which leverages the predictions by Liu and Yang’s deep learning-based frame-level auto-tagging model. To assess how users feel about this system, we have conducted a user study with a questionnaire and a demo website. Experimental results show that: i) the proposed sketch-based system outperforms the two non-sketch-based baselines we implemented in “interestingness” and “satisfaction in user experience”; ii) our proposed method is especially beneficial to multimedia content creators.

碩士
國立成功大學
資訊管理研究所
107
With the development of the Internet and technology, online music platforms and music streaming services are booming, the large number of digital music makes users face the problem of information overloading. In order to solve this problem, these platforms need to construct a comprehensive recommendation system by using user information and meta data to help users in searching, querying or discovering new music. Social tags are considered to help the music recommendation system to make better recommendations. However, social tags face the problem of tag sparsity and cold start, limiting their effectiveness in helping the recommendation system. To solve these problems, it is necessary to supplement the shortage of the tags through a music auto-tagging system. In the past, most of the research on auto-tagging used only audio for analysis. However, many studies have proved that the lyrics can help the music classification system to obtain more information and improve the classification accuracy. This study proposed a method of music auto-tagging, which use both audio and lyric for analysis. Besides, we also experimented the different architecture of tag classification, the result shows that the structure using late fusion model and multi-task classification method has the best performance.

碩士
國立中興大學
資訊科學與工程學系
106
Music tags are a set of descriptive keywords that convey high-level information about a music clip, such as emotions(sadness, happiness), genres(jazz, classical), and instruments(guitar, vocal). Since tags provide high-level information from the listener’s perspectives, they can be used for music discovery and recommendation. However, in music information retrieval (MIR), researchers need to have expertise based on acoustics or engineering design in order to analyze and organize music informations, classify them according to music forms, and then provide music information retrieval. In recent years, people have been paying more attention to the feature learning and deep architecture, thus reducing the required of the engineering works and the need for prior knowledge. The use of deep convolutional neural networks has been successfully explored in the image, text and speech field. However, previous methods for music auto-tagging can’t accurately discriminate the type of music for the distortion and noise audio, it will have the bad results in the auto-tagging. Therefore, we will propose a robust method to implement auto-music tagging. First, convert the music into a spectrogram, and find out the important information from the spectrogram, that is, the audio fingerprint. Then use it as the input of convolutional neural networks to learn the features, in this way to get a good music search result. Experimental results demonstrate the robustness of the proposed method.

The supervised and semi-supervised machine learning paradigms hinge on the idea that the training data is labeled. The label quality is often brought into question, and problems related to noisy, inaccurate, or missing labels are studied. One of these is an interesting and prevalent problem in the semi-supervised classification area where only some positive labels are known. At the same time, the remaining and often the majority of the available data is unlabeled, i.e., there are no negative examples. Known as Positive-Unlabeled (PU) learning, this problem has been identified with increasing frequency across many disciplines, including but not limited to health science, biology, bioinformatics, geoscience, physics, business, and politics. Also, there are several closely related machine learning problems, such as cost-sensitive learning and mixture proportion estimation. This dissertation explores the PU learning problem from the perspective of density estimation and proposes a new modular method compatible with the relabeling framework that is common in PU learning literature. This approach is compared with two existing algorithms throughout the manuscript, one from a seminal work by Elkan and Noto and a current state-of-the-art algorithm by Ivanov. Furthermore, this thesis identifies two machine learning application domains that can benefit from PU learning approaches, which were not previously seen that way: predicting length of stay in hospitals and automatic music tagging. Experimental results with multiple synthetic and real-world datasets from different application domains validate the proposed approach. Accurately predicting the in-hospital length of stay (LOS) at the time of admission can positively impact healthcare metrics, particularly in novel response scenarios such as the Covid-19 pandemic. During the regular steady-state operation, traditional classification algorithms can be used for this purpose to inform planning and resource management. However, when there are sudden changes to the admission and patient statistics, such as during the onset of a pandemic, these approaches break down because reliable training data becomes available only gradually over time. This thesis demonstrates the effectiveness of PU learning approaches in such situations through experiments by simulating the positive-unlabeled scenario using two fully-labeled publicly available LOS datasets. Music auto-tagging systems are typically trained using tag labels provided by human listeners. In many cases, this labeling is weak, which means that the provided tags are valid for the associated tracks, but there can be tracks for which a tag would be valid but not present. This situation is analogous to PU learning with the additional complication of being a multi-label scenario. Experimental results on publicly available music datasets with tags representing three different labeling paradigms demonstrate the effectiveness of PU learning techniques in recovering the missing labels and improving auto-tagger performance.
Graduate

Deep learning has become the standard procedure to deal with Music Information Retrieval problems. This category of machine learning algorithms has achieved state-of-the-art results in several tasks, such as classification and auto-tagging. However, obtaining a good-performing model requires a significant amount of data. At the same time, most of the music datasets available lack cultural diversity. Therefore, the performance of the currently most used pre-trained models on underrepresented music genres is unknown. If music models follow the same direction that language models in Natural Language Processing, they should have poorer performance on music styles that are not present in the data used to train them. To verify this assumption, we use a well-known music model designed for auto-tagging in the task of genre recognition. We trained this model from scratch using a large general-domain dataset and two subsets specifying different domains. We empirically show that models trained on specific-domain data perform better than generalist models to classify music in the same domain, even trained with a smaller dataset. This outcome is distinctly observed in the subset that mainly contains Brazilian music, including several usually underrepresented genres.