Academic literature on the topic 'Big Data et algorithmes'
Create a spot-on reference in APA, MLA, Chicago, Harvard, and other styles
Consult the lists of relevant articles, books, theses, conference reports, and other scholarly sources on the topic 'Big Data et algorithmes.'
Next to every source in the list of references, there is an 'Add to bibliography' button. Press on it, and we will generate automatically the bibliographic reference to the chosen work in the citation style you need: APA, MLA, Harvard, Chicago, Vancouver, etc.
You can also download the full text of the academic publication as pdf and read online its abstract whenever available in the metadata.
Journal articles on the topic "Big Data et algorithmes"
1
Benavent, Christophe. "Big Data, algorithme et marketing : rendre des comptes." Statistique et société 4, no. 3 (2016): 25–35. https://doi.org/10.3406/staso.2016.1009.
Full textAbstract:
Cet article s’intéresse à la question de la mise en oeuvre à vaste échelle d’algorithmes utiles au marketing, et s’intégrant dans une logique de plateforme. Prenant en compte des observations répétées d’externalités négatives produites par les algorithmes : ségrégation, biais de sélection, polarisation, hétérogénéisation, mais aussi leurs faiblesses intrinsèques résultant de la dette technique, de dépendances des données, et du contexte adversial dans lequel ils s’exercent, nous aboutissons à la nécessité d’une redevabilité algorithmique et nous nous questionnons sur la manière dont les algorithmes doivent être gouvernés.
2
Jauréguiberry, Francis. "L’individu hypermoderne face aux big data." Sociologie et sociétés 49, no. 2 (December 4, 2018): 33–58. http://dx.doi.org/10.7202/1054273ar.
Full textAbstract:
Les big data, le datamining et le profilage, avec l’ensemble des applications d’aide à l’action individuelle et collective qui en découle, suscitent à juste titre des inquiétudes en ce qui concerne, d’une part la protection de la vie privée dans un environnement capteur des faits et gestes de chacun, d’autre part les formes de gouvernance de plus en plus informées par des algorithmes prédictifs. Sans négliger ces dangers, une position presque inverse sera ici défendue sous forme d’hypothèse : loin d’entraîner le déclin de l’autonomie individuelle, de soi comme personne singulière capable de réflexivité et en position de faire des choix autonomes, la confrontation renouvelée à une image personnelle purement quantitative et utilitaire (profil) peut conduire à un ressaisissement de soi visant à ce que les choix soient non plus seulement guidés par une logique narcissique, utilitaire et quantitative, mais tout autant par des principes de cohérence individuelle, éthiques et moraux qui, in fine, donnent du sens à la vie.
3
Koch, Olivier. "Les données de la guerre. Big Data et algorithmes à usage militaire." Les Enjeux de l'information et de la communication N° 19/2, no. 2 (2018): 113. http://dx.doi.org/10.3917/enic.025.0113.
Full text
4
Gori, Roland. "La biopolitique à l’ère des algorithmes." Cliniques méditerranéennes 110, no. 2 (September 25, 2024): 147–65. http://dx.doi.org/10.3917/cm.110.0147.
Full textAbstract:
Michel Foucault a largement montré comment l’herméneutique du sujet moderne le conduisait à une externalisation des dispositifs de « technique de soi », mais que dirait-il aujourd’hui face à une connaissance du sujet par lui-même qui passe par Facebook, les Big Data et le numérique, bref un sujet à l’ère des selfies et de l’intelligence artificielle ? Où en sommes-nous aujourd’hui de la manière de gouverner les humains et des techniques de formation pour leur apprendre à se gouverner eux-mêmes ? Il y a véritablement un nouage entre les manières de se connaître, de se fabriquer comme sujet éthique, et les manières de gouverner. C’est la raison pour laquelle à la manière de se connaître par le numérique et les algorithmes correspond un gouvernement politique des humains par la prédiction et la gestion numériques. Face à la crise de confiance des peuples dans les gouvernements démocratiques, les États se réfugient dans une gestion technocratique, une administration quasiment algorithmique des populations, non sans se soumettre aux exigences des marchés avec lesquels le gouvernement technocratique des humains peut faire bon ménage. Avec une telle transition vers la post-démocratie allant vers une ère post-politique, déjà présente dans les nouvelles manières de soigner les âmes, nous serions face à une crise de la vérité et de la subjectivité. Et plus encore en présence de l’émergence d’un nouveau système, d’une nouvelle épistémè politique et subjective, d’une nouvelle relation du sujet au pouvoir.
5
Besse, Philippe, Céline Castets-Renard, and Aurélien Garivier. "L’IA du Quotidien peut elle être Éthique ?" Statistique et société 6, no. 3 (2018): 9–31. https://doi.org/10.3406/staso.2018.1083.
Full textAbstract:
Associant données massives (big data) et algorithmes d’apprentissage automatique (machine learning), la puissance des outils de décision automatique suscite autant d’espoir que de craintes. De nombreux textes législatifs européens (RGPD) et français récemment promulgués tentent d’encadrer les usages de ces outils. Laissant de côté les problèmes bien identifiés de confidentialité des données et ceux d’entrave à la concurrence, nous nous focalisons sur les risques de discrimination, les problèmes de transparence et ceux de qualité des décisions algorithmiques. La mise en perspective détaillée des textes juridiques, face à la complexité et l’opacité des algorithmes d’apprentissage, révèle la nécessité d’importantes disruptions technologiques que ce soit pour détecter ou réduire le risque de discrimination ou pour répondre au droit à l’explication. La confiance des développeurs et surtout des usagers (citoyens, justiciables, clients) étant indispensable, les algorithmes exploitant des données personnelles se doivent d’être déployés dans un cadre éthique strict. En conclusion nous listons, pour répondre à cette nécessité, quelques possibilités de contrôle à développer : institutionnel, charte éthique, audit externe attaché à la délivrance d’un label.
6
Viglino, Manon. "La présomption d’innocence à l’ère du numérique." Revue de la recherche juridique, no. 2 (January 5, 2021): 1039–63. http://dx.doi.org/10.3917/rjj.190.1039.
Full textAbstract:
Les nouvelles technologies se développent, les formes d’intelligence artificielle se perfectionnent. Algorithmes, Big Data et justice prédictive sont aujourd’hui au cœur des débats sur l’avenir de la justice. Loin de dénier les progrès permis par ces nouvelles technologies et les bénéfices des infinies possibilités qu’elles offrent, il convient toutefois d’en souligner les dangers. En particulier, à l’heure de leur officialisation et de leur développement, il semble indispensable de veiller au respect des droits et libertés les plus fondamentaux. En particulier, dans le cadre pénal, la présomption d’innocence semble déjà souffrir d’un certain affaiblissement, induit par la facilité du partage d’informations non vérifiées en particulier par l’utilisation des réseaux sociaux, mais également par les nombreux biais et la nature même de certains algorithmes. Une réflexion éthique devrait ainsi être initiée, pour garantir l’existence d’un état de droit sans pour autant freiner le progrès.
7
Nazeer, Mohammed Yaseer, and Mohammad Tarik Nadir. "Data Deluge Dynamics: Tracing the Evolution and Ramifications of Big Data Phenomenon." International Journal of Research and Innovation in Social Science VIII, no. V (2024): 2147–56. http://dx.doi.org/10.47772/ijriss.2024.805157.
Full textAbstract:
This paper presents a comprehensive review of the evolution, methodologies, challenges, and implications of Big Data in various domains. Big Data has emerged as a critical resource, offering unprecedented opportunities for decision-making, innovation, and societal advancement. The analysis delves into the historical trajectory of Big Data, examining its evolution from the early 2000s to its current status as a cornerstone of contemporary data-driven practices. Drawing on seminal works by Chen et al. (2014), Manyika et al. (2011), and Kitchin (2014), the review highlights the fundamental characteristics of Big Data, encapsulated by the “three Vs” – Volume, Velocity, and Variety – along with the emerging dimensions of Veracity and Value. Methodologically, the paper surveys the diverse approaches and technologies employed in Big Data analytics, ranging from descriptive and predictive analytics to advanced machine learning algorithms. Provost and Fawcett (2013) and Zikopoulos et al. (2011) provide valuable insights into the practical applications of these methodologies across sectors such as healthcare, finance, marketing, and governance. However, amidst the promise of Big Data lies a myriad of challenges, including data quality issues, scalability constraints, and ethical dilemmas. Davenport and Harris (2007) discuss the imperative of organizations to compete on analytics while navigating the complexities of managing large and heterogeneous datasets. Moreover, the paper examines the ethical, legal, and social considerations inherent in Big Data practices, emphasizing the importance of privacy, consent, fairness, transparency, and accountability. These concerns are further underscored by recent controversies surrounding data privacy breaches and algorithmic biases, prompting calls for enhanced regulatory frameworks and ethical guidelines. Looking ahead, the paper outlines future research directions in Big Data, including the development of ethical frameworks for governance, the integration of diverse data sources, and the exploration of emerging applications in smart cities, precision agriculture, and autonomous vehicles. In conclusion, while Big Data holds immense potential for driving innovation and progress, its responsible and ethical utilization is paramount to ensuring equitable and sustainable societal outcomes.
8
Berriche, Amira, Dominique Crié, and Michel Calciu. "Une Approche Computationnelle Ancrée : Étude de cas des tweets du challenge #Movember en prévention de santé masculine." Décisions Marketing N° 112, no. 4 (January 25, 2024): 79–103. http://dx.doi.org/10.3917/dm.112.0079.
Full textAbstract:
• Objectif L’objectif de cette étude est de présenter l’approche méthodologique computationnelle ancrée qui repose sur une démarche d’interprétation par les chercheurs des thèmes détectés par les algorithmes d’intelligence artificielle (IA) puis de l’appliquer au cas #Movember. • Méthodologie Une classification non supervisée par LDA et une analyse de sentiment ont été réalisées sur 144 906 tweets provenant de différents pays participants (France, Italie, Belgique, Australie, USA, UK, Arabie Saoudite, etc.). • Résultats Les résultats montrent que le processus de l’engagement individuel au mouvement social #Movember est composé de trois principaux éléments : (1) 4 segments d’engagement individuel (sympathisants, conscients, engagés et maintiens), (2) émotions collectives (positives et négatives) et (3) facteurs cognitifs et motivationnels (calcul bénéfices-coûts, efficacité collective et identité). • Implications managériales Les résultats proposent des actions marketing adaptées à chaque segment pour aider à la fois les organisateurs du mouvement #Movember et les professionnels de santé (PS) à atteindre deux principaux objectifs : (1) dépistage et (2) notoriété, recrutement et collecte de dons, grâce au big data, par le ciblage des personnes avec antécédents familiaux. • Originalité Les recherches sur #Movember utilisent habituellement les algorithmes supervisés qui présentent plusieurs limites tels que biais de confirmation, manque de répétabilité et une exigence en temps. Ce travail utilise le modèle non supervisé LDA pour identifier des concepts latents par la machine dans une perspective computationnelle ancrée (Computational Grounded Theory, CGT).
9
Polton, Dominique. "Les données de santé." médecine/sciences 34, no. 5 (May 2018): 449–55. http://dx.doi.org/10.1051/medsci/20183405018.
Full textAbstract:
En matière de santé comme dans d’autres secteurs, une masse croissante de données numérisées provenant de diverses sources est disponible et exploitable. C’est l’un des domaines où le potentiel du Big data apparaît très prometteur, avec de multiples innovations au bénéfice des patients et du système (accélération de la recherche et développement, connaissance des maladies, des facteurs de risque, médecine personnalisée, aide au diagnostic et au traitement, rôle accru des patients, pharmacovigilance, etc.), même si des inquiétudes s’expriment aussi vis-à-vis des impacts sociétaux, économiques et éthiques que le recours croissant aux algorithmes et à l’intelligence artificielle pourrait induire. Développer l’usage de ces données constitue un objectif stratégique de tous les systèmes de santé, et de ce point de vue le Système national de données de santé (SNDS) constitue pour la France un patrimoine intéressant, mais qui demande à être complété et enrichi.
10
Bullich, Vincent, and Viviane Clavier. "Production des données, « Production de la société ». Les Big Data et algorithmes au regard des Sciences de l’information et de la communication." Les Enjeux de l'information et de la communication N° 19/2, no. 2 (2018): 5. http://dx.doi.org/10.3917/enic.025.0005.
Full textDissertations / Theses on the topic "Big Data et algorithmes"
1
Ho, Zhen Wai Olivier. "Contributions aux algorithmes stochastiques pour le Big Data et à la théorie des valeurs extrèmes multivariés." Thesis, Bourgogne Franche-Comté, 2018. http://www.theses.fr/2018UBFCD025/document.
Full textAbstract:
La thèse comporte deux parties distinctes. La première partie concerne des modèles pour les extrêmes multivariés.On donne une construction de vecteurs aléatoires multivariés à variations régulières. La construction se base sur une extension multivariée d'un lemme de Breiman établissant la propriété de variation régulière d'un produit $RZ$ de variable aléatoire avec $R$ positive à variation régulière et $Z$ positive suffisamment intégrable. En prenant $mathbf{Z}$ multivarié et suffisamment intégrable, on montre que $Rmathbf{Z}$ est un vecteur aléatoire à variations régulières et on caractérise sa mesure limite. On montre ensuite que pour $mathbf{Z}$ de loi bien choisie, on retrouve des modèles stables classiques comme le modèle t-extremal, Hüsler-Reiss, etc. Puis, on étend notre construction pour considérer la notion de variation régulière multivariée non standard. On montre ensuite que le modèle de Pareto (qu'on appelle Hüsler-Reiss Pareto) associé au modèle max-stable Hüsler-Reiss forme une famille exponentielle complète. On donne quelques propriétés du modèle Hüsler-Reiss Pareto puis on propose un algorithme de simulation exacte. On étudie l'inférence par le maximum de vraisemblance. Finalement, on considère une extension du modèle Hüsler-Reiss Pareto utilisant la notion de variation régulière non standard. On étudie l'inférence par le maximum de vraisemblance du modèle généralisé et on propose une méthode d'estimation des paramètres. On donne une étude numérique sur l'estimateur du maximum de vraisemblance pour le modèle Hüsler-Reiss Pareto. Dans la second partie qui concerne l'apprentissage statistique, on commence par donner une borne sur la valeur singulière minimale d'une matrice perturbée par l'ajout d'une colonne. On propose alors un algorithme de sélection de colonne afin d'extraire les caractéristiques de la matrice. On illustre notre algorithme sur des données réelles de séries temporelles où chaque série est pris comme étant une colonne de la matrice. Deuxièmement, on montre que si une matrice $X$ à une propriété d'incohérence alors $X$ possède aussi une version affaiblie de la propriété NSP (null space property). Puis, on s'intéresse au problème de sélection de matrice incohérente. A partir d'une matrice $Xin mathbb{R}^{n imes p}$ et $mu>0$, on cherche la plus grande sous-matrice de $X$ avec une cohérence inférieure à $mu$. Ce problème est formulé comme un programme linéaire avec contrainte quadratique sur ${0,1}^p$. Comme ce problème est NP-dur, on considère une relaxation sur la sphère et on obtient une borne sur l'erreur lorsqu'on considère le problème relaxé. Enfin, on analyse l'algorithme de gradient stochastique projeté pour l'analyse en composante principale online. On montre qu'en espérance, l'algorithme converge vers un vecteur propre maximum et on propose un algorithme pour sélectionner le pas de l'algorithme. On illustre ensuite cet algorithme par une expérience de simulationThis thesis in divided in two parts. The first part studies models for multivariate extremes. We give a method to construct multivariate regularly varying random vectors. The method is based on a multivariate extension of a Breiman Lemma that states that a product $RZ$ of a random non negative regularly varying variable $R$ and a non negative $Z$ sufficiently integrable is also regularly varying. Replacing $Z$ with a random vector $mathbf{Z}$, we show that the product $Rmathbf{Z}$ is regularly varying and we give a characterisation of its limit measure. Then, we show that taking specific distributions for $mathbf{Z}$, we obtain classical max-stable models. We extend our result to non-standard regular variations. Next, we show that the Pareto model associated with the Hüsler-Reiss max-stable model forms a full exponential family. We show some properties of this model and we give an algorithm for exact simulation. We study the properties of the maximum likelihood estimator. Then, we extend our model to non-standard regular variations. To finish the first part, we propose a numerical study of the Hüsler-Reiss Pareto model.In the second part, we start by giving a lower bound of the smallest singular value of a matrix perturbed by appending a column. Then, we give a greedy algorithm for feature selection and we illustrate this algorithm on a time series dataset. Secondly, we show that an incoherent matrix satisfies a weakened version of the NSP property. Thirdly, we study the problem of column selection of $Xinmathbb{R}^{n imes p}$ given a coherence threshold $mu$. This means we want the largest submatrix satisfying some coherence property. We formulate the problem as a linear program with quadratic constraint on ${0,1}^p$. Then, we consider a relaxation on the sphere and we bound the relaxation error. Finally, we study the projected stochastic gradient descent for online PCA. We show that in expectation, the algorithm converges to a leading eigenvector and we suggest an algorithm for step-size selection. We illustrate this algorithm with a numerical experiment
2
Bach, Tran. "Algorithmes avancés de DCA pour certaines classes de problèmes en apprentissage automatique du Big Data." Electronic Thesis or Diss., Université de Lorraine, 2019. http://www.theses.fr/2019LORR0255.
Full textAbstract:
De nos jours, le Big Data est devenu essentiel et omniprésent dans tous les domaines. Par conséquence, il est nécessaire de développer des techniques innovantes et efficaces pour traiter la croissance rapide du volume des masses de données. Nous considérons les problèmes suivants dans le contexte de Big Data : la sélection de groupes de variables pour la régression logistique multi-classes, la réduction de dimension par t-SNE (« t-distributed Stochastic Neighbor Embedding « en anglais) et l'apprentissage en profondeur pour la classification non-supervisée (« Deep Clustering « en anglais). Nous développons des algorithmes DC (Difference of Convex functions) avancés pour ces problèmes, qui sont basés sur la programmation DC et DCA (DC Algorithm) -- des outils puissants pour les problèmes d'optimisation non-convexes non-différentiables. Dans la première partie, nous étudions le problème de la sélection de groupes de variables pour la régression logistique multi-classes. Nous résolvons ce problème en utilisant des DCAs avancés – Stochastic DCA et DCA-Like. Plus précisément, Stochastic DCA se spécialise dans le problème de la minimisation de la grande somme des fonctions DC, et ne nécessite qu'un sous-ensemble de fonctions DC à chaque itération. DCA-Like relaxe la condition de convexité de la deuxième composante DC en assurant la convergence. Accelerated DCA-Like intègre la technique d'accélération de Nesterov dans DCA-Like pour améliorer sa performance. Les expériences numériques sur plusieurs jeux de données benchmark de grande taille montrent l'efficacité de tous les algorithmes proposés en termes de temps d'exécution et de qualité de la solution. La deuxième partie concerne t-SNE, une technique efficace de réduction de dimension non linéaire. t-SNE est modélisé sous forme d'un problème d'optimisation non-convexe. Motivés par le caractère novateur de DCA-Like et Accelerated DCA-Like, nous développons ces deux algorithmes pour résoudre le problème t-SNE. La supériorité de nos algorithmes, appliqués à la visualisation de données, par rapport aux méthodes existantes est illustrée via des expériences numériques réalisées sur les jeux de données de très grande taille. La troisième partie est consacrée à la classification non-supervisée par l'apprentissage en profondeur. Dans la première application, nous proposons deux algorithmes basés sur DCA pour combiner t-SNE avec MSSC (Minimum Sum-of-Squares Clustering) par ces deux approches : « tandem analysis » et joint-clustering. La deuxième application considère le clustering en utilisant l'auto-encodeur. Nous avons proposé une extension d'une classe d'algorithmes de joint-clustering pour résoudre le problème de mise à l'échelle de données (« scaling problem » en anglais), et appliqué pour un cas spécifique de joint-clustering avec MSSC. Les résultats numériques sur plusieurs jeux de données benchmark montre l'efficacité de notre algorithme comparé aux méthodes existantesBig Data has become gradually essential and ubiquitous in all aspects nowadays. Therefore, there is an urge to develop innovative and efficient techniques to deal with the rapid growth in the volume of data. This dissertation considers the following problems in Big Data: group variable selection in multi-class logistic regression, dimension reduction by t-SNE (t-distributed Stochastic Neighbor Embedding), and deep clustering. We develop advanced DCAs (Difference of Convex functions Algorithms) for these problems, which are based on DC Programming and DCA – the powerful tools for non-smooth non-convex optimization problems. Firstly, we consider the problem of group variable selection in multi-class logistic regression. We tackle this problem by using recently advanced DCAs -- Stochastic DCA and DCA-Like. Specifically, Stochastic DCA specializes in the large sum of DC functions minimization problem, which only requires a subset of DC functions at each iteration. DCA-Like relaxes the convexity condition of the second DC component while guaranteeing the convergence. Accelerated DCA-Like incorporates the Nesterov's acceleration technique into DCA-Like to improve its performance. The numerical experiments in benchmark high-dimensional datasets show the effectiveness of proposed algorithms in terms of running time and solution quality. The second part studies the t-SNE problem, an effective non-linear dimensional reduction technique. Motivated by the novelty of DCA-Like and Accelerated DCA-Like, we develop two algorithms for the t-SNE problem. The superiority of proposed algorithms in comparison with existing methods is illustrated through numerical experiments for visualization application. Finally, the third part considers the problem of deep clustering. In the first application, we propose two algorithms based on DCA to combine t-SNE with MSSC (Minimum Sum-of-Squares Clustering) by following two approaches: “tandem analysis” and joint-clustering. The second application considers clustering with auto-encoder (a well-known type of neural network). We propose an extension to a class of joint-clustering algorithms to overcome the scaling problem and applied for a specific case of joint-clustering with MSSC. Numerical experiments on several real-world datasets show the effectiveness of our methods in rapidity and clustering quality, compared to the state-of-the-art methods
3
Chuchuk, Olga. "Optimisation de l'accès aux données au CERN et dans la Grille de calcul mondiale pour le LHC (WLCG)." Electronic Thesis or Diss., Université Côte d'Azur, 2024. http://www.theses.fr/2024COAZ4005.
Full textAbstract:
La Grille de calcul mondiale pour le LHC (WLCG) offre une infrastructure informatique distribuée considérable dédiée à la communauté scientifique impliquée dans le Grand Collisionneur de Hadrons (LHC) du CERN. Avec un stockage total d'environ un exaoctet, le WLCG répond aux besoins de traitement et de stockage des données de milliers de scientifiques internationaux. À mesure que la phase du High-Luminosity LHC (HL-LHC) approche, le volume de données à analyser augmentera considérablement, dépassant les gains attendus grâce à l'avancement de la technologie de stockage. Par conséquent, de nouvelles approches pour un accès et une gestion efficaces des données, telles que les caches, deviennent essentielles. Cette thèse se plonge dans une exploration exhaustive de l'accès au stockage au sein du WLCG, dans le but d'améliorer le débit scientifique global tout en limitant les coûts. Au cœur de cette recherche se trouve l'analyse des journaux d'accès aux fichiers réels provenant du système de surveillance du WLCG, mettant en évidence les véritables schémas d'utilisation.Dans un contexte scientifique, la mise en cache a des implications profondes. Contrairement à des applications plus commerciales telles que la diffusion de vidéos, les caches de données scientifiques traitent des tailles de fichiers variables, allant de quelques octets à plusieurs téraoctets. De plus, les associations logiques inhérentes entre les fichiers influencent considérablement les schémas d'accès des utilisateurs. La recherche traditionnelle sur la mise en cache s'est principalement concentrée sur des tailles de fichiers uniformes et des modèles de référence indépendants. Au contraire, les charges de travail scientifiques rencontrent des variations de taille de fichier, et les interconnexions logiques entre les fichiers influencent de manière significative les schémas d'accès des utilisateurs.Mes investigations montrent comment l'organisation hiérarchique des données du LHC, en particulier leur compartimentation en "datasets", influence les schémas de demande. Reconnaissant cette opportunité, j'introduis des algorithmes de mise en cache innovants qui mettent l'accent sur la connaissance spécifique des datasets et je compare leur efficacité avec les stratégies traditionnelles axées sur les fichiers. De plus, mes découvertes mettent en évidence le phénomène des "hits retardés" déclenché par une connectivité limitée entre les sites de calcul et de stockage, mettant en lumière ses répercussions potentielles sur l'efficacité de la mise en cache.Reconnaissant le défi de longue date que représente la prédiction de la Popularité des Données dans la communauté de la Physique des Hautes Énergies (PHE), en particulier avec les énigmes de stockage à l'approche de l'ère du HL-LHC, ma recherche intègre des outils de Machine Learning (ML). Plus précisément, j'utilise l'algorithme Random Forest, connu pour sa pertinence dans le traitement des Big Data. En utilisant le ML pour prédire les futurs schémas de réutilisation des fichiers, je présente une méthode en deux étapes pour informer les politiques d'éviction de cache. Cette stratégie combine la puissance de l'analyse prédictive et des algorithmes établis d'éviction de cache, créant ainsi un système de mise en cache plus résilient pour le WLCG.En conclusion, cette recherche souligne l'importance de services de stockage robustes, suggérant une orientation vers des caches sans état pour les petits sites afin d'alléger les exigences complexes de gestion de stockage et d'ouvrir la voie à un niveau supplémentaire dans la hiérarchie de stockage. À travers cette thèse, je vise à naviguer à travers les défis et les complexités du stockage et de la récupération de données, élaborant des méthodes plus efficaces qui résonnent avec les besoins évolutifs du WLCG et de sa communauté mondialeThe Worldwide LHC Computing Grid (WLCG) offers an extensive distributed computing infrastructure dedicated to the scientific community involved with CERN's Large Hadron Collider (LHC). With storage that totals roughly an exabyte, the WLCG addresses the data processing and storage requirements of thousands of international scientists. As the High-Luminosity LHC phase approaches, the volume of data to be analysed will increase steeply, outpacing the expected gain through the advancement of storage technology. Therefore, new approaches to effective data access and management, such as caches, become essential. This thesis delves into a comprehensive exploration of storage access within the WLCG, aiming to enhance the aggregate science throughput while limiting the cost. Central to this research is the analysis of real file access logs sourced from the WLCG monitoring system, highlighting genuine usage patterns.In a scientific setting, caching has profound implications. Unlike more commercial applications such as video streaming, scientific data caches deal with varying file sizes—from a mere few bytes to multiple terabytes. Moreover, the inherent logical associations between files considerably influence user access patterns. Traditional caching research has predominantly revolved around uniform file sizes and independent reference models. Contrarily, scientific workloads encounter variances in file sizes, and logical file interconnections significantly influence user access patterns.My investigations show how LHC's hierarchical data organization, particularly its compartmentalization into datasets, impacts request patterns. Recognizing the opportunity, I introduce innovative caching policies that emphasize dataset-specific knowledge, and compare their effectiveness with traditional file-centric strategies. Furthermore, my findings underscore the "delayed hits" phenomenon triggered by limited connectivity between computing and storage locales, shedding light on its potential repercussions for caching efficiency.Acknowledging the long-standing challenge of predicting Data Popularity in the High Energy Physics (HEP) community, especially with the upcoming HL-LHC era's storage conundrums, my research integrates Machine Learning (ML) tools. Specifically, I employ the Random Forest algorithm, known for its suitability with Big Data. By harnessing ML to predict future file reuse patterns, I present a dual-stage method to inform cache eviction policies. This strategy combines the power of predictive analytics and established cache eviction algorithms, thereby devising a more resilient caching system for the WLCG. In conclusion, this research underscores the significance of robust storage services, suggesting a direction towards stateless caches for smaller sites to alleviate complex storage management requirements and open the path to an additional level in the storage hierarchy. Through this thesis, I aim to navigate the challenges and complexities of data storage and retrieval, crafting more efficient methods that resonate with the evolving needs of the WLCG and its global community
4
Défossez, Gautier. "Le système d'information multi-sources du Registre général des cancers de Poitou-Charentes. Conception, développement et applications à l'ère des données massives en santé." Thesis, Poitiers, 2021. http://theses.univ-poitiers.fr/64594/2021-Defossez-Gautier-These.
Full textAbstract:
Les registres du cancer sont au plan international l’outil de référence pour produire une vision exhaustive (non biaisée) du poids, de la dynamique et de la gravité du cancer dans la population générale. Leur travail de classification et de codage des diagnostics selon des normes internationales confère aux données finales une qualité spécifique et une comparabilité dans le temps et dans l’espace qui les rendent incontournables pour décrire l’évolution et la prise en charge du cancer dans un environnement non contrôlé. Leur travail repose sur un processus d’enquête rigoureux dont la complexité est largement dépendante des capacités à accéder et à rassembler efficacement toutes les données utiles concernant un même individu. Créé en 2007, le Registre Général des Cancers de Poitou-Charentes (RGCPC) est un registre de génération récente, débuté à une période propice à la mise en œuvre d’une réflexion sur l’optimisation du processus d’enregistrement. Porté par l’informatisation des données médicales et l’interopérabilité croissante des systèmes d’information, le RGCPC a développé et expérimenté sur 10 ans un système d’information multi-sources associant des méthodes innovantes de traitement et de représentation de l’information fondées sur la réutilisation de données standardisées produites pour d’autres finalités.Dans une première partie, ce travail présente les principes fondateurs et l’implémentation d’un système capable de rassembler des volumes élevés de données, hautement qualifiantes et structurées, et rendues interopérables sur le plan sémantique pour faire l’objet d’approches algorithmiques. Les données sont collectées pluri annuellement auprès de 110 partenaires représentant sept sources de données (cliniques, biologiques et médico-administratives). Deux algorithmes assistent l’opérateur du registre en dématérialisant une grande partie des tâches préalables à l’enregistrement des tumeurs. Un premier algorithme crée les tumeurs et leurs caractéristiques (publication), puis un 2ème algorithme modélise le parcours de soin de chaque individu selon une séquence ordonnée d’évènements horodatés consultable au sein d’une interface sécurisée (publication). Des approches de machine learning sont testées pour contourner l’éventuelle absence de codification des prélèvements anatomopathologiques (publication).La deuxième partie s’intéresse au large champ de recherche et d’évaluation rendu possible par la disponibilité de ce système d’information intégré. Des appariements avec d’autres données de santé ont été testés, dans le cadre d’autorisations réglementaires, pour enrichir la contextualisation et la connaissance des parcours de soins, et reconnaître le rôle stratégique des registres du cancer pour l’évaluation en « vie réelle » des pratiques de soins et des services de santé (preuve de concept) : dépistage, diagnostic moléculaire, traitement du cancer, pharmaco épidémiologie (quatre publications principales). L’appariement des données du RGCPC à celles du registre REIN (insuffisance rénale chronique terminale) a constitué un cas d’usage veillant à expérimenter un prototype de plateforme dédiée au partage collaboratif des données massives en santé (publication).La dernière partie de ce travail propose une discussion ouverte sur la pertinence des solutions proposées face aux exigences de qualité, de coût et de transférabilité, puis dresse les perspectives et retombées attendues pour la surveillance, l’évaluation et la recherche à l’ère des données massives en santéPopulation-based cancer registries (PBCRs) are the best international option tool to provide a comprehensive (unbiased) picture of the weight, incidence and severity of cancer in the general population. Their work in classifying and coding diagnoses according to international rules gives to the final data a specific quality and comparability in time and space, thus building a decisive knowledge database for describing the evolution of cancers and their management in an uncontrolled environment. Cancer registration is based on a thorough investigative process, for which the complexity is largely related to the ability to access all the relevant data concerning the same individual and to gather them efficiently. Created in 2007, the General Cancer Registry of Poitou-Charentes (RGCPC) is a recent generation of cancer registry, started at a conducive time to devote a reflection about how to optimize the registration process. Driven by the computerization of medical data and the increasing interoperability of information systems, the RGCPC has experimented over 10 years a multi-source information system combining innovative methods of information processing and representation, based on the reuse of standardized data usually produced for other purposes.In a first section, this work presents the founding principles and the implementation of a system capable of gathering large amounts of data, highly qualified and structured, with semantic alignment to subscribe to algorithmic approaches. Data are collected on multiannual basis from 110 partners representing seven data sources (clinical, biological and medical administrative data). Two algorithms assist the cancer registrar by dematerializing the manual tasks usually carried out prior to tumor registration. A first algorithm generate automatically the tumors and its various components (publication), and a second algorithm represent the care pathway of each individual as an ordered sequence of time-stamped events that can be access within a secure interface (publication). Supervised machine learning techniques are experimented to get around the possible lack of codification of pathology reports (publication).The second section focuses on the wide field of research and evaluation achieved through the availability of this integrated information system. Data linkage with other datasets were tested, within the framework of regulatory authorizations, to enhance the contextualization and knowledge of care pathways, and thus to support the strategic role of PBCRs for real-life evaluation of care practices and health services research (proof of concept): screening, molecular diagnosis, cancer treatment, pharmacoepidemiology (four main publications). Data from the RGCPC were linked with those from the REIN registry (chronic end-stage renal failure) as a use case for experimenting a prototype platform dedicated to the collaborative sharing of massive health data (publication).The last section of this work proposes an open discussion on the relevance of the proposed solutions to the requirements of quality, cost and transferability, and then sets out the prospects and expected benefits in the field of surveillance, evaluation and research in the era of big data
5
Brahem, Mariem. "Optimisation de requêtes spatiales et serveur de données distribué - Application à la gestion de masses de données en astronomie." Thesis, Université Paris-Saclay (ComUE), 2019. http://www.theses.fr/2019SACLV009/document.
Full textAbstract:
Les masses de données scientifiques générées par les moyens d'observation modernes, dont l’observation spatiale, soulèvent des problèmes de performances récurrents, et ce malgré les avancées des systèmes distribués de gestion de données. Ceci est souvent lié à la complexité des systèmes et des paramètres qui impactent les performances et la difficulté d’adapter les méthodes d’accès au flot de données et de traitement.Cette thèse propose de nouvelles techniques d'optimisations logiques et physiques pour optimiser les plans d'exécution des requêtes astronomiques en utilisant des règles d'optimisation. Ces méthodes sont intégrées dans ASTROIDE, un système distribué pour le traitement de données astronomiques à grande échelle.ASTROIDE allie la scalabilité et l’efficacité en combinant les avantages du traitement distribué en utilisant Spark avec la pertinence d’un optimiseur de requêtes astronomiques.Il permet l'accès aux données à l'aide du langage de requêtes ADQL, couramment utilisé.Il implémente des algorithmes de requêtes astronomiques (cone search, kNN search, cross-match, et kNN join) en exploitant l'organisation physique des données proposée.En effet, ASTROIDE propose une méthode de partitionnement des données permettant un traitement efficace de ces requêtes grâce à l'équilibrage de la répartition des données et à l'élimination des partitions non pertinentes. Ce partitionnement utilise une technique d’indexation adaptée aux données astronomiques, afin de réduire le temps de traitement des requêtesThe big scientific data generated by modern observation telescopes, raises recurring problems of performances, in spite of the advances in distributed data management systems. The main reasons are the complexity of the systems and the difficulty to adapt the access methods to the data. This thesis proposes new physical and logical optimizations to optimize execution plans of astronomical queries using transformation rules. These methods are integrated in ASTROIDE, a distributed system for large-scale astronomical data processing.ASTROIDE achieves scalability and efficiency by combining the benefits of distributed processing using Spark with the relevance of an astronomical query optimizer.It supports the data access using the query language ADQL that is commonly used.It implements astronomical query algorithms (cone search, kNN search, cross-match, and kNN join) tailored to the proposed physical data organization.Indeed, ASTROIDE offers a data partitioning technique that allows efficient processing of these queries by ensuring load balancing and eliminating irrelevant partitions. This partitioning uses an indexing technique adapted to astronomical data, in order to reduce query processing time
6
Jlassi, Aymen. "Optimisation de la gestion des ressources sur une plate-forme informatique du type Big Data basée sur le logiciel Hadoop." Thesis, Tours, 2017. http://www.theses.fr/2017TOUR4042.
Full textAbstract:
L'entreprise "Cyres-group" cherche à améliorer le temps de réponse de ses grappes Hadoop et la manière dont les ressources sont exploitées dans son centre de données. Les idées sous-jacentes à la réduction du temps de réponse sont de faire en sorte que (i) les travaux soumis se terminent au plus tôt et que (ii) le temps d'attente de chaque utilisateur du système soit réduit. Nous identifions deux axes d'amélioration : 1. nous décidons d'intervenir pour optimiser l'ordonnancement des travaux sur une plateforme Hadoop. Nous considérons le problème d'ordonnancement d'un ensemble de travaux du type MapReduce sur une plateforme homogène. 2. Nous décidons d'évaluer et proposer des outils capables (i) de fournir plus de flexibilité lors de la gestion des ressources dans le centre de données et (ii) d'assurer l'intégration d'Hadoop dans des infrastructures Cloud avec le minimum de perte de performance. Dans une première étude, nous effectuons une revue de la littérature. À la fin de cette étape, nous remarquons que les modèles mathématiques proposés dans la littérature pour le problème d'ordonnancement ne modélisent pas toutes les caractéristiques d'une plateforme Hadoop. Nous proposons à ce niveau un modèle plus réaliste qui prend en compte les aspects les plus importants tels que la gestion des ressources, la précédence entre les travaux, la gestion du transfert des données et la gestion du réseau. Nous considérons une première modélisation simpliste et nous considérons la minimisation de la date de fin du dernier travail (Cmax) comme critère à optimiser. Nous calculons une borne inférieure à l'aide de la résolution du modèle mathématique avec le solveur CPLEX. Nous proposons une heuristique (LocFirst) et nous l'évaluons. Ensuite, nous faisons évoluer notre modèle et nous considérons, comme fonction objective, la somme des deux critères identifiés depuis la première étape : la minimisation de la somme pondérée des dates de fin des travaux ( ∑ wjCj) et la minimisation du (Cmax). Nous cherchons à minimiser la moyenne pondérée des deux critères, nous calculons une borne inférieure et nous proposons deux heuristiques de résolution"Cyres-Group" is working to improve the response time of his clusters Hadoop and optimize how the resources are exploited in its data center. That is, the goals are to finish work as soon as possible and reduce the latency of each user of the system. Firstly, we decide to work on the scheduling problem in the Hadoop system. We consider the problem as the problem of scheduling a set of jobs on a homogeneous platform. Secondly, we decide to propose tools, which are able to provide more flexibility during the resources management in the data center and ensure the integration of Hadoop in Cloud infrastructures without unacceptable loss of performance. Next, the second level focuses on the review of literature. We conclude that, existing works use simple mathematical models that do not reflect the real problem. They ignore the main characteristics of Hadoop software. Hence, we propose a new model ; we take into account the most important aspects like resources management and the relations of precedence among tasks and the data management and transfer. Thus, we model the problem. We begin with a simplistic model and we consider the minimisation of the Cmax as the objective function. We solve the model with mathematical solver CPLEX and we compute a lower bound. We propose the heuristic "LocFirst" that aims to minimize the Cmax. In the third level, we consider a more realistic modelling of the scheduling problem. We aim to minimize the weighted sum of the following objectives : the weighted flow time ( ∑ wjCj) and the makespan (Cmax). We compute a lower bound and we propose two heuristics to resolve the problem
7
Saffarian, Azadeh. "Algorithmes de prédiction et de recherche de multi-structures d'ARN." Phd thesis, Université des Sciences et Technologie de Lille - Lille I, 2011. http://tel.archives-ouvertes.fr/tel-00832700.
Full textAbstract:
L'ARN (acide ribonucléique) est une molécule ubiquitaire qui joue plusieurs rôles fondamentaux au sein de la cellule: synthèse des protéines avec les ARN messagers, activité catalytique ou implicationdans la régulation, les ARN non-codants. Les nouvelles technologies de séquençage à haut-débit permettent de produire des milliards de séquences à moindre coût, posant de manière cruciale la question de l'analyse de ces données. L'objectif de cette thèse est de définir de nouvelles méthodes computationnelles pour aider à l'analyse de ces séquences dans le cas des ARN non-codants. Dans cette perspective, la "structure secondaire" d'un ARN, formée par l'ensemble des appariements entrebases, délivre des informations utiles pour étudier la fonction de l'ARN. Notre travail se concentre plus particulièrement sur l'ensemble des structures potentielles que peut adopter une séquence d'ARN donnée, ensemble que nous appelons "multi-structure". Nous apportons deux contributions: un algorithme pour générer systématiquement toutes les structures localement optimales composantune multi-structure, et un algorithme basé sur la recherche d'unemulti-structure pour identifier un ARN non-codant dans une séquence génomique. Ces résultats ont été mis en oeuvre dans deux logiciels, Alterna et Regliss, appliqués avec succès à des ensembles de test.
8
Phan, Duy-Hung. "Algorithmes d'aggrégation pour applications Big Data." Electronic Thesis or Diss., Paris, ENST, 2016. http://www.theses.fr/2016ENST0043.
Full textAbstract:
Les bases de données traditionnelles sont confrontées à des problèmes de scalabilité et d'efficacité en raison d’importants volumes de données. Ainsi, les systèmes de gestion de base de données modernes, tels que Apache Hadoop et Spark, peuvent désormais être distribués sur des clusters de milliers de machines: ces systèmes sont donc devenus les principaux outils pour le traitement des données à grande échelle. De nombreuses optimisations ont été développées pour les bases de données conventionnelles, cependant celles-ci ne peuvent être appliquées aux nouvelles architectures et modèles de programmation. Dans ce contexte, cette thèse vise à optimiser une des opérations les plus prédominantes dans le traitement des données : l'agrégation de données pour ces systèmes à grande échelle. Nos principales contributions sont les optimisations logiques et physiques de l'agrégation de grands volumes de données. Ces optimisations sont fortement interconnectées : le problème d'optimisation d'agrégation de données ne pourrait être entièrement résolu si l’une d’entre elles venait à manquer. Par ailleurs, nous avons intégré les optimisations dans le moteur d'optimisation multi-requêtes, ce qui est transparent pour les usagers. Le moteur, les optimisations logiques et physiques proposées dans cette thèse forment une solution complété exécutable et prête à répondre aux requêtes d'agrégation de données à grande échelle. Nos optimisations ont été évaluées de manière théorique et expérimentale. Les résultats d'analyses ont démontré que le passage à l’échelle et l’efficacité de nos algorithmes et techniques surpassent les résultats des études antérieuresTraditional databases are facing problems of scalability and efficiency dealing with a vast amount of big-data. Thus, modern data management systems that scale to thousands of nodes, like Apache Hadoop and Spark, have emerged and become the de-facto platforms to process data at massive scales. In such systems, many data processing optimizations that were well studied in the database domain have now become futile because of the novel architectures and programming models. In this context, this dissertation pledged to optimize one of the most predominant operations in data processing: data aggregation for such systems.Our main contributions were the logical and physical optimizations for large-scale data aggregation, including several algorithms and techniques. These optimizations are so intimately related that without one or the other, the data aggregation optimization problem would not be solved entirely. Moreover, we integrated these optimizations in our multi-query optimization engine, which is totally transparent to users. The engine, the logical and physical optimizations proposed in this dissertation formed a complete package that is runnable and ready to answer data aggregation queries at massive scales. We evaluated our optimizations both theoretically and experimentally. The theoretical analyses showed that our algorithms and techniques are much more scalable and efficient than prior works. The experimental results using a real cluster with synthetic and real datasets confirmed our analyses, showed a significant performance boost and revealed various angles about our works. Last but not least, our works are published as open sources for public usages and studies
9
Malekian, Hajar. "La libre circulation et la protection des données à caractère personnel sur Internet." Thesis, Paris 2, 2017. http://www.theses.fr/2017PA020050.
Full textAbstract:
La protection des données à caractère personnel (DCP) constitue un droit fondamental autonome au sein de l’Union européenne (article 8 de la Charte des droits fondamentaux de l’Union européenne). En outre, la libre circulation de ces données et des services de la société de l’information, notamment des plateformes en ligne, est primordiale pour le développement de l’économie numérique dans le cadre du marché unique numérique européen. C’est dans ce contexte qu’un point d’équilibre entre la libre circulation et la protection des DCP fait l’objet du cadre juridique européen et français en matière de protection des DCP. Ainsi, dans cette étude, nous nous sommes intéressés en particulier aux enjeux liés à la mise en balance de ces deux intérêts. Ces enjeux suscitent une attention particulière notamment à l’ère des plateformes en ligne, du Big Data et de l’exploitation en masse des données à travers des algorithmes sophistiqués dotés de plus en plus d’autonomie et d’intelligenceFree flow of data and personal data protection on the Internet Protection of personal data is an autonomous fundamental right within the European Union (Article 8 of the Charter of Fundamental Rights of European Union). Moreover, free flow of personal data and free movement of information society services in particular online platforms is essential for the development of digital single market in European Union. The balance between free movement of data and personal data protection is subject of the European legal framework. However, the main challenge still remains to strike the right balance between effective personal data protection and free flow of this data and information society services. This balance is not an easy task especially in the age of online platforms, Big Data and processing algorithms like Machine Learning and Deep Learning
10
Kopylova, Evguenia. "Algorithmes bio-informatiques pour l'analyse de données de séquençage à haut débit." Phd thesis, Université des Sciences et Technologie de Lille - Lille I, 2013. http://tel.archives-ouvertes.fr/tel-00919185.
Full textAbstract:
Nucleotide sequence alignment is a method used to identify regions of similarity between organisms at the genomic level. In this thesis we focus on the alignment of millions of short sequences produced by Next-Generation Sequencing (NGS) technologies against a reference database. Particularly, we direct our attention toward the analysis of metagenomic and metatranscriptomic data, that is the DNA and RNA directly extracted for an environment. Two major challenges were confronted in our developed algorithms. First, all NGS technologies today are susceptible to sequencing errors in the form of nucleotide substitutions, insertions and deletions and error rates vary between 1-15%. Second, metagenomic samples can contain thousands of unknown organisms and the only means of identifying them is to align against known closely related species. To overcome these challenges we designed a new approximate matching technique based on the universal Levenshtein automaton which quickly locates short regions of similarity (seeds) between two sequences allowing 1 error of any type. Using seeds to detect possible high scoring alignments is a widely used heuristic for rapid sequence alignment, although most existing software are optimized for performing high similarity searches and apply exact seeds. Furthermore, we describe a new indexing data structure based on the Burst trie which optimizes the search for approximate seeds. We demonstrate the efficacy of our method in two implemented software, SortMeRNA and SortMeDNA. The former can quickly filter ribosomal RNA fragments from metatranscriptomic data and the latter performs full alignment for genomic and metagenomic data.
Books on the topic "Big Data et algorithmes"
1
Davenport, James Harold. Calcul formel: Systèmes et algorithmes de manipulations algébriques. Paris: Masson, 1987.
Find full text
2
Avenati, Olaf. Datalogie: Formes et imaginaires du numérique. Paris]: Éditions Loco, 2016.
Find full text
3
Divay, Michel. Algorithmes et structures de donne es ge ne riques: Cours et exercices corrige s en langage C. 2nd ed. Paris: Dunod, 2004.
Find full text
4
Italy) International Conference "Law Via the Internet" (2018 Florence. Knowledge of the law in the big data age. Amsterdam: IOS Press, 2019.
Find full text
5
Marcuse, Groupe. La liberté dans le coma: Essai sur l'identification électronique et les motifs de s'y opposer. Paris: Éditions La Lenteur, 2012.
Find full text
6
Menger, Pierre-Michel, and Simon Paye, eds. Big data et traçabilité numérique. Collège de France, 2017. http://dx.doi.org/10.4000/books.cdf.4987.
Full text
7
IAFRATE. Intelligence Artificielle et Big Data. ISTE Editions Ltd., 2018.
Find full text
8
Monino, Jean-Louis. Big Data, Open Data et Valorisation des Données. ISTE Editions Ltd., 2016.
Find full text
9
SEDKAOUI. Econo de Partage et le Big Data Analyc: L'Economie de Partage et le Big Data Analytics. ISTE Editions Ltd., 2019.
Find full text
10
UM, Yannick, and Joel NGUENA. Cohabitation des Technologies de Business Intelligence et de Big Data: Devenez développeur Business Intelligence et Big Data Par la Pratique. Independently Published, 2018.
Find full textBook chapters on the topic "Big Data et algorithmes"
1
Liu, Zheng, and Hao Wang. "Research on Process Diagnosis of Severe Accidents Based on Deep Learning and Probabilistic Safety Analysis." In Springer Proceedings in Physics, 624–34. Singapore: Springer Nature Singapore, 2023. http://dx.doi.org/10.1007/978-981-99-1023-6_54.
Full textAbstract:
AbstractSevere accident process diagnosis provides data basis for severe accident prognosis, positive and negative effect evaluation of Severe Accident Management Guidelines (SAMGs), especially to quickly diagnose Plant Damage State (PDS) for operators in the main control room or personnel in the Technical Support Center (TSC) based on historic data of the limited number of instruments during the operation transition from Emergency Operation Procedures (EOPs) to SAMGs. This diagnosis methodology is based on tens of thousands of simulations of severe accidents using the integrated analysis program MAAP. The simulation process is organized in reference to Level 1 Probabilistic Safety Analysis (L1 PSA) and EOPs. According to L1 PSA, the initial event of accidents and scenarios from the initial event to core damage are presented in Event Trees (ET), which include operator actions following up EOPs. During simulation, the time uncertainty of operations in scenarios is considered. Besides the big data collection of simulations, a deep learning algorithm, Convolutional Neural Network (CNN), has been used in this severe accident diagnosis methodology, to diagnose the type of severe accident initiation event, the breach size, breach location, and occurrence time of the initial event of LOCA, and action time by operators following up EOPs intending to take Nuclear Power Plant (NPP) back to safety state. These algorithms train classification and regression models with ET-based numerical simulations, such as the classification model of sequence number, break location, and regression model of the break size and occurrence time of initial event MBLOCA. Then these trained models take advantage of historic data from instruments in NPP to generate a diagnosis conclusion, which is automatically written into an input deck file of MAAP. This input deck originated from previous traceback efforts and provides a numerical analysis basis for predicting the follow-up process of a severe accident, which is conducive to severe accident management. Results of this paper show a theoretical possibility that under limited available instruments, this traceback and diagnosis method can automatically and quickly diagnose PDS when operation transit from EOPs to SAMGs and provide numerical analysis basis for severe accident process prognosis.
2
Azrour, Mourade, Mohammed Ouanan, Yousef Farhaoui, and Azidine Guezzaz. "Security Analysis of Ye et al. Authentication Protocol for Internet of Things." In Studies in Big Data, 67–74. Cham: Springer International Publishing, 2019. http://dx.doi.org/10.1007/978-3-030-12048-1_9.
Full text
3
Otmani, Ayoub, and Taoufik Benkaraache. "Towards a Strategy of Knowledge Management Within the Agence Nationale de la Conservation Foncière du Cadastre et de la Cartographie (ANCFCC)." In Studies in Big Data, 296–306. Cham: Springer International Publishing, 2019. http://dx.doi.org/10.1007/978-3-030-12048-1_30.
Full text
4
Tseng, Yi-Fan, and Chun-I. Fan. "Cryptanalysis on the Anonymity of Li et al.’s Ciphertext-Policy Attribute-Based Encryption Scheme." In Security with Intelligent Computing and Big-data Services, 98–104. Cham: Springer International Publishing, 2018. http://dx.doi.org/10.1007/978-3-319-76451-1_10.
Full text
5
Cáceres, Santiago, Francisco Valverde, Carlos E. Palau, Andreu Belsa Pellicer, Christos A. Gizelis, Dimosthenes Krassas, Hanane Becha, et al. "Towards Cognitive Ports of the Future." In Technologies and Applications for Big Data Value, 453–74. Cham: Springer International Publishing, 2021. http://dx.doi.org/10.1007/978-3-030-78307-5_20.
Full textAbstract:
AbstractIn modern societies, the rampant growth of data management technologies—that have access to data sources from a plethora of heterogeneous systems—enables data analysts to leverage their advantages to new areas and critical infrastructures. However, there is no global reference standard for data platform technology. Data platforms scenarios are characterized by a high degree of heterogeneity at all levels (middleware, application service, data/semantics, scalability, and governance), preventing deployment, federation, and interoperability of existing solutions. Although many initiatives are dealing with developing data platform architectures in diversified application domains, not many projects have addressed integration in port environments with the possibility of including cognitive services. Unlike other cases, port environment is a complex system that consists of multiple heterogeneous critical infrastructures, which are connected and dependent on each other. The key pillar is to define the design of a secure interoperable system facilitating the exchange of data through standardized data models, based on common semantics, and offering advanced interconnection capabilities leading to cooperation between different IT/IoT/Objects platforms. This contribution deals with scalability, interoperability, and standardization features of data platforms from a business point of view in a smart and cognitive port case study. The main goal is to design an innovative platform, named DataPorts, which will overcome these obstacles and provide an ecosystem where port authorities, external data platforms, transportation, and logistics companies can cooperate and create the basis to offer cognitive services. The chapter relates to knowledge and learning as well as to systems, methodologies, hardware, and tools cross-sectorial technology enablers of the AI, Data and Robotics Strategic Research, Innovation & Deployment Agenda (Milano et al., Strategic research, innovation and deployment agenda - AI, data and robotics partnership. Third release. Big Data Value Association, 2020).
6
Charvolin, Florian. "Chapitre 5. Le numérique et les big data : de la promesse à la réalité." In Les Sciences participatives au secours de la biodiversité, 57–66. Paris: Éditions Rue d’Ulm, 2019. http://dx.doi.org/10.4000/11syz.
Full text
7
Tseng, Yi-Fan. "Cryptanaylsis to Sowjanya et al.’s ABEs from ECC." In 2021 International Conference on Security and Information Technologies with AI, Internet Computing and Big-data Applications, 287–94. Cham: Springer International Publishing, 2022. http://dx.doi.org/10.1007/978-3-031-05491-4_29.
Full text
8
Stielike, Laura. "Migration Multiple? Big Data, Knowledge Practices and the Governability of Migration." In Research Methodologies and Ethical Challenges in Digital Migration Studies, 113–38. Cham: Springer International Publishing, 2021. http://dx.doi.org/10.1007/978-3-030-81226-3_5.
Full textAbstract:
AbstractThis chapter explores the big-data-based production of knowledge on migration. Following Mol (2002) and Scheel et al. (2019), it is analysed how migration and migrants are enacted through big-data-based research papers. The emerging sub-discipline of big-data-based migration research enacts migration and migrants in multiple ways that open up possibilities to rethink migration. However, this multiplicity of migration is held together by reference to three migration narratives—demography, integration and humanitarianism—which stand in stark contrast to these alternative enactments, as they all frame migration as something that needs to be governed. As the research papers aim at contributing to these research fields, they inscribe themselves into these migration narratives and thereby adopt the assumption of migration as an object of government.
9
Grall, Matthieu. "CNIL (Commission Nationale de l’Informatique et des Libertés) and Analysis of Big Data Projects in the Health Sector." In Healthcare and Artificial Intelligence, 235–39. Cham: Springer International Publishing, 2020. http://dx.doi.org/10.1007/978-3-030-32161-1_29.
Full text
10
Bates, Jo, Alessandro Checco, and Elli Gerakopoulou. "Worker Perspectives on Designs for a Crowdwork Co-operative." In Transforming Communications – Studies in Cross-Media Research, 415–43. Cham: Springer International Publishing, 2022. http://dx.doi.org/10.1007/978-3-030-96180-0_18.
Full textAbstract:
AbstractCrowdwork platforms such as Amazon Mechanical Turk (AMT) are a crucial infrastructural component of our global data assemblage. Through these platforms, low-paid crowdworkers perform the vital labour of manually labelling large-scale and complex datasets, labels that are needed to train machine learning and AI models (Tubaro et al., Big Data & Society, 7(1), 2020) and which enable the functioning of much digital technology, from niche applications to global platforms such as Google, Amazon and Facebook.In this chapter, we reflect on how a ‘design justice’ approach might be valuable to build on insights gained from a series of exploratory discussions we have engaged in with US-based crowdworkers about how a crowdworker co-operative might work in practice, and begin to sketch out a potential software architecture that could form the basis of future participative approaches to the design and development of a crowdworker co-operative.We begin by describing and reflecting on our own evolving methodology and how it fits with the ‘design justice’ lens we propose for future work. Following this, we present findings from our discussions with crowdworkers about how a crowdwork co-operative might work in practice, including what values workers would like to see embedded in the design. We then finish with the outline of a prototype software architecture for a crowdworker co-operative that could be used as a starting point in future design work in collaboration with crowdworkers.
Conference papers on the topic "Big Data et algorithmes"
1
Lu, Zongyu, Zhenxin Jiang, Zhe Wu, Xianzhi Song, Shanlin Ye, and Zihao Liu. "A Novel Rock Drillability Characterization and Prediction Method Based on Drilling Big Data and Unsupervised Clustering Algorithm." In 57th U.S. Rock Mechanics/Geomechanics Symposium. ARMA, 2023. http://dx.doi.org/10.56952/arma-2023-0394.
Full textAbstract:
ABSTRACT Rock drillability measures the ability of rock to resist damage by the drill bit. Accurate evaluation of rock drillability is crucial for understanding formation properties and selecting appropriate drill bits and drilling parameters. However, many existing intelligent evaluation methods for rock drillability mainly rely on supervised learning algorithms, which require a large number of rock core samples for drillability analysis data as labels. Core acquisition is difficult and expensive. To address this issue, this study proposes a method for continuous formation drillability evaluation using logging big data and unsupervised clustering algorithms. Firstly, self-organizing mapping (SOM) neural network is used to cluster logging data. The formation is then divided into six drillability grades by analyzing the rate of penetration (ROP) distribution of each type of logging data. We used this method to grade the drillability of the test well formation. The results show that the ROP of the formation decreases gradually with the increase of drillability grade, which verproves the effectiveness of our method. INTRODUCTION Rock drillability is a crucial factor that determines drilling efficiency, reflecting how easily the rock breaks during drilling operations. Accurate evaluation of rock drillability can help drilling engineers select drilling tools and parameters to improve drilling efficiency (Kong, et al. 2022). The microdrill bit experimental method is a classic approach for evaluating drillability. It involves recording the time it takes for a miniature gear or PDC bit to drill 2.4mm under a specific drilling pressure and rotation speed condition, and calculating the formation drillability based on standard formulas and recorded time. (Zhang & Xue, 2019; Chen, et al. 2010). In the experimental method, the test rock core samples are not in the real underground environment, and the accuracy of the results is difficult to guarantee. Statistical analysis is another method for evaluating rock drillability, which predicts the drillability of rock by establishing a mathematical model that incorporates logging data, drilling parameters, and other factors that influence drillability (Andrews, et al.; Tang, et al.). However, a significant amount of rock core data is required to support the model and ensure accurate predictions.
2
Marrone, Teresa, and Pierpaolo Testa. "Brand algorithms and social engagement in digital era." In 13th International Conference on Applied Human Factors and Ergonomics (AHFE 2022). AHFE International, 2022. http://dx.doi.org/10.54941/ahfe1002562.
Full textAbstract:
The world we live in today is pervaded by digital, the net is increasingly present and mixes the dimensions of the physical and the virtual, changing the way we understand, decide and evaluate things and also the way we do business. Artificial intelligence (AI) and related technologies are transforming the way we think and do marketing and the way companies relate to consumers and society.Internet has assumed a key role in nurturing innovation within business ecosystems. AI, big data and Internet of things (IoT) are key drivers of the current revolution in the way of communicating and relating among both individuals and products. This change is mainly due to the impact of algorithms’ mediations on the creation of value and customer engagement.Recent years, growing attention has been devoted to consumer brand engagement through emerging technological platforms (e.g., social media/artificial intelligence-based). However, despite important knowledge advancement, much remains unknown regarding the effect of Consumers’ Technology-Facilitated Brand Engagement (CTFBE) on individuals’ wellbeing, thus determining an important research gap (Hollebeek and Belk, 2021). CTFBE comprises a vital social facet. Hollebeek and Belk (2021) define CTFBE as a consumer’s bloodedly volitional resource investment in technology-mediated brand interactions (Kumar et al., 2019; Hollebeek et al, 2020). Online behavioral customer engagement occurs because of the rise of the new media and the advancement of technology, which have changed the way customers connect and interact with firms (Jahn and Kunz, 2012). One of the most active channels for such an aim are social media (Gummerus et al, 2012) where customers share their own experiences, information, review brands and manifest enthusiasm, delight, or disgust about a brand with others (Hollebeek and Chen, 2014).Digital transformation has totally transformed the value creation process (Reinartz et al., 2019) revolutionizing the way of doing business using the large mass of available data and information, through sophisticated service platforms that increase both effectiveness and efficiency in the value creation processes. AI has been a key component of digital transformation, substantially affecting consumer decision-making (Duan et al., 2021).AI, big data and the IoT are supporting and / or automating many decision-making processes: product, price, channel, supply chain, communication, etc. The customer experience is also redesigned starting from new value creation objectives and can become a stimulus for the creation of new business models. This, in turn, can provide a customized experience that is highly valued by consumers (Lemon and Verhoef, 2016). While new technologies have brought more ways for customers to interact with brands and companies, digital technologies have similarly enabled the automation of company’s interactions with customers (Kunz et al., 2017).According to Kumar et al (2010), AI represents the enabling technology for the transformation of marketing theory and practices: the enormous availability of data, the explosion of the possibilities to reach and interact on the markets and an increased speed of transactions. AI-enabled digital platform helps organizations to attract their customers (Bag et al, 2021; Chawla and Goyal, 2021).An increasing number of marketing decisions already use artificial intelligence in some way, and with the rise of big data is becoming easier to incorporate AI into business practices. Marketers may develop a more effective and personalized communication approach (Mogaji et al., 2020). For this reason, today AI is adopted in all activities where classification, forecasts and clustering are useful or necessary to solve problems and support decisions (management of anomalies in processes, logistics and optimization planning, customer service and customization).In the contemporary world the ubiquity of digital has made fluid the distinctions between channels and has integrated two dimensions of reality (physical and virtual one in phygital), the management of complex processes has become agile and adaptive, the advantages of integration and dynamic use of resources condition the operation of entire businesses. Well, what influence all this changes, new technologies and brand algorithms will have on social engagement?Prior studies on artificial intelligence in service and marketing research have not addressed customer engagement (Kaartemo & Helkkula, 2018). Perhaps, even Kaartemo & Helkkula (2018) specifically called for more research to answer the question: “How can we improve customer engagement through AI?”The article proposal is theoretical/conceptual in nature and starts from an updated review of academic literature on the aforementioned topics, mainly within marketing and business management disciplines, to achieve an interpretative attempt of Brand algorithm and social engagement (role) in digital era. References on request.
3
Sandunil, K., Z. Bennour, H. Ben Mahmud, and A. Giwelli. "Effects of Tuning Hyperparameters in Random Forest Regression on Reservoir's Porosity Prediction. Case Study: Volve Oil Field, North Sea." In 57th U.S. Rock Mechanics/Geomechanics Symposium. ARMA, 2023. http://dx.doi.org/10.56952/arma-2023-0660.
Full textAbstract:
ABSTRACT Ensemble learning is a recent development in machine learning. Random forest regression (RFR) is one such widely utilized ensemble learning algorithm. However, the current literature lacks studies that primarily focus on the effects of hyperparameter tuning in RFR when predicting reservoir properties of hydrocarbon reservoirs. Thus, in this study we investigated the effects of three commonly used hyperparameters; namely, n_estimators, max_features and min_samples_leaf to predict porosity of Volve oil field in North Sea. Four parameters; depth, gamma ray logs, neutron porosity logs and resistivity logs were used as inputs, while calculated porosity was used as target outputs to develop the RFR models. The RFR models were developed through: (i) tuning each hyperparameter individually, (ii) tuning hyperparameters by coupling them into three groups and, (iii) tuning all three hyperparameters at once. Results showed that the highest performing model had an R2 value of 0.8517 with n_estimators of 100, max_features of 0.5 and min_samples_leaf of 1. Furthermore, it was observed that tuning max_features had a higher impact on improving the performance of the RFR model when predicting porosity of Volve oil field in North Sea. INTRODUCTION World is gradually moving towards an era of artificial intelligence (AI) where every major sector possibly be supplemented by man-like machines. Machine learning (ML) is a branch of AI which has the ability to predict or forecast outputs, decrease computational time and extract features from complex and high-dimensional data sets (Zhan and Kitchin, 2021). These properties of ML are favourable in industries where big data is utilized in several stages of the production line. Oil and gas industry is a perfect match for ML since it uses huge number of data to analyse and interpret both in upstream and downstream sectors. There are several ML algorithms which have been utilized in the literature for reservoir characterization including prediction of petrophysical properties. Artificial neural network (ANN) is one such ML algorithm. ANN's originated from mimicking biological neural systems (Wang et al., 2019). They are widely used in predicting petrophysical properties of hydrocarbon reservoirs (Al Khalifah et al., 2020, Urang et al., 2020, Okon et al., 2021). Supportive vector machine is another commonly used ML algorithm in the literature to estimate reservoir properties (Jamalian et al., 2018, Zhong & Carr, 2019, Wu et al., 2020). Deep learning is an ML algorithm which can be defined as an advanced version of ANN's. This algorithm is also being used in reservoir characterization (Chen et al., 2020, Zhang et al., 2021, Arigbe et al., 2019). 2019).
4
V. B. Jeronymo, Pedro, and Carlos D. Maciel. "Fast Markov Blanket Discovery Without Causal Sufficiency." In Congresso Brasileiro de Automática - 2020. sbabra, 2020. http://dx.doi.org/10.48011/asba.v2i1.1663.
Full textAbstract:
Faster feature selection algorithms become a necessity as Big Data dictates the zeitgeist. An important class of feature selectors are Markov Blanket (MB) learning algorithms. They are Causal Discovery algorithms that learn the local causal structure of a target variable. A common assumption in their theoretical basis, yet often violated in practice, is causal sufficiency: the requirement that all common causes of the measured variables in the dataset are also in the dataset. Recently, Yu et al. (2018) proposed the M3B algorithm, the first to directly learn the MB without demanding causal sufficiency. The main drawback of M3B is that it is time inefficient, being intractable for high-dimensional inputs. In this paper, we derive the Fast Markov Blanket Discovery Algorithm (FMMB). Empirical results that compare FMMB to M3B on the structural learning task show that FMMB outperforms M3B in terms of time efficiency while preserving structural accuracy. Five real-world datasets where used to contrast both algorithms as feature selectors. Applying NB and SVM classifiers, FMMB achieved a competitive outcome. This method mitigates the curse of dimensionality and inspires the development of local-toglobal algorithms.
5
Du, Yuxuan, Tongliang Liu, Yinan Li, Runyao Duan, and Dacheng Tao. "Quantum Divide-and-Conquer Anchoring for Separable Non-negative Matrix Factorization." In Twenty-Seventh International Joint Conference on Artificial Intelligence {IJCAI-18}. California: International Joint Conferences on Artificial Intelligence Organization, 2018. http://dx.doi.org/10.24963/ijcai.2018/289.
Full textAbstract:
It is NP-complete to find non-negative factors W and H with fixed rank r from a non-negative matrix X by minimizing ||X-WH^Τ ||^2. Although the separability assumption (all data points are in the conical hull of the extreme rows) enables polynomial-time algorithms, the computational cost is not affordable for big data. This paper investigates how the power of quantum computation can be capitalized to solve the non-negative matrix factorization with the separability assumption (SNMF) by devising a quantum algorithm based on the divide-and-conquer anchoring (DCA) scheme [Zhou et al., 2013]. The design of quantum DCA (QDCA) is challenging. In the divide step, the random projections in DCA is completed by a quantum algorithm for linear operations, which achieves the exponential speedup. We then devise a heuristic post-selection procedure which extracts the information of anchors stored in the quantum states efficiently. Under a plausible assumption, QDCA performs efficiently, achieves the quantum speedup, and is beneficial for high dimensional problems.
6
Venkatesan, Sibi, James K. Miller, Jeff Schneider, and Artur Dubrawski. "Scaling Active Search using Linear Similarity Functions." In Twenty-Sixth International Joint Conference on Artificial Intelligence. California: International Joint Conferences on Artificial Intelligence Organization, 2017. http://dx.doi.org/10.24963/ijcai.2017/401.
Full textAbstract:
Active Search has become an increasingly useful tool in information retrieval problems where the goal is to discover as many target elements as possible using only limited label queries. With the advent of big data, there is a growing emphasis on the scalability of such techniques to handle very large and very complex datasets. In this paper, we consider the problem of Active Search where we are given a similarity function between data points. We look at an algorithm introduced by Wang et al. [Wang et al., 2013] known as Active Search on Graphs and propose crucial modifications which allow it to scale significantly. Their approach selects points by minimizing an energy function over the graph induced by the similarity function on the data. Our modifications require the similarity function to be a dot-product between feature vectors of data points, equivalent to having a linear kernel for the adjacency matrix. With this, we are able to scale tremendously: for n data points, the original algorithm runs in O(n^2) time per iteration while ours runs in only O(nr + r^2) given r-dimensional features. We also describe a simple alternate approach using a weighted-neighbor predictor which also scales well. In our experiments, we show that our method is competitive with existing semi-supervised approaches. We also briefly discuss conditions under which our algorithm performs well.
7
Krasniuk, Svitlana, and Svitlana Goncharenko. "BIG DATA IN PHILOLOGY." In DÉBATS SCIENTIFIQUES ET ORIENTATIONS PROSPECTIVES DU DÉVELOPPEMENT SCIENTIFIQUE. European Scientific Platform, 2024. http://dx.doi.org/10.36074/logos-20.09.2024.031.
Full text
8
Gjertsen, Ole, Ryan Mushinski, Preston Wolfram, Jeffrey Leisey, Mani Bandi, Roberta Santana, Gregory Andreasen, Paul Pastusek, and Dustin Daechsel. "IADC Dull Code Upgrade: Photometric Classification and Quantification of the New Dull Codes." In SPE/IADC International Drilling Conference and Exhibition. SPE, 2023. http://dx.doi.org/10.2118/212533-ms.
Full textAbstract:
Abstract In the world of drilling, the drill bit dull condition contains our best forensic evidence of the drilling assembly's interaction with the formation. Dull grading forensics is the first place to look to identify drilling dysfunction yet commonly overlooked or misunderstood by operators. The drill bit dull condition can be leveraged to learn about the formation, drilling dynamics and drilling practices (Watson et. al. 2022). The IADC bit dull grading classification system received its most recent revision in 1992 and currently consists of an average inner and outer dull grade severity, rated from 0 – 8 with a major and other dull characteristic along with a reason pulled. These grades can be used to make critical operational and bit design decisions to overcome drilling challenges thereby improving performance and allowing drilling teams to drill consistently further and faster. The oil and gas industry is becoming more reliant on digitally enabled applications to improve performance through big data, machine learning and automation, but at the time of this paper, the critical IADC dull grading system has remained the same. It is still a crude and subjective characterization of the complex drill bit dull condition. A key challenge with the current classification system and industry standard grading technique is that it is highly dependent on the person grading the bit. Personal subjectivity and lack of training can result in key forensic evidence being overlooked that otherwise could have aided in understanding the root cause of drilling dysfunction. A cross disciplinary committee of subject matter experts (SME's) from operators, drill bit providers, cutter manufacturers, and digital solution providers have convened to define and introduce a new standard dull grading system as replacement for the current outdated IADC dull grading. The new dull grading system will allow for an objective cutter-by-cutter dull grading to be stored with relevant drilling data with reduced subjectivity and enhanced accuracy. With recent advancements in mobile phone hardware and applications, a solution was developed that delivers high quality, cutter-by-cutter dull grading automatically and connecting with drilling meta data from a drilling records database containing over 1.8 million well records with over 5 million bottom-hole assembly (BHA) runs. It leverages videos with machine learning combined with an algorithm to deliver cutter specific, major dull characteristics of a scanned bit. This high quality photographic digital dull information is incorporated into workflows allowing for rapid improvement in cutting structure and cutter development lifecycle timelines leading to rapid improvements in drilling performance for operators.
9
Jin, W., T. Atkinson, G. Neupane, T. McLing, C. Doughty, N. Spycher, P. Dobson, and R. Smith. "Influence of Mechanical Deformation and Mineral Dissolution/precipitation on Reservoir Thermal Energy Storage." In 56th U.S. Rock Mechanics/Geomechanics Symposium. ARMA, 2022. http://dx.doi.org/10.56952/arma-2022-2068.
Full textAbstract:
ABSTRACT: Reservoir thermal energy storage (RTES) is a promising technology to balance the mismatch between energy supply and demand. In particular, high temperature (HT) RTES can stabilize the grid with increasing penetration of renewable energy generation. This paper presents the investigation of the mechanical deformation and chemical reaction influences on the performance of HT-ATES for the Lower Tuscaloosa site. Thermo-hydraulic (TH), thermo-hydro-mechanical (THM), and thermo-hydro-chemical (THC) coupled simulations were performed with different operational modes and injection rates for a fixed five-spot well configuration and a seasonal cycle. The results show that (1) geomechanical-induced porosity change is mainly contributed by effective stress change, and the porosity change is distributed through the whole system; (2) geochemistry-induced porosity change is located near the hot well, and its change is one order of magnitude higher than the geomechanical effect; (3) both the operation mode and the injection rate have a huge influence on the RTES performance and lower injection rate with push-pull operation mode has the best performance with recovery factor around 70% for this RTES system. These results shed light on the deployment of HT-RTES in the US and around the world. 1 INTRODUCTION The concept of reservoir thermal energy storage (RTES), also known as geological thermal energy storage (GeoTES) or aquifer thermal energy storage (ATES), to mitigate the mismatch between energy supply and demand has been applied around the world since the 1960s with mixed success. Given its nearly unlimited storage capacity and easy accessibility, RTES has the potential to become an indispensable component to achieve the goal of carbon-neutral energy. Most successful deployments of RTES are operated at low temperatures (LT) (< 25°C), mainly to heat buildings by storing excess thermal energy during the low-use periods (summer) and recovering it during peak energy demand periods (winter). As reviewed by Fleuchaus et al. (2018), there are currently more than 2800 RTES systems worldwide, and 99% are LT-RTES. However, only high-temperature (HT) RTES has the capacity to serve as an earth battery for stabilizing the grid as indicated in McLing et al. (2019). The research and development of HT-RTES have mainly focused on site suitability studies and performance optimization by only considering fluid flow and heat transfer. For example, Schout et al. (2014) extended the widely adopted Rayleigh number - recovery factor relationship for identifying site suitability of LT-RTES systems (Gutierrez-Neri et al., 2011) to HT-RTES systems. Sheldon et al. (2021) further improved the Rayleigh number relationship to consider daily cycles for HT-RTES systems. In addition to recovery factor, the performance metrics of HT-RTES include storage capacity, operational duration, etc. Jin et al. (2021, 2022) performed stochastic thermo-hydraulic simulations and used a machine learning algorithm to directly correlate formation parameters and operational conditions with multiple HT-RTES performance metrics using the simulated big data. All these investigations can facilitate the deployment of HT-RTES. However, geomechanical response and geochemical reactions involved during the operation of a HT-RTES system can potentially induce risks as identified by Fleuchaus et al. (2020), and their effects on HT-RTES performance have not been systematically reported.
10
Aliguer, I., I. Oliver, C. de Santos, F. Vara, and J. Gomez. "GEMINI, a Novel Software System to Improve the Penetration Rate of a Tunnel Boring Machine." In 58th U.S. Rock Mechanics/Geomechanics Symposium. ARMA, 2024. http://dx.doi.org/10.56952/arma-2024-1007.
Full textAbstract:
ABSTRACT: This technical paper presents GEMINI, a software system designed to optimize the performance of tunnel boring machines (TBMs) using machine learning-based techniques. The system key features include an advanced architecture for querying and processing data from the TBM database, big data analysis and visualization in a web browser, a ground-machine interaction model to predict the TBM advance rate and operation anomalies detection via machine learning algorithms to enhance the efficiency and reliability of tunnel excavation operations. In collaboration with ACCIONA, the system has been validated in the Mularroya (Spain) hydraulic tunnel (small diameter) in fractured rock mass with faulted and sheared zones, and implemneted in a larger diameter tunnel in Sao Paulo (Brazil) in weathered granite materials. In both cases, the ground-machine excavation model was trained with excavation data from each tunnel and the model was deployed to provide real-time predictions of the penetration rate during the TBM operation. 1. INTRODUCTION Generally., in tunnelling operations regardless the excavation method, penetration rate and advance rate are the most relevant indicators to deliver the project on time and within budget. In this context, an appropriate methodology to predict their values during operation is a challenge that ACCIONA as a contractor is willing to tackle for mechanized excavations with TBMs in collaboration with SAALG Geomechanics. Geotechnical back-analysis to characterize ground behaviour has received attention from both the academia and the industry since the 80s (Gioda and Maier, 1980) within the context of forensic geotechnics and the Observational Method. It consists in comparing a set of ground measurements that capture the actual ground response with a conceptual model that is able to predict such response by means of some ground-related parameters. The parameters that best represent the ground behaviour are those that minimize the difference between the measurements and model results, expressed as an objective function. SAALG Geomechanics develops and commercializes DAARWIN (Acosta et al. 2023), a cloud-based web application that implements the general workflow of geotechnical back-analysis at the same pace as construction progresses, also referred as real-time back-analysis – RTBA). Its inputs are: i) Finite Elements (FE) as a predictive model of ground behaviour, ii) data from sensors installed within the construction site and uses genetic algorithms as the optimization technique to minimize the objective function (de Santos, 2015).
Reports on the topic "Big Data et algorithmes"
1
Greenberg, Jane, Samantha Grabus, Florence Hudson, Tim Kraska, Samuel Madden, René Bastón, and Katie Naum. The Northeast Big Data Innovation Hub: "Enabling Seamless Data Sharing in Industry and Academia" Workshop Report. Drexel University, March 2017. http://dx.doi.org/10.17918/d8159v.
Full textAbstract:
Increasingly, both industry and academia, in fields ranging from biology and social sciences to computing and engineering, are driven by data (Provost & Fawcett, 2013; Wixom, et al, 2014); and both commercial success and academic impact are dependent on having access to data. Many organizations collecting data lack the expertise required to process it (Hazen, et al, 2014), and, thus, pursue data sharing with researchers who can extract more value from data they own. For example, a biosciences company may benefit from a specific analysis technique a researcher has developed. At the same time, researchers are always on the search for real-world data sets to demonstrate the effectiveness of their methods. Unfortunately, many data sharing attempts fail, for reasons ranging from legal restrictions on how data can be used—to privacy policies, different cultural norms, and technological barriers. In fact, many data sharing partnerships that are vital to addressing pressing societal challenges in cities, health, energy, and the environment are not being pursued due to such obstacles. Addressing these data sharing challenges requires open, supportive dialogue across many sectors, including technology, policy, industry, and academia. Further, there is a crucial need for well-defined agreements that can be shared among key stakeholders, including researchers, technologists, legal representatives, and technology transfer officers. The Northeast Big Data Innovation Hub (NEBDIH) took an important step in this area with the recent "Enabling Seamless Data Sharing in Industry and Academia" workshop, held at Drexel University September 29-30, 2016. The workshop brought together representatives from these critical stakeholder communities to launch a national dialogue on challenges and opportunities in this complex space.
2
Heurich, Manuel, and Anne Demond. Whitepaper: Real World Evidence. Medizinisch Wissenschaftliche Verlagsgesellschaft mbH & Co. KG, 2024. http://dx.doi.org/10.32745/wp-1.
Full textAbstract:
Als Real World Evidence (RWE) bezeichnet man die gewonnenen medizinischen Erkenntnisse, die aus Beobachtungsstudien und der Darstellung der Versorgungsrealität mit Massendatenquellen (Big Data) gewonnen werden (Behrendt 2019). Sie erlauben im Gegensatz zu randomisierten klinischen Studien häufig eine höhere Generalisierbarkeit (externe Validität) (Ziemssen et al. 2017). Die interne Validität hingegen ist bei der Real World Evidence im Vergleich zu klassischen randomisierten klinischen Studien (interventionelle Studien) als geringer anzusehen, da im Vorfeld der Studien bzw. Beobachtungen keine geplanten Eingriffe (Interventionen) erfolgen, um mögliche Verzerrungen auszuschließen (Vogelmann 2019).
3
Gruson-Daniel, Célya, and Maya Anderson-González. Étude exploratoire sur la « recherche sur la recherche » : acteurs et approches. Ministère de l'enseignement supérieur et de la recherche, November 2021. http://dx.doi.org/10.52949/24.
Full textAbstract:
• Introduction Dans le cadre du deuxième Plan National de la Science Ouverte, le Comité pour la science ouverte a souhaité mener une étude pour la préfiguration d’un Lab de la science ouverte (LabSO) afin de mieux comprendre le périmètre de la recherche sur la recherche (research on research) dans un contexte international. • Objectifs de l’étude : L’étude répond ainsi à trois objectifs : 1. repérer des grands courants de recherche sur la recherche (RoR) dans le paysage plus général de la recherche universitaire en Europe et outre-atlantique, en reconnaître les principaux acteurs institutionnels et différencier les approches mobilisées par les uns et les autres ; 2. proposer une méthodologie d’analyse dans une démarche de science ouverte (méthodes mixtes et cartographie numérique) pour faciliter l’appropriation de son contenu ; 3. émettre des recommandations pour faciliter le positionnement du LabSO et s’inspirer d’initiatives existantes. • Méthodologie Une série de treize entretiens et une collecte de données d’une sélection thématique de sites web ont permis de dresser un paysage d’acteurs et d’approches impliqués dans des recherches sur la recherche. Ce dernier s’est nourri d’une démarche de cartographie numérique pour repérer et visualiser les liens existants entre différentes communautés, mouvements, réseaux et initiatives (financeurs, projets, centres de recherche, fournisseurs de données, éditeurs, etc.). • Résultats Le rapport présente différents courants de « recherche sur la recherche » issus des traditions théoriques et méthodologiques de la sociologie, de l’économie, des sciences politiques, de la philosophie, des sciences de l’information et des mesures (biblio/scientométrie). Des courants plus récents sont aussi décrits. Ils s’inscrivent dans un contexte de politiques publiques favorables à la science ouverte et ont émergé dans le champ des sciences sociales computationnelles, des Big Data ou encore des domaines biomédicaux. Si certaines de ces approches s’appuient sur des courants académiques (STS, sciences des mesures) établis depuis de nombreuses décennies, d’autres comme ceux de la « métascience » ou de la « science de la science », se sont structurées plus récemment avec une visée prescriptive et de changement fondé sur des preuves (evidence-based) se basant sur un engagement normatif pour une science plus ouverte, inclusive et diverse. Bien loin d’un paysage statique, l’étude fait ressortir des recherches en mouvement, des débats tout autant que des mises en garde afin que certains courants « ne réinventent pas la roue » en faisant fit d’une longue tradition académique de l’étude des sciences et de la production scientifiques. De nouvelles alliances entre centres de recherche et laboratoires, institutions subventionnaires, décideurs politiques et fournisseurs de données ont été repérées. Elles participent à une dynamique actuelle d’équipement des politiques publiques par des outils d’évaluation et des protocoles de recherche pour guider les actions menées, on parle d’évidence-based policies. Un des exemples les plus récents étant laa seconde feuille de route du RoRI1 poussant notamment à la formation d’un réseau international d’instituts de recherche sur la recherche, fondé sur le partage et la mutualisation de données, de méthodes et d’outils. Outre la présentation de ces différents acteurs et courants, le rapport pointe le rôle joué par les infrastructures et les fournisseurs de données scientifiques (publications, données, métadonnées, citations, etc.) dans la structuration de ce paysage et les équilibres à trouver. • Recommandations 1. Accompagner la construction d’indicateurs et de métriques par le biais d’un regard critique et de discussions collectives pour mesurer leurs impacts sur les comportements des professionnels de la recherche (mésusages, gaming). 2. Porter attention aux étapes de diffusion des résultats scientifiques issus des « recherches sur la recherche » pour les adapter aux différents publics ciblés (chercheurs, responsables des politiques publiques de recherche, journalistes, etc.). 3. Articuler les travaux de « recherche sur la recherche » avec une démarche de science ouverte en questionnant notamment les choix faits concernant les fournisseurs de données, les infrastructures et outils d’évaluation, de découvrabilité et d’analyse de la production scientifique (gouvernance, utilisation des données, etc.). 4. Soutenir les approches thématiques et transversales plutôt que disciplinaire de manière collaborative entre les différents membres du Lab de la science ouverte et aider le dialogue entre les différentes approches et mouvements (STS, research on research, science of science, scientométrie, etc.)
4
Choquette, Gary. PR-000-16209-WEB Data Management Best Practices Learned from CEPM. Chantilly, Virginia: Pipeline Research Council International, Inc. (PRCI), April 2019. http://dx.doi.org/10.55274/r0011568.
Full textAbstract:
DATE: Wednesday, May 1, 2019 TIME: 2:00 - 3:30 p.m. ET PRESENTER: Gary Choquette, PRCI CLICK DOWNLOAD/BUY TO ACCESS THE REGISTRATION LINK FOR THIS WEBINAR Systems that manage large sets of data are becoming more common in the energy transportation industry. Having access to the data offers the opportunity to learn from previous experiences to help efficiently manage the future. But how does one manage to digest copious quantities of data to find nuggets within the ore? This webinar will outline some of the data management best practices learned from the research projects associated with CEPM. - Logging/capturing data tips - Techniques to identify 'bad' data - Methods of mapping equipment and associated regressions - Tips for pre-processing data for regressions - Machine learning tips - Establishing alarm limits - Identifying equipment problems - Multiple case studies Who Should Attend? - Data analysts - Equipment support specialists - Those interested in learning more about 'big data' and 'machine learning' Recommended Pre-reading: - PR-309-11202-R01 Field Demonstration Test of Advanced Engine and Compressor Diagnostics for CORE - PR-312-12210-R01 CEPM Monitoring Plan for 2SLB Reciprocating Engines* - PR-309-13208-R01 Field Demonstration of Integrated System and Expert Level Continuous Performance Monitoring for CORE* - PR-309-14209-R01 Field Demo of Integrated Expert Level Continuous Performance Monitoring - PR-309-15205-R01 Continuous Engine Performance Monitoring Technical Specification - PR-000-15208-R01 Reciprocating Engine Speed Stability as a Measure of Combustion Stability - PR-309-15209-R01 Evaluation of NSCR Specific Models for Use in CEPM - PR-000-16209-R01 Demonstration of Continuous Equipment Performance Monitoring - PR-015-17606-Z02 Elbow Meter Test Results* *Documents available to PRCI member only Attendance will be limited to the first 500 registrants to join the webinar. All remaining registrants will receive a link to view the recording after the webinar. Not able to attend? Register anyway to automatically receive a link to the recording after the webinar to view at your convenience! After registering, you will receive a confirmation email containing information about joining the webinar. Please visit our website for other webinars that may be of interest to you!
5
Diakonova, Marina, Corinna Ghirelli, and Juan Quiñónez. Economic Policy Uncertainty in Central America and the Dominican Republic. Madrid: Banco de España, August 2024. http://dx.doi.org/10.53479/37524.
Full textAbstract:
The advent of Big Data and computational tools has transformed macroeconomic analysis, introducing real-time, high-frequency text-based indicators such as the economic policy uncertainty (EPU) index pioneered by Baker et al. (2016). However, constructing the EPU index for developing economies remains a challenge, mostly due to limited press coverage. Our study focuses on the Central American region, comprising Costa Rica, El Salvador, Guatemala, Honduras, Nicaragua, Panama and the Dominican Republic (CAPADR). We construct country-specific EPU indices using a combination of local and regional sources and validate them using the narrative approach in order to ensure variation accurately reflects relevant economic policy events. We offer further empirical validation by computing impulse response functions for key macroeconomic variables, at both country and representative country level. We show that EPU shocks lead to a decline in economic activity, foreign direct investment (FDI) and tourism levels. Our findings underline the importance of EPU monitoring in Central America and offer a solution through our indices.
6
Shamblin, Robert, Kevin Whelan, Mario Londono, and Judd Patterson. South Florida/Caribbean Network early detection protocol for exotic plants: Corridors of invasiveness. National Park Service, July 2022. http://dx.doi.org/10.36967/nrr-2293364.
Full textAbstract:
Exotic plant populations can be potentially catastrophic to the natural communities of South Florida. Aggressive exotics such as Brazillian Pepper (Schinus terebinthifolius) and Melaleuca (Melaleuca quinquinervia) have displaced native habitats and formed monocultures of exotic stands (Dalrymple et al. 2003). Nearby plant nurseries, especially the ones outside the boundaries of Biscayne National Park (BISC) and Everglades National Park (EVER), are a continuous source of new exotic species that may become established within South Florida’s national parks. Early detection and rapid response to these new species of exotic plants is important to maintaining the integrity of the parks’ natural habitats and is a cost-effective approach to management. The South Florida/Caribbean Network (SFCN) developed the South Florida/Caribbean Network Early Detection Protocol for Exotic Plants to target early detection of these potential invaders. Three national parks of South Florida are monitored for invasive, exotic plants using this protocol: Big Cypress National Preserve (BICY), Biscayne National Park (BISC), and Everglades National Park (EVER). These national parks include some 2,411,000 acres (3,767.2 square miles [mi2]) that encompass a variety of habitat types. To monitor the entire area for new species would not be feasible; therefore the basic approach of this protocol is to scan major “corridors of invasiveness,” e.g., paved and unpaved roads, trails, trail heads, off road vehicle (ORV) trails, boat ramps, canals, and campgrounds, for exotic plant species new to the national parks of South Florida. Sampling is optimized using a two- to three-person crew: a trained botanist, a certified herbicide applicator, and optionally a SFCN (or IPMT [Invasive Plant Management Team]) staff member or park staff to take photographs and help with data collection. If infestations are small, they are treated immediately by the herbicide applicator. If large, they are reported to park staff and the Invasive Plant Management Team. The sampling domain is partitioned into five regions, with one region sampled per year. Regions include the terrestrial habitats of Biscayne National Park, the eastern region of Everglades National Park, the western region of Everglades National Park, the northern region of Big Cypress National Preserve, and the southern region of Big Cypress National Preserve. Monitoring of roads, trails, and canals occurs while traveling into and through the parks (i.e., travel at 2–10 mph) using motorized vehicles, airboats, and/or hiking. Campgrounds, boat launches, trailheads, and similar areas, involve complete searches. When an exotic plant is observed, a GPS location is obtained, and coordinates are taken of the plant. Photographs are not taken for every exotic plant encountered, but photographs will be taken for new and unusual species (for example a coastal exotic found in inland habitats). Information recorded at each location includes the species name, size of infestation, abundance, cover class, any treatment/control action taken, and relevant notes. During the surveys, a GPS “track” is also recorded to document the areas surveyed and a field of view is estimated. Field notes, pictures, and GPS data are compiled, entered, and analyzed in a Microsoft Access database. Resource briefs (and optional data summary reports) and associated shapefiles and data are then produced and sent to contacts within the corresponding national parks.
7
Holland, Darren, and Nazmina Mahmoudzadeh. Foodborne Disease Estimates for the United Kingdom in 2018. Food Standards Agency, January 2020. http://dx.doi.org/10.46756/sci.fsa.squ824.
Full textAbstract:
In February 2020 the FSA published two reports which produced new estimates of foodborne norovirus cases. These were the ‘Norovirus Attribution Study’ (NoVAS study) (O’Brien et al., 2020) and the accompanying internal FSA technical review ‘Technical Report: Review of Quantitative Risk Assessment of foodborne norovirus transmission’ (NoVAS model review), (Food Standards Agency, 2020). The NoVAS study produced a Quantitative Microbiological Risk Assessment model (QMRA) to estimate foodborne norovirus. The NoVAS model review considered the impact of using alternative assumptions and other data sources on these estimates. From these two pieces of work, a revised estimate of foodborne norovirus was produced. The FSA has therefore updated its estimates of annual foodborne disease to include these new results and also to take account of more recent data related to other pathogens. The estimates produced include: •Estimates of GP presentations and hospital admissions for foodbornenorovirus based on the new estimates of cases. The NoVAS study onlyproduced estimates for cases. •Estimates of foodborne cases, GP presentations and hospital admissions for12 other pathogens •Estimates of unattributed cases of foodborne disease •Estimates of total foodborne disease from all pathogens Previous estimates An FSA funded research project ‘The second study of infectious intestinal disease in the community’, published in 2012 and referred to as the IID2 study (Tam et al., 2012), estimated that there were 17 million cases of infectious intestinal disease (IID) in 2009. These include illness caused by all sources, not just food. Of these 17 million cases, around 40% (around 7 million) could be attributed to 13 known pathogens. These pathogens included norovirus. The remaining 60% of cases (equivalent to 10 million cases) were unattributed cases. These are cases where the causal pathogen is unknown. Reasons for this include the causal pathogen was not tested for, the test was not sensitive enough to detect the causal pathogen or the pathogen is unknown to science. A second project ‘Costed extension to the second study of infectious intestinal disease in the community’, published in 2014 and known as IID2 extension (Tam, Larose and O’Brien, 2014), estimated that there were 566,000 cases of foodborne disease per year caused by the same 13 known pathogens. Although a proportion of the unattributed cases would also be due to food, no estimate was provided for this in the IID2 extension. New estimates We estimate that there were 2.4 million cases of foodborne disease in the UK in 2018 (95% credible intervals 1.8 million to 3.1 million), with 222,000 GP presentations (95% Cred. Int. 150,000 to 322,000) and 16,400 hospital admissions (95% Cred. Int. 11,200 to 26,000). Of the estimated 2.4 million cases, 0.9 million (95% Cred. Int. 0.7 million to 1.2 million) were from the 13 known pathogens included in the IID2 extension and 1.4 million1 (95% Cred. Int. 1.0 million to 2.0 million) for unattributed cases. Norovirus was the pathogen with the largest estimate with 383,000 cases a year. However, this estimate is within the 95% credible interval for Campylobacter of 127,000 to 571,000. The pathogen with the next highest number of cases was Clostridium perfringens with 85,000 (95% Cred. Int. 32,000 to 225,000). While the methodology used in the NoVAS study does not lend itself to producing credible intervals for cases of norovirus, this does not mean that there is no uncertainty in these estimates. There were a number of parameters used in the NoVAS study which, while based on the best science currently available, were acknowledged to have uncertain values. Sensitivity analysis undertaken as part of the study showed that changes to the values of these parameters could make big differences to the overall estimates. Campylobacter was estimated to have the most GP presentations with 43,000 (95% Cred. Int. 19,000 to 76,000) followed by norovirus with 17,000 (95% Cred. Int. 11,000 to 26,000) and Clostridium perfringens with 13,000 (95% Cred. Int. 6,000 to 29,000). For hospital admissions Campylobacter was estimated to have 3,500 (95% Cred. Int. 1,400 to 7,600), followed by norovirus 2,200 (95% Cred. Int. 1,500 to 3,100) and Salmonella with 2,100 admissions (95% Cred. Int. 400 to 9,900). As many of these credible intervals overlap, any ranking needs to be undertaken with caution. While the estimates provided in this report are for 2018 the methodology described can be applied to future years.
8
Big Data: Applications, technologies et reflexions sociétales. Résumé du Programme national de recherche ≪Big Data≫ (PNR 75). Fonds national suisse, Berne, March 2023. http://dx.doi.org/10.46446/publication_pnr75.2023.fr.
Full text