Thematische Bibliographien / Distributed optimization and learning

Inhaltsverzeichnis

Zeitschriftenartikel
Dissertationen
Bücher
Buchteile
Konferenzberichte
Berichte der Organisationen

Auswahl der wissenschaftlichen Literatur zum Thema „Distributed optimization and learning“

Autor: Grafiati

Veröffentlicht am 15. Februar 2025

Geben Sie eine Quelle nach APA, MLA, Chicago, Harvard und anderen Zitierweisen an

Wählen Sie eine Art der Quelle aus:

Machen Sie sich mit den Listen der aktuellen Artikel, Bücher, Dissertationen, Berichten und anderer wissenschaftlichen Quellen zum Thema "Distributed optimization and learning" bekannt.

Neben jedem Werk im Literaturverzeichnis ist die Option "Zur Bibliographie hinzufügen" verfügbar. Nutzen Sie sie, wird Ihre bibliographische Angabe des gewählten Werkes nach der nötigen Zitierweise (APA, MLA, Harvard, Chicago, Vancouver usw.) automatisch gestaltet.

Sie können auch den vollen Text der wissenschaftlichen Publikation im PDF-Format herunterladen und eine Online-Annotation der Arbeit lesen, wenn die relevanten Parameter in den Metadaten verfügbar sind.

Zeitschriftenartikel zum Thema "Distributed optimization and learning"

Kamalesh, Kamalesh, und Dr Gobi Natesan. „Machine Learning-Driven Analysis of Distributed Computing Systems: Exploring Optimization and Efficiency“. International Journal of Research Publication and Reviews 5, Nr. 3 (09.03.2024): 3979–83. http://dx.doi.org/10.55248/gengpi.5.0324.0786.

Der volle Inhalt der Quelle

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Mertikopoulos, Panayotis, E. Veronica Belmega, Romain Negrel und Luca Sanguinetti. „Distributed Stochastic Optimization via Matrix Exponential Learning“. IEEE Transactions on Signal Processing 65, Nr. 9 (01.05.2017): 2277–90. http://dx.doi.org/10.1109/tsp.2017.2656847.

Der volle Inhalt der Quelle

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Gratton, Cristiano, Naveen K. D. Venkategowda, Reza Arablouei und Stefan Werner. „Privacy-Preserved Distributed Learning With Zeroth-Order Optimization“. IEEE Transactions on Information Forensics and Security 17 (2022): 265–79. http://dx.doi.org/10.1109/tifs.2021.3139267.

Der volle Inhalt der Quelle

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Blot, Michael, David Picard, Nicolas Thome und Matthieu Cord. „Distributed optimization for deep learning with gossip exchange“. Neurocomputing 330 (Februar 2019): 287–96. http://dx.doi.org/10.1016/j.neucom.2018.11.002.

Der volle Inhalt der Quelle

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Young, M. Todd, Jacob D. Hinkle, Ramakrishnan Kannan und Arvind Ramanathan. „Distributed Bayesian optimization of deep reinforcement learning algorithms“. Journal of Parallel and Distributed Computing 139 (Mai 2020): 43–52. http://dx.doi.org/10.1016/j.jpdc.2019.07.008.

Der volle Inhalt der Quelle

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Nedic, Angelia. „Distributed Gradient Methods for Convex Machine Learning Problems in Networks: Distributed Optimization“. IEEE Signal Processing Magazine 37, Nr. 3 (Mai 2020): 92–101. http://dx.doi.org/10.1109/msp.2020.2975210.

Der volle Inhalt der Quelle

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Lin, I.-Cheng. „Learning and Optimization over Robust Networked Systems“. ACM SIGMETRICS Performance Evaluation Review 52, Nr. 3 (09.01.2025): 23–26. https://doi.org/10.1145/3712170.3712179.

Der volle Inhalt der Quelle

Annotation:

Research Summary: 1 Introduction Networked systems are ubiquitous in our daily lives, playing a critical role across a wide range of scientific fields, including communication, machine learning, optimization, control, biology, economics, and social sciences. In machine learning and optimization, distributed learning and distributed optimization exploit the structure of such networked systems, which enables training models across multiple users [1] and solving complex problems by partitioning tasks among interconnected agents [2]. This significantly enhances both scalability and efficiency of the learning/optimization task. In control systems, for example, networked architectures synchronize the operations of different parts of the system like sensors, controllers etc. across distributed environments, ensuring the overall system's stability and performance [3]. In economics, networked systems were utilized to capture the interdependencies between different part of the economic system, allowing further insights of market dynamics [4]. In the social sciences, specifically social networks, study the propagation of idea, behaviors and opinions, offering a deeper understanding of societal structures and its dynamics [5].

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Gao, Hongchang. „Distributed Stochastic Nested Optimization for Emerging Machine Learning Models: Algorithm and Theory“. Proceedings of the AAAI Conference on Artificial Intelligence 37, Nr. 13 (26.06.2023): 15437. http://dx.doi.org/10.1609/aaai.v37i13.26804.

Der volle Inhalt der Quelle

Annotation:

Traditional machine learning models can be formulated as the expected risk minimization (ERM) problem: minw∈Rd Eξ [l(w; ξ)], where w ∈ Rd denotes the model parameter, ξ represents training samples, l(·) is the loss function. Numerous optimization algorithms, such as stochastic gradient descent (SGD), have been developed to solve the ERM problem. However, a wide range of emerging machine learning models are beyond this class of optimization problems, such as model-agnostic meta-learning (Finn, Abbeel, and Levine 2017). Of particular interest of my research is the stochastic nested optimization (SNO) problem, whose objective function has a nested structure. Specifically, I have been focusing on two instances of this kind of problem: stochastic compositional optimization (SCO) problems, which cover meta-learning, area-under-the-precision recall-curve optimization, contrastive self-supervised learning, etc., and stochastic bilevel optimization (SBO) problems, which can be applied to meta-learning, hyperparameter optimization, neural network architecture search, etc. With the emergence of large-scale distributed data, such as the user data generated on mobile devices or intelligent hardware, it is imperative to develop distributed optimization algorithms for SNO (Distributed SNO). A significant challenge for optimizing distributed SNO problems lies in that the stochastic (hyper-)gradient is a biased estimation of the full gradient. Thus, existing distributed optimization algorithms when applied to them suffer from slow convergence rates. In this talk, I will discuss my recent works about distributed SCO (Gao and Huang 2021; Gao, Li, and Huang 2022) and distributed SBO (Gao, Gu, and Thai 2022; Gao 2022) under both centralized and decentralized settings, including algorithmic details about reducing the bias of stochastic gradient, theoretical convergence rate, and practical machine learning applications, and then highlight challenges for future research.

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Choi, Dojin, Jiwon Wee, Sangho Song, Hyeonbyeong Lee, Jongtae Lim, Kyoungsoo Bok und Jaesoo Yoo. „k-NN Query Optimization for High-Dimensional Index Using Machine Learning“. Electronics 12, Nr. 11 (24.05.2023): 2375. http://dx.doi.org/10.3390/electronics12112375.

Der volle Inhalt der Quelle

Annotation:

In this study, we propose three k-nearest neighbor (k-NN) optimization techniques for a distributed, in-memory-based, high-dimensional indexing method to speed up content-based image retrieval. The proposed techniques perform distributed, in-memory, high-dimensional indexing-based k-NN query optimization: a density-based optimization technique that performs k-NN optimization using data distribution; a cost-based optimization technique using query processing cost statistics; and a learning-based optimization technique using a deep learning model, based on query logs. The proposed techniques were implemented on Spark, which supports a master/slave model for large-scale distributed processing. We showed the superiority and validity of the proposed techniques through various performance evaluations, based on high-dimensional data.

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Yang, Peng, und Ping Li. „Distributed Primal-Dual Optimization for Online Multi-Task Learning“. Proceedings of the AAAI Conference on Artificial Intelligence 34, Nr. 04 (03.04.2020): 6631–38. http://dx.doi.org/10.1609/aaai.v34i04.6139.

Der volle Inhalt der Quelle

Annotation:

Conventional online multi-task learning algorithms suffer from two critical limitations: 1) Heavy communication caused by delivering high velocity of sequential data to a central machine; 2) Expensive runtime complexity for building task relatedness. To address these issues, in this paper we consider a setting where multiple tasks are geographically located in different places, where one task can synchronize data with others to leverage knowledge of related tasks. Specifically, we propose an adaptive primal-dual algorithm, which not only captures task-specific noise in adversarial learning but also carries out a projection-free update with runtime efficiency. Moreover, our model is well-suited to decentralized periodic-connected tasks as it allows the energy-starved or bandwidth-constraint tasks to postpone the update. Theoretical results demonstrate the convergence guarantee of our distributed algorithm with an optimal regret. Empirical results confirm that the proposed model is highly effective on various real-world datasets.

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Mehr Quellen

Dissertationen zum Thema "Distributed optimization and learning"

Funkquist, Mikaela, und Minghua Lu. „Distributed Optimization Through Deep Reinforcement Learning“. Thesis, KTH, Skolan för elektroteknik och datavetenskap (EECS), 2020. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-293878.

Der volle Inhalt der Quelle

Annotation:

Reinforcement learning methods allows self-learningagents to play video- and board games autonomously. Thisproject aims to study the efficiency of the reinforcement learningalgorithms Q-learning and deep Q-learning for dynamical multi-agent problems. The goal is to train robots to optimally navigatethrough a warehouse without colliding.A virtual environment was created, in which the learning algo-rithms were tested by simulating moving agents. The algorithms’efficiency was evaluated by how fast the agents learned to performpredetermined tasks.The results show that Q-learning excels in simple problemswith few agents, quickly solving systems with two active agents.Deep Q-learning proved to be better suited for complex systemscontaining several agents, though cases of sub-optimal movementwere still possible. Both algorithms showed great potential fortheir respective areas however improvements still need to be madefor any real-world use.
Förstärkningsinlärningsmetoder tillåter självlärande enheter att spela video- och brädspel autonomt. Projektet siktar på att studera effektiviteten hos förstärkningsinlärningsmetoderna Q-learning och deep Q-learning i dynamiska problem. Målet är att träna upp robotar så att de kan röra sig genom ett varuhus på bästa sätt utan att kollidera. En virtuell miljö skapades, i vilken algoritmerna testades genom att simulera agenter som rörde sig. Algoritmernas effektivitet utvärderades av hur snabbt agenterna lärde sig att utföra förutbestämda uppgifter. Resultatet visar att Q-learning fungerar bra för enkla problem med få agenter, där system med två aktiva agenter löstes snabbt. Deep Q-learning fungerar bättre för mer komplexa system som innehåller fler agenter, men fall med suboptimala rörelser uppstod. Båda algoritmerna visade god potential inom deras respektive områden, däremot måste förbättringar göras innan de kan användas i verkligheten.
Kandidatexjobb i elektroteknik 2020, KTH, Stockholm

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Konečný, Jakub. „Stochastic, distributed and federated optimization for machine learning“. Thesis, University of Edinburgh, 2017. http://hdl.handle.net/1842/31478.

Der volle Inhalt der Quelle

Annotation:

We study optimization algorithms for the finite sum problems frequently arising in machine learning applications. First, we propose novel variants of stochastic gradient descent with a variance reduction property that enables linear convergence for strongly convex objectives. Second, we study distributed setting, in which the data describing the optimization problem does not fit into a single computing node. In this case, traditional methods are inefficient, as the communication costs inherent in distributed optimization become the bottleneck. We propose a communication-efficient framework which iteratively forms local subproblems that can be solved with arbitrary local optimization algorithms. Finally, we introduce the concept of Federated Optimization/Learning, where we try to solve the machine learning problems without having data stored in any centralized manner. The main motivation comes from industry when handling user-generated data. The current prevalent practice is that companies collect vast amounts of user data and store them in datacenters. An alternative we propose is not to collect the data in first place, and instead occasionally use the computational power of users' devices to solve the very same optimization problems, while alleviating privacy concerns at the same time. In such setting, minimization of communication rounds is the primary goal, and we demonstrate that solving the optimization problems in such circumstances is conceptually tractable.

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Armond, Kenneth C. Jr. „Distributed Support Vector Machine Learning“. ScholarWorks@UNO, 2008. http://scholarworks.uno.edu/td/711.

Der volle Inhalt der Quelle

Annotation:

Support Vector Machines (SVMs) are used for a growing number of applications. A fundamental constraint on SVM learning is the management of the training set. This is because the order of computations goes as the square of the size of the training set. Typically, training sets of 1000 (500 positives and 500 negatives, for example) can be managed on a PC without hard-drive thrashing. Training sets of 10,000 however, simply cannot be managed with PC-based resources. For this reason most SVM implementations must contend with some kind of chunking process to train parts of the data at a time (10 chunks of 1000, for example, to learn the 10,000). Sequential and multi-threaded chunking methods provide a way to run the SVM on large datasets while retaining accuracy. The multi-threaded distributed SVM described in this thesis is implemented using Java RMI, and has been developed to run on a network of multi-core/multi-processor computers.

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Patvarczki, Jozsef. „Layout Optimization for Distributed Relational Databases Using Machine Learning“. Digital WPI, 2012. https://digitalcommons.wpi.edu/etd-dissertations/291.

Der volle Inhalt der Quelle

Annotation:

A common problem when running Web-based applications is how to scale-up the database. The solution to this problem usually involves having a smart Database Administrator determine how to spread the database tables out amongst computers that will work in parallel. Laying out database tables across multiple machines so they can act together as a single efficient database is hard. Automated methods are needed to help eliminate the time required for database administrators to create optimal configurations. There are four operators that we consider that can create a search space of possible database layouts: 1) denormalizing, 2) horizontally partitioning, 3) vertically partitioning, and 4) fully replicating. Textbooks offer general advice that is useful for dealing with extreme cases - for instance you should fully replicate a table if the level of insert to selects is close to zero. But even this seemingly obvious statement is not necessarily one that will lead to a speed up once you take into account that some nodes might be a bottle neck. There can be complex interactions between the 4 different operators which make it even more difficult to predict what the best thing to do is. Instead of using best practices to do database layout, we need a system that collects empirical data on when these 4 different operators are effective. We have implemented a state based search technique to try different operators, and then we used the empirically measured data to see if any speed up occurred. We recognized that the costs of creating the physical database layout are potentially large, but it is necessary since we want to know the "Ground Truth" about what is effective and under what conditions. After creating a dataset where these four different operators have been applied to make different databases, we can employ machine learning to induce rules to help govern the physical design of the database across an arbitrary number of computer nodes. This learning process, in turn, would allow the database placement algorithm to get better over time as it trains over a set of examples. What this algorithm calls for is that it will try to learn 1) "What is a good database layout for a particular application given a query workload?" and 2) "Can this algorithm automatically improve itself in making recommendations by using machine learned rules to try to generalize when it makes sense to apply each of these operators?" There has been considerable research done in parallelizing databases where large amounts of data are shipped from one node to another to answer a single query. Sometimes the costs of shipping the data back and forth might be high, so in this work we assume that it might be more efficient to create a database layout where each query can be answered by a single node. To make this assumption requires that all the incoming query templates are known beforehand. This requirement can easily be satisfied in the case of a Web-based application due to the characteristic that users typically interact with the system through a web interface such as web forms. In this case, unseen queries are not necessarily answerable, without first possibly reconstructing the data on a single machine. Prior knowledge of these exact query templates allows us to select the best possible database table placements across multiple nodes. But in the case of trying to improve the efficiency of a Web-based application, a web site provider might feel that they are willing to suffer the inconvenience of not being able to answer an arbitrary query, if they are in turn provided with a system that runs more efficiently.

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Ouyang, Hua. „Optimal stochastic and distributed algorithms for machine learning“. Diss., Georgia Institute of Technology, 2013. http://hdl.handle.net/1853/49091.

Der volle Inhalt der Quelle

Annotation:

Stochastic and data-distributed optimization algorithms have received lots of attention from the machine learning community due to the tremendous demand from the large-scale learning and the big-data related optimization. A lot of stochastic and deterministic learning algorithms are proposed recently under various application scenarios. Nevertheless, many of these algorithms are based on heuristics and their optimality in terms of the generalization error is not sufficiently justified. In this talk, I will explain the concept of an optimal learning algorithm, and show that given a time budget and proper hypothesis space, only those achieving the lower bounds of the estimation error and the optimization error are optimal. Guided by this concept, we investigated the stochastic minimization of nonsmooth convex loss functions, a central problem in machine learning. We proposed a novel algorithm named Accelerated Nonsmooth Stochastic Gradient Descent, which exploits the structure of common nonsmooth loss functions to achieve optimal convergence rates for a class of problems including SVMs. It is the first stochastic algorithm that can achieve the optimal O(1/t) rate for minimizing nonsmooth loss functions. The fast rates are confirmed by empirical comparisons with state-of-the-art algorithms including the averaged SGD. The Alternating Direction Method of Multipliers (ADMM) is another flexible method to explore function structures. In the second part we proposed stochastic ADMM that can be applied to a general class of convex and nonsmooth functions, beyond the smooth and separable least squares loss used in lasso. We also demonstrate the rates of convergence for our algorithm under various structural assumptions of the stochastic function: O(1/sqrt{t}) for convex functions and O(log t/t) for strongly convex functions. A novel application named Graph-Guided SVM is proposed to demonstrate the usefulness of our algorithm. We also extend the scalability of stochastic algorithms to nonlinear kernel machines, where the problem is formulated as a constrained dual quadratic optimization. The simplex constraint can be handled by the classic Frank-Wolfe method. The proposed stochastic Frank-Wolfe methods achieve comparable or even better accuracies than state-of-the-art batch and online kernel SVM solvers, and are significantly faster. The last part investigates the problem of data-distributed learning. We formulate it as a consensus-constrained optimization problem and solve it with ADMM. It turns out that the underlying communication topology is a key factor in achieving a balance between a fast learning rate and computation resource consumption. We analyze the linear convergence behavior of consensus ADMM so as to characterize the interplay between the communication topology and the penalty parameters used in ADMM. We observe that given optimal parameters, the complete bipartite and the master-slave graphs exhibit the fastest convergence, followed by bi-regular graphs.

APA, Harvard, Vancouver, ISO und andere Zitierweisen

El, Gamal Mostafa. „Distributed Statistical Learning under Communication Constraints“. Digital WPI, 2017. https://digitalcommons.wpi.edu/etd-dissertations/314.

Der volle Inhalt der Quelle

Annotation:

"In this thesis, we study distributed statistical learning, in which multiple terminals, connected by links with limited capacity, cooperate to perform a learning task. As the links connecting the terminals have limited capacity, the messages exchanged between the terminals have to be compressed. The goal of this thesis is to investigate how to compress the data observations at multiple terminals and how to use the compressed data for inference. We first focus on the distributed parameter estimation problem, in which terminals send messages related to their local observations using limited rates to a fusion center that will obtain an estimate of a parameter related to the observations of all terminals. It is well known that if the transmission rates are in the Slepian-Wolf region, the fusion center can fully recover all observations and hence can construct an estimator having the same performance as that of the centralized case. One natural question is whether Slepian-Wolf rates are necessary to achieve the same estimation performance as that of the centralized case. In this thesis, we show that the answer to this question is negative. We then examine the optimality of data dimensionality reduction via sufficient statistics compression in distributed parameter estimation problems. The data dimensionality reduction step is often needed especially if the data has a very high dimension and the communication rate is not as high as the one characterized above. We show that reducing the dimensionality by extracting sufficient statistics of the parameter to be estimated does not degrade the overall estimation performance in the presence of communication constraints. We further analyze the optimal estimation performance in the presence of communication constraints and we verify the derived bound using simulations. Finally, we study distributed optimization problems, for which we examine the randomized distributed coordinate descent algorithm with quantized updates. In the literature, the iteration complexity of the randomized distributed coordinate descent algorithm has been characterized under the assumption that machines can exchange updates with an infinite precision. We consider a practical scenario in which the messages exchange occurs over channels with finite capacity, and hence the updates have to be quantized. We derive sufficient conditions on the quantization error such that the algorithm with quantized update still converge."

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Dai, Wei. „Learning with Staleness“. Research Showcase @ CMU, 2018. http://repository.cmu.edu/dissertations/1209.

Der volle Inhalt der Quelle

Annotation:

A fundamental assumption behind most machine learning (ML) algorithms and analyses is the sequential execution. That is, any update to the ML model can be immediately applied and the new model is always available for the next algorithmic step. This basic assumption, however, can be costly to realize, when the computation is carried out across multiple machines, linked by commodity networks that are usually 104 times slower than the memory speed due to fundamental hardware limitations. As a result, concurrent ML computation in the distributed settings often needs to handle delayed updates and perform learning in the presence of staleness. This thesis characterizes learning with staleness from three directions: (1) We extend the theoretical analyses of a number of classical ML algorithms, including stochastic gradient descent, proximal gradient descent on non-convex problems, and Frank-Wolfe algorithms, to explicitly incorporate staleness into their convergence characterizations. (2)We conduct simulation and large-scale distributed experiments to study the empirical effects of staleness on ML algorithms under indeterministic executions. Our results reveal that staleness is a key parameter governing the convergence speed for all considered ML algorithms, with varied ramifications. (3) We design staleness-minimizing parameter server systems by optimizing synchronization methods to effectively reduce the runtime staleness. The proposed optimization of a bounded consistency model utilizes the additional network bandwidths to communicate updates eagerly, relieving users of the burden to tune the staleness level. By minimizing staleness at the framework level, our system stabilizes diverging optimization paths and substantially accelerates convergence across ML algorithms without any modification to the ML programs.

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Lu, Yumao. „Kernel optimization and distributed learning algorithms for support vector machines“. Diss., Restricted to subscribing institutions, 2005. http://uclibs.org/PID/11984.

Der volle Inhalt der Quelle

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Dinh, The Canh. „Distributed Algorithms for Fast and Personalized Federated Learning“. Thesis, The University of Sydney, 2023. https://hdl.handle.net/2123/30019.

Der volle Inhalt der Quelle

Annotation:

The significant increase in the number of cutting-edge user equipment (UE) results in the phenomenal growth of the data volume generated at the edge. This shift fuels the booming trend of an emerging technique named Federated Learning. In contrast to traditional methods in which data is collected and processed centrally, FL builds a global model from contributions of UE's model without sending private data then effectively ensures data privacy. However, FL faces challenges in non-identically distributed (non-IID) data, communication cost, and convergence rate. Firstly, we propose first-order optimization FL algorithms named FedApprox and FEDL to improve the convergence rate. We propose FedApprox exploiting proximal stochastic variance-reduced gradient methods and extract insights from convergence conditions via the algorithm’s parameter control. We then propose FEDL to handle heterogeneous UE data and characterize the trade-off between local computation and global communication. Experimentally, FedApprox outperforms vanilla FedAvg while FEDL outperforms FedApprox and FedAvg. Secondly, we consider the communication between edges to be more costly than local computational overhead. We propose DONE, a distributed approximate Newton-type algorithm for communication-efficient federated edge learning. DONE approximates Newton direction using classical Richardson iteration on each edge. Experimentally, DONE attains a comparable performance to Newton’s method and outperforms first-order algorithms. Finally, we address the non-IID issue by proposing pFedMe, a personalized FL algorithm using Moreau envelopes. pFedMe achieves quadratic speedup for strongly convex and sublinear speedup of order 2/3 for smooth nonconvex objectives. We then propose FedU, a Federated Multitask Learning algorithm using Laplacian regularization to leverage the relationships among the users' models. Experimentally, pFedMe excels FedAvg and Per-FedAvg while FedU outperforms pFedMe and MOCHA.

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Reddi, Sashank Jakkam. „New Optimization Methods for Modern Machine Learning“. Research Showcase @ CMU, 2017. http://repository.cmu.edu/dissertations/1116.

Der volle Inhalt der Quelle

Annotation:

Modern machine learning systems pose several new statistical, scalability, privacy and ethical challenges. With the advent of massive datasets and increasingly complex tasks, scalability has especially become a critical issue in these systems. In this thesis, we focus on fundamental challenges related to scalability, such as computational and communication efficiency, in modern machine learning applications. The underlying central message of this thesis is that classical statistical thinking leads to highly effective optimization methods for modern big data applications. The first part of the thesis investigates optimization methods for solving large-scale nonconvex Empirical Risk Minimization (ERM) problems. Such problems have surged into prominence, notably through deep learning, and have led to exciting progress. However, our understanding of optimization methods suitable for these problems is still very limited. We develop and analyze a new line of optimization methods for nonconvex ERM problems, based on the principle of variance reduction. We show that our methods exhibit fast convergence to stationary points and improve the state-of-the-art in several nonconvex ERM settings, including nonsmooth and constrained ERM. Using similar principles, we also develop novel optimization methods that provably converge to second-order stationary points. Finally, we show that the key principles behind our methods can be generalized to overcome challenges in other important problems such as Bayesian inference. The second part of the thesis studies two critical aspects of modern distributed machine learning systems — asynchronicity and communication efficiency of optimization methods. We study various asynchronous stochastic algorithms with fast convergence for convex ERM problems and show that these methods achieve near-linear speedups in sparse settings common to machine learning. Another key factor governing the overall performance of a distributed system is its communication efficiency. Traditional optimization algorithms used in machine learning are often ill-suited for distributed environments with high communication cost. To address this issue, we dis- cuss two different paradigms to achieve communication efficiency of algorithms in distributed environments and explore new algorithms with better communication complexity.

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Mehr Quellen

Bücher zum Thema "Distributed optimization and learning"

Jiang, Jiawei, Bin Cui und Ce Zhang. Distributed Machine Learning and Gradient Optimization. Singapore: Springer Singapore, 2022. http://dx.doi.org/10.1007/978-981-16-3420-8.

Der volle Inhalt der Quelle

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Wang, Huiwei, Huaqing Li und Bo Zhou. Distributed Optimization, Game and Learning Algorithms. Singapore: Springer Singapore, 2021. http://dx.doi.org/10.1007/978-981-33-4528-7.

Der volle Inhalt der Quelle

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Joshi, Gauri. Optimization Algorithms for Distributed Machine Learning. Cham: Springer International Publishing, 2023. http://dx.doi.org/10.1007/978-3-031-19067-4.

Der volle Inhalt der Quelle

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Tatarenko, Tatiana. Game-Theoretic Learning and Distributed Optimization in Memoryless Multi-Agent Systems. Cham: Springer International Publishing, 2017. http://dx.doi.org/10.1007/978-3-319-65479-9.

Der volle Inhalt der Quelle

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Oblinger, Diana G. Distributed learning. Boulder, Colo: CAUSE, 1996.

Den vollen Inhalt der Quelle finden

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Majhi, Sudhan, Rocío Pérez de Prado und Chandrappa Dasanapura Nanjundaiah, Hrsg. Distributed Computing and Optimization Techniques. Singapore: Springer Nature Singapore, 2022. http://dx.doi.org/10.1007/978-981-19-2281-7.

Der volle Inhalt der Quelle

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Giselsson, Pontus, und Anders Rantzer, Hrsg. Large-Scale and Distributed Optimization. Cham: Springer International Publishing, 2018. http://dx.doi.org/10.1007/978-3-319-97478-1.

Der volle Inhalt der Quelle

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Lü, Qingguo, Xiaofeng Liao, Huaqing Li, Shaojiang Deng und Shanfu Gao. Distributed Optimization in Networked Systems. Singapore: Springer Nature Singapore, 2023. http://dx.doi.org/10.1007/978-981-19-8559-1.

Der volle Inhalt der Quelle

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Abdulrahman Younis Ali Younis Kalbat. Distributed and Large-Scale Optimization. [New York, N.Y.?]: [publisher not identified], 2016.

Den vollen Inhalt der Quelle finden

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Otto, Daniel, Gianna Scharnberg, Michael Kerres und Olaf Zawacki-Richter, Hrsg. Distributed Learning Ecosystems. Wiesbaden: Springer Fachmedien Wiesbaden, 2023. http://dx.doi.org/10.1007/978-3-658-38703-7.

Der volle Inhalt der Quelle

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Mehr Quellen

Buchteile zum Thema "Distributed optimization and learning"

Joshi, Gauri, und Shiqiang Wang. „Communication-Efficient Distributed Optimization Algorithms“. In Federated Learning, 125–43. Cham: Springer International Publishing, 2022. http://dx.doi.org/10.1007/978-3-030-96896-0_6.

Der volle Inhalt der Quelle

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Jiang, Jiawei, Bin Cui und Ce Zhang. „Distributed Gradient Optimization Algorithms“. In Distributed Machine Learning and Gradient Optimization, 57–114. Singapore: Springer Singapore, 2021. http://dx.doi.org/10.1007/978-981-16-3420-8_3.

Der volle Inhalt der Quelle

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Jiang, Jiawei, Bin Cui und Ce Zhang. „Distributed Machine Learning Systems“. In Distributed Machine Learning and Gradient Optimization, 115–66. Singapore: Springer Singapore, 2021. http://dx.doi.org/10.1007/978-981-16-3420-8_4.

Der volle Inhalt der Quelle

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Joshi, Gauri. „Distributed Optimization in Machine Learning“. In Synthesis Lectures on Learning, Networks, and Algorithms, 1–12. Cham: Springer International Publishing, 2022. http://dx.doi.org/10.1007/978-3-031-19067-4_1.

Der volle Inhalt der Quelle

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Lin, Zhouchen, Huan Li und Cong Fang. „ADMM for Distributed Optimization“. In Alternating Direction Method of Multipliers for Machine Learning, 207–40. Singapore: Springer Nature Singapore, 2022. http://dx.doi.org/10.1007/978-981-16-9840-8_6.

Der volle Inhalt der Quelle

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Jiang, Jiawei, Bin Cui und Ce Zhang. „Basics of Distributed Machine Learning“. In Distributed Machine Learning and Gradient Optimization, 15–55. Singapore: Springer Singapore, 2021. http://dx.doi.org/10.1007/978-981-16-3420-8_2.

Der volle Inhalt der Quelle

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Scheidegger, Carre, Arpit Shah und Dan Simon. „Distributed Learning with Biogeography-Based Optimization“. In Lecture Notes in Computer Science, 203–15. Berlin, Heidelberg: Springer Berlin Heidelberg, 2011. http://dx.doi.org/10.1007/978-3-642-21827-9_21.

Der volle Inhalt der Quelle

APA, Harvard, Vancouver, ISO und andere Zitierweisen

González-Mendoza, Miguel, Neil Hernández-Gress und André Titli. „Quadratic Optimization Fine Tuning for the Learning Phase of SVM“. In Advanced Distributed Systems, 347–57. Berlin, Heidelberg: Springer Berlin Heidelberg, 2005. http://dx.doi.org/10.1007/11533962_31.

Der volle Inhalt der Quelle

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Wang, Huiwei, Huaqing Li und Bo Zhou. „Cooperative Distributed Optimization in Multiagent Networks with Delays“. In Distributed Optimization, Game and Learning Algorithms, 1–17. Singapore: Springer Singapore, 2021. http://dx.doi.org/10.1007/978-981-33-4528-7_1.

Der volle Inhalt der Quelle

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Wang, Huiwei, Huaqing Li und Bo Zhou. „Constrained Consensus of Multi-agent Systems with Time-Varying Topology“. In Distributed Optimization, Game and Learning Algorithms, 19–37. Singapore: Springer Singapore, 2021. http://dx.doi.org/10.1007/978-981-33-4528-7_2.

Der volle Inhalt der Quelle

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Konferenzberichte zum Thema "Distributed optimization and learning"

Patil, Aditya, Sanket Lodha, Sonal Deshmukh, Rupali S. Joshi, Vaishali Patil und Sudhir Chitnis. „Battery Optimization Using Machine Learning“. In 2024 IEEE International Conference on Blockchain and Distributed Systems Security (ICBDS), 1–5. IEEE, 2024. https://doi.org/10.1109/icbds61829.2024.10837428.

Der volle Inhalt der Quelle

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Khan, Malak Abid Ali, Luo Senlin, Hongbin Ma, Abdul Khalique Shaikh, Ahlam Almusharraf und Imran Khan Mirani. „Optimization of LoRa for Distributed Environments Based on Machine Learning“. In 2024 IEEE Asia Pacific Conference on Wireless and Mobile (APWiMob), 137–42. IEEE, 2024. https://doi.org/10.1109/apwimob64015.2024.10792952.

Der volle Inhalt der Quelle

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Chao, Liangchen, Bo Zhang, Hengpeng Guo, Fangheng Ji und Junfeng Li. „UAV Swarm Collaborative Transmission Optimization for Machine Learning Tasks“. In 2024 IEEE 30th International Conference on Parallel and Distributed Systems (ICPADS), 504–11. IEEE, 2024. http://dx.doi.org/10.1109/icpads63350.2024.00072.

Der volle Inhalt der Quelle

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Shamir, Ohad, und Nathan Srebro. „Distributed stochastic optimization and learning“. In 2014 52nd Annual Allerton Conference on Communication, Control, and Computing (Allerton). IEEE, 2014. http://dx.doi.org/10.1109/allerton.2014.7028543.

Der volle Inhalt der Quelle

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Hulse, Daniel, Brandon Gigous, Kagan Tumer, Christopher Hoyle und Irem Y. Tumer. „Towards a Distributed Multiagent Learning-Based Design Optimization Method“. In ASME 2017 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference. American Society of Mechanical Engineers, 2017. http://dx.doi.org/10.1115/detc2017-68042.

Der volle Inhalt der Quelle

Annotation:

Complex engineered systems create many design challenges for engineers and organizations because of the interactions between subsystems and desire for optimality. In some conceptual-level optimizations, the design problem is simplified to consider the most important variables in an all-in-one optimization framework. This work introduces a stochastic optimization method which uses a distributed multiagent design method in which action-value based learning agents make individual design choices for each component. These agents use a probabilistic action-selection strategy based on the learned objective values of each action. This distributed multiagent system is applied to a simple quadrotor optimization problem in an all-in-one optimization framework, and compared with the performance of centralized methods. Results show the multiagent system is capable of finding comparable designs to centralized methods in a similar amount of computational time. This demonstrates the potential merit of a multiagent approach for complex systems design.

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Li, Naihao, Jiaqi Wang, Xu Liu, Lanfeng Wang und Long Zhang. „Contrastive Learning-based Meta-Learning Sequential Recommendation“. In 2024 International Conference on Distributed Computing and Optimization Techniques (ICDCOT). IEEE, 2024. http://dx.doi.org/10.1109/icdcot61034.2024.10515699.

Der volle Inhalt der Quelle

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Vaidya, Nitin H. „Security and Privacy for Distributed Optimization & Distributed Machine Learning“. In PODC '21: ACM Symposium on Principles of Distributed Computing. New York, NY, USA: ACM, 2021. http://dx.doi.org/10.1145/3465084.3467485.

Der volle Inhalt der Quelle

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Liao, Leonardo, und Yongqiang Wu. „Distributed Polytope ARTMAP: A Vigilance-Free ART Network for Distributed Supervised Learning“. In 2009 International Joint Conference on Computational Sciences and Optimization, CSO. IEEE, 2009. http://dx.doi.org/10.1109/cso.2009.63.

Der volle Inhalt der Quelle

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Wang, Shoujin, Fan Wang und Yu Zhang. „Learning Rate Decay Algorithm Based on Mutual Information in Deep Learning“. In 2024 International Conference on Distributed Computing and Optimization Techniques (ICDCOT). IEEE, 2024. http://dx.doi.org/10.1109/icdcot61034.2024.10515368.

Der volle Inhalt der Quelle

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Anand, Aditya, Lakshay Rastogi, Ansh Agarwaal und Shashank Bhardwaj. „Refraction-Learning Based Whale Optimization Algorithm with Opposition-Learning and Adaptive Parameter Optimization“. In 2024 Third International Conference on Distributed Computing and Electrical Circuits and Electronics (ICDCECE). IEEE, 2024. http://dx.doi.org/10.1109/icdcece60827.2024.10548420.

Der volle Inhalt der Quelle

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Berichte der Organisationen zum Thema "Distributed optimization and learning"

Stuckey, Peter, und Toby Walsh. Learning within Optimization. Fort Belvoir, VA: Defense Technical Information Center, April 2013. http://dx.doi.org/10.21236/ada575367.

Der volle Inhalt der Quelle

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Nygard, Kendall E. Distributed Optimization in Aircraft Mission Scheduling. Fort Belvoir, VA: Defense Technical Information Center, Mai 1995. http://dx.doi.org/10.21236/ada300064.

Der volle Inhalt der Quelle

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Meyer, Robert R. Large-Scale Optimization Via Distributed Systems. Fort Belvoir, VA: Defense Technical Information Center, November 1989. http://dx.doi.org/10.21236/ada215136.

Der volle Inhalt der Quelle

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Shead, Timothy, Jonathan Berry, Cynthia Phillips und Jared Saia. Information-Theoretically Secure Distributed Machine Learning. Office of Scientific and Technical Information (OSTI), November 2019. http://dx.doi.org/10.2172/1763277.

Der volle Inhalt der Quelle

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Graesser, Arthur C., und Robert A. Wisher. Question Generation as a Learning Multiplier in Distributed Learning Environments. Fort Belvoir, VA: Defense Technical Information Center, Oktober 2001. http://dx.doi.org/10.21236/ada399456.

Der volle Inhalt der Quelle

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Voon, B. K., und M. A. Austin. Structural Optimization in a Distributed Computing Environment. Fort Belvoir, VA: Defense Technical Information Center, Januar 1991. http://dx.doi.org/10.21236/ada454846.

Der volle Inhalt der Quelle

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Hays, Robert T. Theoretical Foundation for Advanced Distributed Learning Research. Fort Belvoir, VA: Defense Technical Information Center, Mai 2001. http://dx.doi.org/10.21236/ada385457.

Der volle Inhalt der Quelle

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Chen, J. S. J. Distributed-query optimization in fragmented data-base systems. Office of Scientific and Technical Information (OSTI), August 1987. http://dx.doi.org/10.2172/7183881.

Der volle Inhalt der Quelle

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Nocedal, Jorge. Nonlinear Optimization Methods for Large-Scale Learning. Office of Scientific and Technical Information (OSTI), Oktober 2019. http://dx.doi.org/10.2172/1571768.

Der volle Inhalt der Quelle

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Lumsdaine, Andrew. Scalable Second Order Optimization for Machine Learning. Office of Scientific and Technical Information (OSTI), Mai 2022. http://dx.doi.org/10.2172/1984057.

Der volle Inhalt der Quelle

APA, Harvard, Vancouver, ISO und andere Zitierweisen

Wir bieten Rabatte auf alle Premium-Pläne für Autoren, deren Werke in thematische Literatursammlungen aufgenommen wurden. Kontaktieren Sie uns, um einen einzigartigen Promo-Code zu erhalten!