Bibliografias temáticas / Distributed optimization and learning

Índice

Artigos de revistas
Teses / dissertações
Livros
Capítulos de livros
Trabalhos de conferências
Relatórios de organizações

Literatura científica selecionada sobre o tema "Distributed optimization and learning"

Autor: Grafiati

Publicado: 15 de fevereiro de 2025

Crie uma referência precisa em APA, MLA, Chicago, Harvard, e outros estilos

Selecione um tipo de fonte:

Consulte a lista de atuais artigos, livros, teses, anais de congressos e outras fontes científicas relevantes para o tema "Distributed optimization and learning".

Ao lado de cada fonte na lista de referências, há um botão "Adicionar à bibliografia". Clique e geraremos automaticamente a citação bibliográfica do trabalho escolhido no estilo de citação de que você precisa: APA, MLA, Harvard, Chicago, Vancouver, etc.

Você também pode baixar o texto completo da publicação científica em formato .pdf e ler o resumo do trabalho online se estiver presente nos metadados.

Artigos de revistas sobre o assunto "Distributed optimization and learning"

Kamalesh, Kamalesh, e Dr Gobi Natesan. "Machine Learning-Driven Analysis of Distributed Computing Systems: Exploring Optimization and Efficiency". International Journal of Research Publication and Reviews 5, n.º 3 (9 de março de 2024): 3979–83. http://dx.doi.org/10.55248/gengpi.5.0324.0786.

Texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

Mertikopoulos, Panayotis, E. Veronica Belmega, Romain Negrel e Luca Sanguinetti. "Distributed Stochastic Optimization via Matrix Exponential Learning". IEEE Transactions on Signal Processing 65, n.º 9 (1 de maio de 2017): 2277–90. http://dx.doi.org/10.1109/tsp.2017.2656847.

Texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

Gratton, Cristiano, Naveen K. D. Venkategowda, Reza Arablouei e Stefan Werner. "Privacy-Preserved Distributed Learning With Zeroth-Order Optimization". IEEE Transactions on Information Forensics and Security 17 (2022): 265–79. http://dx.doi.org/10.1109/tifs.2021.3139267.

Texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

Blot, Michael, David Picard, Nicolas Thome e Matthieu Cord. "Distributed optimization for deep learning with gossip exchange". Neurocomputing 330 (fevereiro de 2019): 287–96. http://dx.doi.org/10.1016/j.neucom.2018.11.002.

Texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

Young, M. Todd, Jacob D. Hinkle, Ramakrishnan Kannan e Arvind Ramanathan. "Distributed Bayesian optimization of deep reinforcement learning algorithms". Journal of Parallel and Distributed Computing 139 (maio de 2020): 43–52. http://dx.doi.org/10.1016/j.jpdc.2019.07.008.

Texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

Nedic, Angelia. "Distributed Gradient Methods for Convex Machine Learning Problems in Networks: Distributed Optimization". IEEE Signal Processing Magazine 37, n.º 3 (maio de 2020): 92–101. http://dx.doi.org/10.1109/msp.2020.2975210.

Texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

Lin, I.-Cheng. "Learning and Optimization over Robust Networked Systems". ACM SIGMETRICS Performance Evaluation Review 52, n.º 3 (9 de janeiro de 2025): 23–26. https://doi.org/10.1145/3712170.3712179.

Texto completo da fonte

Resumo:

Research Summary: 1 Introduction Networked systems are ubiquitous in our daily lives, playing a critical role across a wide range of scientific fields, including communication, machine learning, optimization, control, biology, economics, and social sciences. In machine learning and optimization, distributed learning and distributed optimization exploit the structure of such networked systems, which enables training models across multiple users [1] and solving complex problems by partitioning tasks among interconnected agents [2]. This significantly enhances both scalability and efficiency of the learning/optimization task. In control systems, for example, networked architectures synchronize the operations of different parts of the system like sensors, controllers etc. across distributed environments, ensuring the overall system's stability and performance [3]. In economics, networked systems were utilized to capture the interdependencies between different part of the economic system, allowing further insights of market dynamics [4]. In the social sciences, specifically social networks, study the propagation of idea, behaviors and opinions, offering a deeper understanding of societal structures and its dynamics [5].

Estilos ABNT, Harvard, Vancouver, APA, etc.

Gao, Hongchang. "Distributed Stochastic Nested Optimization for Emerging Machine Learning Models: Algorithm and Theory". Proceedings of the AAAI Conference on Artificial Intelligence 37, n.º 13 (26 de junho de 2023): 15437. http://dx.doi.org/10.1609/aaai.v37i13.26804.

Texto completo da fonte

Resumo:

Traditional machine learning models can be formulated as the expected risk minimization (ERM) problem: minw∈Rd Eξ [l(w; ξ)], where w ∈ Rd denotes the model parameter, ξ represents training samples, l(·) is the loss function. Numerous optimization algorithms, such as stochastic gradient descent (SGD), have been developed to solve the ERM problem. However, a wide range of emerging machine learning models are beyond this class of optimization problems, such as model-agnostic meta-learning (Finn, Abbeel, and Levine 2017). Of particular interest of my research is the stochastic nested optimization (SNO) problem, whose objective function has a nested structure. Specifically, I have been focusing on two instances of this kind of problem: stochastic compositional optimization (SCO) problems, which cover meta-learning, area-under-the-precision recall-curve optimization, contrastive self-supervised learning, etc., and stochastic bilevel optimization (SBO) problems, which can be applied to meta-learning, hyperparameter optimization, neural network architecture search, etc. With the emergence of large-scale distributed data, such as the user data generated on mobile devices or intelligent hardware, it is imperative to develop distributed optimization algorithms for SNO (Distributed SNO). A significant challenge for optimizing distributed SNO problems lies in that the stochastic (hyper-)gradient is a biased estimation of the full gradient. Thus, existing distributed optimization algorithms when applied to them suffer from slow convergence rates. In this talk, I will discuss my recent works about distributed SCO (Gao and Huang 2021; Gao, Li, and Huang 2022) and distributed SBO (Gao, Gu, and Thai 2022; Gao 2022) under both centralized and decentralized settings, including algorithmic details about reducing the bias of stochastic gradient, theoretical convergence rate, and practical machine learning applications, and then highlight challenges for future research.

Estilos ABNT, Harvard, Vancouver, APA, etc.

Choi, Dojin, Jiwon Wee, Sangho Song, Hyeonbyeong Lee, Jongtae Lim, Kyoungsoo Bok e Jaesoo Yoo. "k-NN Query Optimization for High-Dimensional Index Using Machine Learning". Electronics 12, n.º 11 (24 de maio de 2023): 2375. http://dx.doi.org/10.3390/electronics12112375.

Texto completo da fonte

Resumo:

In this study, we propose three k-nearest neighbor (k-NN) optimization techniques for a distributed, in-memory-based, high-dimensional indexing method to speed up content-based image retrieval. The proposed techniques perform distributed, in-memory, high-dimensional indexing-based k-NN query optimization: a density-based optimization technique that performs k-NN optimization using data distribution; a cost-based optimization technique using query processing cost statistics; and a learning-based optimization technique using a deep learning model, based on query logs. The proposed techniques were implemented on Spark, which supports a master/slave model for large-scale distributed processing. We showed the superiority and validity of the proposed techniques through various performance evaluations, based on high-dimensional data.

Estilos ABNT, Harvard, Vancouver, APA, etc.

Yang, Peng, e Ping Li. "Distributed Primal-Dual Optimization for Online Multi-Task Learning". Proceedings of the AAAI Conference on Artificial Intelligence 34, n.º 04 (3 de abril de 2020): 6631–38. http://dx.doi.org/10.1609/aaai.v34i04.6139.

Texto completo da fonte

Resumo:

Conventional online multi-task learning algorithms suffer from two critical limitations: 1) Heavy communication caused by delivering high velocity of sequential data to a central machine; 2) Expensive runtime complexity for building task relatedness. To address these issues, in this paper we consider a setting where multiple tasks are geographically located in different places, where one task can synchronize data with others to leverage knowledge of related tasks. Specifically, we propose an adaptive primal-dual algorithm, which not only captures task-specific noise in adversarial learning but also carries out a projection-free update with runtime efficiency. Moreover, our model is well-suited to decentralized periodic-connected tasks as it allows the energy-starved or bandwidth-constraint tasks to postpone the update. Theoretical results demonstrate the convergence guarantee of our distributed algorithm with an optimal regret. Empirical results confirm that the proposed model is highly effective on various real-world datasets.

Estilos ABNT, Harvard, Vancouver, APA, etc.

Mais fontes

Teses / dissertações sobre o assunto "Distributed optimization and learning"

Funkquist, Mikaela, e Minghua Lu. "Distributed Optimization Through Deep Reinforcement Learning". Thesis, KTH, Skolan för elektroteknik och datavetenskap (EECS), 2020. http://urn.kb.se/resolve?urn=urn:nbn:se:kth:diva-293878.

Texto completo da fonte

Resumo:

Reinforcement learning methods allows self-learningagents to play video- and board games autonomously. Thisproject aims to study the efficiency of the reinforcement learningalgorithms Q-learning and deep Q-learning for dynamical multi-agent problems. The goal is to train robots to optimally navigatethrough a warehouse without colliding.A virtual environment was created, in which the learning algo-rithms were tested by simulating moving agents. The algorithms’efficiency was evaluated by how fast the agents learned to performpredetermined tasks.The results show that Q-learning excels in simple problemswith few agents, quickly solving systems with two active agents.Deep Q-learning proved to be better suited for complex systemscontaining several agents, though cases of sub-optimal movementwere still possible. Both algorithms showed great potential fortheir respective areas however improvements still need to be madefor any real-world use.
Förstärkningsinlärningsmetoder tillåter självlärande enheter att spela video- och brädspel autonomt. Projektet siktar på att studera effektiviteten hos förstärkningsinlärningsmetoderna Q-learning och deep Q-learning i dynamiska problem. Målet är att träna upp robotar så att de kan röra sig genom ett varuhus på bästa sätt utan att kollidera. En virtuell miljö skapades, i vilken algoritmerna testades genom att simulera agenter som rörde sig. Algoritmernas effektivitet utvärderades av hur snabbt agenterna lärde sig att utföra förutbestämda uppgifter. Resultatet visar att Q-learning fungerar bra för enkla problem med få agenter, där system med två aktiva agenter löstes snabbt. Deep Q-learning fungerar bättre för mer komplexa system som innehåller fler agenter, men fall med suboptimala rörelser uppstod. Båda algoritmerna visade god potential inom deras respektive områden, däremot måste förbättringar göras innan de kan användas i verkligheten.
Kandidatexjobb i elektroteknik 2020, KTH, Stockholm

Estilos ABNT, Harvard, Vancouver, APA, etc.

Konečný, Jakub. "Stochastic, distributed and federated optimization for machine learning". Thesis, University of Edinburgh, 2017. http://hdl.handle.net/1842/31478.

Texto completo da fonte

Resumo:

We study optimization algorithms for the finite sum problems frequently arising in machine learning applications. First, we propose novel variants of stochastic gradient descent with a variance reduction property that enables linear convergence for strongly convex objectives. Second, we study distributed setting, in which the data describing the optimization problem does not fit into a single computing node. In this case, traditional methods are inefficient, as the communication costs inherent in distributed optimization become the bottleneck. We propose a communication-efficient framework which iteratively forms local subproblems that can be solved with arbitrary local optimization algorithms. Finally, we introduce the concept of Federated Optimization/Learning, where we try to solve the machine learning problems without having data stored in any centralized manner. The main motivation comes from industry when handling user-generated data. The current prevalent practice is that companies collect vast amounts of user data and store them in datacenters. An alternative we propose is not to collect the data in first place, and instead occasionally use the computational power of users' devices to solve the very same optimization problems, while alleviating privacy concerns at the same time. In such setting, minimization of communication rounds is the primary goal, and we demonstrate that solving the optimization problems in such circumstances is conceptually tractable.

Estilos ABNT, Harvard, Vancouver, APA, etc.

Armond, Kenneth C. Jr. "Distributed Support Vector Machine Learning". ScholarWorks@UNO, 2008. http://scholarworks.uno.edu/td/711.

Texto completo da fonte

Resumo:

Support Vector Machines (SVMs) are used for a growing number of applications. A fundamental constraint on SVM learning is the management of the training set. This is because the order of computations goes as the square of the size of the training set. Typically, training sets of 1000 (500 positives and 500 negatives, for example) can be managed on a PC without hard-drive thrashing. Training sets of 10,000 however, simply cannot be managed with PC-based resources. For this reason most SVM implementations must contend with some kind of chunking process to train parts of the data at a time (10 chunks of 1000, for example, to learn the 10,000). Sequential and multi-threaded chunking methods provide a way to run the SVM on large datasets while retaining accuracy. The multi-threaded distributed SVM described in this thesis is implemented using Java RMI, and has been developed to run on a network of multi-core/multi-processor computers.

Estilos ABNT, Harvard, Vancouver, APA, etc.

Patvarczki, Jozsef. "Layout Optimization for Distributed Relational Databases Using Machine Learning". Digital WPI, 2012. https://digitalcommons.wpi.edu/etd-dissertations/291.

Texto completo da fonte

Resumo:

A common problem when running Web-based applications is how to scale-up the database. The solution to this problem usually involves having a smart Database Administrator determine how to spread the database tables out amongst computers that will work in parallel. Laying out database tables across multiple machines so they can act together as a single efficient database is hard. Automated methods are needed to help eliminate the time required for database administrators to create optimal configurations. There are four operators that we consider that can create a search space of possible database layouts: 1) denormalizing, 2) horizontally partitioning, 3) vertically partitioning, and 4) fully replicating. Textbooks offer general advice that is useful for dealing with extreme cases - for instance you should fully replicate a table if the level of insert to selects is close to zero. But even this seemingly obvious statement is not necessarily one that will lead to a speed up once you take into account that some nodes might be a bottle neck. There can be complex interactions between the 4 different operators which make it even more difficult to predict what the best thing to do is. Instead of using best practices to do database layout, we need a system that collects empirical data on when these 4 different operators are effective. We have implemented a state based search technique to try different operators, and then we used the empirically measured data to see if any speed up occurred. We recognized that the costs of creating the physical database layout are potentially large, but it is necessary since we want to know the "Ground Truth" about what is effective and under what conditions. After creating a dataset where these four different operators have been applied to make different databases, we can employ machine learning to induce rules to help govern the physical design of the database across an arbitrary number of computer nodes. This learning process, in turn, would allow the database placement algorithm to get better over time as it trains over a set of examples. What this algorithm calls for is that it will try to learn 1) "What is a good database layout for a particular application given a query workload?" and 2) "Can this algorithm automatically improve itself in making recommendations by using machine learned rules to try to generalize when it makes sense to apply each of these operators?" There has been considerable research done in parallelizing databases where large amounts of data are shipped from one node to another to answer a single query. Sometimes the costs of shipping the data back and forth might be high, so in this work we assume that it might be more efficient to create a database layout where each query can be answered by a single node. To make this assumption requires that all the incoming query templates are known beforehand. This requirement can easily be satisfied in the case of a Web-based application due to the characteristic that users typically interact with the system through a web interface such as web forms. In this case, unseen queries are not necessarily answerable, without first possibly reconstructing the data on a single machine. Prior knowledge of these exact query templates allows us to select the best possible database table placements across multiple nodes. But in the case of trying to improve the efficiency of a Web-based application, a web site provider might feel that they are willing to suffer the inconvenience of not being able to answer an arbitrary query, if they are in turn provided with a system that runs more efficiently.

Estilos ABNT, Harvard, Vancouver, APA, etc.

Ouyang, Hua. "Optimal stochastic and distributed algorithms for machine learning". Diss., Georgia Institute of Technology, 2013. http://hdl.handle.net/1853/49091.

Texto completo da fonte

Resumo:

Stochastic and data-distributed optimization algorithms have received lots of attention from the machine learning community due to the tremendous demand from the large-scale learning and the big-data related optimization. A lot of stochastic and deterministic learning algorithms are proposed recently under various application scenarios. Nevertheless, many of these algorithms are based on heuristics and their optimality in terms of the generalization error is not sufficiently justified. In this talk, I will explain the concept of an optimal learning algorithm, and show that given a time budget and proper hypothesis space, only those achieving the lower bounds of the estimation error and the optimization error are optimal. Guided by this concept, we investigated the stochastic minimization of nonsmooth convex loss functions, a central problem in machine learning. We proposed a novel algorithm named Accelerated Nonsmooth Stochastic Gradient Descent, which exploits the structure of common nonsmooth loss functions to achieve optimal convergence rates for a class of problems including SVMs. It is the first stochastic algorithm that can achieve the optimal O(1/t) rate for minimizing nonsmooth loss functions. The fast rates are confirmed by empirical comparisons with state-of-the-art algorithms including the averaged SGD. The Alternating Direction Method of Multipliers (ADMM) is another flexible method to explore function structures. In the second part we proposed stochastic ADMM that can be applied to a general class of convex and nonsmooth functions, beyond the smooth and separable least squares loss used in lasso. We also demonstrate the rates of convergence for our algorithm under various structural assumptions of the stochastic function: O(1/sqrt{t}) for convex functions and O(log t/t) for strongly convex functions. A novel application named Graph-Guided SVM is proposed to demonstrate the usefulness of our algorithm. We also extend the scalability of stochastic algorithms to nonlinear kernel machines, where the problem is formulated as a constrained dual quadratic optimization. The simplex constraint can be handled by the classic Frank-Wolfe method. The proposed stochastic Frank-Wolfe methods achieve comparable or even better accuracies than state-of-the-art batch and online kernel SVM solvers, and are significantly faster. The last part investigates the problem of data-distributed learning. We formulate it as a consensus-constrained optimization problem and solve it with ADMM. It turns out that the underlying communication topology is a key factor in achieving a balance between a fast learning rate and computation resource consumption. We analyze the linear convergence behavior of consensus ADMM so as to characterize the interplay between the communication topology and the penalty parameters used in ADMM. We observe that given optimal parameters, the complete bipartite and the master-slave graphs exhibit the fastest convergence, followed by bi-regular graphs.

Estilos ABNT, Harvard, Vancouver, APA, etc.

El, Gamal Mostafa. "Distributed Statistical Learning under Communication Constraints". Digital WPI, 2017. https://digitalcommons.wpi.edu/etd-dissertations/314.

Texto completo da fonte

Resumo:

"In this thesis, we study distributed statistical learning, in which multiple terminals, connected by links with limited capacity, cooperate to perform a learning task. As the links connecting the terminals have limited capacity, the messages exchanged between the terminals have to be compressed. The goal of this thesis is to investigate how to compress the data observations at multiple terminals and how to use the compressed data for inference. We first focus on the distributed parameter estimation problem, in which terminals send messages related to their local observations using limited rates to a fusion center that will obtain an estimate of a parameter related to the observations of all terminals. It is well known that if the transmission rates are in the Slepian-Wolf region, the fusion center can fully recover all observations and hence can construct an estimator having the same performance as that of the centralized case. One natural question is whether Slepian-Wolf rates are necessary to achieve the same estimation performance as that of the centralized case. In this thesis, we show that the answer to this question is negative. We then examine the optimality of data dimensionality reduction via sufficient statistics compression in distributed parameter estimation problems. The data dimensionality reduction step is often needed especially if the data has a very high dimension and the communication rate is not as high as the one characterized above. We show that reducing the dimensionality by extracting sufficient statistics of the parameter to be estimated does not degrade the overall estimation performance in the presence of communication constraints. We further analyze the optimal estimation performance in the presence of communication constraints and we verify the derived bound using simulations. Finally, we study distributed optimization problems, for which we examine the randomized distributed coordinate descent algorithm with quantized updates. In the literature, the iteration complexity of the randomized distributed coordinate descent algorithm has been characterized under the assumption that machines can exchange updates with an infinite precision. We consider a practical scenario in which the messages exchange occurs over channels with finite capacity, and hence the updates have to be quantized. We derive sufficient conditions on the quantization error such that the algorithm with quantized update still converge."

Estilos ABNT, Harvard, Vancouver, APA, etc.

Dai, Wei. "Learning with Staleness". Research Showcase @ CMU, 2018. http://repository.cmu.edu/dissertations/1209.

Texto completo da fonte

Resumo:

A fundamental assumption behind most machine learning (ML) algorithms and analyses is the sequential execution. That is, any update to the ML model can be immediately applied and the new model is always available for the next algorithmic step. This basic assumption, however, can be costly to realize, when the computation is carried out across multiple machines, linked by commodity networks that are usually 104 times slower than the memory speed due to fundamental hardware limitations. As a result, concurrent ML computation in the distributed settings often needs to handle delayed updates and perform learning in the presence of staleness. This thesis characterizes learning with staleness from three directions: (1) We extend the theoretical analyses of a number of classical ML algorithms, including stochastic gradient descent, proximal gradient descent on non-convex problems, and Frank-Wolfe algorithms, to explicitly incorporate staleness into their convergence characterizations. (2)We conduct simulation and large-scale distributed experiments to study the empirical effects of staleness on ML algorithms under indeterministic executions. Our results reveal that staleness is a key parameter governing the convergence speed for all considered ML algorithms, with varied ramifications. (3) We design staleness-minimizing parameter server systems by optimizing synchronization methods to effectively reduce the runtime staleness. The proposed optimization of a bounded consistency model utilizes the additional network bandwidths to communicate updates eagerly, relieving users of the burden to tune the staleness level. By minimizing staleness at the framework level, our system stabilizes diverging optimization paths and substantially accelerates convergence across ML algorithms without any modification to the ML programs.

Estilos ABNT, Harvard, Vancouver, APA, etc.

Lu, Yumao. "Kernel optimization and distributed learning algorithms for support vector machines". Diss., Restricted to subscribing institutions, 2005. http://uclibs.org/PID/11984.

Texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

Dinh, The Canh. "Distributed Algorithms for Fast and Personalized Federated Learning". Thesis, The University of Sydney, 2023. https://hdl.handle.net/2123/30019.

Texto completo da fonte

Resumo:

The significant increase in the number of cutting-edge user equipment (UE) results in the phenomenal growth of the data volume generated at the edge. This shift fuels the booming trend of an emerging technique named Federated Learning. In contrast to traditional methods in which data is collected and processed centrally, FL builds a global model from contributions of UE's model without sending private data then effectively ensures data privacy. However, FL faces challenges in non-identically distributed (non-IID) data, communication cost, and convergence rate. Firstly, we propose first-order optimization FL algorithms named FedApprox and FEDL to improve the convergence rate. We propose FedApprox exploiting proximal stochastic variance-reduced gradient methods and extract insights from convergence conditions via the algorithm’s parameter control. We then propose FEDL to handle heterogeneous UE data and characterize the trade-off between local computation and global communication. Experimentally, FedApprox outperforms vanilla FedAvg while FEDL outperforms FedApprox and FedAvg. Secondly, we consider the communication between edges to be more costly than local computational overhead. We propose DONE, a distributed approximate Newton-type algorithm for communication-efficient federated edge learning. DONE approximates Newton direction using classical Richardson iteration on each edge. Experimentally, DONE attains a comparable performance to Newton’s method and outperforms first-order algorithms. Finally, we address the non-IID issue by proposing pFedMe, a personalized FL algorithm using Moreau envelopes. pFedMe achieves quadratic speedup for strongly convex and sublinear speedup of order 2/3 for smooth nonconvex objectives. We then propose FedU, a Federated Multitask Learning algorithm using Laplacian regularization to leverage the relationships among the users' models. Experimentally, pFedMe excels FedAvg and Per-FedAvg while FedU outperforms pFedMe and MOCHA.

Estilos ABNT, Harvard, Vancouver, APA, etc.

Reddi, Sashank Jakkam. "New Optimization Methods for Modern Machine Learning". Research Showcase @ CMU, 2017. http://repository.cmu.edu/dissertations/1116.

Texto completo da fonte

Resumo:

Modern machine learning systems pose several new statistical, scalability, privacy and ethical challenges. With the advent of massive datasets and increasingly complex tasks, scalability has especially become a critical issue in these systems. In this thesis, we focus on fundamental challenges related to scalability, such as computational and communication efficiency, in modern machine learning applications. The underlying central message of this thesis is that classical statistical thinking leads to highly effective optimization methods for modern big data applications. The first part of the thesis investigates optimization methods for solving large-scale nonconvex Empirical Risk Minimization (ERM) problems. Such problems have surged into prominence, notably through deep learning, and have led to exciting progress. However, our understanding of optimization methods suitable for these problems is still very limited. We develop and analyze a new line of optimization methods for nonconvex ERM problems, based on the principle of variance reduction. We show that our methods exhibit fast convergence to stationary points and improve the state-of-the-art in several nonconvex ERM settings, including nonsmooth and constrained ERM. Using similar principles, we also develop novel optimization methods that provably converge to second-order stationary points. Finally, we show that the key principles behind our methods can be generalized to overcome challenges in other important problems such as Bayesian inference. The second part of the thesis studies two critical aspects of modern distributed machine learning systems — asynchronicity and communication efficiency of optimization methods. We study various asynchronous stochastic algorithms with fast convergence for convex ERM problems and show that these methods achieve near-linear speedups in sparse settings common to machine learning. Another key factor governing the overall performance of a distributed system is its communication efficiency. Traditional optimization algorithms used in machine learning are often ill-suited for distributed environments with high communication cost. To address this issue, we dis- cuss two different paradigms to achieve communication efficiency of algorithms in distributed environments and explore new algorithms with better communication complexity.

Estilos ABNT, Harvard, Vancouver, APA, etc.

Mais fontes

Livros sobre o assunto "Distributed optimization and learning"

Jiang, Jiawei, Bin Cui e Ce Zhang. Distributed Machine Learning and Gradient Optimization. Singapore: Springer Singapore, 2022. http://dx.doi.org/10.1007/978-981-16-3420-8.

Texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

Wang, Huiwei, Huaqing Li e Bo Zhou. Distributed Optimization, Game and Learning Algorithms. Singapore: Springer Singapore, 2021. http://dx.doi.org/10.1007/978-981-33-4528-7.

Texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

Joshi, Gauri. Optimization Algorithms for Distributed Machine Learning. Cham: Springer International Publishing, 2023. http://dx.doi.org/10.1007/978-3-031-19067-4.

Texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

Tatarenko, Tatiana. Game-Theoretic Learning and Distributed Optimization in Memoryless Multi-Agent Systems. Cham: Springer International Publishing, 2017. http://dx.doi.org/10.1007/978-3-319-65479-9.

Texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

Oblinger, Diana G. Distributed learning. Boulder, Colo: CAUSE, 1996.

Encontre o texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

Majhi, Sudhan, Rocío Pérez de Prado e Chandrappa Dasanapura Nanjundaiah, eds. Distributed Computing and Optimization Techniques. Singapore: Springer Nature Singapore, 2022. http://dx.doi.org/10.1007/978-981-19-2281-7.

Texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

Giselsson, Pontus, e Anders Rantzer, eds. Large-Scale and Distributed Optimization. Cham: Springer International Publishing, 2018. http://dx.doi.org/10.1007/978-3-319-97478-1.

Texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

Lü, Qingguo, Xiaofeng Liao, Huaqing Li, Shaojiang Deng e Shanfu Gao. Distributed Optimization in Networked Systems. Singapore: Springer Nature Singapore, 2023. http://dx.doi.org/10.1007/978-981-19-8559-1.

Texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

Abdulrahman Younis Ali Younis Kalbat. Distributed and Large-Scale Optimization. [New York, N.Y.?]: [publisher not identified], 2016.

Encontre o texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

Otto, Daniel, Gianna Scharnberg, Michael Kerres e Olaf Zawacki-Richter, eds. Distributed Learning Ecosystems. Wiesbaden: Springer Fachmedien Wiesbaden, 2023. http://dx.doi.org/10.1007/978-3-658-38703-7.

Texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

Mais fontes

Capítulos de livros sobre o assunto "Distributed optimization and learning"

Joshi, Gauri, e Shiqiang Wang. "Communication-Efficient Distributed Optimization Algorithms". In Federated Learning, 125–43. Cham: Springer International Publishing, 2022. http://dx.doi.org/10.1007/978-3-030-96896-0_6.

Texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

Jiang, Jiawei, Bin Cui e Ce Zhang. "Distributed Gradient Optimization Algorithms". In Distributed Machine Learning and Gradient Optimization, 57–114. Singapore: Springer Singapore, 2021. http://dx.doi.org/10.1007/978-981-16-3420-8_3.

Texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

Jiang, Jiawei, Bin Cui e Ce Zhang. "Distributed Machine Learning Systems". In Distributed Machine Learning and Gradient Optimization, 115–66. Singapore: Springer Singapore, 2021. http://dx.doi.org/10.1007/978-981-16-3420-8_4.

Texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

Joshi, Gauri. "Distributed Optimization in Machine Learning". In Synthesis Lectures on Learning, Networks, and Algorithms, 1–12. Cham: Springer International Publishing, 2022. http://dx.doi.org/10.1007/978-3-031-19067-4_1.

Texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

Lin, Zhouchen, Huan Li e Cong Fang. "ADMM for Distributed Optimization". In Alternating Direction Method of Multipliers for Machine Learning, 207–40. Singapore: Springer Nature Singapore, 2022. http://dx.doi.org/10.1007/978-981-16-9840-8_6.

Texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

Jiang, Jiawei, Bin Cui e Ce Zhang. "Basics of Distributed Machine Learning". In Distributed Machine Learning and Gradient Optimization, 15–55. Singapore: Springer Singapore, 2021. http://dx.doi.org/10.1007/978-981-16-3420-8_2.

Texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

Scheidegger, Carre, Arpit Shah e Dan Simon. "Distributed Learning with Biogeography-Based Optimization". In Lecture Notes in Computer Science, 203–15. Berlin, Heidelberg: Springer Berlin Heidelberg, 2011. http://dx.doi.org/10.1007/978-3-642-21827-9_21.

Texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

González-Mendoza, Miguel, Neil Hernández-Gress e André Titli. "Quadratic Optimization Fine Tuning for the Learning Phase of SVM". In Advanced Distributed Systems, 347–57. Berlin, Heidelberg: Springer Berlin Heidelberg, 2005. http://dx.doi.org/10.1007/11533962_31.

Texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

Wang, Huiwei, Huaqing Li e Bo Zhou. "Cooperative Distributed Optimization in Multiagent Networks with Delays". In Distributed Optimization, Game and Learning Algorithms, 1–17. Singapore: Springer Singapore, 2021. http://dx.doi.org/10.1007/978-981-33-4528-7_1.

Texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

Wang, Huiwei, Huaqing Li e Bo Zhou. "Constrained Consensus of Multi-agent Systems with Time-Varying Topology". In Distributed Optimization, Game and Learning Algorithms, 19–37. Singapore: Springer Singapore, 2021. http://dx.doi.org/10.1007/978-981-33-4528-7_2.

Texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

Trabalhos de conferências sobre o assunto "Distributed optimization and learning"

Patil, Aditya, Sanket Lodha, Sonal Deshmukh, Rupali S. Joshi, Vaishali Patil e Sudhir Chitnis. "Battery Optimization Using Machine Learning". In 2024 IEEE International Conference on Blockchain and Distributed Systems Security (ICBDS), 1–5. IEEE, 2024. https://doi.org/10.1109/icbds61829.2024.10837428.

Texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

Khan, Malak Abid Ali, Luo Senlin, Hongbin Ma, Abdul Khalique Shaikh, Ahlam Almusharraf e Imran Khan Mirani. "Optimization of LoRa for Distributed Environments Based on Machine Learning". In 2024 IEEE Asia Pacific Conference on Wireless and Mobile (APWiMob), 137–42. IEEE, 2024. https://doi.org/10.1109/apwimob64015.2024.10792952.

Texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

Chao, Liangchen, Bo Zhang, Hengpeng Guo, Fangheng Ji e Junfeng Li. "UAV Swarm Collaborative Transmission Optimization for Machine Learning Tasks". In 2024 IEEE 30th International Conference on Parallel and Distributed Systems (ICPADS), 504–11. IEEE, 2024. http://dx.doi.org/10.1109/icpads63350.2024.00072.

Texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

Shamir, Ohad, e Nathan Srebro. "Distributed stochastic optimization and learning". In 2014 52nd Annual Allerton Conference on Communication, Control, and Computing (Allerton). IEEE, 2014. http://dx.doi.org/10.1109/allerton.2014.7028543.

Texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

Hulse, Daniel, Brandon Gigous, Kagan Tumer, Christopher Hoyle e Irem Y. Tumer. "Towards a Distributed Multiagent Learning-Based Design Optimization Method". In ASME 2017 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference. American Society of Mechanical Engineers, 2017. http://dx.doi.org/10.1115/detc2017-68042.

Texto completo da fonte

Resumo:

Complex engineered systems create many design challenges for engineers and organizations because of the interactions between subsystems and desire for optimality. In some conceptual-level optimizations, the design problem is simplified to consider the most important variables in an all-in-one optimization framework. This work introduces a stochastic optimization method which uses a distributed multiagent design method in which action-value based learning agents make individual design choices for each component. These agents use a probabilistic action-selection strategy based on the learned objective values of each action. This distributed multiagent system is applied to a simple quadrotor optimization problem in an all-in-one optimization framework, and compared with the performance of centralized methods. Results show the multiagent system is capable of finding comparable designs to centralized methods in a similar amount of computational time. This demonstrates the potential merit of a multiagent approach for complex systems design.

Estilos ABNT, Harvard, Vancouver, APA, etc.

Li, Naihao, Jiaqi Wang, Xu Liu, Lanfeng Wang e Long Zhang. "Contrastive Learning-based Meta-Learning Sequential Recommendation". In 2024 International Conference on Distributed Computing and Optimization Techniques (ICDCOT). IEEE, 2024. http://dx.doi.org/10.1109/icdcot61034.2024.10515699.

Texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

Vaidya, Nitin H. "Security and Privacy for Distributed Optimization & Distributed Machine Learning". In PODC '21: ACM Symposium on Principles of Distributed Computing. New York, NY, USA: ACM, 2021. http://dx.doi.org/10.1145/3465084.3467485.

Texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

Liao, Leonardo, e Yongqiang Wu. "Distributed Polytope ARTMAP: A Vigilance-Free ART Network for Distributed Supervised Learning". In 2009 International Joint Conference on Computational Sciences and Optimization, CSO. IEEE, 2009. http://dx.doi.org/10.1109/cso.2009.63.

Texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

Wang, Shoujin, Fan Wang e Yu Zhang. "Learning Rate Decay Algorithm Based on Mutual Information in Deep Learning". In 2024 International Conference on Distributed Computing and Optimization Techniques (ICDCOT). IEEE, 2024. http://dx.doi.org/10.1109/icdcot61034.2024.10515368.

Texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

Anand, Aditya, Lakshay Rastogi, Ansh Agarwaal e Shashank Bhardwaj. "Refraction-Learning Based Whale Optimization Algorithm with Opposition-Learning and Adaptive Parameter Optimization". In 2024 Third International Conference on Distributed Computing and Electrical Circuits and Electronics (ICDCECE). IEEE, 2024. http://dx.doi.org/10.1109/icdcece60827.2024.10548420.

Texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

Relatórios de organizações sobre o assunto "Distributed optimization and learning"

Stuckey, Peter, e Toby Walsh. Learning within Optimization. Fort Belvoir, VA: Defense Technical Information Center, abril de 2013. http://dx.doi.org/10.21236/ada575367.

Texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

Nygard, Kendall E. Distributed Optimization in Aircraft Mission Scheduling. Fort Belvoir, VA: Defense Technical Information Center, maio de 1995. http://dx.doi.org/10.21236/ada300064.

Texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

Meyer, Robert R. Large-Scale Optimization Via Distributed Systems. Fort Belvoir, VA: Defense Technical Information Center, novembro de 1989. http://dx.doi.org/10.21236/ada215136.

Texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

Shead, Timothy, Jonathan Berry, Cynthia Phillips e Jared Saia. Information-Theoretically Secure Distributed Machine Learning. Office of Scientific and Technical Information (OSTI), novembro de 2019. http://dx.doi.org/10.2172/1763277.

Texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

Graesser, Arthur C., e Robert A. Wisher. Question Generation as a Learning Multiplier in Distributed Learning Environments. Fort Belvoir, VA: Defense Technical Information Center, outubro de 2001. http://dx.doi.org/10.21236/ada399456.

Texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

Voon, B. K., e M. A. Austin. Structural Optimization in a Distributed Computing Environment. Fort Belvoir, VA: Defense Technical Information Center, janeiro de 1991. http://dx.doi.org/10.21236/ada454846.

Texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

Hays, Robert T. Theoretical Foundation for Advanced Distributed Learning Research. Fort Belvoir, VA: Defense Technical Information Center, maio de 2001. http://dx.doi.org/10.21236/ada385457.

Texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

Chen, J. S. J. Distributed-query optimization in fragmented data-base systems. Office of Scientific and Technical Information (OSTI), agosto de 1987. http://dx.doi.org/10.2172/7183881.

Texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

Nocedal, Jorge. Nonlinear Optimization Methods for Large-Scale Learning. Office of Scientific and Technical Information (OSTI), outubro de 2019. http://dx.doi.org/10.2172/1571768.

Texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

Lumsdaine, Andrew. Scalable Second Order Optimization for Machine Learning. Office of Scientific and Technical Information (OSTI), maio de 2022. http://dx.doi.org/10.2172/1984057.

Texto completo da fonte

Estilos ABNT, Harvard, Vancouver, APA, etc.

Oferecemos descontos em todos os planos premium para autores cujas obras estão incluídas em seleções literárias temáticas. Contate-nos para obter um código promocional único!