東華大學圖書館 |

語系: 繁體中文

說明(常見問題)

回圖書館首頁

手機版館藏查詢

登入

回首頁

切換: 標籤 | MARC模式 | ISBD

FindBook

Google Book

Amazon

博客來

Toward Robust and Communication Efficient Distributed Machine Learning.

紀錄類型:	書目-電子資源 : Monograph/item
正題名/作者:	Toward Robust and Communication Efficient Distributed Machine Learning./
作者:	Wang, Hongyi.
出版者:	Ann Arbor : ProQuest Dissertations & Theses, : 2021,
面頁冊數:	273 p.
附註:	Source: Dissertations Abstracts International, Volume: 83-03, Section: B.
Contained By:	Dissertations Abstracts International83-03B.
標題:	Computer science. -
電子資源:	http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=28719050
ISBN:	9798538110582

Toward Robust and Communication Efficient Distributed Machine Learning.
Wang, Hongyi.

Toward Robust and Communication Efficient Distributed Machine Learning. - Ann Arbor : ProQuest Dissertations & Theses, 2021 - 273 p.

Source: Dissertations Abstracts International, Volume: 83-03, Section: B.

Thesis (Ph.D.)--The University of Wisconsin - Madison, 2021.

This item must not be sold to any third party vendors.

Distributed machine learning (ML) is emerging as its own field at the heart of MLSys due to the exploding scale of modern deep learning models and the enormous amounts of data. However, distributed ML suffers from low communication efficiency and is vulnerable to adversarial attacks. This dissertation focuses on improving the communication efficiency and robustness of distributed ML for two popular use cases, i.e., centralized distributed ML and federated learning (FL). The first part of this dissertation focuses on communication efficiency. For centralized distributed ML, we start by presenting Atomo, a general framework to compress the gradients via atomic sparsification. Improving Atomo, we present Pufferfish, which bypasses the need for gradient compression via integrating it into model training. Pufferfish trains the factorized low-rank model starting from its full-rank counterpart, which achieves both high communication and computation efficiency without the need of using any gradient compression. For FL, we propose FedMA, which uses matched averaging in a layer-wise manner instead of one-shot coordinate-wise averaging for model aggregation. FedMA effectively reduces the number of FL rounds needed for the global model to converge.The second part of this dissertation focuses on robustness. In the centralized setting, we present Draco, which leverages algorithmic redundancy to achieve Byzantine resilience with black-box convergence guarantees. To improve Draco, we present Detox, which combines robust aggregation with algorithmic redundancy. Detox can be used in tandem with any robust aggregation methods and enhances their Byzantine resilience and scalability. For FL, we demonstrate its vulnerability to training-time backdoors. We establish that robustness to backdoors implies model robustness to adversarial examples, a major open problem in itself. Furthermore, detecting the presence of a backdoor in an FL model is unlikely. We couple our results with edge-case backdoors, which forces a model to misclassify on seemingly easy inputs that are however unlikely to be part of the training or test data. We demonstrate that edge-case backdoors can lead to unsavory failures and may have serious repercussions on fairness and bypass all existing defense mechanisms.

ISBN: 9798538110582Subjects--Topical Terms:

523869
Computer science.
Subjects--Index Terms:

Communication Efficiency

Toward Robust and Communication Efficient Distributed Machine Learning.
LDR:03465nmm a2200361 4500 001 2348643
005 20220912135625.5
008 241004s2021 ||||||||||||||||| ||eng d
020 $a 9798538110582
035 $a (MiAaPQ)AAI28719050
035 $a AAI28719050
040 $a MiAaPQ $c MiAaPQ
100 1 $a Wang, Hongyi. $3 3688012
245 1 0 $a Toward Robust and Communication Efficient Distributed Machine Learning.
260 1 $a Ann Arbor : $b ProQuest Dissertations & Theses, $c 2021
300 $a 273 p.
500 $a Source: Dissertations Abstracts International, Volume: 83-03, Section: B.
500 $a Advisor: Papailiopoulos, Dimitrios.
502 $a Thesis (Ph.D.)--The University of Wisconsin - Madison, 2021.
506 $a This item must not be sold to any third party vendors.
520 $a Distributed machine learning (ML) is emerging as its own field at the heart of MLSys due to the exploding scale of modern deep learning models and the enormous amounts of data. However, distributed ML suffers from low communication efficiency and is vulnerable to adversarial attacks. This dissertation focuses on improving the communication efficiency and robustness of distributed ML for two popular use cases, i.e., centralized distributed ML and federated learning (FL). The first part of this dissertation focuses on communication efficiency. For centralized distributed ML, we start by presenting Atomo, a general framework to compress the gradients via atomic sparsification. Improving Atomo, we present Pufferfish, which bypasses the need for gradient compression via integrating it into model training. Pufferfish trains the factorized low-rank model starting from its full-rank counterpart, which achieves both high communication and computation efficiency without the need of using any gradient compression. For FL, we propose FedMA, which uses matched averaging in a layer-wise manner instead of one-shot coordinate-wise averaging for model aggregation. FedMA effectively reduces the number of FL rounds needed for the global model to converge.The second part of this dissertation focuses on robustness. In the centralized setting, we present Draco, which leverages algorithmic redundancy to achieve Byzantine resilience with black-box convergence guarantees. To improve Draco, we present Detox, which combines robust aggregation with algorithmic redundancy. Detox can be used in tandem with any robust aggregation methods and enhances their Byzantine resilience and scalability. For FL, we demonstrate its vulnerability to training-time backdoors. We establish that robustness to backdoors implies model robustness to adversarial examples, a major open problem in itself. Furthermore, detecting the presence of a backdoor in an FL model is unlikely. We couple our results with edge-case backdoors, which forces a model to misclassify on seemingly easy inputs that are however unlikely to be part of the training or test data. We demonstrate that edge-case backdoors can lead to unsavory failures and may have serious repercussions on fairness and bypass all existing defense mechanisms.
590 $a School code: 0262.
650 4 $a Computer science. $3 523869
650 4 $a Sparsity. $3 3680690
650 4 $a Accuracy. $3 3559958
650 4 $a Breakdowns. $3 3682712
650 4 $a Methods. $3 3560391
650 4 $a Datasets. $3 3541416
650 4 $a Communication. $3 524709
650 4 $a Experiments. $3 525909
650 4 $a Artificial intelligence. $3 516317
653 $a Communication Efficiency
653 $a Distributed Machine Learning
653 $a Federated Learning
653 $a Robustness
690 $a 0984
690 $a 0459
690 $a 0800
710 2 $a The University of Wisconsin - Madison. $b Computer Sciences. $3 2099760
773 0 $t Dissertations Abstracts International $g 83-03B.
790 $a 0262
791 $a Ph.D.
792 $a 2021
793 $a English
856 4 0 $u http://pqdd.sinica.edu.tw/twdaoapp/servlet/advanced?query=28719050