File Download
There are no files associated with this item.
Supplementary
-
Citations:
- Appears in Collections:
Conference Paper: Distributed Machine Learning through Heterogeneous Edge Systems
Title | Distributed Machine Learning through Heterogeneous Edge Systems |
---|---|
Authors | |
Issue Date | 2020 |
Publisher | AAAI Press. The Journal's web site is located at https://aaai.org/Library/AAAI/aaai-library.php |
Citation | Proceedings of the 34th Association for the Advancement of Artificial Intelligence (AAAI) Conference on Artificial Intelligence (AAAI-20), New York, NY, USA, 7-12 February 2020, v. 34 n. 5, p. 7179-7186 How to Cite? |
Abstract | Many emerging AI applications request distributed machine learning (ML) among edge systems (e.g., IoT devices and PCs at the edge of the Internet), where data cannot be uploaded to a central venue for model training, due to their large volumes and/or security/privacy concerns. Edge devices are intrinsically heterogeneous in computing capacity, posing significant challenges to parameter synchronization for parallel training with the parameter server (PS) architecture. This paper proposes ADSP, a parameter synchronization model for distributed machine learning (ML) with heterogeneous edge systems. Eliminating the significant waiting time occurring with existing parameter synchronization models, the core idea of ADSP is to let faster edge devices continue training, while committing their model updates at strategically decided intervals. We design algorithms that decide time points for each worker to commit its model update, and ensure not only global model convergence but also faster convergence. Our testbed implementation and experiments show that ADSP outperforms existing parameter synchronization models significantly in terms of ML model convergence time, scalability and adaptability to large heterogeneity. |
Description | AAAI-20 Technical Tracks 5 / AAAI Technical Track on Multiagent Systems |
Persistent Identifier | http://hdl.handle.net/10722/301296 |
ISSN |
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Hu, H | - |
dc.contributor.author | Wang, D | - |
dc.contributor.author | Wu, C | - |
dc.date.accessioned | 2021-07-27T08:09:01Z | - |
dc.date.available | 2021-07-27T08:09:01Z | - |
dc.date.issued | 2020 | - |
dc.identifier.citation | Proceedings of the 34th Association for the Advancement of Artificial Intelligence (AAAI) Conference on Artificial Intelligence (AAAI-20), New York, NY, USA, 7-12 February 2020, v. 34 n. 5, p. 7179-7186 | - |
dc.identifier.issn | 2159-5399 | - |
dc.identifier.uri | http://hdl.handle.net/10722/301296 | - |
dc.description | AAAI-20 Technical Tracks 5 / AAAI Technical Track on Multiagent Systems | - |
dc.description.abstract | Many emerging AI applications request distributed machine learning (ML) among edge systems (e.g., IoT devices and PCs at the edge of the Internet), where data cannot be uploaded to a central venue for model training, due to their large volumes and/or security/privacy concerns. Edge devices are intrinsically heterogeneous in computing capacity, posing significant challenges to parameter synchronization for parallel training with the parameter server (PS) architecture. This paper proposes ADSP, a parameter synchronization model for distributed machine learning (ML) with heterogeneous edge systems. Eliminating the significant waiting time occurring with existing parameter synchronization models, the core idea of ADSP is to let faster edge devices continue training, while committing their model updates at strategically decided intervals. We design algorithms that decide time points for each worker to commit its model update, and ensure not only global model convergence but also faster convergence. Our testbed implementation and experiments show that ADSP outperforms existing parameter synchronization models significantly in terms of ML model convergence time, scalability and adaptability to large heterogeneity. | - |
dc.language | eng | - |
dc.publisher | AAAI Press. The Journal's web site is located at https://aaai.org/Library/AAAI/aaai-library.php | - |
dc.relation.ispartof | Proceedings of the AAAI Conference on Artificial Intelligence | - |
dc.title | Distributed Machine Learning through Heterogeneous Edge Systems | - |
dc.type | Conference_Paper | - |
dc.identifier.email | Wu, C: cwu@cs.hku.hk | - |
dc.identifier.authority | Wu, C=rp01397 | - |
dc.identifier.doi | 10.1609/aaai.v34i05.6207 | - |
dc.identifier.hkuros | 323517 | - |
dc.identifier.volume | 34 | - |
dc.identifier.issue | 5 | - |
dc.identifier.spage | 7179 | - |
dc.identifier.epage | 7186 | - |
dc.publisher.place | United States | - |