Deep learning has revolutionized a wide range of fields. In spite of its success, most deep learning systems are proposed in the cloud, where data are processed in a centralized manner with abundant compute and network resources. This raises a problem when deep learning is deployed on the edge where distributed compute resources are limited. In this dissertation, we propose three distributed systems to enable collaborative deep learning on the edge. These three systems target different scenarios and tasks. The first system dubbed Distream is a distributed live video analytics system based on the smart camera-edge cluster architecture. Distream fully utilizes the compute resources at both ends to achieve optimized system performance. The second system dubbed Mercury is a system that addresses the key bottleneck of collaborative learning. Mercury enhances the training efficiency of on-device collaborative learning without compromising the accuracies of the trained models. The third system dubbed FedAce is a distributed training system that improves training efficiency under federated learning setting where private on-device data are not allowed to be shared among local devices. Within each participating client, FedAce achieves such improvement by prioritizing important data. In the server where model aggregation is performed, FedAce exploits the client importance and prioritizes important clients to reduce stragglers and reduce the total number of rounds. In addition, FedAce conducts federated model compression to reduce the per-round communication cost and obtains a compact model after training completes. Extensive experiments show that the proposed three systems are able to achieve significant improvements over status-quo systems.
Read
