publications

2025

1. Minder: Faulty Machine Detection for Large-scale Distributed Model Training

   Yangtao Deng ,  Xiang Shi ,  Zhuo Jiang , and 12 more authors

   Apr 2025

   Abs HTML PDF

   Large-scale distributed model training requires simultaneous training on up to thousands of machines. Faulty machine detection is critical when an unexpected fault occurs in a machine. From our experience, a training task can encounter two faults per day on average, possibly leading to a halt for hours. To address the drawbacks of the time-consuming and labor-intensive manual scrutiny, we propose Minder, an automatic faulty machine detector for distributed training tasks. The key idea of Minder is to automatically and efficiently detect faulty distinctive monitoring metric patterns, which could last for a period before the entire training task comes to a halt. Minder has been deployed in a production environment for over nine months, monitoring daily distributed training tasks where each involves up to thousands of machines. In our real-world fault detection scenarios, Minder can accurately and efficiently react to faults within 3.6 seconds on average, with a precision of 0.904 and F1-score of 0.893.

2023

1. fKPISelect: Fault-Injection Based Automated KPI Selection for Practical Multivariate Anomaly Detection

   Xingjian Zhang ,  Yinqin Zhao ,  Chang Liu , and 9 more authors

   In 2023 IEEE 34th International Symposium on Software Reliability Engineering (ISSRE) , Oct 2023

   ISSN: 2332-6549

   Abs PDF Code Slides

   IT services are now popularly hosted in cloud systems. In order to enhance the availability of cloud services, an emerging approach for detecting failures of cloud components is to monitor Key Performance Indicators (KPIs) of the components and apply Neural Network based AI technologies to detect KPI anomalies. Multivariate Time Series Anomaly Detection (TSAD) models have been designed for this purpose. However, when applying such models directly to real-world cloud systems the anomaly detection performance is not as good. This is because the number of KPIs in real cloud systems is typically much more than the number of KPIs in the datasets used for model evaluation, and the larger number of KPIs bring about a performance loss of the models’ anomaly detection. Therefore, selecting KPIs properly is essential for applying multivariant KPI data for any practical anomaly detection. This paper studies this performance loss issue when TSAD models are applied onto real-world cloud systems, and proposes fKPISelect, a mechanism of automated KPI selection based on fault injection. We implemented fKPISelect, deployed it to a real cloud system, and created a real-world KPI dataset. We conducted extensive experiments, and the experimental results show the effectiveness and practicality of fKPISelect: it improves the F1 score of anomaly detection from 0.68 to 0.91 for real-world KPI data.

2022

1. Modeling Composition of Cloud Services with Complex Dependencies for Availability Assessment

   Xingjian Zhang ,  and  Long Wang

   In 2022 52nd Annual IEEE/IFIP International Conference on Dependable Systems and Networks - Supplemental Volume (DSN-S) , Jun 2022

   Abs PDF

   This paper presents a methodology to assess availability of cloud services through Bayesian network-based availability model. We propose a modeling technology to represent heterogeneous dependencies of cloud services in the methodology. We show that our modeling technology provides more expressive capability for complex dependencies than classic tools like RBD.

2. OAG-BERT: Towards a Unified Backbone Language Model for Academic Knowledge Services

   Xiao Liu ,  Da Yin ,  Jingnan Zheng , and 5 more authors

   In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining , Aug 2022
3. Parameter-Efficient Prompt Tuning Makes Generalized and Calibrated Neural Text Retrievers

   Weng Lam Tam ,  Xiao Liu ,  Kaixuan Ji , and 6 more authors

   Jul 2022

   arXiv:2207.07087 [cs]

   Abs

   Prompt tuning attempts to update few task-specific parameters in pre-trained models. It has achieved comparable performance to fine-tuning of the full parameter set on both language understanding and generation tasks. In this work, we study the problem of prompt tuning for neural text retrievers. We introduce parameter-efficient prompt tuning for text retrieval across in-domain, cross-domain, and cross-topic settings. Through an extensive analysis, we show that the strategy can mitigate the two issues – parameter-inefficiency and weak generalizability – faced by fine-tuning based retrieval methods. Notably, it can significantly improve the out-of-domain zero-shot generalization of the retrieval models. By updating only 0.1% of the model parameters, the prompt tuning strategy can help retrieval models achieve better generalization performance than traditional methods in which all parameters are updated. Finally, to facilitate research on retrievers’ cross-topic generalizability, we curate and release an academic retrieval dataset with 18K query-results pairs in 87 topics, making it the largest topic-specific one to date.