AI Workflow Optimization: From Data Labeling to HPC Resource Management

The lack of an operational environment for retraining and redeploying existing models results in continuous performance degradation. The process of labeling new image data and retraining is lengthy and inefficient. Furthermore, optimizing HPC (High Performance Computing) resources for training is crucial after this process.

The AI platform, Runway, addressed these challenges by offering an auto annotation tool to automate image data labeling and a continuous training (CT) environment to automate model retraining and redeployment. Additionally, a scheduler managed HPC resources and integrated with legacy systems to optimize resource utilization.

Automating the data labeling and retraining-redeployment cycle reduced process time by approximately 80% compared to traditional methods. The cycle of data labeling, model training, performance monitoring, relabeling misclassified data, retraining, and redeployment operated efficiently on the AI platform, maximizing resource utilization and enhancing overall productivity.