The RustNet can be used to detect the wheat stripe rust with images from drone, smartphone and videos. It was developed with a semi-automatic image labeling strategy which combines automatic labeling and human correction together. In labeling stage, we started with automatically labeling two hundred 100% non-disease images and two hundred 100% non-disease images. These images help to transfer the ResNet-18 that has been pre-trained with the ImageNet into the wheat stripe rust detection mission. We used the model of Stage 1 to predict dozens of images with almost all leaves infected or without infected leaves. Some labels were correct and some were not. Labels adjustment of expert were involved by under the help of ROOSTER . The Stage 1 version of RustNet was retrained with these new labeled images and got the Stage 2 version of RustNet. We continue use the Stage 2 version of RustNet to predict dozens of new partial infected images. More labels were corrected in this round of prediction than before. Experts adjustment was added again. The new images labeled were add into the images of Stage 2 and used to retrained RustNet and got the Stage 3 version of RustNet.