Sex determination in zebrafish by manual approaches according to current guidelines relies on human observation. These guidelines for sex recognition have proven to be subjective and highly labor-intensive. To address this problem, we present a methodology to automatically classify the phenotypic sex using two machine learning methods: Deep Convolutional Neural Networks (DCNNs) based on the whole fish appearance and Support Vector Machine (SVM) based on caudal fin coloration. Machine learning techniques in sex classification provide potential efficiency with the advantage of automatization and robustness in the prediction process. Furthermore, since developmental plasticity can be influenced by environmental conditions, we have investigated the impact of elevated water temperature during embryogenesis on sex and sex-related differences in color intensity of adult zebrafish. The estimated color intensity based on SVM was then applied to detect the association between coloration and body weight and length. Phenotypic sex classifications using machine learning methods resulted in a high degree of association with the real sex in nontreated animals. In temperature-induced animals, DCNNs reached a performance of 100textpercent, whereas 20textpercent of males were misclassified using SVM due to a lower color intensity. Furthermore, a positive association between color intensity and body weight and length was observed in males. Our study demonstrates that high ambient temperature leads to a lower color intensity in male animals and a positive association of male caudal fin coloration with body weight and length, which appears to play a significant role in sexual attraction. The software developed for sex classification in this study is readily applicable to other species with sex-linked visible phenotypic differences.