Non-destructive in-season grain yield (GY) prediction would strongly facilitate the selection process in plant breeding but remains challenging for phenologically and morphologically diverse germplasm, notably under high-yielding conditions. In recent years, the application of drones (UAV) for spectral sensing has been established, but data acquisition and data processing have to be further improved with respect to efficiency and reliability. Therefore, this study evaluates the selection of measurement dates, sensors, and spectral parameters, as well as machine learning algorithms. Multispectral and RGB data were collected during all major growth stages in winter wheat trials and tested for GY prediction using six machine-learning algorithms. Trials were conducted in 2020 and 2021 in two locations in the southeast and eastern areas of Germany. In most cases, the milk ripeness stage was the most reliable growth stage for GY prediction from individual measurement dates, but the maximum prediction accuracies differed substantially between drought-affected trials in 2020 (R2 = 0.81 and R2 = 0.68 in both locations, respectively), and the wetter, pathogen-affected conditions in 2021 (R2 = 0.30 and R2 = 0.29). The combination of data from multiple dates improved the prediction (maximum R2 = 0.85, 0.81, 0.61, and 0.44 in the four-year*location combinations, respectively). Among the spectral parameters under investigation, the best RGB-based indices achieved similar predictions as the best multispectral indices, while the differences between algorithms were comparably small. However, support vector machine, together with random forest and gradient boosting machine, performed better than partial least squares, ridge, and multiple linear regression. The results indicate useful GY predictions in sparser canopies, whereas further improvements are required in dense canopies with counteracting effects of pathogens. Efforts for multiple measurements were more rewarding than enhanced spectral information (multispectral versus RGB).