Often, in practice, the data distribution at test time differs, to a smaller or larger extent, from that of the original training data. Consequentially, the so-called source classifier, trained on the labeled data, deteriorates on the test, or target data. Domain adaptive classifiers aim to alleviate this problem, but typically assume a particular type of domain shift. Most are not robust to violations of domain shift assumptions and may perform even worse than the non-adaptive source classifier. We construct robust parameter estimators for discriminant analysis that guarantee performance improvements of the adaptive classifier over the source classifier.