Information Maximization Clustering via Multi-View Self-Labelling
Score-based diffusion models (SBDMs) have achieved the SOTA FID results in
unpaired image-to-image translation (I2I). However, we notice that existing
methods totally ignore the training data in the source domain, leading to
sub-optimal solutions for unpaired I2I. To this end, we propose energy-guided
stochastic differential equations (EGSDE) that employs an energy function
pretrained on both the source and target domains to guide the inference process
of a pretrained SDE for realistic and faithful unpaired I2I. Building upon two
feature extractors, we carefully design the energy function such that it
encourages the transferred image to preserve the domain-independent features
and discard domain-specific ones. Further, we provide an alternative
explanation of the EGSDE as a product of experts, where each of the three
experts (corresponding to the SDE and two feature extractors) solely
contributes to faithfulness or realism. Empirically, we compare EGSDE to a
large family of baselines on three widely-adopted unpaired I2I tasks under four
metrics. EGSDE not only consistently outperforms existing SBDMs-based methods
in almost all settings but also achieves the SOTA realism results without
harming the faithful performance. Furthermore, EGSDE allows for flexible
trade-offs between realism and faithfulness and we improve the realism results
further (e.g., FID of 51.04 in Cat to Dog and FID of 50.43 in Wild to Dog on
AFHQ) by tuning hyper-parameters. The code is available at
https://github.com/ML-GSAI/EGSDE.