Reconstructing multiple modalities from each other, e.g., predicting the visual modality from proprioceptive input or vice versa, not only creates a correspondence between multiple modalities but also provides a representation that generalizes to out-of-distribution examples such as different grippers, colors, and objects, even though the system is only trained with a single instance of such properties.
Gokay, D., Simsar, E., Atici, E., Ahmetoglu, A., Yuksel, A.E., Yanardag, P. (2021).
Graph2Pix: A Graph-Based Image to Image Translation Framework.
In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops (pp. 2001-2010).
project page,
video,
code
An image translation framework from a graph of images.
Aksoy, C., Ahmetoglu, A., Gungor, T. (2020).
Hierarchical Multitask Learning Approach for BERT.
arXiv:2011.04451.
An analysis of BERT's downstream performance when trained with the auxiliary tasks at different layers.
Ahmetoglu, A., Alpaydin, E. (2020).
Hierarchical Mixture of Generators for Adversarial Learning.
In 2020 25th International Conference on Pattern Recognition (ICPR) (pp. 316-323).
project page,
video,
slides,
code
Training a hierarchical mixture of generators increases both the quality and coverage of generated samples in the context of GANs. The learned tree structure also allows for the interpretation of generators.
Ahmetoglu, A., Irsoy, O., Alpaydin, E. (2018).
Convolutional Soft Decision Trees.
In Proceedings of the 27th International Conference on Artificial Neural Networks (ICANN) (pp. 134-141).
slides
A differentiable combination of a deep net with (cooperative) hierarchical mixtures of experts slightly increases the classification performance and provides interpretability of decisions.
Theses