Çağatay Yıldız

profpic.png

AI Research Building

Maria-von-Linden-Str. 6

72076 Tuebingen, Germany

I’m a postdoctoral researcher at the Bethge Lab, University of Tuebingen. During my doctoral studies, I was supervised by Harri Lähdesmäki at Aalto University, Finland. Before that, I worked with Taylan Cemgil in my Master’s degree at Bogazici University, Istanbul.

Ongoing projects

  • Lie auto-encoders with Tolga and Matthias.
  • LLM domain adaptation with Firat and Beyza.
  • Continuous-time optimal control with David Leeftink and Steffen Ridderbusch.

MSc theses I currectly supervised

  • Realistic online continual learning (with Joschka Strüber).
  • Knowledge organization for continual question answering (with Atahan Özer).
  • Mechanistic understanding of forgetting in LLMs (with Nitin Sharma).

Looking for projects?

  • Linear algebraic compression of transformers for efficient inference.
  • Continual learning of transformers by low-rank reduction.
  • Personalized LLMs via RAG.
  • Continual handwriting recognition and its adaptation.

news

Jul 15, 2024 👨‍🏫 Our NeurIPS 2024 workshop proposal on Scalable Continual Learning for Lifelong Foundation Models has been accepted!
Jul 10, 2024 🏖️👨‍🏫💶 Our ML summer school to be organized in 2025 received funding from Centre International de Mathématiques Pures et Appliquées.
Jul 09, 2024 🎤 Talk on 6th Cluster Conference “Machine Learning in Science”. My slides are here.
Jun 15, 2024 📝 Two papers submitted to EMNLP 2024: Investigating Continual Pretraining in LLMs (stay tuned for the second preprint).
May 26, 2024 📝 Identifying latent state transition in non-linear dynamical systems paper is submitted to NeurIPS 2024!
May 20, 2024 📝 Infinite dSprites for disentangled continual learning is accepted to CoLLAs 2024!
Dec 05, 2023 📜 I co-organized the first Tübingen pre-NeurIPS event.
Oct 28, 2023 👨‍💻 Modulated neural ODEs paper accepted to NeurIPS 2023!
Jul 27, 2023 🎤 Organized a summer school on ML and mathematical foundations in Bilimler Köyü.
Feb 06, 2023 🎤 Talks on PCA/VAEs and diffusion models in Nesin Village.

latest posts

Oct 06, 2024 Peer reviewing in ML
Apr 14, 2024 Translanguaging
Mar 17, 2024 Sınırların ötesi

selected publications

  1. submitted
    Identifying latent state transition in non-linear dynamical systems
    Çağlar Hızlı ,  Çağatay Yıldız ,  Matthias Bethge , and 2 more authors
    In Advances in Neural Information Processing Systems , 2024
  2. submitted
    Investigating Continual Pretraining in Large Language Models: Insights and Implications
    Çağatay Yıldız ,  Nishaanth Kanna Ravichandran ,  Matthias Bethge , and 1 more author
    In Empirical Methods in Natural Language Processing , 2024
  3. CoLLAs
    Infinite dSprites for Disentangled Continual Learning: Separating Memory Edits from Generalization
    Sebastian Dziadzio ,  Çağatay Yıldız ,  Gido Ven , and 3 more authors
    In Lifelong Learning Agents , 2024
  4. NeurIPS
    Invariant Neural Ordinary Differential Equations
    Ilze Amanda Auzina ,  Çağatay Yıldız ,  Sara Magliacane , and 2 more authors
    In Advances in Neural Information Processing Systems , 2023
  5. ICLR
    Latent Neural ODEs with Sparse Bayesian Multiple Shooting
    Valerii Iakovlev ,  Çağatay Yıldız ,  Markus Heinonen , and 1 more author
    In International Conference on Learning Representations , 2023
  6. NeurIPS
    Learning Interacting Dynamical Systems with Latent Gaussian Process ODEs
    Çağatay Yıldız ,  Melih Kandemir ,  and  Barbara Rakitsch
    In Advances in Neural Information Processing Systems , 2022
  7. UAI
    Variational multiple shooting for Bayesian ODEs with Gaussian processes
    Pashupati Hedge ,  Çağatay Yıldız ,  Harri Lahdesmaki , and 2 more authors
    In Uncertainty in Artificial Intelligence , 2022
  8. thesis
    Differential Equations for Machine Learning
    Çağatay Yıldız
    2022
  9. ICML
    Continuous-time Model-based Reinforcement Learning
    Çağatay Yıldız ,  Markus Heinonen ,  and  Harri Lahdesmaki
    In International Conference on Machine Learning , 2021
  10. NeurIPS
    ODE2VAE: Deep generative second order ODEs with Bayesian neural networks
    Çağatay Yıldız ,  Markus Heinonen ,  and  Harri Lahdesmaki
    In Advances in Neural Information Processing Systems , 2019
  11. ICML
    Learning unknown ODE models with Gaussian processes
    Markus Heinonen ,  Çağatay Yıldız ,  Henrik Mannerström , and 2 more authors
    In International Conference on Machine Learning , 2018
  12. ICML
    Asynchronous Stochastic Quasi-Newton MCMC for Non-Convex Optimization
    Umut Simsekli ,  Çağatay Yıldız ,  Than Huy Nguyen , and 2 more authors
    In International Conference on Machine Learning , 2018