publications

2021

1. ICML

   Continuous-time Model-based Reinforcement Learning

   Yildiz, Cagatay, Heinonen, Markus, and Lahdesmaki, Harri

   In International Conference on Machine Learning 2021

   Abs PDF Code

   Model-based reinforcement learning (MBRL) approaches rely on discrete-time state transition models whereas physical systems and the vast majority of control tasks operate in continuous-time. To avoid time-discretization approximation of the underlying process, we propose a continuous-time MBRL framework based on a novel actor-critic method. Our approach also infers the unknown state evolution differentials with Bayesian neural ordinary differential equations (ODE) to account for epistemic uncertainty. We implement and test our method on a new ODE-RL suite that explicitly solves continuous-time control systems. Our experiments illustrate that the model is robust against irregular and noisy data, is sampleefficient, and can solve control problems which pose challenges to discrete-time MBRL methods.

2019

1. NeurIPS

   ODE2VAE: Deep generative second order ODEs with Bayesian neural networks

   Yildiz, Cagatay, Heinonen, Markus, and Lahdesmaki, Harri

   In Advances in Neural Information Processing Systems 2019

   Abs HTML PDF Supp Code Poster

   We present Ordinary Differential Equation Variational Auto-Encoder (ODE2VAE), a latent second order ODE model for high-dimensional sequential data. Leveraging the advances in deep generative models, ODE2VAE can simultaneously learn the embedding of high dimensional trajectories and infer arbitrarily complex continuous-time latent dynamics. Our model explicitly decomposes the latent space into momentum and position components and solves a second order ODE system, which is in contrast to recurrent neural network (RNN) based time series models and recently proposed black-box ODE techniques. In order to account for uncertainty, we propose probabilistic latent ODE dynamics parameterized by deep Bayesian neural networks. We demonstrate our approach on motion capture, image rotation, and bouncing balls datasets. We achieve state-of-the-art performance in long term motion prediction and imputation tasks.

2018

1. ICML

   Learning unknown ODE models with Gaussian processes

   Heinonen, Markus, Yildiz, Cagatay, Mannerström, Henrik, Intosalmi, Jukka, and Lähdesmäki, Harri

   In International Conference on Machine Learning 2018

   Abs HTML PDF Supp Code Poster

   In conventional ODE modelling coefficients of an equation driving the system state forward in time are estimated. However, for many complex systems it is practically impossible to determine the equations or interactions governing the underlying dynamics. In these settings, parametric ODE model cannot be formulated. Here, we overcome this issue by introducing a novel paradigm of nonparametric ODE modelling that can learn the underlying dynamics of arbitrary continuous-time systems without prior knowledge. We propose to learn non-linear, unknown differential functions from state observations using Gaussian process vector fields within the exact ODE formalism. We demonstrate the model’s capabilities to infer dynamics from sparse data and to simulate the system forward into future.

2. ICML

   Asynchronous Stochastic Quasi-Newton MCMC for Non-Convex Optimization

   Simsekli, Umut, Yildiz, Cagatay, Nguyen, Than Huy, Cemgil, Taylan, and Richard, Gael

   In International Conference on Machine Learning 2018

   Abs HTML PDF Supp Poster

   Recent studies have illustrated that stochastic gradient Markov Chain Monte Carlo techniques have a strong potential in non-convex optimization, where local and global convergence guarantees can be shown under certain conditions. By building up on this recent theory, in this study, we develop an asynchronous-parallel stochastic L-BFGS algorithm for non-convex optimization. The proposed algorithm is suitable for both distributed and shared-memory settings. We provide formal theoretical analysis and show that the proposed method achieves an ergodic convergence rate of O(1/\sqrtN) (N being the total number of iterations) and it can achieve a linear speedup under certain conditions. We perform several experiments on both synthetic and real datasets. The results support our theory and show that the proposed algorithm provides a significant speedup over the recently proposed synchronous distributed L-BFGS algorithm.

3. MLSP

   Learning Stochastic Differential Equations With Gaussian Processes Without Gradient Matching

   Yildiz, Cagatay, Heinonen, Markus, Intosalmi, Jukka, Mannerström, Henrik, and Lähdesmäki, Harri

   In IEEE 28th International Workshop on Machine Learning for Signal Processing 2018

   Abs HTML PDF Code

   We introduce a novel paradigm for learning non-parametric drift and diffusion functions for stochastic differential equation (SDE). The proposed model learns to simulate path distributions that match observations with non-uniform time increments and arbitrary sparseness, which is in contrast with gradient matching that does not optimize simulated responses. We formulate sensitivity equations for learning and demonstrate that our general stochastic distribution optimisation leads to robust and efficient learning of SDE systems.

4. Workshop

   A Nonparametric Spatio-temporal SDE Model

   Yildiz, Cagatay, Heinonen, Markus, and Lähdesmäki, Harri

   In NeurIPS Spatiotemporal Workshop 2018

   Abs PDF Poster

   We propose a nonparametric spatio-temporal stochastic differential equation (SDE) model that can learn the underlying dynamics of arbitrary continuous-time systems without prior knowledge. We augment the input space of the drift function of an SDE with a temporal component to account for spatio-temporal patterns. The experiments on a real world data set demonstrate that the spatio-temporal model is better able to fit the data than the spatial model and also reduce the forecasting error.

5. DSP

   A Bayesian change point model for detecting SIP-based DDoS attacks

   Kurt, Baris, Yildiz, Cagatay, Ceritli, Taha, Sankur, Bulent, and Cemgil, Taylan

   In Digital Signal Processing 2018

   Abs HTML

   Session Initiation Protocol (SIP), as one the most common signaling mechanism for Voice Over Internet Protocol (VoIP) applications, is a popular target for the flooding-based Distributed Denial of Service (DDoS) attacks. In this paper, we propose a DDoS attack detection framework based on the Bayesian multiple change model, which can detect different types of flooding attacks. Additionally, we propose a probabilistic SIP network simulation system that provides a test environment for network security tools.

6. SoftwareX

   A real-time SIP network simulation and monitoring system

   Yildiz, Cagatay, Kurt, Baris, Ceritli, Taha, Sankur, Bulent, and Cemgil, Taylan

   In SoftwareX 2018

   Abs HTML PDF Code

   In this work we present a real time SIP network simulation and monitoring system. The SIP network simulator is based on a probabilistic generative model that mimics a social network of VoIP subscribers calling each other at random times. The monitoring system, installed at a SIP server, provides services for collecting network data and server statistics in real time. The system provides a robust framework for developing SIP network applications such as security monitors.

2017

1. SIU

   A Bayesian change point model for epileptic seizure detection

   Yildiz, Cagatay, Bingol, Haluk, Irim-Celik, Gulcin, Aktekin, Berrin, and Aykut-Bingol, Canan

   In 25th Signal Processing and Communications Applications Conference 2017

   Abs HTML PDF

   Epilepsy is a chronic neurological disorder in which the normal pattern of neuronal activity in the brain becomes disturbed. Identification of the brain region that is abnormally active during an epileptic seizure is vital for epilepsy surgery. One way of achieving so is to collect electroencephalography (EEG) signals from epileptic people and then to identify the active region as a seizure occurs. In this work, we present a Bayesian change point model that detects when seizures occur. We applied our method to a data set that contains 48 EEG and electrocardiography (EKG) record pairs collected from epileptic people and observed that the model is able to detect all seizures.

2016

1. Workshop

   A Dictionary Learning Based Anomaly Detection Method for Network Traffic Data

   Ceritli, Taha, Kurt, Baris, Yildiz, Cagatay, Sankur, Bulent, and Cemgil, Taylan

   In ICML Anomaly Detection Workshop 2016

   Abs PDF Poster

   In this paper we propose a dictionary learning scheme to extract network traffic pattern templates for different types of anomalies and the normal traffic via nonnegative matrix factorization. We employ Bayesian change point models on the representation of the running network traffic in terms of those templates to detect network anomalies. Our proposed methods are tested and evaluated on a simulated SIP network with attacks generated by a commercial network vulnerability scanning tool.

2. SIU

   Attack Detection in VOIP Networks Using Bayesian Multiple Change-Point Models

   Yildiz, Cagatay, Ceritli, Taha, Kurt, Baris, Sankur, Bulent, and Cemgil, Taylan

   In IEEE Singal Processing and Communications Applications 2016

   Abs HTML PDF

   One of the most commonly used network protocols in Internet telephony services is SIP. As the popularity of the protocol increases, SIP networks have become targets of DDoS attacks more frequently. In this work, we propose a Bayesian change point model to detect anomalies due to such attacks. The model monitors the network and alarms when a change in the network traffic occurs. We test the model with a data set generated by network traffic and attack simulators.

3. SIU

   A Probabilistic SIP Network Simulation System

   Kurt, Baris, Yildiz, Cagatay, Ceritli, Taha, Yamac, Mehmet, Semerci, Murat, Sankur, Bulent, and Cemgil, Taylan

   In IEEE Singal Processing and Communications Applications 2016

   Abs PDF

   Experimenting with large-scale real world data is crucial for the development of network protocol and investigate their performance. However, collecting such data from real networks, and especially to annotate them with ground truth proves to, if not impossible, too tedious. In such cases use of simulated data, generated for various network scenarios, becomes a plausible alternative. To this purpose, we have developed a SIP network simulation system that employs probabilistic network model and we have demonstrated its use for the performance analysis of statistical (D)DOS attack detectors.