Research

My main research targets the study of fundamental performance limitations of networked control systems and the privacy aspects of such systems or other dynamical systems. The study of such complex systems can be characterized as interdisciplinary because it combines tools from control and estimation theories, communication and information theories, signal processing and optimization. My ongoing expertise span the areas of: i) information theory, ii) stochastic control and estimation theory, iii) signal processing, iv) convex optimization.



Properties of information measures

MapReduce
In this research, we study various functional and topological properties of information measures, such as, directed information, causally condition directed information or its variants. In information theory, directed information or its variants was primarily used to characterize the capacity of channels with feedback, some lossy compression problems and in network information theory. However, the last decade such information measures are widely used to characterize the fundamental performance limitations of delay constrained dynamical systems, like for instance, in networked control systems. This is because by definition, such measures can take into account the dynamics in systems where the components are modeled by (stochastic or deterministic) processes. The utility of directed information is not exhaustive to the previous research directions. Instead, it can be used in an anthology of problems like gambling, portfolio theory, or in biology as an alternative to Granger's measure of causality.

Selected Publications
Book Chapters:
[3] C. D. Charalambous, P. A. Stavrou, and C. K. Kourtellaris, "Directed information on abstract spaces: Properties and extremum Problems," Coordination Control of Distributed Systems (Edt. by Jan Van Schuppen and Tiziano Villa) in ser. Lecture Notes in Control and Information Sciences, Vol. 456, pp. 307-315, November 2014.
Full Text

Journal papers:
[2] C. D. Charalambous, and P. A. Stavrou, "Directed information on abstract spaces: properties and variational equalities," IEEE Transactions on Information Theory, vol. 62, Issue 11, pp. 6019-6052, November 2016.
Full Text

Conference papers:
[10] P. A. Stavrou and C. D. Charalambous, "Variational equalities of directed information and applications " in proceedings of IEEE International Symposium on Information Theory (ISIT), 2013.
Full Text
[7] C. D. Charalambous and P. A. Stavrou, "Directed information on abstract Alphabets: properties and extremum problems," in proceedings of IEEE International Symposium on Information Theory (ISIT), 2012.
Full Text


Source coding and rate distortion theory

In this research direction, we study generalizations of classical source coding and Shannon's rate distortion theory but we are not limited to such problems. On the contrary, we ask the question how source coding and rate distortion theory can be applied to delay-constrained dynamical systems. Our question is motivated by the utility need of the next generation systems to process information as quick as possible and as reliably as possible. Those standards are the epitomy of the next generation of engineering systems, such as, cyber-physical systems or networked control systems. Our study is focued in identifying the fundamental performance limitations of systems that operate using causal and/or instantaneous codes, and in understanding practical coding schemes that can support such frameworks.

Zero-delay source coding system
Fig. An example of zero-delay processing of a Gauss-Markov source using variable-length coding.

Selected Publications
Book Chapters:
[2] P. A. Stavrou, C. K. Kourtellaris, and C. D. Charalambous, "Information nonanticipative rate distortion function and its applications," Coordination Control of Distributed Systems (Edt. by Jan Van Schuppen and Tiziano Villa) in ser. Lecture Notes in Control and Information Sciences, Vol. 456, pp. 317-324, November 2014.
Full Text

Journal papers:
[6] P. A. Stavrou, T. Charalambous, and C. D. Charalambous, "Optimal estimation via nonanticipative rate distortion function and applications to time-varying Gauss-Markov processes," SIAM Journal on Control and Optimization, Vol. 56, Issue 5, pp. 3731-3765, 2018.
Full Text
[5] P. A. Stavrou, J. Østergaard, and C. D. Charalambous, "Zero-delay rate distortion via filtering for vector-valued Gaussian sources," IEEE Journal of Selected Topics in Signal Processing , Vol. 12, Issue 5, pp. 841-856, October 2018.
Full Text
[4] P. A. Stavrou, T. Charalambous and C. D. Charalambous, "Finite-time nonanticipative rate distortion function for time-varying scalar-valued Gauss-Markov sources," IEEE Control Systems Letters, Vol. 2, Issue 1, pp. 175-180, January 2018.
Full Text

Conference papers:
[24] H. Ghourchian, P. A. Stavrou, T. Oechtering, M. Skoglund, Block source coding with sequential encoding, in proceedings of IEEE Information Theory Workshop (ITW), 2019.
[22] P. A. Stavrou, T. Charalambous, C. D. Charalambous, S. Loyka and M. Skoglund, Asymptotic reverse-waterfilling solution of nonanticipative rate distortion function for vector-valued Gauss-Markov sources, in proceedings of IEEE 57th Conference on Decision and Control (CDC), 2018.
Full Text
[20] P. A. Stavrou, J. Østergaard, and M. Skoglund, "On zero-delay source coding of LTI Gauss-Markov systems with covariance matrix distortion constraints," in proceedings of European Control Conference (ECC), 2018.
Full Text
[19] P. A. Stavrou and J. Østergaard, "Fixed-rate zero-delay source coding for stationary vector-valued Gauss-Markov sources," in proceedings of Data Compression Conference (DCC), 2018.
Full Text
[18] P. A. Stavrou, J. Østergaard, C. D. Charalambous, and M. Derpich, "An upper bound to zero-delay rate distortion via Kalman filtering for Vector Gaussian sources," in proceedings of IEEE Information Theory Workshop (ITW), 2017.
Full Text
[16] P. A. Stavrou and J. Østergaard, "A lower bound on causal and zero-delay rate distortion for scalar Gaussian autoregressive sources," in proceedings of International Symposium on Information Theory and Signal Processing in Benelux (SITB), 2017.
Full Text
[12] P. A. Stavrou, C. K. Kourtellaris, and C. D. Charalambous, "Applications of information nonanticipative rate distortion function," in proceedings of IEEE International Symposium on Information Theory (ISIT), 2014.
Full Text


Joint source-channel coding

In this research direction, we study source-channel codes which combined can achieve optimal or near-optimal performance within a point-to-point system or a network. We are primarily interested in delay constrained dynamical systems although we are not limited to those. Our motivation is partly to understand if such coding paradigms can be applied to the next generation of engineering systems hence providing meaningful answers related to the performance of such systems. Another key aspect that triggers our research interest herein, is that the high level joint construction of a transmitter, the (real-world) channel and the receiver that model a communication system is shown to be very hard to achieve even for simple networks.

Realization
Fig. Pictorial view of the joint construction of an (encoder, AWGN channel, and MMSE decoder with causal side information) that model a scalar-valued Gaussian dynamical system.

Selected Publications
Book Chapters:
[4] C. D. Charalambous, C. K. Kourtellaris, and P. A. Stavrou, "On Shannon's duality of a source and a channel and nonanticipative communication and communication for control," Coordination Control of Distributed Systems (Edt. by Jan Van Schuppen and Tiziano Villa) in ser. Lecture Notes in Control and Information Sciences, Vol. 456, pp. 291-305, November 2014.
Full Text
[1] C. K. Kourtellaris, C. D. Charalambous, and P. A. Stavrou, "Nonanticipative duality of sources and channels with memory and feedback," Coordination Control of Distributed Systems (Edt. by Jan Van Schuppen and Tiziano Villa) in ser. Lecture Notes in Control and Information Sciences, Vol. 456, pp. 325-335, November 2014.
Full Text

Conference papers:
[12] P. A. Stavrou, C. K. Kourtellaris, and C. D. Charalambous, "Applications of information nonanticipative rate distortion function," in proceedings of IEEE International Symposium on Information Theory (ISIT), 2014.
Full Text


Feedback capacity of channels with memory

In this research direction, we are interested in computing the (ergodic) channel capacity of channels with memory and/or feedback. Ergodic feedback capacity is one of the fundamental characterizations in communication theory because it gives theoretical limitations on the least upper bound of allowable rate conveyed through a communication channel. However, characterizing the ergodic channel capacity of a communication system that is often expressed as an intractable optimization problem is not an easy task, let alone finding analytical expressions to such characterizations. In information theory, ergodic channel capacity is computed in closed form expressions for simple memoryless channels (defined either on finite alphabet or continuous alphabet spaces) For these simple channel models it is well-known that the presence of noiseless feedback often makes the coding much simpler but it cannot outperform the ergodic channel capacity without noiseless feedback. Unfortunately, this is not the case for channels with memory and, thus, to arrive to similar conclusions as those deduced for the class of memoryless channels, one has to characterize and then find analytical or computable expressions for specific classes of channels with memory. The previous challenging task is the motivation behind this study.

Feedback Capacity
Fig. Convergence of the binary erasure channel with memory to its feedback capacity after a finite number of transmissions.

Selected Publications
Journal papers:
[3] P. A. Stavrou, C. D. Charalambous and C. K. Kourtellaris, "Sequential necessary and sufficient conditions for capacity achieving distributions of channels with memory and feedback," IEEE Transactions on Information Theory, Vol. 63, Issue 11, pp. 7095-7115, November 2017.
Full Text

Conference papers:
[14] P. A. Stavrou, C. D. Charalambous, and C. K. Kourtellaris, "Sequential necessary and sufficient conditions for optimal channel input distributions of channels with memory and feedback," in proceedings of IEEE International Symposium on Information Theory (ISIT), 2016.
Full Text


Communication and Coding for Networked Control Systems

In this research direction, we are interested in identifying fundamental performance limitations and practical coding schemes for simple networked control systems, for instance, a closed loop control system where the information plant conveys information via a communication channel (noisy or noiseless) that in turn connects the observer to the controller. In the case where the controller is fully or weakly separated from the communication part of the closed loop system, we wish to design low-delay communication strategies which ensure that at the decoder of the communication system, the estimated process obtained based on an optimal minimum mean squared estimator (a la Kalman filter) satisfies an end-to-end average fidelity or distortion criterion.

Closed_loop
Fig. A scalar-valued closed-loop control system with causal side information at the decoder of the communication system.

Selected Publications
Book Chapters:
[4] C. D. Charalambous, C. K. Kourtellaris, and P. A. Stavrou, "On Shannon's duality of a source and a channel and nonanticipative communication and communication for control," Coordination Control of Distributed Systems (Edt. by Jan Van Schuppen and Tiziano Villa) in ser. Lecture Notes in Control and Information Sciences, Vol. 456, pp. 291-305, November 2014.
Full Text

Journals:
[6] P. A. Stavrou, T. Charalambous, and C. D. Charalambous, "Optimal estimation via nonanticipative rate distortion function and applications to time-varying Gauss-Markov processes," SIAM Journal on Control and Optimization, Vol. 56, Issue 5, pp. 3731-3765, 2018.
Full Text
[5] P. A. Stavrou, J. Østergaard, and C. D. Charalambous, "Zero-delay rate distortion via filtering for vector-valued Gaussian sources," IEEE Journal of Selected Topics in Signal Processing, Vol. 12, Issue 5, pp. 841-856, October 2018.
Full Text
[1] C. D. Charalambous, P. A. Stavrou, and N. U. Ahmed, "Nonanticipative rate distortion function and relations to filtering theory," IEEE Transactions on Automatic Control, Vol. 59, Issue 4, pp. 937-952, April 2014.
Full Text

Conference papers:
[22] P. A. Stavrou, T. Charalambous, C. D. Charalambous, S. Loyka and M. Skoglund, Asymptotic reverse-waterfilling solution of nonanticipative rate distortion function for vector-valued Gauss-Markov sources, in proceedings of IEEE 57th Conference on Decision and Control (CDC), 2018.
Full Text
[21] C. D. Charalambous, P. A. Stavrou, C. K. Kourtellaris and I. Tzortzis, "Directed Information subject to a fidelity: applications to conditionally Gaussian processes," in proceedings of European Control Conference (ECC), 2018.
Full Text
[17] M. Barforooshan, J. Østergaard, and P. A. Stavrou, "Achievable performance of zero-delay variable-rate coding in rate constrained networked control systems with channel delay," in proceedings of IEEE Conference on Decision and Control (CDC), 2017.
Full Text
[15] P. A. Stavrou, T. Charalambous, and C. D. Charalambous, "Filtering with fidelity for time-varying Gauss-Markov processes," in proceedings of IEEE 55th Conference on Decision and Control (CDC), 2016.
Full Text
[11] C. D. Charalambous and P. A. Stavrou, "Optimization of directed information and relations to filtering theory," in proceedings of 13th European Control Conference (ECC), 2014.
Full Text
[9] C. D. Charalambous and P. A. Stavrou, "On the relation of nonanticipative rate distortion function and filtering theory," in proceedings of 12th Biannual European Control Conference (ECC), 2013.
Full Text