Designing optimized policies in large scale wireless networks is challenging due to unknown or time-varying dynamics. While wireless communication networks can be well-modeled by Markov Decision Processes (MDPs), this approach induces a large state space which challenges policy optimization.
Herein, we design new, graph signal representations for wireless network behavior. A novel on-line/in-real-time Q-learning methodology is proposed based on the new graph representation for wireless networks. The graph representations allow for the efficient creation of synthetic MDPs, or digital cousins, that accurately capture network behavior without the need for excessive trajectory sampling of the actual network.
Urbashi Mitra,
University of Southern California
The approach enables the ensemble learning of multiple policies which can be efficiently fused. The new learning method is distinctly different from more classical learning strategies that mix off-line collected data with real-time trajectory tracking.
The proposed mixed strategy offers significantly improved convergence rates and performance. We provide theoretical results on convergence as well as the ranking of the informativeness of the synthetic environments via learning analyses based on coverage. We provide preliminary results on a multi-agent version of the learning approach. Digital cousins can be applied to a multitude of graph learning problems and not just wireless.
Urbashi Mitra received her bachelors and master’s degrees from the University of California at Berkeley and her Ph.D. from Princeton University. Dr. Mitra is currently the Gordon S. Marshall Professor in Engineering at the University of Southern California with appointments in electrical engineering and computer science.
Dr. Mitra is a fellow of the IEEE and a foreign member of Academia Europaea. She is a co-principal investigator of the recently funded National Science Foundation Center on Pandemic Insights. Dr. Mitra was the inaugural editor-in-chief for the IEEE Transactions on Molecular, Biological and Multi-scale Communications.
Her honors include: the 2024 IEEE Information Theory Society’s Aaron D. Wyner Distinguished Service Award, the 2021 USC Viterbi School of Engineering Senior Research Award, the 2017 IEEE Communications Society Women in Communications Engineering Technical Achievement Award, a 2016 UK Royal Academy of Engineering Distinguished Visiting Professorship, a 2016 US Fulbright Scholar Award, 2012 Globecom Signal Processing for Communications Symposium Best Paper Award, 2012 US National Academy of Engineering Lillian Gilbreth Lectureship, Student Best Paper Award, as co-advisor, at the International Conference on Signal Processing and Communications, Bangalore India 2012, the 2009 DCOSS Applications & Systems Best Paper Award, 2001 Okawa Foundation Award, 2000 OSU College of Engineering Lumley Award for Research, 1997 OSU College of Engineering MacQuigg Award for Teaching, and a 1996 National Science Foundation CAREER Award.
Her research interests are in: model-based machine learning, wireless communications, communication and sensor networks, biological communication systems, and the interface of communication, sensing and control.