Understanding Why Multimodal AI Learns More Robustly

Abstract A surge of recent advancements has consistently highlighted the superiority of multimodal learning over unimodal approaches across a variety of tasks. However, the theoretical foundations elucidating this advantage remain underexplored: existing theoretical analyzes are often constrained by tight assumptions, and lack empirical validation. In this paper, we bridge this gap by proposing a novel theoretical framework grounded in convolutional smoothing, offering a new perspective on how multimodal learning contributes to a smoother loss landscape compared to unimodal learning. Building upon this theoretical foundation, we introduce a simple yet effective distributional training strategy based on stochastic modality pairing instead of a fixed pairing; Thus, further promoting a flatter landscape via convolutional smoothing. Our empirical results across various multimodal datasets demonstrate that multimodal models not only achieve higher performance but also exhibit flatter loss landscape, which represent better generalization and robustness. Artificial intelligence models trained on combinations of images, speech, and text consistently outperform those that learn from a single type of data. While this advantage has been widely observed, the reason multimodal learning produces more accurate and reliable models has remained largely unexplained. Researchers at UNIST have now provided a mathematical explanation for this phenomenon and used it to develop a new training strategy that further improves multimodal learning. The study, led by Professor Sung Whan Yoon of the Graduate School of Artificial Intelligence, shows that learning from multiple data modalities produces a flatter loss landscape—a property associated with models that generalize more effectively to unfamiliar data and remain more resilient to noise and perturbations. The researchers demonstrate that this effect arises through a mathematical mechanism known as convolutional smoothing. By learning from different types of information simultaneously, multimodal models naturally smooth abrupt changes in the optimization process, creating a more stable learning landscape than models trained on a single modality. Building on this theoretical framework, the team developed Distributional Multimodal Learning (DML), a simple training strategy that replaces fixed image-text or image-audio pairs with randomly sampled combinations drawn from the same class. This broader variety of training examples further enhances the smoothing effect, leading to stronger generalization and more robust learning. Across multiple multimodal benchmark datasets, DML consistently outperformed conventional multimodal training. The method improved both classification accuracy and cross-modal retrieval tasks, including matching images with their corresponding text descriptions and retrieving images from textual queries. “Our work provides a theoretical foundation for understanding why multimodal learning consistently outperforms unimodal learning,” the research team said. “It also demonstrates how that understanding can be translated into a simple yet effective training strategy that improves both robustness and generalization.” The study's first author is Jaejun Lee, a researcher in the UNIST Graduate School of Artificial Intelligence. The work has been accepted for presentation at the International Conference on Machine Learning (ICML) 2026, one of the world's leading conferences on artificial intelligence, to be held in Seoul from July 6–11. The research was supported by the National Research Foundation of Korea (NRF) and the Institute for Information & Communications Technology Planning &Evaluation (IITP) through programs funded by the Ministry of Science and ICT. Journal Reference Jae-Jun Lee and Sung Whan Yoon, "Understanding Multimodal Learning: A Loss Landscape Smoothness Perspective," ICML '26 ., (2026).

2026.07.09

New Deep Learning Model Extends Reliable Wildfire Forecasts Beyond Two Weeks

Abstract The 2025 Los Angeles wildfires highlighted growing risks from climate-driven extremes and the need for reliable wildfire forecasting beyond short lead times. Although the European Center for Medium-Range Weather Forecasts provides global fire weather index forecasts, these products are primarily designed for long-range climate outlooks, and their practical effectiveness remains uncertain in regions with limited local forecasting infrastructure. Here we show that a global deep learning framework, forecasting daily fire weather index values up to 31 days ahead, consistently improves forecast accuracy and reduces bias relative to operational numerical forecasts. The framework learns nonlinear and lagged fire–weather relationships by integrating past fire weather index dynamics and future meteorological conditions. Importantly, forecast bias is reduced in 85% of grid cells where high wildfire exposure coincides with high socioeconomic vulnerability. This demonstrates that data-driven forecasting can help bridge critical information gaps in underserved regions and support more equitable climate risk management. Reliable wildfire forecasts become increasingly difficult beyond the first few weeks, limiting their value for medium-range decision-making. A research team, led by Professor Jungho of the Department of Civil, Urban, Earth, and Environmental Engineering at UNIST, has shown that a global deep learning framework can extend reliable daily wildfire forecasts to 31 days while reducing forecast bias compared with existing operational forecasting systems. Known as FWI-Net, the framework predicts daily values of the Fire Weather Index (FWI), a widely used measure of wildfire danger. Unlike conventional forecasting methods, it combines historical fire-weather conditions with future meteorological forecasts, allowing it to capture the cumulative effects of prolonged heat and drought on future wildfire risk. Across the full 31-day forecasting period, FWI-Net reduced prediction error by 6.6% compared with the operational forecasting system of the European Center for Medium-Range Weather Forecasts (ECMWF). During the first week, prediction error decreased by 12.4%. The model also extended the period of meaningful forecasts under very high wildfire danger by five days. The model performed particularly well in regions where high wildfire exposure coincides with high socioeconomic vulnerability. Forecast bias was reduced across 85% of these areas, and in regions with limited forecasting infrastructure, FWI-Net maintained useful forecasting skill for an average of 22 days. “By combining historical fire-weather conditions with future weather forecasts, the model captures patterns that conventional forecasting systems often miss,” the research team said. “Most importantly, it provides more reliable forecasts for regions facing high wildfire risk despite limited forecasting infrastructure.” “As climate change increases wildfire risk around the world, reliable forecasting is becoming essential for disaster preparedness,” said Professor Im. “We expect this framework to support medium-range wildfire planning while helping reduce information gaps in regions with limited forecasting capacity.” The study was co-led by Professor Yoojin Kang of Kookmin University and Sihyun Lee of UNIST, who served as first authors. The findings were published online in Communications Earth & Environment on June 28, 2026. The research was supported by the Ministry of Environment (ME), the Korea Forest Service, and the National Research Foundation of Korea (NRF). Journal Reference Yoojin Kang, Sihyun Lee, Dongjin Cho, and Jungho Im, "Deep learning-based forecasting provides a pathway to closing wildfire information gaps in underserved regions," Commun. Earth Environment. , (2026).

2026.07.08

New Training Method Improves Knowledge Distillation for Compact Generative AI

Abstract We demonstrate that in knowledge distillation for diffusion models, the teacher network's highly complex denoising process—stemming from its substantially larger capacity—poses a significant challenge for the student model to faithfully mimic. To address this problem, we propose a coarse-to-fine distillation framework with LInear Fitting-based distillation (LIFT) and Piecewise Local Adaptive Coefficient Estimation (PLACE). First, LIFT decomposes the objective into a coarse'' alignment and afine'' refinement. The student is then trained on coarse alignment before proceeding to hard refinement. Second, LInear Fitting-based distillation extends LIFT to address spatially non-uniform errors by partitioning outputs into error-based groups, providing locally adaptive guidance. Our comprehensive experimental results demonstrate that ours, \core~with \pick, outperforms previous knowledge distillation on diffusion models based on both U-Net and DiT architectures. Furthermore, as compression rates become exceedingly high, conventional knowledge distillation fails to provide sufficient guidance, thereby preventing lightweight diffusion models from achieving stable training. In contrast, our method demonstrates stable convergence even under such extreme compression ratios. Compact generative AI models promise to bring image generation to personal devices, but reducing model size often comes at the expense of image quality and training stability. Researchers at UNIST have developed a new knowledge distillation framework that enables lightweight diffusion models to learn more effectively from much larger models without increasing inference costs. Led by Professor Jaejun Yoo of the Graduate School of Artificial Intelligence, the study addresses a key challenge in knowledge distillation—the process of transferring the capabilities of a large AI model to a smaller one. The researchers found that as teacher models become increasingly capable, the complexity of their denoising process makes it progressively more difficult for compact student models to reproduce faithfully. Rather than asking the student model to reproduce every detail from the outset, the framework restructures learning into successive stages, allowing it to first capture an image's overall structure before progressively refining finer details. The approach further improves learning by providing additional guidance where prediction errors are greatest while adapting supervision throughout training. This strategy is implemented through two complementary techniques, LIFT (LInear Fitting-based Distillation) and PLACE (Piecewise Local Adaptive Coefficient Estimation). The approach changes only the training strategy, requiring neither additional model parameters nor extra computation during image generation. Across multiple image-generation benchmarks, the proposed framework consistently outperformed existing knowledge distillation methods. It proved effective for text-to-image synthesis, class-conditional generation, and unconditional image generation, while remaining compatible with multiple diffusion architectures, including U-Net, DiT, and MMDiT, the architecture used in Stable Diffusion 3. The advantages became particularly evident under extreme model compression, where the widening gap between teacher and student models often leaves conventional knowledge distillation unable to provide sufficient guidance for stable learning. When compressing a teacher model containing 78.7 million parameters into a student model with just 1.3 million parameters, the proposed method maintained stable convergence and generated substantially higher-quality images. “Rather than modifying model architectures, our approach improves how compact models learn from larger ones,” said Professor Yoo. “Because it introduces no additional model parameters or inference-time computation, it can be readily applied across a wide range of generative AI models, making compact generative AI practical across a wide range of computing environments.” Hyunsoo Han and Sangyeob Yeo of the Graduate School of Artificial Intelligence at UNIST contributed to the study. The findings have been accepted for presentation at the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2026. Journal Reference Hyunsoo Han, Sangyeop Yeo, and Jaejun Yoo, "LIFT and PLACE: A Simple, Stable, and Effective Knowledge Distillation Framework for Lightweight Diffusion Models," CVPR'26

2026.07.07

New Reinforcement Learning Framework Makes Cooperative AI More Resilient

Abstract Cooperation is central to multi-agent reinforcement learning (MARL), yet learned coordination can be fragile when external perturbations disrupt inter-agent interactions. Prior robust MARL methods have primarily considered value-oriented attacks, leaving a gap in robustness when interaction structures themselves are corrupted. In this paper, we propose an interaction-breaking adversarial learning (IBAL) framework that takes an information-theoretic view to construct attacks that impede coordination by perturbing agents' observations and actions, and trains agents to perform reliably under such disruptions. Empirically, our approach improves robustness over existing robust MARL baselines across diverse attack settings and yields stronger performance even under agent-missing scenarios. From autonomous drone swarms to industrial robots and self-driving vehicles, many AI systems depend on multiple agents working together. Yet when communication fails or individual agents become unavailable, coordination can quickly break down. Researchers at UNIST have developed a new reinforcement learning framework that trains AI agents to adapt to these disruptions, enabling them to continue working together under conditions that would typically cause conventional systems to fail. Led by Professor Seungyul Han of the Graduate School of Artificial Intelligence, the research team developed Interaction-Breaking Adversarial Learning (IBAL) , a multi-agent reinforcement learning framework that deliberately disrupts coordination during training. Rather than exposing agents only to ideal conditions, IBAL repeatedly simulates communication failures and missing teammates, encouraging the remaining agents to reorganize their roles and develop new cooperative strategies. Unlike previous robust reinforcement learning methods, which primarily strengthen individual agents against noisy observations or unfavorable actions, IBAL focuses on the interactions between agents. During training, the framework identifies information that is essential for coordination and selectively disrupts it, preparing AI systems to maintain cooperation even when communication becomes unreliable. "Existing robust multi-agent reinforcement learning methods mainly interfere with individual agents," said Sunwoo Lee, the study's first author. “IBAL instead challenges the interactions that enable cooperation, allowing the remaining agents to discover new ways of working together when communication is interrupted or some agents fail.” The researchers evaluated the proposed IBAL on the StarCraft II Multi-Agent Challenge (SMAC). In scenarios where some allied units suddenly became unavailable, conventional models often experienced cascading failures, with some achieving win rates as low as 13.3%. By contrast, agents trained with IBAL adapted their strategies in real time—reorganizing team formations and redistributing roles—and achieved a win rate of 87.0%. “Many real-world AI systems rely on multiple agents working together, making them vulnerable to equipment failures or communication loss,” said Professor Han. “Our framework trains AI agents to adapt to these unexpected disruptions and continue accomplishing their task, providing a foundation for safer and more reliable multi-agent systems.” The study has been accepted for presentation at the International Conference on Machine Learning (ICML) 2026, which will be held in Seoul from July 6 to 11, 2026. The research was supported by the Ministry of Science and ICT (MSIT), the Institute for Information & Communications Technology Planning & Evaluation (IITP), and the National Research Foundation of Korea (NRF). Journal Reference Sunwoo Lee, Mingu Kang, Yonghyeon Jo, and Seungyul Han, "Interaction-Breaking Adversarial Learning Framework for Robust Multi-Agent Reinforcement Learning," ICML '26 , (2026).

2026.07.03

New Semiconductor Device Brings Communication and AI Computing onto a Single Chip

Abstract Next-generation 6G communication systems necessitate hardware that simultaneously manages high-frequency signal routing and massive matrix-level computations under extreme constraints. However, conventional digital architectures suffer from high power dissipation and area overhead. Here, we report an oxidized molybdenum disulfide (MoS2)-based memristive platform that integrates nonvolatile radio-frequency (RF) switching and energy-efficient vector–matrix multiplication. By implementing a controlled thermal oxidation process, we achieve stable resistive switching with low switching energy and zero-static power consumption, while maintaining a high cutoff frequency of 33.2 THz, outperforming existing phase-change and microelectromechanical systems (MEMS) technologies. Using system-level simulations, we demonstrate the robustness of this hardware through successful 1024-quadrature amplitude modulation (1024-QAM) demodulation, spectral analysis, and multiple-input multiple-output (MIMO) signal reconstruction, even when accounting for intrinsic device non-idealities. This monolithic integration of high-frequency switching and analog computing provides a scalable solution for energy-efficient deployment of intelligent wireless systems. Future 6G networks and satellite communication systems will require hardware that can process growing amounts of data while operating under strict power and space constraints. Professor Myungsoo Kim and his research team from the Department of Electrical Engineering at UNIST introduced a new semiconductor device that combines high-frequency signal switching and computation on a single platform, offering a more compact and energy-efficient approach to next-generation wireless systems. At the heart of the platform is a multifunctional memristor based on oxidized molybdenum disulfide (MoS2), a two-dimensional semiconductor. The device functions as both a radio-frequency (RF) switch and hardware for in-memory matrix operations, bringing communication and computation onto a single chip. Conventional communication systems typically separate these functions, requiring signals to move between switching circuits and processors. This data movement increases power consumption, processing delays, and chip size. By integrating both functions into a single device, the new platform simplifies signal processing while improving energy efficiency. The memristor also operates without standby power. Once switched between its ON and OFF states, it retains that state even after power is removed, making it particularly well suited to energy-constrained applications such as satellite communications and future 6G infrastructure. The device demonstrated a switching energy of just 140 picojoules per operation and switching power below one milliwatt. It maintained stable operation for more than 40,000 seconds and over 1,000 switching cycles, while achieving a calculated cutoff frequency of 33.2 THz. Experimental measurements verified high-frequency switching up to 67 GHz. To evaluate its computing capability, the researchers performed system-level simulations of wireless communication tasks. The device successfully carried out 1024-quadrature amplitude modulation (1024-QAM) demodulation and multiple-input multiple-output (MIMO) signal reconstruction, demonstrating that a single hardware platform can efficiently support both communication and signal processing. "Commercial RF switches continuously consume standby power and often suffer signal loss at high frequencies," said Juhyo Son, the study's first author. "Our device addresses these limitations while remaining competitive with existing phase-change memory and MEMS technologies in energy efficiency, operating speed, and scalability." Professor Kim added, "Our study demonstrates that oxidized MoS₂ memristors can function not only as high-performance RF switches but also as hardware for the in-memory matrix operations required for next-generation wireless communications. By combining nonvolatile operation, low power consumption, and high-frequency performance, this platform could contribute to more compact and energy-efficient systems for satellite communications, radar, defense electronics, and future 6G RF front ends." The research was supported by the National Research Foundation of Korea (NRF) through the Ministry of Science and ICT (MSIT), the Institute for Information & Communications Technology Planning & Evaluation (IITP), the Space-K BIG Project, and a regional talent development program. The findings were published online in Advanced Functional Materials on June 6, 2026. Journal Reference Juho Son, Changwoo Pyo, Sungmoon Park, et al., “ Oxidized MoS2-Based Multifunctional Memristive Hardware for Energy-Efficient mmWave Signal Processing and In-Memory Matrix Multiplication ,” Adv. Funct. Mater., (2026).

2026.07.02

Beyond Algorithms

《Editor's Note: Scientific discovery has always depended on our ability to observe and understand the world. Increasingly, however, many of science's most pressing questions—from interpreting human behavior and dynamic environments to advancing healthcare and understanding Earth's changing climate—have grown too complex for conventional approaches alone. Across disciplines, UNIST researchers are using artificial intelligence to reveal patterns, improve prediction, and open new pathways for discovery. The latest work—from enabling machines to understand human emotion and perceive the physical world to revealing new insights into cities and climate—reflects a broader shift in science: AI is no longer simply a field of research. It is becoming part of how research itself is done.》 From Data to Insight AI is becoming increasingly capable of interpreting the world with greater nuance and context. At UNIST, researchers are advancing AI's ability to understand both people and the physical world through innovations in computer vision and multimodal learning. One research team developed an AI framework that translates emotional cues from speech into realistic facial expressions, enabling more natural human–AI interaction. Another explored how robots can better interpret social touch and express emotions through movement, making human–robot interaction feel more intuitive and lifelike. Researchers also introduced a new camera calibration method that significantly improves how machines perceive dynamic environments. Together, these studies show how AI is moving beyond recognizing data toward interpreting people, interaction, and the physical world. ■ AI Turns Speech into Natural Facial Expressions (CVPR 2026) ■ Making Human-Robot Interaction More Natural (ICRA 2026) ■ AI Advances Event Camera Calibration (CVPR Highlight 2026) From Insight to Discovery AI is emerging as an essential research tool, helping scientists uncover patterns and relationships that would be difficult to detect through conventional methods alone. Rather than replacing scientific expertise, AI is expanding how researchers investigate complex systems across disciplines. At UNIST, researchers developed LightSplat, enabling AI to recognize objects within complex 3D scenes from natural language descriptions in just five seconds. Another team combined deep learning with hyperspectral satellite imagery to detect methane emissions faster and more accurately, improving how researchers monitor one of the world's most potent greenhouse gases. Together, these studies illustrate how AI is accelerating scientific discovery across both digital and environmental systems. ■ AI Recognizes 3D Objects in Five Seconds (CVPR 2026) ■ AI Detects Methane from Space (npj Clim. Atmos. Sci. l Mar. 2026) Across disciplines, AI is becoming more than a field of study. It is emerging as a common language for discovery—expanding not only what researchers can discover, but the questions they are able to ask.

2026.07.01