Richard Cornelius Suwandi

Open to chat about research ideas and potential collaborations

I’m a fully-funded PhD student at School of Artificial Intelligence, CUHK-Shenzhen, advised by Prof. Feng Yin and Prof. Tsung-Hui Chang. Prior to my PhD, I obtained my BSc degree in Statistics (with first-class honors) from CUHK-Shenzhen.

What I’m currently working on:

Adaptive intelligence for sequential decision-making and optimization
PlugBO, a modular framework that lets an agent adapt the optimization configuration on the fly
CAKE, an LLM-driven evolutionary framework for adaptively evolving kernel functions
OpenEvolve, an evolutionary coding agent for discovering and optimizing algorithms
Kai, an autonomous AI engineer for codebase security and optimization
Awesome Bayesian Optimization, a curated repo of Bayesian optimization resources

Some other things worth mentioning:

I co-founded the Institute for AI-Driven Discovery of Algorithms (AIDDA)
I am dev ambassador at Qwen (Alibaba Cloud)
I am a recipient of the IEEE Signal Processing Society Scholarship and the Guangdong Government Outstanding International Student Scholarship
I also received research funding from the Shenzhen Universiade International Scholarship Foundation
I was previously a community leader for the AI4Science community at alphaXiv

Research

2026

Abstract Paper alphaXiv Code Thread

GRAPE: Gradient Refinement and Progress-Aware Exploitation for Query-Efficient High-Dimensional Bayesian Optimization

Richard Cornelius Suwandi, Feng Yin

PREPRINT 2026

TL;DR

Local Bayesian optimization often takes tiny steps that are almost sure to descend. GRAPE first sharpens the local gradient, then moves in the direction with the largest expected decrease, giving a 5.4x speedup on adversarial attacks and 3.8 log-units better regret on LLM prompt optimization.

Abstract

Optimizing expensive, high-dimensional black-box functions remains a central challenge in modern machine learning and scientific discovery. While local Bayesian optimization mitigates the curse of dimensionality, existing techniques often prioritize the probability of descent over the magnitude of progress. This leads to overly conservative steps that yield negligible improvement, wasting queries on directions that are nearly certain to descend but offer little decrease. We introduce Gradient Refinement and Progress-Aware Exploitation (GRAPE), a two-stage framework that first sharpens the local gradient posterior via a closed-form acquisition function, then selects update directions by maximizing the expected decrease conditional on descent. Theoretical analysis proves that this gradient refinement stage monotonically minimizes local uncertainty and that the progress-aware direction converges to true steepest descent as the posterior sharpens. Empirically, GRAPE demonstrates superior query efficiency across high-dimensional tasks: in black-box adversarial attacks, it achieves an average 5.4× speedup over baselines, and on large language model prompt optimization tasks, it outperforms the second best method by a reduction of 3.8 log-units in the final average regret.
2026

Abstract Paper alphaXiv

MIMOMamba: From Scalar Duality to Matrix-Valued Attention

Yanbo Li, Richard Cornelius Suwandi, Feng Yin, 3 more authors Yiyong Sun, Wei Huang, Wenqiang Pu

43rd International Conference on Machine Learning (ICML), 2026

TL;DR

Mamba’s state-space duality is limited to single-input systems, so dimensions cannot mix inside the state. MIMOMamba extends it to MIMO with a matrix polynomial, matching or beating Transformers at about one-third the parameters.

Abstract

The state space duality (SSD) framework, central to modern state-space models (SSMs) such as Mamba, has established an efficient attention-like mechanism by leveraging the commutative property of linear recurrences. However, existing formulations are limited to single-input single-output (SISO) systems that enforce commutativity with a restrictive scalar-identity constraint, which prevents cross-dimensional interactions within the state dynamics. In this work, we generalize SSD to the multi-input multi-output (MIMO) setting by introducing a matrix polynomial parameterization. This approach not only provides a principled way to ensure commutativity for generalized duality but also induces a shared algebraic structure across state transitions, thereby significantly reducing parameter redundancy. Building on this foundation, we present MIMOMamba, a multi-head SSM architecture that captures rich cross-dimensional dynamics while retaining linear-time training. Empirical results on a sequence modeling benchmark show that MIMOMamba matches or exceeds the performance of standard Transformers with only approximately one-third the parameters of the baseline.
2026

Abstract Paper alphaXiv Code

Breaking the Curse of Dimensionality in Gaussian Process Training With Zeroth-Order Adaptive Perturbation

Richard Cornelius Suwandi, Feng Yin, Tsung-Hui Chang

ORAL 51th IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2026

TL;DR

Training a Gaussian process gets expensive as the number of hyperparameters grows. ZAP estimates the full gradient from two loss evaluations, independent of dimension, with about 20x lower error and 118x less optimization time.

Abstract

Hyperparameter optimization remains a core challenge in training Gaussian processes (GPs), primarily due to the high computational cost and numerical instabilities associated with gradient-based optimizers. These issues are exacerbated in high-dimensional spaces, where the optimization landscapes become increasingly complex and difficult to navigate efficiently. We propose zeroth-order adaptive perturbation (ZAP), a scalable gradient-free algorithm that leverages simultaneous perturbations sampled from a Bernoulli distribution to obtain accurate gradient estimates with only two function evaluations per iteration, regardless of the hyperparameter dimensionality. Our theoretical analysis establishes that the gradient estimator is asymptotically unbiased and that ZAP converges to a stationary point under standard assumptions. Empirically, ZAP outperforms state-of-the-art gradient-based and gradient-free baselines on real-world datasets, achieving an average of 20 times reduction in mean-squared-error and a 118 times speedup in optimization time.
2025

Abstract Paper alphaXiv Code Thread

Adaptive Kernel Design for Bayesian Optimization Is a Piece of CAKE with LLMs

Richard Cornelius Suwandi, Feng Yin, Juntao Wang, 3 more authors

39th Conference on Neural Information Processing Systems (NeurIPS), 2025

TL;DR

A poorly chosen GP kernel slows Bayesian optimization. CAKE uses an LLM to evolve kernels during the search, and BAKER picks which one to query next, improving hyperparameter optimization, controller tuning, and photonic chip design.

Abstract

The efficiency of Bayesian optimization (BO) relies heavily on the choice of the Gaussian process (GP) kernel, which plays a central role in balancing exploration and exploitation under limited evaluation budgets. Traditional BO methods often rely on fixed or heuristic kernel selection strategies, which can result in slow convergence or suboptimal solutions when the chosen kernel is poorly suited to the underlying objective function. To address this limitation, we propose a freshly-baked Context-Aware Kernel Evolution (CAKE) to enhance BO with large language models (LLMs). Concretely, CAKE leverages LLMs as the crossover and mutation operators to adaptively generate and refine GP kernels based on the observed data throughout the optimization process. To maximize the power of CAKE, we further propose BIC-Acquisition Kernel Ranking (BAKER) to select the most effective kernel through balancing the model fit measured by the Bayesian information criterion (BIC) with the expected improvement at each iteration of BO. Extensive experiments demonstrate that our fresh CAKE-based BO method consistently outperforms established baselines across a range of real-world tasks, including hyperparameter optimization, controller tuning, and photonic chip design. Our code is publicly available at https://github.com/richardcsuwandi/cake.
2025

Abstract Paper alphaXiv Code

Sparsity-Aware Distributed Learning for Gaussian Processes with Linear Multiple Kernel

Richard Cornelius Suwandi, Zhidi Lin, Feng Yin, 2 more authors Zhiguo Wang, Sergios Theodoridis

IEEE Transactions on Neural Networks and Learning Systems, 2025

TL;DR

Hyperparameters of the GSM kernel come out sparse. SLIM-KL uses that sparsity, plus quantized ADMM, so many agents can train a large GP kernel privately and with less communication.

Abstract

Gaussian processes (GPs) stand as crucial tools in machine learning and signal processing, with their effectiveness hinging on kernel design and hyper-parameter optimization. This paper presents a novel GP linear multiple kernel (LMK) and a generic sparsity-aware distributed learning framework to optimize the hyper-parameters. The newly proposed grid spectral mixture (GSM) kernel is tailored for multi-dimensional data, effectively reducing the number of hyper-parameters while maintaining good approximation capabilities. We further demonstrate that the associated hyper-parameter optimization of this kernel yields sparse solutions. To exploit the inherent sparsity property of the solutions, we introduce the Sparse LInear Multiple Kernel Learning (SLIM-KL) framework. The framework incorporates a quantized alternating direction method of multipliers (ADMM) scheme for collaborative learning among multiple agents, where the local optimization problem is solved using a distributed successive convex approximation (DSCA) algorithm. SLIM-KL effectively manages large-scale hyper-parameter optimization for the proposed kernel, simultaneously ensuring data privacy and minimizing communication costs. Theoretical analysis establishes convergence guarantees for the learning framework, while experiments on diverse datasets demonstrate the superior prediction performance and efficiency of our proposed methods.
2022

Abstract Paper alphaXiv Code

Gaussian Process Regression with Grid Spectral Mixture Kernel: Distributed Learning for Multidimensional Data

Richard Cornelius Suwandi, Zhidi Lin, Yiyong Sun, 3 more authors Zhiguo Wang, Lei Cheng, Feng Yin

25th International Conference on Information Fusion (FUSION), 2022

TL;DR

We introduce a grid spectral mixture kernel for multidimensional Gaussian processes and train it with distributed ADMM, so agents fit the kernel without sharing raw data.

Abstract

Kernel design for Gaussian processes (GPs) along with the associated hyper-parameter optimization is a challenging problem. In this paper, we propose a novel grid spectral mixture (GSM) kernel design for GPs that can automatically fit multidimensional data with affordable model complexity and superior modeling capability. To alleviate the computational complexity due to the curse of dimensionality, we leverage a multicore computing environment to optimize the kernel hyper-parameters in a distributed manner. We further propose a doubly distributed learning algorithm based on the alternating direction method of multipliers (ADMM) which enables multiple agents to learn the kernel hyper-parameters collaboratively. The doubly distributed learning algorithm is shown to be effective in reducing the overall computational complexity while preserving data privacy during the learning process. Experiments on various one-dimensional and multidimensional data sets demonstrate that the proposed kernel design yields superior training and prediction performance compared to its competitors.
2021

Abstract Paper alphaXiv

Demystifying Model Averaging for Communication-Efficient Federated Matrix Factorization

Shuai Wang, Richard Cornelius Suwandi, Tsung-Hui Chang

46th IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2021

TL;DR

Model averaging had not been applied to federated matrix factorization. FedMAvg combines alternating minimization with averaging, and communicating less often with fewer clients cuts cost on heterogeneous data.

Abstract

Federated learning (FL) is encountered with the challenge of training a model in massive and heterogeneous networks. Model averaging (MA) has become a popular FL paradigm where parallel (stochastic) gradient descent (GD) is run on a small sampled subset of clients multiple times before uploading the local models to a server for averaging, which has been proven effective in reducing the communication cost for achieving a good model. However, MA has not been considered for the important matrix factorization (MF) model, which has vast signal processing and machine learning applications. In this paper, we investigate the federated MF problem and propose a new MA based algorithm, named FedMAvg, by judiciously combining the alternating minimization technique and MA. Through analysis, we show that gradually decreasing the number of local GD and only allowing partial clients to communicate with the server can greatly reduce the communication cost, especially in heterogeneous networks with non-i.i.d. data. Experimental results by applying FedMAvg to data clustering and item recommendation tasks demonstrate its efficacy in terms of both task performance and communication efficiency.

Projects

algorithmicsuperintelligence/openevolve

Open-source implementation of AlphaEvolve

Python 7.3k 1.1k

richardcsuwandi/sift

Find, understand, and organize your files with private local AI

Python 4 1

richardcsuwandi/cake

[NeurIPS 2025] Context-Aware Kernel Evolution (CAKE)

Python 32 4

richardcsuwandi/plugbo

A modular framework for agentic Bayesian optimization

Python 6 1

richardcsuwandi/awesome-bo

Curated books, papers, and software for Bayesian optimization

20 5

richardcsuwandi/zap

[ICASSP 2026] Zeroth-order adaptive perturbation for scalable Gaussian process training

Python 2 1

richardcsuwandi/grape

A two-stage local method for query-efficient high-dimensional Bayesian optimization

Python 2 1

pycaret/pycaret

Open-source, low-code AutoML platform for Python

Python 9.8k 1.8k

anyoptimization/pymoo

Multi-objective optimization algorithms in Python

Python 2.9k 478

feature-engine/feature_engine

Feature engineering and selection library for scikit-learn

Python 2.3k 371

Latest Blog Posts

Aug 20, 2026

PlugBO: A Modular Framework for Agentic Bayesian Optimization

Aug 12, 2026

World Models for Scientific Discovery

Mar 24, 2026

Metacognitive Self-Modification in Self-Improving Agents

News

Aug 28, 2026

Invited to serve as a reviewer for ICLR 2027

Aug 10, 2026

Joined Qwen as a dev ambassador

Jun 09, 2026

Co-organized AIDDA 2026, a two-day virtual technical conference focusing on AI-driven algorithm discovery. Recordings of the talks are available on our YouTube channel

May 01, 2026

Our paper titled “MIMOMamba: From Scalar Duality to Matrix-Valued Attention” has been accepted to ICML 2026

Apr 24, 2026

Successfully passed my PhD candidate exam

Apr 13, 2026

Joined ByteDance’s Creation & Music Recommendation Team as a Research Intern to work on generative recommendation, LLM4Rec, and AIGC-powered content discovery

Mar 25, 2026

Invited to serve as a reviewer for NeurIPS 2026

Jan 18, 2026

Our paper titled “Breaking the Curse of Dimensionality in Gaussian Process Training With Zeroth-Order Adaptive Perturbation” has been accepted to ICASSP 2026 as oral

Jan 05, 2026

Transferred to the School of Artificial Intelligence, CUHK-Shenzhen with a fully-funded PhD scholarship

Nov 17, 2025

Selected as the recipient of the Guangdong Government Outstanding International Student Scholarship

Nov 01, 2025

Gave a talk on “Adaptive Kernel Design for Bayesian Optimization Is a Piece of CAKE with LLMs” at the 2025 Greater Bay Area Graduate Research Forum in Shenzhen, China

Oct 09, 2025

Invited to serve as a reviewer for ICASSP 2026

Sep 24, 2025

Invited to serve as a reviewer for ICLR 2026

Sep 19, 2025

Our paper titled “Adaptive Kernel Design for Bayesian Optimization Is a Piece of CAKE with LLMs” has been accepted to NeurIPS 2025

Sep 15, 2025

Joined Dria as a Research Intern to work on evolutionary coding agents

Jul 20, 2025

Won the 2nd prize award at the 2025 Doctoral Research and AI Innovation Conference held by CUHK-Shenzhen

May 09, 2025

Our latest work on grid spectral mixture product (GSMP) kernel has been featured in the “Machine Learning: From the Classics to Deep Networks, Transformers and Diffusion Models” book