Hongxu (Danny) Yin

I am a Senior.II (Staff) Research Scientist at NVIDIA Research, with the learning and perception research team led by Jan Kautz. I did my Ph.D. at Princeton University, where I was advised by Prof. Niraj Jha.

I am a recipient of Princeton Yan Huo 94* Graduate Fellowship, Princeton ECE Best Dissertation Finalist, Princeton Natural Sciences and Engineering Fellowship, Defense Science & Technology Agency gold medal, and Thomson Asia Pacific Holdings gold medal.

I have also been featured by Forbes as Top 60 Elite Chinese in North America and 36Kr with Global Outstanding Chinese Power 100 Award.

Google Scholar  /  Twitter /  CV  

I'm interested in a deeper understanding of neural nets.

This makes neural net efficient and secure. Topics cover resource efficient (data/computation) deep learning, overseeing CNNs and transformers, leveraging model inversion, knowledge distillation, dynamic inference, neural architecture search, pruning, quantization, model adaptation, etc.

I aim for fast and reliable networks for multi-modality tasks, autonomous driving, large language models, e-commerce, smart healthcare, among others.

We always welcome great research interns. Reach out to us if interested.

Dec. 2023 - I have research intern slots for VLM next spring/summer/fall 2024, and please drop a CV if interested.

[2024 Apr] We will host the Efficient Foundation Model workshop at ECCV 2024. See you in Milan.

[2024 Mar] FedBPT won the best paper award at AAAI symposium. Congrats to Jingwei!

[2024 Feb] Two papers accepted at CVPR 2024 on vision-language models.

[2024 Feb] We will host the Efficient Computer Vision workshop and GPU-based DL Acceleration tutorial at CVPR 2024. See you in Vancouver.

[2024 Jan] Two papers accepted at ICLR 2024 on efficient and robust large models.

[2023 Apr] Two papers accepted at ICML 2023 on improving transformer and diffusion models.

[2023 Apr] We will host the Data-efficient Learning Tutorial at ICCV 2023. See you in Paris.

[2023 Mar] Three papers accepted at CVPR 2023 on resource efficient deep learning.

[2023 Feb] We will host the GPU-based Acceleration Tutorial at CVPR 2023. See you in Vancouver.

[2022 Nov] Featured in 36Kr 2022 Global Outstanding Chinese Power 100 Awards.

[2022 Oct] I will be at Forbes Dinner 2022. Congrats to new awardees and see you in San Francisco.

Full list here. * equal contribution. ^ our great research interns at NVIDIA Research.

With an enhanced pre-training recipe we build VILA, a Visual Language model family that consistently outperforms the state-of-the-art models, e.g., LLaVA-1.5, across main benchmarks without bells and whistles.

We introduce RegionGPT that enables complex region-level captioning, reasoning, classification, and expression comprehension capabilities for the multimodal large language model.

Diffusion model for Plug-and-Play controllable generation.

Continual learning is now enabled for evolving model architecture upgrades. Your current model and new data are all you need to swap into a stronger architecture.

Transformers are vastly redundant, and one-click global structural pruning offers speed up right away. Stacking same block across depth is lazy, and a new distribution rule is proposed based on embedding only.

Deep equilibrium models smooth out flickering effect in video, enabled through recurrence without temporal-aware training, yielding efficient early stopping.

We show viability to train an invert network that maps intermediate tensors back to inputs. Works smoothingly for GANs and classifiers on high resolution tasks.

Sturcutral pruning is a quick knapsack problem to maximize accuracy through combining latency-guided parameter chunks.

We show that transformers can quickly drop redundant tokens and reserve computation on only informative ones, offering off-the-shelf cost saving.

We show Vision Transformer gradient encode sufficient information such that private original images can be easily reconstructed via inversion.

We push structural pruning into earlier training, cutting down on training costs.

We argue for a Train-Large-Swap-Faster model acceleration paradigm. Quickly adapting a large model to varying constraints in CPU-second search yields the quick finding of Pareto front.

We show under strong inversion, gradients are in essence original data via inversion, even for large datasets, large nets, for high resolution.

We aim at code-book oriented best scaled optimal quantization for deep nets.

We show that trained deep nets are in essence datasets. One can quickly invert from net outputs to synthesize a new dataset for off-the-shelf models. See ResNet-50 dreamed objects as left.

A genetic algorithm to quickly adjust the hyper architecture of a base model to target platforms (DSP, CPU, GPU) given constraints such as latency or memory.

Wearable medical sensors, backed by extremely efficient NNs, offers around-the-clock diabetes diagnosis.

We show that grow-and-prune yields faster yet more accurate H-LSTM family, surpassing LSTMs and GRUs.

We observe high degree of non-monoticity in latency surface given shrinking model dimensions, and propose a systematic structural Grow-and-Prune way to exlpoit this for faster inference.

Human brains grow before age 2 before pruning neurons for efficient synapsis afterwards. We propose grow-and-prune accordingly, and show consistent improvements over conventional pruning that always start from full models.

We aim to lay out foundations for pervasive healthcare from a wearables angle. Book now available at Amazon.

An FPGA solution augmented by a novel real-time tri-core SVD design for multi-coset signal reconstruction.

Partial list here.