Efficient MOE inference and training

We explore developing a \texttt{H}eterogeneous-aware \texttt{EX}pert \texttt{A}llocation framework, \textbf{\texttt{HEXA-MoE}}, with significantly enhanced computing efficiency.

We propose Lightening-Transformer, the first light-empowered, high-performance, and energy-efficient photonic Transformer accelerator.

We design and tape-out the SpAtten architecture in TSMC 28nm technology digital chip.

This paper provides an overview of efficient deep learning methods, systems and applications.