The rise of generative AI has increased the size of LLMs, escalating computing costs for services. While Datacenter LLM services use larger batch sizes to improve GPU efficiency, self-attention block processing is still suffering from low efficiency. SK hynix's AiM device, utilizing Processing in Memory technology, offers high bandwidth and high energy efficiency, reducing operational costs significantly compared to GPUs regardless of batch size. Additionally, applying AiM in on-device services enables high performance and low energy consumption, enhancing competitiveness. SK hynix has developed the AiMX accelerator prototype for Datacenters, showcasing single batch operations last year and planning multi-batch operations with larger models this year. The AiMX structure can be similarly applied to on-device AiM implementations. SK hynix's AiM/AiMX solutions address cost, performance, and power challenges in LLM services for both Datacenters and on-device AI applications.
Euicheol Lim
Eui-cheol Lim is a Research Fellow and leader of Solution Advanced Technology team in SK Hynix. He received the B.S. degree and the M.S. degree from Yonsei University, Seoul, Korea, in 1993 and 1995, and the Ph.D. degree from Sungkyunkwan University, suwon, Korea in 2006. Dr.Lim joined SK Hynix in 2016 as a system architect in memory system R&D. Before joining SK Hynix, he had been working as an SoC architect in Samsung Electronics and leading the architecture of most Exynos mobile SoC. His recent interesting points are memory and storage system architecture with new media memory and new memory solution such as CXL memory and Processing in Memory. In particular, he is proposing a new computing architecture based on PIM, which is more efficient and flexible than existing AI accelerators, to process generative AI and LLM (large language Model) that is currently causing a sensation.