Neuro-hardware systems on a chip (NeuSoC) for micro-LLM: integration of specialized accelerators into slow industrial MC

The concept of Neuro-Hardware Systems on a Chip (NeuSoC) is proposed, aimed at the efficient execution of microscopic language models (Micro-LLM) on industrial microcontrollers (MC) with limited computing resources and frequency. Unlike approaches that require high-performance central processors, NeuSoC integrates specialized, super-energy-efficient hardware accelerators directly into the chip of existing MC, acting as specialized peripherals (similar to SPI/I2C). The article details the architecture of such accelerators, focusing on blocks for 8-bit matrix multiplications, activation functions (Softmax), and attention operations. The interaction of accelerators with the main core of the MC through standardized interfaces and the issues of compiling models for the heterogeneous NeuSoC system is considered. The fundamental possibility of significantly speeding up Micro-LLM output while maintaining extremely low power consumption is shown.