Edge-AI chips for the next generation of IoT devices

In recent years, edge-based computing has increasingly emerged as an attractive solution to provide Artificial Intelligence (AI) applications and Internet of Things (IoT) devices with valuable sensor data in a time- and energy-efficient manner. However, in order to effectively implement these innovative technologies at scale, researchers and integrated circuit manufacturers must first develop new, specialised chips that can support their computationally-intensive demands.

One actor working towards this goal is the Chinese startup “Reexen Technology”, which was founded in 2018 by ETH Zurich graduate Dr. Hongjie Liu. It has since established itself as a rising global player in the field of edge-AI Application-Specific Integrated Circuits (ASICs) for consumer, medical, and industrial markets.

Despite their considerable potential impact on our daily life, however, it is not always easy to navigate the extensive assortment of terms used to denote the latest technological trends, such as “edge-AI ASICs” or “embedded DNN functions in low-power IoT sensors”. In order to address this shortcoming, the following article therefore has two main goals. First, it aims to present a quick overview of the most important concepts relating to the emerging field of “Artificial Intelligence of Things” (AIoT), which broadly encompasses all of the terms mentioned above. Second, it aims to provides a few practical examples of the “real-world” implementation of several of the technologies, based on the work of Reexen.

IoT, Edge Computing & AI

The Internet of Things (IoT) has quickly established itself as one of the most promising emerging paradigms over the past decade. Broadly defined, it is a “network of intelligent objects capable of automatically organising and sharing information, data, and resources, as well as of making decisions and reacting to changes in the environment”. This highly publicised concept promises to unify everything in our world under a common infrastructure, thereby enabling us to connect and communicate with just about anything from anywhere in the world. This, in turn, has sparked the development and proliferation of a myriad of “smart” devices for a wide range of sectors, such as healthcare, energy, urban planning, and industrial manufacturing, to name a few. 

Although definitions vary as to what exactly makes an object “smart”, one of their key features undoubtedly includes their ability to collect data about their environment via embedded sensors. This, however, quickly results in the production of massive datasets – particularly when numerous sensor devices are interconnected within an IoT network – that need to be analysed in a timely manner. This raises the question of which type of computing is best suited for the job. 

One popular option is cloud computing, which outsources the task of storing, managing, and processing data to a network of remote servers hosted on the Internet rather than a local server or a personal computer. However, although this method is appropriate for some IoT sectors, it also comes with several drawbacks, including privacy concerns, increased latency, decreased bandwidth and potentially even loss of data. 

As a result, edge computing has emerged as a promising solution for more time-sensitive applications, whereby data is processed and analysed by smaller computing devices located close to the data source – i.e., the sensors. These “edge devices”, in turn, open the door to a plethora of interesting applications that involve the use of Artificial Intelligence (AI), which has led to the emergence of a new field, known as Artificial Intelligence of Things (AIoT). This could potentially prove to be a transformational advancement, as researchers and industry experts envision that AIoT systems will one day be able to not only detect events and failures but also to collect data and make appropriate decisions based on that data – all without human involvement.

Despite significant advances in this domain, many IoT sensor devices continue to employ conventional processor chips, which are not very well-suited to implementing many of the modern, computationally intensive algorithms that sensors require to perform on edge. These, for example, include cutting-edge machine-learning algorithms, and deep neural networks (DNNs) in particular, which have been responsible for a multitude of recent AI breakthroughs, such as DeepMind’s AlphaGo. As a result, significant efforts are currently underway to develop new “application-specific integrated circuits” (ASICs), which, as their names imply, are specifically designed for a particular task or application.  This is where startups, such as Reexen Technology, come in, who are working to develop innovative solutions to efficiently deploy cutting-edge AI technologies at scale. 

Reexen Technology & Neuromorphic Engineering

As explained by Dr. Liu, Reexen is active in neuromorphic engineering – sometimes also called neuromorphic computing – which aims to emulate the neural structure and operation of the human brain with software and hardware. In the case of Reexen, their focus is on mimicking how the brain and the eye, or the cochlea in our ear, work – also known as “neuromorphic sensing and processing” – by developing new “mixed-signal in-memory computing and inference sensing” solutions. 

Without going into too much detail, mixed-signal in-memory computing chips solve problems of latency and high energy consumption in analogue-to-digital (A/D) conversion and data-intensive digital signal processors (DSPs) in two main ways. First, unlike conventional Central Processing Units (CPUs) or Graphics Processing Units (GPU), which can only process information in the digital, “computer-readable” domain, mixed-signal computing chips can directly process sensory signals in real-time in their analogue representation, as well as in the digital domain. Second, by directly integrating computing cells into the memory cells, processing-in-memory solutions can overcome the deficiencies of the so-called “von Neumann” architecture of conventional computers, which expend considerable amounts of time and energy to transfer data from the memory to the CPU for computation. 

Inference sensing, in turn, means that the inputs derived from the physical world are converted and processed at the side of the sensors, rather than in the cloud or a big computer, which is useful for a wide range of applications, such as smartwatches, earphones, or smart IoT devices, to name a few. 

In this context, Reexen collaborated with a top MEMS (micro-electromechanical systems) microphone company to integrate its new audio-processing chip into the MEMS sensor itself, thereby enabling the microphone to use keyword spotting. This is essential for a plethora of speech recognition applications – a noteworthy example of which includes so-called “wake words”, such as “Okay Google”, “Hey Alexa”, or “Hey Siri”, which enable digital assistants to respond to peoples’ queries. In addition to this audio-processing chip, Reexen is currently focusing on developing a vision-processing chip, which can be used in smartphones or AR/VR glasses. 

Conclusion

In conclusion, edge-based computing has unquestionably established itself as an attractive solution to provide IoT devices with high-quality, actionable sensor data in a time and energy-efficient manner. In order to achieve this, however, researchers and industry leaders alike have been scrambling to develop new, specialised chips that can complete increasingly demanding machine learning tasks on-device – either completely or by employing a hybrid strategy – in real-time. 

One such player includes the Chinese startup Reexen Technology, which develops “mixed-signal in-memory computing and inference sensing” solutions that aim to emulate the neural structure and operation of the human brain. This, for example, has resulted in an innovative audio-processing chip, which is being used to produce MEMS microphones with integrated keyword-spotting functions and is currently being used to develop a vision-processing chip for smartphones and AR/VR glasses.

Video Conversation with Dr. Liu, Reexen Technology

References

DeepMind. AlphaGo [online]. [n.d.]. https://deepmind.com/research/case-studies/alphago-the-story-so-far (accessed on 30.08.2021).

DRYFHOUT, Matt and HEWER, Scott. What Is Cloud Computing? [online]. 11.04.2019. https://scouttg.com/blog/articles/what-is-cloud-computing/ (accessed on 30.08.2021).

HERTZ, Jake. AI Chip Strikes Down the von Neumann Bottleneck With In-Memory Neural Net Processing [online]. 10.07.2020. https://www.allaboutcircuits.com/news/ai-chip-strikes-down-von-neumann-bottleneck-in-memory-neural-network-processing/ (accessed on 30.08.2021).

HUČ, Aleks, ŠALEJ, Jakob and TREBAR, Mira. “Analysis of Machine Learning Algorithms for Anomaly Detection on Edge Devices”. Sensors, 2021, 21(14), 4946. Available at: https://www.mdpi.com/1424-8220/21/14/4946.

LAPEGNA, Marco, BALZANO, Walter, MEYER, Norbert and ROMANO, Diego. HUČ, Aleks, ŠALEJ, Jakob and TREBAR, Mira. “Clustering Algorithms on Low-Power and High-Performance Devices for Edge Computing Environments”. Sensors, 2021, 21(16), 5395. Available at: https://www.mdpi.com/1424-8220/21/16/5395. 

MADAKAM, Somayya, RAMASWAMY, R., and TRIPATHI, Siddharth. “Internet of Things (IoT): A Literature Review”. Journal of Computer and Communications, 2015, 3, pp. 164-173. Available at: https://www.scirp.org/journal/paperinformation.aspx?paperid=56616.

MAJUMDER, Sumit, AGHAYI, Emad, NOFERESTI, Moein, MEMARZADEH-TEHRAN, Hamidreza, MONDAL, Tapas, PANG, Zhibo and DEEN M. Jamal. “Smart Homes for Elderly Healthcare—Recent Advances and Research Challenges”. Sensors, 2017, 17(11), 2496. Available at: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5712846/.

Reexen Technology. Homepage [online]. [n.d.]. http://reexen.ch/ (accessed on 30.08.2021).

Reexen Technology. Neural Network Processing IP/ASIC [online]. [n.d.]. http://reexen.ch/?page_id=4632 (accessed on 30.08.2021).

Sensory. Embedded Wake Word & Phrase Recognition Engine [online]. [n.d.]. https://www.sensory.com/wake-word/ (accessed on 30.08.2021).

SILVERIO-FERNÁNDEZ, Manuel, RENUKAPPA, Suresh and SURESH, Subashini. “What is a smart device? – a conceptualisation within the paradigm of the internet of things”. Visualisation in Engineering, 2018, 6(3). Available at: https://viejournal.springeropen.com/articles/10.1186/s40327-018-0063-8.

STEWART, Duncan, LOUCKS, Jeff, CASEY, Mark and WIGGINTON, Craig. Bringing AI to the device: Edge AI chips come into their own [online]. 09.12.2019. https://www2.deloitte.com/us/en/insights/industry/technology/technology-media-and-telecom-predictions/2020/ai-chips.html (accessed on 30.08.2021).

VAUGHAN, Jack. Chip-Enabled Edge AI Drives Next-Gen IoT [online]. 23.02.2021. https://www.iotworldtoday.com/2021/02/23/chip-enabled-edge-ai-drives-next-gen-iot/ (accessed on 30.08.2021).