In terms of intelligent sensing and computing, this research center leverages our university's strengths in semiconductor and integrated circuit design.
We focus on the research and development of three key technologies: intelligent sensing systems, artificial intelligence computing chips, and robots and sensory synthesis.
From the sensing end, we extract important features and develop AI computing chips and systems through hardware-software co-design, achieving high-performance, rapid-response intelligent sensing and computing systems, which are applied to robotic perception.
Our forward-looking research goal is to develop high-performance intelligent sensing and computing chips.
For intelligent sensing chips, we utilize in-sensor computing to reduce data transmission requirements.
For intelligent computing chips, we employ non-Von Neumann architectures and in-memory computing technologies to achieve high energy efficiency computing chips at the POPs/W level.
Intelligent Sensors
In wireless edge devices, such as drones, the need for low power consumption is increasing. Traditional methods of acquiring motion and depth using CMOS image sensors and digital signal processors require high power consumption and introduce latency. To address this, Processing-in-Sensor (PIS) technology is becoming increasingly important for implementing low-power, low-latency real-time computing. We propose a frame-based Motion Detection vision sensor that uses longitudinal parallel region binarization for spatial feature acquisition, eliminating the need for initial analog-to-digital conversion. This approach significantly reduces power consumption.
By combining frame difference and area binarization, the chip enables the acquisition of both spatial and temporal features, which can be used for obstacle detection and avoidance. In the application of obstacle detection and avoidance, this spatial and temporal information allows for the calculation of dynamic stereo vision. Notably, this method is the first to calculate the depth of dynamic objects while filtering out the depth of static objects. The research result was published in IEEE Journal of Solid-State Circuits.
By combining frame difference and area binarization, the chip enables the acquisition of both spatial and temporal features, which can be used for obstacle detection and avoidance. In the application of obstacle detection and avoidance, this spatial and temporal information allows for the calculation of dynamic stereo vision. Notably, this method is the first to calculate the depth of dynamic objects while filtering out the depth of static objects. The research result was published in IEEE Journal of Solid-State Circuits.
Computing in Memory (CIM)
With the increasing demand for high-performance and low-power AI hardware, the next-generation AI chip design has to break through the limitations of the traditional von Neumann architecture. The traditional architecture faces a memory wall bottleneck when executing AI deep learning algorithms, leading to frequent data transfers between memory and computing units, which consume significant power and introduce delays that hinder system performance.
To overcome these challenges, our research group focuses on in-memory computing to break the memory wall and minimize data movement. In-memory computing reduces data transfer and energy consumption, providing a significant breakthrough in the circuit design . This approach contributes to the development of AI edge devices with constrained energy and hardware resources (see Figure B-5). In particular, our research group has developed various in-memory computing architectures tailored for different memory types (volatile and non-volatile). These architectures support fixed-point computing needs ranging from 1-bit to 8-bit, achieving world-leading energy efficiency in current in-memory computing technologies. Our research results were published in Nature Electronics, Proceedings of IEEE International Solid-State Circuits Conference (ISSCC), and Proceedings of IEEE Symposium on VLSI Technology and Circuits (VLSI Technology and Circuits).
To overcome these challenges, our research group focuses on in-memory computing to break the memory wall and minimize data movement. In-memory computing reduces data transfer and energy consumption, providing a significant breakthrough in the circuit design . This approach contributes to the development of AI edge devices with constrained energy and hardware resources (see Figure B-5). In particular, our research group has developed various in-memory computing architectures tailored for different memory types (volatile and non-volatile). These architectures support fixed-point computing needs ranging from 1-bit to 8-bit, achieving world-leading energy efficiency in current in-memory computing technologies. Our research results were published in Nature Electronics, Proceedings of IEEE International Solid-State Circuits Conference (ISSCC), and Proceedings of IEEE Symposium on VLSI Technology and Circuits (VLSI Technology and Circuits).
CIM Self-Test and Self-Restoration
Computing in memory (CIM) supports weight stationary dataflow and local computation, and has the potential to be more efficient than the von Neumann architecture. However, it is not optimal to rely on weighted stationary dataflow because the convolutional neural network layer with high depth direction or input suffers from high feature maps and low utilization. In addition, in-memory computing, unlike traditional memory, does not need to be 100% accurate and thereby demands for a different approach to memory testing. We develop Morphable CIM which is capable of switching between weighted static or input static data streams on its own. We propose early termination, input-inversion, and bit-line inversion mechanisms for in-memory operations. We have also developed self-assessment and self-restoration of in-memory operations. Our research results were published in Proceedings of ACM/IEEE Design Automation Conference (DAC’23), Proceedings of IEEE International Conference on Computer Design (ICCD’23), and Proceedings of Design, Automation & Test in Europe Conference & Exhibition (DATE’23).
A New Operational Unit and Number Format for Integer and Floating-Point Numbers
To address the high demand for memory access and computational resources in AI computing, we propose a new computing unit and novel number formats for integer and floating-point numbers. First, we introduce the Bucket-based Processing Engine, utilizing multiple small accumulators (buckets) to handle a narrow range of exponential values of floating-point numbers. This approach reduces power-intensive operations from alignment and format conversion units.
Second, the Bit-Serial Cache leverages localized data caching to enhance performance and energy efficiency. It stores partial sums of recently computed input bit vectors, minimizing redundant computations through cache accesses. We introduce two new number formats: 1). BSFP (Bit-Serial Floating-Point), which approximates quantized floating-point numbers by linearly stacking multiple groups of two's complement integers. 2). FM-P2L (Fixed-Point to Linear), using fixed-point numbers for the highest significant bit and powers of two for the lowest significant bit. Detailed research findings were published in Proceedings of IEEE/ACM International Symposium on Microarchitecture (MICRO’23), Proceedings of International Conference on Learning Representations (ICLR’23), and Proceedings of ACM/IEEE Design Automation Conference (DAC’23).
Second, the Bit-Serial Cache leverages localized data caching to enhance performance and energy efficiency. It stores partial sums of recently computed input bit vectors, minimizing redundant computations through cache accesses. We introduce two new number formats: 1). BSFP (Bit-Serial Floating-Point), which approximates quantized floating-point numbers by linearly stacking multiple groups of two's complement integers. 2). FM-P2L (Fixed-Point to Linear), using fixed-point numbers for the highest significant bit and powers of two for the lowest significant bit. Detailed research findings were published in Proceedings of IEEE/ACM International Symposium on Microarchitecture (MICRO’23), Proceedings of International Conference on Learning Representations (ICLR’23), and Proceedings of ACM/IEEE Design Automation Conference (DAC’23).
High-Definition Image Chip Architecture
In AI applications, high-definition image processing finds use in diverse scenarios like smartphones, cameras, car dashcams, surveillance systems, and televisions. Achieving higher image quality necessitates deeper and more complex network models, which pose significant challenges in terms of power consumption and cost as depicted in Figure B-10. Addressing these challenges is a key focus of our research.
To tackle these issues, we propose several innovative solutions. For example, we proposed block-based overlapped stripe inference for high-resolution image processing to reduce external memory bandwidth requirements. Utilizing structure-sparse kernel to decrease computational complexity at the same time with the use of a 40nm wafer to achieve a power efficiency of 4.6-8.3 TOPS/W, thereby to develop an ultra-high-definition 4K-UPVC chip which is capable of delivering high-quality images efficiently. Furthermore, we explore mixed-precision computing to reduce computational complexity. Our biased mixed-precision approach allows for 8-bit or 4-bit mixed-precision computing with only a 10% increase in area, saving 33% of power consumption while maintaining image quality. Research findings were published in IEEE Solid-State Circuits Letters, as well as Proceedings of Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC).
Figure B-10: Impact of Artificial Intelligence Computing Applications and Network Model Size on Quality
To tackle these issues, we propose several innovative solutions. For example, we proposed block-based overlapped stripe inference for high-resolution image processing to reduce external memory bandwidth requirements. Utilizing structure-sparse kernel to decrease computational complexity at the same time with the use of a 40nm wafer to achieve a power efficiency of 4.6-8.3 TOPS/W, thereby to develop an ultra-high-definition 4K-UPVC chip which is capable of delivering high-quality images efficiently. Furthermore, we explore mixed-precision computing to reduce computational complexity. Our biased mixed-precision approach allows for 8-bit or 4-bit mixed-precision computing with only a 10% increase in area, saving 33% of power consumption while maintaining image quality. Research findings were published in IEEE Solid-State Circuits Letters, as well as Proceedings of Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC).
Figure B-10: Impact of Artificial Intelligence Computing Applications and Network Model Size on Quality
Friction Nanosensor Development
The Internet of Things (IoT) has significantly enhanced our daily life by integrating activities into a smart network. This transformation has particularly impacted healthcare, environmental monitoring, and home security, driving advancements in wearable sensor technology. However, the growing demand for wearable sensors has intensified challenges related to power supply. Traditional battery-powered sensors are constrained by high energy consumption, limited lifespan, bulkiness, and environmental concerns. To overcome these limitations, sustainable energy technologies such as nanogenerators, which harvest energy from the environment, have emerged. Among these, the Triboelectric Nanogenerator (TENG) (see Figure B-11) stands out for its ability to convert mechanical energy into electrical energy using tribocharging and electrostatic induction. The generated electrical energy not only serves as a power source but also as a signal for chemical sensing.
This application, termed a Triboelectric Nanosensor (TENS), detects analytes based on changes in electrical output influenced by adsorbed substances on the TENG's contact layer. While previous studies have explored TENS for sensing analytes like caffeine and pathogenic bacteria, few have demonstrated its potential for on-site detection critical in safety and environmental applications. In response, we propose integrating self-powered TENS with robotic hands for fingertip detection of mercury (Hg2+) ions. Conventional Hg2+ ion detection methods face challenges such as off-site detection, high power consumption, complex sample preparation, and lengthy processing times. Our solution involves a tellurium nanowire-based self-powered TENS capable of selectively detecting Hg2+ ions. Integrated with a robotic hand, the TENS comes into contact with and separates Hg2+ ion solutions, altering triboelectric output due to substance-specific triboelectric polarity. Real-time output signals are wirelessly transmitted via a Bluetooth system integrated into the robotic hand to a smartphone, demonstrating on-the-spot detection of Hg2+ ions in real samples. Research finding was published in ACS Nano.
Figure B-11, Friction Nanosensor
This application, termed a Triboelectric Nanosensor (TENS), detects analytes based on changes in electrical output influenced by adsorbed substances on the TENG's contact layer. While previous studies have explored TENS for sensing analytes like caffeine and pathogenic bacteria, few have demonstrated its potential for on-site detection critical in safety and environmental applications. In response, we propose integrating self-powered TENS with robotic hands for fingertip detection of mercury (Hg2+) ions. Conventional Hg2+ ion detection methods face challenges such as off-site detection, high power consumption, complex sample preparation, and lengthy processing times. Our solution involves a tellurium nanowire-based self-powered TENS capable of selectively detecting Hg2+ ions. Integrated with a robotic hand, the TENS comes into contact with and separates Hg2+ ion solutions, altering triboelectric output due to substance-specific triboelectric polarity. Real-time output signals are wirelessly transmitted via a Bluetooth system integrated into the robotic hand to a smartphone, demonstrating on-the-spot detection of Hg2+ ions in real samples. Research finding was published in ACS Nano.
Figure B-11, Friction Nanosensor
Mechanical Finger Sensing and Control
For robots to assist healthcare workers in high-risk environments, the flexibility of the robotic hand is crucial (see Figure B-12 for the modular soft actuator finger architecture). Previous research on robotic fingers has focused on fully actuated fingers with high degrees of freedom to mimic the human hand and underactuated fingers designed for adaptive motion. Fully actuated fingers often struggle with grasping objects of varying shapes, while underactuated fingers face challenges in performing precise gripping maneuvers. To address these issues, we combined the advantages of both Fully Actuated and Self-Adaptive (FASA) modes. The FASA finger is in-line driven, allowing for both adaptive gripping and precise angular positioning with the same finger. Through kinematic-static analysis, we optimized the torsional rigidity and functionality of FASA fingers. In full drive mode, controlling two motors enables independent movement of two joints, allowing the FASA finger to perform precise angular movements and adaptive grasping without altering the mechanical structure.
Additionally, to ensure safe human-robot interactions, we developed soft material-actuated robot fingers. Traditional soft robot fingers often lack precise control over fingertip positions, so we introduced innovative Modular Soft Actuators (MSA). These MSAs offer both adaptive behavior and precise joint angle control for anthropomorphic robot fingers. Each air chamber's pressure is independently controlled, allowing for more accurate joint angles compared to traditional PneuNets soft actuators. A rigid skeleton enhances force transfer and measurement capabilities, ensuring each module can effectively withstand and transfer forces during use. Experimental results demonstrate that the modularized soft actuator performs exceptionally well and is adaptable to various applications. The research results related to robotic finger drive mechanisms were published in ASME Journal of Mechanisms and Robotics, Proceedings of the 2023 IET International Conference on Engineering Technologies and Applications (IET IECTA 2023), and Proceedings of the IFTOMM World Congress (IFToMM WC2023).
Figure B-12, Modular Flexible Actuator Finger Structure.
Additionally, to ensure safe human-robot interactions, we developed soft material-actuated robot fingers. Traditional soft robot fingers often lack precise control over fingertip positions, so we introduced innovative Modular Soft Actuators (MSA). These MSAs offer both adaptive behavior and precise joint angle control for anthropomorphic robot fingers. Each air chamber's pressure is independently controlled, allowing for more accurate joint angles compared to traditional PneuNets soft actuators. A rigid skeleton enhances force transfer and measurement capabilities, ensuring each module can effectively withstand and transfer forces during use. Experimental results demonstrate that the modularized soft actuator performs exceptionally well and is adaptable to various applications. The research results related to robotic finger drive mechanisms were published in ASME Journal of Mechanisms and Robotics, Proceedings of the 2023 IET International Conference on Engineering Technologies and Applications (IET IECTA 2023), and Proceedings of the IFTOMM World Congress (IFToMM WC2023).
Figure B-12, Modular Flexible Actuator Finger Structure.
