Dec 11, 2020

Abstract The human sensory system has a fascinating stimulus-detection capability attributed to the fact that the feature (pattern) of an input stimulus can be extracted through perceptual learning. Therefore, sensory information can be organized and identified efficiently based on iterative experiences, whereby the sensing ability is improved. Specifically, the distributed network of receptors, neurons, and synapses in the somatosensory system efficiently processes complex tactile information. Herein, we demonstrate an artificial tactile sensor system with a sensory neuron and a perceptual synaptic network composed of a single device: a semivolatile carbon nanotube transistor. The system can differentiate the temporal features of tactile patterns, and its recognition accuracy can be improved by an iterative learning process. Furthermore, the developed circuit model of the system provides quantitative analytical and product-level feasibility. This work is a step toward the design and use of a neuromorphic sensory system with a learning capability for potential applications in robotics and prosthetics. Introduction Over the past half-century, device electronics have been successfully advancing the Information Age thanks to consistent performance improvements based on the downscaling of digital devices that can provide reliable logic-gate operations, thus benefiting from their robustness to high levels of noise. Recently, research paradigms have shifted the focus to conventional devices with analog characteristics, which were previously considered a drawback. With an innovative computing architecture used to overcome the energy inefficiency of the conventional von Neumann architecture—typically referred to as a neuromorphic system 1 —the analog behaviors of these devices have begun to be rehighlighted as essential features for the implementation of neuromorphic systems 2 . Given that the numberless internal states of an analog device can be adjusted with minimal energy consumption and given that they can be maintained over the long term (i.e., they are nonvolatile) 3 , analog devices can store more data in a single device with better energy efficiency than digital devices. Specifically, recent advances in these analog devices allow them to emulate the functionality of biological synapses and neurons, while the ability of their crossbar arrays to solve cognitive tasks related to learning and recognition has been demonstrated experimentally 4 , 5 , 6 , 7 , 8 . A neuromorphic system will provide energy-efficient computing power for artificial intelligence that can replicate cognitive function up to the human brain level. Interestingly, some analog devices exhibit volatile behaviors for input stimuli; i.e., they yield a temporal state enhancement that quickly decays to its initial state, which constitutes one of the essential characteristics of a neuromorphic system. Early research on this volatile behavior focused on the emulation of short-term plasticity (STP) in biological synapses, such as paired-pulse facilitation (PPF) 9 , 10 . Moreover, recent studies have emulated biological neurons, including their capability to integrate temporal input stimuli 11 , 12 based on the leaky integrate-and-fire (I&F) neuron model 13 . By exploiting the response of the volatile device depending on the frequency of the input stimuli, the spatial or temporal patterns in the input stimuli can be classified. For example, the integration of sensor devices for various external stimuli (e.g., pressure, tactile, or visual stimuli) with processing components (i.e., a volatile analog device as a sensory neuron) has been proposed to capture the similarities of the human sensory perception system 14 , 15 , 16 , 17 , 18 , 19 . Notably, unlike the volatile or nonvolatile analog devices mentioned above, a specific device can achieve switchable volatile/nonvolatile behaviors, as desired 20 , 21 . Although this is not a generic term, we will refer to it as a “semivolatile” characteristic (in fact, in the case of a two-terminal-based analog device (memristor), a device with semivolatile behavior has been referred to as a second-order memristor 22 , 23 ). Previous studies of semivolatile behavior focused only on the capture of a psychological model related to the human brain’s processes of memorizing and forgetting (i.e., the transition from short-term memory to long-term learning) 20 , 21 . However, more attention should be paid to the potential of a semivolatile device to simulate both synaptic and neuronal functions simultaneously, i.e., the processes of leaky integration (in the neuron) and learning/recognition (in the synaptic network) of input stimuli and their emulation with a single type of device. To date 24 , neuromorphic systems have been implemented with an artificial neural network based on the integration of two different types of devices (for neuronal and synaptic devices). If the entire artificial neural network with neuronal/synaptic devices can be implemented with a single device type, a simpler fabrication process for high-density integration will be achieved. Consequently, the goal of achieving large-scale and on-chip implementation of neuromorphic systems can be realized much sooner than expected. In this paper, we demonstrate a biorealistic tactile sensor system wherein both the sensory neurons and perceptual synaptic network are implemented by a semivolatile carbon nanotube (CNT) transistor. The semivolatile transistor can switch the operation mode according to the bias condition. In this way, a single device type is allowed to play two different roles (neuronal and synaptic functions) simultaneously. In our tactile sensor system (Fig. 1 ), the sensing receptor, action potential in the axon, sensory neuron that processes information, and synaptic network for perception learning are emulated by a tactile sensor, a voltage-controlled oscillator (VCO) circuit, one neuronal CNT transistor, and the synaptic CNT transistor array, respectively. The tactile sensor converts pressure stimuli into resistance changes. These are then converted to digital signals and modulated such that their frequencies vary with the pressure intensity based on the use of the VCO. Subsequently, the output of the VCO is delivered to the neuronal CNT transistor that operates in volatile mode. A leaky-integrating output corresponding to the frequency of the VCO output is then generated. Finally, the sampled output of the neuronal CNT transistor is fed to the network of synaptic CNT transistors that operate in a nonvolatile mode, and the learning/recognition processes for distinguishing the input stimuli pattern are conducted in a supervised learning manner. Fig. 1: Conceptual design of our tactile sensor system compared with a biological sensor system. In biological systems, pressure stimuli applied to mechanoreceptors change the receptor potential of each mechanoreceptor. The receptor potentials initiate action potentials. Action potentials from multiple nerve fibers combine through neurons and contribute to information processing. Finally, the synaptic network in the brain recognizes the input pressure pattern. Our artificial tactile sensory system is composed of a tactile sensor device, voltage-controlled oscillator (VCO), neuronal device, and synaptic network. Fabrication of the tactile sensor A flexible and transparent polydimethylsiloxane (PDMS) substrate (thickness of 1 mm) was prepared by mixing PDMS prepolymer (Sylgard 184A, Dow Corning) and a curing agent at a ratio of 10:1 by weight. Subsequently, the substrate was cleaned by oxygen plasma treatment, functionalized with a poly-L-lysine solution (0.1% w/v in H2O, Sigma-Aldrich), and acted as an effective adhesion layer for the CNTs. The substrate was thoroughly rinsed with deionized (DI) water and dried with flowing nitrogen gas. Subsequently, the 99% metallic CNT network film was directly formed on the PDMS substrate by spray coating the preseparated metallic CNT solution (concentration of 0.01 mg/mL) on a hot plate (at 100 °C) followed by thorough rinsing with isopropanol and DI water. Two Cu electrodes were then formed at both ends of the CNT network film with silver paste for reading and inducing electrical signals. Subsequently, a top PDMS layer was cast with liquid PDMS with a thermal curing agent onto the CNT network film to fabricate a sandwich-like structure. In the final step, all the layers were annealed at 100 °C for 1 h. Fabrication of the semivolatile CNT transistor array CNT transistors were fabricated on p-doped rigid silicon substrates with a thermally grown SiO2 layer with a thickness of 50 nm. We used the local back-gate structure for the modulation of the channels in the CNT transistors. To form the local back-gate, a Ti layer with a thickness of 20 nm was deposited by e-beam evaporation and patterned by a subsequent lift-off process. Subsequently, an Al2O3 layer (thickness of 40 nm) and a SiO2 layer (thickness of 10 nm) were deposited sequentially to form a gate insulator by atomic layer deposition. The top surface of the SiO2 layer was then functionalized with a 0.1 g/mL poly-l-lysine solution for 20 min to form an amine-terminated layer, which acted as an effective adhesion layer for the deposition of the CNTs. The CNT network channel was then formed by immersing the chip into a 0.01 mg/mL 99% semiconducting CNT solution (NanoIntegris, Inc.) for 8 min at an elevated temperature of 100 °C. The source/drain electrodes that consisted of Ti and Pd layers (each 2 nm and 30 nm, respectively) were then deposited and patterned using conventional thermal evaporation and a lift-off process, respectively. Finally, additional photolithography and oxygen plasma etching steps were conducted to remove unnecessary CNTs other than those in the channel area, thus isolating the devices from one another. In the case of the crossbar array, Cu (thickness of 80 nm) and SiOx (thickness of 150 nm) were sequentially deposited and patterned for the metal line and interlayer dielectric layer (ILD), respectively. Results and discussion The operational principle of our semivolatile CNT transistor is based on different hole-movement mechanisms in the traps (i.e., interface and surface traps). It has been shown in other studies 25 that typical CNT transistors have a drain current (ID) hysteresis that is related to the gate voltage (VG) sweep (Fig. 2a ). The hysteresis is attributed to the trapping/detrapping of holes at (1) the interface trap through the tunneling process and (2) the surface trap through the diffusion process in the lateral direction (see Fig. 2b , Fig. S1 , and Supplementary Information Note 1 for a more detailed discussion). Upon the application of negative VG values, traps are filled by holes. The positively charged traps bend the energy band of the CNT downward, thereby resulting in the suppression of ID owing to the enlarged Schottky barrier width at the drain/CNT junction (left inset of Fig. 2a ). A positive VG ejects the trapped holes, and the consequent upward band bending leads to the increase in the ID owing to the narrowing of the Schottky barrier width (right inset of Fig. 2a ). Notably, because the interface trap is adjacent to the CNT channel, the tunneling of holes between them is relatively fast. Conversely, the diffusion of holes between surface traps in the lateral direction is relatively slow. These different hole-movement mechanisms lead to a semivolatile behavior in the CNT transistor. Figure 2c shows one example of a ID response following the application of a single VG pulse. The low level of the VG pulse (Vlow = −3 V) is the undisturbed read voltage for the channel conductance (i.e., −3 V cannot lead to any trapping/detrapping process). Accordingly, the pulse width was minimized to 5 μs to eliminate any time-cumulative effects. Before the onset of the measurements, a negative VG (−6 V, direct current (DC)) was applied for 1 s to ensure that all the traps were filled by holes at the initial state. When a single VG pulse was applied, the trapped holes were ejected to the CNT channel. If the high level of the VG pulse (Vhigh) was large enough to cause the ejection of all the trapped holes at the interface and surface traps (given that there are no holes to refill the traps after the pulse is removed), the ID change, i.e., the nonvolatile mode operation, would be maintained over the long term. By contrast, if Vhigh is not sufficient to cause the ejection of all the trapped holes (e.g., Vhigh = 0.5 V, as shown in Fig. 2c ), ID temporarily increases and decays to its initial state subsequently, i.e., the volatile mode operation (Istart, Ipeak, and Iend, denote the initial, peak, and final ID levels, respectively). During volatile mode operation, only the holes at the interface trap are ejected by applying the pulse, but most of the holes at the surface trap remain. After the pulse ends, surface traps act as reservoirs of holes and help refill the empty interface traps, which results in the gradual recovery of ID. Consequently, the level of Vhigh determines the operation mode of the semivolatile CNT transistor. Figure 2d shows the summarized channel conductance change (defined as ΔG) according to the level of Vhigh. Here, the amount of short-term enhancement in ID is defined as ΔGST = (Ipeak − Istart)/Istart, and the amount of long-term change in ID is defined as ΔGLT = (Iend − Istart)/Istart. Notably, ΔGLT is zero until Vhigh attains a value of 2.5 V, which indicates that there is only a temporary change in the drain current. Moreover, ΔGLT gradually increases once Vhigh exceeds 2.5 V. Therefore, the semivolatile CNT transistor has two switchable operation modes, which can be controlled by adjusting the Vhigh level. Fig. 2: Switchable operation modes of the semivolatile CNT transistor. a Counterclockwise hysteresis of ID according to VG sweep (VD = 2 V and VS = 0 V). Depending on the presence of trapped holes, the width change in the Schottky barrier at the drain junction results in ID hysteresis. b Schematics of the trapping/detrapping of holes at the interface/surface traps. c Temporal ID enhancement and decaying behavior when a single VG pulse is applied. The use of Vhigh = 0.5 V is not sufficient to allow the ejection of all the trapped holes from the interface/surface traps. d The summarized short-term ID change (ΔGST) and long-term ID change (ΔGLT) according to the Vhigh level. When Vhigh > 2.5 V is applied to the gate of the CNT transistor, the CNT transistor operates in a nonvolatile mode. Moreover, when Vhigh < 2.5 V is applied, the CNT transistor operates in volatile mode. e Cumulative ID enhancement behavior based on the application of a train of pulses. Because the CNT transistor operates in volatile mode (Vhigh = 0.5 V), the final current level (Iend) decays to the initial level (Istart) again (these data are not shown in this graph). f Variation in ΔGST as a function of fpre enables the differentiation of the pressure intensity. As mentioned above, the volatile behavior can be exploited to emulate neuronal functions. When a series of pulses (referred to as presynaptic spikes, Vpre) are applied to the volatile CNT transistor, ΔGST is dependent on the frequency of Vpre (fpre). Figure 2e shows one example of the transient ID behavior when a train of five pulses for which fpre = 105 Hz is applied, and Fig. 2f shows the summarized ΔGST values according to fpre. As the interval of each pulse becomes shorter (fpre increases), larger cumulative ID enhancement leads to higher Ipeak values. By contrast, as the interval of the pulse train increases, the decaying ID becomes more dominant, and Ipeak is suppressed. Therefore, the volatile CNT transistor can emulate the neuronal leaky-integration function that corresponds to the frequency of the input stimulus. Additionally, the compact circuit model for the volatile CNT transistor is implemented (see also Fig. S2 and Supplementary Information Note 2 ). Different temporal responses in the tunneling/diffusion processes can be emulated with two resistor–capacitor circuits that have different time constants. As shown in Fig. 2c, e (yellow dotted curves), the circuit model can capture the measured results with high accuracy. This circuit model can be combined with the circuit model for other components in our tactile sensor system, which allows a quantitative analysis of the entire system through the circuit simulation (this concept will be discussed later). The operations of each component in our tactile sensor system will be discussed in the order of signal flow, wherein all signal flows are manipulated with custom-made software (the detailed experimental setup is presented Fig. S3 in Supplementary Information Note 3 ). First, the pressure stimulus was detected by a previously demonstrated tactile sensor device (Fig. 3a ) 26 that was fabricated with the use of the percolated solution that processed 99% of the metallic CNTs owing to their high bendability and material uniformity. The pressure stimulus leads to resistance changes in the tactile sensor (ΔRsensor) in the range of 2.5–10 MΩ (Fig. 3b ). The value of ΔRsensor is then converted to the frequency of the presynaptic spike (Vpre) by the VCO (Fig. 3c —the detailed design of the VCO is presented Fig. S4 in Supplementary Information Note 4 ). As the pressure intensity increases, Vpre oscillations are generated at an increasingly higher frequency (Fig. 3d ). Consequently, fpre varies proportionally to the pressure intensity (Fig. 3e ). Notably, because the VCO is designed based on a conventional digital circuit, the signal conversion (Rsensor versus fpre) can be characterized accurately by circuit simulations (Fig. 3e ). Fig. 3: The tactile seonsor device co-integrated with the VCO circuit. a Photograph of the tactile sensor device. b Resistance change in the tactile sensor (ΔRsensor) according to the pressure intensity. c Photograph of the VCO integrated on the printed circuit board. d The output of the VCO (Vpre) according to ΔRsensor. Higher pressure intensity leads to higher oscillation frequency fpre. e Variation in fpre as a function of Rsensor that shows the consistency with the simulation results obtained based on the circuit model of the VCO. Subsequently, the output of the VCO (Vpre) was delivered to the neuronal device, i.e., to the volatile CNT transistor (Fig. 4a ). Among our 10 × 10 CNT transistors 27 , 28 , one selected CNT transistor served as the neuronal device that operated in volatile mode. The other 10 × 4 CNT transistors acted as a synaptic network that operated in a nonvolatile mode to classify the pattern of input pressure (as discussed in a subsequent part of this section). When the pressure stimulus was detected by the tactile sensor, Vpre was applied to the gate electrode of the neuronal device, and a temporal ID enhancement and its subsequent decay were observed, as shown in Fig. 4b . Obviously, this transient ID behavior can be captured accurately through the circuit simulation from the combined VCO model and the volatile CNT transistor model discussed above. Moreover, the neuronal device can integrate temporally correlated tactile stimuli. As a proof of concept, two different tactile patterns in one row (convex or flat) were used as the target of recognition (Fig. 4c ), wherein we defined the convex pattern in the pair with the number “1” and the flat pattern with “0”. Therefore, the pattern pairs were labeled “’00”, “01”, “10”, and “11”. In the experiment, the tactile sensor was attached to a finger, and the finger was brought close to the patterns and was moved from left to right. A complete move/touch action was completed in ≤1 s. When the tactile sensor passes through the “1” pattern, the pressure stimulus at the sensor causes a temporal increase in ID. Moreover, the “0” pattern cannot cause any change in ID. Figure 4d shows the transient ID responses according to the different tactile patterns. Note that the “11” pattern leads to the largest ID change because this pattern provides two successive pressure stimuli. Interestingly, although both the “01” and “10” patterns have only one convex pattern, the timing information of the two patterns are different. The response to the “10” pattern decays earlier than the response to the “01” pattern. Therefore, the ID value attributed to the “01” pattern is higher than that attributed to the “10” pattern. Because the responses to the pattern pairs are distinguishable, these responses can be used as specific features for recognition. Fig. 4: Two different tactile patterns as the target of recognition. a Photograph of the entire tactile sensor system mounted on a hand in which the tactile sensor is attached to the fingertip and the VCO is attached to the back of the hand. The output of the VCO is transmitted to the CNT transistor array through the measurement equipment (see Supplementary Information Note 3 ). Among the 10×10 CNT transistors, one selected transistor serves as the neuronal device. The other 10×4 transistors serve as the synaptic network. b Measured ID response of the neuronal device according to the pressure stimuli, where the simulation obtained through the combined circuit model between the VCO and the neuronal device can accurately capture the measured data. c Photographs showing the pattern pairs and their corresponding two-bit binary code labels. d Typical ID responses of the neuronal devices for three types of pattern pairs. Finally, the biorealistic perceptual learning and recognition processes are demonstrated. In principle, the transient ID responses after the completion of the move/touch action are sampled. For example, ID is sampled after a period of 1 s at 0.1 s intervals (i.e., ID(ti) where i = 1 to m, Fig. 5a ). The input vector Ui is defined as the normalized ID(ti): Ui = ID(ti)/Iref, where 0 ≤ Ui ≤ 1. Next, the boundary vectors (Wj, where j = 1 to n) are defined as the references for the classification of Ui. In our experiment, four boundary vectors, namely, W1, W2, W3, and W4 (n = 4), correspond to the tactile patterns of “00”, “01”, “10”, and “11”, respectively (the dotted curves in Fig. 5a ). Based on the Euclidean distance between Ui and Wj, the tactile pattern that corresponds to Ui can be inferred. Therefore, two processes need to be performed iteratively: (1) the learning process for determining Wj with the labeled Ui and (2) the recognition process based on the calculation of the Euclidean distance between Ui and Wj. These learning and recognition processes are similar to the k-nearest neighbors (KNN) algorithm, and they can be performed efficiently with a resistive crossbar array 6 . Wj can be directly mapped as a set of weights in the nonvolatile CNT transistor array in a columnwise fashion, i.e., to the W matrix (W = [W1 W2 W3 W4]; thus, the size of W is m × n, Fig. 5b ). The weights are in turn linearly mapped to the device conductance values. Additionally, to calculate the Euclidean distance, the S matrix (of size 1 × n) should be included 6 . Thus, an (m + 1) × n nonvolatile CNT transistor array (referred to as the synaptic network wherein m = 9 and n = 4) was exploited in our experiment. Fig. 5: The learning and recognition processes in the CNT transistor array. a Example of boundary vectors (Wj) and the measured data. By calculating the Euclidean distance between the measured data and each boundary vector and by finding the minimum distance, the tactile pattern that generates the measured data can be inferred. b Mapping the process of the Euclidean distance calculation onto the 10×4 CNT transistor array (synaptic network). All the boundary vectors W1, W2, W3, and W4 can be directly mapped as weights in the 9×4 CNT transistor array (W matrix) in a columnwise fashion. Additionally, the 1×4s matrix is included in the synaptic network. The input vector is coded using pulses with different levels and is then applied to the drain of the CNT transistor in the row direction. The integrated currents at the column outputs allow direct comparisons of the Euclidean distances. The highest column current (∑IS,Cj) represents the shortest Euclidean distance between Ui and Wj. c Flowchart of the learning and recognition processes. d The measured evolution of the W matrix weights (i.e., device conductance of each CNT transistor) during the iterative learning process. Each column represents the corresponding boundary vector. e The recognition error rate as a function of the learning iterations. Each data point is the averaged error rate following 30 repetitions.