DEVELOPMENT OF HAPTIC ELECTROTACTILE RENDERING DEVICE: DESIGN IMPLEMENTATION AND TESTING By John W. Gregory A THESIS Submitted to Michigan State University in partial fulfillment of the requirements for the degree of MASTER OF SCIENCE Electrical Engineering 2011 ABSTRACT DEVELOPMENT OF HAPTIC ELECTROTACTILE RENDERING DEVICE: DESIGN IMPLEMENTATION AND TESTING By John William Gregory The purpose of this research is to build and test a haptic rendering system using skin surface electro-tactile stimulation for a fingertip interface. A new driver output stage for delivering tactile signals is implemented to improve power inefficiency and reduce circuit area relative to traditional open-loop constant current driver (CCD)-based haptic renders. This will be realized in a constant voltage driver (CVD) method for haptic rendering. The challenge will be to make the system aware of the skin impedance loading conditions and to use this information to introduce a closed-loop solution that is necessary due to implementing this type of driver. The system will be made load-aware by identifying parameters of a proposed electronic impedance model of skin that is done online and in real-time with customized software and hardware. Output sensors will be used to measure the voltage and current at the loaded output. An extended least squares algorithm will identify the parameters from the measurements. The paper will introduce a model reference adaptive control law, using the direct method, with nonlinear feedback to test a closed-loop CVD design. To aid in determining normal operating conditions, and validating parameter estimation and simulated closed-loop control, related experiments will be carried out. This work is based on an older version electro-tactile haptic rendering system used in the laboratory. ACKNOWLGEMENTS A small feat dedicated to the giant character of my father, William L. Gregory, whose death eventually motivated me to start building my own for the better, and continues still, with the aid and comfort of my lovely wife, Tracy M. Gregory. I would also like to thank Drs. Ning Xi and Yantao Shen in their support for this work, as well as, for National Science Foundation and the Army Research Lab for their generous funding of this project. iii TABLE OF CONTENS LIST OF TABLES ........................................................................................................... VI LIST OF FIGURES ........................................................................................................ VII 1. INTRODUCTION ......................................................................................................... 1 1.1 BACKGROUND AND MOTIVATION .............................................................. 1 1.2 RELVAVENT BIOLOGY OF RECEPTORS CONCERNING ELECTRONIC STIMULATION OF THE FINGERTIP ................................................................... 2 1.3 TYPES OF DISPLAYS, SENSORY CONTENT, AND MECHANISMS ........... 4 1.4 ORGANIZATION OF THESIS ........................................................................ 7 2. LITERATURE REVIEW ............................................................................................... 8 2.1 ELECTRICAL IMPEDANCE PROPERTIES AND CHARACTERSTIC OF SKIN ..................................................................................................................... 8 2.2 TACTILE INTENSITY AND PERCEPTION .................................................. 12 2.3 TYPES OF ELECTROTACTILE DRIVERS .................................................. 16 2.4 DESIGN CHALLENGES WITH EXISTING OUTPUT DRIVER METHODS .. 18 3. SYSTEM DESIGN ..................................................................................................... 23 4. HARDWARE, SOFTWARE, AND SYSTEM CALIBRATION ..................................... 25 4.1 HARDWARE DESIGN .................................................................................. 27 4.1.1 DIGITAL ACQUISITION (DAQ) BOARD ........................................................... 27 4.1.2 HIGH VOLTAGE DC-TO-DC CONVERTER POWER UNIT ............................. 27 4.1.3 CONSTANT VOLTAGE DRIVER (CVD) ........................................................... 31 4.1.4 CVD OUTPUT VOLTAGE SENSOR ................................................................. 33 4.1.5 CVD OUTPUT CURRENT SENSOR ................................................................ 35 4.1.6 ELECTRO-TACTILE DISPLAY HARDWARE DESIGN..................................... 43 4.2 SOFTWARE DESIGN .................................................................................. 46 4.2.1 GUI FRONT END AND RESULTS GRAPHICAL DISPLAY .............................. 47 4.2.2 SOFTWARE DESIGN AND PROGRAM FLOW ................................................ 51 4.2.3 SOFTWARE WAVEFORM GENERATOR AND TYPICAL WAVEFORM PRIMITIVES............................................................................................................... 57 4.3 HARDWARE CALIBRATION ........................................................................ 59 5. MODELING AND ANALYSIS OF ADAPTATION LAW .............................................. 66 5.1 CONTINUOUS TIME BIO-IMPEDANCE (CTBI) MODELING ....................... 66 5.2 DISCRETE TIME CONDUCTANCE (DTC) MODELING .............................. 72 5.2.1 DTC MODEL PARAMETER ESTIMATION AND ADAPTATION ALGORITHM IN Z-DOMAIN ................................................................................................................. 73 5.2.2 ITERATIVE EXTENDED LEAST SQUARES WITH FORGETTING FACTOR ADJUSTMENT (ELSFF) MODEL ALGORITHM ........................................................ 75 iv 5.3 PARAMETER ESTIMATION FOR THE CONTINUOUS TIME LINEAR PARAMETERS BIO-IMPEDANCE (CT-LPBI) MODEL ...................................... 82 6. EXPERIMENTAL SETUP AND TESTING................................................................. 86 6.1 HARDWARE AND ADAPTATION ALGORITHM TESTING.......................... 86 6.1.1 EXPERMINETAL SETUP ................................................................................. 86 6.1.2 EXPERMINETAL RESULTS AND ANALYSIS .................................................. 89 6.2 EXPERIMENTATION ................................................................................... 99 6.2.1 EXPERIMENTAL SETUP ............................................................................... 100 6.2.2 EXPERIMENTAL RESULTS USING STATISTICAL ANALYSIS .................... 105 6.2.3 ELECTRO-TACTILE DYNAMIC RANGE STUDY ........................................... 106 6.2.4 CVD POWER OUTPUT STUDY ..................................................................... 115 6.2.5 STATISTICAL STUDY BETWEEN ESTIMATION MODELS AND PLANT OUTPUT ESTIMATION ERROR ............................................................................. 118 6.2.6 OBSERVATIONAL RESULTS FOR LARGE SIGNAL I-V CHARACTERISTIC ................................................................................................................................. 124 6.2.7 STATISTICAL RESULTS FOR LARGE SIGNAL I-V CHARACTERISTIC AND LARGE SIGNAL MODELING................................................................................... 131 6.2.8 OUTPUT ESTIMATION UNDER TIME VARIATION STUDY .......................... 139 6.2.9 ESTIMATION MODEL OUTPUT AS A FUNCTION OF CVD WAVEFORM TYPE STUDY ..................................................................................................................... 145 7. ADAPTIVE CONTROL MODELING, SIMULATION AND ANALYSIS ..................... 160 7.1 MODEL ....................................................................................................... 160 7.2 STOCHASTIC PARAMETER IDENTIFICATION FROM POPULATION DATA ......................................................................................................................... 162 7.3 PROPOSED BIO-IMPEDANCE DYNAMICS MODEL ................................ 164 7.4 DERIVATION OF ADAPTIVE CONTROL LAW .......................................... 165 7.5 SIMULATION OF THE BIO-IMPEDANCE MODEL (OPEN-LOOP) ............ 167 7.6. SIMULATION ON CLOSED LOOP STIMULATION CURRENT CONTROL ......................................................................................................................... 173 8. CONCLUSIONS AND FUTURE WORK .................................................................. 178 9. REFERENCES ........................................................................................................ 181 v LIST OF TABLES Table 1. Bio-impedance parameters for 10 individuals using offline identification [34]: dry and damp Index fingertip case. ............................................................................... 88 Table 2. Known RC circuit elements of impedance networks built on breadboard with known analog pole and zero frequency locations used in online identification testing: built using table 1 parameter sets. ................................................................................ 89 Table 3. Resulting online estimation of circuit element parameters and analog pole and zero frequency locations for the corresponding known parameters of the plants in table 2. ................................................................................................................................... 90 Table 4. Absolute error and standard deviation between plant and estimated parameters: first column shows which variable held constant for calculating the associated row statistics. ............................................................................................... 91 Table 5. Pain threshold sensation description reported from subjects and frequency of occurrence for the three CVD waveform types. ........................................................... 115 Table 6. Two sample KS test for similarity between ECDFs of apparent power distributions for CVD waveform type (5% Significance): samples of apparent power for each waveform type are calculated at sensation and pain threshold for each subject (N=21). ........................................................................................................................ 118 Table 7. Experimentally estimated bio-impedance parameters for subject #2 with square wave voltage pulse amplitude conditions shown ......................................................... 168 vi LIST OF FIGURES Figure 1. Illustration of fingertip electro-tactile stimulation via an electrode array. ......... 1 Figure 2. Improved Howland current pump with master-slave configuration adapted from [3]. Rs = 1 kΩ, R1 = R3 = 5 kΩ, R2 = 10 kΩ, R4 = 9 kΩ, and ZL is load impedance. .... 20 Figure 3. System level diagram of implemented project. ............................................... 26 Figure 4. High voltage DC-to-DC converter circuit. ....................................................... 30 Figure 5. Constant voltage driver (CVD) circuit. ............................................................ 33 Figure 6. CVD voltage output sensor circuit. ................................................................. 35 Figure 7.CVD current output sensor circuit. .................................................................. 37 Figure 8. Tactile display: dimensions are 10.16x7.62x2.54 centimeters with an active area of 2.74x2.31 centimeters. ...................................................................................... 45 Figure 9. Ground-isolated return electrode.................................................................... 46 Figure 10. GUI front end................................................................................................ 48 Figure 11. Identification results dialogue. ...................................................................... 50 Figure 12. Program flow control: top level. .................................................................... 52 Figure 13. Flow control for D/A output thread and DAQ shared memory write. ............. 53 Figure 14. Flow control for D/A output thread and DAQ shared memory read. ............. 54 vii Figure 15. Flow control for initiating or stopping A/D and identification threads along with identification shared Memory read from results display thread. .................................... 55 Figure 16. Flow control, shared memory access and data accessing for online identification thread. ...................................................................................................... 56 Figure 17. CVD output pulses generated from software waveform primitives utilized in testing tactile display. .................................................................................................... 58 Figure 18. Calibration of the CVD input-output voltage amplitudes. .............................. 61 Figure 19. Calibration of the load voltage sensor. ......................................................... 63 Figure 20. Calibration of load current sensing circuit. ................................................... 65 Figure 21. Continuous time bio-impedance (CTBI) model of the fingertip under electrotactile stimulation. .......................................................................................................... 68 Figure 22. Overview of parameter identification. ........................................................... 74 Figure 23. Top level block diagram of parameter estimation using iterative ELS algorithm in Z-Domain. .................................................................................................. 77 Figure 24. Measured and decomposition of estimated output current from terms in (5.4a) plotted by driving load #5-Damp in table 2: output for one triangle wave CVD pulse shown at 60 Hz, 11% duty cycle at Vpp = 140 volts, λ = 1. ................................. 94 Figure 25. 10 second estimation of measured state current at pulse midpoints using model (5.4a). Results produced from square wave pulses driven under load #1-Dry in table 2. The mth step is the mth duty cycle pulse midpoint: output is at 60 Hz, 11 percent duty cycle at Vpp = 100 volts, λ = 1. ................................................................. 96 viii Figure 26. Typical output estimation error convergence characteristic versus time calculated from (5.21) when driving LTI loads in table 2: assessed for #5-Damp impedance load. ............................................................................................................ 98 Figure 27. View of user forearm resting comfortably on gel pad while index fingertip is placed firmly on tactile display for stimulation and measurement. ............................... 102 Figure 28. View of anodic return electrode proper placement secured with a Velcro strap on thenar eminence..................................................................................................... 103 Figure 29. Mean dynamic range ratio for RMS stimulation current for each type of CVD output waveform pulse (N=21). The 95% confidence interval is included. The sample mean and standard deviation for the square saw tooth, and triangle wave are: M = 4.78, SD = 1.86; M = 7.21, SD = 2.69; M = 7.29, SD = 2.74, respectively. .......................... 107 Figure 30. Average sensation and pain thresholds for stimulation currents versus wave form (N=21). The 95% confidence interval is shown for the mean. ............................. 109 Figure 31. Five statistic box and whiskers plot for mean dynamic range ratios for RMS stimulation current versus waveform shape (N = 21). ................................................. 111 Figure 32. Five statistic box and whiskers plot for mean sensation and pain thresholds concerning RMS stimulation current versus waveform shape (N = 21). ...................... 112 Figure 33. Mean CVD Voltage Amplitudes for corresponding RMS current thresholds versus CVD pulse type (N = 21). ................................................................................. 113 Figure 34. Apparent power bar chart shown at mean sensation and pain stimulation current thresholds for each individual (N = 21) and each CVD waveform type: standard error bars are shown. .................................................................................................. 117 Figure 35. Modeling error given by power inefficiency. Percentages of average power consumed in the CED filter normalized to average power estimated in the DTC model for all waveform types calculated at reported threshold levels: results graphed at the sensation and pain thresholds of subjects (N=21)....................................................... 122 ix Figure 36. Mean power efficiency of DTC model estimation output normalized to measured power delivered to skin-tactor load for each waveform type: results calculated at the reported sensation and pain thresholds of subjects (N = 21). ........................... 123 Figure 37. Mean power efficiency of physical impedance (CT-LPBI) model estimation output normalized to measured power delivered to skin-tactor load for each waveform type: results calculated at the reported sensation and pain thresholds of subjects (N = 21). .............................................................................................................................. 124 Figure 38. Identification of Rp and Rs as a function of stimulation current and subject (N=21): results are from square wave CVD pulses starting at Vpp = 30V and incremented to pain threshold for each subject. .......................................................... 126 Figure 39. Measured and DTC estimation model (from 5.4a) I-V characteristic curves for each subject as a function of square wave CVD pulse amplitude and subject (N=21). .................................................................................................................................... 128 Figure 40. Identification of Cp as a function of stimulation current and subject (N=21): results are from square wave CVD pulses starting at Vpp = 30V and incremented to pain threshold for each subject. .................................................................................. 130 Figure 41. Five statistic box and whiskers plot produced from each average Rs value per individual from 30 volts to maximum pain threshold voltage (N = 21). .................. 132 Figure 42. Scatter plot of all identified Rs values versus stimulation current independent of subject (N=278): results generated from applied square wave pulses from 30 volts to maximum pain threshold voltage per individual. .......................................................... 133 Figure 43. Scatter plot of identified Rp values for sample population (N = 278) as a function of stimulation current with power-fitted regression curve using square wave CVD pulse: lower 5th and upper 95th percentile confidence intervals are shown for identified Rp calculated using the mean of the Rp power-fitted regression curves for each individual (shown as the dashed line). ................................................................ 135 x Figure 44. Histogram of percent error between the identified Rp and Interpolated Rp values using square wave CVD pulse (N = 278). ........................................................ 136 Figure 45. Scatter plot of measured CVD Vpp for sample population (N = 278) as a function of stimulation current with log-fitted regression curve using square wave CVD pulse: lower 5th and upper 95th percentile confidence intervals are shown for measured CVD Vpp calculated using the mean of the Vpp log-fitted regression curves for each individual (shown as the dashed line).......................................................................... 138 Figure 46. Histogram of percent error between the measured CVD and interpolated CVD amplitude values using square wave CVD pulse (n = 278)................................. 139 Figure 47. Output estimation of measured stimulation current for human subject using DTC model in (5.4a): square wave CVD pulses with λ = 1.0, time = 10 sec, and Vpp = 90V. ............................................................................................................................. 142 Figure 48. Output estimation of measured stimulation current for human subject using DTC model in (5.4a): square wave CVD pulses with λ = 0.9975, recording time = 10 sec, and Vpp = 90V. .................................................................................................... 144 Figure 49. One of 60 square wave pulse of measured and estimated model output current, using models in (5.3) and (5.4a), for CVD square wave pulse train administered at reported pain threshold for subject #4: Vpp = 125V, λ = 0.99975 and recording time = 1 sec............................................................................................................................ 148 Figure 50. Estimated Rp, Rs, and Cp as a function of time and square wave CVD pulses for subject #4: Vpp = 125 volts, λ = 0.99975 and recording time = 1 sec. ........ 150 Figure 51. One of 60 saw tooth wave pulse of measured and estimated model output current, using models in (5.3) and (5.4a), for CVD saw tooth wave pulse train administered at reported pain threshold for subject #2: Vpp = 135V, λ = 0.99975 and recording time = 1 sec. ................................................................................................ 152 Figure 52. Estimated Rp, Rs, and Cp as a function of time and saw tooth wave CVD pulses for subject #2: Vpp = 135 volts, λ = 0.99975 and recording time = 1 sec. ........ 154 xi Figure 53. One of 60 triangle wave pulses of measured and estimated model output current, using models in (5.3) and (5.4a), for CVD triangle wave pulse train administered at reported pain threshold for subject #20: Vpp = 142V, λ = 0.99975 and recording time = 1 sec. ....................................................................................................................... 156 Figure 54. Estimated Rp, Rs, and Cp as a function of time and triangle wave CVD pulses for subject #20: Vpp = 142 volts, λ = 0.99975 and recording time = 1 sec. ...... 158 Figure 55. Illustration of bio-impedance model of skin and frequency model for constant voltage pulse driver (CVD) .......................................................................................... 161 Figure 56. Large-signal stochastic representation of Rp and A and B power law coefficients for sample population (N = 21) versus RMS stimulation current for 60 Hz, 5% duty cycle square wave pulses.............................................................................. 163 Figure 57. The mean of the average values of Rs for each member of the population 164 Figure 58. Measured and estimated output current pulse shape to subject #2 at 150 volt square wave pulse; measured and estimated Istim at steady state equals 0.63 mA .. 169 Figure 59. Simulated current pulse from square wave voltage pulses inputted into equations 7.2a- 7.2b using experimental data in Table 7 for one subject; Istim steady state equals 0.5 mA..................................................................................................... 171 Figure 60. Simulated value of Rp stratum corneum resistance (Ohms) using equation 7.2b during the current pulse time in Figure 59; the minimum of Rp equals 231 kΩ ... 172 Figure 61: Reference Current ...................................................................................... 174 Figure 62. Simulation results: Tracked stimulation current. ......................................... 175 Figure 63. Simulation results: behavior of Rp. ............................................................. 176 Figure 64. Simulation Results: Voltage output of closed loop CVD. ............................ 177 xii 1. INTRODUCTION 1.1 BACKGROUND AND MOTIVATION Electrodes designed to substitute touch sensation have been performed on various locations on the body. For example, the tongue [16] [17], forearm [20], abdomen [12] [15], and fingertip [9]-[14] [24] are areas most studied to assess the ability of human subjects to detect and identify patterns using these devices. Research in this area has shown that electrical stimuli can effectively deliver a significant amount of information about patterns and physical properties of stimuli such as texture, geometry, contour, movement, pressure, elasticity, etc; thus, ample research supports the notion that electrical stimuli are a viable means for delivering sensory information [4]. Our research interest pertains to investigation of the ability to induce touch sensations on the fingertip skin using an electrical array. This mechanism, also called electro-tactile or electrocutaneous stimulation is illustrated in Figure 1. Figure 1. Illustration of fingertip electro-tactile stimulation via an electrode array. 1 1.2 RELVAVENT BIOLOGY OF RECEPTORS CONCERNING ELECTRONIC STIMULATION OF THE FINGERTIP The fingertip skin consists of several horizontal layers of skin each consisting of receptors: four classes of mechanoreceptors, two classes of thermo-receptors, four classes of nociceptors, and three classes of proprioceptors. All are found within these layers of the skin. The four classes of mechanoreceptors are located in the first three layers under the stratum corneum, which are the epidermis, dermal and subcutis (connective) tissue layers [22]. To an extent, all provide the perceived tactile sensations during mechanical or electrical stimulation [3] [4] [14] [21] [22] [24]. Different receptors sense different tactile modalities such as pressure, vibration, temperature or pain to name a few. Especially, when considering the different classes of mechanoreceptors, the most commonly investigated tactile receptors include: Merkel cells that encode pressure sensation and edge detection over a small distance; Meissner corpuscle that encode low frequency vibration, touch, changes in pressure and motion over small regions on the skin; and Pacinian corpuscle that register high frequency vibration and changes in shear stretching on the skin over large distances. A fourth type not often discussed as the previous are the Ruffini endings that encode static shear stretching on the skin over large distances [4] [14] [22]. The mechanoreceptor afferents can be categorized by their orientation, depth, innervation densities, adaptation, receptive field and frequency response. In terms of innervation density and receptive field, Merkel and Meisner Corpuscles have a higher 2 density and smaller receptive field in the hand and fingers than compared to both Pacinian Corpuscles and Ruffini endings [22]. This suggests that the first two types of receptors are responsive to finer details in stimuli, in contrast to Pacinian and Ruffini afferents which are fewer and have a much larger, less well-defined boundary for their detection region. As well, the action potential firing characteristic from an applied mechanical deformation to the skin, such as a ramped square wave, differ for the classes of receptors [7]. The Merkel and Ruffini receptors are referred to as slow adapting one and two (SA-I and SA-II), respectively, since their action potentials fire continually while the deformation is present. Alternatively, the Meisner and Pacinian receptors, classified as fast adapting one and two (FA-I and FA-II), respectively, fire only during the application and removal from the mechanical indentation. These adaptation schemes along with the different receptive fields to an extent define the mechanoreceptors‟ roles in encoding the tactile dynamics on the skin that are sent to and interpreted by somatosensory areas in the brain to elicit perception. It is noteworthy that the classes of mechanoreceptors are not mutually exclusive to their primary function since the mechanical detection thresholds of each versus their frequency response overlap [24]. Hence, temporal integration of the induced action potentials by mechanical stimuli, for instance by scanning the fingertip over a fine texture, probably lends to a richer encoding scheme, such as frequency encoding, for higher-level functions in the brain to process. 3 Several research findings in the areas of psychophysics, neurocytology, electrochemistry, and cognitive science have shown that mechanical or electrical stimulations to these mechanoreceptors produce or mimic tactile feelings in humans [7] [13] [18] [19] [25] [26]. Following these explorations, extensive research in rehabilitation engineering has been performed to elucidate the properties, characteristics, and mechanisms underlying the sensation of touch delivered through electrode arrays on the fingertip skin [10]-[14] [24]. As a result, electro-tactile research and development have been driving the potential to advance assistive, diagnostic, and rehabilitative devices such as Braille readers, sensory substitution systems, teleoperation and telepresence, and computer games [1] [2] [5] [6] [9] [10] [20] [23] [24]. 1.3 TYPES OF DISPLAYS, SENSORY CONTENT, AND MECHANISMS Broadly, a haptic renderer senses tactile information originating from a real or virtualmodeled source and encodes it to be delivered to the touch sense via the user interfacing with a peripheral device called a tactile display. One class of haptic renderers are used to display tactile information encoded from a virtual world, such as in computer gaming or texture patterns, or as a tactile feedback display in teleoperation and telepresence applications [23] [20]. These devices are meant to allow the user to be knowledgeable about or allow for better interaction with the physical or media construct in question using relatively high resolution tactile displays. Another classification of sensory substitution haptic displays have a long tradition and include applications for visual or auditory substitution, such as Braille readers, acoustic encoders in the frequency domain [12], or tactile vision substitution systems (TVSS) [16] [17]. Here, 4 sensory information that is intended for a particular sense is measured and inputted in to the haptic renderer for re-encoding as output to the tactile display for recognition by the sense of touch. The principle relies on neuroplasticity of the brain which, in this case, is the ability of the somatosensory cortical area to interface with and transmit the relevant information to the cortical area(s) responsible for processing the sensory information that is now substituted for, as studied by Bach-y-Rita in the 1960‟s. A third class of applications for haptic renders involves information delivery systems which allow the user to process media-rich content through the sense of touch. One such system discussed in [6] is a donned vest with arrays of vibrating tactors used as a navigation-assistance system that delivered tactically-encoded orientation and speed content for the user in his surrounding environment. The last classification for haptic rendering applications, in the area rehabilitative engineering, concerns sensory feedback integration in and neuromuscular activation of prostheses [12] [29]. Whereas the origins of content or media amenable for haptic encoding are nearly endless to exploit, the traditional means of delivery to the tactile sense are specific. The means for traditional tactile interfaces are accomplished using either servo driven pins, called vibrotactile displays, or electrode stimulators, called electrotactile displays that are applied to various surface regions of the skin [4]. Each type has its advantages and disadvantages. For instance, the vibrotactile displays can control pressure and vibration sensation intensity and gradients delivered to the cutaneous area directly by current spatial configuration and frequency modulation of its pins. However, tactile information conveyed electronically seems more complex since the perceived intensities 5 of rudimentary sensations, such as pressure and vibration must be coded indirectly to elicit a desired mechanical sensation; thus, attempting to replicate these physical sensation and intensities is done using amplitude, pulse width or frequency modulation of the electronic signals [12]. They are also more difficult to provide a stable sensation level due to coupling effects between the electro-chemistry of the skin locus under stimulation and amplitude and modulation properties of the signals [29] [32]; whereas mechanical stimulation does not change drastically the inherit mechanical properties of the cutaneous region interfaced. However, electrotactile terminals are amenable to compact construction, improved compliance to the skin, finer tactor pitch, and are more energy efficiently versus mechanical displays due to their lack of moving parts. A recent and emerging class of haptic rendering devices uses smart materials as actuators to build the tactile display tactor elements. For instance, research into high density tactile displays utilizing dielectric electro-active polymers (D-EAPs) are discussed in [8]. These devices have the potential for being compact in size and possess a dense active area of stimulators, with substrates that are mechanically compliant to the skin. They can also be made energy efficient, while delivering mechanical stimulation to the skin surface. In the case of D-EAPs, miniature diaphragms are used as actuators against the skin. Unfortunately, acrylics or silicone polymers used for this purpose requires voltages in excess of 1 kV to actuate, with low current levels compared to the mechanical pin actuators. 6 1.4 ORGANIZATION OF THESIS In section two, the electronic impedance properties of dermal skin, and tactile perceptive parameters and sensation control of the tactile sense are overviewed. A brief survey of previous haptic designs employed by others using electrotactile haptic systems is presented along with some of the design challenges and possible solutions; the overview for a new electrotactile driver design is given in Section 3. As well, this laboratory‟s specific applications for the original, fully-implemented haptic display system are presented; section 4 details the hardware and software implementation, and output signals used for experimentation by haptic rendering system. Additionally, the last subsection in section 4 goes over the method for calibrating the driver output and measurement circuitry; in section 5, biologically-inspired formulation of electronic impedance models for skin surfaces under electronic stimulation is introduced and analyzed. The last part of the section is devoted for elaborating the real-time, online adaptation law used to identify the parameters for the models; section 6 is the start of the experimental section to test the hardware and software, validate the proposed models and test the adaptation law. The first major subsection explains the experimental setup to test the haptic system and validate the load identification algorithm which is then carried out; in the second major part of this section, the setup and motivations for human experimentation are explained and the types of experiments carried out presented. The experimental results and analysis are given in this section, as well; in Section 7, the adaptive control rule for controlling driver output to an electrode-skin interface is derived based on experimental and theoretical findings. Simulations of the indirect MRAC law are provided for demonstration of the control; 7 concluding remarks about the results and future work are given in Section 8; finally, all references used in this work are given in Section 9. 2. LITERATURE REVIEW 2.1 ELECTRICAL IMPEDANCE PROPERTIES AND CHARACTERSTIC OF SKIN The dead, multi-layered protective barrier of skin that is exposed to the environment is known as the stratum corneum (SC). The SC cells are densely packed lattice of layered corneocytes composed of relatively high resistivity keratin (pk = 105 Ωm at DC) compared to other tissues in the body, such as muscle, bone and underlying dermal tissue which averages pc = 255 Ωm [39]; therefore, it is considered the main insulative barrier to transcutaneous current flow. Cellular membranes that enveloped corneocytes are electrically insulative and composed of dielectric lipid bilayers. The intercellular spacing is heterogeneous consisting of in-series and stacked (approx. 100) dielectric laminar lipid bilayers, perforated with pores and sweat gland pathways in an ionic aqueous fluid [32]. Hence, there are at least two distinct types of electrical conductance through the skin in reference to the SC model presented in [27]: 1) the intercellular route, which is mainly due to ionic conduction in aqueous electrolyte whose flow is dependent on applied voltages magnitudes [32]; 2) the transcellular pathway formed between SC mater, dielectric lipid bilayers, and extracellular ionic fluid which can be modeled to form a capacitive element for displacement current [32]. The complex electro-physiologic makeup mean that electronic bulk impedance of SC skin exhibits nonlinearities dependent on waveform stimulation dynamics and time history, frequency 8 dependency, spatial and time variation. For square wave electronic pulses, empirical evidence indicates that the electronic resistivity of the SC layer decreases almost inversely with increasing current amplitude [3] [4] [12] [29]. SC thickness and moisture content also tend to increase and decrease skin bulk resistance, respectively [34] [39]; the electrical resistance and capacitance of the skin are extrinsic properties; hence, skin impedance is dependent on effective electrode area against the stimulated skin surface which in turn depends on applied force between skin-electrode interfacing. In contrast, the resistivity and non-linear dynamics of subdermal skin tissue and other soft tissues is shown to be mostly invariant of frequency and amplitude (i.e. it exhibits an ohmic relationship) and is much lower and constant spatially in value compared to the SC [29] [30] [39] [43]. For related statistical comparison studies of resistivities and moisture content of various organs and tissues, including skin, as a function of frequency, see [43]. An observed effect in SC electrical impedance under square wave stimulation for current amplitudes greater than 2 mA is a recovery in skin resistance over time during the pulse. This second order phenomenon becomes more dramatic as current amplitudes are increased. The illustrations of the effect are shown in [3] and [29]. The researchers in [3] demonstrated that by applying square wave current pulses of increasing magnitude to the skin surfaces of participants, specific resistance of the epidermis decreased as a power function of current density with an exponent of 0.80. Extrapolating the authors‟ relationship by normalizing their results to a 1 mm 2 electrode area, resistance is predicted to be 5 MΩ at 50 µA down to about 100 kΩ at 1mA. Investigators in [39] determined an exponential law inversely relating DC skin resistivity 9 to skin depth by experimentally determining the resistivity distribution by successively stripping away SC layers. As well, skin impedance measured at high frequency stimulation of unmodified SC asymptotically approached low frequency impedance of completely stripped SC. The authors of [32] state that effective shunting of each stacked lipid layer in the SC totaling about 100 occurs from electrical field induced formation of aqueous pores leading to increased ionic conduction in each layer; the number of effectively shorted SC layers was indicated as a linear function of voltage amplitude. The studies in [32] and [39] could indicate the physiological reason for the inverse functional relationship between SC bulk impedance and pulsed current amplitude The investigators of [30] studied the resistance of un-cleansed, dry, and un-abraded epidermal skin using a 0.550 cm2 electrode stimulator as function of sinusoidal frequency from 1 Hz to 1 MHz and held independent of current amplitude by applying current in the microamperes; multiple subjects and body locations were used for testing. If the electrode geometry used in the study is normalized to a 1 mm2 area in this review, then impedances ranged from a mean of 10 MΩ at 1 Hz down to 12 kΩ at 1 MHz for various subjects and body locations. Interestingly, findings showed a decrease in the variance of impedance versus subject and body location as frequency increased that was attributed to increased capacitive conduction at higher AC frequencies; hence, the variation of real impedance due to differing SC skin thicknesses are not as apparent at higher frequencies as should be expected. The residual measurable resistance at high frequency is likely from deep tissue which is modeled in series with the SC resistive and capacitive modeled impedance in parallel. 10 A physiologically-inspired model used to predict time hysteresis in SC skin impedance from intra-modulated square wave current pulses is investigated in [29]; however, the electrode areas, stimulation current amplitudes, and pulse widths used in this work were larger than ones used for typical tactile information delivery. The psyological or electrochemical changes to the SC structure over time that can occur from electronic pulsing include: the electrolytic effect, ion migration, and spontaneous (with or without recovery) generation of pores [32]. The later is called electroporation by the same authors. These effects cause the skin bulk impedance to exhibit time variation, usually having the effect of lowering impedance magnitudes over time; in general higher magnitudes of stimulation usually result in more drastic time variation and longer lasting effects. Investigators in [32] observed a marked reduction in barrier resistance with high voltage pulses applied to the SC with the recovery time after stimulation (if any) dependent on amplitude and pulse history. Hence, the skin impedance exhibits remarkable hysteretic dependency on pulsing history. Since electrotactile displays are usually a matrix of spaced electrodes, spatial variation in impedance is a function of localized epidermal morphology, such as thickness, porous regions, and hydration levels. As such, the rate of time variation changes according to electrode stimulation position. Localized hydration concentration of the epidermal layer may change over time due to biofeedback or sweat buildup due to electroporation and ion migration at the stimulation interface. The thickness of the SC shows slight variation in individuals and, more significantly, relative to specific body 11 locations as observed from population studies from [40] and [41]. Susceptibility to changes in hydration-to-mass percentage of SC was investigated with Raman spectroscopy in [41]. The lower granular layer directly underneath the SC, in contrast, is hydrated homogeneously and with a high ratio showing little variation between individual and body location [41] (indicated by Raman spectroscopy results) and has low electrical resistance [39]. For skin bulk resistance and capacitance measurements as a function of dampness from applying electronic pulsing using skin surface electrodes, see the results in [34]. Finally, the distribution of natural pores over the skin does not seem to be uniform as is indicated by studies [31]. Pores in general, lead to marked reduction in impedance when stimulated and can act as positive feedback for current density leading to stinging sensations. 2.2 TACTILE INTENSITY AND PERCEPTION Coding for tactile sensation is based on measured or created reference stimuli and is done by tailoring the spatial configuration and temporal properties of the stimulating waveforms. In turn, the delivered waveform content to the tactile display should emulate the reference when interfaced. For stimulators using electronic actuation, transcutaneous current is thought to directly stimulate the afferent nerve fibers in the dermis. In relation to time, one of the most relevant parameters for predicting perceived intensity of stimulation is found by the time integral of delivered charge which is discussed and illustrated in [4]. The amount of delivered charge versus intensity perception relationship is non-linear and increases monotonically. Because of this wellestablished phenomenon, square-wave pulses are traditionally a popular method for 12 encoding tactile sensation. In terms of perceived intensity from encoded stimuli, the square wave signal can be coded by modulating the electronic pulse amplitude, width, or burst rate per cycle, as is explained in [12]. An experiment relating the sensation of intensity of physical roughness to electronic current pulse width and amplitude modulation is carried out in [20]. The quantification of just noticeable detection levels (JNDs) that relates gradations in perceived intensity to that of the intensity range of the reference stimuli has also been studied using mechanical and electronic stimulators for the sense of touch. Similar to visual and auditory senses, the JND for skin under tactile stimulation seems to follow either a Weber‟s or Steven‟s power law in terms of amplitude or pulse width modulation [22]. The JND levels can be controlled to an extent by manipulating the same waveform parameters as discussed above. Further the number of differentiated JND levels increases with the fundamental frequency of the stimulating waveform. The tactile sense has both time and spatial resolution. It has limited resolution for both when consideration of minimum pulse dead-time and spatial separation for two-point discrimination for the CNS to discern new tactile content. JND resolution is based on the fundamental frequency with which square wave electronic pulses are delivered to a single contact point on the skin. The optimal JND frequency resolution is said to fall between 1 and 100 Hz. At 100 Hz, the number of identifiable JND levels will not follow the increasing trend versus frequency found for frequencies below 100 HZ [12]. Additionally, waveform parameters can be set to target mechanoreceptors to evoke specific sensations to the user such as pressure, vibration, or softness by combined 13 coding of spatial, temporal and fundamental frequency of square wave signals, as was theorized and demonstrated in [24]. An example of spatial coding using two spaced electrodes involves their respective current magnitudes that average together to form an intermediate location for a new phantom point where the stimulation is perceived and is discussed in [12]. Papers elucidating typical experiments in pattern recognition, directionality, and tactile resolution of skin using electronic or mechanical stimulation can be found in [17], [20], [42] and [7]. Dynamic range of sensation for tactile perception is defined as the ratio of pain threshold over the initial registration as stimulation intensity is increased using some consistent coding scheme. The metric is a psychophysically assessed measurement for an individual. The ratio is shown to increase as the user habituates to the applied waveform pulses over time or with the amount of experience the user has on the stimulator device [4]. The voltage dynamic range of the driver to achieve a given maximum current magnitude scales inversely with the effective electrode-skin contact area [3] and hydration level [34] of the epidermis since skin resistance likewise under these conditions. However, there is a tradeoff between a desire to decrease the electrode area and increase the pain sensation threshold, and dynamic range of sensation in general, since current densities will also increase with smaller electrodes. According to [4], pain sensation threshold lowers with increasing current density; however, electrodes larger than a given area can also cause unwanted concentration of current density. Larger areas can increases the probability that low resistance pores or 14 areas around hair follicles, which are relatively highly conducive, could shunt current and cause a runaway condition at a point [12]. Electronic stimulators can be configured to generate either biphasic or monophasic square wave pulses where the phase is the polarity of the voltage signal referenced to the return electrode. The associated current signal outputs can have a DC component or be configured as pure AC in either case. If monophasic pulses configured as pure AC signals are used, then a small DC voltage offset of opposite polarity in the inactive pulse region would be required. A biphasic voltage pulse is mainly used to avoid electrobiological alterations to stimulated skin areas that would occur from net DC currents. Current Pulses applied that are pure AC and for charge transfers much less than 4 µC/mm2 do not produce convective migration of ions, electrolysis of water in the tissue, or non-native injection of ions in to cutaneous tissue [4] [12] [32]. Interestingly, monophasic pulses exhibit longer adaptation times versus biphasic pulses [12]. Biphasic electronic pulses usually include a transition time duration in-between the change in polarity on the order of microseconds. The short duration is needed to allow time for charge accumulation to initiate action potential firings by depolarizing the targeted nerve endings before changing polarity [12]. This helps to set the intended sensation intensity. Studies have shown that electrode pulse polarity and also the spatial superposition of nearby electrode polarities can influence the comfort and coded perception by targeting and isolating specific classes of mechanoreceptors [24], dynamic range threshold of sensation [4], and lasting alterations in skin biology from the stimulation pulses. 15 2.3 TYPES OF ELECTROTACTILE DRIVERS The stimulator topologies used for electrotactile haptic rendering are prevalently openloop constant current driver (CCDs) as seen in literature reviews. CCDs deliver and maintain constant current square wave pulses to an unknown and potentially changing bio-impedance load and are modeled as current sources. The CCD circuitry design and output stimulation signals are dictated by how where on the human body the tactile display is interfaced. For instance, electrodes for the display can be implanted subcutaneously or placed on untreated skin surfaces meaning that voltage compliance and current magnitude would differ greatly between these cases. When interfaced as a skin surface display, the CCD must be able to drive potentially high and changing bioimpedance during application. The voltage compliance of the driver must be high under these conditions, usually in the hundreds of volts to produce the required current in the milliamp regime. However, CCD drivers interfaced to dermal surfaces with much lower impedance with comparably less variable loading, such as on the tongue in [17] or with abraded or hydrated SC do not require high voltage drivers. Resulting Output voltage during active current pulsing can be under 10 volts on the tongue and under 100 volts for stripped SC. Other systems can completely bypass impedance barriers of the skin to stimulate nerve fibers directly via electrode implantation. CCD drivers in this case require very low voltage compliance that results in stimulation currents in the microamperes. For a system that uses subcutaneous microelectrode implantations as a tactile display interface which targets SAI afferent nerve fibers directly and uses low amplitude stimulation currents in the microamperes, see [10]. 16 For surface interfaces that are designed for non-moistened and unaltered skin loci, many high voltage systems have been proposed. The traditional electronic driver is composed of a choice of OpAmp at the input stage with ether passive or active voltage to current conversion (VIC) circuitry at its output stage. CCDs biased using passive networks use a high voltage amplifier with feedback resistors for VIC. For one such design that uses a Howland current pump configuration, see [3]. Alternatively, active VIC networks use a low voltage operational amplifier with a high voltage transistorbased output biasing stage. Such classes of drivers are popular since they control current with accuracy and precision over passive types since they can be designed for extremely high output impedance. For detailed designs of such systems used in application settings, see [20], [24] or [28]. The authors in [28] developed CCDs with a 16-channel scanning to a display matrix of terminal electrodes. The output of the VIC topology could deliver high current square wave pulses using 8 customizable waveform parameters for each channel that also aided in reducing skin irritation by having a zero net DC current. Conversely, an alternative to CCD-based systems would be open-loop voltage source square wave pulsed drivers. Such drivers can be referred to as open-loop constant voltage drivers (CVDs). Here, output voltage signals are independent of the variable bio-impedance loading conditions. However, as skin surface stimulators, CVDs are not conducive to a non-moistened and unaltered SC resistive barrier. Intact and dry stratum corneum barriers would cause positive feedback in CVD current output because the 17 barrier resistance falls near-inversely to current amplitude in this state. When the electrode is interfaced directly to the nerve fiber subcutaneously, or if the stratum corneum of the skin is dampened or removed through abrasion, open-loop CVDs could be considered. Skin impedance is either no longer a factor in the former case, and the current output no longer influences appreciably the impedance of the skin in the latter, which shows a near ohmic relationship under such conditions [29]. Also, by hydration of skin, time variation of skin resistance is reduced and quality of sensation increased [12] [15]. In reality, open-loop CVD drivers are not seen in application settings, since ultimately a strong need to control precise current gradations and accurate ranges are needed for most tactile information coding schemes and proper regulation of pain/sensation (P/S) thresholds. Furthermore, the current signal that is directly responsible for delivering the tactile information by some form of encoding would be largely unknown without knowledge of the impedance state of the skin. 2.4 DESIGN CHALLENGES WITH EXISTING OUTPUT DRIVER METHODS Open-loop CCD stimulators must be calibrated to a user‟s pain/sensation dynamic range which differs between individuals, over time and with electrode contact location as was previously discussed. To resolve this issue, researchers in [20] allowed users to individually set current magnitude ranges for each electrode using dials for potentiometers, thus allowing the individuals to customize their P/S dynamic range per electrode on the display. The amount of pressure that the user applies to the electrotactile display can influence the effective electrode contact areas and thus concentrate or diffuse localized current density accordingly. With CCDs, sudden stinging, burning or 18 uncomfortable sensations could occur from insufficiently applied pressure since P/S dynamic range ratio is inversely related to contact-area; such a feedback condition would not occur with open-loop CVD-based displays since dermis bulk impedance increases with decreased contact effective area for an electrode. Complicating factors of this approach are the number of set-points, and possible readjustments if the locus under stimulation shifts relative to the electrodes, to name a few. To compensate for P/S dynamic range ratio thresholds versus applied pressure, the researchers in [24] incorporated load cells for pressing force feedback to dynamically adjust current amplitude levels based on overall pressing force; however, this approach was not electrode-specific but controlled P/S dynamic range display-wide likely because of prohibitive size and cost of load cells. A passive stage VIC for CCD design is in shown in Figure 2, and adapted from [3]. The human-equivalent DC resistance will typically be much higher in magnitude than the feedback resistor magnitudes shown. Power loss from biasing increases through this path as the equivalent load magnitude decreases, as will typically happen when the load current is increased. However, using higher resistance resistors in the feedback path reduces bandwidth, introduces Johnson noise and increases voltage offset. Other design complications include precise resistor matching in the feedback path, and choosing an appropriate Rs, which, depending on the value, can reduce load current SNR if set too low or reduce load voltage compliance of the HV OpAmp if set too high in value. 19 Figure 2. Improved Howland current pump with master-slave configuration adapted from [3]. Rs = 1 kΩ, R1 = R3 = 5 kΩ, R2 = 10 kΩ, R4 = 9 kΩ, and ZL is load impedance. For active VIC circuitry, the output stage usually consists of a current mirror. The mirror serves two purposes: 1) converts a low voltage, low reference current amplitude waveform to a high voltage, low current output; 2) sets the output impedance of the driver to a high value. The advantage of using this approach is the availability of low voltage OpAmps and high voltage transistors to design for frequency response, output impedance, and power efficiency of the output stage when compared to a passive VIC solution. However, significant power can go in to transistor biasing, and losses from transistor switching, reference current generation due occur. Specifically, if the SC electrical impedance approaches that of the current mirror output impedance, power efficiency of the circuit decreases, and transistor saturation may occur. One final consideration for open-loop CVD and CCD designs is the layout size. The current mirror components for CCD add to the overall layout size. However, open-loop CVDs require no additional real-estate other than that of a few resistors in their feedback path if HV OpAmps are available. 20 Open-loop CVD designs are not generally used as surface displays because of previously stated reasons. Since both current and bio-impedance are coupled and inversely related unknown variables in a CVD topology, an open-loop method is unstable and an offline calibration approach to P/S dynamic range ratio unrealistic. However, the design advantages for CVDs warrant investigation. CVDs are more energy efficient, simpler to design, and take up smaller layout real-estate versus CCDs as discussed. As the number CCDs in a system increases, these tradeoffs become considerable enough to explore the CVD method. Designing for CVD stimulation implies some prior knowledge skin bio-impedance where interfaced and necessitates a closed-loop solution. Real-time identification of bioimpedance parameters according to a reasonable load impedance model, along with closed-loop design methodologies, means that current output tracking can be accomplished under load-varying conditions. Moreover if a relationship between P/S dynamic range and the known bio-impedance can be found, current signals delivered to each electrode can be adjusted dynamically. Such a possibility means: 1) offline calibration is minimal or no-longer required; and 2) that the user could spatially scan the display and that the respective magnitudes of current signals would be adjusted to the changing bio-impedance conditions during the process. Power efficiency is increased over CCDs because the output impedance of the driver is low and not dependent on load magnitude. The design of the driver in theory is simple since high-voltage OpAmps with adequate slew-rates for square wave generation exist; further, the power needed for driving unmodified and dry SC skin are possible with OpAmps on the market due to 21 the relatively small current requirements. A line of OpAmps for such a purpose are available by Cirrus Logic [37]. Since electrically stimulated intact-SC tissue is a complex system, one possibility for the use of CVDs as surface stimulators would require monitoring of I-V outputs combined with a real-time adaptive control solution. The tradeoff for the desirable power efficiency and minimal layout area with CVD drivers would be the additional circuitry for monitoring outputs, the control scheme for regulating output current, and the real-time software and systems overhead to integrate these aspects. With a load-aware system, the parameters of the skin impedance models for each electrode-skin contact point could be known at all times; hence, further power reduction and cost saving could be achieved by turning off driver power to un-interfaced electrodes, as sensed by the high impedance. Though the process is natural for a closed-loop CVD system, as of the writing of this paper, the author could find no papers for review for such an approach. The synthesis relies on using CVD drivers that are simple to design, manage, and, are above all, compact, and consume less power than the CCD topologies. Therefore, with closedloop design, the CVD can be accomplished with minimal driver overhead if output measurement circuitry is introduced. 22 3. SYSTEM DESIGN The haptic renderer was designed for surface electrotactile stimulation of the index fingertip using a matrix of electrodes for untreated and dry SC skin boundaries. This work is a redesign of an earlier developed electrotactile renderer whose design is detailed in [1]. The previous version has successfully functioned as the haptics display in a telediagnostic and telepresence system for researching breast tumor pathology diagnosis (see [1] for a description of the overall system). In [2], it is presented as a tool for haptics investigation in to perception of car seat leather qualities coded for using electro-tactile stimuli. The physical properties of the leather were measured using a high-resolution optical sensor that included roughness, geometry, and contour information for haptic encoding and rendering. The application was for quality control purposes regarding car seat inspection in the automobile industry as a telepresence application. The newly developed haptic system in this paper features a CVD OpAmp stimulator as proof-of concept for a new type of stimulator topology. It adds intelligence to the system by making it a real-time, load-aware module for bio-impedance by incorporating customdesigned I-V sensors necessary for online system identification of the given impedance model. Pertaining to the output, voltage waveform output parameters that can be adjusted in software include: voltage amplitude, duty cycle, fundamental frequency, and three types of waveform pulse shape geometries; the last setting is research-specific that pertains to psychodynamic and output power experiments that were carried out. 23 This is in contrast to the older system which does not have software settings for output amplitude; the possible pulse states to any given electrode are either on or off. The motivation for the new haptic system is to maximize power efficiency in the output driver that accompanies CCD design and study bio-impedance identification and its implications. Power analysis of the driver will be presented under real loading conditions. As a first step, system identification of bio-impedance loading models using the CVD hardware was accomplished through custom software, online and in real-time, which will be demonstrated. As well, a model reference adaptive control (MRAC) law will be introduced as a means to overcome the challenge of tracking the output current under changing load conditions. Simulations of the control law will be shown as a first step to prove efficacy. In summary, the first version of the haptic renderer is constructed with hardware including: CVD stimulator, output sensors, high voltage power supply, and a revamped electro-tactile display terminal. It also includes a high speed DAQ card for sampling data from the I-V sensors and output signal rendering. As well, the initial system software and GUI was designed in Windows to carry out preliminary tests of the hardware and load identification algorithm. Simulations and preliminary analysis of the closed-loop algorithm will be presented based on realistic scenarios from bio-impedance identification experiments and results. 24 4. HARDWARE, SOFTWARE, AND SYSTEM CALIBRATION Figure 3 shows the system-level diagram of the implemented haptic renderer. The main hardware modules can be grouped as follows: PC and high speed digital acquisition board (DAQ), CVD stimulator, load measurement circuitry (current and voltage), high voltage bi-polar power supply, and designed electro-tactile display unit. The CVD amplifies the voltage waveforms to appropriate stimulation levels with a set gain. The CVD interfaces to one electrode on the electrotactile display at its output. The multiplexing circuitry for time-division scanning the output to the electrotactile array was not developed at this time. The two functions of the measurement circuitry are signalconditioning and scaling down load signals for sampling by the DAQ. The software is custom designed. It serves as the software-controlled waveform function generator and controls the sampling and processing of load measurement circuitry for the DAQ. Other main functionalities of the software include real-time load impedance identification, a GUI front-end used for setting I/O and waveform generator parameters and displaying identification results. The design details on, and interrelationships between each block diagram are summarized in Figure 3 and will be expanded upon in sections to follow. As well, the details of calibrating the measurement sensors and the CVD amplifier will be explained. 25 Figure 3. System level diagram of implemented project. 26 4.1 HARDWARE DESIGN The following subsections discuss the haptics hardware design criteria, such as for the selection of DAQ, OpAmps for output sensors and CVD stimulator, and also elaborate on the custom hardware built, for this project. As well, the specifications and construction for the revised tactile display are presented. 4.1.1 DIGITAL ACQUISITION (DAQ) BOARD The DAS 6025 DAQ from Measurement Computing was selected for I/O interfacing between haptic renderer hardware and software. The DAQ board features 12-bit resolution, 10,000 samples/sec D/A output channels, and 12-bit resolution, A/D input channels with conversion rate of 200,000 channel-samples/sec. A high speed A/D sampling rate is necessary to capture the output current transient response delivered to the skin-electrode interface. Based on known output dynamics under usual human loading conditions, a DAQ board with twice the sampling rate was desired; however, budget limitations constrained the choice. The project required one analog out channel for inputting analog data to the CVD input and two A/D conversion channels (100,000 samples/sec) for sampling the load current and voltage measurement circuitry outputs, respectively. As well, bipolar voltages can be outputted and sampled by the DAQ allowing for biphasic pulsing to the skin and measurement. 4.1.2 HIGH VOLTAGE DC-TO-DC CONVERTER POWER UNIT The haptic renderer requires a high voltage, bipolar power source for powering the high voltage CVD stimulator OpAmp which can be powered at ±450 volts. For this, a custom 27 bipolar, high voltage converter configuration was designed; the schematic is shown in Figure 4. The core components are duel unregulated 300 volt EMCO DC-DC converters supplying 10 watts output at full scale (U3 and U4 in the schematic) [36]; the output voltage is proportional to the input voltage, and the isolated output of the converter allows for polarity selection. Thus, one converter functions as the high voltage positivereferenced power supply, while the other functions as the symmetric negativereferenced counterpart. The duel reference sources are added for the option of producing biphasic pulses on the CVD stimulator output if desired. The positive and negative output voltage levels are adjusted symmetrically at the input by tuning a set point Vset with output dynamic range from ±50V to ±300V. Since the output is unregulated, each converter was designed with negative feedback regulation circuitry as to reduce the dependency of current draw on the output voltage. The negative feedback from the outputs of U3 and U4 to the corresponding inverting terminal of U1 and U2 are used to bias the Darlington BJT transistors configurations, Q1, Q2 or Q3, Q4, for regulation of input power to the converters based on the high-side power draws, respectively. The effect is to keep the high-side output voltages invariant of output current draws. The feedback capacitors, C2 and C12, are used as low pass filters to reject high frequency influence on the regulation. Q2 and Q4 are transistors with high collector current ratings in case it is necessary to source input current to the converters operating at full power rating. High voltage tank capacitors, C1 and C4, with large capacitances at the converter output side are included to filter out high frequency switching noise attributed to DC-DC boost 28 conversion and to isolate the regulated voltages from transients that can occur from high current draws. These capacitors assist the regulation circuitry, which is only able to respond to low frequency changes in current draw. An important safety concern is the isolation of the DC-DC converters from ground as to avoid possible electrical shock to the user. The electro-tactile display and, hence, the user is referenced to COM2 of this module and must be isolated from ground well. The isolation of the converters was tested empirically and provided at least 5 MΩ of ground isolation during operation and human use. 29 Figure 4. High voltage DC-to-DC converter circuit. 30 4.1.3 CONSTANT VOLTAGE DRIVER (CVD) The CVD component of the haptic renderer delivers high voltage pulsatile signals to the electro-tactile display. The schematic for the CVD circuit is shown in Figure 5. The OpAmp component U1 is high voltage, which is configured as a non-inverting amplifier. The output is given by: VDAQ_OUT is the software-generated waveform from the DAQ D/A output channel to the CVD input terminal; VCVD is the output voltage signal available for application to a user‟s fingertip as referenced to COM2. The PA97 OpAmp manufactured by Cirrus Logic was selected for its maximum ±450V voltage output swing, high 8Volts/µs slew rate, and capability to source or sink 10 mA of DC current [37]. The current and voltage output ratings far exceed the normal electro-tactile stimulation ranges based on prior work that identified haptic design parameters now pertinent to this work (see [34]) as well as values discussed in the literature review section. From the slew rate specification, the minimum rise time of the OpAmp output at full-scale is 37.5 µs. The rise time is insignificant since the voltage pulse durations in this application ranges from 0.5 to 2.2 ms. The OpAmp closed-loop gain is set at approximately 50 and can deliver bipolar waveform pulses of approximately 300 volts in amplitude; the bipolar voltage sources powering these OpAmps, discussed in the previous subsection, sets this upper magnitude limitation. Vbias is a low voltage source that can be adjusted to cancel DC offset at the CVD output. A large 4.7 MΩ feedback resistor (R3) minimizes power loss in 31 the feedback path since it will be shown to be much greater than the typical DCequivalent impedance of human skin for this application. It is important that the large value of R3 will not inject instability in to the circuit. According to [37] the conservative gain-bandwidth product is: where Av is the designed closed-loop gain of the circuit and fGB is the maximum largesignal bandwidth frequency for a certain Av (in this case about 50). The open-loop unity gain bandwidth for the PA97 from (4.2) is 1 MHz. Per the specification sheet [37], the internal input capacitance Cin is equal to 4 pF. With the value of the resistor used for R1 set at 10 kΩ, the added pole to the active network is related by: The values introduce a closed-loop pole at 4 MHz. Consequently, the additional pole far exceeds the bandwidth of the OpAmp and is not a concern for closed-loop instability. 32 Figure 5. Constant voltage driver (CVD) circuit. 4.1.4 CVD OUTPUT VOLTAGE SENSOR Figure 6 shows the schematic for the developed CVD voltage sensor configured as a voltage follower with a voltage divider stage for scaling the high input voltage. The purpose of this circuit is to proportionally scale down the CVD voltage output pulses for compliance and buffer the signal to one of the A/D channels of the DAQ for measurement. The circuit can measure voltages of both polarities, as well. COM1 is the input node and is referenced to the voltage delivered to the tactor under measurement. The voltage divider network has a resistor R1 of high value to minimize loading and effectively keep electrical isolation between COM1 and COM2. Since the OpAmp has input capacitance, low-pass filtering of CVD pulses under measurement from the large resistor values shown could occur. The analysis will follow. 33 The equivalent common mode input capacitance for U OpAmp is a CIN of 2.5 pF. The CVD output voltage seen at the non-inverting input of U in Figure 6 is the output of the filter given by: where Vcom1 (COM1 in the figure) is the voltage measured at a given tactor on the electro-tactile display. The relationship in (4.4) should approximate a flat band pass filter over the dominant frequencies in the CVD output pulse waveform. The 3 dB down impedance pole of (4.4) is located at: Using (4.2) and the value of Av equaling 50, the large signal frequency cutoff of the CVD OpAmp is roughly 20 kHz so that voltage divider scaling will not depend on frequency in this case. At the output side of the OPA37, the slew rate is 11.9 V/μs which is greater than that of the PA97 CVD OpAmp. In summary, the voltage sensing circuit will not act as a low pass filter because of the location of the pole given by (4.5) and higher slew rate versus that of the CVD OpAmp. 34 Figure 6. CVD voltage output sensor circuit. 4.1.5 CVD OUTPUT CURRENT SENSOR Figure 7 shows the schematic of the designed high-side current sense amplifier circuit used for measuring human load AC currents from applied CVD voltage pulses. Rsense is a high-precision resistor of 1.000 kΩ in series with the fingertip load. Load current measurements occur across the Rsense resistor by transduction to low differential voltages since RMS load currents sourced to the display are typically in the low to sub milliamp range. OpAmp U4 buffers the differential voltage across Rsense. The voltage at the sensor output U5 is linearly related to an acceptable degree to the stimulation load current. U1, which is a standard voltage DC-to-DC converter, powers the currentsensing circuitry and has galvanic isolation between its primary and secondary sides. The ground reference COM1 for the converter on the secondary side is galvanically isolated from COM2 and is the reference potential for the current sense resistor and circuitry powered by it. When viewed from COM2, high common mode voltages due to CVD high voltage pulses referenced to COM2 are canceled on the input to U4 and 35 circuit elements powered by U1 since these elements will float on the pulsed voltages. However, U1 possesses some inter-winding capacitance which means noticeable capacitive current will flow between secondary and primary windings if the CVD voltage pulses changes quickly and with great magnitude in a short duration of time, as is the case with square wave voltage signals. This could result in transient high voltage spikes across circuit element nodes referenced to COM1. The selection of U1 was based on choosing one with small interwinding capacitance as to reduce the effects of capacitivecoupled currents on the isolation and measurement process. The analysis of the current sense circuit and why it is necessary will now follow. 36 Figure 7.CVD current output sensor circuit. 37 The high common mode (CM) input voltage for this application would be problematic for conventional high side current sense amplifiers. Most affordable OpAmps do not have inputs that are rated for high common mode voltages in the 100s of volts. The common mode introduced from the CVD pulses is usually well above the 100 volt level and can be as high as 250 volts. There were two observed ways to contend with this issue: 1) use a current sense difference amplifier that scales the high common mode to a compliant range; 2) construct a floating voltage reference for the current sense circuit using the CVD pulse as the voltage reference. Solution one is idea in theory, due to the minimal number of components needed, but is difficult to implement in practice for this application. There are many differential OpAmps on the market that achieve the desired CM attenuation using laser-trimmed resistors for good matching leading to accurate current measurement. However, differential amplifiers not only scale down the common mode voltage, they also scale down the differential mode input across R sense. Since the typical stimulation currents are in the milliamp range, Rsense would have to be increased in value proportional to the CM magnitude attenuation scaling so not to decrease SNR. However, doing so would add unintended loading to the CVD stimulator voltage which would negate one of the intended purposes of its use. Therefore, approach two is promising because it does not rely on scaling down voltages across Rsense and also cancels the common mode seen by the input of U4 in Figure 7. The tradeoff is an increased complexity in the accompanying circuit design. Choosing a DC-to-DC converter to provide the necessary reference isolation is the main design challenge for the second approach. The component U1 met such a requirement, 38 which also has a three-tap secondary winding to provide a positive and negative power supply to the OpAmps incorporated for measuring bipolar current across Rsense. The secondary side of U1 provides +12 and -12 volt duel power with a center-tapped common reference terminal and 2.5 pF inter-winding capacitance, while also providing galvanic isolation. The effect is twofold: 1) the high voltage common mode is canceled when powering the current sense circuitry by using U1; 2) the current sense amplifier, U4 only sees the differential voltage across Rsense. Therefore, small voltage OpAmps can now be employed to measure current while keeping Rsense small in value as well. The barrier capacitance of the converter is 2.5 pF. With the maximum slew rate of 8V/μs for the HV CVD OpAmp [37], an expected leakage current from the transient response would be 2 μA. This is negligible leakage for the application. However R1 being high forms a low pass filter with this barrier capacitance and should be kept as small as possible as not introduce a significant time constant in the measurement circuitry; the effect would result in transient high voltages on the circuit from the CVD pulses which could damage the measurement circuitry. Selection of R1 has another constraint that will be discussed later. The last consideration is reducing switching noise on the secondary side of U1. For this, a CLC filter (π-filter) was employed for both the positive and negative voltage references on the secondary side. The inductor was chosen to have low winding resistance to minimize power dissipation, and the decoupling capacitors are low ESR tantalum types to minimize high frequency impedance. The capacitors on the other side 39 of the inductor choke are low-valued ceramic capacitors with very low ESR to filter out the highest frequency noise. The current measurement voltage signal outputted on U6 must be re-referenced to COM2 for DAQ sampling. The isolation linear optocoupler U7 serves this purpose. The optocoupler has an LED emitter with a feedback photodiode referenced to COM1 and another photodiode on the opposite barrier referenced to COM2. The servo diode is used as a negative feedback to amplifier U6 to regulate the LED emitter‟s forward current IF and to make this current highly linear since it is directly related to the current through R1. The photodiode current referenced to COM2 is also proportional to IF since it is directly related to the current through feedback resistor R2. U5 is the transconductance linear amplifier on the output side of U7 and buffers this signal to the input of the analog DAQ channel. The isolation amplifier is configured in the photovoltaic state. This state was selected to give the best linearity, and lowest noise characteristic. Having a high SNR for 12 bit resolution and good linearity is paramount for accuracy in the identification algorithm when compared to the photoconductive state. The bandwidth of the optocoupler in photovoltaic operation is about 50 kHz [38]. This exceeds the bandwidth requirement for measuring load current since it is greater than the bandwidth of the CVD stimulator of 20 KHz given by (4.2) with Av equaling 50. The circuit topologies of U2 and U3 provide precision current sources whose output is given by: 40 where the voltage reference in the equation is produced by D1 and D2 with good precision. The purpose of the current sources allow for bipolar current measurement. They can also cancel unwanted DC offset voltage at the output of U5 via adjustment of each ones biasing resistor (Rbias). For the load identification algorithm to be valid, it is important to measure any potential current undershoot even if bipolar voltage pulses are not used. Referring again to Figure 7, the relationships for the isolation amplifier circuitry are: The diode forward current IF is responsible for the non-linear, time and temperature dependencies using optocouplers without servo-current feedback. However, with feedback, (4.13) shows that IF is factored out. The ratio of R2 and R1 is kept close to 41 unity as well for measurement purposes. Now, the output at U5 is only dependent on resistor values and the transfer gain K3, which itself is a ratio of the output transfer gain K1 and the transfer gain K2. The transfer gain K3 is close to unity and drifts very slightly with respect to forward current magnitude and ambient temperature but otherwise remains constant. The resistor values, Rsense, and R1, must be selected in consideration of the optocoupler‟s maximum IF current rating for best linearity, knowledge of the maximum load current levels to be measured so not to damage the IC, and the SNR of the input signal. The latter of which is based on the magnitude of Rsense. In addition, the voltage magnitude going to the A/D channel of the DAQ cannot exceed 10 volts. From previous studies in [34], the absolute maximum current amplitude of 5 mA will be taken as the upper limit. In actuality, the typical current amplitude to a human load is about 1 to 4 mA with dry skin. For this reason, the sense resistor was selected at 1.000 kΩ, which will give a good SNR while also minimizing loading influence on the CVD stimulator. Taking 5 volts to be the maximum voltage across R1 and K1 in (4.9) with a nominal value equal to 0.007, R1 was selected to be 20 kΩ to give a maximum IF of 36 mA by using to (4.8) and (4.9). According to [38], the linearity of the optocoupler is not compromised with this maximum value of IF. R2 was then selected to give unity gain. However, the biasing current for bipolar current measurement can effectively reduce the maximum current rating (60 mA) and upper bound in forward photodiode current IF for optimal linearity (30 mA) of the optocoupler. The current biasing of U2 took in to consideration that negative 42 current pulses are much lower in magnitude than the positive pulses to the anode electrode for the intended uses of the system; hence, the biasing is optimized for positive non-symmetric bipolar or positive monophasic voltage pulses. In summary, the isolation amplifier should provide a highly linear relationship between the measured stimulation current across Rsense, and the output of U5 in the face of high common mode voltage stimulation pulses delivered to the electro-tactile display. 4.1.6 ELECTRO-TACTILE DISPLAY HARDWARE DESIGN The electro-tactile display is the interface for the fingertip surface to elicit various sensations to the somatosense by electronic stimulation. The tactile display shown in Figure 8 was developed to make improvements to the previous one presented in [1]. The enhancements include doubling the electrode density, more intuitive fingertip placement, ergonomic design, such as fingertip and arm rest padding, and making the unit more compact. The dimensions of the encasement of the display are 10.16x7.62x2.54 centimeters. Soft rubber padding resides at the base of the display to serve as a stable resting position for the index finger. The electrode active area, where stimulation occurs on the fingertip, is 2.74x2.31 centimeters. The active area is made intentionally larger than the average area of actual index fingertip regions for ability to temporally scan the matrix. Scanning the electrodes with the fingertip has been shown to improve richness of perceptive experience in some cases [7] [42]. To determine the lower bound of the active area corresponding to the index fingertip, subjects were asked to make fingerprints with this region on paper so that the resulting ink perimeter that occurred from pressing against a hard surface could be measured and averaged. The 43 pressing force was kept between 4 to 8 ounces to make the prints, which were typical preferred forces by users as determined beforehand. The display features 192 copper electrodes with tin plating. Tin platting is necessary so that the electrodes will not oxidize, which would increase interface resistances and, hence, voltage amplitudes for the same current stimulation levels. The electrodes are 0.762 mm in diameter with 1 mm pitch. The pitch of the electrodes is less than reported distances of two point discrimination tests (DPDT) with static-placed fingertip, which according to [4] ranges from 2 to 7mm, and can be greater for temporal scanning. From [22], the innervation density of the densest receptors, the FA1s, is on average 1.4 units per square millimeter, which is on the order of the tactor density of the active area of this display. Choosing the electrode diameter was done to maximize the electrode density without compromising physical or quality of sensation. The electrode diameter determination took in to consideration dielectric breakdown of air, and the effects of electrode geometry on sensation intensities. In general, the breakdown voltage of air is 3000 volts/mm with rounded electrodes if voltage potential is static but will decrease as a function of increasing change in voltage or air humidity. Increasing the electrode area increases the pain-to-sensation (P/S) dynamic ratio because the current density over the skin-electrode contact area decreases on average [4]. Keeping the electrode diameter small minimizes the chance of dielectric breakdown between electrodes by keeping electrode pitch large at the cost of reduced P/S threshold ratio. However, the 44 electrode diameter used in the former tactile display of comparable dimension gave good quality of sensation over a wide range of current amplitude and pulse width. Quality of sensation in this case was a blunt vibration sensation experienced by the individual without notable stinging or sharpness present. Figure 8. Tactile display: dimensions are 10.16x7.62x2.54 centimeters with an active area of 2.74x2.31 centimeters. The electro-tactile display is isolated from ground for the safety of the user. Having high impedance isolation assures that no current flows through the cardiac tissue. In the unlikely possibility that the system forms a path to ground through the human user, the 45 maximum current that can be delivered from the CVD is 10 mA. Figure 9 shows the developed return electrode for the isolated system. It features a large copper cathode return with Velcro strap so that the user can easily attach it to the thenar eminence region of the palm for minimal interference. The lack of hair follicles in this area helps with making low impedance contact to the return. Without a connection to the isolated return, no stimulation can be felt which supports the conclusion that most of the stimulation current flows between the index fingertip and the electrode return path. Figure 9. Ground-isolated return electrode. 4.2 SOFTWARE DESIGN The software for this project was custom designed and programmed in the Windows environment. The three main features of the software are: 1) a GUI front-end for configuring voltage pulse output parameters, I/O sampling rates, load identification algorithm and to display the real-time results dialog; 2) control of system-level components such as DAQ functions, and algorithm for real-time identification of human 46 impedance models; 3) and software-based waveform generator for pulsed stimulation. Each of these features and functions in relation to software design will be discussed in the subsections to follow. 4.2.1 GUI FRONT END AND RESULTS GRAPHICAL DISPLAY The GUI front end, shown in Figure 10, allows the user to configure settings for the waveform generator and identification algorithm. Referring to the “Initial Driver Output Parameters”, the output waveform settings include: pulse shape, CVD pulse amplitude at output, fundamental frequency of the waveform, and duty cycle. One additional setting is the number of equally-spaced, intra-duty cycle modulations in an active duty cycle pulse. As well, the calibration coefficients necessary for scaling the input from the DAQ output to achieve desired CVD amplitude at its output are entered here. The other settings, including waveform pulse shape, are provided for the researcher to study configurations that impact psychophysical perception of the media being displayed, such as pressure, edge, tickle, vibration, or motion to name a few. Also, the researcher can experiment with the parameter space to determine their influence on perception. Referring to the “Identification Algorithm Settings”, the responsiveness of the identification algorithm to time variation in the plant is done by setting the sample memory N. As seen under the heading, “Record ELS Results in Seconds”, the program can be used to record results and relevant data as a function of time for offline analysis. As well, gain measurement calibration coefficients can be entered which are necessary to proportionally rescale the current and voltage output sampled data to actual values. 47 Figure 10. GUI front end. 48 Figure 11 shows the dialogue of the GUI that displays the real-time metrics of the identification algorithm. The results dialogue provides visual feedback for output error convergence, most recent set of identified parameters in the S and Z domain, and other states for the identification algorithm in real-time and as a function of relative time. The metrics can be stored when the operator uses the software record feature. 49 Figure 11. Identification results dialogue. 50 4.2.2 SOFTWARE DESIGN AND PROGRAM FLOW Figure 12 illustrates the top-level hierarchical structure and interactions between the GUI, underlying process, and child threads. The software processes GUI front-end requests, controls I/O and access to DAQ shared memory, selects (de)activation of the load identification algorithm based on GUI request, and generates digital waveform primitives from desired GUI inputted parameters. Digital waveform primitives are defined as DAQ analog output waveforms inputted to the CVD stimulator. Waveform D/A conversion and output are handled by the “DAQ D/A Out” thread. Programming for the DAQ was done using the C routines of the Universal Library API by Measurement Computing. GUI requests may be disregarded based on in-program software interlocks if they would lead to unsafe operating conditions for the user. The “DAQ A/D In” thread intermediates the run/stop state of the “Online Identification” thread based on GUI request. The load identification thread is programmed for speed and priority level since it must process the load model in real-time. As well, the load identification thread grants access to inter-thread read requests by the “Results Display” dialogue thread for displaying the most current load identification results. Figure 13, Figure 14, Figure 15 and Figure 16 expand graphically the program logic and shared memory read/write schemes inside the block diagrams of each child thread and their interrelation to one another. 51 Figure 12. Program flow control: top level. 52 Figure 13. Flow control for D/A output thread and DAQ shared memory write. 53 Figure 14. Flow control for D/A output thread and DAQ shared memory read. 54 Figure 15. Flow control for initiating or stopping A/D and identification threads along with identification shared Memory read from results display thread. 55 Figure 16. Flow control, shared memory access and data accessing for online identification thread. 56 4.2.3 SOFTWARE WAVEFORM GENERATOR AND TYPICAL WAVEFORM PRIMITIVES Waveform primitives are generated from GUI-inputted parameters and stored in PC memory as a digital period template of one period. The digital waveform generation corresponds to desired CVD output pulse characteristics entered in the GUI such as shape, amplitude, frequency and pulse duration (duty cycle). Digital waveforms must be generated for D/A converted input to the CVD stimulator; hence, the relationship via calibration from CVD stimulator output settings to corresponding analog input must be known. This is discussed in the next sub-section. Figure 17 illustrates the different waveform primitive types as seen at the CVD output that the can be generated in software; they are square, saw tooth, and triangle wave pulses. The graphs were generated by sampling and magnitude rescaling the voltage output and saving the measurements with the software save feature; only one pulse for each waveform primitive type is shown out of the multiple pulses that occurred in the sampling duration The CVD output parameters settings were 60 Hz, 11 percent duty cycle and between 140 to 170 volts in amplitude. Each displayed measurement was sampled using a load impedance of 50 kΩ at output. As seen, the voltage pulses had low noise, and well defined geometry at high voltage. The quantization levels from D/A conversion are apparent in the waveform, especially with the non-square wave shaped geometries; however, when applied as stimulation to the skin, the quantization levels were not experienced tactilely likely do to smoothing from both action potential response time and JND level gradation. The shown waveform primitive shapes were selected 57 primarily for testing hardware dynamics capability, load parameter identification results as a function of waveform shape, sensation threshold levels and psychophysical responses from stimulation to be presented in later sections. They do not necessarily reflect the optimal choice of waveforms for delivery of tactile content in application settings. The hardware is fully capable of generating biphasic or monophasic voltage pulses which have no net average DC current when in application settings. Researchers have shown that these later waveform classes are less electro-chemically reactive to the skin which lessens creation of sweat pores (electroporation), irritation of the skin, injection of non-native ions, and net ionic flow inside the dermal tissue over extended time of operation [12] [28] [32]. Figure 17. CVD output pulses generated from software waveform primitives utilized in testing tactile display. 58 4.3 HARDWARE CALIBRATION This section explains calibration of the CVD output voltage and measurement circuitry. The identified relationships are programmed in to the software with respective calibration coefficients that can be set before or at run-time. The result is that the CVD output voltage amplitude can now be reliably set programmatically by the GUI interface and that output measurements sampled by the DAQ can be scaled back to original magnitudes for load identification purposes. The calibration process utilized a 50 MHz analog oscilloscope and two voltage probes which allowed for simultaneous measurements of I/O voltages at either the CVD or one of sensor networks, respectively. Best-fit regression analysis between the input and output of a given network was then done using obtained multiple sampled voltage pairs to characterize the network accurately. A known high precision load resistance of RL = 50 kΩ was used for calibration in all cases, which was chosen since it is the minimum allowable impedance the haptic renderer can reliably drive. The types of pulses used for calibration were square wave voltage pulses at 60 Hz with an 11 percent duty cycle. The CVD voltage output amplitude versus voltage amplitude seen at the DAQ analog output channel is illustrated in Figure 18 for multiple measured pairs. The fitted regression line and equation of the measurements points is given. CVD output voltage measurements were taken before the current sensor and directly at CVD output so that the relationship was independent of loading conditions. The CVD output voltage 59 amplitude spanned from 50 to about 180 volts with good full-scale linearity. A power fitting curve gave the best regression model (R2 = 0.996). 60 Figure 18. Calibration of the CVD input-output voltage amplitudes. 61 Figure 19 shows the plot relating voltage amplitude seen at the voltage sensor output versus that at the load standard. The regression line and corresponding equation are shown. The relationship is used in program to rescale the DAQ sampled CVD measurement voltages to load values. Measurements of the CVD output voltages were taken after voltage dropped across the current sense resistor since the interest is in the voltage relationship due to loading of the CVD, as is the case under actual fingertip impedance loading conditions. For calibration purposes, the standard resistor aforementioned is used. Regression analysis was performed and the result shows the lower bound of linearity. Pairs of voltage samples were collected under CVD output that ranged from 50 to 180 volts. An affine linear model gave the best coefficient of determination (R2 = 0.997). The constant DC term is a negligible percentage of the scaling term which is likely due to either probe measurement or OpAmp offset voltage error from the measurement circuitry. 62 Figure 19. Calibration of the load voltage sensor. 63 The results for the calibration of the current sensing circuit voltage output amplitude to that of the load current are given in Figure 20. The horizontal axis ordinals are the steady-state load currents obtained by dividing the measured voltages at the load with the known 50 kΩ impedance standard. The vertical axis corresponding ordinals are voltage points from measurements at the current sensor output. The CVD output voltages were chosen to emulate typical current magnitudes flowing through human skin at the region of application. An affine linear regression model gave the best goodness-of-fit for the plotted data points (R2 = 0.9996). The DC offset voltage for the load current-measuring circuitry is only 0.0206 volts. The low DC offset is due to offset canceling circuitry as discussed in section 4.1.5. The transconductance gain coefficient in the linear regression is 1051 kΩ and close in value to the precision sense resistor of 1000 kΩ. Ideally, the measurement gain should equal the value of the current sense resistor; however, the offset depends on the isolation amplifier used in the current sensing circuit having a gain of exactly unity. Finer precision instruments for voltage calibration could be used to correct this disparity. 64 Figure 20. Calibration of load current sensing circuit. 65 5. MODELING AND ANALYSIS OF ADAPTATION LAW The subsections to follow will introduce and analyze two electronic conductance models of the skin when current is applied. Both models have the CVD voltage pulse as input and the estimated current response to the skin as output. The first model to be discussed is parameterized as a transfer function in the S-domain and is called the continuous time bio-impedance (CTBI) model, where the “bio” prefix emphasizes that it represents some of the skin‟s bulk electronic properties. The parameters for this model correspond to passive electronic circuit elements that have been shown to reproduce the dynamics of the response of skin to electronic stimulation. The other model is parameterized as a conductance transfer function in the Z-domain whose parameters are estimated online in the discrete time domain. The parameter estimates corresponding to the CTBI model are then derived from a function of the identified Z-domain model parameters. The Z-domain model will be referred to as the discrete time conductance (DTC) model; its parameters do not have any direct physical correspondence to the underlying plant. However, the model order and number of unknown parameters are based on the CTBI model structure. Subsequently, the adaptation law for online parameter estimation that occurs in the Z-domain will be introduced and analyzed. 5.1 CONTINUOUS TIME BIO-IMPEDANCE (CTBI) MODELING The CTBI model is shown in Figure 21 as a first order impedance network with time and spatial varying properties. The elements are functions of measured stimulation current 66 Istim, skin location of electrode contact δloc, effective skin-electrode contact area Aeff, and the fingertip pressing force on the electrode Fapp. As seen, Rp and Cp are coupled to the stimulation current Istim. The network comprises a resistor Rp in parallel with a capacitor Cp in series with another resistor Rs. The electrical impedance model reproduces the shape of the output current from high-voltage monophasic square wave voltage pulses to the skin. The much larger return electrode is attached to thenar area on the palm to complete the circuit. Istim will be taken as the measurable output to the interface with VLoad as the measurable input to the system for simplicity. Referring to (4.1) and (4.7), VLoad is related to the CVD haptic renderer output VCVD as: 67 Figure 21. Continuous time bio-impedance (CTBI) model of the fingertip under electrotactile stimulation. 68 Rp models electrical resistance of the natural protective barrier of the skin, called the stratum corneum (SC). Large signal analysis done on Rp indicates that it is an approximate power function of the RMS value of Istim, which will be called ISTIM_RMS. The characteristic is: , for A and B to be identified, which are themselves assumed to be a functions of time, space, electrode area and pressing force. Therefore, Rp is inversely related to the stimulation current amplitude. The physical interpretation of the dependency can be viewed as increased ion flow in to the pores of the stratum corneum due to the higher voltage potential, which increases conductivity in the SC [29]. Rp has been shown to be related to the effective skin contact area at an electrode [3]; that is, with all other variables held fixed: where rp is the specific resistance of the SC; hence, the fingertip pressing force Fapp against the electrode can influence the value of Rp since it may alter Aeff. Last, time and spatial variation δloc of Rp arises from changes in electro-chemistry of the stratum corneum from the applied signal over time, and the structure and moisture content of SC tissue at different regions on the skin, respectively. Cp models the bulk capacitance that is a result of the skin cell membrane structure in relation with water and ion content in the extra- and intra-cellular regions of the skin. 69 The cellular membranes are built from lipid-bilayers, which are two lipid layers that each has outward-facing hydrophilic boundaries and mutually inward-facing hydrophobic molecules when viewed in cross-section. Likely, the cellular membrane causes polarization of molecules from an applied electric field, since the membrane boundary will attempt to oppose the flow of ionized material through or into the cellular membrane. The cellular capacitance is assumed to be a function of the same parameters as Rp. A relationship between Cp and stimulation current is not as clear when compared to that of Rp. However, the variation in order of magnitude versus current is about two to six, maximum, where as for Rp it can be from 10 to 100 orders of magnitude. Based on experiment, Cp is assumed to have a slower time variation, and be well within a tight lower and upper bound, depending on skin location, user, and stimulation current magnitude, as compared to that of Rp. Rs model the deep tissue resistance in the skin. Rs is usually measured 10 to 100 orders less in magnitude compared to Rp at quasi-static DC frequencies; this is likely a result from better conduction in this tissue because of the higher water and ion content compared to the stratum corneum. In contrast to Rp and Cp, Rs is relatively constant versus skin location, user, and current stimulation amplitude because of increased isotropy of the environment compared to the stratum corneum. As well, ion conduction is not influenced by the stimulation levels in this tissue because of the relative homogeneity [29] [32]. It will be assumed that Rs is dependent on the pressing force Fapp and electrode skin contact area Aeff. 70 The coupling between Rp and the stimulation current level, as given in (5.1), will influence the accuracy in output estimation if the relationship is not included in the modeling and adaptation law. This is especially true for current output generated with long rise and fall times where the effect will be noticeable. In general, if this second order effect is neglected in the identification process, then the CTBI model cannot be used as a model estimator for output current since the estimation error will not even converge. However, it can be shown through experiment and analysis that sufficient conditions exist for when the effects due to (5.1) can be lumped in to time variation of the model so that the output estimation error will still converge. The set of assumptions that make the model linear in the parameters are (5.1.1): 1) The input signal voltages are square monophasic pulses less than 5 milliseconds in duration. 2) The stratum corneum of the index fingertip to be stimulated is intact without damage and not overly damp. 3) The current amplitude is in an experimentally determined range, usually from 0.5 mA to 5 mA. Assuming (5.1.1) is true, the CTBI model now consists of three linear parameters to be estimated online as a function of time and space, Rp(t, δloc), Cp(t, δloc), and Rs(t, δloc), that have been shown to reproduce the measured transient and steady state current response to a high degree. Here, the pressing force Fapp and effective electrode skin contact area Aeff are lumped in to time variation of the model. 71 In lieu of (5.1.1), the continuous time linear parameters bio-impedance (CT-LPBI) model in transfer function form is given by: δ where is the model output when the parameters are known. 5.2 DISCRETE TIME CONDUCTANCE (DTC) MODELING Parameter estimation occurs in the Z-domain for the DTC model. The parameterization and form of the DTC model is specified for: 1) using this model to calculate the parameter estimates for the CT-LPBI model (5.3) at each time k; 2) introducing a term that will give improved minimization of output estimation error for classes of input or output not specified in (5.1.1). It will be argued that when conditions in (5.1.1) are met, along with conditions on the sampling rate Ts of the DAQ, the CT-LPBI and DTC models are equivalent. The DTC model is given by: δ 72 where Y(Z, k, δloc) will be referred to as the discrete time small signal conductance (DTSSC) model, and Ψ(Z, k, δloc) the correlated error dynamics (CED) filter; e(k δloc) is the output error at time k caused by all un-modeled dynamics by the DT-SSC model; the filtered error term improves output estimation error convergence for when large signalnonlinearity is present due to violation of (5.1.1). is the model stimulation current output when the parameters are known; and VLoad(k) is the measured tactor voltage. The coefficients α1(k, δloc), β2,(k, δloc) β1(k, δloc), c1(k, δloc) are the Z-domain parameters to be estimated at time k. As in (5.3), the model parameters vary with time and space δloc, where the variation of the parameters due to pressing force Fapp and effective skin-electrode contact area Aeff have been lumped into the time variation of the parameters. 5.2.1 DTC MODEL PARAMETER ESTIMATION AND ADAPTATION ALGORITHM IN Z-DOMAIN Figure 22 shows the system-level scheme for real-time parameter identification. The goal is to use an adaptation law derived from a Lyapunov function V(t, θ) that gives the ˆ set of estimated parameter θ for (5.4a) that minimizes the difference between measured and estimated output, termed the residuals ε, versus time. As illustrated, a function is used to map the Z-domain estimated parameters to the S-domain estimated model parameters of (5.3). 73 Figure 22. Overview of parameter identification. 74 5.2.2 ITERATIVE EXTENDED LEAST SQUARES WITH FORGETTING FACTOR ADJUSTMENT (ELSFF) MODEL ALGORITHM Let: which is the set of parameter estimates for (5.4a) at a fixed tactor-skin location C1 at time k; and let: be the regressor vector with entries that hold the time-delayed estimated current output, delayed and most recent measured voltage input, and the error input term from (5.4a) at time k. The error input, which is not measurable, can be approximated by the a posteriori residual εp(t), which is the measurable most up-to-date output error estimation: where Istim(k) is the measured stimulation current output at time k. Then, the finalized regressor vector becomes: The estimation model for (5.4a) to follow is in the form of a difference equation so that the adjustment model for it can be implemented online. Utilizing (5.5) and (5.8), the estimation linear parametric model (LPM) estimation is: 75 The iterative ELSFF adjustment model to update the parameters online in (5.5), under the constraint that the output estimation error should converge over time, is derived by minimizing the function: where N is the final time and the output estimation error at the kth step is given by the a priori residual: Here, λ is the forgetting factor that weights the recent residuals exponentially more significantly than older ones. Including the forgetting factor is necessary in the adaptation law because of the assumed time variation of the model parameters in (5.4a), which will help make the parameter estimates more responsive to reflect the electro-biological changes in the skin. The minimization problem is to drive the a priori residual εa(k) to zero over time. In this respect, the model output Îstim should approach physical plant output Istim. The ELSFF adaptive law was chosen because its good ability to filter measurement noise injected from the sensors since it accounts for previous sampled data as well as current. Figure 23 illustrates the inputs to the adjustment model to estimate the parameters for (5.9). Software gains K1 and K2 rescale the measured and sampled voltage and current from the skin-electrode tactor back to their original magnitudes once acquired and stored by the DAQ. The bio-impedance parameters estimates corresponding to (5.3) 76 are extracted per iteration by a one-to-one and onto function in the frequency domain that will be disused in the next section. Figure 23. Top level block diagram of parameter estimation using iterative ELS algorithm in Z-Domain. The iterative ELSFF algorithm has the following initial conditions: The P covariance matrix of the iterative ELSFF algorithm is selected to be a large magnitude, positive definite matrix as required. The adjustment model to identify the parameters every iterative cycle is calculated by the following algorithm: For k = 1,…,n, where n is an index for an even subset of the circular input buffer queue of length M for storing samples of VCVD_IN(VLoad(k)) and VCVD_IN(Istim(k)), 77 1. magnitude rescale the measured and sampled load current and voltage to the nearest quantization level: 2. λ 3. 4. υ 5. 6. 7. 8. Extract bio-impedance parameters from Z-model parameters. The algorithm is repeated by indexing the start of the next data subset in the circular input buffer queue, and so on, until the identification is stopped at time N. Normally, P(k) would have to be reset to P(1) in (5.12) if the minimum eigenvalue of P(k) becomes too small which implies adaptation slows by observing step 5. However, to save computation time, and the fact that almost always the forgetting factor is set to 78 be less than one, this can be neglected. Clearly, with λ < 1, the P(k) in step 3 grows exponentially fast when the CVD does not produce a pulse. This in effect allows P(k) to be reset autonomously. Also, since ϕ is always persistently exciting and bounded, P(k) will not become unbounded. K(k) in (5.17) is the adaptive gain matrix weight on the a priori error output estimation in (5.18). Ideally, the adaptive gain matrix should tend to zero as the error output estimation converges so that parameter estimates remain set. However, when ϕ is “turned off”, P(k) grows exponentially so that K(k) will be nonzero when ϕ “turns on” again, which injects noise in to the output and parameter estimation process. The tradeoff is that the smaller the forgetting factor value used, the more time responsive the ELSFF algorithm is to correcting for estimation error due to time varying plant. In contrast, if the forgetting factor is too small, the parameter estimation signal will be noisy due to gain sensitivity on the estimates. It is up to the operator to decide on an optimal λ for the application. The resetting moving mean square error derived from (5.10) is used as the goodnessof-fit criterion on the output estimation error over some time period given by: which resets for every time interval of length L. This gives the average power in the error output estimation over during the interval. 79 Assume the parameters and error input in 5.4b are known and C1 is a fixed skin-tactor interface location, the change in the estimated output current to that of the parameters or input as a function of time is related by: Assume that initially that the model (5.4a) estimates the measured current exactly and that the input is fixed. Suppose the plant becomes time varying; the first and second term, illustrate that the magnitudes of the resulting output estimation error around time k will depend on how quickly the adaptation law updates the respective model parameters to account for the plant time variation. As well, the last term output and its influence on the estimation error will be coupled to the time variation. Assume again that for the last two terms there is perfect output estimation and that now the plant is not time varying, then the third term relates the change in the output estimate to a change in the voltage input. In the absence of all other influences such as modeling error and noise, the change in output estimation from this model equals the change in measured current. The last term and its input account for the change in output estimation from the change in estimated output related to all model uncertainty. It acts as a correcting factor when the model fails to account for plant dynamics. In general, the magnitude of this term becomes large if (5.1.1) is not guaranteed or for sudden and unexpected time variation of the plant. As is implied by the assumption in (5.1.1), the nonlinear properties of the unknown physical plant become increasingly apparent as the stimulation current amplitude or 80 transition times of the CVD pulse increases. This later condition makes apparent the non-Ohmic DC resistance dependency on output current given (5.2) during the transitions of the pulse. The most important factor is that the pulse transition times are fast and that the stimulation current pulse amplitudes change slowly from pulse-topulse, then the adaptation law can construct an estimation model that gives good output estimation without the need of the last term in (5.4b); appropriately set forgetting factor aids in the responsiveness to steady state changes in the pulse. The empirical results of these phenomena will be shown in the experimental section 6 of this paper. The forgetting factor can be related to a designing time constant that helps with selecting the forgetting factor for the adjustment model to keep parameter estimation responsive to a time varying plant, assuming that the time constant for plant time variation is known. Let N represent the approximate current and past discrete time memory that are significantly weighted by the ELSFF in the adaptation law. The relationship to the forgetting factor is [33]: If N is neither too large nor small, then a time constant in the continuous time domain can be defined, which is used for approximating the lag time in the adaptation law it takes to respond to a continual time variation of the plant. The time constant is: Ts is the A/D sampling rate. N is the value from (5.23) selected by the operator based on experiment that is a tradeoff between tracking sensitivity and stability in the output and parameter estimation. If by assumption, the electro-biological plant of the skin has a 81 minimum time variation constant of τθ, then the time constant in (5.24) should be designed so that: That is the time lag in the adjustment model should be less than that of the significant variation in time of the plant so that the output estimation can track the actual output. Experimentally determining this condition helps with optimally tuning the adaptation law. Experiments that determined the acceptable choices of λ from N to obtain the lower bound for τθ in (5.25) were carried out in the experimental section 6 of this paper. 5.3 PARAMETER ESTIMATION FOR THE CONTINUOUS TIME LINEAR PARAMETERS BIO-IMPEDANCE (CT-LPBI) MODEL To iteratively estimate the bio-impedance parameters associated with the CT-LPBI model in (5.3) from the Z-domain model (5.4a), assume that the inequality from (5.25) holds, and that: which means that ELSFF algorithm should have a much greater updating time constant than the DAQ sampling rate time so that the adaptation law estimation is not prone to noise. The estimated bio-impedance parameters should model the underlying dynamics, which under ideal circumstances implies: 82 for all k in (5.22), where K1 is arbitrary time when the estimated parameters converge to the true parameters in (5.3) and (5.4a), respectively. As well, when (5.25-5.27) hold, the following terms in (5.22) become negligible with influencing output estimation: Based on experiment, (5.27) it is not guaranteed at high stimulation current levels from CVD pulses with slow rise and fall time transition. This is because the current signal must transition through its non-ohmic I-V characteristic, as seen by the relationship (5.2), which becomes more apparent as amplitude increases. The CVD square wave pulse on the other hand has fast rise and fall time compared to its steady state time duration versus the transition times of the other types of pulses explored, such as triangle- and saw tooth-shaped.; hence, (5.27) is almost always valid when estimating the CVD square wave load current when using the CT-LPBI model, which validates the set of assumptions presented in (5.1.1). However the stringency of the conditions for (5.1.1) can be relaxed depending on the specific application. Given that (5.25-5.27) hold over some time interval including time k, (5.22) reduces to the small-signal model term: The bio-impedance parameters can be found from the estimated parameters of (5.29) to get the physical impedance model: so that the following filters are equivalent: 83 To produce the filter equivalence, the Tustin transform is used on the current estimate of the model in (5.4a): By assuring that the DAQ A/D sampling rate Ts of the system is: model uncertainty that would occur from frequency aliasing due to under-sampling the output is eliminated since the pole location of the CT-LPBI model is always closer to the imaginary axis. Here, and the „starred‟ parameters is the „true‟ zero location and parameter set in (5.30a), and ωA and ωD are the radian analog and digital frequencies corresponding to the filters in (5.31b), respectively. Further, since the mapping (5.32) is one-to-one and onto, the frequency response is the same in both domains. The mapping functions from Z-domain parameter estimates to the S-domain are: So that the pole and zero locations of (5.30a) are, respectively: 84 where for all estimates: 85 6. EXPERIMENTAL SETUP AND TESTING There were two main classes of experiments that were undertaken with the haptic renderer and real-time identification of load parameters: proof-of-concept and human experimentation. Proof-of-concept experiments will be presented first to obtain a benchmark for the modeling, adaptation law and haptic rendering system when using known LTI circuit models attached to the output. In subsequent sections, the experimental setup, undertaking, and analysis involving human testing with the haptic rendering system, in regard to bio-impedance identification, plant modeling and psychophysical experiments, will be presented 6.1 HARDWARE AND ADAPTATION ALGORITHM TESTING What follows are proof-of-concept experiments to establish the workings of the hardware and identification algorithm to identify load impedances connected to the haptic driver with a model structure corresponding to (5.3). The method is to identify known parameters in RC networks and compare empirically these sets of values and model output estimations to the actual values and measured current responses, respectively. The purpose is to obtain evidence for convergence of parameter identification and output estimation under known loads assuming the ideal human bioimpedance model in (5.3). 6.1.1 EXPERMINETAL SETUP 86 Table 1 shows impedance parameter estimation sets from 10 subjects versus index fingertip condition that were fitted to the CT-LPBI model given in (5.3). The impedance values were obtained by using the offline bio-impedance identification setup and method, which is described in previous work of [34]. RC impedance networks with circuit component values used from Table 1 were built on a breadboard and re-identified using the online method described in section 5.2.1; the built loads to be driven by the haptic renderer are shown in Table 2. The element values for each network were measured with an LCR meter to assure accuracy of the standards. The set of values seen in Table 2 were chosen because they represent a range of typical human loading conditions that the revised haptic renderer should encounter. In Table 1, the index fingertips of the subjects were dry or moistened with tap water to simulate sweat. The parameter sets in Table 1 selected to build the RC networks were: #5 Damp, #7 Damp, #1 Dry, and #9 Dry. The two damp fingertip circuit models were chosen because the corresponding circuit impedances will load the haptic driver and measurement circuitry the greatest. Any Rp values in Table 1 that are below 100 KΩ were not used for testing due to safeguarding the hardware at this time. The measured impedance sets in Table 2 differ slightly from the corresponding values in Table 1 due to resistor or capacitor tolerances. All online identification experiments consisted of applying voltage pulses to the loads at a fundamental frequency of 60 Hz and 11 percent duty cycle. Online identification experiments were carried out with using the pulse types shown in Figure 17, the square, triangle and saw tooth waveform shapes, and also varying the CVD voltage amplitude pulse for each load under investigation. 87 Table 1. Bio-impedance parameters for 10 individuals using offline identification [34]: dry and damp Index fingertip case. 88 Table 2. Known RC circuit elements of impedance networks built on breadboard with known analog pole and zero frequency locations used in online identification testing: built using table 1 parameter sets. subject number Finger Model Rp (Ω) Cp (F) Rs (Ω) Fp (kHz) Fz (kHz) #5 Damp 1.25E+05 2.16E-10 1.59E+04 5.895 52.23 #7 Damp 1.65E+05 1.60E-10 1.98E+04 6.028 56.27 #1 Dry 1.94E+05 1.70E-10 1.90E+04 4.825 54.10 #9 Dry 2.72E+05 1.40E-10 2.22E+04 4.179 54.79 6.1.2 EXPERMINETAL RESULTS AND ANALYSIS The following experimental results were calculated from N=60 trials of one second recordings of load impedance identification, which were the four loads, each driven under the three CVD waveform pulses, with voltage amplitude that ranged from 100 to 140 volts using 10 volt increments. The rows of Table 3 show the average of an identified parameter in an impedance network for each known corresponding entry in Table 2, independent of amplitude and waveform type. 89 Table 3. Resulting online estimation of circuit element parameters and analog pole and zero frequency locations for the corresponding known parameters of the plants in table 2. Mean Parameter Values Sample Rp (kΩ) Cp (pF) Rs (kΩ) Fp (kHz) Fz (kHz) #5-DAMP 124.47 245.45 16.85 5.21 44.55 #7-DAMP 163.15 192.98 20.52 5.06 45.73 #1-DRY 192.21 195.97 18.80 4.23 47.98 #9-DRY 264.95 169.28 22.27 3.55 46.26 Table 4 summarizes the average absolute errors between the plant parameters and their corresponding estimated values for fixed haptic driver condition or driven impedance load. The entry in first column of the table indicates the condition that was held constant to calculate the corresponding row statistics. Error was not seen to be a function of pulse amplitude levels due to the high SNR and not held constant for this reason when calculating the descriptive statistics. There were no significant differences found in the mean absolute error statistics by looking down each of the columns in Table 4; thus for typical loading conditions on the haptic renderer, parameter identification accuracy is not influenced by amplitude, waveform pulse shape, or impedance load. As well the repeatability is sufficiently bounded when viewing the columns on reported standard deviation in identification for each parameter and condition. 90 Table 4. Absolute error and standard deviation between plant and estimated parameters: first column shows which variable held constant for calculating the associated row statistics. Variable Held Constant Mean Absolute % Error Rp Cp Rs Fp Absolute % Error S.D. Fz Rp Cp Rs Fp Fz None 1.62% 13.08% 9.89% 12.65% 13.05% 1.04% 7.47% 7.13% 2.76% 9.16% #5-DAMP 1.59% 13.64% 12.22% 11.54% 15.71% 0.96% 4.00% 9.00% 1.83% 11.04% #7-DAMP 1.35% 2.48% 9.23% 11.67% 8.66% 0.99% 1.75% 6.68% 2.32% 5.89% #1-DRY 0.96% 15.27% 9.66% 12.31% 11.33% 0.65% 4.46% 6.28% 3.55% 8.74% #9-DRY 2.59% 20.91% 8.46% 15.07% 16.49% 0.84% 2.11% 6.38% 1.45% 8.76% square 1.88% 12.21% 10.17% 11.55% 16.08% 0.97% 7.36% 7.89% 2.73% 9.46% saw 1.45% 15.65% 9.35% 13.81% 12.47% 1.06% 7.87% 8.01% 3.14% 9.48% triangle 1.55% 11.36% 10.15% 12.59% 10.59% 1.10% 6.79% 5.57% 1.94% 8.05% The absolute errors tended to be small for the identified Rp and larger for Cp and Rs. The actual Cp and Rs on the breadboard largely determine the high frequency zero location in the S domain, since Rp >> Rs in reference to (5.30a). Consequently, the Nyquist sampling rate of the DAQ (50 KHz) is close to the zero frequencies in Table 2. The increased absolute errors in the identification of Cp and Rs are likely associated with frequency aliasing due to under-sampling the load current and voltage measurements and not bandwidth limitations of the measurement circuitry itself; the measurement circuitry was designed to exceed the bandwidth of the CVD driver. Conversely, the estimated Rp has relatively small absolute error because its value is 91 mostly determined at DC frequency, dominates the much lower frequency pole location, and is not strongly coupled to the zero frequency location. The equations used to calculate the estimated parameters are given in (5.34a – 5.34c). Overall, the methods presented in chapter 5 for parameter identification for the CT-LPBI model given in (5.3) give good estimation of plant parameters when the plant is a known LTI impedance load. Parameter convergence for Cp and Rs could be improved by using DAQ cards with higher sampling rates. The ability of any one of the models presented in section 5 to estimate the output load current is another important assessment of the online ELS-FF adaptive law and modeling used for output estimation. In general, each model will give different accuracy of output estimation depending on the extent for which nonlinearity present in the human skin is excited under load. In the case of LTI plants used here for testing, that have CT-LPBI model structure, (5.3) should give equivalent estimates to that of the full DTC model structure in (5.4a). Hence, the impedance loads in Table 2 were driven by the haptic renderer to assess the output estimation accuracies for the measured outputs from each known impedance plant. Figure 24 shows the plot of the estimated output current produced from the DTC model in (5.4a) along with measured plant current versus time for one triangle wave pulse. The model output has been decomposed in to the outputs from the DT-SSC and CED filter components, as explained in section 5.2, to show that, with a known LTI plant, the DTSSC model is a highly accurate estimator. Here, the output from the CED filter is 92 negligible. The decomposed model outputs in the graph are called Imodel and IunModeled, respectively, and the measured current is called Imeas. The attached load was #5-Damp with 140 volt CVD amplitude pulses applied. The figure also gives the RMS values of measured current and output estimation current from the DT-SSC model that was averaged over the entire one second of recorded data. Their values are: ImeasRMS = 0.18991 mA and ImodelRMS = 0.18978 mA, respectively. (6.1) Also seen, the RMS value of the CED filter output, whose output corrects for unmodeled dynamics, is negligible in the micro-amperes since the load is highly linear: IunModeledRMS = 2.37 μA. (6.2) Figure 24 also demonstrated that the DT-SSC model accounts for the transient response of the measured current pulse to a good degree. These results were typical for all LTI loads in Table 2, relevant haptic renderer settings and waveform pulse types. 93 Figure 24. Measured and decomposition of estimated output current from terms in (5.4a) plotted by driving load #5-Damp in table 2: output for one triangle wave CVD pulse shown at 60 Hz, 11% duty cycle at Vpp = 140 volts, λ = 1. 94 Figure 25 was created by using recorded data from 600, 100 volt square wave pulses acquired in 10 seconds by driving the #1-Dry load impedance plant in Table 2. The graph shows the estimated and measured steady state current taken at each current pulse midpoint, which represents 600 trials of steady state output estimation. The known process parameters are clearly time invariant as is evidenced by the nonfluctuating measured output current. Also shown, are the expected values of the measured and estimated current using model (5.4a) averaged over the 10 second recording. The values are almost identical at: E(ImeasSS) = 0.47638 mA, and E(IcalcSS) = 0.47657 mA, respectively. (6.3) This result provides evidence that the estimations from (5.4a) are unbiased in the mean if the load is linear. Figure 25 also illustrates that the typical repeatability in the DTC adaptive model for estimating the 600 measured current pulses at steady state over time is high, since the standard deviation is in line with that of the measured steady state value: sn(ImeasSS) = 2.2988 μA, and sn(IcalcSS) = 2.0386 μA. (6.4) Thus, Figure 24 and Figure 25 taken together provide evidence that the models used for estimating the measured current are consistent in the mean when LTI loads are used. Experiments using the other test impedance loads in Table 2 gave similar results. Hence, the ELS-FF real-time adaptation law applied to the model structures in section 5 with associated driven LTI plants gives good output estimation of the measured current. 95 Figure 25. 10 second estimation of measured state current at pulse midpoints using model (5.4a). Results produced from square wave pulses driven under load #1-Dry in table 2. The mth step is the mth duty cycle pulse midpoint: output is at 60 Hz, 11 percent duty cycle at Vpp = 100 volts, λ = 1. 96 Another desirable quality of the adaptive algorithm is to update the model in a way that produces exponential convergence in output estimation. The output estimation error convergence time was assessed using (5.21) for the driven LTI plants in Table 2 for one recording of data. Figure 26 shows the results obtained for #5-Damp impedance load. The algorithm converges in about 100 milliseconds and shows exponential convergence. This is the nominal convergence time for the other impedance loads in Table 2. 97 Figure 26. Typical output estimation error convergence characteristic versus time calculated from (5.21) when driving LTI loads in table 2: assessed for #5-Damp impedance load. 98 6.2 EXPERIMENTATION Tactile stimulation was administered to human subjects to test the capabilities and quantify the power output of the haptic renderer, as well as to test the identification algorithm using a sample population. By using a sample population, the haptic renderer was subjected to robust loading conditions because of the wide differences in electronic skin impedance between subjects. This was one of the motivating factors behind human testing. Another reason for using a sample of humans for testing was so to gain initial insight on and study the comfortable stimulation ranges with the haptic renderer versus waveform pulse type and stimulation level, which can differ between electronic tactile display devices. Doing such an experiment can help aid in future calibration of the device by knowing the statistical bounds in stimulation ranges. The comfortable range is defined as the sensation and pain threshold and is a subjective measurement based on the user‟s verbal response to a known stimulation level. In terms of incrementing the voltage pulse amplitude for a fixed waveform pulse shape, the sensation threshold is defined as when a user first notices a stimulus at an amplitude level and the pain threshold is when the user feels that the sensation at the current level is just barely tolerable. Besides the thresholds varying between individuals, it changes based on how well the user is trained on the device so that the user should have some initial training before these bounds are recorded. 99 The study can also be used to gain insight in to the ways of reducing power consumption of the haptic driver by statistically analyzing how driver power output is related to the average population dynamic range or threshold limits versus waveform pulse shape. Dynamic range is the defined as the ratio of the pain over the sensation threshold. Due to time constraints, psychophysical assessment could not be carried out for just noncable detection and pulse width duration, which would also be important for calibration. Thus, human experimentation can be subcategorized as either collection of objective measurements or psychophysical assessment. Both of these experiments were carried out concurrently for every participant to compose possible statistical relationships. As well, the results of real-time identification on the models introduced in section 5 are explored both statistically and in the time domain when applied to the human population under study. 6.2.1 EXPERIMENTAL SETUP In the following sections, it is assumed that the skin region to be stimulated with the tactile display is on the surface of the index fingertip. The sample size for human experimentation is N = 21. The sample group consisted of individuals 19 to 33 years of age with approximately two-thirds of participants male. The finger condition was kept free of sweat or moisture and otherwise not altered any other way for the entire experiment. The required voltages to stimulate the underlying tactile receptor axons for these conditions, and with the given electrode diameter of roughly one millimeter, are typically between 100 to 300 volts [3] [4]. A visual inspection of the fingertip also took place to assure no abrasions, scratches or open wounds; these defects may cause 100 painful stimulation to occur at low voltages due to compromised stratum corneum. A damaged stratum Corneum can decrease resistance an order of magnitude. The user was first trained how to use the tactile display including: showing the subject how to properly place the index fingertip on the electro-tactile display, hooking up the return electrode, and letting the user experience the tactile sensation beforehand. Figure 27 shows the typical arm, hand, and index fingertip placement for the display. Figure 28 shows the area on the palm to affix the return electrode for best skin contact and minimal skin resistance due to the large planar area and lack of hair follicles. The tactile display is designed so that the index finger can rest comfortably in the same position over multiple trials of an experiment. 101 Figure 27. View of user forearm resting comfortably on gel pad while index fingertip is placed firmly on tactile display for stimulation and measurement. 102 Figure 28. View of anodic return electrode proper placement secured with a Velcro strap on thenar eminence. Before testing commenced, each individual was given the experience of electronic stimulation on the device for a short period by using the three voltage pulse shapes (square, saw and triangle waveforms). The voltage amplitude was varied for each waveform pulse type so that the user could become accustomed to the sensation. This served two purposes: 1) it helped to determine if the subject‟s fingertip would spontaneously sweat after application of high voltage; 2) it gave the user a priori knowledge about the range of sensations that would be experienced from electronic tactile stimulation. In the first case, the user was rejected as a participant. The training helped to reduce the novelty of sensations and psychological apprehension to high voltages brought about by electronic stimulation. This gave each person a similar point of reference for the questions that were asked for the psychophysical portion of the 103 actual experiment. It also helped reduce the chance of the subject becoming startled, which would undoubtedly influence the results due to variation of pressure on the skinelectrode interface. This was sometime noticed in earlier offline experiments. A weight scale was placed underneath the tactile display to indicate the amount of pressing force applied from the user‟s fingertip to the tactile display. A participant was asked to keep the pressure between 8 and 16 oz for consistency between measurements and the sample population as a whole. The subject was also asked to keep the fingertip still during all trials in an experiment as bio-impedance can change over potentially small distances on the skin. Over time, cellular residue deposits on the electrodes were observed that influenced the measurements by increasing the effective resistance between the fingertip and electrode interface. Therefore, the electrodes were cleaned with isopropyl between each experiment (three experiments per individual). As well, since the return electrode is made of copper, it was cleaned after a session with a subject was completed using vinegar to remove oxidation. An experiment consisted of applying one of three CVD voltage waveform pulses in the following order: saw tooth, triangle, and square wave, for a total of three experiments. Each experiment consisted of multiple trials. A trial involved administering the voltage pulses at given amplitude, recording the results, and noting the feeling reported by the user. Voltage pulses started at 30 volts. The bio-impedance parameters and waveforms 104 were recorded for one second once steady state was reached before the next voltage increment. In this case, steady state meant that the observed identified impedance parameters did not change significantly. A small recording time was used to decouple possible influence of time variation at the active skin-tactor interface in the recordings. The amplitude was then incremented or the experiment concluded based on whether the pain threshold of sensation for the user was reached. The recorded parameters include continuous and discrete time, Z- and S-domain estimated model parameters for the models in (5.3) and (5.4a), and the current and voltage measurement samples versus time. For a trial, when the user first noticed a sensation, this value was noted as the sensation threshold. When the trial reached the subjective pain threshold experiment was stopped and level noted. The user was then asked to report what made the stimulation level uncomfortable for him or her for later analysis. After an experiment was completed, the subject was given a few minutes of rest period. This helped to reset any short-term conditioning from sensory adaptation that might bias the results. After this rest period, the next waveform in the set was applied and the process repeated; it was not necessary that the fingertip be in the exact location between experiments as this would be difficult to accomplish. Each experiment had the duty cycle set at 11 percent with fundamental frequency of 60 Hz, regardless of waveform type. Only changes in amplitude or selection of waveform pulse shape were studied. 6.2.2 EXPERIMENTAL RESULTS USING STATISTICAL ANALYSIS 105 The first study involved determining how the three different waveform shapes influenced the dynamic range for the sample population statistically. The stimulation dynamic range is of interest and is a function of the RMS stimulation current. The second study exams the predicted power based on the estimation models presented in section 5 that is delivered from driver to the interfaced tactor versus waveform pulse shape. The actual power transferred to the fingertip load predicted by a particular estimation model will be assessed based on the waveform pulse shape and threshold levels. Finally, the third experiment evaluates how the small signal bio-impedance model parameters vary as a function of the output current. This should help aid in developing the model used in the design of and adaptive control system for the implemented CVD method. 6.2.3 ELECTRO-TACTILE DYNAMIC RANGE STUDY Figure 29 shows the mean stimulation dynamic range ratio in decibels for the sample population as a function of waveform type. The graph is a result of subjective reporting by the subject of his or her sensation thresholds relative to the physical measurement of current stimulation amplitude and waveform type at time of recording; though the reporting is subjective, the sensations experienced at pain threshold among subjects share some commonality, as will be discussed. The samples of dynamic range ratios for the population from each waveform type appeared normally distributed when analyzed, which is supported by Q-Q plot analysis (R2 > 0.90 lowest) and rejection of significant skewness and kurtosis at the 5% significance level for data collected for each waveform type. The mean of the dynamic range ratio in stimulation current for applied square wave pulses is significantly lower than the mean of the saw tooth (t(20) = 3.46, p < 0.05 106 two-tailed) and triangle wave (t(20) = 3.49, p < 0.05 two-tailed) dynamic range ratios for the population. Figure 29. Mean dynamic range ratio for RMS stimulation current for each type of CVD output waveform pulse (N=21). The 95% confidence interval is included. The sample mean and standard deviation for the square saw tooth, and triangle wave are: M = 4.78, SD = 1.86; M = 7.21, SD = 2.69; M = 7.29, SD = 2.74, respectively. Figure 30 shows the average of the RMS currents measured at the reported sensation and pain thresholds levels resulting from the applied pulse type to the sample population. The measured current levels of the thresholds all appeared normally distributed supported by Q-Q plot analysis (R2 > 0.93 lowest) and rejection of significant skewness and kurtosis at the 5% significance level for each empirical distribution. The 107 mean of the measured currents at sensation thresholds for the square wave is level shifted significantly higher compared to the respective means of the saw tooth (t(20) = 6.10, p < 0.05 two-tailed) and triangle wave (t(20) = 6.79, p < 0.05 two-tailed) counterparts. This is also the case for the applied square wave pulses when compared to the means of the RMS current levels at pain threshold for applied saw tooth (t(20) = 3.40, p < 0.05 two-tailed) and triangle wave pulses(t(20) = 2.82, p < 0.05 two-tailed). The means of the current stimulation levels from applied saw tooth and triangle pulses to individuals showed no difference at the 5% significance level for the respective threshold level reports by the individuals. Figure 30 indicates that subjects are more sensitive to stimulation currents that are produced from saw tooth and triangle wave CVD pulses versus the square wave; this is indicated by observing the shifting in the levels of both the threshold means when comparing each applied waveform pulse type. However, the subjects appeared more sensitive to the gradations in the stimulation current levels for produced by applying square wave pulses, which is supported by the calculated dynamic range ratios for each waveform type in Figure 29. 108 Figure 30. Average sensation and pain thresholds for stimulation currents versus wave form (N=21). The 95% confidence interval is shown for the mean. Figure 31 and Figure 32 display the variance in the data for the tabulated results corresponding to the means displayed in Figure 29 and Figure 30, respectively, in the form of box and whiskers plots. Figure 31 shows the box and whiskers plot for the variation in calculated dynamic range ratios corresponding to the means in Figure 29. When the minima and maxima data under saw tooth and triangle waveforms pulses are considered outliers, the percentiles in the box and whiskers indicate approximately equal variance in the dynamic range ratio in all three waveform types under study for the population. Figure 32 shows the box and whiskers plot for the variation in measured stimulation current threshold levels in the sample population corresponding to the 109 means in Figure 30. As can be seen, the sensation and pain thresholds for the same waveform type show no overlap at their upper and lower quartile boundaries, respectively, which indicates that there is a statistical commonality in reporting these levels for a given applied waveform pulse type for the population as a whole. When the graph is viewed across applied waveform pulse type, there is negligible to no overlap in terms of the variances in the measured sensation and pain thresholds, which now represents a commonality in reporting of these sensations for all applied waveform types. The pain threshold for the applied square wave has significantly greater variance compared to all other sensation and pain thresholds whose respective percentile variances are about the same. This may be related to subjects being more sensitive the change in stimulation current levels for the square wave pulse type leading to greater variation in reporting, as was discussed in relation to Figure 29. 110 Figure 31. Five statistic box and whiskers plot for mean dynamic range ratios for RMS stimulation current versus waveform shape (N = 21). 111 Figure 32. Five statistic box and whiskers plot for mean sensation and pain thresholds concerning RMS stimulation current versus waveform shape (N = 21). Figure 33 displays the means of the applied lower and upper CVD voltage pulse amplitudes to the subjects corresponding to the measured lower and upper stimulation current thresholds, respectively, for each waveform type. These results did not fit well with a normal distribution. However, from the graphs it can be inferred that lower and upper CVD voltage pulse amplitudes for saw tooth and triangle wave pulses must be higher to achieve the respective stimulation thresholds than the square wave CVD pulses on average. Also, the square wave voltage pulse amplitude dynamic range is seen to be much narrower than that of the other waveforms. 112 Figure 33. Mean CVD Voltage Amplitudes for corresponding RMS current thresholds versus CVD pulse type (N = 21). Once the pain threshold for the individual was reached, the experiment was stopped. The subject was then asked to describe what made the experienced stimulation current level uncomfortable for the given waveform pulse type applied in his or her own words; no pre-defined set of keywords were provided. The summary of the keywords are shown in Table 5 along with the reported keyword frequency. The groupings were created by the researcher based on commonalities. By far, the most reported unpleasant sensation was needle poking, followed by intense vibration and stinging. In many instances, these tactile experiences transitioned or occurred concurrently. For instance, for some, a pleasant vibration transitioned to an intense vibration with a 113 concentrated needle poking sensation that became noticeable as stimulation levels increased. For this reason, the groupings are not mutually exclusive. The importance of Table 6 will be to help determine if the experiences of maximum tolerable sensation have a common theme for future analysis even though the descriptions are subjective. In this respect, a list of precise keywords can be formed. A list of the well-defined sensation keywords can be shown to the user once his or her upper threshold sensation level is reached. In this respect, an individual who trained in calibration of the haptic renderer can help the user determine his or her maximum tolerable sensation levels based on objective keywords. As well, it gives the user an initial appraisal of what to expect at the typical level of upper sensation threshold. 114 Table 5. Pain threshold sensation description reported from subjects and frequency of occurrence for the three CVD waveform types. Sensation needle poking intense vibration stinging numbness buzzing muscle tension or twitching or shaking burning pinching unpleasant slight electrical shock itching cut in flesh Friction Description Keywords for Pain Threshold Saw Tooth Wave Triangle Wave Square Wave #1(M), #2(F), #3(M), #4(F), #5(M), #9(M), #11(M), #18(M), #21(F) #1(M), #5(M), #7(M), #10(M), #14(M) #6(M), #12(M), #1(M) tot al #1(M), #4(F), #5(M), #1(M), #4(F), #5(M), 23 #6(M),#9(M), #11(M) #6(M), #8(M), #9(M), #11(M), #18(M) #1(M), #3(M), #5(M), #7(M), #14(M), #16(M) #12(M), #15(M) #1(M), #11(M), #14(M), #16(M) 15 #12(M), #20(F) 7 N/A #13(F), #19 (M) #11(M), #18(M) #2(F), #13(F), #14(M) N/A #19(M) #2(F), #14(M) #13(F), #19(M) #15(M) 5 4 4 #8(M) #20 (F) #17(M) #8(M) #21(F) #17(M) #8(M) #21(F) #17(M) 3 3 3 N/A N/A N/A N/A #18(M) #20(F) #3(M) N/A N/A 1 1 1 6.2.4 CVD POWER OUTPUT STUDY Figure 34 gives a bar chart of the mean and standard error in apparent power delivered from the driver, which is shown tabulated at reported sensation and pain thresholds, as a function CVD waveform pulse type. For each applied waveform pulse type, the apparent power data was calculated using the measured values at the reported sensation and pain thresholds for the individual (N = 21) under study, respectively, and averaged accordingly. The apparent power is the product of the RMS recorded current 115 and voltage for a delivered waveform pulse type. The amount of power delivered from the CVD is comparable to the power outputted by a typical red LED laser pointer. It is easily seen, that no matter which waveform pulse type is delivered to the fingertipelectrode interface, the means in apparent power are about equal when calculated at the respective subjectively-reported threshold level for the population. A two sample KS test was done to statistically test the hypothesis that the apparent powers calculated at a particular threshold came from the same empirical CDF to justify the above observation. In neither case, could the hypotheses be rejected at the 5 % significance level; see Table 6 for the test results. This is in contrast to the statistical results of physical measurements (such as current and voltage magnitude) at subjectivelyreported threshold levels in previous sections, which were a function of delivered waveform pulse type for the sample population. However, the mean apparent power at the sensation and pain threshold levels remains constant in the mean irrespective of wave form pulse shape. This suggests that the power delivered to the fingertip is more of a reliable predictor of the reported threshold levels than stimulation current levels no matter the shape of the delivered waveform pulse type for the ones used in this study. 116 Figure 34. Apparent power bar chart shown at mean sensation and pain stimulation current thresholds for each individual (N = 21) and each CVD waveform type: standard error bars are shown. 117 Table 6. Two sample KS test for similarity between ECDFs of apparent power distributions for CVD waveform type (5% Significance): samples of apparent power for each waveform type are calculated at sensation and pain threshold for each subject (N=21). df sample comparison SW. vs. STW S. Threshold Apparent Power SW. vs. TW S. Threshold Apparent Power STW. vs. TW S. Threshold Apparent Power SW. vs. STW P. Threshold Apparent Power SW. vs. TW P. Threshold Apparent Power STW. vs. TW P. Threshold Apparent Power p KS H0: The two independent samples come from same ECDFs 20 0.1545 0.333 not rejected 20 20 0.1545 0.9728 0.333 0.143 not rejected not rejected 20 0.9728 0.143 not rejected 20 20 0.9728 0.9728 0.143 0.143 not rejected not rejected 6.2.5 STATISTICAL STUDY BETWEEN ESTIMATION MODELS AND PLANT OUTPUT ESTIMATION ERROR This section concerns presenting statistical results for how well the estimations models presented in section 5 estimates the measurable dynamics of the underlying human electrical bio-impedance plant. The discrete time conductance (DTC) estimation model in (5.4a) is an additive combination of the discrete time small signal conductance (DTSSC) model and correlated error dynamics (CED) filter introduced in section 5.2, where the CED filter and its modeling error input correct for un-modeled dynamics, which are heavily influenced by the CVD voltage pulse shape delivered to the skin-electrode interface. The DT-SSC estimation model is parameterized and structured in such a way that its estimation parameters map to the continuous time linear parameters bioimpedance (CT-LPBI) model presented in section 5.1 and given in (5.3); further, when the assumptions of (5.1.1) are met, these estimation models will be equivalent since 118 the CED filter should be negligible. It is desired that the CT-LPBI model and its parameters reflect the measurable dynamics of the plant when in varies in time and thus should estimate the output accurately, independent of the shape of the voltage pulses to an extent. The statistical approach used in this subsection to study efficacy of the estimation models, based on the stated conditions, involved analyzing the power inefficiency or efficiency derived from the estimation outputs from either the DTC or CT-LPBI model, or measured load current. What will be used is statistical mean square analysis for quality of fit between the selected model compared to human plant estimated or measurable stimulation currents, respectively, for the individuals under study (N= 21). In this respect the modeling error power is equivalent to the CED filter output. Time domain results and analysis for estimating measured bio-impedance load current in terms of the efficacy of which estimation model is used versus the types and magnitude of input waveform applied to the skin are presented in sections 6.2.9 and 6.2.10 of this paper. The CED filter output power, and the DTC model and CT-LPBI model output power in the graphs that follow are normalized to a one Ohm test load so that the power calculations are the square of the estimated current outputs for each model type. Since the method deals with power inefficiencies or efficiencies, the results were normalized 119 to either DTC model estimated or measured power consumption depending on the context to from the ratios. The average of the power inefficiencies was calculated from the CED filter power output in modeling error normalized to the corresponding modeling power output calculated from the DTC model. The power output calculations from each of the model components was done using the output estimation in stimulation currents given by the respective model component estimated for every subject at a specific reported threshold level; then the averages in the modeling error represented by power inefficiency per delivered waveform pulse type at a reported threshold level by the subjects could be calculated. The calculations were done at the sensation and pain thresholds for each member of the sample population and for each waveform type. The results of the power inefficiency in modeling are shown in Figure 35. The smaller the power inefficiency for a given applied voltage waveform to the fingertip, the more accurately the output estimation from the CT-LPBI impedance model accounts for the measured load current dynamics in the mean square sense (assuming that the CT-LPBI model reflects the dynamics of the plant). This is because the CED filter function has as its input the difference between measured and a posteriori estimation current, as is seen in (5.7). As the L2 norm of this estimated error input decreases, then so too does modeling output estimation error. The illustrations in Figure 35 give evidence that the physical impedance, as represented by the CT-LPBI model, best represent the dynamics of the measured output current for square wave CVD pulses. Referring to the same figure, the mean square power inefficiency for either applied saw tooth saw tooth or triangle 120 waveform pulses are relatively worse. The error of the physical impedance modeling rises with voltage amplitude except for square wave input when normalized to the estimated power from the DTC model. 121 Figure 35. Modeling error given by power inefficiency. Percentages of average power consumed in the CED filter normalized to average power estimated in the DTC model for all waveform types calculated at reported threshold levels: results graphed at the sensation and pain thresholds of subjects (N=21). Figure 36 is included to show that the DTC model output is statistically predictive in the mean square to the measured stimulation current output. This is indicated by the high power efficiency between the estimated and actual power delivered to the fingertiptactor interface by the CVD for each voltage waveform pulse type and current amplitude level. However, the square wave input gives the best overall power efficiency for the DTC model. The efficiencies were calculated at each subject‟s sensation and pain threshold levels. Thus, the DTC model is valid for the human loaded tactor in the mean square. A desirable indicator is that the DTC model power efficiency improves with increased voltage amplitude for all three waveform types, which is shown to occur. 122 Figure 36. Mean power efficiency of DTC model estimation output normalized to measured power delivered to skin-tactor load for each waveform type: results calculated at the reported sensation and pain thresholds of subjects (N = 21). Figure 37 is similar to Figure 36. Instead, it displays the mean power efficiency using the physical impedance (CT-LPBI) model output estimations normalized to the corresponding measured powers delivered to the human-loaded tactor under study. As with the previous figures, the efficiencies are derived as a function of applied CVD waveform pulse type and reported threshold levels in the sample population. Comparing Figure 37 to Figure 36, the efficiencies are slightly less for each waveform as expected and the variance shown by the standard error intervals larger. The reason being is that the CED filter component is no longer included in the estimated power output as was 123 the case in the previous results. The effects on efficiency by dropping the un-modeled dynamics term is least pronounced for the square wave input, as expected. However, the physical impedance models are still a good predictor for the actual plant impedance in the mean square and on the average for all waveform types and current stimulation levels, as is represented by the calculation at reported threshold levels. Figure 37. Mean power efficiency of physical impedance (CT-LPBI) model estimation output normalized to measured power delivered to skin-tactor load for each waveform type: results calculated at the reported sensation and pain thresholds of subjects (N = 21). 6.2.6 OBSERVATIONAL RESULTS FOR LARGE SIGNAL I-V CHARACTERISTIC It is desirable to quantify and model the large signal behavior for: 1) how the bioimpedance parameters change with stimulation current level; 2) and how the trends in 124 current versus voltage characteristic curves behave. Though the bio-impedance model is updated online, and has RC circuit elements associated with the electricalbiochemistry of the skin, it is not robust. It cannot account for the physics that cause variation of its parameters, in general, unless the conditions in 5.1.1 are met. Statistical approaches to be presented in the next subsection were used to study the large-signal behavior in the measured output based on the collected data of the sample population to study future improvements that may be made to modeling the dynamics. In this section, the observations in large signal I-V characteristics are presented. Figure 38 plots the identification of the Stratum Corneum-modeled electrical resistance Rp and the deep dermal tissue modeled electrical resistance Rs for each subject as a function of RMS stimulation current. The CVD input was square wave for one second identification and recording time, as previously stated. No participant experienced a sensation below a 60 Volt pulse. The voltage pulses began at 30 volts for all subjects and ended at the increment when the subjective pain threshold for the subject was reached. Rp decreases at least an order of magnitude over the entire stimulation region for almost all individuals. Another observation is that the variability in Rp for low stimulation currents is initially large but seems to take on similar minimal magnitudes as stimulation current increased to each subject‟s maximum threshold. The Rp minima magnitude seems asymptotic to about 100 kΩ at the maximum pain stimulation threshold possible for any member of sample population. The shapes of the Rp characteristic curves are in agreement with others who have studied the electrical impedance models for tactile displays. 125 It was also observed from Figure 38 that Rs is always smaller in magnitude than Rp over the entire stimulation current domain. As well it shows hardly any variation in magnitude as a function of stimulation current or subject. This is also in agreement with others who have studied similar models. Figure 38. Identification of Rp and Rs as a function of stimulation current and subject (N=21): results are from square wave CVD pulses starting at Vpp = 30V and incremented to pain threshold for each subject. Figure 39 displays the I-V characteristic curves associates with each subject as a function of the CVD square wave input voltage amplitude. The values plotted are for 126 measured and process model predicted load current. This shows that the predictor can track RMS current over a large variation in load current. The current is observed to rise rapidly in most cases as the voltage pulse amplitude increases past a certain threshold depend on the subject. From this, we can observe that even small changes in input voltage can lead to large changes in stimulation current. For small voltages near 30 volts, the RMS current tends to be similar in value for all individuals. The nonlinear increase in current is a result of positive feedback from Rp as it is an inverse function of the stimulation current. 127 Figure 39. Measured and DTC estimation model (from 5.4a) I-V characteristic curves for each subject as a function of square wave CVD pulse amplitude and subject (N=21). 128 Figure 40 plots each identified and recorded intracellular capacitance Cp value for the physical impedance model spanning the voltage amplitude increments for each subject. This graph is shown for square wave input and plotted against the measured stimulation currents. The trend shows an increase in capacitance as stimulation current increases that occurred for almost every individual. Prima facie, the increase appears affine linear for most individuals as a function of current. The rate of increase is less than that of Rp over the region of interest; Except for two outliers, the change in value of Cp was at most a factor of 3 for any individual, whereas Rp can vary greater than a factor of 10 for the same stimulation current domain. As well, the capacitance values are grouped closer together over the current stimulation domain versus Rp. The outliers are likely a result of increased ionic moisture content at the point of stimulation that facilitates dielectric polarization. 129 Figure 40. Identification of Cp as a function of stimulation current and subject (N=21): results are from square wave CVD pulses starting at Vpp = 30V and incremented to pain threshold for each subject. 130 6.2.7 STATISTICAL RESULTS FOR LARGE SIGNAL I-V CHARACTERISTIC AND LARGE SIGNAL MODELING For the following results assume that all CVD outputs were generated from square wave pulses starting at 30 volts and incremented to the point of pain threshold for a specific subject. As well, it must be assumed that the results are valid only if the fingertip is mostly dry and not altered or abraded and that dimensions of the tactors (stimulating electrodes) on the PCB are of the dimensions discussed in the hardware design section of this paper. Figure 41 shows the box and whiskers plot for the average of all identified R s values per individual (N=21). As seen, the median average Rs value is 62 kΩ for the sample population. For this tactile display most of the averages of the Rs values fall between 45 and 70 kΩ. 131 Figure 41. Five statistic box and whiskers plot produced from each average Rs value per individual from 30 volts to maximum pain threshold voltage (N = 21). Figure 42 is the bivariate plot of all identified Rs values (N = 278) versus stimulation current, independent of subject. This scatter plot clearly indicates that there is no correlation between Rs and stimulation current. A Spearman Rank correlation test showed that there is only a small correlation (rs(276) = 0.1877, p < 0.05 two-tailed). 132 Figure 42. Scatter plot of all identified Rs values versus stimulation current independent of subject (N=278): results generated from applied square wave pulses from 30 volts to maximum pain threshold voltage per individual. Figure 43 displays the scatter plot of all identified Rp values versus stimulation current irrespective of subject (N=278). The stimulation current is a function of the voltage amplitude. This graph displays two related regression curves for the data for the following two cases: 1) the power-fitted regression curve calculated from the Rp scatter plot that gave the best goodness-of-fit (R2 = 0.874); 2) the mean of the fitted power function regression curves to each subject‟s data, which itself is a vector of entries produced from a power function (R2 = 1). The motivation for case 2 was to build the lower 5 percent and upper 95 percent confidence intervals for any identified Rp value as 133 a function of stimulation current. As seen, this regression curve is qualitatively similar to the scatter-plot regression curve. The goodness-of-fit for case 2 ranged from R2 = 0.9587 to R2 = 0.9981 with a mean of R2 = 0.9730 and standard deviation of 0.02991 for the sample population (N =21). Testing the values of these curves at a particular stimulation current for normality was done at 4 different current magnitudes using Lilliefors (0.080 ≤ K(20) ≤ 0.139, 0.332 ≤ pCalc ≤ 0.979, p < 0.05), Q-Q plot analysis (R2 >0.9509),kurtosis (-0.104 ≤ γ2(20) ≤ 1.70, p < 0.05), and skewness ((-0.0256 ≤ γ1(20) ≤ 0.496, p < 0.05). Testing indicated that the interpolated Rp values for the population could not be rejected for normally; hence, the conclusion is that the confidence intervals represent the lower 5th and upper 95th percentile for the identified Rp values referenced to the power-fitted curve of the second case. As well, the mean sensation and pain stimulation thresholds at 0.0998 and 0.176 mA corresponded to 273±103 kΩ and 186±67.9 kΩ stratum corneum skin resistances, respectively, with tolerances at the 5 percent significance level. 134 Figure 43. Scatter plot of identified Rp values for sample population (N = 278) as a function of stimulation current with power-fitted regression curve using square wave CVD pulse: lower 5th and upper 95th percentile confidence intervals are shown for identified Rp calculated using the mean of the Rp power-fitted regression curves for each individual (shown as the dashed line). 135 The histogram of Figure 44 is included to compare the percent error between all calculated Rp and Interpolated Rp over the respective means of perceptive stimulation thresholds (sensation and pain) for the population derived in the psychophysics experimental section. The histogram shows evidence of a slight positive bias for the Rp interpolation estimator. Otherwise, the figure illustrates that the error between identified and interpolated Rp values is small. This result further suggests that the confidence intervals on the calculated Rp are reliable bounds for the sample population versus stimulation current if the interpolated Rp is used to calculate the bounds. Figure 44. Histogram of percent error between the identified Rp and Interpolated Rp values using square wave CVD pulse (N = 278). 136 Figure 45 displays the scatter plot for CVD square wave amplitude versus the corresponding RMS measured stimulation current (V-I) irrespective of subject (N=278). This graph displays two related regression curves for the data for the following two cases: 1) the logarithmic-fitted regression curve calculated from the V-I scatter plot that gave the best goodness-of-fit (R2 = 0.6924); 2) the mean of the fitted logarithmic regression curves to each subject‟s data, which itself is a vector of entries produced from a logarithmic function (R2 = 1). The motivation for case 2 was to build the lower 5 percent and upper 95 percent confidence intervals for any measured V-I pair. As seen, this regression curve is qualitatively similar to the scatter-plot regression curve. The goodness-of-fit for case 2 ranged from R2 = 0.9594 to R2 = 0.9989 with a mean of R2 = 0.9866 and standard deviation of 0.01165 for the sample population (N=21). Testing the interpolated voltage amplitudes produced from these curves at a particular stimulation current for normality was done at 4 different current magnitudes using Lilliefors (0.113 ≤ K(20) ≤ 0.139, 0.350 ≤ pCalc ≤ 0.703, p < 0.05), Q-Q plot analysis (R2 >0.9523), kurtosis (0.320 ≤ γ2(20) ≤ 0.597, p < 0.05), and skewness ((-0.202 ≤ γ1(20) ≤ 0.738, p < 0.05). Testing indicated that the interpolated voltage amplitudes for the population could not be rejected for normally; hence, the conclusion is that the confidence intervals represent the lower 5th and upper 95th percentile for the measured voltage amplitudes referenced to the logarithmic-fitted curve of the second case. As well, the mean sensation and pain stimulation thresholds at 0.0998 and 0.176 mA corresponded to 97.12±18.93 volts and 120.92±22.28 volts in amplitude, respectively, with tolerances at the 5 percent significance level. 137 Figure 45. Scatter plot of measured CVD Vpp for sample population (N = 278) as a function of stimulation current with log-fitted regression curve using square wave CVD pulse: lower 5th and upper 95th percentile confidence intervals are shown for measured CVD Vpp calculated using the mean of the Vpp log-fitted regression curves for each individual (shown as the dashed line). Figure 46 gives the histogram included to compare the percent error between all measured CVD voltage amplitudes and Interpolated values over the mean perceptive stimulation thresholds for the population derived in the psychophysics experimental section. The figure illustrates that the error between the measured and corresponding interpolated CVD voltage amplitude values is small. This result further suggests that the confidence intervals on the actual CVD amplitude are reliable bounds for the sample 138 population versus stimulation current if the interpolated CVD amplitude is used to calculate the bounds. Figure 46. Histogram of percent error between the measured CVD and interpolated CVD amplitude values using square wave CVD pulse (n = 278). 6.2.8 OUTPUT ESTIMATION UNDER TIME VARIATION STUDY The experiment carried out was to test how responsive the ELS adjustment model could be with updating the parameters in the DTC model given in section 5.2 under simulated time variation of the plant by manipulating the value of its forgetting factor. The motivation for the experiment originates from notable time variation in human bio- 139 impedance loading on the tactor that increased with increasing stimulation current amplitude level for the participants. Two different forgetting factor coefficients were used to see how well the process model could track the measured load current. Namely, λ = 1, where previous data is not exponentially weighted, and λ = 0.99975, where previous data is exponentially weighted as a function of discrete time. The previous data becomes less influential in the update model for this latter case. Only one subject was used for this test. The conditions for the test were administered square wave CVD pulses to the index fingertip-electrode interface set at 90 volt amplitude. For each forgetting factor experiment, the fingertip of the subject was placed on the tactile display with the estimation and measured output results of the stimulation current recorded for 10 seconds in each case. In the two experiments, the subject simulated his impedance time variation by scanning his index fingertip slowly across the stimulating electrode under measurement for the 10 second duration. The motivation for the experiment was to determine the proper forgetting factor value to use so that applying the haptic renderer and model estimation algorithm to a sample population gives accurate results. Figure 47 shows the real-time results of the output estimation of measured stimulation current over time with λ = 1. Only the steady state points, taken at the midpoints of the measured and estimation model output current pulses, for each pulse are plotted for clarity. The midpoint is a point in the active duty cycle of square wave voltage pulse. The dashed line is the measured, and the solid line the estimated output current to load, 140 respectively. The measured current changed as a function of time when constant voltage pulses were applied, as illustrated, when fingertip scanning occurred. The figure shows that the DTC model output time lags in estimating the output with plant time variation present when the adaptation law has no forgetting factor set when updating the model parameters. 141 Figure 47. Output estimation of measured stimulation current for human subject using DTC model in (5.4a): square wave CVD pulses with λ = 1.0, time = 10 sec, and Vpp = 90V. 142 Figure 48 illustrates the estimation results of the second experiment with the exponential forgetting factor set slightly less than unity: λ = 0.99975. By setting the forgetting factor, and repeating the fingertip scanning procedure over the relevant electrode, the estimated model now estimates the measured current pulses to a high degree. By using equation (5.23) in section 5, the number of data points that the algorithm is sensitive to is approximately 8000 of the most recently sampled points. This corresponded to about 80 milliseconds of estimation time in the continuous domain. The voltage pulse width at 60 Hz frequency and 11 percent duty cycle for both these experiments was about 1.83 milliseconds or about 183 sampled points; therefore, it is reasonable to assume that the adjustment model can update the DTC model in less than 80 milliseconds of time for it to become valid (accurate). 143 Figure 48. Output estimation of measured stimulation current for human subject using DTC model in (5.4a): square wave CVD pulses with λ = 0.9975, recording time = 10 sec, and Vpp = 90V. 144 6.2.9 ESTIMATION MODEL OUTPUT AS A FUNCTION OF CVD WAVEFORM TYPE STUDY This section examines and illustrates some experimental cases concerning model uncertainty and model validity in estimation of output current when the haptic renderer drives a human skin-electrode load as a function of the three CVD waveform pulse types. For reference, the three types of voltage pulses were shown in Figure 17. The results that will follow can be compared to the results when the load is a known ideal LTI impedance network driven by the haptic renderer, which was studied in section 6.1 of this paper. The motivation for the experiments was to empirically assess to what extent the shapes of the voltage pulses excite differently the underlying non-linear plant dynamics. For this purpose, the measured output current was estimated in real-time using the small signal DT-SSC estimation model and the generalized DTC model given in (5.4a) for applied square wave, saw tooth, and triangle wave pulses. The output estimation from the CED filter in (5.4a) was also plotted alongside the results to visualize the magnitude of plant uncertainty that cannot be modeled alone using DTSSC estimation model versus CVD waveform type. Again, the CED filter is the term that corrects for this plant uncertainty in its output for stimulation current estimation. It should be noted that the CT-LPBI model given in (5.3) and the DT-SSC model will be assumed interchangeable for all estimations, whose sufficient conditions for validity rests on the assumption put forth in (5.1.1); the equivalency relation between these two models is given by (5.31b) with parameter transformations given by equations (5.34a – 5.34c). 145 In this study, the collected data from three subjects were used that accounted for each of the three applied CVD waveform types. The recorded data was taken at each subject‟s reported pain threshold in stimulation current amplitude so that the effects of model uncertainty would be apparent in the time domain plots. Selecting the maximum stimulation current data also showed the extreme case for the ability of the models to estimate the load current. The fundamental frequency in the output was 60 Hz with 11 percent duty cycle for all plots, as in previous cases. As well, a forgetting factor of: λ = 0.99975 was used. The total recording time for all the following plots was one second. In the time domain plots, only one pulse of the resulting 60 is shown for each case. Figure 49 displays one of the output current pulses in the recording of CVD square wave voltage pulses delivered to subject #4 for one second. Along with this plot, the estimated current outputs, ICorrError, ICalcModeled, and ICorrError + ICalcModeled are shown, which correspond to outputs from: the CED filter, DT-SSC model and the superposition of the model terms in (5.4a), respectively. Here we see that the measured current response is approximately first order by examining the transient and steady state responses. The small-signal DT-SSC model is accurate in estimating the output response on a small time resolution even at the rising and falling edge of the pulse. The output from the CED filter is negligible for the entire pulse duration indicating that: 1) the DT-SSC estimation model and CT-LPBI estimation model derived in section 5.1 are equivalent for square wave pulses as was argued in section 5; 2) the electrical bioimpedance plant effectively shows linear dynamics under square wave pulses. The conclusion empirically supports the conditions set forth in (5.1.1). The CED filter output 146 is greatest at the transient response of the pulse as expected, since the curve must transition over the true non-ohmic I-V relationship to get to steady state (see equation (5.2) for the general relationship). It should be noted that these results were observed for all subjects and over all valid stimulation current dynamic ranges. Therefore, the parameter estimates of the physical bio-impedance parameters, shown in the model structure in Figure 25, model the lumped effects of the underlying electro-biological chemistry of the fingertip to a reasonable degree when CVD square wave pulses are applied to the skin. 147 Figure 49. One of 60 square wave pulse of measured and estimated model output current, using models in (5.3) and (5.4a), for CVD square wave pulse train administered at reported pain threshold for subject #4: Vpp = 125V, λ = 0.99975 and recording time = 1 sec. 148 The parameter estimates for impedance values corresponding to the previous figure are given in Figure 50. These were the recorded parameters that estimated the dynamics for the fingertip-electrode interface for the one second square wave pulse application. The locations of the active duty cycle in the pulse train are apparent by the seen “spikes” in the parameter estimates because of the exponential rise in the norm of the covariance P matrix used in the ELS-FF adaptation law during the inactive pulse region, which is explained in section 5.2.2. Rp is shown to be the parameter most affected by the time varying plant, as expected. This graph can be compared to the subsequent saw tooth and triangle wave parameter estimates over time in Figure 52 and Figure 54. It is seen, that at pain threshold stimulation for all three subjects, the respective Rp and Rs values are similar. This result was repeatable. However, Cp does not, in general, take on similar magnitude values between the different CVD pulse types, as is exemplified by these plots. The likely reason for this phenomenon is the coupling in estimates of the parameters specific to the DT-SSC model and CED filter, which should be independent if the load is ideally linear. However as will be seen in the saw tooth and triangle wave graphs in Figure 51 Figure 53 respectively, the shapes of the CVD pulses violate the assumptions set forth in (5.1.1) to guarantee minimizing the effects of non-linear dynamics of the electrobiological plant. Hence, the equivalency between the DT-SSC and CT-LPBI model are not guaranteed when transforming the DT-SSC model parameters from the discrete time domain to the CT-LPBI model impedance parameters in the continuous time domain. 149 Figure 50. Estimated Rp, Rs, and Cp as a function of time and square wave CVD pulses for subject #4: Vpp = 125 volts, λ = 0.99975 and recording time = 1 sec. 150 Figure 51 shows the plot based of the same conditions as the previous case except with saw tooth CVD pulse trains administered to subject #1. Here the CED filter output estimation compensated noticeably for the significant estimation error present in the small-signal DT-SSC output estimation; hence, both DT-SSC and CT-LPBI are no longer valid for the plant dynamics on such a small time resolution. The generalized DTC model, however, estimated the load current well. In contrast, the DT-SSC model first over estimates the measured current output and then underestimates it at higher current levels. This is likely due to the exponential rise in measured stimulation current as a function of rising voltage which is visible in the graph over time; equivalently the underlying physical phenomenon can be viewed as the fall in stratum corneum skin resistance as the measured stimulation current increases over time, which is related in equation (5.2). This suggests that (5.2) can be used to model Rp during transition time and not only in steady state if this in fact the trend. In summary, the phenomenon becomes apparent over the long duration of I-V transition as a function of time. The result emphasizes the inadequacy of the CT-LPBI physical model, in the large signal, in terms of long I-V transition time. Moreover, the results illustrates the need for a more robust physical model that accounts for the large signal I-V transition times as it relates to the electro-biological dynamics of the skin. In this respect, the estimation error could be reduced when only the reformulated model is considered. 151 Figure 51. One of 60 saw tooth wave pulse of measured and estimated model output current, using models in (5.3) and (5.4a), for CVD saw tooth wave pulse train administered at reported pain threshold for subject #2: Vpp = 135V, λ = 0.99975 and recording time = 1 sec. 152 Figure 52 shows that, at reported pain stimulation threshold for subject #1, Rp and Rs resistances are the same magnitudes, to their square wave counterparts in Figure 50, as expected. However, the estimation of Cp confirms that it is an order of magnitude greater than expected. Rp is shown to be the parameter most influenced by the time varying process, as expected. It is known that Cp is larger than should be expect because of the larger than expected recovery time erroneously estimated by the DTSSC model output in Figure 51. 153 Figure 52. Estimated Rp, Rs, and Cp as a function of time and saw tooth wave CVD pulses for subject #2: Vpp = 135 volts, λ = 0.99975 and recording time = 1 sec. 154 The last time domain recording, corresponding to CVD triangle wave pulse trains administered to subject #20 at recorded pain threshold, is graphed in Figure 53. The small error in output estimation using the generalized DTC model is in line with the previous time domain results. As well, the calculated Cp extracted from the DT-SSC estimated parameters as a function of time is clearly over estimated. As with the saw tooth current, the bio-impedance model of (5.3) has significant error in estimation as is apparent by the bowing in the CED filter output correction. 155 Figure 53. One of 60 triangle wave pulses of measured and estimated model output current, using models in (5.3) and (5.4a), for CVD triangle wave pulse train administered at reported pain threshold for subject #20: Vpp = 142V, λ = 0.99975 and recording time = 1 sec. 156 Figure 54 shows that for subject #20, the calculated capacitance is of the same order as that of the capacitance value derived from the physical model when saw tooth waveforms are applied to the skin in the previous results. However, Rp and Rs are of the same order of magnitude as the previous two cases. This is as expected from the discussion for the saw tooth wave analysis. 157 Figure 54. Estimated Rp, Rs, and Cp as a function of time and triangle wave CVD pulses for subject #20: Vpp = 142 volts, λ = 0.99975 and recording time = 1 sec. 158 In conclusion, the DTC model estimates the resulting stimulation current output pulses regardless of the applied CVD pulse shape extremely well. However, the CT-LPBI model is valid for the square wave pulses for which the model was originally developed. As well, Rp and Rs appear accurate for all the cases. The calculated Cp, however, for the saw tooth and triangle wave pulses is clearly inaccurate. 159 7. ADAPTIVE CONTROL MODELING, SIMULATION AND ANALYSIS The following sections rely on experimental results from section 6 of this paper. Stochastic characterization for identified Rp in section 6.2.7 will be generalized to the instantaneous current case and tested for validity. The motivation will be to have a full bio-impedance dynamics model for closed-loop adaptive control to track square-wave voltage pulse induced current. Simulation will be presented to provide evidence for the proposed model. 7.1 MODEL The CVD amplifier circuit low pass filter along with the skin bio-impedance model first presented in Figure 21 is shown in Figure 55. The circuit will be the used as the overall model for simulation. 160 Figure 55. Illustration of bio-impedance model of skin and frequency model for constant voltage pulse driver (CVD) The models for skin bio-conductivity and CVD OpAmp, along with the large signal stochastic modeling of Rp are stated as: λ λ δ δ δ Coefficients A and B are experimentally determined stochastic parameters that fit the large-signal characteristic curve of Rp versus RMS output current to a power law regression curve. It holds for any member in the sample population to a reasonable degree. The value of λ is the pole location of the low pass filter of the OpAmp 161 determined by its gain-bandwidth product. Va is taken as the input to the system and Istim the stimulation current output. It is important to note that the relationship in (7.1b) is derived from average current and not instantaneous. In our system, Istim and the postfiltered output Va, termed VLoad (not shown), are observable via the corresponding current and voltage sensors measuring signals to an electrode on the display. 7.2 STOCHASTIC PARAMETER IDENTIFICATION FROM POPULATION DATA Figure 56 illustrates the identified Rp versus RMS stimulation current scatter plot for 21 participants Rp in was identified online using extended least squares and the measured stimulation current averaged over the sampling interval time. As seen, the data can be fitted to a power law (R2 = 0.874) , which ranges in value from 450 kΩ to 1550 kΩ near 0.03 mA RMS, and 100 kΩ to 200 kΩ near the maximum RMS current applied. The variance decreases as function of current. The mean values of A and B coefficients for the entire sample population are Aavg = 40.903 and Bavg = -0.798. Analysis on the sample population data for identified Cp indicates that the value over the same RMS current range increases approximately linearly from 200 pF at the low end of RMS current to 600 pF at maximum RMS current level. The average value of Cp for the population over the current range shown in Figure 56 is about 400 pF. 162 Figure 56. Large-signal stochastic representation of Rp and A and B power law coefficients for sample population (N = 21) versus RMS stimulation current for 60 Hz, 5% duty cycle square wave pulses Figure 57 shows a box and whiskers plot of the averages for each set of identified values of Rs for each member of the population (N = 21) identified over the same current range as in the previous figure. The population mean of the averages equals 63 kΩ. Rs exhibits a small variation in value compared to Rp. In summary, this value is used as the stochastic representation of Rs in (7.1a). 163 μ (Rs) kΩ Box and Whiskers Plot for Random Variable: Average of the Estimated Rs Values for Each Subject (N = 21) (Square Wave) 100 90 80 70 q1 60 50 min 40 median 30 20 max 10 q3 0 Square Wave CVD Pulse Type Figure 57. The mean of the average values of Rs for each member of the population (N = 21) 7.3 PROPOSED BIO-IMPEDANCE DYNAMICS MODEL It is desirable to generalize the model (7.1b) by replacing RMS with instantaneous current to account for dynamic on Rp. Without prior knowledge and as a first assessment, (7.1b) is assumed to hold for instantaneous time. The bio-impedance dynamics model to be studied then becomes: λ λ Ω Ω Here, the physical parameters Rp, Cp, and Rs are defined as before. In part 5 of this work, an online ELS identification algorithm was introduced as a means to estimate 164 these parameters in real-time. Section 6 demonstrated real-time identification for Rp, Cp and Rs for square wave voltage pulses in addition to other pulse geometries. (7.2b) is obtained by the graph in Figure 56. In (7.2b), the instantaneous output current term is normalized by duty cycle and re-dimensioned to amperes. As well, Rp is re-dimensioned from kilo-ohms to ohms. 5 MΩ is taken as the DC resistance of the skin when no current is applied and is also the isolation resistance of the actual haptic rendering device from ground. Now, Istim is defined as the measurable instantaneous current to the skinelectrode interface due to measurable voltage pulses from the driver. 7.4 DERIVATION OF ADAPTIVE CONTROL LAW Assume that the condition of the skin is slowly changing and the control is for one electrode on the array; Rp introduces nonlinearity in the system. Neglecting the dynamics of the OpAmp, which will be assumed to be much quicker than the output response time, taking (7.2a) in to the time domain and substituting Rp in to the original model yield the following differential equation: In this equation, we are assuming that since it was found experimentally to be relatively close to unity and will greatly simplify the control law. VLoad ≈ Va is the system input and Istim is the output, which are both measurable. The parameters Rs and Cp are estimated online using an extended least squares algorithm and are assumed to be known for this reason; A is unknown. The system is linearized with nonlinear feedback 165 by canceling out the square term and the coupled term on the original differential equation using the substitution: which leaves: The introduced term u is the input to the new system with coefficients as previously defined in (7.4).The input will be controlled by using adaptation on the model with the indirect MRAC law approach. The following is the reference model to be tracked: π where T is the period and TD the duty cycle for a train of square wave pulses. The inequality constraint in (7.7a) is so that output tracking does not exceed the slew rate of the OpAmp. The virtual control for u will be: where: The error is defined as: An MRAC law with σ-switching modification applied to 7.6 gives: γ γ σ 166 γ γ σ σ σ σ The σ-switching modification is necessary to correct for parameter drift due to model uncertainty from approximations made in deriving the nonlinear feedback. Using data collected previously, the constant, A, in (7.4) is substituted by the known sample population coefficient Aavg computed for the regression analysis in Figure 56 giving: Once u is updated, the real input Va must be computed to produce the desired output. There is not direct access to in (7.5); so instead, taking into consideration the sample rate of the DAQ card Ts, the implementation is: Note at the end of the active duty pulse cycle duration, the control is shutoff for reasons to be explained. 7.5 SIMULATION OF THE BIO-IMPEDANCE MODEL (OPEN-LOOP) The instantaneous model in (7.2a – 7.2b) will be simulated in Matlab using filled in parameters found experimentally for subject #2 which are given in 167 Table 7. One instance of the measured and ELS-estimated current pulses corresponding to the identified parameters is shown in Figure 58 for the subject. The first order zero and pole in the impedance model are clearly evident in the transient response; As well the addition in Rp and Rs values at the voltage amplitude predict the steady state current component accurately. The effects of current nonlinearity versus R p are not apparent because of the fast rise and fall times in the pulse. If the model is a reflection of the dynamics of skin under electronic stimulation, then the simulated output should replicate the actual output to a reasonable extent. The simulated CVD pulses should also account for the low-pass filter characteristics of the OpAmp or the simulation could result in large error in the transient response. The low pass filter corner frequency of the driver OpAmp is based on its data sheet and observed time differences between voltage rise and fall times under known resistive load. The low pass filter is: λ λ λ π λ π Table 7. Experimentally estimated bio-impedance parameters for subject #2 with square wave voltage pulse amplitude conditions shown 168 kΩ; Vpp = 150 Volts 430 pF; Vpp = 150 Volts 65 kΩ; Vpp = 150 Volts 0.63 mA; Vpp = 150 Volts IstimSS A 61.613; Vpp = 30…150 Volts B 0.741; Vpp = 30…150 Volts Rp Cp Rs 168 Figure 58. Measured and estimated output current pulse shape to subject #2 at 150 volt square wave pulse; measured and estimated Istim at steady state equals 0.63 mA 169 A simulated current pulse is shown in Figure 59 for a train of simulated voltage pulses inputted in to the dynamics model with parameters taken from Table 7. In addition, Figure 60 illustrates the simulated value of Rp. The absolute error between experimental and simulated steady state current is 0.13 milliamps. The error may be attributed to the imperfect and statistical power law approximation between current and Rp that is nonetheless assumed in the dynamical model. The value of R p drops to 231 kΩ during the simulated steady state current. Comparing this to R p from experiment, the SC resistance for the subject was 168 kΩ. The transient response between experiment and simulation agree well during the rise time and could be improved with better dynamical modeling of the driver circuitry and additional modeling of measurement sensors. The discrepancy seen during the fall-time of the simulated current pulse is likely due to the need of a larger time constant than was used since the ELS estimate of current matches with experimental measurement. The simulation makes apparent that the SC skin resistance drops quickly for square wave pulses; hence, the effects of Rp nonlinearity are negligible during such a short transient response time. In summary, the results are promising for generalizing the stochastic model presented in (7.1a – 7.1b) to the instantaneous current impedance model given in (7.2a – 7.2b). However, the model affords improvement as seen in the steady state current discrepancy. 170 Figure 59. Simulated current pulse from square wave voltage pulses inputted into equations 7.2a- 7.2b using experimental data in Table 7 for one subject; Istim steady state equals 0.5 mA 171 Figure 60. Simulated value of Rp stratum corneum resistance (Ohms) using equation 7.2b during the current pulse time in Figure 59; the minimum of Rp equals 231 kΩ 172 7.6. SIMULATION ON CLOSED LOOP STIMULATION CURRENT CONTROL The following simulations use values from Table 7. Figure 61 shows the desired reference current. Reference current is designed with the pole at 2000 KHz so that rise and fall time are at 175 μs. The frequency is at 60Hz, amplitude at 0.63mA and with 11 percent duty cycle. 173 Figure 61: Reference Current 174 Figure 62 shows the simulation result on the stimulation current output of the closed loop control system. It displays that the designed stimulation current control system was able to track the reference current plotted in Figure 61 very well at the beginning of every pulse. There is a spike at the end of every pulse. This spike is the result of abruptly turning off the voltage control of the system. This is a result of the capacitive effects of the skin. This is a necessary step to ensure correct tracking of the current. Given the behavior of Rp, when the current is low, it increases dramatically as seen in Figure 63. This results in a dramatic increase in the time constant of the system and so there is very slow convergence. Figure 62. Simulation results: Tracked stimulation current. 175 Figure 63. Simulation results: behavior of Rp. Figure 64 plots the simulated closed loop CVD voltage pulses. In the figure, the voltage output Va approximates to VLoad. This agrees with our assumption in simplification of the dynamic model. 176 Figure 64. Simulation Results: Voltage output of closed loop CVD. 177 8. CONCLUSIONS AND FUTURE WORK This paper presents development of a complete electrotactile haptic renderer using a constant voltage driver (CVD) method with closed loop design. The hardware for powering the driver as well as measurement circuitry to monitor voltage and current output were developed that must operate under high voltage conditions were successfully developed. The CVD output driver as well as the measurement circuitry was also proven to be able to operate under typical human fingertip bio-impedance loading. In addition, the on-line identification algorithm of fingertip skin bio-impedance and associated models are presented. The algorithm is based on a discrete-time extended least squares (ELS) iterative approach with forgetting factor, which serves as an adaptive law to estimate parameters of the first-order bio impedance model of the fingertip. Experimental results demonstrate the performance of the developed CVD based electrotactile haptic renderer, as well the effectiveness of the algorithm for oneline identification of parameters of human fingertip skin bioimpedance. This work will greatly improve the tactile/haptic rendering performance of electrical stimulation (electrotactile) systems and facilitate the development of electrotactile based haptic rendering devices and applications. The purpose of this work is also to establish a generalized dynamic model of the electronic bio-impedance of the fingertip skin, based on experimental research for which closed-loop MRAC control could be applied using the direct method. The goal of the control is to regulate stimulation current to the reference current. The model is 178 presented to instantaneous stimulation current, which is necessitated by the type of input our electrotactile driver used to deliver sensory content to the skin, namely constant voltage pulses. This is in contrast to drivers that use constant current pulses whose outputs are mainly independent of the current state of the electronic bioimpedance of the skin. The results of this work will be used to develop the advanced controller in an application where an electrotactile haptic renderer adapts the stimulation output current from changing electro-bioimpedance conditions of the skin. This will allow reduced driver complexity design and increased power efficiency. The tradeoff will be in the complexity of sensor design to monitor the output, external DAQ hardware, and software complexity necessary for implementing the adaptive control and estimation law. The modeling of the dynamics for the generalized model was shown to be preliminarily valid from simulation when compared to experimental results. 179 REFERENCES 180 9. REFERENCES [1] Y. Shen, et al., “Supermedia Interface for Internet Based Tele-Diagnostics of Breast Pathology”, International Journal of Robotics Research, vol. 26, No. 11-12, pp. 12351250, 2007. [2] Y. Shen, C. A. Pomeory, and N. Xi, et al., “Quantification and Verification of Automobile Interior Textures by a High Performance Tactile-Haptic Interface”, Proceedings of the 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3773-3778, 2006. [3] C. J. Poletto and C. L. Van Doren, “A High Voltage, Constant Current Stimulator for Electrocutaneous Stimulation Through Small Electrodes”, IEEE Transactions On Biomedical Engineering, Vol. 46, No. 8, pp. 929-936, 1999. [4] K. A. Kaczmarek, J. G. Webster, et al., “Electrotactile and Vibrotactile Displays for Sensory Substitution Systems”, IEEE Transactions On Biomedical Engineering, Vol. 38, No. 1, pp. 1-15, 1991. [5] K. M. Benali-Khoudja, et al., “Tactile Interfaces: a State-of-the-art Survey”, Proceedings of the 35th International Symposium on Robotics, March 23-26, 2004. [6] F. Gemperle, et al., “Design of a Wearable Tactile Display”, Proceedings of the 2001 Fifth International Symposium on Wearable Computers, pp. 5-12, 2001 [7] J. R. Phillipsa and K. O. Johnson, “Neural Mechanisms of Scanned and Stationary Touch”, J. Acoust. Soc. Am., Vol. 77, No. 1, pp. 220-224, 1985. [8] C. Bolzmacher, et al., “Polymer Based Actuators for Virtual Reality Devices”, Proceedings of SPIE, Vol. 5385, pp.281-289, 2004 [9] I. Koo1, et al., “Wearable Fingertip Tactile Display”, SICE-ICASE International Joint Conference, pp. 1911-1916, 2006. [10] M. Shimojo, et al., “Development of a System for Experiencing Tactile Sensation from a Robot Hand by Electrically Stimulating Sensory Nerve Fiber”, Proceedings of the 2003 IEEE International Conference on Robotics and Automation, Vol. 1, pp. 12641270, 2003. [12] A. Y. J. Szeto and F. A. Saunders, “Electrocutaneous Stimulation for Sensory Communication in Rehabilitation Engineering”, IEEE Transactions on Biomedical Engineering, Vol. 29, pp. 300-308, 1982. [13] L. R. Bobich, J. P. Warren, J. D. Sweeney, S. I. Helms Tillery, and M. Santello, “Spatial Localization of Electro-tactile Stimuli on the Fingertip in Humans”, Somatosensory and Motor Research, Vol. 24, No. 4, pp. 179-188, 2007. 181 [14] H. Kajimoto, N. Kawakami, S. Tachi, and M. Inami, “SmartTouch: Electric Skin to Touch the Untouchable”, IEEE Computer Graphics and Application, Vol. 24, Issue 1, pp. 36–43, 2004 [15] S. J. Haase, K. A. Kaczmarek, “Electrotactile Perception of Scatterplots on the Fingertips and Abdomen”, Medical and Biological Engineering and Computing, Vol. 43, pp. 283-289, 2005. [16] P. Bach-y-Rita, K. A. Kaczmarek, and K. Meier, “The Tongue as A Man-machine Interface: A Wireless Communication System”, Proceedings of the 1998 International Symposium on Information Theory and its Applications, pp. 79-81, 1998. [17] P. Bach-y-Rita, K. A. Kaczmarek, M. E. Tyler, and J. Garcia-Lara, “Form Perception with a 49-point Electrotactile Stimulus Array on the Tongue: A Technical Note”, Journal of Rehabilitation Research and Development, Vol. 35, pp. 427-430, 1998. [18] M. R. Neuman, “Biopotential Electrodes”, In J.G. Webster (Ed.), Medical instrumentation, application and design, pp. 183-232, New York: John Wiley and Sons, Inc, 1998. [19] J. P. Reilly, “Electrical Stimulation and Electropathology”, Cambridge: Cambridge University Press, 1992. [20] O. Yarimaga, J. Lee, B. Lee, and J. Ryu, “Tactile Sensation Display with Electrotactile Interface”, ICCAS 2005, KINTEX, Gyeonggi-Do, Korea, June 2-5, 2005. [21] N. Asamura, N. Yokoyama, and H. Shinoda, “A Method of Selective Stimulation to Epidermal Skin Receptors for Realistic Touch Feedback”, Proceedings of the 1999 IEEE Virtual Reality Conference, pp. 274-281, 1999. [22] A. B. Vallbo and R. S. Johansson, “Properties of Cutaneous Mechanoreceptors in the Human Hand Related to Touch Sensation”, Human Neurobiology, Vol. 3, pp. 314, 1984. [23] O. Yarimaga, B. Lee, J. Ryu, and N. Mahalik, “An Electro-tactile Device for Broadcasting and Game Applications”, HCI2005 Korea, Vol. 1, pp. 388-394, 2005. [24] H. Kajimoto, et al., “Electro-Tactile Display with Tactile Primary Color Approach”, IROS, 2004 [25] A. B. Vallbo, “Sensations Evoked from the Glabrous Skin of the Human Hand by Electrical Stimulation of Unitary Mechanosensitive Afferents”, Brain Research, Vol. 215, pp. 359-363, 1981. [26] F. Rattay, “Electrical Nerve Stimulation”, Springer-Verlag, 1990. 182 [27] H. Moghimi, N. Noorani, A. Zarghi, “Stereoselective Permeation of Tretinoin and Isotretinoin through Enhancer-Treated Rat Skin. II. Effects of Lipophilic Penetration Enhancers”, Iranian Journal of Pharmaceutical Research, Vol. 3, pp. 17-22, 2004. [28] K. A Kaczmarek, “A 16-Channel 8-Parameter Waveform Electrotactile Stimulation System”, IEEE Trans. On Biomedical Engineering, Vol. 38 No. 10, pp. 933-943, Oct. 1991. [29] Stephen J. Dorgan and Richard B. Reilly, “A Model for Human Skin Impedance During Surface Functional Neuromuscular Stimulation”, IEEE Transactions on Rehabilitation Engineering, Vol. 7, No. 3, pp. 341-348, Sep. 1999. [30] S. J. Dorgan and R. B. Reilly, “Skin Impedance From 1 Hz to 1 MHz”, IEEE Transactions on Biomedical Engineering, Vol. 35, No. 8, pp. 649-652, Aug. 1988. [31] E. F. Prokhorov et al, “In Vivo Electrical Characteristics of Human Skin, Including at Biological Active Points”, Med. Biol. Eng. Comput., Vol. 38, pp. 507-51, 2000. [32] U. Pliquett, R. Langer, & J. C. Weaver, “Changes in the passive electrical properties of human stratum corneum due to electroporation”, Biochimica et Biophysica Acta, 1239 pp. 111-121, 1995. [33] K. Astrom and B. Wittenmark, “Adaptive Control (2nd Edition)”, Prentice-Hall, 1994. [34] J. Gregory and Y. Shen, et al., “Towards on-line fingertip bio-impedance identification for enhancement of electro-tactile rendering”, Proceedings of, the 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 3685-3690, 2009. [35] P. Ioannou and B. Fidan, “Adaptive Control Tutorial (Advances in Design and Control)”, SIAM, Society for Industrial and Applied Mathematics, 2006. [36] EMCO High Voltage Power Supply Manufacturer: (n.d.) [Online]. Available: http://www.emcohighvoltage.com/ [37] Very Low Standby Current 600µA, High Voltage 400V Power Amplifier: (n.d.) [Online]. Available: http://www.cirrus.com/en/products/pa97.html/ [38] Vishay - Optocouplers/Isolators - IL300 - Linear Optocoupler, High Gain Stability, Wide Bandwidth: (n.d.) [Online]. Available: http://www.vishay.com/optocouplers/list/product-83622/ [39] T. Yamamoto and Y. Yamamoto, “Electrical properties of the epidermal stratum corneum”, IEEE Transactions on Biomedical Engineering, Vol. 14, No. 5, pp. 494500,1976 183 [40] J. Sandby-Møller, T. Poulsen and H. Wulf, “Epidermal Thickness at Different Body Sites: Relationship to Age, Gender, Pigmentation, Blood Content, Skin Type and Smoking Habits”, Acta Derm Venereol, 83, pp. 410–413, 2003 [41] M. Egawa, T. Hirao and M. Takahashi, “In vivo Estimation of Stratum Corneum Thickness from Water Concentration Profiles Obtained with Raman Spectroscopy”, Acta Derm Venereol, 87, 2007, pp. 4–8 [42] M. Shimojo, M. Shinohara, and Y. Fukui, “Human Shape Recognition Performance for 3-D Tactile Display”, IEEE Transactions on Systems, Man, and Cybernetics—Part A: Systems and Humans, VOL. 29, NO. 6, pp. 637 - 644, Nov. 1999 [43] T J C Faes, H A van der Meij, J C de Munck and R M Heethaar, “The electric resistivity of human tissues (100 Hz–10 MHz): a meta-analysis of review studies”, Physiol. Meas., Vol. 20, 1999, pp. R1–R10 184