EARIES

\\ “WI

L
mom «was “5 m

N \\\\\\\\\\\...\ mm

31293

 

 

 

 

 

 

 

 

 

 

i

This is to certify that the
dissertation entitled
IMPLEMENTATION OF
FEEDFORWARD ARTIFICIAL NEURAL NETWORKS
WITH LEARNING USING STANDARD CMOS TECHNOLOGY
presented by

Myung-Ryul Choi

has been accepted towards fulﬁllment
of the requirements for

Ph.D degree in Eleggrical Engineering

SLAP—‘2

Major professor

 

 

Date M

MS U is an Afﬁrmative Action/Equal Opportunity Institution 0-12771

 

 

LIBRARY
Michigan State
University

 

 

 

PLACE IN RETURN BOX to remove this checkout from your record.
TO AVOID FINES return on or before date due.

_____________________
DATE DUE DATE DUE DATE DUE

1 1995

v
i ;
L
i

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

J—ji J

MSU Is An Afﬁrmative Action/Equal Opportunity Institution
ammo—o.-

 

 

.—-

 

IMPLEMENTATION OF
FEEDFORWARD ARTIFICIAL NEURAL NETWORKS
WITH LEARNING USING STANDARD CMOS TECHNOLOGY

By

Myung-Ryul Choi

A DISSERTATION

Submitted to
Michigan State University
in partial fulﬁllment of the requirements
for the degree of

DOCTOR OF PHILOSOPHY

Department of Electrical Engineering

1991

("75—3‘4 A 4-; «77¢? .3"

ABSTRACT

IMPLEMENTATION OF
FEEDFORWARD ARTIFICIAL NEURAL NETWORKS
WITH LEARNING USING STANDARD CMOS TECHNOLOGY

By
Myung-Ryul Choi

A feedforward artiﬁcial neural network with a supervised learning rule is imple-
mented using standard CMOS technology. Programmable synapse cells are imple-
mented by employing a simple tunable all-MOS analog multiplier, where its inputs and
its output are given in voltage units. Employing this multiplier, we can implement
large scale artiﬁcial neural networks (ANN s) using fewer MOS transistors than are
1‘6an by implementations using the so-called Gilbert multiplier.

A modiﬁed error back-propagation learning rule is used in order to realize on—
Chip learning circuits using standard CMOS technology. Using this learning rule,
sigmoid-derivative circuits are not necessarily required for the implementation of feed-
forward artiﬁcial neural networks (ANN s) with learning. This learning circuit has

learned a desired input/output pattern successfully for arbitrary initial weights.

A modular design is described for a large scale implementation of feedforward

artiﬁcial neural networks (ANN s) with learning. Two implementations are designed

using the MAGIC VLSI editor: 4x1 module chip on the M0815 Tinychip and 9x3
module chip on the M0818 small standard chip. The module chips can be connected
vertically and horizontally to realize large scale feedforward artiﬁcial neural networks
(ANN s) with optional use of on-chip or off-chip leaming capability.

 

To my parents,
Yun-Jin, Jae-Joon, and Jae-Sig

for their love, support, and sacriﬁce

iii

 

ACKNOWLEDGMENTS

I would like to thank my thesis advisor, Dr. Fathi M. A. Salam for his patience,
guidance and encouragement throughout the course of this research.

I would like to express my gratitude to all the members of my Ph.D. guidance
committee, Dr. Frank Hoppenstadt, Dr. Hassan K. Khalil, Dr. Lionel M. Ni and Dr.
Gregory Wierzba for their comments and suggestions. '

I would like to express my sincere thanks and appreciation to my parents, my
wife parents, my sister, and my brother-in-law for their continuous love, understanding,
and encouragement. Special thank to my wife, Yun-Jin, and my children, Jae-Joon,
Jae-Sig, for help and understanding throughout my studies.

I also gratefully acknowledge the partial support of this work under grant from
ONR and the Michigan Research Excellence Fund (REF).

iv

 

 

TABLE OF CONTENTS

LIST OF TABLES ..................................................................................................... viii
LIST OF FIGURES ................................................................................................... x
Chapter 1: INTRODUCTION .................................................................................. 1
Chapter 2: BACKGROUND .................................................................................... 10
2.1 The Model of a Neuron ......................................................................... 10

2.2 Neural Network Models ......................................................................... 13

2.2.1 Feedback Models ..................................................................... 13

2.2.2 Feedforward Models ................................................................ 17

2.3 MOS Transistor ...................................................................................... 19
Chapter 3: BASIC ANALOG SUBCIRCUITS ....................................................... 24
3.1 A Simple Tunable MOS Analog Multiplier .......................................... 25

3.2 A CMOS Operational Ampliﬁer ............................................................ 31

3.2.1 A Voltage Follower ................................................................. 35

3.2.2 A Voltage Integrator ................................................................ 36

3.3 Voltage Shifters/Attenuators .......................... I ........................................ 36

3.4 A MOS Capacitor ................................................................................... 41

3.5 A CMOS Double Inverter ...................................................................... 42

3.6 An Analog Multiplexer .......................................................................... 43
Chapter 4: ANALOG SCALAR/VECTOR MULTIPLIERS .................................. 45

4. 1
4.2
4.3

4.4

Chapter 5:
5. 1

5.2

5.3
5.4

Chapter 6:

6. 1
6.2
6.3
6.4

Analog Multiplier Cells ..........................................................................
Performance of Analog Multipliers .......................................................
Implementation of an MD Analog Multiplier .....................................
Applications ............................................................................................

4.4.1 Programmable Synapse Cells ..................................................

4.4.2 A Sigmoid—Derivative Cell ......................................................

4.4.3 Analog Adder Cells .................................................................
ANALOG FEEDFORWARD ANNS WITH LEARNING ..................
A Modiﬁed Learning Rule for Feedforward ANN s ..............................
Feedforward ANNS with Sequential Leaming ......................................

5.2.1 Feedforward ANNS with the Sequential Learning

Circuit #1 .................................................................................

5.2.2 Feedforward ANNS with the Sequential Learning
Circuit #2 .................................................................................

Feedforward ANNS with Simultaneous Leaming .................................
Discussion ...............................................................................................

A MODULAR DESIGN OF FEEDFORWARD ANNS
wrm LEARNING ........................................

A Prototype 2x2x2 Feedforward ANNS with Learning ........................
Tire Implementation of the Module Chip ..............................................
The Operation of the Module Chip .......................................................

Implementation of Module Chips for Large Scale
Feedforward ANNS with Learning ........................................................

45
49
54
57
57
58
6O
61
61

63

63

68
74

81

83
83
92

97

101

6.4.1 The 4x1 Module Chip on a 40-pin MOSIS Tinychip ............ 101

6.4.2 The 9x3 Module Chip on a 64-pin MOSIS Small

Chip .......................................................................................... 105
Chapter 7: SUMMARY ............................................................................................ 109
APPENDICES ............................................................................................................ 112
LIST OF REFERENCES ........................................................................................... 156

3.2.1

3.2.2

3.3.1

3.4.1

4.1.1

4.2.1

4.2.2

4.3.1

4.3.2

5.2.1

5.2.2

5.2.3

5.3.1

LIST OF TABLES

(a) The device sizes and (b) the characteristics of

the CMOS Op-amp ........................................................................................
The device sizes of a voltage follower ...........................................................
(a) The operating ranges and (b) the device sizes of

the w_shifter and the x_shifter .....................................................................
The device size of MOS capacitors ................................................................
The device sizes and the applied control voltages of each standard

analog multiplier cell ....................................................................................
The speciﬁcations of 1-D multiplier and AD53ZJ multiplier ........................
Output offset voltage and maximum percentage error of

analog multiplier cells ..................................................................................
(a) The number of pins used and (b) pin-assignment of

the ll-D chip ................................................................................................
The tuning signals for each cell of the ll-D chip .........................................
The PSPICE transient analysis of the feedforward ANN with

the sequential learning circuit #1 .................................................................
The PSPICE transient analysis of the feedforward ANN with

the sequential leaming circuit #2 .................................................................
The PSPICE dc analysis of the feedforward ANN with the

sequential learning circuit #2 .......................................................................

The PSPICE transient analysis of the feedforward ANN with

33

35

40

42

47

53

53

56

56

67

72

73

5.3.2

5.3.3

6.1.1

6.1.2

6.2.1
6.2.2
6.2.3
6.2.4
6.4.1
6.4.2
6.4.3
6.4.4
6.4.5

6.4.6

simultaneous learning when a = 50. ............................................................

The PSPICE transient analysis of the feedforward ANN with

simultaneous learning when a = l. ..............................................................

The PSPICE transient analysis of the feedforward ANN with

simultaneous learning when a = l. ..............................................................

The PSPICE transient analysis of 2x2x2 feedforward ANN s with

the sequential learning circuit #2 .................................................................

The PSPICE dc analysis of 2x2x2 feedforward ANN s with the

sequential learning circuit #2 .......................................................................
The list of the extra signals for the nxm module chip ..................................
The four MOSIS standard chip sizes ..............................................................
The number of pins used for the nxm module chip ......................................
The nxm module chip vs. the M0818 standard chip .....................................
The number of pins used for the 4x1 module chip .......................................
The cells for the 4x1 module chip ..................................................................
Pin-assignment of the 4x1 module chip .........................................................
The number of pins used for the 9x3 module chip .......................................
The cells for the 9x3 module chip ..................................................................

Pin-assignment of the 9x3 module chip ........................................................

79

80

86

90

96

97

97

97

104

104

104

107

107

107

2.1.1
2.1.2
2.2.1
2.2.2
2.2.3
2.3.1
2.3.2
3.1.1
3.1.2
3.1.3
3.2.1

3.2.2

3.2.3
3.3.1
3.3.2
3.4.1

3.5.1

3.6.1

LIST OF FIGURES

A biological neuron .........................................................................................
Neural model circuits ......................................................................................
The Hopﬁeld feedback model .........................................................................
The Hoppensteadt model .................................................................................
General two-layer feedforward ANN s ............................................................
A simpliﬁed 3-dimensional view of a MOS transistor ..................................
Convention for electrical variables of MOS transistors .................................
A simple four-quadrant analog multiplier ......................................................
All-MOS simple four-quadrant analog multiplier ..........................................
All-MOS n-D analog multiplier ......................................................................
A CMOS operational ampliﬁer .......................................................................

(a) The open loop transfer function and (b) the frequency
response of the CMOS op-amp .....................................................................

The voltage integrator circuit ..........................................................................
The w_shifter ........................................................... i ........................................
The x_shifter ....................................................................................................
A MOS capacitor .............................................................................................

(a) A CMOS double inverter and (b) the transfer characteristics

of the double inverter .....................................................................................

A 1x8 analog multiplexer ................................................................................

10
12
13
16
17
19
21
25
27
30

32

34

36

38

39

42

43

44

 

4.1.1 A 1-D multiplier cell ....................................................................................... 46
4.1.2 A lOO-D multiplier cell ................................................................................... 48

4.2.1 (a) Multiplication graphs, (b) the x-nonlinearity, (c) the

w-nonlinearity, and (d) step response of 1-D multiplier .............................. 51
4.3.1 The ll-D vector multiplier chip on a 40-pin Tinychip ................................. 55
4.4.1 A 1-D programmable synapse cell ................................................................. 57
4.4.2 A sigmoid-derivative cell ................................................................................ 59
4.4.3 The input/output characteristics of the sigmoid-derivative cell ..................... 59
4.4.4 A voltage adder cell ........................................................................................ 60
5.2.1 The feedforward ANN with the sequential learning circuit #1 ..................... 64
5.2.2 The PSPICE transient analysis of the circuit in Figure 5.2.1 ........................ 67
5.2.3 The feedforward ANN with the sequential learning circuit #2 ..................... 68
5.3.1 The feedforward ANN with simultaneous learning circuit ............................ 77
6.1.1 A 2x2x2 feedforward ANN with the sequential learning circuit #2 ............. 84
6.2.1 The block diagram of the nxm module chip .................................................. 93
6.2.2 nx2nxm two-layer feedforward ANNs with learning .................................... 94
6.3.1 The block diagram of interface circuitry ........................................................ 99
6.4.1 The 4x1 module chip on the 40-pin Tinychip ............................................... 102
6.4.2 The circuit for each weight of the 4x1 module chip ..................................... 102
6.4.3 The 9x3 module chip on a 64-pin MOSIS small pad frame ......................... 106
6.4.4 The circuit for each weight of the 9x3 module chip ........ 108

 

CHAPTER 1

INTRODUCTION

Conventional digital computers are very useful in solving well-deﬁned problems
since they can be represented by sequences of instructions. However, conventional
computers can not compete with the performance of human beings in solving ill-
deﬁned random problems such as pattern recognition, classiﬁcation, speech understand-

ing, vision and so on.

The human brain can solve certain ill-deﬁned problems easily, even though it usu-
ally takes much longer time to solve well-deﬁned problems than does the digital com-
puter. The human nervous system and the brain have the following characteristics:
adaptive learning, generalization, error correction, robustness, and creativity. Its com-
putational power is postulated to result from its massive interconnection and asynchro-

nous parallel communication among neurons.

In the human brain, neuron cells become defective or dying out everyday. Yet,
the human brain still functions correctly without any apparent degradation in perfor-
mance. However, in a digital computer if one digital circuit element is not functioning
correctly, the whole computer may be malfunctioning. The human nervous system is
the best existing intelligent system because of its adaptive and learning ability. It can

learn or adapt by itself.

In the nervous system, a neuron is the basic unit. It consists of dendrites, a cell

body, an axon, and synapses. Synapses appear to be the most crucial component in

1

neural systems since the interaction between neurons appears to result from the synap-
tic connection and the behavior of the neural system depends critically on them.
Synapses are often considered to be the disuibuted storage elements and their opera-
tional mechanism is still under investigation. The concept of distributed storage is

quite different from that of present digital computers.

With the development of VLSI technology and with improved understanding of
the human nervous system, it is possible to implement models of neural nets by mim-
icking some aspects of the nervous system of mammals. Models of neural nets are
considerably simpliﬁed from their biological counterparts, however. When a nervous
system is imitated and implemented, it is usually called artiﬁcial neural networks
(ANN s). The implementation of artiﬁcial neural networks is based on a large number
of simple computational components. The proposed artiﬁcial neural networks available

have quite different architectures from the ones used in present day digital computers.

Many researchers have worked on the implementation of artiﬁcial neural networks
(ANNs). Currently, there are many artiﬁcial neural network models. These models can
be implemented via software or hardware. Software implementation usually employs
an algorithm, which is based on the architecture and understanding of neural networks,
mapped onto conventional digital computers. Software implementation is very ﬂexible
since one can easily modify codes or algorithms in order to implement another

artiﬁcial neural network model or learning mechanism.

There are at least two disadvantages to software implementations using present
day digital computers. One is that the computations among a large number of simple
processors in artiﬁcial neural nets require huge communication paths between proces-
sors, which can not be supplied or which may take huge communication time by
present parallel or serial digital computers. The other is that when artiﬁcial neural net-
works (ANN s) are modeled with differential equations, solving the differential equa-

tions of the neural network model may take a great deal of computation time.

 

Hardware implementation is suitable to implementing large ANNs and its compu-
tation speed is much faster than that of software implementation. Huge computation
paths between processors can be supplied by connecting those processors with wire.
Hardware implementation can be realized via electronics, optics, or electro—optics. This
implementation, however, is not ﬂexible but is speciﬁcally dedicated for a given appli-

cation or a given artiﬁcial neural network model.

Among the many hardware implementations of artiﬁcial neural network models,
the Hopﬁeld-type feedback ANN model and the feedforward ANN model are fre-
quently used. Most implementations are distinguished by the particular implementation
of the synapse. Some implementations have implemented the synaptic weight multipli-
cation with binary inputs and discrete set of weights. Here are some hardware imple-
mentations of artiﬁcial neural network in the literature. In [1], only one ﬁxed weight
is used. In [2, 3], three different weights, -1 (inhibitory), 0 (no connection) and +1
(excitatory), are used. Other implementations have employed the synaptic weight mul-
tiplication with binary inputs and analog programmable weights [4] or with analog
inputs and binary programmable weights [5] or analog synaptic weights [6-10].

The ENN (Elecu'onic Neural Network) by Hubbard et al. [1] has been designed
with 256 neurons on a single chip using a mixture of analog and digital VLSI technol-
ogy and a custom microfabrication process. This circuit employs amorphous silicon
for a resistor material. Accordingly, only one ﬁxed connection weight is available and

its weight strength should be decided and ﬁxed before fabrication.

The ANN (Artiﬁcial Neural Network) by Sage et a1. [4] has been designed with
the Charge-Coupled Device (CCD) and the Metal-Nitride-Oxide-Semiconductor
(MN OS). This circuit operates with binary inputs and analog programmable synaptic
weights and executes analog operations, but it can not be implemented with standard
VLSI technology.

1161

cui

spe

310'

Ml

The ASSOCMEM (Associative Memory) by Sivilotti et al. [2, 3] has been imple-
mented and tested in NMOS technology. This circuit has been implemented with 22
nem'ons on 6700 x 5700 ttm2 with 53 U0 pads. This circuit has on-chip learning cir-
cuits by using the so-called truncated Hebb rule. During learning, all the desired
memory contents load onto the chip, the outer product rule is applied, and its ressults
are truncated one at a time. Finally, three different synaptic weights are obtained,

speciﬁcally -1 (inhibitory), 0 (no connection), and +1 (excitatory).

Using the Hebb rule, the interconnection weights are calculated from all the
stored vectors. Based on simulation studies [11], the Hebb outer-product rule has been

found to succeed in storing data vectors equal to, 0.15 N, where N is the number of

neurons.

The AT&T group [10] uses SRAM (static random access memory) cells to store
prespeciﬁed memories and one bit of each memory is stored at one SRAM cell. Here
the stored data are presented as the interconnection mau'ix and only one output, which
has the largest inner product value between the input vector and the stored vectors, is
chosen. This design does not generate spurious steady states since it physically stores
the desired data as weights. However, it requires a large number of extra memory

cells to store the desired data.

At Naval Research Laboratory, researchers [6, 7] have illustrated several
approaches for implementing programmable synapse multiplier circuits that perform
multiplication with analog inputs and analog synapse weights. The synapse weights
are calculated with a host computer using a suitable neural network algorithm and are
then stored in digital form in EPROM or RAM. The digital data are loaded to
digital-to-analog converter by a memory controller and thus analog voltages are written
to each synapse cell by a decoding circuit, i.e., an analog voltage supplies a charge to

the capacitor at the gate of an MOS transistor at each synapse cell.

 

At Jet Propulsion Laboratory, researchers [8] have developed hardware implemen-
tation of feedforward neural systems using several custom analog VLSI building block
chips. They have implemented three different kinds of chips: a multiplexing input
neuron chip, a synapse chip, and a hidden/output layer variable neuron chip. All three
chips are using CMOS n-well 2 micron process with 8.2 x 8.2 mm2 die size and 84-
pin package. The overall operation is similar to the operation of the Naval Research
Laboratory group [6, 7]. The synapse weights are calculated with a host computer
using a suitable neural network algorithm. The basic programmable synapse cell con-
sists of a pair of sample-and-hold circuits with address select circuitry and a folded
Gilbert four—quadrant multiplier [21]. The input voltages and synapse weight voltages
are stored on the gate of a 5550 um2 transistor and the output currents of the synapses
are summed via wire connection, which is simply connected together. That is, the
charge stored on the capacitor is converted into conductance through the multiplier.
Then a variable-gain neuron chip sums current signals coming from the previous layer
and then produces output voltages which are used for the input voltages of the next
layer.

Furman and Abidi have described an analog CMOS backward error-propagation
LSI (Large Scale Integration) [9]. This design is a novel cascadable CMOS IC chip
with on-chip supervised learning. Each IC chip consists of 48 analog inputs and 10
analog outputs which are fully connected together. Each pin is used for two signals :
one is feedforward signal as voltage and the other is backward error-propagation signal
as current. Using this common pin for both processing signals, they can save a large
number of pins. This IC chip can be added vertically or horizontally to increase the
number of neurons per layer between a hidden layer and the next hidden layer or the
output layer. An analog Gilbert multiplier [19] is used for the implementation of each
synapse cell. The synapse weight voltages are stored as charges on capacitors with

values 0.7 pF.

For the implementation of artiﬁcial neural nets (ANN s) using standard CMOS
VLSI technology, the implementation of synapses is the most crucial and difﬁcult one.
A synapse can be implemented using an analog multiplier which multiplies incoming
analog input by the stored synaptic weight. Currently, the analog MOS multiplier is
implemented with the inputs in voltage and the output in current [6-9]. Then its out-
put current is converted to voltage via current-voltage converters. Moreover, for higher
dimensions, the implementation becomes complex and requires larger number of

transistors.

In this thesis, feedforward artiﬁcial neural net (ANN) is implemented with on-
chip learning circuit by using a modiﬁed error back-propagation rule [43]. We employ
a simple tunable all-MOS analog multiplier whose inputs and output are given in vol-
tage units in order to implement the programmable synapse cell which operates with
analog inputs and analog synaptic weights. This multiplier can be used for a scalar
multiplication and a vector—vector multiplication. In the implementation of feedfor-
ward and feedback neural circuit, this multiplier executes the product of a vector of
(output) signals and their corIeSponding weights, i.e., a vector-vector multiplication.
With this multiplier, we can implement larger scale artiﬁcial neural networks using

fewer MOS transistors than are required by Other implementations.

In this thesis, an implementation of feedforward ANNs with a supervised learning
using standard analog CMOS VLSI/L81 technology is presented. Contributions of this

thesis are as follows:

1. A simple tunable analog multiplier is designed using enhancement-mode MOS

transistors for the implementation of feedforward ANN s with learning.

2. A CMOS operational ampliﬁer from the literatm'e [33] is considered and designed
for the multiplier. The multiplier with CMOS operational ampliﬁer is simulated
using the PSPICE circuit simulator and is implemented using the MAGIC VLSI
editor.

3.

A voltage follower is used for the driving inputs for the drain of the enhancement

MOS transistors of the simple multiplier.

Two voltage shifters/attenuators are considered and designed for the following
two reasons: The operating voltage ranges are mismatched between the inputs and
the output of the multiplier. In order to reduce the nonlinearity error of the multi-
plier, which may be caused by mobility of MOS transistor, the operating range of
the multiplier is reduced in order to reduce the variation of mobility.

The multiplier is designed with additional circuits such as voltage followers and
voltage shifters/attenuators. The performance of the multiplier is simulated using
the PSPICE circuit simulator. A very high dimensional multiplier may be
designed hierarchically, i.e., connecting lower-dimensional multiplier cells verti-
cally and horizontally together.
A derivative of the sigmoid function is implemented as one circuit element of the
feedforward learning circuitry using one 2—dimensional multiplier by assuming the
sigmoidal function to be a hyperbolic tangent function, S (x) = tanh x. That is, we
have implemented the function, S (x )’ = (1 + S (x ))(1 - S (x )).
Implemented the modiﬁed enor back-propagation learning rule [43] for a proto-
type two-layer feedforward ANN s with learning using standard MOS technology.
PSPICE simulation is performed in order to demonstrate the learning capability of
the circuits.

A module chip is considered for implementation of feedforward ANN s with on-
chip learning. A module chip for a hidden layer and one for the output layer are
separately designed using MOS transistors, since the learning circuitry of the out-
put layer has sigmoid-derivative circuits but the hidden layer does not. In this
case two different chips should be available to implement a two-layer feedforward
ANN s with learning. Another module chip has been considered for the use of
any layer since the design of two different module chips has required more design

10.

11.

eﬂ‘ort than that of one module chip. The module chip has been implemented and
simulated using the PSPICE circuit simulator.

In order to provide the ﬂexibility of the learning capability, the module chip is
designed with two learning features: on-chip learning and off-chip learning. The
on-chip learning is executed by using on-chip learning circuits which realize the
modiﬁed learning rule [43]. The off-chip learning is executed by using software
on an off-line personal computer (PC). Given any learning rule, a PC computes
the learning process and controls, via interface circuitry, the modiﬁed weights

written on the on-chip analog storages.

A MOS capacitor is employed as a analog storage device in order to store the
modiﬁed weights via the learning process. Due to leakage currents, these MOS
capacitors are required to be refreshed regularly in order not to lose the stored
information. This refreshing is done periodically through interface circuitry under
the control of a PC or a microprocessor.

Two chips have been designed on a 40-pin TINYCHIP with analog pads using
the MAGIC VLSI editor and have been sent for fabrication via MOSIS using 2
pm n-well CMOS process. One is an ll-dimensional multiplier chip which is
implemented using 39 out of the 40 pins. The other is a 4x1 module chip which

is implemented using 32 out of the 40 pins.

12. A 9x3 module chip is designed on a 64-pin MOSIS small chip with analog pads

using the MAGIC VLSI editor and ready to be submitted for fabrication via
MOSIS using 2 um n-well CMOS process. This chip has used 46 out of the 64
pins.

This thesis is organized as follows. Chapter 2 reviews the model of a neuron and

two artiﬁcial neural network models, namely, fwdback and feedforward models. In

Chapter 3, analog subcircuits are discussed for the implementation of the basic stan-

dard cells. In Chapter 4, analog multiplier cells are implemented and designed using

the MAGIC VLSI editor. PSPICE simulation is executed for the performance of this
multiplier and its applications are discussed. In Chapter 5, a modiﬁed error back-
propagation rule [43] is discussed and two—layer feedforward ANN s with a modiﬁed
learning rule are designed and simulated using a PSPICE circuit simulator. In chapter
6, a modular design is described, then a design using the MAGIC VLSI editor is dis-
cussed. This module chip can be connected vertically and horizontally for the imple-

mentation of a large scale artiﬁcial neural system. Chapter 7 summarizes the conclu-

sions of this research work.

CHAPTER 2

BACKGROUND

2.1 The Model of a Neuron

The basic building block in the brain is the neuron which can be abstractly and
schematically depicted in Figure 2.1.1. Each neuron consists of synapses, dendrites, a
cell body (or soma), and axons. Neurons communicate across synapses, receive inputs

via dendrites, integrate the received information over the cell body, and transmit the

signal through their axon.

 

 

Figure 2.1.1 A biological neuron

10

11

Among these processing elements, synapses are believed to be the primary
information-processing element in the neuron. The human brain is said to have about
1012 netuons [48] and each neuron has on average about 103 synaptic connections to
other neurons. The collective functioning through synapses is determined largely by
the way in which neural units are connected physically.

Dendrites are intricate uee of largely passive "process" or elements which aggre-
gate synaptic input current from other neurons. This aggregate form of computation is
done with slowly varying potentials control by chemical activities of neurotransmitter
ions. The input charges the cell body and changes the cell membrane potential. If this
potential, say u,- , is larger than a threshold voltage, say U, , then an action potential is
ﬁred at the axon hillock located at the end of the cell body and the beginning of the

axon.

This action potential arrives at the presynaptic side of synapses, and consequently
neurouansmitters are released from the inside of the presynaptic membrane. The
synapse is thus said to be excitatory if the postsynaptic potential becomes more posi-
tive, and inhibitory if the postsynaptic potential becomes more negative, than before.

The rate of ﬁring of action potentials is determined by the net synaptic input
cmrent. Let Til- be the synaptic conductance between the input of neuron i and the
output of neuron j , and let u,- be the cell membrane input potential of neuron i. Then
action potential pulses are generated at a rate dependent on the value of u,-. If the neu-
ron has a large input, then these pulses are generated at a high rate. If the input is
weak or absent, then the pulses are generated at a very low rate. The ﬁring rate is usu-
ally represented by a smooth sigmoid function of the input u,- , say S;(ui).

The augmented neural system is now modeled as a continuous dynamical system
represented by a set of differential equations. Let C,- be the cell membrane capacitance.
Then the rate of change of u; is represented by

12

d“,- u,-
Ci—d—t- = Z TijVj " ? +1: (2.1.1)
1

8
“i = Si-1(Vi)

Ri-l = [35-1 + z lTij l,
I

where Tij is the linear conductance modeling the synaptic weight, p,- is the local resis-
tance at the input of the cell body, I; is an external stimulation or excitation at the
input of the cell body, and Ti,- Vj is the postsynaptic current from nemon j to neuron
i.

This mathematical model can be described by the neural model circuit in electri-
cal elements, which is shown in Figure 2.1.2. A cell body is replaced by an ampliﬁer
whose input-output characteristics is a sigmoid function. Axons and dendrites are each
replaced by a uansmission wire and synapses are substituted by variable conductance

devices.

VjTij

 

S(.)_Vi_,

 

 

 

 

Figure 2.1.2 Neural model circuits

In the neural model circuit, excitatory and inhibitory synapses can be represented
by two approaches: One is by using the sign of a conductance, i.e., a positive value for
an excitatory synapse and a negative value for an inhibitory synapse. The other is by
connecting a linear resistor to the output of a normal operational ampliﬁer for an exci.

tatory synapse and to the output of an inverting operational ampliﬁer for an inhibitory

13

synapse.
2.2 Neural Network Models

Many researchers have uied to model neural network systems with current
knowledge of biological neurons since it is still under research. Neural network

models can be speciﬁed by the network topology, learning rule, and computational ele-
ment characteristics. Currently, there are several artiﬁcial neural network models.
Among them, the feedback model and feedforward model, which are discussed next.

2.2.1 Feedback Models

Two feedback neural network models are brieﬂy reviewed: the Hopﬁeld model
and the Hoppensteadt model. The Hopﬁeld model is shown in Figure 2.2.1.

 

 

 

 

o
ai-l

l

 

 

 

 

 

 

 

 

..<\lr=

 

 

 

 

 

 

 

 

 

 

 

Figure 2.2.1 The Hopﬁcld feedback model

14

The Hopﬁeld model has a simple topology and is represented with binary or analog
inputs and outputs. Each nemon has a nondecreasing sigmoid nonlinearity at each
node. Its output is fed back to all other neurons via synaptic weights. Each weight
denoted by T5,- connects the output of neuron j to the input of neuron i. There is a
symmetry requirement on the connections, namely, we must have T5,- = Tji . I ,- is the

external input current of nemon I'.

The dynamic behavior of the Hopﬁeld model can be examined by considering the

energy function of equation (2.1.1). Equation (2.1.1) can be rewritten then as

dui OE
. — - - _ 0 g 1
C‘ _ av, , (2 2 )

where the energy function, E, is given by

1 V _
=‘ i2: n,- v.- v,- + 2: Fl ' 3,- ‘WV - 21.41.. (2.2-2)
5" i t i

The time derivative of the energy function along trajectories is

 

dB 35 du.
-d_t-—-§au,- d!
315 th' dut
23am. at, at

 

S 0, (2.2.3)

since C,- 2 O and S,- (u;) is a monotone nondecreasing sigmoid function. Therefore,

this system is a gradient-er system. That is, this energy function decreases along tra-

jectories and its time-derivative equals zero at 33%:— = 0, which is an equilibrium point
i

15

of this system.
The Hopﬁeld model, as proposed by Hopﬁeld [11], employs the Hebb rule as its
learning law. The Hebb rule is simply described by Hopﬁeld [11] as
M
s=l
where T”- is the synapse weight from the output of neuron j to the input of neuron i,
V;' is the ith bit of stored memory 5 and M is the number of the stored memories.
Since equation (2.2.4) is the addition of the outer product terms, equation (2.2.4) can
be represented in matrix form as
M T
T = 2 vs V‘. (2.2.5)
3:1
If further learning is required with additional desired pattern V“ , then equation (2.2.5)

can be recursively written as
Tm = T,“ + V“ V“, (2.2.6)

where Tm is the new synapse matrix and To“ is the old synapse matrix which was
modiﬁed by the new desired patterns.

The operation of this neural network model is described follows: First, synapse
weights are set using some learning, such as the Hebb rule, from desired patterns.
Then an unknown pattern is applied to initialize the net. After initialization, the net
evolves through its dynamics. Finally, the outputs of the net converge to their steady
state values.

The Hopﬁeld model is proposed as an associative memory or to solve optimiza-
tion problems [1 1-13]. However, the Hopﬁeld model with the Hebb rule is severely
limited by the number of desired patterns which can be stored and accurately remem-
bered when it is used as an associative memory [11]-

16

SYN
d(V(Y)) S(V (X) + W) V(X)
VO’) — VCO >

 

 

 

 

 

 

 

Ill—9

 

 

 

 

 

Figure 2.2.2 The Hoppensteadt model

Another feedback neural network model is introduced by Hoppensteadt, say the
Hoppensteadt model, which is closer to the biological neuron. Figure 2.2.2 shows a
voltage-controlled oscillator neuron (V CON), which is the neuron circuit model of the
Hoppensteadt model. A VCO behaves analogous to the cell body, where repetitive
action potentials are generated. A SYN is emulating the synapse function. A SYN is
realized with a low-pass ﬁlter, which models the synaptic gap, and a comparison
ampliﬁer using a voltage adder and a linear ampliﬁer. A comparison ampliﬁer sums
the output of the low-pass ﬁlter, say W , and the postsynaptic potential, say U . This
circuit model is represented by the following mathematical model [32]:

RCEW. = _ w + avg,» (2.2.7)

d:

;g. = m, + P[V(x) + W], (2.2.8)

Where (no is the VCO’s center frequency, P is a bounded continuous differential
monotone increasing function, and d (V) is the positive part of V through a rectifying
diode.

A VCON generates its output, V(x ), at the rate of the phase x which is the func-
tion of W and V(x ). That is, V(x) is generated at a high rate when W is positive
(CXCitatory) and generated at a low rate when W is negative (inhibitory). In other

l7

wards, W is positive if the synapse is excitatory and negative if the synapse is inhibi-
tory.
The VLSI implementation of the Hoppensteadt model is not presently available

since a VCON circuit is not simple to implement using standard MOS technology.
2.2.3 Feedforward Models

Multilayer fwdforward neural nets have one or more layers of computational
nodes between the input and output nodes. Consider the basic structure of multilayer
feedforward ANNs in Figure 2.2.3, where outputs of any layer are weighted and

summed as an input to a neuron in the next layer [31].

w \1’

l 7i
X2 O

/\ A

Figure 2.2.3 General two—layer feedforward ANNs

 

 

x1

 

 

 

 

Xno

An external input is applied to the ﬁrst layer, which is called the input layer and fed
forward to the last layer, which is called the output layer. Any layer between the input
layer and the output layer is called a hidden layer. These layers are cascaded to form

18

the multilayer feedforward ANN s. The governing static equation for each neuron unit

in any layer may be described as
y,- = 5,-(ijtxt + 0,). (2.2.9)
I

where y j is the output of the j -th neuron, S ,- (.) is a nonlinear monotone nondecreasing
sigmoid function, w},- is the connection weight from the i-th output of the previous
layer to the j -th neuron input, 1,- is the i-th output of a neuron unit in the previous
layer, and 6,- is a threshold at the input of the j-th neuron.

For a supervised learning nrle of feedforward ANNs, the error back-propagation
rule was proposed by Rumelhart et al. [31]. For training the desired output or target
vector, the input vector is applied to this network and its output vector is produced,
which is compared with the desired output vector. If there is an error, the weights are
modiﬁed to reduce the error. And this process is iteratively continued until its error is

The total squared error function is given by

E = 2 3‘, (2.2.10)
11

= 2‘; g. (,pj_ypj)2, (2.2.11)

where Ep is the squared error function for the desired target p, say I} = [tpl - - ° rm]
and ypj is the actual output for the target p. l

The error back-propagation rule is given as follows [31]:

 

w; = wit—t + 2 prﬁ. (2.2.12)
P
35,
AP W}; "-= '11 aw

= 116,, jypi, (2.2.13)

19

where prﬁ is the change of the weight w,.,. at the k-th iteration for the desired target
p , 11 is the learning rate which is sufﬁciently small positive value, and ypi is the output
of the previous layer.

If a unit j is in the output layer, 8”. is given by
5 - = +[r . - ypj]. (2.2.14i)

If a unit j is in the hidden layer, 5],]- is given by

d5,- ..
Opj = 'du— : spkwkja (2.2.1411)
J k

where k is the index for the units in the next layer to which a unit j is connected.

This learning rule is iteratively computed in discrete time using equation (2.2.13)

and equation (2.2.14).

2.3 MOS Transistor

The MOS transistor plays a central role for all our analog implementation of
artiﬁcial neural nets [6, 7, 8, 9, 27, 28, 29, 30]. Here we present its basic features as
available in the literattue for completeness. Consider the simpliﬁed 3-dimensional

structure shown in Figure 2.3.1.

 

Gate
Gate Oxide W
7’ — — — - I A I
Source——-“"""‘ " = \\ Drain
Substrate

 

 

 

Figure 2.3.1 A simpliﬁed 3—dimensional view of a MOS transistor

20

It consists of several layers: from top to bottom, it contains a metal layer, a silicon
dioxide (S,- 0;) layer, a p-type or n-type silicon layer, and another metal layer. It is
called a metal-oxide-semiconductor (MOS) su'ucture, where the silicon dioxide layer is
an excellent insulator and the bottom metal layer is connected to the lowest voltage,
V” , for an n-well process and the highest voltage, Vdd , for a p-well process.

There are two heavily doped n-type (n +-type) regions in the p-type material,
which are respectively called drain, whose voltage is denoted by VD , and source,
whose voltage is denoted by vs. These are separated by L , which is called channel
length. The top metal electrode is called the gate whose voltage is denoted by VG .
The body of the semiconductor is usually called the substrate or bulk whose voltage is
denoted by v, . Since this structure is symmeuic, the roles of the source and the drain
are interchangable. This overall device is called the nMOS u'ansistor.

For the nMOS transistor, when a positive voltage is applied to the gate, electrons
start to deplete the substrate near the surface under the gate and form a depletion
region under the gate. If more positive veltage is applied to the gate, then more elec-
trons will be attracted to the subsuate surface under the gate and this region is changed
from p-type to n-type, i.e., inverted. This n-type region is called an inversion layer or
channel. The gate-source voltage which is required to create the inversion layer is
called the threshold voltage, VT .

The pMOS uansistor is similarly obtained by forming p+-type region in the n-
type material. Both nMOS and pMOS transistors are four-terminal devices. The MOS
transistor size is represented by the channel width, W , and the channel length, L ,
which are depicted in Figure 2.3.1. W/L is called the aspect ratio of the MOS transis-

tor.

The convention of electrical variables of MOS transistors are shown in Figure
2.3.2. Along with this convention, a mathematical MOSFEI‘ dc model is obtained in
order to predict the experimental characteristics for a given aspect ratio, process

21

Drain Drain
+ +
VDs i1” VD {In
Gate Bulk Gate °_—_i
+ + : +
Vos +1 3 VHS Vos +15 VBs
Source Source

(a) (b)

Figure 2.3.2 Convention for electrical variables for MOS uansistors

(a) nMOS transistor (b) PMOS transistor

parameters, and applied voltages. There are three different operating regions: cutoff

region, ohmic (or triode) region, and saturation region.

Assuming that the channel is sufﬁciently long and wide and the substrate is uni-

formly doped. For an nMOS transistor, the drain current is approximated by the fol-

lowing quadratic function:
Cutoff: if (v65 - V1) 5 0

i0 = 0
Triode: if (sz - v7) 2 (V05)

v 2
in = uoCa,(W/L)[(VGS - vT)vDs - if] ( 1+ tvDs )

Saturation: if (sz - v1») 5 (V05)

. v.6...
tn- 2 (WILchr-vrftnxvas)

 

with the threshold voltage
VT =Vro +‘K‘i9-Vas ‘45).

When ll.Ca,,A, V10, y,and¢areprocess parameters:

(2.3.1)

(2.3.2)

(2.3.3)

(2.4.4)

 

 

u = channel mobility

C“ = capacitance density of the gate-channel capacitor
1 = channel length modulation

V10 = threshold voltage for v33=0

y = bulk threshold parameter

4) = strong inversion surface potential.

For a pMOS transistor, we can obtain the drain current I'D similarly by taking the

negative sign of the cunent and voltage values, i.e., I'D is substituted for -I'D and VI is

replaced by -vx, where v, is any one of IQ, , vs, and VT. That is,

Cutoff: if (VGS - VT) 2 0
i0 = 0

Triode: if (v65 — vr) s (V05)
v 2
. DS
10 = — u0C0x(W/L)[(VGS - VT)VDS — T] ( 1 - AVDS )
Saturation: if (v63 - VT) 2 (V05)

u C
iD _ - ”2“ (W/L)(—sz +v,)2( 1 -1.VDS )

 

with the threshold voltage

VT=VTo "KV¢+Vas ”‘15).

(2.3.5)

(2.3.6)

(2.3.7)

(2.3.8)

nMOS devices become more popular because of higher mobility of electrons, which

makes nMOS transistors faster than pMOS transistors.

As VLSI MOS technology is deve10ping, the device minimum size is getting

smaller. In analog circuit designs using MOS technology, the channel length and the

channel width are very important. The above equations are derived with the assump-

tion of a sufﬁciently long and sufﬁciently wide channel. If the channel is not

sufﬁciently long or sufﬁciently wide, these equations are changed a little [36].

23

Moreover, for short channels, MOS transistors face another problem, namely, limita-
tions on voltages. For example, when an analog circuit is designed using 2 11m
minimum feature size of a transistor, as supplied via MOSIS, the maximum allowable

voltage difference is only 10 volts.

CHAPTER 3

BASIC ANALOG SUBCIRCUITS

The following basic analog subcircuits are described in order to integrate analog
multiplier and feedforward artiﬁcial neural nets (ANN s) with learning as building
block scheme: a simple tunable analog MOS multiplier, a CMOS operational
ampliﬁer, and voltage shifters/attenuators. These basic analog MOS subcircuits have

been designed and implemented using standard CMOS technology:

A simple tunable analog scalar/vector multiplier [15, 16, 17, 23] is used to
implement an artiﬁcial synapse cell by performing a multiplication analog inputs by
analog synaptic weights. In order to implement an artiﬁcial synapse cell of large
scale ANN s, the multiplier is designed with a CMOS operational ampliﬁer, a voltage
follower, and voltage shifters/attenuators. A follower integrator is used to perform the
integration of the learning algorithm and is realized with a voltage follower and a
MOS capacitor. A MOS capacitor is also used as an analog storage element to store
a synaptic weight. A CMOS double inverter is used to realize a cell body, whose
input-output characteristics are represented by a sigmoid function. An analog multi-
plexer is addressed to read or write synaptic weights on on-chip analog memories via
off-chip interface circuitry.

All the basic analog MOS subcircuits are simulated using the PSPICE circuit

simulator, which is a commercial circuit simulator.

24

25

3.1. A Simple Tunable MOS Analog Multiplier

The output voltage of a multiplier is uniquely determined by any pair of inputs
which are in the operating range. Four-quadrant multipliers respond to all four com-
binations of input signals and provide output signals of correct polarity. Presently,
some analog MOS multipliers have been implemented using the MOS version of the
Gilbert multiplier which performs multiplication with very good accuracy. However,
these multipliers seem to take a large number of transistors for the implementation of
a vector-vector multiplier which implements a synapse cell for large scale ANN s.

Here we employ a simple tunable four-quadrant analog multiplier [15, 16, 17,
23] in order to implement a synapse cell of a large scale ANN. This multiplier is
easy to implement as high dimensional vector—vector multiplier with less number of

transistors than the existing multipliers.

Consider four matched nMOS transistors which are inside the box in Figure

3.1.1 [15, 16, 17, 23].

 

5'2:
I
l:

a. .2
_1 l
:31:

h
3

 

 

 

 

 

 

 

$9.”;

 

 

 

g1
II

 

 

 

Figure 3.1.1 A simple four-quadrant analog multiplier

26

The complete expression of the drain current in the nonsaturation ohmic region is

represented by [36]

’0 =XCVGrVD) -8(VG.Vs)r (3.1.1)

which shows the symmetry of drain and source and where

;
3(ervy)=K[ (V: " VB '"VFB -¢B)Vy _ é—(Vy -VB)2"' %Y(Vy —VB +¢B)2]

«1 Ne
withK=ttCole—andy=ié"—S.
at

Assuming all transistors are matched, operate in the ohmic region, have the same
constant mobility, and the operational ampliﬁer is ideal. Then given terminal inputs

X1, X2, W1 and W2 [16],
[1-12:10‘ +102—103—ID‘
= g(X2,W1) - 8(X2.V) + 8(X1rW2) - 8(erV)

' 8 (X2,W7_) + 8 (erV) " 8(X1»W1)+ 8(X1»V)
.3.
=Kl (X2— Va " VFB '¢B)W1'-;'(W1'VB)2-'§—KW1- V3 +¢a)2
2 2.
+(X1- V3 "VFB ’¢B)W2- %(W2'VB)2- 3W2'VB +¢a)2

3
2 _
'(Xz- VB "" VFB -¢B)W2+ ';—'(W2-VB)2+ EWZ—VB +¢B)2

le

1 2
-(X1- Va "VFB '¢B)W1+ ‘2'(W1’Va)2- 3W1 "VB +93) 1

= K (X2 -X1)(W2 ' W1)-

That is,

11'12=K(X2‘X1)(W2"W1)- (3.1.2)

27

Therefore, its output V," can be represented by

V," = R (I 1 - I 7)

=KR(Wl-W,)O(1-X2), (3.1.3)

where K = 11C“ (W/L) is the transconductance parameter of the nMOS transistors
for the feedforward devices. Since equation (3.1.1) is subject to operate in the uiode
region, the following operating conditions should be satisﬁed [27]:

W1. Wzsmin I: “I “V10, (XZ-VT)]
or

Xi '- Wj 2 VT > 0 for all l,j = 1,2. (1°14)

The multiplier shown in Figure 3.1.1 can be implemented with only MOS
transistors by replacing the two resistors by another matched four nMOS transistors.

This all-MOS analog multiplier is depicted in Figure 3.1.2.

 

 

 

 

 

 

 

 

 

 

 

 

_L_£

TXé l I2 Xi Xi

M4 —> E ii
T

_1_E w,1 l-D
TX} p. ‘62 w;
vc, V

 

 

 

Figure 3.1.2 All-MOS simple four-quadrant analog multiplier

28

It is assumed that all uansistors are matched, all transistors operate in the ohmic
region, have the same constant mobility, and the operational ampliﬁer is ideal. For

the terminal voltages Vc l, Vc 2, V,” , and V, = 0, we can derive the following equation

from equation (3.1.1):
— (ll — 12) = Kch, — Vc2)(V,,, - Vt). (3.1.5)
Therefore, the output of this multiplier can be represented by [27 , 28]

V = K1(W1’ W2)(X1-X2)

m K, (Vc‘ _ ch) a (3.1.6)

 

where K 1 = 11C” (W/L )1 is the transconductance parameter of the nMOS transistor
for the feedforward devices and K, = “Cwr (W/L ), is the transconductance parameter
of the nMOS transistor for the feedback devices. Vcl and Vc2 are used for tuning and
to specify the multiplier constant for given K 1 and K, . This multiplier may be used
as a divider when Vcl and Vc; are used as the dividend with ﬁxed W,- J- . The operat-
ing conditions of this multiplier are given by [27, 28]

Xi-WjZVT>0 forallt',j=1,2,and

Vc‘ — VT 2 0 and V6, - VT 2 V," for all i = 1,2. (3.1.7)

The multipliers in Figure 3.1.1 and Figure 3.1.2 can be extended for the multi-
plication of two n-dimensional vectors, i.e., n -D vector inner product when additional
(n - l)-four matched nMOS transistors are connected to the input nodes of the
oP‘Bl'ﬁtional ampliﬁer [27]. This multiplier will be called the n -D multiplier. Then
equation (3.1.3) can be extended to read as follows [27]:

V... =R 2": K.- (Wt — W5) (Xi —Xi>. (3.1.8)
.gl

Where the superscript 1' is used as an index. Assume all nMOS transistors have the

same aspect ratio (W IL ), equation (3.1.8) becomes [27]

29
ﬂ . . . .
V," = uCaR (WIL) 2 (W‘, - ‘2) (X‘l -X‘2) (3.1.9)
i=1
under the following operating conditions [27, 28]:

x} - wj" 2 VT > 0 for all i, j =1,2 and k =1,2,...,n. (3.1.10)

Similar to the above n-D analog multiplier, this multiplier can be implemented
using MOS transistors only. An all-MOS n-D four-quadrant multiplier is shown in
Figure 3.1.3 [27, 28]. Similarly with n -four matched nMOS uansistors, equation
(3.1.6) can be extended to the vector inner product [27, 28]

 

_ 1 " . 1' _ 1' i _ 1‘
V", - K,(Vc, __ V“) £1 K, (W1 W,)(Xl X2), (3.1.11)

where the superscript I' is used as an index. Assume all the nMOS transistors of the
feedforward devices have the same aspect ratio (W/L ), equation (3.1.11) becomes
[27, 28]

(WIL),- n . . . .
= ‘ - ' ' - X' . 3.1.12

 

Similar to equation (3.1.7), the operating conditions are given by [27, 28]
it," — wj" 2 VT > 0 for all i,j = 1,2 and k =1,2,...,n and
Vc‘ — V7- 2 0 and V6, — VT 2 V," for all 1' = 1,2. (3.1.13)

Since the output of this multiplier ranges from negative to positive voltages, the

above constraints can be simpliﬁed by

x,‘ - W]! 2 VT > o for all i,j =1,2 and k =1,2,...,n and

V6, - VT 2 V," for all i = 1,2. (11.14)

This analog multiplier has complete cancellation of nonlinearities except for a mobil-

ity term. A mobility is represented using SPICE parameters as follows:

 

1 1
I": E
w: O—J ' 7

 

bl

 

 

 

 

W

"i

.519

a;
T
aft-i

 

 

Figure 3.1.3 All-MOS n-D analog multiplier

31

(UCR’Tkri
ColeGs ' Vr - (UTRAWDsl

 

“'3 = “0|: ]UEXP’

if (UCRIT )e,i
Car

 

< VGS " VT " (UTRA ”’05

11, = 110, otherwise,
where UCRIT, UT RA , and UHF are process parameters :
UCRIT = mobility degradation critical ﬁeld
UTRA = mobility degradation transverse ﬁeld coefﬁcient
UEXP = mobility degradation exponent.

That is, this term is still dependent upon the terminal voltages, VG , VD , and VS , and
this attributes to the imperfection of the multipliers in real-world implementation. A
complete implementation of the analog scalar/vector multiplier is discussed in

Chapter 4.1.
3.2. A CMOS Operational Ampliﬁer

A CMOS operational ampliﬁer is considered in order to implement the above
simple tunable all-MOS analog multiplier using standard CMOS technology. We
have designed a CMOS Operational ampliﬁer to approximate an ideal Operational
ampliﬁer with a sufﬁcient open-loop gain, very small input Offset voltage, and a quite
large output current. However, a real Operational ampliﬁer is implemented with ﬁnite
gain, nonzero output resistance and nonzero input offset voltage. These factors attri-
bute to some errors or imperfection of the multipliers.

Presently, many CMOS Operational ampliﬁers are available in the literature. Fig-

ure 3.2.1 depicts the schematic circuit of a simple p-channel input, unbuffered two

Stage CMOS Operational ampliﬁer [33].

32

 

 

 

 

1! V011!

1‘.
Vl—itiMl M2 l_v2

 

 

 

 

M6

 

 

 

 

 

' in

Figure 3.2.1 A CMOS Operational ampliﬁer

In the design of this operational ampliﬁer, it is crucial to select a proper bias
voltage, Vm , and the aspect ratios Of the MOS transistors. MOS transistor M1 and

M2 are assumed identical and M3 and M4 are also assumed identical. Then the

overall linear gain of the Operational ampliﬁer is given by [33]
A, = A,l A,2

8”“ 803‘
g0; + 804 go; + 807 ’

l

2
Where 8!!! = [Isucaxlzi] 811d 30 = “(1'
This Operational ampliﬁer is usually designed with the aspect ratio relationship

[33]:

(WIL)3 -_ (WIL)4 _ _1_(W/L)5
(IV/L)6 ’ (WIL)6 ‘ 2 (W/L),‘
For low noise, M3 and M4 are designed with low transconductance, i.e., with small

33

aspect ratios. For a better frequency response, M6 is designed with high transconduc-

tance, i.e., with large aspect ratio.

The bias voltage of the CMOS operational ampliﬁer is chosen to 3.2 v. The
device sizes and the characteristics Of the CMOS operational ampliﬁer, based on our
PSPICE simulations, are summarized in Table 3.2.1. Figure 3.2.2 shows the PSPICE
simulation result of the Open loop input/output characteristics Of the CMOS opera-

tional ampliﬁer and its frequency response. The PSPICE input ﬁle for the frequency
response of the CMOS op-amp is shown in Appendix 3.1.

Table 3.2.1 (a) The device sizes (unit: um) and (b) the characteristics of the CMOS op-arnp

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

(a)
16 4 4
M1 and M2 7 M3 and M4 '1'; M5 10 - -
240 200 4 40
M6 4 M7 4 M8 and M9 4 M10 40
(b)
voltage supplies V“ = 5.0 v and V. = —5.0 v
input voltage Offset —0.1 mv
input common-mode range -4.5 v to 4.5 v
open-loop voltage gain 70dB
unit-gain bandwidth (GB) 8 MHz
phase margin 70 degree
slew rate 2.5 v(11.sec)'l
Output cunent at V*—V'= o 390 M

 

 

 

 

 

Figure 3.2.1(a) The open loop transfer function and
(b) the frequency response of the CMOS op—amp

35

3.2.1. A Voltage Follower

The vector multiplier circuit is suitable to implement an artiﬁcial synapse in
artiﬁcial neural nets since it is simple and executes linear multiplication. However,
the multiplier circuit has some practical difﬁculties when it is implemented. The
multiplier circuit has a driving input to the drain Of an MOS transistor, whose input

impedance is low. This will create problems of loading.

In order to overcome this problem, a voltage follower is desired. The output
impedance of the voltage follower is so small that it can supply sufﬁcient current to
the low input impedance node of the drain Of the MOS transistor Of the vector multi-
plier circuit.

A voltage follower can be implemented with any standard (MOS) Operational
ampliﬁers. The voltage follower has been designed as the same structure as Figure
3.2.1. It has been implemented with a lower output impedance than the impedance of
the driven MOS transistor in order to supply sufﬁcient current to low input
impedance node of the drain of the MOS transistor of the multiplier. The PSPICE
simulation has been executed and has shown prOper Operation of the voltage follower
supporting our contention. The device sizes Of the voltage follower according to our

design are summarized in Table 3.2.2 with setting the bias voltage of the voltage fol-

lower to 3.2v.

Table 3.2.2 The device sizes (unit: um) of a voltage follower

 

MlandMZ M3andM4 -— M5

 

M10

 

 

 

 

 

Ala» 31].».

 

 

E.
4
M6 5&9- M7 1— M8andM9

ta'g

 

 

 

36

3.2.2. A Voltage Integrator

A voltage integrator will be used to perform the integration of the learning algo-
rithm. This is identical to a voltage follower except that it has a capacitor load [30],
which is shown in Figure 3.2.3. The capacitor and the output impedance of the
Operational ampliﬁer will determine the time constant for the learning dynamics. The
device sizes of a voltage integrator is the same as those Of a voltage follower which

are shown in Table 3.2.2. The PSPICE simulation has been executed to verify its
proper function.

 

I

Figure 3.2.3 The voltage integrator circuit

3.3. Voltage Shifters/Attenuators

In order to use the vector multiplier in the implementation Of a synapse cell, the
operating range Of its input must be considered carefully. The input of the multiplier
in ANN s is applied from either an external voltage source or the output of a neuron.
However, the output and input of the (vector) multiplier are not properly matched. In
addition to input-output matching or compatibility, one must always satisfy the

Proper Operation constraints of the multipliers.

The multipier has been designed to operate in a small operatin g range in order

to reduce the nonlinearity-error effects which are caused by the variation in the

37

mobility parameters. Therefore, the operating range of the multiplier is required to be
ampliﬁed to the level of range of a neuron in order to use the output of a neuron as
an input signal. The difference in unmatched voltages and the increase in the operat-

ing range can be adjusted by employing a voltage shifter/attenuator.

There are several kinds of level shifters in the literature [17, 25, 26]. A level
shifter can be implemented by using an operational ampliﬁer and resistors. However,
it takes a large area and is difﬁcult to implement using MOS technology because of
the implementation of the linear resistors. Simple and small-area level shifters are

suitable for the implementation of neural nets [1'], 26].

Two level shifter/attenuator circuits are considered. One shifter/attenuator cir-
cuit is for the input of the vector multiplier w,- . We label this shifter the w_shifter.
The other shifter/attenuator circuit is for x,- . We label this shifter the x_shifter. Figure
3.3.1 shows the level shifter/attenuator for the w; input. It is assumed that the
uansistors, M1, M5, and M7 are operating in the saturation region and the transistor

M3 is Operating in the ohmic region [25]. Then, its drain current is represented as

follows:

K
1'41: 71 (Va — V3 - VT1)2 (1 + 2w,“ - V3))

. V3" vs:
1113 = K3 (Vin - V” - VTo- T) (V3 - Vu)(1+ A'(‘13 — V83»

Since 1",,! = 1'“, and assuming 2. = 0, one obtains

1 V 2 VS-Vu
‘3‘( in’V3-Vr) =K3 (Vin ‘Vrr “W." ——2_—) (V3'Vrr)

K
= 33- 1 (V... —Vr,-V.. )2 - (V... ’Vr.’V3)2l-

Simplifying the above equations, one gets

(K, + K3) (Vin-1134,)2 = 1r3 (vi, 4170-41”)2

38
VDD
1
new
.— vom
t n.

Vin V3‘F——| 1M5

that

 

 

 

 

 

v w_Dhiﬁer Von

 

 

 

 

 

 

 

 

i

VSS

Figure 3.3.1 The w_shifter

K3
Vin - V3 — VT, = Vm (Vin " VT, - Vss)°

Rearranging the last equation containing V3, and assuming VT1 = VTO, one obtains

K3 K3
V = 1- —— V. —V +‘\/———V. (3.3.1)
3 ( VK1+K3 )( m To) K1+K3 SS

Since the pMOS transistors, M5 and M7, are in the saturation region, then id, = 1°47.

 

Assuming 2. = 0, one gets

K5 K7 2
70/01:! - V3 + VTs)2 = To,“ . Vb + VT?)

K
VM=V3—VT+V—7(Vdd—Vb+VT1)
5 K5

—V + _: V -V + -——V —V (332)
[V T T]- ~-
3 VK ( dd b) K 1 5

Substituting equation (3.3.1) for equation (3.3.2),

K
M (l VK1+K3) m VK5( d b) K1+K3 SS

 

 

 

39

 

 

VK 13K 1 V V ‘\/ 7V 333
—- — + — . . .
+( 1 3 ) r, T, K, T, ( )
That is, the attenuation factor, aw , is given by

l ’\/ 3 334
a”. - K1+K3' (°')

The level shifting, V,,, , is given by

V-VK?(V V>lv K3 V
d' K5 dd b K1+K3 ’5
+ V—3 -1)V ’V +V—7V 335
O O.
( K1+K3 To T5 K5 T7 ( )

Figure 3.3.2 shows the level shifter/attenuator for the 1 inputs, namely the
x_shifter. This circuit is the as same as the ﬁrst stage of Figure 3.3.1 except that
pMOS transistors are used, in stead of nMOS transistors, for two input transistors.

Similarly, the output of this level shifter/attenuator is given as

VDD

__{M3

vout

 

 

 

—4| Ml

VSS

 

 

 

 

 

 

Figure 3.3.2 The x_shifter

 

v -(1\/ K2 (V v )+\/ K2 V 336
0111‘ Kl't'Kz) in T0 K1+K2 dd. ()

 

The attenuator factor for the x_shifter, a, , is given in the same form as aw . How-

ever, these values will be different since the K ,- values are dependent upon the

40

transistor type and the aspect ratio of the MOS transistors used.

These level shifters/attenuators have been simulated using the PSPICE circuit
simulator. Their device sizes and PSPICE simulation results are summarized in Table

3.3.1 with setting the bias voltage of the w_shifter to 3.0V.

Table 3.3.1 (a) The operating ranges and (b) the device sizes (unit: um) of the w_shifter and

 

 

 

 

 

 

 

 

 

the x_shifter
(a)
shifter range of inputs shifting attenuation factor max. enor(%)
w_shifter -2.5 to 2.5 4566 0.0813 0.0019 (0.47)
x_shifter -2.5 to 2.5 4.686 0.0766 0.0013 (0.34)
(b)

 

M1&M2 M3&M4

 

 

 

 

—s
O

 

 

. 11 10 4 4

sin __ _ _. __
w- m 22 4 4 4
x_shifter —§— 249 - -

 

 

 

 

41

3.4. A MOS Capacitor

Theoretically, the capacitance is represented as

A
= —’ .4.
C e d (3 1)

where A is the area of the electric plate and d is the distance between two electric
plates. A capacitor can be designed by several different ways using standard MOS
technology [35]. One method uses double metals or double poly. However, this
method requires large chip area due to the thick oxide layer between double metals
or double poly. Another method uses the gate oxide capacitance when a MOS
transistor is operated in the ohmic region. This method requires less chip area than
the ﬁrst method because of the thin oxide layer of the gate of a MOS transistor.

We have employed the second method in order to implement an MOS capacitor
which is shown in Figure 3.4.1. The MOS transistor is biased in the ohmic region
by connecting its drain, source, and substrate nodes to the same voltage level, say V”
and by satisfying Vs: - VT > 0. Then the gate forms one plate and the source, drain,
and channel form another plate. This capacitor can be used very effectively in non-
critical applications. However, the sheet resistance of the bottom plate formed by

the channel is high since the underlying subsu'ate is lightly doped.

Using equation 3.4.1, the gate oxide capacitance, C8“, is represented as

801
C80, =A Co, = A 7"“ (3.4.2)

a

Therefore, if we want to determine the actual transistor size for a capacitance, say

C 1, then the actual chip area, A is found by using

A= C1=C1Tax

Ca, ea, ’

 

(3.4.3)

whereA = WL.

 

 

42

 

 

 

 

 

 

 

 

 

0<
V

 

 

 

 

 

 

 

 

 

VSS

Figure 3.4.1 A MOS capacitor

Given the SPICE parameters in Appendix A.2, Table 3.4.1 below summarizes

the device size corresponding to different capacitance value.

Table 3.4.1 The device size of MOS capacitors

 

capacitanceﬂlpF 2pF 3pF 4pF 5pF
area(um2) " 1200 2400 3600 4800 6000

 

 

 

 

 

 

 

 

3.5. A CMOS Double Inverter

A CMOS push-pull double inverter is shown in Figure 3.5.1(a). This can be
used to implement a cell body by cascading two inverters and forming a double
inverter since its i/o characteristics shows a sigmoid function (Figure 3.5.1(b)).
Sometimes a simple operational ampliﬁer is used for the implementation of a cell
body.

43
Vdd2
M2 M4
Vdd Vdd
V in v”
M1 M3
VII VII
V1.32
(a)

 

Figure 3.5.1 (a) A CMOS double inverter and
(b) Transfer characteristics of the double inverter

The maximum and the minimum output of this double inverter are given as +2.5
v and -2.5 v, respectively, in order to be matched with the maximum and the
minimum input of the analog multiplier. For latch-up protection of MOS transistors,

p-well contact and n-well contact are connected by +5 v and -5 v, respectively.

3.6. An Analog Multiplexer

An analog multiplexer is one of the important elements for the implementation
of ANNs. This is especially due in the design of ANNs with a large number of neu-
rons since standard chips have a limited number of pins. In Chapter 6, an analog
multiplexer is used to be addressed to read or write synaptic weights stored on on-
chip analog memories via off-chip interface circuitry. This analog multiplexer can be
easily designed with CMOS analog switches and a digital decoder. When an address
is applied to the decoder, its corresponding analog switch is turned on and an analog
signal is passed through the analog switch from the Digital/Analog converter (DAC)
or Analog/Digital converter(ADC) on the interface circuitry. Figure 3.6.1 shows the

44

block diagram of a 1x8 analog multiplexer, where switches are implemented by a
CMOS analog switch.

W0 W1 W2 W3W4 W5 W6W7

 

 

 

 

 

 

 

118 8

 

 

 

 

A3

 

 

 

mo

Figure 3.6.1 A 1x8 analog multiplexer

CHAPTER 4

ANALOG SCALAR/V ECT OR MULTIPLIERS

The following standard cells are described in this chapter: analog multiplier cells,

analog adder cells, programmable synapse cells, and a sigmoid-derivative cell.

4.1. Analog Multiplier Cells

An all-MOS four-quadrant analog multiplier is implemented using the basic ana-
log MOS circuits which are described in Chapter 3. The output Of this n-dimensional
analog vector multiplier is given by [27 , 28]

W/L - n . . . .
m 2 (ML; (V )‘_ V 2) 2 (W1 - W5) (Ir'1 -x',) (4.1.1)
f C; C i=1

 

with the following operating consuaints

X,‘ - Wj" 2 VT > 0 for all 1',j =12 and k =1,2,...,n and

Vc, -- VT 2 V", for all 1’ = 1,2. (4.1.2)
Due to the operating constraints of the multiplier, equation (4.1.2), the output

range of the multiplier does not match its operating input range. The multipier has

been designed to Operate in the small Operating range in order to reduce the

nonlinearity-errors caused by the variation of the mobility. The Operating range of the

multiplier is required to be compatible with the operating range of a neuron in order to

45

use the multiplier as an artiﬁcial synapse.

There are two problems to be considered for this multiplier in order to implement
it as a synapse in artiﬁcial neural nets : One is the multiplier’s driving input to the
drain of a ﬂoating MOS transistor, whose input impedance is low; the other is the

operating range of the multiplier and its input-output comparability.

The ﬁrst problem is solved by employing a voltage follower, which is described
in section 3.2.1. In order to make the operating input range match the output range of
the multiplier and/or increase the operating input range of the multiplier, voltage

shifters/attenuators are employed, which are described in section 3.3.

The 1-D multiplier cell is designed with additional circuits such as voltage fol-
lowers and voltage shifters/attenuators, which is depicted in Figure 4.1.1. The inputs
are applied through the voltage shifters/attenuators. A voltage follower is placed in
prior to the drain input of the multiplier in order to drive sufﬁcient current to the low-
input-impedance drain inputs of the multiplier. The Operating range of the multiplier is
given by -2.5 S w;, x;, V," S 2.5.

vxl vx2

l i

x_shifter x_shiﬁa

i

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

- Vblta V
VW l w_shifter 10110:: 7 Mi] .1) 1i ‘ m
. up er
VW2 w_shifter PM" 7 V
xi

 

 

 

 

 

Figure 4.1.1 A 1-D multiplier cell

This multiplier is deﬁned and designed as a standard analog multiplier cell which
may be up to 20-dimensional vector multiplier, called n -D multiplier. The PSPICE

47

simulations have been executed up to 20-D vector multiplier with the SPICE parameter
in Appendix A2. The device sizes and the tuning voltages are summarized in Table
4.1.1.

Table 4.1.1 The device sizes (unit: um) and the applied tuning voltages (unit: volts) of each
standard analog multiplier cell

 

 

 

W W
"‘D [I], [7:] V0: ' V62
4 4
1-Dto 10-D 24 48 0.041 n
4 4
- D _ _ .
11 D to 20- 48 48 002 n

 

 

 

 

 

 

All the standard multiplier cells are designed with the same size of the feedfor-
ward nMOS transistors and the feedback nMOS uansistors for ID to 10-D and for
MD to 20—D, respectively. They are controlled by setting Vc, — Vc2 to 0.041n volts
for the former and to 0.0211 volts for the latter. However, these tuning voltage values
are determined by the SPICE parameters used

When a very high dimensional vector multiplier is required, this design requires
large chip area. That is, from equation (3.1.12), as n is increased, L,- is also increased
with ﬁxed W3, W,, and L,. This results in a large chip area.

Hierarchical design may be suitable to design a very high dimensional vector
multiplier. The standard multiplier cells, n -Ds and the standard adder cells, n -DAs,
are used for hierarchical design. The standard adder cells are implemented using the
standard multiplier cells, which will be described in section 4.4.3.

Figure 4.1.2 shows one example with the implementation of a loo-dimensional
vector multiplier. This multiplier is implemented using ten 10—D s and one IO—DA .

Each 10—D cell multiplies two 10-dimensional vectors, i.e., lO—dimensional vector

48

inner product. A 10-DA cell combines ten vector inner products which are obtained

from ten IO-DM cells and results in loo-dimensional vector inner product.

Using this hierarchical design with standard cells, we can reduce the chip area
and its design time for a very high dimensional vector multiplier. However, this design
has two disadvantages. One is a time delay since the computation is done through
more than one layer. The time delay is proportional to the number of layers. The

other is that the error becomes larger than that of the non-hierarchical design since

each layer generates errors.

 

 

 

 

 

 

31-10
to
w to
to
w 10
..... ms
to
to
wn-n E3
to
to
”am I!!!
re
re
W41-30
IE .1“, _....
to
10
when m
to
to
"61-70 I!!!
to
w 10
mo rm x
o to
1 rec
“mo m l.
81.10:) 100
m 10 W-+o- lm-D —>
"m m3

 

 

 

 

 

 

Figure 4.1.2 A lOO-D multiplier cell

49

4.2. Performance of Analog Multipliers

The performance of the multiplier [47] is usually checked by ﬁnding the follow-
ing measurements : output offset, X or W nonlinearity, X or W feedthrough, small sig-

nal bandwidth, slew rate, and settling time.
First, the output function is determined by the anticipated functional relationship

among two inputs, AV, , AV, , the output, VWW , and the scale constant, V, . The

multiplier in Section 4.1 produces the following output function.

AV, AVw
Vanticr'pated = T
AV AV
= L‘s—.0 “' , - (4.2.1)
where
AV, - V, l V,2
AVw = Vw‘ - Vw2

with the operating range, -2.5 _<. v v,“ s 2.5 volts and V,“ = 0.

113’

All errors are deﬁned as the deviations from this output function. Percentage

error is calculated as follows:

8
% E = 1 _, . 4.2.2
rror 00 F S ( )

Where a = Vm - anticipated and FS means full-scale range.

The output oﬁ‘set is measured at the output voltage by applying V, = Vw = 0.
X or W nonlinearity is measured with the maximum percentage error of full-scale
between the actual multiplier output and the anticipated output with the full-scale range
0f the corresponding input and the full-scale dc value on the other input. Speciﬁcally.
X nonlinearity is measured with a 10.0 Vp-p 50-Hz triangular waveform on AV, and

50

2.5 dc voltage on AV“, . W nonlinearity is measure with a 5.0 Vp—p SO-Hz triangular

waveform on AV“, and a 5.0 dc voltage on AV, .

X or W feedthrough is usually speciﬁed by applying a 50 Hz full-scale sine wave

on one input and zero on the other.

Small-signal bandwidth is achieved by measming the frequency at which the out-
put has decreased to -3 dB of the full-scale output with a full-scale dc voltage (5 volts)
on one input and 5% of full-scale (0.25 volts) sine wave on the other. Slew rate is the
maximum rate Of change of output voltage and settling time is the time that the output
takes to approach within a speciﬁed percentage of its ﬁnal value in response to a full-
scale step input and a full-scale dc voltage.

The above speciﬁcations are measured using the PSPICE circuit simulator for 1-
dimensional analog multiplier which is shown in Figure 4.1.1. Table 4.2.1 compares
the speciﬁcations of 1-dimensional analog multiplier and a commercial multiplier
AD532J [51], which is designed with a Gilbert cell using bipolar technology. Scalar
to ll-D vector multiplier is simulated to measure their output offset and maximum
error. Its results is summarized in Table 4.2.2. Figure 4.2.1 depicts the graphs which
are achieved from the PSPICE simulation: the multiplication in Figure 4.2.1(a), the x-
nonlinearity error in Figure 4.2.1(b), the w—nonlinearity error in Figure 4.2.1(c), and
the slew rate in Figure 4.2.1(d). The PSPICE inputs for the above measurements are
shown in Appendix B.2-B.6.

51

owns?» 5.50530: 255* .25. "8E 828m
Raw 59:: ”come 0622
can 39:0 ”8E mom.
annr>a e5 a §un>a so
cause a; La 556528: 2:. 3 3.4 ear

,. c: from al. 3

................. + E ._.n.-

_
_
_
_
_
_
_
_
.
.
2
2

 

22322: or do mica 82.82352 3 3.2. 2:32

 

52

a.
A
H,
I.
T

 

scans: e5
o>a3 855m Nmoow A32, 3 u x><
Bane—=8 D-“ we 8.5%“: @on AB ﬁne Baum

ow§o> 553:5: 39:4» 2E. ”cog Eoeom
Rama; ”cog 22:2
Rama. 39:0 Hooch nob
.>nu..>< e5 arrowhead ca
2232...: a; dc chaise? 2F 3 3.4 cameo

 

53

Table 4.2.1 The speciﬁcations of 1-D multiplier and AD5321 multiplier [51]

 

 

 

 

 

 

 

 

 

 

 

l-D multiplier AD5321
Power Supply :t 5v :1: 15v
Input-Output Range 50va 10v,_,
Output offset voltage -0.087v 10.04v
X-input nonlinearity 2.0%FS :t 0.8%
W-input nonlinearity 3.0%FS :t 0.3%
X-feedthrough 0.02 mvpﬂ, 50 mv
W-feedthrough 9 mvh, 30 mv
-3dB Bandwidth lMHz
for the entire circuit SkHz
multiplier cell only 2MHz
Slew Rate 0.1 votsecr‘ 45 vqlsec)“
Settlingtime 117usect02% lusecto2%

 

 

 

 

 

Table 4.2.2 Output offset voltage and maximum percentage error of analog multiplier cells

 

 

 

 

 

 

l-D 2-D 3-D 4-D S-D 6-D
Output Offset (v) -0.087 -0.094 -0.096 -0.097 -0.0976 -0.0977
96 Error (96) 2.26 3.82 4.14 4.34 4.49 4.596
7-D 8-D 9-D lO-D 1 LD -
Output Offset (v) -0.0976 -0.0975 -0.0973 -0.097 -0.0967 -
% Error (96) 4.69 4.75 4.83 4.88 4.95 -

 

 

 

 

 

 

 

 

 

54

4.3. Implementation of an ll-D Analog Multiplier

An MD analog multiplier is designed using the MAGIC VLSI editor and sent
for fabrication on a 40-pin MOSIS TINYCHIP with analog pads using 2 pm CMOS
n-well process. The ll-dimensional vector multiplier is the largest vector multiplier we
can obtain because of the pin limitation of the 40-pin on the M0818 TINYCHIP. Fig-
ure 4.3.1 shows its layout in the 40-pin MOSIS TINYCHIP pad frame. The 40-pin
MOSIS TINYCHIP pad frame contains 34 analog i/o pins, 3 pins for Vss, and 3 pins
for Vdd .

33 pins out of 34 pins are used and Table 4.3.l(a) summarizes the number of pins
used for inputs and outputs, where their notations are the as same as we used in Figure
4.1.1. Table 4.3.l(b) tabulates the pin-assignment of this chip. This chip contains three
different cells to test an up to ll-dimensional vector multiplier, with on-chip CMOS
operational ampliﬁer or with off-chip operational ampliﬁer, and a CMOS operational
ampliﬁer. For the purpose of testing each cell, some of the control voltages are set as

in Table 4.3.2.

xro

YIO

Y9

X8

Y8

x7

Y7

Vdd

urN
2_0 3

3_16

315

3_ 14

3_13

I

3_12

I

311

310

3_9

3_8
IIN

5_0

c
Yrr J"11 Y2

I I I

‘09 IO

VSS y6 V-

BCIOQ "CI

-28 _27 3_28 3_

55

C

vaf VSS Vb“, X2

I I I

ddN ‘ ‘
6_0 3_ 3

V+ Vdd GND C1

x1

I

I

(VIII

Y1

I

_0 3_1 3_2 3_3 4_0 3_4 3_5 3_6 3_

I

   

IIN
5_1

C2 Vss

Figure 4.3.1 The MD vector multiplier chip on a 40-pin Tinychip

Vdd
UN]
7 2_1
I
3_25 ‘2
I
3_24 Y2
I
3_23 x3
I
3_22 Y3
I
3_21 x4
I
3_20 Y4
I
3_19 "5
I
3_18 Y5
I
3_17 vbcop

56

Table 4.3.1 (a) The number of pins used and (b) pin-assignment of the ll-D chip
(8)

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

total

33

 

 

Pin# 1 2 3 4 5 6 7 8 9 10

 

Signal W3 X3 W2 12 Vdd W1 11 15 Vw VSS

 

Pin # 11 12 13 14 15 16 17 l8 19 20

 

Signal va W5 In W" Vdd 110 W10 19 W9 18

 

 

 

Pin # 21 22 23 24 25 26 27 28 29 30

 

81“ W 8 17 W7 16 V88 W6 * V- V+ Vdd

 

Pin # 31 32 33 34 35 36 37 38 39 40

 

 

 

 

 

 

 

Signal GND V‘.1 V, V

c, VSS Veg, W5 1:; W4 X4

 

 

 

 

 

 

 

(* means ”do not use")

Table 4.3.2 The tuning signal for each cell in the ll-D chip (units are in volts)

 

 

 

 

 

 

 

 

 

 

 

 

Test Cell vcl V“ W V' V“, v,
11-1) with on-chip op-amp ** ** ‘ * " output
ll-D with off-chip op-amp -5 -5 0W output " "
CMOS op-amp -5 -5 input input ' ** output

 

(* means "do not use” and “ means ”tune properly.")

 

57

4.4. Applications

Analog multipliers can be used to implement other circuits which are used for the
implementation of feedforward ANN s with learning such as programmable synapse
cells, a sigmoid-derivative cell, and analog voltage-adder cells.

4.4.1. Programmable Synapse Cells

Programmable analog synapse cells are implemented using standard analog MOS
multiplier cells and MOS capacitors. n -S stands for the n-dimensional synapse cell and
is realized with one u -D multiplier cell and n MOS capacitors. Figure 4.4.1 shows

the circuit description of 1-8 cell.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

MW... V
e o m
I w.“ '11 ' m 1 Mllltlpllef
VW2

Figure 4.4.1 A 1-D programmable synapse cell

A MOS capacitor is employed as an analog storage device. In Table 3.4.1, in
order to realize the 1 pF MOS capacitor, the transistor size is given by W/L= 40/20 for
nMOS transistor and WIL= 40/20 for pMOS transistor.

A large scale ANN may require a huge number of synapses and neurons. In this
case, a very high dimensional synapse cell should be available. A very high

 

58

dimensional synapse cell can be built with a very higher dimensional vector multiplier
which is discussed in section 4.1.

4.4.2. A Sigmoid-Derivative Cell

A derivative of the sigmoid function of a neuron may be required to implement a
learning rule for feedforward ANN. The sigmoid-derivative cell is the circuit which
computes the differentiation of a sigmoid function by the input voltage of the sigmoid
function, not by time.

Usually, a cell body is implemented using a double inverter or a simple opera-
tional ampliﬁer. When a neuron is implemented on silicon, it is impossible to obtain
the exact expression of the sigmoid function of a cell body. In order to obtain a
derivative of the sigmoid function of a cell body, it is assumed that a sigmoid function
of a cell body is represented by a hyperbolic tangent function such as

S (V;) = m tanh(kV,- ). (4.4.1)
Then its derivative can be represented

”(Vi)

4V: = k m sech 2(kV,.)

 

=km(l-—tanh2(kV,-))

(m2 ”520’s”

u
5 la-

n
a In-

(In -S(V.-))(m -(-S(Vi))- (4.4.2)

Equation (4.4.2) can be implemented using a 2-D analog multiplier by setting

x11=x12=m,x21=x§=0,w11=w%=S(V,~),and w12=w21 =0, where S(V,-) is

 

 

59

supplied from the output of a double inverter. Figure 4.4.2 shows the block diagram
of a sigmoid-derivative cell. In order to achieve the derivative of the sigmoid function
of a double inverter, this cell is simulated with the PSPICE input ﬁle in Appendix B.7
using the PSPICE circuit simulator. Figure 4.4.3 shows the sigmoid function of a dou-

ble inverter and its derivative.

x; -S(V) X? = 25v

- xi-xg-Ov
i JL JL 1

d S(V)
dV

 

 

 

 

2-D

 

 

 

l 2
Wz-Wz IO'V

Figure 4.4.2 A sigmoid-derivative cell

 

Figure 4.4.3 The input/output characteristics of the sigmoid-derivative cell

4.4.3. Analog Adder Cells

An analog voltage adder can be achieved using the analog multiplier cell by
ﬁxing Aw,- = 2.5. If w,- is supplied from the outside voltage source directly, two
w_shifters and two voltage followers will not be required any more. Figure 4.4.4
shows the block diagram of an n -D analog adder, called n ~DA. The operating range of

an I: -DA is given by

-2.5 s xii, V," s 2.5

with the ﬁxed wj1= 2.5v and wf =

X2 =X22 =0.0V

 

wi =w¥= 2.5v_,(_,i

l__.v0

 

 

1 2 2
W2 =w2 = 0.0V —¥—"

 

Figure 4.4.4 A voltage adder cell

 

 

 

xu- ix?
2

-A —>Vo

 

 

 

 

 

CHAPTER 5

FEEDFORWARD ANNS WITH LEARNING

A modiﬁed error back-propagation learning rule [43] is realized with circuits
which can be implemented using MOS transistors. The circuit is simulated using the
PSPICE circuit simulator for a prototype two-layer feedforward ANN s with learning
using standard CMOS VLSI/L81 technology.

Four feedforward ANNs with learning circuits are suggested by Dr. Salam.
These circuits have been implemented and simulated using the PSPICE circuit simu-
lator in order to prove this modiﬁed learning scheme is successfully implementable
using standard CMOS technology. Each circuit is described with block diagram and
its PSPICE simulation result is summarized. In order to save simulation time using
PSPICE, control voltage sources are used to model an ideal operational ampliﬁers
and level shifters in the circuits. A 1000 resistor is used between an ideal opera-
tional ampliﬁer and an integrating capacitor in order to ensure the integrating opera-

tion of the capacitor with the output of a multiplier.
5.1. A Modiﬁed Learning Rule for Feedforward ANNs

The squared error function and the governing static equation of each neuron are

deﬁned as follows [31]:

61

62

E - 1 n 2
P ‘ E E1 “pi ‘ypﬂ (5.1.1)
N
y, = 5,-(2wjtxt + 0i). (5.1.2)

where I” is the value of the desired target p for the j-th output component, ypj is the

output of neuron j in the output layer for the pattern p , y j is the output of neuron j,

S j is the sigmoid function of neuron j, N is the number of neurons in the previous

layer, W]; is the synapse weight from the output of neuron i in the previous layer to

the input of neuron j, x,- is the output of neuron i in the previous layer, and 9,- is the

threshold weight of neuron j .

A modiﬁed learning rule is obtained by removing the sigmoid derivative func-

tion term, de/duj, from the equation (2.2.14) in section 2.2.3, which is the error

back-propagation rule [31]. Then the modiﬁed learning rule [29, 43] is given for any

weight wj; as follows:

._ as,-_1 as, a
wji--n(auj) E319}; ﬁwjl

= 71 2 epj 2p; - ajini. (5.1.3)
19
where, if the neuron j is in the output layer, then
w=w-m (mm
and. if the neuron j is in any hidden layer, then
(131 " (5 1 5)
e - = -——- e w -. . .
m g dl'i,‘ pt *1
That is, when the neuron k is in the output layer, then
(5.1.6)

Wt,- = n 2 (tpt - 521:) in; ’ “jiWii
P

63

and when the neuron j is in any hidden layer, then
. d5] _ ..
2;: = 11 2‘. -: kj 2 (tpt ’ka) ypi - gim- (5.1.7)
A: duk p

where 57,, is the nondecreasing differentiable monotone sigmoid function of the neu-
ron k for the output layer. )7” is the output of the neuron k in the output layer, M
is the output of the neuron j in a hidden layer, and yp; is the output of the neuron i
in a previous layer. If the previous layer is the input layer, then ypi is the sigmoidal
output to the external input p-pattern xp; . g

It is also shown that the derivative terms iii/dd} in the equation (5.1.7) may be
removed without loss of stability and convergence of the update law (5.1.6)-(5.1.7).

5.2. Feedforward ANNs with Sequential Learning

Two different feedforward ANN s with sequential learning are suggested by Dr.
Salam and have been are implemented. Their PSPICE simulation results are dis-
cussed.

5.2.1. Feedforward ANNs with the Sequential Learning Circuit #1

The block diagram of the feedforward ANN s with the sequential learning circuit
#1 is shown in Figure 5.2.1 [29]. This circuit is a 2x2x1 feedforward ANN employ-
ing a version of the modiﬁed learning rule. That is, it has two input nodes, two neu-
rons in the hidden layer, and one neuron in the output layer. This circuit does not
include circuits for the update of the threshold parameters. Its circuit implementation
is described by the following equations:

ll

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

l-D l-D l-D l-D
v 0-—-'
X‘ 2-D >
21
2-D ~0—«>—>i
, 2.1) —O;/d
X. ,- . y,
r) I , ..
an; E: an 3 “i2
1) “—4
l-D l-D l-D 1-D
T T [T A .
X1 X2 ‘

Figure 5.2.1 The feedforward ANN with the sequential learning circuit #1

RCﬁ’Lll = kl [ (‘ll ' 711) 2111] " W11 (5.2.1i)
Raf-12 = 1‘1 1 (‘11 ’ 5’10112] ’ l"712 (SJ-Iii)
Rcﬁll = kl 79-11 [(111 ‘ 71015111" 211 (5.2-2i)
Rcrllz = kl W11 [(tll — 5’11) x12] ' 19.12 (5-2-2ii)
RCEZI = kl W12 [ (I11 " 5’30111] " 19.21 (5-2-2ﬁi)
R5222 = k 1 W12 [ (t 11 " 711) 112] ’ 1222- (5-2-2iV)

The circuit implementation of the governing equation results in

Zpl = Sn (k2(2. ﬂljxpj » (5.2.31)
.I

52 = Sn(k2(z 1221*ij (5.2.3ii)
J

65
Y} = Sn(k2(2 W 1,1,,- )). (5.2.3iii)
I

where S,, (.) is a sigmoid function of a neuron and It:” is a n-dimensional vector multi-

plier constant. In the above equations, 1”- and xpj are deﬁned as follows:

In 5 EN " 2-5
xpj EXP]. -‘ 2.5,

where Zpi is an actual output of the hidden layer, 17,, is an actual output of the output
layer, and ij is an actual input of the input layer for the pattern p . These same

notations are used throughout this paper.

The full circuit is simulated using the PSPICE circuit simulator. Four learning
tasks using four distinct input-output patterns have been executed separately as fol-

lows:

1. Initialize the dynamic feedforward ANN circuit with the same initial condition,
(311 312 121 3322 W11 W12) = ( —0.5 0.6 -0.5 0.5 0.7 0.5 ) for all the learning
tasks.

2. Each input data and its desired target (i.e., the input-output pattern) are applied
and the PSPICE transient analysis is performed. Then measure the steady state

weight values and the output error, I - Y, for each pattern pair.

3. The PSPICE dc analysis is performed by setting the weights to the steady state
weight values. Then we measure the error 1 - 5" again.

The results are summarized in Table 5.2.1 and the PSPICE input ﬁles are shown
in Appendix B.8-B.9. The results show that this ANN circuit succeeds to learn each
applied input-output pattern. However, when the four distinct input-output patterns
are applied sequentially, we found that the overall ANN circuit did not retain all pre-

vious patterns as the weights are updated to learn a new pattern. The PSPICE

66

simulation for the sequential learning circuit is executed as follows: we initialize the
ANN circuit for an input-output pattern which it would learn successfully by con-
verging to a set of (equilibrium) weights. We then use the attained (equilibrium)
weights as initial weights for the next input-output pattern. Continuing this process,
until we employ all the input-output patterns and reapply the ﬁrst pattern again. In
this process, we found that the ANN circuit would learn each pattern that is presently
applied, but it does not necessarily retain other input-output patterns that it has previ-

ously learned.

The PSPICE transient analysis is executed for the ANN circuit with the PSPICE
input ﬁle in Appendix 3.10 by applying pulse input waveforms for the external
inputs, X 1, X 2, and the desired target, 1:, i.e., continuous inputs and target pulse sig-
nal. Figme 5.2.2 shows the actual output curve corresponding to an applied input
vector and its desired target. The convergent weights are the as same as the results in

Table 5.2.1.

67

Table 5.2.1 The PSPICE transient analysis results of the feedforward ANN with the sequen-
tial learning circuit #1. The initial and steady state weights are tabulated for each pattern (units
are in volts).

 

 

 

 

 

X 1 0.5 0.5 4.5 4.5

X 2 0.5 4.5 0.5 4.5

t 1.0 4.5 4.5 1.0

yr 1.81 5.0 5.0 1.81

’t - 7 -0.81 -0.5 -0.5 -0.81
weights init. s.s. irlit. s.s. init. s.s. init. s.s.

 

In 0.5 0.053 0.5 0.02 0.5 0.019 0.5 0.052
1,2 0.6 0.053 0.6 0.019 0.6 0.02 0.6 0.052
22, 0.5 0.053 0.5 0.02 0.5 0.019 0.5 0.052
222 0.5 0.053 0.5 0.019 0.5 0.02 0.5 0.052
w“ 0.7 0.276 0.7 0.168 0.7 0.168 0.7 0.276
W12 0.5 0.276 0.5 0.168 0.5 0.168 0.5 0.276

 

 

 

 

 

 

 

 

 

 

 

s.s means steady state and init. means initial value.

 

Figure 5.2.2 The PSPICE transient analysis of the circuit in Figure 5.2.1,
where v(301) and v(302) are the input vector, v(303) is its desired
target, and v(13) is the actual output of the circuit.

68

5.2.2. Feedforward ANNs with the Sequential Learning Circuit #2

The block diagram of the feedforward ANN s with the sequential learning circuit
#2 is shown in Figure 5.2.3. This circuit is designed with a threshold weight learning

circuit for each neuron and a nonlinearity for each weight, in addition to the feedfor-
ward ANNs with with sequential learning circuit #1.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

1 X1 X2 ‘ - < t
j i L ll _.1 -1
l-D l-D 1-1) 1-1) ID ID 1
V 1*“J 1'
if? :1: 2; “159529
X1 £1 211‘.“ _ - W11- 61
3'9 Y1
1'] ' l_.t 3-1) .h—o—ﬁH-y-
923-1) M L
X2 y: W;
gain as
J 1
1-D 1-1) l-D ID 11)
T T T “—1
1 x1 x2 I < t

Figure 5.2.3 The feedforward ANN with the sequential learning circuit #2

The circuit implementation is described in the following equations [43]:

RC T11 = k1 (‘ 11471) 211 ‘ T11 (5.2.40

RC 7‘12 = k, (tn-71) 2,, - 1"}, (5.2.4ii)

RC :11 = k1 W11 (ill-ilkll " Ill (5.2.5i)

69

RC :1. = 1:1 Wu (tn-fins: - In - (5.2.511)

RC :21 = 1‘1 W12 (’ 11*Y11x11 " :21 (5-2-5iii)

RC ['22 = It, 1712 (tn-171p: ,2 - 122 (5.2.5iv)
and

Wu = S (7})- ).

The modiﬁed threshold weights are given as follows [43]:

91 + RC Q; = 1017110,, - 17,.)1 (5.2.61)

92 + RC Q; = 1:1 W12 (tp — 17;.) 1 (5.2.6ii)

6', + RC '6, = k, (t, — 17,) 1 (5.2.6iii)
and

e,- = S(0,-),

where S (.) represents a nonlinearity for 9,- and wij, Ti} is the modiﬁed synaptic
weight before it passes a nonlinearity, and 0,- is the modiﬁed threshold weight before

it passes a nonlinearity.

The circuit implementation of the governing equations now results in

g“ = s,,( 13(2 11ij + 9_,)) (5.2.71)
i

sz '5 Sn( “(2." ﬂzjxpj + _9_7)) , (5.2.7ii)
I

)7}? = Sn( k3(2 WUIPJ. + 51)). . (5.2.7iii)
I

This circuit is simulated using the PSPICE circuit simulator. The learning tasks

are executed sequentially in the following steps:

70

1. Initialize the ANN circuit with the initial condition which is at the top of each
table.

2. The ﬁrst input-output pattern is applied and the PSPICE transient analysis is

executed. The measure the steady state weight values.

3. Use these steady state weight values as the initial condition and run the PSPICE

transient analysis for the next input-output pattern. Then measure the steady

state weight values.
4. GotoStepBuntilallthelearningtasksaredone.

5. After completing the learning tasks for all the input-output patterns, the PSPICE
dc analysis is executed by setting the weights to the steady state weight values

for each learning task. Then measure the output error 1: - )7.

Table 5.2.2 and Table 5.2.3 summarize the results of the PSPICE simulations.
One of the PSPICE input ﬁles for Table 5.2.3 is shown in Appendix 3.11. The

simulation results show that this circuit does not learn all the desired targets simul-

taneously.

For example, let’s consider Table 5.2.2. Table 5.2.2(a) contains the convergent
connection weights of the feedforward ANN s with the sequential learning after exe-
cuting the PSPICE transient analysis. With these weights, the PSPICE dc analysis is
executed and the results are summarized in Table 5.2.2(b). Note that in Table
5.2.2(b). column #l,5 means that the dc analysis is performed using the data of
column #1 and the data of column #5 in Table 5.2.2(a), separately, and their results

are summarized in the same column since both results are the same.

In column #1,5, the pattern (X 1 X 2 t)=(4.5 4.5 0.5) is learned with an error
equal to -0.0002 and the pattern (X 1 X 2 t)=(l.0 1.0 0.5) seems to be learned with an
error equal to 0.5., but the patterns (X 1 X 2 1:)=(1.0 4.5 4.5) and (4.5 1.0 4.5) fail to
be learned. In column #3,4, the patterns (X 1 X 2 't)=(l.0 4.5 4.5) and (4.5 1.0 4.5) are

71

learned with an error equal to -0.001, the pattern (X1X2 t)=(1.0 1.0 0.5) seems to be
learned with an error equal to 0.5, however the pattern (X 1 X 2 1:)=(4.5 4.5 0.5) fails
to be learned.

72

Table 5.2.2 (a) The PSPICE transient analysis results of the feedforward ANN with the sequaltial learning circuit

112(on ampliﬁcation) (units areinvolta).‘tia atargetandy'isanaemaloumt'l'heinitialconditionsare givenby
(111(0) 111(0) 111(0) Tam) 111(0) 711(0) 91(0) 23(0) 01(0) ) a: ( 0.5 -0.5 -05 0.5 0.5 0.5 -0.5 -0.5 0.5 )
(311(0) 311(0) 331(0) 1111(0) 1711(0) 1711(0) 21(0) 22(0) 51(0) ) = ( 1.0 -l.0 ~l.0 1.0 1.0 1.0 -l.0 -l.0 1.0).

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Column #1 Column #2 Column #3 Column #4 Column #5
x 1 4.5 1.0 1.0 4.5 4.5
x; 4.5 1.0 4.5 1.0 4.5
t 0.5 0.5 4.5 4.5 0.5
y 0.5002 0.4999 4.501 4.501 0.5002
t-y" -0.002 0.001 -0.001 -0.001 -0.002
In 4.210E-04 45118-04 29018-04 3.8l7E-04 45655-04
In 4.210E-04 -2.51 113-04 3.817E-04 -2.901E-04 4.56513-04
In 4.210E-04 -2.51 18-04 29015-04 3.817E-04 4.565E-04
In 4.210E-04 45118-04 3.817E-04 -2.901E-04 4.565E-04
Tn 3.410303 3.41 113-03 3.410E-03 3.4105-03 3.410E-03
T1; 3.410503 3.41 1E-03 3.4105-03 3.410E-03 3.410E-03
Q, 421013-04 33048-04 3.8 1715-04 3.817E-0-4 4.565E-04
Q; 421015-04 330413-04 3.817E-04 3.817E-04 4.565E-04
0, -3 26915-03 273915-03 27238-03 -2.723E-03 -3.77 15-03
g u -l -l -l -l -l
g 12 -l -l -l -1 -l
:21 -l -l -l -l -l
222 ~l -l -l -l -l
W 11 01393 0.1396 0.1663 0.1663 -0. 1393
W 12 «0.1393 0.1396 0.1663 -0. 1663 -0.l393
Q, -l -l -l -l -1
Q; -1 -l -l -l -l
51 -l -l -l -1 -l

 

 

 

 

 

 

 

 

Table 5.2.2(b) The PSPICE DC malyaia results of the feedforward ANN circuit with the convergent weights in
Table5.2.2(a).‘t'natarget,y, is anoutputofthehiddallaya.andy'iaqulutofdleoutputlayer.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Column 11.5 Column :2 Column 13.4 7
x, x; 7: y, y; y 1: - I 11 1; Y 1 " Y 21 22 Y " " Y
= E =—
10 1.0 05 5.0 5.0 0.0 0.5 5.0 5.0 0.4995 0.005 5.0 5.0 0.0 0.5
1.0 4.5 4.5 0.0 0.0 0.5002 3.998. 0.0 0.0 0.0 4.5 0.0 0.0 4.50l 0.001
4.5 1.0 4.5 0.0 0.0 0.5002 3.998 0.0 0.0 0.0 4.5 0.0 0.0 J 4.501 0.001
45 4.5 0.5 0.0 0.0 0.5002 0.002 0.0 0.0 0.0 0.5 0.0 0.01 4.501 4.001

 

73

Table 5.2.30) The PSPICE tranaialt analysis results of the feedforward ANN with the sequential learning circuit
92(wlo ampliﬁcatim)(unitareinvolta).‘tiaatargetandy'iaarlacmalouqalt'l'heinitialoonditionsaregiven

by

(1,,(0)1,;(0)zu(0)za(0)1'u(0)131(0) 91(0) ya) 61(0) ) a ( 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 )
(3,,(0) 33(0) 311(0) 33(0) Wn(0) 17,,(0) 9(0) 9(0) 6,(0) ) =- ( 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0).

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Column #1 Column #2 Column #3 Column #4 Column #5

x, 4.5 1.0 1.0 4.5 4.5

I; 4.5 1.0 4.5 1.0 4.5

‘t 0.5 0.5 4.5 4.5 0.5

y 05099 0.4928 451 451 05099
t—y 0.099 0.0072 0.01 0.01 0.099
1,, 45071104 25071104 2.851304 3.284E-04 45351-204
1:12 4.607504 4.507504 3 .75 lE-04 ~2.906E04 4.535E-04
13 4.607504 -2.507E-04 0.85504 3.284E04 4.535504
In 4.607304 -2.507E04 3.751E04 0.906504 4.535504
Tu 3.410503 3.411503 3.410E03 3.410503 3.410503
Tu 3.4105-03 3.41 1503 3.410503 3.410E03 3.410503
9'1 4.607E-04 3.3001504 3.751504 3.824504 4.535E04
2; 4.607504 3 300504 3.751504 3.824E04 4.5351504
01 -3.804E-03 2.748503 0.723503 0.723503 0.739503
£11 -1 -1 -l -1 -1

_,, -l -1 -1 -1 -1

33,, -1 -1 -1 -1 -1

3,, -1 -1 -1 -1 -1

Wu 0.1396 0.1394 0.1663 0.1663 0.1396

Wu 0.1396 0.1394 0.1663 0.1663 0.1396

g, -1 -1 -1 -1 -1

9, -1 -1 -1 -1 -1

6, -1 -l -1 -1 -1

 

 

 

 

 

 

 

Table 513(b) The PSPICE DC analysis results of the feedforward ANN circuit with the convagent weights in
Table5.23(a).tiaatarget.z, iaanoutputofthehiddenlaya.andiiamouqaltoftheoutwtlayer.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Column 415 Column 42.6 Column 43.4.7.8 7
x; I: T l1 l; y 6". 11 11 Y i ‘—y z! 22 ’- Thy
j=== ==_ r—n—“l
1.0 10 05 5.0 5.0 0.0 05 5.0 5.0 0.4931 0.0069 5.0 5.0 J 0.0 05 I
1.0 45 45 0.0 0.0 05107 3.9893 00 0.0 0.0 45 0.0 0.0 i 4.511 I 00W
45 10 45 0.0 0.0 05107 3.9893 0.0 0.0 0.0 45 0.0 0.0 J' 4511 I 0.011]
45 45 05 0.0 0.0 05107 0.0107 0.0 0.0 0.0 05 0.0 Md 45117 4.01ﬂ

 

74

5.3. Feedforward ANN s with the Simultaneous Learning

From the above simulation results, we may conclude that feedforward ANN s
with the sequential learning do not (easily) completely learn all the desired patterns.
In this section, feedforward ANN s with the simultaneous learning are considered by
expanding the circuit for feedforward ANNs with the sequential learning, i.e., by
employing a number of feedforward ANNs with the sequential learning circuit equal
to the number of patterns.

Two different feedforward ANN with simultaneous learning circuits are sug-
gested by Dr. Salam and have been implemented. These two implementations are the
same except that there is an ampliﬁcation unit before each capacitor (or after a multi-
plier). Accordingly, when these circuits are represented as model equations, they have

the same form except this ampliﬁcation constant, a.

Prototype 2x2xl feedforward ANN s with the simultaneous learning are imple-
mented when the number of pattern is four, i.e., p = 4. Figure 5.3.1 depicts the
feedforward ANN with simultaneous learning circuit with an ampliﬁcation unit
(a = 50) and without one (a = l), where the ampliﬁcation unit is represented by the

rectangular box. The circuit implementation is described by the following equations:

RC 77-11 = 01 *4 0114-1 ‘21-'72 t31"173 '41")7 41 [X11221 13124117 " T11 (5.3.li)
RC 1712 = G In [tn-71 t21472 ‘ 31"7 3 ‘41-’74] L212 X22 232 X421T " T12 (5-3-1ii)
RC :11 = 01 *4 W11 [‘11—’71 ‘21“‘72 ‘31-’73 '41‘174] [X11121 X31141JT " In (5.3.20
RC _1_' 12 = 01 144 1711 [I 11-71 t21")72 ‘31’73 ‘41‘741 [x 12 122 x 32 144217 " 1'12 (5-3-21i)
RC :21 = 01 k4 W12 [tn-171 ‘21“7-2 ‘31‘Y-3 ‘41’741 [11112113114111 ' I21(5.3.21ii)

RC :22 = 01 k4 W12 [111-171 ‘21’172 ‘31’73 ‘41-’74] [112 x 22 Jr32 x47JT *- 122 (5.3.2iv)

75

and
Wij = S(Tt'j)'

The modiﬁed threshold weights are given as follows:

91 + RC 9, = 01 [:4 Wu 2‘, (1, — 1;) 1 (5.3.31)
P

92 + RC 92 = 01 [:4 ran 2 (1,, - 17,) 1 (5.3.311)
P

61 + RC 0, = 01 1:4 2 (1p - 17p) 1 (5.3.3111)

P
and
91 = 5(91).

where S(.) represents a nonlinearity for wii and 6;, Ti} is the modiﬁed synaptic
weight, and 6,- is the modiﬁed threshold weight, before it passes through the non-
linearity.

The circuit implementation of the governing equation now results in

Zpl = Sn( k3<2 Kljxpj + 21)) (5.3.41)
1

52 = SJ k3(2 Ezjxpj + 9.2)) (5.3.4ii)
j

y; = Su< k3<2 W1 1‘ij 4‘ E»- (5.3.4iii)
J

The feedforward ANN with simultaneous learning circuit with an ampliﬁcation
unit (a = 50) is simulated using the PSPICE circuit simulator as follows:
1. Initialize the ANN circuit with the initial condition which is at the tap of each

table.

76

2. All the input-output patterns are applied and the PSPICE transient analysis is
performed.

3. Measure the steady state weight values and the output errors, namely, 1: -— 5".

The results are summarized in Table 5.3.1. Table 5.3.1(a) and Table 5.3.1(c)
conclude that the feedforward ANN with simultaneous . learning circuit is simulated
with the same initial condition but with different logic 0 value. And similarly for
Table 5.3.1(b) and Table 5.3.1(d). As shown in Table 5.3.1, all four input—output

patterns are successfully learned.

Next, the feedforward ANN with simultaneous learning circuit without an
ampliﬁcation unit (a = l) is simulated in the same manner. Four different initial con-
dition sets are given for the simulation of this circuit. Two of them are the same as
in Table 5.3.1 and the other two sets are obtained from dividing the initial conditions
of Table 5.3.1 by the ampliﬁcation factor of the ﬁrst feedforward ANN with simul-
taneous learning circuit, i.e., 50. Their results are summarized in Table 5.3.2 and
5.3.3. However, depending on the initial conditions, this circuit may not always learn
all the desired targets but learn some desired targets. For example in Table 5.3.2(a)
and Table 5.3.2(c), Table 5.3.2(c) shows that this circuit learns all the desired targets
but not in Table 35.3.2(a). The difference of these two simulations is the logic 0
value, i.e., Table 5.3.2(a) is the result of using 1.0V as the logic 0 value and Table
5.3.2(c) is the result of using 0.5V as the logic 0 value.

The PSPICE input ﬁles for Table 5.3.1 and Table 5.3.2 are shown in Appendix
3.12 and Appendix B.13, respectively.

77

A—H

3711
3721

731

 

01. or #4 Pattern §41

XpZ w12

p—D p-D p X2 p-D

P Yp
t

 

P

Figure 5.3.1 The feedforward ANN with the simultaneous leaming circuit with
threshold weight circuits, nonlinearity circuits, and ampliﬁcation units

78

Table 5.3.1 The PSPICE transient analysis results of the feedforward ANN with the simultaneous learning when
a=50(unitsareinvolts).‘risatargetandy'isanactualouqsut.
(a)Logic 1 =45 andlogic0= 1.0
(111(0) [1(0) [11(0) 13(0) 711(0) 711(0) 91(0) 92(0) 51(0) ) = ( 0.5 -0.5 -0.'> 05 0.5 0.5 -0.5 -0.5 0.5 )
( 311(0) 312(0)!21(0) 33(0) 1711(0) 1712(0) §1(0) 21(0) 51(0) ) = ( 1.0 -l.0 -l.0 1.0 1.0 1.0 -l.0 -l.0 1.0)

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

‘t 1.0 4.5 4.5 1.0
y 1 .0 4.498 4.499 1 .005
n-y 0.0 0.002 0.001 0.005
111 It: In In 2 ll 7 12 21 92 1
8.440504 —1 .287503 3.412503 3.412503 3.411503 3.410503 6.068505 1.012503 3.819503
£11 £12 221 ‘12: W11 W12 21 92 61
-1.0 -1.0 0.1661 0.1628 0.01825 0.314 ~1.0 -1.0 -1.0
(1)) Logic 1 = 4.5 and logic 0 =1.0
( 211(0) 112(0) [21(0) [22(0) 711(0) 712(012103) 22(0) 61(0) ) = ( -0-5 05 0-5 43-5 -0-5 -05 70-5 -0-5 05 )
(311(0) 23(0) 23(0) 13(0) 1711(0) {9711(0) 91(0) 22(0) 61(0) ) = ( -1.0 1.0 1.0 -1.0 -1.0 -1.0 -1.0 -1.0 1.0)
‘l 1.0 4.5 4.5 1.0
y 1.0 4.499 4.499 1.004
1-7' 0.0 0.001 0.001 0.004
In In 12.1 In 11 i 12 21 22 51
1.6765-04 0.701503 3.4125-03 3.4125-03 3.4115-03 3.410503 0.141505 +1552503 -5.8255-03
£11 £12 £21 1’32 “’11 W12 21 22 3l
-1.0 -1.0 0.1646 0.1643 0.01826 0.314 -1.0 -1.0 -1.0
(6)1..ogic1345 “103100205
( 111(0) 112(0) I3(0) 13(0) 111(0) 132(0) 91(0) 91(0) 01(0) ) = ( 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 )
( 211(0) 23(0) 23(0) 13(0) 9711(0) W130) 91(0) gm) 61(0) ) B ( 1.0 ~1.0 -1.0 1.0 1.0 1.0 -1.0 -1.0 1.0)
‘t 0.5 45 4.5 0.5
9' 0.5003 4.499 4.498 0.5041
z-y 0.0003 0.001 0.002 0.0041
=
=1. 1.. 1. 1. =r—.==E=ﬁ 1. g. 21%
‘ 3.412503 3.412503 -1.053503 -2.428503 3.410503 3.411503 2.161503 -2.443504 -5.490503
211 2,12 :21 32:: W11 W12 21 92 31
0.1714 0.1733 -1.0 -1.0 0.3046 0.02738 -1.0 -1.0 -1.0
(d)1.ogicl=4.5mdlogic0=0.5 '
( 111(0) 1:3(0) 23(0) 13(0) 111(0) 113(0) 91(0) 93(0) 01(0) ) I= ( 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5)
(211(0)!12(0)y_11(0)_u_13(0) 17,,(0) 17,,(0) 9(0) 9(0) 6,(0) ) .. ( -1.0 1.0 1.0 -1.0 -1.0 .1.0 -1.0 -l.0 1.0)
1 0.5 4.5 4.5 0.5
Y 0.5197 4.498 4.499 0.4818
z-y 0.0197 0.002 0.001 0.0182
In In In In 11 12 21 92 1
3.410543 3.4135-03 3.414503 3.336503 3.411503 3.41 1503 7.826505 7.7965-05 3.410503
=3. 2.. Fer—TE .., 9. g. a.
0.2915 0.4228 0.6703 0.9676 0.02708 0.02697 ~1.0 -1.0 0.1826

 

 

 

79

Table 5.3.2 The PSPICE Innsimt mnlysis results of the feedforward ANN with the. simultaneous leaning when
u=l(1nﬁtsareinvolts).1isamgetmdy'ismacmalontmt.
(1)1.ogic 1 =45 u1dlogic0=l.0
(111(0) [11(0) 13(0) 13(0) 71((0)Tu(0) 91(0) 92(0) 61(0) ) = ( 0.5 ~05 -0.5 0.5 0.5 0.5 -0.5 -0.5 0.5 )
( 311(0) 313(0) 3;,(0) 23(0) ﬁnal) 1740) §1(0) 22(0) 61(0) ) = ( 1.0 -1.0 -1.0 1.0 1.0 1.0 -1.0 ~1.0 1.0)

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

‘t 1.0 4.5 45 1.0
y 0.9617 3.332 3.332 3.332
1-y 0.0383 1.168 1.168 .2332
I11 :12 In In 1- 11 i 12 Q1 Q2 61
6.645304 -5.698E04 1.380503 1.380E03 3.411E03 3.411503 1.143E05 -2.820E-05 3.410E-03
£11 £12 221 22: W11 W12 21 22 S1
o1.0 -l.0 -1.0 -1.0 3.955803 0.753503 -1.0 -1.0 0.2586
(b)Logic 1 =45 mdlogic0= 1.0
( 111(0) 112(0) 13(0) 13(0) 111(0) T130) 0(0) 91(0) 01(0) ) = ( 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 )
( 211(0) 212(0) 33(0) 33(0) 17;,(0) 1711(0) 91(0) 9(0) 61(0) ) 8 ( -1.0 1.0 1.0 -1.0 -1.0 -1.0 -1.0 ~1.0 1.0)
1 1.0 4.5 4.5 1.0
9’ 0.9617 3.337 3.325 3.334
t-y' 0.0383 1.163 1.175 .2334
I11 112 In In 1 11 i 12 9,1 92 81
3.358504 3.288503 4.532503 4.508503 3.411303 3.411503 0.659505 4.982505 3.410503
=== ._._L
211 £12 221 22: W11 W12 91 22 31
0.9503 0.9894 -1 .0 -1 .0 0.02359 0.01954 ~1.0 -1.0 0.2586
(c)Logic 1 =45 mdlogic0=05
( 111(0) 112(011'2100 122(0) 711(0) 712(0) 91(0) 92(0) 51(0) ) = ( 0-5 ‘05 '05 05 0-5 05 '05 ~05 0-5 )
( 211(0) 212(0) 23(0) 23(0) 1711(0) 17,30) 21(0) 9(0) 61(0) ) = ( 1.0 -1.0 -1.0 1.0 1.0 1.0 -1.0 -1.0 1.0)
‘t 0.5 4.5 4.5 0.5
Y 0.5126 4.434 4.434 0.677
z-y 0.0126 0.066 0.066 0.177
I11 I12 121 In a 11 i 12 91 93 81
3.412503 3.412503 2.516304 3.047304 3.410503 3.41 1303 1.215503 9.873505 4.714803
‘ _=_ =1 jun—— _=
211 212 221 222 ‘”11 W12 9.1 .93 81
0.168 0.1679 -1.0 -1.0 0.3032 0.0253 -1.0 -1.0 -1.0
(d)Logic1:-45mdlogi00=0.5 ‘
( 111(0) 1,1(0) 13(0) 13(0) 711(0) 111(0) 91(0) 93(0) 0K0) ) = ( 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5 0.5)
(1111(0)gn(0)_11_ru(0)!n(0) 17,10) 17,,(0) 9(0) _e_,(0) 6,(0) ) = ( -1.0 1.0 1.0 -1.0 -1.0 -1.0 -1.0 -1.0 1.0)
‘t 0.5 4.5 4.5 0.5
1 0.4652 3.157 3.157 3.180
1-y 0.0348 1343 1343 .2680
I11 I12 111 In a 11 i 12 21 92 :1
1.477503 1.477503 1.411803 1.411503 3.411303 3.411E03 4.6433-07 1.176506 3.410303
211 1'1: 221 £22 “’11 712 9.1 22 1
-11) -1.0 -l.0 -1.0 -7.957E-03 3.612503 -1.0 -1.0 0.2835

 

 

 

 

 

 

 

 

 

 

 

80

Table 5.3.3 The PSPICE transient analysis results of the feedforwud ANN with simultaneous leaning when a = l
(mitsminvolts).sissmgetmdy’ismscunloutpm.
(s)Logic 1 =45 sndlogic0=l.0
(111(0)111(°)Iﬂ(0)122(0)111(0)T11(0) 91(0) 92(0) 01(0) ) s ( 0.01 -0.01 -0.01 0.01 0.01 0.01 -0.01 -0.01 0.01 )
( 311(0)!u(0)221(0)gn(0)W110» 1711(0) 9(0) 92(0) 640) ) =( 1.0 .10 ~10 1.0 1.0 1.0 -l.0 -l.0 1.0)

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

1 1.0 4.5 4.5 1.0
y 0.9617 3.331 3.330 3.335
1-y' 0.0383 1.169 1.17 4.335
In In In In 1 11 1 12 Q1 93 61
14645.04 24633—04 5.757E-04 5.758E-04 3.411E-03 3.41 15—03 6.021506 -1 .175E-05 3.4105-03
£11 3212 221 3222 “'11 W12 21 92 a1
-1.0 ~1.0 -1.0 -1.0 43375-03 4.066503 -1.0 -1.0 0.2586
(b) Logic 1 = 45 and logic 0 . 1.0
( [11(0) I;1(0) 13(0) 13(0) 111(0) Tum) 91(0) 92(0) 01(0) ) = ( -0.01 0.01 0.01 -0.01 -0.01 -0.01 0.01 -0.01 0.01)
( 211(0) 211(0) 231(0) 2&0) an) W110) 21(0) 22(0) 6(0) ) = ( -1.0 1.0 1.0 -1.0 -1.0 -1.0 -1.0 -1.0 1.0)
1 1.0 4.5 45 1.0
y 1 .008 4.429 4.429 1.184
s-y' -0.(X)8 0.071 0.07 1 -0.184
== ———-——————————.—__——
I11 I12 In In 1 11 i 12 21 92 51
3.412E-03 3.4125—03 13053-04 13553-04 3.410503 3.4115-03 1.017503 16483-05 4.0875403
=
211 £12 221 22: W11 ‘71: Q1 91 81
0.1607 0.1610 -1.0 -1.0 -0.3122 -0.01631 -1.0 -1.0 -1.0
(6)1631: 1 =45 undlogic0=05
( 111(0) 112(0) [11(0) 13(0) 111(0) Tu(0) 21(0) 21(0) 03(0) ) 8 ( 0.01 -0.01 -0.01 0.01 0.01 0.01 -0.01 -0.01 0.01 )
(gum) gum) gum) 3,,(0) wum) 15,,(0) 91(0) 9(0) 3(0) ) = ( 1.0 -1.0 -1.0 10 1.0 1.0 -1.0 -1.0 1.0)
‘t 0.5 4.5 4.5 0.5
y 0.5126 4.435 4.434 0.6764
z-y 0.0126 0.065 0.066 01164
= _ 1— == —————1
I11 112 In In 1 11 a 13 9,1 9: an
14125-03 3.4123-03 2.674E-04 28255-04 3.410503 3.4115-03 12065-03 5.581505 4.681503
= $
211 2:11 211 Pin Wu Wu 9.1 9.1 1
0.1672 0.1687 -1.0 -1.0 43.3133 0.02531 -1.0 -1.0 -1.0
(musicnumbgscoaos '
(1.,(0)z.,(0)zn(0)z,(0) 1',,(0)1',,(0)9_.(0) 91(0)6.(0) ). ( 0.01 0.01 0.01 0.01 .001 0.01 0.01 0.01 0.01)
( 211(0) 313(0) 331(0) 23(0) Wuw) 0711(0) 21(0) 23(0) 51(0) ) 8 ( ~1.0 1.0 1.0 -1.0 ~1.0 ~1.0 ~1.0 -1.0 1.0)
1 0.5 45 4.5 0.5
Y 0.4653 3.17 3.133 3.191
s-y 0.0347 133 1.367 .2691
==
111 I12 [21 In 11 12 21 9,: 91
3.410E-03 3.4105—03 -3 3923—04 4.647504 3.41 15-03 3.41 13-03 43658-05 65678-06 3.4108413
‘ g; — _— f
211 £12 3,21 22: W11 W12 21 9,2 1
0.2767 0.2033 -1.0 -1.0 0.01821 2.271503 -1.0 -1.0 -0.2844

 

 

 

 

 

 

 

 

 

 

 

81

5.4. Discussion

The modiﬁed learning rule [43] is employed in order to implement feedforward
ANNs with learning in electronic VLSI/L81 circuits. Two feedforward ANNs with
sequential learning circuits and two feedforward ANN s with simultaneous learning
circuits have been realized and simulated using the PSPICE circuits simulator. The
PSPICE simulation results show that the former circuits may not learn all the input-
output patterns simultaneously but the latter ones can easily learn all the input-output

patterns simultaneously.

Consider the total squared error function and the squared error function for the

desired target p as follows:

1
E=7§g, a“)

For feedforward ANN s with simultaneous learning circuit, all the weights con-
verge to an equilibrium point in order to achieve a minimum value of the total
squared error function E . On the Other hand, feedforward ANN s with sequential
learning force all the weights to converge to an equilibrium point where the squared
error function for the desired target p, E, , has a minimum value. These two equili-
brium points are not necessarily the same. That is, the ﬁnal weights of the feedfor-
ward ANN with simultaneous learning circuit may not be a minimum for the feedfor-
ward ANN with sequential learning circuit. Therefore, feedforward AN Ns with
sequential learning circuit may forget or ignore the previous learned weights. This
demonstrates the need for feedforward ANN with simultaneous learning circuits in
the analog implementation of feedforward ANN s with learning. However, the feed-
forward AN Ns with simultaneous learning circuit take larger chip area than the

82

feedforward ANN s with sequential learning circuit.

CHAPTER 6

A MODULAR DESIGN OF
FEEDFORWARD ANNS WITH LEARNING

A module chip is designed in order to build a large scale feedforward AN N s
with learning. The implementation of a module chip is initiated by designing two
different module chips for feedforward ANN s with learning: One is for the hidden
layer and the other is for the output layer. That is because the learning circuitry of

the output layer has sigmoid-derivative circuits but the hidden layer does not.

Dr. Salam suggests one module chip for the use of any layer since the design of
two different module chips has required more design effort than that of one module
chip. Then the module chips are connected vertically in order to increase the number

of neurons in any layer and horizontally for additional layers.
6.1. A Prototype 2x2x2 Feedforward ANNs with Learning

In the modiﬁed learning rule, the removal of sigmoid-derivative terms in the
output layer does not affect the stability of the system. Dr. Salam suggests that
another modiﬁcation be taken from the modiﬁed learning rule by removing sigmoid-
derivative terms in the output layer. A prototype 2x2x2 feedforward ANN with

learning has been implemented without sigmoidderivative circuits in the output layer

83

34

and simulated using the PSPICE circuit simulator in order to prove that this modiﬁed

learning scheme is successftu implementable.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

0‘2
2-D “’11
“’21
X
l
X r
'6
2_1) ‘12
MT]. 22

 

 

 

 

 

 

 

 

m2

Figure 6.1.1 A 2x2x2 feedforward ANN with the sequential learning circuit #2
with threshold weight circuits and nonlinearity circu1ts

The feedforward ANN with the sequential learning circuit #2 in section 5.2.2 is
employed to implement a prototype 2x2x2 feedforward ANN with learning, which is
shown in Figure 6.1.1. There are two inputs, two outputs, and two neurons in the
hidden layer. Since there are two inputs and two desired targets, all sixteen different
learning tasks are available for the simulation of this ANN circuit using the PSPICE
circuit simulator. The PSPICE simulation is performed along the following pro-

cedures :
1. Initialize the circuit with an initial condition, which is the same for all the tasks.

2. Each input-output pattern is applied and the PSPICE transient analysis is exe-

cuted. Measure the steady state weight values and the output error, namely,

85
r-Y.

3. The PSPICE dc analysis is performed by setting the weights to the steady state

weight values. Measure the output of each layer and the output error I - Y.

One of the PSPICE input ﬁles for Table 6.1. 1(a) is shown in Appendix B. 14 and
one of the PSPICE input ﬁles for Table 6.1.l(c) is shown in Appendix B.15. The
results are summarized in Table 6.1.1. This simulation result shows that each task
has learned its own input-output pattern successfully although this ANN circuit does
not include sigmoid-derivative circuits. That is, as expected, a sigmoid-derivative cir-
cuit does not affect the learning process of the implementation of feedforward ANN
with learning.

Another PSPICE simulation is executed for the sequential learning of the feed-
forward ANN in Figure 6.1.1. Four training input-output vectors are given for this
simulation. First, the ﬁrst pair of input vector and its corresponding desired target
vector are applied with a given initial condition. The PSPICE transient analysis is
executed. The steady state weight values are measured and used as the initial condi-
tion for the next test vector. The same procedure is followed for the other three train-
ing vectors. After completing the fourth training process, the steady state weight
values are used as the initial condition of the ﬁrst training vector and this circuit is
simulated again. This circuit then converged to the steady state weight values which
are the same as those of the ﬁrst simulation result of the same training pattern.
Simulation of the next training pattern will result in the same steady state weight
values for each case. This simulation results are tabulated in Table 6.1.2. This
shows that the sequential learning appears to ignore the learning information of the
previous training vector and has learned only the present training vector as we have

encountered in Section 5.2.2.

050

 

_..l.

l

_T:._T]_T1_.T:_fF.E 8...... it = 0; EELM; 0. ~ 9.5;; age; 9:6 Na:

86

Table6.l.l(a)Tr-IsithSHCB simuhtimd'amfeedfaw-dANNwiththelequmtialleamingdrwitﬂ (unitsarein
«mayhem-ray, isnsemalotnput'lhehiﬁaleonditimuegivmhy

(11.101540115011510) 11.011140) 131(0) 1510121101940) 6.8» 61(0) 1 =19: 9: 0.5 05 05 05 as o: 95 9.5
0.5 0.5 )

1 2111012128!) 221(0) 222(0) 10.110117180117510) 1751012101240) 61(0) 52(0) 1 -- ( 1.0 4.0 -1.0 1.0 1.0 1.0 1.0 1.0 4.0

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

.10 10 1.0 ).
Calm l Calm 2 Column 3 Column 4 Cohnnn 5 Column 6 Colman 7 Column 8
x, 45 1.0 1.0 45 4.5 1.0 1.0 4.5
x, 4.5 1.0 4.5 1.0 4.5 1.0 45 1.0
r, 05 0.5 4.5 45 4.5 45 05 0.5
4, 0.5 05 4.5 45 4.5 45 05 05
y, 0.5098 04928 4510 4510 4.513 4.490 05098 0.5098
7, 05098 04928 4510 4.510 4513 4.490 05098 05098
4,-y, 00098 00072 0.01 0.01 0013 0.01 0.0098 0.0098
try; 0.0098 00072 0.01 0.01 0013 0.01 0.0098 0.0098
In 5.477504 0.997504 0.476504 4574504 3.411503 0.490504 4.178504 5.510504
1,, 5.477504 0.997504 4574504 0.476504 3.411503 0.490504 5510504 4.178504
1,, 5.477504 0.997504 0.476504 4574504 3.411503 0.490504 4.178504 5.510504
1,, 5.477504 0.997504 4574504 0.476504 3.411500 0.490504 5510504 4.178504
1,, 3.410503 3.411503 3.413540 3.408540 6925504 3.412503 3.411540 3.410503
1,, 3.410503 3.411500 3.408540 3.413500 6.925504 3.412500 3.410500 3.411503
1,, 3.410503 3.411503 3.413540 3.400540 6925504 3.412503 3.411500 3.410503
1,, 3.410503 3.411503 3.408540 3.413500 6.925504 3.412503 3.410540 3.411503
2, 5.477504 3.951504 4574504 4.574504 3.411503 4590504 5.510504 5.510504
9, 5.477504 3.951504 4.574504 4.574504 3.411503 4590504 5510504 5.510504
0, 0.771500 2.752540 0.716540 0.716540 0.412503 2.733500 0.770500 0.710500
0, 0.771500 2.752503 0.716500 0.716500 0.412500 2.733503 0.770500 0.770503
211 -l -l -l -1 005046 -l -l -l
3,, .1 -1 -1 -1 006046 -1 -1 -1
221 -l -1 .1 -1 005046 .1 .1 .1
3,, .1 .1 .1 .1 006046 -1 .1 -1
0,, 01396 01394 0.4721 07921 -1 0.1664 0.00945 03176
0,, 01396 01394 07921 0.4721 -1 01664 03176 000944
0,, 0.1396 01394 0.4721 0.7921 .1 0.1664 003945 03176
11,, 01396 01394 0.7921 04721 -1 0.1664 00176 010944
2, -1 -1 .1 .1 005046 -1 .1 -1
g, -1 -1 .1 -1 006046 -1 .1 -1
6, -1 -1 -1 -1 -1 -1 -1 -1
6, -1 -1 -1 -1 -1 -1 .1 -1

 

 

 

 

 

 

 

 

 

 

 

Table 6.1.l(b)deeluSPICB Wdamfudfmmmqunaﬁdkuniudmn (uninueinvolu).

87

nilsmgeundﬁ hmmﬂm'lbehiﬂdcmdiﬁmmﬁmby

(111(0)I12(°)121(°) 15(0) 111(0)11501150110010015016101 52(0) ) = ( 05 0.5 0.5 05 05 05 05 05 05 0.5

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

05 05 )
(3,,(0) 3,,(0) 3,,(0) 3,,(0) 0,,(0) 0,,(0) 0,,(0) 0,,(0) 9(0) Q00) §,(0) 600) ) = ( 1.0 0.0 0.0 1.0 1.0 1.0 1.0 1.0 0.0
0.0 1.0 1.0 1
Colm9 Column 10 Columnll Columnlz Column Colman“ Column 15 Colnmnl6
x, 4.5 1.0 1.0 45 4.5 1.0 1.0 45
x, 45 1.0 45 1.0 4.5 1.0 4.5 1.0
1, 05 05 45 45 4.5 45 05 05
1, 45 45 05 05 05 0.5 45 45
9, 05098 05098 4.490 4.490 4.490 4.490 0.4928 0.4928
9, 4510 4510 0.4928 0.4928 04928 0.4928 4.490 4.490
4,—0, 0.098 00098 0.01 0.01 001 0.01 00072 0.0072
4,0, 0.01 001 0.0072 00002 00072 0.0072 0001 0.001
In 4.673501 7.776500 0591540 3.409503 3.411503 0.675501 0014502 7.731502
1,, 4.673501 7.776502 3.409540 0589503 3.411503 0575501 8001540 0.872500
1,, 0.819501 1.406501 0.459540 3240540 3.240500 3.408540 461813-02 9.231502
1,, 0.819501 1.406501 32405-00 0.462502 3.240502 3.408540 7.397502 0.011502
1,, 3.413503 3.410503 0.147500 0.195503 0.129503 3.414540 3.411503 3.411540
1,, 3.406503 3.410503 3.411500 3.411503 3.411503 3.409540 3.411503 3.411500
1,, 3.412503 3.411503 0.194540 0.189503 0.193503 3.412500 3.412503 3.412503
1,, 3.409503 3.410503 3.411540 3.411503 3.411503 3.411540 3.411503 3.412503
1, 4.673501 0.651540 3.409500 3.409503 3.411503 2.208501 5001502 7.731502
9, 0.819501 0.382501 3240540 3240540 3.240502 4.486540 7.397502 9231540
5, 0.760503 0.757500 2.729500 2.730540 2.731503 2.732540 2.748503 2.750503
6, 0.724503 0.704500 2.748540 2.747503 2.747503 2.748540 2731503 2.732503
:11 -1 1 0 05879 006081 0 0 1
:1: -1 1 0.5879 0 006081 0 1 0
3,, 0 1 0 1 1 06348 0 1
3,, 0 1 1 .1 1 0.6348 1 0
0,, 05867 01396 0 0 0 0.7427 01345 0.1394
0,, 08496 01396 0.946540 0.933500 0947500 04009 01443 0.1394
0,, 01837 008064 0 0 0 02704 01996 0.1664
0,, 05124 02517 0.290540 0289500 0289500 8.942540 01332 0.1664
g, 0 0 05879 05879 006081 1 1 1
g, 0 0 1 1 1 0 1 1
6, 0 0 0 0 0 0 0 0
6, 0 0 0 0 0 0 0 0

 

 

 

 

 

 

 

 

 

 

 

 

 

 

H

 

88

Table 61.1(c) DC Analysis damfeedfuwudANN drmiwilhdlemgmvdahuh'hbk 611(6). 1,13 8131361.”
isnoutputdtbhiddmllya.mdy,ismompmohheompmhyu.

 

31 12 t1 *0 11 1: Y1 72 ‘1’?1 12"72

 

 

 

 

1.0 1.0 - - 5.0 5.0 0.0 (10 - -
Column 1 1.0 45 - - 0.0 0.0 05107 0.5107 - -
45 1.0 - - 0.0 0.0 05107 0.5107 - -

 

4.5 45 0.5 05 0.0 0.0 05107 0.5107 0.0107 0.0107
1.0 1.0 0.5 05 5.0 5.0 0.4931 0.4931 01069 0.11169

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Cohan 2 1.0 45 - - 0.0 0.0 0.0 0.0 - -
45 1.0 - - 0.0 0.0 0.0 0.0 - -
45 4.5 - - 0.0 0.0 0.0 0.0 - -
1.0 1.0 - - 5.0 5.0 0.0 0.0 - -
Column 3 1.0 45 4.5 4.5 0.0 0.0 4.51 1 4.51 1 0.01 1 -0.011
45 1.0 - - 0.0 0.0 4511 4.511 - -
45 45 - - 0.0 0.0 4511 4.511 - -
1.0 1.0 - - 5.0 5.0 0.0 0.0 - -
Column 4 1.0 45 - - 0.0 0.0 4.511 4.51 1 - -
4.5 1.0 4.5 45 0.0 0.0 4511 4.51 1 -0.01 1 -0.01 1
45 4.5 - - 0.0 0.0 4511 4.511 - -
1.0 1.0 - - 4.951 4.951 0.0 0.0 - -
Column 5 1.0 45 - - 4.881 4.881 0.0 0.0 - -
45 1 .0 - - 4.881 4.881 0.0 0.0 - -

 

45 4.5 4.5 4.5 2.091 2.091 4511 4.511 -0.011 -0.011
1.0 1.0 4.5 4.5 5.0 5.0 4.490 4.490 0.01 0.01

 

 

 

 

 

 

 

 

 

 

 

Colmm 6 1.0 45 - - 0.0 0.0 0.0 0.0 - -

45 1.0 - - 0.0 0.0 0.0 0.0 - -

45 45 - ~ 0.0 0.0 0.0 0.0 - -

=77: 1.0 - - 5.0 5.0 0.0 Off-d - -
Column 7 1.0 4.5 0.5 05 0.0 0.0 05099 0.5099 0.11199 0.1189

4.5 1.0 - - 0.0 0.0 0510 0.510 - -

45 45 - — 0.0 0.0 0.5099 0.5099 - -

1.0 1.0 - - 5.0 5.0 0.0 0.0 - -

Column 8 45 - - 0.0 0.0 05099 0.5099 - -

 

 

 

 

 

 

 

 

 

 

 

 

1.0
45 1.0 0.5 0.5 0.0 0.0 0510 0.510 -0.01 -0.01
45 45 - - 0.0 0.0 05099 0.5099 - -

 

 

 

 

 

89

Table 611.1(11) DCAndydsofanZfoeedforwudANNdrwhwhhmemmweighuinTAble 611.1(b). 1,. batsmen»-
isnonqmdlhehiddmhyet.mdy}ismompmof1hcwwhyen

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

1‘1 32 ‘1 t2 21 1: Y1 Y2 t1 - Y1 T1 ' Y2
1.0 1.0 - - 5.0 5.0 0.0 0.0 - a
Column 9 1.0 4.5 - - 0.0 0.0 05112 5.0 - -
45 1.0 - - 0.0 0.0 0.5109 5.0 - -
45 4.5 05 45 0.0 0.0 0.5109 5.0 0.0109 0.5
1.0 1.0 0.5 45 0.0 0.0 05107 4511 0.0107 0.01 1
Cohan 10 1.0 4.5 - - 0.05664 0.05664 0.4112 4.361 - -
45 1.0 - - 0.05665 0.05665 0.41 12 4.361 - -
4.5 4.5 - - 5.0 5.0 0.0 0.0 - -
1.0 1.0 . - 5.0 5.0 0.0 0.0 - -
Colmnn 1 1 1.0 4.5 4.5 0.5 1.646 5.0 4.594 0.6174 0.094 0.1 174
45 1.0 - - 0.0 0.0 5.0 5.0 - -
4.5 4.5 - - 0.0 5.0 5.0 5.0 - -
1.0 1.0 - - 5.0 5.0 0.0 0.0 - -
Cohm 12 1.0 4.5 - - 0.0 0.0 5.0 5.0 - -
45 1.0 4.5 0.5 1.646 5.0 4594 0.6174 0.094 0.1 174
4.5 4.5 - - 0.0 5.0 5.0 5.0 - -
1.0 1.0 - - 4.951 0.0 0.0 0.0 - -
Column 13 1.0 4.5 - - 4.881 5.0 0.0 0.0 - -
45 1.0 - - 4.881 5.0 0.0 0.0 - -
45 4.5 45 0.5 1.661 5.0 4.498 05012 0.002 0.012
1.0 1.0 45 0.5 5.0 5.0 4.489 0.4931 0.01 1 0.11369
Column 14 1.0 4.5 - - 5.0 0.0 5.0 0.25 - -
45 1.0 - - 5.0 0.0 5.0 0.25 - -
45 4.5 - - 0.0 0.0 0.0 0.0 - -
m 1.0 - - 20" 5.0 (14930 4.489 - -
Column 15 1.0 4.5 0.5 45 5.0 5.0 0.4931 4.489 0W9 0.011
4.5 1.0 - - 0.0 0.0 0.0 0.0 - -
45 4.5 - - 5.0 5.0 0.4920 4.488 - -
1.0 1.0 - - 5.0 5.0 0.4930 4.489 - -
Column 16 1.0 4.5 - - 0.0 0.0 0.0 0.0 - -
45 1.0 05 45 5.0 5.0 0.4931 4.4% 0W9 0.010
4.5 4.5 - - 5.0 5.0 (14920 4.488 - -

 

 

 

 

 

 

 

 

 

 

 

 

 

9O

Tnble 6.1.2 (1) Transient PSPICE simulation of a 2x2x2 feedforward MOS circuit with the sequential learning cir-
cuit“(units”11117011811111amgetandy'ismacmdwtpuLT'ln'nﬁtialcmdiﬁmmgivenby

(111(0) 112(0) [21(0) 13(0) 131(0) 712(0) 721(0) 722(0) 91(0) 92(0) 51(0) 52(0) ) = ( 0.5 ~05 -05 0.5 0-5 05 0-5
05 0.5 0.5 05 0.5 )

( 211(0) 112(0) 221(0) 322(0) Wn(0) “712(0) 1721(0) W22(0) 21(0) 22(0) 51(0) 52(0) ) = ( 1.0 -1-0 -1.0 1-0 1-0
1.0 1.0 1.0 -l.0 -1.0 1.0 1.0).

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

1 Column #1 Column #2 Column #3 Column #4 Column 45
x, 45 1.0 1.0 45 45
x, 45 1.0 45 1.0 45
r 05 0.5 45 45 05
y. 0.5098 0.4928 4510 4510 05098
9', 0.5098 0.4928 4510 4510 05098
1-y. 0.0098 0.0072 0.01 0.01 0.0098
r-y'; 0.0098 0.0072 0.01 0.01 0.0098
1,, 5.473504 5.002504 5.410504 4575504 5.428504
1,, 5.473504 5.002504 4.488504 5.476504 5.428504
1,. 5.473504 5.002504 5.410504 4575504 5.428504
In 5.473504 5.002504 4.488504 5.476504 5.428504
1'" 3.410503 3.411503 3.410503 3.410503 3.410503
13, 3.410503 3.411503 3.410503 3.410503 3.410503
13, 3.410503 3.411503 3.410503 3.410503 3.410503
1'22 3.410503 3.411503 3.410503 3.410503 3.410503
0, 5.473504 3.951504 4.488504 4575504 5.428504
9, 5.473504 3.951504 4.488504 4575504 5.428504
6. 5.771503 2.748503 5.73503 5.73503 5.739503
6, 5.771503 2.748503 5.73503 5.73503 5.739503
33.11 '1 -l -1 -1 -1
3,, -1 -1 -1 -1 -1
w_v ,1 -1 -1 -1 -1 -1
an -1 -1 -1 -1 .1
.7" 0.1396 0.1394 0.1663 0.1663 0.1396
51, 0.1396 0.1394 0.1663 0.1663 0.1396
49,. 0.1396 0.1394 0.1663 0.1663 0.1396
.7, 0.1396 0.1394 0.1663 0.1663 0.1396
g, -1 -1 -1 -1 -1
9, -1 -1 -1 .1 -1
6. -1 .1 .1 .1 .1
76',— -1 -1 .1 -1 -1

 

 

Table 6.1.203) DC Analysis of I 2x2x2 feedforward ANN circuit with the convergau weighs Table 6.1.201). ‘t is a

91

target.” ismwtputoftluhiddailayu.unifiamoutputofthewtwtlaya.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Jr1 x2 ‘1 12 21 22 Y1 Y2 11’Y1 H‘Yz
1.0 1.0 - - = 5.0 0.0 0.0 - —
COhImn 51.5 1.0 45 - - 0.0 0.0 05107 05107 - -
4.5 1.0 - - 0.0 0.0 05107 05107 - -
4.5 45 05 05 0.0 0.0 05107 0.5107 0.0107 0.0107
1.0 1.0 05 05 5.0 5.0 0.4931 0.4931 0.0069 0.0369
Column 52 1.0 45 - - 0.0 0.0 0.0 0.0 - -
45 1.0 - - 0.0 0.0 0.0 0.0 - -
45 45 - - 0.0 0.0 0.0 0.0 - -
1.0 1.0 - - 5.0 5.0 0.0 0.0 - -
Column 83.4 1.0 45 45 45 0.0 0.0 4511 4511 0.01 1 0.01 l
4.5 1.0 45 45 0.0 0.0 451 1 451 1 0.011 0.011
4.5 45 - - 0.0 0.0 4511 4511 - -

 

 

92

Another modiﬁcation of the modiﬁed learning rule results in the following
update law:

When the neuron k is in the output layer,
“’ij = 11 (‘k "YIJ 2,; - 091% (61-1)
and when the neuron j is in any hidden layer,

1.59: = TI 2‘. Wm,- (t... - )7...) y.- - ajiﬂjis (6.1.2)
"I

where )T,‘ is the output of the neuron k in the output layer, 1,- is the output of the
neuron j in a hidden layer, and y,- is the output of neuron i in a previous layer. If
the previous layer is the input layer, then y,- is the sigmoidal output to the external

input x,- .
6.2. The Implementation of the Module Chip

One module chip is designed for the use of any layer since the learning circuitry
of the output layer is the as same as that of the hidden layer. The block diagram of
n x m module chip is shown in Figure 6.2.1, where n is the number of inputs and m
is the number of outputs. Its block representation is shown in the box depicted on the
left corner at the bottom of Figure 6.2.1, where x is an n-dimensional input vector, y
is an m-dimensional output vector, e is an m-dimensional error vector from the next
higher layer, and E’ is an n-dimensional back-propagated error vector to the previous
lower layer. N denotes the nonlinearity of each neuron, 19,-" = [Wu 4 - - win ].

W1C = [W 1,- ' ° - WM], 71 -D represents n -dimensional vector multiplier, l-DI
ropresents l-dimensional multiplier with an integrating capacitor, and MUX denotes a

2x1 analog multiplexer. z,- is the output of MUX whose inputs are y,- and 0.

93

 

 

 

 

i”
j...

 

 

,..
é
a:
l.
2
g.

 

 

 

 

Figure 6.2.1 The block diagram of the n x m module chip

The module chip consists of two subcircuits: the feedforward subcircuit and the
learning subcircuit. The feedforward subcircuit generates n1 outputs, y,- ’s, and these
outputs are applied to the inputs of the next higher layer or used to modify the con-
nection weights, wij’s, and the threshold weights, 0,-’s, in the learning subcircuit.
These modiﬁed weights are used to generate back-propagated error signals for the
previous lower layer and also applied to the feedforward subcircuit.

The circuit implementation of the nxm module chip results in the following

equations:

yi = Sn (kanQ; Wijxj 4' 91)) (6.2.1)
1

RkaJ- = ’61 (8k - zkkj - Wk} (6.2.2)

94
2': = k... 2‘. Wade.- - 2;). (6.2.3)

where z,- = y,- when a module chip is used in the output layer and z,- = 0 when a
module chip is used in a hidden layer.

Input Layer Hidden Layer Output Layer

X

 

Figure 6.2.2 nx 2m x m two-layer feedforward ANNs with learning

Figure 6.2.2 illustrates a simple example of how this module chip can be used
to expand vertically and horizontally. Three nxm module chips are used to implement
an n x 2m x m two—layer feedforward ANNs with learning. Generally, if n == km,
then an n x km x m fully-connected two-layer feedforward ANN s with learning can
be implemented using (1: +1) :1 xm module chips.

The input layer is composed of just the input voltage nodes, 1. The module
chips in the middle form a hidden layer. The outputs of the hidden layer are applied
to the inputs of the output layer. In the output layer, the desired target vector is sup-
plied through the error vector, e . Then a back-propagated error vector is generated
and applied to the error vector node, e , of the module chips in the hidden layer. 71

M
back-propagated error signals are represented by E} = 2 w]; (t,- —j’1j), i = l, ..., n.
j.=l

95

If there is no lower layer, then the 2' vector is not propagated backward any further.
Otherwise, it is propagated through the error vector, or , of the module chip in that
lower layer.

The module chip has four diﬂ'erent-type signals: 1' lo signals, tuning signals, con-
trol signals, and voltage sources. There are four 1' lo signals, 1:, y, e, and e”.

There are ﬁve tuning signals, namely, Veg, V”, Vb”, vaf, and wap. Vc2
represents the tuning voltage for a multiplier and “y Vc2’s are required for 7 different
number of multipliers. For example, since the 4x1 module chip consists of two
different kinds of multipliers, a 5—D multiplier and 1-D multipliers, y = 2. Vb”, is a
bias for the w_shifter, vaf is a bias for a voltage follower, and Vbcop is a bias for a
CMOS operational ampliﬁer. The value of these tuning signals is determined by the
PSPICE simulation after (MOSIS) fabrication process of a module chip is completed
and the PSPICE parameters are provided (by MOSIS).

There are three control signals: freeze , select , and desel . freeze is set to +5
for the on-chip learning mode and -5 for the off-chip learning mode. select is set to
+5 when a module chip is used as the output layer and -5 for the hidden layer.

desel is set to +5 to turn on the analog multiplexer and —5 for turning it off.

[There are ﬁve different voltage sources (VS): +2.5 voltage source (Vddz), -2.5
voltage source (Vss,), and 0 voltage source (GND ), +5 voltage source (Vdd ), and -5

voltage source (Vss ).

Two different kinds of pins are reserved for interface circuitry, namely, r -w
and address . p r-w pins are used as the paths to read or write the synaptic weights
from the off-chip by selecting address in sequential order. These extra signals are
summarized in Table 6.2.1.

Table 6.2.1 The list of the extra signals for the nxm module chip

 

 

 

 

 

 

 

 

 

 

 

 

 

Type Signal Use
Vt: uming voltage for multipliers
V...2 common w; input for multipliers
tuning V1,, bias voltage for the w_shifter
V“, bias voltage for a voltage follower
Vb“, bias voltage for a CMOS op-amp
freeze +5v (the on-chip learning), -5v (the off-chip learning)
control select +5v (the output layer), -5v (a hidden layer)
desel +5v (MUX on), -5v (MUX off)
Vdd +5v
Vss -5v
voltage source GND 0v
Vdd2 +2.5v
Vss2 -2.5.v

 

 

 

 

 

The design of the module chip is critically dependent upon the number of pins
of the standard chip and its chip area. Table 6.2.2 shows the four MOSIS standard
chip sizes for 2 pm process. Table 6.2.3 tabulates the number of pins used for nxm
module chip.

The module chip can be connected vertically and horizontally to increase the
number of outputs of each layer. However, the number of inputs of each layer can
not be increased for fully-connected ANN circuits. Accordingly, the module chip is
designed so that n is at least three times bigger than 771. Table 6.2.4 summarizes the
nxm module chip vs. the MOSIS standard chip when the only limitation is given by

the number of pins.

97

Table 6.2.2 The four MOSIS standard chip sizes for 2 microns process

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Chip Die Sizeatm) Max. Project Size(mm) Area Normalization Price (Quantities)
Tiny 2540 x 2667 2.22 x 2.25 1 $500 (4)
Small 4826 x 7112 4.6 x 6.8 6.26 $2.5m (12)
Medium 7112 x 7112 6.9 x 6.8 9.39 35.400 (24)
Large 8218 x 9779 7.9 x 9.2 14.55 810.9(1) (32)
Table 6.2.3 The number of pins used for the nxm module chip
Signals I] x y e I tuning control address r-w V.S. H Total
#ofpinsﬂn m m n 1+4 3 a p 5 H2n+72u+10+1+a+p

 

 

 

 

 

 

 

 

 

 

Table 6.2.4 The nxm module chip vs. the MOSIS standard chip

 

 

40 Tiny

4x1

 

 

 

 

21116 30x10

 

 

40x10

 

 

6.3. The Operation of the Module Chip

In order to provide flexibility for learning, this module chip is designed with

two learning features: on-chip learning and off-chip learning. The on-chip learning is

executed by using on-chip learning circuits which realize the modiﬁed learning rule

[9]. The off-chip learning is executed by using off-line personal computer (PC).

 

98

Given any learning rule, a software program on the PC will compute the learning
process and produce the appropriate weights. These ﬁnal weights are represented as
analog voltage values and are written onto the analog storage devices by the control

of the PC. The analog storage devices we use are simply capacitors.

A capacitor takes a large chip-area when it is implemented using MOS technol-
ogy (section 3.4). A MOS capacitor is designed with a small capacitance for our ana-
log storage device. However, a small capacitance MOS capacitor is not a good ana-
log storage device since it loses its stored information due to its leakage currents.
Therefore, each stored weight will need to be refreshed regularly in order not to dis-
sipate its stored value. In order to decide the refreshing interval, ﬁrst, we have to
specify the allowable loss rate of the weight information, say 1% or 4%. Then deter-
mine the decay rate, or speciﬁcally decay time, of 1% or 4% of the desired weight,
called ‘dway‘ We need to refresh each weight with a time interval shorter than the

decay time, the“, .

Large scale ANNs have a huge number of weights. We have to specify how
many weights can be refreshed within the allowable refreshing time. By assuming
appropriate refreshing time, t, , we can estimate the number of weights to be
refreshed within the allowable fall time it takes the voltage level stored in the capaci-
tor to decrease by about 1% to 4% from its original rate tam. The block diagram
of the interfacing circuitry is shown in Figure 6.3.1. There are some interfacing ele-
ments to generate time delay such as digital-analog-converters (DAC), multiplexers,
sample-and-hold (S/H) circuits, and so on. Also there are on-chip time-delay elements
such as analog multiplexer, CMOS switches, and so on. Let’s denote the critical time
delay as ted . Then the maximum number of weights can be approximated by

t
(# of weights)“ = “my ,
trf

 

 

 

 

 

PC <—> RAM

”W 1‘ l

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

> MUX
off-chip 3
on-chip —
address —> MUX

 

 

 

 

 

 

 

 

 

 

 

 

A
O O 0
freeze ——
Learning ‘ ’ Synapse
"T Circuits —D—J Circuits —’

 

 

 

 

 

 

 

 

 

 

 

Learning . V Synapse
'” Circuits + f Circuits ”’

Figure 6.3.1 The block diagram of interface circuitry

100

where t,, > ted . We can refresh (# of weights )M weights within the allowable

loss of weight information.

Under the control of a PC, the operation of the module chip proceeds as fol-

lows:

To initiate the operation of the module chip,

1. Set all the tuning signals properly: VCz’s, V”, V”, vaf, and Vbcop.

2. select is set to +5v for the use of the output layer and set to -5v for the use of
the hidden layer.

3. Decide or specify the reﬁeshing time.

To perform the learning process using the on-chip learning circuits,

1. Set freeze = 5v and desel = -5v (the on-chip learning mode).

2. Apply an input vector x and its corresponding target vector t.

3. After convergence, set desel = 5v (the analog MUX is on).

4. Read the steady state values by setting address in sequential order through the
interface circuitry and store them in the off-chip digital memory.

To refresh or write each weight which is achieved from the on-chip learning process

or from the off-chip learning process,

1. Set freeze = -5v and desel = +5v (the off-chip learning mode).

2. Write each weight by setting address in sequential .order through the interfacing
circuitry.

3. Refresh each weight every refreshing interval.

101

6.4. Implementation of Module Chips for Large Scale Feedforward
ANN s with Learning

A modular design has been described for a large scale implementation of feed—
forward ANN s with learning. Two implementations are designed using the MAGIC
VLSI editor : 4x1 module chip on a 40-pin MOSIS Tinychip and 9x3 module chip
on a 64—pin MOSIS small chip. The module chips can be connected vertically and
horizontally to realize a large scale feedforward ANN s with optionally using on-chip
learning circuit or off-chip learning capability.

6.4.1. The 4x1 Module Chip on a 40-pin TINYCHIP

A 4x1 module chip is designed on a 40-pin Tinychip with analog pads and has
been sent for fabrication via MOSIS using 2 tun CMOS n-well process. There are
612 MOS transistors on the 2220x2250 pmz project area. Figure 6.4.1 shows the
layout on the 40-pin Tinychip pad ﬁame. Since the chip area of the Tinychip is small
(see Table 6.1(b) in Section 6.0), the value of n and m are limited by the chip area
and not by the number of pins of the chip.

The 4x1 module chip contains four inputs, one output, one error signal, and four
back-propagation errors. There are ﬁve weights : four synaptic weights and one thres-
hold weight. These weights can be read and written through one r—w pin using a
three-bit addressing analog multiplexer which is controlled by a PC. The refreshing
time can be chosen up to l msec for 1% loss of weight value because of small

number of weights to refresh.

The 40-pin Tinychip consists of 34 1/0 pins available for a designer and 6 pins
for Vdd and Vss, which are ﬁxed by MOSIS. 26 out of the 34 pins are used for the

Vdd F2 VSS cl sz V dd

Vdd2
GND

V01

select
Vss2
V02

vb,”

 

33 Vdd E4 Vuw VIM e Vss

Figure 6.4.1 The 4x1 module chip on a 40-pin Tinychip

ﬁeeze
l-DI

5111
II:— ...,...
Circuits

1:.

I" I

 

 

ﬂ

 

 

 

 

 

 

Figure 6.4.2 The circuit for each weight of the 4x1 module chip

103

4x1 module chip. Table 6.4.1 summarizes the number of pins used. Table 6.4.2 lists
what kind of and how many cells are used for the 4x1 module chip, where l-DI is
the 1-dimensional multiplier with an integrating capacitor, ID is the 1-dimensional
multiplier, and DI is a double inverter for a nonlinearity and a cell body. Figure 6.4.2
shows the circuit for each weight of the 4x1 module chip. Table 6.4.3 tabulates the
pin-assignment of the 4x1 module chip.

The operation of the 4x1 module chip, with the proper setting of tuning signals,
is as follows:
To perform the learning process using the on-chip learning circuits,
1. Set freeze = 5v and desel = ~5v (the on-chip learning mode).
2. Apply an input vector x and its corresponding target t.

3. After convergence, desel = 5v and read the steady state values by setting the
address (a2 a1 a0) in sequential order.

To refresh or write each weight (which is achieved from the on-chip learning process
or the off-chip learning process),

1. Set freeze = ~5v and desel = +5v (the off-chip learning mode).
2. Write each weight by setting the address (a2 01 do) in sequential order.

3. Refresh each weight every refreshing interval.

104

Table 6.4.1 The number of pins used for the 4x1 module chip

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Signal e control address r-w V.S. Total
a”

s of pins 1 3 3 1 26

Table 6.4.2 The cells used for the 4x1 module chip

Cells [rs-D l-Dl 1x8 MUX 1x2 MUX

it of cells I] 1 5 1 1

Table 6.4.3 Pin-assignment of the 4x1 module chip
Pin ll 2 5 6 7 8 10
Signal V, 25 GND Vdd v.1 '* * Vss
Pin # ll 12 15 16 17 18 20
Signal * Vdd X 3 r-w do a;
Pin # 22 24 25 26 27 28 30
Signal X3 desel Vss * * * Vdd
Pin it 31 32 35 36 37 38 40
Signal [1 v... Vss v... v, :1 Vss2 y

 

 

 

 

 

 

 

 

 

 

 

(* means "do not use”)

 

105

6.4.2. The 9x3 Module Chip on a (54-pin MOSIS Small Standard Chip

A 9x3 module chip is designed on a 64-pin MOSIS small standard chip with
analog pads using 2 11m CMOS technology. The project area of the MOSIS small
standard chip is 4600x6800 111712, which is 6.26 times larger than that of the
Tinychip. The 9x3 module chip is designed with 2,973 MOS transistors for the pur-

pose of future fabrication.

The 9x3 module chip contains nine inputs, three outputs, three error signals, and
nine back-propagation errors. There are thirty weights: twenty-seven synaptic weights
and three threshold weights. These weights can be read or written through two pins
using four-bit addresses by the control of a personal computer. The refreshing time
can be chosen up to 0.2 msec for 1% loss of weight value because of the relatively

small number of weights to be refreshed.

The layout of this chip is shown in Figure 6.4.3. 44 out of the 64 pins are used
for the 9x3 module chip. Table 6.4.4 summarizes the number of pins used and Table
6.4.5 lists what kind of and how many cells are used for the 9x3 module chip. Table
6.4.6 tabulates the pin-assignment of this module chip.

The 9x3 module chip has two different circuits from 4x1 module chip. One
difference is that deer is not used for the enable signal of the analog multiplexer in
order to reduce the chip-area and one pin. Instead (a3 a2 a1 a0) is set to (5v 5v 5v
5v) to turn off the analog multiplexer, i.e., when this address is selected, the other
outputs of the analog multiplexer are off and the corresponding output is on but not
connected (or used). Accordingly, this address plays a role of desel .

The other is that only one capacitor is used for each weight while the 4x1
module chip has two capacitors for each weight (See Figure 6.4.4 and Figure 6.4.2).
The PSPICE simulation has been performed by modifying a 2x2x2 feedforward ANN

Va? e3 t32 el x1 X2 x3 x4 x5 x6 x7 X8 x9 “"2

'61 Y1
6. v.21
53 var
E4 Vhws
35 vbcop
35 GND
57 5.5
Vdd
Es Vss
39
r-wl
”0 Y2
a1 chlo
a2 freeze
a3 select
r-wz +2.5

 

Y3

Figure 6.4.3 The 9113 module chip on a 64~pin MOSIS small pad frame

107

Table 6.4.4 The number of pins used for the 9x3 module chip

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

1x2 MUX
3
Pin # l] l 2 3 4 5 6 7 8 9 10
Signal * 55v GND v,” v... v”, V.,‘ yr V4, *
Pin # 11 12 13 14 15 W?TT
Signal x9 x. 17 15 15 14 x3 12 x1 *
‘ Pin# 21 22 T23 24 f???? 30
Signal e1 e2 e3 Veg3 Ft 6'2 63 Va: 3.1 52
Pin # 31 32 33 34 35 36 37 38 39 40
Signal 5, 5, r, r, z. Vdd a, 2', ml a,
Pin it 41 42 43 44 45 46 47 48 49 50
Signal 4: 4- 4: e a e 4- e e 4:
Pin 4 51 52 53 54 55 56 57 58 59 60
Signal [I )2 * " VSS ' ' - - - -

 

 

 

 

(" means ”do not use")

108

with learning in Figure 6.1.1 with the circuit in Figure 6.4.4. The results show that
the circuit of the 9x3 module chip converges to the same weight values as the circuit

of the 4x1 module chip does.
The operation of the 9x3 module chip (with the proper setting of tuning signals)

is as follows:

To perform the learning process using the on-chip learning circuits,

1. Set freeze = 5v and, set (03 a2 al ao)=(5v 5v 5v 5v) (the on-chip learning
mode).

2. Apply an input vector x and its corresponding target t.

3. After convergence, set freeze = SV and read the steady state values by setting
(a3 a2 a1 an) in sequential addressing order.

To reﬁesh or write each weight (which is achieved from the on-chip learning process

or the off-chip learning process),

1. Set freeze = -5v (the off-chip learning mode).

2. Write each weight by setting (03 02 al an) in sequential addressing order.

3. Reﬁesh each weight every refreshing interval.

freeze

 

10x3

_. - . ———_I-——-> Synapse»
1D . Circuits

 

 

 

 

 

 

 

Figure 6.4.4 The circuit for each weight of the 9x3 module chip

CHAPTER 7

SUMMARY

A feedforward neural network with a supervised learning rule is implemented
using standard CMOS technology. In order to implement a programmable synapse

cell, we employ a simple tunable all-MOS analog multiplier, voltage

shifters/attenuators, voltage followers, and MOS capacitors.

This multiplier uses voltage signals for its inputs and its output and executes the
multiplication between analog inputs and analog synaptic weights. Employing this
multiplier, we can implement large scale ANN s .with learning using fewer MOS
transistors than are required by implementations using the so-called Gilbert multi-
plier. Voltage shifters/attenuators adjust applied input voltages to meet the input
operating range of the multiplier, and they increase the input operating range of the
multiplier cell.

In order to test the performance of this scalar/vector analog multiplier, an 11-
dimensional vector multiplier is designed on a 40-pin MOSIS Tinychip with analog
pads, and it has been sent for fabrication via MOSIS using 2 11m CMOS n-well pro-
cess on Oct. 3, 1990. After receiving this chip, scalar to a maximum of an 11-

dimensional vector multiplier will be tested and compared to the results obtained
from the PSPICE simulation.

109

110

A modiﬁed error back-propagation learning rule [43] is used in order to realize
on-chip learning circuits using standard CMOS technology. Using this modiﬁed
learning rule, sigmoid-derivative circuits are not necessarily required for the imple-
mentation of feedforward ANN s with learning.

A prototype 2x2xl feedforward ANN with learning has been implemented and
simulated using the circuit simulator in order to prove that the learning scheme is
successquy implementable. The PSPICE simulation has been done by applying
several test vectors which are formed of a given input vector and its corresponding
desired pattern. During the learning process the connection weights converge to their

steady states that achieve the learning of the given pattern [29].

Two 2x2x1 feedforward ANN s with sequential learning circuits and two 2x2xl
feedforward ANN with simultaneous learning circuits are implemented and simulated
using the PSPICE circuit simulator. The results show that the former circuits may not
learn all the desired targets concurrently but that the latter can easily learn all the
desired targets. That is, a feedforward ANN with simultaneous learning circuit has
achieved all the weights, which are an equilibrium point, in order to achieve an
acceptable minimum value of the total squared error function, E . On the other hand,
a feedforward ANN with sequential learning circuit forces all the weights to converge
to an equilibrium point where only the squared error function for the desired target p,
8,, , has an acceptable minimum value. The two equilibrium points obtained are

expected to be different. That is, the ﬁnal weights of the feedforward ANN with

simultaneous learning circuit may not be a minimum for the feedforward ANN with

sequential learning circuit. Therefore, feedforward ANN s with sequential learning
circuits may ignore the previous learned input-output patterns.

A modular design is described for a large-scale implementation of feedforward

ANN s with learning. Two implementations are designed using the MAGIC VLSI

lll

editor : a 4x1 module chip on the MOSIS Tinychip and a 9x3 module chip on a
MOSIS small standard chip. The module chips can be connected vertically and hor-
izontally to realize a large scale feedforward ANNs with learning.

In order to provide ﬂexibility in the learning process, this module chip is
designed with two learning features : on-chip learning and off-chip learning. The on-
chip learning is executed by using on-chip learning circuits which realize the
modiﬁed learning rule [43]. The off-chip learning is executed by using any choice of
a learning rule as a software on a personal computer (PC).

The modiﬁed weights are represented as analog voltages and are written onto
the analog storage devices by the control of the PC. A MOS capacitor is employed
as a simple analog storage device over a Short interval of time. Due to leakage
currents, however, the MOS capacitors are required to be refreshed regularly in order
not to lose the stored weight values. This refreshing process is executed periodically
through interface circuitry under the control of the PC.

A 4x1 module chip is designed on a 40-pin MOSIS Tinychip with analog pads
using the MAGIC VLSI editor and was sent for fabrication via MOSIS using 2 11m
CMOS n—well process on Oct. 31, 1990. After receiving the 4x1 module chip, it will
be tested by forming two-layer feedforward ANNs with learning. If the test results
are successful, then the 9x3 module chip design will be ﬁnalized on a 64-pin MOSIS
small standard chip with analog pads and sent for fabrication.

112

APPENDICES

113

APPENDIX A
SPICE PARAMETERS

The SPICE parameters from MOSIS 3 micron CMOS process on June 1987.

114

APPENDIX A.l

 

 

 

 

 

NMOS TRANSISTOR PMOS TRANSIst
LEVEL 2 2
TPG -1
LD 280.000000E-09 280.000000E-09
VTO .827125 -.894654
KP 32.866490E-06 15.264520E-06
GAMMA 1.3596 .879003
PHI .6 .6
LAMBDA .01605 .047087
RSH 25 95
CJ 320.000000E-06 200.000000E-06
CJ SW 900.000000E-12 450.000000E-12
M] 0.5 0.5
MJSW 0.33 0.33
CGSO 520.000000E-l2 400.000000E-12
CGDO 520.000000E- 12 400.000000E- 12
NSUB 10.000000E+15 1 12.108800E+12
N SS 0 0
NFS 1.234795E+12 878.861700E+09
TOX 500000001509 50.000000E-09
XJ 400.000000E-09 400.000000E-09
UO 200 100
UCRIT 999.000000E+03 16.376500E+03
UEXP 1.001000E-03 .153441
VMAX 100.000000E+03 100.000000E-103
NEFF .01001 .01001
DELTA 1.2405 1.93831

 

 

115

APPENDIX A.2

The SPICE parameters from MOSIS 2 micron CMOS process on June 1989

 

 

 

 

 

NMOS TRANSISTOR PMOS TRANSIst
LEVEL 2 2
TPG -1
LD 212.340000E-O9 2500000001509
VTO .783736 -.807
KP 54.600000E-06 21.300000E-06
GAMMA .5262 .5644
PHI .6 .6
LAMBDA .035333 .056595
RSH 33.4 121.6
CJ 113.400000E-06 254.000000E-06
CJ SW 477.000000E-12 331.000000E-12
MI .708 .553
MJSW .253 .352
CGSO 266.956500E-12 314.303100E-12
CGDO 266.956500E-12 314.30310013—12
CGBO 425.025500E-12 457 .437700E-12
NSUB 5.860000E+15 6.740000E+15
NSS 1.000000E+12 1.000000E+12
NFS 954.427000E+09 100.000000E+09
TOX 41.200000E-09 41 .200000E-09
XJ 150.000000E-09 50.000000E-09
UO 651 253.997
UCRIT 30.664600E+03 16.929200E+03
UEXP .177364 ' .2458
VMAX 57.874100E+03 37 .028200E+03
NEFF 1.001
DELTA 1.99612 1.001368E-06

 

 

116

APPENDIX A.3

The SPICE parameters from MOSIS 2 micron CMOS process on Oct. 3, 1990

 

 

 

 

 

NMOS TRANSISTOR PMOS TRANSISTOR
LEVEL 2 2
TPG -1
LD 250.000000509 209.610000E-09
VTO .771327 -.78821
KP 56.060000E-06 22.300000E-06
GAMMA .53 .5486
PHI .6 .6
LAMBDA .030894 .049817
RSH 29.6 90.7
CI 113.650000E-06 251.360000E-06
CJSW 531.870000E-12 294.4700005-12
MJ .6862 .5471
MJSW .2651 .3188
CGSO 322924900512 270753200552
CGDO 322924900512 270753200552
CGBO 672642900512 583962500512
NSUB 6.257042E+15 6.7220925+15
NSS 100000005209 100000005+09
NFS 4.834708E+12 100.0000005+09
TOX 40.100000509 40100000509
XJ 250000000509 250000000509
U0 650.996 259
UCRIT 41.9315005+03 12.594500E+03
UEXP .153507 ' .182346
VMAX 67.660800E+03 41.222600E+03
NEFF 1.001
DELTA .472546 .844423

 

 

117

APPENDIX B
PSPICE INPUT FILES

118

APPENDIX B.l

PSPICE input ﬁle for the frequency response of the CMOS op-amp

.MODEL N NMOS LEVEL-2.00000 LD=O.212340U TOX=412.000E-10
+NSUB-5.860000£+15 VTO-0.783736 KP-5.4GOOOOE-05
+GAMMA=0.5262 PHI-0.600000 UO=651.000 UEXP=O.177364
+UCRIT-30664.6 DELTA-1.99612 VMAX=57874.1 XJ=O.1500000U
+LAMBDA=3.533329E-02 NFS=9.54427OE+11 NEFF=1
+NSS-1.000000E+12 TPG=1.00000 RSH=33.400 CGSO=2.669565E-10
+CGDO-2.66956SE-10 CGBO=4.250255E-10 CJ=1.134OOE-4
+MJ=0.70800 CJSW-4.77000E-10 MJSW=0.253000 PB=0.80000

.MODEL P PMOS LEVEL=2.00000 LD=0.250000U TOX=412.000E-10
+NSUB-6.74OE+15 VTO--0.807 KP=2.13000E-05

+GAMMA-0.5644 PHI-0.60 UO=253.997 UEXP=0.2458

+UCRIT-16929.2 DELTA-1.001368E-06 VMAX=37028.2 XJ=0.0500U
+LAMBDA-5.659491E-02 NFS=1.000E+11 NEFF=1.001
+NSS-1.000000E+12 TPG=-l.00000 RSH=121.600 CGSO=3.14303lE-1O
+CGDO-3.143031E-10 CGBO=4.574377E-10 CJ=2.5400E-4 MJ=O.55300

+CJSW-3.3100E-10 MJSW=O.35200 PB=O.80

V6 6 0 0.0

V5 5 0 0 AC 1.0
VBIAS 70 0 3.2
VDD 90 0 5.0
VSS 80 0 -5.0

M1 2 6 1 90 P L=4U W=16U
M2 3 5 l 90 P L=4U W=16U
M3 2 2 80 80 N L-16U W=4U
M4 3 2 80 80 N L-16U W-4U
M5 1 7O 90 90 P L=10U W=4U
M6 8 3 80 80 N L-4U W-24OU
M7 8 4 90 90 P L-4U W=200U
M8 4 2 80 80 N L-4U W=4U
M9 4 4 90 90 P L=4U W=4U

MCl 8 3 8 90 P L-lSU W=120U

.AC DEC 25 1 lOOMEG
.OPTION LIMPTS=10000
.PROBE

.END

119

APPENDIX B.2

PSPICE input ﬁle for the 1-D multiplier cell

VYl 1 0

VY2 2 0 .0

VXl 6 0

BX2 7 0 0 6 1

V02 60 0 4.959

XVWl 1 2 3 5 VSW

XVFl 3 4 4 COPl

XVF2 5 55 55 COPl

XVXl 6 7 8 9 VSWX

XMULl 4 55 8 9 11 12 10 60 MULl
.SUBCKT VSW 1 3 20 4O

VSS 80 0 -5.0

VDD 90 0 5.0

V70 70 0 3.0

M1 90 1 2 80 N L=22U W=11U
M3 2 1 80 80 N L-4U W=IOU
M5 80 2 20 90 P L-4U W=4U
M7 20 70 90 90 P L-4U W=4U
M2 90 3 4 80 N L=22U W=llU
M4 4 3 80 80 N L-4U W=10U
M6 80 4 40 90 P L=4U W=4U
M8 40 70 90 90 P L=4U W=4U
.BNDS VSW

.SUBCKT VSWX 1 3 2 4

VSS 80 0 -5.0

VDD 90 o. 5.0

M1 80 1 2
M3 2 1 90
M2 80 3 4
M4 4 3 90
.ENDS VSWX

.SUBCKT MULl

VSS 80 0 -5.0
VCl 50 0 5.0

MGlX 11 14
MDlX 12 13
MGZX 12 14
MD2X 11 13
MR61 60
MR62 50
MR51 60
MRSZ 50

00000001010303

wounds

90 P L-16U W=BU
90 P L-4U W-ZOU
90 P L-lGU W=80
90 P L-4U W-ZOU

11 12 13 14 5 6 8 6O

80 N L=24U W=4U
80 N L-24U W=4U
80 N L-24U W=4U
80 N L-24U W-4U
80 N L-480 W-4U
80 N L-48U W=4U
80 N L-48U W-4U
80 N L-48U W=4U

XCOPl 5 6 8 COP

.ENDS MULl

120

.SUBCKT COP1 5 6 8
VBIAS 70 0 3.2

VDD 9O 0 5.0

VSS 80 0 -5.0

M1 2 6 1 90 5 L-4U W-8U
M2 3 5 1 90 5 L-4U w-su

M3 2 2 8o 80 N L=16U W=4U
M4 3 2 80 80 N L=16u W=4U
M5 1 7o 90 90 P L-IGU W-4U
M6 8 3 80 80 N L=4U W=60U
M7 8 4 90 90 P L-4U w=470
M8 4 2 80 80 N L-4U w—4U
M9 4 4 90 90 p L-4U w-4U

MCI 8 3 8 90 P L-20U W-lOU
.ENDS COP1

.SUBCKT COP 5 6 8
VBIAS 70 O 3.2
VDD 90 O 5.0

VSS 80 O -5.0

M1 2 6 1 90 P L=4U W=160
M2 3 5 1 90 P L-4U W=16U
M3 2 2 80 80 N L-16U W=4U
M4 3 2 80 80 N L-16U W=4U
M5 1 7O 90 90 P L=1OU W=4U
M6 8 3 80 80 N L-4U W-240U
M7 8 4 9O 90 P L=4U W=200U
M8 4 2 80 80 N L=4U W=4U
M9 4 4 90 90 P L=4U W=4U
MCl 8 3 8 90 P L-4OU W=4OU
.ENDS COP

.MODEL N NMOS LEVEL=2.00000 LD=O.21234OU TOX=412.000E-10
+NSUB=5.860000E+15 VTO=0.783736 KP=5.460000B-05
+GAMMA=0.5262 PHI*0.600000 UO=651.000 UEXP=O.177364
+UCRIT=30664.6 DELTA=1.99612 VMAX=57874.1 XJ=0.1500000U
+LAMBDA-3.533329E-02 NFS-9.544270E+11 NEFF=1
+NSS=1.000000E+12 TPG-1.00000 RSH=33.4OO CGSO=2.669565E-10
+CGDO=2.669565£-10 CGBO=4.250255£-10 CJ=1.134OOE-4
+MJ=0.70800 CJSW=4.77000E-10 MJSW=0.253000 PB=0.80000

.MODEL P PMOS LBVEL=2.00000 LD=O.2500000 TOX=412.000E-10

+NSUB-6.74OE+15 VTO=-0.807 KP=2.13000E-05
+GAMMA=O.5644 PHI=0.60 UO=253.997 UEXP=O.2458
+UCRIT=16929.2 DELTA=1.001368E-O6 VMAX=37028.2 XJ=0.0500U

+LAMBDA=5.659491E-02 NFS=1.000E+11 NEFF=1.001
+NSS-1.000000E+12 TPG--1.00000 RSH=121.600 CGSO=3.143031E-10

+CGDO-3.l43031E-10 CGBO-4.574377E-10 CJ=2.54OOE-4 MJ-0.55300
+CJSW=3.3IOOE-10 MJSW-0.35200 PB=0.80

.PROBE

.DC vr1 -2.5 2.5 0.1 vx1 —2.5 2.5 2.5
.OPTIONS LIMPTS=100000

.PRINT DC V(3) V(4) V(4,55) v(10)

.END

121

APPENDIX B.3

PSPICE input ﬁle for the X-nonlinearitv of the 1-D cell

VYl l O 2.5

VY2 2 0 0.0

VXl 6 0 PWL(O -2.5 10M 2.5 20M -2.5 30M 2.5 40M -2.5 50M 2.5)
EX2 7 0 O 6 1

VC2 60 0 4.959

XVWl 1 2 3 5 VSW
XVFl 3 4 4 COP1
XVF2 5 55 55 COP1
XVXl 6 7 8 9 VSWX
XMULl 4 55 8 9 11 12 10 6O MULl

.SUBCKT VSW l 3 20 4O

VSS 8O 0 -5.0

VDD 90 O 5.0

V70 70 O 3.0

M1 90 1 2 80 N L=22U W=1lU
M3 2 l 80 80 N L=4U W=10U
M5 80 2 2O 90 P L=4U W=4U
M7 20 7O 9O 90 P L=4U W=4U
M2 90 3 4 80 N L=22U W=llU
M4 4 3 80 80 N L=4U W=1OU
M6 80 4 4O 90 P L=4U W=4U
M8 40 7O 9O 90 P L=4U W=4U
.BNDS VSW

.SUBCKT VSWX 3 2 4

l
VSS 80 O -5.0
VDD 9O 0 5.0

M1 80 1 2 90 P L=16U W=BU
M3 2 1 9O 90 P L=4U W=20U
M2 80 3 4 90 P L=16U W=8U
M4 4 3 9O 90 P L=4U W=20U
.ENDS VSWX

.SUBCKT MULl 11 12 13 14 5 6 8 60
V83 80 O -5.0
VCl 50 O 5.0

MGlX ll 14 6 80 N L=24U W=4U
MDlX 12 13 6 80 N L=24U W=4U
MG2X 12 14 5 80 N L=24U W=4U
MD2X 11 13 5 80 N L=24U W=4U
MR61 6 60 O 80 N L=48U W=4U
MR62 6 50 8 80 N L=48U W=4U
MR51 5 6O 8 80 N L=48U W=4U
MR52 5 50 O 80 N L=48U W=4U

XCOPl 5 6 8 COP
.ENDS MULl

122

.SUBCKT COP1 5 6 8
VBIAS 70 0 3.2

VDD 90 O 5.0

VSS 80 0 -5.0

M1 2 6 1 90 P L=4U W=8U

M2 3 5 1 90 P L=4U W=8U

M3 2 2 80 80 N L=16U W=4U
M4 3 2 80 80 N L=16U W=4U
M5 1 70 9O 90 P L=16U W=4U
M6 8 3 80 80 N L=4U W=6OU
M7 8 4 9O 90 P L=4U W=47U
M8 4 2 80 80 N L=4U W=4U
M9 4 4 90 90 P L=4U W=4U
MCl 8 3 8 90 P L=20U W=10U
.ENDS COP1

.SUBCKT COP 5 6 8
VBIAS 7O 0 3.2
VDD 9O 0 5.0

VSS 80 0 -5.0

M1 2 6 1 90 P L=4U W=16U
M2 3 5 1 90 P L=4U W=16U
M3 2 2 80 80 N L=16U W=4U
M4 3 2 80 80 N L=16U W=4U
M5 1 7O 9O 90 P L=10U W=4U
M6 8 3 80 80 N L=4U W=24OU
M7 8 4 90 90 P L=4U W=200U
M8 4 2 80 80 N L=4U W=4U
M9 4 4 9O 90 P L=4U W=4U
MCl 8 3 8 90 P L=4OU W=4OU
.ENDS COP

.MODEL N NMOS LEVEL=2.00000 LD=O.21234OU TOX=412.000B-10
+NSUB=5.86000OE+15 VTO=O.783736 KP=5.46000OE-05
+GAMMA=O.5262 PHI=O.6000OO UO=651.000 UEXP=O.177364
+UCRIT=30664.6 DELTA=1.99612 VMAX=57874.1 XJ=0.1500000U
+LAMBDA=3.533329E-02 NFS=9.54427OE+11 NEFF=1
+NSS=1.000000E+12 TPG=1.00000 RSH=33.400 CGSO=2.669565E-10
+CGDO=2.669565E-10 CGBO=4.250255E-10 CJ=1.13400E-4
+MJ=O.70800 CJSW=4.77000E-1O MJSW=O.253000 PB=0.80000

.MODEL P PMOS LEVEL=2.00000 LD=O.250000U TOX=412.000E-1O
+NSUB=6.74OE+15 VTO=-O.807 KP=2.13000E-05

+GAMMA=O.5644 PHI=O.6O UO=253.997 UEXP=O.2458
+UCRIT=16929.2 DELTA=1.00l368E-O6 VMAX=3ZOSSi2 XJ=0.0500U
+LAMBDA=5.659491E-02 NFS=1.000B+11 NEFF= .

+NSS=1.000000E+12 TPG=-1.00000 RSH=121.600 CGSO=3.14303lE-10
+CGDO=3.143031E-10 CGBO=4.574377E-10 CJ=2.54OOE-4 MJ=O.553OO

+CJSW=3.3lOOB-1O MJSW=0.35200 PB=0.80

.PROBE

.TRAN lOOU 55M
.OPTIONS LIMPTS=1000OO
.END

123

APPENDIX B.4

PSPICE input ﬁle for the W-nonlinearity of the 1-D cell

VYl 1 0 PWL(O -2.5 10M 2.5 20M -2.5 30M 2.5 40M -2.5 50M 2.5)
VY2 2 0 0.0

VXl 6 O 2.5

EX2 7 0 O 6 l

VC2 6O 0 4.959

XVWl l 2 3 5 VSW
XVFl 3 4 4 COP1
XVF2 5 55 55 COP1
XVXl 6 7 8 9 VSWX
XMULl 4 55 8 9 11 12 10 60 MULl

.SUBCKT VSW 1 3 20 4O

VSS 80 O -5.0

VDD 90 O 5.0

V70 70 0 3.0

M1 90 1 2 80 N L=22U W=11U
M3 2 1 8O 80 N L=4U W=10U
M5 80 2 20 90 P L=4U W=4U
M7 20 7O 9O 90 P L=4U W=4U
M2 90 3 4 80 N L=22U W=11U
M4 4 3 80 80 N L=4U W=lOU
M6 80 4 4O 90 P L=4U W=4U
M8 40 7O 90 90 P L=4U W=4U
.ENDS VSW

.SUBCKT VSWX 1 3 2 4

V88 80 O -5.0

VDD 9O 0 5.0

M1 80 l 2 90 P L=16U W=BU
M3 2 1 9O 90 P L=4U W=20U
M2 80 3 4 90 P L=16U W=8U
M4 4 3 90 90 P L=4U W=20U

.ENDS VSWX

.SUBCKT MULl ll 12 13 14 5 6 8 6O
VSS 80 0 -5.0
VCl 50 O 5.0

MGlX 11 14 6 80 N L=24U W=4U
MDlX 12 13 6 80 N L=24U W=4U
MGZX 12 14 5 80 N L=24U W=4U
MD2X ll 13 5 80 N L=24U W=4U
MR61 6 60 O 80 N L=48U W=4U
MR62 6 50 8 80 N L=48U W=4U
MR51 5 6O 8 80 N L=4BU W=4U
MR52 5 50 0 80 N L=48U W=4U

XCOPl 5 6 8 COP
.ENDS MULl

124

.SUBCKT COP1 5 6 8
VBIAS 7O 0 3.2

VDD 90 0 5.0

VSS 80 0 -5.0

M1 2 6 1 90 P L=4U W=BU

M2 3 5 1 90 P L=4U W=8U

M3 2 2 80 80 N L=16U W=4U
M4 3 2 80 80 N L=16U W=4U
M5 1 70 90 90 P L=16U W=4U
M6 8 3 80 80 N L=4U W=6OU
M7 8 4 9O 90 P L=4U W=47U
M8 4 2 80 80 N L=4U W=4U
M9 4 4 90 90 P L=4U W=4U
MCl 8 3 8 90 P L=20U W=1OU
.ENDS COP1

.SUBCKT COP 5 6 8
VBIAS 7O 0 3.2
VDD 9O 0 5.0

VSS 80 O -5.0

M1 2 6 1 90 P L=4U W=16U
M2 3 5 1 90 P L=4U W=16U
M3 2 2 80 80 N L=16U W=4U
M4 3 2 8O 80 N L=16U W=4U
M5 1 7O 9O 90 P L=lOU W=4U
M6 8 3 80 80 N L=4U W=24OU
M7 8 4 9O 90 P L=4U W=200U
M8 4 2 80 80 N L=4U W=4U
M9 4 4 9O 90 P L=4U W=4U
MCl 8 3 8 90 P L=4OU W=4OU
.ENDS COP

.MODBL N NMOS LEVEL=2.00000 LD=O.21234OU TOX=412.000E-1O
+NSUB=5.86000OE+15 VTO=O.783736 KP=5.46000OE-05
+GAMMA=O.5262 PHI=O.6000OO UO=651.000 UEXP=O.177364
+UCRIT=30664.6 DELTA=1.99612 VMAX=57874.1 XJ=O.1500000U
+LAMBDA=3.533329E-02 NFS=9.54427OE+11 NEFF=1
+NSS=1.000000E+12 TPG=1.00000 RSH=33.4OO CGSO=2.669565E-1O
+CGDO=2.669565E-10 CGBO=4.250255E-10 CJ=1.13400E-4
+MJ=0.70800 CJSW=4.77000E-10 MJSW=O.253000 PB=0.8000O

.MODEL P PMOS LEVEL=2.00000 LD=O.250000U TOX=412.000E-10
+NSUB=6.74OE+15 VTO=-O.807 KP=2.13000E-05

+GAMMA=O.5644 PHI=O.60 UO=253.997 UEXP=0.2458
+UCRIT=16929.2 DELTA=1.00l368E-O6 VMAX=37028.2 XJ=0.0SOOU
+LAMBDA=5.65949lE-02 NFS=1.000E+11 NEFF=1.001
+NSS=1.000000E+12 TPG=-1.00000 RSH=121.600 CGSO=3.143031E-10
+CGDO=3.143031E-10 CGBO=4.574377E-10 CJ=2.5400E-4 MJ=O.55300

+CJSW=3.31003-1O MJSW=O.35200 PB=0.80

.PROBE

.TRAN lOOU 55M
.OPTIONS LIMPTS=1000OO
.END

125

APPENDIX B.5

PSPICE input ﬁle for the step response of the 1-D cell

ULSE(-2.5 2.5 0.1N 0.1N 0.1N 1M 2M)

8 6O

VYl 1 O 2.5

VY2 2 O 0

VXl 6 O P

EX2 7 0 O 6 l

VC2 60 0 4.959

XVWl 1 2 3 5 VSW

XVFl 3 4 4 COP1

XVF2 5 55 55 COP1

XVXl 6 7 8 9 VSWX

XMULl 4 55 8 9 11 12 10 6O MULl
.SUBCKT VSW 1 3 20 4O

VSS 80 0 -5.0

VDD 90 0 5.0

V70 70 0 3.0

M1 90 1 2 80 N L=220 W=llU
M3 2 1 80 80 N L=4U W=10U
M5 80 2 20 90 P L=4U W=4U
M7 20 7O 9O 90 P L=4U W=4U
M2 90 3 4 80 N L=22U W=1lU
M4 4 3 80 80 N L=4U W=1OU
M6 80 4 4O 90 P L=4U W=4U
M8 40 7O 9O 90 P L=4U W=4U
.ENDS VSW

.SUBCKT VSWX 1 3 4

VSS 80 0 -5.0

VDD 9O 0 5.0

M1 80 1 2 90 P L=16U W=8U
M3 2 1 9O 90 P L=4U W=ZOU
M2 80 3 4 90 P L=16U W=8U
M4 4 3 9O 90 P L=4U W=20U
.ENDS VSWX

.SUBCKT MULl 11 12 13 14 5 6
VSS 80 O -5.0

VCl 50 0 5.0

MGlX ll 14 6 80 N L=24U W=4U
MDlX 12 13 6 80 N L=24U W=4U
MGZX 12 14 5 80 N L=24U W=4U
MD2X ll 13 5 80 N L=24U W=4U
MR61 6 6O 0 80 N L=48U W=4U
MR62 6 50 8 80 N L=48U W=4U
MR51 5 6O 8 80 N L=48U W=4U
MR52 5 50 O 80 N L=48U W=4U
XCOPl 5 6 8 COP

.ENDS MULl

126

.SUBCKT COP1 5 6 8
VBIAS 7O 0 3.2

VDD 9O 0 5.0

VSS 80 0 -5.0

M1 2 6 1 90 P L=4U W=8U

M2 3 5 1 90 P L=4U W=8U

M3 2 2 80 80 N L=16U W=4U
M4 3 2 80 80 N L=16U W=4U
M5 1 70 9O 90 P L=16U W=4U
M6 8 3 80 80 N L=4U W=6OU
M7 8 4 90 90 P L=4U W=47U
M8 4 2 80 80 N L=4U W=4U
M9 4 4 90 90 P L=4U W=4U
MCl 8 3 8 90 P L=20U W=10U
.ENDS COP1

.SUBCKT COP 5 6 8
VBIAS 70 O 3.2
VDD 90 O 5.0

VSS 8O 0 -5.0

M1 2 6 1 90 P L=4U W=16U
M2 3 5 1 90 P L=4U W=16U
M3 2 2 8O 80 N L=16U W=4U
M4 3 2 80 80 N L=16U W=4U
M5 1 70 90 90 P L=10U W=4U
M6 8 3 8O 80 N L=4U W=24OU
M7 8 4 9O 90 P L=4U W=200U
M8 4 2 80 80 N L=4U W=4U
M9 4 4 9O 90 P L=4U W=4U
MCl 8 3 8 90 P L=4OU W=40U
.ENDS COP

.MODEL N NMOS LEVBL=2.00000 LD=O.21234OU TOX=412.000E-10
+NSUB=5.86000OE+15 VTO=O.783736 KP=5.46000OE-05
+GAMMA=O.5262 PHI=O.6000OO UO=651.000 UEXP=O.177364
+UCRIT=30664.6 DELTA=1.99612 VMAX=57874.1 XJ=0.1500000U
+LAMBDA=3.533329E-02 NFS=9.544270E+11 NEFF=1
+NSS=1.000000E+12 TPG=1.00000 RSH=33.400 CGSO=2.669565E-10
+CGDO=2.669565E-1O CGBO=4.2502553-10 CJ=1.134OOE-4
+MJ=0.70800 CJSW=4.77000E-1O MJSW=O.253000 PB=0.80000

.MODEL P PMOS LEVEL=2.00000 LD=O.250000U TOX=412.000E-1O
+NSUB=6.74OE+15 VTO=—O.807 KP=2.13000E-05

+GAMMA=O.5644 PHI=O.60 UO=253.997 UEXP=O.2458
+UCRIT=16929.2 DELTA=1.001368E-06 VMAX=37028.2 XJ=0.0500U
+LAMBDA=5.659491E-02 NFS=1.000E+11 NBFF=1.001
+NSS=1.000000B+12 TPG=-1.00000 RSH=121.600 CGSO=3.143031E-10
+CGDO=3.143031E-10 CGBO=4.574377E-10 CJ=2.54OOB-4 MJ=0.553OO

+CJSW=3.31OOB-1O MJSW=O.35200 PB=0.80

.PROBE

.TRAN lOUS 6M3
.OPTIONS LIMPTS=lOOOOO
.END

127

APPENDIX B.6

PSPICE input ﬁle for the MD cell

VYl 1 0

VYZ 2 O 0.0

VX1 6 0

BXZ 7 0 0 6 1

VC2 60 O 4.549

XVWl 1 2 3 5 VSW

XVWZ 1 2 31 51 VSW

XVW3 1 2 32 52 VSW

XVW4 1 2 33 53 VSW

XVWS 1 2 34 54 VSW

XVW6 1 2 35 55 VSW

XVW7 1 2 36 56 VSW

XVW8 1 2 37 57 VSW

XVW9 1 2 38 58 VSW

XVWlO 1 2 39 59 VSW

XVW11 1 2 41 61 VSW

XVXl 6 7 8 9 VSX

XMULl 3 5 8 9 11 12 MULF
XMULZ 31 51 8 9 11 12 MULF
XMUL3 32 52 8 9 11 12 MULF
XMUL4 33 53 8 9 11 12 MULF
XMULS 34 54 8 9 11 12 MULF
XMUL6 35 55 8 9 11 12 MULF
XMUL7 36 56 8 9 11 12 MULF
XMUL8 37 57 8 9 11 12 MULF
XMUL9 38 58 8 9 11 12 MULF
XMULlO 39 59 8 9 11 12 MULF
XMULll 41 61 8 9 11 12 MULF
XMULB 11 12 10 60 MULB

.SUBCKT VSW 1 3 21 41

VSS 80 O -5.0

VDD 9O 0 5.0

V70 70 O 3.0

M1 90 1 2 80 N L-22U W-llU
M3 2 1 80 80 N L=4U W=10U
M5 80 2 20 90 P L-4U W=4U
M7 20 70 90 90 P L-4U W=4U
M2 90 3 4 80 N L-22U W-llU
M4 4 3 80 80 N L=4U W=10U
M6 80 4 40 90 P L-4U W-4U
M8 40 7O 9O 90 P L-4U W-4U

XVFZO 20 21 21 COP1
XVF40 4O 41 41 COP1
.ENDS VSW

.SUBCKT VSX 1 3
VSS 80 0 -5.0

VDD 90 0 5.0
M1 80 1 2 90
M3 2 1 90 90
M2 80 3 4 90
M4 4 3 9O 90
.ENDS VSX

.SUBCKT MULF 11
VSS 80 0 -5.0
MGlX 11 14 6 80
MDlX 12 13 6 80
MG2X 12 14 5 80
MD2X 11 13 5 80
.ENDS MULF

.SUBCKT MULB 5
VSS 80 0 -5.0
VC1 50 0 5.0
MR61 6 60 0 80
MR62 6 50 8 80
MR51 5 60 8 80
MR52 5 50 O 80
XCOPl 5 6 8 COP
.ENDS MULB

.SUBCKT COP1 5
VBIAS 70 0 3.2
VDD 90 0 5.0
VSS 80 O -5.0

1
1
80 80 N
80 80 N
70 9O 90 P
3 80 80 N
4 90 90 P
M8 2 80 80
M9 4 9O 90
MCl 8 3 8 90
.ENDS COP1

M1
M2
M3
M4
M5
M6
M7

Nwmm

obobCDCDD-‘CONUJN

N
P
P

.SUBCKT COP 5 6
VBIAS 70 0 3.2
VDD 90 0 5.0
VSS 80 0 -5.0

6
5
2 80 80 N
2 80 80 N
70 9O 90 P
3 80 80 N
4 90 90 P
M8 2 80 80 N
M9 4 9O 90 P
MCI 8 3 8 90 P
.ENDS COP

M1
M2
M3
M4
M5
M6
M7

1

bubCDCDI-‘WNMN

128

P L-lGU W=8U
P L=4U W-ZOU
P L*16U W=8U
P L=4U W-ZOU

12 13 14 5 6

W=4U
W=4U
W=4U
W=4U

N L-24U
N L-24U
N L=24U
N L-24U

6 8 60

W=4U
W=4U
W=4U
W=4U

N L=48U
N L=48U
N L=48U
N L=48U

6 8

90 P L=4U W=8U
90 P L=4U W=8U

L=16U W=4U
L=16U W=4U
L-16U W=4U
L=4U W=6OU
L=4U W=47U
L-4U W=4U
L=4U W=4U
L=ZOU W=10U

8

1 90 P L=4U W=16U
90 P L=4U W=16U

L=16U W=4U
L-lGU W=4U
L-lOU W-4U
L=4U W=240U
L-4U W=ZOOU
L=4U W-4U
L-4U W=4U
L-4OU W-4OU

129

.MODEL N NMOS LEVEL-2.00000 LD=O.21234OU TOX=412.000E-10
+NSUB=5.860000E+15 VTO=O.783736 KP=5.46000OE-05
+GAMMA=0.5262 PHI=0.600000 UO=651.000 UEXP=O.177364
+UCRIT-30664.6 DELTA=1.99612 VMAX=57874.1 XJ=0.1500000U
+LAMBDA-3.533329E-02 NFS-9.54427OB+11 NEFF=1
+NSS-1.000000E+12 TPG-1.00000 RSH=33.4OO CGSO=2.669565E-10
+CGDO-2.669565E-10 CGBO-4.250255E-10 CJ=1.13400E-4
+MJ-0.70800 CJSW-4.77000E-10 MJSW=0.253000 PB=0.80000

.MODEL P PMOS LEVEL-2.00000 LD=O.2500000 TOX=412.000E-10
+NSUB-6.74OE+15 VTO--0.807 KP=2.13000E-05

+GAMMA=O.5644 PHI-0.60 UO=253.997 UEXP=O.2458

+UCRIT=16929.2 DELTA=1.001368E-06 VMAX=37028.2 XJ=0.0500U
+LAMBDA=5.659491E-02 NFS=1.000E+11 NEFF=1.001
+NSS=1.000000E+12 TPG=-1.00000 RSH=121.600 CGSO=3.143031E-10
+CGDO=3.143031E-10 CGBO=4.574377B-10 CJ=2.5400E-4 MJ=0.55300
+CJSW=3.3100E-10 MJSW=O.35200 PB=0.80

.PROBE

.DC VY1 -2.5 2.5 0.1 VX1 -2.5 2.5 2.5
.OPTIONS LIMPTS-lOOOOO

.PRINT DC V(3,5) V(8,9) V(ll) V(1l,12) V(10)
.END

130

APPENDIX B.7

PSPICE input ﬁle for the sigmoid-derivative cell

VY1 1 0

V111 111 0 2.5

V117 117 0 2.5

VC2 60 0 4.99

XDSl 1 101 NEU

XVWl 101 0 103 105 VSW
XVFl 103 104 104 COP
XVF2 105 106 106 COP
XVXl 0 101 108 109 VSX
XVW2 111 O 113 115 VSW
XVF3 113 114 114 COP
XVP4 115 116 116 COP
XVX2 117 0 118 119 VSX
XMULl 104 106 108 109 11 12 MULl
XMULZ 114 116 118 119 11 12 MULl
XFDRl 11 12 10 60 FDR
.SUBCKT NEU 1 3

VSS1 80 0 -5.0

VDDl 90 0 5.0

VSS2 60 0 -2.5

VDD2 7O 0 2.5

M1 2 1 60 80 N L‘ZU W-2U
M2 2 1 70 90 P L-2U W-4U
M3 3 2 60 80 N L820 W=2U
M4 3 2 70 90 P L=ZU W-4U
.ENDS NEU

.SUBCKT VSX 1 3 2 4
VSS 80 O -5.0
VDD 9O 0 5.0
M1 80 1 2 90
M3 2 1 9O 90
M2 80 3 4 90
M4 4 3 90 90
.ENDS VSX

P L-lGU W=8U
P L-4U W-20U
P L=16U W=8U
P L=4U W=ZOU

.SUBCKT VSW 1 3 20 40

VSS 80 0 -5.0
VDD 90 0 5.0
V70 70
M1 90
M3 2 1
M5 80 2
M7 20

N L-22U W-11U
N L-4U W-lOU
P L-4U W-4U
P L-4U W-4U

80
80
90
90

131

M2 90 3 4 80 N L-ZZU W=11U
M4 4 3 80 80 N L-4U W=1OU
M6 80 4 40 90 P L-4U W=4U
M8 40 70 90 90 P L=4U W=4U
.ENDS VSW

.SUBCKT MUL1 11 12 13 14 5 6
VSS 80 0 -5.0
MGlX 11 14 6 80 N L-24U W=4U

MDlX 12 13 6 80 N L=24U W=4U
MG2X 12 14 5 80 N L-24U W=4U
MDZX 11 13 5 80 N L=24U W=4U
.ENDS MULl

.SUBCKT FDR 5 6 8 60

VSS 80 O -5.0

VC1 50 0 5.0

MR61 6 60 0 80 N L-44U W=4U
MR62 6 50 8 80 N L=44U W=4U
MR51 5 60 8 80 N L-44U W=4U
MR52 5 50 0 80 N L-44U W=4U
XCOPl 5 6 8 COP

.ENDS FDR

.SUBCKT COP 5 6 10

VBIAS 70 0 -3.5

VDD 90 0 5.0

VSS 80 0 -5.0

M1 7 5 1 80 N L=4U W=24U

M2 8 6 1 80 N L=4U W=24U

M3 3 2 7 80 N L-24U W=4U

M4 2 2 8 80 N L-24U W=4U

M5 3 2 90 90 P L-24U W=4U

M6 2 2 90 90 P L=24U W=4U

M7 1 7O 80 80 N L-16U W-4U

M8 10 7O 80 80 N L-4U W=200U
M9 10 3 90 90 P L=4U W=280U
MR 3 80 9 90 P L=4U W=16U
MCN 10 9 10 80 N L=4OU W-4OU
MCP 10 9 10 90 P L-4OU W-4OU
.ENDS COP

.MODEL N NMOS LEVEL-2.00000 LD=O.21234OU TOX-412.000E-10
+NSUB-5.86000OE+15 VTO=O.783736 KP=5.460000E-05
+GAMMA-0.5262 PHI-0.600000 UO=651.000 UEXP-0.177364
+UCRIT-30664.6 DELTA-1.99612 VMAX=57874.1 XJ=0.1500000U
+LAMBDA-3.533329E-02 NFS=9.544270E+11 NEFF=1
+NSS-1.000000E+12 TPG-1.00000 RSH=33.400 CGSO-2.669565E-10
+CGDO-2.669565E-10 CGBO-4.250255£—10 CJ=1.134OOB-4
+MJ-0.70800 CJSW-4.77000E-10 MJSW=0.253000 PB=0.80000

.MODEL P PMOS LEVEL-2.00000 LD=O.250000U TOX-412.000E-10
+NSUB-6.74OE+15 VTO--0.807 KP=2.13000E-05

+GAMMA-0.5644 PHI-0.60 UO=253.997 UEXP=O.2458
+UCRIT-16929.2 DELTA-1.001368E-06 VMAX-37028.2 XJ-0.0500U
+LAMBDA-5.659491E-02 NFS-1.000E+11 NEFF-1.001
+NSS-1.000000E+12 TPG--1.00000 RSH8121.600 CGSO-3.143031E-10
+CGDO-3.143031E-10 CGBO-4.574377E-10 CJ-2.5400E-4 MJ‘0.55300

+Csz-3.31003-10 MJSW-0.35200 PB=0.80

.PROBE

.DC VYl -0.05 0.05 0.0001
.OPTIONS LIMPTS-lOOOOO
.PRINT DC V(101) V(10)
.END

132

133

APPENDIX B.8

PSPICE input ﬁle for the Table 5.2.1

V31 1 O 4.5

VLl 2 0 0.5

VL2 100 0 1.0

VRR 120 0 2.5

XMHl 3 1 4 2 5 MUL2

XNHl 5 9 101 NEUl

XMH2 6 1 7 2 8 MUL2

XNH2 8 10 103 NEUl

XMOl 11 9 12 10 13 MUL2

XNOl 13 14 NEU2

XL11 101 100 14 102 11 MUL11
XL12 103 100 14 104 12 MUL11
XL21 11 100 14 105 MUL12
XL24 12 100 14 106 MUL12
XL23 105 1 120 108 3 MUL11
XL22 105 2 120 107 4 MUL11
XL26 106 1 120 110 6 MUL11
XL25 106 2 120 109 7 MUL11
.SUBCKT MUL2 1 3 11 13 8

VSS 80 O -5.0

V2 2 0 -3.0

V4 4 0 2.5

MGlX 1 4 6 80 N L=4OU W=4U
MDlX 2 3 6 80 N L-4OU W-4U
MG2X 2 4 5 80 N L-4OU W-4U
MDZX 1 3 5 80 N L-40U W-4U
MGl 11 4 6 80 N L=4OU W=4U
MD1 2 13 6 80 N L-4OU W-4U
MG2 2 4 5 80 N L-4OU W-4U
MD2 11 13 5 80 N L-4OU W=4U

EMl 8 0 5 6 1E08
RI 5 6 lOOMEG

R1 8 6 140K

R2 5 0 140K
.ENDS MUL2

.SUBCKT NEUl 1 5 7

VDD 90 0 5.0

VR2 2 O 2.5

EVSl 3 2 1 0 1

MNl 4 3 O O N L-lOU W-4U
MP1 4 3 90 90 P L-4U W-10U
MN2 5 4 0 0 N L-4U W=4U
MP2 5 4 90 90 P L-lOU W-lOU
VR 6 0 -4.0

EVSZ 7 6 5 O 0.4

.ENDS NEUI

134

.SUBCKT NEU2 1 5

VDD 90 0 5.0

VR2 2 0 2.5

EVS 3 2 1 0 1

MNl 4 3 0 0 N L-4U WHIOU
MP1 4 3 90 90 P L-4U W540
MN2 5 4 0 0 N L-4U W-lOU
MP2 5 4 90 90 P L-4U W=4U
.ENDS NEU2

.SUBCKT MUL11 1 3 4 8 10

VSS 80 0 -5.0

VYC2 2 0 -3.0

MGlX 1 4 6 80 N L=4OU W-4U
MD1X 2 3 6 80 N L-4OU W=4U
MGZX 2 4 5 80 N L-4OU W-4U
MD2X 1 3 5 80 N L-4OU W-4U
RI 5 6 100MEG

EMl 7 0 5 6 1E08

R1 7 6 100K

R2 5 0 100K

RR 7 8 100

CC 8 0 lPF

EVS 10 2 8 0 1

.ENDS MUL11

.SUBCKT MUL12 1 3 4 10

VSS 80 -5.0

VYC2 2 -3.0

MGlX 1 6 80 N L=4OU W=4U
MD1X 2 6 80 N L=4OU W=4U
MGZX 2 5 80 N L=4OU W=4U
MD2X 1 5 80 N L=4OU W=4U
RI 5 6 100MEG

EM1 7 0 5 6 1308

R1 7 6 100K

R2 5 0 100K

EVS 10 2 7 0 1

.ENDS MUL12

WQWDOO

.MODEL N NMOS LEVEL-2.00000 LD-0.280000U TOX-500.000E-10
+NSUB-1.000000E+16 VTO-0.827125 KP-3.286649E-05
+GAMMA'1.35960 PHI-0.600000 UO-200.000 UEXP-1.001000E-03
+UCRIT=999000. DELTA-1.24050 VMAX-100000. XJ-O.4000OOU
+LAMBDA-1.604983E-02 NFS-1.23479SE+12 NEFF-1.001000E-02
+NSS-0.000000E+00 TPG-1.00000 RSH-25 CGSO=5.ZE-10
+CGDO‘5.2E-10 CJ-3.2E-4 MJ‘0.5 CJSW-9E-10 MJSW=O.33

.MODEL P PMOS LEVEL-2.00000 LD-0.28000OU TOX-500.0003-10
+NSUB=1.121088E+14 VTO--0.894654 KP-1.526452E-05
+GAMMA-O.879003 PHI-0.600000 UO=100.000 UEXP-0.153441
+UCRIT-16376.5 DELTA-1.93831 VMAX-100000. XJ-0.400000U
+LAMBDA-4.708659E-02 NFS-8.788617E+11 NEFF-1.001000E-02
+NSS-0.000000E+00 TPG--1.00000 RSH-95 CGSO'4E-10
+CGDO=4E-10 CJ-2E-4 MJ-0.5 CJSW=4.5E-10 MJSW-0.33

.IC V(108)--O.5 V(107)-0.6 V(110)--0.5 V(109)-0.5
.IC V(102)-0.7 V(104)-0.5

.PROBE

.TRAN 0.1ns lns

.END

135

APPENDIX B.9

PSPICE input ﬁle for the dc analysis of the Table 5.2.1

VX1 1 0

VX2 2 0

VWll 3 0 -4.0
VW12 4 0 -4.0
VW21 6 0 -4.0
VW22 7 0 -4.0
VW31 11 0 -2.8603
VW32 12 0 -2.8609
VTl 16 0 -4.0
VT2 17 0 -4.0
VT11 18 O -4.0
XFFl 1 2 3 4 6 7 9 10 11 12 14 16 17 18 FFXR

.SUBCKT FFXR 1 2 3 4 6 7 9 10 11 12 l4 16 17 18

XMHl 3 1 4 2 16 5 MUL3

XNHl 5 9 NEUl

XMHZ 6 1 7 2 17 8 MUL3

XNH2 8 10 NEUl

XMOl 11 9 12 10 18 13 MUL3
XNOl 13 14 NEUl

.ENDS FFXR

.SUBCKT MUL3 l 3 11 13 21 8
VSS 80 0 -5.0

V2 2 0 -3.0

V4 4 0 2.5

V23 23 0 3.5

MG11 1 4 6 80 N L=4OOU W=4U
MDll 2 3 6 80 N L=4OOU W=4U
M612 2 4 5 80 N L=4OOU W=4U
MD12 1 3 5 80 N L=4OOU W=4U
M621 11 4 6 80 N L=4OOU W=4U
MD21 2 13 6 80 N L=4OOU W=4U
MG22 2 4 5 80 N L=4OOU W=4U
MD22 11 13 5 80 N L=4OOU W=4U
MG31 21 4 6 80 N L=4OOU W=4U
MD31 2 23 6 80 N L=4OOU W=4U
MG32 2 4 5 80 N L=4OOU W=4U
MD32 21 23 5 80 N L=4OOU W=4U

EMl 8 0 5 6 1E08
RI 5 6 lOOMEG

R1 8 6 400K
R2 5 0 400K
.ENDS MUL3

136

.SUBCKT NEUl l 5

VDD 90 O 5.0

VR2 2 0 2.5

BVSl 3 2 l 0 l

MNl 4 3 0 0 N L=ZU W=ZU
MP1 4 3 90 90 P L=2U W=4U
MN2 5 4 0 O N L=2U W=2U
MP2 5 4 90 90 P L=2U W=4U
.ENDS NEUl

.MODEL N NMOS LEVEL=2.00000 LD=O.28000OU TOX=500.000E-10
+NSUB=1.000000E+16 VTO=O.827125 KP=3.286649E-05
+GAMMA=1.3596O PHI=O.6000OO UO=200.000 UEXP=1.001000E-O3
+UCRIT=999000. DELTA=1.24050 VMAX=100000. XJ=0.400000U
+LAMBDA=1.604983E-02 NFS=1.23479SE+12 NEFF=1.001000E-02
+NSS=0.000000B+00 TPG=1.00000 RSH=25 CGSO=5.2E—10
+CGDO=5.2£-10 CJ=3.2£-4 MJ=O.5 CJSW=9E-1O MJSW=O.33

.MODEL P PMOS LEVEL=2.00000 LD=O.2800000 TOX=500.000E-10
+NSUB=1.121088E+14 VTO=-0.894654 KP=1.526452E-05
+GAMMA=O.879003 PHI=0.600000 UO=100.000 UEXP=O.153441
+UCRIT=16376.5 DELTA=1.93831 VMAX=100000. XJ=O.400000U
+LAMBDA=4.708659E-02 NFS=8.788617E+11 NEFF=1.001000E-02
+NSS=0.000000E+OO TPG=-l.00000 RSH=95 CGSO=4E-10
+CGDO=4B-10 CJ=2E-4 MJ=O.5 CJSW=4.5E-1O MJSW=O.33

.PRINT DC V(2) V(9) V(10) V(14)
.PROBE

.OPTIONS ITL4=4O RELTOL=0.01 ITL5=0
.DC VX1 0.5 4.5 4.0 VX2 0.5 4.5 4.0
.END

137

APPENDIX B.10

PSPICE input ﬁle for Figure 5.2.2

VX1 1 0 PULSE(0.5 4.5 0.02ns 0 0 0.01ns 0.04ns)
VX2 2 0 PULSE(0.5 4.5 0.01ns 0 0 0.01ns 0.02ns)
VT 100 O PULSE(1.0 4.5 0.01ns 0 0 0.02ns 0.04ns)

VRR 120 0 2.5

XMHl 3 1 4 2 5 MUL2

XNHl 5 9 101 NEUl

XMH2 6 1 7 2 8 MUL2

XNH2 8 10 103 NEUl

XMOl 11 9 12 10 13 MUL2

XNOl 13 14 NEU2

XL11 101 100 14 102 11 MUL11
XL12 103 100 14 104 12 MUL11
XL21 11 100 14 105 MUL12
XL24 12 100 14 106 MUL12
XL23 105 1 120 108 3 MUL11
XL22 105 2 120 107 4 MUL11
XL26 106 1 120 110 6 MUL11
XL25 106 2 120 109 7 MUL11
.SUBCKT MUL2 1 3 11 13 8

VSS 80 0 -5.0

V2 2 0 -3.0

V4 4 0 2.5

MGlX 1 4 6 80 N L=4OU W=4U
MDlX 2 3 6 80 N L-4OU W=4U
MGZX 2 4 5 80 N L=4OU W=4U
MDZX 1 3 5 80 N L-4OU W=4U
M61 11 4 6 80 N L-4OU W-4U
MD1 2 13 6 80 N L-4OU W=4U
MGZ 2 4 5 80 N L-4OU W=4U
MD2 11 13 5 80 N L-4OU W-4U

EMl 8 0 5 6 1E08

RI 5 6 100MEG
R1 8 6 140K
R2 5 0 140K
.ENDS MUL2

.SUBCKT NEUl 1 5 7

VDD 90 0 5.0
VR2 2 0 2.5
EVS1 3 2 1 0 1
MN1 4 3 0 0 N L-lOU W-4U
MP1 4 3 90 90 P L-4U W-lOU
MN2 5 4 0 0 N L=4U W-4U

MP2 5 4 90 90 P L-lOU W-lOU
VR 6 0 -4.0

EVS2 7 6 5 0 0.4

.ENDS NEUl

138

SUBCKT N202 1 5
VDD 90 0 5.0
VR2 2 0 2.5
EVS 3 2 1 0 1
MN1 4 3 0 0 N L-4U W-lOU
MP1 4 3 90 90 P L-4U W-4U
MN2 5 4 0 0 N L-4U W-lOU
MP2 5 4 90 90 P L-4U W=4U
.ENDS NEUZ

.SUBCKT MUL11 1 3 4 8 10
VSS 80 0 -5.0
VYC2 2 0 -3.0
MGlX 1 4 6 80 N L=4OU W=4U
MDlX 2 3 6 80 N L-40U W-4U
MGZX 2 4 5 80 N L-4OU W-4U
MD2X 1 3 5 80 N L-40U W=4U

RI 5 6 100MEG
EMl 7 0 5 6 1E08
R1 7 6 100K

R2 5 0 100K

RR 7 8 100

CC 8 0 1PF

EVS 10 2 8 0 1
.ENDS MUL11

.SUBCKT MUL12 1 3 4 10

VSS 80 0 -5.0

VYC2 2 0 -3.0

MGlX 1 4 6 80 N L=4OU W=4U
MDlX 2 3 6 80 N L=4OU W=4U
MG2X 2 4 5 80 N L-4OU W-4U
MD2X 1 3 5 80 N L=4OU W-4U
RI 5 6 lOOMEG

EMl 7 0 5 6 1E08

R1 7 6 100K

R2 5 0 100K

EVS 10 2 7 0 1

.ENDS MUL12

.MODEL N NMOS LEVEL-2.00000 LD=O.280000U TOX-500.000E-10
+NSUB-1.000000E+16 VTO-0.827125 KP-3.286649E-05
+GAMMA-1.35960 PHI-0.600000 UO=200.000 UEXP-1.001000E-03
+UCRIT-999000. DELTA-1.24050 VMAX-100000. XJ-0.400000U
+LAMBDA-1.604983E-02 NFS-1.234795E+12 NEFF=1.001000E-02
+NSS-0.000000E+00 TPG-1.00000 RSH=25 CGSO-S.ZE-10
+CGDO-5.ZB-10 CJ‘3.28-4 MJ-0.5 CJSW-9E-1O MJSW-0.33

.MODEL P PMOS LEVEL=2.00000 LD-0.2800000 TOX-500.000E-10
+NSUB-1.121088£+14 VTO--0.894654 KP-1.526452E-05
+GAMMA-0.879003 PHI-0.600000 UO-100.000 UEXP-0.153441
+UCRIT-16376.5 DELTA-1.93831 VMAX-100000. XJ-0.400000U
+LAMBDA-4.708659E-02 NFS-8.788617E+11 NEFF-1.0010003-02
+NSS-0.000000E+00 TPG--1.00000 RSH-95 CGSO-4E-10
+CGDO-4E-10 CJBZE-4 MJ‘0.5 CJSW-4.58-10 MJSW-0.33

.IC V(108)--0.5 V(107)-0.6 V(110)--0.5 V(109)-0.5
.IC V(102)-0.7 V(104)-O.5

.PROBE

.TRAN 0.1ns 1ns

.END

139

APPENDIX B.11

PSPICE input ﬁle for Table 5.2.3

VH1 301 0 4.5
VL1 302 0 4.5
VTT 303 0 0.5
VR1 304 0 4.5

***FF Circuits***

XFFl 301 302 3 4 5 6 11 12 1 2 13 8 9 7 FFXR
***Learning Circuits***

XL11 11 303 13 121 101 1 MUL11

XL12 12 303 13 122 102 2 MUL11

XLTl 304 303 13 127 107 7 MUL11

XC11 1 303 13 131 MUL13
XC12 2 303 13 132 MUL13

XI11 131 301 123 103 3 MUL12
XI12 131 302 124 104 4 MUL12
XI21 132 301 125 105 5 MUL12
X122 132 302 126 106 6 MUL12
XTH12 131 304 128 108 8 MUL12
XTHZZ 132 304 129 109 9 MUL12

.SUBCKT FFXR 1 2 3 4 6 7 9 10 11 12 14 16 17 18
XMHl 3 1 4 2 16 5 MUL3

XNHl 5 9 NEUl

XMHZ 6 1 7 2 17 8 MUL3

XNHZ 8 10 NEUl

XMOl 11 9 12 10 18 13 MUL3

XNOl 13 14 NEUl

.ENDS FFXR

.SUBCKT MUL3 1 3 11 13 21 8
VSS 80 0 -5.0

V2 2 0 -3.0

V4 4 0 2.5

V23 23 0 3.5

MG11 1 4 6 80 N L=4OOU W=4U
MD11 2 3 6 80 N L-400U W=4U
MG12 2 4 5 80 N L=4OOU W=4U
MD12 1 3 5 80 N L=4OOU W=4U
MG21 11 4 6 80 N L=4OOU W=4U
MD21 2 13 6 80 N L=4OOU W=4U
MG22 2 4 5 80 N L-4OOU W-4U
MD22 11 13 5 80 N L-4OOU W-4U

140

MG31 21 4 6 80 N L-4OOU W-4U
MD31 2 23 6 80 N L-400U W-4U
MG32 2 4 5 80 N L-4OOU W-4U
MD32 21 23 5 80 N L-400U W=4U

EMl 8 0 5 6 1308
RI 5 6 100MEG

R1 8 6 400K

R2 5 0 400K
.ENDS MUL3

.SUBCKT NEUl 1 5

VDD

90 0

5.0

VR2 2 0 2.5
BVSl 3 2 1 0 1

MN1 4 3 0 0
MP1 4 3 90 90

MN2 5 4 0

MP2 5 4 90 90

N L=2U W=2U
P L=2U W=4U
0 N L=2U W=2U
P L=2U W=4U

.ENDS NEUl

.SUBCKT MUL12

11 3 8 10 12

VSS 80 O -5.0

VSS1 50 0 -1.0

VDD1 60 0 1.0

VYC2 2 0 -3.0

VXC2 4 0 2.5

EVXl 1 2 11 0 1

MGlX 1 4 6 80 N L=4OU W=4U
MDlX 2 3 6 80 N L=4OU W=4U
MGZX 2 4 5 80 N L=4OU W=4U
MD2X 1 3 5 80 N L=4OU W=4U
RI 5 6 lOOMEG

EMl 7 0 5 6 1E08
R1 7 6 100K
R2 5 0 100K
R00 7 8 100

C8 8 0 lPF

9 8 50 50 N L=1OU W=4U
9 8 60 60 P L-4U W-10U
10 9 50 50 N L-lOU W=4U
MP2 10 9 60 60 P L-4U W=1OU
BVS 12 2 10 0 1

.ENDS MUL12

MN1
MP1
MN2

.SUBCKT MUL11 101 3 4 8 10 12

VSS 80 0

VSSl
VDD1
VYC2
VCCl
EVXl

50 0
6O 0
2 0
20 0
1 20

MG11X 1 4
MD11X 2 3
MGlZX 2 4
MD12X 1 3

-5.0
-1.0
1.0
-3.0
-4.0
101 0 0.4

6 80 N L-4OU W-4U
6 80 N L-4OU W=4U
5 80 N L-40U W-4U
5 80 N L-40U W=4U

141

R1 7 6 100K

R2 5 0 100K

EVOP 7 0 5 6 1208
RIN 5 6 100MEG
ROU 7 8 100

C8 8 0 lPF

MN1 9 8 50 50 N L-lOU W-4U
MP1 9 8 60 60 P L=4U W=10U
MN2 10 9 50 50 N L=10U W=4U
MP2 10 9 60 60 P L-4U W-lOU
EVS 12 2 10 0 1

.ENDS MUL11

.SUBCKT MUL13 1 3 4 7
VSS 80 0 -5.0
VYC2 2 O -3.0

MGllX 1 4 6 80 N L=4OU W=4U
MD11X 2 3 6 80 N L=4OU W=4U
MG12X 2 4 5 80 N L=4OU W=4U
MD12X 1 3 5 80 N L=4OU W=4U

R1 7 6 100K

R2 5 0 100K

EVOP 7 O 5 6 1308
RIN 5 6 lOOMEG
.ENDS MUL13

.MODEL N NMOS LEVEL=2.00000 LD=O.280000U TOX=500.000E-10
+NSUB=1.000000E+16 VTO=0.827125 KP=3.286649E-05
+GAMMA-1.35960 PHI-0.600000 UO=200.000 UEXP=1.001000E-03
+UCRIT-999000. DELTA=1.24050 VMAX=100000. XJ=0.400000U
+LAMBDA-1.604983£-02 NFS=1.234795E+12 NEFF=1.001000E-02
+NSS-0.000000E+00 TPG-1.00000 RSH=25 CGSO=5.2E-10
+CGDO-5.ZE-10 CJ=3.2E-4 MJ-O.5 CJSW=9E-10 MJSW=O.33

.MODEL P PMOS LEVEL-2.00000 LD=O.280000U TOX=500.000E-10
+NSUB-1.121088£+14 VTO--0.894654 KP=1.526452E-05
+GAMMA-0.879003 PHI-0.600000 UO=100.000 UEXP=O.153441
+UCRIT-16376.5 DELTA-1.93831 VMAX=100000. XJ=0.4000OOU
+LAMBDA-4.708659E-02 NFS=8.788617E+11 NEFF=1.001000E-02
+NSS-0.000000E+00 TPG--1.00000 RSH=95 CGSO=4E-10
+CGDO=4E-10 CJ-2E-4 MJ=O.5 CJSW=4.5E-10 MJSW=O.33

.IC V(123)-3.4lZE-03 V(124)-3.412E-03 V(125)=2.516E-04 V(126)=3.407E-04

.IC V(121)-3.4loE-03 V(122)-3.411E-03
.IC V(128)-1.215£-03 V(129)=9.873E-05 V(127)=—4.714E-03

.print tran v(ll) v(12) v(13) v(l)

.print tran V(2) v(3) v(4) v(S) V(6)

.print tran v(108) v(109) v(107) v(102)

.print tran v(103) v(104) v(105) v(106) V(101)
.print tran v(123) v(124) v(125) v(126) v(121)
.print tran v(128) v(129) v(127) v(122)

.PROBE

.OPTIONS ITL4-40 RELTOL=0.01 ITL5=0

.TRAN 0.2ns 3.0ns

.END

142

APPENDIX B.12

PSPICE input ﬁle for Table 5.3.1

VH1 301 0 4.5
VL1 302 O 1.0
VL2 303 O 1.0
***FF Circuits***

11 12 1 2 13 8 9 7 FFXR

XFFl 302 302 3 6
6 21 22 1 2 23 8 9 7 FFXR
6
6

4 5
XFFZ 301 302 3 4 5
XFF3 302 301 3 4 5
XFF4 301 301 3 4 5

31 32 1 2 33 8 9 7 FFXR
41 42 1 2 43 8 9 7 FFXR

***Learning Circuits***

XL11 11 21 31 41 303 301 301 303 13 23 33 43 121 101 1 MULP
XL12 12 22 32 42 303 301 301 303 13 23 33 43 122 102 2 MULP
XLTl 301 301 301 301 303 301 301 303 13 23 33 43 127 107 7 MULP

XL111 302 301 302 301 303 301 301 303 13 23 33 43 113 MULP2
XIll 113 101 123 103 3 MUL12
XL112 302 302 301 301 303 301 301 303 13 23 33 43 114 MULP2
XI12 114 101 124 104 4 MUL12
XL121 302 301 302 301 303 301 301 303 13 23 33 43 115 MULP2

XI21 115 102 125 105 5 MUL12
XL122 302 302 301 301 303 301 301 303 13 23 33 43 116 MULP2

XI22 116 102 126 106 6 MUL12

XLTll 301 301 301 301 303 301 301 303 13 23 33 43 118 MULP2

XTHll 118 101 128 108 8 MUL12
XLT22 301 301 301 301 303 301 301 303 13 23 33 43 119 MULP2

XTH22 119 102 129 109 9 MUL12

.SUBCKT FFXR 1 2 3 4 6 7 9 10 11 12 14 16 17 18

XMHl 3 1 4 2 16 5 MUL3

XNHl 5 9 NEUl

XMHZ 6 1 7 2 17 8 MUL3

XNHZ 8 10 NEUl

XMOl 11 9 12 10 18 13 MUL3

XNOl 13 14 NEUl

.ENDS FFXR

.SUBCKT MUL3 1 3 11 13 21 8

VSS 80 0 -5.0

V2 2 0 -3.0
5
5

V4 4 0 2.
V23 23 0 3.
MG11 1 4 6 80 N L-400U W-4U
M011 2 3 6 80 N Lt400U W=4U
MG12 2 4 5 80 N L-4OOU W=4U
MD12 1 3 5 80 N L=4OOU W=4U

143

MG21 11 4 6 80 N L=4OOU W=4U
MD21 2 13 6 80 N L=4OOU W=4U
M622 2 4 5 80 N L-4000 W=4U
MD22 11 13 5 80 N L=4OOU W-4U
MG31 21 4 6 80 N L-4OOU W=4U
MD31 2 23 6 80 N L=4OOU W=4U
MG32 2 4 5 80 N L-4000 W=4U
MD32 21 23 5 80 N L=4OOU W=4U
EMl 8 0 5 6 1E08

RI 5 6 100MEG

R1 8 6 400K

R2 5 0 400K

.ENDS MUL3

.SUBCKT NE01 l 5

VDD 90 0 5.0

VR2 2 0 2.5

EVSl 3 2 1 0 1

MN1 4 3 0 0 N L=20 W=20

MP1 4 3 90 90 P L=20 W=4U

MN2 5 4 0 0 N L=2U W=20

MP2 5 4 90 90

P L=20 W=4U

.ENDS NEUl

.SUBCKT MUL12

11 13 8 10 12

VSS 80 0 -5.0

VSSl 50 0 -1.0

VDDl 6O 0 1.0

VYC2 2 0 -3.0

VXC2 4 O 2.5

EVXl 1 2 11 0 1

EVYl 3 4 13 0 2.5

MGlX 1 4 6 80 N L=4OU W=4U
MDlX 2 3 6 80 N L=4OU W=4U
MG2X 2 4 5 80 N L=4OU W=4U
MD2X 1 3 5 80 N L=4OU W=4U
RI 5 6 lOOMEG

EMl 7 0 5 6 1E08

R1 7 6 100K

R2 5 0 100K

*EVAl 77 0 7 0 50

R00 7 8 100

C8 8 0 lPF

MN1 9 8 50 50 N L-lOU W=4U
MP1 9 8 60 60 P L=4U W=100
MN2 10 9 50 50 N L=100 W=4U
MP2 10 9 60 60 P L=4U W-100
EVS 12 2 10 0 1

.ENDS MUL12

.SUBCKT MULP 101 111 121 131
VSS 80 0 -5.0

VSSl 50 0 -1.0

VDD1 60 0 1.0

VYC2 2 0 -3.0

VCCl

20 0 -4.0

3 13 23 33 4 14 24 34 8 10 12

EVX1 1 20 101 0

EVXll
EVX21
EVX31

MGllX
MD11X
MGlZX
MD12X

MGZlX
MD21X
MG22X
MD22X

MG31X
MD31X
MG32X
MD32X
MG41X
MD41X
MG42X
MD42X

11 20 111
121

131

6 80 N
6 80 N
5 80 N
5 80 N
11 14 6 80
2 13 6 80

2 14 5 80
11 13 5 80

21 24 6 80
2 23 6 80
2 24 5 80
21 23 5 8O
31 34 6 80
2 33 6 80
2 34 5 80
31 33 5 80

R1 7 6 300K
R2 5 0 300K

EVOP 7 0 5 6

RIN 5 6 100MEG

.SUBCKT MULP2 101 111 121 131 3 13 23 33 4 14 24 34 7

1E08

0.4

0 0.4
O 0.4
0 0.4

144

L=4OOU W-4U
L=4OOU W-4U
L=4OOU W=4U
L=4OOU W=4U

2532525225252 223252

L=4OOU
L=4OOU
L=4OOU
L=4OOU

L=4OOU
L=4OOU
L=4OOU
L=4OOU
L=4OOU
L=4OOU
L=4OOU
L=4OOU

W=4U
W=4U
W=4U
W=4U

W=4U
W=4U
W=4U
W=4U
W=4U
W=4U
W=4U
W=4U

8 50 50 N L=100 W=4U
8 60 60 P L=4U W=100

0.4

0 0.4
0 0.4
O 0.4

10 9 50 50 N L=100 W=4U
10 9 60 60 P L-40 W-lOU

L=4OOU W=4U
L-4000 W=4U
L=4OOU W=4U
L=4OOU W-4U

N L=4OOU W=4U
N L=4OOU W-40
N L-4000 W-40

*EVAl 77 0 7 0 50
R00 7 8 100

C8 8 0 lPF

MN1 9

MP1 9

MN2

MP2

EVS 12 2 10 0 1
.ENDS MULP

VSS 80 0 -5.0
VYC2 2 0 -3.0
VCC1 20 0 -4.0
EVXl 1 20 101 0
BVXll 11 20 111
EVX21 21 20 121
EVX31 31 20 131
MGllX 1 4 6 80 N
MD11X 2 3 6 80 N
MG12X 2 4 5 80 N
MD12X 1 3 5 80 N
MG21X 11 14 6 80
MD21X 2 13 6 80
MGZZX 2 14 5 80
MD22X 11 13 5 80

N L-4000 W=4U

145

MG31X 21 24 6 80 N L=4OOU W840
MD31X 2 23 6 80 N L=4OOU W-4U
MG32X 2 24 5 80 N L=4OOU W-4U
MD32X 21 23 5 80 N L=4OOU W=4U
MG41X 31 34 6 80 N L=4OOU W=4U
MD41X 2 33 6 80 N L-4000 W=4U
MG42X 2 34 5 80 N L=4OOU W=4U
MD42X 31 33 5 80 N L=4OOU W=4U
R1 7 6 300K

R2 5 0 300K

EVOP 7 0 5 6 1308
RIN 5 6 lOOMEG
.ENDS MULP2

.MODEL N NMOS LEVEL-2.00000 LD=O.2800000 TOX=500.000E-10
+NSUB=1.000000E+16 VTO=0.827125 KP=3.286649E-05
+GAMMA=1.35960 PHI=0.600000 UO=200.000 UEXP=1.001000E-03
+UCRIT-999000. DBLTA=1.24050 VMAX=100000. XJ=0.4000OOU
+LAMBDA-1.604983E-02 NFS=1.23479SE+12 NEFF=1.001000E-02
+NSS-0.000000E+00 TPG=1.00000 RSH=25 CGSO=5.2E-1O
+CGDO=5.ZE-10 CJ=3.2E-4 MJ=O.5 CJSW=9E-1O MJSW=O.33

.MODEL P PMOS LEVEL=2.00000 LD=O.2800000 TOX=500.000E-10
+NSUB=1.121088£+14 VTO=-0.894654 KP=1.526452E-05
+GAMMA-0.879003 PHI=O.600000 UO=100.000 UEXP=O.153441
+UCRIT=16376.5 DELTA=1.93831 VMAX=100000. XJ=0.4000000
+LAMBDA=4.708659E-02 NFS=8.788617E+11 NEFF=1.001000E-02
+NSS-0.000000E+00 TPG=-1.00000 RSH=95 CGSO=4E-10
+CGDO=4E-10 CJ-ZE-4 MJ=O.5 CJSW=4.5E-1O MJSW=O.33

.IC V(123)-0.01 V(124)--0.01 V(125)=-0.01 V(126)=0.01
.IC V(121)-0.01 V(122)-0.01 V(128)=-0.01 V(129)=-0.01 V(127)=0.01

.print tran v(113) v(114) v(115) v(116)

.print tran v(ll) v(21) v(31) V(4l)

.print tran v(12) v(22) v(32) v(42)

.print tran v(123) v(124) v(125) v(126) v(121)
.print tran v(128) v(129) v(127) v(122)

.print tran v(103) v(104) v(105) v(106) V(101)
.print tran v(108) v(109) V(107) V(lOZ)

.print tran v(13) v(23) v(33) V(43)

.print tran v(118) v(119)

.PROBE

.OPTIONS ITL4-40 RELTOL=0.01 ITL5=O

.TRAN 0.4ns 8.0ns

.END

146

APPENDIX B.13

PSPICE input ﬁle for Table 5.3.2

VHl 301 o 4.5
VL1 302 o 1.0
VL2 303 o 1.0
***FF Circuits***

XFFl 302 302 3 4 5 6 11 12 1 2 13 8 9 7 FFXR
XFFZ 301 302 3 4 5 6 21 22 1 2 23 8 9 7 FFXR
XFF3 302 301 3 4 5 6 31 32 1 2 33 8 9 7 FFXR
XFF4 301 301 3 4 5 6 41 42 1 2 43 8 9 7 FFXR

***Learning Circuits***

XL11 11 21 31 41 303 301 301 303 13 23 33 43 121 101 1 MULP
XL12 12 22 32 42 303 301 301 303 13 23 33 43 122 102 2 MULP
XLTl 301 301 301 301 303 301 301 303 13 23 33 43 127 107 7 MULP

XL111 302 301 302 301 303 301 301 303 13 23 33 43 113 MULP2
X111 113 101 123 103 3 MUL12
XL112 302 302 301 301 303 301 301 303 13 23 33 43 114 MULP2
X112 114 101 124 104 4 MUL12
XL121 302 301 302 301 303 301 301 303 13 23 33 43 115 MULP2
X121 115 102 125 105 5 MUL12
XL122 302 302 301 301 303 301 301 303 13 23 33 43 116 MULP2
X122 116 102 126 106 6 MUL12

XLTll 301 301 301 301 303 301 301 303 13 23 33 43 118 MULP2
XTH11 118 101 128 108 8 MUL12
XLT22 301 301 301 301 303 301 301 303 13 23 33 43 119 MULP2
XTH22 119 102 129 109 9 MUL12

.SUBCKT FFXR 1 2 3 4 6 7 9 10 11 12 14 16 17 18
XMHl 3 1 4 2 16 5 MUL3

XNHl 5 9 NE01

XMHZ 6 1 7 2 17 8 MUL3

XNHZ 8 10 NEUl

XMOl 11 9 12 10 18 13 MUL3

XNOl 13 14 NEUl

.ENDS FFXR

.SUBCKT MUL3 1 3 11 13 21 8
VSS 80 0 -5.0

V2 2 0 -3.0

V4 4 0 2.5

V23 23 0 3.5

MG11 1 4 6 80 N L-4000 W=4U
MD11 2 3 6 80 N L-4000 W-4U
MG12 2 4 5 80 N L-4000 W=4U
MD12 1 3 5 80 N L-4000 W=4U
MG21 11 4 6 80 N L-4000 W=4U
MD21 2 13 6 80 N L-4000 W-4U
MG22 2 4 5 80 N L-4000 W=4U
MD22 11 13 5 80 N L-4000 W-4U

147

MG31 21 4 6 80 N L=4OOU W=4U
MD31 2 23 6 80 N L-4OOU W=4U
M632 2 4 5 80 N L-4000 W=4U
MD32 21 23 5 80 N L-4000 W=4U

EMl 8 O 5 6 1E08

RI 5

R1 8
R2 5
.END

6 lOOMEG

6 400K
0 400K
3 MUL3

.SUBCKT NEUl 1 5

VDD
VR2
BVSl
MN1
MP1
MN2
MP2
.END

.SUBCKT MUL12

VSS

VSSl
VDD1
VYC2
VXC2
EVXl
EVYl
MGlX
MDlX
MGZX
MDZX
RI 5
EMl

R1 7
R2 5

90 0 5.0
2 0 2.5

3 2 1 0 1
4 3 0 0
4 3 90 90
5 4 0 0
5 4 90 90
S NEUl

N L=20 W=20
P L-20 W=4U
N L-ZU W-20
P L=20 W=4U

11 13 8 10 12
80 0 -5.0
50 0 -1.0
60 0 1.0
-3.0
2.5
11 O
13 0
6 80
6 80
5 80
5 80
100MEG
7 0 5 6 1E08
6 100K
0 100K

.5

L=4OU W=4U
L'4OU W=4U
L-4OU W=4U
L=4OU W-4U

O‘HBONDHCUF‘bPO
asotucsarocao

*EVAl 77 0 7 0 50

R00
C8

MN1
MP1
MN2
MP2
EVS

7 8 100
8 0 1PF

9 8 50 50 N L-lOU W-40
9 8 60 60 P L=4U W-lOU
10 9 50 50 N L-100 W=4U
10 9 60 60 P L-40 W-lOU
12 2 10 0 1

.ENDS MUL12

.SUBCKT MULP 101 111 121 131 3 13 23 33 4 14 24 34 8 10 12

VSS

VSS1
VDDl
VYC2
VCC1
EVXl
EVX1
EVX2
EVX3

80 0 -5.0
50 0 -1.0
60 0 1.0
2 0 -3.0
20 0 -4.0
1 20 101 0

1 11 20 111

1 21 20 121

0
0
0
1 31 20 131 0

4

0 4
0.4
0 4

MGllX 1 4 6 80 N L-4000 W-40
MD11X 2 3 6 80 N L=4OOU W=4U
MGlZX 2 4 5 80 N L=4OOU W=4U
MD12X 1 3 5 80 N L=4OOU W=4U
MGZlX 11 14 6 80 N L-4000 W-4U
MD21X 2 l3 6 80 N L-4000 W-4U
MGZZX 2 14 5 80 N L-4OOU W-4U
MD22X 11 13 5 80 N L-4000 W=4U
MG31X 21 24 6 80 N L=4OOU W=4U
MD31X 2 23 6 80 N L=4OOU W-40
MG32X 2 24 5 80 N L-4OOU W-4U
MD32X 21 23 5 80 N L=4OOU W=4U
MG41X 31 34 6 80 N L-4000 W=4U
MD41X 2 33 6 80 N L-4000 W=4U
MG42X 2 34 5 80 N L-4000 W=4U
MD42X 31 33 5 80 N L=4OOU W=4U
R1 7 6 300K

R2 5 0 300K

EVOP 7 0 5 6 1E08

RIN 5 6 lOOMEG

*EVAl 77 0 7 0 50

R00 7 8 100

C8 8 0 1PF

MN1 9 8 50 50 N L-lOU W=4U

MP1 9 8 60 60 P L=4U W=100

MN2 10 9 50 50 N L=1OU W=4U
MP2 10 9 60 60 P L=4U W=100
EVS 12 2 10 0 1

.ENDS MULP

.SUBCKT MULP2 101 111 121 131 3 13 23 33 4 14 24 34 7
VSS 80 0 -5.0

VYC2 2 0 -3.0

VCC1 20 0 -4.0

EVXl 1 20 101 0 0.4

EVX11 11 20 111 0 0.4

EVX21 21 20 121 0 0.4

EVX31 31 20 131 0 0.4

MGllX 1 4 6 80 N L=4OOU W=4U
MD11X 2 3 6 80 N L=4OOU W=4U
MG12X 2 4 5 80 N L=4OOU W=4U
MD12X 1 3 5 80 N L=4OOU W=4U
MG21X 11 14 6 80 N L=4OOU W=4U
MD21X 2 13 6 80 N L-4000 W=4U
MG22X 2 l4 5 80 N L-4OOU W-4U
MDZZX 11 13 5 80 N L=4OOU W=4U
MG31X 21 24 6 80 N L-4OOU W-4U
MD31X 2 23 6 80 N L-4000 W=4U
MG32X 2 24 5 80 N L-4000 W=4U
MD32X 21 23 5 80 N L-4000 W=4U

148

149

MG41X 31 34 6 80 N L=4OOU W=4U
MD41X 2 33 6 80 N L-4000 W=4U
MG42X 2 34 5 80 N L=4OOU W=4U
MD42X 31 33 5 80 N L=4OOU W=4U

R1 7 6 300K

R2 5 0 300K

EVOP 7 O 5 6 1E08
RIN 5 6 lOOMEG
.ENDS MULP2

.MODEL N NMOS LEVEL=2.00000 LD=O.2800000 TOX=500.000E-10
+NSUB-1.000000E+16 VTO=0.827125 KP=3.286649E-05
+GAMMA=1.35960 PHI=0.600000 UO=200.000 UEXP=1.001000E-03
+0CRIT=999000. DELTA=1.24050 VMAX=100000. XJ=0.4000000
+LAMBDA-1.604983E-02 NFS=1.234795E+12 NEFF=1.001000E-02
+NSS-0.000000E+OO TPG=1.00000 RSH=25 CGSO=5.ZE-10
+CGDO-5.ZE-1O CJ-3.ZB-4 MJ=O.5 CJSW=9E-10 MJSW=O.33

.MODEL P PMOS LEVEL=2.00000 LD=O.2800000 TOX=500.000E-10
+NSUB-1.121088£+14 VTO=-O.894654 KP=1.526452E-05

+GAMMA-0.879003 PHI=0.600000 UO=100.000 UEXP=O.153441
+0CRIT-l6376.5 DELTA=1.93831 VMAX=1000OO. XJ=0.4000000
+LAMBDA-4.708659E-02 NFS=8.788617E+11 NEFF=1.001000E-02
+NSS-0.000000£+00 TPG=-1.00000 RSH=95 CGSO=4E-10
+CGDO-4E-10 CJ-ZE-4 MJ=O.5 CJSW=4.5E-1O MJSW=O.33

.IC V(123)-0.01 V(124)--0.01 V(125)=-0.01 V(126)=0.01
.IC V(121)-0.01 V(122)=0.01 V(128)=-0.01 V(129)=-0.01 V(127)=0.01

.print tran v(113) v(114) v(115) v(116)

.print tran v(11) v(21) v(31) v(41)

.print tran v(12) v(22) v(32) v(42)

.print tran v(123) v(124) v(125) v(126) V(121)
.print tran v(128) v(129) v(127) v(122)

.print tran v(103) v(104) v(105) v(106) V(101)
.print tran v(108) v(109) v(107) v(102)

.print tran v(13) v(23) v(33) V(43)

.print tran v(118) v(119)

*.PROBE

.OPTIONS ITL4-40 RELTOL=0.01 ITL5=0

.TRAN 0.4ns 8.0ns

.END

150

APPENDIX B.14

PSPICE input ﬁle for Table 6. 1. 1 (a)

VHl 301 0 4.5
VL1 302 0 4.5
VTT 303 0 0.5
VR1 304 0 4.5
***FF Circuits***

XFFl 301 302 3 4 5 6 13 14 l 2 11 12 15 16 8 9 7 10 FFXR

***Learning Circuits***

XL11 13 303 15 121 101 1 MUL11
XL12 14 303 15 122 102 2 MUL11
XL21 13 303 16 131 111 11 MUL11
XL22 14 303 16 132 112 12 MUL11

XLTl 304 303 15 127 107 7 MUL11
XLT2 304 303 16 137 117 10 MUL11

XC11 1 303 15 2 303 16 141 MUL2
XC12 11 303 15 12 303 16 142 MUL2

MUL12
MUL12
MUL12
MUL12

X111 141 301 123 103
X112 141 302 124 104
X121 142 301 125 105
X122 142 302 126 106

mount»

MUL12
MUL12

XTH12 141 304 128 108
XTHZZ 142 304 129 109

£000

.SUBCKT FFXR 1 2 3 4 6 7 9 10 11 12 15 16 14 18 21 22 23 24
XMHl 3 1 4 2 21 5 MUL3

XNHl 5 9 NBUl

XMH2 6 1 7 2 22 8 MUL3

XNHZ 8 10 NEUl

XMOl 11 9 12 10 23 13 MUL3

XN01 13 14 NE01

XMOZ 15 9 16 10 24 17 MUL3

XN02 17 18 NEUl

.ENDS FFXR

.SUBCKT MUL3 1 3 11 13 21 8
VSS 80 0 -5.0

V2 2 0 -3.0

V4 4 0 2.5

V23 23 0 3.5

M611 1 4 6 80 N L-4000 W=4U
MD11 2 3 6 80 N L-4000 W-40
M612 2 4 5 80 N L-4000 W-4U
MD12 1 3 5 80 N L=4OOU W-4U

M621

MD21 2 13

M622
MD22

MG31
MD31
M632
MD32

EMl 8 0 5 6 1E08
RI 5 6 100MEG

R1 8 6 400K
R2 5 0 400K
.ENDS MUL3
.SUBCKT MUL2
VSS 80 0 -5.
V2 2 0 -3
M611 1 4 6
MD11 2 3 6
M612 2 4 5
MD12 1 3 5
M621 11 4 6
MD21 2 13 6
M622 2 4 5
MD22 11 13 5

EMl 8 0 5 6 1E08
RI 5 6 lOOMEG

R1 8 6 60K

R2 5 0 60K
.ENDS MUL2
.SUBCKT NEUl 1 5
VDD 90 0 5.0
VR2 2 O 2.5
EVSl 3 2 1 0 1
MN1 4 3 0 0
MP1 4 3 90 90
MN2 5 4 0 0
MP2 5 4 90 90
.ENDS NE01
.SUBCKT MUL12
VSS 80 0 -5.0
VSSl 50 0 -1.0
VDDl 60 0 1.0
VYC2 2 0 -3.0
VXC2 4 0 2.5
EVXl 1 2 11 0 1
MGlX 1 4

MDlX 2 3

MGZX 2 4

MD2X 1 3

11 4

2 4
11 1

21 4
2 2
2 4

21 23

6
6
5
3 5
6
3 6
5
5

.0

80 N L=4OOU W=4U
80 N L=4OOU W=4U
80 N L=4OOU W=4U
80 N L=4OOU W=4U
80 N L=4OOU W=4U
80 N L=4OOU W=4U
80 N L-4OOU W=4U
80 N L=4OOU W=4U
1 3 4 11 13 14

0

80 N L=4OU W=4U
80 N L=4OU W=4U
80 N L-400 W=4U
80 N L=4OU W=4U
80 N L=4OU W=4U
80 N L=4OU W=4U
80 N L=4OU W=4U
80 N L=4OU W=4U

N L-ZU W=20
P L=20 W=4U
N L-20 W-ZU
P L=20 W=4U

11 3 8 10 12

6 80 N L-400 W=4U
6 80 N L-400 W=4U
5 80 N L=4OU W=4U
5 80 N L-400 W=4U

151

152

R1 5 6 100ME6

EMl 7 0 5 6 1E08
R1 7 6 100K

R2 5 0 100K

*EVAl 77 0 7 0 50
R00 7 8 100

C8 8 0 1PF

MN1 9 8 50 50 N L=1OU W=4U
MP1 9 8 60 60 P L=4U w=1ou
MN2 10 9 50 50 N L-lOU W=4U
MP2 10 9 60 60 P L=4U W=100
EVS 12 2 10 0 1

.ENDS MUL12

.SUBCKT MUL11 101 3 4 8 10 12
VSS 80 0 -5.0

VSSl 50 0 -1.0

VDD1 60 0 1.0

VYCZ 2 0 -3.0

VCCl 20 0 -4.0

EVXl 1 20 101 0 0.4

MGllX 1 4 6 80 N L=4OU W=4U
MD11X 2 3 6 80 N L-400 W=4U
M612X 2 4 5 80 N L=4OU W=4U
MD12X 1 3 5 80 N L=4OU W=4U

R1 7 6 100K

R2 5 0 100K

EVOP 7 0 5 6 1E08
RIN 5 6 100MEG
*EVAl 77 0 7 O 50
R00 7 8 100

C8 8 O 1PF

MN1 9 8 50 50 N L=1OU W=4U
MP1 9 8 60 60 P L=4U W=100
MN2 10 9 50 50 N L-lOU W=4U
MP2 10 9 60 60 P L-40 W=100
EVS 12 2 10 0 1

.ENDS MUL11

.SUBCKT MUL13 1 3 4 7
VSS 80 0 -5.0
VYC2 2 0 -3.0

MGllX 1 4 6 80 N L=4OU W=4U
MDllX 2 3 6 80 N L=4OU W=4U
M612X 2 4 5 80 N L-400 W=4U
MD12X 1 3 5 80 N L-400 W=4U

R1 7 6 100K

R2 5 0 100K

EVOP 7 0 5 6 1E08
RIN 5 6 IOOMEG
.ENDS MUL13

153

.MODEL N NMOS LEVEL-2.00000 LD=O.2800000 TOX=500.000E-10
+NSOB-1.000000E+16 VTO=0.827125 KP=3.286649E-05
+6AMMA-1.35960 PHI=0.600000 UO=200.000 UEXP=1.001000E-03
+0CR1T-999000. DELTA-1.24050 VMAX=100000. XJ=0.4000000
+LAMBDA-1.604983E-02 NFS-1.234795E+12 NEFF=1.001000E-02
+NSS-0.000000E+OO TPG-1.00000 RSH=25 CGSO=5.ZE-10
+CGDO-5.ZE-10 CJ-3.ZE-4 MJ-0.5 CJSW=9E-10 MJSW=O.33

.MODEL P PMOS LEVEL-2.00000 LD=O.2800000 TOX=500.000E-10
+NSUB-1.121088E+14 VTO--0.894654 KP=1.526452E-05
+GAMMA-0.879003 PHI-0.600000 UO=100.000 UEXP=O.153441
+0CRIT-16376.5 DELTA=1.93831 VMAX=100000. XJ=0.4000000
+LAMBDA-4.708659E-02 NFS=8.788617E+11 NEFF=1.001000E-02
+NSS-0.000000£+00 TPG=-1.00000 RSH=95 CGSO=4E-10
+CGDO=4E-10 CJ-2E-4 MJ=O.5 CJSW=4.5E-10 MJSW=O.33

.IC V(123)-0.5 V(124)--0.5 V(125)=-0.5 V(126)=0.5
.IC V(121)-0.5 V(122)-0.5 V(131)=0.5 V(132)=0.5
.IC V(128)--0.5 V(129)--0.5 V(127)=0.5 V(137)=0.5

v(13) v(14) v(15) v(16)
v(l) V(2) v(11) v(12)
V(3) V(4) V(S) v(6)
v(8) v(9) v(7) V(10)

tran
tran
tran
tran

.print
.print
.print
.print

.print
.print
.print

.print
.print
.print
.PROBE

.OPTIONS ITL4-40 RELTOL=0.

tran
tran
tran

tran
tran
tran

V(101)
v(103)
v(121)
v(123)
v(108)
v(128)

.TRAN 0.2ns 3.0ns

.END

v(102)
v(104)
v(122)
v(124)
v(109)
v(129)

v(lll)
v(105)
v(131)
v(125)
v(107)
v(127)

v(112)
v(106)
v(132)
v(126)
v(117f
v(137)

01 ITL5=0

154

APPENDIX B.15

PSPICE input ﬁle for Table 6.1.l(c)

V1 1 0

V2 2 0

VWHll 3 0 -4.0

VWH12 4 0 -4.0

VWHZl 6 0 -4.0

VWHZZ 7 0 -4.0

VW011 11 0 -3.1396
VWOlZ 12 0 -3 1396
VW021 15 0 -3.1396
VW022 16 0 -3 1396
VTHl 21 0 -4.0

VTHZ 22 0 -4.0

VTOl 23 0 -4.0

VTOZ 24 0 -4.0

XMHl 3 1 4 2 21 5 MUL3
XNHl 5 9 N301

XMHZ 6 1 7 2 22 8 MUL3
XNH2 8 10 N301

XMOl 11 9 12 10 23 13 MUL3
XNOl 13 14 N301

XMOZ 15 9 16 10 24 17 MUL3
XN02 17 18 N301

.SUBCKT MUL3 l 3 11 13 21 8
VSS 80 0 -5.0
V2 2 0 -3.0
V4 4 0 2.5

V23 23 O .5

M611 1 4 6 80 N L=4OOU W=4U
MD11 2 3 6 80 N L=4OOU W=4U
M612 2 4 5 80 N L=4OOU W=4U
MD12 1 3 5 80 N L=4OOU W=4U
M621 11 4 6 80 N L=4OOU W=4U
MD21 2 13 6 80 N L=4OOU W=4U
M622 2 4 5 80 N L=4OOU W=4U
MD22 11 13 5 80 N L-4OOU W=4U
MG31 21 4 6 80 N L-4000 W=4U
MD31 2 23 6 80 N L=4OOU W=4U
M632 2 4 5 80 N L'4000 W=4U
MD32 21 23 5 80 N L=4OOU W=4U

3M1 8 O 5 6 1308
R1 5 6 100M36

R1 8 6 400K

R2 5 0 400K
.ENDS MUL3

155

.SUBCKT N301 1 5

VDD 90 0 5.0

VR2 2 0 2.5

EVSI 3 2 1 0 1

MN1 4 3 0 0 N L-ZU W=20
MP1 4 3 90 90 P L-ZU w-4U
MN2 5 4 O 0 N L=20 W=20
MP2 5 4 90 90 P L-20 W-4U
.3NDS N301

.MODEL N NMOS LEVEL-2.00000 LD=O.2800000 TOX=500.0003-10
+NSUB-1.0000003+16 VTO-0.827125 KP=3.2866493-05
+6AMMA-l.35960 PHI=O.600000 UO=200.000 UEXP=1.0010003-03
+0CRIT-999000. D3LTA=1.24050 VMAX=100000. XJ=0.4000000
+LAMBDA-1.6049833-02 NFS=1.23479SE+12 N3FF=1.0010003-02
+NSS-0.0000003+00 TPG=1.00000 RSH=25 CGSO=5.23-10
+CGDO-5.23-10 CJ-3.23-4 MJ=O.5 CJSW=93-10 MJSW=O.33
.MODEL P PMOS L3V3L=2.00000 LD=O.2800000 TOX=500.0003-10
+NSUB-1.1210883+14 VTO=-0.894654 KP=1.5264523-05
+GAMMA-0.879003 PHIBO.600000 UO=100.000 UEXP=O.153441
+UCRIT-16376.5 D3LTA=1.93831 VMAX=100000. XJ=0.4000000
+LAMBDA-4.7086593-02 NFS=8.7886173+11 N3FF=1.0010003-02
+NSS=0.0000003+00 TPG=-1.00000 RSH=95 CGSO=43-10
+CGDO=43-10 CJ-23-4 MJ=O.5 CJSW=4.53-10 MJSW=O.33

.print dc v(9) V(10) v(14) v(18)
.PROBE

.OPTIONS ITL4-40 R3LTOL=0.01 ITL5=0
.DC V1 1.0 4.5 3.5 V2 1.0 4.5 3.5
.3ND

156

LIST OF REFERENCES

[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

W. Hubbard, J. S. Denker, H. P. Graf, R. E. Howard, L. D. Jackel, B. Straughn,
and D. Schwartz, "Electronic Neural Networks", AIP Neural Networks for Com-
puting, pp. 227-234, 1986.

M. Sivilotti, M. Emerling, and C. Mead, "A Novel Associative Memory Imple-
mented Using Collective Computation", 1985 Chapel Hill Conference on VLSI,
pp. 329-342.

M. Siviloni, M. Emerling, and C. Mead, "VLSI Architecture for implementation
of Neural Networ ", AIP Neural Networks for Computing, pp. 408-413, 1986.

I. P. Sage, K. Thompson, and R. S. Withers, "An Artiﬁcial Neural Network
Integrated Circuit Based on MNOS/CCD Principles", AIP Neural Networks for
Computing, pp. 381-385, 1986.

A. P. Thakoor, A. M00pen, J. Lambe, and S. K. Khanna, "Electronic hardware
implementations of neural networ ", Applied Optics, Vol. 26, pp. 5085-5092.

F. Kub, I. Mack, K Moon, C. Yao, and J. Modolo, "Programmable analog
synapses for microelectronic neural networks using a hybrid digital-analog

approach", poster presented at IEEE ICNN, San Diego, July 24-27, 1988.
F. Kub, K. Moon, I. Mack, and F. M. Long, "Programmable Analog Vector-
Matrix Multipliers", IEEE J. Solid-State Circuits, 80-25, pp. 207-214, Feb. 1990.

S. Eberhardt, T. Duong, and A. Thakoor, "Design of Parallel Hardware Neural
Network Systems from Custom Analog VLSI ‘Building Block’ Chips", UCNN
1989. PP. II-183 - II-190.

B. Furman and A. A. Abidi, "An Analog CMOS Backward Error-Propagation
LSI", Asilomar Conf. on Signal, Control, and Systems, pp. 645-648, 1988.

157

[10] H. P. Graf and P. deVegvar, "A CMOS Implementation of a Neural Network
Model", Proc. Stanford Confer. Advanced Research in VLSI, MIT Press, pp.
351-367, 1987.

[11] J. J. Hopﬁeld, "Neural networks and physical systems with emergent collective

computational abilities", Proc. Natl. Acad. Sci. U.S.A., vol. 79, pp. 2554-2558,
1982.

[12] J. J. Hapﬁeld, "Neurons with graded response have collective computational pro-
perties like those of two-state neurons", Proc. Natl. Acad. Sci. U.S.A., vol. 81,
pp. 3088-3092, 1984.

[13] J. J. Hopﬁeld and D. W. Tank, "Simple neural optimization networks: an AID
converter, a signal decision circuit, and a linear programming circuit", IEEE
Trans. on Circuits and Systems, vol. GAS-33, no.5, May 1986, pp. 533—541.

[14] M. Banu and Y. Tsividis, "Fully-integrated active-RC ﬁlters in MOS technology",
IEEE J. Solid-State Circuits, SC-18, pp. 644-651, Dec. 1983.

[15] Y. Tsividis, M. Banu, and J. Khoury, "Continuous-time MOSFET-C Filters in
VLSI", IEEE Trans. on Circuits and Systems, vol. GAS-33, pp. 125-140, Feb.
1986.

[16] Z. Czarnul, "Modiﬁcation of the Banu-Tsividis continuous-time integrator struc-
ture", IEEE Trans. on Circuits and Systems, vol. GAS-33, pp. 714-716, July 1986.

[17] B.-S. Song, "CMOS RF Circuits for Data Communications Applications", IEEE J.
Solid-State Circuits, SC-21, no. 2, pp. 310-317, April 1986.

[18] P. R. Gray and R. G. Meyer, "MOS operational ampliﬁer design - A tutorial
overview", IEEE J. Solid-State Circuits, SC-17, pp. 969-982, Dec. 1982.

[19] B. Gilbert, "A high-performance monolithic multiplier using active feedback",
IEEE J. Solid-State Circuits, SC-9, pp. 364-373, Dec. 1974.

158

[20] D. 800 and R. G. Meyer, "A four-quadrant NMOS analog multiplier", IEEE J.
Solid-State Circuits, SC- 17, pp. 1174-1178, Dec. 1982.

[21] J. N. Babanezhad and G. C. Temes, "A 20-V Four-Quadrant CMOS Analog Mul-
tiplier", IEEE J. Solid-State Circuits, 8020, pp. 1158-1168, Dec. 1985.

[22] P. J. Ryan and D. G. Haigh, "Novel Fully Differential MOS Transconductor for
Integrated Continuous-time ﬁlters", Elect. Lett., pp. 742-743, June 1987.

[23] M. Ismail, "Four-transistor continuous-time MOS transconductor", Electron. Lett.,
vol. 23, no. 20, pp. 1099-1100, Sept. 1987.

[24] N. I. Khachab and M. Ismail," Novel Continuous-Time All MOS Four Quadrant
Multipliers", IEEE Int. Symp. Circuits and Systems, May 1987, pp. 762-765.

[25] M. VanHorn and R. L. Geiger, "A CMOS OTA for Voltage-Controlled Analog
Signal Processing", Proc. 28th Midwest Symposium on Circuits and Systems,
Louisville, KY, pp. 596—599, Aug. 1985.

[26] S.-C. Qin and R. L. Geiger, "A :tS-V CMOS Analog Multiplier", IEEE J. Solid-
State Circuits, SC-22, no. 6, pp. 1143-1146, Dec. 1987.

[27] F. M. A. Salam, N. Khachab, M. Ismail, and Y. Wang, "An Analog MOS Imple-
mentation of The Synaptic Weights For Feedback Neural Nets," IEEE Int. Symp.
on Circuits and Systems, Portland, Oregon, CA, May 1989, pp. 1223-1226.

[28] F. M. A. Salam, M. R. Choi, and Y. Wang, "An Analog MOS Implementation of
Synaptic Weights for Feedforward/Feedback Neural Nets", Proc. of 32nd Midwest
Symposium on Circuits and Systems, Champaign, Illinois, August, 1989.

[29] F. M. A. Salam and M. R. Choi, "An All-MOS Analog Fwdforward Neural Cir-
cuit With Learning", IEEE Int. Symp. on Circuits and Systems, May 1990, pp.
2508-251 1.

159

[30] C. Mead, Analog VLSI and Neural Systems, Addison Wesley, 1989.

[31] D. E. Rumelhart, J. L. McClelland, and the PDP Research Group Eds, "Parallel
distributed processing-Explorations in the microstructure of cognition", vol. 1,
Foundations. Cambridge, MA: MIT Press, 1986.

[32] F. C. Hoppensteadt, An Introduction to the mathematics of neurons, Cambridge
University Press, 1986.

[33] P. E. Allen and D. R. Holberg, CMOS Analog Circuit Design, HRW, 1987.

[34] R. Gregorian and G. C. Temes, Analog MOS Integrated Circuits For Signal Pro-
cessing, John Wiley & Sons, 1986.

[35] P. R. Gray and R. G. Meyer, Analysis and Design of Analog Integrated Circuits,
John Wiley & Sons, 1984.

[36] Y. Tsividis, Operation and Modeling of the MOS Transistor, McGraw-Hill, 1986.

[37] Y. Tsividis and P. Antognetti, Eds, Design of MOS VLSI Circuits for Telecom-
munications, Prentice-Hall, 1985.

[38] R. L. Geiger, P. E. Allen, and N. R. Strader, VLSI Design Techniques for Analog
and Digital Circuits, McGraw-Hill, 1990.

[39] C. Mead and L. Conway, Introduction to VLSI Systems, Addison Wesley, 1980.

[40] N. Weste and K. Eshraghian, Principles of CMOS VLSI Design, Addison Wesley,
1985. '

[41] F. M. A. Salam, M. R. Choi, Y. Wang, and B. Javidi, "On the Design of Neural
Nets", 25th Annual Allerton Conference on Communication, Control, and Com-
puting, Sep. 30 - Oct. 2, 1987.

[42] F. M. A. Salam, "A Formulation For The Design of Neural Processors", The
Procwdings of the 2nd IEEE Annual International Conference on Neural

 

160

Networks (ICNN), San Diego, CA, July 24-27, 1988, pp. I-l73-180.

[43] F. M. A. Salam, "A Modiﬁed Learning Rule for Feedforward Artiﬁcial Neural
Nets For Analog Implementation," Memorandum No. MSU/EE/S 90/02, Depart-
ment of Electrical Engineering, Michigan State University, East Lansing, MI
48824-1226, 26 January 1990.

[44] F. M. A. Salam, "A Model of Neural Circuits For Programmable VLSI Imple-
mentation", 1989 IEEE International Symposium on Circuits and Systems
(ISCAS), Portland, Oregon, CA, May 9-11, 1989.

[45] F. M. A. Salam and Y. Wang, "Some Properties of Dynamic Feedback Neural
Nets", in the session on Neural Networks and Control Systems, the 27th IEEE
Conference on Decision and Control, December 1988, pp. 337-342.

[46] R. P. Lippmann, "An Introduction to Computing with Neural Nets", IEEE ASSP
Magazine, April 1987. PP. 4-24.
[47] D. H. Sheingold, Nonlinear Circuits Handbook, Analog Devices, 1976.

[48] J. Darnell, I. Lodish, and D. Baltimore, Molecular Cell Biology, Scientiﬁc Books,
1986.

[49] F. M. A. Salam, S. Bai, and J. Hou, "Dynamic of Feedback Neural Nets with
Unsupervised Learning", 1990 IEEE International Joint Conference on Neural
Networks (IJCNN), San Diego, CA, June 17-21, 1990, pp. II-239-244.

[50] F. M. A. Salam and S. Bai, "A Feedback Neural Network with Supervised Learn-
ing", 1990 IEEE International Joint Conference on Neural Networks (IJCNN),
San Diego, CA, June 17-21, 1990, pp. III-263-268.

[51] Linear Products Databook, Analog Devices Inc., 1988.

 

HICHIGQN STATE UNIV. LIaRﬁTrEs
llllllWlHlllllNIIHWIWII‘NWlllllllllllllllllllll
31293009022843