Calhoun
iniutuiiaiul AKliiv« ou tfit Nilvdl Poi($ra{jua(« School
Calhoun: The NPS Institutional Archive
DSpace Repository
Theses and Dissertations
1. Thesis and Dissertation Collection, all items
1981-03
An experiment in voice data entry for imagery
interpretation reporting.
Jay, Gregory T.
Monterey, California. Naval Postgraduate School
http://hdl.handle.net/10945/20526
This pubiication is a work of the U.S. Government as defined in Titie 17, United
States Code, Section 101. Copyright protection is not avaiiabie for this work in the
United States.
Downloaded from NPS Archive: Calhoun
DUDLEY
KNOX
LIBRARY
http://www.nps.edu/ljbrary
CsMwun is the Neval Postgraduate School's public access distal repository for
research oiateriels and tnstitutjiooal pubftcatiions created by the NPS community.
Cathouni is named for Professor of Mathematcs Guy K. CatHiuo, NPS's first
appointed — and publi^d — scholar^ author.
Dudley Knox Library / Naval Postgraduate School
411 Dyer Road / 1 University Circle
MontereVr California USA 93943
c MOX
^U.Va r>Oj H PADUATf SCHOOL
M'imKmy caljp <? 3')40
;b-.
Ki:
I]:i[
1 < %
NAVAL POSTGRADUATE SCHOOL
Monterey, California
THESIS
AN EXPERIMENT IN VOICE DATA ENTRY FOR
IMAGERY INTERPRETATION REPORTING
by
Gregory T. Jay
March 1981
Thesis Advisor G. K. Poock
Approved for public release; distribution unlimited.
SCCURITY CLASSlfICATION Of THIS PAGE (Wh9t% Dmtm entmrmd)
REPORT DOCUMENTATION PAGE
READ INSTRUCTIONS
BEFORE COMPLETING FORM
1. fCfONT NUMEEN
2. GOVT ACCESSION NO.
3. PECiPiCNT'S CAT ALOG NUMBER
4. TITLE fand
An Experiment in Voice Data Entry for
Imagery Interpretation Reporting
5. TYPE Of REPORT ft PERIOD COVERED
Master's Thesis; March 1981
S. PERfORMIMG ORG. REPORT NUMBER
7. AUTHOnr*;
Gregory T. Jay
ft. CONTRACT OR GRANT NUMBER|«>
f. fCNfONMING ONOANIZATION NAME ANO AOONEtS
Naval Postgraduate School
Monterey, California 93940
10. PROGRAM ELEMENT. PROJECT TASK
AREA ft WORK UNIT NUMBERS
1 1. CONTMOLLINO orrice NAME ANO AOONCSS
Naval Postgraduate School
Monterey, California 93940
12. AEPOAT DATE
March 1981
1). NUMBER Of PAGES
1 6 7
14. MONiTOfINC AGENCY NAME ft AOOm€%%(ti dlUmrwnt from ConirolfiAft OHIem)
IS. security class. (oI ihl» tiport)
Unclassified
iSa. OECLASSifiC ATI on/down grading
SCHEDULE
U. OlSTRiSuTlON STATEMEN T ro/mi* X«porO
Approved for public release; distribution unlimited.
17. OlSTfiEUTlON STATEMENT (ol th9 mtfrmd In Block 30, II dilloront Ifom Rmport)
It. SUPfLEMENTANY NOTES
19. KEY WOPOt iConlInu* on rororto old# II noeooooiT dnd Idonillf ky klotk t%iMmbmr)
Automatic Speech Recognition
Voice Data Entry
Imagery Interpretation
Intel 1igence
Experiment
20. ABSTPACT |Conllm«o on rovoroo oldo II nocooonrr md Idonlllp ftp ftlocft mnoftorj
This thesis investigated the feasibility of voice data entry for imagery
intelligence order of battle reporting. Time, accuracy, and efficiency were
measured for 20 subjects in an experiment physically simulating the use of a
light table, optics, and an interactive computer system for reporting. A
Threshold Technology Inc. T600 voice recognition system was used for a large,
unstructured vocabulary (255 words) of unclassified Soviet/Warsaw Pact
equipment names, alphanumerics, and editing commands. The T600 recognition
DD 1473 EDITION Of 1 NOV ft It OttOLETE
S/N 0t03-014* 6601 I
1
ttCUNlTY CLAttIfiCATlON Of THIS f AOE Entfd)
I
I
» w:.?
f
I
I
I
!
I
I
I
Cl
'M
I
accuracy for this experiment was 97.0^ without rejects, and 95.5% with
rejects.
Buffered voice and unbuffered voice modes of the T600 were evaluated
with typing: buffered voice was 58% faster, and unbuffered voice 41% faster
than typing. Voice was also found to be as accurate as typing for writing
short order of battle reports. Finally, subjects preferred voice for
several criteria evaluated before and after the experiment.
Form 1473
J All * 3
0102-014-6601
2 _
CWAMIflCATlOM or Txif
1
1
rcvad for pi-llic release; aisir ilut ioa uLiinlted.
An Ixperiment in Voice Date Zntry for
Imagery Interpretation Repcrtins-
by
Sregor/ T. Jay
Captain, United States Air Force
2.S. Id., ^'iemi University of Ohio, 1970
r'l.S. Fd., Miami University of Ohic, 1971
Submitted in partial fulfillment of the
requirements for the degree of
MASTER OF SCI2NCE IN SYSTEMS TECHNOLOGY - C3
from the
NAVAL PCSTGRAUUATE SCKCCL
March 19S1
ABSTRACT
This thesis investigated the feasibility of voice data
entry for imagery intelligence order of battle reporting.
Time, accuracy, and efficiency were measured for 20 subjects
in an experiment physically simulating the use of a light
table, optics, and an interactive computer system for
reporting. A Threshold Technology Inc. T600 voice
recognition system, was used for a large, unstructured
vocabulary (255 words) of unclassified Scviet/Warsaw Pact
equipment names, alphanumerics, and editing commands. The
Te00 reccgniton accuracy for this experiment was 97.0%
without rejects, and 95.5% with rejects.
Buffered voice and unbuffered voice modes of the T600
were evaluated with typing: buffered voice was 58% faster,
and unbuffered vcice 41% faster than typing. Voice was alsc
found to be as accurate as typing for writing short
order of battle reports. Finally, subjects preferred
voice for several criteria evaluated before and after the
experiment.
4
TAPLS OF CONTINTS
I. BACKGROUND LIADING TO FXPZRIMENTATION -
A. INTRODUCTION -
B. IMAGERY INTERPRETATION REPORTING SYSTEMS -
1. functicns -
2. Examples of Imagery Interpretation
Reporting Systems -
3. Requirement for Voice Data Entry -
C. AUTOMATIC SPEECH RECOGNITION -
1. Overview -
2. Value of Speech Recognition Systems -
3. Military Research and Applications -
D. SUMMARY -
II. DESCRIPTION OF THE EXPERIMENT -
A. OBJECTIVES AND CONSTRAINTS -
B. SUBJECTS -
C. EQUIPMENT -
1. Voice Recognition System -
2. Tachistoscope -
3. Scenario Cards and Vocabulary -
4. Interactive Computer System: ARPANET -
D. SUBJECT PREPARATION -
1. Te00 Vocabulary Training -
2. Typing Test -
12
12
17
17
19
28
29
29
32
36
39
41
41
42
43
43
4c
52
54
58
58
60
5
4
Subjective Questicnnaire and Data Sheet
61
'Z
Z. IXPZRIMZMAL FHOCZDURZ - 52
J. DZPZNDINT VARIABLZS - 66
G. EYPOTEZSZS- 68
1. Hypotheses Regarding Time- 68
2. Hypotheses P.egardiag Accuracy- 56
3. Hypotheses P.egardiag Efficiency- 59
4. Hypotheses P.egardiag T600 P.eccgniticn
Accuracy without P.ejects- 69
5. Hypotheses Regarding T600 Recognition
Accuracy with Rejects- 70
6. Hypothesis Regarding Subject Attitudes — 70
H .
I .
ZXPZRIMZNTAL DESIGN -
RESULTS -
1. Results for Reporting Time -
2. Results for Reporting Accuracy -
3. Results for Reporting Efficiency -
4. Results for T600 Recognition Accuracy -
5. Results for Subject Attitudes -
70
72
72
80
84
86
III. DISCUSSION
90
GENERAL
90
3. RECOKt^ENDATIONS
1. Research
2. Applications
C. CONCLUSIONS -
94
95
6
APPENDIX A:
APPENDIX B:
APPENDIX C :
APPENDIX D;
APPENDIX S;
APPENDIX f:
APPENDIX G;
APPENDIX H:
APPENDIX I:
USSR/\vARSAW PACT ORDER CF BATTLE (OB)
VOCABULARY - 96
SCENARIO CARDS - 102
T600 TRAINING INSTRUCTIONS - 120
TYPING TEST-123
PRS/POST SUBJECTIVE QUESTIONNAIRE - 125
SUBJECT DATA SHEET -128
INSTRUCTIONS BRIEFED TO SUBJECTS - 129
VOCABULARY '*CRDS MISRSCCGNIZED OR REJECTED 135
RESULTS FOR PRE/POST SUBJECTIVE
QUESTIONNAIRE - 146
LIST OF REFERENCES -150
INITIAL DISTRIBUTION LIST - 152
7
LIST OF TABLES
I. I^SAN REPOHTING TIKE- 74
II. ANALYSIS OF VARIANCE FOR REPORTING TIKE
(SECONDS) - 75
III. KEAN REPORTING ACCURACY (%) - 76
IV. ANALYSIS OF VARIANCE FOR ARCS IN-TRANSFORMED
REPORTING ACCURACY - 79
V. MEAN REPORTING EFFICIENCY (%) - 61
VI. ANALYSIS OF VARIANCE FOR ARCS IN-TRANSFORMED
REPORTING EFFICIENCY - 82
VII. MEAN T600 RECOGNITION ACCURACY (%)
WITHOUT REJECTS - 85
VIII. MEAN T600 RECOGNITION ACCURACY (%)
WITH REJECTS- 85
IX. ANALYSIS OF VARIANCE FOR ARCS IN-TRANSFORMED
T600 RECOGNITION ACCURACY WITHOUT REJECTS - 86
X. ANALYSIS OF VARIANCE FOR ARCS IN-TRANSFORMED
T600 RECOGNITION ACCURACY WITH REJECTS - 87
8
LIST Cf FIGURES
1. Basic Coirnand and Control Model- 15
2. CATIS Ixagery Exploitation Support - 21
3. TIPI IiT;agery Interpretation Systerr (IIS)- 22
4. TIPI Manual Radar Reconnaissance Exploitation
Systerr (MARPES) - 23
5. QSR Reconnaissance Reporting Facility (RRF) - 25
6. Compass Previev/ Digital Exploitation System- 26
7. Threshold Technology Inc., T602 Voice Recognition
System with Ann Arbor Terminal (facing left) and
Keyboard, and Shure SM-10 Microphone - 45
6. Tachistoscope Interfaced to Ann Arbor Display and
Motorized Card Presentation Peripheral - 49
9. Tachistoscope Viewport Used to Simulate Optics
and Light Table- 51
10. Sample Scenario Cards - 53
11. ARPANET MAP- 56
12. ADM Terminal Attached to ISI Computer via the
ARPANET- 57
13. Monitor Station- 59
14. CB Reporting Format Eased on Cards in Figure 10 — 64
9
Concept\;a] ijesi,?n of the Siperinent
71
16.
17.
18.
yeaa Reporting; Time by Data Entry Mode-
Mean Reporting: Time by Trial -
Mean Reporting Efficiency by Data Entry Mode
76
77
63
ACSNO’A'LEDGSf^iNTS
I happily take this opportunity to express well-deserved
thanks to the many kind people who supported rte in this
thesis research. Special thanks to Professor Gary Poock, an
"ideal advisor," to Professor Bill Mcroney, my patient and
helpful second reader, and to Mr. Paul Sparks who faithfully
set up the equipment and graciously assisted me whenever
necessary.
I also thank all the people who participated in the
experiment, unselfishly dedicating approximately eight hours
of their free time to help further voice research.
host importantly, warm thanks to my wife Joy, and our
children Heather, 2ric, and Sam who stood by me physically
and spiritually through the seemingly endless hours of
thesis experimentation and writing. Finally, as the
psalmist wrote:
I give thanks to the Lord, for He is good? for His
lovingkindness is everlasting [Psalm 11S:29].
11
I. BACKGROUND LEADING TO EXPERIMENTATION
A. INTRODUCTION
This thesis investigates the potential application of
automatic speech recognition (ASP.) technology to military
imagery interpretation reporting. It stems from the
author's background in three areas: imagery interpretation,
Intelligence Data Handling Systems (IDES), and recent
exposure to the benefits of voice data entry as an
alternative modality for interacting with m^achines,
especially computers.
The need for the thesis arises from two areas: the need
to evaluate and advance current ASH technology without major
redesign of systems* and the need for faster, reliable
reporting systems for the intelligence community. Dr. Wayne
Lea and Dr. Gary Poock called for the evaluation of state-
of-the-art ASR equipment, specifically, to evaluate input
modalities, e.g. voice versus typing [Refs. 1 and 2]. The
intelligence community is continually seeking ways to
improve performance of imagery sensors and exploitation and
reporting systems, and is very interested in ways of
reducing costs while improving the quality of intelligence
to tactical and strategic users.
The Soviet Union and the Warsaw Pact countries are
expected to employ mass, mobility, and surprise tactics in
12
aa/ future European attack scenario on our North Atlantic
Treaty Organization (NATO) Allies. The speed and ran^je of
rrodern weaponry leave little or no room for iristakes in
responding to crisis situations. Decision-making in minutes
or even seconds is a requirement today, and is likely to be
more critical in the future with the increased use of
microelectronic components for sensor and weapons control,
and faster, more redundant, survivable, and interoperable
communications facilities. National Command Authorities,
U.S. Strategic and Tactical Forces, and NATO Theater Forces
must have accurate, timely, and complete indications and
warning (I&W) intelligence of the enemy's real intentions
and capabilities. Once hostilities begin, with today's
warfighting technology, military commanders will require
near-real-time (NRT) combat information to enable them to
provide effective command and control of their forces to
counter the enemy.
Globally, intelligence must be available for
national security decisions regarding appropriate
responses to international terrorism and the unwarranted
intervention of foreign powers into the affairs of
other nations. Additionally, intelligence is
required for long-range planning estimates to support the
acquisition of the best possible mix of forces to
meet mission requirements in support of basic U.S.
policy and objectives. Finally, intelligence must
13
continually support Strategic Nuclear Commend and Control
forces which rust always be at a sufficient state of
readiness to provide nuclear deterrence.
The following basic commend and control model in Figure
1 was adapted frcm the work of Dr. o^cel Lawson, Technical
Director, Naval Electronics Systems Command [Hef. 3]. It is
shown here to illustrate the importance of the intelligence
process in providing support to command and control of
forces in war and peace. Note that it does little good to
provide better sensors without also improving the ability to
compare the information derived with objectives and
historical information in conjunction with intelligence
analysts, inherent in the "compare” process. In the
reconnaissance area, imiagery exploitation and reporting
would fall under the "compare" function of the system, and
as such can be a major information "bottleneck" if not
capable of effectively processing the sensor output to meet
the information needs of the decision-maker.
14
^ S|C jjc JjC sj{ 3^ 3|« sfs ?|« 3yS 3{S Jjs jJS 3(S Sj? 3y« 3^ ?J% 3|$ 3^ jJS 3f! SjS JyS SjC 3|C 3(5 5{5 5(4 3jC 3^ 5j4 5(5 5{5 3{5 3(4 3^ SjC 3^ 3^ 3^ SjC 3^5
3^- 5:t->->-Jts 3;;
SiNSZ
— y — y^
5;< 3;5 3i53;4 3^;3{«3;5>;4 3i55;5 3;4 5;5 3;5 5;5 3;4 3;5 5;5 5;5 5(C3’,5 5j5>;5
ENVIRONMENT
nS
nC
3{C
n*
5!C3;5 3iC3{C3j5 3;5 5iC3;S5}C3;4 3j5 Sjc >;5 5;4 3)5 >}C ;{f jIC 354 >;5 5^5
- COMPARE «
3(c 3^:
3;4:iS3}c:{5 5{55;4 3;C3;5 3{5 3jC 3|C 3:5 3*5 SjS 3;5 3^5 sjC 3>:c
3;5 3:5
3^5 5^5 3:* 3|5 314 3^4 5:5 ^ 3(5 3:4 515 3^5 ^ 3:5 Sl* ^ 3^5 s:* 3:5 5:4
5(5 5:5
* DECIDE
# >!«
3^4 3:43:4 3:4 3:5 5^5 3^5 3:4 314 3(4 314 3:4 314 3(4 3:4 314 3:5 314 3(5 3:5 s:5
3:4 5:4 3(4 3^4 ^ 5 : 45 : 55 : 45 : 45:4 3;4 5^5 3:5 5:4 3:4 3 :c 3:4 5:4 5:5 3:4 5:5
-- 5:4
3{«
ACT *
^ #—<—<_# Jit
3:5 3(5 3:4 5:4 3^ 3:4 3^4 514 3:5 3:4 5:5 5:4 3(4 3(4 3:6 5:5 3:4 3:43:5 5:5 3:5 3:5 3:5 3:55:4 3:53:5 3:4 5*143:45:4 5:4 51; 5(4 5(4 3:5 5^5 3:4 5:5 5:5 515 3:4 5:4 5:4 3:4 3:4 3:4 3:4 3:4 5(4 5:5 5:4 3;4
Figure 1. Basic Command and Control Model
Regarding the central importance of the command and
control process, Dr. Lawson states,
...the central prohlem of command control is producing
an up-to-date geographic display of the location of
’’things." Besides purely the location of things he [the
commander] needs to know what [the] things are, what is
their identity, or who do they Belong to and what is
their status.
Imagery is a key source of such information, and is thus a
major contrihutor to the comm,and and control process.
Automated imagery interpretation reporting systems have
been employed for strategic and theater support for over 10
years, and new systems which include exploitation aids are
being deployed to tactical units now. They have
15
significantly reduced the tine tc exploit and report all
types of imagery intelligence. However, the man-machine
interface research and development of these systems must
continue to meet future challenges facing the intelligence
community. Significant volumes of imagery intelligence will
be available from NRT digital imagery sensors in the future,
and the best possible man-machine interface must be sought
to effectively exploit IS’*, order of battle, targeting, and
damage assessment intelligence available from imagery.
Reporting speed and accuracy, manpower reducticns,and
increased throughput are worthy design goals for new or
improved imagery exploitation and reporting systems. Voice
data entry is one newly evolving technology that offers
significant potential toward these goals. Dr. Wayne Lea, in
the introduction to his book Trends in Speech Recognition,
1980, said:
Speech input seems to offer a truly natural mode of
human-machine communication that, if attainable in a
cost-effective way, would be unsurpassed in making
computers and other mechanical devices truly cooperative
servants of mankind, rather than increasing the demands
on the human to adapt to the machine [Ref. 4].
The next section briefly overviews the functions of
imagery reporting systems, provides some examples of
systems for today and tomorrow, and mentions some specific
requirements which lead to the desirability of voice
data entry for imagery intelligence reporting.
16
B. IMAGERY INTERPRETATION REPORTING SYSTEMS
1. Func tions
A milixary imagery interpretation system basically
functions to provide support for first, second, and third
phase exploitation of multi-sensor imagery in response to
tasking from parent or outside user organizations. These
phases represent three levels of depth of imagery analysis
in accordance with Defense Intelligence Agency (DIA)
standard reporting procedures, data elements, and
requirements.
First and second phase reports represent the hulk of
the work, and are called Initial/Supplementary Photo
Interpretation Reports (IPIRs/SUPIRs). The IPIR m:ay he
thought of as a quick, concise response to time-sensitive
requirements. It .is often followed hy the SUFIR, which
represents a more detailed and thorough exploitation effort.
Third phase reporting is the most detailed, and includes
special analyses and reporting of selected installations of
specific interest to users of imagery products.
Such reporting standards and systems grew out of
requirements forced hy large increases in the volume of
available imagery during the sixties. During the sixties,
the volume of imagery exceeded the exploitation capabilities
hy a factor of five to ten [Ref. 5]. This drove the
development of a variety of imagery exploitation and
reporting systems which came into operation in the
17
seventies, and forced standards for reporting cn the imagery
intelligence corr.muni ty as a whole. These developments
permitted the sharing of imagery intelligence via magnetic
tape files and bvlk data transfers over communications
circuits. It also facilitated the integration of imagery
intelligence intc more general data bases, and enhanced the
corporate memory" of intelligence units, since interpreters
often kept installation data in small personal files, net
easily accessed by others. With better data bases,
exploitation was enhanced and duplication of effort was
reduced.
Today, imagery exploitation systems are located
worldwide in support cf U.S. military commanders. The focus
now is on providing more integrated data bases, which are
optimally dynamic, complete, and timely. Kulti-scurce
imagery reports may be telecommunicated to and from many of
the sites, and distributed tc users with a valid
requirement. Integrated data bases will afford producers
and users with more responsive, coordinated information in
time of need.
Imagery systems range from national level to
tactical reconnaissance squadron level systems. They have
become increasingly capable of supporting many tasks
associated with exploitation and reporting: responding tc
tasking transmitted over telecommunications networks?
managing interpretation hardware, software, and data base
18
resources; exploiting the imagery to include making
measurements on the imagery, correlating imagery with maps,
composing reports, editing them, and other support
functions; disseminating reports; and automatic screening
and updating cf local imagery and multi-source data bases.
2. Examples of Imagery Interpretation Reporting Systems
The DIA uses the Automated Imagery Related
Exploitation System (aIRES), modeled after the PACER system
used by the Strategic Air Command's 544th Aerospace
Reconnaissance Technical Wing. PACER means Program Assisted
Console
Evaluation
and Review,
and oonsists of a
dual
Honeywell
6080
based computer
system
and UNIVAC
1652
consoles
supporting the interpretation
process .
Both
systems
support a
wide variety of
analyst
functions.
A
s ys t em
developed and
installed in the
late
seventies for theater and tactical user support is the
Computer Assisted Tactical Information System (CATIS). This
system is used by fixed-site, imagery exploitation units in
the Pacific Air Forces (PACAE), the Tactical Air Command
(TAC), the Fleet Intelligence Center for Europe and the
Atlantic (FICEURLANT), the United States Air Forces in
Europe (USAFE), and the training site in Air Training
Command (ATC). The imagery exploitation support provided by
CATIS may be viewed in Figure 2.
To provide highly mobile support, the Tactical
Information Processing and Interpretation, Imagery
19
Interpretation System (TIPI IIS) was developed, and is being
deployed to Air force, Marine, and Army tactical
reconnaissance support units worldwide. The photo
interpretation console of the TIPI IIS may be viewed in
figure 3, displaying a great deal of modular, ruggedized
support equipment for imagery interpretation reporting and
communications. This system provides mobile automation at
the squadron level, not previously available. For example,
an interpreter can use a cursor in the light table to make
rapid, accurate measurements of objects such as bridges,
runways, and storage tanks and store the answer on an
electronic scratch pad for later insertion into a report.
Reports are filled in quickly, using a fill-in-the-blank
online report composer. They may then be edited by a
supervisor, and distributed over secure communications
links.
To perform side-looking airborne radar (SLAR)
exploitation and reporting the TIPI Manual Radar
Reconnaissance Exploitation System (MARRES) was
developed, but with a different ccnscle (figure 4). This
system provides special readout of radar imagery that may be
used in good or bad weather, and is useful for discovering
enemy force movements in inclement weather, such as that
found in Europe. Unique man-machine systems have been
20
'i«-ijre'£. CATIS Irc§er/ r.xplc i te ti or. Suppc:
(Acar.ted frorr CATIS User's I^arual, 1979)
C 1
1
figure 3. TIPI Imagery Interpretation Syste- (IIS)
(Courtesy of Texas Instruments, Inc.)
2c
W¥$'
s^S
S<-^%
M
wmm^Wf0MM$f40mm
v;)
23
‘lli'l (■■ar.L.al Radar Re co nna i s sa nc t Exploitation S./ster, (f ARF.E
tCouriesy of ‘lexas I nslrnt enls Ino.)
provided to assist in providing detection of changes in the
landscape or order of battle.
New NRT digital imagery reconnaissance sensors,
such as foward-looking infrared imagery (FLIR), Synthetic
Aperature Radar \SAR), or other types of imagery
which can he supported by sensors on tactical aircraft
will result in increased NRT imagery. Exploitation
system.s to support the sensors must be developed to provide
the additional support required. The Air Force has
initiated advanced developmental models to prepare for
such a requirement.
One system is the Reconnaissance Reporting
Facility developed to support the Quick Strike
Reconnaissance concept whereby the reporting facility would
receive NRT hardcopy and softcopy (digital) imagery from
reconnaissance aircraft over the forward edge of the battle
area. When advancing enemy forces posed themselves as
targets of opportunity, imagery reports would notify the
strike center to order nearby airborne loitering
aircraft to destroy the target. Figure 5, top and
bottom, gives views of the shelter developed to test
the NRT reporting concept.
24
Figure 5. QSR Reccnnaissance Reporting
(Courtesy of Texas Instruir.ents,
Facility
Inc.)
(RRF)
25
le
Fi^vire 6. Compass Preview
Digital Exploitation System
(Courtesy of Northrop Corporation)
The RRF contains computers, communications, and both
hardcopy and softcopy imagery exploitation and reporting
stations. Used during exploitation of a target-rich
wartime environment, this facility would pose a
challenging work environment for the best of interpreters
and supervisors. Efforts to optimize the man-computer
interface could only result in improved responsiveness and
greater system capability.
Another system, for strategic use, is the Compass
Preview digital imagery exploitation system shown in Figure
6. For the first time, interpreters will be able to view
stereo images without the aid of a light table, hardcopy
imagery, or a stereoscope. The interpreter can use computer
support to enhance the image to improve its interpretability
in terms of scale, contrast, sharpness, and other image
qualities. Simultaneously, historical data base information
and reporting formats are available for reporting what is
seen on the image and correlated with other data.
Measurements may also be made using a Joystick and cursor.
The imagery systems discussed represent a large leap
forward in imagery intelligence since the late sixties. The
results from current systems such as PACER and CATIS are
encouraging with 3:1 and 12:1 increases in output as
compared to their predecessors, less duplication of effort,
increased validity of reporting, and most importantly,
better responsiveness to specific user questions.
27
Imagery reporting systems are quite
sophisticated, having incorporated not only state-of-
the-art exploitation techniques, tut others as
well from computer, communications, and other
intelligence disciplines. Significant skill and
training are required to operate them effectively.
Interpreters are not trained typists, and thus their speed
may slow the reporting process. Additionally, they may
nave an inherent fear of working- with computers.
Continuing attention must he given to improving the
man-machine interface to optimize the system product:
complete, accurate, and timely imagery intelligence. Though
not a panacea, voice data entry may be part of the solution
for improving the imagery interpretation systems, by
improving man's interface with the machine, and making
optimal use of man's skills as an image analyst.
3. Requirement for Voice Data Zntry
During the author's recent assignment at the Armed
Forces Air Intelligence Training Center, he was responsible
for managing the initial development of the TIPI IIS
Operator and Supervisor Courses. As he observed
interpreters training on the prototype, it was often
apparent that they were deficient in typing skills. It was
painfully obvious that the multi-million dollar IIS would
not produce reports any faster than the few words-per-minute
of the "hunt and peck" typist. Certainly, with practice
26
individuals may improve their typing speed and accuracy as
they adapt to a system, hut as we have seen, the trend is
toward faster reporting, and somehow the problem of data
entry must he attacked or critical resources will he wasted
on systems limited by the the mac-in-the-1oop.
One simple and effective way may he to conduct
typing classes to improve interaction with the computer. In
fact, online routines for teaching better typing could he
developed to improve the interpreters' skills between
missions. Another way may be to use voice data entry, which
offers a great potential beyond even the fastest typists for
data entry, should be easier and faster to train, and could
be used in conjunction with typing, function keys, or a
variety of other input modalities.
G. AUTOMATIC SP3ICH R2C0GNITICN
1. Overview
Automatic Speech Recognition (ASR) is no longer a
dream of the future, hut a technology being applied around
the world by people who use machines, allowing effective
machine control and data entry into computers. ASR is not
without problems or limitations however, and must he
carefully examined before trying to apply it. Human factors
must he studied and tailored to the application to allow ASR
to have the appropriate im.pact it affords. Failure to
attend to operator considerations such as microphone
29
4*ii
rrounting, recognition accuracy, error correction, response
time and delay, feedback and prompting, stability of
reference data, and training procedures can have
catastrophic effects on system performance for both the
voice system and the system it aids [Ref. 7],
The ultimate goal for speech recognition
science is to develop "speech understanding systems"
which give the appropriate response to the user's
request, and do not just recognize tne elements of speech or
words and phrases [Ref. 8]. Admittedly, the technology is
not that far along, but many applications dc not need cr
cannot afford the ideal speech system. The question that
must be asked now is "what applications can be
accomplished in a more cost-effective manner witn voice
recognition systems that are available now or will be
available within the next few years?"
Speech scientists have been working on ASR for about
28 years. Commercially available speech recognizers became
available in 1972 with Scope Slectrcnics, Inc. and Threshold
Technology Inc. delivering quality systems which achieved
significant results under a variety of conditions. In
general, recognition accuracy scores from 99.0% to 99.9%
accuracy have been achieved in laboratory conditions of no
noise, adequate talker training, and consistent talking
habits. Field testing, however has usually achieved results
in the neighborhood of 97% recognition accuracy, generally
30
as a result cf high background noises or speaking to the
system in a manner different than the way the system was
trained initially.
All ASR systems fall into either of two
categories: continuous (connected) or isolated (discrete)
speech systems [Ref. 9]. Continuous speech systems work on
the extraction of information from strings of words
that may be run together in natural speech in the form of
strings of digits, phrases, or sentences. Isolated-word
recognizers require that a short minimum-duration pause
be inserted between digits, words or phrases which must be
spoken within a given period of time, e.g. two seconds.
These isolated-word recognizers are more prevalent
today as they are less expensive, more accurate, work in
real-time, ana are more readily available. Continuous
speech systems, however, may be available within the next
few years offering 250 word vocabularies and recognition
in real-time at a reasonable price. Centinucus speech
systems, in the upper end of the cost spectrum, are
approximately $100,000. Eigh quality isolated-word speech
recognizers normally cost in the tens of thousands of
dollars today? however, a few companies are also introducing
systems on the market for a few thousand dollars that
can recognize vocabularies of about 250 words with
recognition accuracies of 97^ or better, according to
Dr.Poock, who intends to compare such systems at NFS for
31
command and control applications. At the bottom end
of the cost spectrum, hobby systems are currently available
for a few hundred dollars.
Dr. Lea, well recognized for his work in speech science
at the University of Southern California and the Speech
Communication Research Laboratory said this about the future
of speech recognition technology:
The next ten years or more would seem to offer a growing
spectrum of available devices, ranging from very low
cost isolated word recognizers, through digit string
recognizers, recognizers of strictly formatted word
se«iuences, task-restricted speech understanding systems,
and more powerful research systems for continuous speech
recognition. All such systems will take advantage of
low-cost miniaturization hardware that puts speech
recognizers within the reach of most potential users...
User acceptance of voice input will approach the
"matter-of-fact" attitudes now prevalent with limited
keyboard entry, even though full versatility and
"habitability" of input languages will not have been
attained to any major degree... Despite all these
advances, we will be far from the science fiction image
of fully versatile voice interaction with machines, and
I doubt that unrestricted "phonetic typewriters" are a
part of the next decade or more of practical work on
speech recognition [Ref 10] .
2.
Value of Speech
Recognition System.
5
Speech input
to machines can be of signif
icant
value,
but under what conditions or
situations?
This
section
discusses
some of the
advantages
and
disadvantages of speech
input described by
Dr. Lea.
32
Speech systems offer the potential to capitalize
on the best of man's communicative abilities, give him
compatibility with unusual circumstances, and help him gain
additional mobility and freedom in some situations [Ref.
ll]. Speech is said to be the human's most natural
communication miodality. It is familiar, convenient, and can
be used spontaneously because the individual uses it often
in all types of situations. Though performance with voice
may degrade under situations of stress, it may not degrade
as much as a less learned, less frequently used skill.
Since voice is familiar to the user, it is less difficult to
train him to use the system. Additionally, voice is the
human's highest-capacity output channel, and permits
simultaneous communications with humans and machines. For
example, a speaker in a large auditorium or a command center
can display the next visual on a large screen display by
saying some key phrase or word which has meaning to both
listener and display system. To illustrate, when Dr. Pocck
recently briefed a group of senior naval officers in the
Pacific, he used such key phrases as "Good Morning
Admiral..." to begin his briefing, and "here you see the
(pause) SKIPS ..." to convey briefing information and tell
the command and control graphics display system to present
the next graphic in his presentation on the subject of Voice
Input for Command and Control. This is just one
33
illustration of the creative ways man can use voice input to
his advantage.
Navy feasibility studies sponsored by Naval
Ilectronics Systems Command, and conducted by Dr. Poock,
examined the potential for voice data entry fcr commend,
control, communications, and intelligence. Two voice
recognition systems were installed in late 1960 at Fleet
Headquarters, Commander-in-Chief of the Pacific (CINCPAC) in
Hawaii to examine the benefits and limitations of voice
input for operation of the Worldwide Military Command and
Control Time-Sharing System (WWMCCS TSS) and the nearby
Ocean Surveillance Intelligence Systerr (OSIS). One
advantage of many of the new voice terminals is that they
are stand-alone, intelligent terminals with standard
communications interfaces and character sets that can be
interfaced rapidly with computers possessing those same
generic interfaces. Voice units may be moved around easily
and installed as simply as most ether modern RS-232 plug-
compatible terminals. Voice may also be used remotely as
much as 2000 feet from the main computer, free from any
panel space, displays, or complex apparatus.
The advantages of voice input fcr
complementing the communicative abilities of man are
offset somewhat today since a user cannot speak totally
naturally, but must insert pauses in between utterances, and
must use utterances within the constraints of the voice
34
system's stored vocabulary. This requires the user to
be very familiar with the vocabulary in use, not
unlike knowing the letters of the alphabet.
Speech input for machines is also of value in
helloing man cope with unusual circumstances. For example,
it can be used in complete darkness, around obstacles, by
the blind and other handicapped individuals, is unaffected
by weightlessness, and only slightly affected by high
acceleration and mechanical constraints. On the negative
side, it is often sensitive to dialect, and also susceptible
to background noise and distortions. Additionally, a
microphone must either be worn or held in close proximity to
the speaker. And finally, a display or synthesized voice
feedback may be necessary for tasks requiring data entry
validation.
The mobility possible with voice input is one of its
greatest attributes. It enables operation of devices from a
distance and from various orientations, permits simultaneous
use of hands and eyes for other tasks, and can even permit
the telephone to be used as a computer terminal. Some
degree of privacy is lost, although users often operate in
the laboratory at NFS inconspicuously running graphics
displays and other command and control applications without
bothering other nearby terminal operators.
The key questions to keep in mind when considering
the value of speech input are: "is there an application that
35
I
coiald be done more cost-effectively using vcice as a single
or additional input modelity?...and, " Is the current
technology adequate to provide the quality, naturalness, and
speed that the application of interest requires?" A brief
loch at the military's efforts in voice technology may help
the reader to further assess the value of speech technology
for his own application.
3. Military Researcn and Applications
Research supported by the Advanced Research Projects
Agency (DARFA), which funds leading-edge technology, was a
prime ingredient contributing to the development of voice
technclcgy. However, a large number of military projects,
such as the ARPA Speech Understanding Research, met with
limited success as a great deal of work in acoustic-
phonetics, speech perception, linguistics, and
psychcaccustic equipment is still necessary to provide the
foundation for ASR to approach human performance [Ref 12].
Most of the research in the military has turned to
taking off-the-shelf isolated-word recognizers and adapting
them to particular applications. Recognition studies in the
military have been done for applications in aircraft
cockpits, tactical field data entry, military training
systems, cartography, command and control of networks,
wargames and graphics, keyword spotting of communications
channels, emergency action message composition, and imagery
interpretation tasks such as mensuration and reporting. The
36
applications most clcsel/ related to this thesis are the
cartography, command and control of displays, and imagery
interpretation reporting.
A significant amount of research was performed for
the Defense flapping Agency(D^'A) by contractors under the
program management of the Air force's Rome Air Development
Center(RADC ). The Defense flapping Agency Aerospace Center
(DMAAC) and the Defense Mapping Agency Aerospace and
Hydrographic Center (DMAHC) produce large volumes of
cartographic products for the military and other users.
Research has been performed for such applications as voice
data entry for the processing of Digital Landmass System
(DIMS) data, preparation of Ilight Information Publications
(FLIPS) data, and ocean-depth neasurements for digitized
cartographic applications. In these applications analysts
were performing tasks in an "eyes busy, hands busy
environment," sometimes with stereo optics and or other
special devices. Voice was shown experimentally to be
faster, easier, and a less fatiguing mode of data entry than
the more conventional modes used [Refs. 13, 14, and 13].
User acceptance and system support can be significant
problems, as explained by DMAAC officials to the author
during a recent visit to their facilities.
The NPS is currently performing voice data entry
research in the area of command and control applications.
In a study by Pocck, twenty-four command and control
37
students operated the AEPA network or AP.PANIT, a distributed
network of computers in the U.S. and Purope, using voice and
typing as a comparison between the two modes [Ref. 16].
Voice was significantly faster and more accurate for
entering commands into the system. Additionally, students
were given an secondary transcription task to perform while
operating the ARPANET. The voice mode permitted
substantially more data to be transcribed than the typing
mode. On the other hand, McSorley recently demonstrated
that voice was no faster than typing for entering commands
into a vargame. This was due in part to the poor editing
features of the game, but demonstrates that voice is not for
everything [Ref. 17] .
In the area of imagery interpretation, interest in
voice data entry is growing. RADC recently completed a
study which evaluated a voice recognition system known as
"Talk and Type," built by Threshold Technology Inc., to
study the application of voice data entry to the problem of
imagery interpretation and intelligence report generation
[Ref. 18], The innovation by Threshold required the user to
type the first letter of the word to be recognized. In this
manner the voice system restricted the size of the
vocabulary to be searched, thereby increasing recognition
accuracy. Four varied tests were performed looking at small
and large vocabularies, and especially tasks where the
subject was describing scenes the way an interpreter might
38
describe a bridp^e or a runway. The results showed the Talk
and Type system to be superior over typiUis: for unskilled
typis t s.
Soon the new ground station for the Tactical
Reconnaissance-1 iTR-1) aircraft is expected to be built to
provide exploitation and reporting support for the sensors
aboard the U-2 derivative aircraft which is expected to
provide NRT reconnaissance support tc theater forces.
According to the program manager, voice data entry is a
serious consideration for inclusion into the program.
D. SUI^MARY
The purpose of this thesis is to investigate the
potential application of ASR technology to military imagery
interpretation. The research responds to the need for
rapid, concise, valid information for command and control
of forces in peace and war. The functions of the imagery
reporting systems include support for a variety of
tasks, especially composing reports. The specific focus
cf the thesis is to examine the feasibility of
writing order of battle reports using a large voice
vocabulary of 255 words of USSR/Warsaw Pact military
equipment names, editing commands, and alphanumerics.
Several examples cf modern operational and
developmental imagery exploitation and reporting systems
were briefly discussed which represent potential systems
39
4
■m
for applioation of voice technology. Incorporation of ASH
technology could resnlt in improved capabilities in terms
of speed, accuracy, and completeness of imagery
reporting. ASR technology makes optimal use of the fact
that speech is man's most natural input modality,
while the limited speeds of interpreters typing may not
optimize advanced reporting system capabilities.
The advantages and disadvantages of speech were
presented. Some of the value of speech input awaits
technological breakthroughs and may not be realized in this
decade.
The
military
is
not
waiting
however,
and seems
unwilling
to
pay for
ail
the
has ic
research
to push
continuous speech systems. Instead, the military is hard at
work with applications efforts with limited-vocabulary,
isolated-word, speaker-dependent voice recognition systems,
proven to be reliable and accurate for the right
applications, while monitoring and sometimes supporting work
by private contractors, hopefully leading to practical
continuous speech systems.
The objective of this thesis is to support military
applications research efforts aimed at comparing input
modalities, and afford the intelligence community an
independent data point regarding the overall evaluation
of ASR. This research began independent of the related
RADC research, and thus serves to underscore the
appropriateness of voice data entry support to the task.
40
II.
LESCRIPTIOK OE THE EXPERIMENT
A. OBJECTIVES ANE CONSTRAINTS
The objective of this experiment was to determine if
state-of-the-art voice data entry equipment wes feasible for
reporting imagery-aerived order of battle (03) intelligence
using an interactive computer system. The experiment was
designed to determine if there wes any significant
difference in speed, accuracy, efficiency, and subject
attitudes regarding manual keyboard and voice data entry for
this task. A large unclassified vocabulary cf 255 words
containing alphanumerics, commands, and representative
USSR/Varsaw Pact equipment names was selected for the
reporting scenario (see Appendix A). Based on recent
research, voice data entry was expected to be faster, more
accurate, and preferred by subjects over manual keyboard
data entry [Ref. 18].
Accomplishment of this objective was constrained within
the research facilities of the Naval Postgraduate School
(NPS). In the interest cf time and money, the process of
reporting was simulated to the maximum degree possible
within the constraints of available subjects and laboratory
facilities. This simulation, though not ideal, afforded an
effective, economical tool to accomplish this objective.
41
B. SUBJECTS
Twenty subjects participated cn a vclunteer basis.
The group was composed of 13 military officers, and
two civilians. The military officers, representing the
Army, Navy, Air Force, and Marines included 17 males and 1
female; tneir grades ranged from Lieutenant to Commander
and from Captain to Lieutenant Colonel, The civilians
included an employee of the National Security Agency and
a professor from the NFS Operations Research Department.
The subjects' ages ranged from 28 to 45, with an average age
of 33.
Seventeen of the subjects were enrolled in the Command,
Control, and Communications (C3) Curricula at NFS, while the
other two students were from the Intelligence and Computer
Science curriculas. The background of the subjects were
quite varied: special warfare; ground combat; communications
maintenance and staff; logistics staff; automatic data
processing; training; intelligence; C3 research; language
analysis; electronic warfare; Joint Chiefs of Staff; field
artillery; destroyer group staff; combat development; C3
training and operations; and tactical C3 flight operations.
Nineteen of the subjects had experience with interactive
computer systems at NFS. Eighteen of the subjects were
experienced in use of the ARFANZT, a network of computer
systems available for use by the C3 Curricula and other
researchers at NFS. The two subjects without ARPANET
42
I
i
4
i
experience were trained to tne level necessary to
participate in the experiment with their contemporaries,
since a computer on the ARPANET was chosen as tne host for
the experiment.
The subjects were, as a whole, familiar with speech
reccgnition as many had either seen, used, or even studied
automatic speech recognition. Eighteen subjects had seen a
voice recognition system demonstrated? 12 subjects had used
voice, primarily as subjects in one other experiment? and 11
had studied voice for a term paper, thesis, or work at their
previous duty station.
C. EQUIPMENT
1. Voice Recognition System
A Threshold Technology Inc. Model T602 voice
reccgnition system was used to represent commercially
available, state-of-the-art equipm.ent. The T600 is a
speaker-dependent, isolated-word recognizer which
automatically recognizes spoken words or phrases. These
words or phrases are called utterances and must be in a
range of 0.1-2.0 seconds in duration and must be separated
by very short pauses of 0.1 second or more [Ref. 19].
The terminal consists of a threshold analog speech
preprccesscr, an LSI-11 microcomputer and a digital RS-232
input/output interface, an Ann Arbor large character
display and operator console, an operator console/microphcne
43
preamplifier, and a tape cartridfe^e unit. The speech
preprocessor, microcomputer, and interfacing elements are
contained in the main terminal unit which was table mounted.
The remaining components, the display console, and tape were
disc table mounted and located with the main terminal (see
Figure 7). A Shure SN-10 noise-cancelling microphone with
headset was used for the voice input to the preamplifier.
The T600 combines analog and digital signal
processing technology to perform the recognition function.
The energy from the spoken utterance is passed through 19
bandpass filters spanning the speech spectrum. The presence
or absence of each of 32 acoustic features is determined,
and the appropriate feature information is extracted by a
combination of analog and binary logic. The features are
either primary features or phonetic-event features. Primary
features describe the energy spectrum by measuring local
maxima and the energy rate-of-change relative to the
frequency of the voice signal. Phonetic-event features
result
from measurements
corresponding
to
phoneme-1 ike
events:
vowels, nasals and
fricatives.
The
preprocessor
also must determine the beginning and ending of each word.
44
, It St. old 'i 11 iiriol Of-
The T600 has two primary nodes of operation:
training mode, and recognition mode. In the training mode,
the 1600 extracts a time-normalized template for each given
word. Ihis template consists of two arrays referred to as
the most significant tit (hSB) and non extremum tit (NEB).
The h'SB indicates whether a particular feature has occurred
and the NEB indicates the frequency of occurrence. These
arrays combine to form the reference array (HAR) . When the
T600 is in recognition mode, the preprocessor functions as
before: features are extracted, digitized, and time
normalized. The resultant feature array (JAR) is correlated
with the stored RARs in the current active vocabulary and
the best correlation is selected as the recognized word.
As previously mentioned, for each utterance 32
acoustic features represented in binary form and their time
of occurrence are fed from the preprocessor to the
microcomputer short-term memory. The pattern-matching
algorithm subsequently compares these feature occurrence
patterns to the stored reference patterns for the various
vocabulary words and determine the "best fit" for a wcrd
decision. The JAR of a test word requires 512 bits of
information (32 features m.apped into 16 time segments). The
RARs include 1024 bits per word because of the two part
arrays.
When the Te00 recognizes a word in its vocabulary it
will output a preprogrammed string of up to 16 characters
46
associated with the spoken worn. These output strings can
he modified hy the user at any time via his ASCII console,
which may also he used instead of voice to interact with the
host computer. Also associated with each word are training
prompts which are strings of up to 12 ASCII characters
displayed on the CP.T terminal to notify the user of the word
to he trained. The T600 used in this experiment rec^uired 10
training utterances per word.
Two types of errors can occur with the Te00:
misrecognition and rejection. Misrecognition errors are
those where an output string is selected for output that
does not match the utterance. When the system rejects the
utterance as not part of the vocabulary it signals the
operator with a "heep." These two cases assume the word was
in the vocabulary and properly trained. Other errors are
called operator errors and arise from mispronunciation,
using words not in the vocabulary, or a variety of other
errors such as speaking too fast or slow.
The T600 used had enough memory modules to maintain
an active working vocabulary of 256 utterances.
Vocabularies were input and output using the tape cartridge
unit. The system reads and stores prompt and output strings
and reference patterns from semiconductor random access
memory onto rugged, high-quality magnetic tapes similar to
cassette tape cartridges. A complete 256 word vocabulary
may be recorded or loaded in a few minutes.
47
Two recognition modes are available on the 1622 :
unbuffered and buffered. In unbuffered mode, the T600 will
send any output immediately to the host computer. No
internal processing is performed on the output strings.
However, the buffered mode permits up to 126 utterance
output strings to be sequentially stored in a T 600 buffer
for subsequent output as a composite block of characters.
An "erase function" may be used to delete the last
utterance; an "interrupt" function sends a special user-
defined string to the host and deletes the remainder of the
buffer contents; a "cancel" function may be used to delete
the buffer contents; and a "transmit" function will cause
the T600 to send the buffer contents to the host. The
utterance assigned to these functions may be independent of
their function name.
2. Tachistoscope
To provide a simulation of the light table
and optics portion of the imagery interpreter's work
environment, the G-1130 Harvard Tachistoscope was
selected from the man-machine laboratory facilities,
(see Figure 6) The tachistoscope is an instrument that
can present images of material presented on cards and,
as modified in this experiment, a CRT display. The card
images may be presented by a timer or changed at will by the
subject using a button switch. Lighting may be regulated
46
49
'ure b. ’lacaisioscopfe liiierfaced lo hnn Arbcr LhT Disilay chu
end rriult i-ims^es overlayed. The three primary uses of the
device are studies on learning, perception, and attention
[Ref. 20] .
However, in this experiPient the tachi stoscope was
used in the following manner. The viewport seen in Figure 9
simulates the optics through which an interpreter must get
much of his/her data. The 4" x 6” cards seen through it
simiulated the imagery the interpreter was tasked to analyze
and report. The CRT presented three lines of data (40
characters each) providing visual feedback for voice data
entry. (Note; Rome Air Cevelcpment Center has developed an
eyepiece for a Bausch & Lomb stereoscope that displays 16
characters of data while viewing the optics) thus the author
assumed that more data could be displayed in the next few
years to support such visual feedback, if required.)
The tachistoscope viewport permitted the viewing of
the scenario cards and the Ann Arbor CRT. The card image
was centered above the three bottom lines of the large-
character CRT. The CRT displayed the responses of the Te00
to the subject's utterances, thereby providing visual
feedback to him/her performing the task.
50
1
Figure 9. Tachistoscope Viewport
Scenario Cards and Vocdbulary
The cards for the reporting scenario were used to
simulate frares of irrager/. Because no imagery interpreters
were available in large numbers for the experiment at NFS,
the author created the cards with a to represent the
equipment location and annotated the with the number
and description of the equipment at the point. All subjects
were provided with the same information, i.e. they were
"perfect imagery interpreters" and any experience level was
held constant.
Figure 10 illustrates the format of two sample cards
which had five to eight objects and an installation number,
lach card was divided into four quadrants to simplify and
standardize the reporting process and scoring.
Thirty-six cards were required for the experiment.
Their content was governed by four criteria: realism, an
even mix of ground, air, and naval terms, full use of the
USSP./Warsaw Fact vocabulary selected for the experiment, and
maintaining a balance in number of characters among sets of
cards to be used in experimental trials. The cards used in
the experiment are shown in reduced size in Appendix 3. The
larger, actual size cards seen in Figure 10 were produced
using large print on a Tektronix 4014 terminal and its
associated thermal printer.
52
INSTALLATION e298-T14217
50 CONFIRPIED ASU-85
27 CONFIRHED ASU-57
tt
tt
AIRBORNE ASSAULT GUNS
AIRBORNE ASSAULT GUNS
tt 20 POSSIBLE n-20
62 PROBABLE 122-nn D
48 CONFIRNED 240-nn
tt
HEAUY nORTARS
30 FIELD HOUITZERS tt
Bn-24 ROCKET LAUNCHERS
INSTALLATION 0199-U14197
16 CONFIRNED ni-4_H0UND HELICOPTERS
tt
tt
11 CONFIRNED ni-12_
tt 5 PROBABLE ni-6J
HOnER HELICOPTERS
I
HOOK HELICOPTERS
21 CONFIRNED ni-10_HARKE HELICOPTERS
19 PROBABLE ni-24_HIND HELICOPTERS
7 ^' r r p 1
i-c.
(actLdl size
crple Seer.eric ‘'avis
= ^
A O
includinsT 'oordsr''
A USSR/Wersaw Pact vocabulary was used because of
available unclassified source inforration in large quantity
[Refs, cl, 22, 23,and 24]. A full vocabulary cf 255 words
was used containing the phonetic alphabet, nurnbers 2-25,
administrative aIphanumerics, special symbols and control
characters, and ground, air, and naval forces equipment
vocabulary. Appendix A contains a complete listing of the
vocabulary by number, training prompt, and output string.
The vocabulary was not structured in term.s of
recognition sets. Rather, the T620 operated on the entire
vocabulary each time an utterance was spoken.
4. Interactive Computer System: ARPANZT
To provide an interactive text editing environment
for the reporting scenario, the facilities of the ARPANET
were selected because of their reliability and also to
demonstrate how reporting niight be done over a distributed
network of computers, rather than a local host system. The
ARPANET, now managed by the Defense Communications Agency,
was used by 18 cf the subjects during 5 quarters of their C3
Curricula prior to the experiment.
Two host computers were used: Information Sciences
Institute Systems E and C (ISIE 6. ISIC), located in southern
California. The experimental text editor (XED), photoscript
(PHOTO), directory linking (TALK), file transfer protocol
(ETP), and file archival (ARCHIVE) were the major programs
54
used to conduct and manage the experimental data end
interactive computer environment. ISIC was the primary
system used, because the "system load level" was generally
lower thereby offering a more responsive system. The load
level was checked during experimentation to assure a
consistent response time was available to all subjects.
Both systems were supported by the TOPS-cZ Operating System,
on Digital Equipment Corporation (DEC) Model 20 Computers.
These computers were linked to NFS terminals
equipped with phone moders or acoustic couplers via
the ARPANET distributed communications facilities.
These facilities include a terminal interface processor
(TIP) at NPS connecting school terminals with ISI via
the ARPANET. The author gained access to the network
via the TIP and selecting the network computer to be
used. The ARPANET providea a myriad of facilities
supporting the administration of the experiment. Figure 11
is a map of the ARPANET adapted from, the ARPANET Information
Brochure, 1979.
CRT terminals and the T600 were attached to the
ARPANET via 30£ bps acoustic couplers. A lier-Siegler
ADM CRT display was situated near the tachistoscope to
provide keyboard entry of the OB data obtained from the
cards via the viewport (see Figure 12). The ADM terminal on
53
^Lectmber, 19Vf
c7
Aita(iied tc ISI Ccnipiiter via the ARPANET
the ARPANET was used to sirr.ulate the text editing facilities
cf an imagery reporting system for the order of tattle entry
portion of the report. All keystroke entries into the
terminal were copied ty a typescript program, during the
experiment to provide a record of the subject's performance.
A monitor station with a hardcopy Computer Devices
Miniterm and an Alanthus V-203 CRT display were used to
record and observe the subject's actions, whether by voice
or keyboard entry (see Figure 13). The Alanthus display,
connected to the 1602, provided the author with a copy cf
the data being displayed to the operator via the Ann Arbor
display used in the tachistoscope viewport for visual
feedback. This was essential for recording, recognition and
rejection errors in the voice-buffered mode; such errors
could not be analyzed from the hardcopy record if edited
from the voice buffer prior to transmiission of the buffer
contents to the text editor.
D. SUBJECT PREPARATION
1. T600 Vocabulary Training
Prior to the experiment, subjects were individually
trained in the use of the T6e0 to a level of knowledge and
competence to allow them to operate it to train the large
vocabulary of 255 words. Each subject was briefed on the
59
r ini tern haracop,/ ternincl
proper training of the TtZd, and received a derronstrat icn
and written instructions with the training (see Appendix C).
Cnee the subject had deironstrated proficiency in operating
and training the 1600, he/she was allowed tc proceed
independently, with the author remaining nearby to answer
questions and correct training pitfalls. Once training was
complete, the subject tested the vocabulary by saying each
word three times. Any words which were misrecognized or
rejected more than once were retrained until a goed training
pattern was established. Most retraining was required
because the subject forgot how the word was pronounced when
initially trained.
The training was normally accomplished in two
sessions of approximately two hours each. Thus by the time
the training was complete, the subject was very familiar
with the Te00. Approximately four hours was the average
time each subject spent with the vocabulary prior to
experimentation. The training patterns were stored on a
cassette tape for each subject and retained by the author
until experimentation.
2 . Typing Test
A five minute typing test was given to each subject
to group the subjects into 'FAST" and "SLO" typing ability
groups; these groups were necessary for the experimental
design. The typing test required only upper case letters
and symbols (Appendix D), as did tne experiment.
60
The typine' test was administered and scored
similarly to the civil service test used to screen clerk-
typist applicants to determine their typing- ability.
The typing tests were scored for speed and accuracy. A
raw score in words per minute was assigned according to
the number of lines typed. Credit was given for all lines
typed, including any portion of the last line started.
The number of words per minute was based on an average word
length of five characters. For each mispelled word, 0.2
words per minute were subtracted from the raw score,
tnereby decreasing the final score to deduct for errors. For
eianple, if a subject had a raw score of 40 wpm, but made 5
typing errors, the final score would be 39 wpm.
Subject typing speeds ranged uniform.ly from 17 to
58 words per minute, with an average speed of 43 words per
minute. The SLOW typist scores ranged from 17 to 32
with an average of 25? FAST typists scores ranged from 33 to
53 with an average of 43.
3. Subjective Questionnaire and Data Sheet
To assess the attitudes of each subject before and
after experimentation regarding their assessment of voice
data entry versus typed data entry, a 10 item subjective
•luestionnaire was developed (see Appendix I). The
questionnaire asked for the subject's opinions regarding the
61
voice and typing modes on concerns relating to usability
such as speed, accuracy, flexibility, training, and other
criteria.
Subjects also completed a short data sheet regarding
age, previous job, background, next assignment, and voice
experience. Appendix I contains the data sheet format.
E. EXPERIMENTAL PROCEDURE
As soon as the subject completed the vocabulary
training, he/she was scheduled to perform the experiment
which lasted between two and four hours, depending on the
speed of the subject. The experiments were conducted in the
iNPS Man-Machine Lab at times most convenient to the subject,
generally in the evening.
The subject was briefed concerning the general purpose
for the experiment and the three major parts of the
experiment: typing mode, voice-unbuffered mode, and voice-
buffered mode experimental conditions (see Appendix G).
Each experimental condition consisted of a practice card and
three trials. A Latin-Square determined the order of the
experimental conditions such that a balance was maintained
in the numbers of people starting each experimental mode.
This balance was also maintained on the second and third
experimental conditions for the subjects. In other words,
care was taken that no experimental condition received an
62
advantage froni always being first, second, or third-
Subjects were assigned randomly tc the orderings.
The subject's task for each aata entry mode was to write
12 simplistic on-line imagery interpretation reports of the
USSR/Warsaw Pact OB obtained from the cards by looking
through the viewport of the tachistoscope. Four cards were
included per trial for the three trials per mode.
Recall the sample cards in Figure lo; they were used for
typing (top) and voice (bottom) modes respectively, and
differed slightly. Since some utterances were actually two
or three words, (e.g. MIG-25 F0X3AT) and since the
vocabulary of equipment names were so large, it was
unrealistic to eipect the subject to recall which ones were
multiple words without greater familiarity with the
vocabulary. A convention was adopted to link such words
with an underscore symbol (_), such as MIG-25_F0XBAT, to
remind the subject that the name was tc be said in a single
utterance vice two or three utterances. The underscore was
the only distinction between the cards for voice and typing
modes.
The report format is shown in Figure 14. The subject
was required to report the installation number and C3
location (**) by quadrant in the order shown: UPPFR LEFT,
UPPER RIGHT, LOWER LEFT, LOWER RIGHT. Reports were to be
separated by a blank line.
63
I
4
INSTALLATION 0298-114217
UPPER LEFT
27 CONFIRMED ASU-57 AIRBORNE ASSAULT GUNS
UPPER RIGHT
£0 CONFIRMED ASU-85 AIRBORNE ASSAULT GUNS
LOWER LEFT
20 POSSIBLE M-20 HEAVY MORTARS
48 CONFIRMED 240-MM BM-24 ROCKET LAUNCHERS
LOWER RIGHT
62 PROBABLE 122-MM D-30 FIELD HOWITZERS
INSTALLATION 0199-V14197
UPPER LEFT
11 CONFIRMED MI-12 HOMER HELICOPTERS
6 PROBABLE MI-6 HOOK HELICOPTERS
UPPER RIGHT
16 CONFIRMED MI-4 HOUND HELICOPTERS
LOWER LEFT
19 PROBABLE MI-24 HIND HELICOPTERS
LOWER RIGHT
21 CONFIRMED MI-10 HARKE HELICOPTERS
Figure 14. OB Reporting Format Based on Cards in Figure 10
64
Subjects were allowed short breaks between trials and
longer breaks between the entry modes as they moved for
exapple from the typing portion to the voice-unbuffered
portion or vice-versa.
The number of characters per trial was balanced to a
very high degree within 10-lb characters and 10-15
utterances for all modes. The average number of keystrokes
per trial for the typing mode was 1170. The average number
of utterances per trial for the voice-unbuffered mode was
220/trial, slightly less than the 22e/trial for voice-
buffered. These keystrokes and utterances did not count any
editing keystrokes or utterances, but included all carriage
returns required. To perform the 3 modes i 3 trials, a
minimum of approximately 3510 keystrokes and 1344 utterances
would be required, plus any editing.
Prior to beginning each experimental condition the
subject was briefed on the entry mode, reminded of the
editing features available (delete character, delete word,
delete line, and repeat line), and allowed to practice the
entry mode by writing a report for a practice card.
The experimenter monitored the entire experiment at the
station illustrated in Figure 13. The elapsed time to
complete each trial was measured using a digital stopwatch
and recorded. The Miniterm provided a typescript for
analysis of the reports for missing or extra Information,
resulting from typing or voice recognition errors. Fxtra
65
I
4
I
I
I
■'i;?
typing -ceystrokes or voice utterances used for editing out
errors were noted for subsequent analysis for an efficiency
reasurement. The CRT display was used for the unbuffered
voice mode to record the misrecognitions and rejects since
they did not appear on the typescript if they were edited
prior to buffer transmission.
At the conclusion of the experiment the subject
completed the subjective questionnaire again. The subject
was asked not to discuss the experiment with others.
F. DEPENDENT VARIABLES
The following variables were recorded or calculated in
per cent for each trial:
NCC
Report Accuracy (RA)= - X 100
NCC + OS + CE
where NCC: Number of Characters Correct
OE: Omission Errors/missing data
CE: Commission Srrors/extra data
NCE/U
Mode Efficiency (MS) = -X 120
NCE/U + EK/U + SDK/U
where NCI/U: Number of Correct
Keystrokes/Utterances (Typing/Vcice)
SK/U: Error Keystrokes/Utterances
EDK/U: Editing Keystrokes/ Utterances
used to recover errors
66
NCR
Reccgniiicn Accuracy (RA) = -X ICC
NCR + NM
where NCR: Number of Correct Recognitions
NM; Number of Misreccgnitions
NCR
Recognition Accuracy (RAR) = - X 100
with Rejects NCR + NM + NR
where NCR: Number of Correct Recognitions
NM: Number of Misrecognitions
NR: Number of Rejects
Perhaps the most imiportant variable was the time it took
for a subject to complete the trials in the experiment.
Close behind time is accuracy, since reports must be valid
in addition to timely. Thus it is important to look at
report output in terms of accuracy as a system product.
Frequently experimenters examine the errors made with voice
and typing and report the results as percentage of error.
However in this experiment the final test is in the report
produced ... is it accurate? Next, how efficient is the
data entry mode? This is also a useful statistic for
judging the merits of each system. Accuracy and efficiency
were basic measures of the total system capability, l.e. the
man and the machine. Recognition accuracy was a measure of
TS00 performance alone, with operator errors such as
mispronunciation removed. Two recognition accuracy measures
were examined, but the first is considered most appropriate
in this experiment since the T60e did not output Incorrect
67
data but beeped when it rejected what should have been a
valid vocabulary utterance.
G. HYPOTHESIS
The following hypotheses were tested:
1. Hypotheses Regarding Tib’S
a. Eo :
There is no difference in TIME to
complete reports between FAST
and SLOW typists.
H, :
Eo false.
b. Ho :
There is no difference in TIME to
complete reports among the
THBEE DATA ENTRY MODES.
E, :
Eo false.
c. Ho :
There is no difference in TIME to
complete reports among the
THREE TRIALS.
H/ :
Ho false.
2. Hypotheses Regarding ACCURACY
a. Ho :
There is no differenoe in ACCURACY of
reports between FAST and SLOW typists.
H, :
Ho false.
b. Ho :
There is no difference in ACCURACY of
reports among the THREE DATA ENTRY MODES.
K/ :
Ho false.
68
I
i
I
c. Ho : There is no difference in ACCURACY cf
reports among the THREE TRIALS.
Ej : Ho false.
Hyootheses Reeiardins EFFICIENCY
a. H/
h.
H/
K,
There is no difference in EFFICIENCY
between FAST and SLOW typists.
Ho false.
There is no difference in EFFICIENCY
among the THREE DATA ENTRY MODES.
Ho false.
There is no difference in EFFICIENCY
among the THREE TRIALS.
Ho false.
4. Hypotheses Regarding T60e RECOGNITION ACCURACY
WITHOUT REJECTS
a.
b.
Ko
K/
Eo
There is no difference in RECOGNITION
ACCURACY between FAST and SLOW typists
Ho false.
There is no difference in RECOGNITION
ACCURACY among the TWO VOICE MODES.
Ho false.
There is no difference in RECOGNITION
ACCURACY among the THREE TRIALS.
Ho false.
69
Hypotheses Regaraing T600 RECOGNITION ACCURACY
WITH REJECTS
6 .
a.
Ho
c .
H/
Ho
H/
Hypothe
: There is no difference in RECOGNITION
ACCURACY WITH REJECTS between FAST and
SLOW typists.
: Ho false.
: There is no difference in RECOGNITION
ACCURACY WITH REJECTS amonfir the TWO
VOICE MODES.
: Ho false.
: There is no difference in RECOGNITION
ACCURACY WITH REJECTS arr.ong the THREE
TRIALS .
: Ho false.
sis Regarding SUBJECT ATTITUDES
Ho : There is no difference in SUBJECT
ATTITUDES regarding a preference for
VOICE DATA ENTRY over TYPED DATA
ENTRY after the experiment.
K/ ; Ho false.
H. EXPERIMENTAL DESIGN
The conceptual design for the experiment is illustrated
in Figure 15. This is a three-factor nested design with
repeated measures over trials. The subject is nested within
only one typing ability condition. Recall that one-third of
70
FAST
TYPING
ABILITY
(TA)
SLOW
TYPING V0IC3 VOICE
UNEUFEIRED BUFFERED
DATA ENTRY MODE (DEM)
Figure 15. Conceptual Design of the Experiment
71
the subjects started typing first; another third started
voice-unbuffered first, and another third started
voice-buffered first.
An analysis of variance procedure was selected to test
the hypotheses for reporting times, accuracy, and
efficiency, and T60? recognition rates. A significance
level of oc = 0.0c was used as the experimental threshold. A
sign test was chosen to evaluate the subjective
Questionnaire results at a significance level ofac= 0.10.
I. RESULTS
1. Results for Reporting Time
The results for reporting time were the most
significant, with an analysis of variance (ANOVA) indicating
SIGNIFICANT DIFFIRINCFS in the DATA ENTRY f^ODES and TRIALS
^p < .0005). The mean reporting times in Table I show the
average time in minutes to complete each of the reporting
trials for each of the three data entry modes. Table II
displays the results of the ANOVA for reporting time, and
Figures 16 and 17 illustrate the significant differences.
On the average, voice-unbuffered was 41% faster and
voice-buffered was 56% faster than typed data entry. Thus
voice data entry, averaging the two modes, was 50% faster
overall than typing. Voice data entry was faster because the
7Z
subject was able tc simultaneously receive information
through the viewport while composing the report. The
feedback received on the monitor enabled immediate
confirmation of the T6i30 response to his/her utterances.
The typist, in the conventional reporting mode, was forced
to return often to the viewport to get additional items of
information, since there was too much to memorize. The
illustrated differences may be seen in Figure 16.
Learning over trials is apparent in all three data
entry modes. Figure 17 illustrates the differences in time
to complete the scenario by trials. No significant
differences were noted between typing abilities. All
subjects adapted to the reporting task well. The voice-
buffered mode was the most natural for subjects to use,
since they could simply speak the report into the system,
and make corrections most easily. Thus they learned to use
it quickly, and improved slightly thereafter. The voice-
unbuffered and typing modes, with more room for improvement,
showed more learning as the subjects adapted to the
reporting scenario.
No significant difference was apparent between fast
and slow typists for this eiperinent. This was primarily
because the amount of information that the subject could get
from the tachisto scope was limited to the amount he/she
could memorize when moving back and forth to the manual
keyboard.
73
I
J
I
4
ABLE I
MEAN REPORTING TIME (MINUTES)
TYPING
VOICE
UNBUFFERED
VOICE
BUFFERED
FAST TYPISTS
Trial 1
16.2
11.6
10.5
Trial 2
13.6
10.5
10 .1
Trial 3
13.2
9.6
9.1
All Trials
14.3
10.6
9.9
SLOW TYPISTS
Trial 1
16 .0
12.7
10.0
Trial 2
16.6
10.6
9.8
Trial 3
15.6
10.5
9.2
All Trials
16.7
11.3
9.7
ALL SUBJECTS
Trial 1
17.1
12.2
10 .3
Trial 2
16.1
10.7
10.0
Trial 3
14.4
10.1
9.2
All Trials
15.6
11.0
9 .8
For
the
foil owing
analysis of
variance several
abbeviations
are
used for
the sake of
brevity. Their
ireauing is expanded below:
SS: Sum of Squares
df: degrees of freedom
MS; Mean Square
F: F Ratio
p: significance level
74
.'■fe
.l-M
TABLE II
ANALYSIS CF VARIANCE FCR REPORTING TI^'E (SECONDS)
SOURCE
SS
df
MS
F
P
BETWEEN SUBJECTS
: 3,558,£01.60
19
Typing Ability
(TA)
149,472.05
1
149,492.05
0 .78
NS
Error
3,439,329.61
18
191,073.87
WITHIN SUBJECTS:
6,588,801.20
160
Data Entry
rede (DEM)
3,969,141.26
2
1 ,964,570.64
61.61
TA X DEM
167,215.63
2
93,607.82
2.91
NS
Error(1)
1,159,579.54
36
32,210.54
Trials (Tr)
424,688.41
2
212,444.21
33.22
T A X T r
2766.70
2
1,383.35
0.22
NS
Error(2)
230,255.50
36
6,395.99
DEM I Tr
66,396.02
4
16,599.01
2.28
NS
TA I DEM X Tr
17,872.27
4
4,468.07
0.61
NS
Error(3}
525,207 .79
72
7,294.55
TOTAL
10,172,124.80
179
p < 0.0005
r NS: NOT SIGNIFICANT for p < 0.05 ]
75
^■ZAN TIME
(rrinutes)
17 +
9
TYPING VCICE-UNBUZFZRZD VOICZ-BUFFERZD
DATA ENTRY MODES (DEM)
Figure 16. Mean Reporting Time by Data Entry Mode
76
MEAN TIME
(minutes )
19 +
I
I
I
I
f
IS +
I
[TYPING
I
I
17 +MODE
j
I
I
I
I
I
15 +
I
14
I
+
I
I
I
13 +
I
jUNBUPFSRSD
12 +MODE
Figure 17. Mean Reporting Tirre iDy Trial
77
'i
'i
ck
' i
i
' i
j
‘ i
'I
(1
n
i i
■ »
■ 1
, »■
M
■ i
->
2. Results for Reporting Accuracy
The results for reporting accuracy are shown in
Tables III and IV. The analysis of variance fcr the
arcsin-transforired efficiency data revealed NO SIGNIFICANT
DIFFERENCES in ALL CONDITIONS investigated. The subjects,
whether fast or slow typists, did near perfect reporting in
each mode, over all trials. The reporting accuracy was
expected to be high, but exceeded the author's expectations.
An average of 99.5% accuracy was achieved for the
experiment.
Subjects were told to go as fast as possible, while
maintaining accurate reporting. I^ost errors were errors cf
omission, where a letter or worn was missing from a report.
Sven greater speeds could be expected, especially from
voice, in situations where m.ore errors could be tolerated.
But in the case of imagery reporting, accuracy was deemed
essential, even though operationally reports are normally
edited before being sent cut to the agencies.
TAELI III
MEAN REPORTING ACCURACY (%}
VOICE VOICE
TYPING
UNBUFFERED
BUFFERED
FAST TYPISTS
99.6
99.6
99.7
SLOW TYPISTS
99.2
99.4
99.6
ALL SUBJECTS
99.5
99.5
99.6
78
TABLE IV
ANALYSIS OF VARIANCE
FCR ARCSU-TEANSFORriD REPORTING ACCURACY
Y = 2 ARCS IN [SQRT( ACCURACY %)]
SOURCE
SS
df
MS
Y
P
BETWEEN SUBJECTS:
3.786
19
Tyt'in^ Ability
(TA)
0 .0ii34
1
0.004
0 .02
NS
Error
3.784
18
0.210
WITHIN SUBJECTS:
24.030
160
Bata Entry
Mode (DEM)
0.346
2
3.173
1.16
NS
TA X DEM
0.407
2
0.204
1.40
NS
Error(1)
5.262
36
0.146
Trials (Tr)
0.352
2
0.176
1.18
NS
TA I Tr
0.202
c
0.101
0.68
NS
Err or(2 )
5.362
36
0.149
DEM X Tr
0.395
4
0.099
0 .64
NS
TA I DEM I Tr
0.326
4
0.082
0.53
NS
Error(3 )
11.078
72
0.154
TOTAL
27.518
179
[ NS: NOT SIGNIFICANT for p
Note: Arcsin transform above normalizes
< 0.05 ]
the per
cent data
*
79
Results for Reporting Efficiency
The results fcr reporting efficiency are shewn in
Tables V and VI. The analysis of variance indicated
SIGNIFICANT DIFFFRZNCSS between the DATA ENTRY MODES.
Figure 18 shows the differences with typing being the most
efficient at 95%, voice-buffered next with an efficiency of
85%, and finally voice-unbuffered with an efficiency of 60%.
The author attributes the efficiency difference, in
part, to the level of experience with the mode. The reader
may recall that the subjects had, in general, extensive
keyboard experience during five quarters at NFS. In
comparison with typing, the subjects had very little
experience with voice. It is expected that if subjects were
more skilled and efficient in the use of voice data entry,
the time advantages reported earlier would be even more
dramatic. Voice-buffered was more efficient than voice-
unbuffered because the subject could edit out an entire
incorrect utterance, vice deleting it by voice a word at a
time in the unbuffered mode.
80
TABLZ V
f^EAN REPORTING
TYPING
EEEICISNCY {%)
VOICE
UNBUFFERED
VOICE
BUFFERED
iAST TYPISTS
Trial 1
93.6
77.2
63.5
Trial 2
95.1
80.5
85.7
Trial 3
93.6
81.6
83 .3
All Trials
94.2
79.6
84.2
SLOW TYPISTS
Trial 1
94.4
80.0
66 .3
Trial 2
95.8
64.4
64.4
Trial 3
96.7
76.9
68 .4
_——
All Trials
95.6
80.4
86.4
ALL SUBJECTS
Trial 1
94.0
78.6
84.9
Trial 2
95.4
62.5
85.0
Trial 3
95.3
79.3
65.8
All Trials
94.9
60.1
65 .2
ei
TABLE VI
ANALYSIS OE VARIANCE
FOR ARCSIN-TRANSFORMED REPORTING EFFICIENCY
Y = 2 * ARCSIN [SQRT(EFFICISNCY %)]
SOURCE
SS
df
MS
P
BETWEEN SUBJECTS:
3.059
19
Typing Ability
(TA)
0.134
1
0.134
0 .82
NS
Error
2.925
18
0.163
WITHIN SUBJECTS:
15.689
160
Data Entry
Mode (DE^^)
7.102
2
3.551
44 .95
TA X DEM
0.023
2
0.011
0.14
NS
Error(1)
2.660
36
0.079
Trials (Tr)
0.17 0
2
0.085
3.54
NS
TA I Tr
0 .020
2
0.010
0 .42
NS
Srror(2)
0.860
36
0.860
DIM X Tr
0.167
4
0.042
1.40
MS
TA X DEM X Tr
0.301
4
0.075
2.50
NS
Error(3)
2.186
72
0.030
TOTAL
16.748
179
** p <
0.001
[ NS: NOT SIGNIFICANT fcr p < 0.05 ]
82
.rEAN EFFICIENCY
100 % +
I
I
I
I
I
I
j
95 % +
I
I
I
I
I
I
I
90% +
I
I
I
I
I
t
I
85% +
I
I
I
I
I
I
I
80% +
I
I
I
f
1
I
I
I
I w/b ^
I
I
I
I
I
I
II I I
II I I
^2% _
TYPING VOICE-UNBUJFEP.ED VOICE-BUFFERED
DATA ENTRY MODES (DEM)
Figure 18. Mean Reporting Efficiency ty Data Entry Mode
83
Results for T600 Recognition Accuracy
4.
The results fcr the Te00 Recognition Accuracy are
shown in Tables VII, VIII, IX, and X. Analysis of variance
cf the results revealed NO SIONIIICAMT DIFFERENCES fcr ALL
CONDITIONS considered. Thus the T600 recognized all
subjects equally well during all trials of the experiment.
The 1600 recognition accuracy was 97.0% overall if an error
is defined as a misreccgnition only. If rejects are
included, the recognition accuracy drops to 96.5% as an
overall average.
These results are based on an average of 1519
utterances per subject giving 30,380 utterances for the
entire experiment using 20 subjects. This number includes
the utterances required, plus misrecognitions and reject
utterances, and finally the editing utterances used to
correct errors. A list of misrecognitions and rejects is
contained in Appendix n.
The author had expected the recognition
accuracy to get worse in later trials from fatigue or
frustration, since the experiment was two tc four hours in
length. One procedure that may have helped was to
allow subjects to, upon their request, retrain troublesome
words during the course cf the experiment. The time tc
retrain was counted against the trial time to account for
realistic retraining that would take place on the job.
64
Table vii
hEAN T622 RECOGNITION ACCURACY (%)
WITHOUT REJECTS
VOICE
UNBUFFERED
VOICE
BUFFERED
FAST
TYPISTS
97.0
97.1
SLOW
TYPISTS
97.0
96.9
ALL
SUBJECTS
97.0
97.0
TABLE
VIII
MEAN T600 RECOGNITION ACCURACY
WITH
REJECTS
VOICE
UNBUFFERED
VOICE
BUFFERED
FAST
TYPISTS
95.8
95.4
SLOW
TYPISTS
95.2
95.4
ALL
SUBJECTS
95.5
95.4
65
{
TABLE IX
ANALYSIS 0? VARIANCE
ARCSIN-TRANSFORhED T6ee RECOGNITION ACCURACY
WITHOUT REJECTS
Y = 2 - ARCSIN [SORT(ACCURACY %)]
SOURCE
SS
df
MS
F
P
BETWEEN SUBJECTS:
0.S64
19
Typing Ability
(TA)
0.001
1
0.001
0.02
NS
3rrcr
0.663
18
0.048
WITHIN SUBJECTS:
1.033
100
Data Entry
Mode (DSM;
0.000
' 1
0.000
0 .00
NS
TA I DEM
0.009
1
0.009
0.69
NS
Error(1)
0.231
18
0.013
Trials (Tr)
0 .009
2
0.005
0.63
NS
TA I Tr
0.037
2
0.019
2.38
NS
Error(2)
0.261
36
0.008
DEM. X Tr
0.053
2
0.027
2 .45
NS
TA I DEM I Tr
0.032
2
0.016
1 .45
NS
Error(3)
0.381
36
0.011
TOTAL
1.697
119
[ NS: NOT SIGNIFICANT for p < 0.05 ]
66
TABIS X
ANALYSIS C? VARIANCE
ARCSIN-TRANSFOR^’ED RECOGNITION ACCURACY
'*ITH REJECTS
Y = 2 * ARCSIN [SORT(ACCURACY %)]
SOURCE
SS
df
MS
Y
P
BETWEEN SUBJECTS:
0 .926
19
TyDin^ Ability
(TA)
0.000
1
0.000
0.00
NS
Error
0.926
16
0.051
WITHIN SUBJECTS:
1.106
100
Bata Entry
Mode (DEM)
0.000
1
0.000
0.00
NS
TA X DEM
0.004
1
0.004
0 .33
NS
Error(1)
0.224
18
0.012
Trials (Tr)
0.001
2
0.000
0 .00
NS
TA I Tr
0.034
2
0.017
2 .43
NS
Err or(2 )
0.256
36
0.007
DEM X Tr
0 .046
2
0.023
1.64
NS
TA X DEM X Tr
0.018
2
0.009
0.64
NS
Error(3)
0.521
36
0.014
TOTAL
1.977
119
[ NS :
NOT SIGNIFICAN
T for
p < 0.05 ]
87
During the experirent the author observed that
subjects occasionally became frustrated when the
T600 was misrecognizing their utterances. This
frustration appeared to, at times, generate a lack of
confidence in the T600, along with a change in the
overall pitch, rate, and inflection of the voice.
The frustration seemed more prevalent in the
unbuffered than the buffered mode. ?or this reason, the
T600 buffered mode was expected to have a better
recognition rate, since it was faster and somewhat easier
to use. However the results indicate there is no
difference in the recognition rate. One explanation is that
subjects went faster in the buffered mode since they could
correct the misrecognitions more easily. With the
consequence of a misreccgnition reduced, they were less
afraid to make mistakes.
5. Results for Subject Attitudes
The scores from the subjective questionnaire given
before and after the experiment were tested for any general
change in opinion regarding voice versus typed data entry.
These scores were evaluated using a two-tailed nonparametric
sign test, flc = 0.10. A significant snift in favor of voice
data entry over typing occured for half of the criteria
covered by the questionnaire. No significant shifts toward
typing resulted from the analysis. Appendix I contains the
results of the pre/post questionnaire.
88
S uirrnar i z ins the results, subjects either were
neutral or favored voice before and after the eiperirT:ent.
After the experiment, they preferred voice even more for
ease of use, speed, flexibility, intermittent use, and
finally ease of learning to use as an input modality. They
continued to believe that voice was a more accurate,
sustaining, relaxed man-machine interface fcr on-line
reporting of critical, tine-sensitive information such as
intelligence obtained in a high-pressure work environment.
The subjects' positive attitudes about voice arise
from their fresh experience and observations of speech
recognition equipment in the C3 Lab at NPS, where it is used
with the Wargame Effectiveness Simulator (WES) with graphics
and other ARPANET and laboratory facilities to demonstrate
its potential for command, control, and communications
applications.
S9
III. DISCUSSION
A. DEMRAL
This thesis investigated tne potential application of
automatic speech recognition technology to military imagery
interpretation reporting. Only the order of tattle portion
of reporting was investigated because of limited time and
resources. The overall results of the experiment are
supportive of the application of voice data entry for
imagery interpretation reporting systems. Voice-buffered
mode was 58% faster than typing, while voice-unbuffered was
41% faster. On the average, voice was 50% faster than
typing.
Voice was faster because it allowed the operator to view
the image while reporting. This experiment modeled
conventional imagery reporting systems where a light table
is located next to a computer console. The operator
must move back and forth between the light table and the
console, or two operators work together, with one
interpreting the imagery, and the other writing the report
via the console. For these situations, it appears voice
data entry would significantly improve reporting speeds
and/or require only one person per station to perform the
task. For never systems with the keyboard and function keys
built into a computer console with a light table or digital
90
displa/, voice na/ aot have as significant an irrpact for
improvin? reporting speeds.
Both voice and typing were very accurate for
the experimental task, with no significant difference
letveen modes and an overall accuracy of 99.5%. It is
interesting - to note these speeds and accuracies were
obtained even though subjects were less efficient with
either mode of voice. Voice-unbuffered had 80.1%
efficiency, voice-buffered had 65% efficiency, and
typing had 95% efficiency. These results were all
attained at a significance level of x = 0.05 or better.
In terms of recognition accuracy, the results were
tetter tnan the author expected. Poorer results were
expected because short phrases consisting of several
utterances were used rather than simple one or two utterance
commands. It was anticipated that subjects would run words
together more than they actually did, and it was also
anticipated that the Te00 would have more trouble with
similar sounding terms such as MICt-25 FOXBAT and f^IG-25R
?0XEAT...or CEARLIZ I CLASS and CHAHLIF II CLASS. Though
the 1500 did misrecognize such words at times, subjects
quickly adapted to the situation, emphasizing the portion of
the utterance that was unique, thereby achieving
better results. The 97% overall recognition accuracy
would likely improve with practice and increased usage.
Additionally, new high-speed recognition systems, like
91
1
4
i
m
m
4
m
Threshold's QUICLTALK (Trademark), require a much shorter
pause between utterances, thus permitting the operator to
speak faster. QUICKTALL is advertised to reach speeds of
16? vords-per-minute, and 99^ accuracy for moderately
trained speakers. Vocabulary structuring may also be
performed which allows the system to search only a subset of
the vocabulary, thus increasing the speed and accuracy of
recognition. This system, as advertised, has twice the
speed of the T600 used in the experiment.
Subjects tendea to prefer voice before and after the
experiment (even more). For tne vast majority of subjects,
this was the first use of voice continuously for an extended
period of time. Even though it did not meet some of their
more lofty expectations, they continued to give voice the
edge in the subjective questionnaire, and actually
strengthed their opinions toward it on several criteria.
Thus this experiment, though outside an operational
setting, supports further research and possible applications
of ASR for imagery interpretation reporting systems, and
perhaps other similar intelligence and tactical cc^mand end
control data systems. The results are certainly not new,
but add credence to the related results achieved by RADC,
NFS, and others.
Use of the ARPANET facilities in this
experiment demonstrated, to a limited degree, that
reporting can be performed without the benefit of a
92
local host computer. This may be very beneficial in the
future if department of defease organizations want to
remctely query cr update a ccmmcn data base.
E. RICOrfiENDATICNS
1. Research
The time is perhaps ripe for the military to perform
some research using voice data entry as a keyboard assist
for one or mere of the current imagery reporting systems,
such as TIPI IIS, f^AH.RZS, CATIS, PACZR, AIRZS, and others.
Ey beginning now to look at the use of voice fer these
systems, the intelligence community may be able to identify
the specific questions needing to be addressed to most fully
adapt voice as an input mcdality. In the next five cr ten
years, the outlook for "matter-of-fact" use of voice is
good. By studying the problems associated with training,
user acceptance, physical interfacing, vocabulary size,
vocabulary data-base maintenance, respense times, and other
areas now, voice will be more easily applied later.
Additionally, voice input may be applied to other
tasks associated with the other intelligence disciplines
using interactive computer-controlled devices. Command
center applications are also receiving increased attention
as natural language query systems coupled with graphics
displays commanded by voice are now a reality in terms of
advanced applications technology.
93
All new irr.agery exploitation s/sterrs being developed
or o’Cdified snculd fully consider the benefits of voice
recognition technology. Considering the three to eight years
it takes to develop a new syster;, it is highly likely that
by the time it is fielded, significantly more voice
capabilities will be available. Special consideration
should be given to not only to how it might aid interpreters
in the reporting process, but also how they might be able to
use it to enhance, manipulate, annotate, and otherwise
modify digital softcopy imagery on systems such as Compass
Preview.
2. Applications
Practical applications using voice data entry on a
large scale will require a significant amount of work. It
must also be proven that while voice may be as fast or
faster than typing that the time differential achieved
contributes commensurately with the additional cost of such
new technology. Careful attention must be paid to involving
the users, since they will ultimately "sell" the system,
even though proven in the lab.
The author recommends a small application first with
a few of the best interpreters who know the imagery
system well, and are ambivalent regarding voice data entry.
Py allowing them to use voice on a daily basis, they can
develop the in-house expertise at the level needed to apply
it on a large scale later...or they may be able to assess
94
that it just won't work for that particular application.
The oiilitary is fortunate, having excellent research
people involved with voice technolo,ey. RADC and NFS are
just two military institutions able to provide consultation
and assistance.
C. CONCLUSIONS
Since 1972, automatic speech recognition has
proven to
be
valuable for
a number
of limited
applications .
The
future for the
technclogy
is bright.
The author concludes voice is not only feasible, but
desirable as a means toward the best imagery
interpretation reporting possible. It is not so much a
question of whether voice can be used, but rather ...
how can it be used?...how extensively can it be used?...and
how cost-effective will it be?
95
APP2NDIX A
USSR/WARSAW PACT ORDER Of BATTLE (OB, VOCABULARY
INSTRUCTIONS: TRAIN TEE WORDS IN THE GIVEN SEQUENCE, USING
TEE GIVEN PROMPT. WORD NUMBERS MARKED WITH AN ASTERISK MAY
EE TRAINED WITH THE GIVEN PROMPT OR YOU MAY USE YOUR OWN.
(THESE WORDS WILL BE USED fOR TEXT EDITING, AND THUS SHOULD
BE FAMILIAR, EASY TO REMEMBER) BE SURE TO WRITE IN THE
ONE THAT YOU USE ON TEE VOCABULARY LISTING SO THAT YOU MAY
HAVE IT FOR FUTURE REFERENCE.
WORD
PROMPT
OUTPUT
ZERO
0
1
ONE
1
2
TWO
c
3
THREE
3
4
FOUR
4
FIVE
c.
5
S IX
6
7
SEVEN
7
6
EIGHT
8
9
NINE
9
10
ALPHA
A
11
BRAVO
B
12
CHARLIE
C
13
DELTA
D
14
ECHO
E
15
FOXTROT
A
16
GOLF
G
17
HOTEL
H
18
INDIA
I
19
JULIET
J
20
KILO
K
21
LIMA
L
22
MIKE
M
23
NOVEMBER
N
24
OSCAR
0
25
POPPA
P
26
QUEBEC
Q
27
ROMEO
R
28
SIERRA
S
29
TANGO
T
30
UNIFORM
U
31
VICTOR
V
32
WHISKEY
w
33
XRAY
X
96
til
34
YANKE3
Y
35
ZULU
Z
36
POSSIBLI
POSSIBLE
37
PROBABLE
PROBABLE
36
COMFIRhSD
_CONFIRMED_
39
DASH
-
40-
IRAS Z
BKSP <CTRL A>
41
GO OR CARRIAGE RETURN
^CARRIAGE RETUR
42
SLASH
/
43-
KILL IvORD
<CTRL V>
44*
KILL LINE
<CTRL X>
45*
REPEAT LINE
<CTRL R>
46
SPACE
<SPACE CHARACTE
47
TEN
10
46
INSTALLATION
INSTALLATION
49
ELEVEN
11
50
UPPER LEFT
UPPER LEFT
51
TANKS
TANKS
52
LIGHT
LIGHT
K"?
W W
MEDIUM
MEDIUM
54
HEAVY
HEAVY
55
T72
T-72
56
Tc2
T-62_
57
T54/55
T—54/55
56
T10
T-10
59
T34/65
T-34/85
60
TWELVE
12
61
PT76
PT-76
62
AMPHIBIOUS
AMPHIBIOUS
63
UPPER RIGHT
UPPER RIGHT
64
APC
APC
65
AT GW
ATGW
66
ERDM
BRDM
67
BTR6ePK
BTR-60PK
68
BMP76PB
BMP-76PB
69
ETR152
BTR-152
70
ETR50PK
BTR-50PS
71
FIELD HWTZRS
FIELD HOWITZERS
72
ASU65
ASU-&5
73
SU100
SU-100
74
AIRBORNE
AIRBORNE
75
LOWER LEFT
LOWER LEFT
76
D3e
D-30
77
AT3 SAGGER
AT-3 SAGGER
78
ANTI-TK GUNS
ANTI-TANK GUNS
79
D74
D-74
£0
B20
D-20
61
M1955
M-1955
82
D44
D-44
63
EM21
EM- 2 I
64
M1976
M-1976
97
65
be
67
66
69
90
91
92
93
94
95
96
97
96
99
100
101
102
103
104
105
106
107
10S
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
126
129
130
131
132
133
134
135
3f^24
EM-24
f?.0G3
FROG-3
FH0G4
FROG-4
FR0G7
FROG-7
SCUD A
SCUD-A
SCUD E
SCUD-E
SS12 SCL3RD
SS-12 SCALSBOAED
SSM
SSM
ATI SNAPPIR
AT-1 SNAPPER
65 MIDIMETER
85-MM
100 MILIMETR
100-MM
SA4 GANZF
SA-4 GANEF
SA6 GAINFUL
SA-6 GAINFUL
SA6 GECKO
SA-6 GECKO
SA9 GASKIN
SA-9 GASKIN
LAUNCHERS
LAUNCHERS
THIRTEEN
13
ASW
ASW
FOURTEEN
14
AA GUNS
AA GUNS
FIELD GUNS
FIELD GUNS
ZU23/2
ZU-23/2
ZSU23/4
ZSU-23/4
ZSU57/2
ZSU-57/2
S60
s-6e
M44
M-44
M49
M-4 9
57 MILIMETER
57-MM
SU15 FLAGON
SU-15 FLAGON
YAK28P FRBAR
TAK-2SP FIREEAR
TU28P FIDLR
TU-28P FIDDLER
MIG19 FARMER
MIG-19 FARMER
MIG21 FSHBED
MIG-21 FISHBED
MIG23 FLGGSR
MIG-23 FLOGGER
MIG25 FOXBAT
MIG-25 FOXBAT
MIG27 FLGGER
MIG-27 FLOGGER
TU20 BEAR
TU-20 BEAR
TU126 MOSS
TU-126 MOSS
SU9 FISHPOT
SU-9 FISHPOT
MIG25R FXBAT
MIG-25R FOXBAT
TU22 BLINDER
TU-22 BLINDER
TU16 BADGER
TU-16 BADGER
TU26 BACKFIR
TU-26 BACKFIRE
MI4 HOUND
MI-4 HOUND
MI12 HOMER
MI-12 HOMER
MI6 HOOK
MI-6 HOCK
MI6 HIP
MI-6 KIP
MI10 HARKE
MI-10 HARKE
Mi 124 HIND
MI-24 HIND
IL38 MAT
IL-38 MAY
M-4 BISON
M-4 BISON
96
136
137
136
139
142
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
156
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
161
162
183
184
185
186
SU19 FENCE?.
FIFTEEN
AN8 CAFP
AN 12 CUB
AN22 COCK
AN26 CURL
KA15 HEN
KA18 HOG
KA25 HORMONE
IL12 COACH
IL14 CRATE
IL28 BEAGLE
IL76 CANDID
AWACS
BE12 MAIL
TRANSPORTS
FIGHTERS
BOMBERS
FIGKTER-BMRS
STRIKE/ATTCK
HELICOPTERS
RECONNAISNC
SS
FRIGATE
SSB
SSGN
SSBN
CARRIER
CRUISERS
DESTROYERS
MINESWEEPERS
FRIGATES
CORVETTES
MISSLE
TORPEDO
BOATS
LANDING
SIXTEEN
INTELLIGENCE
SHIPS
SEVENTEEN
EIGHTEEN
KIEV CLASS
MOSKVA CLASS
SSN
DELTA CLASS
DELTA2 CLASS
H0TEL2 CLASS
HOTELS CLASS
ASU57
VICTOR CLASS
SU-19 FENCSR
15
AN-e CAMP_
AN-12 CU3_
AN-22 COCK_
AN-26 CURL_
^A-15 HEN_
KA-IS EOG_
KA-25 HORMONE_
IL-12 CCACK_
IL-14 CRATE_
IL-2S BEAGLE_
IL-76 CANDID_
AWACS
BE-12 MAIL_
TRANSPORTS
FIGHTERS
BOMBERS
FIGHTER-BOMBERS
STRIKE/ATTACK
HELICOPTERS
RECONNAISSANCE
SS
FRIGATE
SSB
SSGN
SSBN
CARRIER
CRUISERS
DESTROYERS
MINESWEEPERS
FRIGATES
CORVETTES
MISSLE,
TORPEDO,
BOATS
LANDING,
16
INTELLIGENCE,
SHIPS
17
16
KIEV CLASS,
MOSKVA CLASS,
SSN
DELTA CLASS,
DELTA II CLASS,
HOTEL II CLASS,
HOTEL III CLASS
ASU-57,
VICTOR CLASS
99
■.«
i
i
187
ise
1S9
19 0
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
ccc
223
224
225
226
227
22S
229
230
231
232
233
234
235
236
237
TANKZE CLASS
GOLFl CLASS
G0LF2 CLASS
ZULU4 CLASS
KRZSTAl CLAS
:<HZSTA2 CLaS
hlRAAl CLASS
MIRKA2 CLASS
PETYAl CLASS
PETYA2 CLASS
JULIET CLASS
LOWER RIGHT
YANKEE CLASS_
GOLF I CLASS,
GOLF II CLASS,
ZULU IV CLASS
KRSSTA I CLASS,
KRESTA II CLASS
hiRKA I CLASS,
MIRKA II CLASS,
PETYA I CLASS,
PETYA II CLASS,
JULIET CLASS,
LOWER RIGHT
122 MILIMETR
122-MM
FOXTROT CLAS
FOXTROT CLASS
ROMEO CLASS
ROMEO CLASS
SSG
SSG
BRAVO CLASS
BRAVO CLASS
ECEOl CLASS
ECHO I CLASS
SCH02 CLASS
ECHO II CLASS
152 MILIMETR
152-MM
TANGO CLASS
TANGO CLASS
WHISKEY CLAS
WHISKEY CLASS
CEARLIEl CLS
CHARLIE I CLASS
CHARLIE2 CLS
CHARLIE II CLASS
KARA CLASS
KARA CLASS
SVERDLOV CLS
SVERDLOV CLASS
KYNDA CLASS
KYNDA CLASS
KRIVAK CLASS
KRIVAK CLASS
KASHIN CLASS
KASHIN CLASS
240 MILIMETR
240-MM
KANIN CLASS
KANIN CLASS
INTERCEPTORS
INTERCEPTORS
KOTLIN CLASS
KOTLIN CLASS
KOTLN SAM CL
KOTLIN-SAM CLASS
SKORY CLASS
SKORY CLASS,
RIGA CLASS
aIGa CLASS
GRISHA CLASS
GRISHA CLASS
NANUCKKA CLS
NANUCHKA CLASS
POTI CLASS
POTI CLASS
OSAl CLASS
OSA I CLASS
0SA2 CLASS
OSA II CLASS
KOMAR CLASS
KOMAR CLASS
STENKA CLASS
STENKA CLASS
NINETEEN
19
TWENTY
20
SHERSHSN CLS
SHERSHEN CLASS,
TWENTY-ONE
21
NATYA CLASS
NATYA CLASS
YURKA CLASS
YURKA CLASS
ALLIGATOR CL
ALLIGATOR CLASS
POLNOCNY CLS
POLNOCNY CLASS
100
236
239
240
241
242
243
244
245
24c
247
24S
249
250
251
252
253
254
T’aENTY-T’aO
PRIMORYS CIS
TWINTY-THRZZ
TWZNTY-ZOUR
SS16
SS20
5514 SCFGOA?
5515 SCROOGE
ICEr
IRBh
MOBILE
M240
MORTARS
ASSAULT GUNS
ROCKET LCHRS
AIRCRAFT
TWENTY-FIVE
22
PRIMORYE CLASS,
23
24
ss-ie_
SS-20_
SS-14 SCAPEGOAT,
SS-15 SCROOGE,
I CBM
IRBM
MOBILE
M-240
MORTARS
ASSAULT GUNS
ROCKET LAUNCHERS
AIRCRAFT
25
101
APPENDIX 3
SCENARIO CARDS
TYPING CARDS ->>>>>>>> FIRST TWELVE
installation 0613-T11214
I
** 4 CONEIRMED 3MP-76PB APC
7 CONEIRNiSD BRLM APC
I
3 CONFIRMED AT-3 SAGGER ATGW **
4 PROBABLE ZSU-23/4 AA GUNS
j
40 CONFIRMED T-54/55 MEDIUM TANKS **
j V 'T'
4 PROBABLE SA-9 GASKIN LAUNCHERS
I
I
I
I
I
6 PROBABLE ZU-23/2 AA GUNS
_I __
I ” “
INSTALLATION 0115-T12314
5 CONFIRMED M-4 BISON BOMBERS
j
** 1 POSSIBLE TU-2e BEAR RECONNAISSANCE
AIRCRAFT
12 CONFIRMED TU-20 BEAR BOMBERS
1 CONFIRMED TU-126 MOSS AWACS
i
_I_
2 CONFIRMED BE-12 MAIL RECONNAISSANCE
AIRCRAFT i
I
7 CONFIRMED IL-2b BEAGLE BOMBERS
17 CONFIRMED TU-IS BADGER BOMBERS **
3 PROBABLE TU-16 BADGER RECONNAISSANCE
AIRCRAFT
102
INSTALLATION 0128-T13213
! /
2 CONFIRMED KHESTA II CLASS CRUISERS
3 CONflRMED ERESTA I CLASS CRUISERS
1
1 POSSIBLE TANGO CLASS SS
>
I
1
j
--"l2"CCNFrRMED“WErSKlY"CLASS“SS
I
2 PROBABLE CEARLIE ll CLASS SSGN
1
I
S
I
j
1 CONFIRMED CHARLIE I CLASS SSGN
I
I
I
1_______I__ _—______
I INSTALLATION 0298-T14218
I }
I 50 CONFIRMED ASU-65 AIRBORNE ASSAULT GUNS
I j
1 27 CONFIRMED ASU-57 AIRBORNE ASSAULT GUNS
I I
[ 1
I I
I I
I i
I I
I_i_I_
I t
I 20 POSSIBLE M-240 HEAVY MORTARS
1 j
162 PROBABLE 122-MM D-30 FIELD HOWITZERS
i I
1 I
j I
146 CONFIRMED 240-MM EM-24 ROCKET LAUNCHERS
1 i
I I
I JU I
____ 1 ....._......___
I
103
INSTALLATION 0S27-T21253
6 CONFIRMED FOXTROT CLASS SS
I
12 CONFIRMED JULIET CLASS SSG
** 2 PROBABLE DELTA II CLASS SSEN
j
3 PROBABLE DELTA CLASS SSBN
I
I
”CONHRMSD"GOLf I l”CLASS”SS
I
I
j
5 CONBiRMEi} POTI CLASS CORVETTES
I
I
2 POSSIBLE YANKEe'cLASS SSBN
I
7 PROBABLE ROMEO CLASS SS
INSTALLATION 0405-T22217
40 CONFIRMED T-10 HEAVY TANKS
I
57 CONFIRMED T-34/85 MEDIUM TANKS
43 CONFIRMED T-54/55 MEDIUM TANKS
I
I
_I_
3 CONFIRMED PT-76 LIGHT AMPHIBEOUS TANKS
j
e CONFIRMED BTR-152 APC
I
j
6 CONFIRMED BRDM RECONNAISSANCE APC
104
INSTALLATION •2352-T23224 **
I /
11 CONflRMEL TU-22 BLINDER BOMBERS
I
I
20 CONJIRMSD TU-2e BACKFIRE BOMBERS
5 probable IL-28 BEAGLE BOMBERS
{ >!£#
2 CONFIRMED IL-76 CANDID TRANSPORTS
I
I
j *Sr
15 CONFIRMED AN“12 CUB TRANSPORTS
I
I
j SrS}*
7 CONFIRMED MI-8 HIP HELICOPTERS
_____ _ _ • _ __—__—__
INSTALLATION 0247-T24283
5 probable koi^ar'class i^issle boats
17 CONFIRMEE OSA I CLASS MISSLE BOATS
I
5 CONFIRMED OSA Il'cLASS MISSLE BOATS
!
I
I
j
7 CONFIRMED STENKA CLASS TORPEDO BOATS
I
I
j
11 POSSIBLE NANUCHKA CLASS TORPEDO BOATS
I
6 POSSIBLE GRISHA'class CORVETTES
2 PROBABLE SHSRSHEN CLASS TORPEDO BOATS*-
105
1 INSTALLATION 0243-T31£7£
j j
! 12 CONFIRr^ED MIG-27 FLCGGZR STRIKE/ATTACK
1 AIRCRAFT I
il6 CONFIRMED SU-19 FENCER STRIKE/ATTACK
i AIRCRAFT I
I I
! 2 POSSIBLE MIG-25R FOXEAT RECONNAISSANCE
1 AIRCRAET ** !
** 1 CONFIRMED IL-38 MAY ASV AIRCRAFT
{
3 CONFIRMED AN-6 CAMP TRANSPORTS
t
I
5 CONJIRNSD AN-26 CURL TRANSPORTS
j
I
I
_I___
INSTALLATION 0657-T32179 **
I / **
2 CCNFIRMPD HOTEL II CLASS SSBN /
1 CONFIRMED HOTEL III CLASS SSBN
I
I
I
I
I
I
_I_
[
1 PROBABLE GOLF I CLASS SSB
1 PROBABLE MIRKA I CLASS LIGHT FRIGATE
I
I
j
1 POSSIBLE ZULU IV CLASS SS
!
I
106
INSTALLATION 0410-T33252
** 4 CONFIRMED 100-MM M-49 AA GUNS
4 CONFIRMED ZSU-57/2 AA GUNS ---
6 CONFIRMED 65-MM M-44 AA GUNS
I ##
j
/ !
8 CONFIRMED FROG-4 SSM MOBILE LAUNCHERS
I
_ ___I __
I
j
6 PROBABLE AT-1 SNAPPER ATGW
I
*5^ j
4 CONFIRMED 122-MM D-74 FIELD GUNS
I
I
21 CONFIRMED 85-MM D-44 ANTI-TANK GUNS
_I_
I
INSTALLATION 0173-T34246
1 CONFIRMED TU-126 MOSS AWACS /
1 CONFIRMED TU-16 BADGER RECONNAISSANCE
AIRCRAFT !
16 CONFIRMED AN-22 COCK TRANSPORTS
>!s5it {
I
18 CONFIRMED TU-20 BEAR BOMBERS
/
12 CONFIRMED TU-22 BLINDER BOMBERS
2 CONFIRMED TU-20 BEAR RECONNAISSANCE
AIRCRAFT
! ❖’i'
107
VC ICI-
UNBUFFERED CARDS -> > > NEXT TWELVE
INSTALLATION glSe-VllLSC
I /
9 probable YAK-28P_FIREBAR fighter-boneers
** 12 CONFIRrSD SU-15_FLAG0N INTERCEPTORS
20 CONFIRMED TU-28P_FIDDLER INTERCEPTORS
j ##
j **
T3“PR0£ABLE"F!Ig-25“F0XBAT“InTERCEPTCRS
11 POSSIBLE SU-9_FISEP0T FIGHTERS
I
I
15 PROBABLE MIG-21_FISHBED FIGHTERS**
_I___
INSTALLATION 0357-V12252
j
1 CONFIRMED MOSKVA_CLASS CARRIER
I
Sjc# I
1 CONFIRMED KIEV_CLASS CARRIER
** 2 PROBABLE KARA_CLASS CRUISERS
I
__________ ^ _____________________
** 3 CONFIRMED KASEIN_CLASS DESTROYERS
** 4 CONFIRMED KRIVAK_CLASS FRIGATES
>!«={! j
8 CONFIRMED MIRKA_II_CLASS LIGHT FRIGATES
108
_I___
INSTALLATION 0168-V13259
6 probable S-60 l^;ZDIUM AA GUNS
I
4 CCNFIRI^ED SA-8_GZCK0 LAUNCHERS
I
I
I
3 CONFIRMEL SA-4^GANEF LAUNCHERS
I
I
I
_I___
:Jc3;s I ^ -
4 CONFIRMED SA-6^GAINFUL LAUNCHERS
I
3 CONFIRMED SS-12_SCALEBGARD MOBILE SSM
I
5 CONFIRMED FROG-3 MOBILE SSM
I
I
I
4 CONFIRMED SA-9^GASKIN LAUNCHERS
— —_ j _ __ —
INSTALLATION C199-V14197
I jjcsie
le CONFIRMED MI-4 HOUND HELICOPTERS
>!!# I
11 CONFIRMED MI-12_E0MER HELICOPTERS
I
** 5 PROBABLE MI-6 HOOK HELICOPTERS
[ *>!!
21 CONFIRMED MI-10 EARKE HELICOPTERS
Jltsie
19 PROBABLE MI-24 HIND HELICOPTERS
I
I
I
I
___I _____
109
INSTALLATION 02e6-V21221
I :jt#
1 CONFIEMSL SS-ie MOBILE ICEM
j
** 1 POSSIBLE SS-lb_SCROOGE MOBILE IRBM
1 CONFIRMED SS-14_S CAPEGO AT MOBILE IRBM
2 probable SS-20 MOBILE IRBM
** 1 CONFIRMED FROG-7 SSM
** 3 CONFIRMED SCUD_A SSM
I
I
I
1 POSSIBLE SCUD_B SSM
I
I
I
I
INSTALLATION 0195-722231
10 CONFIRMED KA-25_H0RM0NE HELICOPTERS
] ~
[
11 CONFIRMED MI-S HIP HELICOPTERS
T
I
I
I
I
_I_
I
4 CONFIRMED KA-i5_HEN HELICOPTERS
6 CONFIRMED KA-18 HOG HELICOPTERS
23 CONFIRMED IL-12 COACH TRANSPORTS
22 CONFIRMED IL-14_CRATE TRANSPORTS **
___ I ___
110
INSTALLATION 0327-V23249
2 PRCBAEIE PRIN’ORYE CLASS INTELLIGENCE
SHIPS !
I
I
3 CCNElRt^.ED POLNOCNY.CLASS LANDING SHIPS
2 CONFIRMED ALLIGATOR_CLASS LANDING SHIPS
!
I
_ \ _
I
3 CONFIRMED YUHKA_CLASS MINESWEEPERS
I
5;:5;s }
2 POSSIELE NATYA^CLASS MINESWEEPERS
4 PROBABLE PETYA^I^CLASS FRIGATES
I
I
I
_ I_
_j_
INSTALLATION 01S7-V24Z77
60 CONFIRMED BTR-60PK AMPHIBEOUS APC
25 CONFIRMED T-62 MEDIUM TANKS
I
23 CONFIRMED SS-MMi I)-44 ANTI-TANK GUNS
I
IS PROBABLE BM-21 ROCKET_LAUNGEERS
1
__I_
I
«*22 CONFIRMED 122-MM D-3e FIELD_HOWITZERS
I
1
5JC5}; j
19 CONFIRMED M-1955 FIELD^HOWITZERS
I
17 CONFIRMED M-1976 AIRBORNE
ASSAULT GUNS i
111
I i
_j_
INSTALLATION 0528-V31176
I
** 11 CCNFIEMED PETYA_II_CLASS FRIGATES
2 PROBABLE 3RAV0_CLASS SS
I
I
j
3 CONFIRMED ECHO_I_CLASS SSGN
I
___I_
j JjCJiS
12 CONFIRr^ED ICHC^I I^CLASS SSGN
j
5 CONFIRriD RIGA.CLASS FHIGATIS
I
I
1
i INSTALLATION
0410-732237
❖ j
1 22
CONFIRMED
BTR-50PK AMPEIBEOUS
APC !
! 40
CONFIRMED
T-72 HEAVY TANKS --
j
1 IB
PROBABLE SU-100 ASSAULT GUNS
I
j **£,5
CONFIRMED
152-MM D-20 FIELD,
«
HOWITZERS I
j
24 CONFIRMED 100-MM M-1955 FIELD GUNS
I
I
[
13 PROBABLE M-1976 AIRBORNE ASSAULT^GUNS
I
I
I
I
I
I
_I_
I
112
i INSTALLATION
0276-V33264
1 15 CONFIRMED
MIG-21 FISHBED
FIGHTERS !
j
1
1
j
112 CONFIRMED MIG-19_FARMER FIGHTER-BOMBERS!
1 ! i
|*’i‘ 11 PROBABLE
»
MIG-23 FLOGGSR
1
1
FIGHTERS i
I J>:5;s
17 CONFIRMED MIG-£7_fLOOSER STRIKE/ATTACK
AIRCRAFT 1
21 CONFIRMED MIO-25 FOXBAT INTERCEPTORS
T
3 POSSIBLE TD-28P FIDDLER INTERCEPTORS
INSTALLATION 0362-V34273
I
** 2 PROBABLE SKORY^CLASS DESTROYERS
I
3 CONFIRMED KOTLIN_CLASS DESTROYERS
i
2 CONFIRMED KYNDA_CLASS CRUISERS
5 CONFIRMED KANIN CLASS DESTROYERS
I ^
— ^
I
2 CONFIRMED SVERDLOV CLASS DESTROYERS
❖ sis
I
I
I
I
2 PROBABLE SHERSHEN^CLASS TORPEDO BOATS
I
4 CONFIRMED KOTLIn’saM CLASS DESTROYERS
1
I
❖❖
113
i
i
EUi'FIRSr-VCICI CARLS ->>>>> NEXT TWELVE
I
1 INSTALLATION 0613-V51214
4 CONFIRMED BMP-76PE APC
1 7 CONFIRMED BRDM
APC !
1 i
!3 CONFIRMED AT-3 SAGGER ATG¥ **
1 1
1 4 PROBABLE ZSU-23/4
1 1 '
AA_GUNS1
1 1
i 43 CONFIRMED T-54/5E MEDIUM TANKS
1
^ j
! I
I I
4 FROEABLS'SA-9^GASKIM LAUNCHERS
I
} 51 ^^
6 PROBABLE ZU-23/2 AA^GUNS
_I_
I
INSTALLATION 0115-V52314
3 CONEIRNIED M-4_:BIS0N BOMBERS
1 POSSIBLE TU-20^BEAR RECONNAISSANCE
AIRCRAFT j
I
12 CONFIRMED TU-20_BEAR BOMBERS
1 CONFIRMED TU-126 MOSS AWACS
I ^
_I_
j
C CONIIR^EI) BS-12_MAIL RECONNAISSANCE
AIRCRAFT I
7 CCNEIRMER IL-2S_EEAGLE BOMBERS --
1? CONFIRMED TU-16_BADGER BOMBERS
’i'* 3 PROBABLE TU-16^BADGER RECONNAISSANCE
AIRCRAFT
114
i
INSTALLATION 012e-V53113
i /
2 CONFIRMED SRESTA_II_CLASS CRUISERS
3 CONIIRMED KRESTA_I_CLASS CRUISERS
1 POSSIBLE'taNGO.CLASS SS
I
I
I
I
I
'5*"r2"CGNfrRFifD"WHrsZSYlCLASS"SS
I
2 PROEAPLZ CHAHLII^il^CLiiSS SSGN
!
1
1 CONFIRMED CHARLIE^I^CLASS SSGN
j
I
I
_I___
INSTALLATION 0298-V54216
j 3^5
50 CONFIRMED ASU-a5 AIRBORNE ASSAULT^GUMS
I
27 CONfIRM.ED ASU-57 ' AIRBORNE ASSAULT^GUNS
JicXi I
I
I
I
_I_
1
20 POSSIBLE M-240 HEAVY MORTARS
52 PROBABLE 122-MM D-30 FIELD^HOWITZERS
I
I
I
48 CONFIRMED 240-MM BM-24 ROCKET-LAUNCHERS
I
j
j
115
INSTALLATION £827-761253
6 CCNflRMEr FOXTROT.CLASS SS
I -V.
12 CCNFIRMED JULIIT_CLASS SSG
** 2 PRCEAELE DELTA 11_CLASS SSEN
T
3 PROBAELZ DELTA^CLASS SSEN
1
1
4“coneIrn’Sd”golf_I!Iclass“sssn~^^
I
1
j
5 CCNEIH^^ED POTI^CLASS CORVETTES
1
2 POSSIBLE YANKEE^CLASS SSEN
I
7 PROBABLE ROMEO CLASS SS
______I ______
INSTALLATION e405-V62217
j
40 CONEIRMSD T-10 HEAVY TANKS
57 CONFIRMED T-34/65 MEDIUM TANKS
43 CONFIRMED T-54/55 MEDIUM TANKS
1
1
!
1
1
_I_
3 CONFIRMED PT-76 LIGHT AMPHIEEOUS TANKS
I
6 CONFIRMED BTR-152 APC
I
6 CON5IRMSD BRDM RECONNAISSANCE APC
I
I
I
1
I
I
116
INSTALLATION 0352-V63224
sis#
1 /
11 CONJIRMZI) TU-22 BLINDER BOMBERS
20 CONEIRMED TU-26_BACKFIRE BOMBERS
5 PROBABLE IL-28_BEAGLE BOMBERS-’’'’^
I
I
I
_I_
j
2 CONFIRMED IL-76_CANDID TRANSPORTS
I
I
15 CONFIRMED AN-12,CUE TRANSPORTS
I
I
7 CONFIRMED MI-S HIP HELICOPTERS
" I
I
___I_______
INSTALLATION 0247-V642S3
I
5 PROBABLE KOMAR^CLASS MISSLE BOATS
i
{
17 CONFIRMED OSA^I.CLASS MISSLE BOATS
I
5 CONFIRMED OSA^II^CLASS MISSLE BOATS
3^5{C I
I
I
j
7 CONFIRMED STENKA^CLASS TORPEDO BOATS
I
I
I
11 POSSIBLE NANUCHKA^CLASS TORPEDO BOATS
I
6 POSSIBLE GRISHa'cLASS CORVETTES
2 PROBABLE SHSRSHEnIcLASS TORPEDO EOATS-^
______ I___—
I
117
INSTALLATION 0242-V71278
5|t# j
12 CONFIRhSD iNlG-27_FLOGGER STRISE/ATTACK
AIRCRAFT I
16 CONFIRMED SU-19_FENCER STRIKE/ATTACK
AIRCRAFT 1
I
2 P0SSI2LE MIG-25R_F0XBAT RECONNAISSANCE
AIRCRAFT ** I
1 CONFIRMED IL-38_MAY ASV AIRCRAFT
❖ sis j
3 CONnR^^ED AN-9_CAMP TRANSPORTS
I
I
I
5 CONIIRMEP AN-26^CURL TRANSPORTS
j 5;s5;s
I
I
_1_
INSTALLATION 0657-772179
j /
2 CONFIRMSI) HOTEL.II.CLASS SSBN /
1 CONFIRMED HOTEL III CLASS SSBN
— 1 —
I
I
I
I
I
I
I
_I_
[
1 PROBABLE GOLF.I^CLASS SSB
j
1 PROBABLE MIEKA.I.CLASS LIGHT FRIGATE
I
I
j
1 POSSIBLE ZULU IV CLASS SS
I
I
lis
INSTALLATION 0410-V73252
** 4 CONFIRr^LD 100-Mr^ M-49 AA_GUNS
4 CONFIRMED ZSU-57/2 AA_GUNS -
5 CONFIRMED S5-MM M-44 AA_GUNS —
I
I
J
/
6 CONFIRMED FRCG-4 SSM MOEILE LAUNCHERS
_I_
I
I
5!!J!5 j
6 PROBABLE AT-1_SNAPPER ATGW
I
I
I
4 CONFIRMED 122-MM D-74 FISLD_GUNS
I
I
j !it«
21 CONFIRMED 65-MM D-44 ANTI-TANK GUNS
_______ I _______
INSTALLATION 0173-V74246
1 CONFIRMED TU-126_M0SS AWACS /
1 CONFIRMED TU-16_BADGSR RECONNAISSANCE
AIRCRAFT '
16 CONFIR^^ZD AN-22^CCCK TRANSPORTS
j
I
I
I
I
I
_I_
le CONFIRMED TU-20 BEAR BOMBERS
/ I
j
I
12 CONFIRMED TU-22_BLINDER BOMBERS
I
I
I
2 CONFIRMED TU-20 BEAR RECONNAISSANCE
AIRCRAFT ^ I
_____I_____
1
I
119
APPENDIX C
Te00 TRAINING INSTRUCTIONS
For this experiment a 254 word vocabulary will be used
with the Threshold 600 (T600} voice recognition system. You
will be required to speak each utterance ten times to train
the Te00 tc recognize ycur voice. Two sessions of
approximately 90 minutes will be required to complete the
training prior to experimentation.
Please observe the following guidelines during training
and operation of the T600, as they will improve performance
and reduce the time required for retraining.
a. Use variety. Say the repetitions with the
variety of intonation, emphasis, and volume
you would expect to use in normal speech.
d. Speak crisply without pausing. 2e natural
and relaxed. Don't exaggerate or
overemphasize; for example when saying the
word "five", don't say "FI-I-VIK", thereby
overemphasizing the end of the wcrd in an
unnatural way.
120
b. Do the repetitions in e?rcups to avoid breath
noise and help you count the reps. For
example to train the word "zero” group the
zeros as follows:
022 - 002 - 002-0
or
000 - 000-0000
rather than -
0000000000
or
0 - 0 - 0 - 0 - 0 - 0 - 0 - 0 - 0-0
c. Adjust the microphone carefully, as
demonstrated { see the picture).
e. Leave a distinct pause between words. Ycu
must wait for the green RIADY light to come
cn before saying the next utterance.
f. Use the proper volume. Watch the meter;
the needle should be in the green area cr
just slightly in the red on the peek parts
of the word. Words trained in the lower
white or upper red will give poorer results.
121
Cnee you are corfortaole with training the TeCC, I will
ask you to operate the keyboard for the remainder of the
training. I will remain nearby to provide assistance as
required. Be sure to ask for help if you have any
questions. Take breaks as you need them» a convenient place
to break is every few pages.
Operating the T600 ate 5*s sjs 5;s :^€ 5{c sis :{c 3;s sis
To train a word - YOU TYPE
T6O0 RESPONSE
CTRL-U
<word number>
.e.g. 0
WDi?:
<word prorrpt>
ZERO
Now you say the word or phrase 10 times. Once the current
phrase disappears you are ready to go onto the next word of
the vocabulary. Again you type CTRL-U and continue as
before .
122
APPENDIX D
TYPING TEST
TEE SOVIET NAVAL AIH FORCE
FOR THE FIRST TII^F IN ITS HISTORY, THE SOVIET NAVAL
AIR FORCE WILL BE PUTTING TO SEA WITH ITS OWN AIRCRAFT
EMBARKED ON TEE FIRST OF TEE NSW SOVIET AIRCRAFT CARRIERS,
THE KIEV, WHICH HAD ALREADY BEGUN ITS WORKING-UP TRIALS IN
THE AUTUMN OF 1974. DISPLACING SOME 36,0^0 TONS WITH AN
OVERALL LENGTH SLIGHTLY IN EXCESS OF 900 FEET, THE KIEV
IS PRESUMED TO EMBARK 40-50 AIRCRAFT IN ALL, COMPRISING
A MIX OF HELICOPTERS AND FIXED-WING V/STCL AIRCRAFT
(TEE KIEV SHOWS NO SIGNS OF ARRESTER CABLES OR LAUNCH
CATAPULTS). THE SUGGESTED VERSION OF THE STRIKE AND
RECONNAISSANCE FIGHTER TO BE EMBARKED ON TEE KIEV IS TEE
YAK-36, A VERSION OF WHICH WAS TESTED ON THE AIRFIELDS NEAR
MOSCOW AND GIVEN SEA TRIALS ON THE SOVIET HELICOPTER-
CARRIER MOSKVA. TEE YAK-36 UTILIZES VECTORED THRUST
AND DIRECT LIFT IN COMiBINATION. SUCH AN AIR COMPLEMENT
MIGET EE BROKEN DOWN INTO 30 KA-25 ASW HELICOPTERS AND
15-20 V/STOL FIXED-WING AIRCRAFT. HOW MANY OF THESE
CARRIERS WILL BE PRODUCED ?
AT LEAST TWO OF THESE KIEV-CLASS AIRCRAFT CARRIERS ARE
DUE TO ENTER SERVICE, WITH THE POSSIBILITY OF THE SOVIET
NAVY PRODUCING A WHOLE CLASS OF SOME 6-e SHIPS, THEREBY
123
FACILITATING CCiNTINUCUS DFPLCYMENT OF ONE VESSEL IN BOTH THE
^S^ITERRANEA^] ANL THE INLIAN OCEAN. TEE HELICOPTER
COMPLEMENT PROVIDES INTENSIVE AS'.v CAPABILITY INTO DISTANT
SEA AREAS (FOR DEFENSIVE AND OFFENSIVE PURPOSES), AS WELL AS
FURNISHING AIRBORNE TARGET GUIDANCE FOR SURFACE-TO SURFACE
ANTISHIP MISSLES. TEE V/STOL AIRCRAFT, WHILE PROVIDING A
STRIKE CAPABILITY, MUST OBVIOUSLY INCREASE THE
RECONNAISSANCE COVERAGE OF THE SOVIET NAVAL AIR ARM IN AREAS
WHICH ARE BEYOND THE RANGE OF EXISTING LAND-BASED AIRCRAFT.
MEANWHILE, THE ARMAMENT OF TEE KIEV-CLASS SHIPS IS ITSELF
SIGNIFICANT. IT CONSISTS OF A TWIN LAUNCHER FOR ASW
MISSLES, TWO 12-BARRELL MSU AS ROCKET LAUNCHERS, TWO SA-N-3
SAM TWIN LAUNCHERS, A NUMBER OF RETRACTABLE SA-N-4 SAM
LAUNCHERS, MULTIPLE 57-MM AAA MOUNTS AND SMALLER WEAPONS FOR
CLOSE-IN PROTECTION AGAINST MISSLES AND OTHER GUIDED
WEAPONS.
124
1
i
‘I
I
I
I
I
i
I
i
t
i
I
APPENDIX E
PP.S/POST SUBJECTIVE CUESTIONNAIRS
Subjective Ciiesticnnaire Narre:_
INSTRUCTIONS; Express your feelira^^s regarding typed data
entry and voice data entry. CIRCLE TEE NUN’BSR which BEST
DESCRIBES your opinion for each question.
1. Which data entry ncde do you think is the easiest to use
to enter character strings and corrnands?
Typed
Data
Entry
<= <= <=
Neutral
_ V
=>
Voice
Da ta
Ent ry
=>
12 3
4
5
7
2. Which data entry
fcr entering character
mode do you
strings and
think is the
commands?
fas
test mode
Typed
Data
Entry
<= <= <=
Neutral
*
=>
=>
Voice
Data
Ent ry
=>
12 3
4
p;
6
7
3. Which data entry
character strings and
mode is the
conmands?
mcst accurate
fcr
entering
Typed
Data
Entry
<= <= <=
Neutral
=>
=>
Voice
Data
Ent ry
=>
12 3
4
c;
6
7
125
4. 'fthich data entry mode provides the most flexibilit
general, for interaction with a computer?
in
Typed
Neutral
Voice
Data
Data
Entry
Entry
< =
<=
< =
*
=>
=>
=>
1
2
3
4
5
6
7
£. «hich data
several hours, if
entry mode would you prefer
required?
to op
erate for
Typed
Data
Entry
<= <=
<=
Neutral
- =>
=>
Voice
Da ta
Entry
=>
1 2
*7
4 5
a
w
7
6. 'A'hich data entry mode would you prefer
more sporadic user of a computer system?
to operate as a
Typed
Data
Entry
<= <=
<=
Neutral
« =>
=>
Voice
Data
Snt ry
=>
1 2
4 £
6
7
7. Which data
operation?
entry
mode promotes the
most
relaxed
Typed
Data
Entry
<= < =
< =
Neutral
* =>
=>
Voice
Data
Snt ry
=>
1 2
3
4 5
6
7
126
Which data entry rrode would be the irost advantageous to
use to update
information?
an on*
line data
base of
intelligenoe
Typed
Da t a
Entry
<=
<=
<=
Neutral
=>
=>
Voice
Data
Ent ry
=>
1
2
4
5
6
7
9. Which data
interface in
environment?
entry mode provides the best man'
a time-critical, high-pressure
-me chine
work
Typed
Data
Entry
<=
<=
<=
Neutral
=>
=>
Voice
Data
Entry
=>
1
C
3
4
6
7
10. Which data
learn?
entry
mode do you
think is
the easiest to
Typed
Data
Entry
<=
<=
< =
Neutral
sit
=>
=>
Voice
Da ta
Entry
=>
1
2
0
4
c;
6
7
127
APPENDIX F
SUBJECT DATA SHEET
Subject Data Sheet
Date:
Narr.e:_ Age;
Service:_ PLank/Grade:_
Joh/Specialty Description (last job / next jcb)
Prior to this experiment wnat has been ycur experience with
voice data entry systems 7 Check one or more.
_a. I have used a voice data entry system.
_b. I have seen a voice data entry system demonstrated.
_c. I have studied voice data entry systems (class,
report, thesis, etc.)
_d. I have no experience with voice data entry systems.
If you checked a. above, circle the term that best describes
your experience and skill with voice data entry.
Experience - Skill-
Considerable High
(Moderate Average
Minimal Novice
Explain :
If you checked c. above, please briefly state the extent
of your studies.
12S
I
I
AFPINDIX G
INSTRUCTIONS BRIRFEr TO SUPJICTS
TYPING f^CDI
1. During this portion of the experir'ent you vill view 12
cards and use the ADM terminal to write a report on each
card similar to the one you saw in the sample (or other
portion of the experiment). I will stop you after every
four cards. This will give you a break end allow me to
collect some data.
2. You will be using a text editor at the ISII host
computer. The edit keys discussed during training which may
be used are shown on the carl at the terminal. You may edit
errors only if you are on the line with the error in it,
i.e. if you notice an error on the previous line, do not
attempt tc correct it. However, I will demonstrate how you
may void the previous line if you wish to do it over.
3. Pencil and paper are provided if you want to use them to
take notes as you look in the viewport.
4. Now practice on this card.
5. <critique the repcrt>
6. Ycu are tc go as fast as you can while trying to
minimize errors. Keep in mind you are writing an
intelligence report which should be timely, accurate, and
complete. Cuestions?
7. Ck, start.
6. <Trial rfl>
9. Ok, stop. Rest a moment, then you will do four more.
IZ. Ck, start.
11. <Trial #2>
129
12. Ok, step. Rest a rrement , this is the last set cf feur
you will type for the experiment.
13. Ok, start.
14. <.Trial ff3>
15. Stop. You deserve a hreak. Relax a while. You may
=^et up and move arcund, get a drink, etc.
130
V0ICS-UN3UFFSRED I^.ODE
1. Luring this portion of the experiment you will view 12
cards, ana use the T600 in unbuffered rrcde to write a report
for each card like the one you saw in the sample (or other
part of the experiment). I will stop you after every four
cards. This will save you a break and allow me to collect
some data.
2. The Te03 unbuffered mode allows you to send the output
corresponding to an utterance immediately to the host
computer. So for example, when you say "CONFIRKID, ” it is
sent immediately to the computer, and in this case, becomes
a part of the text in tne text editor at the ISIS computer.
You may edit your input as long as you are on the line that
has the error using the edit commands you trained. A list
of tne edit commands you use is provided for you here, along
with a list of the vocabulary as reference material.
5. If you look in the viewport at this time, you will see
that the three bottom lines of tne 1600 display may be seen.
These will provide a visual feedback of the text editor
contents, and allow you to view the editing process as well
as the carl.
4. Now practice using the sample card provided.
5. <critique the report>
6. You are to go as fast as you can while trying to
niinimize errors. Keep in mind you are writing an
intelligence report which should be timely, accurate, and
complete. Questions?
7. Ck, start.
6. <Trial #1>
9. Ok, stop. Rest a moment, then you will do four more.
10. Ok, start.
11. <Trial #2>
131
12. C>, Stop. Rest a mc.'nent , this will be your last set of
four to enter for the unbuffered rnode pert of the
experindent.
13 . Ck, start.
14. <Trial #3>
15. Stop. You deserve a break. Relax a while. You nay
get up and move around, get a drink, etc.
132
vcics-BuriSRzr
1. During this portion of the eiperirent you will view 12
cards, and use tne T600 in buffered mode tc write a report
for each card like the one you saw in the sample (or other
part of the exper- iment). I will stop you after every four
cards. This will ^ive you a break and allcw me to collect
some data.
2. The 1622 bufferea mode allows you to speak a chain of
phrases prior to sending: them tc tne nest computer. Ycu ma^
edit the last utterance in the buffer by saying "kill line'
cr its equivalent fer your vocabulary. If you make several
errors, the entire buffer nay be erased with the command
"kill line." Once you are ready to send the contents of the
buffer, you say "go* or "carriage return," whichever you
trained, and the character string will be sent tc the text
editor at ISIS. However, you will not be able to use the
editing features of the text editor at ISIS while in the
buffered mode. I will demonstrate the buffered mode for ycu
now .
3. If you look in the viewport at this time, you will see
that the three bottom lines of tne T600 display may be seen.
These will provide a visual feedback of the buffer contents,
and allow you to view the editing process as well as the
card.
4. Now practice using the sample carl provided.
5. <critique the report>
6. You are to go as fast as ycu can while trying to
minimize errors. Seep in mind you are writing an
intelligence report which should be timely, accurate, and
complete. Questions?
7. Ok, start.
S. ^Trial ffl>
9. Ck, stop. Rest a moment, then you will do four more.
10. Ok, start.
11. <Trial #2>
133
12. Ck, stop. Rest c rcrent, this l^ill be your Last set of
four to enter for the buffered T’ode part of the ej.perinent.
13. Ck, start. »
14. <Trial ff3.
15. Stop. You deserve a break. Relax a while. You may
^et up and move around, get a drink, etc.
134
APPEND IX H
VOCABULAHY WORDS KISRECOGNIZED OR REJECTED
NOTE: THE EOLLOWING LIST IS IN ASCENDING COLLATING SEOUENC
BY UTTERANCE AND MISRECOGNITON . THE ISRECOGNI TI ONS HAV
TEE FOLLOWING FORMAT:
A (B) X N
WHERE
A = UTTERANCE ASSOCIATED WITH Te00
MISRECOGNITION
E = SPECIFIC TSee OUTPUT, IF DIFFERENT
THAN A above; S.G.”(2)’’ MEANS
THAT A NUMERAL WAS OUTPUT RATHER
THAN THE WORD "TWO"
N = NUMBER OF OCCURENCES
s;: 5;s 5*,c 5^ s;: 3ic 3}: ^ 3^ sjc six >1?
* UTTERANCE *
* MISRECOGNITION(S ) -
* :|e ^ :{s s^c sj: # # j*! sis s;£
31: 3^S 3;c 3 ( 1 : 5}C sjc 3|« 3;c 5jc sfc sjc 3}: 3;« 3{s s;« 3|« 3}C 5;« sic 3;s 3^
122-MM
100-MM X 2
12 2-MM
152-MM X 3
152-MM
122-MM X S
ee-MM.
57-MM
AA GUNS
AN-8 CAMP X 6
AA GUNS
ANTI-TANK GUNS X 3
AA GUNS
YAK-2eP FIREBAR
AIRCRAFT
ANTI-TANK GUNS
AIRCRAFT
CARRIER
AIRCRAFT
TU-26 BACKFIRE
AMPHIBIOUS
FRIGATES X 3
AM-e CAMP
AA GUNS
ANTI-TANK GUNS
ANPHIBIOUS
ANTI-TANK GUNS
AN-6 CAMP X 4
ANTI-TANK GUNS
BEEP* X e
ASSAULT GUNS
BEEP* X 16
ASSAULT GUNS
MISSILE
ASU-57
AT-3 SAGGER
AT-1 SNAPPER
BEEP*
AT-3 SAGGER
APC
BM-21
M-44
BM-24
BM-21
BMP-75PB
BTR-6ePK
BOATS
BEEP*
135
•i*' v^
^ “r* n* ^ ^ 'c ^ ^ ^ n*
- UTTZRANCZ -
'T* *T* •T* 'it^ or 'i
:}c ?;«5;c sis ;;s s;s >;: 5;c ;|s:{: s;? sjs >;« ^ >;t s*^ ?|s ^ ?;s
»!= ^'ISRZCO&NITICN(S) -
«/« S*-* vl^ «i« ayV
O” or or -r or* ^ O' ^ ^ O' -T* "r o^ O'’ or ^ O' or ^r ^r o' ^r
ROf^REPS
30MBZRS
BOMBERS
BOMBERS
BRAVO
BRAVO CLASS
BRAVO CLASS
BRAVO CLASS
ERDM
BRBM
ETR-152
BTR-5e?K
BTR-50PiC
BTR-60PK
CARRIAGE RETURN
CARRIAGE RETURN
CARRIAGE RETURN
CARRIAGE RETURN
CARRIAGE RETURN
CARRIAGE RETURN
CARRIAGE RETURN
CARRIAGE RETURN
CARRIAGE RETURN
CARRIAGE RETURN
CARRIAGE RETURN
CARRIAGE RETURN
CHARLIE I CLASS
CHARLIE I CLASS
CHARLIE I CLASS
CONFIRMED
CONFIRMED
CONFIRMED
CONFIRMED
CONFIRMED
CONFIRMED
CONFIRMED
CONFIRMED
CONFIRMED
CONFIRMED
CONFIRMED
CONFIRMED
CONFIRMED
CONFIRMED
CONFIRMED
CONFIRMED
CONFIRMED
CONFIRMED
ALPHA (A)
BEEP’:^ X 7
IL-14 CRATE X 2
LAUNCHERS
FRIGATE
GOLF I CLASS
KOMAR CLASS X 2
KOTLIN CLASS X 2
TORPEDO
YANKEE
NINETEEN (19)
BTR-6ePK X 4
D-20
BTR-50PE X 3
AN-e CAM? X 4
BEEP’!' X 2
BRDM
CARRIER X 11
BRIGATE X 2
FRIGATES
HEAVY X 3
SSN
VICTOR (V)
XRAY (X)
YAK-2SP FIREBAR X 2
ZSU-23/4
FOXTROT CLASS
KOTLIN CLASS
MIRKA I CLASS X 2
AIRBORNE
BEEP* X 148
BOMBERS
BRAVO (B) X 4
BRDM
ELEVEN (11) X 6
FIVE (5) X 7
FOUR (4)
HEAVY X 4
KOTLIN CLASS
LANDING
LIMA (L) X 5
MI-4 HOUND X 5
MIKE (M) X 2
NINE (9)
NOVEMBER (N) X 2
SA-6 GECKO
SEVEN (7) X 2
136
%JL» hJU «A«
'TT *1- O' O' 'i' ^r *¥• O' ^ O' 'I'
- UTTERANCE *
.J« *J« y« »u «V oA*
'i' O' ^ O' O' O' ^ O' ^ ^ O' O' ^
V' *V V' 'A# W^ 'A' V*'' 'V
O' ^ O' O' O' O' O' O^ ^ O' * 1 ' O' •'«' O' O' O' o'* O' ^ O' O'
- MISRECCGN ITIGN(S ) *
5;c5',cj;s5,*t3;i>;i>;s?{«?{?3;«5;c5{c5;??’^:Cc5;:5;<5(c5;«5^5ji5;?
CCNEIRMED
CONFIRMEE
CONFIRMEE
CONFIRMED
CONFIRMEE
CONFIRMEE
CONFIRMED
CONFIRMEE
CONFIRMEE
CRUISERS
E-30
D-44
DASH
EAS E
DELETE LINE
DELETE WORD
DELETE WORE
DELTA CLASS
DELTA II CLASS
ECHO I CLASS
ECHO II CLASS
ECHO II CLASS
ECEC II CLASS
EIGHT
EIGHT
EIGHT
EIGHT
EIGHT
EIGHT
EIGHT
SIGHT
EIGHT
EIGHT
EIGHT
SIGHT
EIGHT
SIGHT
ELEVEN
ELEVEN
ELEVEN
ELEVEN
ELEVEN
ELEVEN
ELEVEN
ERASE
FIELD GUNS
FIELD GUNS
TEN (le)
TWELVE (12) X 7
TWENTY (23)
TWENTY-FIVE (25)
TWENTY-ONE (21)
UNIFORM (U)
UPPER RIGHT
XRAY (X) X 6
Z SU-23/4
TWENTY-THREE (23)
D-74
TWENTY-FOUR (24)
QUEBEC (Q)
TEN (10)
LIMA (L)
DELETE LINE (CTRL X)
TWENTY-THREE (23)
KOTLIN CLASS X 2
GOLF II CLASS X 2
PETYA I CLASS X 2
DELTA II CLASS X 2
PETYA II CLASS
SEERSHEN CLASS
AA GUNS X 4
AMPHIBIOUS
AN-8 CAMP X 3
APC
ASU-e5
BEEP^ X 4
EIGHTEEN (l8) X 4
FIFTEEN (15)
FOUR (4) X 5
HEAVY X S
KA-15 HEN X 14
MEDIUM
SA-S GECKO
YANKEE (Y) X 7
BEEP’*' X 6
D-20
FIVE (5) X 2
FOUR (4)
ONE (1) X 3
UPPER LEFT
UPPER RIGHT
EIGHT (6)
BEEP’!' X 2
JULIETT X 2
137
ii:
- UTTERANCE *
*;!«Si: >J::(: # # :!s :{s 5j5 s;: sf:
EIEIB GUNS
EIEID HOWITZERS
FIELD HOWITZERS
FIELD HOWITZERS
FIFTEEN
FIFTEEN-
FIGHTER
FIGETER-EOf^BFRS
FIGHTER-BOMBERS
FIGHTER-BOMBERS
FIVE
FIVE
FIVE
FIVE
FIVE
FORTY
FOUR
FOUR
FOUR
FOUR
FRIGATE
FRIGATES
FRIGATES
FRIGATES
FROG-3
FROG-3
FROG-4
GO
GO
GO
GO
GO
GO
GO
GOLF I CLASS
GRISHA CLASS
GRISHA CLASS
GRISHA CLASS
HEAVY
HELICOPTERS
HELICOPTERS
HELICOPTERS
HELICOPTERS
HOTEL III CLASS
IL-14 CRATE
INSTALLATION
INSTALLATION
^ ^ ^ ^ sis
* MISRECOGNITION(S) *
5JC 5}S 3^6 sjs sis jjs :0c jjc J{s Sic 5{S si: Jjs 3ic ;$c5{« sjs Jjs ijc sjs 3{e
T-IC
BEEP’!' X 5
HELICOPTERS X 4
INTELLIGENCE
EIGHTEEN (18)
THIRTEEN (13) X 5
FRIGATES
ROCKET LAUNCHERS
BEEP* X 3
TWENTY-ONE (21)
AN-e CAMP
BEEP* X 2
NINE (9) X 2
PAPA (P) X 2
QUEBEC (Q) X 2
THREE (3)
BEEP* X 33
FROG-4
LOWER RIGHT X 4
MOBILE
IL-38 MAY
BEEP*
FRIGATE
SHSRSHEN CLASS
BEEP*
D-20
PROBABLE
BEEP* X 24
BRAVO (B) X 2
DELTA (D)
ECHO (E) X 3
GOLF (G)
TWELVE (12)
ZERO (0)
OSA I CLASS
KYNDA CLASS
RIGA CLASS
VICTOR CLASS
SCUD B
BEEP* X t
BRAVO (B)
FOXTROT (F)
MI-4 HOUND
HOTEL II CLASS X 3
MI-24 HIND
BEEP* X 3
S-60
138
1
i!
I
I
3;c ^ ^
=<= UTTERANCE -
INTELLIGENCE
INTERCEPTORS
INTERCEPTORS
JULIET CLASS
KA-16 HOG
KANIN CLASS
KANIN CLASS
KANIN CLASS
KANIN CLASS
KANIN CLASS
KANIN CLASS
KARA CLASS
KARA CLASS
KARA CLASS
KARA CLASS
KARA CLASS
KASHIN CLASS
KASHIN CLASS
KASHIN CLASS
KASHIN CLASS
KASHIN CLASS
KASHIN CLASS
KIEV CLASS
KIEV CLASS
KIEV CLASS
KIEV CLASS
KIEV CLASS
KIEV CLASS
KIEV CLASS
KILL LINE
KILL LINE
KILL LINS
KILL LINE
KILL LINE
KILL LINE
KILL WORE
KILL VCRD
KILL WORD
KILL WORD
KOMAR CLASS
KOMAR CLASS
KOTLIN CLASS
KOTLIN CLASS
KOTLIN CLASS
KOTLIN CLASS
KOTLIN CLASS
KOTLIN CLASS
^ 3;::(e
* MISRSCOGNITICN (S )
3;e 3;csj; Jje 3;c 3^ 3 ;: 3}: ;;c3;c 3ic :;c
BEEP’:^ X 6
BEEP* X 3
HELICOPTERS X 6
YURKA CLASS
EIGHT (8)
CARRIER
KASHIN CLASS X 3
KIEV CLASS
KYNDA CLASS
SHERSHEN CLASS X 4
YANKEE CLASS X 5
KANIN CLASS
KOMAR CLASS
KOTLIN CLASS X 3
STENKA CLASS
YURKA CLASS X 2
JULIET CLASS X 8
KANIN CLASS
KOTLIN CLASS
NATYA CLASS X 3
SHERSHEN CLASS X 2
YANKEE CLASS
AIRCRAET
JULIET CLASS
KANIN CLASS
KARA CLASS
KYNDA CLASS X 2
SHERSHEN CLASS
STENKA CLASS X 6
CHARLIE (C) X 6
DELETE (CTRL X)
KANIN CLASS X 2
KOTLIN CLASS
M-44
MI-4 HOUND
BEEP* X 6
FIELD HOWITZERS
KILL LINE X 2
SEVEN t?)
KARA CLASS X 2
MIRKA I CLASS
BRAVO CLASS
CHARLIE II CLASS
DELTA CLASS
KASHIN CLASS
KOMAR CLASS X 2
MOSKVA CLASS
139
^:tc :{s sj?3;c Xs
- UTTERANCE =*'
si; j;« sj; 5{« ** 5S !(:❖>!«## Jr >}=# =<5 5r # 5r n!’!'>!«
’i' MISRSCCGNITICN(S) -
s;c 3 }: :{: >;c ^ Xc s|s 5j: s;s »;<
KOTLIN
CLASS
POLNOCNY CLASS
KOTLIN
CLASS
FOTI CLASS
KOTLIN
CLASS
UPPER LEFT
KRESTA
I CLASS
CHARLIE I CLASS
KRESTA
I CLASS
ECHO I CLASS
KRESTA
I CLASS
OSA I CLASS X 4
KRESTA
I CLASS
RIGA CLASS
KRESTA
II CLASS
NANUCKKA CLASS
KRESTA
II CLASS
OSA II CLASS
KRIVAK
CLASS
KANIN CLASS
KRIVAK
CLASS
KARA CLASS X 2
KRIVAK
CLASS
KOTLIN CLASS
KRIVAK
CLASS
KYNDA CLASS
KYNDA
CLASS
KANIN CLASS X 6
KYNDA
CLASS
NATYA CLASS
KYNDA
CLASS
RIGA CLASS
KYNDA 1
CLASS
STENKA CLASS X 7
LANDING
BEEP*
LAUNCHERS
LIGHT
LIGHT
LIGHT
LIGHT
LIGHT-
LIGHT
LOWER LEFT
LOWER LEFT
LOWER LEFT
LOWER LEFT
LOWER LEFT
LOWER LEFT
LOWER LEFT
LOWER LEFT
LOWER LEFT
LOWER LEFT
LOWER RIGHT
LOWER RIGHT
LOWER RIGHT
LOWER RIGHT
LOWER RIGHT
LOWER RIGHT
LOWER RIGHT
LOWER RIGHT
M-1955
1^-1955
M-1955
M-1976
^;ORTARS X 6
BEEP* X E
FIVE (5)
LOWER RIGHT X 2
MIKE (N) X 4
ONE (1)
TWENTY (20) X 3
BEEP=<' X 19
BTR-50PK
CORVETTES X 2
KOTLIN CLASS
LIGHT
LOWER RIGHT X 9
MOBILE X 3
THREE (3)
TWENTY-ONE (21)
UPPER LEFT X 4
BEEF*
CONFIRMED
LIGHT
LOWER LEFT
ONE (1) X 3
SEVEN (7)
SEVENTEEN (17)
UPPER RIGHT X 8
ASU-85
BEEP* X 2
SU-15 FLAGON
PT-76
140
yu «v «v* ju Up «««
=5* UTTERANCE '■'
«ju %>« mj^ aju y« yu u« yy u# yu yo u^
T n' •^•* V ^ <T‘ ^ ‘^■* -nr <Y* ^ ^
*•» %<* U« y« «•« Up »’p y« Up Up Up Up ..'p Up Up Up Up %»p %»p Up Up
■T* n' nr ^r *’r nr >^r ^r ur n» nr n-* -n* n-* n» -nr nr n» n» ^r n»
- r^ISRECOGNITION (S) -
-T“ ^ ^ ^ ^ ^ -T* ^ n' •^r'r -"i*^ ^ ^ ^
h-242
t^-4 EISOM
^-4 BISON
^"-4 BISON
y'-44
^-49
r-49
r^EDiur-’
M-ie HARKE
MI-10 HARKE
MI-12 HOMER
MIG-19 FARMER
MIG-21 FISHBED
MIG-21 FISHBED
MIG-23 FLOGGER
MIG-2E FOXEAT
MIG-2ER FOXBAT
MIG-25R FOXBAT
MIRKA I CLASS
MIRKA I CLASS
MRKA II CLASS
MIRKA II CLASS
MIRKA II CLASS
MIRKA II CLASS
MIRKA II CLASS
MISSILE
MOBILE
MOBILE
MOBILE
MOBILE
MORTARS
MOSKVA CLASS
MOSKVA CLASS
MOSKVA CLASS
MOSKVA CLASS
NANUCHKA CLASS
MANUCEKA CLASS
NANUCHKA CLASS
NANUCHKA CLASS
NANUCHKA CLASS
NANUCHKA CLASS
NATYA CLASS
NATYA CLASS
NATYA CLASS
NATYA CLASS
NATYA CLASS
NATYA CLASS
EEEP’^
BEEP*
CHARLIE I CLASS
MI-6 HOOK
TWENTY-FOUR (24) X 3
M " 19 5 5
TWENTY-FIVE (25)
BEEP*
1-11-24 HIND X 3
MI-6 EIP
MIG-19 FARMER
RIGA CLASS
IL-14 CRATE
MIG-27 FLOGGER
KA-25 HORMONE
MIG-25R FOXBAT X 2
KA-25 HORMONE
MIG-25 FOXBAT X 3
ECHO II CLASS
PETYA I CLASS
CHARLIE II CLASS X 2
DELTA II CLASS
KANIN CLASS
KOTLIN CLASS X 2
POLNOCNY CLASS
TWELVE (12)
BEEP- X 3
BRAVO X 2
HOTEL (H)
PROBABLE X 6
LAUNCHERS
BEEP’!' X 2
GOLF I CLASS X 3
NATYA CLASS
POLNOCNY CLASS X 5
KOTLIN-SAM CLASS
KYNDA CLASS
SHERSHEN CLASS
STENKA CLASS X 2
YANKEE CLASS
YURKA CLASS
ALLIGATOR CLASS X 2
BEEP’!'
KANIN CLASS X 3
KASHIN CLASS X 2
KOTLIN CLASS
KYNDA CLASS
141
^
#y» #|« ^ ^ ^ 'J' ?k» V ^
yu yu y^ y^ •JU •ju yu y^ y^ y# y# y^
>p #P rp ^p ^p ^1^ ^p ^p >p #p ^p #p
- UTTiRANCI *
* NISRSCCGNITION (S ) ’i-'
Jjj sis j;; 5i: };£ J(s # ^ # j): j;: 5S !!< >!« jIs >;t 3i5sis
KATYA CLASS
POTI CLASS
MNI
BEEP* X 2
MINI
FIVE (5) X 5
M N i
LIGHT
NINE
MI-6 HIP
NINE
MIKE (M)
NINE
TWENTY (20) X e
NINETEEN
EIGHTEEN
NINETEEN
MIKE (M)
NINETEEN
THIRTEEN (13)
ONE
BEEP- X 4
ONE
FIVE (5) X 7
ONE
FOUR (4)
ONE
FOURTEEN (14) X 2
ONE
LIGHT X 2
ONE
M-44
ONE
UPPER RIGHT
OSA I CLASS
MIRKA I CLASS
OSA II CLASS
K,RESTA II CLASS X 2
PETYA I CLASS
NANUCHKA CLASS
PETYA I CLASS
YANKEE CLASS
PETYA II CLASS
ECHO II CLASS
PETYA II CLASS
HOTEL II CLASS
PETYA II CLASS
KASHIN CLASS
PETYA II CLASS
SEERSHEN CLASS
POLNOCNY CLASS
ALLIGATOR CLASS
POLNOCNY CLASS
BSEP’i' X 2
POLNOCNY CLASS
ECHO II CLASS
POLNOCNY CLASS
HOTEL II CLASS
POLNOCNY CLASS
HOTEL III CLASS
POLNOCNY CLASS
KOTLIN CLASS
POLNOCNY CLASS
MOSKVA CLASS
POSSIBLE
BEEP*
POTI CLASS
KANIN CLASS
POTI CLASS
KOTLIN CLASS X 3
POTI CLASS
MOSKVA CLASS X 6
POTI CLASS
ROMEO CLASS
POTI CLASS
WHISKEY CLASS X 2
FRIMORYE CLASS
ECHO I CLASS
PRIMORYE CLASS
MIRKA I CLASS
PRINORYE CLASS
MIRKA II CLASS
PROBABLE
BEEP*
PROBABLE
FRAVO (B) X 7
PROBABLE
MOBILE X 3
PROBABLE
POSSIBLE
PROBABLE
TORPEDO
PROBALBLS
TWENTY-FOUR (24)
142
I
I
n' ^ ^ 5j5 SjS >}l 3{C ^ Jjt 5{i 3{i
%<« V-'
-1^ ^ ^ ^ n' nr ^ V V n'* ^ -Y* •'r n^ V nr ^r
UTTERANCE
- MISRECOGNITICN(S) *
?;« Sli 5;x :*s :Js ?j! 5}: ^ 5j: 5{C 5^5 3;s >;c 5tc>;s 5j! Jjc 5;c
RECONNAISSANCE
SEEP- X 4
RECONNAISSANCE
CRUISERS
RECONNAISSANCE
GRISHA CLASS
RECONNAISSANCE
INTELLIGENCE X 2
REPEAT LINS
CARRIAGE RETURN (CTRL M)
REPEAT LINE
D-20
REPEAT LINE
M-240
REPEAT LINS
THREE (3)
RETURN
BEEF’S^ X 4
RETURN
CONFIRMED X 5
RETURN
ELEVEN (li;
RETURN
SEVEN (7) X 2
RETURN
TEN (10) X 2
RIGA CLASS
GRISHA CLASS X 5
RIGA CLASS
VICTOR CLASS X 3
ROMEO CLASS
POLNOCNY CLASS
S-60
SS-16
S-60
SSG X 2
SEVEN
ASSAULT GUNS
SEVEN
BEEP* X 3
SEVEN
ELEVEN
SEVEN
FIVE (5)
SEVEN
SCUD A X 11
SEVEN
SEVENTEEN (17)
SEVEN
SIERA (S) X 2
SEVEN
WHISKEY CLASS
SEVEN
ZSU-57/2
SEVENTEEN
SCUD A X 3
SEERS ESN CLASS
KANIN CLASS X 4
SHIPS
SIX (6) X 2
SIX
BEEP*
SIX
DESTROYERS X 3
SIX
FRIGATES
SIX
INDIA (I)
SIX
SCUD B
SIX
SHIPS X 9
SIX
SPACE ( ) X 20
SIX
T-72
SIXTEEN
BEEP*
SIXTEEN
FIFTEEN (15) X 4
SPACE
AMPHIBIOUS X3
SPACE
BACKSPACE (CTRL A)
SPACE
FRIGATES X 2
SPACE
SHIPS
SPACE
T-10
SS
SSGN
SS
SSM
143
J
4
{
j;: ^*5 >;s sis :js
UlTIRANCI
%f* V.» %l^ »•* V''
'j- *1' -T* •r* 'n 'r- "T* 'i» ^ *r» ^
- N-ISRECCGNITION ( S ) -
***^ Vi* V* ^'*
*r» ^ Of* *1 *i»' ^ *T* ^-v ^ ^ ^ ^ ^ *1^ *f*
SS-14 SCAPIGOAT
SS-C0
SS2
SSSN
SSBK
SSG
SSG
SSGN
SSGN
SSGN
SSGN
SSGN
SSH
SSN
SS N
STiNKA CLASS
ST2NKA CLASS
STENKA CLASS
STENKA CLASS
STENKA CLASS
STENKA CLASS
STRIKE/ATTACK
STRIKE/ATTACK
STHIKE/ATTACK
SU-15 FLAGON
SU-19 FENCER
SU-A9 FISHPOT
SVERDLOV
SVERDLOV CLASS
SVERDLOV CLASS
SVERDLOV CLASS
SVERDLOV CLASS
T-34/65
T-34/S5
T-34/B5
T-34/e5
T-34/B5
T-54/B5
T-54/55
T-54/55
T-54/55
T-62
TANGO
TANGO CLASS
TANKS
TANKS
TANKS
SA-6 GECKO X 3
SSF
SSG X 6
SSB X 5
SSGN X 10
SSE X 2
SSGN
CHARLIE I CLASS
SS3N X 31
SSG
SSM
SSN X 4
SA-6 GAINFUL
SSN X 20
SSiN
JULIET CLASS
KANIN CLASS
KYNDA CLASS
NATYA CLASS X 2
SEERSHEN CLASS
VICTOR CLASS X 2
AN-6 CAN’P
BEEP’^' X 2
M-1955
IL-36 MAY
SS-16
BEEP*
BEEP*
OSA I CLASS X 2
POLNOCNY CLASS
STENKA CLASS X 2
YURKA II CLASS
ASU-85
M-1955 X 2
MIG-21 EISE3ED
T-54/55
TU-26 BACKFIRE X 3
BEEP* X 2
BMP-76PB
I)-44
T-34/S5 X 20
TU-16 BADGER
NINE (9)
NATYA CLASS
BEEP*
BEEP* X 2
HEAVY
144
- UTTERANCE -
4^ #1^ rp #1%
^ .^ISRSCCGMTION (S ) *
j;< ?;€};« 55e 5;c»;«sjc 5;s;Jc :;s5ls s;? 5)5 3;? :Jc ?;?
TANKS
TANKS
TEN
TEN
THIRTEEN
THREE
THREE
THREE
THREE
THREE
THREE
THREE
THREE
THREE
TEHEE
TU-22 BLINDER
TU-28 BEAGLE
TU-28P FIDDLER
TWELVE
TWELVE
TWENTY
TWENTY
TWENTY
TWENTY
TWENTY
TWENTY
TWENTY
TWENTY
TWENTY-FOUR
TWENTY-ONE
TWENTY-ONE
TWENTY-THREE
TWENTY-TWO
LOWER LEFT
TWENTY-TWO
TWO
TWO
TWO
TWO
TWO
TWO
TWO
TWO
TWO
TWO
TWO
TWO
NINETEEN (19)
TWENTY (20)
BEEP’>=
ELEVEN (11)
jilFTEEN (15)
CARRIER
FOURTEEN (14) X 2
FRIGATE X 8
F ROG-3
HEAVY X 2
Ml-e HIP
THIRTEEN (13)
TWENTY (20)
TWO (2)
WHISKEY (W)
TU-2e BEAR
IL-14 CRATE X 2
hIG-23 FLOGGER
GOLF
BEEP’!' X 4
D-30
FOURTEEN (14)
LIGHT
^^IKE (N-)
NINETEEN (19)
ONE (1)
UPPER RIGHT
TWENTY-ONE (2l)
mirTP3^« Y
TWENTY-FOUR (24) X 2
FROG-3
BEEP’*' X 2
LOWER RIGHT X 9
TWENTY-THREE
BEEP- X 12
BTR-152
EIGHT (8)
FIFTEEN (15)
FOUR (4) X 2
FRIGATE
HEAVY
LIGHT
MEDIUM X 4
T-10
T-e2
T-72
145
^ ^ y<*
#1% #1% #*y^ r|%
* UTTIRANCE -
A ils il2 3ic iJs sdc 2lr 5*r i!r ^£ i!t 2i a!s
TWO
TWO
TWO
TWO
TWO
TWO
TWO
UNIFORM
UPPZR LEFT
UPPER LEFT
UPPER LEFT
UPPER LEFT
UPPER LEFT
UPPER LEFT
UPPER LEFT
UPPER RIGHT
UPPER RIGHT
UPPZR RIGHT
UPPZR RIGHT
UPPER RIGHT
VICTOR
VICTOR
VICTOR
VICTOR
VICTOR
VICTOR
VICTOR
VICTOR
VICTOR
VICTOR
VICTOR
VICTOR
VICTOR
VICTOR
VICTOR
VICTOR CLASS
VICTOR CLASS
VICTOR CLASS
VICTOR CLASS
VICTOR CLASS
WHISKEY CLASS
YAK-26P FIREBAR
YAK-26P FIREBAR
YANKEE CLASS
YURKA CLASS
YURKA CLASS
YURKA CLASS
y^ y* y^ y^ sy y^ yy y^
# 1 ^ > 1 % > 1 ^ #|« < 1 ^ ^ 1 % ^ 1 %
MISRSCCGNITION (S)
:;5 3;^ jlc 5;< 5;: 5{c 5{; sjt 3^ 5^ 5|c 5;c
TEN (le) X 4
TERES ^3) X 6
TU-22 BLINDER
TWENTY-TWO (22) X 5
YAK-26P FIREBAR
2SU-57/2 X 12
ZU-23/2 X 8
BRDM
BEEP* X 21
BRAVO CLASS X 3
EIGHTEEN (18)
ELEVEN (11)
KOTLIN CLASS X 3
LOWER LEFT X 5
UPPER RIGHT X £
BEEP’!' X 8
LIGHT
LOWER LEFT X 4
LOWER RIGHT X 9
UPPER LEFT X 7
BEEP’S' X 17
CARRIAGE RETURN
CARRIER X 2
D-30
FIFTEEN (15) X 2
FRIGATE X 2
HEAVY
INDIA (I) X 2
M-1955
NOVEMBER (N)
QUEBEC (Q) X 2
SA-6 GECKO
THREE (3) X 2
TU-2e BEAR
WHISKEY
BEEP’S*
KARA CLASS
KYNDA CLASS X 2
MIRKA II CLASS
NANUCHKA CLASS
KANIN CLASS
MIG-19 FARMER
TU-28P FIDDLER
KANIN CLASS X 4
KYNDA CLASS
PRIMORYF CLASS
VICTOR CLASS X 3
146
^
* UTTERANCE ^
^ ^ ju ^ ^
0\^ ^1% r,% #1%
5;c:;:s;c^:^j;c5;:5jc^3{s3;c3t:5lc5;2 5;s;;c:;cs5t:;cs;c:;:^
- MSRSCOGMTION (S)
O'** V* ^V
#1% ^|4 #1% #1^ ^1% #1% #1%
ZERO
ZERO
ZERO
ZERO
ZSU-23/4
ZSU-57/2
ZSU-57/2
ZSU-57/2
ZU-23/2
ZU-23/2
ZU-23/2
backspace (CTRL A)
KILO (K) X 2
hOBILE
ZSU-57/2 X 2
ZU-23/2 X 2
ASU-57
ZSU-23/4
ZU-23/2
SU-19 FENCER
TU-22 BLINDER
TU-26 BACKFIRE
147
APPENDIX I
RESULTS FOR PRE/POST SUBJECTIVE QUESTIONNAIRE
The followine,' data reflect whether subjects' attitudes
shifted either toward typing or voice as a result cf the
experiment. A two-tailed si^^ test was used. Note: Means
for the pre/post given below may be misleading if thought to
be indicative of the shift. Tne sign test looks at the fact
of whether their was a shift or not, and ignores the amount
of shift in the analysis, since the amount may be somewhat
arbitrary.
QUESTIONS and
PRS / POST MEANS
for 20 subjects.
SHIFTS
towa rd
TYPING
SHIFTS
towa rd
VOICE
NO
SHIFT
OC =.10
S ignif?
1. Which data entry m.ode
do you think is the
easiest to use to enter
character strings and
commands? (5.1/6.1]
T
12
p
YES
2. Which data entry mode
do ycu think is the
fastest mode for entering
character strings and
commands? (5.1/5.6)
3
13
4
YES
3. Which data entry mode
is the most accurate for
entering character
strings and commands?
(4.1/4.8)
4
9
7
NO
4. Which data entry mode
provides the most
flexibility, in general,
for interaction with a
computer? (5.1/5.1)
4
12
4
YES
5. Which data entry mode
3
8
9
NO
would you prefer tc
operate for several
hours, if required?
(4.3/4.3)
146
I
i
Id
I
I
I
CUESTIONS and
PRI / POST r.EANS
for 2i2 subjects.
SHIFTS SHIFTS
toward toward NO
TYPING VOICE SHIFT
6. Which data entry rrode
would you prefer to
operate as a more
sporadic user of a
computer system?
(4.2/4.3)
7. Which data entry mode
promotes the most relaxed
operation? (5.0,5.1)
6. Which data entry mode
would be the most
advantageous to use to
update an on-line data
base of intelligence
information? (5.1/5.0)
9. Which data entry mode
provides the best
man-machine interface in
a time-critical, high
pressure work
environment? (5.0/5.0)
10. Which data entry
mode do you think is the
easiest to to learn?
(4.9/5.6)
3 10 7
5 6 7
3 9 6
1 =,
12
2 13 5
OC =.10
Signif?
YES
NO
NO
NO
YES
149
1
0
ft
LIST OF REIERENCiS
1. Lea, W. A., Treads in Speech Recogaitioa . Prentice-Hall
Inc., p. 68-c9, I960.
2. Naval Postgraduate School Report NPS-55-60-016,
Experiments with Voice Input for CoriTiand and Control ,
hy G.K.Poock, April 1960.
3. Lawson,S., "Naval Tactical C3 Architecture 1985-
1995," Signal, p. 72-76, August 1976.
4. Lea, op. cit., p. 3.
5. Rome Air Development Center Briefing, "dOD Automated
Exploitation System," February 1961.
6 .
Ibid
•
7.
Lea,
op. cit..
P.
30-3
6 .
Lea,
op. cit.,
? •
40.
9.
Lea,
op. cit..
P •
24.
10 .
Lea,
op. cit.,
P-
92.
11.
Lea ,
op. cit..
P-
4-7.
12. Peek, B., Meuberg, E.P. and Eodge, D.C., 'An Assessment
of the Technology of Automatic Speech Recognition for
Military Applications," IEEE Transactions Acoustics,
Speech, and Signal Processing, ASSP-25, Number 4, p.
310-322, 1977.
13. Rome Air Development Center Report RADC-TR-80-220, DIMS
Voice Data Entry , by Phillips E. Scott, Threshold
Technology Inc., June 1980.
14. Rome Air Development Center Report RADC-TR-7S-209, Word
Recognition . by Phillips B. Scott, Threshold
Technology Inc., September 1976.
15 ?
15. Potre Air Devsloprrect Center Report RADC-TH-77-164,
Alpha/Numeric Sxtraction Techr.lQiie Phase II . by
Phillips B. Scott, Threshold Technolot^y Inc., Key
1977.
16 . Foock, op. loc.
17. KcSorley, W., Using Voice Kecoignition to Run the Warfare
Environment Simulator Masters Thesis , Naval
Postgraduate School, Monterey, California, March 1961.
IS. Rome Air Development Center Report RADC-TR-80-74,
Advanced Image Exploitation Aids , by John R. Welch
and E. Shamsi, March 1950.
19. Pccck, op. Icc.
20. Threshold 600 User's Manual, Threshold Technology Inc.,
1978.
21. Conners, E., and Vance C., "Operation of tne ^0-1130
Harvard Tachistoscope and Peripheral Equipment," Navel
Postgraduate School CS 3565 Paper, p. 1-3, September
1978.
22. United States Strategic Institute Report, USSI 76-2,
Soviet Warsaw Pact Force levels , by J. Erickson, 1976.
23. Donnely, C., and others. The Soviet War Machine; An
Encyclopedia of Russian Military Equipment and
Strategy , Chartwell Becks, Inc. 1976.
24. Defense Intelligence Agency DBB-2680-40-78, Handbook on
the Soviet Armed Forces . February 1976.
25. Wiener, E., The Armies of the Warsaw Pact Nations . 2nd
ed., Carl Ueberreuter Publishers, 1978.
151
INITIAL DISTF.IEUTICN LIST
No.
1. Defense Technical Information Center
Cameron Station
Alexandria, Virginia 22134
2. Superintendent
ATTN: Litrary, Cede 0142
Naval Postgraduate School
Monterey, California 93940
3. Superintendent
ATTN: Department Chairman, Code 55
Naval Postgraduate School
Monterey, California 93940
4. Superintendent
ATTN: Professor J. Arima, Code 55Aa
Naval Postgraduate School
Monterey, California 93940
5. Superintendent
ATTN: Professor R. Sister, Code 55Sa
Naval Postgraduate School
Monterey, California 93940
5. Superintendent
ATTN: CDR W. Moreney, USN, Code 55Mp
Naval Postgraduate School
Monterey, California 93940
7. Superintendent
ATTN: Professor D. Neil, Code 55Ni
Naval Postgraduate School
Monterey, California 93940
6. Superintendent
ATTN: Professor G. PcccR, Code 55Pk
Naval Postgraduate School
Monterey, California 93940
9. Superintendent
ATTN: Code 312A
Naval Postgraduate School
Monterey, California 93940
C cpies
2
2
1
1
1
1
1
10
152
10. Superintendent 1
ATTN; Cede 39
Naval Postgraduate School
^’onterey, California 93940
11. Air Force Institute of TecnncIcgy/CIS? 1
ATTN: Major Charles Farnhart, USAF
Wright Patterson Air Force Base
Ohio 45433
12. Air Force Intelligence Service/INT 2
ATTN: It Ccl W. Cray, USAF
Bolling AFB, Washington, D.C. 20332
13. Rorre Air Development Center/IRE 2
ATTN: E. Benfeld, IDES
Driffiss AFB, New York 13441
13. Rome Air Development Center/IRRA 2
ATTN: Lt J. Woodard
Criffiss AFB, New York 13441
14. Aeronautical Systems Division/RWT 1
ATTN: Lt Col J. Turinetti
Tactical Reconnaissance Projects
Wright Patterson AFB, Ohio 45433
15. Strategic Air Command/INXY 2
ATTN: Capt Greg Jay, USAF
Offutt AFB, Nebraska 6ftll3
16. American Sterilizer Company 1
ATTN: T. Brendgord
2424 West 23rd St
Erie, Pennsylvania 16152
17. Crew Systems Technology 1
ATTN: Donald L. Parks
Seeing Commercial Airplane Co.
P.O. Box 3707
MS 47-08
Seattle, Washington, 98124
IS. Dipl.-ing Eartmut Mutschler 1
Wissencraftl.Mitarbeiter
Fraunhofer-Institute Fur
Informations-Und Da tenverabelting
Rintheimer Strabe 19
D-7500 Karlsruhe 1
Germany
153
1
19. SeneraL Dynarrics
ATTN: J. Mke Byrd
hail Zone 8227-1
P.O. Box S5106
San Diego, California 92136
20. Babcox and Wilcox 1
Nuclear Power Generation Division
ATTN: Robert L. Starkey
P.O. Box 1260
Lynchburg, Virginia 24505
21. International Telephone and Telegraph 1
Great Pasters House
Human Factors
ATTN: Barry Drake
Edinburgh Way
Harlow, Essex
England
22. Texas Instruments, Inc. 1
Human Factors
ATTN: Kenneth C. Bice
P.O. Box 2909
MS 2201
Austin, Texas 76763
23. Walt Goede 1
Consultant
31051 Hawksmoor Drive
Rancho Palos Verdes, California 90274
24. International Telephone and Telegraph 1
ATTN: H. Rudy Ramsey
1000 Oronoque Lane
Stratford, Connecticut 06497
25. Lear Siegler, Inc 1
ATTN : Ivan Belya
4141 Eastern Avenue S. E.
hS 128
Grand Rapids, Michigan 49506
26. TRW 1
MTS-Men-Machine Interface
System Design Department
ATTN: C. E. (NED) Wilkins
Cne Space Park
Redondo Beach, California 90278
154
27 .
1
TRW
Systems Analysis Eepartment
ATTN: (Matthew F. Carroll
MS 75-190
One Space Park
Redonio Beach, California 9027S
26. American Telephone and Telegraph 1
ATTN : R . E. Cochrane
Engineering Maniger for Human Factors
Room 4C154
Eelminster, New Jersey 07921
29. general Motors Corporation 1
Industrial Relations Staff
Director-Ergonomics
Health Services
ATTN: Roger L. Kuhn
3044 West Grand Elvd.
Detroit, Michigan 46202
30. Northrop Corporation 1
ATTN; Jeffrey E. Miller
2301 West 120th Street
Eawthorne, California 90250
31. Northrop Corporation 1
Electronics Division
ATTN: Compass Preview Program Manager
1 Research Park
Palos Verdes Peninsula, California 90274
32. John Herchenroder 1
204 North Wakefield Street
Arlington, Virginia, 22203
33. Sperry Univac 1
ATTN: Michael L. Schneider
Computer Scientist
P.O. Box 500
Blue Bell, Pennsylvannia 19424
34. U.S. Army Human Engineering Lab 1
ATTN: Richard N. Armstrong
Box 476
Fort Rucker, Alabama, 35362
15 ^
Ohio State University
1
Industrial and Systems J!.ngineerin?
ATTN: Gayle L. Berry
1971 Neil Avenue
Columbus, Ohio 43210
26 .
University of Nebraska at Lincoln
1
Industrial Engineering
ATTN: David J. Cochran
Lincoln, Nebraska 65586
37. Systems Research laboratories
Human Factors Engineering
ATTN: Chris Hale
2600 Indian Ripple Road
Dayton, Ohio 45440
1
38. University of California
1
Computer Science and Applied Math Department
Lawrence Berkeley Lab
ATTN: Aaron Marcus
Building 503
Room 2238
Berkeley, California 94720
39. Bunker Ramo 1
Electronic Systems Division
ATTN: CATIS Program Manager
31717 La Tienda Drive
Box 5009
Westlake Village, California 91359
40. Texas Instruments, Inc. 1
Electronic Systems Division
ATTN: TIPI Program Manager
Lewisville, Texas 75067
41. Harris Corporation 1
ATTN: Larry Lamb
MS 22/2419
P.O. Box 37
Melbourne, Florida 32901
42. Rodney Elden 1
Management Consultant
5 Middle Road
Broniville, New York 10708
156
43. Defence and Civil Institute of 1
Invironmental i^edicine
ATTN: Ing. L. Van Breda
P.O. Box 2002
1133 Sheppard Avenue West
Downsview, Ontario
Canada MM 3B9
44. Naval Ocean Systerrs Center 1
ATTN: Carl Rcsengrant, Code 6141
San Diego, California 92152
45. Merck and Corrpany 1
ATTN: Nancy Woo
R 84-17
Box 2000
Rahway, New Jersey 27065
4c. National Aeronautics and Space Administration 1
ATTN: 2. L. Weiner
MS 23903
Moffett Field, California 94035
47. Naval Undersea Systems Center 1
ATTN: Anthony Bessacini, Code 3522
Newport, Rhode Island 02S42
48. National Security Agencey 1
ATTN: John F. Boehm
R542
Fort Meade, Maryland 20750
49. Naval Training and Equipment Center 1
ATTN: R. Breaux, Code M-711
Orlando, Florida 32613
50. Office of the Undersecretary of Defense 1
Research and Engineering
ATTN: Cdr Paul Chatelier, USN
Room 3D129
Pentagon, Washington, D.C. 20301
51. National Aeronautics and Space Administration 1
ATTN : Clay Coler
MS 23902
Moffett Field, California 94235
157
I
52. t\dval Undersea Systems Center 1
ATTN: Dianne Davis, Cede 3522
Newport, Rhode Island
53. Naval Undersea Systems Center 1
ATTN: Idward De Gregario, Code 3522
Newport, Rhode Island 22340
54. U.S. Army Engineer Topographic and 1
and Research Institute
ATTN : Tice De Yeung
Fort Eelvoir, Virginia 22060
55. U.S. Army Applied Technology Lab 1
Fort Sustis, Virginia 23662
56. United States Postal Service 1
Research and Develcnment Lab
ATTN : Harold C. Glass
11711 Parklawn Drive
Rockville, Maryland 20852
57. Air Force Aerospace Medical Research lab 1
ATTN; Don Me Kechnie
Wright Patterson AFB, Chio 45433
58. Air Force Aerospace Medical Research Lab/EEA 1
ATTN: Thomas J. Moore
Wright Patterson AFB, Ohio 45433
56. Air Force Aerospace Medical Research Lat/EBM 1
ATTN; Capt Vince Mortimer, USAF
Wright Patterson AFB, Ohio 45433
59. Naval Aerospace Medical Research Lab 1
Acoustical Sciences Division
ATTN; James Mesko
Naval Air Station
Pensacola, Florida 52506
60. U.S. Army Signal Center 1
Directorate of Training
ATTN: Capt Leslie Scofield, USA
Fort Forden, Georgia 30905
61. Naval Air Development Center 1
ATTN; C. Skriver, Code 6021
Warminster, Pennsylvania 13974
156
m
62 . i\aval Undersea S/star's Center
ATTN: S. Nils Straaveit, Code 317
New London, Coaneoticnt 26322
63 . Fleet Material Support Office
ATTN: Leahrrond Tyre, Code 9333
Mechanicsturg , Pennsylvania 17C55
64. Air Force Weapons Analysis Laboratory/FGR
ATTN: Eric Werkowitz
Wrieht Patterson AF3, Chio 45433
c5. Naval Air Development Center
ATTN: T. Weiner, Code 4043
Warminster, Pennsylvania 16974
66 . Naval Air Systems Command
ATTN: Cdr Chuck: Hutcnins , Air-340F
Jefferson Davis Highway
Arlington, Virginia 20360
57. Army Communicative Technology Office
ATTN: Major W. MacHarrie
Box 4337
Fort Sustis, Virginia 23624
cS. National Securtity Agency
ATTN: Charles Wayne
.R54
Fort Meade, Maryland 20755
69. DAVAA-S
Avionics Research and Development
ATTN: Lockwood Reed
Fort Monmouth, New Jersey 07703
70. Army Research Institute
PERI-0U
ATTN: J. N. McConnell
5001 Eisenhower Avenue
Alexandria, Virginia 22333
71. Air Force Aerospace Medical Research Lab/BBA
ATTN: P.ichard McKinley
Wright Patterson AFB, Chio 45435
1
1
1
1
1
1
1
1
■1
±
1
72. Texas Instruments, Inc.
ATTN: George roddinglon
Eox 225956
MS 371
Dallas, Texas 75243
73. IBM Research Center
ATTN: N. Hex Dixon
Box 2lS
Yorktown Heights, New York 10596
74. Massachusetts Institute cf Technology
ATTN : Victor Zue
Room 36-543
Camhridge, Massachusetts 02139
75. Bolt, Beranek, and Newman, Inc.
ATTN: Jared Wclf
50 Moulton Street
Camhridge, Massachusetts 02238
76. Naval Air Development Center
ATTN; Norm Warner, Code 6021
Warminster, Pennsylvania 18974
77. Robert Lynchard
3165 McCrory Place
Orlando, Forida 32603
78. Naval Aerospace Medical Research Lab
ATTN: Cdr James Goodson
Naval Air Station
Pensacola, Florida 32508
79. Armament Division/XRC
ATTN; H. F. Brown
Fglin AF3, Florida 32541
80. Naval Aerospace Medical Research Lab
Acoustical Sciences
ATTN; Carl Williams
Naval Air Station
Pensacola, Florida 52508
81. Naval Air Test Center
Systems Engineering Test Directorate
ATTN: Andrew Cruce, Code 57030
Patuxent River, Maryland 20670
1
1
1
1
1
1
1
1
1
1
160
62. Internal Revenue Service 1
ATTN; Thcn^as Cullen
1201 East Street NW
■A'ashingtcn, D.C. 20224
£3. Internal Revenue Service 1
ATTN: Xlaus Erosius
1201 East Street NW
Washington, E.C. 20224
54. Vertex Corporation 1
ATTN: Janet Baicer
Two Oak Park
Bedford, [Massachusetts 01730
65. United States Postal Service 1
Research and Development Lab
ATTN; Arnold Craft
11711 Parklawn Drive
Rockville, Maryland 20852
£6. Army Research Institute 1
ATTN: Irv Alderman
5001 Eisenhower Avenue
Alexandria, Virginia 22333
£7. Navel Ocean Systems Center 1
ATTN: John Phillips, Code 7232
San Diego, California 92152
£8. Wayne Lea 1
£89 Sanford Court
Santa Barbara, California 93111
£9. Naval Electronics Systems Center 1
ATTN: Erank Deckelman, Code 330
2511 Jefferson Davis Highway
Arlington, Virginia 20360
90. Naval Ocean Systems Center 1
ATTN: Bruno White, Code £302
San Diego, California 92152
91. Naval Ocean Systems Center 1
ATTN: William Dejka, Code 8302
San Diego, California 92152
161
92. Advanced Research Projects Agency/IPTO 1
ATTN: Lcdr J. Dietzler, USN
1420 Wilson Blvd.
Arlington, Virginia 22362
93. Bolt, Beranek, and Newman, Inc. 1
ATTN: Richard Pew
50 houlton Street
Cambridge, Massachusetts 20138
94. University of Missouri at Columbia 1
Industrial Engineering Department
ATTN: Marlin Thomas
Room 113, Electrical Engineering Building
Columbia, Missouri 64211
95. Office of Naval Research 1
ATTN: M. Talcott, Code 455
£00 North Cuincy Street
Arlington, Virginia 22217
96. Office of Naval Research 1
ATTN: G. Maleclci, Code 455
Arlington, Virginia 22217
97. Bolt, Beranek, and Newman, Inc. 1
ATTN: N. Greenfield
50 Moulton Street
Cambridge, Massachusetts 02136
98. Air Force Human Resources Lab/TT 1
ATTN: Col Richard Shelton, USAF
Lowry AFB, Colorado 60230
99. Armed Forces Air Intelligence Training Center 1
ATTN: Capt Dick Rewalt
Building 360
Lcwry AFB, Colorado 60230-
100. University of Southern California 1
Information Sciences Institute
ATTN: R. Bisbey
4676 Admiralty Way
Marina Del Ray, California 90291
101. Naval Ocean Systems Center 1
ATTN: R. Kolb, Code 624
San Diego, California 92152
162
1
102. Coinmander in Chief Pacific Fleet
ATTN: Car R. heinhold, Code £4
3cx 6
Pearl Earbcr, Hawaii 96960
103. Stanford Research Institute
Artificial Intellignce Center
ATTN: Daniel Sagalowicz
33 Ravenswood Avenue
I^enlo Park, California 94025
104. Science Applications Incorporated
ATTN: Russ Hammond
Suite 1200
1911 North Fort Meyer Drive
Arlington, Virginia 22209
105. S. Parsons
19740 Via Fscuela Drive
Saratoga, California 95070
106. University of Michigan
Industrial Engineering Department
ATTN: Don Chaffin
Ann Arbor, Michigan 4S104
106. University of Michigan
Industrial Engineering Department
ATTN: Walt Hancock
Ann Arbor, Michigan ^48104
107. Noval Ocean Systems Center
ATTN: Dennis hcCall, Code 6242
San Diego, California 92512
108. Office of Naval Research
ATTN: Marvin Denicoff, Code 437
800 North Quincy Street
Arlington, Virginia 22217
109. Naval Ocean Systems Center
ATTN: John Schill, Code 92152
San Diego, California 92152
110. University of Pennsylvania
Wharton School of Business
ATTN: E. Morgan
Room W-63, Dietrich Hall
Philadelphia, Pennsylvania 19104
1
1
1
1
1
1
1
1
1
163
111. Naval Electronics Systems Command
ATTN: Dan Schutzer, PME 12£
2511 Jefferson Davis Highway
Washington, D.C. 22360
112. Advanced Research Projects Agency/IPTO
ATTN: R. Kahn
1400 Wilson Elvd.
Arlington, Virginia 22209
113. Threshold Technology, Inc.
ATTN : Tom Martin
1629 Underwood Blvd.
Delran, New Jersey 06075
114. Threshold Technology, Inc.
ATTN: Joe Bove
lt529 Underwooa Blvd.
Delran, New Jersey 06075
115. Threshold Technology, Inc.
ATTN: Phillips Scott
1529 Underwood Blvd.
Delran, New Jersey 06075
115. Advanced Research Projects Agency/CTO
ATTN : Craig Fields
1400 Wilson Blvd.
Arlington, Virginia 22209
117. Comm.andant of the Marines
Scientific Advisor
ATTN: A. L. Slafkosky, Code RD-1
Washington, D.C. 20360
118. Institute for Defense Analysis
ATTN: Jesse Crlansky
400 Arny-Navy Drive
Arlington, Virginia 22202
119. Naval Ocean Systems Center
ATTN: Clen Allgaier, Code 6242
San Diego, California 92152
120. Air Force Aerospace Medical Research Lah/KEF
ATTN : Den Topmiller
Wright Patterson AFB, Chio 45433
■1
1
1
1
1
1
1
1
1
1
164
1
121. Naval electronics Systems Command
ATTN: R. Jratilla, Code 330
2511 Jefferson Davis Highway
Arlington, Virginia 20360
122. Naval Electronics Systems Commend 1
ATTN: J. hachodo, Code 330
2511 Jefferson Davis Highway
Arlington, Virginia 20362
123. Capt John Armstrong 1
6445 Sugar Creek Way
Orleans, Ontario
Canada KIC lYl
124. Department of National Defence 1
NDHS DAS Eng. 4
ATTN: Lt Col J. A. Wellington, CAF
101 Colonel by Drive
Ottawa, Ontario
Canada KIA 0E2
125. Romie Air Development Center/IRRA 1
ATTN: R. Vonusa
Sriffiss AF3, New York 13441
126. Lt Col r^ark Smith, USAF 1
9500 Braddock Road
Fairfax, Virginia 22032
12?. Computer Corporation of America 1
ATTN: Chris Eerot
575 Technology Square
Cambridge, Massachusetts 02139
128. Digital Equipment Corporation 1
ATTN: Paul Thordarson, ML3-2/E41
146 Main Street
Maynard, Massachusetts 02139
129. Thomas J. Watson Researcn Center 1
ATTN: John Gould
Box 2lS
Yorktown Heights, New York 10596
130. Lockheed Missile and Space Division 1
Department 86-10
ATTN: Leon Herman
Box 182
Building 504
165
Sunnyvale, California 94266
131. Biotechnology, Inc. 1
ATTN: Harold Price
3027 Rosemary Lane
Falls Church, Virginia 22042
132. f'iaj Warren Watkins, USAF 1
1 STRAD/D02
Vandenburg AFE, California 93437
133. Olin Campbell 1
Suite 201
1150 South State Street
Orem, Utah 84057
134. Commander in Chief Pacific Fleet 1
TRW Field Office
ATTN: George Harris, Code N-34
Pearl Harbor, HI 96860
135. Air Force Aerospace Medical Research Lab/HFC 1
ATTN: Lew Hahn
Wright-Patterson AFB, Ohio 45433
136. California Institute of Technology 1
Jet Propulsion Laboratory
Systems Analysis Section
ATTN: Robert L. Sohn
4600 Oak Grove Drive
Pasadena, California 91103
137. Massachusetts Institute of Technology 1
Lincoln Laboratories
ATTN : Cliff Weinstein
Room B-335
Lexington, Massachusetts 02173
138. David Jcly 1
2180 Bryant Street
San Francisco, California 94110
139. Naval Training Equipment Center 1
Human Factors Laboratory
ATTN: Elizabeth Lambert, Code N-711
Orlando, Florida 32813
166
1
140. Threshold Technclcsy, Incorporated
ATTN: Fred Gladney
Suite 4 - Cl
1440 South State College 31vl.
Anaheim, California 92606
141. Heuristics, Inc. 1
ATTN : Tom Imperato
1285 Hammerwood Avenue
Sunnyvale, California 94266
142. XYBICN 1
ATTN: Paul Frost
7 Ridgedale Avenue
Cedar Knolls, New Jersey 07927
143. Threshold Technology, Incorporated 1
ATTN: John V/elch
1829 Underwood Elvd.
Delran, New Jersey 26075
144. Cffice of Naval Research 1
ATTN: Robert Wisher, Code 456
800 North Quincy Street
Arlington, Virginia 22217
145. Honeywell, Incorporated 1
Systems and Researcn Center
ATTN: Robert North
2600 Ridgeway Elvd.
Minneapolis, Minnesota 55413
146. National Bureau of Standards 1
Information-Communications Systems Technology
ATTN: Dave Pa 11ett
A219 Technology Building
Washington, D.C. 22234
167
i
5
Thesis
J345
c.l
Jay
i
192411
An experiment in
yoice data entry for
'"lagery interpretation
reporting.
> (3 f tti
^ Jc* o5
28 1 a g
^‘0123
^ 2 5 2 P
Thes is 192411
J345 Jay
c.l An experiment in
voice data entry for
imagery interpretation
reporting.
Live Music Archive
Librivox Free Audio
Metropolitan Museum
Cleveland Museum of Art
Internet Arcade
Console Living Room
Books to Borrow
Open Library
TV News
Understanding 9/11