Emacspeak Direct Speech Access T. V. Raman Senior Computer Scientist Adobe Systems c #1996 Adobe Systems Incorporated.All rights reserved. 1 Outline # Overview of speech applications. # Emacspeak --Architecture. # Emacspeak --The user experience. c #1996 Adobe Systems Incorporated.All rights reserved. 2 Screen Access User explores visual display to: # Construct a mental model of the interface. # And interpret intent of the UI. Aural output --Derived from visual display. c #1996 Adobe Systems Incorporated.All rights reserved. 3 Screen Access Design # Encapsulate display in an off-screen model. # Present this model aurally. # Enable navigation of this model. Aural feedback is application independent. c #1996 Adobe Systems Incorporated.All rights reserved. 4 Available Screen Access Character-based interfaces :::GUI. Richer layout of GUI l Harder to build OSM l But richer OSM l Enables possibly richer feedback c #1996 Adobe Systems Incorporated.All rights reserved. 5 Screen Access Innovations # Speak specific areas of the screen. # Attach meaningful labels to icons. # Navigate window hierarchy. Access separate from application. c #1996 Adobe Systems Incorporated.All rights reserved. 6 What Is UI? 1. Obtain user input 2. Compute on the information 3. Display the results UI = Input+Output c #1996 Adobe Systems Incorporated.All rights reserved. 7 Speech Enabling Applications # Treat speech as a first class medium. # Application produces its own feedback. # Exploit features of the spoken medium. Audio output independent of visual display. c #1996 Adobe Systems Incorporated.All rights reserved. 8 Contrasting Approaches Read Screen Speech Enable Speak display Speak information No context Full context Independent Integrated Global scope Local context Uses display Uses environment c #1996 Adobe Systems Incorporated.All rights reserved. 9 Advantages # Produce intuitive feedback. # Provide a simpler user model. # Reduces users cognitive load. User works with one --not two-- applications. c #1996 Adobe Systems Incorporated.All rights reserved. 10 Example March 1996 Sun Mon Tue Wed Thu Fri Sat 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 Easy to see relevant information. c #1996 Adobe Systems Incorporated.All rights reserved. 11 EMACS A customizable, extensible user environment: # Editing and browsing electronic information. # Software development. Unified interface!Cooperating applications c #1996 Adobe Systems Incorporated.All rights reserved. 12 Emacspeak # Device independent speech interface. # Core modules for basic spoken feedback. # Extensions for various applications. Does not modify Emacs code-base. c #1996 Adobe Systems Incorporated.All rights reserved. 13 Advice Overview BEFORE Called before advised function. AFTER Called after advised function. AROUND Called instead of advised function. Advice enables code re-use. c #1996 Adobe Systems Incorporated.All rights reserved. 14 Advice Let x ! f ! y. BEFORE AROUND AFTER Result in function f 0 that computes y 0 . x ! f --- #z # f 0 ! y 0 c #1996 Adobe Systems Incorporated.All rights reserved. 15 The User Experience # Succinct contextual speech feedback. # Auditory icons augment interaction. # User action flushes prior speech immediately. User focuses on task at hand. c #1996 Adobe Systems Incorporated.All rights reserved. 16 Demonstration --Editing # Simple editing, search and replace. # Completion and spell checking. # Syntax coloring using voice-lock mode. Intuitive interface enables fluent interaction. c #1996 Adobe Systems Incorporated.All rights reserved. 17 Demonstration --Browsing Information # Browsing the file system. # Reading and responding to email and news. # Browsing the WWW. Window to digital information. c #1996 Adobe Systems Incorporated.All rights reserved. 18 WWW --Speech Style Sheets # Voice properties, # Auditory icons, # Sound cues for document elements. Generate richly formatted audio documents. c #1996 Adobe Systems Incorporated.All rights reserved. 19 Demonstration --System Tasks # Running a shell. # Running terminal based applications. Behaves like a traditional screen-reader. c #1996 Adobe Systems Incorporated.All rights reserved. 20 Device Independent Speech # Dectalk Express and MultiVoice # Software Dectalk --DEC-ALPHA # DECFACE --A Talking Face Supporting other devices is straightforward c #1996 Adobe Systems Incorporated.All rights reserved. 21 Core Modules Speech enables # Navigation, editing, and prompting. Sufficient to implement a working system. c #1996 Adobe Systems Incorporated.All rights reserved. 22 Application Specific Extensions The icing on the cake! # WWW browser, Mail and News reader # Tools: Calendar, Spell checker, ::: Code-size is less than 1% of original. c #1996 Adobe Systems Incorporated.All rights reserved. 23 Advice Statistics Package Code Ext % Advice Core 217,295 3,152 1.45% 188 W3 17,384 515 2.96% 12 Mail 17,943 270 1.50% 14 GNUS 35,528 501 1.41% 37 Emacspeak --A model of code re-use. c #1996 Adobe Systems Incorporated.All rights reserved. 24 Conclusion # Treat speech I/O as a first-class medium. # Separates UI and computation. # Enables rich, well-structured interfaces. c #1996 Adobe Systems Incorporated.All rights reserved. 25