Source
Automatically imported from: http://commons.somewhere.com:80/rre/1994/HOTT.Issue.94.425.Part.1.html
Content
| | | | --- | --- | | Red Rock Eater Digest | Most Recent Article: Thu, 13 Sep 2001 |
HOTT: Issue 940425, Part 1 of 3
``` I won't forward HOTT to RRE regularly; those who are interested can subscribe directly.
Date: Tue, 3 May 1994 17:39:11 -0700 From: hott-list-relay@ucsd.edu To: hott-list@ucsd.edu Subject: HOTT: Issue 940425, Part 1 of 3
HOTT (Hot Off The Tree), Issue 940425 Part 1 of 3 (each part is about 25-35 pages in length) Monday, 25 April 1994
IN THIS ISSUE
Note: It's best to download this "message" to a standard word processing package. This allows a reader to select specific articles uniquely identified by the form [x], e.g., [19], [27], etc., or search articles by keywords. To assist reader selection, actual article titles may be augmented or replaced with descriptive article titles.
Parts 2 & 3 will be transmitted over the next two to three weeks, and will include a report from Interop. If part 3 is transmitted in three weeks, it will include a report from Meckler's Virtual Reality Conference & Exhibition. BTW, subscribers, especially FidoNet sysops and those receiving HOTT via RadioMail, have suggested that I limit "messages" to 40 pages. As a result, this issue has been divided into thirds.
[1] Inaugural issue editorial (please read this) [2] HOTTpixels: A column by founding editor Susan Jurist
---
SPECIAL: A FULL-TEXT TUTORIAL ON NEURAL NETWORKS (see item [X], the second to last article in Part 1)
---
VIRTUAL REALITY
[3] Virtual warriors (military apps of VR) [4] Ready for a wild ride? (Digital adventures in entertainment) [5] Virtual reality shapes surgeons skills [6] Virtual reality: Immerse yourself (Wall Street applications) [7] VR becoming reality for everyone (technical issues) [8] Virtual reality moves into design (market projections)
INTELLIGENT AGENTS
[9] Software valets that will do your bidding in cyberspace [10] Robo-software reports for duty [11] The metaphor is the message (Microsoft's agent SW project) [12] The butlers of the digital age will be just a keystroke away [13] Software 'agents' will make life easy (General Magic) [14] Just like Magic? (General Magic) [15] Agent technology stirs hope of magical future (Magic & AT&T) [16] Telescript eases cross-network communication (Magic & AT&T)
SPEECH RECOGNITION
[17] Computer: Take a memo (IBM's Personal Dictation System) [18] Number please: Speech recognition over the phone [19] Talking to computers: Time for a new perspective (from WiReD)
INFORMATION SUPERHIGHWAY
[20] There's no reason to subsidize access to information highway (Michael Schrage column) [21] Information highway will create jobs [22] Many PC makers steer clear of information highway [23] Superhighway into the home (interactive TV) [24] No space in cyberspace? (online services & the Internet) [25] Information superhighway: A refreshing approach, but some old questions (survey results)
PCS (PERSONAL COMMUNICATIONS SERVICES)
[26] All segments of PCS industry expected to see robust growth [27] Sweet successes in wireless attracting investors to PCS [28] PCS: Integrated wireless telephone-computer opportunities [29] PCS: Will it end cellular telephones? [30] PCS: Hands-on communications for all (telephony aspects) [31] Satellite-based Personal Communication Services
MOBILE COMPUTING
[32] Telcos may decide PDA fate [33] Going on-line when you're off the beaten path [34] Michael Finley column on pen computing
COMPANY PROFILES
[35] How Mac changed the world [36] Microsoft hits the gas (Microsoft's R&D activities)
NEURAL NETWORKS
[37] Neural network enhances ICU patient monitoring [X] Full-Text: The basic ideas in neural networks
ANNOUNCEMENT
[Z] VR company managing partner is the presenter at the next session of the Caltech/MIT Enterprise Forum (Avatar Partners; Peter Rothman, Managing Partner)
[1] FROM THE EDITOR (& PUBLISHER)
Since time is the one commodity that's in very short supply, I've decided to "kill two birds with one stone" and write a single editorial for the first issue of the "reinvented" HOTT (Hot Off The Tree) "e-zine", my new Internet-based electronic magazine, and IEEE Engineering Management Review (EMR). (Most editorials will be a screen or less, but this inaugural editorial is about three Microsoft Word pages.)
HOTT has evolved since December when I assumed the helm as editor and publisher. (IEEE is the publisher of EMR; I'm only the editor.) Rather than focusing on what it was, I'd like to state what it is (and will be). Thanks to the power of reader feedback through Internet e-mail, I've been able to adapt HOTT to meet a lot of different needs.
Coverage has been expanded from 100 sources to over 3,000 sources! I'm using CD-ROMs, floppies, online databases, document delivery services, electronic and manual clipping houses, printed and faxed current awareness services, multi-authored books, conference proceedings, and over a hundred subscriptions for my information pool. (Don't ask which sources I'm scanning -- that's my competitive advantage.) These sources cost about US$20,000 per year! And, if all goes well, I'll be adding services costing at least US$50,000 yearly. HOTT covers trade magazines, research journals, trade shows, conferences, U.S. and international dailies, multi-authored books, business magazines, news weeklies, and selected U.S. television broadcasts. Eventually, I hope to expand coverage of conferences (and conference proceedings), global broadcast media, and foreign language sources ... provide a HOTTWire news service ... and, publish HOTT in several languages. It's possible to do all of this NOW, but I don't have the financial resources.
HOTT will occasionally include exclusive interviews, transcripts of keynote speeches, and "think tank" technological forecasts. HOTT's interviews will be more "techie" than those in business magazines or dailies. Transcripts of keynote speeches may be transmitted as separate messages, similar to the January transmission of the Winter Consumer Electronics Show keynote speech.
But it's the forecasts that are HOTT's key value-added service. Bottom line: Too many published forecasts use questionable forecasting techniques. That's not to say that my techniques are fool-proof. But only fools believe some of the garbage that passes for legitimate market research. (Example: If you want to know how many PDAs will be sold in the year 2000, you can consult one of many market research reports. Or, you may get better results by reading tea leaves or tarot cards.) Not all far-flung projections are worthless, but you just can't tell ahead of time. Hence, caution is warranted. BTW, watch out for so-called industry gurus. Their gut feeling, "I know everything about the industry" projections can often be ripped apart by their biases and other faulty decision making processes. (To Tversky and Slovic: I'll be calling!)
Two sections that were originally planned are being shelved. Too many other e-serials and mailing lists cover forthcoming conferences and threads in Usenet discussion groups. I want to provide original material (even if it's in the form of original article summaries); I don't want to regurgitate what so many others are doing on the 'Net. Books are a trickier issue. I've decided to list books received, and provide special notations for highly recommended and "Must Buy" books. (This is similar to what I've been doing for years in EMR.)
Length and frequency (it's hard ... sorry, difficult ... to resist making a pun) are still two open issues. I'm likely going to shoot for 32 to 40 pages every three or so weeks (about 16 times per year), similar to most U.S. book clubs in frequency, but easily digestible at 40 pages. (Even Edupage has increased their page count; I've come across some recent issues that were nearly 20 pages ... and they're semi-weekly!) This is an area where reader feedback is especially appreciated.
Topical coverage, though, is NOT open for discussion. HOTT will focus on bleeding-edge telecomputing technologies. Examples of HOTT stuff: PDAs (personal digital assistants), interactive multimedia, wireless communications, speech recognition, HDTV and agent-oriented software on the commercialization side; neural nets, fuzzy computing, genetic algorithms, nanotechnology and molecular electronics on the research side; and, transitional technologies such as VR (virtual reality) and telepresence. HOTT is a "what's new" source of information on cutting- edge computer and communications (i.e., "telecomputing") technologies ... it's NOT a PC or Mac digest.
However, HOTT's article summaries should NEVER be viewed as a substitute for original articles. HOTT points to worthy sources; it doesn't pretend to replace them. I encourage our readers to obtain tear sheets and photocopies from a document delivery service or through interlibrary loan. Three of my personal favorites are Ask*IEEE, the British Library, and UMI (University Microfilms). Ask*IEEE may be reached at askieee@ieee.org . The snail mail address for the British Library Document Supply Centre is: Boston Spa, Wetherby, West Yorkshire LS23 7BQ. UMI can be faxed at +1.313.665.7075. Each has a different pricing structure, so it's best to get pricing information before placing an order.
And I encourage our readers to subscribe to those periodicals (and purchase those books and proceedings) that yield the most useful hits. Action item: If you like the summary, please consider subscribing to (or purchasing) the original source.
Before I touch upon EMR, a few closing comments on HOTT. HOTT will remain a FREE, controlled circulation publication. We've had to expand our sponsorship base to include approved, informational advertisements, not just advertorials. (PR budgets for advertorials are too limited to realistically support HOTT.) But don't worry: No hyped display ads will be allowed. Also, no more than 12.5% (1/8th) of HOTT will be ads. Believe it or not, some business magazines are 60%/40% ... 60% ads and only 40% editorial! Once again, the ads will be informational, NOT hype.
We're pushing HOTT as a way to reach over 35,000 innovators and early adopters with individual subscriptions ... and as many as 100,000 through mail exploders, BBSes, Usenet gating, and e-mail forwarding. Also, by limiting HOTT to 7:1 editorial coverage versus ads, sponsors and advertisers will no longer get lost in ad clutter. We're also hoping to provide three "bingo" numbers per ad (versus only one in most printed pubs), with instant Internet-On-Demand (tm) response. A single placement reaches a global audience (rarely true with printed publications), and ad production cost is virtually nil. How about video demos in response to an e-mail "bingo" number request? That's possible through Mosaic, especially for those firms that have already created VNRs (video news releases) or B-rolls (video clips) for the new Silicon Valley-based CommerceNet. The Internet is a whole new, wonderful world for advertisers and agencies. Are you listening General Magic, Microsoft, Apple, IBM, Intel, Motorola, Compaq, AST, ALR, Dell, DEC, Silicon Graphics, Sun, HP, Sharp, Canon, NEC, Sony, Novell, Lotus, General Instrument, BellSouth (as "Simon" says), Scientific Atlanta, ... ?
Information to keep handy: HOTT will be moved from its current host site at UCSD beginning with the first issue with paying sponsorships (i.e., probably the third issue in July). I'm talking with Novalink, Downtown Anywhere, and Netcom. And I plan to talk with the IEEE Computer Society sometime in late May. Internet wizard Daniel Dern has recommended that I place a call for redistribution points. I need to further explore this option. But just in case you don't hear from this list after two or so months, contact me at d.s.lewis@ieee.org or callewis@netcom.com .
As a necessary disclaimer, please note that there are no official ties
between the UCSD administration and HOTT. The list management is NOT an
official function of the UC system. Final comment on HOTT: For the time
being, we're operating as a de facto non-profit organization. (Actually,
High On Technology
For a FREE subscription, send your e-mail request to: listserv@ucsd.edu (The "Subject" line is ignored.) In the body of message type: SUBSCRIBE HOTT-LIST (do NOT include first or last names). Note that we'll likely use a more flexible Majordomo list manager when we move the list from UCSD.
Many HOTT subscribers have asked me about EMR. EMR is positioned as the Harvard Business Review for "high tech" managers and executives. It's primarily (but not exclusively) a reprint journal. Perhaps the best way to describe it is as a cross between HBR and Reader's Digest -- but for "high tech" execs and execs-in-training. EMR is a 100 page quarterly. I view EMR as the management counterpart to HOTT. For information on Engineering Management Society membership and EMR, contact Bill Burke at em.mem@ieee.org , the U.S. member hot line at +1.800.742.0432, tel: +1.518.382.5512, or fax: +1.518.382.5531. For e-mail requests, please copy m.mckoy@ieee.org , em.pres@ieee.org , and em.pub@ieee.org .
I'd also like to recommend membership in the IEEE Computer and Communica- tions Societies. For Computer Society membership and general information call +1.714.821.8380, or e-mail membership@compmail.com . If you're interested in IEEE Computer Society publications (and you should be!), call +1.714.821.4641, or e-mail cs.books@compmail.com .
[2] HOTTpixels by Susan Jurist
HOTTstuff: for those of you new to HOTT, and that is most of you, first a bit of history. HOTT was started as an in-house e-mail publication for the UCSD libraries in 9/90. The sponsoring group was the Technology Watch Information Group (TWIG) and the publication was called Hot Off The Tree. ("tree" "twig" get it?)
HOTT, a weekly, then became available on MELVYL, the University of California online catalog. After people started seeing it there & started requesting subscriptions, a listserv for HOTT was started. It continued as an almost-weekly until last October when my pixels were fried, and I stopped doing it. David asked to take it over, and the rest, as they say is HOTTstory.
My column in HOTT will focus on the pixel end of the story -- that is I am most interested in what the end-user sees and notices and primarily, the end-users I'm interested in are those on Mac's.
ROLL 'EM, ROLL 'EM, ROLL 'EM: well the news for the last 2 weeks has centered around the unveiling of the PowerPCs, or PowerMacs as Apple is calling their 3 new models. Short and sweet: I want one.
Why is a bit more complicated. For those like me who do a lot of graphics work -- especially with programs like Adobe Photoshop or Fractal Arts Painter, a PPC is the answer to an impatient person's dream. Running those programs in emulated mode -- that is running existing Mac software -- is somewhat faster -- how faster depends on exactly which Mac you are now on. But using the programs in "native mode" -- that is programs directly programmed to take advantage of the PPCfeatures can be as much as 10 times faster for some operations.
Those of you who primarily use word processors may not be all that impressed with those figures. But just last week, I was able to fill out the 1040 form in the time it took my IIsi to open a 1.14 meg Adobe Illustrator file in Photoshop. Luckily I only have to fill out taxes once a year & even then, that wait was pretty tedious.
But there's the rub. Not only do you have to spend $many-K to buy a new machine, you then have to spend $many-more-K more to buy software that will make it faster than a speeding bullet. And how much you have to spend will depend on the software publisher. It will vary. According to various articles in the 3.14.94 MacWeek, upgrades will be from free, or nominal charges like $15 for Painter to as much as $195 for Quark. In addition, some publishers, like Quark are only going to make the upgrades available for a short period of time. After that, you're back to full price. So you might have to guess ahead that you will be buying a PPC & buy the software before the machine. Yuck. (again, for those of you new to HOTT, I am not above editorializing)
But there's still lot's of good news. Except for the really professional graphic artists who will want the 801, the rest of us might be quite happy with the 601 or 701 models (the 701 has more expansion). They seem to be compatible with the majority of the currently available Mac software and are reasonably priced for the power you get.
In the formal Apple roll out last Monday, they showed even 601s going up against high-end Pentium machines. Of course the PPCs were faster, or they wouldn't have done that demo. Now this goes for spread-sheets (or spread-shits as the French-born Apple employee kept calling them) as well as graphics applications.
HOTT wants to know though, are software publishers going to stop upgrading Mac software & just come out with PPC versions? This is not clear from anything I've read.
SPEAKING OF SOFTWARE: my second favorite software in the world , Macromedia's Director, has a major new upgrade. Some of the new features include increasing the number of channels to 48 (previously a limiting 21) and increasing the maximum number of cast members from 512 to 32,000. Also, for those of us who find ourselves in multi-platform worlds (and who isn't?), Director 4.0 documents and the (not yet released) Director for Windows can swap documents between them for editing. Upgrades from previous versions will be $199 through 8/31/94.
More on this if I can ever convince Macromedia to give me a review copy. --> information from a Macromedia press release.
A PICTURE MAY BE WORTH 1000 WORDS: but the merger between two major graphic software producers was worth many, many millions. Adobe & Aldus announced a merger which has some people worried, even though the companies swear their merger doesn't violate any federal anti-trust laws.
From my view, it looks like Adobe & Aldus only go head-to-head with Illustrator vs. FreeHand. Sure Aldus has some products that lets you manipulate photos, but I sure don't know anyone who uses it instead of Photoshop.
I'm a big fan of products from both companies and only hope the merger makes using products together easier. Instead of a bunch of separate applications, I wish they'd work more on document-based computing so I could do my work without trying to have Photoshop, SuperPaint, Illustrator & PageMaker all open at the same time. Creating an umbrella program that could let a user add the needed functionality without having to change programs is my dream for the future.
Adobe seems to be going in that direction when they create programs like Dimensions, a 3D program which has you do your drawing in Illustrator & your finishing up in Illustrator or Photoshop. It's the only 3D program I've ever understood simply because all you have to learn are the 3D pieces.
At this writing, they have yet to announce a new name. But what's in a name anyway?
TOYS R US: I indulged in (that means paid my own money) two new Mac toys in the last month. The first was the "Little Mouse ADB" -- an optical mouse for the Mac. Point 1 -- the new Apple mouse, which is billed as "ergonomic" may in fact be so if you've got larger hands than I do. But as one of the people who really loved the old mouse, when my old one died, it was hard to find a new one small enough. Point 2 -- those optical mouses are really neat. They are much smoother than a traditional mouse any I find I have much more control over it in tight drawing situations. So far the only downside I could find was the optical mouse pad was a bit small for playing Crystal Crazy. I'm losing lots of points in the bonus round because of this. Oh well...
The second toy was a Microtek IIsp scanner. If you've been wanting a scanner and have always thought the prices were too too much, look again -- the prices have really come down. This new scanner is fast (even on my IIsi) and more important, accurate. Out of the box, no special software but the Photoshop plug-in, it gave me the best color I've seen from a low-end scanner. Except when I used Microtek's extra special DCR color-matching software. Then everything went west. Why would they include special software that makes everything worse? (removing it solved all the problems)
WHO'S REALITY IS THIS?: speaking of software, when are the software publishers going to get real? I recently acquired some expensive ($600) software for work (which shall remain nameless) which is specifically licensed for one machine. That of course means, if I need to do work at home, I've got to come up with another $600 for a second copy. And this is the "educational" price -- a good $500 less than list. Come on guys. As far as I know, WordPerfect is the only major publisher that licenses software per user -- you can take your software and use it on your machines -- you just can't give it to anyone else. Now this is fair and doable. The software publishers are always (rightfully) complaining about pirated software, but they also make it very hard for the normal person to be honest.
REVIEWS: A review copy of Pixel Paint Pro 3 came in the mail today -- a full review in the next issue of HOTT.
[3] Virtual warriors Frank Oliveri Air Force Magazine, Jan 94
Some VR enthusiasts believe that VR will be to the '90s what PCs were to the '80s. Synthetic environments are the core of any VR system, and the more elaborate ones provide natural surroundings outside. The best- selling VR devices today are in the entertainment field, but the Air Force has been experimenting with VR for years in a major program called the Visually Coupled Airborne Systems Simulator (VCASS). Other efforts were added as part of the Super Cockpit program. But scientists such as Dr. Robert Eggleston (Armstrong Aerospace Medical Research Laboratory, Wright-Patterson AFB, Ohio) say it will take thirty years or so for the military to achieve a full VR cockpit.
An all-aspect, opaque helmet system might allow a pilot to be placed in a horizontal position. This would protect from blinding lasers, and enable pilots to pull more Gs and designers to build more maneuverable aircraft.
And engineers and scientists at Armstrong Laboratory have created a prototype cockpit that makes it possible for pilots to actuate flight controls with brain waves alone.
Reference: Applying virtual reality technology to cockpits of future fighter aircraft by Michael Haas (Human Engineering Division) and Lawrence Hettinger (Senior Human Factors Engineer, Logicon Technical Services, Inc.)
Contact: Fusion Interfaces for Tactical Environments (FITE) lab, Armstrong Laboratory
[4] Ready for a wild ride? (Digital adventures in entertainment, Part 2 of 2) Barbara Robertson Computer Graphics World, Feb 94
The LBE (Location-Based Entertainment) industry is excited about VR applications. Why? First, VR experiences are reprogrammable. Second, the experiences can be labeled with the so-called magic marketing words: "virtual reality." Third, the market is expanding to include not only LBEs being built by Sega, but LBE experiences appropriate for sports bars, zoos, and museums. Fourth, there are already a handful of successes.
4th Wave Inc. (Alexandria, VA) estimates that purchases of VR equipment by operators of LBE centers will grow from $33 million in '93 to $64.5 million in '96, and public spending to play with this equipment will increase from $18.8 million in '93 to $91.3 million in '96.
[5] Virtual reality shapes surgeons' skills Linda Carroll Medical World News, Feb 94
Researchers at universities and start-up companies are developing surgical simulators that mimic the look and feel of an operation. One such simulator by High Techsplanation (Baltimore) was developed for prostate surgery. It's so real that if the physician makes a mistake and cuts a blood vessel, s/he will see it bleed. (High Techsplanation has received funding from the National Institutes of Health and Merck & Co.)
To further the development of VR in medicine, the National Library of Medicine (a part of NIH) will soon be releasing a computerized descrip- tion of the human anatomy. This atlas will be available via the Internet, but beware before you ftp at 14.4: It will take up 40 gigabytes!
[6] Virtual reality: Immerse yourself Carrie R. Smith Wall Street & Technology, Dec 93
No more spreadsheets ... here comes VR! It's appealing to traders because it puts the same data into a 3D format; traders can physically "experience" the market at their workstations. The primary advantage is data recognition, and a time advantage in reacting to changing market conditions.
[7] VR becoming reality for everyone R. Colin Johnson Electronic Engineering Times, 10 Jan 94
VR technology has begun to proliferate down to the PC. For a designer on a shoestring, Shooting Star Technology (Burnaby, BC) offers the $1,500 ADL-1. The head-tracking device is connected to a robotic arm with six degrees of freedom. A microcontroller communicates head location and orientation through a serial port to the VR rendering PC. Shooting Star also supplies the MR Toolkit for constructing virtual realities; it's free to anyone as long as it isn't used for profit-making activities. The MR Toolkit provides a set of subroutine libraries, device drivers, and a language for describing the appearance and behaviors of a VR environment.
The lowest-cost solution for 3D VR displays (but without full-immersion) is the Cyberscope from Simsalabim Systems, Inc. (Berkeley, CA). The Cyberscope is a 3D hood that attaches directly to a standard PC monitor. The cost: $179 per seat.
[8] Virtual reality moves into design Andrea Baker Design News, 7 Feb 94
Market projections by Find/SVP (New York, NY): Virtual reality product sales by application Largest market segment in '95: Information @ $35 million Largest market segment in '99: Tie Training & simulation and Entertainment, each @ $150 million Total projected sales in '95: $85 million Total projected sales in '99: $575 million
Contact: Chris Codella, Manager, Virtual Worlds Department, IBM Thomas J. Watson Research Center (Yorktown Heights, NY)
[9] Software valets that will do your bidding in cyberspace Evan I. Schwartz The New York Times, 9 Jan 94
Larger and well-financed companies get the ink, but smaller companies have been selling agent software for the past 18 months or so, agent software that schedules meetings, responds automatically to incoming e-mail, ... and even optimizes computer network configuration.
Hoover, from Sandpoint Corporation (Cambridge, MA), is a PC-compatible program for information gathering. Hoover's search results are compiled into a customized electronic newsletter, with headlines that can be clicked on with a mouse to retrieve full-text articles. Microsoft's Office suite includes Intellisense for realtime spelling error correction. And Apple's equivalent to Hoover is Applesearch.
The term "intelligent agent" was coined in the '60s by then MIT Lincoln Laboratory computer scientist Oliver Selfridge. Today, MIT's Media Lab is pursuing Selfridge's vision. Other software packages include Beyondmail from Beyond, Inc. and Open Sesame! from Charles River Analytics (Cambridge, MA). Beyondmail automates responses to incoming e-mail. Open Sesame! monitors repetitive PC activity ... and then, in essence, automatically creates intelligent, autonomous macros.
[10] Robo-software reports for duty John W. Verity with Richard Brandt Business Week, 14 Feb 94
At Carnegie Mellon University (CMU), Stanford, and software think-tank Interval Research, researchers are attempting to create so-called "believable agents" that embody human character and present themselves as animated cartoon faces. The agents react to stimuli with convincing human expressions.
[11] The metaphor is the message Barbara Kantrowitz Newsweek, 14 Feb 94
Microsoft is working on its own version of a third generation interface. Its design project is code named "Utopia."
Contact: Virginia Howlett, Director of Visual Interface Design, Microsoft
[12] The butlers of the digital age will be just a keystroke away Barbara Kantrowitz Newsweek, 17 Jan 94
About a dozen companies, including America Online, AT&T, and OAG (Official Airline Guide), expect to have agent-based products by the end of the year. Matt Kursh, president of eShop, Inc., says his company is designing electronic stores that consumers -- or their agents -- will enter via PCs or PDAs.
[13] Software 'agents' will make life easy Andrew Kupfer Fortune, 24 Jan 94
When AT&T introduces Telecript e-mail this year, users will be able to type in the addressee's phone number; an agent will then look up the e-mail address corresponding with that number and deliver the message to the addressee's computer. General Magic plans to license Telescript freely to companies throughout the computer and telecommunications industries.
[14] Just like Magic? Tom R. Halfhill and Andy Reinhardt Byte, Feb 94
After four years in development, General Magic (Mountain View, CA) is bring several technologies to market. The technologies include: Telescript, a communications-oriented programming language; Magic Cap, an OOPS designed for PDAs; and a new, third generation GUI. Motorola's Envoy, due for release in 3Q '94, will use Magic Cap as its OS.
What PostScript did for cross-platform, device-independent documents, Telescript aims to do for cross-platform, network-independent messaging. Telescript protects programmers from many of the complexities of network protocols.
Competitors for Magic Cap include Windows for Pens/Winpad (Microsoft), PenPoint, Newton Intelligence (Apple), and GEOS (GeoWorks). The competition for Telescript is more fragmented.
[15] Agent technology stirs hope of magical future Michael Fitzgerald Computerworld, 31 Jan 94
Analysts believe that agent technology will present a powerful example for remote computing, especially in a client/server environment. In a client/server environment where the client leaves, the message has to be smarter because the client isn't there to guide it.
Contacts: Ken Dulaney, Gartner Group (Stamford, CT); Kimball Brown, Dataquest, Inc. (San Jose, CA); Bruce Stephen, International Data Corp. (Framingham, MA)
[16] Telescript eases cross-network communication Yvonne L. Lee Infoworld, 17 Jan 94
Contact: Thomas Cantrell, West Coast senior editor, Computer Applications Journal newsletter
[17] Computer: Take a memo Wendy Pickering Datamation, 7 Jan 94
IBM has released the under-$1,000 Personal Dictation System (PDS), which it claims has a 95 to 98% accuracy rate. Its easy to operate: Simply speak into a microphone and the software converts speech to text on screen. Its speced at 70 WPM (words per minute), but users can push it to 110 WPM. It runs on most 486 platforms and Pentium PCs. Users train the system by reading "A Ghost Story" by Mark Twain for about 90 minutes. After it's been trained, the system can distinguish between homophones, such as "to" and "two." The PDS has a vocabulary of 32,000 words and up to 2,000 words can be added. And macros can be assigned. Don't worry about accents: The PDS doesn't care, as long as the user consistently mispronounces a given word or phrase.
[18] Number please: Speech recognition over the telephone Judith Markowitz PC AI, Apr 94
Automatic speech recognition (ASR) can provide rotary telephone owners the same services that touch-tone phone owners enjoy. Surprisingly, most industrial countries still rely overwhelmingly on rotary telephone, and this situation is not likely to change in the near future.
Ameritech automates operator services for collect calls and calls billed to a third party. AT&T is scheduled to complete conversion of all "0+" (collect and third party billing) long distance calls to the system. NYNEX added ASR to its call intercept network in 1990. (Intercepts are calls that cannot be completed due to changed numbers, equipment failures, and a variety of other problems.)
Subscriber services include household dialing directories; recognition is linked to the voices of household members. In February '93 AT&T became the first major long distance carrier to offer ASR on its 800 network. But it's the Sprint FONCARD that has received the most media attention. It was first built as a security device, but it can also be used for speed dialing.
[19] Talking to computers: Time for a new perspective Column by Nicholas Negroponte (MIT's Media Lab) Wired, Feb 94
In contrast to the microphone-based IBM PDS described earlier, Nicholas Negroponte believes that computers should be in "earshot" -- but this requires the segregation of speech from surrounding sounds. Also, ideal systems should be able to understand sarcasm, passion, exasperation, and other emotions. Negroponte doesn't believe that speaker independence is really important, and that vocabulary size is manageable by sub-setting vocabularies, such that specific vocabularies are culled at any given time.
Contact: Nicholas Negroponte, MIT Media Lab (nicholas@media.mit.edu)
[20] There's no reason to subsidize access to information highway Michael Schrage San Jose Mercury News, 10 Jan 94
(Editor's Note: If I'm going to do justice to Michael's column, I have to use his own words. He's a terrific writer -- readers of his column syndicated by The Los Angeles Times are fortunate to have access to his words of wisdom. The excerpts below flow in the same order as presented in his original column.)
Excerpts:
"Of all the misconceptions surrounding multimedia innovation and digital superhighway metaphors, none is more misguided or misleading than the belief that access to new telecommunications technologies is somehow central to determining wealth and poverty in the Information Age."
"It's bizarre that certain pundits and prognosticators want to focus on high-tech network access subsidies for the masses barely three months after the Department of Education published a survey claiming to show that fully half of American adults are close to functionally illiterate. What does network access mean to them?"
"Just what do we want to subsidize here, and why? Is it really, as the White House official said, information access? If that were true, it would be far more logical and cost-effective to give people subsidies for newspaper and magazine purchases and keep the public libraries open longer."
"Is access to CNN or C-SPAN an essential public service, like dialing 911?"
[21] Information Highway will create jobs James Coates (Chicago Tribune) Washington Post, 17 Jan 94
Needed:
EEs Software programmers* "Information technicians" (people who will use the worldwide networks to gather data)
Studies by the Hudson Institute and the U.S. Department of Labor estimate that by 2005 jobs for computer systems analysts will grow by 79%; for programmers by 56%; and for EEs by 34%. The Labor Department also estimates that 200,000 new telecommunications jobs will emerge by 2005.
[22] Many PC makers steer clear of Information Highway Kyle Pope The Wall Street Journal, 28 Feb 94
While IBM, Apple, Compaq, and Packard Bell have jumped onto the Info Superhighway bandwagon, the rest of pack, which accounts for about two-thirds of all computers sold, has shied away from Highway mania. "It's all hype," says Safi U. Qureshey, CEO of AST Research (Irvine, CA). At the core of his statement is the belief that the PC, and not the TV or a hybrid of the two, will be the preferred vehicle for cruising the Highway.
Activities:
IBM -- Already in set-top box trial with Bell Atlantic. Sees hybrid of PC and TV
Apple -- Also in set-top box trial, but with a Mac version
Compaq -- Negotiating 20 deals with telecom and software companies
Packard Bell -- Already offers PC TV. Interested in JV with a major online service
AST -- Convinced most of the megamergers will fail. Focused on core PC business
Dell -- Focusing on computer servers
Gateway 2000 -- Believes PC will be focused on text and TV on video
[23] Superhighway into the home Louise Kehoe Financial Times (London), 8 Mar 94
Interactive television (I-TV) will be the foundation of a $3.5 trillion digital consumer electronics industry by the turn of the century. Perhaps the most intriguing aspect of I-TV is the prospect of being able to design a personalized TV channel (similar to selecting topics within Clarinet).
According to analysts at Salomon Brothers, games and will edutainment will bring I-TV into the home. But HP consumer research suggests that educational apps for children may be the initial attrac- tion. Yet, it may be pornographic materials that may lead the charge, as it was with the video rentals business.
On the issue of advertising, 94% in a survey said that they wanted no solicitations on I-TV and 73% said no to advertising.
Obstacles:
1) Huge capital investments are required to build the infrastructure;
2) Neither cable TV or telephony companies have networks that are well suited to the task; and,
3) There are, as yet, no standards for I-TV.
Contact: Ms. Casey Lumus, Market Development Manager, Interactive Television, Hewlett-Packard
[24] No space in cyberspace Aaron Zitner Boston Globe, 6 Feb 94
Interesting tidbits: More than 12 million homes have computers with modems; in New England, 15% of residential customers who request additional phone lines are using them for computer modems. Jupiter Communications (a market research and newsletter publishing firm) projects that households with online access will increase from 4 million now to over 17 million by '98, and fees paid to online services will top $3 billion in '98, up from $550 million now.
(Editor's Note: Jupiter is the publisher of the excellent newsletter titled Consumer Information Appliance. They've also launched a new newsletter on online services. I'm going to invite them to place a sample issue of each newsletter on our ftp site. If they accept my offer, I'll place an availability notice in HOTT.)
[25] Information Superhighway: A refreshing approach, but some old questions James J. Mitchell San Jose Mercury News, 13 Jan 94
Survey results courtesy of MCI Telecommunications Corp. and Reuters:
The two Info Superhighway features with the most appeal are --
1) Interactive educational programs that let you take courses and participate in classes through TV, and
2) Information system with home access to library resources
[26] All segments of PCS industry expected to see robust growth Anonymous R C R: Radio Communications Report, 14 Feb 94
The Personal Communications Industry Association's "1994 PCS Market Demand Forecast" predicted that total subscriptions could reach 167 million by 2003, up from 88.3 million in '98 and 33.7 million in '93. PCS will consist of eight key elements: new 1.8-2.2 GHz PCS; cellular; paging; enhanced and standard Specialized Mobile Radio (SMR); dedicated data; satellite; wireless PBX; and, cordless telephone service.
Cellular and paging will continue to dominate the market. Low price points will keep paging/messaging services in demand. Satellite services, ESMR and dedicated data will grow by nearly 221%, 73%, and 68% respectively between '98 and '03, according to the PCIA market study. All three categories will be dominated by business users.
[27] Sweet success in wireless attracting investors to PCS Jeffrey Silva R C R: Radio Communications Report, 14 Feb 94
Congress has mandated that the FCC make room for small businesses, women, minorities and rural telephone companies in the upcoming PCS auction process.
[28] PCS: Integrated wireless telephone-computer opportunities Thomas K. Crowe Voice Processing, Jan 94
The FCC defines PCS as "a family of mobile and portable communications services for individuals and businesses that may be integrated with a variety of competing networks."
In its Broadband PCS Order, the FCC allocated 160 MHz in the 2 GHz range, commonly referred to as "broadband PCS." Of this 160 MHz, the FCC designated 120 MHz of spectrum for licensed broadband PCS operations, and the remaining 40 MHz for unlicensed devices.
In its Narrowband PCS Order, the FCC allocated 3 MHz in the 900 MHz range, commonly referred to as "narrowband PCS." The FCC has channelized the spectrum into 50 kHz or 12.5 kHz paired or unpaired channels. Narrowband PCS is particularly well suited for existing and enhanced paging and messaging services.
[29] PCS: Will it end cellular telephones? Harry Caul Popular Communications, Mar 94
Four times the spectrum originally allocated to cellular is being allocated to PCS. PCS standards allow licensed base stations to operate at powers up to 100 watts with an antenna height up to 300 meters. Licensees will be required to offer service to at least one-third of the population in each market area within five years of being licensed, two-thirds within seven years, and 90% within ten years.
Cellular markets are allowed to be served by only two companies. Major urban markets could have their two cellular services and, in addition, as many as seven competing PCS service suppliers. For cellphone companies, this represents competition for subscribers by others providing a newer technology that offers more services at less cost.
Bottom line: Cellular will have to make dramatic changes in its services and costs, otherwise cellular will end up as one more outdated technology like 35 MHz carphones, the 152 MHz IMTS phones that followed, and the 454 MHz carphones that arrived on the market a few years before the dawning of cellphones.
[30] PCS: Hands-on communications for all Randy Oster and Gary Brush Telephony, 28 Feb 94
Wireless communications continues to grow by 25% per year, as compared to 7% to 8% for long-distance revenues and 3% to 4% for local telephone service. Some analysts predict that total end user penetration could increase from today's 5% (for cellular) to 20% to 25% (for cellular and PCS) by the end of the decade.
Winners in the PCS auction will have to pay for the spectrum it won, compensate incumbent users for moving from their spectrum band (e.g., microwave companies), procure radio equipment, acquire sites for the radio equipment, and acquire customers. Many wireless providers may seek network partners since 80% of the cost of delivering PCS will be for the network component.
With PCS, messages are sent to personal identifiers rather than to physical locations. Market research that polled trial participants concludes that users want and need mobility, a single personal phone number and an automatic follow-me capability. Trial results indicate that there is potential for a 1,000 minutes-per-month mass market service. Bell Atlantic will soon be testing an extension of the concept that will allow users to register at both wireless and wireline phones, perhaps using smart card and intelligent badge technologies.
Sources: PCS Network Access Services to PCS Providers + Technical Report on PCS Network Capabilities, Architectures, and Interfaces for PCS both are available from Bellcore Customer Service, 8 Corporate Place, Room 3C-138, Piscataway NJ 08854-4156 USA Tel: +1.800.521.CORE (in the U.S.) check for report prices before placing an order
[31] Special Report: Satellite-based Personal Communications Services Rob Frieden Microwave Journal (Wireless Supplement), Jan 94 adapted from Telecommunications, Dec 93
Low and middle Earth orbiting satellite projects, including Motorola's IRIDIUM, Odyssey, Globalstar and Project 21, will make personal communication networks virtually ubiquitous. Each venture is comprised of a constellation of between 12 and 66 nongeostationary orbiting satellites. However, satellite-based, global PCS will not compete with cheaper terrestrial options, including cellular radio. The reason is cost, estimated at no less than $3.00 per minute.
Global PCS has international support through the International Telecommunications Union (ITU). At the '92 World Administrative Radio Conference (WARC), representatives endorsed the concept of "universal personal communications" with both terrestrial and satellite components. And the FCC has begun a program to reward innovators of licensed but unused spectrum. The new program encourages licensees to deploy spectrum-conserving technologies, like compression and circuit multipli- cation.
[32] Telcos may decide PDA fate Michael Fitzgerald Computerworld, 14 Feb 94
Captain Kirk's flip-open communicator -- and not Apple's Newton -- may represent the future of PDAs. Likewise, future PDAs may come from a telephone company, not a PC manufacturer. For example, the Bell Northern Research lab in Ottawa is developing a cellular phone with ASR, faxing, e-mail and PIM (Personal Information Manager) functions. But a shake- out is likely to occur. According to Dataquest (San Jose, CA) analyst Kimball Brown, there may be as many as ten competing handheld platforms by year's end, two of which will survive over time.
Contacts: Kimball Brown, Dataquest, Inc. (San Jose, CA) Andrew M. Seybold, Editor, "Outlook on Mobile Computing"
[33] Going on-line when you're off the beaten path Bart Ziegler Business Week, 6 Dec 93
There will be new PCMCIA cards allowing wireless, untethered communications. Partly as a result, the number of wireless-data users will grow from 1 million this year to 7.3 million in 1998, says the Yankee Group, a New England-based market research firm.
Another hot topic is CDPD, which functions by sending millisecond bursts of data during normal pauses in conversation on cellular networks. According to Motorola's Bob Growney, GM of their Paging & Wireless Data Group, the CDPD equipment will fit onto a card by the end of the year.
However, not all is rosy. The forthcoming PCMCIA cards that house the entire radio won't work with many notebook computers due to electrical interference.
[34] Michael Finley column on pen-based computing St. Paul Pioneer Press via INDIVIDUAL 13 Mar 94
Michael recommends a new pen system from Communication Intelligence called Handwriter for Windows. This system allows any desktop or laptop PC to become a pen-based system. The components: A wafer-thin writing pad (tablet?), a plastic-tipped writing stylus, and a few disks.
Drawback: You have to print very plainly; handwriting is out.
For information contact: Communication Intelligence +1.800.888.9242
[35] How Mac changed the world Philip Elmer-Dewitt Time, 31 Jan 94
Although Apple didn't invent the mouse or windows, Apple was the first to pioneer their commercialization. Now, the essence of the Mac is being felt in three key areas:
1) Internet -- Mosaic is a Mac-like guide.
2) Pocket computers -- Mac metaphors are being used to design new generation devices.
3) Interactive TV -- The Mac metaphor of the desktop has been adopted by General Magic with their metaphor of a street, by Time Warner with their metaphor of an electronic shopping mall, and by GTE with their metaphor of Main Street. Even Apple's new online service, eWorld, adopts the metaphor of a village.
Steven Levy chronicles the meaning of the Mac in his new book, Insanely Great (Viking, $20.95).
[36] Microsoft hits the gas Richard Brandt with Julia Flynn and Amy Cortese Business Week, 21 Mar 94
Microsoft's 500-person Advanced Technology Group (ATG) has a $100 million budget. Its Microsoft Research unitconducts long-range research into intelligent and human-computer interface technologies. The Advanced Consumer Technology unit explores new markets, including I-TV, handheld computers, and set-top boxes.
Key products and projects include Microsoft At Work, their software blueprint for the office of the future; WinPad, Microsoft's answer to Magic Cap; and, set-top boxes in a JV with General Instrument and Intel.
[37] Neural network enhances ICU patient monitoring F.G.B. Dodd and N.A. Dodd Medical Electronics, Dec 93
Instruments in an intensive care unit (ICU) generally operate indepen- dently of each other. They generally act as limit alarms by calling attention to outliers. Little use is made of intelligent software technologies to combine readings and build a more sophisticated internal representation of the patient's true state.
Neural networks are one way to train a system by iteratively improving the weights and biases by applying I/O pairs to the system. A Neural Network Alarm Monitor developed by the authors does not attempt to indicate what is wrong with a patient; it simply signals a change to a state that has not been encountered in training data. But the neural network looks at systemic effects, rather than single parameter readings.
The article has an excellent "deck" with additional background information on artificial neural networks.
---
Note: The following full-text article is reprinted with the permission of the Association for Computing Machinery. It should be noted that the reprint is not made or distributed for direct commercial advantage; to copy otherwise, or to republish, requires a fee and/or specific permission.
[X] FULL-TEXT
The basic ideas in neural networks Rumelhart, David E; Widrow, Bernard; Lehr, Michael A Communications of the ACM v37n3 PP: 86-92 Mar 1994 ISSN 0001-0782 7 Pages
ABSTRACT: In the study of neural networks, the strategy has been to develop simplified mathematical models of brain-like systems and then to study these models to understand how various computational problems can be solved by such devices. Although the details of the proposals vary, the most common models take the neuron as the basic processing unit. Each such processing unit is characterized by an activity level, an output value, a set of input connections, a bias value, and a set of output connections. Each of these aspects of the unit is represented mathematically by real numbers. Thus, each connection has an associated weight which determines the effect of the incoming input on the activation level of the unit. A brain-style computational device consists of a large network of such units, richly connected to one another. Such a network is a general computing device. The function it computes is determined by the pattern of connections. Thus, the configuration of connections is the analog of a program. The goal is to understand the kinds of algorithms that are naturally implemented by such networks.
TEXT: The strategy has been to develop simplified mathematical models of brain-like systems and then to study these models to understand how various computational problems can be solved by such devices. The work has attracted scientists from a number of disciplines: neuroscientists who are interested in making models of the neural circuitry found in specific areas of the brains of various animals; physicists who see analogies between the dynamical behavior of brain-like systems and the kinds of nonlinear dynamical systems familiar in physics; computer engineers who are interested in fabricating brain-like computers; workers in artificial intelligence (AI) who are interested in building machines with the intelligence of biological organisms; engineers interested in solving practical problems; psychologists who are interested in the mechanisms of human information processing; mathematicians who are interested in the mathematics of such neural network systems; philosophers who are interested in how such systems change our view of the nature of mind and its relationship to brain; and many others. The wealth of talent and the breadth of interest have made the area a magnet for bright young students. Although the details of the proposals vary, the most common models take the neuron as the basic processing unit. Each such processing unit is characterized by an activity level (representing the state of polarization of a neuron), an output value (representing the firing rate of the neuron), a set of input connections, (representing synapses on the cell and its dendrite), a bias value (representing an internal resting level of the neuron), and a set of output connections (representing a neuron's axonal projections). Each of these aspects of the unit are represented mathematically by real numbers. Thus, each connection has an associated weight (synaptic strength) which determines the effect of the incoming input on the activation level of the unit. The weights may be positive (excitatory) or negative (inhibitory). Frequently, the input lines are assumed to sum linearly yielding an activation value for unit i at time t, given by
(Equation omitted)
where w sub ij is the strength of the connection from unit sub j to unit sub i , beta sub i is the unit's bias value; and x sub j is the output is value of unit sub j .
Note that the effect of a particular a unit's output on the activity of another unit is jointly determined by its a output level and the strength (and sign) of its connection to that unit. If ) the sign is negative, it lowers the activation; if the sign is positive it raises the activation. The magnitude of the output and the strength of the connection determine the amount of the effect. The output of such a unit is normally a nonlinear function of its activation value. A typical choice of such a function is the sigmoid. The logistic,
(Equation omitted)
illustrated in Figure 1, will be employed in the examples illustrated later. The parameter of the logistic T, yields functions of differing slopes. (Figure 1 omitted) As approaches zero the logistic becomes a simple logical threshold function which takes on the value of 1 if the activity level is positive and zero otherwise.
A brain-style computational device consists of a large network of such units, richly connected to one another. In real brains there are tens of billions of such units and tens of trillions of such connections. Such a network is a general computing device. The function it computes is determined by the pattern of connections. Thus, the configuration of connections is the analog of a program. The goal is to understand the kinds of algorithms that are naturally implemented by such networks.
Although there has been a good deal of activity recently, the study of brain-style computation has its roots over 50 years ago in the work of McCullock and Pitts [7] and slightly later in Hebb's famous Organization of Behavior [4]. The early work in artificial intelligence was torn between those who believed that intelligent systems could best be built on computers modeled after brains [9, 13, 17], and those like Minsky and Papert [7] who believed that intelligence was fundamentally symbol processing of the kind readily modeled on the von Neumann computer. For a variety of reasons, the symbol-processing approach became the dominant theme in AI. The reasons for this were both positive and negative. On the one hand, the stored-program digital computer became the standard of the computer industry. Such computers were easy to design and easy to program. The symbol-processing/logic-based approach to AI is well suited for such an architecture. On the other hand, the fundamentally parallel neural network systems, such as Rosenblatt's perceptron system, were not well suited to implementation on serial computers. Moreover, the perceptron turned out to be rather more limited than first expected [8], and this discouraged both scientists and funding agencies. Although work continued throughout the 1970s by a number of workers including Amari, Anderson, Arbib, Fukushima, Grossberg, Kohonen, Widrow, and others, and although a number of important results were obtained during this period, the work received relatively little attention.
The 1980s showed a rebirth in interest. There seem to be at least five reasons for this. Three of the reasons are essentially pragmatic and two theoretical. First, on the more pragmatic side:
1. Today's computers are much faster than those of the 1950s and 1960s. It is thus possible to use conventional computers to simulate and experiment with much larger and more interesting networks than ever before.
2. Everyone believes that the future for faster computers must be in parallel computation. Unfortunately, there is no generally accepted paradigm for parallel computation. It is generally easier to build parallel computers than to find algorithms that are efficient for them. There is a hope that algorithms which prove efficient and effective on brain-style computers may prove a useful general paradigm for parallel computation.
3. The basic empirical tools of neuroscience are expanding, and we are learning more and more about how the neuron functions and how neurons communicate with one another. But little is known about how to go from this information about specific neurons to a theoretical account of how large networks of such neurons might function. It is hoped that the theoretical tools developed in the study of neural network computational systems will allow for the modeling of real neural networks.
In addition to the preceding three reasons, there have been two theoretical results which have been developed well enough to be appreciated.
1. The first of these results is due to Hopfield [6] and provides the mathematical foundation for understanding the dynamics of an important class of networks. In particular, Hopfield pointed out that recurrent networks with symmetric weights have a point-attractor dynamics, making their behavior relatively simple to understand and analyze. This observation has been extended and applied by Hinton and Sejnowski [5], Cohen and Grossberg [1], Smolensky [14], and a number of others to provide us with a useful mathematical understanding of how networks such as these might be configured to solve important optimization problems.
2. The second result is an extension of the work of Rosenblatt and Widrow and Hoff, to deal with learning in complex, multilayer networks and thereby provide an answer to one of the most severe criticisms of the original perceptron work. In this case, it was observed that by selecting differentiable, nonlinear functions (such as the sigmoid described earlier) it was possible to use the gradient search methods of Widrow and Hoff for nonlinear and multilayer networks. This provided a technique by which multilayer perceptron-like devices could be reliably trained. This procedure, known as the backpropagation learning algorithm, has had a major impact on the field and is the primary method employed in most of the applications we will discuss [11, 16]
Here we focus on the learning results, since they have had the greatest influence on applications.
LEARNING BY EXAMPLE
The problem of learning in neural networks is simply the problem of finding a set of connection strengths which allow the network to carry out the desired computation. In this section we focus on backpropagation, currently the most popular form of learning system and the one on which virtually all of the applications are based. The usual network architecture is illustrated in Figure 2. (Figure 2 omitted)
There is a set of input units which are connected, through a set of so-called hidden units, to a set of output units. In the general case, there may be any number and configuration of hidden units and connections among the units. Generally, the hidden units are configured as a set of hidden-unit layers--most often there is a single layer of hidden units, but in some applications it is convenient to have two or more layers of hidden units. (For simplicity, we will restrict discussion here to the case of feedforward networks in which the activity of a given unit cannot influence, even indirectly, its own inputs.) The network is provided with a set of example input/output pairs (a training set) and is to modify its connections in order to approximate the function from which the input/output pairs have been drawn. The networks are then tested for ability to generalize.
The error correction learning procedure is simple enough in conception. The procedure is as follows: During training an input is put into the network and flows through the network generating a set of values on the output units. Then, the actual output is compared with the desired target, and a match is computed. If the output and target match, no change is made to the net. However, if the output differs from the target a change must be made to some of the connections. The problem is to determine which connections in the entire network were at fault for the error--this is called the credit assignment (or perhaps better, the blame assignment) problem. Although the solution to this problem for the case of networks without hidden layers has been known for some time, this is, in general, a difficult problem, and the lack of a satisfactory solution was a major factor in the earlier loss of interest in neural network systems. The 1980s has led to the development of a rather simple, yet powerful, solution to this problem. The basic idea is to define a measure of the overall performance of the system and then to find a way to optimize that performance. In this case, we can define the performance of the system as
(Equation omitted)
where i indexes the output units; p indexes the I/O pairs to be learned; t sub ip indicates the target for a particular output unit on a particular pattern; y sub ip indicates the actual output for that unit on that pattern; and E is the total error of the system. The goal, then, is to minimize this function. It turns out, if the output functions are differentiable, that this problem has a simple solution--namely, we can assign a particular unit blame in proportion to the degree to which changes in that unit's activity lead to changes in the error. That is, we change the weights of the system in proportion to the derivative of the error with respect to the weights. The change in w sub ij is thus proportional to
(Equation omitted)
This simple procedure works remarkably well on a wide variety of problems. The problem of learning is thus reduced to the problem of parameter estimation.
A key advantage of neural network systems is that these simple, yet powerful learning procedures can be defined, allowing the systems to adapt to their environments. Work on the learning aspect of these neurally inspired models is what first led to an interest in them [93, and it was the conjecture that learning procedures for complex networks could never be developed that contributed to the loss of interest [8]. Although the perceptron convergence procedure and its variants had been around for some time, these learning procedures were limited to simple one-layer networks involving only input and output units. There were no hidden units in these cases and no internal representation. The coding provided by the external world had to suffice. Nevertheless, these networks have proved useful in a wide variety of applications (see [18]). Perhaps the essential character of such networks is that they map similar input patterns to similar output patterns. This characteristic is what allows these networks to make reasonable generalizations and perform reasonably on patterns that have never before been presented. The similarity of patterns in a connectionist system is determined by their overlap. The overlap in such networks is determined outside the learning system itself whatever produces the patterns.
The constraint that similar input patterns lead to similar outputs can lead to an inability ofthe system to learn certain mappings from input to output. Whenever the representation provided by the outside world is such that the similarity structure of the input and output patterns is very different, a network without internal representations (i.e., a network without hidden units) will be unable to perform the necessary mappings.
In a multilayer network, the information coming to the input units is recoded into an internal representation, and the outputs are generated by the internal representation rather than by the original pattern. If we have enough connections from the input units to a large enough set of hidden units, we can always find a representation that will perform any mapping from input to output through these hidden units.
The existence of multilayer networks illustrates the potential power of hidden units and internal representations. The problem, as noted by Minsky and Papert [8], is that whereas there is a very simple guaranteed learning rule for all problems that can be solved without hidden units, namely, the perceptron convergence procedure (or the variation due originally to Widrow and Hoff [17]), there has been no equally powerful rule for learning in multilayer networks. We are thus not assured of optimal solutions--local minima are always a possibility. Nevertheless, the a backpropagation procedure is sufficiently robust that local minima rarely turn out to be serious limitations.
Although the learning results do not guarantee that we can find a solution for all solvable problems, our analyses and simulation results have shown that as a practical matter, the backward-error propagation scheme leads to solutions in virtually every case.
GENERALIZATION
The backpropagation learning procedure sketched earlier has become, perhaps, the single most popular method to train networks. The procedure has been used to train networks in problem domains including character recognition, speech recognition, sonar detection, mapping from spelling to sound, motor control, analysis of molecular structure, diagnosis of eye diseases, prediction of chaotic functions, playing backgammon, the parsing of simple sentences, and many more areas of application (see [18]). Perhaps the major point of these examples is the enormous range of problems to which the backpropagation learning procedure can usefully be applied. In spite of the rather impressive breadth of topics, and the success of some of these reapplications, there are a number of serious open problems. The theoretical issues of primary concern fall into four main areas:
1. The learning problem--can the network learn how to solve the problem at hand
2. The architecture problem--are there useful architectures, beyond the standard three-layer network employed in most of these areas, which are appropriate for certain areas of application?
3. The scaling problem--how can we cut down on the substantial training time that seems to be involved for the more difficult and interesting problem application areas?
4. The generalization problem--how can we be certain that the network trained on a subset of the example set will generalize correctly to the entire set of exemplars?
The original efforts were focused on the first of these problems. The primary applications of our learning algorithms were to see if a network could learn some complex nonlinear function. Thus we focused on such problems as parity, exclusive--or, and other similar analytically defined problems. We found that with a sufficiently large network we could learn essentially any function. The initial worries about the role of local minima is and similar problems turned out to be much less serious than we originally thought. However, we have come to understand that the "generalization" problem is much more serious than we might have thought. This, of course, is just the mirror image of the learning problem. The more general our learning procedure, the less constraints we have on the way the network actually solves the problem and therefore the less certain we can be about the network's ability to properly generalize to new cases. In the statistics literature this is known as the "overfitting" problem. Models of many parameters can fit essentially any function in many different ways. Our problem is to fit the function in such a way that it maximizes its ability to generalize to an as yet unseen collection of data. There have been essentially two strategies in the connectionist literature to deal with this problem.
The first strategy is a version of "Occam's Razor"--i.e., the notion that the simplest hypothesis consistent with the data is the one that should be chosen. In the world of connectionist networks this involves the view that the simplest network consistent with the data should be chosen. There are a number of measures of simplicity in a network. We, for example, have suggested that the following variables covary with simplicity: number of weights, number of units, number of symmetries among the weights, number of bits per weight, and so forth. It is possible to define cost functions which lead to minimal-complexity networks as measured by any or all of these measurements. Generally, we find that minimal networks offer better generalization performance than more complex networks [15].
The second basic scheme for network training and, in fact, the most commonly used scheme is a version of cross-validation. In this scheme, the data are divided into three parts. One part is used for training; one part is used to evaluate the generalization performance and is set aside for a final test; and one part of the data is used for cross-validation.
The procedure is as follows: following each training epoch, the performance of the network is evaluated on the validation set. As long as the network continues to improve on the validation, set training is c continued. If over-fitting is occurring, the network will at some point begin to show poorer performance on the validation data. At that point we stop training and select the weights which give optimal performance on the validation set for testing against the "test set," and the performance on this set is used as a measure of the quality of the g generalization. This method is reasonably powerful and simple and often leads to good results. The results are nearly as good for this method as for the more complex method described earlier, and the training time is generally much less [2].
HINTS FOR SUCCESSFUL APPLICATIONS
Although some authors have suggested that neural networks are simple black boxes that can be applied without much consideration of the details of the problem, most successful applications require great care in approaching the problem at hand. Following are a number of considerations that have proved useful in some areas of application.
1. Be certain to have enough data to constrain your model sufficiently for the problem at hand.
2. Carefully design appropriate input data. This will often require theory-based data reduction of the number of input variables. This was important in the work of Rumelhart [10] on cursive handwriting. In this case a coupled oscillator model of handwriting was used followed by the parameters of this model, rather than the underlying temporal data. This allowed a five-fold reduction in the size of the input space.
3. Build known symmetries (often through weight linking) into your network wherever possible. This allows a substantial reduction in the number of weights in the network and allows the network to learn without having each region of the network see each input pattern.
4. Build a probabilistic model of the task. Make use of "forward models" to map from a representation of the input that you want to discover to a target set that is easy to construct. This method was also used in the cursive handwriting work. Here the problem was that we did not want to have to tell the network exactly where each character in the word was. Rather we wanted to simply tell the network which characters were in the word. The original network tried to predict the location of each character in the word, but we attached a second network that took the first network's guesses as to the location of the character and computed the probability that the character was "anywhere" in the word. This was a fixed network and was used to compute these probabilities. Thus while the targets could is be simple information about which characters were in the word, the network could determine where each character was. The details of this are given in Rumelhart [10].
5. Use the network to solve problems it is good at, but feel free to combine the network with other statistical methods. Making certain you can offer a clear probabilistic/Bayesian interpretation of the behavior will help in interfacing the network with other statistical methods. It is very useful to have the network provide output values which are reasonably interpreted as probabilities. These probabilities can then be used to determine confidence levels and to combine with other sources of evidence. See Curry and Rumelhart [2] for a useful example.
REFERENCES
1. Cohen, M.N. and Grossberg, S. Absolute stability of global pattern formation and parallel memory storage by competitive neural networks. IEEE Trans. Man Cybernet. 13 (1983).
2. Curry, B. and Rumelhart, D.E. MSNET: A neural network that classifies mass spectra. HPL Tech. Rep. 90-161. MPL.
3. Grossberg, S. Adaptive pattern classification and universal recoding: Part I: Parallel development and coding of neural feature detectors. Bio. Cybernet. 23 (1976), 121-134.
4. Hebb, D.O. The Organization of Behavior. Wiley, New York, 1949.
5. Hinton, G.E. and Sejnowski, T. Learning and relearning in Boltzmann Machines. In Parallel Distributed Processing: Exploration in the Microstructure of Cognition. Vol. 1: Foundations, D.E. Rumelhart, J.L. McClelland, and PDP Res. Grp. MIT Press/Bradford Books, Cambridge, Mass., 1986.
6. Hopfield, J.J. Neural networks and to physical systems with emergent collective computational abilities. In Proceedings of the National Academy of Sciences, USA. Vol. 79. National Academy of Sciences, Washington, D.C., 1982.
7. McCulloch, W.S. and Pitts, W. A logical calculus of the ideas imminent in nervous activity. Bull. Math. Biophys. 5, 115-133.
8. Minsky, M. and Papert, S. Perceptrons. MIT Press, Cambridge, Mass., 1969.
9. Rosenblatt, F. Principles of Neurodinamics. Spartan, New York, 1962.
10. Rumelhart, D.E. Theory in practice. A case study--Recognizing cursive handwriting. In Proceedings of the Third NEC Research Symposium. SIAM, 1993.
11. Rumelhart, D.E., Hinton, G.E., and Williams, R.J. Learning internal representations by error propagation. In Parallel Distributed Processing Explorations in the Microstructure of Cognition. Vol. 1 Foundations, D.E. Rumelhart, J.L. McClelland, and PDP Res. Grp. MIT Press/Bradford Books, Cambridge, Mass., 1986.
12. Rumelhart, D.E., McClelland, J.L., and the PDP Research Group. Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Vol. 1. Foundations. MIT Press/Bradford Books, Cambridge, Mass., 1986.
13. Selfridge, O.G. Pattern recognition in modern computers. In Proceedings of the Western Join Computer Conference. ACM, New York, 1955.
14. Smolensky, P. Information processing in dynamical systems: Foundations of harmony theory. In Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Vol. 1. Foundations, D.E. Rumelhart, J.L. McClelland, and the PDP Res. Grp. MIT Press/Bradford Books, Cambridge, Mass., 1986.
15. Weigend, A.S., Rumelhart, D.E., and Hubberman, B. Generalization by weight-elimination with applications to forecasting. In Advances in Neural Information Processing. Vol. 3, R.P. Lippman, J. Mody, and D.S. Touretsky. Morgan Kaufman, San Mateo, Calif., 1991, pp. 875-882.
16. Werbos, P. Beyond Regression. New Tools for Prediction and Analysis in the Behavioral Sciences. Harvard Univ., Cambridge, Mass.
17. Widrow, B. and Hoff, M.E. Adaptive switching circuits. In Institute ofRadio Engineers, Western Electronic Show and Convention, Convention Record. Part 4, Inst. Radio Eng., 1960, pp. 96-104.
18. Widrow, B., Rumelhart, D.E., and Lehr, M.A. Commun ACM 7, 4 (March 1994) (this issue).
ABOUT THE AUTHORS: DAVID E. RUMELHART is professor of psychology at Stanford University. His research has focused on how people learn complex skills such as reading, and how that knowledge is represented in the mind. He is coauthor of the well-known 2-volume set of connectionist texts, Parallel Distributed Processing.
BERNARD WIDROW is professor of electrical engineering at Stanford University. He does research and teaching in the fields of digital signal processing, adaptive signal processing, adaptive control systems, pattern recognition and neural networks. He is coinventor of the LMS algorithm and the neural element ADALINE and various MADALINE networks.
MICHAEL LEHR is a doctoral candidate in electrical engineering at Stanford University. His research involves the application of second- order training techniques to large neural networks.
Authors' Present Addresses: David Rumelhart can be reached at Stanford University Department of Psychology, Bldg. 420, Room 414, Stanford, CA 94305-2130. Bernard Widrow and Michael Lehr can be reached at Stanford University Department of Electrical Engineering, Durand Bldg., Stanford, CA 94305-4055.
Copyright Association for Computing Machinery 1994
---
[Z] ANNOUNCEMENT
Peter Rothman (avatarp@well.sf.ca.us), Managing Partner of Avatar Partners (a PC VR company) will be the presenter at the next session of the Caltech/MIT Enterprise Forum on Tuesday evening, 17 May, beginning at 7:00 p.m.
Peter's company has received several contracts under the Defense Department's Small Business Innovation Research program. Their key commercial product is a VR software package for trading stocks and commodities. HOTT's editor will be a panelist. Editors, journalists, and columnists are invited at no charge. The session is open to all others for about $20. For additional informaton contact Caltech's Industrial Relations Center; their area code in Pasadena is 818.
David Scott Lewis, Editor & Publisher HOTT electronic magazine HOT (High On Technology) Media Ventures President, Cellsys, Inc. (a wireless communications company) VOX: +1.818.786.0420 Fax: +1.818.994.5026 VoiceMail: +1.714.662.7037 E-Mail: d.s.lewis@ieee.org or callewis@netcom.com ```