A Self-Interview
by George Michael
I started at the Laboratory on the 16th April in 1953. Someone met me at the
gate and told me I would be sitting in the "Cooler" until my Q clearance came
through. It's been downhill since then.
I can honestly state that interviewing myself is boring. The purpose of this
recording is to talk about some of the earliest memories I have of things that
had to do with graphics. To do that, I will talk first about the UNIVAC. Now,
the UNIVAC arrived about a week after I got there. I got my clearance in one
month - never having been anywhere interesting, nor done anything even
moderately contentious - but it took six months or so to get the UNIVAC
cleared (to run). And while we were waiting, we practiced writing UNIVAC
programs and did a collection of manual calculations and plots to produce,
exactly, the numbers that we expected the UNIVAC to produce when it ran.
Some of the bigger design calculations, then as now, took between 20 and
40 hours to run. Of course, they were very simple compared to what's being
done now. Simple or otherwise, they produced reams and reams of output. It
took a long time to get at the output because it was printed, one character at
a time on typewriters. These typewriters were somewhat modified Remington
Rand typewriters with very wide carriages. We produced, literally, reams of
output on these printers. We would then take the output, plot the numbers
appropriately, by hand, on graph paper and, truthfully, it took too long. One run
would take over a week to plot, so that you could see just the crudest of the
trends in the calculation, or what was much worse, find out that there was
some sort of error and all the computer time had been, in a sense, wasted.
However, this intimacy with ones output seemed to give a special kind of
intuition about how the program was running.
We decided that we needed some kind of automatic plotter. Now, the chief
engineer on the UNIVAC, Lou Nofrey, and Bob Crew, Chet Kenrich, and Dick
Karpen, all assistant engineers, found a commercially available flatbed
plotter. A crude vacuum held down the paper, and you could move an X-Y
plotter on top of it, and a little hammer would smack a die through a wet ink
ribbon, and leave a symbol where it hit. One of the problems with this plotter,
which was about the only thing then available, is that it had no interface to
the UNIVAC; the only thing it understood was IBM punched cards.
So—complexity on complexity—what was done was to feed the data written on an ordinary output UNIVAC tape (a metal oxide coating on a metal tape) into a
converter that could produce punched cards. For output, the UNIVAC
recorded on tapes at a density of 20 pulses per inch—20 bits per inch—20
characters per inch— in each of seven parallel tracks. The trouble was that
there was no such converter. So, Lou Nofrey and his crew designed and built
such a machine. It would accept input from the tape and produce punched
cards—old-fashioned, ordinary IBM cards. We then took the IBM cards and
put them through an IBM "Summary Punch" that was hooked up to the
plotter; (I think I remember that Benson Lehrner made the plotter.) After a
suitable setup, which included a lot of twiddling and diddling with multiplier
potentiometers (pots) and scaling pots, and so forth, and much effort to
ensure that the paper was aligned exactly with the X-Y axes of the plotter,
you could actually get it to read the cards and produce a series of points. And
then, by carefully drawing the axes, you could read values from the plot. Not
surprisingly, it turns out, that really wasn't much faster than doing it by hand.
An added "feature" was that the plotter refused to stay adjusted, so
comparison of successive graphs was essentially impossible.
The entire procedure was too complicated and time consuming to be useful
in our daily work but that, I believe, is a first instance of computer graphics at
the Lab. It was between 1953 and '54 that this work was done. The plotter
didn't survive too long because it represented much too much complex hand
waving to get the plot. It most cases, it was actually faster to plot
something by hand, even though it took a week to do, say, the fifty plots that
characterized a problem—that is, reduce all of the results from an entire
problem to the graph paper.
The other thing that certainly contributed to the early death of that plotter
was the fact that in 1956 we took delivery of some 704s from IBM, IBM
704s, on one of which there was a cathode ray tube that was imaged by a
camera. It was called the Model 740. So, this was the Model 740 on the IBM
704. And there was also a direct-view tube that went with it, the Model 780—a
modified television set. And one could look at the results as they were being
played out at the same time that the objects were being plotted by the 740
and recorded on film. The difference in speed between the Benson Lehrner
plotter and the IBM 740/780 was so dramatic that it quickly became the
favorite way to produce graphical output from the design calculations. Of
course, there's no free lunch. As delivered, the 740 was loaded with
inadequacies. The unit while faster than anything else we had, was
impressively slow, the distance between frames was not constant, the lens
and the film used did not match the phosphor, a single frame advance took
on the order of half a second. Initially, we were treated to the slow production
of poor quality pictures. However, fixing these problems turned out to be
easy, and it was stimulating and gave us the nerve to go to bigger and better
things.
Before discussing some software efforts, I would like to just mention some
other graphical developments from these early years. Starting with the IBM
740/780, we will be discussing the Digital Equipment Corporation PDP-1, the
Data Display Incorporated dd80, the Information International Incorporated
FR-80, the Scientific Data Systems SIGMA-7, and the Television Monitor
Display System. Some of the persons being interviewed will discuss, more
extensively, these and other aspects of the development of graphics
capabilities at the Laboratory. Even including these discussions, not all the
graphics work that went on at the Lab is going to be covered. The ones that
are mentioned are intended to highlight some particular aspect of the
graphical work that went on. It is not my intention to slight the other work. I
am simply not as familiar with it as I should be. Where possible, I hope later
to interview some of those who were more involved, but were missed for one
reason or another in my initial set of interviews.
In the abstract, the early display devices were ones that supply some
hardware solutions for elementary actions like:
- Set the beam to the point (X,Y)
- Move the beam to the point (X, Y)
- Change the intensity of the beam
- Move the film
With such primitives, it is necessary to develop programs that can display
characters, draw lines and, generally, produce pictures that satisfy some set
of requirements. The early display devices were delivered without such
software; a situation that was both awkward and liberating. One might say that,
instead of delivering, manufacturers abandoned their hardware at our door,
and it became both our job and our pleasure to write the software—it didn't
have that name then—that we actually got to use.
One of the most interesting pieces of software developed very early was a
subroutine called "Plotla" written by Norman Hardy. Its only function was that
between any two points in the lattice, it could plot a "best straight" line
represented by a series of points that were spaced either every point, every
other point, every fourth point, and so on. It was a very tight and fast routine.
The speed of the 740 was such that it could only plot, in 151 microseconds,
either a point at some location (X,Y) in a raster of 1024 by 1024 points, or
starting at a given point, it could draw either a horizontal line, a vertical line,
or a 45-degree diagonal line going from lower left to upper right from that
given point. With that capability, one had to see all of things that you'd like to
plot in terms of those three or four simple little capabilities. Norman's routine,
even though there was some nontrivial amount of calculation being done to
get the points, was a tight routine.
Being as fast as possible, Plotla was used extensively. So, everything,
practically speaking, was built out of that and one other subroutine. We used
some data tables to design characters that could be plotted in a 5 by 7 matrix
of points. With this capability, we were able to build routines that simulated
all kinds of graph paper and plotted all kinds of points, and also, of course,
with some trickery, produced strange kinds of surface texturing features. So
an area could be textured or otherwise marked according to some prescription.
Another interesting thing—this is now about late 1956 or early 1957—is the
so-called discovery of motion pictures. It happened this way: Every week, the
group of designers would get together and talk about their designs and what
problems were current. At these meetings, people would discuss and show
how their calculations were progressing. Among those was Chuck Leith. So,
one day he was showing some of the results that he had obtained by using
Ng triangular zoning [1]. Beside allowing
some very interesting approximations
and averaging techniques, some theorists believed tiling with triangular
zones was the most natural way to the difference equations in a region.
Anyway, his results, showing the movement of the mash, were recorded on a
strip of 35mm film and in a slide projector. When he was finished he just
pulled the filmstrip out of the projector and the image on the screen moved! It
electrified everyone in the room. This was incredible! The images were
moving! And, at once, everybody saw the value that motion would contribute
to the presentation of such results.
The use of motion pictures in science was common enough, but it seems
that no one had yet thought of them as being useful when doing mathematics
and physics with a computer model. But there it was—a powerful new way to
perceive one's results. So, several of us began looking into the uses of
computer generated motion sequences. In general, however, computer time
was too valuable to use it making movies. So, the movie-making mechanisms
had to be careful not to use too much computer time and, certainly, to never
waste time. Among other things, this inspired efforts to develop things away
from the big computers, and a search for a small computer that could be
used for movie making. One other approach was followed—the movie making
capability on the big computers was improved.
One of the first things we decided was that the camera was not acceptable
for movie making and had to be replaced. At this point enters our Technical
Photography Division. Then, Bill Jordan was in charge of it, but the guys who
did the work included Dave Dixon and perhaps one or two others that I don't
remember now. But what they did was to interface a real movie camera to
the computer. The camera had pin registration and a claw pull-down, so that
the amount of film that was moved at each frame advance was very, very
accurately controlled to within a few ten-thousandths of an inch. And with that
camera, we started producing motion pictures. The next problem that
showed up was that the film being used was not acceptable. It had, in one
case, a bluish cast, and the images were very blurry. I started dealing with
some representatives from Kodak, complaining about the need for a much
better film. They started bringing out samples of new coatings that would
respond better to the kind of phosphor that was on the 5-inch tube in the
Model 740. It was an exciting, interesting thing to do, to learn how to match
the capabilities of the lenses, and the film, and the film processing, and the
phosphor in the tube, and the amount of energy that was being produced by
the unblanked signal, and so forth, and get that all to work together
harmoniously, so that we got sharper images. But we did it. I should add that
some of these developments found their way into a neat Kodak handbook on
CRT photography, and some very excellent films were developed, giving
more resolution and more photographic speed.
It was a tremendous amount of fun, but not being satisfied with just that, the
next thing that people started talking about was, wouldn't it be neat if we
could produce color pictures? By color-coding the data in each frame, one
could see very quickly lots of new information that was not so obvious from
looking at numbers or even monochrome pictures. This was, I would say, in
the middle to the latter half of 1957.
The problem of producing color was, in general, much more difficult. One can
think of the customer as the human eye, a very high precision device. It had
to see the computer colors as comparable with every day color. The
variables in this study included the CRT phosphors, the color films, lenses
and film processors. We considered various things, but it was a suggestion
of Dave Dixon in our Technical Photography group that the color processes
that were available on film could be exploited best by organizing the
information we wanted to plot into separate frames of monochrome film,
each frame being destined for a unique, logical color. We called this the
Color Separation Method. Thus, you could imagine the information on one
frame would be meant to be projected onto the color film through a green
filter, and the next frame go through blue, and the next frame it would be red,
and so on. And, with the help of an optical-effects printer in the darkroom, it
would merge these pictures, one on top of the other, through appropriate
color filters, onto some ordinary, standard color film, and produce a
color image. We tried that, and although it took a long time, it yielded superb
color pictures. The problems of directly producing color pictures and movies
took considerably longer to overcome, so in the interim this Color Separation
Method was used, albeit sparingly. Nonetheless over the years, thousands of
colored movies were produced.
Our very first attempts at this, other than just test runs and so forth, were
done with the help of Leith's program. Chuck modified the code so that some
of the contents of one frame would be designated as green data, and the
next would be in blue, and so forth. Of course, one could merge, for instance,
green and blue and produce yellow and things like that. This gave a usable
color capability without having to wait for the making of a special color CRT
and special films and lenses and so on. It turned out that something over six
years went by before one could produce color pictures directly from a
computer controlled CRT.
I took the first set of black and white (monochrome) runs that were
produced. Since we had no equipment of our own at the Lab to do this sort
of thing, I took them down to a place in Hollywood called Film Effects. The
person who had invented the optical-effects printer during the Second World
War ran it. His name was Linwood Dunne. And he thought we were crazy
when I told him what we wanted to do and how to do it, but he said, "It's your
money, you can do what you want." So, in this manner, he produced our first
Color Separated movies, through green, blue, and red filters sequentially.
And, lo and behold, we had real color output! It wasn't the best, most slick
color in the world, but it was very usable color output. And with it, you could
see, for instance, the hottest spots in a field in red, or, when you chose, to
show a shock position in yellow. It came out beautifully.
We brought this film back to the Laboratory and showed it to all the
designers, and while it was generally very well received, the impression,
again was like the reaction to the Benson-Lehrner plotter several years
earlier: Ho Hum. It was too involved to become a part of the regular
production cycling that had to be done with these design codes.
Dave Dixon built an optical-effects printer that would do the job at the Lab.
With that little thing, many, many, movies in color were made, and when the
work outgrew the homemade version, we acquired a commercial version
where even more elaborate effects were possible. We tried putting the
effects printer under control of a computer, but it was a step too far into the
future; digital control was still too foreign to the film industry. And, as usual, by the time they caught up, the entire approach had been passed by.
The color films that Kodak and others now started producing for direct
exposure by CRTs got to be better and better, along with the processing that
was being done, and by our learning about better lenses, and better films,
and better CRTs and so forth, we could blend all of these things together and
produce the best color film exposures that were possible.
In 1960, say, we finalized the design of what was called a high-precision
cathode ray tube. Its raster was 4,096 by 4,096 points, and it had many
levels of intensity, and it could draw characters and draw lines, and so forth.
This machine, this precision thing, was to be an integral part of our first
acquisition of a PDP-1 computer from the Digital Equipment Corporation,
which had two basic goals. It was to be kind of a romper room for us to try
weird ideas, and it was to do all of the plotting that came off the larger
machines. You'd just bring over a tape and plot the stuff. The PDP-1 we got
was truly a romper room. With it, one could explore many areas that were for
us, really new. It was made to talk, play music, and do high-precision film
recording. It could accept magnetic or paper tape or punched-card input or
output and had one other unusual feature: It was able to digitize
photographic records. This was done initially using the 740 CRT on the IBM
704, but the PDP-1 was faster and far more accurate. This facility was called
the Eyeball, and was used for almost ten years to digitize many test films.
These films were held in the Eyeball's film mount device. A point (X,Y) was
displayed on the CRT and imaged on the film sample. One measured the
amount of light getting through the film and knew thereby, the photographic
density at that point. All of the readings were written onto a tape and taken
over to a larger computer where various filtering and analysis techniques
were used.
Somewhat independently, at Information International, Ed Fredkin, Ben
Gurley, and later Bob Waller produced a series of much more elaborate
Eyeballs, which they called Programmable Film Readers (PFR). The PFR-2
was the most precise of these, having effectively,
a 218 by 218 addressable raster,
and this was used by an AEC contractor, EGG, in
Nevada to digitize the films that were being produced there. The PFR-3 and
the PFR-1, at least, were used in rocket research to digitize rocket traces
and other shock analyses that came out of White Sands and North American
Rockwell and places like that. And PFR-0 and PFR-3 were used also in
various and sundry of the DOE laboratories to digitize things like bubble
chamber films that were produced by the big accelerators. Even with that
precision, it was decided by the persons who were doing the work, that that
was not sufficient. So, while they did a great deal of analysis, higher
precision measurements were needed. These were being done by other
kinds of specialized semiautomatic film analyzers based on the on-axis
measurement of photographic density. But I won't go further into those here.
Many years later, I met a person, Phil Peterson, who was then at Lincoln
Laboratory working on the TX 2. He independently had the same idea and
proceeded also to build an "eyeball," which he called the Flying Spot
Digitizer and used, among other things, to scan a slide of the Mona Lisa. He
then achieved some added fame by producing a very large picture of the
Mona Lisa on a Cal Comp 30-inch plotter. He did this while he was an
employee of CDC, and it had a great deal of publicity all over the United
States. Everybody wanted a copy of the Cal Comp Mona Lisa picture. It
became an instant collector's item.
Well, again, in the time between its design and delivery, new things came
along, not the least of which was, in our case, a display called the dd80,
which was built for us by Data Display Incorporated, in St. Paul. Data Display
was absorbed into CDC subsequently but, at the time, it was a separate little
company, and they were using an electrostatic deflection system instead of
the standard electromagnetic deflection. This allowed the dd80 to be much
faster than any other display system then in existence. And these machines
allowed us to return the display function to the production machines. On the
IBM computers, the interface was a data channel that delivered 36 bits every
6 microseconds to the display controller. We went through a typical
improvement sequence, getting better films and lenses.
With the help of Leonard Nelson, vice president of Ehrenreich Photo-optical,
we were able to get a CRT lens that dramatically improved exposures. The
lens was so good that it was produced in great number and used throughout
the AEC test facilities.
The dd80 CRT had a P-24 phosphor on it. Just as a measure of how things
had improved, when we started with the 740 on the 704, the cycle time to
plot a point was 151 microseconds. On the dd80 it was about 4.0 microsec.
Even with that short time, the new films were able to record the exposure. In
addition, much better looking characters picked out of an ASCII matrix, could
be plotted in 9 microseconds. And lines drawn from anywhere to anywhere
on the face of the tube could be done between 6 and 30 microseconds each.
So, the machine was a very powerful, fast, and stable thing. And it led to a
major change in how we produced output on the computers for the design
calculations.
We needed a fast camera. We found it in a company called Flight
Research, run by Bob Woltz. We got him to slightly modify a version of the
Flight Research camera that had been developed to photograph spark
chambers used at LBL and other accelerator sites. Our version of it had
thousand foot magazines, and was capable of up to 30 frame advances a
second—that's almost two feet per second. So the frame advance, which
used to be a half a second or so on the 740, became a more acceptable
time. In consequence, we could run some calculations that were more or less
simple and photograph the output at 30 frames a second, and some of the
movies that are still in the archive today were made precisely that way.
I want back up a little bit. Starting back in the early '60s, when we had the
PDP 1, and even on the 740 earlier, in addition to Norman Hardy, other
programmers like Bob Cralle, Garret Boer, and Gale Marshall were producing
useful software such as, given a large collection of values, plot a series of
isopleths in color. I remember also, David Mapes, who had an intuitive grasp
of the binary character of the display routines. We put together great
quantities of little routines that could be pulled out of a repertoire and stuck
into the physicist's FORTRAN code to do all kinds of plotting and graphics
massaging and so forth. Pat Crowley and I produced a thing called
CRT-Batch, which was a collection of many of the plotting subroutines existing at
the time. These routines ran mostly on the production computers that used
the dd80s, until they had outgrown their usefulness. There are, however, still
versions of CRT Batch running and the dd80s were never superseded in
speed. It was one of the fastest things on wheels then, and probably still is,
but there arose a desire for more precision and flexibility.
We got this higher precision in the '70s by our acquisition of the FR-80s
from the Information International Incorporated. The FR-80s—I think there
were six of them at the Lab at one point—provided a resolution of 80 line
pairs per millimeter. They had the ability of recording on a variety of film
formats and directly on paper. They took over essentially 80 percent of the
output load of the entire computational facility. The remaining 20 percent
were produced on a high-speed printer, the Radiation Printer, a 7 page per
second device. Its description will be covered at another time. The FR-80s
picked up a load of about one million microfiche per year with an average of
over 120 pages per fiche produced at a reduction of 42X. The FR-80s were
not anywhere near as fast as the dd80, but they produced an incredibly finer
image. As the people aged at the Lab, the speed got to be less important
than the quality of the output. There may be some sort of ontological
message here, but I'll leave that for some sociologists to ponder.
To return to the question of software, I think what should be inferred from all
this is that, given the opportunity to fiddle around with some very strange
equipment, people tend to do interesting things especially if they're not being
hassled by deadlines or other things. We had some young and very bright
people who came to work at the Laboratory during the summers. One of
them, Ken Bertran, was a cooperative engineering student at Berkeley. And
he and I worked a lot together. We started this adventure on the 709 or
7090, I don't remember. But the idea was, given a display device and given
the fact that the computer is a powerful logic engine, produce, with Kenneth
doing most of the work, a program that could do Symbolic Algebra or
simulate things like an oscilloscope probe. Such capability is present today in
things like Macsyma and Mathematica. It was really important, in his version,
that the functions you were working with could be displayed on the screen of
a direct view CRT, and in stereo if needed.
We moved fairly quickly from the 704/709/7090 sequence to the PDP 1
as soon as it got in, and started producing things there, because there was
much less pressure on the programmer to get out of the way and let some
production work get done. With Ken's program, you could get a stereoscopic
view of the function and, now, if you trained your eyes the right way, you
could see the thing in livid, three dimensions and understand what the
function looked like. You could do calculations with the program and
understand the consequences of changing a variable or changing a parameter.
We did two other things from which not much came. One was an
attempt to digitize voice such that you could synthesize new words from the
phonemes that were being spoken. This used the Eyeball, and the idea was
these sounds would be strung together to produce speech. First, it seemed
that we could read things and produce speech use by persons who had a
vision problem and to let the computer instruct the operators about what was
needed. Like telling them to mount a tape, or replenish a paper supply.
Again, a complete solution of the problem was too tedious for people to use;
there were simpler but less capable things available and more generally,
this sort of dabbling didn't really fit in with the Laboratory's programmatic
work.
We tried reading with music with the Eyeball. We could read the music and
select certain voices to have the PDP-1 play through a set of attached
speakers. But again, it was too tedious; the machine wasn't fast enough for
the job we wanted to do. All it was really good for was a "demonstration of
principle." Actually, an MIT student, Peter Samson, developed an encoding
scheme for writing and playing music on the PDP 1.
I should mention also that, when the Eyeball was developed, some of the
people—Gale Marshall, Roger Fulton, Ray De Saussure, David Mapes, and
Larry Gales—were very instrumental in making this piece of hardware a
usable tool for the purposes that we had intended. I'm not sure where any of
these persons are now, but the work they did in the early '60s should not be
forgotten; it was truly a pioneering effort.
I think it would be fun to end this diatribe with a little anecdote that I think is
charming. After we had the dd80 working fully, it was one of the sources of
illustrations never before seen. There was a program that Bob Cralle
produced called "Jahke." Inside this Jahke you could house all kinds of
programs.
At one point, I was asked to give a talk at an Association for Computing
Machinery (ACM) dinner meeting on St. Patrick's Day.
We put together samples of films and a first clip that said "Happy St.
Patrick's Day." Since we controlled the size of the characters, Bob made
them vanishingly small to begin with, and he displayed them backwards.
And then, as the film played, this phrase approached the viewer, and at an
appropriate time was supposed to turn around and become readily
recognizable as "Happy St. Patrick's Day." Well, so fine and good.
It was, of course, printed in all livid green, just beautiful. And it was the first of many little clips of film that were on this sampler that I had made up to
illustrate our work. We were privileged then to have such beautiful
equipment. And it was, well, it was fun to show others what you could do with
it. So, we went to the banquet, and we gave the talk, but something
happened.
The projectionist looked at the film before he put it into the camera, and
noticed that it was backwards, because he could see that it was "Happy St.
Patrick's Day," but it was backwards. So, he thought that the film had been
wrapped wrong on the reel. So he rewrapped it. He reversed it. I was
supposed to stand up and say, "Well, I want you to see that this is a special
day, because St. Patrick got rid of some of the snakes in Ireland, so we want
to wish you all a Happy St. Patrick's Day." And we show this thing marching
onto the screen, and I say, "Oh, it's backwards. Turn it around." But it was
already turned around, and now it went backwards. So, everybody got a large
laugh out of that thing. But it just goes to show you—you can't assume too
much about how others will view your work. It's working around corners that
you hadn't otherwise thought about.
Finally, there is a story to tell about graphics for the masses. In the mid
1960s I was shown some work at CDC being done by Malcolm MaCaulley
and Joseph Hood. Briefly, they were building interfaces in a CDC 3600 to go
from video to digital data and from digital data to video. The key point is that
the vision organ was an ordinary television display. Back at the Lab, we were
actively looking for a way to provide graphics capability directly in the user's
office that was affordable. Television monitors clearly were the lowest cost
displays we could find. The question of how and where to manage the
refresh did not have so obvious an answer. Then, some low cost disks came
available. The one we chose was a made by Armin Miller at Data Disk, Inc.
There was a head per track and the heads rode directly on the disk surface,
a cobalt-nickel alloy. The capacity was such that at least 16 video channels
could be placed on each disk. Then the vector display lists were scan
converted using a program running on the production computer, and
characters were produced through a video character generator. The scan
converter software was highly optimized; it produced raster displays faster
than a television camera. Thus was born the first model of the Television
Monitor Display System (TMDS). With the refresh of the screen being served
by writing the rasterized images on the disk, and the TV signals distributed to
individual offices over high-grade coaxial cable, the users finally had
excellent graphics in their offices. Later models of the TMDS grew from 16 to
almost 256 video channels, and then through some cross bar switches to
almost 2048 terminals. Also, later on, the refresh disks were replaced by
solid-state memories.
As with all the other graphics devices mentioned in this interview, a more
detailed treatment is given in some of the other interviews.
Finally, I would like to reiterate that this discussion of some early graphics
adventuring at the Lab is not meant to be complete.
[1] At that time, and even today, most difference equations were written specifically for quadrilateral zoning.
