Henry Spencer
Giganews Usenet History > Henry Spencer
Henry Spencer is a freelance consultant and avid founding member of the Canadian Space Agency. He also does some work in small spacecraft engineering.
Spencer co-authored C News with Geoff Collyer in 1987 and wrote The Ten Commandments for C Programmers. As a staff member of the University of Toronto, Spencer ran the first international news server, starting in 1981. Spencer also took over a Usenet feed from Duke University in 1981 and he began to actively archive technical newsgroup posts that came through that feed. As a result of this project, he accumulated over 2 million newsgroup posts which were acquired by Google in 2001.
Spencer’s notoriety amongst the Usenet community culminated with Vernor Vinge’s 1992 novel A Fire Upon the Deep which featured a character modeled after Spencer and a communications medium very similar to Usenet.
As of 2007, Spencer still runs the news server at the University of Toronto’s Zoology department, although it’s role amongst the Usenet network is smaller than it was in its heyday.
Interview (5/1/2007) with Henry Spencer:
1. You operated what is widely regarded as the first international Usenet site. What impact did joining the network have?
It meant being part of an online community for the first time. The importance of that, initially, was mostly technical. Being able to offer it to our users was secondary; we saw clear benefits to ourselves! That probably calls for some explanation.
We were running the new Unix Version 7, aka V7 (not to be confused with System V, which came much later). V7, including full source code, was essentially free to educational sites, but the hidden snag was that it came with no support — none. You were on your own. The natural response was that sysadmins and developers started banding together for mutual assistance; the Usenix Association came out of that. There was a lot of expertise in the community, but the problem was how to access it: Usenix’s conferences came only twice a year, and its newsletter had a somewhat irregular schedule. Being able to ask a question and get answers within a day or two made a huge difference. The “net.v7bugs” newsgroup alone made a lot of sysadmins get their sites onto Usenet as soon as they could.
2. What benefits were you expecting Usenet to bring to the internet? Did it meet your expectations?
Not sure what you’re asking here. Usenet, and our participation in it, pre-dated the Internet… and long pre-dated widespread access to the Internet and its predecessor the ARPAnet. In its early years, Usenet was mostly carried over UUCPnet, a store-and-forward network with transmission primarily by autodialing modems (with long-haul connections made mostly after midnight, when long-distance charges were low).
Assuming what you really meant to ask about was benefits to the user communities, in the beginning most of us saw Usenet mostly as a medium for technical communication. It easily met our initial expectations there. We had no particular expectations for the non-technical side — it seemed pretty incidental — and were a bit startled by its explosive growth.
I should probably elaborate slightly on that last. Whether you saw the ARPAnet as widely available depended a lot on whether *you* were already on it (or had reasonable expectations of getting onto it soon). Many prominent sites and people were on it… but they were vastly outnumbered by the ones who weren’t, and probably weren’t going to be. Even if your university was on it, *you* probably weren’t, unless you were in just the right department. It was exclusively for DoD-funded research, at least in theory — although the exact rules and the strictness with which they were enforced varied with time — and if you weren’t involved in any of that, you couldn’t get on.
(Also, unless you were quite close to an existing site, getting on was expensive — the leased lines and the routers and other equipment cost a bundle.)
3. What benefits did Usenet provide for your professional or academic life?
Directly, mostly what I mentioned in question 1: access to near-real-time bug reports, fixes, and expert consultation, plus occasional useful bits of new software. Given that Unix sites then were strictly on their own for software maintenance, this made a big difference.
Indirectly, an unexpected benefit of participating in the more technical newsgroups was that it spread my reputation far and wide, which was of no great immediate importance but paid dividends later. In particular, one of my first Usenet postings was a diagnosis and fix for a subtle but serious bug in some of the C library’s long-arithmetic routines, and a lot of people noticed that.
4. How did you and Geoff Collyer become involved with co-developing C News?
B News, the then-reigning news software, was somewhat buggy and quite slow. The real beginning of C News was when B News’s “expire” program stopped working entirely. After a quick look at the source convinced me that finding the bug wasn’t going to be easy, I decided to just rewrite the program. The resulting code was simpler and cleaner and a lot more reliable, and it also turned out to be quite noticeably faster.
That got Geoff interested in rewriting the core of B News, the code that received, filed, and relayed news. It still worked, but it was awfully slow, and the steadily growing news volume was getting to be a real problem. He managed to get quite large speedups with modest effort.
After that, things gradually evolved toward a complete package, partly because word got out on the grapevine and other suffering B News users started harassing us about releasing our code.
5. How did you collaborate while developing C News?
Although we worked in adjacent buildings, we didn’t actually visit each other much. For routine stuff, a lot of email went back and forth. For things that required serious discussion, we mostly talked about it over dinner at one restaurant or another.
In particular, we were both late sleepers and late workers, and both then single with limited social lives, so it wasn’t uncommon to find us both at our keyboards on Saturday evening. We both lived in the same direction and usually walked home, so we developed the habit of quitting at the same time Saturday night and having midnight dinner together at our favorite late-night restaurant on the way home. A good many basic C News design decisions got argued out in that restaurant.
6. Was the development of C News an official project or a personal pursuit?
A bit of both. Some official time got spent on it because we needed better news performance to keep load on our servers under control. But particularly later on, when the server situation was under control and the C News work had more to do with packaging, portability, bug fixes, and enhancements, it became more of a spare-time activity.
7. What were your main technical challenges when developing C News?
(a) Performance. We knew we wanted something faster than B News, but determined efforts (some of it written up in our old Usenix paper, “News Need Not Be Slow”) produced performance that really startled people who were used to B News.
(b) Portability. This came considerably later, when we started doing actual software distribution, especially after user feedback started coming in from the field. We were amazed at some of the stupid ways in which system suppliers had broken Unix software. Dealing with some of this presented real challenges, especially we had a strong preference for trying to write one body of code that worked everywhere, rather than a different version for each system. (See another of our papers, “#ifdef Considered Harmful”.)
(c) 16 bits. When we started doing this, we were both running Unix on 16-bit machines, the old PDP-11s. Making everything work in very limited amounts of memory was sometimes a real constraint. (Mind you, we think a lot of modern programming projects are far too sloppy about memory use, and the PDP-11’s constraints forced you to *think* about the issue. But once in a while they were a headache.)
(d) Figuring out what the spec was! Often it wasn’t obvious just what the software had to do to be compatible with B News, or to interoperate gracefully. B News contained lots of stupidity, so we didn’t want to just blindly copy everything it did. RFC 1036 helped, but it wasn’t complete, and it had at least one outright lie in it. It took considerable effort, and sometimes a few iterations, to sort out just what the software should do about certain issues.
8. Were you involved with other Usenet development projects after C News?
No, we were both pretty much burned out on news software! I did a Usenet-administration book for O’Reilly (with Dave Lawrence), and have been involved off and on with efforts to produce a successor to RFC 1036, but no more software.
9. For about a decade, you archived technical Usenet posts and discarded the rest. What was the signal-to-noise ratio of technical vs. other posts during that period?
It wasn’t so much that we archived only technical posts, as that the non-technical newsgroups were the first targets for cuts when we started to feel the need to reduce the tape consumption of archiving. We got gradually more stringent about that as volume grew; the main point of the archiving was, after all, to preserve things that might be of technical interest at some point.
While I never compiled statistics on this, my personal impression is that after a short early period of rapid evolution, the ratio of technical to non-technical material held roughly constant, at least in the “mainstream” Usenet groups. The volume of both grew tremendously, of course, and both splintered into more and more newsgroups to help keep the volume manageable, but I don’t think one grew more than the other. (I think that much of the moaning about how “Usenet has deteriorated” comes from people who don’t really know what Usenet used to be like.)
10. What do you envision for the future of Usenet?
Much the same, I think — perhaps some gradual changes, but nothing drastic. The fact that it’s seen as a bit stodgy and old-fashioned quite likely will work in its favor, encouraging the airheads and spammers to follow the latest fads and go elsewhere.
Individual newsgroups will continue to succeed or fail much the same way as they have in the past: a core of patient, knowledgeable contributors is crucial to making (and keeping) a group a success.
11. Where are you currently employed? What is your current role? What are you currently working on?
I’m still a freelance consultant and occasional author, as I have been for about fifteen years now. I do all sorts of software-related stuff, and also have a growing sideline in small-spacecraft engineering (which unfortunately isn’t yet steady enough to consistently pay the bills).
I’ve just finished being lead engineer for a preliminary study of a small Mars orbiter, funded by the Canadian Space Agency; if that project gets further funding, it may absorb much of my time for a couple of years. I’m also consulting on architecture and design of a complex embedded control system, at a local startup company (Gedex Inc.) that’s trying to build an airborne gravity-mapping instrument for mineral surveying.
12. Are you currently involved with the operation of any Usenet servers? Which cluster?
I actually still run the same news server — the one at the U of Toronto Zoology department — that I always have, although it’s now a quiet backwater, almost a leaf node, rather than the major regional hub that it used to be. (That’s probably just as well, since I haven’t kept up with recent developments in news software and would have to do some serious catching up before I could run a busy site properly.)
13. Do you actively participate in any newsgroups? Which ones?
These days, my news participation is largely confined to the sci.space newsgroups and a few of the rec.arts.comics groups. I intermittently read a few others, but seldom post to them.
14. I noticed that you and Geoff Collyer have asteroids named for you. What’s the story behind that?
It’s a side effect of having a good friend who’s a serious amateur astronomer. Asteroids are the one class of astronomical object where names are pretty much up to the discoverer, subject to some minimal rules and an approval process. So as birthday presents last year, Geoff and I each got an asteroid, so to speak!