David: [00:00:00] Something that people don't usually talk about and I think is really important and relevant when talking to someone is where did you come from? You graduated in 1958 from the Imperial College in physics and mathematics but that doesn't tell me who Ben Segal was, growing up like in school, in high school. Well what kind of kid were you?

Ben: [00:00:19] So I was an only child and brought up without a father in my grandparents' house in Manchester. I was spoiled because I was the only boy. My mother worked which was unusual in those days: she had a profession - she'd been to university, the first in the family. I was what we would call in English in those days a "swot" that is someone who loved books. I had a copy of "The Children's Encyclopedia". It was like ten or twelve volumes which I would just love to read.

Ben: [00:00:56] So I was a very studious kid but I also loved taking my bicycle apart and putting it together and that sort of thing.

Ben: [00:01:03] So I was a fairly typical only child I think. At school I was happy at school. It was a very good school. I was very well treated. The only problem was that they wanted so badly for me to win a scholarship to Cambridge or Oxford. This was the holy grail for British schools at that time.

Ben: [00:01:23] And so they sent me a year early to Cambridge to sit this exam and it was a disaster. It was freezing cold. It was Christmas. I hated it. The interview went very badly as well so I only got only got a place, not a scholarship.

Ben: [00:01:42] So I went back to school. The Headmaster said: "That's perfect you know, next year you'll get the scholarship". And I said: "No, I don't want to get there and I'm not staying". And anyway at that time if you didn't get a scholarship to University you had to do your military service. So I wasn't going to accept the simple place anyway - two years of military service was long. So, to cut a long story short I took the exam against the school's wishes but they had to allow me and I took the exam for Imperial College in London which was so easy compared to the Cambridge exams. So I got 100 percent on everything and I got a scholarship and I got this and that. So that's how I went to Imperial College and there I had the experience that for the first time I met lots of kids who were as smart as I was.

Ben: [00:02:30] And that was... I also was in a place where it was not nourishing like the school I'd been to, the high school, the grammar school where they'd taken care of the kids, particularly kids who were at the top of the class. Whereas at university I was there one of 100. It was a very hostile atmosphere because there was no group work allowed. Everyone for himself - they knew that they were going to chop like 30 in the first year and 20 in the second year and it was only a three year course anyway. So it was a very unpleasant atmosphere and the teaching was very poor. So at least by the age of 21 when I got out of there I understood something about good teaching and bad teaching and I also understood that I knew very little. Nevertheless I was very lucky to get this first job in the UK Atomic Energy Authority which sounds very military but it wasn't. I worked in the Industrial Division working on nuclear reactors. There I realised that I didn't know much physics but I started to get interested in computing. So - like many things in life - accidents and accidental meetings and accidental changes to your career made a huge difference to me. I came from, if you like, a fairly protected background - a privileged background in the sense that education was known that it was a good thing and I benefited from the free education at that time. This is England in the ‘50’s and ‘60’s recovering from the war. Education was a big thing. It was a good society. Like many, I thought it was going to turn into a socialist paradise and so on. But unfortunately it didn't. But that's another story.

David: [00:00:00] How do you get started? How would you advise a person that just graduated - or maybe even perhaps just a younger self - as to what to do with your life when you find yourself being 21, 22, 23 and a whole wide world and now you're not being told what to do. It's up to you now.

Ben: [00:00:16] Yeah, it's a very good question and a hard question.

Ben: [00:00:20] In my case, I just followed the line of least resistance, if you like. I was good at physics. I was good at actually everything at school. But physics I was good at. I wasn't a top mathematician at school. I had friends who were better mathematicians. But I just did what was easy. And getting into Imperial College as I said was very easy. So this was the second best place to go after Cambridge or Oxford. So I just went with the flow.

Ben: [00:00:48] Getting the job was luck. Luckily, they came interviewing at the university, which was not common in those days. I remember quite clearly the interview I had. It was fairly formal and I didn't know after that they were going to offer me a job or not, but they did. I took another interview too, it was Philips. Philips had a set of interviewers from Holland, very dry guys and I quickly decided I didn't want to go and work with Philips.

Ben: [00:01:19] So again, it was a ... (??) I got this job offer. It was prestigious. Paid next to nothing but it was perfectly fine for me. And it led... one thing led to another. What it led to actually was two things. First I got interested in computers, even though I was not a programmer. Programmers at that time were lower forms of life for physicists, so I would write the equations and go and discuss with the programmer.

Ben: [00:01:47] So I saw the Ferranti Mercury computer which used paper tape. You had to repeat the problem twice if you wanted to be sure, or three times if the results were different the first two times, this sort of thing. So I saw IBM come. I mean, this was a computer where you didn't have paper tape. You didn't have to run the problem twice - it was very reliable. But it was also the center of a big social, rather frightening atmosphere that we ... OK we were the physicists, but we couldn't go into the computer centre. It was sort of a private thing.

Ben: [00:02:24] We did have, I had a programmer friend - this is a nice story. At that time, we were very young and carefree. And I don't know if you ever heard of what is called a football pool. It's betting on football matches and there were several companies. I remember one was called Vernon's - I forgot what the other one was called - and you would fill out your form by matches. Every match was like a penny or something, depending... people would just fill out 20 lines. And if one of the lines won, you'd get some money back.

Ben: [00:02:56] So. One of us had decided (it wasn't me) that we could do this by computer. And I had the idea that we should not have any system trying to guess the true results. What we should do is use random selections because if it was a random thing, it was much, much more likely to win when nobody else did. Because if there was some logic to guessing the better teams, then more people would win. So our idea, my idea, was: let's do it randomly. So my friend wrote a program and every week it would print out thousands ... and we were entering thousands of matches each week - we had a little syndicate, six or eight guys - and the company agreed to take our paper listings and process them (it was all done by hand). And if we won, we were going to win big. Well, I think we won two or three times but we won really nothing. But it was fun. Every Monday morning, we would all go and we would press our noses against the window, the glass outside the computer centre. Our friend would go in, print out, run a little program to do the checking and everything and we'd know if we'd won or not.

Ben: [00:04:10] That was my first introduction to a computer being fun. That was an IBM. So that was in Risley, near Warrington near where I lived. That was one of the social aspects of computing that I remember from that time.

Ben: [00:00:00] Another thing which I learned, I made friends with a very, very good programmer, called Tony Hassitt, who had written a program to solve simplified equations of reactor theory to design reactor cores. And I became very interested in this program and I became an expert in handling input and output for this program. You know, being a liaison between the physics group for running this program, presenting the results, etc. If you like I was I was ... I was not programming, but I would talk a lot to him and feed back ideas from us. And so I helped to design this program, helped to develop it (he did all the programming). And it was a very advanced program. And what that led to, after I moved to the States, which we'll talk about later, it led me to take this program to Los Alamos Laboratory, where our program was much faster than their program, though it was less accurate, but it was a very, very fast program.

Ben: [00:01:03] So Tony let me take a tape. Well, he let me take a tape with me from England to this company. And we used his program inside the company I worked for - the private company that we'll talk about. But soon the word got out that we had this very fast program and Los Alamos was interested. And so they flew me out to Los Alamos in 1963...

David: [00:01:26] With the physical tape?

Ben: [00:01:27] With the physical type, yeah. And it was a wonderful experience. So I was 26 years old. I had never been to the West before - I'd been to the Midwest, I was in that. And it was the most romantic trip. So I flew to Albuquerque, which was for me like a different world.

David: [00:01:45] It is pretty.

Ben: [00:01:46] Yeah. And I arrived there in the night, in the evening, it was before sunset. And I had instructions what to do. It was rather mysterious. I had to go to a certain desk, OK. And it was called Carco. I remember the name - "Go to the Carco desk and wait". So there was I with my tape and my bag. Nothing much happened at the Carco desk. The airport was pretty empty anyway. And soon, though, a guy comes just in jeans, jeans and sort of maybe a cowboy hat, I don't remember. "Mr. Segal?". I said "Yes". "Come with me". So we go out the side entrance to a little plane and he flies me in this little Cessna or whatever it was, up from Albuquerque up to the Mesa, which is where Los Alamos is. And I was, you know at that time, of course, I didn't have the guilt feelings about Hiroshima. Being a young physicist through the war and everything, I still thought that the Manhattan Project was the most exciting thing in the world. It was a great achievement and so on, and here I was. And we land there and a car comes for me and we're delivered to the old lodge ...

Ben: [00:03:04] The guy who came in the car was also, he wasn't in jeans but he was in a sort of a hunting type shirt and a string tie, I remember. He was the Director of the Computing Division - Carroll Zabel he was called, I remember his name from all that time ago. He welcomed me like I was his friend. You know, a long lost ... It was so unexpected and so charming as a young man.

*** ERRATUM: Carroll Zabel was the Division Leader of the Reactor Development Division, not of the Computing Division (BS) ***

[00:03:32] So he took me to the Lodge. The Lodge at that time was a log cabin type building, which was the original school building where Oppenheimer and company had .. it was what existed when Oppenheimer discovered it. In fact, Oppenheimer's family used to go there to ride horses. Oppenheimer came from a very rich family and so Oppenheimer knew this part of the world from before, and when they needed a lab in the back of beyond, in remote places, he thought of putting it there. So what we were staying in was the original building there. It was a school or - I don't know what it was. It was a school, but it was a ranch. And that building was pulled down since. So I stayed there. And later that evening for supper, the guy comes for me and we go to eat supper in his house with his family. The next morning, I go in with my tape, I deliver the tape and they gave me a tour of Los Alamos's Computer Center. I still remember the machine there. It was called the IBM Stretch machine, the biggest machine in the world. It was literally very big. It was very long. It had lights flashing. It looked like the biggest computer in the world.

[00:04:47] Now I went back in '91, it must have been, back to Los Alamos for meetings with our Division Leader of the time, by the way. And we couldn't get in - the meeting was held outside the fence. It was part of Los Alamos's meeting rooms. But we were not allowed in - there was no question of going inside to see the computer centre, no question.

[00:05:12] So that was very exciting. And that shows how accidents lead to interesting things. In exchange, by the way, we got a copy of their neutron transport program written by a Swede, called Bengt Carlson, I remember, and I could take that back, too, with me to the company in which I was working.

David: [00:00:00] It may be interesting for someone listening to realize that carrying a program with you physically is not insane, given that this was in the '60s and that ...

Ben: [00:00:10] There's no other way because it was a program that only ran on IBM machines, OK, so it wasn't portable, it was written in FORTRAN.

David: [00:00:17] OK.

Ben: [00:00:18] But it was built for running on IBM. By the way, what's interesting: IBM computers at that time couldn't fit in a big program or all of a big program into memory.

David: [00:00:31] Right.

Ben: [00:00:31] So they used mag tapes - what was called chain tapes. So you would actually load the first part of the program, maybe the part that would read the data, you'd load that first and then you'd read your data in. And then the operator would have to take the tape off and put another tape on. It would read maybe the main program in. And I certainly remember when the program had finished running, and it was ready to print, you would have to change the tape to the printing the output, you know, things like that. So the resources were so restricted in memory and power.

*** ERRATUM: Chain tapes had a set of files on them so there was no need to physically change the tapes between program sections (BS) ***

Ben: [00:00:00] I worked in Detroit with this British program. We would prepare the data for that program. We didn't have an IBM computer in our company, and you couldn't sort of send it out - there was no Internet. So I and one or two other guys would fly out to the cheapest IBM computer we could get hold of, which was in Philadelphia at an oil company. I can still smell the oil smell in the air.

Ben: [00:00:26] So we would arrive. Computer time is cheapest at night. We would arrive in the evening with our tapes and everything. And we'd go and eat some supper and we'd go out to the oil company and we'd go into the computer room. And they would be just finishing the shift and it'd be our computer for all night. So we'd get in, we could load our tapes, we'd do everything. And we'd have our cards, our punched cards and everything, and we'd leave with stacks of printout. But we had to babysit the computer: for instance when you ran a big program, the convergence, this was part of the thing - it was a big numerical mesh. You were solving what was called the diffusion equation. So it's like solving a heat equation, it's more or less the same equation. And so you start with a guess of the solution, and you'd iterate, and you'd check from time to time if the errors were getting smaller and the thing was converging, or not. If not, you'd have to change the input and start again.

David: [00:01:28] Yeah.

[00:01:28] So that was done hands-on, hands-on by... There was no screen. So from time to time you'd print a bit of results, you'd print some metric of what was going on. And to print, you'd have to stop the computer because the computer was single tasking. So if it was in compute mode, OK, it would remember. And then you'd stop it and you'd go into print mode, you'd print out what you had and so on. So you'd have the whole computer centre to yourself.

David: [00:00:00] And you also spent close to 10 years in California around the '70s. How does it compare? And you've been back to California relatively recently. So how does it compare? And this is we're talking about this time for San Francisco Bay Area. How is it different now from how it was back then? And perhaps it's not so different now? Right.

Ben: [00:00:21] Actually, it was only six years in California - for four years I was in Detroit.

David: [00:00:24] OK, yes.

Ben: [00:00:24] Well, we saw California the first time on a trip from Detroit. And so we both fell in love with California, my first wife more than I did. But we never managed in the end to live in San Francisco. It was a complete chance that I went back to Stanford to do a PhD. That's how we really lived in California.

Ben: [00:00:48] Now, how was it? Well, the Stanford campus was probably still much the same, many less buildings, but it was still charming. We found a lovely apartment in Menlo Park, the centre of Menlo Park. I do remember we found this lovely apartment. We'd been paying in Detroit, I'd been paying, I think, 100 dollars a month. And we didn't have much money. I earned 300 dollars a month tax free. I was paid that much plus all my tuition, everything. But we were poor. My wife worked. She got a job working for British Airways. She'd worked for British Airways in Detroit in the reservations. And she managed to get a job in San Francisco. So we had some money, but, you know... We thought, with our budget, we could go to $120 but this apartment cost $150 and we were so torn about this and we loved it so much. But in the end, we said yes. There was another condition: no children. The landlady was very uptight about that sort of thing. So we had no choice. But when we did have a child, we had to move. We moved into the Stanford married student housing.

[00:01:59] So Stanford is very recognizable when I go back - it's the same. What's different is the students. I mean, it was all white then. And now it's like going to Google. I mean, talk about rainbow, and young. So that's different. For instance, I still of course get the Stanford Magazine, the alumni magazine and just looking at the names...

David: [00:02:23] Yeah.

Ben: [00:02:24] ...of the classes and everything... It's a completely radical change.

Ben: [00:02:30] Berkeley, I remember, Berkeley always looked a bit more hippie and rainbow and strange.

David: [00:02:35] Still does.

Ben: [00:02:36] Still does, OK. But Stanford was, you know, rather white and uptight. When the student uprisings came, Stanford did participate, but behind Berkeley. We did have the National Guard on campus at least once. '69 I think that was. But Berkeley had them often!

David: [00:02:56] Yeah.

Ben: [00:02:57] So we were ... it was, you know, Stanford is nicer looking and so a bit more comfortable than Berkeley.

Ben: [00:03:06] Now the Bay Area itself. Yes. Well, first of all, when I arrived there was no Bay Shore Freeway ***. The 280 was built while I was there I think - it went right across SLAC. I didn't work at SLAC, I worked at the University. Because I was doing ... I was in the Mechanical Engineering department, but I was doing Nuclear Engineering, we were actually, our offices were at the Ryan Lab, which was a lab that had been an electrical engineering lab with great big equipment and they'd built a test reactor there so Stanford had a test reactor - water cooled reactor - on campus. It's gone now. And that's where I worked.

*** ERRATUM: I meant the Interstate 280 – the Bayshore Freeway (Route 101) existed since the 1920’s (BS) ***

Ben: [00:03:47] So, yeah, the traffic now, the traffic was still heavy, everything being relative in those days. And I do remember very well going back on my sabbatical in '77 and thinking "My God. It's the rush hour", you know. And then I realized it's always the rush hour compared to what I think.

Ben: [00:04:08] The traffic, of course, in the States is better managed than in Europe. Even in Detroit, when I went to Detroit in '62, the roads, the lights, the traffic lights were synchronized for miles. Of course, it was pretty easy in Detroit because they had these long square grids of roads. But the idea of having a synchronized traffic light in Switzerland, I mean, that was unheard of, just unheard of. I remember also coming back from the States to Switzerland one time and I turned right on red, which was a very, very good habit. And then in those days, there were practically no cameras. And I just happened to do it where there was a camera.

David: [00:04:45] Of course.

Ben: [00:04:46] And I remember taking the trouble to go to the judge and explain that, where I'd come from, turning right on red was legal and it was a very good idea and why don't the Swiss adopt it - but he still made me pay the fine.

David: [00:00:00] So continuing with CERN, of course, you've told me about a lot of people that you've worked with over the years, most of them really into highly accomplished... Is there anything that you perceive is common to all of them? Like is there some that... is there a tell tale of all these people that you worked with? Is there something common to all of them that you can pinpoint like, oh, yeah, all these successful people, they share this trait? Everything, everybody comes in a different...

Ben: [00:00:36] There's a certain subset of people that I liked to work with, people like you that are open, enthusiastic, plenty of energy - a lot of them Spanish, by the way.

David: [00:00:47] Really?

Ben: [00:00:47] Oh, yeah... When we started with BOINC, almost all of the people working on BOINC were Spanish. A lot of my students were Spanish, and the CERN people were Spanish. Ignacio Reguero. Miguel Marquina - he wasn't strong technically but he got us visitors from all over the place in Spain from... Yeah, and one of the very, very brightest guys. Daniel Lombrana...

David: [00:01:19] Yeah. Yeah, yeah.

Ben: [00:01:20] He was... it was Miguel who had a relationship with, what was the university that he was at? Extremadura, Merida. And that was through his contact... There was definitely a big Spanish contribution in that area. So a lot of, a lot of people .. I've had a lot of very good Spanish people. I don't know why, they have this, but like you, I don't know - they're hungry, they're quick...

Ben: [00:01:54] Whereas I have often been disappointed by my own compatriots whom I've worked with at certain times. You know.. "It's 5:30 and David is not going to be in his office...". It's awful. I remember when we were running the satellite group, we were running on this satellite project. We had a big collaborator was the Rutherford Lab in Oxford. That was so British. Now, it's true that the main collaborator there was a very good, very bright guy who was very unorthodox, but... It was such a pain to go there. So we'd go there. I mean, we were debugging this CIM, the CIM was this hardware interface between the modem and the minicomputer with the tapes, OK. And I went with the hardware designer, a wonderful guy, a Swedish guy. And it was probably, I would guess it was like Friday morning. So Friday morning we started to set this thing up and we needed to make a correction - so the technology is wire wrap. Remember wire wrap? Ever seen one...? You have a board...

David: [00:03:07] Yeah.

Ben: [00:03:07] This is low, low level integration. So you have components... maybe the processor... and it's all .. the sockets, the pins are under the board. And the pins you can interconnect with wires: it's done by computer. But if you need to make a fix... So I remember

• there we are on a Friday. What it would have been is it was Friday, 11:00. By 11:30 it was time to go to the pub for lunch. So we get out of the pub at 2 o'clock, right. And it's "virtual weekend" already at 2 o'clock in England on a Friday. And we desperately need to make this fix to test the box. And we need a wire wrap tool. He's lost his wire wrap tool, my friend. And we say to Chris, our collaborator: "Chris, we need a wire wrap tool - where is there one?". "Oh yes.. ah, Bernard will have one of those. Let's go and see". So off we go, and we had to go through a long rigamarole: "Hello Bernard, how are you..." (First of all Bernard was in, which was good). It's like negotiation for an arms treaty. "Would you by any chance have a wire wrap tool?". "Might I ask what you want to do with it?". The guy had a wire wrap tool. He didn't want to lend it to us. He probably did in the end. But other similar cases would be: you'd go and Bernard's left. Back on Monday, maybe Tuesday. It's infuriating. So that was the sort of thing that I didn't like.

Ben: [00:04:42] So what they had in common? Yes. Now, what was most important for me, though, for me personally, was mentors. I had a wonderful mentor when I arrived at CERN, an Australian guy, no one's ever heard of him - Mark Palandri - this is a guy who, he could code anything. He'd write beautiful, beautiful code. He would sit down and mostly... he would write .. he would sit with a yellow pad and he would write, like Mozart you know, comments to the right of every line. And he would write this code, pages and pages and pages for whatever. So typically it would be assembler on some machine or FORTRAN, etc. Whatever. He was the guy who, everything was possible. So there were no networks so he designed a network, he made a network project. He got some guys, guys would work for him that weren't in his operation. He wasn't for most of his career... He wasn't a Group Leader, he had no particular... But he was a really nice guy, he was brilliant, he would speak slowly and so on, I loved to work for him.

David: [00:05:45] He was pleasant..

Ben: [00:05:46] Pleasant, but fantastic. Fortunately, he managed - in those days there was enough space to do stuff like this and he got support.

Ben: [00:05:55] So OK, I landed at CERN in 1971, July. I'd never seen this guy. I'd been hired in a very unorthodox manner, which I mentioned to you. And I arrived and there was nobody there. So they gave the CDC manuals, I had to learn the machine code of CDC. Maybe I would have to program the CDC (I never did), and one or two other manuals. I had to wait for two weeks before.. Because they'd said you could work on this, this or this. And I'd said, well, communications, that sounds interesting. (I knew nothing about communications ...). And so when Mark gets back from holiday - he has a Spanish wife by the way, Rosa, a wonderful lady - they gave me my project. And my project consisted of twelve minicomputers, which are going to be put in twelve important sites in CERN with a card reader, a line printer and a teletype and a modem and a communication line to the Centre. And those had one program emulating a Remote Job Entry terminal. D'you know what that means?

David: [00:07:06] Well, I understand the words separately, but...

Ben: [00:07:09] Well, in those days when you submitted a job, well you went to the Centre, they gave you an operator, you gave your cards to the operator, and you came back and you looked in the thing on the wall, a rack for your output, hopefully... And this was a revolutionary idea where instead of everybody having to come to the Centre, they could stay near their offices and there would be a Remote Input/Output Station...

David: [00:07:37] Oh, yeah.

Ben: [00:07:39] ...and there were twelve. All of them had been chosen, purchased, designed - before I arrived so I didn't have to do that - by Mark.

Ben: [00:07:48] The Centre had these big Control Data machines as the central machines. No IBM at that time in the Centre. And they had Remote Job Entry terminals with a card reader and line printer and they would sell it to you. But it was very expensive. And Mark hated this. It was slow. He saw them as slow, horrible. And so he looked around - there was no Web. And he found this little company in England building a revolutionary very fast machine that no one had ever heard of, called the Modular One. And very, very fast. Completely different, of course. Its own machine code. Its own everything. Its own operating system - no, it didn't have an operating system, it had a monitor. It only had 16 Kilobytes of memory. And it was a very nice design. Each unit was about the size of that, right? (SHOWS A LARGE DESK). Maybe bigger. And so there was a CPU unit and there was a disk.. no, no disk. It was this memory unit - it was magnetic core memory. And there was a unit to control the teletype. It was big. So, very expensive. And Mark found the most beautiful card reader: beautiful, smooth wonderful card reader, and a beautiful line printer - from different companies.

Ben: [00:09:08] And Mark's actually ordered all this stuff. The stuff had arrived, right. And I've just arrived too. So my job was to integrate all this stuff. And in probably half of these places, there was an operator as well, who would help the poor physicists to use this thing because you know the physicists... They were just as horrible as they are now. They were impatient. "No, it's not working... ", et cetera. So there was a nice lady operator to calm them all down and they could leave their cards and she would submit jobs. So I had this big thing, big operation. I was alone. So I had to manage all of that. I had to make sure that this thing worked. So this was a good idea.

Ben: [00:09:59] So, Mark, OK, had to write a driver for the card reader, which was not supported by the company, and for the line printer which was not supported by the company. Now, this machine didn't have any switches or keyboard or whatever. It had .. in fact I just sent him an email the other day to figure out how he booted it, and he thought there must have been a paper tape reader. The only thing. It had no switches and no lights, nothing. The only external interrupt you could give it was switch it off and on. Switch it off and on. Talk about Ockam's Razor - very elegant, very British, very much from Cambridge.

David: [00:10:40] And it had no ROM to read from?

Ben: [00:10:43] There must have been a little ROM. Yes. Sure. Sure. To run the paper tape. Yeah. OK. After that, you had to boot it. So what we wanted was that the machine would boot out of the card reader. You'd put some binary cards in, load the program from the cards, and you got that to work. And then I remember him struggling so much to get the printer to work. Remember he's a brilliant programmer. And he came to this machine and he would go: click-click. Nothing happened. Curses, more work, and again: click-click. And one day after - it must have been a week - he'd taken a huge amount of time: click-click. And one character came on the printer! He was so happy!

Ben: [00:11:31] It was that sort of environment, apart from the political problems with this, because there was also a political part. So these things started to work. What I did was actually quite clever. It turned out they were emulating the existing remote station of the Control Data, and the Control Data it had this reader, line printer and it had a teletype. Now, how it worked was you went to the original machine and you put your cards into the hopper of the card reader, and then you were supposed to go to the teletype and type "Read", OK. And then it would, then it would read the first 12 cards. It had a buffer of 12 cards. And then you had to type something else and it would start to communicate with the host, with the central machine, and it would print something on the teletype and then it would read the rest of the cards, hopefully. It would read them, by the way, 12 at a time. I remember 12 because it had a buffer of 12. It wasn't probably double buffered.

Ben: [00:12:33] Actually I never saw the original one, I'm describing how... But then it's emulating this example. And so we thought, gosh, this isn't very good. The physicists... we can't have them typing on this thing - and they'd get so impatient. So, so we really needed to do this better. So I looked at the code in this thing. There wasn't much code. It was all in assembler. The strangest, of course. And Mark, by the way, being a genius, had also quickly whipped up a cross-compiler *** for it on the main CDC - the main CDC had a beautiful compiler, a beautiful one with macros - a really powerful compiler. So whenever Mark got a new minicomputer, which was often, the first thing he'd do is write a cross-compiler for it. So he wrote the cross-compiler.

*** ERRATUM: All the references here, and below, to cross-compilers should be to cross-assemblers (BS) ***

Ben: [00:13:16] And I looked at this code and I saw it was written by a most dreadful programmer, in fact, an Irish guy we'd met when we went to the company's site. No subroutine calls, all in-line code. Sooo, wonderful, there's all that space that I can economise. So I snipped out, ripped out all that, made subroutines, and I saved a lot, a lot of space, maybe a Kilobyte (LAUGHTER!). But a lot. And I wrote a little emulator, a human emulator in this case. So what would happen? You’d put the cards in, it read the first 12…

David: [00:13:57] Exactly.

Ben: [00:13: 58] …and then it sent down the line what the guy would have typed.

David: [00:14:01] Yeah.

Ben: [00:14:02] And so the reply came, and it sent the next message, and the rest of the... So, so you could walk up to it, put your cards in, press the button, and away you go. But we knew if they hadn't made that mistake there wouldn't have been enough space to write that, things like that. So that's how you learn: when you have a master mentor who writes beautiful code.

David: [00:14:27] That's so true.

Ben: [00:14:29] And you learn by reading other people's code. Same thing with the CIM, this thing to connect to the satellite. There was a raw, raw, raw box built by my friend with a 6800 in it, (not a 6809, a 6800, very limited instruction set). What you learn is it's very simple, but you have to write everything. There was no system, there was a monitor of a few hundred instructions, but after that ... So you know, you have an idle loop and you're going around checking interrupts and checking stuff. And so. I've still got the code in a box somewhere. You can write any code. And it worked beautifully, because I worked with the hardware guy. He was explaining, you know, what you have to do is put this in this register and do that. And yeah, we worked together. Yeah.

David: [00:15:24] It takes sometimes people like that to show you what's possible. It reminds me of like the four minute mile. Yeah. No one thought it was possible, so it took forever for it to happen. Then one day it happens. Next year, two more guys do it and then the next one ten and so on.

Ben: [00:15:38] Right. But I was so lucky to have a guy like that.

David: [00:15:40] Well, yeah,.

Ben: [00:15:41] He left after less than two years, he left in '73. After just over two years he left and he went to Australia, back to Australia. I was so lucky - if I'd finished up with a dead-head boss, I wouldn't have done anything here. So, he introduced me to communications. And he'd started another project, which I inherited with another colleague of mine when he left, and again, would you believe it, you had to write this and build it yourself? Okay, what was the application? They wanted to connect terminals, asynchronous terminals, to the Control Data timesharing system - they had a primitive timesharing system on the Control Data. You either bought their expensive terminals, or if you wanted to connect your own terminals, ASCII terminals, as the Control Data was not an ASCII machine, you had to do it yourself. So Mark says: "Right. We're going to build a terminal concentrator, OK? And we're going to have 32 input ports, for ASCII terminals, asynchronous ASCII terminals". (I think they could connect synchronous, complicated synchronous terminals of their own manufacture). And how do you connect this concentrator to the machine? On the channel. On the channel of the mainframe! Mark knew a bunch of people that had already connected all sorts of channel attached equipment.

David: [00:17:10] Right. OK,.

Ben: [00:17:11] So a channel adapter, built by CERN, existed and could be modified. And so it needed to be modified to go on this particular Hewlett Packard minicomputer, that Mark had chosen because it was the only one that could do micro-coding. You couldn't multiplex 32 terminals all entering characters without handling the characters super-fast, because if it did it in 70 milliseconds, that was too slow. He could do in 10 with his micro-code. So Mark bought the machine that would do that and coded the micro-code - with his own cross compiler of course.

David: [00:17:48] Of course.

Ben: [00:17:49] That was beyond me. I mean, that was just... But it was an attitude like that. And he wasn't hyper, he wasn't crazy. He was just a great guy, a perfect mentor.

David: [00:18:02] Yeah.

Ben: [00:18:03] When he left, I and my German colleague, the German colleague had done the code on the CDC side, on the Control Data side, which would read the stuff coming from the terminal concentrators and put it into the timesharing system and get the responses to come back. Okay. So he and I thought, well, we can run this project. Mark has left. Instead of which, the Division management, who didn't like us, had just hired a useless Dutch guy who had no function at all. I don't know how he got in. And they said, well, he can manage this project. We had one meeting with the guy and my colleague resigned and left CERN, and I refused to work with him.

David: [00:18:53] That meeting went well.

Ben: [00:18:54] Well, maybe it took more than one meeting. The first meeting we realized the guy was a loser. He was very charming, but he was completely useless. He had no function and we very much resented it.

David: [00:19:02] Yeah, it's a contrast also.

Ben: [00:19:05] Yeah. So. So Hans left, became a multi-millionaire, multi- multi-millionaire. And he started a company and now he buys other companies. I just met him recently.

Ben: [00:19:20] And I sort of went on strike and said I wouldn't work on it, but I consented to advise a replacement programmer. The system was basically finished at the time and... by the time Mark left. And that wasn't good at all. And so I became under a cloud. And that's why I took a sabbatical.

Ben: [00:00:00] So in 1983, I finished a five year project on satellite communication, which had taken me out of the main line work in the Computing Division at that time. It'd been a great project, but when the satellite project finished, I needed a job. So what I did do was find a new boss. The boss I'd had during the satellite project was an ex-Director of CERN, an Englishman called Mervyn Hine, who, together with the famous John Adams, had designed, the two of them, the PS - CERN's big Proton Synchrotron - the first big machine at CERN. They had come from Harwell in England as young men and they'd done that design basically. So when Mervyn Hine finished being a Director - Director of Research, Director by the way, of the Computing Division as well - he needed a job and he came back as a technical person and he had the idea to build this satellite project: the idea being that CERN could broadcast its data, measured in experiments at CERN, simultaneously to European laboratories over a satellite.

David: [00:01:20] Wow...

Ben: [00:01:21] And we did that. It was a five year project. I can tell about that in more detail later. But at the end of that project, I needed a job. And so I'd had one illustrious boss that everyone was afraid of. Now I had to find another boss who would, you know, nourish me and protect me and so on, which I'd always needed - a good boss. And I found this in the shape of the then Leader of the Software Group of the IT Department, a guy called Les Robertson, a Scottish guy, and Les - during conversations we'd had during the satellite project and other times - Les had decided that distributed computing was the way to go.

[00:02:11] Now, Les was the head of software in a department which was dominated by mainframes, mainframes and proprietary computers. So: IBM mainframes and Digital Equipment mini-computers, and so on. But he had been bitten by the bug of distributed computing. So he stepped down from being a Group Leader, became a humble Section Leader, and he took me and another younger guy called Frederic Hemmer, who is now the Head of the Computing Division at CERN, and we formed the DC Section - Distributed Computing Section - three people in this very big Software Group. And the first thing we started to look at, and I started to look at, was Remote Procedure Call (RPC). We were looking for protocols that would help us connect all these very disparate computers all over the campus at CERN, for which we only had a home made network, which connected a few of these computers, but we really didn't have a proper network. So it was in looking at that, after looking at that, that it became known we were working on this, that a young man called Tim Berners Lee some years later, probably three or four years later, came to me to learn about RPC. (Now, ah sorry, it wasn't so much - it was maybe in '84, '85). Because he had been given a project using RPC to work on in a different group. It was in the same Division at that time. So he'd heard we'd worked on it and that's how I met Tim Berners-Lee. And I noticed, you know, this bright young guy: nobody knew what he was going to do, of course. But the first contact was made then. Later, when Tim was working seriously on the Web design, he came back to me to talk about that design. But that was later. That was in '88 or so.

David: [00:00:00] What I find fascinating is how deep that effort to distribute the data through satellite was, how did it work out?

Ben: [00:00:06] Right. Wonderful. So we did not use Internet protocols, although, funnily enough...

David: [00:00:11] Which year is this?

Ben: [00:00:12] '78 to '83.

David: [00:00:15] OK.

Ben: [00:00:16] 1978, it started. The challenge was this. If you want to distribute data across a satellite link, obviously you need communication protocols. But all we got... Mervyn Hine, my boss, he made a deal with Eutelsat, the European satellite outfit, to get free time on a satellite channel most days for like an hour - it was a different time each day. That's all we got. He got some equipment from Marconi in England to actually beam towards the satellite, but for the rest we had no other support and we didn't have dedicated computers.

Ben: [00:00:59] We were five collaborating labs. It was CERN ... maybe six, CERN and five other labs. Nobody had a special computer. We didn't have a standard computer to use. We didn't have a standard programming language and we didn't have equipment to do the necessary time division multiplexing of this channel, we had nothing. So in the spirit of the time, we built our own hardware, we built our own software, which we could port using a new language called BCPL...

David: [00:01:27] Yeah.

Ben: [00:01:27] ...the only portable language we knew. And this was another ... this was our English collaborator who was the expert on that from Rutherford Lab...

David: [00:01:35] Which, if I'm not wrong, is a predecessor to C.

Ben: [00:01:37] Absolutely. In fact, Dennis Ritchie, er.. Ken Thompson, called it the B language.

David: [00:01:43] Right.

Ben: [00:01:44] Yeah. BCPL was invented by an Englishman called Martin Richards at Cambridge. Very elegant language. There was one exponent of it at CERN called Ian Willers, who'd graduated from the Computer Lab in Cambridge and came to CERN with knowledge of this. So he helped us with this. But so .. the idea was this: we would have a computer... each site got itself a computer, they were all different. We had a Norwegian computer. The Brits had a GEC, a General Electric, computer. The Germans had a little IBM computer, not a mainframe. The Italians had a PDP-11. And the Austrians, I forget what they had. The French... the French dropped out...

Ben: [00:02:31] So we needed software, the main software which would read the tape - the data was on mag tape - read the tape and transmit it logically.

David: [00:02:39] Yeah.

Ben: [00:02:40] Okay. And that software was written in BCPL. Then we needed a modem controller. We had the modem and we had the transmitter, we had a dish and we had to have a controller. And that controller we built from hardware ourselves, we had a very, very good hardware designer, and he built it in such a way that it would adapt to each computer so that we had a DMA channel...

David: [00:03:07] Yeah.

Ben: [00:03:07] ... and we had a PIO (programmed I/O) channel, and he would adapt it hardware-wise to each host computer hardware...

David: [00:03:15] Hardware.

Ben: [00:03:17] Hardware, OK?

David: [00:03:17] Right.

Ben: [00:03:18] And we had a fast serial channel to drive and it had to have timing gates on it. So because we'd invented the protocol, a time division multiplexing protocol. You each, you have six sites all using the same channel. So they have to multiplex and they have to tell each other... So everyone got a basic small time window dedicated to them. And then with a dialogue you could say "OK, I need a big transmission window" or "I want to receive now", or whatever. Right. So all this software was programmed in the mini-computer and gated by the controller, and the controller was a Motorola 6800 controlled device with special hardware, special time gating to handle the transmission. And because basically the mini-computer would do a DMA with big data and it would be held in the buffer of the controller until the time when the time was right for that particular thing to take the channel. The channel was, it was like an Ethernet without collisions possible.

Ben: [00:04:31] So all that was developed in a couple of years and I did the controller. I did the software for the little controller working very tightly with the hardware designer, a close friend of mine, and other people did ... Now we didn't use TCP/IP because .. it was actually available, just becoming available here, but we hadn't heard about it, although our Italian partners had heard about it. So they used .. in the second part of the project, by the way, when we got this to work, the big problem was an operational problem. The idea was: CERN would mount a tape, a round tape. The whole tape, read full speed, would be read in 10 minutes. So a mini-computer with the tape spinning would transmit this data in 10 minutes and all the other stations would receive it, acknowledging the receipt. It was like a TCP type protocol.

David: [00:05:27] Yeah.

Ben: [00:05:28] With the time division multiplexing underneath it so that people didn't get in each other's way on the satellite. So the problem was that in order to make this work properly, you had to have an operator at each site in real time mounting the tapes because there was no memory for buffering. No hard disks. Not enough for a whole tape anyway. So it used to usually fail because an operator wouldn't turn up at one site or another site. And so rarely would it really work. And during this, after like three years out of the five years, we realized that a better way to do this would be to have fixed disks on a bigger computer that would have the data on them. And we'd use sort of a batch system. And we would do it, we would use the satellite channel as basically an Internet channel. So in the second part of the project we actually did this, we demonstrated a Local Area Network in three sites using different networks with our Internet protocol connecting them. It wasn't the IP network of the Internet, but it was equivalent. The Italian guys designed it knowing about IP. So in fact, at the end of the project, which was really exciting, we had at CERN, we had .. I guess we had Ethernet at CERN, in England they had a Cambridge ring, which was an early LAN. Okay, a token ring .. an early token ring. In Italy.. Oh no, sorry, at CERN we had CERNET, our local home made network, and the Italians Ethernet. So we had data going across this thing, three different Local Area Networks linked by the satellite channel, and that worked. And that really was quite an advanced thing. So we learned a lot.

*** ERRATUM: The STELLA Internet actually bridged:

(1) a Cambridge Ring between CERN and the Rutherford Lab

and:

(2) a CERNET segment between CERN and CNUCE-Pisa (BS) ***

David: [00:07:18] Especially because you have huge latency if you go through a satellite, and that's how you know if you've really designed a protocol well.

Ben: [00:07:25] 250 milliseconds, yes.

David: [00:07:27] Yeah.

Ben: [00:07:27] So the actual TCP-level protocol was a windowed protocol with big windows to take care of it, which we all designed ourselves. So a big learning .. So in five years it seems to me, using this portable language as well, it was quite amazing.

Ben: [00:07:45] So anyway, so coming back from that was the time to get a new job. People had, you know, they knew this was going on, this project. But it was treated you know, it was run by the old Director. It was his hobby. We had these people, these spoiled people like me who didn't have to run services - they just do the exciting developments - and there was quite a bit of jealousy at times. But anyway, coming back into the mainline, then I had found Les Robertson as a boss, and very soon we decided we needed proper networking - RPC wasn't enough. We were going to find it. And so the choice came down to TCP/IP or Xerox Network Systems (XNS). So I took a look at both of these and XNS was more elegant, very nice, but not properly developed. And you couldn't get implementations of it, whereas TCP/IP - which seemed a bit messy coming from the ARPAnet, you know, was it military or was it not, etc. - implementations were coming all over for everything. And so we chose that very quickly. And then we ran into the problem that, at that time there was huge debate, controversy and conflict around the choice of computer communication protocols. So it turns out TCP/IP, OK, is American, and was being developed in universities, bottom-up and not at all sort of official, but very dynamically being developed dynamically. And the "official" future protocols were the ISO protocols. Seven layers, all that sort of stuff, and they were being developed top-down and they didn't seem to be arriving. And you couldn't get implementations. You could spend a lot of money. You could get a partial implementation on one solution. It wasn't practical, but politically, we were seriously criticized for pushing ahead with this. And however, at the price of signing a document that Les and I had to sign saying we promised never to use these protocols outside CERN, we were allowed to go ahead. And I was even named the official coordinator of these protocols at CERN. And so we were left to do our stuff with very little help or cooperation from the main networking people - more or less harassment, criticisms that we were using connectionless protocols in the IP layer. There was a lot of "religion" - you had to be "connection-oriented" to be taken seriously and so on. Oh yeah. I mean X.25...

David: [00:10:30] Right. I mean that makes no sense... now...

Ben: [00:10:34] And even then it made no sense.

David: [00:10:36] What do you get? You only get the downside from it.

Ben: [00:10:40] You get the downside. And it's not scalable.

David: [00:10:41] Exactly right.

Ben: [00:10:42] So anyway, it was on the level of religion and politics. And the main networking group had all the money, they had the budget for leasing lines all over the place and running proprietary protocols on them. And we were sort of looked down on, but they let us get on with it. And we finished up, yes, we connected all the important machines at CERN. I used to handle the licenses when we had to pay. I would have the licenses and go around, give out the licenses mostly for the VAX-VMS machines and so on. And we had software that we found on each major implementation, and sometimes from universities. On the IBM, we started off with software from the University of Wisconsin and soon IBM came up with their own.

Ben: [00:11:28] Funnily enough, the kicker in the story is this: IBM, who I always felt didn’t want to share, they switched quite early on and decided that TCP/IP was a good thing, before CERN did. And IBM started to encourage their big customers to use TCP/IP and CERN was part of something called, I forget the name of it, but IBM had a certain class of European customer, the very very big centres. Like Bologna, and CERN and so on, running the biggest mainframes, and CERN offered, sorry IBM offered, to pay for TCP/IP lines to the US if people would adopt it. I mean, this was before CERN had done it. So that was the final kicker that made the politicians in CERN switch and admit that after all the ISO protocols were not going to come, and that what we had been messing around with since '84 for five years was a good thing after all, it was working. And that took off very quickly in CERN, very, very quickly. So Tim, then, was sitting inventing the Web, HTTP in particular, and he could see all this happening. And I taught him socket programming. Because what came with the Internet protocols in the software implementations, came an API, called sockets, which all these other products didn't have - standard sockets. No, how do you interface with them? If you had X.25 or so on, how do you interface with them? Well, each of them would have its own method, all the usual stuff. But you couldn't just write a little test program and connect it to our test program on another machine with a different operating system. But with TCP/IP you could. If there was a socket library for a decent runtime library, you could do it and you could do it between VMS, the IBM... And so this .. I was also running around teaching and preaching all this. To me, it was so obviously necessary. And that's why the other stuff never, they never planned the implementation all the way up to the application level. They didn't do that. So they couldn't succeed.

David: [00:13:52] Yeah.

David: [00:00:00] I remember reading about IBM offering to pay for the "pipe" and the bandwith was one and a quarter Megabits per second ..

Ben: [00:00:08] 1.5

David: [00:00:09] 1.5 Megabits per second. 200 Kilobytes...

Ben: [00:00:12] It's called a T1 line,.

David: [00:00:14] Right.

Ben: [00:00:14] Yeah. And it was like gold and it must have cost .. I don't know - it'd be an interesting thing to research just what it cost. But I would think one or two million a year, I would think.

David: [00:00:25] Yeah. And this was a transatlantic line.

Ben: [00:00:28] Yes, it was going to .. again it .. having to remember, might have been to MIT, I forget where.

David: [00:00:36] Probably somewhere in New England. Because that was the closest.

Ben: [00:00:39] Right. Right. Then, of course, things in the States happened very quickly, NSFnet and so on. But no, in those days to get connected to the Internet, you simply rang up Joyce Reynolds in USC. "Hello, Joyce. Can I have some some IP addresses? I'd like class B". She gave us two Class B networks just like that! Before that, because I was only allowed to work on the inside of CERN and I couldn't have external connectivity, I numbered the network myself. I used class A 100, so we were on network 100 all over CERN. Another thing is worth mentioning at that time. The Ethernet at CERN, because there was no IP, wasn't routed.

David: [00:01:24] Oh my God..

Ben: [00:01:24] It was one big broadcast network...

David: [00:01:29] It was a cluster of collisions, right.

Ben: [00:01:31] Yeah. So any single node anywhere on the CERN Ethernet that decided to broadcast...

David: [00:01:39] Oh, my..

Ben: [00:01:39] So in fact, there was a person - who was a friend of mine actually, in the main network group - part of his job was monitoring this thing to see broadcast storms happening. And zap the person. And you could get broadcast storms from various malfunctions. So we didn't have DNS or anything like that. If you needed, you know, we used something called ARP. Every system had a host .. an IP host table.

David: [00:02:07] That's your DNS.

Ben: [00:02:10] But on the Ethernet level we used ARP. So .. we had certain applications which were using a raw Ethernet. Raw. You can take a raw Ethernet packet, then define your own protocol and go with that. So it was a zoo really. It was hard to maintain and of course it grew quite quickly. But before we had IP allowed, properly, there were no routers, no routers doing the main job of routing..

David: [00:02:39] How many machines are we talking? It's like ..

Ben: [00:02:44] Like a thousand. PC's and stuff..

David: [00:02:45] Oh, OK. Well, that's a fair amount of collisions.

Ben: [00:02:49] No, no, no. Now remember, there were switches..

David: [00:02:53] Oh, OK.

Ben: [00:02:55] Were there switches...?

David: [00:02:57] Switches are dumb in that they just forward everything they see.

Ben: [00:03:00] No, no, no, no. Wait a moment. No, there were switches. No, no. It was on each segment, the collisions are on each segment. So a guy on one side of it doesn't see the collisions on the other. But it was a mess.

Ben: [00:00:00] For instance, when the Cray came, when the Cray came in '86, '87, we had to sign this very rigorous license with the Americans, with both the Department of Commerce and the Department of Defense, that we would secure this machine. Okay, so we did use three-level security. The lowest level was: I put the Cray on a secure Ethernet with IP filtering, which I bought from Cisco, which had 20 people working for them at that time. And I'd met the guy from Cisco at a USENIX exhibition in San Francisco..

David: [00:00:35] Cisco had 20 employees?

Ben: [00:00:36] Twenty employees in 1986 - '85 probably I met him then, '85 - and I saw .. the guy came from Stanford, Len Bosack he was called. And I was interested and I went to talk to him. "What are you selling?" "Ah, selling a router". "Oh yeah?" - because in those days IP routing was done in minicomputers, inside your own computer, OK.

David: [00:01:00] Right.

Ben: [00:01:00] Okay. You didn't have a dedicated router, it wasn't necessary. People, you know, they passed the packets. So, very interesting - and he gave me the brochure about it. And this was called "cisco", (San Fran-cisco). And I took that brochure home and I remembered. And it had IP filtering. And at home we were thinking about the Cray and I thought surely, we need - all we do is put this on a filtered Ethernet. And so, you know, only certain people can get into the network. And it was very nicely done: you configured the thing, you know, it could .. you could go out to anywhere and only certain address groups and certain ports, you know, could come in, and so on. So this was wonderful. And I bought the first two Cisco boxes probably in Europe.

David: [00:01:54] Wow.

Ben: [00:01:55] Again, I had a problem with CERN, because CERN was thinking about routing and they had decided they were going to buy their routers from a company called Bridge. I think it was Bridge - anyway it wasn't Cisco. Cisco was American, it was too small, you hadn't heard of them. So .. but I wanted that because it had the filtering. So I managed to buy them in the end.

Ben: [00:02:15] Another story which goes from that is that some years later, a guy comes to see me from Amsterdam and says: "Ben, I'm looking for a router that has X.25 capacity on it so that we can route across X.25". And I pulled my Cisco catalogue off the shelf and said: "Look, they have X.25" - because the US military actually needed X.25

• even in the States there was discussion going on about whether we should use the Internet. So this guy then went back: he was running the main gateway of USENET in Europe, in Amsterdam, and his idea had been subversively to convert from uucp protocol .. You know what was uucp?

David: [00:03:06] Unix to Unix copy.

Ben: [00:03:08] .. to IP. His friends, he had about five or six important friends who were running the nodes in each country of USENET. And all he had to do was give them a Cisco box each, they could interconnect across X.25, which was allowed, you know, so that they could use that, they wouldn't have to use dial-up any more. And away they went. And they did that without telling the PTT's who were ferociously against it. It turned out the Germans, who were the strongest against it, were not actually monitoring what was going over X.25. Because the Germans had threatened: "If you use TCP/IP we'll cut you off" etc.. It was really, it was hardball politics. This was '88 or so. And so I introduced him to the Cisco salesman who rushed off to Holland and sold them the boxes. And that's how the Internet started in Europe without people noticing.

David: [00:04:11] Essentially, you were .. if you ask for forgiveness, not permission, essentially..

Ben: [00:04:16] Absolutely!

David: [00:04:17] .. you would have never been able to get the permission, so you just went away with it and hoped for the best.

Ben: [00:04:21] So that's what he did. Also, it was the Unix club remember also?

David: [00:04:25] Yeah.

Ben: [00:04:25] Because Unix was important because Unix was there, it was a good technology and Unix had - Berkeley Unix - had TCP/IP in it.

David: [00:04:34] Yeah. Yeah. Yeah.

Ben: [00:04:35] So this is what helped. It was a sort of symbiosis. Whereas Unix at CERN was still a curiosity. OK, we'd put it on the Cray. That was a momentous decision, that again. That was my boss, Les Robertson, who'd pushed that.

David: [00:00:00] How did Linux become a thing at CERN? If it's a completely bottoms up...

Ben: [00:00:05] OK. It didn't. Linux came later to CERN. Well, Unix ...

David: [00:00:10] Well, Unix came for sure because...

Ben: [00:00:11] Unix had all its dialects and almost all the big development was done with different flavours of proprietary Unix.

David: [00:00:20] Right.

Ben: [00:00:21] Unix came in 1980. Let me see. I just came back from my sabbatical...

Ben: [00:00:32] Probably '82 we had the first Unix machine at CERN. I should remember because a guy was just talking about it. When I came back from my sabbatical in '78, Unix came a couple of years later. '80, '81...

Ben: [00:00:48] And we got a Berkeley flavored one because we had a Unix guru, a German guy. Very good. And he had a good sense of what Unix to get. And I believe, I was told by another guy who worked on this, that Bill Joy came as well early on and helped CERN with it. So that was just one machine at CERN and it was used for cross compiling, just an obscure thing. It wasn't a mainline thing at all. But Unix came slowly and the big drive was with the Cray.

Ben: [00:01:24] And at that point, because Unix didn't have features for running batch jobs that time. I remember very well when IBM was asked - because we were a big IBM shop. Do you have Unix - we could have a Unix? Yes, yes, we've got Unix. And do you have tape support? So the salesman went away and came back. Yes, we have tape support: "tar".

David: [00:01:48] No, no, no...

Ben: [00:01:50] I mean all that of course we developed with Cray. Cray came and Cray understood about batch jobs and NQS scheduling – job scheduling. And Cray understood you needed to schedule tapes and so on. So they, Cray, did a lot of development with the CERN colleagues.

Ben: [00:02:07] And it was in doing that development around Cray's Unix, they called it UNICOS, between '86, '87 and ... '89 that we understood enough about how you can put these features into Unix to make Unix work on a mainframe. And then the idea was as follows. In the early '80's we had obtained, CN had obtained an Apollo, Apollo DN10000. Have you ever heard of the Apollo? It's very important. The Apollo was the first serious workstation from a little company in Chelmsford - it later got bought by Hewlett Packard. I'll never forget this: the guy came in 19... must have been 1980 and he gave a talk with nice transparencies about this new workstation. And this workstation came on a ring. It had a token ring.

Ben: [00:03:04] There was a 12 megabit token ring called Apollo Domain and these Apollo machines were so advanced they could page memory over the network. So you had virtual memory: virtual network memory. In 1980, '81. And not only that. The guy was asked: "D'you run FORTRAN?"

Ben: [00:03:26] "Ah, yes". D'you run this, that?. "Yes. Yes".

Ben: [00:03:30] And people thought it was just just, you know, a snow job.

Ben: [00:03:36] So CERN... It was very impressive, the presentation. But was it real? So CERN sent one of their very best programmers to Chelmsford and ... to see if this worked. And he took his tape, of course, with the biggest simulation program of that time with him, and went off to run some tests. First thing he found out is he'd brought the wrong tape. He'd brought the IBM version instead of the VAX version and the ifdefs for the VAX would be almost right for the Apollo but the ifdefs for the IBM were not. So you had to edit 100,000 lines of code. OK. Now, you couldn't do that at CERN on any machine. Well, maybe with Wilbur. But it was super difficult to do. Certainly had no workstation of any sort yet or even the mainframe.

Ben: [00:04:28] So the CERN guy thought he was finished. But they said: "What's the problem? He said "I need an editor". "What's the problem?". And not only it wasn't a line editor, it was a screen editor. Yeah, it was a beautiful thing. And in half an hour or so, he'd done this. And then he could compile his 100,000 lines of FORTRAN which no-one thought it would ever do and it did it. And the thing worked.

Ben: [00:04:53] Then CERN got interested. Bitmap display, ... it'd never been seen. Okay it was expensive. So CERN bought a few of these. Anyway by the end of the '80's, they were very well established at CERN, the physicists liked them. It wasn't running Unix, it was running the Domain operating system - it was Unix-like. Later they did .. they were very clever. Later they put shells in.

Ben: [00:05:27] They put a BSD shell and a System III / System V shell as well. So you could run two Unix flavors or Domain - very cleverly done. So it was a great hit, this machine. Anyway, they announced their latest model, the DN10000 it was called: four CPU's and a one Gigabyte disk on it, I remember. And I had this idea: we'd seen these things could run serious computing and I had the idea that why don't we try and run batch physics on this thing and see how fast it would do it? The batch jobs were scripts basically, big scripts and they would need data from, you know, they would need to read tapes and things like this. Now the way the mainframes did this, the Cray in particular, they would "stage" the data. So you'd get a job and in the job control you would see that it was going to need tapes. So it would, if it had never seen this tape before, it would order that the tape be mounted. The tape would be mounted and the data would be read from the tape (or the files you needed, maybe all of the tape) onto a disk - this was called staging. Okay. And then the machine, then the job would go into execution with the computation done against the disk and the tape could be dismounted. Yeah. So it wouldn't be slowed down by that.

Ben: [00:06:55] This model of running jobs was standard in the computer center, like in many centers. So I said why shouldn't we do this with an Apollo? And all we had to do is connect it to the Cray. Now, the tapes, I should tell you, the tapes were all on the IBM. The Cray didn't have tapes. But the Cray had very expensive software and channel connections under the floor between the Cray and the IBM. So the Cray could order a tape to be mounted before a job ran, and that tape would actually be staged, mounted on the IBM and staged across, across a channel or whatever. So why not do this with an Apollo? I mean, my idea was to offer the users a service, and they wouldn't know if their job was being done on the Cray or the Apollo. IBM would be different because the job control was all different. But the Cray was Unix and the Apollo was Unix enough ... So HP liked this idea - we'd proposed this idea to HP that we would try this. And HP gave us the top end machine, probably a 100,000 franc machine, and they gave us one guy for three years of our choice. So we chose a physicist from I forget which collaboration, one of the LEP collaborations, whose contract had just expired, and who was the expert on Apollo because the Apollo machines were being used by the collaborations.

Ben: [00:08:27] And he came with us and he did the necessary liaison with the experiment - we worked with the OPAL experiment. I remember going to the operations manager, a guy called Jean-Claude Juvet, a Swiss guy. "Jean-Claude, I need a place to put this machine, this new machine, and it needs to be pretty near the Cray?". He said: "They've just taken all the big round tape drives away and now it's all on cassette tape. Look, all the middle of the floor is empty. Help yourself". So we took a table. We put it in the middle of the computer centre, right under the visitors' gallery. We put our Apollo on it, and under the floor we ran our Ethernet cable to the Cray. And all we had to do was to use "rsh", (rshell you know, get this tape or whatever). And so you could stage tapes just using the Cray. And it worked like a charm. So anyway, so this thing starts working. And when it was clear that it was working, somebody connected it to the accounting system at the centre. After a couple of months, the Director rings up my boss and says, "What is this?". You know what - 25 percent of the whole centre's cycles were from this. It was called HOPE. HP, Apollo, something or other. It was an acronym. "What's this HOPE thing? That's 25 percent of all the CPU cycles of the whole centre". Cray, IBM etc. So my boss says: "Well, that's er, that's our little project with the Apollo. So he starts asking questions. Then he rings me up and says, "Segal, I've heard your machine, you're connected to the Cray. I want to know exactly what resources you're using out of the Cray for this". He didn't like this idea at all. He thought we were cheating or something. How could one machine do 25 percent of all these mainframes? So I sat down. I worked it out. It was 1 percent or half a percent or something just for driving the network. And so I don't know if I called him back or something, sent him a message. I don't know. I didn't hear from him again. And so that was the beginning of the end of the mainframes. Now, OK, it was easy because we were emulating a Unix, a Cray mainframe. But so we sat down and wrote a proposal. Why don't we have a network of Apollos - or any sort of machines as Apollos were actually rather expensive because they were workstations. You don't need a bitmap display to do crunching. And we wrote this proposal. And it had a better history than Tim's proposal, because it was actually looked at seriously and accepted by the Division Head, David Williams, on condition that we could get at least one of the four LEP physics collaborations to work with us and to give us money. If so CERN, the IT Division, would match that money.

Ben: [00:11:32] So we went round... I've told this story elsewhere, but we went round the four collaborations. One didn't know what we were talking about. The other one, the richest one, L3 they said: "We've got enough computing at present". (They had their own IBM let alone, and a lot of Apollos). The third one said yes, as long as it's a VAX, a VAX...

Ben: [00:11:51] And the fourth one said "Yes". The fourth one had just got a new physicist, collaborator, a woman professor from the University of Indiana. And she had some money because when you join a collaboration you have to give some money. And she even, or she or somebody else , they had SCSI disks that they were going to attach to their own system. But we persuaded them to attach it to our system. And so we built a system - actually Silicon Graphics were the main machines at the beginning - and from starting from '91 it took off. And by '96 I guess we switched off the IBM. The Cray unfortunately got booted out before, even though it was our model and our beloved machine.

Ben: [00:12:38] I know what we were talking about. So let's go back to Cisco. So I ordered the two Cisco boxes to filter the Cray network. That was where Cisco came in. Certainly the first at CERN, maybe in Europe. And then later helped Europe, with USEnet etc... Right.

Ben: [00:12:58] So about the Cray and about security. We'd had to sign this agreement with the Americans. Now it turned out most of the big European centers signed the same things we did, but they didn't do anything about it, like Bologna and so on. They didn't actually implement anything. They just signed. Nobody checked that they were securing the machine or not. But we did. First of all we had the Ethernet filtering, the IP filtering, that worked.

David: [00:13:26] Couldn't you spoof ... it's easy to spoof an IP address in a lab. It has collisions. But you can spoof the IP and bypass that.

Ben: [00:13:36] If someone could spoof an IP address, they could get through up to a point, I suppose.

Ben: [00:13:45] I don't know how strict we were on that. We were never hacked, by the way. But then we had another thing. We introduced a SecurID card so that when you logged on, it wasn't just enough to put a name and a password. You also had to have a one-time password.

David: [00:14:08] Oh, wow!

Ben: [00:14:08] I don't know who it was that discovered this little company called SecurID in the US. They make this card. It was very clever. It would time out and all sorts of nice things. So we bought that code and I had to adapt it. What was so interesting was the kernel of it was beautiful code, the actual code that was doing the encryption, the rest was just rubbish code. Just crap.

David: [00:14:36] That's how it goes.

Ben: [00:14:37] Yeah. But anyway, we could adapt it and we lived with it. It was very unpopular with the physicists. First of all, in principle, because they hated the idea that Iranian colleagues couldn't have one of these cards and therefore couldn't have an account on the Cray. There were certain nationalities even then - Iran was on the blacklist - that we weren't allowed to give accounts to, so that the administration of giving out these cards was very strictly done by CERN.

Ben: [00:15:04] The first time CERN ever did restrict things like that. But anyway for CERN it was very strict. And in fact before the Cray came, there was no physical security in the centre. You could just walk into the CERN Computer Centre with your briefcase and put it next to the biggest machine you wanted that would blow up. No problem. It was when the Cray came that the doors were secured and you had to have little badges. Yeah, that was all due to the Cray. That was in the middle of the '80's. And we had a third level ... I don't know what it was... It was quite tight and... But we weren't hacked as far as we know.

David: [00:15:50] Yeah? Well, that's remarkable. Especially as this is such a highly visible target, especially in Europe.

Ben: [00:15:56] Yeah. There've been hacking attempts, successful hacking attempts at CERN, but not on the Cray. One of the very, very first hacking attempts was done on our VAX system with a famous case: it's mentioned in the book called "The Cuckoo's Nest". Have you ever read that? It's mentioned in there. A friend of mine, again on the VAX team, saw funny activity one evening and he followed the guy. And later he made friends with the guy. Online. And they got him because of that finally. He was hacking various systems. That was quite early days. On the Cray we'd have a look. Yeah, they would check, they would make checks on what what people were running, that sort of thing. It was fairly...

Ben: [00:16:49] OK, that was, so that's how Cisco came here. Another story about the Cisco box. So when the Cray left, (probably 1991 when the Cray went, '91, it went before the IBM, '92 probably) the two Cisco boxes were there. And I took them and I put them under my desk because they were my babies, right? And there they stayed for a long time until one day one of my friends from the operations group came to me. "You know, we've got this museum. Yeah. You know, I have a little museum. And can we have, you know, a Cisco router?". So I gave him - it was Cisco A and Cisco B - I gave him Cisco A, and some weeks later I went to see him, David, David Underhill, yeah, "Where's my Cisco box?" "Oh, it was too big to go in the glass cabinet. We're making a new cabinet for it". And then a day after he came in, he said "Ben, I've lost it. When we realised it wouldn't fit in the cabinet, it got taken and it was put too near the recuperation".

David: [00:17:58] Oh, no!

Ben: [00:18:00] It was just taken and thrown away.

David: [00:18:03] It got recuperated.

Ben: [00:18:05] Yes. So the second one was there. I was furious. The second one was there. So the second one, some time later, I gave to the CERN Microcosm (museum), in the days when the Microcosm did have computer stuff and it was there for a long time. Anyway now it's been... I got it before it was taken away from Microcosm. Now it's in the CERN archives storage ready to be put on show again. So that was the Cisco story.

David: [00:18:32] Yeah, I remember that museum. That was impressive. I visited there a decade ago. But there were a number of devices there.

Ben: [00:18:40] Well, it's less good now. There's a little museum in the IT Department. But there's a girl who's supposed to put that up again. They're very slow.

Ben: [00:18:54] So anyway, those were exciting days.

David: [00:18:57] Yeah.

Ben: [00:18:58] And at the end of that then was that I think we were the first to run serious Unix clusters instead of mainframes. Now you asked about Linux. Now, Linux did not come in early. We used proprietary Unix and we built our SHIFT system using different sorts of proprietary Unixes. For pure crunching you could use... HP had a very fast machine with what you didn't need for a disk server. The architecture was that you had compute servers, disk servers and tape servers. The tape servers would do the interface to the tape robots and stage tapes to the disk servers which were physically separate and the compute servers would compute against the disks. And for that you needed a very fast network. We used network attached storage. We didn't use NFS because NFS had a bad name. It wasn't very reliable. It wasn't fast and so on. NFS Version 4 would have been better. And AFS would have been much too slow. So we built our own network attached protocol. It was called RFIO. We just built file connections and all the tape management software that we'd built for the Cray, we just transferred it.

Ben: [00:20:16] So a job could be submitted, arrive on a compute server, give its instructions that it needs these tapes. It was a huge tape management system. A whole cache system, managed cache system on the disk servers. So you could easily check with the particular files you needed: were they there already? If not some of them, you would stage what you needed. And then the job would go into execution all across the network.

Ben: [00:20:44] Now, I was in charge of the network. Now the network is very demanding because .. At first I remember doing the calculations for this: if you have a system... Our target was to build a system - our first system would be three times the capacity of the centre, the existing centre, right. People would say "Three times! You're arrogant, you're stupid, you'll never do that. Your Unix is rubbish...". So we had this little proposal. And so I did my arithmetic. We have a job which is using ... needs to consume such and such an amount of data, and the data has to be first staged from tape, and then computed against, and probably only you know, once,.. it depends. There are different models. And so I figured out what this network had to be capable of. So obviously it wouldn't work with just one Ethernet. So I designed a network that was actually a hybrid network. So there were three levels of it.

Ben: [00:21:45] There was a very, very fast special network called UltraNet, which even had protocal processing on it. From the application it just looked like an ordinary socket. But actually your protocol overhead was done for you and you could do very fast transfers and they had interfaces for Sun, Silicon Graphics, a few other systems, and the Cray of course. So that was expensive, but we needed that for the core network. And then we had a 100 Megabit network - Fast Ethernet wasn't supported at CERN - they had a horrible expensive 100 Megabit network called FDDI. Very expensive. But there was a group in the network team that liked that and pushed it everywhere. So we had to use that for 100 Megabits. And we had ordinary Ethernet that was 10 megabits. Now in the end all that got replaced by Gigabit Ethernet and now faster than that. But at the time that was very advanced for the mid, mid '90s and early '90s because it wouldn't work without a fast network like that. You couldn't do it.

Ben: [00:22:54] So that was a success and so this OPAL experiment were the first people - they were very happy. And then the other experiments, the ones that had said they weren't interested - they came. And by the end of the '90s, everybody was on that system and there were no more mainframes anymore. And then it was at that point that we said, right, now this proprietary stuff is too expensive. Like for a tape server we used the VAX station or something. It's very expensive. Or like SCSI discs. We learned that SCSI disks were just the same as PC disks. It was the same disk unit with an expensive controller. So we didn't realize, we learned all of this. So we decided we were going to go to PC's. So the politicians said to us, "Yes, you can go to PC's, but you must use NT".

David: [00:23:50] Windows NT?

Ben: [00:23:51] Yes, Windows NT.

Ben: [00:23:54] So the smartest, one of the smartest programmers (who was the same guy who is now the Head of the Division, the IT Department), spent almost a year trying to make a system running NT. And he failed.

David: [00:24:09] Well, yeah, I mean it's not... nothing wrong with NT. That is not Unix like ...

Ben: [00:24:13] We needed Unix. But he tried...

Ben: [00:24:16] At that point we were allowed and then we went to Linux. We didn't know a lot about Linux. And again, the management had been quite rude about Linux. I'd maybe told you that the remark was: "Yeah, if there's a bug, you have to fix it yourself".

David: [00:24:29] All right. Yeah.

Ben: [00:24:31] As though, oh, we'd be spending all our time fixing bugs. They didn't know about communities on the network. We did know about that...

Ben: [00:24:36] And we decided we needed to use RAID, RAID technology on the disks, which all the expensive disk servers had. We found you could get a nice PCI interface with RAID on it, as long as you had the driver for Linux, OK. And not only that, but you didn't have to use SCSI. You could use commodity disks, exactly the same mechanically. And it cost a quarter of the price or something. And so, with a PCI interface and a good driver and a RAID interface and these cheap disks, we could solve the disk server problem.

Ben: [00:25:17] The tape problem. Well, the tape interface was some SCSI one, I don't know what it was. So we did all this. And for the network, we needed a Gigabit Ethernet interface. And there wasn't one, OK. So I hired an Internet guru, kernel guru, Danish guy called Jes Sorensen you may have met. And he wrote the first Gigabit Ethernet driver for a very nice Gigabit Ethernet board. A very very nice one. And we kicked off with those, as always. So we were really, you know, pushing on the envelope at that point and we got this thing going. So by the end of the '90s, we were all basically all Linux. And it's never looked back from there.

David: [00:26:03] Yeah, that was my experience in 2005. I came in and it was I don't know about it. It was based on RedHat, perhaps.

Ben: [00:26:11] RedHat. Again, they didn't have the real courage to go Fedora or something. They wanted support, wanted support.

David: [00:26:17] I think it does make sense ultimately - that's RedHat recently bought by I forget who, but RedHat's immensely successful as a company providing support for Linux. It's an example of a win-win situation. Right.

Ben: [00:26:29] So we've got the Linux expertise now. So that was 20 years - no, 25 years - since I would say things like: "We should run Unix on everything" and people would say: "Shut up, you're arrogant". So anyway, that's some of the more recent stuff.