This is part of the talk “UNIX archeology” I gave at OpenFest 2023.

If you, dear reader, are a vim user, you’ve probably seen this keyboard:

It explains a lot about our beloved text-editor. CTRL is just where it has to be in the first place, the Esc is so near and the : symbol we use for commands is not hidden behind a modifier like nowadays. But why the arrows?

So, one might say: “Well, because Bill Joy created vi using this ADM-3A terminal where the arrows just happened to be located there”. And that person would be absolutely right. But why were they located there?

To be fair, if you look at other terminals from these days, where to put the arrows was not really standardized. Not even the triangular shape, we’re so used with today. Most machines would put them in one line, but in different order. For instance, some would go with [↓] [↑] [←] [→]. So far, the answer that they just had to put them somewhere still holds.

ASCII

It was not random, though. You see, in ASCII characters are not just put at some point on random. There are fairly good decisions taken there that go mostly unnoticed today as people no longer work directly with ones and zeroes really. If you want to get the code for an upper-case letter you should add 64 to the position of that letter in the alphabet. If you want it to be lower-case, add 96 instead. That’s pretty clever, because it all boils down to flipping a bit:

This first bit is always 0, as ASCII is a 7-bit character set. If the second bit is up, we are dealing with a letter. If the third is up, then this letter is a lower-case. It’s that simple. But look what happens if we set both of those 2nd and 3rd bits to zero:

When pressing any of those four keys, the terminal has to take two decisions:

• If [Shift] is pressed, set the 3rd bit to 0
• If [CTRL] is pressed, set the 2nd and 3rd bits to 0

But wait, there’s more

Legacy we still observe from this terminal doesn’t end here, though.

~/

Take a look at the top right button. Yep. That’s why we are shortening the path to user’s home folder as ~/.

Regular expressions

If you ever wondered why ^ is the beginning of line or “home” position in regular expression dialects, well… now you know :)

This is part of the talk “UNIX archeology” I gave at OpenFest 2023.

In modern contexts, /usr is essential for housing the majority of user utilities and applications. This includes system-wide software and files like standard programs, libraries, documentation, and much more. Unlike the root filesystem, which contains the minimum necessary for booting and repairing the system, /usr provides the bulk of the operating system’s functionality.

However, have you ever thought why do we have seemingly duplicated with / directories in there? For instance, there is /bin and /usr/bin, /lib and /usr/lib, and lately we’ve started symlinking one to another. Surely, we could’ve just avoided the need to have them both in the first place. But why are they there?

The Filesystem Hierarchy Standard (FHS) is a reference describing the conventions used for the layout of Unix-like systems. Latest release defines its purpose as:

/usr is the second major section of the filesystem. /usr is shareable, read-only data. That means that /usr should be shareable between various FHS-compliant hosts and must not be written to. Any information that is host-specific or varies with time is stored elsewhere.

Large software packages must not use a direct subdirectory under the /usr hierarchy.

Actually, we only found out that we could mount /usr in read-only mode and even re-use it between a bunch of machines, decades after /usr was part of Unix.

no space left on device

Unix was created on a PDP-7 machine in 1969 and later migrated (which included rewriting a bunch of things, but that’s a story for another time) to the much more powerful PDP-11 , which had one RK05 disk pack (1.5MB):

Arnold Reinhold | CC-BY-SA

That was all. The entire / of Unix was on one of those drives, less than 1.5MB. Eventually it grew, with its userbase, and they needed another one. It was mounted into /usr in order to contain user home directories. There were /usr/ken, /usr/dmr, /usr/bwk, etc. However, multiple users were writing useful tools which they wanted to share and thus /usr/bin has been created. Eventually man pages started getting into shape, fonts started being interesting, people experimented with graphics and they all meant to be shared. The law was that nothing on that drive should be crucial for the machine to boot. For instance, you could not place the mount command in there, because something had to mount /usr in the first place. This limitation no longer exists thanks to initrd and initramfs.

Eventually, this drive grew bigger as well and they needed a 3rd one. It was mounted into /home and all of the home directories have been moved there, leaving /usr only for shareable files. That meant whole 3MB solely for the operating system. Hell, yeah!

And then we just kinda gone with it. Once Unix was made (not quite) generally available and the idea to split the OS into multiple drives was still legit. This was no longer the case only decades later. Why going through a revolution for that? Instead, people just made up rules and conventions on how to deal with /usr.

This is part of the talk “UNIX archeology” I gave at OpenFest 2023.

Hidden files are fairly popular these days, especially in our home folders. We store configuration, cache, logs and whatnot in there. Here’s what’s going on in the home folder on the machine I’m writting this right now:

❯ find ~ -name '.*' -type d | wc -l
5768
❯ find ~ -name '.*' -type f | wc -l
30573

But why?

Suppose you’re working on this new cool OS and receive it as a feature request: “We should introduce hidden files”. How would you do it? Knowing that files generally need attributes, why not create a new one for it? Exactly what Microsoft has one for their cool OS:

But that is not how we do that in Unix. Instead, soon after the beginning of time, in Unix hidden files are those which name starts with a dot. Looks odd when you think about it, trying to ignore the fact you’re so used with them. But why?

It’s not a bug, it’s a feature!

Rob Pike wrote an explanation on Google+. Even though Google killed that service long ago, The Wayback Machine has a record of that :

A lesson in shortcuts.

Long ago, as the design of the Unix file system was being worked out, the entries . and .. appeared, to make navigation easier. I’m not sure but I believe .. went in during the Version 2 rewrite, when the file system became hierarchical (it had a very different structure early on). When one typed ls, however, these files appeared, so either Ken or Dennis added a simple test to the program. It was in assembler then, but the code in question was equivalent to something like this:

if (name[0] == '.') continue;

This statement was a little shorter than what it should have been, which is

if (strcmp(name, ".") == 0 || strcmp(name, "..") == 0) continue;

but hey, it was easy.

Two things resulted.

First, a bad precedent was set. A lot of other lazy programmers introduced bugs by making the same simplification. Actual files beginning with periods are often skipped when they should be counted.

Second, and much worse, the idea of a “hidden” or “dot” file was created. As a consequence, more lazy programmers started dropping files into everyone’s home directory. I don’t have all that much stuff installed on the machine I’m using to type this, but my home directory has about a hundred dot files and I don’t even know what most of them are or whether they’re still needed. Every file name evaluation that goes through my home directory is slowed down by this accumulated sludge.

I’m pretty sure the concept of a hidden file was an unintended consequence. It was certainly a mistake.

How many bugs and wasted CPU cycles and instances of human frustration (not to mention bad design) have resulted from that one small shortcut about 40 years ago?

Keep that in mind next time you want to cut a corner in your code.

(For those who object that dot files serve a purpose, I don’t dispute that but counter that it’s the files that serve the purpose, not the convention for their names. They could just as easily be in $HOME/cfg or $HOME/lib, which is what we did in Plan 9, which had no dot files. Lessons can be learned.)

This is part of the talk “UNIX archeology” I gave at OpenFest 2023.

Have you ever thought why the 79 characters limit per line is so popular in programming and pretty much nowhere else? Python with their notable PEP-0008, the Linux kernel, PostgreSQL and the GNU project, just to name a few. Sounds like an arbitrary one. But why?

tty

People tend to believe that this is because terminals were able to print up to 80 symbols per line and this just got carried over. The most popular one being VT100:

Jason Scott | CC BY 2.0

To be fair, not all of them did, but VT100 is iconic enough to inspire such a tradition. It was able to operate in two modes - 80 columns by 24 rows or 132 columns by 14 rows. So, we might as well agree with that and call it a night. But why?

IBM 80-column punched card

Okay, so seems like VT100 and some of its predecessor were not the first ones sticking with 80 symbols. Here’s the IBM 12-row/80-column punched card format which came to dominate the industry:

Pete Birkinshaw | CC BY 2.0

This makes sense. The EOL is an actual characters and this is how we’ve ended up with 79 characters limit, depending on whether you’re counting the EOL as one. So, this makes sense to be the initial reason for that limitation - the most popular punched cards were limited to 80 symbols. But why?

Initially, punched cards allowed far less characters per-line in order not to tear the card with too many consecutive holes. In fact, way before that they weren’t used to store lines of text in the first place. The only thing that never changed was the size of those cards.

Hollerith tabulating machine

Adam Schuster | CC BY 2.0

The Hollerith Tabulating Machine, an ingenious invention of the late 19th century, marked a pivotal moment in the history of data processing. Conceived by Herman Hollerith, this revolutionary device was inspired by the challenges of efficiently processing vast amounts of data from the 1890 U.S. Census. The machine’s core innovation lay in its use of punched cards, a concept borrowed from the Jacquard loom, to store and sort data.

Each punched card represented a unique data point, with various patterns of holes encoding specific information. When fed into the Hollerith machine, a series of mechanical and electrical systems interpreted these holes, tabulating and summarizing data with unprecedented speed and accuracy.

Yep, U.S. Census. Each card did NOT encode text, but the details for each counted person. Here’s what it looked like:

If you wonder what any of this means, early computers have a great article about 1890 Census: The Meaning of the Hollerith Card Codes . Some of the entries here might not be approved by today’s standards. So, the cards for that machine were that big and every single machine dealing with punched cards, kind of just went with it. But why were the cards for that machines so big?

U.S. dollar bills. See those boxes next to the machine? They were used to transport bills which before 1920 were kind of huge. This way, you don’t get to manufacture new type of boxes and new type of paper cutters for those new special cards.

So, the next time somebody asks you why they should abide to that 79-characters per-line rule, you can tell them the old U.S. dollar bills are to blame. That’s why!