In the dim reaches of Unix history, the first shell was written. It attached file descriptor 0 as a pipe from the TTY device to a process. That became “stdin”. File descriptor 1 is a pipe from the process out to the TTY. That’s “stdout”. I don’t know when FD 2 became “stderr” but it was early.

When you write a Unix program, you don’t have to open these file descriptors. They’re opened by the parent process, before it uses “exec” to load the new program’s code. So by the time “main()” is called in the child program, FDs 0, 1, and 2 are already connected.

For back end services, we kind of abandoned stdout for a long time, in favor of logging frameworks that wrote output into files. Then we added log scrapers and aggregators to gather those logs on a server.

That looked like this:

Recently, stdout has had a bit of a renaissance with the advent of sidecars. Before your application starts (usually in a container now), the container platform connects a pipe to FD 1. The other end of that pipe goes to a socket which is connected to a “sidecar”. The sidecar reads from the socket and passes the data along to a log collector.

So now instead of this linkage that requires a logging framework inside your application, you just use builtin functions like printf() or System.out.println(). You still have to format the log line, which might want a library function in your app. But now different libraries that each spit to stdout can compose nicely. We’ll leave it up to the log collector and indexer to ingest different formats.

Let’s pursue this idea further. What else could we simply provide to an application by hooking up file descriptors before executing it?

Messaging Topics

When an application wants to use messaging, it has to include a client library that knows how to connect to the messaging service. That requires authentication so the application has to manage credentials to supply to the client library to connect to the messaging service.

Those credentials are not part of the application code base, they have to get mixed in by some build or deployment step.

Because the application has to include a client library, the application becomes specific to a particular messaging product.

What if we said “fd 3 and up are for messaging topics?” Each FD could be bound to a topic as either input or output. The application would just use “send” and “recv” socket operations on those FDs. (If we used “read” and “write” file operations on the FD, we’d have to figure out how many bytes to read. What we really want is “a message” as a unit.)

It would then be the responsibility of the runtime platform to supply “pipes” that connect those FDs for the application to the actual messaging infrastructure. We would certainly implement that connection via sidecars again.

With this approach, the application no longer needs a client library. The platform would be responsible to provide some messaging ability. Applications that need precise control over acknowledgements might not be able to use this but simple applications that don’t worry about batching or distributed transactions could go a long way with basic send and receive operations.

Databases

Similar situation with databases. Why do we need all kinds of specific wire protocols? How about a file descriptor that is connected directly to a database. Write to “stddb” and the DB gets it as a SQL statement or query. Read from “stddb” to read results.

Now the application doesn’t need driver libraries in it. Nor does it need to manage credentials. That would be part of the platform configuration for the application, so we’re separating concerns in a different way.

Other Uses

What else could we simplify if we renew the idea that a program’s environment is set up by the runtime that launches the program?