Rust Creator Graydon Hoare Talks About Security, History, and Rust – InApps Technology 2022

We’re approaching the 9th anniversary of the day Graydon Hoare (and numerous contributors) first revealed to the world the newly-designed Rust programming language. So we thought it’d be a good time to assess our current landscape.

Hoare graciously agreed, sharing his thoughts on everything from the state of systems programming, to the difficulty of defining safety on ever-more complex systems — and whether we’re truly more secure today, or confronting an inherited software mess that will take decades to clean up.

And of course, we also took a moment to look back at the history of Rust…

What’s your assessment of the state of the systems programming today?

To my eyes, the “systems niche” seems reasonably healthy today, at least compared to the first couple of decades of my career. Certainly, it’s easier to sell language work that’s concerned with performance or correctness now than it was a while back. Lots of good new languages, lots of dialogue between industry and academia.

So when it comes to safety, are things improving (and how much of that is due to safer programming languages)?

I think safety is improving in some ways (including, sure, deployment of safer languages; though many other technical measures also help) but in other ways, we’re just treading water, or getting worse. Partly this is because we keep building more and more complex new computing systems: complexity beyond comprehension means we often can’t even define safety, much less build mechanisms that enforce it. And partly we just have such a massive backlog of existing incorrect software in the field that computers really only continue to function on a day-to-day basis with extensive human supervision, on-call ops teams, tech-support teams and maintenance coders and so forth. To say nothing of bad actors: there’s so much vulnerable software in the field that they can always find an exploit. Like much of the legacy of the 20th century, there’s just a tremendous mess in software that’s going to take generations to clean up, assuming humanity even survives that long.

Are there any additional safety features that you’d still like to see brought to the world of systems programming?

There’s still lots and lots to do safety-wise. Rust just picks some of the lower-hanging fruit. Plenty of language tech and plenty of non-language tech. I’d need a much bigger word-count to give a real answer here but for a short list focused on languages, let’s say: information flow control systems, effect systems, refinement types, liquid types, transaction systems, consistency systems, session types, unit checking, verified compilers and linkers, dependent types.

The challenge isn’t really in coming up with features that help — there’s quite a backlog of good ideas from academia — it’s in embedding such features in a balanced, niche-adapted language that’s palatable enough to industrial programmers to be adopted and used.

I’ve already seen calls to start replacing C code with Rust. But do you have any broader thoughts about how we should be approaching our backlogs of memory-unsafe legacy code?

Rust rewrites of C or C++ code can work if there’s sufficient motive and budget, and if they’re otherwise appropriate to the technical context. But that’s a lot of if’s. Rewriting can also be focused, partial, one library or subsystem at a time. The various modules of Firefox being rewritten in Rust are a case in point: it’s now a hybrid C++-and-Rust program.

But I think it’s not so likely that we’ll see rewrites happen at a rate much faster than the natural turnover of software, as some old programs are always being replaced at some rate. Some of the next generation will be Rust, but that’s about all I’m expecting to see: most programs in maintenance mode don’t have the budget to be rewritten even in their original language, much less a new one.

This could theoretically change if there were a change in the regulatory regime. Like if we found ourselves facing a future where the law treated memory-unsafety as an obvious form of negligence on the part of software manufacturers or something. But we’re not in such an environment now, so it’s mostly just motivated by people’s individual evaluation of the importance of the problem, and that’s usually quite constrained by context.

Last year you wrote on Reddit that members of the Rust and Swift teams “know each other, talk to one another, trade ideas and implementation insights, and generally coexist peacefully; and they’re both fantastic groups to work with.” You’re perhaps uniquely qualified to satisfy the idle curiosity of developers who may be wondering: What exactly goes on in the day-to-day development of a major programming language?

I’m not actually on the Swift team anymore. But the day-to-day work on a language (and its compiler) is a mix of lots of tasks. By volume, the majority is programming, just like in any other program: fixing bugs and making enhancements to the compiler. The bugs are things like incorrect safety checks, mis-compilations, compiler crashes, bad diagnostics, slow compilations, slow generated code, etc. Enhancements are things like new language features, new platform support, new safety checks, new optimizations, etc. And like any large program there’s often a lot of just plain refactoring, cleanup, documentation and infrastructure work to do inside a compiler. Often some people on a compiler team are doing language-design work, but that tends to taper off as the language matures. Often there’s a lot of liaison with other groups doing work that needs to coordinate with the compiler: the linker, the kernel, the debugger, the libraries, etc.

So what was it like working on the Swift team?

As with my time on the Rust team, I enjoyed it immensely and felt extremely privileged to work with such bright, kind, dedicated colleagues. Both teams are full of really outstanding engineers, and just genuinely nice people.

It’s interesting that Rust made the leap from a research project to a widely-used programming language.

I’m not sure it’s made the leap yet, but I should also point out that Rust wasn’t intended to be a research project. We got put in “Mozilla research” as a sort of organizational matter — a team with a few more academics than usual — but the idea was always to ship something that would be usable in mainstream contexts. Its design is quite constrained in a lot of ways in order to accommodate those requirements!

You also wrote last year that Rust “will always have a special place in my heart given the unusually intense effort I put into its first seven or so years.” So how would you tell the history of Rust?

Hard to summarize! I’m not good at storytelling. I would say we got lucky in a lot of ways: that Mozilla was willing to fund such a project for so long; that Apple, Google and others had funded so much work on LLVM beforehand that we could leverage; that so many talented people in academia, industry and just milling about on the internet were willing to volunteer to help out. It was really a group effort at every level. Perhaps a compelling narrative to the story is that of cross-disciplinary and cross-organizational collaboration.

It’s been amazing to watch; I certainly had no idea it would work out as well as it did.