Ruby Creator Yukihiro Matsumoto on the Challenges of Updating a Programming Language – InApps 2022

A recent presentation from Yukihiro Matsumoto, the creator and chief designer of the Ruby programming language — and Chief Architect of Ruby at the cloud platform-as-a-service company Heroku — offered a clear example of the thoughtful care with which Matsumoto leads his Ruby community. The 30-minute presentation, given at the NoRuko “virtual-only Ruby unconference,” also offered a glimpse of exciting new developments that are just over the horizon for the language.

“The last four or five years, I have been talking about the future of Ruby 3,” Matsumoto told his audience, “for a long, long time. Maybe you are sick of hearing about Ruby 3 from me,” he joked humbly, listing the other conferences where he’d covered the same topic. “But this year — this year, this is real,” he said, confirming that Ruby 3 will finally be released this Christmas, Dec. 25 (as part of an ongoing tradition of releasing Ruby language upgrades on Christmas day).

“If everything is okay. Unless something very, very bad happens, we will release Ruby 3 in December of this year.”

The Designer’s Dilemma

Hundreds of viewers were watching the livestream of Matsumoto’s “Ruby 3 and Beyond” from around the world — from India to Indonesia, and from Brazil to Berlin — leaving dozens of enthusiastic comments in the event’s chat window. They’d gathered to watch the man who’d spent the last quarter of a century shepherding the development of Ruby, beaming in from in front of a shelf of books. “Matz” had said in the past that Ruby 3 would be available “on the eve of Tokyo’s Olympics,” but since the Olympics were postponed until 2021, some core developers had wondered whether Ruby 3 should also be postponed a year.

“But after some discussions, we decided to release Ruby 3 this year, because as an open source community, we have to move forward. We have to keep moving. The open source community in general cannot stop evolving. Otherwise, it will die…”

Then Matsumoto poignantly described the dilemma faced by all language designers. “Everyone likes new things — so as a programmer, as an engineer, I like new things. I’m also a language geek, so I like to study about the new programming languages — like Elixir, Rust, Go, whatever. They’re very exciting.” But that also seems to create a kind of pressure for change. “Ruby is kind of old,” Matsumoto continued. “It was released in 1995.” He adds later that “We are mere mortals. We made mistakes. So we language designers want to fix the mistakes in the past.”

Yet at the same time, “no one wants pain.” With all the web applications already using existing versions of Ruby, there’s always a danger of forcing developers into a difficult upgrade, and “No one wants pain. In that sense, Ruby should be stable. It’s kind of a contradiction, right?”

“So language design, the language plan — it’s very hard to overcome this kind of contradiction. That kind of contradiction is from our heart.”

He provided examples of other programming languages confronting similar dilemmas — including Python, PHP, and ECMAScript — noting how many years its took for their communities to embrace the changes. With Python 3, the community took over a decade to move on, while PHP 6 “was even canceled. They just gave up on making a big change before the release — so they started PHP 7 from PHP 5. EMCAScript 4 was also canceled.”

People might leave the community if changes aren’t made, Matsumoto explained. “They stop using Ruby, just because they’re bored.” But he’s also concerned they could leave the community if an upgrade to later versions involves too much pain, leading them to decide “maybe our next project will be written in, say, Rust, Go, or whatever …”

“So as I said, designing a language is hard, but we need to move forward anyway, to create the future.”

Making Ruby Great

Then he outlined his plan for keeping compatibility while “making Ruby faster and making Ruby programmers more productive.” He identified three clear and specific goals that characterize the upcoming release of Ruby 3: being fast, being concurrent, and being correct.

To make Ruby faster, the development team will start by improving its just-in-time compiler (which generates native code). For concurrency, they’ll avoid the solutions from other languages to try one of their own. While Node.js added promises of completion from asynchronous operations, and ECSMAScript 7 added controlling mechanisms with the keywords async and await, Ruby 3 will instead implement its long-awaited async i/o “fibers” as a better way to control asynchronous threads. “Since the context switch between fibers is fast, we can improve the performance.”

Another reason they expect it to speed up performance: It’s being developed by the creator of Ruby’s falcon application server (which Matsumoto describes as “brilliantly fast.”) But that’s not the only way they’ll speed up performance. They’re also implementing “Ractors” (“Ruby Actors”) — similar to the way JavaScript offers background “web worker” scripts. “Each ractor can run in parallel,” Matsumoto explains, taking advantage of the parallel processing now possible in modern multicore systems.

And the third goal of a more-correct Ruby will come from checking errors earlier, courtesy of a type profiler which will generate Ruby signatures (likened by Matsumoto to the type-describing d.ts files in TypeScript). Ruby 3 will ship with type signatures for its core libraries, available both for type checking as well as for enhancing future IDEs.

But it will have its limitations. “We don’t pursue completeness nor soundness of the type systems, because, you know, Ruby is Ruby,” Matsumoto explained. “Ruby is basically dynamically typed … That means that even though we have some of the gradual types, we cannot do any sound type checks, so we just give up. … We are not going to add a type declaration in the syntax.”

Three New Features

But there are syntax changes coming in Ruby 3. “We are making Ruby even better,” Matsumoto said, touting three new features. The first is an intricate pattern matching syntax, “so you don’t have to deconstruct the arrays and hashes. Instead of that, you can write the patterns … it’s quite handy and convenient.”

The second new feature allows assigning values to a variable on the right side of the equals sign. He called it “unnatural” for our eyeballs — and our IDEs — to always confront expressions beginning with the name of a variable, even when it will be assigned to the resulting value of a much longer chunk of code. “If instead we can right-hand the assignment … That’s natural … We don’t recommend it everywhere. You don’t have to replace every assignment with a righthand-side assignment, but in some cases, this is very convenient.”

And the third new feature coming in Ruby 3 is numbered block parameters, “so you don’t have to worry about naming the temporary local block parameters.”

But Matsumoto’s next slide read, “And Beyond …”

He began by saying “stability matters,” so he’s not predicting any big syntax changes after Ruby 3. Instead, they’ll focus on working on improving Ruby’s existing supporting tools, like Solargraph for the language server protocol, Sorbet for static type-checking, and Rubocop for the code linter. “Those tools proved that better tools enable a better user experience. So we have to keep moving. I think we need more tools, and then we need to improve those tools.” Matsumoto also wants to see better type checkers and formatting tools, or maybe better performance tuning and debugging tools. And he also has some other ideas on how to make Ruby faster — for example, a second lightweight just-in-time compiler that sits above the virtual machine — possibly MIR or DynASM.

Smaller Ruby

But Matsumoto ended his talk about what lies beyond Ruby 3 with what he called “One more idea. This is just a crazy idea. But I’m thinking about a smaller subset of Ruby, which is simpler and then more strict, and hopefully faster or easier to optimize.”

It’d be downward compatible – that is, a subset of Ruby that can run in all existing, current versions of Ruby (though not every version of Ruby could be run in Smaller Ruby). And explaining his rationale, Matsumoto puts up a slide that says “Smaller Ruby Can Run Faster (Hopefully).” He laughs as he reads it out loud, then adds “It’s just an idea. I don’t know yet. We have to keep compatibility … But if we stay in this kind of subset of Ruby … this is kind of inflexible, but much easier to optimize …”

To emphasize the point, he puts up a slide that says “It’s Just an Idea.”

“But I have many of those crazy ideas, and we try and experiment with those things, one by one, to make Ruby great. ”

WebReduce