Using Rust for an Undergraduate OS Course

Perhaps the most controversial decision about this course was the choice of Rust, a very new and immature programming language being developed by Mozilla, as the primary language for course assignments. In general, I am very happy with how this worked out, although there were significant drawbacks to using such a new language and student opinions on this were fairly mixed. Next, I'll describe more why I decided to use Rust and what impact it had on the course. (For more on the overall context and design of the class, see the Course Wrapup.)

Unacceptability of C

The default language choice for Operating Systems courses is C. Nearly all (at least 90% from my cursory survey, the remainder using Java; if anyone knows of any others, please post in the comments!) current and recent OS courses at major US universities use C for all programming assignments. C is an obvious choice for teaching an Operating Systems course since is it the main implementation language of the most widely used operating systems today (including Unix and all its derivatives), and is also the language used by nearly every operating systems textbook (with the exception of the Silberschatz, Galvin, and Gagne textbook which does come in a Java version). To paraphrase one syllabus, "We use C, because C is the language for programming operating systems."

C is an obvious choice, but its also an obviously bad choice. C was a great language when it was designed in the late 1960s, to make up for not having a machine powerful enough to run languages like Algol 60 which were already available, but too complex to compile without a huge (by 1960s standards) amount of memory.

An example of a design decision made to save memory in C, is the use of = as the assignment operator (previous languages, such as Algol, used an arrowlike two-character sequence, :=, which makes the directional nature of assignment clear). From kindergarten on, people learn the mathematical meaning of the = symbol as a Boolean operator that is true when the values on both sides match. Using it to mean assignment is confusing and misleading, but was a "good" decision for C since it saved one byte for each assignment in program text. Since assignments are much more common than equality tests, using == for equality and = for assignment saves bytes in source code files, which was a lot more important than providing a coherent model or understandable notation for Dennis Ritchie and Ken Thompson in the 1960s.

Computers have changed a bit since 1972 - our programming tools should too!
(We should still study and learn from history, though.)

By the time students make it to cs4414, they have recovered from the atrocities of = meaning assignment, but many others don't make it over this (unnecessary) hurdle and suffer greatly because of it, if they have the misfortune of starting out with a programming language that uses C-like syntax.

More important problems with C for teaching an OS course are its lack of type safety and memory safety, and its lack of any intrinsic support for concurrency. It was the correct decision to not include these things in C for the purpose for which it was designed in the 1960s and 1970s, but there have been a few advances in programming languages, compilers, and architecture since then, and the relative costs of things have changed, as well as the increased importance of security and robustness.

C may be the language of operating systems today (and since the 1970s), but it is also the language of buffer overflow vulnerabilities, dangling pointers, double frees, memory leaks, undefined behavior, and race conditions. To me, it would seem inconscionable to have students use C in a way that is likely to lead to programs riddled with security vulnerabilities and robustness problems. On the other hand, actually teaching students to use C in a responsible way would probably require at least the full semester as well as use of many other tools¹, including some that are only available commercially.

As far as I could tell, none of the OS courses that use C had any time for teaching students how to write reasonably safe C code (and none of the C-based OS textbooks I looked at made anything beyond superficial attempts to address safe programming practices in C), which is understandable given how much time it takes to learn the other topics in an OS class, but from my perspective, seems reckless and irresponsible. A C programmer who knows how to hack on an OS kernel, but doesn't know how to avoid memory leaks, double frees, and undefined behavior, should not be unleashed on an unsuspecting code base!

Alternatives to C

Having eliminated C as an irresponsible choice for teaching an OS course, I had to select another language. (I briefly considered not using a particular language, but since it seemed important to provide starting code to have interesting assignments and to cover some things in class using a particular language, this didn't seem like a viable option to me.)

I considered five possibilities: Java, Python, D, Go, and Rust.

Java. Okay, I didn't really consider Java, since I personally find it to be unpleasant to program in Java, but need to mention it since it is the only commonly used alternative to C for OS courses (some examples: University of Massachusetts CMPSCI 377: Operating Systems, University of Texas: CS 372: Operating Systems). The advantage of using Java over C is that it provides type safety and automatic memory management, so does not suffer from most of the security and robustness problems in typical C code. It does not, however, provide enough low-level access to system resources to adequately cover many important operating systems concepts, and is not a language anyone would use to write an OS kernel. Java also suffers from lack of useful expressiveness, only finally and awkwardly adding lambda expressions in Java 8 (expected to be released in March 2014, neary 19 years after the initial Java release!)

Python. Unlike Java, Python is a quite tasty and elegant language, and has the advantage that many complex algorithms can be expressed clearly in a few lines of Python code. When I was on leave at Udacity, I'd had some discussions with Matt Welsh about teaching an open on-line systems programming course using Python, and he came up with a plan for teaching many of the concepts needed to build scalable, robust, distributed systems using Python. It seems possible to do a good job teaching some of the topics in an OS course using Python labs, including distributed systems and scheduling, but, like Java, not well-suited to exploring lower-level topics like memory management or a language one could imaging using to implement an OS kernel.

D. D is a programming language designed as a successor to C++ that has been around since 2001. D has some very nice features: it provides reasonable support for functional programming, static types with type inference, and some concurrency constructs. The biggest disadvantage of D compared to Rust is that it does not have the kind of safety perspective that Rust does, and in particular does not provide safe constructs for concurrency. I prefer Rust's memory management options, although D provides a fairly similar combination of garbage collected objects and objects with explicit memory management, it does not provide any of the safey guarantees or lifetime model that Rust does. The other argument against using D is that it has been around more than 10 years now, without much adoption and appears to be more likely on its way out rather than increasing popularity. This shouldn't be a major factor for an academic course, but I'd much rather be teaching something that is in an early phase of exponential popularity growth rather than something that appears to be declining.

Go. Go is a systems programming language first released in 2007 that was initially designed by a team including Ken Thompson (C co-designer) and Rob Pike at Google. Unlike D, it seems to have a lot of momentum behind it, no doubt partly due to Google's support. There are some preliminary, although tastelessly named, attempts to write operating systems in Go, but the intertwining of the runtime with the compiler makes this quite difficult in Go (in comparison to Rust where the runtime can be easily separated). I'm not aware of any undergraduate operating systems courses that have used Go, but MIT's graduate 6.824: Distributed Systems course and CMU's 15-440: Distributed Systems course used Go. Go is a fairly new language, but mature enough to have a stable, industrial-strength compiler with great performance, and good documentation (and mature enough to have an idiom guide). Go is also similar enough to C to be easy for C programmers to learn and has no major learning curve hurdles.

So, Go definitely has a lot of things going for it, and initially I planned to use Go for cs4414 until learning about zero.rs. After looking into Rust more, it seemed like Rust had several key advantages both as a systems programming language in general, and as a primary language for teaching an operating systems course.

Reasons to Use Rust

Rust is a new systems programming language being developed at Mozilla. It is primarily driven by the needs of the Mozilla's Servo project to build an experimental multi-core browser engine. Rust adopts a C-like syntax, but is much less tied to C's semantics or being easy for C programmers to learn than Go or D, and its overriding design goal is to support safe concurrency in a practical way. The first public release was in January 2012, so Rust is a very new and immature language, but it has some innovative and very appealing features for use in systems programming and teaching an operating systems course.

1. Rust changes the way you think.

   At least for "ivory-tower" types, the only reason to learn new languages is for their power to change the way you think.² Neither Go nor D really do this (at least for anyone who has previously programmed in C, Java, and Python), but Rust does a great deal. Rust requires programmers to think carefully about how objects are shared and used, and this changes the way you think about programs and algorithms.

2. Rust provides simple and safe concurrency mechanisms, and a useful and intellectually interesting task abstraction.

   I'm not aware of any other language that has concurrency constructs as elegant and easy to use as Rust's spawn, and I don't know of any language that comes close to the race-free safety guarantees provided by Rust. The alternative with language like C and Go, is to have students get in the habit of writing programs riddled with data race bugs. Such programs tend to work well enough to be satisfactory for course projects (where the occasional crash or corrupted data is acceptable), but are increasingly unacceptable in the real world. The next step is to use available tools to detect races (e.g., Go's race detector), but these present another tool to use and learn about, are incomplete (only detect races that occur on executions), and much less satisfying than the static, language-based mechanisms provided by Rust.

   Rust's task abstraction is somewhere between a thread and process: it provides the memory isolation safety of a process while allowing safe memory sharing, but with the lightweight costs of a thread requiring no context switching. This is a useful thing for programmers, but also a great thing for students to learn about and understand in an operating systems course. Along with the task abstraction, Rust provides good abstractions for communication between tasks including a simple channel abstraction and abstractions for shared mutable state.

3. Rust provides strong memory safety guarantees, but still allows programmers explicit control over memory management.

   The main alternatives provide two extremes: C provides no memory safety but fully explicit control over memory management; Go and D provide memory safety but with all objects being automatically managed with a garbage collector (over which languages users have little control). Rust provides a way for programmers to declare objects that are automatically managed or explicitly managed, and statically checks that explicitly managed objects are used safely. This is done using a notion of ownership, and type rules for controlling how ownership is transferred between references (e.g., you can pass an owned object temporarily to a function as a borrowed reference, and can have more complex ownership types to enable controlled sharing). These rules guarantee all memory is safely deallocated, and the sharing restrictions are essential for providing safe concurrency.

4. Rust provides good mechanisms for higher-order procedures, and many library types are designed in ways that encourage students to use them.

   One of the biggest embarrassments about our department's standard curriculum is that students can reach a 4000-level class without ever encountering a higher-order procedure (as opposed to the sadly rarely-offered alternative intro course which introduces higher-order procedures in the first assignment), so I thought it was essential to use a language that supports higher-order procedures and provides good opportunities to use them effectively. Rust provides a simple syntax for lambda expressions, and lots of good ways of using them including in the RWArc abstractions we used in the starting code for PS3.

5. With it strong emphaisis on safety, Rust still provides an escape hatch to allow students to experiment with unsafe code.

   The Rust compiler normally disallows any code with implicit data races or unsafe memory use, but the language provides the unsafe construct for surrounding code that may be unsafe. This can certainly be abused (and I regret not providing some guidelines or rules to prevent students from overusing it, which some students did on some assignments), but is also very useful for being able to show simple but unsafe code and for understanding what is necessary to make code safe. For example, we used this in PS1 where students added an unsafe counter to a simple web server, and then in PS3 there was a problem where students had to implement it in a safe (race-free) way. The unsafe escape hatch also makes it possible to include assembly code in Rust programs, and to easily use libc.

6. Rust is open source and has an open development process.

   The Rust source code is available for everyone to read, and the Rust language designers discussions are on an open mailing list. (Go is also open source and has an open developer discussion, so this is not a particular advantage of Rust over Go.)

7. Rust has a vibrant, helpful, and friendly community.

   This is obviously subjective, but the novelty and philosophy of Rust are well-suited to a devoted development team and user base, and I got a good sense that the community would be very helpful in my early interactions on the Rust IRC. For an immature language, having a supportive community is really important, and our class benefitted greatly from help from the Rust community throughout the course.

8. The immaturity of Rust means there are lots of opportunities to make contributions.

   Rust's immaturity is its biggest drawback (more in the next section), but it also has a good side. Using an immature language means there are lots of opportunities to make substantial contributions, and I tried to encourage students to view anything essential lacking from the Rust ecosystem as an opportunity for them to create it. By the end of the course, one of the students had made changes that were accepted into the main Rust compiler, other group had built a regular expression library, true random number generator, and real-time audio, among other projects; and several students are now working on writing tutorials and documentation that will be useful for students in next semester's class and anyone else who wants to learn Rust. It is hard for me to imagine similar contributions being possible to the extremely mature C ecosystem.

   
   

Reasons Not to Use Rust

There are lots of great reasons to use Rust in an OS course, but two major (and one minor) ones not to:

1. Rust is a very immature language (at version 0.7 at the time we started the class).

   The immaturity means there is little documentation, and much of what existed was not consistent with the latest version. It also means the language continues to change in subtantial and non-backwards-compatible ways which mean code we wrote for the class broke when we moved to version 0.8 mid-semester, and many of the libraries available no longer worked. The lack of documentation is a serious problem, and was the main cause of frustration for students in the class. There is a lot of ongoing effort now (including some from students in the class) to improve the available documentation for Rust, so this problem should diminish rapidly.

   The more fundamental problem with an immature language is that we lack the experience to have developed idioms and conventions for using the language effectively. Forty years of experience with C has led to the development of lots of idiomatic ways of doing things, and in most cases, evolutionary pressures should lead to good solutions, and these solutions are documented in lots of places that are fairly easy to search (e.g., stackoverflow).

2. Rust has a very steep learning curve and requires programmers to think differently.

   As mentioned before, for ivory-tower folks the fact that a language makes you think differently is a great property, but this also makes it a lot harder to learn and is often very frustrating for people used to thinking certain ways and unable to do the things they think should be easy. Good tutorials (which hopefully we'll have ready for next semester) and exercises can help, but the way Rust pointers work is different enough from what people with experience using C and Java expect that it seems necessary to get over some pretty big hurdles before writing any non-trivial programs in Rust.

3. Rust is not widely used in industry (yet), so it is disadvantageous to students compared to gaining experience using a commonly used language like C.

   I view this as a minor reason, since I don't think its really the mission of a university CS curriculum to prepare students with particular job skills³, but many people bring this up so it is worth discussing. One of the problems with this view is it leads to a circle of inertia: industry uses C/Java for programming projects because that's what their current team knows and it is easy to hire C/Java programmers, and universities train C/Java programmers because that's what industry wants. Academia should be leading industry, not following it, and one way to do that is by producing graduates who know about things that are not yet widely known in industry.

   For individuals, however, this larger view is not too helpful (trying to change industry doesn't provide much solace for the poor sap who can't get a job, although our graduates usually have a plethora of interesting job offers, at least for the 4th years, often before finishing this class). More pragmatically, I don't see that a one semester course using C is enough to really increase someone's value as a C programmer. There are thousands of programmers with decades of experience using C, and it takes many years using C to be a top-level C programmer, so a single semester of coursework is unlikely to make someone a particularly valuable C programmer if she wasn't already a valuable employee because of her general knowledge, ability, and talents. On the other hand, if Rust takes-off as an industrial systems programming language, being one of a small number of people who already have significant experience with the language should be a considerable advantage.

Programming Assignments and Reactions

Initial student reactions to Rust were largely negative.

The first assignment provided a few exercises to get students started programming in Rust and then expected them to modify a simple web server given code we provided to add an unsafe counter and make it respond to file requests. Responding to file requests required extracting the pathname from the requested URL and implementing simple file I/O. Since Rust doesn't yet have much support for strings, extracting the pathname was much more challenging than students expected, and this caused a lot of frustration. All of the students in the class were able to get the counter to work and 61 out of 65 students were able to get the file request serving to work. Of the four who couldn't, three encountered into problems with types they were not able to resolve and the other one was not able to figure out how to do string extraction.

The submission for PS1 also included an open resonse question,

(Optional) If you have any additional comments about this assignment or about how the course is going so far, here's your chance:

The responses are here. I encourage you to read all of them, but a few representative comments give the prevailing view:

Rust is interesting. I'm not sure if I like it as a choice for the class language yet or not. The safety of the language makes it valuable for operating systems, but the lack of documentation makes doing assignments difficult. To make matters worse, much of the language has been re-factored since the documentation was written, meaning that many of the methods and procedures that are documented no longer work in the current version.

Rust documentation is god awful. The web server portion of this assignment should've taken me all of three minutes, instead I spent 1.5 hrs trying to use undocumented methods. It would be nice if you could provide us with some common code snippits like reading files for instance. That is not something I should have to spend 20 minutes trying to figure out. I have the feeling that rust is going to be counter productive to actually learning anything about operating systems due to the excessive time spent on fruitless google searches/banging my head on the keyboard.

Using Rust was painful. Some library functions from Rust either didn't work correctly or yielded unexpected results that I didn't want, especially the ones from str module. So I decided to custom-develop the library functions by myself instead of being subordinate to language specific issues. I think we can better use our time learning more about the OS instead of wrestling with language specific semantics or bugs.

For the second problem set, which was focused on learning about processes, students implemented a shell. We provided a very minimal starting code that could execute a foreground program, and students had to add internal commands, support for background processes, I/O redirection, pipes, and an extension of their own design to the shell.⁴ Students worked in teams of two for this assignment, and in addition to submitting their code and answer to some questions in a web form did a demo with me or one of the course TAs where they showed off their shell and answered questions about their design and code. Of the 33 teams, 27 were able to succesfully implement all the required problems.

Problem Set 3 focused on synchronization, scheduling, and memory management, and required students to implement a much better zhtta web server than the zhttpo server from PS1. They had to make a safe visitor counter using Rust's concurrency abstractions, implement an improved scheduler for a multi-tasking web server, integrate server-side shell includes (using their shell from PS2) into the server, and improve performance by adding caching. The final problems challenged students to improve the server's performance in any way they could (without sacrificing safety) or add some new functionality. Students worked in teams of three students, and each team did a demo with me after submitting the assignment. All but two of the teams⁵ were able to get everything working (although many did memory caching in a way that actually reduced the performance of their server by requiring too much synchronization), and the remaining teams were able to get most of the extended functionalities working. By this point, many students were starting to really enjoy programming in Rust, and the concurrent programming required for the web server really made some of the advantages of Rust clear.

For the final assignment, students had an open-ended project, and were free to use any language they wanted (or to do something that was not language-specific). A bit over half of the students ended up using Rust for their project. Many did things that I think are very impressive and will be useful to the larger Rust community and to future offerings of this class including Iron Kernel (an ARM kernel written in Rust with a simple file system which I'm hoping to use for an assignment in next semester's class), Rust Raytracer (a concurrent 3D renderer), tRustees: True Random Number Generation, and Regular Expression Library, as well as projects to develop Rust tutorials. See Project Submissions for links to all the projects and credits.

Evaluations

The final survey and official course evaluation included several questions intended to evaluate student's experience using Rust in the class. Some of the results are below; see the Final Survey Results for full details and more comments.

Students overall have mixed views on Rust, with lots of extremes (some people love it, others really hate it). I view that as a fairly positive result for a first run of a course like this, and especially in light of most of the comments that are primarily about lack of documentation that should be much improved next semester. I'll be worried if these results don't improve next semester, though. See the Final Survey Results for more specific comments.

cs4414 Fall 2013 Graduates

Conclusion

In summary, I really enjoyed using Rust and think it is a very promising language with a lot of potential for teaching, as well as for industrial systems programming.

It is irresponsible for universities to continue to teach students to write C code riddled with security vulnerabilities, memory leaks, and race conditions. There are good alternatives available, and Rust appears to be the most promising to me, but I hope others will explore both ways to teach students to write safe C code as well as different languages for teaching systems programming that don't suffer from the legacy flaws of C.

Thanks for reading such a long post! I would be very happy to read comments from any students, people who teach OS courses, or anyone else who is interested in programming languages and operating systems. Feel free to post public comments below, or email me directly with private comments.

More discussion on Reddit (Programming) and Reddit (Rust) and Hacker News.

comments powered by Disqus