CPython eases Rust requirements as assimilation continues

The maintainers of CPython have backed off from making Rust a required dependency for the Python reference implementation, focusing on the less intrusive task of building an optional Rust API for internal Python development. The move “sidesteps a lot of the concerns that people brought up,” Rust Programming Manager Tomáš Šedovič explained at RustConf, held in Montréal last week. Šedovič and others from Rust have been working with the CPython core developers to smooth the integration between the two environments. CPython’s revised approach eschews a more aggressive push to use Rust in Python, an approach that roiled the Linux kernel community’s adoption of Rust last year. The tasty dish is still rejected by the unwilling palate Last November, Python programmer Emma Smith floated the idea of incorporating Rust support into Python, initially for writing extension modules, but with the overall plan of making Rust a required dependency in CPython so it could be used throughout the CPython code base. This would have been similar to the relationship between Rust and the C programming language, upon which CPython is largely built (hence the name). C brings its share of security dangers when it comes to memory, which must be allocated and deallocated manually in code. The memory-safe Rust eliminates this entire class of errors at compile time, hence its interest to the CPythonistas. But, as fellow vulture Liam Proven noted at the time, the mandatory inclusion of Rust created a number of issues. Not all Python implementations were built on platforms that supported Rust. Plus, building Rust itself requires Python, creating cyclic dependencies in the build process. And you have to factor in the usual curmudgeonly reluctance that comes with any forced upgrade. So, in May, Smith amended the proposal to drop the mandate to make Rust a required dependency for CPython, kicking that down the road for a future proposal. Instead, Smith pitched the idea of modifying CPython so its devs could write optional extension modules. They could use Rust to build the functionality if they wanted, but it’d also be cool if they chose not to opt in. Even Python creator Guido van Rossum agreed this easier approach was the way to go, commenting in the proposal: “We all know that a full rewrite in Rust won’t work, but starting to introduce Rust initially for less-essential components, and then gradually letting it take over more essential components sounds like a good plan.” The devil’s details Since then, work has continued on incorporating Rust into the build process, and designing a Rust API that would allow Rust crates, or libraries, to be used to build Python itself. The API is planned for the 3.16 Python release (due October 2027), which will also include the Rust zlib compression library as a test crate. Python and Rust will each have to make some adjustments to work happily together, Šedovič noted. Developers from each camp have been in talks about how Rust could better accommodate Python’s concerns. One element needed from Rust is support for the GCC (GNU Compiler Collection), necessary to CPython’s “long tail” of obscure platforms it supports. The two languages also differ in how they build both standard libraries and individual dynamic shared libraries, which must be reconciled. Python will need some sort of cross language “sanitizer” support to ensure unsafe memory states don’t crop up between how the two languages handle memory allocations. Fortunately, several efforts are already underway along these lines, such as BorrowSanitizer, thanks to the same problem that bedevils C/C++ integrations with Rust. The most complex challenge will be augmenting Rust's Drop trait so that Python objects can receive runtime context when being deallocated from memory. This one will take some major think-em-ups from Rust engineers, he said. Python may also borrow a few innovations from Rust. The original proposal from last November praised Cargo as an “excellent build system.” In fact, it does the work of several separate functions in the Python universe, automating the entire development lifecycle. Not only does Cargo download dependencies, it also compiles and links them to other libraries, and tests them. With Python, you need separate project initialization, build, testing, and publishing tools to do the same tasks. Python is already following Cargo’s lead here with uv, a unified packaging and project manager (written in Rust). It’ll be interesting to see what other Rust innovations Python cops over time. Van Rossum even joked that perhaps CPython may need to be renamed “CRPython.” ®