Google's effort to mitigate memory-safety issues

Posted Feb 18, 2021 22:12 UTC (Thu) by mathstuf (subscriber, #69389)
In reply to: Google's effort to mitigate memory-safety issues by quanstro
Parent article: Google's effort to mitigate memory-safety issues

Do you have more than spooky stories to tell around the campfire here? Maybe some details about the kinds of bugs you tend to encounter? Are you a web developer? Kernel hacker? I'd expect different answers for different fields at least.

Google's effort to mitigate memory-safety issues

Posted Feb 19, 2021 7:20 UTC (Fri) by mti (subscriber, #5390) [Link] (9 responses)

I have seen similar things with respect to Java. When moving a big complex product from C to Java both the development time and number of bugs have increased.

Not what I would have expected!

Quite a few of the bugs are related to timeouts, locks and so on. So part of the problem might be moving from single-threaded to multi-threaded design.

But mostly I think the problem is not related to the programming language at all, but more to changes in the organization, changes in testing, lack of a system architect etc. Maybe a bit of "second system effect".

It would be interesting to make a deeper analysis ...

Google's effort to mitigate memory-safety issues

Posted Feb 19, 2021 13:58 UTC (Fri) by mathstuf (subscriber, #69389) [Link] (8 responses)

Timeouts and locks are problems no language (I know of) has tackled as a compilation guarantee. Maybe some of the verified languages like Idris or the like, but nothing "mainstream".

> But mostly I think the problem is not related to the programming language at all, but more to changes in the organization, changes in testing, lack of a system architect etc. Maybe a bit of "second system effect".

This is certainly more likely. The lack of testing suites in older projects doesn't help for sure.

FWIW, we had this nasty Python script that was handling our git checks upon pushes to MRs and code repositories. The thing was, quite frankly, terrible because it was borne out of "need to get this deployed before we can migrate to GitLab" and was based on what we had running before for Gerrit. Rewriting it in Rust allowed us to hammer down our types, stop fighting Unicode issues, library-ify everything relevant, get a proper test suite up and running, etc. The thing has been rock solid except for things where you're just going to have issues in any language: GraphQL schemas changing, JSON payloads suddenly having `null` where you've never observed them before, etc. But Rust's error handling meant that it shuffled such things off into a place where we got alerts instead of having a `KeyError` bubble up into main (because defensive Python programming is pretty terrible and verbose) and take the thing down during the weekend.

Google's effort to mitigate memory-safety issues

Posted Feb 20, 2021 1:23 UTC (Sat) by LtWorf (subscriber, #124958) [Link] (2 responses)

> stop fighting Unicode issues

Well it's a bit your fault for still using python2…

Google's effort to mitigate memory-safety issues

Posted Feb 20, 2021 3:08 UTC (Sat) by Cyberax (✭ supporter ✭, #52523) [Link] (1 responses)

Python2 actually has no Unicode issues. It works fine. In Py3 I have to put encode/decode in tons of places that mix real world data and the happy-land of Py3 Unicode strings.

With Rust or Go it's back to Py2 level of Unicode usability.

Google's effort to mitigate memory-safety issues

Posted Feb 20, 2021 3:50 UTC (Sat) by mathstuf (subscriber, #69389) [Link]

In this case, it was Python2 (the code died 5 years ago now). We were stuck because of dependencies not supporting Python3. The problem was that 'literal_str' % unicode died a fiery death and we had to catch each field individually because we didn't have a test suite or anything to check our types. Not the greatest code, but the utilities around it were in Python2. The replacement might have been in Python3 if there had been a useful inotify library available at the time (without dragging Twisted around).

Google's effort to mitigate memory-safety issues

Posted Feb 20, 2021 12:23 UTC (Sat) by roc (subscriber, #30627) [Link] (4 responses)

Unlike Java, Rust does statically guarantee no data races in safe code.

Google's effort to mitigate memory-safety issues

Posted Feb 20, 2021 14:51 UTC (Sat) by mathstuf (subscriber, #69389) [Link] (3 responses)

True, but I also read deadlocks in "locking problems" which Rust doesn't aim or claim to solve.

Google's effort to mitigate memory-safety issues

Posted Feb 20, 2021 18:32 UTC (Sat) by ibukanov (subscriber, #3942) [Link]

Deadlocks do not violate memory safety.

Google's effort to mitigate memory-safety issues

Posted Feb 21, 2021 5:45 UTC (Sun) by roc (subscriber, #30627) [Link]

That's true. Fortunately deadlocks are typically a lot easier to debug because the system is in a stuck state that usually shows the problem directly.

compile time deadlock guarentees

Posted Feb 22, 2021 12:03 UTC (Mon) by tim_small (guest, #35401) [Link]

Although Rust doesn't aim to guarantee deadlock free execution at the language level, it's possible to construct such systems on top of Rust using the type system. For instance, in the (bare metal) embedded space, the RTIC framework https://rtic.rs/ features "Deadlock free execution guaranteed at compile time".