Repository navigation
Serializing an EXTERNALSXP can lead to an endless loop #1303
Description
When a WriteItem is called with an EXTERNALSXP that is not Code, Function or DispatchTable, it ends up in an endless loop.
WriteItem calls to an externalCodeWrite function pointer, which is set to serializeRir. Then it calls WriteItem with the result of rirDecompile.
Currently, the rirDecompile function only deserializes Code, Function and DispatchTable types. For all others it returns the SEXP unmodified.
Hence the second call to WriteItem is the same as the first one and it loops endlesly.
Activity
rihafilip commentedon Aug 9, 2024
Reproducing this issue is a bit trickier, easiest way I found was with the recording tool: triggering a
EnvStubMaterializeddeopt, which has aLazyEnvironmentas a trigger, creates a new event with this trigger saved and at the end of execution forces aLazyEnvironmentto be serialized, which leads to an endless loop described above.Smallest example I've been able to find:
RIR_RECORD_SERIALIZE=1 RIR_RECORD=/tmp/rec.rds RIR_RECORD_FILTER=Deopt R -e "library(ggplot2)"By putting an
assert(false)on line 18 ininterpreter/decompile.cpp, the code fails faster on the incorrect inputs.