Repository navigation
undefined becomes null after deserializing #150
Description
const msgpack = require('@msgpack/msgpack');
let original = {x: undefined};
let serialized = msgpack.encode(original, {extensionCodec});
let deserialized = msgpack.decode(serialized, {extensionCodec});
console.log(original, deserialized, original.x === deserialized.x);prints "{ x: undefined } { x: null } false". This is a problem because now code checking for strict equality fails.
I was also not able to handle this in an extension codec because that is only called for non-primitive values.
ignoreUndefined: true doesn't work in all cases:
let original = undefined;
let serialized = msgpack.encode(original, {ignoreUndefined: true});
let deserialized = msgpack.decode(serialized, {ignoreUndefined: true});
console.log(original, deserialized, original === deserialized); // undefined null false
This is stated in the readme, but is there no way to retain this with an extension codec?
Activity
gfx commentedon Dec 23, 2020
Can you elaborate on your use case? In general, it's not a good idea to distinguish
nullandundefinedin JavaScript, and supporting it with extension types could lead to extra overheads, so I'd like to know your use cases are worth adding more overheads.mischnic commentedon Feb 8, 2021
Once case that I found was where we serialize/deserialize an entry of
process.env(so{key: "foo", value: process.env["foo"]}) and then check ifdeserialized.value !== process.env.foo(and this breaks if the value is undefined). It's easy to workaround, but I fear there might be more situations like this in the codebase that haven't surfaced yet.(I'm using msgpack as a browser replacement for
require("v8").(de)serialize)grantila commentedon Oct 23, 2021
May I suggest allowing for overriding this in a codec? Currently
undefinedis handled internally by this package, and you can configure whether to a) ignore it (not include it in the encoded buffer) or b) coerce it into null. I understand that this makes sense when communication between different languages, where there is no distinction between undefined and null, which is why it'snilin the spec.If this could instead be allowed to be overridden though, users can encode it if they want.
My use case is to transfer an object from one place to another and maintain it exactly as is. Turning undefined into null alters the datatype, and allowing undefined to be there is different from not having it.
Object.keys(obj)e.g.genki commentedon Jan 23, 2024
@gfx
Suppose to encode the entire arguments of functions that have optional ones.
The values of the optional arguments are not
nullbutundefined.If the msgpack coearses them into
null, the information that they were not supplied is lost.We have to provide the additional bitmask that informs which parameters were really supplied.
The
undefinedmeans the existence of thekeyin the object.The
nullmeans the absence of thevalue. They can't be treated as same things.WebReflection commentedon Feb 21, 2025
deserialized.value != process.env.foowould solve this in JS land as==withnullandundefinedit's the only case where it'strue, everything else would be false, evenfalseitself so this was trivial to solve to start with but I wonder how comes MessagePack decided to encodeundefinedat all ... I can see the==used here and there, but undefined usually means not defined and even JSON got it right, it's entirely pointless to carry non defined values, especially when every other PL might have just nil or explicit None around, but never undefined.gyzerok commentedon Jun 26, 2026
@gfx can you elaborate on "not a good idea to distinguish"? It's 2 separate types in JavaScript. Any place that checks
value === nullwon't work for undefined and vice-versa. Personally I think that in general people expect serialization/deserialization cycle to be loss-less. I was definitely caught off-guard with this one.It's okay to be opinionated about it by default. But I think at least it'd be nice to allow people to extends it with extension codec instead of not providing them any path forward.