Justin Blank

The fast JavaScript runtime Bun is much faster than Node.js 22 at decoding Base64 inputs. By much faster, I mean *several times* faster. But they both rely on the same underlying library (simdutf) for the actual decoding. So what gives? The problem is that Node.js needs to interact with v8, the underlying JavaScript engine (from Google)... and doing so is not trivial. Before we can start decoding the string, we need to grab the string... so, in this instance, we call String::Value... In turns, this allocates an array inside Node.js and asks v8 to copy the content to it... In an ideal world, we would avoid the trouble entirely and just ask v8 to give us direct access to how it stores the string... and we try to do that if we can... but let me come back to it... How bad can this be, right? Just a copy. Well. Let us do some profiling... So you see, the base64 decoding itself is about about 1/5 of the running time, but the copy takes half of it. What is up with this CopyChars function? Well, it is mostly just a wrapper around the standard high level C++ function std::copy_n as far as I can tell. (see v8/src/utils/memcopy.h) But we are copying for an 8-bit input to a 16-bit output... why is that? Base64 is pure ASCII... and v8 can store ASCII using 8-bit per character. We get there before both IsExternalOneByte() and IsOneByte() are false (see node/src/node_buffer.cc)... We have fast paths for these cases. If IsExternalOneByte() is true, we just get the bytes and everything is great. Unfortunately, it does not always work. So we have a v8 string that is really pure ASCII, but, seemingly, we can't tell that it is the case from Node.js, and so we have to convert it to UTF-16 needlessly, using a function that is maybe not very well optimized... and then we do the base64 decoding of an ASCII string from the UTF-16 input. It is not great. To be fair, this is just one string, created as 'Buffer.alloc(size, "latin1").toString("base64")', basically the base64 encoded version of the string "latin1latin1latin1...". In actual applications, we might have better luck. Yet. Yet. I am telling this complicated story for a reason. The story illustrates why our software is slower than it should be. We have layers of abstractions to fight against. Sometimes you win, sometimes you lose. These layers are there for a reason, but they are not free. To make matters worse... these abstraction layers often thicken over time... and the friction goes up. To be clear, I do not claim that the Node.js code is optimal. In fact, I know it can be better. But it is not trivial to make it go fast. I sometimes hear people say... "well, it is C++ and C++ is hard". No. The C++ part is easy relatively speaking. The difficulty is at a higher level. It is not a matter of syntax. It is a matter of architecture.