Seems to fix or at least make a pretty bad memory leak much slower.
I hate ws but the only other library is written by someone who isn't
a very nice person (putting it on the nice side)
- moved superqemu's "QemuDisplay" here; the VNC VM and Qemu both share it (and it has been renamed to a less goofy dumb name)
- VNC VM has been heavily refactored to just use the VNC display we have (this means only one source of truth, less bugs, and it's generally just Better to share the code imho). this means that future plans to abstract this further (or implement the client in cvm-rs in general) won't cause any explosions, or require duplicate effort
- vms are now in src/vm/... just better organization
- superqemu doesn't manage a display anymore (or care about it, other than making sure the socket is unlinked on stop). Instead now it provides info for us to setup our own VNC client. This is also why we provide our own shim interface
This currently relies on a alpha version of superqemu.
Before this is merged into cvmts main I will publish a stable tag and point cvmts to that new version
i think this was causing it to not be freed because we were holding a live reference (to NAPI/node/v8 side of things) even after we were done so the input buffer just never got freed from the buffer pool
- switch to napi-rs. this mostly affects only the backend side of things, but IMO napi-rs is better (also, way less boilerplate is needed compared to neon).
- jpeg encoding no longer clones the input buffer internally (or wraps it in a Mutex as well), thanks to napi-rs not sucking in this regard. this is *probably* a micro-optimization, but will make it easier to later on do parallel encoding of all rectangles
- guac encoding is weird. This is kind of a painpoint of napi-rs but it's bearable
Much less underhanded than using an IO library's capability to do so. Also, rayon will make encoding multiple jpegs in parallel *much* easier.
tbh if i do that I may also switch to napi-rs, it seems a bit less painful and supports much more async shenigans.
(Also, it will actually bind classes properly... something Neon really sucks at unless you look at strange documentation.)
both superqemu and vncvm have supported this but I never did it for some reason. its noticably faster than before. testing seems to imply that its actually slightly faster without the batcher enabled (maybe because it's actually able to split the work now if I had to guess)
We publish them now, so let's use them in cvmts!
Additionally, this removes the 'shared' module entirely, since it has little purpose anymore. The logger is replaced with pino (because superqemu uses pino for logging itself).
Finally, no more hacky sleep calls are left. Woohoo. This was already
effectively done by the previous commit, but now that I know it works,
I've removed the test path entirely
Simply a more convinent pipe. Additionally, because the pipe will only break when the process exits,
this means we can now remove QMP reconnection logic entirely. Can't exactly have problems
when the problem code is factored out ;)
qmp client is now much more robust (and doesn't fight over itself as much). this should
hopefully completely eliminate the case where display connects but qmp is half connected.
i also forgot QemuDisplay actually emits an event on connection, so we can just use that to
wait for when the display connects. which allows us to set the started state there instead of praying
when the qmp client connects that we are connected to the display roughly at the same time.
i also gated some stuff to require vm state in the server. this is a bit rickity, but does seem to work.
The QMP client has been refactored slightly, mostly just to clean up its
edges slightly.
QemuVM however has seen a big refactor, especially connecting to QMP.
Flattening out this logic is something I should have done a long time ago.
This seemingly has finally hammered out the bugs, although time will tell.
It sucked. The new one is using Sans I/O principles, so it does not
directly do I/O or talk to a net.Socket directly (instead, QemuVM implements
the layer to do I/O). This means in the future this library could actually
be tested, but for now, I'm not bothering with that.
There's also some other cleanups that were bothering me.
doesn't really need to be two seperate libraries. also preperation for other funnies
the build script has been replaced with a much saner justfile which uses much saner "yarn workspace" invocations instead of blindly cding all over the place
this is actually something i need to push to crusttest as well, because
this will affect it as well, though not as badly because it will only break certain buttons
the '@ts-expect-error' is to silence tsc just in case, since this function does in fact exist during bundling. techinically the real solution would be to declare it but /shrug
In short:
- cvmts is now bundled/built via parcel and inside of a npm/yarn workspace with multiple nodejs projects
- cvmts now uses the crusttest QEMU management and RFB library (or a fork, if you so prefer).
- cvmts does NOT use node-canvas anymore, instead we opt for the same route crusttest took and just encode jpegs ourselves from the RFB provoded framebuffer via jpeg-turbo. this means funnily enough sharp is back for more for thumbnails, but actually seems to WORK this time
- IPData is now managed in a very similar way to the original cvm 1.2 implementation where a central manager and reference count exist. tbh it wouldn't be that hard to implement multinode either, but for now, I'm not going to take much time on doing that.
this refactor is still incomplete. please do not treat it as generally available while it's not on the default branch. if you want to use it (and report bugs or send fixes) feel free to, but while it may "just work" in certain situations it may be very broken in others.
(yes, I know windows support is partially totaled by this; it's something that can and will be fixed)
modeco80
