WebAssembly Threads - HTTP 203

Key Takeaways

what better way to start a recording after i don't know two months since the last time we did this and change up everything what could possibly go wrong [Music] should i talk about a bit about webassembly threats jake what do you think well have you written slides about that because if that's what you've written all the slides about then i think that is what you should talk about otherwise it will get very confusing i wrote the title and i'm like well now i wrote the title gotta write the rest around this and so i did well so the the thing the thing i really like about webassembly and this is very much i think a summer thing and that doesn't necessarily apply to everyone is that webassembly has very little surface in itself it's webassembly can't do a lot of things by itself like it's it can very tiny things and i'm going to talk about this what we can do and yet we end up with capabilities like threads and sim d all these things that javascript can't provide and so i kind of wanted to talk about that talk about webassembly what it can and cannot do by itself and how interactions with javascript can open up all these possibilities and for that i thought i'd start a little bit with a small introduction to web assembly at the low level uh before i saw it's probably important to say like this is not what you need to know to use webassembly this is a bit like looking under the hood so so before we continue you said javascript doesn't have threads and javascript doesn't have threads correct but the web has workers node has words i'll talk about that oh okay fair enough yeah that's exactly because that's exactly the interesting bit because they're to an extent yes but on other perspective you would say no and then you combine these two and suddenly magic happens and that's kind of what i want to talk about um so i'm gonna give a bit of an like an incomplete overview enough so that you hopefully understand what webassembly can and cannot do but probably not enough to cover every variant in which it can do these things um but just like a peak under the hood of webassembly something you don't necessarily need to know to use webassembly but could be interesting or useful to know every now and then so this is webassembly kinda this is the human readable assembly languages the human realists i'm gonna have to take issue with human readable it's this human it can't read it it's yes i i i hear you it's it's assembly i mean people who have seen assembly for any or any form of machine code the human rebuild versions are not readable in oh i understand what's happening but at least you can decipher individual words compared to the binary representation of this file so this text presentation is called wat or what webassembly text period i guess this language is literally a text limitation of what is in the file in binary so you can define module and one module will basically end up as one webassembly file a wasm module can contain multiple functions and functions can take numbers as parameters and numbers in webassembly can be 32 and 64-bit integers and 32 and 64-bit floats and you can do math with these numbers and then you can return a number as a result and you can export what more could you possibly want exactly it's all you need right and you can export some of these functions to be callable from the outside i'm going to talk about outside in a bit so javascript is one of the host systems world assembly there's now actually multiple host systems out there some for php some just standalone like wasmtime which runs as a standalone app on your desktop machine but we're going to talk about javascript because we are a web show and so in javascript land you would fetch the dot rather module and instantiate it which means that you know it will compile the module instantiated and you can call these exported functions which is now you know this is what the outside is and here i declare a function that takes two parameters and a return type 32-bit integer is the return type of this function and then i use these two parameters to add them and that is also implicitly the return value of this function and at that point it returns to javascript and the javascript environment knows how to convert between javascript types and web assembly types and pretty much all the wise and times just turn into 64-bit floats because that's the only number type javascript has recently there's been an addition to webassembly where 64-bit integers are now going to map to bigins because a 64-bit float can't represent all the numbers in 64 but integer can assume so that will address that problem but does that mean then javascript can like javascript has a number type that webassembly doesn't support as well because like if you get an arbitrarily big number it will get to a point where webassembly can't once it's beyond the 64 bits yeah that that can happen currently so yeah if the big end grows too big it cannot represent it as an in 64. i actually don't quite remember how that is handled if it throws or if it gets clamped or something like that i would look into that have to look into that you see here that javascript can call webassembly but webassembly has access to absolutely nothing you can pass in the numbers and use these numbers to do arithmetic within webassembly but you don't have access to any of the apis you might be used to the web assembly is completely isolated and that alone is actually surprisingly powerful but we need a bit more to make it actually useful so the next step is that you can declare imports and here i'm saying that i'm expecting an import in the sermons imports namespace and that the import is called alert and i expect it to be a function that takes one 32-bit integer as a parameter later i call it function with a result of our computation sort of returning the result of that computation the instantiation for this webassembly module remains largely unchanged except now we have to provide these imports and that has to happen at instantiation and instantiation will fail if i don't provide all the imports the motor requires so this here is the so called imports object and the alert is obviously a good old alert function that i hope we all remember from our start of javascript's debugging i certainly use that a lot for debugging so this shows now that you can not only export webassembly functions to javascript but also you can expose individual javascript functions to webassembly but still only number types will be able to be passed back and forth because webassembly for example has no built-in understanding of strings now so far these website models have worked with just parameters and did arithmetic on these parameters for any more complex kind of work you actually need a bit of memory and that's why webassembly also has a way to handle chunks of memory that you might already know as array buffers so here we declare that this webinar module expects a memory in our import object and it needs to be at least one page big webassembly measures memory in pages each page being 64 kilobytes that has security and operating system integration reasoning it doesn't really matter but basically the smallest unit of memory is 64 kilobytes and every memory has to be multiple of that size and now we can use load and store to manipulate the values in that memory so instead of adding the two values from the function parameters we are now adding two values that we find in memory webassembly memories are as i mentioned a lot like array buffers but not quite the same they have their own type because you know they grow they can grow they have a different unit of measurement and they need a bit of special setup for security under the hood but in a way they behave exactly the same so here i create a new web assembly memory so you can create a typed array view on this area for just like with normal array buffers and then we can use this data view to put these values into memory and then use our weizen module to add them up this is obviously kind of a useful example just put two values in memory and add them up but it just shows how the interaction with the memory works so the memory in wasm is it it's just like or sending a function into wisem like you do with alerts and sending memory into wasm is just is exactly it's pretty much the same you have to declare what an import is supposed to be because web assembly is strongly typed so add compiler is known that this import needs to be a function and this imported to be memory but it's the same way you as the host system have to make the conscious choice to give something to webassembly website it cannot just grab anything by itself which is one of the securities that talks to what you said before of like of how webassembly is so lightweight like i always assumed like there was a deep integration of how the membrane web assembly works but it is just chucking a javascript object in there and then you're performing operations on it in webassembly lens so for complete transparency a web assembly model can actually declare its mode memory and export it instead but it still functions the same it's just like here's because the website module declares its own memory you also know it doesn't get access to anything it shouldn't get access to because it's created at instantiation time so it doesn't get random access to unknown data so it's all about the security and the primitives that are being exposed here that kind of covers all the things that webassembly can do that we need to know about to talk about this is pretty much what is called the wasm mvp the minimal vibra product which was the synchronized launch between all the browsers there are proposals obviously in webassembly land to augment what webassembly can do but almost all of them are just almost syntactic sugar on top of these things very few of the proposals actually expose new capabilities and if they are they're often limited to arithmetic which i think is very interesting so let's talk about threads because javascript is a bit weird on this topic because it is by design single threaded javascript however supports parallelism at least on the web and a node recently with workers which runs a javascript file in a truly parallel fashion to your main thread however you can only send messages back and forth with the worker and the main thread with post message and there is no way to share a variable between those two threads like you might be used to from other languages that support threads or any form of threading primitive and cynthia oh well except the shared ray buffer right that's well jake that's what i'm getting to okay so if you just if you just uh added shared memory to javascript things would break because many of the permits are designed around the fact that they cannot get interrupted or that there could be race conditions so instead the shared memory concept has to be isolated to a specific type which as you already spoiled jake is the shared array buffer um and that's i'm sorry i'll just i'll just shut up and sit here and let you talk because it's clearly like everything that i've i'm thinking of you've already got covered so i'm just glad you asked this question exactly here because that was my next slide so i did something right at least so yeah shadow buffer is pretty much just like an array buffer in the end um but you can get shared access with the same array buffer from across the threads and both of these threads will see the memory manipulation under the hood in real time so here what i'm doing is i'm running the main thread and basically have a while true loop to wait until the first cell in the memory is bigger or greater than a greater or equal than 100. so this base means the main thread is blocked and in the worker i get axis of the same shared array buffer and i increment the first cell value so even though the method is blocked at some point it's going to get unblocked because the worker is running in parallel for real and can work on the same exact memory chunk as the main thread and so this is called a spin lock where you just keep spinning an endless loop and keep checking your condition to continue um you know their thing but they're obviously quite bad because you just lock in your cpu at 100 for this thread because just that's all the process is doing um and are you going to mention atomics i am now going to mention atomics aren't you on top of things because i i actually think that atomics are not that well known because very few people probably work with shadow array buffers and they only work with shadow array buffers um and basically they have just a couple of operations in atomics to make operating on these short rebuffers more reliable and predictable as for example in a worker here we can block on a memory cell and wait for other threads to notify us that this memory cell is now ready to use it's a form of a mutex so in the worker we can use atomics.weight to wait on a certain cell the first value here is the index and the second parameter is the x at the first memory is the view i should read my own code the first member is the actual memory view the second parameter is the index and the third memory is the expected value that needs to be in the cell before we start blocking and that is a typical mutex programming pattern that you check if the value had been changed before you started waiting now the main thread we can basically just wait on the user to click a button and once they do we use atomics.notify and that will wake up all the threads that are waiting on this memory cell and so this way the cpu is not locked 100 this is not a spin lock it's actually in in cooperation with the operating system and we'll put the thread to sleep and save system resources now that was basically all the prelude all the the building blocks that we need to know to understand the webassembly threats proposal that is now in chrome stable so webassembly uh the web assembly threads proposal is actually much less than i thought it was because when you think about threads in c or rust you think about calling a function and having it run a separate thread webassembly threads is not that what it really is it's just it allows you to declare a memory as shared which basically makes this webassembly memory behave exactly as a normal memory but also like a short array buffer in that you can have multiple views in real time onto the same memory from different threads and additionally it exposes these atomics as web assembly instructions now this is interesting because it doesn't actually allow you to spawn a thread it just gives you these atomics and this is actually solved by the language compiler or the runtime that you use to write your webassembly so i did a diagram i know people find these kind of uml diagrams scary but that's actually what i found interesting but it is more complicated outside of web assembly and it is inside of webassembly so basically you write your javascript and whenever you compile something from c to web assembly with m script and m script will not only generate a.weizen file but also a javascript glue code as it's called and that piece of javascript takes care of loading the webassembly module for you populating the memory with you know all the values it needs to be in there and it provides the integration with the whole system that the c language expects to be in place so for example when in c you call p thread create which is the function to create a thread that is actually a javascript function that is imported into the webassembly so when you call p38 you get the call goes into javascript it will spawn a worker it will send the module and the shared array buffer over to the worker the worker will also load the glue code and instantiate the same module on top of the exact same memory and now we have main thread and worker running on the same memory with the same webassembly module and they can now use the atomics to synchronize so from here on in it actually behaves like a real c program but all the magic really happened so what's the new bit then because we could have like today you know before this wasn't feature became a thing like we can still give uh webassembly a shed labor for his memory so there'd be nothing to stop us instantiating two bits of wasm that are actually using the same bit of memory in different workers and they could be instructed to work on separate things at separate times so is that is that the case and if if so what what's the new bit before you couldn't instantiate webassembly on a shared array buffer that just wasn't possible and you didn't have the the atomics instructions inside webassembly so that that's what i mean like those are really just the new additions and they already exist in javascript they're not new capabilities on the web really the difference is that in javascript we don't have any way of expressing complex objects on top of a shared array buffer but any normal compiled language does exactly that you can build complex classes and structs and they all somehow get represented in linear memory and so that is this combination of old actually kind of old javascript features ray buffers and atomics and webassembly's capability to bring the high-level constructs to a low-level virtual machine that in combination we now have real threads on the web which kind of kinda was possible with workers and generate buffers but not in a comfortable way yeah we've been using it with scrooge now and it worked surprisingly well for some of it and yeah and so i looked into it i was like i was surprised at how small to an extent the proposal really is and yet the combination of these things really turned into something very powerful so i'll put you on the spot a little bit um because i haven't i haven't looked into detail that because it wasn't me doing that that work so it's going to be spinning up these workers when does it get rid of those workers like is it does it generate them per thread and then destroy them per thread or does it have a thread pool i think it has two so i actually no that's not true i think different compilers will handle it differently i know that m scripting takes a worker pool parameter so i wonder if that is to an extent how it works because i mean a computer has you can spawn as many threads as you like technically but you have limited amount of cores that can actually run any of these frets at any given time my hunch is that naively it will probably spin up as many workers as you call create threads and it will kill the worker when the threat is done but there's probably smarter things out there when you create a worker pool that they get recycled and reused i mean workers not workers i think weber from these threads are still so new to an extent that there's so many things to to measure and to optimize and to see how in with actual usage patterns that how that affects performance i know that google earth has been using them for a while so i'm guessing they had and i know they've been talking about so i wonder how much feedback has been flowing back between the m script the engineering team and the google earth engineering team um but so far threats has been quote unquote good enough to run all these use cases with good performance results in the wild and that that makes you kind of hopeful at least so is there any proposal to put in like genuine threads like i don't know i say genuine threads no that's kind of like what you want to spawn something without having to write the glue code right like that's what i thought that that would be the capability to somehow spawn a thread no i don't think so i think what this would be that would be wazi where you have a standardized systems interface so instead of for h language or reinvents how to mock out a threat creation call there would be a standardized interface to the host system which is what wazi is where you can say open the file um create a network connection but probably also spawn a thread and then the wazi implementation may that be in a desktop environment runtime or maybe a javascript layer on the web or wherever would just have this generic implementation but that is still i think bit out being you know worked on being serious there's experiments and that's really really good but nothing that i would say you could settle on for production right now but yeah that was basically the the webassembly handwritten version speedrun with serma and threads did you men you mentioned something about simd along the way as well is that uh what's happening there 7d is an even smaller proposal because it just well it adds one new type which is 128 bits something and you can interpret these 128 bits either as 4 32-bit integers 2 64-bit integers as 8 16-bit integers and just see them as vectors and add them and multiply them and they settled on that because that seems to be what will compile on most actual cpus because webassembly by itself is just an intermediate format right like when you when you download webassembly it will get compiled to real machine code that runs on your actual processor so you need to find a a sim d equivalent will compile as many processors as possible and then all it does just add this new type and add a couple of new instructions and then the compiler can decide whether the cpu supports or not and either pretend to run those instructions in one instruction cycle or just do it in series and pretend that it did it in one cycle and that's yeah that's the other thing that we need to look into but we haven't done yet have we that's still one of the things we want to look into for schools actually so the status of the threads that's shipped right like the well we've so inval our colleague has done some experiments and has gathered to work with both rust and simple plus and we have seen some pretty good performance improvement on those cindy is harder because often cmd needs to be hand written like the compiler often can only figure out very clear cases to auto auto vectorize as it's called to automatic eternal loop into a smd instruction that only works in very few cases the problem is that many codecs that have cmd instructions use handwritten assembly but not handwritten webassembly but for other cpus those don't compile to webassembly so we're kind of in in a niche where we don't know how we can make use of the of the sim d from these codecs to make use of web7d but i think um well we'll play around a bit and maybe maybe he'll find but in terms of this the thread stuff the new thread stuff like what browsers has that landed in it's i know it's in desktop chrome i know it's in firefox desktop if you have co-op and co-op enabled and i think we even have it in android chrome soon if you have crop and crop enabled which we have on scrooge so um we also have feature checking so if you have support on your browser swoosh will just use threads if not you will get the old codec version without threads so i think the reason for this is because like on desktop chrome you get process isolation out of the box whereas you don't get that in firefox and you don't get that on android because it's there's more of a memory concern and that's when you need to sort of exactly yeah close all the doors to outside content which is the coop and co-op stuff we've got another episode on that we can link to that oh do we yeah yeah i mean we just shared array buffer for oh we already did that one you're right with all the the acronyms yeah i mean we lost chad array buffer for a while due to specter and meltdown and now we have found mechanisms how we can bring them back without them being a risk which is co-op and court and we should link that episode because it was good thank you well done um all right that's it that that was my webassembly speedrun cool bye-bye bye all right [Music] oh no