Sending and Receiving data tutorial - Supercomputing and Parallel Programming in Python and MPI 5

Key Takeaways

what is going on everybody Welcome to the fifth video in parallel programming and supercomputing in this video and actually in the last video what we did was we just learned a couple more commands and just did some simple math just to show that it was possible in this video we're going to learn one more command and then we're actually going to send and receive data so that's uh the most exciting part so far is we will finally pass a message that's exciting anyway besides just printing out stuff I suppose so again as usual from MPI 4 Pi import MPI uh then we're going to do com equals mp.com foror if you wonder why I type this every time it's just a good idea to get comfortable with it if you don't want to type it every time that's fine but I suggest you do anyway uh rank equals com. rank size again we equal com. size and this time we're going to learn a new one and that's going to be name name and what this is going to return is the name of the processor so Master 001 node 002 in my case uh if you haven't change your host name then obviously you're going to be you know Raspberry Pi but if you did change it uh you will get uh whatever name it is so MPI do get with a capital gcore processor uncore name and that'll return the name now what we're going to do is we're going to make a message and we're going to call this message shared shared is going to equal rank + 1 * 5 just something nice and simple pretty much now what we're going to say is if rank equals zero so a nice conditional that we've already seen we're going to say data equals shared and now we're going to send uh this data so we're going to say com. send uh and what do we want to send we want to send data what is the destination so D and that's going to equal the rank of where you want it to go in our case that's one we want to send it to node one now if you remember what I was mentioning in the last video was size and we can use this to conditionally uh do some stuff size is a good way to condition or to conditionally configure the destination so right now we're hardcoding the destination whereas what you could do is you can combine Rank and size uh together so the destination could be um size minus rank let's say so you could do something like that so you can share stuff that's really basic and obviously you would run into a problem when the destination is zero minus um or size minus zero for example but that would be Master node to master node and generally that is kind you people would use the master node you can also format that out but but anyway to send a message through you could do something really really simple like that or a lot of people would have a better algorithm than that but just to uh spark your imagination how you might might use that next up you would say so if rank is zero send that data to that destination there's also another command that we might hit on later and that's the tag command and you can tag it with something you can say oh this is you know message X but we're not going to get into that for now so anyways com. send um and then let's just print out some stuff we're going to say from rank rank we are or actually we sent data okay next we're going to say l if rank equals 1 data equal oops data single equal there data equals com. CV for receive and where do we want to receive this from we're receiving it from Source zero so obviously rank zero Master node data is this and it's sending that data to node one which is a worker node and if when this worker node runs this script it's going to realize it's rank one data is going to be comm. receive Source Z and then we're going to say print on node rank we received data okay so uh here we go we're going to save this and again I'm going to move this to uh both pies now SCP five okay so minimize this up arrow and I'm going to change mine to five again if you're not changing your file name obviously you don't have to hit enter and we run so from rank zero we sent a five on node one we received five so uh what did I do with my script so again um it was just rank + one time 5 uh so obviously that five makes a whole lot of sense and so now you can see how we've shared this data between the ranks and as you can see not only have we just shared data we actually shared slightly a bit of an object because we shared whatever data stored so we sent data over and then data represents whatever it represented here so you've sent that stuff over we've stored it in the same named variable and we're able to obviously spit out that exact identical variable so that's going to conclude uh this video and really your first example of truly sending a message over NP and and receiving it and all that and then starting to use it obviously in this case we just printed it out but uh hopefully you can start to see where this might become beneficial and Just for kicks though what if we did something like this what if we flip these around so we'll say one Zer so in this case and then destination 0 one so here we're going to say if we're rank one what do we want to do data is equal to this so this time it'll be rank one 1 + 1 * 5 so 10 then it's going to send that to destination zero which is the master node and then L if you are the master node receive this data right so let's save that and again I'm going to move this over and again and let's just run this one more time and then again it says from rank one we sent 10 on node zero we received 10 so again it it it it was willing to wait so we can run this a few times and it could be assumed that maybe every now and then um something is quicker than the other thing especially rank zero right uh node zero in theory should be quicker than node one because node zero is just trying to receive a message whereas node one is actually trying to calculate something and send it so you would expect that it's totally possible that um that node zero wouldn't have got it in time the other way and then we flipped it so where node one actually you know does this first and it still obviously works um so I just wanted to bring that up but there are there will be times when you're like you're waiting for something to occur uh so for example let us do let's say let's let's just flip this around again so I'm just going to control Z okay so now let's import time and now if rank is zero we're waiting to um or I'm sorry the initial one was we were sending or we were sending data to rank one right from zero so now what we what we can do instead is let's say let's just throw before we do this let's time sleep like five seconds um we'll save that and now let's move that over again close this run it okay so now you can see that both nodes like it literally what it's going to do is it's going to wait for that data so it's going to wait to receive that data so it's not like it's going immediately through everything so there are going to be sometimes when order is somewhat important but that's mostly for like the logistics of sending and receiving stuff and at the rate of calculation but when it comes to actually needing to know like well this code has to be you know the send and receive has to be calculated first in time for that receive for there to be anything to be received and all that if there's nothing there it's just going to it's going to sit there and like wait so anyway just keep that in mind um and then also this one's becoming a little longer than I intended but let me do this so what if we did something like this from rank we sent data and we move the sleep till after that so now let's save and let me move this file again so now it will have sent that data before it begins its little weit time so this up Arrow again and you can see that from rank zero we sent the five we got the five on node one um and obviously we had the sleeper and that's why it took a while for this whole process to end but again it it was everything was done in time and this is a really great representation of processing that's happening still over here even though rank one is is reporting stuff even though this one's over here sleeping it's not running so each each node is really is running this separate script um on each other and we're see seeing the output at the same time so anyways just wanted to show you guys a little bit of that how we can add some stuff to it to get a better understanding of how this process actually works feel free to play around a little bit with it um there's a lot of interesting stuff and so um I'm kind of curious now actually about something I wonder if um like I wonder if you took R if you say you know we run run three nodes which would revert back to the master node as a worker node and you did something like this again to it and made a sleeper if it would continue running the script or not I'm really curious I don't know if I want to add that so maybe I'll add it just Just for kicks so let's say we'll come down here and we'll send this data to two nodes uh this one will come to two I am am just really curious sorry guys um because I haven't actually tried this one yet so now we'll say l if rank equals 2 we'll take that okay cool so now in theory when we run this even though we've got quote unquote three nodes like I said to you guys like you could do this with like making up kind of like a pseudo node um when we run this we're going to say number of processors three and it's going to revert to um and we never used our name variable let's go ahead and put our name variable in there just Just for kicks too just so you can see again what which one we're using so node number three rank two should be Master again so it'll just cycle through so anyways let's just save this and I'm just curious if the sleep will affect here in theory I I would assume it would but I'm curious now so uh let's run that hopefully this video isn't going to be too long um come down here up Arrow come over here well first let's just test this against uh yeah let's see oh invalid rank because we specified rank three ranks so we have no choice but toh say number of processors three okay that is too cool okay so that's interesting because python is singular threaded and it looks to me like we have past that single thread I'll have to use this on one of my more powerful computers and see if if in a single script you can use NPI to um to surpass that so I've never I haven't heard anybody say that they've done that so they can use like your full CPU usually use like threading or something like that maybe in the back end it's threading but if you don't know what I why I'm so happy about this is on the master um here and here as you can see see these are two Master nodes and on our script here this rank zero which is a master node has a command to sleep for five seconds that's why it hangs at the end for a few seconds um meanwhile rank two is um a master node so right here Master 001 so while this was sleeping running some code in sleep time this one still outputed the five I am thoroughly impressed that's interesting that is too cool I'm pretty excited about that I'll have to try that with like a bigger example I'm not really sure if that is if I'm being fooled by something here but it sure looks like you've got threading going on uh when that's the case anyways that's pretty exciting I'll have to play around with that on my main computer so hopefully you guys enjoyed uh I actually enjoyed this video pretty uh pretty well so hopefully you guys did too uh anyways that's going to conclude this video as always thanks for watching thanks for all the support and subscriptions and until next time