Python Tutorial: File Objects - Reading and Writing to Files

Key Takeaways

hey there how's it going everybody in this video we'll be learning how to work with file objects in Python and some of the useful things that we can do with these objects so whether you use Python for desktop or web applications you're likely going to be interacting with files a good bit so it's definitely a good skill to have to know how to properly interact with these file objects okay so let's go ahead and dive in so to get a file object we can use the built-in open command so I have a file here called test.txt in the same directory as my python file now if I open this file we can see that it's just a plain text file with multiple lines so let's see how we can open and read this file from within python now the way I'm going to open up the file right now isn't the way that it's normally recommended uh it's usually recommended to use a context manager which I'll show you here in just a second but to show you why a context manager is useful let me first show you this method for opening files first so what we're going to do here is we're going to say f equals open and we're just going to open that test.txt file now if you're working with files from different directories then you're going to have to pass the path to that file into the open command but since this file is within the same directory as my python file then I can just pass in the name of the file uh but if you want to learn more about how paths work then we uh touch on that a little bit in the tutorial I did on the OS module okay so the open command here allows us to specify whether we want to open this file for reading writing appending or reading and writing now if we don't specify anything then it defaults to opening the file for reading but I usually like to be explicit here so let's go ahead and say that we want to open this file for reading and we can do this by passing in a second argument here and that's just going to be the string of a lower case r and we'll touch on some of these later but if I wanted to write to a file then it would just be a lowercase w that I'd pass in uh appending to a file is a lowercase a and if I wanted to read and write to a file then I could do an R plus but for now we just want to read the contents of the file so let's just pass in a lowercase R okay so now the file is actually open and we can print the name of the file if I was to do a printf. name and also before I run this and print the name of the file out there's one more thing that we have to do here if we open a file like we just did here then we need to explicitly close the file when we're done using it and to do this I'm going to do it by saying f. close so now that we've closed that file let's go ahead and run this and you can see that it printed out the name of the file that we opened and so this has some more information that we can print out also if we wanted to print the mode that the file is currently opened with I can do a f. mode and if I run that you can see it prints out a lowercase R because we open the file for reading okay so now even though that this works the way that we've just now done this let me show you how to instead open the file using a context manager and why for most use cases you'll want to work with files this way so if we open the file like we did here then we have to remember to explicitly close the file if we don't close the file then you can end up with leaks that cause you to run over the maximum allowed file descriptors on your system and your applications could throw an error so it's always important to make sure that you close the files that you open so in order to use a context manager then it's kind of similar but we can do this using the with keyword so we can say with then I'm just going to copy all of this so with open test.txt in read mode and then here at the end I'm going to say as F and then I'm going to put in a opening for our block here now this can be a little confusing to people at first because the variable name is actually over here on the right using as F instead of over on the left when we said f equals open but the benefit of these context managers is that they allow us to work with files from within this block and after we exit that block of code it'll automatically close the file for us so we don't have to worry about whether or not we add in these closes here now this will also close the file if there any exceptions that are thrown or anything like that so that's why using these context managers are considered a best practice and it just automatically takes care of all that cleanup for us so now I'm going to go ahead and delete my outside uh open and close statements there now one thing that some people don't realize is that you actually have access to this F variable for now I'm just going to say pass within this context manager now we actually have access to this file object variable after after we exit the context manager but the file will just be closed so for example if I print the closed method on F now and run that you can say that it you can see that it returns true but even though that we have access to this variable here uh it is closed so it's not like we can read from it like if I try to read the contents from the file and print that out then you can see that it throws a value error here and it says IO operation on a closed file so for what we we're going to have to work with this file from within this context manager and for the rest of the video I'll be using these context managers to work with files since it's a good practice but I wanted to show you the other way first in case you see it in examples or wondered why I wasn't doing it that way okay so back to our file so we just tried to read the contents from the closed file and got our error but let's look at how we can read the contents from the file from here within our context manager so let's create a variable called fcor contents and this will just hold the contents of our file now if we do an f. read and if I print this out oh and actually I need to actually print out that fcor contents so if I save that and print that out then you can see that it printed out all of the contents of our file now if you have a small file then this is probably what you want uh but what if we have an extremely large file that we want to read but you don't want to load all of the contents of that file into memory well there are a couple of other methods here that we have available for reading file contents instead of f. read so just to look at a couple of those I could say f. read lines and if I print this out then you can see that we get a list of all of the lines in the file and it looks a little weird because we have our new line characters in there but if we look through this list then it actually gets every line of the file at a different element of that list now instead of f. read lines I could do f. readline and if I save that and run it then you can see that read line grabbed the first line of our file now every time that we run f. read line it gets the next line in our file so if I was to copy all of this and then do it again and run that now you can see that it got the first and the second lines from the file now this printed out a little weird here because the print statement uh ends with a new line by default but if I go up here and pass in an empty string to the end of our print statements then it will no longer add in that extra new line and now you can see that those are the way that they are in the file okay but we still haven't solved our problem of how we can read all of the content from an extremely large file if we read the entire file in all at once then we could run out of memory and we don't want go through and do f. readline you know thousands of times so what we're going to do here is instead of using readline or read lines we can simply iterate over the lines in a file by saying for oh let me go to a new line here for line in F and then from here we can just print that line so I'm going to copy that and save that so now let me go ahead and comment out these lines and run this iteration over the lines and you can see that it printed out all of the lines in our file now this is efficient because it's not reading in all of the contents from our file all at once so it's not a memory issue that we have to worry about what it's going to do is it's just going to go through and get one line at a time from the file now this is usually good enough for most people but sometimes you may want more control over exactly what you're reading from the file now if we go back I'm going to go ahead and delete this line if we go back to our f. read line here and I'm going to get rid of that one now I'm going to go back to using f. read and if you remember this read in the entire contents of the file so if I run that you can see that we got the exact same thing but with f. read we can actually specify the amount of data that we want to read at a time by passing in the size as an argument so if I pass in a one 100 to our read method and then print this out you can see that it printed out the first 100 characters of our file instead of printing the whole thing all at once now if I was to copy this and run this again then you can see that it printed out the rest of the file because it picked up where it left off and read 100 more characters of the file now when we reach the end of the file then read will just return an empty string so if I was to copy this for a third time and rerun this then you can see that nothing happens because what happens when we reach the end of the file read just returns an empty string so this print statement is just printing out an empty string okay so how are we going to use this technique in order to read in a large file um so since we don't know exactly how long the file will be we're going to have to use some kind of loop that just iterates over small chunks at a time uh so instead of hardcoding in 100 here I'm going to create a variable here called size to read and for now we'll just go ahead and set that equal to 100 so now instead of passing in 100 to f. read let's just pass in this size to read okay so this will grab the first 100 characters of our file now remember when we hit the end of the file then read will just return an empty string so if we do a a while loop and say while the length of f contents is greater than zero then we will print out the contents that we got from read now don't run it like this yet because this will be an infinite loop we're never advancing the contents of the file after it prints the contents then we want to read in the next chunk of characters so in order to do that then we just have to again say F contents equals f. read of that sized chunk now what it's going to do after this line here is that it's going to kick us back out to the while loop and it will check if we've hit the end of the file because f. read will return an empty string and it won't meet this conditional so now if I go ahead and run this then you can see that it printed out all of the contents of our file so to get a better idea of what's going on here let's change the size to read to 10 characters instead of 100 characters and every time that we print out f. contents here instead of an empty string uh let's make this an asteris so now if I print this out then you can see it's a little bit more clear That We're looping through 10 characters at a time and it's printing out these asteris through every Loop so you can see that it came through the loop here and it printed out these and then the asteris that we know that it's just that chunk then it printed out the next 10 characters and then the next 10 characters and so on until we got to the end of the file now when we read from files you can see that it advances its position every time so we can actually see the current position using F.T so what I'm going to do is I'm going to comment out this while loop here and down here I'm going to say print and we'll print out F.T so if I go ahead and run that you can see the F.T returned 10 so it's saying that we're currently at the 10th position of the in the file and that's because we've already read in 10 characters here and we can manipulate our current position using the seek method so to show an example of this uh let me print the first 20 characters of the file by running f. read twice so I'm going to go ahead and print out the contents after the first 10 characters there and then I'm going to do this a second time to get the next 10 characters and I'm going to go ahead and take out this second uh empty string there so that it pushes our finished statement out of the way so now actually let me get rid of F.T here and go ahead and run this so we can see that it printed out the first 20 characters of our file now when we read in this second chunk here it picked up at the 10th position and read in the next 10 characters like we would expect but what if I wanted that second read to instead start back at the beginning of the file and we can do this with f. seek so between these two reads if I was to do an f. seek of zero and save that and ran it now you can see that it set our position back to the beginning of the file so the second time we read in our contents it starts back at the beginning instead of picking up where we left off after the first read now we used seek zero to start at the beginning of the file but you can use it to change the position to any location that you'd like okay so now let's take a look at writing to files and a lot of this will be similar to reading so first of all what happens if we try to write from within a file that we have opened in read mode so let's let's go ahead and try that so I'll do an f. write and I'll just do an f. write of test I'm going to go ahead and get rid of that while loop also and save that so you see when I have a file open in read mode and try to write that we get an error that says that this is not writable so we have to have the file open in write mode so now back up here within our open statement let's create a new file called test2.txt and and instead of reading we are going to write to it now in order to do that we can just say a lowercase w instead of that lowercase R now you can see over here in our directory that we don't have a test2.txt yet now if the file doesn't exist already then this will go ahead and create it now if the file does exist then it will overwrite it so be careful if you're writing to a file that already exists now if you don't want to overwrite a file then you can use a lowercase a for a pending to the file but we're going to go ahead and overwrite this file if it exists so first of all instead of writing to this file I'm just going to go ahead and put in a past statement here which basically says don't do anything so I'm going to go ahead and run this and you can see that it created test2.txt so I didn't actually have to write anything to the file in order to create it just using the open with the right mode we'll create the file so now in order to write to this file then we can just do what we did before we can do an f.r test.txt so I'm going to go ahead and run that now if we go over here to test2.txt then you can see that it wrote test to our file now if I go back here and do another write to this file then it's going to pick up where we left off just like the read method did so now if I run this and go back to the file then you can see that it wrote test twice back to back now you can actually use seek when writing files also to set the position back to the beginning of the file and we can do this if I go back here between these two WR statements and I was to do an f. seek of zero now if I run this then you can see that the second test overwrote the first one so seek can get a little confusing for file WR because it doesn't overwrite everything only what it needs to overwrite so for example if instead of writing the same thing twice if I was to do an f. seek at the beginning and write out an r as my second one there and now if I run that and go back to the file then you can see that the r only overwrote the T in test it didn't delete the rest of the content so using file seek whenever I am writing to files it can get a little confusing and I don't use it a whole lot uh but maybe there are some use cases out there that you guys will find it uh useful for okay so let's go ahead and pull all of this together and use read and write on multiple files at the same time so we're going to use this to make a copy of our test.txt file um so let's go ahead and delete our test 2.txt file here so that we don't confuse the two and I'm going to go ahead and close that there okay so I'm going to go ahead and get rid of these here so first let's open our original test.txt file in a read mode and instead of f here I'm going to use RF and I'll just say RF there for read file since this is the file that we're going to read from in order to write to our copy so now within this width statement here I'm going to go ahead and let's go ahead and copy all of this and paste another uh open within here and I'm going to call this a testore copy. txt and I'm going to open this in write mode and I'm going to call this WF for WR file now you can actually put both of these open statements on a single line separated by a comma but I think readability here is pretty important and mixing those two on one line is sometimes difficult to understand at least for me uh so this is usually how I work with multiple files at a time is putting them on two different lines um one nested Within in the other okay so now within here we have two files open RF for reading our original file and WF for writing to our copy now to do this it's just as easy as saying for line in RF we want to do a WF do write of that line okay so now let's walk over this one more time so we have our original file opened and we're reading from from that file and we have a file that doesn't exist yet that's our copy and we're writing to that file and we're saying for each line in our original file write that line to WF which is the file that we are copying to so if I go ahead and run that then you can see that it created this test copy. text file and if I open this you can see that it is an exact copy of our original okay and lastly let's look at how we can do something similar and copy a large picture file now this is going to be slightly different so if I look in my current directory that has my python script that I'm currently running I also have a picture of my dog here when he was a puppy and let's go ahead and try to copy this picture file using file objects in Python now this file here is called bronx. JPEG and if I just try to replace our text files with um these picture files and down here I'll call this Bronx copy. jpeg now this is exactly the same as our previous example but we're trying to use a picture instead of a text file now if I try to run this you can see that we got an error down here that says uh UTF codec can't decode bite in position zero so in order to work with images we're going to have to open these files in binary mode and all that means is that we're going to be reading and writing bytes instead of working with text now I'm not going to go into the specifics but if anyone is curious about the differences then I'll try to leave some resources in the description section as to what exactly that means but for this case in order to work with these pictures to use binary mode we can just append a b to our read uh R here and our write w there so now with that one simple change if I save that and now run it then you can see that we do have this copied picture file here and if I go over to finder then you can see that that file um copied exactly the way that the original is okay so last thing here now I said earlier that sometimes you want more control over exactly what you're reading and writing so instead of doing this line by line let's instead uh do this in specific chunk sizes and we saw something like this earlier when we were learning how to read our files so to do this let's just do a chunk size and we'll set this equal to 496 now you can choose different sizes but this is the one that I'm choosing here um so now let's do an RF chunk and we're just going to read in a chunk of our read file here so I'll say rf. read and I'll pass in this chunk size so now we're reading that much data from our original picture so now let's create a loop that will write these chunks to our copy until there's nothing left to read from the original and if you remember from earlier to do this we can do a while loop and while the length of this chunk here is greater than zero then we want to take our copy file and write that chunk to it so I'm going to write this chunk to our copy and then to keep this from being an infinite loop I have to read in the next chunk size so I'll paste that in there to read in uh the next Chunk from the original file so now if I come up here and I delete this copy that we just created so I'm going to delete that and now I'm going to go ahead and rerun it using the code that we just wrote and you can see that it made that copy there and if I go back over to finder and open up that copy then you can see that it made an exact copy of our original original okay so I think that's going to do it for this video uh there's a lot more that we could look at with files and I'll plan on putting together some tutorials in the near future for how to work with uh temporary files inmemory files and things like that but I hope that this was a good introduction into working with files and some of the useful things that we can do with them now if you do have any questions then feel free to ask in the comment section below and I'll do my best to answer those uh be sure to subscribe for future videos and thank you all for watching