PY0101EN-4-2-WriteFile.ipynb

{"cells":[{"cell_type":"markdown","metadata":{},"source":["\u003cp style=\"text-align:center\"\u003e\n","    \u003ca href=\"https://skills.network/?utm_medium=Exinfluencer\u0026utm_source=Exinfluencer\u0026utm_content=000026UJ\u0026utm_term=10006555\u0026utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkPY0101ENSkillsNetwork1005-2022-01-01\" target=\"_blank\"\u003e\n","    \u003cimg src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/assets/logos/SN_web_lightmode.png\" width=\"200\" alt=\"Skills Network Logo\"  /\u003e\n","    \u003c/a\u003e\n","\u003c/p\u003e\n","\n","\n","# Write and Save Files in Python\n","\n","\n","Estimated time needed: **25** minutes\n","    \n","\n","## Objectives\n","\n","After completing this lab you will be able to:\n","\n","* Write to files using Python libraries\n"]},{"cell_type":"markdown","metadata":{},"source":["\u003ch2\u003eTable of Contents\u003c/h2\u003e\n","\u003cdiv class=\"alert alert-block alert-info\" style=\"margin-top: 20px\"\u003e\n","    \u003cul\u003e\n","        \u003cli\u003e\u003ca href=\"write\"\u003eWriting Files\u003c/a\u003e\u003c/li\u003e\n","        \u003cli\u003e\u003ca href=\"Append\"\u003eAppending Files\u003c/a\u003e\u003c/li\u003e\n","        \u003cli\u003e\u003ca href=\"add\"\u003eAdditional File modes\u003c/a\u003e\u003c/li\u003e\n","        \u003cli\u003e\u003ca href=\"copy\"\u003eCopy a File\u003c/a\u003e\u003c/li\u003e\n","    \u003c/ul\u003e\n","\n","\u003c/div\u003e\n","\n","\u003chr\u003e\n"]},{"cell_type":"markdown","metadata":{},"source":["\u003ch2 id=\"write\"\u003eWriting Files\u003c/h2\u003e\n"]},{"cell_type":"markdown","metadata":{},"source":[" We can open a file object using the method \u003ccode\u003ewrite()\u003c/code\u003e to save the text file to a list. To write to a file, the mode argument must be set to **w**. Let’s write a file **Example2.txt** with the line: **“This is line A”**\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Write line to file\n","exmp2 = '/resources/data/Example2.txt'\n","with open(exmp2, 'w') as writefile:\n","    writefile.write(\"This is line A\")"]},{"cell_type":"markdown","metadata":{},"source":[" We can read the file to see if it worked:\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Read file\n","\n","with open(exmp2, 'r') as testwritefile:\n","    print(testwritefile.read())"]},{"cell_type":"markdown","metadata":{},"source":["We can write multiple lines:\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Write lines to file\n","\n","with open(exmp2, 'w') as writefile:\n","    writefile.write(\"This is line A\\n\")\n","    writefile.write(\"This is line B\\n\")"]},{"cell_type":"markdown","metadata":{},"source":["The method \u003ccode\u003e.write()\u003c/code\u003e works similar to the method \u003ccode\u003e.readline()\u003c/code\u003e, except instead of reading a new line it writes a new line. The process is illustrated in the figure. The different colour coding of the grid represents a new line added to the file after each method call.\n"]},{"cell_type":"markdown","metadata":{},"source":["\u003cimg src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-PY0101EN-SkillsNetwork/labs/Module%204/images/WriteLine.png\" width=\"500\"\u003e\n"]},{"cell_type":"markdown","metadata":{},"source":["You can check the file to see if your results are correct \n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Check whether write to file\n","\n","with open(exmp2, 'r') as testwritefile:\n","    print(testwritefile.read())"]},{"cell_type":"markdown","metadata":{},"source":[" We write a list to a **.txt** file  as follows:\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Sample list of text\n","\n","Lines = [\"This is line A\\n\", \"This is line B\\n\", \"This is line C\\n\"]\n","Lines"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Write the strings in the list to text file\n","\n","with open('Example2.txt', 'w') as writefile:\n","    for line in Lines:\n","        print(line)\n","        writefile.write(line)"]},{"cell_type":"markdown","metadata":{},"source":[" We can verify the file is written by reading it and printing out the values:  \n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Verify if writing to file is successfully executed\n","\n","with open('Example2.txt', 'r') as testwritefile:\n","    print(testwritefile.read())"]},{"cell_type":"markdown","metadata":{},"source":["However, note that setting the mode to __w__ overwrites all the existing data in the file.\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["with open('Example2.txt', 'w') as writefile:\n","    writefile.write(\"Overwrite\\n\")\n","with open('Example2.txt', 'r') as testwritefile:\n","    print(testwritefile.read())"]},{"cell_type":"markdown","metadata":{},"source":["\u003chr\u003e\n","\u003ch2 id=\"Append\"\u003eAppending Files\u003c/h2\u003e\n"]},{"cell_type":"markdown","metadata":{},"source":[" We can write to files without losing any of the existing data as follows by setting the mode argument to append: **a**.  you can append a new line as follows:\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Write a new line to text file\n","\n","with open('Example2.txt', 'a') as testwritefile:\n","    testwritefile.write(\"This is line C\\n\")\n","    testwritefile.write(\"This is line D\\n\")\n","    testwritefile.write(\"This is line E\\n\")"]},{"cell_type":"markdown","metadata":{},"source":[" You can verify the file has changed by running the following cell:\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Verify if the new line is in the text file\n","\n","with open('Example2.txt', 'r') as testwritefile:\n","    print(testwritefile.read())"]},{"cell_type":"markdown","metadata":{},"source":["\u003chr\u003e\n","\u003ch2 id=\"add\"\u003eAdditional modes\u003c/h2\u003e \n"]},{"cell_type":"markdown","metadata":{},"source":["It's fairly ineffecient to open the file in **a** or **w** and then reopening it in **r** to read any lines. Luckily we can access the file in the following modes:\n","- **r+** : Reading and writing. Cannot truncate the file.\n","- **w+** : Writing and reading. Truncates the file.\n","- **a+** : Appending and Reading. Creates a new file, if none exists.\n","You dont have to dwell on the specifics of each mode for this lab. \n"]},{"cell_type":"markdown","metadata":{},"source":["Let's try out the __a+__ mode:\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["with open('Example2.txt', 'a+') as testwritefile:\n","    testwritefile.write(\"This is line E\\n\")\n","    print(testwritefile.read())"]},{"cell_type":"markdown","metadata":{},"source":["There were no errors but \u003ccode\u003eread()\u003c/code\u003e also did not output anything. This is because of our location in the file.\n"]},{"cell_type":"markdown","metadata":{},"source":["Most of the file methods we've looked at work in a certain location in the file. \u003ccode\u003e.write() \u003c/code\u003e writes at a certain location in the file. \u003ccode\u003e.read()\u003c/code\u003e reads at a certain location in the file and so on. You can think of this as moving your pointer around in the notepad to make changes at specific location.\n"]},{"cell_type":"markdown","metadata":{},"source":["Opening the file in **w** is akin to opening the .txt file, moving your cursor to the beginning of the text file, writing new text and deleting everything that follows.\n","Whereas opening the file in **a** is similiar to opening the .txt file, moving your cursor to the very end and then adding the new pieces of text. \u003cbr\u003e\n","It is often very useful to know where the 'cursor' is in a file and be able to control it. The following methods allow us to do precisely this -\n","- \u003ccode\u003e.tell()\u003c/code\u003e - returns the current position in bytes\n","- \u003ccode\u003e.seek(offset,from)\u003c/code\u003e - changes the position by 'offset' bytes with respect to 'from'. From can take the value of 0,1,2 corresponding to beginning, relative to current position and end\n"]},{"cell_type":"markdown","metadata":{},"source":["Now lets revisit **a+**\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["with open('Example2.txt', 'a+') as testwritefile:\n","    print(\"Initial Location: {}\".format(testwritefile.tell()))\n","    \n","    data = testwritefile.read()\n","    if (not data):  #empty strings return false in python\n","            print('Read nothing') \n","    else: \n","            print(testwritefile.read())\n","            \n","    testwritefile.seek(0,0) # move 0 bytes from beginning.\n","    \n","    print(\"\\nNew Location : {}\".format(testwritefile.tell()))\n","    data = testwritefile.read()\n","    if (not data): \n","            print('Read nothing') \n","    else: \n","            print(data)\n","    \n","    print(\"Location after read: {}\".format(testwritefile.tell()) )"]},{"cell_type":"markdown","metadata":{},"source":["Finally, a note on the difference between **w+** and **r+**. Both of these modes allow access to read and write methods, however, opening a file in **w+** overwrites it and deletes all pre-existing data. \u003cbr\u003e\n","To work with a file on existing data, use **r+** and **a+**. While using **r+**, it can be useful to add a \u003ccode\u003e.truncate()\u003c/code\u003e method at the end of your data. This will reduce the file to your data and delete everything that follows. \u003cbr\u003e\n","In the following code block, Run the code as it is first and then run it with the \u003ccode\u003e.truncate()\u003c/code\u003e.\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["with open('Example2.txt', 'r+') as testwritefile:\n","    data = testwritefile.readlines()\n","    testwritefile.seek(0,0) #write at beginning of file\n","   \n","    testwritefile.write(\"Line 1\" + \"\\n\")\n","    testwritefile.write(\"Line 2\" + \"\\n\")\n","    testwritefile.write(\"Line 3\" + \"\\n\")\n","    testwritefile.write(\"finished\\n\")\n","    #Uncomment the line below\n","    #testwritefile.truncate()\n","    testwritefile.seek(0,0)\n","    print(testwritefile.read())\n","    "]},{"cell_type":"markdown","metadata":{},"source":["\u003chr\u003e\n"]},{"cell_type":"markdown","metadata":{},"source":["\u003ch2 id=\"copy\"\u003eCopy a File\u003c/h2\u003e \n"]},{"cell_type":"markdown","metadata":{},"source":["Let's copy the file **Example2.txt** to the file **Example3.txt**:\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Copy file to another\n","\n","with open('Example2.txt','r') as readfile:\n","    with open('Example3.txt','w') as writefile:\n","          for line in readfile:\n","                writefile.write(line)"]},{"cell_type":"markdown","metadata":{},"source":["We can read the file to see if everything works:\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Verify if the copy is successfully executed\n","\n","with open('Example3.txt','r') as testwritefile:\n","    print(testwritefile.read())"]},{"cell_type":"markdown","metadata":{},"source":[" After reading files, we can also write data into files and save them in different file formats like **.txt, .csv, .xls (for excel files) etc**. You will come across these in further examples\n"]},{"cell_type":"markdown","metadata":{},"source":["**NOTE:** If you wish to open and view the `example3.txt` file, download this lab [here](https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-PY0101EN-SkillsNetwork/labs/Module%204/PY0101EN-4-2-WriteFile.ipynb) and run it locally on your machine. Then go to the working directory to ensure the `example3.txt` file exists and contains the summary data that we wrote.\n"]},{"cell_type":"markdown","metadata":{},"source":["\u003chr\u003e\n"]},{"cell_type":"markdown","metadata":{},"source":["\u003ch2\u003e Exercise \u003c/h2\u003e\n"]},{"cell_type":"markdown","metadata":{},"source":["Your local university's Raptors fan club maintains a register of its active members on a .txt document. Every month they update the file by removing the members who are not active. You have been tasked with automating this with your Python skills. \u003cbr\u003e\n","Given the file `currentMem`, Remove each member with a 'no' in their Active column. Keep track of each of the removed members and append them to the `exMem` file. Make sure that the format of the original files in preserved.   (*Hint: Do this by reading/writing whole lines and ensuring the header remains* )\n","\u003cbr\u003e\n","Run the code block below prior to starting the exercise. The skeleton code has been provided for you. Edit only the `cleanFiles` function.\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["#Run this prior to starting the exercise\n","from random import randint as rnd\n","\n","memReg = 'members.txt'\n","exReg = 'inactive.txt'\n","fee =('yes','no')\n","\n","def genFiles(current,old):\n","    with open(current,'w+') as writefile: \n","        writefile.write('Membership No  Date Joined  Active  \\n')\n","        data = \"{:^13}  {:\u003c11}  {:\u003c6}\\n\"\n","\n","        for rowno in range(20):\n","            date = str(rnd(2015,2020))+ '-' + str(rnd(1,12))+'-'+str(rnd(1,25))\n","            writefile.write(data.format(rnd(10000,99999),date,fee[rnd(0,1)]))\n","\n","\n","    with open(old,'w+') as writefile: \n","        writefile.write('Membership No  Date Joined  Active  \\n')\n","        data = \"{:^13}  {:\u003c11}  {:\u003c6}\\n\"\n","        for rowno in range(3):\n","            date = str(rnd(2015,2020))+ '-' + str(rnd(1,12))+'-'+str(rnd(1,25))\n","            writefile.write(data.format(rnd(10000,99999),date,fee[1]))\n","\n","\n","genFiles(memReg,exReg)\n"]},{"cell_type":"markdown","metadata":{},"source":["Now that you've run the prerequisite code cell above, which prepared the files for this exercise, you are ready to move on to the implementation.\n","\n","#### **Exercise:** Implement the cleanFiles function in the code cell below.\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["  \n","'''\n","The two arguments for this function are the files:\n","    - currentMem: File containing list of current members\n","    - exMem: File containing list of old members\n","    \n","    This function should remove all rows from currentMem containing 'no' \n","    in the 'Active' column and appends them to exMem.\n","    '''\n","def cleanFiles(currentMem, exMem):\n","    # TODO: Open the currentMem file as in r+ mode\n","        #TODO: Open the exMem file in a+ mode\n","\n","        #TODO: Read each member in the currentMem (1 member per row) file into a list.\n","        # Hint: Recall that the first line in the file is the header.\n","\n","        #TODO: iterate through the members and create a new list of the innactive members\n","\n","        # Go to the beginning of the currentMem file\n","        # TODO: Iterate through the members list. \n","        # If a member is inactive, add them to exMem, otherwise write them into currentMem\n","\n","        \n","    \n","    pass # Remove this line when done implementation\n","\n","\n","# The code below is to help you view the files.\n","# Do not modify this code for this exercise.\n","memReg = 'members.txt'\n","exReg = 'inactive.txt'\n","cleanFiles(memReg,exReg)\n","\n","\n","headers = \"Membership No  Date Joined  Active  \\n\"\n","with open(memReg,'r') as readFile:\n","    print(\"Active Members: \\n\\n\")\n","    print(readFile.read())\n","    \n","with open(exReg,'r') as readFile:\n","    print(\"Inactive Members: \\n\\n\")\n","    print(readFile.read())\n","                \n","    "]},{"cell_type":"markdown","metadata":{},"source":["The code cell below is to verify your solution. Please do not modify the code and run it to test your implementation of `cleanFiles`.\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["def testMsg(passed):\n","    if passed:\n","       return 'Test Passed'\n","    else :\n","       return 'Test Failed'\n","\n","testWrite = \"testWrite.txt\"\n","testAppend = \"testAppend.txt\" \n","passed = True\n","\n","genFiles(testWrite,testAppend)\n","\n","with open(testWrite,'r') as file:\n","    ogWrite = file.readlines()\n","\n","with open(testAppend,'r') as file:\n","    ogAppend = file.readlines()\n","\n","try:\n","    cleanFiles(testWrite,testAppend)\n","except:\n","    print('Error')\n","\n","with open(testWrite,'r') as file:\n","    clWrite = file.readlines()\n","\n","with open(testAppend,'r') as file:\n","    clAppend = file.readlines()\n","        \n","# checking if total no of rows is same, including headers\n","\n","if (len(ogWrite) + len(ogAppend) != len(clWrite) + len(clAppend)):\n","    print(\"The number of rows do not add up. Make sure your final files have the same header and format.\")\n","    passed = False\n","    \n","for line in clWrite:\n","    if  'no' in line:\n","        passed = False\n","        print(\"Inactive members in file\")\n","        break\n","    else:\n","        if line not in ogWrite:\n","            print(\"Data in file does not match original file\")\n","            passed = False\n","print (\"{}\".format(testMsg(passed)))\n","    \n","\n"]},{"cell_type":"markdown","metadata":{},"source":["\u003cdetails\u003e\u003csummary\u003eClick here for the solution\u003c/summary\u003e\n","\n","```python\n","def cleanFiles(currentMem,exMem):\n","    with open(currentMem,'r+') as writeFile: \n","        with open(exMem,'a+') as appendFile:\n","            #get the data\n","            writeFile.seek(0)\n","            members = writeFile.readlines()\n","            #remove header\n","            header = members[0]\n","            members.pop(0)\n","                \n","            inactive = [member for member in members if ('no' in member)]\n","            '''\n","            The above is the same as \n","\n","            for member in members:\n","            if 'no' in member:\n","                inactive.append(member)\n","            '''\n","            #go to the beginning of the write file\n","            writeFile.seek(0) \n","            writeFile.write(header)\n","            for member in members:\n","                if (member in inactive):\n","                    appendFile.write(member)\n","                else:\n","                    writeFile.write(member)      \n","            writeFile.truncate()\n","                \n","memReg = 'members.txt'\n","exReg = 'inactive.txt'\n","cleanFiles(memReg,exReg)\n","\n","# code to help you see the files\n","\n","headers = \"Membership No  Date Joined  Active  \\n\"\n","\n","with open(memReg,'r') as readFile:\n","    print(\"Active Members: \\n\\n\")\n","    print(readFile.read())\n","    \n","with open(exReg,'r') as readFile:\n","    print(\"Inactive Members: \\n\\n\")\n","    print(readFile.read())\n","    \n","```\n","\n","\u003c/details\u003e\n","\n","    \n"]},{"cell_type":"markdown","metadata":{},"source":["\u003chr\u003e\n","\u003ch2\u003eThe last exercise!\u003c/h2\u003e\n","\u003cp\u003eCongratulations, you have completed this lesson and hands-on lab in Python. \n","\u003chr\u003e\n"]},{"cell_type":"markdown","metadata":{},"source":["## Author\n","\n","\u003ca href=\"https://www.linkedin.com/in/joseph-s-50398b136/?utm_medium=Exinfluencer\u0026utm_source=Exinfluencer\u0026utm_content=000026UJ\u0026utm_term=10006555\u0026utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkPY0101ENSkillsNetwork1005-2022-01-01\" target=\"_blank\"\u003eJoseph Santarcangelo\u003c/a\u003e\n","\n","\n","### Other Contributors\n","\u003ca href=\"www.linkedin.com/in/jiahui-mavis-zhou-a4537814a\"\u003eMavis Zhou\u003c/a\u003e\n","\n","\n","## Change Log\n","\n","\n","|  Date (YYYY-MM-DD) |  Version | Changed By  |  Change Description |\n","|---|---|---|---|\n","| 2022-01-10  | 2.1  | Malika  | Removed the readme for GitShare|\n","| 2020-10-16  | 1.3  | Arjun Swani  |  Added exercise |\n","| 2020-10-16  | 1.2  | Arjun Swani  |  Added section additional file modes |\n","| 2020-10-16  | 1.1  | Arjun Swani |  Made append a different section |\n","| 2020-08-28  | 0.2  | Lavanya  |  Moved lab to course repo in GitLab |\n","\n","\n","\n","\u003chr\u003e\n","\n","## \u003ch3 align=\"center\"\u003e © IBM Corporation 2020. All rights reserved. \u003ch3/\u003e\n"]}],"metadata":{"kernelspec":{"display_name":"Python","language":"python","name":"conda-env-python-py"},"language_info":{"name":""}},"nbformat":4,"nbformat_minor":4}