Write List to File in Python (7 Best Methods)

File storage is a crucial and dependable storage management technique in real-world systems because it enables effective data storage, organization, and access.

It’s fairly easy to write list to file in Python, and in this article, we’ll go through various methods with examples.

Cases when writing List to File is helpful

1. Reports: Generally in organizations, we need to generate reports on data based on our queries which we’ll share among the people.
   • Ex: Employee IDs list whose birthday is in the present month.
2. Sometimes we will need to persist list data in files so that they can be used in systems for different purposes.
   • Ex: In Data science or machine learning, we need to save the results of one method to a file so that it will be input for another method.
3. Sitemap: In this case, we’ll need to keep our sitemap.xml file updated with all the dynamic data that we want google to crawl.
   • Ex: Most of the popular websites will keep their sitemap.xml updated with all their available pages including the newly added ones for google to crawl.
4. If you want to access only a few items at a time from a large list, keeping them in memory for a long time is not memory efficient. In this case, we can save this large list into multiple files with fewer elements and we can access them based on the requirement.

Recommended Ways to write list to file in Python

Depending upon your use case, you have to decide the best method among all the listed methods in this article.

• If you want to write a list to file and are not bothered about the readability of it rather you want it to persist or read it back, go for Using pickle’s dump
• If you just want to write a list data to file in a human-readable format go for Using writelines(list) method. But adding a delimiter after each item will be hard.
• Though the joblib module takes a comparatively long time to write a list to file, it works faster and it’s memory efficient when dealing with numpy’s arrays. So If you are working on large numpy arrays, Using joblib module’s dump() is the better method.

Time and Memory Utilizations

Method	Time In Seconds	Memory in MBs
Using Pickle’s dump()	17	85
Using write() along with join()	18	459
Using writelines(list)	18	84
Using write() in for loop	24	85
Using json’s dump()	30	85
Using writelines() along with join()	37	272
Using joblib dump()	53	85
Using numpy’s savetxt()	67	3299

How to write list to file in Python?

There are 7 methods to write a list to file in python and will use the below list of popular movies as an example for all the methods.

movies_list = ["The Godfather", "The Batman", "Dune", "Avatar: The Way of Water",
               "Black Adam", "Top Gun: Maverick", "Titanic"]

Method 1: Using write() function

a) using write() function

In this example, we’ll create one function which takes the filename and list of movies as input and write the data to the given file.

def write_list_to_file(file_name, list_data):
    # opening the file in write mode
    with open(file_name, 'w') as file:
        # iterate through list_data and write each item at a time using write() function
        for item in list_data:
            file.write(str(item) + '\n')


movies_list = ["The Godfather", "The Batman", "Dune", "Avatar: The Way of Water",
               "Black Adam", "Top Gun: Maverick", "Titanic"]

write_list_to_file('movies.txt', movies_list)

In this program, we used the open() function and with statement to open the file in write mode. After that, we iterate through every item in the list_data and write the item to the file using file.write() method.

Output:

The Godfather
The Batman
Dune
Avatar: The Way of Water
Black Adam
Top Gun: Maverick
Titanic

Instead of iterating over the list of items every time and writing them one by one, we can use python’s join function to convert list_data into a string separated by new lines and write the whole string at a time.

movies_list = ["The Godfather", "The Batman", "Dune", "Avatar: The Way of Water",
               "Black Adam", "Top Gun: Maverick", "Titanic"]

f = open('movies.txt', 'w')
f.write("\n".join(movies_list))

The Godfather
The Batman
Dune
Avatar: The Way of Water
Black Adam
Top Gun: Maverick
Titanic

Now let’s see how can we read the list from a file in python.

b) Read list from a file using read()

We already wrote list items to file separated by a new line. The below program will read these items from the file and create a Python list.

def write_list_to_file(file_name, list_data):
    # opening the file in write mode
    with open(file_name, 'w') as file:
        file.write("\n".join(list_data))


def read_list_from_file(file_name):
    with open(file_name, 'r') as file:
        # type of file_data is <class 'str'>
        file_data = file.read().splitlines()
        return file_data


movies_list = ["The Godfather", "The Batman", "Dune", "Avatar: The Way of Water",
               "Black Adam", "Top Gun: Maverick", "Titanic"]

write_list_to_file('movies.txt', movies_list)
print(read_list_from_file('movies.txt'))

Output:

['The Godfather', 'The Batman', 'Dune', 'Avatar: The Way of Water', 'Black Adam', 'Top Gun: Maverick', 'Titanic']

Method 2: Using writelines() function

a) write using writelines() function

def write_list_to_file(file_name, list_data):
    with open(file_name, 'w') as file:
        file.writelines(list_data)


movies_list = ["The Godfather", "The Batman", "Dune", "Avatar: The Way of Water",
               "Black Adam", "Top Gun: Maverick", "Titanic"]

write_list_to_file('movies.txt', movies_list)

In the above program after opening the file, we used file.writelines() function which will write every item from the list into a file without a separator.

Output:

The GodfatherThe BatmanDuneAvatar: The Way of WaterBlack AdamTop Gun: MaverickTitanic

writelines() will be a faster method for writing list to file compared to the write() function. But the problem here is data is written to file without a separator. This will make reading back the list from a file hard as there is no separator between the items.

So to overcome this, we can convert the list of items to a string using the join() function and write the data to a file using writelines() function.

def write_list_to_file(file_name, list_data):
    with open(file_name, 'w') as file:
        file.writelines('\n'.join(list_data))


movies_list = ["The Godfather", "The Batman", "Dune", "Avatar: The Way of Water",
               "Black Adam", "Top Gun: Maverick", "Titanic"]

write_list_to_file('movies.txt', movies_list)

Output:

The Godfather
The Batman
Dune
Avatar: The Way of Water
Black Adam
Top Gun: Maverick
Titanic

b) Read with readlines()

We can either use Method 1.b or the below program to read a list from the file using lambda as the output of the two methods is the same.

def write_list_to_file(file_name, list_data):
    with open(file_name, 'w') as file:
        file.writelines('\n'.join(list_data))


def read_file(filename):
    with open(filename, 'r') as file:
        list_data = file.readlines()
        # this will take each item and remove or strip trailing '\n' from the item.
        output_list = list(map(lambda item: item.strip('\n'), list_data))
        return output_list


movies_list = ["The Godfather", "The Batman", "Dune", "Avatar: The Way of Water",
               "Black Adam", "Top Gun: Maverick", "Titanic"]

write_list_to_file('movies.txt', movies_list)

print(read_file("movies.txt"))

If you print list_data from line 9, it will look like the one below.

['The Godfather\n', 'The Batman\n', 'Dune\n', 'Avatar: The Way of Water\n', 'Black Adam\n', 'Top Gun: Maverick\n', 'Titanic']

Every item is having trailing newline character. Here we used lambda to strip or remove the new line ‘\n’ from each item.

Output:

['The Godfather', 'The Batman', 'Dune', 'Avatar: The Way of Water', 'Black Adam', 'Top Gun: Maverick', 'Titanic']

Method 3: Using Python3’s print() function

movies_list = ["The Godfather", "The Batman", "Dune", "Avatar: The Way of Water",
               "Black Adam", "Top Gun: Maverick", "Titanic"]

with open('movies.txt', 'w') as output_file:
    print("\n".join(movies_list), file=output_file)

In this program, after opening the file in write mode, we passed the movies list string and file name as arguments to the print() function. This print() function writes the string to the given file.

Output:

The Godfather
The Batman
Dune
Avatar: The Way of Water
Black Adam
Top Gun: Maverick
Titanic

We can use readlines() or read() functions to read the list from a file as in the 1.b or method 2.b.

Method 4: Using numpy module

a) Using numpy’s savetxt()

import numpy as np

movies_list = ["The Godfather", "The Batman", "Dune", "Avatar: The Way of Water",
               "Black Adam", "Top Gun: Maverick", "Titanic"]

np.savetxt('movies.txt', [movies_list], delimiter=',', fmt="%s")

numpy’s savetxt() function expects input collection in the array format and saves the array to a given text file.

Here in this example, we converted the list to an array using type casting and we are passing the fmt as %s (string), so that the ultimate_movies_of_all_time will be converted to a string separated by a ‘,’ delimiter.

savetxt() also takes the below arguments as input. For more information visit NumPy’s documentation.

numpy.savetxt(fname, X, fmt=’%.18e’, delimiter=’ ‘, newline=’\n’, header=”, footer=”, comments=’# ‘, encoding=None)

Note: In our above code example, We didn’t use a new line as a delimiter as NumPy’s loadtxt() doesn’t allow reading newline delimited file data when we want to read the list from the file.

Output:

The Godfather,The Batman,Dune,Avatar: The Way of Water,Black Adam,Top Gun: Maverick,Titanic

b) Read with numpy.loadtxt()

import numpy as np

movies_list = ["The Godfather", "The Batman", "Dune", "Avatar: The Way of Water",
               "Black Adam", "Top Gun: Maverick", "Titanic"]

np.savetxt('movies.txt', [movies_list], delimiter=',', fmt="%s")

lines = np.loadtxt("movies.txt", delimiter=",", dtype=str)

print(list(lines))

As loadtxt() function returns an array instead of the list we converted it to a list explicitly.

If we try to use ‘\n’ as a delimiter, the interpreter will throw the below error. That’s the reason we used ‘,’ as a delimiter while writing the list to file.

TypeError: control character ‘delimiter’ cannot be a newline (\r or \n).

Output:

['The Godfather', 'The Batman', 'Dune', 'Avatar: The Way of Water', 'Black Adam', 'Top Gun: Maverick', 'Titanic']

Method 5: Using pickle module

pickle module is a built-in library in python. It is used to serialize and deserialize python objects. Before knowing how to write a list to file in python using the pickle module, let’s understand what serialization and deserialization in python mean.

Serialization

Serialization means converting objects or collections of objects into a serialized format typically byte format. And that format can be stored and transmitted over the network and can be reconverted to the original object or collection of objects.

Deserialization

Deserialization is the opposite process of serialization, where serialized format can be converted to objects.

To know more about Serialization, please visit this insightful article about the importance of serialization and deserialization in python.

Now, let’s see how to write a list to file using the pickle module.

a) Using pickle’s dump() function

import pickle

movies_list = ["The Godfather", "The Batman", "Dune", "Avatar: The Way of Water",
               "Black Adam", "Top Gun: Maverick", "Titanic"]
# to write
with open('movies.txt', 'wb') as file:
    pickle.dump(movies_list, file)

First, we opened the movies.txt file in wb mode which means the file is opened for writing in binary mode. This is because pickle.dump() method writes ultimate_movies_of_all_time list to file in bytes format. Try opening the output file once, you will see some binary data.

We can now convert this bytes data in the file to the original python object again using the pickle module’s load() method. Let’s see how.

b) Read using pickle’s load()

import pickle

movies_list = ["The Godfather", "The Batman", "Dune", "Avatar: The Way of Water",
               "Black Adam", "Top Gun: Maverick", "Titanic"]
# to write
with open('movies.txt.txt', 'wb') as file:
    pickle.dump(movies_list, file)
# to read
with open('movies.txt.txt', 'rb') as file:
    movies = pickle.load(file)
print(movies)

Output:

['The Godfather', 'The Batman', 'Dune', 'Avatar: The Way of Water', 'Black Adam', 'Top Gun: Maverick', 'Titanic']

Method 6: Using the joblib module

It works the same as pickle module, which serializes the python object and deserializes the serialized format into an object. But before using the joblib module, you need to install it as it does not come with the python standard library like the pickle module.

Please use the below pip command to install joblib.

pip install joblib

Now let’s see how can we write a list to file and also read a list from the file using the joblib module.

import joblib

movies_list = ["The Godfather", "The Batman", "Dune", "Avatar: The Way of Water",
               "Black Adam", "Top Gun: Maverick", "Titanic"]
# to write
with open('movies.txt', 'wb') as file:
    joblib.dump(movies_list, file)

# to read
with open('movies.txt', 'rb') as file:
    movies = joblib.load(file)

print(movies)

Output:

['The Godfather', 'The Batman', 'Dune', 'Avatar: The Way of Water', 'Black Adam', 'Top Gun: Maverick', 'Titanic']

You might be thinking, what is the difference between joblib and pickle, and which is best?

• joblib comparatively works faster and is more memory efficient than pickle while dealing with large datasets. The reason is joblib uses C based backend to serialize and deserialize.
• joblib needs to be installed separately as it doesn’t come up with python’s standard library whereas the pickle module comes with python’s standard library.
• joblib comes with more features that are not available in the pickle module like compressing the serialized data and the ability to do both serialization and deserialization in parallel.

Method 7: Using the json module’s dump()

In the below example we will see, how can we write a list to file and read list from file using json module’s methods.

import json

movies_list = ["The Godfather", "The Batman", "Dune", "Avatar: The Way of Water",
               "Black Adam", "Top Gun: Maverick", "Titanic"]
# Open movies file for writing
with open('movies.txt', 'w') as file:
    json.dump(movies_list, file)
    # Open movies file for reading
with open('movies.txt', 'r') as file:
    movies_list = json.load(file)
print('movies list:', movies_list)

Output:

movies list: ['The Godfather', 'The Batman', 'Dune', 'Avatar: The Way of Water', 'Black Adam', 'Top Gun: Maverick', 'Titanic']

And movies.txt will have the below data.

["The Godfather", "The Batman", "Dune", "Avatar: The Way of Water", "Black Adam", "Top Gun: Maverick", "Titanic"]

Frequently Asked Questions

For more insightful articles, follow codethreads.dev!!