Neat Python Feature - Pickle!

[makerhacks]

Back when I was learning C and C++, I often reused a piece of code that would write data from memory to a file and read it back again to persist state.

Just dump my struct to a file.

Any time the software needed to restart - say after a system crash - it could just keep on trugging along as if not much had happened :)

It is a very long time since I coded in C++ (outside of Arduino and C#) but that concept stuck with me.

I had no idea there was a similar functionality in Python until today (yes, I am late to the party!)

Introducing Pickle

Now if this is not news to you I do not blame you for rolling your eyes, but if you haven't seen Pickle then you are in for a treat.

Start with, as you would expect, importing pickle.

Then use the following code to persist and recall your data:

def persist(item_list):

    with open('my_list.txt', 'wb') as fp:
        pickle.dump(item_list, fp)

Supply this function with your array/list.

And then to get it back ...

def retrieve():

    item_list = []

    try:

        with open('my_list.txt', 'rb') as fp:
            item_list = pickle.load(fp)
    except:
        print("Pickle bork - ah well!")

    return item_list

So what happens is your array or list goes from memory to a file on your file system, and then you can reverse the process :)

Obviously the file needs to exist, so just use the shell command as follows:

touch my_list.txt

(this will create a placeholder file ready for your script to run)

Neat, eh?

Comments

[lemony-cricket]

Pickle is great. I find it most useful when I am dealing with a lot of manually-entered data and an unstable script I wrote off-the-cuff. If I'm an idiot and break something, as long as I keep pickling it to disk as I go, I can just sort of pick it up where I left off.

[themarkymark]

It’s a really good feature when you are in a pickle and need to get your data back.

Thanks I will be here all week.

[makerhacks]

[veleje]

Beginner question:

with open('my_list.txt', 'wb') as fp:

Shouldn't this command create a file if it is not existing already?

[makerhacks]

Yes but in most cases your scripts will attempt to read first so touching the file is just a precaution, obviously you can check for file existence in your code too and even prepare it with an empty array :)

[crokkon]

.

[makerhacks]

Ooh, nice! Thank you :)

[joseangelperez]

Friend had not noticed that Pickle option, which is efficient. Thanks for the input, Greetings ;))

[geekpowered]

Pickle is highly useful! I'm not sure if I've used it to restore a whole state though. I only really remember using it with big data.

[makerhacks]

I am using it to check for IDs of records to see if I have seen them before, doesn't sound like a big deal but it saved me some migraines ;)

[cynthialee]

Thank you for introducing this very useful tool. I may use this to save some unstructured data or model.

[stelwok]

What this configuration, from your coding?