Recovering Photos from a Memory Card in Linux

Well, this is one of those "mostly for me" posts but, in case you find it useful (and I make no guarantees that you will! :-), here is how I recovered some photos from a memory card.

First: Don't use the card! If you do, there's a chance the data you're interested in may be overwritten.

(Optional: Instead of putting the card on the shelf, I created a backup of the data on it, like so (/dev/sdc being the sdcard; you'll have to check to find out which device is correct by mounting the card and looking at dmesg):

> dd if=/dev/sdc  of=~/image/tempfile

This may take quite a while, depending on the size of your card.)

Second: Get the TestDisk suite. We're going to be using PhotoRec.  If you have Ubuntu, you can do:

> sudo apt-get install testdisk

PhotoRec will actually recover a number of different types of files (see their website for more details). It works by reading the raw data and determining the file type, and then saving the file to a recovery location.

Third: Run PhotoRec.  If you copied your card to a file, you can run it like:

> photorec /path/to/sdcard/file

Otherwise, you can just do:

> sudo photorec

In either case, you'll have to select the "device" (be it file or actual hardware) you want to recover from, and then the partition type (if it's an SD card, you're probably fine with "Intel"), and then the actual partition (again, for an SD card, look for FAT32), and then the filesystem type (probably other), and then whether you want to search the whole partition, or just the free space (ie deleted files). I went for whole partition because I wasn't sure whether the files were deleted or not.  Finally, you have to choose where to put the recovered files (press Y when you've navigated to the correct spot).

Fourth: take a break and let PhotoRec work. Depending on the size of the card and the number of files that are on it (deleted or no!), this may take a little while. (I had about 1500 files recovered in about 20 minutes.)

Great! All your files are back! Woohoo! Now, it's just a matter of sorting through all 1500...

Unfortunately, the files don't have useful names (or mine didn't... names were like "f1071616.jpg"). However, JPG files (can) contain metadata, stored in EXIF. Most digital cameras automatically include information about the camera and the picture, like model, exposure, and... creation time! (Assuming you set the date/time on your camera...)

So we can, at least, give the files somewhat useful names now.

Fifth: Make a backup of your recovered files, just in case something goes wrong. You don't want to have to recover them again.

Sixth: It's time for the magic of Python!  Here's a handy little script I whipped up to pull out the creation time and rename the file. Note that you'll need to have the Python Image Library (PIL) installed.

renameJpgByDateTime.py:

#!/usr/bin/env python

from PIL import Image
from PIL.ExifTags import TAGS

import os, shutil

def renameJpg(fn):
    i = Image.open(fn)
    info = i._getexif()
    for tag, value in info.items():
        decoded = TAGS.get(tag, tag)
        if decoded == 'DateTimeOriginal':
            filePrefix = value + "-"
            shutil.move(fn, filePrefix + fn.replace(filePrefix, ""))

dirlist = os.listdir('.')

for fname in dirlist:
    if '.jpg' in fname:
        renameJpg(fname)

Then, go to the directory that contains the files, and run the script (don't forget to set it to executable):

> ./renameJpgByDateTime.py

And there you go!

Oh give me a home, where the GNU/Linuxen roam...

I'm running Linux at home again.

And I'm so very happy. (Seriously)

I am playing with programming again (currently trying to make my way through the Python Challenge; I'm on level 7), for one.

But I really like the relief and freedom I feel. Even alongside the occasional frustration. :-)

Other news (unrelated; likely to be expounded on later): my paternal grandfather is dead, and Steve and I have a new kitten named 'Rory' that we got last Monday.

Demonstrating Monty Hall in Python

There was a recent article on the Monty Hall problem in the New York Times blog TiernyLab.

The problem is this: You're on a game show, and there are three doors you may choose from. One door contains a car (or something else that's desirable) and the other two doors contain goats (or something else that's undesirable). After you pick your first door (this is the important part), the host must open another door and show you a goat. You then may choose whether to stick with your original choice or switch doors.

Should you switch doors?

Most people's response is "it doesn't matter, because the car could be behind either door, so it's the same as if I'd just been given the choice of the two doors, so it's 50-50". But that's not true, and here's why:

Monty's Constraint, as I call it, changes the odds. When you first selected a door, you had a 1/3 chance of choosing the car, and you had a 2/3 chance of choosing a goat. If you were to switch at this point, before Monty (the host) opens another door, then your odds would still be the same.

But, because Monty has to choose a door with a goat behind it to show you, there are two possible scenarios. First, if you picked the car (a 1/3 probability), then Monty can show you either of the two remaining doors as he pleases. But, if you picked a goat (2/3 probability), then the door he doesn't pick must contain the car.

Therefore, your odds of winning if you switch are 2/3 (because those are the odds that you originally picked a goat) and your odds of winning if you stay are 1/3 (the original odds that you picked the car).

Don't believe me? Steve didn't, at first. So I wrote a little program to prove it. It's in Python, so you shouldn't have any trouble understanding it, even if you're not a programmer.

Link: Python interpretor.

The Code:

import random

switchWins = 0
stayWins = 0

CAR = 1
GOAT = 0

for i in range(0,10000):

        # set up doors. the set of doors is an array of two 0s and one 1.
        # The 1 represents the car.
        # start off with an empty set
        doors = [GOAT, GOAT, GOAT]

        # choose a door to have a car
        carDoor = random.randint(0,2)

        doors[carDoor] = CAR


        # choose door (player)
        playerDoor = 1

        # choose door (monty).  We use this to ensure monty
        # randomly chooses a goat door.
        montyDoor = random.randint(0,2)

        # if the door monty picks has a car, or if the door he picks is
        # the player's door, we have to pick again.
        while doors[montyDoor] == CAR or montyDoor == playerDoor:

                montyDoor = random.randint(0,2)

        # There are only two doors left.  If the player's door doesn't have
        # a car, then the player would win by switching, so count it

        if doors[playerDoor] != CAR:
                switchWins += 1

        # Conversely, if the player's door does have a car, the player
        # would win by staying, so count that
        if doors[playerDoor] == CAR:
                stayWins += 1

print switchWins, stayWins

Leaving Haloscan? Use this handy scraper

I recently decided to use entirely to blogger comments for my blog, but I wanted to retain my old comments for posterity (and/or to occasionally enrage me). So, I went a-googling , and eventually came across a Haloscan comment scraper which performed the task very admirably. You just need to have python installed and be ready to enter your Haloscan username and password.

Note that it may not work forever, but it works as of today.

My next task is figuring out how to automatically put all the comments into blogger. Hoo boy.