Dynamically Generated Flash Photo Gallery

Problem

When it comes to my pictures and the interaction between my collection on my PC, I have quite a few requirements. And as you know, anytime your requirements get long and detailed, you find yourself writing some software.

I had a simple gallery up on my older website that was a traditional html table layout and it worked well. I never had a strong desire to use flash, until I saw SimpleViewer. It is simple and elegant. It preloads images in the background so that users can click through as fast as they want without waiting for images or pages to load. It is XML based - you pass it some XML information for a directory of images and the locations of their thumbnails and it displays the gallery.

So, I decided to give it a shot. Unfortunately, that is when all of my requirements set in.

Requirements

After I came to understand exactly how I wanted my gallery to work and how I wanted the image update mechanism to work, I knew I was going to be writing a lot of scripts and wrapper code around the SimpleViewer utility. Here is how I wanted it to work.
  • Updating
    • Everything must be updateable by running a single command.
    • Everything should update in a global sense, meaning the update utility should update every directory in the entire pictures area. The command should not be run on a per-gallery basis. I must be able to make a new directory on my PC in my pictures folder anywhere and throw my files in it. I must be able to rename a folder on my PC. I must be able to remove a folder on my PC. After all of these, running the update command should sync up all directories with a minimal amount of transfer, adding new ones, syncing any new/modified pictures, and deleting anything that has been removed from my PC (i.e. the source area for pictures).
    • All thumbnails must be stored on the web server only. I don't want to have my pristine pictures area on my PC littered with thumbnails.

  • Viewing
    • The gallery must have browseable directories listed on the left side. I have tons of images available on the website, and I don't want to make a seperate gallery for each of them.
    • The gallery should be able to switch back and forth between flash and non-flash versions while looking at the current folder and must also be able to automatically forward to the non-flash version with a warning message and a download link for the flash plugin.
    • The gallery should provide some sort of "breadcrumbs" or navigation path header at the top that shows the user what directory level they are at and let them click to parent directories.
    • The flash gallery should display a link to the full size image so that my users can download images to their hard drive. Also, it should display the size of the full size image so that users know how large it is before they click the link.
    • The gallery must use 3 levels of resolution - the small thumbnails used as preview images in SimpleViewer, the larger thumbnails used as the main images in SimpleViewer, and the original full size image the users can click to. I need this because many of my images are very large, and it is not feasible to have simpleviewer download a huge image in the background only to display it in a small window. This way, I know exactly how big the images in the flash viewer will be, no matter what the size of the original.
    • The directory browser on the left should list "other files in this folder" which will give links to files not viewable by SimpleViewer. The main reason for this is for movie files. In any event, such as a vacation, or a birthday party, I will likely have pictures as well as movie files. I want to be able to keep all of that information together in a single directory so that the users see movies and images all in one place.
    • XML data for each folder should be generated in real-time.

Since my situation is probably fairly unique, I doubt all of my code together is useful to many people. So, instead of giving it out as a single utility, I will present it as a list of problems I ran into and the code I wrote/used to solve each one. Perhaps that will help more people. You can see my implementation of all this here.

Solutions

Updating/Syncing files

So as you can probably tell, me uploading files and then running some comand to generate thumbnails is not going to cut it. I need a sync command. Something to sync up all of the pictures on my PC with the pictures on my webserver and update any files that I have added, modified, or deleted, including folders. Also, since I have gigs of pictures, the sync must be smart to look only at things that have been modified on the source side (my PC).

After some searching, it was quickly obvious that I wanted to use a utility called rsync. It is a great linux utility that has been around for a while. It has the capability to tunnel through lots of transports, so if you have ssh access you can use that. For mine, I have my PC filesystem available locally, so I don't use a transport. It is very easy to do. Basically you give it a source and a destination and tell it to sync. It does the rest.

Here is an example of how I have scripted my rsync. I need to be able to exclude certain things, so that is done by listing them in an exclude file. This is a csh script. These scripts are all meant to be run on the webserver. You'll notice some lines about the thumbnail updater script as well. More on that below.

rebuild.csh:
#!/bin/csh -f

echo " "

# Make sure to including trailing slash --FG
set source_dir = 'path/to/pristine/picture/repository/'
set dest_dir = 'path/to/pictures/location/on/webserver/'

# rsync options
set rsync_options = '-av --delete --exclude-from rsync_exclude'
set rsync_log_file = 'path/to/rsync.log'

# Thumbnail updater script
set thumbnail_update_script = 'path/to/update_thumbnails.pl'
set thumbnail_update_log_file = 'path/to/update_thumbnails.log'

# Simple check to see if the source_dir drive is mounted and available
# if using ssh transport, you would test the ssh connectivity here
echo -n "Looking for source dir ...        "
if (! -d $source_dir) then
    echo "[FAILED]"
    echo "$source_dir is not available"
    echo "Drive may not be mounted or directory may have moved"
    echo " "
    exit
else
    echo "[  OK  ]"
endif

# Simple check to see if the dest_dir drive is mounted and available
echo -n "Looking for dest dir ...          "
if (! -d $dest_dir) then
    echo "[FAILED]"
    echo "$dest_dir is not available"
    echo "Drive may not be mounted or directory may have moved"
    echo " "
    exit
else
    echo "[  OK  ]"
endif

# Build rsync command string
echo -n "Building rsync command ...        "
set rsync_cmd = "rsync $rsync_options $source_dir $dest_dir"
echo "[  OK  ]"

# Sync the directories
echo -n "Performing rsync ...              "
echo "rsync_cmd is: $rsync_cmd" > $rsync_log_file
$rsync_cmd >> $rsync_log_file 
echo "[  OK  ]"

# chmod on dest_dir
echo -n "Performing chmod on dest dir ...  "
chmod -R ug+w $dest_dir
echo "[  OK  ]"

# Simple check to see if thumbnail script is available
echo -n "Looking for thumbnail script ...  "
if (! -f $thumbnail_update_script) then
    echo "[FAILED]"
    echo "Thumbnail script, $thumbnail_update_script is not available."
    echo " "
    exit
else
    echo "[  OK  ]"
endif

# Run the thumbnail updater
echo -n "Updating thumbnails ...           "
echo "thumbnail_update_script is: $thumbnail_update_script" > $thumbnail_update_log_file
$thumbnail_update_script >> $thumbnail_update_log_file
echo "[  OK  ]"

rsync_exclude:
not on website
Originals
Google Talk
Thumbs.db
Picasa.ini
Desktop.ini
thumbs_big
thumbs_small


The exclude file excludes a directory on the source side called "not on website" which is where I put pictures that I want in my library but don't want to show up on the website. Also, the exclude file ignores some files that commonly show up on the windows machine that we don't want, such as Thumbs.db, Desktop.ini, and the "Originals" folder that Picasa sometimes makes. The thumbs_big and thumbs_small directories are listed there because they exist on the webserver, but not on the source side. Since we are using the --delete option (which causes rsync to delete stuff from the destination that is no longer on the source side), we exclude these so that it doesn't remove them and allows them to stay on the webserver. Updating all these thumbnails is done in a seperate script.

Thumbnails

From that rebuild script above, we also want to go through afterwards and look at every image and make sure that the thumbnails for each image exists. Also, we want to make a large thumbnail, and a small thumbnail (see above for why).

The first question to answer is what utility should we use to generate the thumbnails? Well, I searched around a bit and settled on "mkThumb". It isn't extremely fast, but it gets the job done and it is easy to use.

The script to do this is a perl script. It recursively goes through the directory structure and makes thumbnails for any images that don't have the corresponding big and small thumbnail, which is what would occur after new files were added. The code is fairly readable, so I won't explain it too much.

update_thumbnails.pl:
#!/usr/bin/perl
#------------------------------------------------------------------------------
# A little script to recursively go through directories and generate thumbnails
# for every jpg it finds.  Ideally, I could do this with a Makefile, but I just
# can't figure out how to make it work... oh well.
#------------------------------------------------------------------------------

use strict;

my $dir_to_process   = "path/to/pictures/location/on/webserver";
my $thumb_big_dir    = "thumbs_big";
my $thumb_small_dir  = "thumbs_small";
my $thumb_prefix     = "_tn_";
my $thumb_big_size   = 400;
my $thumb_small_size = 65;
my $file_count_warning = 50;
my $thumbnail_big_command   = "mkthumb -p $thumb_big_dir/$thumb_prefix -o -f $thumb_big_size";
my $thumbnail_small_command = "mkthumb -p $thumb_small_dir/$thumb_prefix -o -f $thumb_small_size";

# Directories to filter out... can include regular expressions. Case sensitive.
my @dirs_to_ignore = ("Originals", $thumb_big_dir, $thumb_small_dir );

# Files to process... case insensitive
my @file_types_to_process = ("\.jpg", "\.jpeg", "\.png" );

print "\n";

# First, change to the dir to process, then call our recursive subroutine
chdir( $dir_to_process ) or die "Can't change into $dir_to_process\n";
generate_thumbnails_recursive();

sub generate_thumbnails_recursive {
    my $cwd = `pwd`; chomp $cwd;

    # Get a list of the files that match what we are looking for
    my @files_to_process = get_list_of_image_files( ".", \@file_types_to_process );

    foreach my $file (@files_to_process) {
        #print "Processing $file in $cwd...\n";
        process_file( $file );
    }

    # Get the sub dir list
    my @sub_dirs = get_sub_dirs();

    # Now perform the same actions on each sub dir
    foreach my $sub_dir (@sub_dirs) {
        my $skip = 0;

        # Set a flag to skip if we are ignoring this directory
        foreach my $dir_ignore_string (@dirs_to_ignore) {
            if ($sub_dir =~ m/$dir_ignore_string/) { $skip = 1; }
        }

        # If we are not supposed to skip, then process
        unless ($skip) {
            #print "Changing into $sub_dir ...\n";
            chdir ( $sub_dir ) or die "Can't change into $dir_to_process\n";
            generate_thumbnails_recursive();
            chdir ("..") or die "Can't change into \"..\" from $sub_dir\n";
        }
    }
    return;
}


# Get List of Files to process
sub get_list_of_image_files {
     my $directory = $_[0];
     my @filters = @{$_[1]};
     my $filter  = "";
     my $num_files = 0;
     my $entry;
     my @image_files = ();
     my $cwd = `pwd`; chomp $cwd;

     opendir (DIR, $directory) or die "can't opendir $directory directory: $!";
     $num_files = 0;
     while (defined($entry = readdir(DIR))) {
          foreach $filter (@filters) {
              if ($entry =~ m/$filter/i) {
                      $image_files[$num_files] = $entry;
                      $num_files++;
              }
          }
     }
     closedir(DIR);

     if ($num_files > $file_count_warning) { 
         print "WARNING: Detected more than $file_count_warning ($num_files) files in $cwd\n";
     }
     #return ($num_files, @image_files);
     return (@image_files);
}

# Get list of sub directories to process
sub get_sub_dirs {
     my $number_of_dirs;
     my $entry;
     my @run_dirs;
     my $run_dir_list;

     opendir (DIR, ".") or die "can't opendir current directory: $!";

     $number_of_dirs = 0;
     while (defined($entry = readdir(DIR))) {
          if (-d "$entry") {
              unless(($entry eq ".") || ($entry eq "..")) {
                  $run_dirs[$number_of_dirs] = $entry;
                  $number_of_dirs++;
              }
          }
     }

     closedir(DIR);
     return @run_dirs;
}

# Process an image file
sub process_file {
    my $file_to_process = $_[0];
    my $cwd = `pwd`; chomp $cwd;

    my $big_tn_filename   = $thumb_big_dir   . "/" . $thumb_prefix . $file_to_process;
    my $small_tn_filename = $thumb_small_dir . "/" . $thumb_prefix . $file_to_process;
    my $file_mtime        = (stat($file_to_process))[9];

    # Unless big thumbnail exists and is newer than the source file, create it
    unless (-f $big_tn_filename && ((stat($big_tn_filename))[9] > $file_mtime)) {
        # First, check to see if thumb dirs exist
        if (! -d $thumb_big_dir) {mkdir ( $thumb_big_dir ); }
        `$thumbnail_big_command \"$file_to_process\"`;
        print "Updating thumbnails for $cwd/$file_to_process ...\n";
    }

    # Unless small thumbnail exists and is newer than the source file, create it
    unless (-f $small_tn_filename && ((stat($small_tn_filename))[9] > $file_mtime)) {
        # First, check to see if thumb dirs exist
        if (! -d $thumb_small_dir) {mkdir ( $thumb_small_dir ); }
        `$thumbnail_small_command \"$file_to_process\"`;
    }
}


That pretty much takes care of the updating part. With these scripts, we are left with a directory on our webserver that has a complete mirror of our source picture repository somewhere, minus what we told it to exclude, plus a large and small thumbnail for every image. Now, let's move on the web side of things as this is where most of the difficulty lies...

Dynamically displaying the directories

First off, I'm going to not show my entire php file as I do have some security concerns. However, if you see something in my implementation here that you also want to do and it isn't made clear here, let me know.

The main issue that we face when trying to dynamically show galleries with SimpleViewer is that we must dynamically generate the XML data. My solution to this is to pass in a variable to my page that says what directory I want to show. No variable means show the top level. From this variable, I ignore all jpg files and thumbnail directories, and list the remaining directories on the left with links to the same script with a different argument. Also, other files in the directory that aren't viewable by SimpleViewer are listed in this column as well, such as movie files. You can see an example of this case here

After listing that on the left, I embed the flash viewer on the right, and I pass it an argument using the "Flashvars" variable. This lets me give the flash viewer a script to run that will return the XML data it wants. Of course, I pass it a script with the same directory name as an argument, and that script should return the XML data for that directory. For example:

<?php
    
# Only embed the flash object if flash is 1... otherwise, use the non-flash version
    
if ( $flash == ) {
        echo 
"<object classid=\"clsid:d27cdb6e-ae6d-11cf-96b8-444553540000\" codebase=\"http://download.macromedia.com/pub/shockwave/cabs/flash/swflash.cab#version=6,0,0,0\" width=\"650px\" height=\"520px\" align=\"middle\">\n";
        echo 
"   <param name=\"movie\" value=\"viewer.swf\" />\n";
        echo 
"   <param name=\"quality\" value=\"high\" />\n";
        echo 
"   <param name=\"scale\" value=\"noscale\" />\n";
        echo 
"   <param name=\"BGCOLOR\" value=\"#A5B6C7\" />\n";
        echo 
"   <param name=\"FlashVars\" value=\"xmlDataPath=getXML.php?folderName=$folderName\" />\n";
        echo 
"   <embed src=\"viewer.swf\" width=\"650px\"  height=\"520px\" align=\"middle\" FlashVars=\"xmlDataPath=getXML.php?folderName=$folderName\" quality=\"high\" scale=\"noscale\" bgcolor=\"#A5B6C7\" type=\"application/x-shockwave-flash\" pluginspage=\"http://www.macromedia.com/go/getflashplayer\" />\n";
        echo 
"</object>\n";

    } else {


So, this brings in "getXML.php". It's code is pretty simple.

getXML.php:
<?php

$basePath 
'path/to/pictures/location/on/webserver';

# Make sure we match this for security: all images MUST be in the images folder!
$matchString $basePath "/images";
#echo "matchString is $matchString<br />\n";

# Parse the query string
parse_str($_SERVER['QUERY_STRING']);
#echo "Query String is ".$_SERVER['QUERY_STRING']."<br />\n";

# If query is blank (no folder specified, use root as default
if ( $_SERVER['QUERY_STRING'] == "" ) { $folderName "images"; }
#echo "folderName is $folderName<br />\n";

# Actual dir looked at is combination of basepath and folder passed in
$dir $basePath."/".$folderName;
#echo "dir is $dir<br />\n";

# Check to make sure dir exists and is a directory
if (! is_dir($dir) ) {
    echo 
"<html>\n";
    echo 
"Error: ".$folderName." does not appear to be a valid directory<br />\n";
    echo 
"</html>\n";
    exit;
}

# If the directory has a ".." in it, or does not contain our matchString, 
# something is screwy.
if ( (strpos$dir'..')) || (strpos($dir$matchString) === false) ) {
    echo 
"<html>\n";
    echo 
"<b>Error. Unauthorized folder request</b><br />\n";
    echo 
"</html>\n";
    exit;
}

# Remove any trailing "\"
$dir preg_replace("/\/$/"""$dir);
#echo "dir without trailing slash is $dir<br />\n";

# Just use the last dir in dir tree as title
$title end((preg_split"/\//"$dir )));
#echo "<b>Title is $title</b><br />\n";

# Create imagePath and thumbPath
$imagePath $folderName."/";
$thumbs_big_path $imagePath."thumbs_big/";
$thumbs_small_path $imagePath."thumbs_small/";


# Go get the filenames of all image files
$numFiles 0;
$fileNames = array ();
$dh opendir($dir);
while (
$file readdir($dh)) {
    
#echo "Found file $file...";

    # Only grab jpegs and skip directories and thumbnails
    
if ( (!is_dir($file)) && (preg_match("/\.JPG$/i"$file)) && (!(preg_match("/^_tn_/"$file))) ) {
        
#echo "not a directory or a thumbnail and is a JPG.\n";
        # Record the filename
        
array_push $fileNames$file );

        
$numFiles++;
    }
    
#echo "<br />";
}
closedir($dh);
#echo "Num showable images is $numFiles<br />\n";

# First sort them
sort($fileNames);

# Now go get the filesizes
$index 0;
$fileSizes = array ();
while (
$index $numFiles) {
    
$file $fileNames[$index];

    
# Get the filesize in KB and round to 2 decimal places
    
array_push $fileSizessprintf("%.2f", ((filesize($dir."/".$file)) / 1024)) );
    
#echo " size is ".sprintf("%.2f", ((filesize($dir."/".$file)) / 1024))."\n";

    
$index++;
}

echo 
'<?xml version="1.0" encoding="UTF-8"?>';
echo 
"\n";
echo 
'<SIMPLEVIEWER_DATA maxImageDimension="550" textColor="0x000000" frameColor="0xFFFFFF" bgColor="0x000000" frameWidth="5" stagePadding="5" thumbnailColumns="3" thumbnailRows="5" navPosition="left" navDirection="LTR" ';
echo 
"title=\"$title  ($numFiles Images)\" imagePath=\"$thumbs_big_path\" thumbPath=\"$thumbs_small_path\">\n";

$index 0;
while (
$index $numFiles) {
    
$fileNameWithoutExt $fileNames[$index];
    
$fileNameWithoutExt preg_replace("/\.jpg$/i"""$fileNameWithoutExt) ;
    echo 
"<IMAGE>\n";
    echo 
"    <NAME>_tn_$fileNames[$index]</NAME>\n";
    echo 
"    <CAPTION><![CDATA[$fileNameWithoutExt\n$fileSizes[$index] KB\n<A href=\"$imagePath"."$fileNames[$index]\" target=\"_blank\"><U>View full size</U></A>]]></CAPTION>\n";
    echo 
"</IMAGE>\n";

    
$index++;
}

echo 
"</SIMPLEVIEWER_DATA>\n";

?>


The SimpleViewer application reads the output of this script as the XML file it normally expects and displays things accordingly. You'll notice that I use this script to play with the caption to create the link to the full-size version of the image as well as print out the filesize.

Navigation breadcrumbs

One requirement was to have "breadcrumbs" bar (or whatever it is called) at the top to show the directory level the user is at. I did this by parsing the directory name passed into the script. And then, I use the unshift command to make an array with each level. Then, to print it out, it is simply a matter of walking through the array, and printing the item with its associated link.

First, we get the array of directory tree elements:

<?php
# This function returns an array of directory names that create a path 
# to the input parameter.  For example, passing in "/path/to/folder"
# returns ("path", "to", "folder").
function get_dir_tree_elements $folder_to_process ) {
    
$debug 0;
    
$output = array ();

    
# Remove any trailing "/"
    
$folder_to_process preg_replace("/\/$/"""$folder_to_process);

    if (
$debug) { echo "<br /><br /><b>dir_tree: parsing $folder_to_process</b>.<br />\n"; }
    
# Go through each element in the string starting on the end, unshifting 
    # the resulting string on to the stack
    
array_unshift$output$folder_to_process );
    while (
preg_match("/\//"$folder_to_process)) {
        
$last_part end((preg_split"/\//"$folder_to_process )));
        
$folder_to_process preg_replace("/\/$last_part/"""$folder_to_process);
        
array_unshift$output$folder_to_process );
    }
    return 
$output;
}
?>


Later, in the output, we make a table and put two columns. One has the breadcrumbs navigation using the array we created above. In the other cell, we post a link to the current folder.

<table align="center" width="99%" border="0">
  <tr>
    <td align="left">
        <br />

<?php
$dir_tree_elements 
get_dir_tree_elements$folderName );
foreach (
$dir_tree_elements as $dir_tree_element) {
    echo 
"                    / ";
    
# If this is the last one in the tree, don't hyperlink it, but save link off
    
if ( end((preg_split"/\//"$dir_tree_element ))) === $title ) {
        echo 
end((preg_split"/\//"$dir_tree_element )));
        
$link_to_folder "index.php?folderName=$dir_tree_element";
        echo 
"\n";
    } else {
        echo 
"<a href=\"index.php?folderName=$dir_tree_element&amp;flash=$flash\">";
        echo 
end((preg_split"/\//"$dir_tree_element )));
        echo 
"</a>\n";
    }
}
?>

    </td>
    <td align="right">
        <a href="<?php echo $link_to_folder;?>">[Link to this folder]</a>
    </td>
  </tr>
</table>


Conclusion

I am really happy with my new set up. It let's me take pictures off of my camera, put them in my pictures folder on my PC whereever I would normally put them, and run one rebuild script on my webserver. The rest takes care of itself. This makes updating very easy. It makes navigation nice. It also makes backups easier because my source repository of pictures doesn't contain tons of thumbnails.

If you have questions about any of the code or how something works, let me know.

One important note is that all of this code was set up to work with SimpleViewer 1.7. There has since been an update to 1.8 and I'm not sure if this code will work with it without minor changes. If anyone sees any problems with the code or security risks, please let me know. Also, if anyone finds the code useful or makes any improvements, I would love to hear about it.

Lastly, thanks to the authors/maintainers of SimpleViewer, mkTthumb, and Picasa (awesome photo software from Google - link at bottom of page). Together they make for an enjoyable picture/web gallery experience.


Last Modified: 09.03.06    Valid XHTML 1.0!    Valid CSS! Powered by Blogger