skettle.com

My productivity dipped, not surprisingly, after a bout of food poisoning coinciding with the brief lull preceding the start of the fall television schedule prompted me to buy an Xbox 360. In an effort to get back on track, I’ve compiled a to-do list, of sorts, to jump-start my flagging… something.

Finish Video Server series.

I got off to a good start with my series of entries on ripping and encoding files for my iTunes/AppleTV video server, but I’m sort of stuck on what was to be the final entry- adding meta-data to the files to make them pretty. It comes down, in part, to not being sure how to distribute a couple of accompanying scripts. They’re too long to post as part of the entry, so they’ll need to be archived and put up for download as a tar file. The real sticking point, though, is that they need more documentation, and I just haven’t been able to bring myself to do it.

Rework skedevel.com.

Moving skettle.com to skedevel.com was far simpler than I’d hoped, but that was only the first step. I need to reduce the complexity of the site, removing the ill-conceived user-centric directory structure for public facing content, and making everything function privately. In other words, go back to a log-in to use the tools model, and decide after the fact what is publicly accessible. The .site.com model is great for complex social-networking type sites, but since I don’t hve much interest in doing that, the complexity is making everything else three times more difficult. While I’m in there, I also need to streamline the database access and see if I can come up with a way to benchmark the capacity of the site. It would be nice to know exactly what kind of pounding the codebase could take.

Implement OpenID on skettle.com.

I was driving back from Sacramento one night when I had a revelation about how one might implement decentralized centralized user authentication for Websites. When I started looking around to see if anyone was doing what I was thinking about, I found that the guys at sxip.org were doing almost exactly what I had thought of, though there were some issues with the implementation at the time. I carefully weighed my options, and rather than using my copious free time and obvious genius to get involved and fix the perceived problems, I think I elected to watch TV and play video games. Eventually the sxip technology morphed in OpenID, and, for the most part, it seems to work. I implemented one of the early incarnations of the library for the skedevel.com code, and for all I know it still works, but I’ve been less successful finding a good WordPress plugin. One guy seems pretty close, and if he hasn’t released a stable version by the time to get around to working on this, I’ll try playing around with his beta.

Unit testing and monitoring precautionmail.com.

One of my chief goals is to build a codebase I can use for rapid development of new sites. I had two sites in mind when I started, and one of them, precautionmail.com, I already built before realizing that it wasn’t very exciting and not something I had any desire to work on long term. However, that doesn’t mean don’t want to it stay up and running. Since it’s running on the same alpha code that runs skedevel.com, there’s a very real chance that any changes I make on skedevel will break precautionmail, and it’s unlikely I’d notice it for weeks, since, let’s face it, precautionmail is boring, and there’s not a lot of incentive to make sure it’s working properly. Therefore, I need to implement some sort of functionality to monitor that the major functions of the site are working correctly, beyond just some sort of simple pattern matching HTTP check (though that would be a good start). Since fully testing the site involves logging in, writing a message, and verifying that it was delivered after a preset period of time, there could be some fairly major engineering involved.

Find a new WordPress theme.

When I first implemented WordPress, I liked the default theme, and I thought that most people using WordPress would change the theme- thus rendering my site, using the default, kind of original. I don’t know if that’s the case or not, but I have decided that it’s always lame to use the default. Even I were the only one in world doing it, I would still be lame, because using the default is lame almost by definition.

While iTunes has its issues, for the most part it makes for a fairly passable media organizer. As long as I let it do its job, I’m pretty happy with how it handles the files on the backend. When I moved my iTunes library to a new external drive, however, I came across a pretty egregious “bug” that left me with thousands of duplicate files and no good way to remove them.

By default, iTunes stores files in “/Users//Music/iTunes/iTunes Music”. When you drag and drop a file from your desktop, or wherever, it copies the file into this directory, tossing it into a subdirectory based on Artist and Album name and renaming it according to the track info you specify in iTunes. When you change the Album name or Track name, the file is is updated and moved automatically, so the location and name of the file always reflect what you see in iTunes. Personally, I like this behavior.

I didn’t run into problems until I attempted to move my files from the default directory to a directory on my new drive, a 500G firewire affair. At first, everything worked properly when I changed the settings; it took while, but it slowly copied all the the files to the new location, “/Data/iTunes”, and once it was done, everything was great. And everything remained great until I made the mistake of opening iTunes while my new drive was unmounted.

Little did I know that when iTunes couldn’t find “/Data/iTunes”, the new directory, it changed itself back to its default: “/Users//Music/iTunes/iTunes Music”. Upon finding all the files still there, it quickly settled in and changed all the entries in the database to point to these files. When I finally realized what had happened, I remounted the new drive and changed the iTunes directory again- and that’s when all my troubles began.

iTunes once again began to copy my files to new the drive- only this time, the files already existed. Rather than simply use the existing files, it created new copies, appended with ” 1″, and I was left with several thousand duplicate files and no easy way to remove them.

To deal with this issue, I wrote a simple PHP script to crawl through my iTunes directory and deal with files that end in ” 1″. In the case where both files exist (with and without ” 1″), the script compares the two files. If they are identical, it tries to rectify the situation by removing the old file and re-adding the remaining file to iTunes using osascript, OS X’s command line Applescript tool. Adding the file triggers iTunes to automatically rename the file properly, thus removing the ” 1″. By default, you’re prompted before any file is deleted. You can disable this by hitting ‘a’ at the prompt.

In the case where the two files are not equal, they are skipped. You’re going to have to deal with those manually. Sorry. It also has problems with files that contain special characters. You’ll have to deal with those as well… but come on- how many Blue Öyster Cult, Mötley Crüe or Björk songs do you have, anyway?

In the case where only one file exists, it will simply re-add the file to iTunes, on the theory that iTunes will fix it if the name is wrong, and ignore it otherwise.

fix_itunes_dupes.php

#!/usr/bin/php
<?php

/*
* Name: fix_itunes_dupes.php
* Author: patrick
* Usage: fix_itunes_dupes.php
*
* This script is provided AS-IS.  No warranty is either expressed or implied.
* Use at your own risk.
* Feel free to use, modify, duplicate, or take credit.
*/
// Set this variable to your iTunes directory.
$itunes_dir = "/Users/{$_ENV['USER']}/Music/iTunes/iTunes Music";

if (!file_exists($itunes_dir)) {
    exit("{$itunes_dir} does not exist.");
}

// Collect itunes files that end in " 1".
$find = "find \\"{$itunes_dir}\\" -type f -name \\"* 1.*\\"";
$files = explode("\n", `$find`);
$file_count = count($files) - 1;
// 'find' comes with a bonus carriage return.  Get rid of it or try to remove
// '.' (that's bad).
unset($files[$file_count]);
// Set some defaults.
$all = false;
$count = 0;

// Loop through the files.
foreach ($files as $file1) {
    $count++;
    print "Processing {$count}/{$file_count}:\n";
    // Skip Movies and TV Shows.
    if (preg_match("/\/Movies\//", $file1) || preg_match("/\/TV Shows\//", $file1)) {
        print "Skipping TV Show or Movie:\n";
        print "{$file1}\n";
    } else {
        preg_match("/^(.+) [0-9].([^.]+)$/", $file1, $matches);
        $file = "{$matches[1]}.{$matches[2]}";
        if (file_exists($file)) {
            $md5 = md5(file_get_contents($file));
            $md51 = md5(file_get_contents($file1));
            print "{$file} ({$md5})\n";
            print "{$file1} ({$md51})\n";
            if ($md5 == $md51) {
                // If the two files are identical, we can space one of them.
                print "{$file} = {$file1}\n";
                if (!$all) {
                    // Options are:
                    //  y - delete the file. default.
                    //  n - skip this file.
                    //  a - stop asking, just do them all.
                    //  q - quit.
                    print "Delete? [Y/n/a/q]: ";
                    $char = substr(fgets(STDIN), 0, 1);
                }
                if ($char == "q") {
                    exit;
                } elseif ($char != "n") {
                    if ($char == "a") {
                        $all     = true;
                    }
                    print "Deleting {$file}\n";
                    unlink($file);
                    add2itunes($file1);
                }
            } else {
                print "Files do not match.  Skipping.\n";
            }
        } else {
            // There is only one file; go ahead and add it to iTunes-
            // shouldn't hurt.
            print "{$file1} exists on its own.\n";
            add2itunes($file1);
        }
    }
    print "\n";
}

function add2itunes($file) {
    $file = preg_replace("/\//", ":", $file);
    // Please forgive the toothpicks- numerous files have 's.
    $command = "osascript -e \\"tell application \\\\"iTunes\\\\" to add file \\\\"{$file}\\\\" to playlist \\\\"Library\\\\" of source \\\\"Library\\\\"\\"\n";
    print $command;
    passthru($command);
    return;
}

?>

Sample Output

Processing 31/39:
/Data/iTunes/The Beastie Boys/Licensed To Ill/13 Time to Get Ill.m4a (42cbc19b9d2feb83899201793ee1e00e)
/Data/iTunes/The Beastie Boys/Licensed To Ill/13 Time to Get Ill 1.m4a (e2f49dfc12db5fb8f8d3954b9671870c)
Files do not match.  Skipping.

Processing 32/39:
/Data/iTunes/The Bloodhound Gang/One Fierce Beer Coaster/07 Asleep at the Wheel.mp3 (6d12ce65c4cba6748224e2262feba67a)
/Data/iTunes/The Bloodhound Gang/One Fierce Beer Coaster/07 Asleep at the Wheel 1.mp3 (6d12ce65c4cba6748224e2262feba67a)
/Data/iTunes/The Bloodhound Gang/One Fierce Beer Coaster/07 Asleep at the Wheel.mp3 = /Data/iTunes/The Bloodhound Gang/One Fierce Beer Coaster/07 Asleep at the Wheel 1.mp3
Delete? [Y/n/a/q]: y
Deleting /Data/iTunes/The Bloodhound Gang/One Fierce Beer Coaster/07 Asleep at the Wheel.mp3
osascript -e "tell application \"iTunes\" to add file \":Data:iTunes:The Bloodhound Gang:One Fierce Beer Coaster:07 Asleep at the Wheel 1.mp3\" to playlist \"Library\" of source \"Library\""
file track id 53184

Processing 33/39:
Skipping TV Show or Movie:
/Data/iTunes/TV Shows/Battlestar Galactica/3-03 Exodus_ Part 1.mp4

Another Short Answer to a Specific Question:

Sometimes, after an upgrade, any PHP script run from the command line causes a segmentation fault, even though the script seems to run fine. For example:

[03:13:01 argon:~]$ php -v
PHP 5.2.4 with Suhosin-Patch 0.9.6.2 (cli) (built: Sep 17 2007 02:28:25)
Copyright (c) 1997-2007 The PHP Group
Zend Engine v2.2.0, Copyright (c) 1998-2007 Zend Technologies
Segmentation fault: 11 (core dumped)
[03:13:02 argon:~]$

Whenever this happens, I can usually resolve the problem by changing the order of the modules listed in extensions.ini. Three times out of four, moving “extension=session.so” to the top of the file fixes the problem.

This is just a little nugget I’ve had around for a while. I’m dusting it off and posting here more to test the WP-Syntax WordPress Plugin than any other reason.

I use screen, but I don’t leave myself logged in when I’m not at my computer. I can usually type ‘screen -R’ to reconnect to the same session the next day, but if I have more than one screen session running at once, I need to ‘screen -ls’, then figure out which one to reattach to. At least I think I need to do that… in addition to being to lazy to do all that, I’m too lazy to scour the man pages looking for the way to make it create or reattach to the same session every time.

If you call ‘screen’ with parameters, they will be used instead… This should allow you to use screen completely normally when necessary.

#!/bin/sh
# Rename your screen binary to something else to get it out of
# the way, and save this script as 'screen' into the same
# directory. Make a note of what you called the original here:
SCREEN="/usr/local/bin/screen1"

# Make $DEFAULT_NAME whatever you want; you'll never actually see it:
DEFAULT_NAME="default_session"

if [ "$#" -eq 0 ]; then
    SCREENLS=`$SCREEN -ls | grep "$DEFAULT_NAME"`
    if [ "$SCREENLS" = "" ]; then
        $SCREEN -S "$DEFAULT_NAME"
    else
        $SCREEN -r "$DEFAULT_NAME"
    fi
else
    $SCREEN $@
fi

I’ve had to make some difficult choices in the last few days.

Though it may not seem like it, this site represents a lot of time and effort; the haphazard design, confusing menus and utter lack of documentation mask a fairly complex and powerful back end. Unfortunately, it’s become clear that this complexity renders it more inefficient than I feared; even with no traffic there’s enough database activity to occasionally bring down the server.

Not coincidentally, these problems started just as my blogging efforts were finally getting underway… efforts which also revealed, in flashing neon visible from space, all the reasons that people use a professional content management packages. For one post, I had to add at least three new features: image uploads, thumnails and comments. Please remember that it was never my intention to build a content management system. Every feature on the site, including the blog, was supposed to be a proof-of-concept test for the underlying code.

That’s when it finally hit me how much work would be to get even minimally usable blogging code written. I wouldn’t be able to blog those efforts, or anything else, until I was finished; and even if I could half-ass it, the code still wouldn’t actually support any traffic… so it better not be good, or god-forbid, popular.

This put me in a funk.

Faced with the prospect of toiling indefinitely and having nothing to show for my efforts but a broke-ass pile of crap, I was forced to concede that there’s only one solution: I’m going to move to WordPress for content management.

Once I finally got the taste of defeat out of my mouth and over the urge to junk everything and move to Miami, I realized that this is actually a great option, and something I should have done a long time ago- it takes a number of things off my plate that I never wanted to do anyway, it removes the pressure of having to use my pre-alpha code to support an actual site, and it will give me a personal homepage I’m not ashamed of.

This is the perfect example of learning to pick my battles; by reducing the scope of what I’m working on, I should be able to accomplish quite a bit more.

Hopefully.