Whither iTunes Plus upgrades (again)

A while ago I wrote about how Apple’s offer to upgrade previously purchased music to iTunes Plus, their “drm-lite” offering that raised recording quality and eliminated copy restrictions, had gone missing.

At that point the issue appeared to be faulty infrastructure, since the link was sometimes available and since you could hit the page directly if you bookmarked it.

Today, it looks like the page may be gone for good. I haven’t seen it in the store in several weeks, and the bookmark now returns a “Could not complete the iTunes Store request” popup.

While I’m not entirely surprised—the promotion has been running for what, four years?—I’m disappointed. Because this is what I see in my smart playlist that contains the old, “FairPlay” encrypted, DRMed iTunes Store downloads:

671items

That’s right. 671 tracks, in about 178 albums, that were never made available for upgrade through the iTunes Plus page. I know, because every time something was made available in Plus, I paid the 30 cents per track or $3 per album to get my music out of DRM jail.

So my question to Apple is: what happened? Did the rest of the music never get “plussed” because of the labels? Or did you just forget?

I’ve read some speculation that iTunes Match is the new “upgrade to iTunes Plus”. If so, I’m still out of luck, because I have more than 25,000 songs in my library—all purchased legally, I might add, though some came from eMusic or Amazon, or were ripped from CDs or vinyl that I own.

So let’s see: I’m stuck with a bunch of DRMed music that I can’t unDRM, contrary to Apple’s iTunes Plus promises, and I can’t take advantage of the other legal path offered to me because I’ve been too good a music customer.

Is it any wonder that people just say “screw it” and download music for free?

Waiting for changes to be applied

So far iOS 5 has been just fine on my iPhone 3GS (yes, still), but for one important exception: I don’t think the phone has ever completed a sync without my having to eject it.

The symptom is one of those things that gives long-term iTunes users pause: text in the iTunes status window that appears at the end of the sync, saying, “Waiting for items to copy,” or “Waiting for changes to be applied.” And stays there, pretty much indefinitely. Turns out it’s a common problem, with no consistent solution. I have tried leaving the phone syncing all night long (both wired and wireless), even tried turning off syncing of all content. Nothing.

So today I tried the ultimate: restore to factory settings, then restore from backup. And, as of right now, things are… “waiting for items to copy,” while syncing podcasts.

Sigh. Wonder how long I have until we can buy the 4S?

There is one note of wonderment though: as I was plowing through the console looking for clues as to what was going on, I found this:

Nov 10 07:11:44 iTunesHelper[248]: AMDeviceConnect (thread 0x7fff7c774960): This is not the droid you're looking for (is actually com.apple.mobile.restored). Move along, move along.

Heh.

UPDATE: Aaaand just as soon as I pushed Send to Blog, I found the answer: voice memos. Specifically, deleting all voice memos on the phone was sufficient to fix the problem and allow the sync to complete. Now, mind, this was after a restore to factory settings and restore from backup, so I don’t know if those steps were necessary, but it worked.

Moving iTunes libraries from one hard drive to another

For the love of me, I don’t know how I ended up here again. The last time I moved my iTunes library to a bigger disk, I was able to use the Consolidate Library command and let iTunes do all the file moves. This time… not so much.

I was trying to do two things at once with this move: put all my music onto a larger external drive (a 1TB MiniStack from OtherWorld Computing. Can’t recommend the enclosure enough for form factor, ports, and reliability; my older 500 MB drive is in the same enclosure), and move to the new iTunes library layout, where there are separate top level folders for music, movies, podcasts, etc. This didn’t work, because partway through the process writing to the disk errored out. I quickly realized that the problem was that I was doing it over the network (the new 1TB drive and my older drive were both connected to my AirPort Extreme). So I directly connected the new drive via FireWire, left the old one on the AirPort, and tried to consolidate again. Only this time, it told me I didn’t have enough disk space. On the new 1TB drive.

What?

Then I realized what was going on. The restructuring of the library wasn’t moving files, it was creating another copy of the files on the same drive. I had started consolidating before I did the library restructuring, so now it was trying to write a second copy of all my music on the drive. Since I probably have about 515 GB of music to work with (some resident on my MacBook Pro’s internal drive), two copies weren’t going to fit on the new drive.

So now I had: a full copy of my library spread across the old drive and my hard drive; one partial copy at the root of my new drive; and another partial copy in the proper location in a Music subdirectory. I didn’t want to delete either of the partial copies because songs in my library were mapped to both locations; I couldn’t reconsolidate on the old drive for lack of space. But I still had over 160 GB free on the new drive, so I could probably copy over the missing files by hand.

So I’m going the manual route to clean up. First, I went to the Terminal and ran the comm command, which compares two text files line by line; I fed it the directory listings of the old external drive and the new final destination, and it spit out about 165 differences, directories that didn’t get copied from the old drive to the new one.

Second, I’m going line by line through the results of the comm listing. For each line, I:

  1. Copy the missing files from the old drive to the new one.
  2. Delete the old files from the old drive.
  3. Go to the matching tracks in iTunes and do a Get Info. Amazingly, since I’m copying the new files into the correct directory structure, a lot of the time I don’t have to do any more work and the library automatically finds the files in the new drive. Sometimes I have to browse one file at a time to link up to the new files, which is a drag.

I expect I’ll be done with the process sometime next week. Painful, but at least no data is lost. Then I can repeat with any files from my laptop’s disk, a much shorter list.

The final cleanup may take some XSLT fu. I will need to triple check the final library to make sure no tracks point to locations on the old drives. I’m going to try using XSLT on the iTunes Library.xml file to see if I can cull out the problem tracks that way; if not, it’ll be a matter of trial and error, because there’s no convenient way to find the file system locations of iTunes tracks from within iTunes itself, other than one at a time.

I’d love to have this be  a more error-free process. I’m beginning to think that iTunes libraries on external drives simply isn’t a well tested scenario by Apple.

Friday Random 15: Out of Rotation

I keep a playlist in iTunes, and on my iPod, that consists of highly rated songs (4 stars or better) that I haven’t heard in at least a year. It’s called Out of Rotation, and it always surprises me in a positive way. Today, when I needed a pickup after car trouble, it came through. Here’s the playlist:

  1. Johnny Cash, “Belshazzar” (Complete Sun Singles, Vol. 2)
  2. Liz Phair, “Chopsticks” (Whip-Smart)
  3. Pernice Brothers, “Waiting for the Universe” (Yours, Mine and Ours)
  4. Sonic Youth, “Radical Adults Lick Godhead Style” (Murray Street)
  5. Ted Leo/Pharmacists, “The High Party” (Hearts of Oak)
  6. Yo La Tengo, “Nothing but You and Me” (Summer Sun)
  7. UNKLE, “Nursery Rhyme Breather” (Psyence Fiction)
  8. The Raconteurs, “Blue Veins” (Broken Boy Soldiers)
  9. The Raconteurs, “Intimate Secretary” (Broken Boy Soldiers)
  10. Pixies, “River Euphrates” (Surfer Rosa)
  11. Gillian Welch, “Revelator” (Time (The Revelator))
  12. Gillian Welch, “My First Lover” (Time (The Revelator))
  13. Chemical Brothers, “Elektrobank” (Dig Your Own Hole)
  14. Prince, “Wherever U Go, Whatever U Do” (Rave Un2 the Joy Fantastic)
  15. Elvis Presley, “New Orleans” (The King of Rock ‘n” Roll: The Complete ’50s Singles)

iTunes Plus de-emphasized in iTunes Store?

Looking at the new iTunes Store experience in iTunes 9, I had difficulty finding any information about iTunes Plus upgrades, Apple’s offer that allows you to upgrade your old DRM-crippled protected files to the new “purchased” format.

Fortunately, it seems that the page is still there and working in the store, just not promoted. Bookmark the link…

Update: In the comments, David C. points out that the link is there, but it looks like it’s not loading reliably–it certainly didn’t ever load for me yesterday. The box in the upper right corner of the store loads progressively, and the bottom links (including iTunes Plus and Complete My Album) load after a delay.

Recovering from an iTunes 13001 error

I hesitate to write this post, but since I found very few reliable aids for surviving this error, I’m writing it up in the hope that it will help someone else.

My MacBook Pro (first generation, dented side resulting in unreliable power cord connection, weak battery) shut off sometime overnight. Unfortunately, when I booted it back up, iTunes told me that the library file was corrupt. Given the size of my library and the fact that I’ve got somewhere close to 100 playlists, and that I just spent about two years going through and listening to everything at least once after the last library deletion, I freaked out.

Then I quit iTunes and started thinking. There are now quite a few files that constitute the “iTunes library,” including iTunes Music Library.xml, iTunes Library Extras.itdb, iTunes Library Genius.itdb, and iTunes Library itself. I knew from past experience that it was iTunes Library that held the playcounts and playlists, so I crossed my fingers, moved everything else out of the folder, and started iTunes. Now it started up, but when it tried to rebuild the Genius data, it told me that it couldn’t save it because of a 13001 error.

I did a lot of research, and through some trial and error I hit upon the following steps:

  1. I moved all my non-Apple codecs out of the Library/Quicktime folder. In my case, this was a DivX decoder and encoder, and three component files from Flip4Mac WMV. (This was nonintuitive; thanks to this Apple support discussion post for suggesting it.)
  2. I deleted the ~/Library/Preferences/com.apple.itunes.plist file. (This is iTunes preferences, including your default settings for importing and your library location. You’ll want to reset these at the end of the process.)
  3. I re-deleted the the iTunes Library Extras.itdb and iTunes Library Genius.itdb files.
  4. I temporarily moved my external music hard drive from my AirPort Extreme and attached it directly to my Mac. (This might not have been a factor, but I figured it would be a lot faster to get through the Genius rebuild if it didn’t have to do it over an 802.11g network connection. Yes, the MacBook Pro 1st gen only has an 802.11 card, alas.)
  5. I restarted iTunes and let it rebuild the Genius database all day.

When I came home, it had succeeded.

I don’t know if all these steps were necessary, but I do know that when I didn’t delete the preferences file or the codecs, rebuilding was not successful. Whatever: it worked.

Genius take II: indeed.

After last night’s disaster, I decided to give iTunes 8 another try. This time I made sure my library settings were correct in advance, and let it collect and submit the information while I ate dinner. No spinning beachball, no issues, this time around. 

So, Genius. I’m not evaluating the sidebar right now (though I will note that the message that appears when Genius can’t find any recommendations in the store is a pretty good predictor of whether the Genius playlist feature will work. And you know what? It does work, quite well.

The feature in a nutshell is a little like an old bar game: pick a song, then identify a bunch of other songs that go with it. All the songs are pulled from your library and you can vary the length of the playlist, and save it into your library if you choose. Pretty simple. So I decided to throw it some curves. First, “Nuki Suki” by Little Richard. It took this slightly profane funk gem from the master and mixed it with “Sexy MF,” “There Was a Time,” “Baby I Love You,” “The Hook and Sling” by Eddie Bo, “Up for the Down Stroke,” and Marvin Gaye’s “You Sure Love to Ball.” There were a few clinkers as well, like Ready for the World’s “Oh Sheila,” but at least everything was in the ballpark. 

So I tried something a little different: a movement from “Jesus’ Blood Never Failed Me Yet” by Gavin Bryars, from the recording with Tom Waits on vocals. What came back seemed to be mostly related to Tom Waits rather than Gavin Bryars: tracks from Waits, Smog, the Black Keys, the Cocteau Twins, Cat Power, and so on. So I tried another Bryars track but Genius couldn’t find matches. Also matchless: tracks from the Virginia Glee Club and the Virginia Gentlemen, probably because they aren’t available in the iTunes Store. But mixes around Hilliard Ensemble, Pink Floyd, Jane Siberry, Steinski, Jeff Buckley, and Neko Case were all pretty solid.

So I think those of us that like genre-busting mixes and unusual juxtapositions are probably safe: Genius doesn’t automate what we do just yet. But for a good 25 song groove it’s not bad, and for most people it will handily replace Shuffle as a way to plumb the depths of their library.

iTunes 8: first impressions

I installed iTunes 8 last night on my home machine, a MacBook Pro with 2 GB of RAM. The update wasn’t in Software Update, so I pulled it off Apple’s website. Then I had to update to get the latest QuickTime, begging the question of why they aren’t packaged together. But that was straightforward enough. Then I rebooted and fired up iTunes.

First it wanted to update all my album art–I suppose to build new thumbnails for the new grid view. When it finished looking at my 26,000 song library in five minutes I was suspicious. Sure enough: it had forgotten that my music lived on a network drive and silently reset the location to my laptop hard drive, causing all the songs in the library to be unplayable. Fortunately I’ve been through this before: Preferences, Advanced, and set the correct location for the folder, then wait fifteen minutes while all the song paths are reset. But man: I was really hoping Apple had fixed this one. I don’t restart iTunes often, but when I do I have to go through this dance more than half the time.

But OK: so far no worse than the old version.

The new grid view seemed nice enough, until I clicked something. Then it locked up tighter than a drum with a spinning beachball. About five minutes later the beachball cleared and I was able to play some music. I found of interesting that the grid view was only present some of the time. If I clicked through on the Jazz genre, it brought up the classic view of tracks next to album art. Maybe this was because of the number of albums (330) in the genre, but I found it a little disorienting.

Then: Genius. I don’t know if I would have called the feature that, since it has to upload the entire library to the cloud before it can work. I let it run for awhile but it wasn’t long before the spinning beach ball returned. I finally killed iTunes but it managed to keep any other application, including QuickTime, from playing any sound until I rebooted.

And when I rebooted, iTunes forgot where the music library was again.

I think Genius has promise–it came up with some interesting recommendations on my work computer. But that only has thirty songs on it. I have a suspicion that it doesn’t scale. At all.

New iPods, new iTunes. It must be September

Apple’s really changed as a company; I remember when September was Back to School month and you’d find out about new iMacs, a new version of iLife, whatever. Now it’s all iTunes and iPod.

I like the look of the new iPod nano, and the price point ($150 for 8 GB) and form factor are sweet. But I’m particularly impressed with the software and the use of an accelerometer in what is basically a low end device. CoverFlow is a killer interface, especially in a small device, and seeing it on the nano is pretty sweet.

I’m keen to see whether the new “genius” features in iTunes scale up to my 26,000 song music collection. Hopefully by the time I’m home tonight, iTunes 8 will be downloadable (it’s still 7.7 from where I sit right now).

But I think my favorite visual from today’s event was this one:

Radiohead on iTunes, five years on

Five years and almost a month ago, I wrote a post called Interesting absence(s) about the disappearance of Radiohead and Sigur Rós from the then-new iTunes Music Store. When I expanded it for Blogcritics, I laid a finger on one of the causes—friction between artists and labels over digital rights and reimbursement. At the time I thought it might take a few months or a year before the Radiohead catalog returned. It only took Sigur Rós until February of 2004 to return, but it took until this week for the Radiohead back catalog to reappear, and then only after In Rainbows proved such a hit on the service.

The Red Herring post makes a few points about the reasons bands hold out, but they don’t answer one question: why is Radiohead only dipping its toe in? There is only one EP on the service, meaning only two non-album tracks, compared to other artists like U2 and Sting who have been far more generous with posting digital rarities—which fans like myself would be motivated to shell out cash for. Perhaps it’s only a matter of time.

(Confidential to Matt F: Hey, Fish, see what you can do about getting Thom and the boys to post some more b-sides and exclusives, won’t you? Much love.)