Yoz Grahame's Cheerleader: Juno MP3 Scraper

April 29, 2003

Juno MP3 Scraper

Juno publish pages with short clips of all their new records. Stef asked for a way to grab them all at once, with their ID3 tags set, so he could wander off with them on his iPod, then come back and buy all the ones he likes. Okay, Stef: here it is. You need the LWP and MP3::Tag modules installed. Edit the script to include your favourite genres from Juno's range, then run it - it'll download all the files to the directory the script is in. The main problem is that extensive use will completely hammer Juno's server, so the next step for this is for someone to stick it on a cronjob and make the zipped-up collections available over BitTorrent or OCN.

A quick note on obtaining and installing modules for Perl: The easiest way of doing it on Unix or Mac OS X systems is the cpan command, which locates, downloads and installs modules automatically, given the name of the module you want. (It doesn't get much simpler than that.) The equivalent for ActivePerl (which most Windows users run) is the PPM command. Both commands are briefly explained here. To download LWP, you should look for Bundle::LWP. However, if you're running on Windows, an extra caveat: the PPM repositories don't seem to have MP3::Tag. Fortunately, installing it manually is simple: Download this tarball, locate your Perl\site\lib folder, make an MP3 folder within it, then drag the Tag.pm and Tag subfolder from the tarball into the new MP3 folder. Done!

Posted by Yoz at April 29, 2003 04:30 PM | TrackBack

Comments

A word of warning from someone who never quite "got" modules... last time I tried running cpan on my Mac OS X (a couple of weeks back) it spent an hour or two upgrading Perl. I was lucky. Apparently this can completely break your ability to start the machine...

Posted by: Phil on April 29, 2003 05:19 PM

Yow. This used to be a regular pain with older versions of perl, but I haven't had it since 5.6. I thought Jagwire was 5.8, anyway?

Posted by: Yoz on April 29, 2003 05:23 PM

I'm with phil on this one; for installs older than 5.6.x, if you try to get any recent modules or bundles without explicitly specifying the version number, it'll get the highest version. This is fine and dandy, but if the bundle or module marks a newer version of perl as a prerequisite, then it'll download the *entire* perl5.whatever, and attempt to install it. Yick.

Posted by: gilbert on April 29, 2003 09:16 PM

On my OS X machine:

$ /usr/bin/perl --version

This is perl, v5.6.0 built for darwin

Posted by: Aaron Swartz on April 30, 2003 01:45 AM

The cpan program (and the CPAN module that powers it, as opposed to CPAN the archive) is something of a relic, and the version with perl 5.6.0 (as default- still!- on Mac OS X) is even more crufty.

Handy hints for the first-time Mac OS X cpan user, then:

* configuration is time-consuming. Stick at it. If you see the question about following prerequisites, answer 'ask' not 'follow'; this should prevent it trying to upgrade the whole of perl without you knowing.

* try and have wget or ncftpget available. They're better at downloading things than plain ftp.

* the first thing you should install is 'CPAN', to upgrade it. Do *not* install Bundle::CPAN; this has more prerequisites, some of which trigger upgrading perl. You'll probably have to reconfigure it, sadly. You could download the distribution manually to avoid the duplication, though: http://search.cpan.org/author/ANDK/CPAN-1.70/

* If you can get LWP to install, do so. It's a somewhat hairy module, but it'll help cpan no end.

* Consider CPANPLUS (whose executable is, confusingly, cpanp) instead. It's a reengineered version of CPAN.pm, which aims to provide a more robust, extensible architecture for fetching modules. http://search.cpan.org/author/KANE/CPANPLUS-0.042/

Hope this helps.

Posted by: paul mison on April 30, 2003 10:33 AM

wow. two bug fixes/essential enhancements:

1. stick "from juno.co.uk" in the album TAG so that you can group them all together.
2. prefix each track name/artist with 'z ' - there might be a less clumsy way to do this (what sorts post z, in ASCII?), but you definitely want these kept away from your normal track/artist listings.

as I've just discovered to my cost

Posted by: stef on May 6, 2003 08:41 AM

re: jaguar and 5.8, I heard from Apple that they royally screwed up there, and released with 5.6 by mistake. The guy in charge apparently got shouted at a bit. Nevertheless, something is rather weird with os x perl - try installing some XML modules and see what I mean. (oh, and any XML modules you do get running will need to be reinstalled after CPAN does it's 5.8 upgradey badness.)

and whatever you do, when you install Bundle::LWP, DO NOT allow get to be aliased. OSX is not case sensitive, so lwp get overwrites GET, (or the other way round, I forget)

AND, upgrading to 5.8 when you have fink installed is just a world of trouble

Posted by: Ben Hammersley on May 6, 2003 09:37 AM

I've made fix #1 that Stef suggested: each file is now album-tagged as "juno.co.uk". Haven't done a fix for the second one: it's easier if you just give the junoscraper its own folder. Plus, I've now added a log feature - every time the scraper runs, it writes each downloaded filename to junolog.txt. It also checks this file for any filename it's about to download, so it can skip files it's done in the past. (So it should be much quicker to run every few days)

Posted by: Yoz on May 6, 2003 09:46 AM

you didn't do number 2, but as http://www.asciitable.com/ so helpfully points out ~ is ASCII 126, and so sorts nicely to the end of the all the playlists.

seriously, this is important :)

Posted by: stef on May 6, 2003 09:13 PM

except it doesn't. damnit apple and their clevercleverness.

use z inztead.

Posted by: stef on May 6, 2003 09:52 PM

I really don't understand what you want here. I mean, if I take your request literally, all you need is this line just before the tagging bit:

$f->{artist} = "z " . $f->{artist};

... but I find it hard to believe that there is no better way of getting iTunes to separate one chunk of MP3s from another chunk of MP3s.

Posted by: Yoz on May 7, 2003 02:27 AM

ok. you'd better come and take a look at my ipod then.

you can browse lists by album, OR song title, or Artist name. the 'juno.co.uk' works fine for gathering together all the juno tracks, when I'm in sample-listening mode, but when I'm listen-to-the-normal-music mode, my artist list is about trebled in size, most of which are 45 second clips. My artist browsing experience is sub-optimal, all of a sudden

If every juno track had z prepended to the song and artist name, the tracks would be at the end of both lists, and my 'pod would still be usable. geddit?

was also thinking it might be good to stick the filename in the comments tag

Posted by: stef on May 8, 2003 02:56 PM

Oh I seeeeee. Okay. Hmmm. Is it working okay with that extra line?

Posted by: Yoz on May 8, 2003 03:47 PM

which line? this would be so much easier by email.

if you mean the juno.co.uk one, then it's fine

Posted by: stef on May 8, 2003 09:49 PM

Online Directory

Posted by: Need List on February 28, 2005 12:56 PM

Online Pharmacy

Posted by: Online Pharmacy on March 15, 2005 02:12 PM