Creating a flac set info file standard

[bacchist]

I'm working on creating a standard file format for holding all of the data that is typically included in txt files in shn/flac file sets.  All the stuff like artist name, date, venue, tracks, source, transfer, etc. I'm going to be developing a system of database and file tools to manage collections of shows.

Now, I've spent some time thinking about what information can and should be a part of the file... but I may be forgetting things. So I wanted to get input on what data it should contain.

[G. Augusto]

You might have better luck in the Tech Forum...

[ytowndan]

Are you talking about new filesets you're transferring, or changing ones that are already in circulation? 

[bacchist]

Are you talking about new filesets you're transferring, or changing ones that are already in circulation? 

The short answer would be changing ones already in circulation... But I really want to avoid THAT conversation.

[twatts]

No offense, but what's wrong with DB.ETREE???  http://db.etree.org/

I would prefer not to have to reorganize or rename my 1000s of FLAC files to conform with another DB... 

I think you would be better off asking Tom Anderson to separate out the PH stuff from DB.ETREE and adapt that as a stand-alone product to be used in conjunction with DB.

Terry

[jephrey]

are you talking like taking the typical .txt file and creating a "standard" with how the data is displayed so it can be imported/exported from a database OR embedding the text file as flags or maybe even as album art or lyrics into the music files?  Or are you trying to essentially replace the database down the road?

After much pondering myself, although I liked the idea of embedding the text file as the "lyrics" tag or a combination of tags, I ended up looking at the text file and taking a screenshot of just that window and using that as the album art.  Basically, I want to be able to see what show I'm listening to easily.  You've got to get close on an old iPod, but it's cool on a touch.  As for song names, I wanted them to be just that, the name of the song, but having the setlist and dates right there in the art is nice.  My text files are uniform too, line 1=band, line 2=date, line 3=location, city, ST, line 4=blank, line 5=first song.  I have a little program I wrote that takes the song names in that file and applies them to the song files and puts them in iTunes.  I'm sure the same can be done if left in flac.

[bacchist]

OK, I'm going to have to clarify myself a bit.  I'll start with a bit of background.

I lost my ENTIRE collection due to flooding, so I'm in the position of having to rebuild.  My collection was all converted to flac and tagged nicely with file names in the same format.  This was a painstaking process, done over a long period of time.

I used some tools in the etree-scripts package, like flacify.  flacify takes a directory of shn files, and converts it to tagged flac files.  The problem is, it parses the txt file to gather the information it needs to tag the files, and that's an ugly process.  Almost no two txt files are in the same format, people leave off information where it would be expected, put things in a different order, add notes in random places, etc. This results in album names including the entire source lineage, song names coming out incorrectly,  and all kinds of other bizarre effects.

So in order to automate a process like converting or tagging a fileset, the info file has to be manually edited into a format that can be parsed properly.  It's not realistic to write something that will parse all the different ways that people might decide to throw together the txt file.  It will be a much cleaner process to develop a standard for info txts to conform to, or alternatively, create something in addition to the current mishmash approach to organizing all the many sources on db.etree.org.

Basically what I'm going to do is start rebuilding my music collection by (re)downloading tons of shows.  I'm going to come up with a standard file to house all the information I can get about the seeds.  I'm thinking it will use YAML, since that's easy for people to read and create by hand, and also easy to parse with a variety of scripting/programming languages.  Then I will use those to convert the shows to flac with tags.  I'll begin to put together a database of these info files, and eventually turn it into some kind of overlay to the etree database.

I have a lot of ideas on where to go from there, but I'll cross that bridge when I get to it.

[bacchist]

I remembered having a similar discussion a couple years ago... All of the same problems exist, as far as I can tell.  My ideas for combatting them have changed a bit, but it's still related enough to mention: http://week4paug.net/index.php?topic=3991

[jephrey]

As I mentioned, I do exactly that.  Instead of worrying about everyone's text files being the same, I just "fix" every text file I get.  I suppose that is painstaking, but I'm not sure you'll get buy-in on a different than text format.  I went through a similar process as you.  I get the set, adjust the text file, then run a script to apply the tags to the files...  It's besides the point that I convert to ALAC along the way.

It'd be nice to set some rules, and you could use a grassroots effort for that.  I'd be on board and maybe the first to suggest a format (since I have text for every show 1994-2004 done in one particular way.)  I also have a database and can output that to excel or whatever

[kellerb]

it should all be in a database, tied to a key (i.e. etree db number), and then it would be relatively easier to pull it in via script,etc.

[jephrey]

sum'm like dat

[bacchist]

As I mentioned, I do exactly that.  Instead of worrying about everyone's text files being the same, I just "fix" every text file I get.  I suppose that is painstaking, but I'm not sure you'll get buy-in on a different than text format.  I went through a similar process as you.  I get the set, adjust the text file, then run a script to apply the tags to the files...  It's besides the point that I convert to ALAC along the way.

It'd be nice to set some rules, and you could use a grassroots effort for that.  I'd be on board and maybe the first to suggest a format (since I have text for every show 1994-2004 done in one particular way.)  I also have a database and can output that to excel or whatever

I'd love to take a look at some of your text files.  If they're all done the same way, then I could write something to parse it and convert it them to YAML. I would actually like to eventually get the format adopted by etree, and available much like the info texts are now.  I'm thinking you should be able to download the txt and md5 plus a new machine readable data file, that would allow quick and easy development of tools to work on entire filesets and collections.

A larger scale buy-in would be possible after it's available for a substantial segment of shows, and some worthwhile applications have been developed.

[DoW]

everytime I read situations like this, I jst want to remind people of one thing.  don't forget to enjoy the music in the middle of all this editing and formatting and tagging.

[DoW]

ok.  I just re-read the post from a while ago that you linked.  I say that whatever you do for personal use is fine, but if your plan is to someone change how bt.etree.org works and only allows flacs, tagged files and a text format, I can basically just guarantee that you are wasting your time.

I won't get inot what makes sense and what doesn;t.  I will just say that it is not going to change.  agree or not, it is reality.  regardless of where you stand on this, it is more of just a question of whether you really think it will change.

but if it is for your personal use, you can obviously do whatever you want.

[bacchist]

I mainly linked the other post because it shows that the problems that were around a couple years ago are still a problem.  That, and there might have been an good idea or two in there that I forgot.  Using par2 files would be sweet.  I'm not trying to force anything on anybody, but I would like to come up with some good solutions for doing things... If people want to adopt them, I'd be ecstatic.. If not, at least I'll have some nice toys to play with.