Skip to main content

Notice

Please note that most of the software linked on this forum is likely to be safe to use. If you are unsure, feel free to ask in the relevant topics, or send a private message to an administrator or moderator. To help curb the problems of false positives, or in the event that you do find actual malware, you can contribute through the article linked here.
Topic: MP3 repacker (Read 586577 times) previous topic - next topic
0 Members and 1 Guest are viewing this topic.

MP3 repacker

Reply #50
Quote
... The red is the size of each frame (constant, of course) but the black dots are the actual amount of data in each frame. The blue is how many bits are in the reservoir per frame. ...
[a href="index.php?act=findpost&pid=298272"][{POST_SNAPBACK}][/a]

How did you produce this graph? Very interesting to me as I'm interested a lot in the maximum actual frame size producable by encoders especially Lame 3.90.3 api and -b320. You show already that actual frame size can be more than the size of a 320 kbps frame (minus frame header minus side info).
(I'm interested in differences between Lame 3.90.3 and 3.96.1/3.97b when using cbr320. There is some evidence that 3.90.3 cbr320 provides better quality. 3.96+ doesn't use bit reservoir which restricts actual frame size to the size of a 320 kbps frame minus frame header minus side info. 3.90.3 however does use the bit reservoir but it is unclear with which restriction concerning maximum actual frame size.)
lame3995o -Q1.7 --lowpass 17

MP3 repacker

Reply #51
Quote
How did you produce this graph?

Omion may have used his very own mp3repacker tool with some of the DEBUG options enabled to output the numbers. You can try that yourself, simply edit the file mp3.pm accordingly.

Quote
There is some evidence that 3.90.3 cbr320 provides better quality.

Keep that discussion out this thread, please.
(for other readers, there is a thread about halb27's search for the ultimate way to create "[...] a universally usable mp3 archive not needing any transcoding any more" - using very high bitrates)

MP3 repacker

Reply #52
Quote
How did you produce this graph?

I made a Perl script which I use as a debugger for my repacker. It outputs a bunch of garbage about each frame, then makes a few Mathematica-readable lists. I then used Mathematica to create the graphs.

@smack: The debug options in mp3.pm do a similar thing, but they don't output a nice list. It's more of a random spamming  .

Quote
Very interesting to me as I'm interested a lot in the maximum actual frame size producable by encoders especially Lame 3.90.3 api and -b320. You show already that actual frame size can be more than the size of a 320 kbps frame (minus frame header minus side info).
(I'm interested in differences between Lame 3.90.3 and 3.96.1/3.97b when using cbr320. There is some evidence that 3.90.3 cbr320 provides better quality. 3.96+ doesn't use bit reservoir which restricts actual frame size to the size of a 320 kbps frame minus frame header minus side info. 3.90.3 however does use the bit reservoir but it is unclear with which restriction concerning maximum actual frame size.)
[{POST_SNAPBACK}][/a]

I don't know about the quality difference, but the fact that newer LAMEs don't use the reservoir for API is why I made the repacker in the first place.For every frame that is less than 320kbps, some space is wasted.

PS. The Perl script I used is [a href="http://people.ucsc.edu/~rswilson/other/mp3reader.pl]here[/url].
"We demand rigidly defined areas of doubt and uncertainty!" - Vroomfondel, H2G2

MP3 repacker

Reply #53
Quote
I don't know about the quality difference, but the fact that newer LAMEs don't use the reservoir for API is why I made the repacker in the first place.For every frame that is less than 320kbps, some space is wasted.
PS. The Perl script I used is here.
[a href="index.php?act=findpost&pid=340552"][{POST_SNAPBACK}][/a]


Thank you very much. I think it will help me a lot.
lame3995o -Q1.7 --lowpass 17

MP3 repacker

Reply #54
Hmm, I seem to be having a strange issue with the repacker.
I was trying to repack some 192kbps CBR files to see the actual frame sizes (not to save any space, I know it won't help) and make it look like nifty VBR files.
However, when I ran it through the repacker, it spit out this error:
ERROR: Can't run frame location on layer 1 or 2 data at mp3.pm line 936.

Not sure what could be causing it (since it's layer III according to foobar and everything else decodes 'em just fine...)
Well, I got the first 4KiB of the file and uploaded it. If you need a bigger clip, I can give you one.
http://firon.site.voila.fr/mp3clip.mp3

I didn't encode these myself, but according to some data at the end of the file, it was made by LAME3.89 (beta)

MP3 repacker

Reply #55
Very strange. I'll take a look. (and see if I still remember what I was thinking when I wrote that  )

[edit]
HA! That's hilarious! It looks like it's all ID3v2's fault... again. Whoever tagged this shoulda used the "unsynchronization" feature...

Short answer: delete (or otherwise mangle) the tag!

Long answer:
The MP3 header sync pattern is 0xFFF. (normally doesn't occur in text)
There is an ID3v2 tag at the beginning of the file. (so far, so good)
That tag uses UTF-16 encoding. (still good)
The Unicode strings all have a 16-bit Unicode "byte order mark" ("BOM", Unicode character U+FEFF)
However, it's little-endian UTF-16 so the character U+FEFF is encoded to the bytes 0xFFFE. (BIG problem)

Now, since the ID3 tag is before everything else in the file, my program scans it for the first sync header. It sees the 0xFFF from the Unicode BOM and parses it as a frame header.

The tag was parsed as an MP1 frame, and since I couldn't find anything on how MP1 or MP2 files store the data I just kill the program when one is encountered. (Nobody here uses MP1 anyway...)

This whole problem could have been avoided if whoever tagged the file used the ID3 unsynchronization scheme. That outputs a 0x00 byte after all 0xFF bytes to break up any false sync headers PRECISELY because of this problem.

Most other programs at least know how to skip over ID3 data, which is why no other programs fail on this. mp3repacker, however, has the attitude that if it ain't an MP3 frame it's junk  (and I don't plan on changing that)

So the problem came from a combination of things:
* The file uses an ID3v2 tag
* The tag does not use the feature that breaks up confusing bytes
* The tag uses the EXACT Unicode encoding that results in lots of 0xFFFE strings
* My program has no idea what a "tag" is, or how to skip one when it sees one

My debugger program gives this as output:
Code: [Select]
Get a bunch of data from the file
Begin reading at 0 - 4096
Found a possible header at 21 (21)
 MPEG1 Layer 1
 160 kbps
 48000 Hz
 Stereo
 CRC present
 Padded frame
 Frame length: 481
Found a possible header at 962 (962)
 MPEG1 Layer 3                            
 192 kbps                                
 44100 Hz                                
 Stereo                                  
 CRC present                              
 Frame length: 626          
Found a possible header at 1588 (1588)
 MPEG1 Layer 3
 192 kbps
 44100 Hz
 Stereo
 CRC present
 Padded frame
 Frame length: 627
Found a possible header at 2215 (2215)
 MPEG1 Layer 3
 192 kbps
 44100 Hz
 Stereo
 CRC present
 Padded frame
 Frame length: 627
Found a possible header at 2842 (2842)
 MPEG1 Layer 3
 192 kbps
 44100 Hz
 Stereo
 CRC present
 Padded frame
 Frame length: 627
Found a possible header at 3469 (3469)
 MPEG1 Layer 3
 192 kbps
 44100 Hz
 Stereo
 CRC present
 Padded frame
 Frame length: 627
No possible headers found
 And the end of the file has been reached!
bitrate={160,192,192,192,192,192};

bpf={3848,5008,5016,5016,5016,5016};

bitsused={,4142,4998,4716,4446,4328};

reservoir={,0,70,34,33,66};


The first frame is the one from the header, and you can see that it is different from all the rest.
"We demand rigidly defined areas of doubt and uncertainty!" - Vroomfondel, H2G2

MP3 repacker

Reply #56
Well, the tags were made by foobar2000 (I'd retagged the files with it), so blame it I guess.
I use ID3v2 and ID3v1 for compatibility reasons.

I'm not sure what to do. Turn off the BOM and rewrite the tags? It's a lot of files so I can't really delete or mangle the tags.

MP3 repacker

Reply #57
Which version of the ID3v2-wrinting component do you have? I just downloaded the 1.17 and it only writes the BOM if it's actually using non-ASCII characters. I noticed that the file you uploaded has no reason to be in Unicode, so getting the newest version of the component and re-saving the tag will help in this case.

I would recommend against disabling the BOM, as it looks like it's the only way a program can tell if the data is actually Unicode and it's technically required (*)

You have a few options:
1. Get version 1.17 of the ID3v2 component and hope you don't run into any Unicode
2. Don't use ID3v2 anymore (ID3v1 and APE tags are both OK)
3. Don't use mp3repacker! It probably won't help much on a CBR 192 file
4. Pester whoever made the Foobar ID3v2 component (probably Peter) to support one of the following:
[span style='font-size:8pt;line-height:100%'].[/span] a. Big-endian Unicode
[span style='font-size:8pt;line-height:100%'].[/span] b. Unsynchronization padding

Note that although #4 would be the best solution, it is the least likely to go through, as evidenced from the ID3v2 'about' box:
Quote
About ID3v2 tag support

You've got your ID3v2, now STFU

Good luck! 

(*) Also, my version puts up a hilarious warning message if you try to disable the BOM. So don't do it.
"We demand rigidly defined areas of doubt and uncertainty!" - Vroomfondel, H2G2

MP3 repacker

Reply #58
I've got version 1.20, which I nabbed off kode54's site. Perhaps I'll bother him about #4 on IRC next time I see him
Anyway, I rolled back to 1.17, no more problems.  Thanks for the tip.

I don't expect the mp3repacker to really help on these particular CBR files, I mostly wanted to use it for fun (and to see what impact the bit reservoir has on so-called CBR files).

TAKE OFF EVERY 'BOM'!!
YOU KNOW WHAT YOU DOING?

MP3 repacker

Reply #59
I am really impressed by this program.  It converts so much faster than EAC.  I am proud of myself simply for figuring how to run PERL.  I wish I could contribute to to the cause for batch processing.  for now all I can do though is wait for someone really smart to save the day.

MP3 repacker

Reply #60
Well, it converts faster than EAC because it doesn't really do anything  . All my program does is re-arrange the bytes in an MP3 file (and might throw out a few unused ones). Although I am also impressed at how fast it goes. Perl is not known for speed.

As for batch processing, asonicboom posted a way to use an external program to do batch processing in this post. As for actually supporting it in my program, I might do something about it. I haven't programmed Perl for a while, so I'll have to dust off Ye Olde Camle Booke and see how best to support multiple files.

I can't work on it right now because I'm in the middle of an OCaml program, and using 2 programming languages at once just leads to confusion (and many, many bugs).
"We demand rigidly defined areas of doubt and uncertainty!" - Vroomfondel, H2G2

MP3 repacker

Reply #61
Quote
Well, it converts faster than EAC because it doesn't really do anything  . All my program does is re-arrange the bytes in an MP3 file (and might throw out a few unused ones). Although I am also impressed at how fast it goes. Perl is not known for speed.

As for batch processing, asonicboom posted a way to use an external program to do batch processing in this post. As for actually supporting it in my program, I might do something about it. I haven't programmed Perl for a while, so I'll have to dust off Ye Olde Camle Booke and see how best to support multiple files.

I can't work on it right now because I'm in the middle of an OCaml program, and using 2 programming languages at once just leads to confusion (and many, many bugs).



over at poineerprodjforums they are telling everyone that the only way to convert a VBR mp3 to a CBR version (for use on the pioneer CDJ-200)  is if they que them all up in Easy CD Creator and do it all in there. 

I sooo want to chime in and say  You know what, not only is that slow as hell, but it's just flat out shitty!.  If only we had a system for batch-processing a list, or a directory mp3s; and  maybe even a GUI....  Oh boy, would I love to go over there and say SHOVE IT!!  WE FIGURED OUT A WORK-AROUND FOR YOUR DAMN MACHINES THAT DON'T SUPPORT VBRMP3!!!    [a href="index.php?act=findpost&pid=357223"][{POST_SNAPBACK}][/a]

for now I will check out asonicboom's method.  good job!  we'll get there some day.

MP3 repacker

Reply #62
Code: [Select]
Name: MP3 Packer
Path: C:\WINDOWS\system32\cmd.exe
Parameter: /c "cd /d "E:\Backup\Programs\MP3 Packer\MP3 Packer v0.04" && perl mp3packer.pl -mst "%_path%" "%_folderpath%\%_filename%vbr.mp3""

What are the specifics for getting this 'inserted' into mp3tag?  I'd liek to do this, but a wee bit more info would be great.

MP3 repacker

Reply #63
jaybee:

Make sure you've got Omion's Perl tool in a permanent folder, and get the latest version of Mp3Tag, install it and open it. Go Tools --> Options. Click on the last option in the tree menu 'Tools'.  Now click on the icon to the right on the text box. And enter this:

Name: MP3 Packer (or whatever you want)
Path: Navigate to C:\Windows\System32\cmd.exe
Parameter: /c "cd /d "C:\Program Files\Encoding\mp3repack" && perl mp3packer.pl -mst "%_path%" "%_folderpath%\%_filename%-vbr.mp3""

N.B Where mine says 'C:\Program Files\Encoding\mp3repack', just change it to whatever folder you've UnRared the MP3 Packer to.

And check 'For all Selected Files'. Press OK and come out of the menu.

Now change the directory (press CTRL+D) and navigate to a folder of mp3's you want to repack. Select the MP3's, right click on them, go Tools --> MP3 Packer. You'll then get loads of command line terminals come up. Just leave it until it's all finished.

Your new files will have -vbr at the end of the filename.


And thanks Omion, you've solved a problem for me or cutting MP3's gaplessly. Now thanks to Sebi's PCutMP3 and this tool, my MP3's play on my Karma without the faintest gap.

MP3 repacker

Reply #64
Quote
And thanks Omion, you've solved a problem for me or cutting MP3's gaplessly. Now thanks to Sebi's PCutMP3 and this tool, my MP3's play on my Karma without the faintest gap.
[a href="index.php?act=findpost&pid=357647"][{POST_SNAPBACK}][/a]

That was a completely unintended benifit, but I suppose it works.  I find it kind of funny that I made the tool to turn CBR into VBR files, and now it seems that most people are using it the other way around!

About the batch processing: I think I'll make it so that if you give it a directory as input, it will process all the .mp3 files within it. Then it'll add a specified string to the end of the filenames, like "-vbr". I should have it ready for testing in a few days.
"We demand rigidly defined areas of doubt and uncertainty!" - Vroomfondel, H2G2

MP3 repacker

Reply #65
Quote
That was a completely unintended benifit, but I suppose it works.  I find it kind of funny that I made the tool to turn CBR into VBR files, and now it seems that most people are using it the other way around!


I do turn my CBR's into VBR 

The reason I use repacker is because when PCut (and I'm not bashing Sebi's prog here, it's superb) cut's CBR MP3's it actually makes them VBR (the first frame is smaller than the rest CBR bitrate) but doesn't put any Xing header on. This messes up my Karma as it can't figure out the length of the file and gapless playback using the Lame tag padding.

Your MP3 packer, aswell as shedding KB's off, makes the file completely VBR compliant meaning gapless playback works like a charm.

 

Batch processing sounds good too...

MP3 repacker

Reply #66
Many thanks senab.  Works like a charm.

And thanks for the interesting tool Omion.  Just tried it on a 185,809kb 320kbps file and it got it down to a 183,263kb 316kbps file.  Not as much as I thought it might, but I'll play with some other files.  Mind you, 2.5mb ain't too bad is it.

MP3 repacker

Reply #67
Quote
I do turn my CBR's into VBR  
[a href="index.php?act=findpost&pid=357668"][{POST_SNAPBACK}][/a]

Oh. Right. I knew that.  It's still not quite what I had in mind...

Batch processing is coming along nicely, but I need to get to work soon. Should be finished by tomorrow night.
"We demand rigidly defined areas of doubt and uncertainty!" - Vroomfondel, H2G2

MP3 repacker

Reply #68
Quote
And thanks for the interesting tool Omion.  Just tried it on a 185,809kb 320kbps file and it got it down to a 183,263kb 316kbps file.  Not as much as I thought it might, but I'll play with some other files.  Mind you, 2.5mb ain't too bad is it.
[a href="index.php?act=findpost&pid=357669"][{POST_SNAPBACK}][/a]

Do you know what was used to encode it? What kind of music is it? I may have to revise my "~10% off" statement...

[ edit: Just did a big test of ~6 hours of music and I got the files 3% smaller. Time to change the claim on the front page... ]
"We demand rigidly defined areas of doubt and uncertainty!" - Vroomfondel, H2G2

MP3 repacker

Reply #69
Quote
Do you know what was used to encode it? What kind of music is it? I may have to revise my "~10% off" statement...[a href="index.php?act=findpost&pid=357735"][{POST_SNAPBACK}][/a]

LAME 3.96 (according to Mr QM)
Hip-hop from DJ Yoda.

MP3 repacker

Reply #70
Kay, 0.07's out, and adds full-directory processing.

Basically, if you specify a directory as the anput file, it will do all MP3s that are in it (NOT recursive).

By default, it will append "-vbr" to the filename right before the extension, so a.b.c.mp3 will turn into a.b.c-vbr.mp3. Change the appended string with the -a option.

Also, it will skip files which already have the appended string on them. So if you specify "-a monkeys" it will skip the file "lots_of_monkeys.mp3". The idea is that such files have already been processed. You can force processing of all files by using the -A option.

Use the -X option to delete files after they are processed. USE WITH CAUTION! Only do this if you trust me to not delete your music. (I wouldn't!  )


The batch processing is by no means perfect. If you cancel processing in the middle and re-run it, it will probably throw an error about "conflicting filenames". In this case. you will need to delete EITHER the previous output files OR the input files which have already benn successfully processed.

[edit]
Just realized, some options don't work when they probably should:
-f doesn't work with batch processing
-X doesn't work with single-file processing
(-a and -A don't work with single-file either, but they're not supposed to)
[/edit]

I think that's about all the warnings I can give out. Good luck! 



@jaybeee:
Hmm... I think I'll have to revise my statement about taking off ~10% from API files. Maybe my testing isn't extensive enough.
"We demand rigidly defined areas of doubt and uncertainty!" - Vroomfondel, H2G2

MP3 repacker

Reply #71
this is an amazing tool!! I think everyone is too busy converting their mp3s (which takes a lot less time now!) and actually playing the music in their player of choice to make posts about how pleased they are.  I expect my pioneer cdj-200 unit in the mail tomorrow, and if it plays all these files that I've converted to CBR from VBR.... you have no idea how happy I will be.  My grin will stretch from ear to ear, all day long.   

I am also super excited about the GUI for this program.  Makes it so easy! Now an even bigger range of folks can use your software.  I'm sure that some companies who sell mp3 cd players, or portable mp3 players... whatever... wish they had thought like this beforehand!  I'm sure there's money in a market like this.  You guys are so rad! 

  where would we be, on the internet- without long-distance collaberation and open-source software like this?  NOT FAR AT ALL!

MP3 repacker

Reply #72
^^ I'm confused: I thought it made CBR files VBR, not the other way round.  At least it does with mine.

MP3 repacker

Reply #73
Quote
^^ I'm confused: I thought it made CBR files VBR, not the other way round.  At least it does with mine.
Primarily, you are correct. According to Omion's initial post: "Also, using this program on VBR files, or anything below ~200kbps is probably useless."

The single exception I know of, is converting any file (including VBR) to 320k CBR:

perl mp3packer.pl -b320 infile.mp3 outfile.mp3

Regards,
Madrigal

MP3 repacker

Reply #74
Quote
^^ I'm confused: I thought it made CBR files VBR, not the other way round.  At least it does with mine.
[a href="index.php?act=findpost&pid=358589"][{POST_SNAPBACK}][/a]

That's the main reason it exists. But as Madrigal says, by forcing the minimum bitrate to 320, you can turn any MP3 into CBR 320. The reason one would do this is to play an MP3 on something which only supports CBR, like, apparently, most MP3 DJ mixing thingies.

Also, ABR files may be able to be turned into CBR at a lower bitrate, but there's no easy way to find out what the lowest CBR bitrate is.

PS. I went on the Pioneer DJ forums to get some more information about this, and all I discovered was how very glad I am that HA exists. A choice quote:
Quote
VBR = not as good as CBR, even if you think you're saving space, you're sacking audio quality.
"We demand rigidly defined areas of doubt and uncertainty!" - Vroomfondel, H2G2