IPB

Welcome Guest ( Log In | Register )

3 Pages V  < 1 2 3  
Reply to this topicStart new topic
WavPack 4.0 Beta Release 3, lots of new goodies
Liisachan
post Jul 11 2004, 16:10
Post #51





Group: Members (Donating)
Posts: 119
Joined: 9-July 04
Member No.: 15225



QUOTE (bryant @ Jul 11 2004, 02:39 PM)
QUOTE
Actually, this same bug was first found in ttaenc 3.1, and I did the same testing in wavpack, and was embarrassed to learn that Wavpack has the same problem...
I hope this will be fixed in the final 4.0 release...

Hmm. This is a tough one because your test files work fine here, and I was not able to find any directly applicable information on the Internet. Was the problem fixed in TTA? Maybe I could look at what they did. It probably won't help, but you might try putting the whole filename in quotes.


Well, I think you can reproduce the problem by setting your locale (system langauge) = Japanese. In Windows XP, control panel -> Regional and Language Options -> Advanced -> Language for non-unicode program.
You dont need to have Japanese version OS to reproduce this bug. My OS is not Japanese version either. And possibly the same problem may happen in another language using multibute char. such as Chinese.

problematic 2nd half of double-byte char is one of the most common problems in i18n. There should be more than one solutions, one of which is to use Unicode internally, like many internationalized apps do.

A double-byte char uses 2 bytes to express 1 glyph. The problem is, the 2nd half of those 2 bytes might be confusing, like, it can be 0x5c and innocent parser thinks 0x5c as the path separator, or the letter for an escape sequence, and will be messed up. But it actually is not such a special (meta)character, but just the 2nd half of a common double-byte character.

The author of TTA, Alexander aka ald, said "I think that I know the reason of this problem. Going to fix it now." a few days ago.
-o switch in ttaenc can already handle double-byte char decently, and WinTTA (tau producer) is more perfect. As for WavPack, this problem is not conspicuous, because it supports pipeline, so if the user uses foobar2000 as the front-end, then the filename handling will be done by foobar2000. TTAENC cannot do that, not supporting pipeline, so it should parse %s from foobar2000 by itself anyway. So you might want to check his source when ready, or someone else's who knows this problem well...

The reason of this problem is very simple, but possibly fixing it is not that easy.
I hope you will find an elegant way to solve it.....
tyia

This post has been edited by Liisachan: Jul 11 2004, 16:11
Go to the top of the page
+Quote Post
bryant
post Jul 12 2004, 07:10
Post #52


WavPack Developer


Group: Developer (Donating)
Posts: 1291
Joined: 3-January 02
From: San Francisco CA
Member No.: 900



QUOTE (Liisachan @ Jul 10 2004, 03:25 PM)
Actually, this same bug was first found in ttaenc 3.1, and I did the same testing in wavpack, and was embarrassed to learn that Wavpack has the same problem...
I hope this will be fixed in the final 4.0 release...

Unfortunately, changing my language settings did not change how that test folder unpacked, but I think I figured out what was going on anyway. Please let me know if this helps:

http://wavpack.com/wp4-jis.zip

Thanks again, Case. smile.gif
Go to the top of the page
+Quote Post
Liisachan
post Jul 12 2004, 08:25
Post #53





Group: Members (Donating)
Posts: 119
Joined: 9-July 04
Member No.: 15225



QUOTE (bryant @ Jul 12 2004, 03:10 PM)
Unfortunately, changing my language settings did not change how that test folder unpacked, but I think I figured out what was going on anyway. Please let me know if this helps:
http://wavpack.com/wp4-jis.zip


Did you "install files for East Asian languages" (Language tab)? Maybe you have to change the "Text services and input languages" too. I can reproduce this problem on my Windows XP English version...

Anyway, thanks a lot for your work.
I tested "Win32 Version 4.0b3x 11-7-04" and the bug related 0x5C has been fixed smile.gif

Yes, fixed! Thanx again!

But this fix has this terrible side effect sad.gif
- now wavpack can not handle all the other double-byte chars (that is, all the double-byte chars that do not include 0x5c in the trailing byte)

Since its not that easy to describe what is happening by words, I upped this pic.
Could you please chack this one?
http://www.faireal.net/tmp/wv4b3x.png

wavpack [965C].wav
DOES work, while
wavpack [82A0].wav
fails.

The error message is: "file [82A0].wav.wav not found!"

The original WavPack 4.0b3 can do:
wavpack [82A0].wav
successfully, so your last tweak includes both debugging and enbugging...

I hope you wont start hating double-byte chars for that.
Go to the top of the page
+Quote Post
Duble0Syx
post Jul 12 2004, 16:33
Post #54





Group: Members
Posts: 465
Joined: 2-May 04
Member No.: 13847



It may be a dumb question, but does wavpack 4.0 support adding tags while encoding? For example when using Exact Audio Copy for extractio directly to wavpack. Would be nice if the files could be tagged directly, kind of like flac, rather than with some other app. If this is possible I'd be pleased to hear the command line string to do so. That would be the only thing slowing my down from switching. So far it looks like damn good work.
Go to the top of the page
+Quote Post
Faelix
post Jul 12 2004, 17:51
Post #55





Group: Members
Posts: 54
Joined: 6-July 02
Member No.: 2496



QUOTE (Duble0Syx @ Jul 12 2004, 12:33 PM)
It may be a dumb question, but does wavpack 4.0 support adding tags while encoding?  For example when using Exact Audio Copy for extractio directly to wavpack.  Would be nice if the files could be tagged directly, kind of like flac, rather than with some other app.  If this is possible I'd be pleased to hear the command line string to do so.  That would be the only thing slowing my down from switching.  So far it looks like damn good work.
*


Well, actually it is through another little application, but in the end it is transparent when using EAC, so learn on this page how to do it.
Go to the top of the page
+Quote Post
bryant
post Jul 12 2004, 18:15
Post #56


WavPack Developer


Group: Developer (Donating)
Posts: 1291
Joined: 3-January 02
From: San Francisco CA
Member No.: 900



QUOTE (Liisachan @ Jul 11 2004, 11:25 PM)
But this fix has this terrible side effect sad.gif
- now wavpack can not handle all the other double-byte chars (that is, all the double-byte chars that do not include 0x5c in the trailing byte)

Yeah, I was afraid that might happen. The problem is that shift-jis cannot be reliably parsed backward because the second byte of the two byte codes can also be above 0x80 (thanks, Bill). What I did should work in all cases except when the double byte code is the last character in the filename (I think).

I know how to fix this correctly, but it's a little more work. Thanks for helping me out on this.

This post has been edited by bryant: Jul 12 2004, 18:16
Go to the top of the page
+Quote Post
bryant
post Jul 12 2004, 23:49
Post #57


WavPack Developer


Group: Developer (Donating)
Posts: 1291
Joined: 3-January 02
From: San Francisco CA
Member No.: 900



Well, it wasn't as hard as I thought. I think I have got every case this time:

http://wavpack.com/wp4-jis.zip

Thanks very much for your assistance and patience on this.
Go to the top of the page
+Quote Post
Liisachan
post Jul 13 2004, 00:01
Post #58





Group: Members (Donating)
Posts: 119
Joined: 9-July 04
Member No.: 15225



QUOTE (bryant @ Jul 13 2004, 02:15 AM)
The problem is that shift-jis cannot be reliably parsed backward because the second byte of the two byte codes can also be above 0x80 (thanks, Bill).

Exactly.... You cannot parse it backward.
Quick hack would be like this:

CODE
    char szArg[] = "C:\\My Music\\[Some DB Charcters]\\foo.wav";
    char szBuf[ MAX_PATH + 1 ];
    char szPathSeparated[ MAX_PATH + 1 ][ MAX_PATH + 1 ];
    int iPathSeparated = 0;
    memset( szBuf, '\0', sizeof( szBuf ) );
    if( GetSystemDefaultLangID() == 0x411 ) // Japanese
    {
 int len = lstrlen( szArg );
 for( int i = 0, j = 0; i < len; )
 {
     if( (UINT) szArg[ i ] > 0x80 )
     {
   szBuf[ j++ ] = szArg[ i++ ];
   szBuf[ j++ ] = szArg[ i++ ];
     }
     else
     {
   if( szArg[ i ] == '\\' )
   {
       lstrcpy( szPathSeparated[ iPathSeparated++ ], szBuf );
       i++;
       j = 0;
       memset( szBuf, '\0', sizeof( szBuf ) );
   }
   else
   {
       szBuf[ j++ ] = szArg[ i++ ];
   }
     }
 }
    }


But this is ugly. You might want to process everything in Unicode internally....
Go to the top of the page
+Quote Post
Liisachan
post Jul 13 2004, 00:44
Post #59





Group: Members (Donating)
Posts: 119
Joined: 9-July 04
Member No.: 15225



ADDED: Sorry, not yet fixed.

I forgot to test the "half-width" case. In Shift_JIS Code Page, 0x80-FF can be used as a single-byte char too, like 0x80-FF used in French or German for special characters. Wavpack cannot handle this case yet.

Heres a demo result (pic):
http://www.faireal.net/tmp/0xb1.png

/ADDED




QUOTE (bryant @ Jul 13 2004, 07:49 AM)
Well, it wasn't as hard as I thought. I think I have got every case this time:

http://wavpack.com/wp4-jis.zip

Thanks very much for your assistance and patience on this.
*


Thank you very much biggrin.gif
You posted that one while I'm posting another one,
and YES, looks like this newest build finally fixes all the problems related to Shift_JIS. smile.gif smile.gif

Im really happy for that, and at the same time, let me add this for the future development:
That build is not yet perfect, as many other same tools are not, when it should handle Unicode filenames.
For instance, it cannot handle the path which contains both Japanese and Latin special characters such as (a umlaut).

Youll get an error like this (yeah, this is Bill's fault, basically):
http://www.faireal.net/tmp/mixed_filename.png

So, users would be in trouble if they had a folder in Japanese name and if they had some music files in German or French titles in it. Moreover, on win2k/xp, you can mix, say, Korean, Japanese, Arabic, and French in one filename. I guess that's why modern windows programs need to be Unicode-based.
It is very difficult to handle this gap for commandline tools because cmd.exe does not support Unicode properly while Windows per se uses Unicode. But ofcos normal Win32 GUI applications can manage unicode file names, like foobar2000 can open files in Japanese and files in Thai and files in Korean in the one same playlist.

Thanx again anyway smile.gif

This post has been edited by Liisachan: Jul 13 2004, 01:00
Go to the top of the page
+Quote Post
bryant
post Jul 13 2004, 04:32
Post #60


WavPack Developer


Group: Developer (Donating)
Posts: 1291
Joined: 3-January 02
From: San Francisco CA
Member No.: 900



Okay, this is obviously not the way to fix this problem because even if I get this language working right, there are hundreds of others. One of these days I will look into this fully.

In the meantime I have generated one final version which might just fix all cases for shift-jis because I now use only 81-9f and e0-fc for the 2-byte introduction and I query the default language.

http://wavpack.com/wp4-jis.zip

Thanks again for assistance... smile.gif
Go to the top of the page
+Quote Post
Liisachan
post Jul 13 2004, 09:47
Post #61





Group: Members (Donating)
Posts: 119
Joined: 9-July 04
Member No.: 15225



Fixed anyway. Thank you, bryant

QUOTE (bryant @ Jul 13 2004, 12:32 PM)
Okay, this is obviously not the way to fix this problem because even if I get this language working right, there are hundreds of others. One of these days I will look into this fully.


Thats exactly what I thought, as there are even "unicode-only" languages, like Tatar, now working on Windows somehow. But on the other hand, this confusing situation is fully understandable when I remember that WavPack has a long history, begining from the time when Unicode was not common at all.

QUOTE (bryant @ Jul 13 2004, 12:32 PM)
Thanks again for assistance...  smile.gif
*


You're welcome smile.gif
I am the one who should be thanking you, for your cool freeware, for its high speed (one of the fastest as lossless), and for its imaginative hybrid mode.
Keep up a good work!

Greetings from the east end of Asia
- Liisachan
Go to the top of the page
+Quote Post
robUx4
post Jul 13 2004, 09:53
Post #62


Matroska Developer


Group: Developer (Donating)
Posts: 410
Joined: 14-March 02
From: Paris
Member No.: 1519



Yeah Liisachan is always a precious help for developpers. You can ask Mosu about it smile.gif
We even named a Matroska release "Liisachan / CHIP release" (as it was also mentioned in the CHIP german magazine) !
biggrin.gif


--------------------
http://www.matroska.org/ : the best vapourware / http://robux4.blogspot.com/
Go to the top of the page
+Quote Post
jcoalson
post Jul 17 2004, 01:42
Post #63


FLAC Developer


Group: Developer
Posts: 1526
Joined: 27-February 02
Member No.: 1408



QUOTE (kuniklo @ Jul 9 2004, 03:56 PM)
QUOTE (bryant @ Jul 9 2004, 08:42 PM)
QUOTE
Bryant - is there any chance you could add a seek function to wvunpack? I rip cds to single flacs now and use flac's seek function to extract individual tracks from the resulting file. I can't do this with wvunpack.
That is a nice feature of FLAC. I'll look into puting something like that in. (What would be nice is if could use the cuesheet and just decompress by track number...)

Automatically splitting from cuesheets would be excellent! One vote from me!
*

yes, this is a useful feature. some squeezebox guys have been asking for this too since they use the command-line to decode and pipe out data. I just checked in just such a --cue option to CVS; will be in the next release pretty soon.

Josh
Go to the top of the page
+Quote Post
kuniklo
post Jul 17 2004, 05:42
Post #64





Group: Developer (Donating)
Posts: 193
Joined: 9-May 02
From: Emeryville, CA
Member No.: 2010



QUOTE (jcoalson @ Jul 17 2004, 12:42 AM)
QUOTE (kuniklo @ Jul 9 2004, 03:56 PM)
QUOTE (bryant @ Jul 9 2004, 08:42 PM)
QUOTE
Bryant - is there any chance you could add a seek function to wvunpack? I rip cds to single flacs now and use flac's seek function to extract individual tracks from the resulting file. I can't do this with wvunpack.
That is a nice feature of FLAC. I'll look into puting something like that in. (What would be nice is if could use the cuesheet and just decompress by track number...)

Automatically splitting from cuesheets would be excellent! One vote from me!
*

yes, this is a useful feature. some squeezebox guys have been asking for this too since they use the command-line to decode and pipe out data. I just checked in just such a --cue option to CVS; will be in the next release pretty soon.

Josh
*



I'll definitely be making use of this. Thanks!
Go to the top of the page
+Quote Post
Duble0Syx
post Jul 20 2004, 09:27
Post #65





Group: Members
Posts: 465
Joined: 2-May 04
Member No.: 13847



I just wanted to comment on how nice wavpack is looking. From what I can tell it seems overall better than most lossless compressors. When compared with things like optimfrog and ape, the compression is bit less, but damn does it encode fast. I've testing it using -hm for encoding. I like how it stores the wav files md5 in the file. Not found any bugs yet either. Nice work. Just waiting for the final release now. biggrin.gif

This post has been edited by Duble0Syx: Jul 20 2004, 09:28
Go to the top of the page
+Quote Post
kuniklo
post Jul 21 2004, 19:27
Post #66





Group: Developer (Donating)
Posts: 193
Joined: 9-May 02
From: Emeryville, CA
Member No.: 2010



QUOTE (jcoalson @ Jul 17 2004, 12:42 AM)
yes, this is a useful feature.  some squeezebox guys have been asking for this too since they use the command-line to decode and pipe out data.  I just checked in just such a --cue option to CVS; will be in the next release pretty soon.

Josh


Just tried this out. Seems to be working great. Thanks!
Go to the top of the page
+Quote Post

3 Pages V  < 1 2 3
Reply to this topicStart new topic
1 User(s) are reading this topic (1 Guests and 0 Anonymous Users)
0 Members:

 



RSS Lo-Fi Version Time is now: 23rd September 2014 - 21:38