dcdev mailing list

About the file list format; One can discuss the best compression ratios
back and forth (I personally think .xml.bz2 would be perfectly fine and
small enough, and prolly a lot smaller than ebml), but it doesn't really
matter. The most important issue (partly since bz2 _is_ a good
compressor)  is usability, maintainability and through-out support for
the format. In that case, nothing beats XML. Also there are _several_
libraries for both zlib, bzip2 and a h*ll lot of libraries for xml, for
any operating system, in any programming language, and even for
web-things such as php etc...

About regexp library choise, I'd say the support for wide charsets
should not only be considered, but required. Regex++ supports it, that's
all I know for now.

About this mailing list; It has become a chat. I'd suggest those of you
"chatting" here go visit "us" on dev.myhub.org:666. You all have
accounts there, or can get one by sending anyone of us a mail.

O

About regexp library choise, I'd say the support for wide charsets
should not only be considered, but required. Regex++ supports it, that's
all I know for now.

about substring vs regex discussion, I'd like to remember regex is not what was proposed initially (regex was only a subset of this proposition). The previous proposition was something like that
( N =~ avi$ || N =~ ogm$ ) && ( T =~ ^video/ ) && ( S > 450000000 )
which can natively mix both regex and substring.

Eric

Opera writes:
> About the file list format; One can discuss the best compression ratios
> back and forth (I personally think .xml.bz2 would be perfectly fine and
> small enough, and prolly a lot smaller than ebml), but it doesn't really
> matter. The most important issue (partly since bz2 _is_ a good
> compressor)  is usability, maintainability and through-out support for
> the format. In that case, nothing beats XML. Also there are _several_
> libraries for both zlib, bzip2 and a h*ll lot of libraries for xml, for
> any operating system, in any programming language, and even for
> web-things such as php etc...

I'm all for .xml.bz2. I don't even see a reason for a binary file
list. If it truly is smaller, it won't be more than a few bytes
considering the bzip2 compression of XML. Therefore, I agree with XML
since it's much more accepted everywhere.

> About regexp library choise, I'd say the support for wide charsets
> should not only be considered, but required. Regex++ supports it, that's
> all I know for now.

Indeed, it should be that way. However, it's not usually a
problem. I'm not sure how Windows works in this area, but on *ix
systems, filenames are still stored as 8-bit byte strings, encoded
using the character set of the current locale. Therefore, when a regex
comes in on the protocol with UTF-8, and it cannot be converted into a
multi-byte string in the current locale's charset, that would
constitute an automatic false expression, since if the regex contains
characters that aren't in the locale's charset, then no filenames can
exist which contains those characters anyway.

Fredrik Tolf

I'm all for .xml.bz2. I don't even see a reason for a binary file
list. If it truly is smaller, it won't be more than a few bytes
considering the bzip2 compression of XML. Therefore, I agree with XML
since it's much more accepted everywhere.

I agree, XML is probably better because sharelist is mainly composed of text and converting size into text won't waste a lot of memory (and moreover will prevent us from dealing with CPU endianness).

 > About regexp library choise, I'd say the support for wide charsets
 > should not only be considered, but required. Regex++ supports it, that's
 > all I know for now.

Indeed, it should be that way. However, it's not usually a
problem. I'm not sure how Windows works in this area, but on *ix
systems, filenames are still stored as 8-bit byte strings, encoded
using the character set of the current locale. Therefore, when a regex
comes in on the protocol with UTF-8, and it cannot be converted into a
multi-byte string in the current locale's charset, that would
constitute an automatic false expression, since if the regex contains
characters that aren't in the locale's charset, then no filenames can
exist which contains those characters anyway.

I don't 100% agree. On *nix, filenames are stored as byte strings as you said but the usage of the current locale charset is just a de facto standard because most of the programs work like this. AFAIK, nothing prevents a program to use UTF8 encoded filename.

Eric

eric writes:
> >  > About regexp library choise, I'd say the support for wide
> >  > charsets should not only be considered, but required. Regex++
> >  > supports it, that's all I know for now.
> >
> > Indeed, it should be that way. However, it's not usually a
> > [...]
> > exist which contains those characters anyway.
> > I don't 100% agree. On *nix, filenames are stored as byte strings
> as you said but the usage of the current locale charset is just a
> de facto standard because most of the programs work like
> this. AFAIK, nothing prevents a program to use UTF8 encoded
> filename.

Certainly, nothing prevents a program from doing that, but I seriously
don't think that any programs would do that, since that would be
stupid. Having filenames in the system stored with different charsets
seems like a really bad thing in my mind. Also, if that is the case,
you won't be able to support wide char filename searches anyway, since
you won't know how to convert filenames into wchar strings.

Fredrik Tolf