dcdev mailing list

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Great stuff there. I like your initiative, but any plans to phase out old or obsoleted features?

I've been working on TTH and a new queueing mechanisms for some time now... But with these changes and future changes, the backwards compatibility problem will keep growing (one of the reasons I made the first DCTNG suggestion). Therefore I suggest we declare certain old and/or obsoleted features as depreciated, to keep code complexity down and in the future we can clean out "dead" code. Let's set a date for when certain features are phased out.
I'm first of all thinking about the MyList.DcList file, which isn't necessary any more, and with the new XML extensions, the MyList.bz2 shouldn't be needed either in a few months.
And while I'm at it, why not scrap the old $get commands and consistently use UTF-8 for filenames. All (?) clients use the filename they got from the $SR without decoding/encoding it so by using UTF-8 (or any other encoding) doesn't matter to the remote system (except that some characters may look like crap :). I've used UTF-8 encoded filenames nearly two years already...

DC++ does have the largest market share, so you decide how to play this game ;)

Cheers

- -j-

- -- - -janvidar-

Dj Offset / QuickDC
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.4 (GNU/Linux)

iD8DBQFAMLJRMLjmoUcyZAoRAigCAJ0X6GpKbhQ5v7sPFWEPlbdK1pBf/QCgmVY2
lqEOHwSHwYbgNQLgpedaxOw=
=Usjx
-----END PGP SIGNATURE-----

On Mon, 16 Feb 2004, Jacek Sieka wrote:

The next dc++ will support a few new things, and if you have any comments on
these, this is a good time to speak up...

Actually, I have a slightly related comment.

Have you considered implementing hublist.org's XML hublist format?

/fnord

Ok, it's time for the dc world to take a step up on the ladder, and since I
happen to own the biggest client I guess if I don't do it, it won't have any
impact...

The next dc++ will support a few new things, and if you have any comments on
these, this is a good time to speak up...

1) Hashing. I've chosen TTH, as it has a few nice properties, the most
important being the ability to check individual file parts. The choice of
Tiger as hash function for the merkle trees is quite arbitrary, but was
chosen because bitzi and bcdc++ already use it, and I don't think the world
needs another hash type...SHA1 could have been used to save a few bytes and
cpu cycles, but the difference is silly since hashing speed is capped by
shitty disk drive speed anyway. The only thing speaking against tiger is
that sha1 has been researched for weaknesses much more thouroughly, and if
anyone knows of anything that would compromise (it's security, not speed)
tiger, speak up now citing the source. Searches are done using type 9, with
TTH:xxxx as prefix to the search string. Hash support is sent to the hub
using $supports, so that hub owners can nuke clients that don't support TTH
/ have it turned off...no use to hash your own files unless everyone else
does it as well...

2) XML File lists. The file list of my documets folder increased ~3.5% (55k
vs 57k) compared to the nmdc style file list compressed with bz2 (in both
cases)...adding hashes to it will obviously grow it more. You can argue all
you want about binary formats and whatnot, I don't care.
Also, dc++ will not offer uncompressed xml lists (at least not in the first
test release), as these can easily grow to silly big files (my 55k list with
8400 files is ~355k uncompressed)
The format is more or less what I wrote earlier, but without the columns (I
don't feel like defining data types for the columns at the moment...version
2 perhaps...)
The file list contain TTH roots for all files that have been hashed so
far...
The file lists use utf-8 as encoding, and therefore also introduce new $get
commands that use utf-8 for filenames (yes, this is unfortunately
necessary), you'll find the specs in extensions.txt or the wiki "soon", but
basically they're called $UGetBlock and $UGetZBlock and have the same
parameters as $GetZBlock...DC++ will not accept any other encoding for the
xml file list.

3) The GetZBlock changed slightly to accommodate for $UGetBlock and
$UGetZBlock. DC++ is also leaving the $GetTestZBlock phase, so the new
version is final.

enjoy/J

Ps on a side note, dc++ finished hashing the 8400 files (~4 gb on a 5400rpm
notebook drive) when I was halfway through this mail, so I don't see any
particular speed issues either...the file list grew to 246k compressed and
770k uncompressed...

Great stuff there. I like your initiative, but any plans to phase out old or obsoleted features?

Not really, I'll scrap everything in one blow when I move to ADC...if it
works out, fine, if not...

And while I'm at it, why not scrap the old $get commands and consistently use UTF-8 for filenames. All (?) clients use the filename they got from the $SR without decoding/encoding it so by using UTF-8 (or any other encoding) doesn't matter to the remote system (except that some characters may look like crap :). I've used UTF-8 encoded filenames nearly two years already...

Well, adding utf8 didn't break anything really, and dc++'s code became only
minimally more complex because of the new commands so...
Not everyone's using bzip2, and not everyone knows how to do it either, so
mandating it for lists would mean fewer clients...(although the
pseudo-huffman encoding is about the most stupid I've seen for a long
time...)

/J

Jonathan Joseph Hess wrote:

Congrats on getting the TTH going. I'm also implementing this hashing strategy and would like both of our clients to be compatible out of the gate so as to not introduce a mess. Are you going to have external behavior identical to BCDC++?

Yes, so far, the code in DC++'s CVS is outwardly compatible - it replaces the hubname with TTH:BASE32ENCODEDHASH in $SR - it also advertises TTHSearch in the hub <-> client Supports (as well as GetZBlock) so that new hubs can know if the clients have those two options enabled.  Currently, there's no client to client support for exchanging hashes, or for verifying completed downloads against them.  I'm sure both will come in good time.

FYI, you might be interested to know that you can query Bitzi with TTH roots now - though you cannot retrieve an XML ticket about the file with them yet:
http://bitzi.com/bboard/message?message_id=97246&forum_id=4076
I think they desire communication with any developer before integration with their service, to make sure it won't generate too much traffic, or so they don't block the application/IPs if it does - I haven't contacted them myself  though.

- Todd

Jacek -

Congrats on getting the TTH going. I'm also implementing this hashing strategy and would like both of our clients to be compatible out of the gate so as to not introduce a mess. Are you going to have external behavior identical to BCDC++?

Thanks -
Jon Hess

...(although the
pseudo-huffman encoding is about the most stupid I've seen for a long
time...)

Come on, give me i break. I was 18.

Jon

Congrats on getting the TTH going. I'm also implementing this hashing strategy and would like both of our clients to be compatible out of the gate so as to not introduce a mess. Are you going to have external behavior identical to BCDC++?

As for searches, yes, it'll be the same as bcdc++, the getmeta I'll have to
change though because getmeta sends twice as much data as is really needed -
the leaves are enough for most purposes...the current dc++ implementation
(tth and xml) is in the cvs for the interested, it's fairly complete apart
from transferring the leaves...there's a small bug in the compression lib
though, it'll fixed in a few days when I have time.

/J