GNU tar now supports lzma compression as a compression method. Please
consider adding lzma support to the tarfile module (either by using the
external lzma program or by adding a lzma extension to the standard
library).

lzma extension at http://svn.fancycode.com/repos/python/pylzma/trunk/

lzma is used in many tools (7zip, dpkg, rpm), offers faster
decompression than bzip2, slower compression than gzip and bzip2.

As for an lzma module - I would prefer one that isn't LGPL'ed. Instead,
it should link against a system-provide lzma library (which then might
or might not licensed under lpgl). I would probably exclude the lzma
module from Windows, as distributing the lzma sources along with the
Python binaries is too painful.

If we support LZMA, we should do so on all platforms; it kind of
restricts usefulness to only have it on some.  Maybe the LZMA code in
one of the many archival tools in existence that supports it is not LGPL'd?

The LZMA implementation from 7-zip has been released as public domain 
(since version 4.62 / Nov 2008) in the LZMA SDK: http://www.7-zip.org/
sdk.html
So, there shouldn't be a license issue for Windows. I am not sure if 
there are already system-provided LZMA libraries on Linux at this time.

> The LZMA implementation from 7-zip has been released as public domain 
> (since version 4.62 / Nov 2008) in the LZMA SDK: http://www.7-zip.org/
> sdk.html

That's good news. Now, if somebody could contribute a Python wrapper for
these...

> So, there shouldn't be a license issue for Windows. I am not sure if 
> there are already system-provided LZMA libraries on Linux at this time.

There are. The Linux version apparently originates from the same
sources, so they might be API compatible. However, I wouldn't mind
if we extracted the entire lzma library from 7zip, and put it into
the source distribution.

I'm the author of the pyliblzma module, and if desired, I'd be happy to help out adapting pyliblzma for inclusion with python.
Most of it's code is based on bz2module.c, so it shouldn't be very far away from being good 'nuff.
What I see as required is:
* clean out use of C99 types etc.
* clean up the LZMAOptions class (this is the biggest difference from the bz2 module, as the filter supports a wide range of various options, everything related such as parsing, api documentation etc. was placed in it's own class, I've yet to receive any feedback on this decission or find any remote equivalents out there to draw inspiration from;)
* While most of the liblzma API has been implemented, support for multiple/alternate filters still remains to be implemented. When done it will also cause some breakage with the current pyliblzma API.

I plan on doing these things sooner or later anyways, it's pretty much just a matter of motivation and priorities standing in the way, actual interest from others would certainly have a positive effect on this. ;)

For other alternatives to the LGPL liblzma, you really don't have any, keep in mind that LZMA is "merely" the algorithm, while xz (and LZMA_alone, used for '.lzma', now obsolete, but still supported) are the actual format you want support for. The LZMA SDK does not provide any compatibility for this.

ps: pylzma uses the LZMA SDK, which is not what you want.
pyliblzma (not the same module;) OTOH uses liblzma, which is the library used by xz/lzma utils

You'll find it available at http://launchpad.net/pyliblzma

> For other alternatives to the LGPL liblzma, you really don't have
> any,

If that's really the case (which I don't believe it is), then this
project stops right here. If the underlying library is LGPL, it would
require us to distribute its sources along with the Windows binaries,
which I'm not willing to do.

The XZ Utils website ( http://tukaani.org/xz/ ) states the following:

"The most interesting parts of XZ Utils (e.g. liblzma) are in the public domain. You can do whatever you want with the public domain parts. 

Some parts of XZ Utils (e.g. build system and some utilities) are under different free software licenses such as GNU LGPLv2.1, GNU GPLv2, or GNU GPLv3."

So, liblzma is not the problem. But the license of PylibLZMA is LGPL3.

> If the underlying library is LGPL, it would
> require us to distribute its sources along with the Windows binaries,
> which I'm not willing to do.

Martin, this is wrong, you don't have to bundle the source *in* the object code package. Making it available on some HTTP or FTP site is sufficient.
(actually, if we don't modify the library source, we can even point at the original download site)

Per, on 2010-03-17, I asked you via email:

"I was looking at

  http://bugs.python.org/issue5689
  http://bugs.python.org/issue6715

and Martin's comments about the licensing of the bindings; is there a special reason for the lgpl3 license of the bindings, given that both python and xz-utils are not gpl'ed?"

Does pyliblzma need to be licensed under the lgpl3?

Am 26.05.2010 10:51, schrieb Antoine Pitrou:
>
> Antoine Pitrou<pitrou@free.fr>  added the comment:
>
>> If the underlying library is LGPL, it would
>> require us to distribute its sources along with the Windows binaries,
>> which I'm not willing to do.
>
> Martin, this is wrong, you don't have to bundle the source *in* the object code package.

That's why I said "along". I'm still not willing to do that: making the 
source available is still inconvenient. More importantly, anybody 
redistributing Python binaries would have to comply also (e.g. on 
CD-ROMs or py2exe binaries); this is a burden I don't want to impose
on our users. Fortunately, we don't have to, as the LZMA compression 
itself is in the public domain. For the Python wrapper, I hope that 
somebody contributes such a module under a PSF contributor agreement.
If nobody else does, I may write one from scratch one day.

if you're already looking at issue6715, then I don't get why you're asking.. ;)

quoting from msg106433:
"For my code, feel free to use your own/any other license you'd like or even public domain (if the license of bz2module.c that much of it's derived from permits of course)!"

The reason why I picked LGPLv3 in the past was simply just because liblzma at the time was licensed under it, so I just picked the same for simplicity.
I've actually already dual-licensed it under the python license in addition on the project page though, but I just forgot updating the module's metadata as well before I released 0.5.3 last month..

Martin: For LGPL (or even GPL for that matter, disregarding linking restrictions) libraries you don't have to distribute the sources of those libraries at all (they're already made available by others, so that would be quite overly redundant, uh?;). LGPL actually doesn't even care at all about the license of your software as long as you only dynamically link against it.

I don't really get what the issue would be even if liblzma were still LGPL, it doesn't prohibit you from distributing a dynamically linked library along with python either if necessary (which of course would be of convenience on win32..)..

tsktsk, discussions about python module for xz compression should anyways be kept at issue6715 as this one is about the tarfile module ;p

> tsktsk, discussions about python module for xz compression should
> anyways be kept at issue6715 as this one is about the tarfile module
> ;p

Ok, following up there.