Known Bugs

From openZIM

Jump to: navigation, search

Please report bugs to http://bugs.openzim.org/!

unspecified 
 
NEW (10)
RESOLVED (26)
37
total37

Current List of Bugs

IDPStatusSeverityVersionProductSummary (37 tasks) ⇒ 
16*
16 I already changed the zimlib to check just the first 2 bytes of the 4 bytes version number. In future versions we can then use the remaining bytes for a minor number. The current zimlib then will ignore, if this minor number is not zero, while zimwriter writes a zero minor number.
P1RESOLVEDenhancementunspecifiedopenZIMVersion & Subversion ZIM format handling 
19*
19

So if the last fixes listed in this following email were commited to the SVN https://intern.openzim.org/pipermail/dev-l/2010-April/000398.html

... the zimlib should be 100% compatible with MS Windows cl.exe compiler and the 2GB bug fixed.
P1RESOLVEDblockerunspecifiedopenZIMWindows+MSVisualStudio: Zimlib is unable to deal with ZIM files larger than 2GB 
1*
1 This is not included in the new ZIM format. I have to test it.
P2NEWenhancementunspecifiedopenZIMHandling of categories inside the ZIM format 
10*
10

I added 2 configure options --with-cluster-cache-size=number and --with-dirent-cache-size to tune default cache sizes. Additionally also the environment variables ZIM_CLUSTERCACHE and ZIM_DIRENTCACHE are read, which when set override the cache sizes.

The cluster cache caches uncompressed clusters. By default zimlib caches the last 16 clusters and with the default cluster size of 1MB this means 16MB of memory usage. For small devices the cache size may be reduced.

The dirent cache is by default 512. Dirents are not that big. They take about 30 bytes, so it makes not that much sense to reduce that value.
P2RESOLVEDenhancementunspecifiedopenZIMcaching on low memory devices 
4*
4

We discussed this at our last developer meeting and decided to add a new namespace for that, which is now documented in the wiki. So there is no need for a change in the zim file format.

A possible enhancement is to add better support for that in zimwriter. And also zimdump should look for the entries when printing information about a zim file.
P3RESOLVEDenhancementunspecifiedopenZIMNew header fields 
11*
11 This is done. The mime-types have to be specified by the ZIM file creator in the DB (or done auto. from the building script).
P3RESOLVEDenhancementunspecifiedopenZIMDynamic mime-types 
26*
26 Implemented
P3RESOLVEDenhancementunspecifiedopenZIMCRC check 
14*
14

Should also be checked if the HTML content do not have any online dependences. For example <img src=http://....

Also in the CSS.
P5ASSIGNEDenhancementunspecifiedopenZIMzim-check 
21*
21

If we don't implement that feature, how do we want to display quickly, allow seeking in a 10MB video or audio file? How do we allow a download window to appear immediately? This is concrete problems I have, as a user, with my ZIM files.

The good news is that not all the articles are compressed... In fact the articles which need this quick random access are mainly not compressed (audio/video/...) in the ZIM itself. So at least, 90% of what I think user need would be reach with such a feature for non compressed content.

For the compressed articles, the problem occurs only with big articles which need a big uncompress time and a big download time... and you already have written about the solution in such cases.
P5NEWenhancementunspecifiedopenZIMZimlib should allow to get/unpack only a part of a content 
27*
27

GNU GPL v2

http://svn.openzim.org/viewvc.cgi/trunk/zimlib/COPYING?view=log http://svn.openzim.org/viewvc.cgi/trunk/zimreader/COPYING?view=log

http://svn.openzim.org/viewvc.cgi/trunk/zimwriter/COPYING?view=log
P5NEWminorunspecifiedopenZIMlicense unclear 
28*
28 there are no downloadable releases of zimlib, zimwriter, ...
P5NEWenhancementunspecifiedopenZIMprovide tarballs 
29*
29

Here the patch:

$ svn diff configure.in Index: configure.in
===================================================================
--- configure.in (revision 335) +++ configure.in (working copy) @@ -10,6 +10,8 @@ 

AC_CONFIG_HEADERS([src/config.h])
AC_CONFIG_SRCDIR([src/file.cpp])

+AM_PROG_CC_C_O + 

AC_PROG_CC
AC_PROG_CXX
AC_PROG_LIBTOOL
P5NEWmajorunspecifiedopenZIMSVN r333 does not always compile 
31*
31

In the trunk (r335) if you call zimwriterdb with -s 1024 (for example) you get the usage(): no zim file with be generated.

$/usr/local/bin/zimwriterdb -s 1024 --db "postgresql:dbname=wiki2b user=kiwix port=5432" titi create zim file from db usage: /usr/local/bin/zimwriterdb [options] output-filename ".zim" is appended to the output filename if not already given.

options: -s <number> specify chunk size for compression in kB (default 1024) --db <dburl> specify a db source (default: postgresql:dbname=zim, tntdb is used here) -Z <articlefile> create a fulltext index for specified article -S <words> search in zim file for articles -I <articlefile> article file for search -X <indexfile> use indexfile as full text search index -D <directory> create zim file from directory content

additional options for full text indexer: -T <file> trivial words file for full text index (a text file with words, which are not indexed) -M <number> memory factor (default 64, smaller factors reduce memory usage but makes indexer slower, try smaller values when you run out of memory) -t <filename> temporary file name (default zimwriter.tmp)

But if you call the same command line but without "-s 1024"... it works.

$ /usr/local/bin/zimwriterdb --db "postgresql:dbname=wiki2b user=kiwix port=5432" titi create zim file from db create directory entries collect articles sort 6 directory entries (aid) remove invalid redirects from 6 directory entries sort 6 directory entries (url) set index translate redirect aid to index 6 directory entries created create title index 6 title index created create clusters 0% ready 10% ready 20% ready 30% ready 40% ready 50% ready 60% ready 70% ready 80% ready 1 clusters created fill header write zimfile ready

zim file ready
P5NEWtrivialunspecifiedopenZIM-s argument seems to be (newly) invalid 
32*
32

make[1]: entrant dans le répertoire « /root/zim/zimlib/src » make all-am make[2]: entrant dans le répertoire « /root/zim/zimlib/src » source='article.cpp' object='article.lo' libtool=yes \ DEPDIR=.deps depmode=none /bin/bash ../depcomp \ /bin/bash ../libtool --tag=CXX --mode=compile g++ -DHAVE_CONFIG_H -I. -I../include -c -o article.lo article.cpp libtool: compile: g++ -DHAVE_CONFIG_H -I. -I../include -c article.cpp -o .libs/article.o ../libtool: line 984: g++ : commande introuvable make[2]: *** [article.lo] Erreur 1 make[2]: quittant le répertoire « /root/zim/zimlib/src » make[1]: *** [all] Erreur 2 make[1]: quittant le répertoire « /root/zim/zimlib/src »

make: *** [all-recursive] Erreur 1
P5NEWtrivialunspecifiedopenZIMconfigure does not check if g++ is installed 
35*
35 Please try with kiwix-serve, this is really easy to use and I can give you a support it (unlike for zimreader).
P5NEWenhancementunspecifiedopenZIMCompletely unclear how to run a .zim file 
36*
36

Few remarks: 1- Distro. dependent... (but could maybe be possible as comment in configure). 2- Lots of templates I think... takes time. 3- If you want only the zimlib, cxxtools and tntdb should not be mandatory (I use it without in kiwix). 4- yes

5- needs to be fixed.
P5NEWenhancementunspecifiedopenZIMSimplify installation of zimlib/zimwriter 
37*
37 We need such tools to provide incremental updates.
P5NEWenhancementunspecifiedopenZIMzimdiff & zimpatch 
2*
2 Fixed
P5RESOLVEDenhancementunspecifiedopenZIMAdd Windows/ReactOS support 
3*
3 There is a uuid-field already in zim files. This is a 16 bytes random field. It is filled by a md5 sum of the current timestamp induding microseconds as returned by the system function gettimeofday.
P5RESOLVEDenhancementunspecifiedopenZIMAdd a unique identifier header field 
5*
5

Seems to have beed fixed by Tommi:

https://intern.openzim.org/pipermail/dev-l/2009-August/000147.html
P5RESOLVEDmajorunspecifiedopenZIMzimreader on arm: miss-aligned dirent header 
6*
6

Seems to have beed fixed by Tommi:

https://intern.openzim.org/pipermail/dev-l/2009-August/000147.html
P5RESOLVEDblockerunspecifiedopenZIMzimreader on arm: cxxtools/m4/asmtype.m4 broken for arm architecture 
7*
7

It works now on GNU/Linux, Symbian and Windows. Compression method can be configured per mime-type.

I close the feature request.
P5RESOLVEDenhancementunspecifiedopenZIMImprove compression 
8*
8 Is that an option to add to the configure an option to avoid the compilation of the test framework like --enable-tests ? Otherwise the dev. is forced to installed cxxtools what is not really great to my opinion.
P5RESOLVEDnormalunspecifiedopenZIMzimlib unable to simply compile without cxxtools 
9*
9

Bug #10 should resolve that. It is not reasonable to create device specific zim files. Instead Bug #10 makes the number of clusters, which are cached when reading configurable. When the default cluster size of 1MB is used, and the cache size is set to 0, this 1MB is needed, which I feel is a reasonable value. I don't think we need to support devices, which have less memory.

1MB is also the value, which is normally used in compression libraries. So it increasing the value does not lead to better compression ratios. On the other side smaller values lead to worse compression ratios.
P5RESOLVEDenhancementunspecifiedopenZIMoptimize cluster size (small devices limitations) 
12*
12

Reason for the problem was a failed data transfer of zim file. Zimlib correctly replied with a exception when trying to read the cluster, which was not in the file.

While investigating the reason for the problem (it was actually no bug), I discovered, thatn zimwriter does not check, if the file was successfully written. This is fixed now.
P5RESOLVEDblockerunspecifiedopenZIMZimReader freezes 
13*
13

zim::File::find returns always the next article when no excact match was found.

There is another method "zim::File::findx(char ns, const QUnicodeString& title, bool collate = false)". This returns a "std::pair<bool, const_iterator>". The first element is true, if a excact match was found and false otherwise. The second element returns the const_iterator. zim::File::find is actually a wrapper around findx, which throws the bool-flag away and returns just the const_iterator.

You may as well use zim::File::getArticle. This uses findx as well but returns a null article (article.good() returns false) if no exact match was found.
P5RESOLVEDnormalunspecifiedopenZIMzim::File::find always returns an article 
15*
15

There is a operator<< for std::ostream, which prints the hex representation of the uuid. If you want to see the hex representation of a uuid do: 

 zim::Uuid myUuid = ...;
 std::cout << myUuid;

If you really need a std::string, use std::ostringstream: 

 std::ostringstream s;
 s << myUuid;
std::string hexUuid = s.str();
P5RESOLVEDenhancementunspecifiedopenZIMzim::File getUuid() should return an HEX encoded MD5 hash 
17*
17 Me too. I just tested with xz 4.999.9beta on opensuse. In lzma.h I can find #include "lzma/check.h", so it should be ok. If you can offer more information please reopen this bug.
P5RESOLVEDnormalunspecifiedopenZIMlzma_check is undefined in lzmastream.h 
18*
18

This is a Kiwix bug and tracked in Kiwix bug tracker.

We hope to be able to fix that soon.
P5RESOLVEDenhancementunspecifiedopenZIMdebian needs an init.d script 
20*
20 Latest version is compling again
P5RESOLVEDmajorunspecifiedopenZIMNot compiling (‘uint16_t’ does not name a type) 
22*
22 The patch adds dependency to boost. This is not really needed. There is a protected constructor and a member in iostream, which allows to initialized the base class without a streambuf and pass the streambuf in the body of the constructor.
P5RESOLVEDblockerunspecifiedopenZIMcrash on opening zim file 
23*
23 ... it's not possible to fill a zimdump specific bug. zimlib seems not to be the right place for that IMO... although currently the zimdump code is delivered with the zimlib code.
P5RESOLVEDenhancementunspecifiedopenZIMWe should have in the bugzilla an item "zimdump" 
24*
24

My examples are not so good, tar has the root directory in the file and unzip has exactly the same problem as zimdump.

What I want to say is if you have a tgz or zip, nobody (except unzip) ask you to create first a directory (default usage) before unpacking the content. tar creates it like it is defined in the file and zip archiver creates it based on the filename. This is true unzip does not work like that at all, and this is really not handfull to use it, because this is common in zip not packaging the root directory itself but only the content.

I will not reopen this bug because this is a minor point, but I think this is not OK to close a feature request without giving at least one argument.

At least please display an error message which is more explicit.
P5RESOLVEDtrivialunspecifiedopenZIMZimdump -d DIRECTORY should be able to create the DIRECTORY if necessary 
25*
25 Thanks for reporting
P5RESOLVEDnormalunspecifiedopenZIMBroken post increment and decrement (patch included) 
30*
30 Bug was confirmed and fixed in SVN. Thx Tommi.
P5RESOLVEDcriticalunspecifiedopenZIMzimdump -d fails on x86_32 with big ZIM files 
33*
33 It works, this bug is invalid
P5RESOLVEDmajorunspecifiedopenZIMfile::findxByTitle() fails 
34*
34

I have fixed this bug of Kiwix, without modifying the zimlib.

So the zimlib is OK.
P5RESOLVEDnormalunspecifiedopenZIMzim files with umlaut in file name do not open 
Retrieved from "http://www.openzim.org/Known_Bugs"
Personal tools
Namespaces
Variants
Actions
Navigation
Toolbox