[Refdb-devel] Docs failing to build

[David Nebauer]

Hi Markus,

All of a sudden refdb docs is failing to build. The fop command
'JAVA_OPTS=-Xmx256M fop -fo refdb-manual.fo -pdf refdb-manual.pdf' is
failing:
-----------------------------------------------------------------------------
CFLAGS="-Wall -g -O2" ./configure \
--host=i486-linux-gnu \
--build=i486-linux-gnu \
--prefix=/usr \
--mandir=\${prefix}/share/man \
--infodir=\${prefix}/share/info \
--sysconfdir=/etc \
--sharedstatedir=/var \
--localstatedir=/var \
--with-classpath-root=/usr/share/java \
--with-xml-declaration=/usr/share/xml/declaration/xml.dcl \
--with-trang-jar=/usr/share/java/trang.jar \
--disable-clients \
--disable-server \
--disable-manpages
checking build system type... i486-pc-linux-gnu

<snip.../>

config.status: executing depfiles commands

Configuration summary:
will build docs

What you should do next:
- run "make" to build RefDB.
- run "make install" as root to install everything.
- create the main database and the configuration files. Either run the
refdb-init shell script as root, or peruse the handbook if you
prefer a manual installation.

dh_testdir
# Add here commands to compile the package.
/usr/bin/make
make[1]: Entering directory
`/home/david/data/computing/projects/refdb/packages/refdb/svn/doc/pkg/debianise/source/refdb-0.0-svn-20060822'
Making all in doc
make[2]: Entering directory
`/home/david/data/computing/projects/refdb/packages/refdb/svn/doc/pkg/debianise/source/refdb-0.0-svn-20060822/doc'
SGML_CATALOG_FILES="" dtdparse --title "citationlistx XML DTD" --output
citationlistx.xml --declaration ../declarations/xml.dcl --system-id
"http://refdb.sourceforge.net/dtd/citationlistx.dtd"
../dtd/citationlistx.dtd

<snip.../>

../scripts/statgen.pl docbook > refdb-manual-statustable.xml
xsltproc -output include/fotitlepages.xsl
http://docbook.sourceforge.net/release/xsl/current/template/titlepage.xsl
include/fotitlepage.templates.xml
Creating PDF manual...
xsltproc -o refdb-manual.fo --nonet --xinclude
../doc/include/manual-fo.xsl ../doc/refdb-manual.xml
Making portrait pages on USletter paper (8.5inx11in)
affiliation encountered in author, but no template matches.
varlistentry encountered in variablelist, but no template matches.

<snip.../>

varlistentry encountered in variablelist, but no template matches.
JAVA_OPTS=-Xmx256M fop -fo refdb-manual.fo -pdf refdb-manual.pdf
[INFO] Using org.apache.xerces.parsers.SAXParser as SAX2 Parser
[INFO] FOP 0.20.5
[INFO] Using org.apache.xerces.parsers.SAXParser as SAX2 Parser
[INFO] building formatting object tree
[INFO] setting up fonts
[ERROR] property - "background-position-horizontal" is not implemented yet.
[ERROR] property - "background-position-vertical" is not implemented yet.
Error creating background image: Error while recovering Image
Informations (Loading Image...) :
Connection timed out
[ERROR] property - "background-position-horizontal" is not implemented yet.
[ERROR] property - "background-position-vertical" is not implemented yet.
[INFO] JAI support was not installed (read: not present at build time).
Trying to use Jimi instead
[ERROR] property - "background-position-horizontal" is not implemented yet.

<snip.../>

[ERROR] property - "background-position-vertical" is not implemented yet.
[INFO] [1]
[INFO] [2]
[INFO] [3]
[INFO] [4]
[INFO] [5]
[INFO] [6]
[INFO] [7]
[INFO] [8]
[INFO] [9]
[ERROR] null
make[2]: *** [refdb-manual.pdf] Error 2
-----------------------------------------------------------------------------

Since nothing in the document source has changed I presume something in
the tools has changed in one of my system updates.

Here are some tool versions:
fop: 1:0.20.5-8
libxerces2-java: 2.6.2-4
libxalan2-java: 2.6.0-6
libavalon-framework-java: 4.2.0-3
libbatik-java: 1.6-2

The classpath for fop is set in ~/.refdbxmlrc:
/usr/share/java/fop.jar:/usr/share/java/batik.jar:/usr/share/java/xercesImpl.jar:\
/usr/share/java/avalon-framework.jar:/usr/share/java/fop-hyph.jar

I've no idea what's going on here. Can anyone else replicate this bug?

Regards,
David.

[Stéphane Téletchéa]

Post by David Nebauer
Hi Markus,
All of a sudden refdb docs is failing to build. The fop command
'JAVA_OPTS=-Xmx256M fop -fo refdb-manual.fo -pdf refdb-manual.pdf' is
-----------------------------------------------------------------------------
CFLAGS="-Wall -g -O2" ./configure \
--host=i486-linux-gnu \
--build=i486-linux-gnu \
--prefix=/usr \
--mandir=\${prefix}/share/man \
--infodir=\${prefix}/share/info \
--sysconfdir=/etc \
--sharedstatedir=/var \
--localstatedir=/var \
--with-classpath-root=/usr/share/java \
--with-xml-declaration=/usr/share/xml/declaration/xml.dcl \
--with-trang-jar=/usr/share/java/trang.jar \
--disable-clients \
--disable-server \
--disable-manpages
checking build system type... i486-pc-linux-gnu
<snip.../>
config.status: executing depfiles commands
will build docs
- run "make" to build RefDB.
- run "make install" as root to install everything.
- create the main database and the configuration files. Either run the
refdb-init shell script as root, or peruse the handbook if you
prefer a manual installation.
dh_testdir
# Add here commands to compile the package.
/usr/bin/make
make[1]: Entering directory
`/home/david/data/computing/projects/refdb/packages/refdb/svn/doc/pkg/debianise/source/refdb-0.0-svn-20060822'
Making all in doc
make[2]: Entering directory
`/home/david/data/computing/projects/refdb/packages/refdb/svn/doc/pkg/debianise/source/refdb-0.0-svn-20060822/doc'
SGML_CATALOG_FILES="" dtdparse --title "citationlistx XML DTD" --output
citationlistx.xml --declaration ../declarations/xml.dcl --system-id
"http://refdb.sourceforge.net/dtd/citationlistx.dtd"
../dtd/citationlistx.dtd
<snip.../>
../scripts/statgen.pl docbook > refdb-manual-statustable.xml
xsltproc -output include/fotitlepages.xsl
http://docbook.sourceforge.net/release/xsl/current/template/titlepage.xsl
include/fotitlepage.templates.xml
Creating PDF manual...
xsltproc -o refdb-manual.fo --nonet --xinclude
../doc/include/manual-fo.xsl ../doc/refdb-manual.xml
Making portrait pages on USletter paper (8.5inx11in)
affiliation encountered in author, but no template matches.
varlistentry encountered in variablelist, but no template matches.
<snip.../>
varlistentry encountered in variablelist, but no template matches.
JAVA_OPTS=-Xmx256M fop -fo refdb-manual.fo -pdf refdb-manual.pdf
[INFO] Using org.apache.xerces.parsers.SAXParser as SAX2 Parser
[INFO] FOP 0.20.5
[INFO] Using org.apache.xerces.parsers.SAXParser as SAX2 Parser
[INFO] building formatting object tree
[INFO] setting up fonts
[ERROR] property - "background-position-horizontal" is not implemented yet.
[ERROR] property - "background-position-vertical" is not implemented yet.
Error creating background image: Error while recovering Image
Connection timed out
[ERROR] property - "background-position-horizontal" is not implemented yet.
[ERROR] property - "background-position-vertical" is not implemented yet.
[INFO] JAI support was not installed (read: not present at build time).
Trying to use Jimi instead
[ERROR] property - "background-position-horizontal" is not implemented yet.
<snip.../>
[ERROR] property - "background-position-vertical" is not implemented yet.
[INFO] [1]
[INFO] [2]
[INFO] [3]
[INFO] [4]
[INFO] [5]
[INFO] [6]
[INFO] [7]
[INFO] [8]
[INFO] [9]
[ERROR] null
make[2]: *** [refdb-manual.pdf] Error 2
-----------------------------------------------------------------------------
Since nothing in the document source has changed I presume something in
the tools has changed in one of my system updates.
fop: 1:0.20.5-8
libxerces2-java: 2.6.2-4
libxalan2-java: 2.6.0-6
libavalon-framework-java: 4.2.0-3
libbatik-java: 1.6-2
/usr/share/java/fop.jar:/usr/share/java/batik.jar:/usr/share/java/xercesImpl.jar:\
/usr/share/java/avalon-framework.jar:/usr/share/java/fop-hyph.jar
I've no idea what's going on here. Can anyone else replicate this bug?
Regards,
David.

As Markus told me before, it is not mandatory to rebuild the docs since
the source tarball already provides them.

I've thus bypassed the complete building by not providing the
buildrequires (programs needed to build the docs in the rpm process), by
commenting them.

That does not help you much directly but may simplify the problem ;-)

Stéphane

[Markus Hoenicka]

Hi David,

Post by David Nebauer
Hi Markus,
All of a sudden refdb docs is failing to build. The fop command
'JAVA_OPTS=-Xmx256M fop -fo refdb-manual.fo -pdf refdb-manual.pdf' is

I didn't have problems lately, but I started to upgrade my FreeBSD box to the
latest version on the weekend. I'm almost done, and I'll have yet to see
whether I'll get similar problems with the updated tools. I'll let you know.

Post by David Nebauer
[INFO] [1]
[INFO] [2]
[INFO] [3]
[INFO] [4]
[INFO] [5]
[INFO] [6]
[INFO] [7]
[INFO] [8]
[INFO] [9]
[ERROR] null

Interestingly, this is the end of the TOC/LOT/whatever section, but I don't see
why this should fail. Just to make sure the recent changes to the doc (included
refdbmanualfig5.svg, replaced some man pages) are not the culprit, you might try
and checkout an older SVN version and try again. If that also fails, it must be
the tools.

regards,
Markus

[David Nebauer]

Hi Markus,

Post by Markus Hoenicka

Post by David Nebauer
All of a sudden refdb docs is failing to build. The fop command
'JAVA_OPTS=-Xmx256M fop -fo refdb-manual.fo -pdf refdb-manual.pdf' is

Interestingly, this is the end of the TOC/LOT/whatever section, but I don't see
why this should fail. Just to make sure the recent changes to the doc (included
refdbmanualfig5.svg, replaced some man pages) are not the culprit, you might try
and checkout an older SVN version and try again. If that also fails, it must be
the tools.

Build fails the same way with revision 120.

Here is the same failure point but with fop debugging output enabled.
Perhaps someone out there can help dissect the java error:

----------------------------------------------------------------------------------
JAVA_OPTS=-Xmx256M fop -d -fo refdb-manual.fo -pdf refdb-manual.pdf
[DEBUG] Input mode:
[DEBUG] FO
[DEBUG] fo input file: refdb-manual.fo
[DEBUG] Output mode:
[DEBUG] pdf
[DEBUG] output file: refdb-manual.pdf
[DEBUG] OPTIONS
[DEBUG] no user configuration file is used [default]
[DEBUG] debug mode on
[DEBUG] dump configuration
[DEBUG] quiet mode on
[INFO] Using org.apache.xerces.parsers.SAXParser as SAX2 Parser
[INFO] base directory:
file:/home/david/data/computing/projects/refdb/packages/refdb/svn/doc/pkg/debianise/source/refdb-0.0-svn-20060823/doc/
[INFO] FOP 0.20.5
[INFO] Using org.apache.xerces.parsers.SAXParser as SAX2 Parser
[INFO] building formatting object tree
[INFO] setting up fonts
[ERROR] property - "background-position-horizontal" is not implemented yet.

<snip.../>

[ERROR] property - "background-position-vertical" is not implemented yet.
[INFO] [1]
[INFO] [2]
[INFO] [3]
[DEBUG] Last page-sequence produced 3 pages.
[INFO] [4]
[INFO] [5]
[INFO] [6]
[INFO] [7]
[DEBUG] Last page-sequence produced 4 pages.
[INFO] [8]
[DEBUG] Last page-sequence produced 1 pages.
[INFO] [9]
[DEBUG] Last page-sequence produced 1 pages.
[ERROR] null
org.apache.fop.apps.FOPException
at
org.apache.fop.apps.CommandLineStarter.run(CommandLineStarter.java:111)
at org.apache.fop.apps.Fop.main(Fop.java:62)

---------

java.lang.NullPointerException
at
org.apache.fop.fo.PropertyManager.getTextDecoration(PropertyManager.java:365)
at org.apache.fop.fo.FObjMixed.<init>(FObjMixed.java:72)
at org.apache.fop.fo.flow.Block.<init>(Block.java:115)
at org.apache.fop.fo.flow.Block$Maker.make(Block.java:79)
at org.apache.fop.fo.FOTreeBuilder.startElement(FOTreeBuilder.java:352)
at org.apache.xerces.parsers.AbstractSAXParser.startElement(Unknown
Source)
at
org.apache.xerces.impl.XMLNSDocumentScannerImpl.scanStartElement(Unknown
Source)
at
org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown
Source)
at
org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown
Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
at org.apache.fop.apps.Driver.render(Driver.java:498)
at
org.apache.fop.apps.CommandLineStarter.run(CommandLineStarter.java:106)
at org.apache.fop.apps.Fop.main(Fop.java:62)

---------

java.lang.NullPointerException
at
org.apache.fop.fo.PropertyManager.getTextDecoration(PropertyManager.java:365)
at org.apache.fop.fo.FObjMixed.<init>(FObjMixed.java:72)
at org.apache.fop.fo.flow.Block.<init>(Block.java:115)
at org.apache.fop.fo.flow.Block$Maker.make(Block.java:79)
at org.apache.fop.fo.FOTreeBuilder.startElement(FOTreeBuilder.java:352)
at org.apache.xerces.parsers.AbstractSAXParser.startElement(Unknown
Source)
at
org.apache.xerces.impl.XMLNSDocumentScannerImpl.scanStartElement(Unknown
Source)
at
org.apache.xerces.impl.XMLDocumentFragmentScannerImpl$FragmentContentDispatcher.dispatch(Unknown
Source)
at
org.apache.xerces.impl.XMLDocumentFragmentScannerImpl.scanDocument(Unknown
Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XML11Configuration.parse(Unknown Source)
at org.apache.xerces.parsers.XMLParser.parse(Unknown Source)
at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
at org.apache.fop.apps.Driver.render(Driver.java:498)
at
org.apache.fop.apps.CommandLineStarter.run(CommandLineStarter.java:106)
at org.apache.fop.apps.Fop.main(Fop.java:62)
make[2]: *** [refdb-manual.pdf] Error 2
----------------------------------------------------------------------------------

Regards,
David.

[Markus Hoenicka]

A few other references suggest this error is caused by invalid
docbook xml. I've attempted to validate the produced refdb-manual-xml
file with xmllint but am unable to do so successfully. I'll send
you a copy off-list to see if you can validate it.

I still believe it is a tool problem. I've more or less finished upgrading my
box to FreeBSD 6.1 with the latest tools and stylesheets available on this
platform. I can still create the PDF manual without any problems. I can also
validate the manual with the following command (all in one line):

xmllint --noout --xinclude --relaxng
/usr/local/share/emacs/21.3/site-lisp/nxml/schema/docbook.rng refdb-manual.xml
2>&1

xmllint only complains about an IDREF with a missing linkend. This is caused by
the refdb-init(8) manpage referencing the RefDB(7) manpage which is not
included in the manual (as it would duplicate some of the general information).
After removing this link (I'll check this in shortly) xmllint reports
"refdb-manual.xml validates". You can remove this link to see whether it helps,
but I guess you won't fare better.

I think the next step is to contact either the docbook-apps list to see whether
other Debian users ran into problems, or find an appropriate Debian list.
Before doing so, did you run any tests with other DocBook documents of a
comparable size?

regards,
Markus

[David Nebauer]

Hi Markus,

Post by Markus Hoenicka
Before doing so, did you run any tests with other DocBook documents of a
comparable size?

This is turning out to be a most puzzling problem. I've tested a number
of documents and refdb-manual is the only document failing so far. The
refdb-elisp manual outputs PDF just fine.

Regards,
David.

[Markus Hoenicka]

Hi David,

Post by David Nebauer
This is turning out to be a most puzzling problem. I've tested a number
of documents and refdb-manual is the only document failing so far. The
refdb-elisp manual outputs PDF just fine.

Drives me nuts. Is the reason of this problem too obvious for us to
see? We have to sort this out, somehow.

I'd suggest to do the following:

- please post the refdb-manual.fo that your tools create on some http
or ftp space (you can abuse the RefDB homepage if you don't have web
space of your own. Use e.g. the htdocs/stuff folder). I'd like to
diff it against what my system creates.

- please post the package versions of the related tools (libxml2,
libxslt2, fop, docbook-stylesheets and so on). I'd like to have
Michael look at this, as he seems to be running Debian as well.

- please try whether the transformation problem is caused by the RefDB
driver file. Transform refdb-manual.xml manually with the stock
DocBook stylesheets.

- please try whether size matters :-) Remove some or all of the
xincludes in refdb-manual.xml and reprocess the document. You'll get
warnings about missing linkends but this shouldn't matter.

- please try whether your Java runtime needs more memory. Have a look
at the fop command in doc/Makefile.am and give it some more memory,
like in:

JAVA_OPTS=-Xmx512M fop -fo refdb-manual.fo -pdf refdb-manual.pdf

thanks,
Markus

[David Nebauer]

Hi Markus,

Post by David Nebauer
This is turning out to be a most puzzling problem.

Post the package versions of the related tools (libxml2,
libxslt2, fop, docbook-stylesheets and so on). I'd like to have
Michael look at this, as he seems to be running Debian as well.

libavalon-framework-java: 4.2.0-3
libbsf-java: 1:2.3.0+cvs20050308a-1
libbatik-java: 1.6-2
liblogkit-java: 1.2.2-8
libxalan2-java: 2.6.0-6
libxerces2-java: 2.6.2-4
libxml2: 2.6.26.dfsg-3
libxslt1.1: 1.1.17-3
fop: 1:0.20.5-8
docbook-xsl: 1.70.1.dfsg.1-0.2
xml-core: 0.09-0.1

I'm running very recent Debian/testing with all packages up-to-date.
Let me know if you want any other package versions.

--------------------------------------------------------------------------------------------

I have discovered the cause of the problem... but am no closer to
understanding it. It has to do with the /doc directory's xsltproc line.

The makefile ends up executing the following command (excepting the
backslash, of course):

xsltproc -o refdb-manual.fo --nonet --xinclude \
../doc/include/manual-fo.xsl ../doc/refdb-manual.xml

The FO file thus created will fail on conversion to PDF with fop. If,
in the xsltproc command, you strip out both instances of '../doc/' you
end up with the following command:

xsltproc -o refdb-manual.fo --nonet --xinclude \
include/manual-fo.xsl refdb-manual.xml

The FO file thus created is different to the one created using the
previous command. This FO file will be converted successfully by fop to
PDF.

The FO files thus generated are different. I do not yet understand why
xsltproc generates different output from the same input and essentially
similar commands.

I have created a tarzipped archive containing both versions of FO output
-- one which will successfully convert to PDF (created by a command with
'../doc/' stripped out) and one which fails to convert to PDF (created
by a command retaining '../doc/'). This archive can be accessed at
<www.nebauer.org/fo-files.tar.gz>.

Regards,
David.

[Markus Hoenicka]

Hi David,

Post by David Nebauer
libavalon-framework-java: 4.2.0-3
libbsf-java: 1:2.3.0+cvs20050308a-1
libbatik-java: 1.6-2
liblogkit-java: 1.2.2-8
libxalan2-java: 2.6.0-6
libxerces2-java: 2.6.2-4
libxml2: 2.6.26.dfsg-3
libxslt1.1: 1.1.17-3
fop: 1:0.20.5-8
docbook-xsl: 1.70.1.dfsg.1-0.2
xml-core: 0.09-0.1
I'm running very recent Debian/testing with all packages up-to-date.
Let me know if you want any other package versions.

What is the version of your xsltproc package? I remember that Debian packages
the binaries separately from the libraries. FWIW the FreeBSD port is also built
from the libxslt 1.1.17 sources.

Post by David Nebauer
I have discovered the cause of the problem... but am no closer to
understanding it. It has to do with the /doc directory's xsltproc line.
The makefile ends up executing the following command (excepting the
xsltproc -o refdb-manual.fo --nonet --xinclude \
../doc/include/manual-fo.xsl ../doc/refdb-manual.xml
The FO file thus created will fail on conversion to PDF with fop. If,
in the xsltproc command, you strip out both instances of '../doc/' you
xsltproc -o refdb-manual.fo --nonet --xinclude \
include/manual-fo.xsl refdb-manual.xml
The FO file thus created is different to the one created using the
previous command. This FO file will be converted successfully by fop to
PDF.

This is really interesting. The "../doc" path component results from the
expansion of $(top_srcdir) which is supposed to allow building RefDB outside of
the source directory. I haven't tried lately whether this works for the docs (it
used to work for the binaries). I also don't know how important this feature is
for end users and for package builders. It is quite possible that xsltproc is
not supposed to be used in this context. But then I don't understand why things
work ok on FreeBSD. As a short-term fix I could disable this feature at least
for the doc subdirectory.

OTOH if this is an xsltproc problem, you should create a simple testcase and
post it to the XML mailing list (***@gnome.org). Do you have any other xslt
processor installed to run a quick test?

Post by David Nebauer
The FO files thus generated are different. I do not yet understand why
xsltproc generates different output from the same input and essentially
similar commands.

One reason may be the internal path handling of xslt processors. The RefDB
manual is processed by a driver file for the DocBook stylesheets. The path is
passed as an argument and should arrive ok. But the driver file xsl:includes
another stylesheet (fotitlepages.xsl) with the titlepage customizations. This
path is specified as a relative path in manual-fo.xsl. I actually suspect that
xsltproc does not properly include this file as the failed output that you
posted contains the code to render the revision history, something that I
switched off in my titlepage customization. A comparison with other xslt
processors could show whether xsltproc is broken or whether I expected too much
from it.

regards,
Markus

[David Nebauer]

Hi Markus,

Post by Markus Hoenicka

Post by David Nebauer
libxslt1.1: 1.1.17-3

What is the version of your xsltproc package? I remember that Debian packages
the binaries separately from the libraries. FWIW the FreeBSD port is also built
from the libxslt 1.1.17 sources.

xsltproc: 1.1.17-3

Post by Markus Hoenicka
I have discovered the cause of the problem... but am no closer to
understanding it. It has to do with the /doc directory's xsltproc line.
The makefile ends up executing the following command (excepting the
xsltproc -o refdb-manual.fo --nonet --xinclude \
../doc/include/manual-fo.xsl ../doc/refdb-manual.xml

More specifically, the error is caused by the inclusion of '../doc/' in
specifying the stylesheet. It makes no difference whether included or
not in the source xml file name.

Post by Markus Hoenicka
OTOH if this is an xsltproc problem, you should create a simple testcase and
processor installed to run a quick test?

xmlstarlet uses the same underlying library ('libxslt1.1') but happily
creates a viable FO file even when the offending '../doc/' is included.
The following command works:

xmlstarlet tr --xinclude ../doc/include/manual-fo.xsl \
../doc/refdb-manual.xml > refdb-manual.fo

Post by Markus Hoenicka
A comparison with other xslt
processors could show whether xsltproc is broken or whether I expected too much
from it.

It would appear to be an xsltproc-specific bug.

Regards,
David.

[Markus Hoenicka]

Hi,

Post by David Nebauer
More specifically, the error is caused by the inclusion of '../doc/' in
specifying the stylesheet. It makes no difference whether included or
not in the source xml file name.

That is what I suspected. To the best of my knowledge the shell does not
canonicalize the path, i.e. the '../doc/<path>' is passed literally to
xsltproc. This works ok if the strings are used for regular "open()" or
"fopen()" calls. Therefore xsltproc finds both the XML file and the XSLT
stylesheet. However, to resolve the "xsl:include" in the stylesheet, an xslt
processor probably applies a different logic (the path could be an URL too). I
suspect that xsltproc expects a canonicalized path at that point.

Post by David Nebauer
xmlstarlet uses the same underlying library ('libxslt1.1') but happily
creates a viable FO file even when the offending '../doc/' is included.
xmlstarlet tr --xinclude ../doc/include/manual-fo.xsl \
../doc/refdb-manual.xml > refdb-manual.fo

This is even more confusing result. The "xsl:include" handling is certainly a
part of the library, not a part of the xsltproc application itself.

Post by David Nebauer
It would appear to be an xsltproc-specific bug.

More specifically, an xsltproc-on-Debian-specific bug. I've compared the patches
that Debian and FreeBSD apply to the sources in their package and port,
respectively. I don't see anything that would explain why things work ok on
FreeBSD but not on Debian.

One last chance: Can you verify with xsltproc --version that you invoke the
right binary and the right libraries?

regards,
Markus

[David Nebauer]

Hi Markus,

Post by Markus Hoenicka
More specifically, an xsltproc-on-Debian-specific bug. I've compared the patches
that Debian and FreeBSD apply to the sources in their package and port,
respectively. I don't see anything that would explain why things work ok on
FreeBSD but not on Debian.
One last chance: Can you verify with xsltproc --version that you invoke the
right binary and the right libraries?

----------------------------------------------------------------------------
$ xsltproc --version
Using libxml 20626, libxslt 10117 and libexslt 813
xsltproc was compiled against libxml 20626, libxslt 10117 and libexslt 813
libxslt 10117 was compiled against libxml 20626
libexslt 813 was compiled against libxml 20626
$
----------------------------------------------------------------------------

Hope that helps.

Regards,
David.

[David Nebauer]

Hi Markus,

Post by David Nebauer
More specifically, the error is caused by the inclusion of '../doc/' in
specifying the stylesheet. It makes no difference whether included or
not in the source xml file name.

As so often happens, this problem has self-corrected after another
system update. I have no idea what the offending package was, nor the
underlying cause.

For what it's worth, here is the current situation regarding
stylesheets. You will see I experimented with putting all stylesheets
in the 'doc' directory rather than the 'include' subdirectory. This
makes a difference.

TOPSRCDIR? USE SUBDIR? RESULT
---------- ----------- ------
Yes Yes --> SUCCEEDED [../doc/include/manual-fo.xsl]
Yes No --> FAILED [../doc/manual-fo.xsl]
No Yes --> FAILED [include/manual-fo.xsl]
No No --> SUCCEEDED [manual-fo.xsl]

The results are perplexing. Using '../doc/' as part of the stylesheet
filepath works when stylesheets are in a subdirectory but not when they
are in the current directory. If you remove the '../doc/' construct the
situation reverses: success when stylesheets are in the current
directory but failure when they are in the subdirectory. Passing
strange indeed.

In any event, the command used in the refdb makefile happens to be of a
form now recognised by xsltproc -- so we are in the clear for now. If,
however, a problem building the manual should arise in the future it
will be worth remembering the fragility of xsltproc's command line.

Regards,
David.

[Markus Hoenicka]

Hi David,

thanks for analyzing this weird problem. Will you follow up on this on
the XML mailing list? As I can't reproduce this problem here, I don't
want to report it myself as I couldn't run any suggested debug
tests. I'd be happy though to help writing a description of the
problem. In any case, DocBook and xsltproc are a likely
combination for building the documentation of an autotools-based
software. It is always annoying to see an installation fail because
the docs won't build properly.

regards,
Markus

Post by David Nebauer
Hi Markus,

Post by David Nebauer
More specifically, the error is caused by the inclusion of '../doc/' in
specifying the stylesheet. It makes no difference whether included or
not in the source xml file name.

As so often happens, this problem has self-corrected after another
system update. I have no idea what the offending package was, nor the
underlying cause.

[David Nebauer]

Hi Markus,

Post by Markus Hoenicka
thanks for analyzing this weird problem. Will you follow up on this on
the XML mailing list?

I started putting together a test case but got lost trying to reduce the
complexity of the xsl files. I found myself drowning in error messages
and gave up. It's working for refdb now and that's the main thing.

Regards,
David.

[David Nebauer]

Hi Markus,

Post by David Nebauer
[ERROR] null
org.apache.fop.apps.FOPException
at
org.apache.fop.apps.CommandLineStarter.run(CommandLineStarter.java:111)
at org.apache.fop.apps.Fop.main(Fop.java:62)
---------
java.lang.NullPointerException
at
org.apache.fop.fo.PropertyManager.getTextDecoration(PropertyManager.java:365)
at org.apache.fop.fo.FObjMixed.<init>(FObjMixed.java:72)
at org.apache.fop.fo.flow.Block.<init>(Block.java:115)
at org.apache.fop.fo.flow.Block$Maker.make(Block.java:79)
at org.apache.fop.fo.FOTreeBuilder.startElement(FOTreeBuilder.java:352)
at org.apache.xerces.parsers.AbstractSAXParser.startElement(Unknown
Source)

I've found reports of a similar error here:
<http://mail-archives.apache.org/mod_mbox/xmlgraphics-fop-users/200302.mbox/%***@yahoo.de%3e>.
In that instance the advice given was:

This means you have content outside a flow, which is
illegal. This should not happen. Upgrade your DocBook XSL,
be sure your DocBook docs validate, and complain on the
DocBook list if this persists.

A few other references suggest this error is caused by invalid
docbook xml. I've attempted to validate the produced refdb-manual-xml
file with xmllint but am unable to do so successfully. I'll send
you a copy off-list to see if you can validate it.

Regards,
David.

[Markus Hoenicka]

Hi David,

A few other references suggest this error is caused by invalid
docbook xml. I've attempted to validate the produced refdb-manual-xml
file with xmllint but am unable to do so successfully. I'll send
you a copy off-list to see if you can validate it.

Please fill me in how the file was "produced". It looks like the
refdb-manual.xml master file in the svn repository. As this file uses a couple
of xinclude commands to import the contents of the other xml files in the same
directory, you cannot expect the file to be valid by itself. I ran it through
xmllint anyway and got the expected results: several warnings about missing
external entities (the xincluded files), and a few errors about missing
linkends, also caused by the missing xincluded files.

If this file was "produced" in order to create a flattened version of the manual
to get rid of the xincludes, then I reckon your tool did not succeed.

However, if you accidentally sent the wrong file, I'll be happy to have another
look.

regards,
Markus

[Michael(tm) Smith]

Post by David Nebauer

Post by David Nebauer
[ERROR] null
org.apache.fop.apps.FOPException
at
org.apache.fop.apps.CommandLineStarter.run(CommandLineStarter.java:111)
at org.apache.fop.apps.Fop.main(Fop.java:62)

I've not been able to reproduce this in my environment with FOP
0.20.5. FOP generates the PDF file successfully.

Post by David Nebauer
This means you have content outside a flow, which is
illegal. This should not happen. Upgrade your DocBook XSL,
be sure your DocBook docs validate, and complain on the
DocBook list if this persists.
A few other references suggest this error is caused by invalid
docbook xml. I've attempted to validate the produced refdb-manual-xml
file with xmllint but am unable to do so successfully.

The patch below adds an internal subset to refdb-manual.xml to
make it possible to validate it with DTD-based validation.

--Mike

Index: refdb-manual.xml
===================================================================
--- refdb-manual.xml (revision 135)
+++ refdb-manual.xml (working copy)
@@ -1,4 +1,20 @@
<?xml version="1.0"?>
+<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.4//EN"
+"http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd" [
+
+<!ELEMENT xi:include (xi:fallback?) >
+<!ATTLIST xi:include
+ xmlns:xi CDATA #FIXED "http://www.w3.org/2001/XInclude"
+ href CDATA #REQUIRED
+ parse (xml|text) "xml"
+ encoding CDATA #IMPLIED >
+
+<!ELEMENT xi:fallback ANY>
+<!ATTLIST xi:fallback
+ xmlns:xi CDATA #FIXED "http://www.w3.org/2001/XInclude" >
+
+<!ENTITY % local.part.class "| xi:include">
+]>
<book>
<bookinfo>
<title><application moreinfo="none">refdb</application> handbook</title>

[Michael(tm) Smith]

Post by David Nebauer

Post by David Nebauer
[ERROR] null
org.apache.fop.apps.FOPException
at
org.apache.fop.apps.CommandLineStarter.run(CommandLineStarter.java:111)
at org.apache.fop.apps.Fop.main(Fop.java:62)

I've not been able to reproduce this in my environment with FOP
0.20.5. FOP generates the PDF file successfully.

Post by David Nebauer
This means you have content outside a flow, which is
illegal. This should not happen. Upgrade your DocBook XSL,
be sure your DocBook docs validate, and complain on the
DocBook list if this persists.
A few other references suggest this error is caused by invalid
docbook xml. I've attempted to validate the produced refdb-manual-xml
file with xmllint but am unable to do so successfully.

The patch below adds an internal subset to refdb-manual.xml to
make it possible to validate it with DTD-based validation.

--Mike

Index: refdb-manual.xml
===================================================================
--- refdb-manual.xml (revision 135)
+++ refdb-manual.xml (working copy)
@@ -1,4 +1,20 @@
<?xml version="1.0"?>
+<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.4//EN"
+"http://www.oasis-open.org/docbook/xml/4.4/docbookx.dtd" [
+
+<!ELEMENT xi:include (xi:fallback?) >
+<!ATTLIST xi:include
+ xmlns:xi CDATA #FIXED "http://www.w3.org/2001/XInclude"
+ href CDATA #REQUIRED
+ parse (xml|text) "xml"
+ encoding CDATA #IMPLIED >
+
+<!ELEMENT xi:fallback ANY>
+<!ATTLIST xi:fallback
+ xmlns:xi CDATA #FIXED "http://www.w3.org/2001/XInclude" >
+
+<!ENTITY % local.part.class "| xi:include">
+]>
<book>
<bookinfo>
<title><application moreinfo="none">refdb</application> handbook</title>