[Refdb-devel] Build failure: HTML manual

[David Nebauer]

Hi Markus,

Revision 218 currently fails to build the HTML manual, reporting the
following error:
-------------------------------------------------------------------------------------
Creating HTML manual...
rm -rf refdb-manual/*
mkdir -p refdb-manual && cp ../doc/manual.css ../doc/refdbmanualfig1.png
../doc/refdbmanualfig2.png ../doc/refdbmanualfig3.png
../doc/refdbmanualfig4.png ../doc/refdbmanualfig5.png refdb-manual/ &&
xsltproc -o refdb-manual/ --nonet --xinclude include/manual-xhtml.xsl
refdb-manual.xml
runtime error: file
file:///usr/share/xml/docbook/stylesheet/nwalsh/xhtml/autotoc.xsl line
190 element apply-templates
The 'select' expression did not evaluate to a node set.
make[2]: *** [refdb-manual/*] Error 10
-------------------------------------------------------------------------------------

The relevant portion of the autotoc.xsl file is:
-------------------------------------------------------------------------------------
<xsl:variable name="subtoc">
<xsl:element name="{$toc.list.type}"
namespace="http://www.w3.org/1999/xhtml">
<xsl:apply-templates mode="toc" select="$nodes">
<xsl:with-param name="toc-context" select="$toc-context"/>
</xsl:apply-templates>
</xsl:element>
</xsl:variable>
-------------------------------------------------------------------------------------

The third line is the one generating the error.

I assume this error is due to a change in the toolset rather than a
change in refdb sources as there appear to have been changes made
recently to underlying java libraries. This became evident when I fixed
another build error due to batik failing to convert svg to png. The
batik error was caused by an unsatisfied dependency -- either the
dependent library (Attributes2.java) had been removed from an existing
jarfile or the dependency is new.

I'd appreciate some help in debugging this build failure as my
understanding of xsl is tenuous at best.

Regards,
David.

[Markus Hoenicka]

Hi David,

did you upgrade xsltproc (or libxslt) or the DocBook XSL stylesheets lately? I
don't think that your problems with the Java tools cause this problem as it is
xsltproc that fails. xsltproc is written in C.

I see a few possible reasons for the failure:

- a new version of xsltproc (or libxslt) may simply be broken

- my customization layer for the DocBook stylesheets relied on a bug/misfeature
which was fixed in a new version of xsltproc. I've tweaked the stylesheets
heavily to make the man pages appear like regular chapters in the table of
contents. These tweaks may contribute to the problem.

- The stylesheets were changed, and my customization layer no longer works

In order to fix this, I'd like to know which of the packages changed on your
system since the last time it worked.

regards,
Markus

Post by David Nebauer
I assume this error is due to a change in the toolset rather than a
change in refdb sources as there appear to have been changes made
recently to underlying java libraries. This became evident when I fixed
another build error due to batik failing to convert svg to png. The
batik error was caused by an unsatisfied dependency -- either the
dependent library (Attributes2.java) had been removed from an existing
jarfile or the dependency is new.
I'd appreciate some help in debugging this build failure as my
understanding of xsl is tenuous at best.

[Markus Hoenicka]

Post by Markus Hoenicka
- my customization layer for the DocBook stylesheets relied on a bug/misfeature
which was fixed in a new version of xsltproc. I've tweaked the stylesheets
heavily to make the man pages appear like regular chapters in the table of
contents. These tweaks may contribute to the problem.

please read that as:

may have relied ... may have been fixed

I have no positive proof that this was indeed the case, but it is possible.

[David Nebauer]

Hi Markus,

Post by Markus Hoenicka
did you upgrade xsltproc (or libxslt) or the DocBook XSL stylesheets lately?
In order to fix this, I'd like to know which of the packages changed on your
system since the last time it worked.

It's difficult for me to provide this information. I update my system
from the Debian repository almost nightly and don't keep track of what
changes occur or when. I presume this information is stored somewhere
in the package management system but, if so, I don't know how to
retrieve it.

Here is the information I've been able to retrieve:

I last built refdb-svn (successfully) on 26 October.

The packages xsltproc and libxslt1.1 are both built from the same
source. According to both packages' changelog (extract follows) those
packages were updated on 27 October or soon thereafter.
------------------------------------------------------------------------------
libxslt (1.1.18-1) unstable; urgency=low

* New upstream release:
+ Fixes xsl:variable with node sets. Closes: #381597.
+ Honors disable-output-escaping in xhtml 1.0 style element.
Closes: #395210.
+ Supports XInclude processing on XSL stylesheets. Closes: #395210.
+ Correctly handles xsl:param names with namespaces. Closes: #389023.
* debian/control:
+ Bumped Standards-Version to 3.7.2.2. No changes required.
+ Build depend on libxml2 >= 2.6.27, and adapt other dependencies
accordingly.

-- Mike Hommey <***@debian.org> Fri, 27 Oct 2006 18:30:38 +0200
libxslt (1.1.17-5) unstable; urgency=low

* xsltproc/xsltproc.c: Reverted patch to allow xsltproc to do XInclude
processing. Closes: #389694. Reopens: #383408.

-- Mike Hommey <***@debian.org> Wed, 4 Oct 2006 16:57:08 +0200
------------------------------------------------------------------------------

According to package docbook-xsl's changelog (extract follows) that
package has not been updated since 26 October.
------------------------------------------------------------------------------
docbook-xsl (1.71.0.dfsg.1-1.1) unstable; urgency=low

* Non-maintainer upload.
* Remove compatibility symlink in
/usr/share/sgml/docbook/stylesheet/xsl/nwalsh and its parent
directories
on removal; closes: #393726.

-- Loic Minier <***@dooz.org> Tue, 17 Oct 2006 22:26:46 +0200

[Markus Hoenicka]

Hi David,

I agree with your analysis that the upgrade of the xsltproc/libxslt
packages are most likely the reason for the build failure. In order to
find out whether it is an xsltproc specific problem, I wanted to
transform the manual with a different xslt processor (if the manual
would cause a similar problem with a different processor on my system
as well, I reckon the problem is with the driver file). However, I'm
apparently not up to this task:

- Saxon by itself does not seem to understand Xincludes

- Saxon-Xerces does not fare any better

- Xalan dies with an error:

(Location of error unknown)XSLT Error
(javax.xml.transform.TransformerConfigurationException):
java.io.EOFException: no more input

I spare you my usual rants about the state of XML tools compared to
SGML tools. Is anyone out there able to transform the manual with
anything else but xsltproc?

regards,
Markus

Post by David Nebauer
Hi Markus,

Post by Markus Hoenicka
did you upgrade xsltproc (or libxslt) or the DocBook XSL stylesheets lately?
In order to fix this, I'd like to know which of the packages changed on your
system since the last time it worked.

It's difficult for me to provide this information. I update my system
from the Debian repository almost nightly and don't keep track of what
changes occur or when. I presume this information is stored somewhere
in the package management system but, if so, I don't know how to
retrieve it.
I last built refdb-svn (successfully) on 26 October.
The packages xsltproc and libxslt1.1 are both built from the same
source. According to both packages' changelog (extract follows) those
packages were updated on 27 October or soon thereafter.
------------------------------------------------------------------------------
libxslt (1.1.18-1) unstable; urgency=low
+ Fixes xsl:variable with node sets. Closes: #381597.
+ Honors disable-output-escaping in xhtml 1.0 style element.
Closes: #395210.
+ Supports XInclude processing on XSL stylesheets. Closes: #395210.
+ Correctly handles xsl:param names with namespaces. Closes: #389023.
+ Bumped Standards-Version to 3.7.2.2. No changes required.
+ Build depend on libxml2 >= 2.6.27, and adapt other dependencies
accordingly.
libxslt (1.1.17-5) unstable; urgency=low
* xsltproc/xsltproc.c: Reverted patch to allow xsltproc to do XInclude
processing. Closes: #389694. Reopens: #383408.
------------------------------------------------------------------------------
According to package docbook-xsl's changelog (extract follows) that
package has not been updated since 26 October.
------------------------------------------------------------------------------
docbook-xsl (1.71.0.dfsg.1-1.1) unstable; urgency=low
* Non-maintainer upload.
* Remove compatibility symlink in
/usr/share/sgml/docbook/stylesheet/xsl/nwalsh and its parent
directories
on removal; closes: #393726.
------------------------------------------------------------------------------
Perhaps changes to xsltproc/libxslt have resulted in the current HTML
manual build problem.
Regards,
David.
-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
_______________________________________________
Refdb-devel mailing list
https://lists.sourceforge.net/lists/listinfo/refdb-devel

[David Nebauer]

Hi Markus,

Post by Markus Hoenicka
I wanted to
transform the manual with a different xslt processor (if the manual
would cause a similar problem with a different processor on my system
as well, I reckon the problem is with the driver file). However, I'm
- Saxon by itself does not seem to understand Xincludes
- Saxon-Xerces does not fare any better
- Xalan dies with an error

I found the following link gives a number of strategies for working
around the incomplete implementation of xincludes in Saxon and Xalan:
http://www.sagehill.net/docbookxsl/Xinclude.html#JavaXIncludes

The three strategies recommended are:
- use Xerces
- use xmllint as a preprocessor to resolve xincludes
- use XIncluder as a preprocessor to resolve xincludes

When using xerces with Saxon or Xalan the Java command requires
additional parameters. Further parameters are needed if catalogs are
used. I have to say the Java commands required are god-awfully hideous.

There is a thread in the docbook-apps mailing list that addresses this
issue. It includes contributions from Bob Stayton and Dave Pawson.
Since the thread navigation links on the site appear to be broken, here
are the thread postings in chronological order:
http://sources.redhat.com/ml/docbook-apps/2004-q1/msg00308.html
http://sources.redhat.com/ml/docbook-apps/2004-q1/msg00312.html
http://sources.redhat.com/ml/docbook-apps/2004-q1/msg00326.html
http://sources.redhat.com/ml/docbook-apps/2004-q1/msg00328.html
http://sources.redhat.com/ml/docbook-apps/2004-q1/msg00329.html

On the other hand, using xmllint to resolve xincludes before processing
appears to be simplicity itself:
xmllint --xinclude original.xml > resolved.xml

My apologies if you are already aware of this information. Your email
didn't specify exactly what strategies or commands you've tried.

Regards,
David.

[Markus Hoenicka]

Hi David,

Post by David Nebauer
On the other hand, using xmllint to resolve xincludes before processing
xmllint --xinclude original.xml > resolved.xml

darn, I completely forgot about this workaround. Thanks for filling me
in. I managed to build the manual using Saxon, and presto - same error
as you report. The following one-line-fix in
doc/include/manual-xhtml.xsl fixes the problem with Saxon:


--- manual-xhtml.xsl (revision 221)
+++ manual-xhtml.xsl (working copy)
@@ -59,7 +59,7 @@

<xsl:call-template name="subtoc">
<xsl:with-param name="toc-context" select="$toc-context"/>
- <xsl:with-param name="nodes" select="refsynopsisdiv | refsect1 | refsect2 | refsect3"/>
+ <xsl:with-param name="nodes" select="refsect1 | refsect2 | refsect3"/>
</xsl:call-template>

</xsl:template>


Could you please try whether this also fixes your build problem?

BTW it's not all milk and honey with Saxon after patching the driver
file. The navigation links generated by the docbook stylesheets are
missing, so you can't really move around, but I don't think that this
is caused by the driver file.

While trying to transform the manual with something else but xsltproc
I was actually using the commands that refdbxml would have
generated. This means that RefDB currently cannot handle documents
using xinclude unless you use xsltpoc. I guess we'll have to fix this
somehow. I'll peruse the links that you send to figure out a fix.

regards,
Markus

[David Nebauer]

Hi Markus,

Post by Markus Hoenicka
The following one-line-fix in
- <xsl:with-param name="nodes" select="refsynopsisdiv | refsect1 | refsect2 | refsect3"/>
+ <xsl:with-param name="nodes" select="refsect1 | refsect2 | refsect3"/>

This solves the problem with xsltproc and I am now able to build the
docs. Thanks for the fix.

Incidentally, the nav links are still present in the output.

Regards,
David

[Markus Hoenicka]

Post by David Nebauer
This solves the problem with xsltproc and I am now able to build the
docs. Thanks for the fix.
Incidentally, the nav links are still present in the output.

Great. The nav links also work in my xsltproc output. I only got this
problem when transforming the manual with Saxon.

regards,
Markus

[Michael(tm) Smith]

I wanted to transform the manual with a different xslt processor
- Saxon by itself does not seem to understand Xincludes

Correct. I think you'd either need to:

A. Add a pre-processing stemp step that first resolves the
Xincludes before handing off the document to Saxon for the
XSLT transformation.

B. Configure Saxon to use a parser that is capable of resolving
Xincludes

As far as B, I seem to remember hearing that Xerces can now
resolve Xincludes if you have it installed (perhaps along with
some extra jars from the Apache XML Commons) and have done some
config stuff to enable it.

As far as A, you can use "xmllint --xinclude foo.xml > bar.xml" to
generate a bar.xml from foo.xml with all Xincludes resolved. You
can then run Saxon on bar.xml.

I spare you my usual rants about the state of XML tools compared to
SGML tools. Is anyone out there able to transform the manual with
anything else but xsltproc?

I've not tried yet, but I will attempt to and let you know.

I will also update my refdb working directory and try running the
transformation (in my Debian environment) with the default
makefile and see what happens.

Depending on whether that works or not, I'll then try swapping out
xsltproc and/or docbook-xsl versions to see if I can either break
it (if it's already working for) or get it working correctly. Then
I'll report back about where the problem appears to be.

I suspect it's in xsltproc/libxslt/libxml2, but we'll see.

--Mike

[Markus Hoenicka]

Post by Michael(tm) Smith
Depending on whether that works or not, I'll then try swapping out
xsltproc and/or docbook-xsl versions to see if I can either break
it (if it's already working for) or get it working correctly. Then
I'll report back about where the problem appears to be.
I suspect it's in xsltproc/libxslt/libxml2, but we'll see.

Thanks for looking into this. However, I think the problem is caused
by a driver file bug (see my reply to David). xsltproc apparently used
to be too lenient to throw an error. A recent version may have
improved the tool, causing it to recognize the error properly. While
you're at it, please try and see whether the one-line patch I posted
fixes the problem if you manage to reproduce it.

regards,
Markus