[Refdb-devel] Docs in split packages

[David Nebauer]

Hi Markus,

I've been thinking about Justus' suggestion to split the Debian package
into server and client. You clearly anticipate such an eventuality with
the 'disable-[server|clients]' configure flags. Most of the machinery,
in fact, is already available to build separate packages. I can see
only one problem: the monolithic documentation. It cannot be split
during the build process. At the moment the choice would be building
full documentation in both, or only one, of the packages. Both
approaches are obviously less than perfect.

Have you considered splitting refdb's documentation into two parts --
able to be installed individually but, when installed together, making a
cohesive whole?

Or perhaps there is another approach?

Regards,
David.

[Markus Hoenicka]

Post by David Nebauer
Hi Markus,
I've been thinking about Justus' suggestion to split the Debian package
into server and client. You clearly anticipate such an eventuality with
the 'disable-[server|clients]' configure flags. Most of the machinery,
in fact, is already available to build separate packages. I can see
only one problem: the monolithic documentation. It cannot be split
during the build process. At the moment the choice would be building
full documentation in both, or only one, of the packages. Both
approaches are obviously less than perfect.
Have you considered splitting refdb's documentation into two parts --
able to be installed individually but, when installed together, making a
cohesive whole?
Or perhaps there is another approach?

Well, a third approach is to build the docs as a separate package and make both
the client and the server package depend on it. This would require a few minor
changes to configure.in.

Splitting the docs is a non-trivial task as I've put a lot of work into
cross-linking the contents. The only part of the docs which could be built
separately is the reference manual which is composed mostly of the man pages.
But then you'd end up with a client reference manual, a server reference
manual, and a maintenance/usage manual. The latter would contain large parts of
the installation and administration stuff that most users would probably expect
in the client or server manuals. I guess this makes the separate docs package
the more obvious choice.

Could you please check whether the Debian policy has specific requirements for
the docs? I mean, the proper man pages would be installed anyway with both the
server and the client package. If the Debian policy doesn't mind, you could
even make the docs package recommended instead of required, for those who want
to access the docs through the internet.

regards,
Markus

[David Nebauer]

Hi Markus,

Post by Markus Hoenicka
Well, a third approach is to build the docs as a separate package and make both
the client and the server package depend on it. This would require a few minor
changes to configure.in.
Could you please check whether the Debian policy has specific requirements for
the docs?

There is no absolute requirement in Debian that general documentation be
installed with the main application files. In fact, it is not uncommon
to have a 'widget' package and an accompanying, recommended but not
dependent, package called 'widget-doc'.

I've just built a version of refdb using 'disable-server' and
'disable-clients' to see if I could produce a document-only install that
might form the basis of a debian package. I note one major and one
minor problem. The minor first: the install includes the file
'refdb-manual.fo'. It is over 3 MB in size and not needed as the pdf
version installs also.

The major problem is the man pages build as part of the documentation
and not alongside the client or server. I quite understand why you
designed it this way. Unfortunately, it violates an inflexible Debian
rule that any executable installing to one of the recognised bin
directories MUST have an accompanying man page. If you could alter the
man page build such that their inclusion depends on the
'disable-[clients|server]' flags (and ditch 'refdb-manual.fo') it would
be quite simple to build a 'refdb-doc' package alongside 'refdb-clients'
and 'refdb-server'.

Regards,
David.

[Markus Hoenicka]

Hi David,

Post by David Nebauer
I've just built a version of refdb using 'disable-server' and
'disable-clients' to see if I could produce a document-only install that
might form the basis of a debian package. I note one major and one
minor problem. The minor first: the install includes the file
'refdb-manual.fo'. It is over 3 MB in size and not needed as the pdf
version installs also.

That's a bummer. I guess this was a backfiring attempt to force the
.fo file into the package to silence make. I've fixed that.

Post by David Nebauer
The major problem is the man pages build as part of the documentation
and not alongside the client or server. I quite understand why you
designed it this way. Unfortunately, it violates an inflexible Debian
rule that any executable installing to one of the recognised bin
directories MUST have an accompanying man page. If you could alter the
man page build such that their inclusion depends on the
'disable-[clients|server]' flags (and ditch 'refdb-manual.fo') it would
be quite simple to build a 'refdb-doc' package alongside 'refdb-clients'
and 'refdb-server'.

I tried to implement this suggestion. You should now be able to build
three packages:

--disable-clients --disable-docs (server package)
--disable-server --disable-docs (client package)
--disable-server --disable-clients (doc package)

Both the client and the server builds should be accompanied by their
respective man pages. Let me know if I got something wrong here.

regards,
Markus

[David Nebauer]

Hi Markus,

Post by Markus Hoenicka
--disable-clients --disable-docs (server package)
--disable-server --disable-docs (client package)
--disable-server --disable-clients (doc package)
Both the client and the server builds should be accompanied by their
respective man pages. Let me know if I got something wrong here.

Have built clients and server packages so far. The doc package is still
to come.

Results of first attempt to build split debian packages:


1. Man pages

No man pages at all built for client or server.


2. Duplicate files

These directories and their contents build in both clients and server:
/usr/share/refdb/sql, and
/usr/share/refdb/examples.
Obviously this is not allowed in package building as the same files
cannot be under the control of multiple packages.

Regards,
David.

[Markus Hoenicka]

Hi David,

thanks for testing the stuff. As usual, version 1 is crap.

Post by David Nebauer
No man pages at all built for client or server.

I've separated the switches for building docs and man pages (actually
I only now understand one of your original comments about this
problem). Now you have to fiddle with these switches:

<no switches> builds everything
--disable-server --disable-docs only clients and their man pages
--disable-clients --disable-docs only server and its man pages
--disable-clients --disable-server only documentation
--disable-docs --disable-manpages builds the binaries, no DocBook
installation required

Post by David Nebauer
2. Duplicate files
/usr/share/refdb/sql, and
/usr/share/refdb/examples.
Obviously this is not allowed in package building as the same files
cannot be under the control of multiple packages.

I've fixed #1 by using a conditional and #2 by including the examples
only in the client package - actually none of the files is required
for the server.

I hope this version builds ok.

regards,
Markus

[David Nebauer]

Hi Markus,

I've separated the switches for building docs and man pages.

More feedback, this time all on the server build:

1. Missing manpage symlink targets.

The following symlinks occur in man1:
refdb-init-pgsql.1 -> ../man8/refdb-init.8
refdb.sh.1 -> ../man8/refdb.8
but there is no man8 directory at all.

2. Missing refdb-init manpage altogether.

There is no refdb-init manpage symlink in man1 and/or no equivalent in
man8 (since man8 is missing entirely.

3. Missing interpreter

The script refdbctl has no interpreter in its first line. I note it is
the only one of the server-built scripts whose '.in' file uses the
'<myshell>' placeholder.

Post by David Nebauer
/usr/share/refdb/sql, and
/usr/share/refdb/examples.

I've fixed #1 by using a conditional and #2 by including the examples
only in the client package - actually none of the files is required
for the server.

These two directories have been 'fixed'.

Regards,
David.

[Markus Hoenicka]

Hi David,

thanks always for your thorough tests.

Post by David Nebauer
1. Missing manpage symlink targets.
refdb-init-pgsql.1 -> ../man8/refdb-init.8
refdb.sh.1 -> ../man8/refdb.8
but there is no man8 directory at all.
2. Missing refdb-init manpage altogether.
There is no refdb-init manpage symlink in man1 and/or no equivalent in
man8 (since man8 is missing entirely.

These problems were due to an incorrect usage of an autoconf/automake
trick. I've fixed that.

Post by David Nebauer
3. Missing interpreter
The script refdbctl has no interpreter in its first line. I note it is
the only one of the server-built scripts whose '.in' file uses the
'<myshell>' placeholder.

The interpreter check ran only if the clients were to be built. I've
moved the check outside of the conditional, and the shell appears
again.


regards,
Markus

[Markus Hoenicka]

Hi David,

All problems are still present building server+manpages from revision
125. To check whether it was a debian problem or a native build problem
I built server+manpages from source and installed them to a dummy
location. Here's what happened in the native build (item numbers match

I'm awfully sorry. I had sent the messages yesterday before checking in the
relevant files. Sure enough there was an intermittent svn problem, so I could
not check in the files yesterday. I'll do it as soon as I get home from work.
Please try again in a couple of hours.

regards,
Markus

[Markus Hoenicka]

Post by David Nebauer
------------------------------------------------------------
.so man8/refdb.8
------------------------------------------------------------
Clearly the symlinks have morphed into bizarre file content. There is
no man8 directory.

The "bizarre file content" is actually sort of a symlink emulation within the
man page system. If a man page is to be available through more than one name,
you create one real man page (in this case man8/refdb.8) and put stub files
everywhere else. The ".so" command simply means to source the mentioned file.
This is equivalent to a symlink on the filesystem level.

regards,
Markus

[David Nebauer]

Hi Markus,

Post by Markus Hoenicka
The "bizarre file content" is actually sort of a symlink emulation within the
man page system.

Oops. Thanks for the manpage tutorial.

Regards,
David.

[David Nebauer]

Hi Markus,

Post by Markus Hoenicka

Post by David Nebauer
1. Missing manpage symlink targets.
refdb-init-pgsql.1 -> ../man8/refdb-init.8
refdb.sh.1 -> ../man8/refdb.8
but there is no man8 directory at all.
2. Missing refdb-init manpage altogether.
There is no refdb-init manpage symlink in man1 and/or no equivalent in
man8 (since man8 is missing entirely.

These problems were due to an incorrect usage of an autoconf/automake
trick. I've fixed that.

Post by David Nebauer
3. Missing interpreter
The script refdbctl has no interpreter in its first line. I note it is
the only one of the server-built scripts whose '.in' file uses the
'<myshell>' placeholder.

The interpreter check ran only if the clients were to be built. I've
moved the check outside of the conditional, and the shell appears
again.

All problems are still present building server+manpages from revision
125. To check whether it was a debian problem or a native build problem
I built server+manpages from source and installed them to a dummy
location. Here's what happened in the native build (item numbers match
those used above):

1. Manpage symlinks

refdb-init-pgsql.1 and refdb.sh.1 are both present in man1 as regular
files, not symlinks. The problem is their content:

refdb-init-pgsql.1:
------------------------------------------------------------
.so man8/refdb-init.8
------------------------------------------------------------

refdb.sh.1:
------------------------------------------------------------
.so man8/refdb.8
------------------------------------------------------------

Clearly the symlinks have morphed into bizarre file content. There is
no man8 directory.


2. refdb-init manpage

Still missing.


3. Missing interpreter

Still missing.


Do these problems occur when building server+manpages on your boxen?

Regards,
David.

[Markus Hoenicka]

All problems are still present building server+manpages from revision
125.

Please give revision 126 a try. I've now checked in the relevant
changes that address the server-only build problems.

regards,
Markus

[David Nebauer]

Post by Markus Hoenicka
Please give revision 126 a try. I've now checked in the relevant
changes that address the server-only build problems.

Thanks, Marcus. Revision 126 worked the charm. All listed problems
resolved.

Regards,
David.

[David Nebauer]

Hi Markus,

Post by Markus Hoenicka
I've separated the switches for building docs and man pages (actually
I only now understand one of your original comments about this
problem).

Building documentation fails on conversion of refdbmanualfig5.svg:
-----------------------------------------------------------------
convert ../doc/refdbmanualfig5.svg ../doc/refdbmanualfig5.png
convert: unbalanced graphic context push-pop `graphic-context'.
make[2]: *** [../doc/refdbmanualfig5.png] Error 1
-----------------------------------------------------------------

You will see the build uses 'convert' as batik-rasteriser is
unavailable. I see you made some changes to that file in the last 48
hours which have presumably caused this error to occur. Unfortunately,
ViewCVS is currently unable to show the changes made in that revision so
I can't look for myself.

Regards,
David.

[Markus Hoenicka]

Hi David,

Post by David Nebauer
-----------------------------------------------------------------
convert ../doc/refdbmanualfig5.svg ../doc/refdbmanualfig5.png
convert: unbalanced graphic context push-pop `graphic-context'.
make[2]: *** [../doc/refdbmanualfig5.png] Error 1
-----------------------------------------------------------------
You will see the build uses 'convert' as batik-rasteriser is
unavailable. I see you made some changes to that file in the last 48
hours which have presumably caused this error to occur. Unfortunately,
ViewCVS is currently unable to show the changes made in that revision so
I can't look for myself.

refdbmanualfig5.svg is a new file which I added a couple of days
ago. I can process it with batik-rasterizer without problems, and even
convert (from ImageMagick 6.2.2, 12/11/05 Q16) works ok. Which version
of convert do you use?

regards,
Markus

[Markus Hoenicka]

Hi David,

It's strange since convert works on all other SVG images. Here's the

I'm afraid this is a ImageMagick bug. I've checked the SVG file in question with
the online validator at http://jiggles.w3.org/svgvalidator/. All I see are error
messages about namespaces in attribute names, but no structural or other errors.
These error messages about namespaces also occur in all other RefDB SVG files.
All of them seem to work ok with your version of convert.

I've also installed the newest Inkscape version (0.44) on my Windoze box, opened
the file (created with 0.43-2), and saved it using a different name. The diff
does not show any substantial differences except version numbers and path
names. It does not look like an Inkscape bug either.

I don't know where to go from here. I don't want to break building the svn
packages on your system. Is there any chance you could move to a different
version of ImageMagick, or even to batik-rasterizer? Otherwise you could try
and remove one object after another from the image and check the results with
convert in order to see what breaks the conversion.

regards,
Markus

[David Nebauer]

Hi Markus,

Is there any chance you could move to batik-rasterizer?

On debian batik is supplied as a library ('libbatik-java') with no
binaries. That's why the refdb configure check would always fail and
switch to 'convert'. Since it is a library there is no main executable
class supplied.

After a moderate amount of googling (whatever did we do before it
existed?) I've created my own 'batik-rasterizer' script which satisfies
'configure' and successfully converts all the SVG files.

To try and help those who come after me, here is the 'batik-rasterizer'
script for debian:

---------------------------------------------------------------------------------
#!/bin/sh

# Script: batik-rasterizer
# Date created: 2006-08-17
# Date last modified: 2006-08-17

# Function: Convert SVG images to PNG
# Wrapper for Debian 'libbatik-java' package

# Credits:
# David Nebauer
# author
# Petter Reinholdtsen
# for the basic script posted as part of debian bug report 152180
# accessible at
<http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=152180>
# Michael F. Lamb
# for the tip on script security in his posting at
# <http://blog.gmane.org/gmane.text.xml.batik.devel/day=20060403>


# VARIABLES

[ "${DESTDIR}" ] || DESTDIR=""
[ "${JAVA}" ] || JAVA="java" # Java executable
XERCES="${DESTDIR}/usr/share/java/xercesImpl.jar" # Xerces
BATIK="${DESTDIR}/usr/share/java/batik-all.jar" # Batik
CLASSPATH="${XERCES}:${BATIK}" # Java classpath


# MAIN

${JAVA} \
-cp ${CLASSPATH} \
org.apache.batik.apps.rasterizer.Main \
-scriptSecurityOff \
$@
---------------------------------------------------------------------------------

Regards,
David.