Re: tests of cdb tools fail on macos X

[ari edelkind <edelkind-debug@episec.com>]

This post is for those who wish to understand why the problem exists.
Those who simply want to correct the issue may skip to the SUMMARY
section, below.

I've debugged the cdbstats program.  One notices that 'klen' becomes
improbably large, larger than the entire database file, which may
suggest a corrupt database, were it not for the fact that the database
itself is intact (as can be verified with a system on which cdb
functions).  It becomes apparent, then, that something else occurs with
the effect of feeding bad data into what will be stored in 'klen'.
Further tracing reveals that the file is continuously repositioned to 0
with every call to seek_set().

In cdbtest.c (and cdbstats.c):
      rest = seek_cur(0);
      [...]
      if (seek_set(0,rest) == -1) die_read();

'rest' should be assigned the current position within the file, but it
is assigned only zero; thus, when seek_set() is called, the descriptor
is always reset to the beginning of the file.

Then examining the seek_cur() function:

seek.h:
      typedef unsigned long seek_pos;

seek_cur.c:
      #include "seek.h"
      [...]
      seek_pos seek_cur(int fd)
      { return lseek(fd,(off_t) 0,CUR); }

A quick check shows:
      $ grep lseek /usr/include/unistd.h
      off_t    lseek(int, off_t, int);
      $ grep 'typedef.*off_t;' /usr/include/sys/types.h
      typedef quad_t          off_t;          /* file offset */
      $ grep 'typedef.*quad_t;' /usr/include/sys/types.h
      typedef u_int64_t       u_quad_t;       /* quads */
      typedef int64_t         quad_t;

And we see that lseek on macos X returns a 64-bit integer (as opposed to
many common implementations that return 32).

Normally, the compiler would automatically cast the type for us.
However, since unistd.h is not included in the file, the function is
declared implicitly as "a function returning an integer".  This is
normally acceptable with the current use, since seek_cur() returns an
unsigned long integer, but causes the following problem:

PowerPC architecture is big-endian.  If lseek(fd, 0, CUR) returns 8192
(0x2000), in memory this will be:

      00 00 00 00 00 00 20 00

However, since the system believes lseek() to return a 32-bit integer,
only the first four bytes will be used:

      00 00 00 00

If the architecture were little-endian, the problem would go by
unnoticed (and considering the fact that the size of a cdb must be
addressable with 32 bits, it would never be a problem).


SUMMARY

The problem is that lseek() returns a 64-bit integer on OS X (darwin),
but it is declared implicitly as a 32-bit integer.  The architecture is
big-endian, so only the first four bytes, all zeros, will be returned.

The solution is simply to add

      #include <unistd.h>

to the beginning of seek_cur.c .


ari



edelkind-debug@episec.com wrote:

> After creating a concurrent database file with cdbmake or cdbmake-sv,
> the 'cdbstats' command fails to gather cdb statistics for the file:
> 
>   % cdbstats <test.cdb
>   cdbstats: fatal: unable to read input: truncated file
> 
> The system on which this fails is a macos X 10.3.6 host, using cdb
> 0.75
> and either of gcc 3.3 or 2.95.  I have tested the software on a few
> other architectures (freebsd, linux, solaris), and it functions
> properly; this seems to occur only on macos X.
> 
> It is also worth noting that the resultant cdb file is identical,
> regardless of the platform involved.  There is no error with the
> cdbmake
> process, only with the cdb statistical gathering process.
> Additionally,
> even on OS X, the cdbget program successfully retrieves entries.
> 
> Researching.
> 
> ari
>