OpenSolaris

an opensolaris user reported the following issue:
---8<---
Date: Tue, 08 May 2007 11:25:35 -0700 (PDT)

Subject: [brandz-discuss] fcntl F_SETOWN error with Opnet

We are trying to get Opnet working in a Linux branded zone.  The zone is running CentOS and the system is running SX b51 (I need to test a newer version when I get a chance).  I checked the change logs since b51 and didn't see anything related to this problem.  Open fails with the following error:

<< Program Abort >>>
  * Time:      14:21:52 Tue May 8 2007
  * Product:   modeler
  * Package:   Vos (Virtual Operating System)
  * Function:  vos_os_ipc_async_io_enable
  * Error:     Error in fcntl() call, F_SETOWN operation.
               (UNIX error: Ô`Óüþþ)

strace shows:
fcntl64(6, F_SETOWN, 6021)              = -1 EINVAL (Invalid argument)

and truss from the global zone gives:
brand(212, 0xFEFF5000, 0x00000020, 0xFEFF657C, 0xFEF337C0, 0x00000000) = 6048
fcntl(6, -2147195268, 0x000017A0)               Err#22 EINVAL

Does anyone have any thoughts on this error?

thanks,
Ben
---8<---

i replied to the user with the following information:

so what should be happening is the linux F_SETOWN should become a
solaris F_SETOWN.  but looking at the truss output the second
parameter to fcntl() (which should be F_SETOWN) is some strange
number.

looking at lx_brand.so.1`lx_fcntl_com(), the function which translates
the linux version to the solaris version i see:
---8<---
        case LX_F_SETOWN:
                rc = fcntl(fd, FIOSETOWN, arg);
                break;
        case LX_F_GETOWN:
                lx_debug("\tioctl(%d, 0x%x - %s, ...)",
                    fd, FIOGETOWN, "FIOGETOWN");
                rc = ioctl(fd, FIOGETOWN, arg);
                break;
---8<---

this seems weird.  afaik FIOSETOWN is an ioctl() commands,
not fcntl() command.  it seems like a simple bug where we've
accidently replaced F_SETOWN with FIOSETOWN.  that said.
we translate LX_F_GETOWN into an ioctl, so it seems that
for LX_F_SETOWN we should be doing one of the following:
        rc = fcntl(fd, F_SETOWN, arg);
or:
        rc = ioctl(fd, FIOSETOWN, arg);

perhaps bill can comment more on this?

you could test out the first idea by manually applying a binary
patch to our emulation library...  first make sure your zone is
stopped, then backup /usr/lib/lx_brand.so.1, then do the following:
---8<---
<<< load up /usr/lib/lx_brand.so.1 into mdb >>>

edp@mcescher$ mdb -w /usr/lib/lx_brand.so.1

<<< dissamble lx_fcntl_com() to find the instruction where setup
    FIOSETOWN as a parameter to fcntl() >>>>

> lx_fcntl_com::dis ! grep 0x8004667c
lx_fcntl_com+0x67:              pushl  $0x8004667c

<<< patch that instruction so that instead of using FIOSETOWN
    we use F_SETOWN >>>

> lx_fcntl_com+0x67 + 1 ? W 0t24
lx_fcntl_com+0x68:              0x8004667c      =       0x18

<<< verify that things look ok >>>

> lx_fcntl_com+0x67/i
lx_fcntl_com+0x67:              pushl  $0x18
---8<---

give that a shot and let me know if that fixes things for you.
if not i can try spinning you a custom binary to test out the
possible ioctl() fix.

N/A

N/A