OpenSolaris

Printable Version Enter a New Search
Bug ID 6630707
Synopsis assertion failed: used >= 0 || dd->dd_used_bytes >= -used, file: dsl_dir.c, line: 904
State 10-Fix Delivered (Fix available in build)
Category:Subcategory kernel:zfs
Keywords
Responsible Engineer Christopher Kirby
Reported Against
Duplicate Of
Introduced In solaris_nevada
Commit to Fix snv_79
Fixed In snv_79
Release Fixed solaris_nevada(snv_79) , solaris_10u6(s10u6_01) (Bug ID:2160921)
Related Bugs 6483677
Submit Date 16-November-2007
Last Update Date 5-December-2007
Description
SunOS Internal Development:  gk 2007-11-15 [onnv-gate]
bfu'ed from /ws/onnv-gate/archives/i386/nightly on 2007-11-15

Our functional test cases
/opt/SUNWstc-zfs/tests/functional/cli_root/zfs_rollback/zfs_rollback_001_pos
hit this assertion failed.

# mdb 1
Loading modules: [ unix genunix specfs cpu.generic cpu_ms.AuthenticAMD.15 uppc pcplusmp scsi_vhci ufs mpt ip hook neti sctp arp usba uhci s1394 qlc fctl nca lofs zfs random ]
> ::status
debugging crash dump vmcore.1 (64-bit) from gears
operating system: 5.11 onnv-gate:2007-11-15 (i86pc)
panic message:
assertion failed: used >= 0 || dd->dd_used_bytes >= -used, file: ../../common/fs
/zfs/dsl_dir.c, line: 904
dump content: kernel pages only
> ::stack
vpanic()
assfail+0x7e(fffffffff8032db8, fffffffff80334d8, 388)
dsl_dir_diduse_space+0x12f(ffffff01c082a400, fffffffff6a00400, 0, 0,ffffff01be538740)
dsl_dataset_set_reservation_sync+0xbe(ffffff01c0828900, ffffff000743d938, ffffff01b922c758, ffffff01be538740)
dsl_dataset_destroy_sync+0xdf(ffffff01c0828900, ffffff01b95012f8, ffffff01b922c758, ffffff01be538740)
dsl_sync_task_group_sync+0xfc(ffffff01bc2a1000, ffffff01be538740)
dsl_pool_sync+0xfc(ffffff01bc99edc0, 148)
spa_sync+0x218(ffffff01b83ce080, 148)
txg_sync_thread+0x221(ffffff01bc99edc0)
thread_start+8()
> ::ps
S    PID   PPID   PGID    SID    UID      FLAGS             ADDR NAME
R      0      0      0      0      0 0x00000001 fffffffffbc251f0 sched
R      3      0      0      0      0 0x00020001 ffffff01bb99f3a8 fsflush
R      2      0      0      0      0 0x00020001 ffffff01bb99b3a8 pageout
R      1      0      0      0      0 0x4a004000 ffffff01bb9993a8 init
R 100047      1 100046 100046      0 0x42020000 ffffff01bdeb53a8 dhcpagent
R 100006      1 100006 100006      0 0x42000000 ffffff01bc59b3a8 svc.configd
R 100004      1 100004 100004      0 0x42000000 ffffff01bc4cf3a8 svc.startd
R 100058 100004 100004 100004      0 0x4a004000 ffffff01c089b3a8 devices-local
R 100093 100058 100004 100004      0 0x4a004000 ffffff01c1db73a8 zfs
> ffffff01c1db73a8::walk thread | ::threadlist -v
            ADDR             PROC              LWP CLS PRI            WCHAN
ffffff01bcc501a0 ffffff01c1db73a8 ffffff01bc12d170   1  58 ffffff01bc99eed2
  PC: _resume_from_idle+0xf1    CMD: /usr/sbin/zfs volinit
  stack pointer for thread ffffff01bcc501a0: ffffff000744aa30
  [ ffffff000744aa30 _resume_from_idle+0xf1() ]
    swtch+0x221()
    cv_wait+0x73()
    txg_wait_synced+0x111()
    dsl_sync_task_group_wait+0x102()
    dsl_dataset_destroy+0x248()
    dmu_objset_destroy+0x5e()
    zfs_ioc_destroy+0x42()
    zfsdev_ioctl+0x12e()
    cdev_ioctl+0x48()
    spec_ioctl+0x86()
    fop_ioctl+0x7b()
    ioctl+0x174()
    sys_syscall32+0x1fc()

Core files: /net/zion.eng/export/dumps/forrest/<bugid>
I could easily reproduce it on x86, onnv-2007-11-15

during zfs_rollback_001_pos, 
the last living operation is destroy a volumn caused this issue.
it also hit this same panic after bootup, so I have to remove /etc/zfs/zpool.cache

Copy the crashdump to
/net/zion.eng/export/dumps/robin/<bug id>/*.8

panic[cpu0]/thread=e2841de0: assertion failed: used >= 0 || dd->dd_used_bytes >= -used, file: ../../common/fs/zfs/dsl_dir.c, line: 904

e2841a7c genunix:assfail+5a (f9acf760, f9acfe0c,)
e2841ad4 zfs:dsl_dir_diduse_space+176 (d5e7a400, f6a00400,)
e2841b38 zfs:dsl_dataset_set_reservation_sync+ea (d5c9fc00, e2841b78,)
e2841c78 zfs:dsl_dataset_destroy_sync+e1 (d5c9fc00, e3d367d0,)
e2841cb0 zfs:dsl_sync_task_group_sync+ec (d527fc80, e42702d8)
e2841cfc zfs:dsl_pool_sync+f9 (dd512380, 48, 0)
e2841d6c zfs:spa_sync+24a (d9f9fec0, 48, 0)
e2841dc8 zfs:txg_sync_thread+25a (dd512380, 0)
e2841dd8 unix:thread_start+8 ()

[0]> ::ptree
fec1e7d0  sched
     ca4b7398  fsflush
     ca4b7c30  pageout
     ca4b84c8  init
          d2d7e040  nfsd
          d615a038  mountd
          d2d81c68  ypbind
          d2d82500  devfsadm
          d5aee3c0  lockd
          d5aedb28  nfsmapid
          d615ba00  statd
          d615dc60  nfs4cbd
          d615c298  java
          d615d3c8  dtlogin
          cabc9140  intrd
          d45564e0  fmd
          d58e5028  sendmail
          d58e6158  sendmail
          cabcc4d0  sshd
          d45539e8  syslogd
          d11da148  automountd
               d11d98b0  automountd
          d58e8c50  inetd
          ca4b5138  cron              
               d4552020  sh
                    dd53d180  sh
                         d2d813d0  sh
                              d2d802a0  zfs-test-script.
                                   d58e7288  stf_execute
                                        dd53f3e0  stf_execute
                                             dbf86508  stf_execute
                                                  d615a8d0  stf_execute
                                                       dd5392b8  stf_timeout
                                                            cabc99d8  
                                                            stf_jnl_context
                                                                 d5aeb030  
                                                                 zfs_rollback_00
                                                                 1
                                                                      cabcb3a0  
                                                                      zfs
[0]> d858d680::threadlist -v
    ADDR     PROC      LWP CLS PRI    WCHAN
d858d680 cabcb3a0 dd4e99e0   1   0 dd51246a
  PC: _resume_from_idle+0xb1    CMD: 
  /usr/sbin/zfs destroy -Rf testpool.455682/testvol455682
  stack pointer for thread d858d680: e7672cd8
    swtch+0x20e()
    cv_wait+0x5d()
    zfs`txg_wait_synced+0x11f()
    zfs`dsl_sync_task_group_wait+0x100()
    zfs`dsl_dataset_destroy+0x208()
    zfs`dmu_objset_destroy+0x52()
    zfs`zfs_ioc_destroy+0x41()
    zfs`zfsdev_ioctl+0xfe()
    cdev_ioctl+0x2e()
    specfs`spec_ioctl+0x65()
    fop_ioctl+0x49()
    ioctl+0x155()
    sys_sysenter+0x1a4()
Work Around
N/A
Comments
N/A