Skip to content

Log reports "AssertionError: bug in code, exclusive lock should exist here" #10532

Open

Description

@noyannus

Likely related with / regression to #3082 and #3590, reported originally as Vorta issue.

Can this be triggered by a particular interplay between borg and Vorta?

Description

Log reports a bug: "AssertionError: bug in code, exclusive lock should exist here".
Seen only once in log so far, see log below. I had run "Check" on both my repos shortly before; maybe that is relevant.

Environment

  • Operating System:
    openSUSE Tumbleweed 20260924
    KDE Plasma Version: 6.7.5
    KDE Frameworks Version: 6.30.0
    Qt Version: 6.11.2
    Kernel Version: 7.2.7-1-default (64-bit)
    Graphics Platform: Wayland

  • Vorta was installed via zypper
    Repository : Main Repository (OSS)
    Name : vorta
    Version : 0.11.6-1.1
    Arch : noarch
    Vendor : openSUSE

  • Borg version: 1.4.5

Logs

...
2026-10-02 08:07:21,805 - vorta.borg.borg_job - INFO - Starting archive consistency check...
2026-10-02 08:07:22,149 - vorta.borg.borg_job - INFO - Analyzing archive foo (1/6)
2026-10-02 08:07:23,135 - vorta.borg.borg_job - WARNING - Exception ignored in: <function Repository.__del__ at 0x7f074a300720>
2026-10-02 08:07:23,147 - vorta.borg.borg_job - WARNING - Traceback (most recent call last):
2026-10-02 08:07:23,147 - vorta.borg.borg_job - WARNING - File "/usr/lib64/python3.13/site-packages/borg/repository.py", line 206, in __del__
2026-10-02 08:07:23,147 - vorta.borg.borg_job - WARNING - self.close()
2026-10-02 08:07:23,147 - vorta.borg.borg_job - WARNING - File "/usr/lib64/python3.13/site-packages/borg/repository.py", line 511, in close
2026-10-02 08:07:23,147 - vorta.borg.borg_job - WARNING - self.lock.release()
2026-10-02 08:07:23,147 - vorta.borg.borg_job - WARNING - File "/usr/lib64/python3.13/site-packages/borg/locking.py", line 431, in release
2026-10-02 08:07:23,147 - vorta.borg.borg_job - WARNING - self._lock.release()
2026-10-02 08:07:23,147 - vorta.borg.borg_job - WARNING - File "/usr/lib64/python3.13/site-packages/borg/locking.py", line 173, in release
2026-10-02 08:07:23,148 - vorta.borg.borg_job - WARNING - raise NotLocked(self.path)
2026-10-02 08:07:23,148 - vorta.borg.borg_job - WARNING - borg.locking.NotLocked: Failed to release the lock /path/to/repo/lock.exclusive (was not locked).
2026-10-02 08:07:23,148 - vorta.borg.borg_job - ERROR - Failed to release the lock /path/to/repo/lock.exclusive (was not locked).
2026-10-02 08:07:23,148 - vorta.borg.borg_job - ERROR - Traceback (most recent call last):
  File "/usr/lib64/python3.13/site-packages/borg/archiver.py", line 205, in wrapper
    return method(self, args, repository=repository, **kwargs)
  File "/usr/lib64/python3.13/site-packages/borg/archiver.py", line 416, in do_check
    if not args.repo_only and not ArchiveChecker().check(
                                  ~~~~~~~~~~~~~~~~~~~~~~^
            repository, repair=args.repair, archive=args.location.archive,
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
            first=args.first, last=args.last, sort_by=args.sort_by or 'ts', glob=args.glob_archives,
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
            verify_data=args.verify_data, save_space=args.save_space):
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/site-packages/borg/archive.py", line 1809, in check
    self.rebuild_refcounts(archive=archive, first=first, last=last, sort_by=sort_by, glob=glob)
    ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib64/python3.13/site-packages/borg/archive.py", line 2280, in rebuild_refcounts
    cdata = self.key.encrypt(data)
  File "/usr/lib64/python3.13/site-packages/borg/crypto/key.py", line 461, in encrypt
    next_iv = self.nonce_manager.ensure_reservation(self.cipher.next_iv(),
                                                    self.cipher.block_count(len(data)))
  File "/usr/lib64/python3.13/site-packages/borg/crypto/nonces.py", line 85, in ensure_reservation
    repo_free_nonce = self.get_repo_free_nonce()
  File "/usr/lib64/python3.13/site-packages/borg/crypto/nonces.py", line 44, in get_repo_free_nonce
    return self.repository.get_free_nonce()
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/usr/lib64/python3.13/site-packages/borg/repository.py", line 374, in get_free_nonce
    raise AssertionError("bug in code, exclusive lock should exist here")
AssertionError: bug in code, exclusive lock should exist here					<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib64/python3.13/site-packages/borg/archiver.py", line 5917, in main
    exit_code = archiver.run(args)
  File "/usr/lib64/python3.13/site-packages/borg/archiver.py", line 5835, in run
    rc = func(args)
  File "/usr/lib64/python3.13/site-packages/borg/archiver.py", line 190, in wrapper
    with repository:
         ^^^^^^^^^^
  File "/usr/lib64/python3.13/site-packages/borg/repository.py", line 231, in __exit__
    self.close()
    ~~~~~~~~~~^^
  File "/usr/lib64/python3.13/site-packages/borg/repository.py", line 511, in close
    self.lock.release()
    ~~~~~~~~~~~~~~~~~^^
  File "/usr/lib64/python3.13/site-packages/borg/locking.py", line 431, in release
    self._lock.release()
    ~~~~~~~~~~~~~~~~~~^^
  File "/usr/lib64/python3.13/site-packages/borg/locking.py", line 173, in release
    raise NotLocked(self.path)
borg.locking.NotLocked: Failed to release the lock /path/to/repo/lock.exclusive (was not locked).

Platform: Linux localhost.localdomain 7.2.7-1-default #1 SMP PREEMPT_DYNAMIC Tue Sep 22 08:00:24 UTC 2026 (e601a2d) x86_64
Linux: Unknown Linux  
Borg: 1.4.5  Python: CPython 3.13.15 msgpack: 1.2.1 fuse: pyfuse3 3.4.2 [pyfuse3,llfuse]
PID: 62545  CWD: /home/<user>
sys.argv: ['/bin/borg', 'check', '--info', '--log-json', '--progress', '/path/to/repo']
SSH_ORIGINAL_COMMAND: None
...

Activity

  1. ThomasWaldmann commented on Oct 15, 2026

    @ThomasWaldmann
    Member

    Did you use break-lock or another borg process in parallel?

  2. noyannus commented on Oct 16, 2026

    @noyannus
    Author

    I am not sure. I ran Vorta, that is supposed to only run one borg instance at a time (AFAIUI). If that is not watertight and two processes can overlap, the answer might be "yes". But I would not know how to catch what might be milliseconds in a race condition, or otherwise reproduce that. Sorry.

  3. ThomasWaldmann commented on Oct 23, 2026

    @ThomasWaldmann
    Member

    Thanks. I went through the borg 1.4.5 locking code and stress-tested it with many concurrent borg lock users on ext4, btrfs, vfat and exFAT, and could not make a held lock disappear.

    The traceback means the lock.exclusive directory in the repo vanished while borg check was running. Since you didn't break the lock, the likely cause is the storage itself.

    Where is the repository stored (internal disk, USB disk, network share, sshfs)?

    Is it inside a synced folder (Nextcloud, Syncthing, …)?

    Could you check journalctl -k --since "2026-10-02 08:00" --until "2026-10-02 08:15" for disk/USB resets or remounts?

  4. noyannus commented on Oct 24, 2026

    @noyannus
    Author

    The repo is on one of those disks (redundant setup):

    lsblk ...
    FSTYPE      NAME                 MOUNTPOINTS
                sdb                  
    ext4        └─sdb1               /media/backups/bup_int_2tb     # SATA
                sdg                  
    ext4        └─sdg1               /media/backups/bup_ext_4tb     # USB3
    

    journalctl unfortunately has no entries from that time any more.
    Vorta logs are also no longer available. Sorry.

    USB is not totally implausible to be the cause (faulty cable, ...), but the internal disk has never shown erratic behaviour since setup 3..4 weeks ago.

    As others have had storage problems that led to the same error message[*], would it make sense to add a test whether storage is still sound before raising the error? "Lock's not there, hmmm, let's see whether other files in that repo are accessible. And if so, let's just try again; maybe it was a temporary glitch." :-)

    [*]
    #7986 (comment)
    #3764 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions