Skip to content

Nanobind migration discussion #101

New issue
New issue
Open
Feature
Open
Nanobind migration discussion#101
Feature
Labels
enhancementquestion

Description

@vincefn
vincefn
opened on Aug 12, 2026

@sbillinge this is to discuss the (possible) migration to nanobind instead of boost.python, e.g. as implemented in the nanobind_migration test branch

Please really consider this as a tentative organisation - I did this as a summer job and as a large vibe-coding exercise, so no worries if it does not match what you would like, it can be redone with different principles.

The main points are:

  • switch to nanobind
  • objcryst++ as submodule and cctbx/newmat code directly embedded, so no more need of libobjcryst
  • pyproject.toml and cmake for compilation (I don't know if you want to stick with scons
  • so far only tested on mac and linux

Also see the extended migration notes (rationale for various implementation decisions, issues with inheritance, etc..)

Activity

  1. vincefn
    added the
    Feature
    issue type on Aug 12, 2026
  2. vincefn
    added
    enhancement
    question
    on Aug 12, 2026

  3. sbillinge commented on Aug 13, 2026

    @sbillinge
    sbillinge
    on Aug 13, 2026
    Contributor

    Thanks @vincefn this is super-helpful. We haven't formulated specific standards for this migration and so lessons learned and recommendations from you are super helpful. As a general issue, moving to pyproject.toml/cmake builds seems like it is taking us in the right direction. I don't think there is any downside for us to drop scons, but as I said, I don't have tons of experience with that. Also, getting rid of the extra layer of libobjcryst seems like a win, so I don't object to that at all.

    I will read the notes in detail with my c++ enabled folks and see if they have any feedback.

  4. vincefn commented on Sep 1, 2026

    @vincefn
    vincefn
    on Sep 1, 2026
    CollaboratorAuthor

    nanobind branch update (September 2026)

    The nanobind_migration_test branch has been updated and is now based on the current main HEAD.

    What was done

    Submodule update

    • extern/objcryst updated from bfdcfda → 30fc8df (8 new upstream commits), adding GetRwHistory, minRwpVar, ReflectionProfilePseudoVoigtTCH, PowderPatternBackgroundHist, aniso PV constructor fix, etc.

    Translated from main branch commits (boost.python → nanobind):

    Translated from open PRs (#107, #113, #115, #116):

    Bug fixes in nanobind bindings:

    • Fixed pre-existing build error in nb_crystal.cpp: replaced non-existent nb::detail::keep_alive() free function (nanobind 3.x) with nb::keep_alive<1,2>() annotation
    • CMakeLists.txt: added homebrew include paths for Boost headers on macOS
    • Added SetProfile() to PowderPatternDiffraction (needed to install concrete profile objects)

    Tests: 171 tests, all passing.

  5. vincefn commented on Sep 24, 2026

    @vincefn
    vincefn
    on Sep 24, 2026
    CollaboratorAuthor

    Nanobind migration update:

  6. vincefn commented on Nov 2, 2026

    @vincefn
    vincefn
    on Nov 2, 2026
    CollaboratorAuthor

    Hi @sbillinge - do you have any update on this ? I'd really like to push this forward and get rid of libobjcryst, and simplify a lot the publication process, and also allow pure pip-installs. Of course it's a big change.

  7. sbillinge commented on Nov 2, 2026

    @sbillinge
    sbillinge
    on Nov 2, 2026
    Contributor

    I am good with this. What do you need from us? I assumed you were just going ahead with this, but were you waiting from word from us? What would the biggest downstream impacts be?

    We looked into also vendoring libdiffpy in a similar way but for us we have a fairly automated release cycle and releasing libdiffy updates and making them a dependency isn't such a big overhead for us and ensures we keep libdiffpy up to date. I need to understand if there are other benefits/downsides to doing it your way. It could, for example, require us to refactor our automated release scripts somewhat?

    tl;dr: please go ahead with libobjcryst changes if you are ready.

  8. vincefn commented on Nov 2, 2026

    @vincefn
    vincefn
    on Nov 2, 2026
    CollaboratorAuthor

    What is proposed here for pyobjcryst completely refactors the library, including all the memory management - so it needs some intense testing to make sure there are no side effects. @clemisch will test this for large Rietveld refinements (>10k patterns in cluding parallel processing), but more tests are warranted in this case given the extensive changes. I don't know how it could affect you.

    Main benefit (besides moving away from boost.python) is getting rid of libobjcryst, and greatly simplifying the release cycle.

  9. sbillinge commented on Nov 3, 2026

    @sbillinge
    sbillinge
    on Nov 3, 2026
    Contributor

    I see. We would be super interested in that for what we can learn for libdiffpy. Shall we make a new migration branch and release and test that before merging into main?

  10. vincefn commented on Nov 3, 2026

    @vincefn
    vincefn
    on Nov 3, 2026
    CollaboratorAuthor

    The https://github.com/diffpy/pyobjcryst/tree/nanobind_migration_test is already there for testing.

    You can test it just using:

    conda create -n pyobjcryst_nanobind python=3.14 compilers pip libboost-devel numpy
conda activate pyobjcryst_nanobind
pip install git+https://github.com/diffpy/pyobjcryst.git@nanobind_migration_test
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementquestion

    Type

    Feature

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions