I’ve mentioned support for pixi a few times, and will soon include it in the macOS and Windows builds. (Probably good to include in Flatpak and AppImage as well.)
So I’m making a little checklist for needed support:
Download and bundle pixi with macOS and Windows builds
Check for pixi on startup and suggest to install it
Is it on many Linux distros yet?
If pixi is available, run pixi install with new plugins (may need to switch some plugins to pyproject.toml)
If pixi is available, and a plugin has a .pixi directory, run scripts as pixi run python … instead of the default python
Should we also offer:
If python is not available, offer to install it via pixi global
I’m trying to think if there’s any other big piece.
So far, I didn’t use pixi yet. According to repology.org it isn’t packaged yet for Debian & relatives, nor Fedora. If adopted to Avogadro2, it would be useful to mention the command curl -fsSL https://pixi.sh/install.sh | sh (and its longer corresponding command for Windows) the project page mentions.
I don’t like their “trust us, and just run the install script.” It seems like a huge security hole. If anyone manages to hack their webpage, you’re running arbitrary code on your account.
It’s much easier to download from https://github.com/prefix-dev/pixi/releases/latest/. Since it’s a static binary, it’s pretty easy to handle.
Since it’s not on repology, it’s clearly worth offering to install on Linux as well (e.g., into ~/.pixi/bin/pixi which seems to be their default.
@ghutchis I agree, this is a valid point of concern. On the other hand, I notice other implementations in Rust equally offer an installation relying on curl (example fortitude for Fortran, or uv for Python). With multiple records of similar name to pixi on PyPI, the alternative route of an installation via pip/pipx probably fell flat in favour of “here is the executable”.
I’m not sure I see the advantage of that, what do you expect to gain from it?
In general I think the best Python experience we can offer is enabled by keeping Avogadro’s activities isolated in its own environment that it can manage how it likes without affecting anything else.
Installation using pip requires Python, and a significant aim of uv and pixi is to make it possible to bootstrap Python with them. Also, pixi specifically aims to be a conda replacement, meaning it’s designed as a toolchain and package manager for multiple languages, not just Python.
The biggest complaint people have with Avogadro2 releases is that they don’t see various input generators (e.g. ORCA) because they don’t have Python installed.
pixi global actually creates a separate environment for it, so it won’t affect anything else.
My concern is that on Mac, creating a separate Python environment for each plugin is ~140MB each time. (The xtb plugin currently is ~210MB). I think many plugins like generators don’t actually require other pre-requisites, so having essentially an avogadro base environment saves a lot of disk space.
Then of course any plugin that needs specific requirements uses requirements.txt or pyproject.toml and will get its own .pixi environment in that install directory.
I agree completely with everything you’re saying It’s just that
is not the same as:
I agree that Avogadro should create and maintain exactly what you said: an Avogadro “base” environment, that is a common environment used for any plugins without special requirements. The only change I’m suggesting is that Avogadro should install Python to that avogadro environment, not to Pixi globally.
Essentially I think that Avogadro should ship with a copy of a Pixi manifest (probably in pyproject.toml form) that is used by Avogadro to create the base avogadro environment on first run, and that that environment is the default used for anything Python-related Avogadro does.
Scripts would be run in it using pixi run --environment avogadro -- python script.py. Package-style plugins should in future be in the form of a proper Python package, and they would be installed to the base environment using pixi add --manifest-path path/to/avogadro/manifest --pypi "someplugin @ file:///absolute/path/to/someplugin".
That way it is entirely isolated and we have complete control over it – we can specify the Python version in the manifest we ship, for example.
Well in fact I would go even further than you. I’d say that we should also use plugins with dependencies with the base avogadro environment as long as they don’t conflict with whatever is already in it.
What you describe is basically how we’ve been doing things with conda - create an avogadro environment (or whatever the user calls it). It’s tricky because some packages have inevitable dependency conflicts.
I wrote an initial reply, but after looking more carefully in the pixi docs, I think it solves our problem cleanly: Pixi Manifest - Pixi by prefix.dev
pixi will start going up parent directories looking for a manifest if it doesn’t find one.
Here’s how we currently install plugins:
[prefix]/avogadro
charges
commands
avo_xtb
avogadro_rdkit
etc.
energy
formatScripts
inputGenerators
So if we add a default manifest into [prefix]/avogadro that serves as the default environment for everything inside. Running pixi install will create a default environment with a base install of python. (I just tried this and it installed 3.13.7 for me.)
If a plugin has its own pyproject.toml then we can run pixi install in that directory when the plugin is installed. It installs any needed dependencies into a separate environment, which is important because many ML packages need to pin down particular versions. (For example my group’s QupKake model had issues with PyTorch ≥ 2.6) This could also install xtb or crest from conda-forge, for example, or antechamber for MD use.
So if I’m inside the avogadro_commands subdirectory and I simply pixi run python replace.py [options] it works even though that plugin has no manifest.
I forgot to mention there’s one special case - running the input generators installed with Avogadro. In that case, we manually use the manifest in the [prefix]/avogadro (user plugin) directory.
Of course if they already have python in their PATH, they can manually select that.
And in reference to my other thread about installing python for macOS and Windows users, the default will be to do that with the bundled pixi.
My thought is that the code to run plugin scripts should migrate to the following logic:
if pixi is available, use pixi run … for the plugin script
if the script ends in .py then use pixi run python script.py
if the script does not end in .py then use pixi run script
This would help enable non-Python plugins / tasks.
And of course for the default input generator scripts (i.e., not in the plugin download directory) add --manifest-path <MANIFEST_PATH>
Seems like that’s all the right logic. If so, the remaining piece is handling requirements, etc. after download. That might require updating some plugins which don’t have pyproject.toml or requirements.txt files.
Yes, the advantage of using pixi run is that we can start to allow other tasks, scripts, etc.
But then Avogadro has to find the manifest path. Right now, if pixi is available, there are two cases:
the internal input generators (which need a manifest)
installed plugin scripts (which get a default manifest installed)
So the new pull request detects the first case and add the manifest. This only happens if the user doesn’t have python installed / configured and does have pixi - which will be macOS and Windows users without python.
For plugin scripts without dependencies (e.g., XSF file reader) the default manifest will install python
It’s just that