Let's decide on a Python version policy (should we align to the Scientific Python policy?)

Development
1

After recent discussion, it seems to me a good idea to standardize on a minimum Python version for users that would likewise be a target for plugin authors. If a specific policy is chosen, rather than an arbitrary version that gets bumped at irregular intervals, it can be clear (e.g. via the documentation) what the Python requirements are at any given time.

Importantly, this doesn’t actually force the user to use anything in particular – as long as plugins/extensions don’t happen to make use of any features not present in the user’s Python version, the user can happily use whatever Python they want without a problem.

However, specifying an official Python version would allow us to say what is supported and what isn’t, and means users, Avogadro devs, and plugin devs can all be on the same page and let’s us rely on the same platform. It means devs can know exactly which features they should avoid using in order to fit the policy. It also allows maintainers to know exactly which dependencies to include when packaging.

I was initially going to suggest standardizing on the second-oldest supported release of Python; now that 3.13 has just released, that would currently be 3.10.

However, there is a cross-party effort, with big names involved, to coordinate the development of scientific Python packages like numpy, scipy etc., and they have agreed on a common time-based policy that involves dropping Python versions three years after release.

I would like to campaign for Avogadro aligning with this effort by having our target Python version (and target versions of numpy, scipy etc.) be the oldest within the Scientific Python support window. This would mean, as of a couple of weeks ago, Python 3.11, numpy 1.24, scipy 1.10, matplotlib 3.7, pandas 2.0, and so on (see the helpful chart at the above link).

Since extensions/plugins are quite likely to rely on the common scientific Python libraries, this only seems sensible to me.

I know this seems like it would cut off a lot of users, but as I said, for people who are really prevented from using newer versions, they can probably still use the majority of plugins without any issue at all. But we get all the benefits of standardization, everyone has the same expectations, and what is and isn’t expected to work is clear.

1 Like
2

@matterhorn In terms of the interpreter and about Avogadro being deployed in Linux Debian/Ubuntu, one could bring forward the argument to stick with Python 3.11.2 because this is the version the stable Debian 12 / bookworm is/was shipped (reference distrowatch) which in turn is one/the foundation of Ubuntu 24.04 LTS noble (which was released with Python 3.12.3, reference distrowatch). Both distributions’ support will last at least untill June 2026 (ref), and May 2029 (ref), respectively. IT helpdesks like them, in part because their update cycles are longer than Debian testing, or the once per six month (intermediate) releases in the ?buntu ecosystem.

From here (i.e., Debian 12 and its packaged Python 3.11.2), I would further propose a reference requirements.txt file to PyPI to stipulate explicitly the dependencies (example on the fragments library) the development should stick to. I equally like the idea by scientific python of a public drop schedule, to announce well in advance when which version of a specific library will be considered as obsolete.

If there were a widespread use of pytest based testing of the (Python based) contributions to Avogadro, one could easily monitor (manually/automatically triggered GitHub action, example) if newer versions of the interpreter and Python libraries introduce a problem, or not. However, in fairness of the Avogadro project, possibly this or a similar approach already is in place and use in many sub repositories (like the test script for the avogenerators), I didn’t yet visited every of the 57 sub repositories.

1 Like
3

Which is fine for those packages. But are there really critical changes in syntax / features that are crucial for Avogadro or plugin scripts?

I’d be okay with 3.10 – mainly because that’s what’s on “previous Ubuntu LTS = 22.04” and because it offers much, much better error messages. That’s what I’m using in classes now.

I don’t think we should go with current LTS because most workstations aren’t updated quickly. IIRC the AppImage builds are still targeting 20.04.

I want to be really careful. Many Avogadro users have much older hardware and OS. There was a recent concern about rendering speed on a 7-year-old Mac. I have no idea what version of Python that used. Maybe 3.7?

I understand that Python tools need to update, but a lot of scientific packages are developed by grad students, and reach some level of static once they graduate. Are we implying that Avogadro shouldn’t support old plugins if they don’t work with the latest numpy or Python version?

I’ll note that RDKit is building packages for Python 3.9 and Open Babel is building packages for Python 3.8.

I agree 100% on a consistent policy, but I don’t support the speed of numpy – I’d go with “second oldest supported version” or something similar.

1 Like
4

I would love to have better testing, for example for the generators:

A few of these should be really easy for automated Python tests if someone can help:

  • run the script without errors
  • run python script.py -h without errors
  • run python script.py --display-name and get a name back
  • run python script.py --print-options and get valid JSON with user options back
  • grab the default options and make sure python script.py --generate-input works

As far as the command scripts, I think I had an example test script here:

1 Like
5

I get that we don’t want to alienate users. But current Python versions apparently support back as far as macOS 10.13, which was released in 2017. So I don’t quite follow how a user with a Mac from 2017 should have any issue using 3.10 or 3.11?

Not at all; maybe I chose the wrong word in “support”. I don’t mean in any way that it should be made impossible to use a version older or newer than the target version. Anyone should be free to use whichever interpreter they like, and for the majority of stuff, it will continue working just fine.

This is my key point really – just because some specific version e.g. 3.11 would be selected as the target, it doesn’t mean 3.10 or 3.12 wouldn’t likely work fine.

I envisaged the following outcomes after selecting a target Python:

  • Full functionality/correct behaviour wouldn’t be unlikely on other versions, but it wouldn’t be guaranteed

  • Users who don’t currently have Python can be given a clear recommendation of which to download and install

  • If we one day either bundle Python or manage the download of interpreters on behalf of the user (e.g. as per my suggestion with uv) then it gives us a clear choice of which to install

  • For maintainers packaging for distros, the Python to have as a dependency is clear and standardized

  • Using a single virtual environment for all plugins becomes much more viable, as they are likely all compatible with the same versions of dependencies

  • Plugin authors will be encouraged to ensure that their plugin runs on the target version, which has two aspects to it:

    • features/syntax that have been deprecated should be dropped to ensure future compatibility;
    • features/syntax that is only introduced in later versions should be avoided

The last point is important for

No – but it’s important to know what the set of useable features is, and to know which version should be tested against. My use of type unions with a syntax that was introduced in 3.10 causing problems for @avervit is one example, but another potential one would be f-strings, which from 3.12 onwards are much more flexible and convenient and it is easy to forget that you have to stick to the old rules (e.g. cool_string = f"Hello {user["name"]}!", which prior to 3.12 has to use different quotes inside the f-string to outside: cool_string = f"Hello {user['name']}!"

The version of Avogadro in the repos for a given distro release can continue to depend on an older version of Python though right? Ubuntu 22.04 has Avo 1.95, and 24.04 has 1.99, so why should 2.0 have its Python dependency dictated by Ubuntu 22.04? And does that mean at the end of 2025 we will still have to continue using Python 3.10?

Sure, but that is something we really ought to sort out. AppImage doesn’t bundle all dependencies, so there is starting to be mismatch between what the AppImage wants and what is available on up-to-date distros. There is always Flatpak available as well, which encapsulates the dependencies much better, and that currently builds against Ubuntu latest, not 20.04. And on older distros there is always the possibility to use Avogadro from the package manager, right?

Having said all that, it seems all three of us agree on at least the principle of picking a target

and second oldest (so 3.10 currently) was also my original instinct. I just felt that it was maybe important to fit in with what the big players in the scientific Python community seem to have agreed on. But from my point of view, 3.10 is fine. :slight_smile:

I also note that despite their new common policy, numpy and scipy are both still currently >= 3.10.

The pattern matching and union type syntax were also pretty nice introductions. The only real thing I tend to miss from 3.11 is the built-in TOML parser, but that’s not very important for plugin dev.

6

Something along these lines would be sensible, I agree. I could add it to the docs.

7

My point is this. Users expect things to work “out of the box” so to speak. We already have issues with Windows users who run Avogadro2 and wonder why they don’t have all the features (because Python isn’t installed).

I try to be very conservative in use of language features because I’m aware that some users have old-stable systems. For example, “Scientific Linux” is used pretty widely (e.g., CERN, Fermilab) and is based on CentOS 7.

On the C++ side, we’re using C++17 and I’m not in a rush to start using C++20 features yet. And we haven’t fully migrated to Qt6 about 4 years later.

I understand that the Python infrastructure updates more readily, but I’m serious in being cautious about upgrading requirements.

I know @erb74 has had this discussion w.r.t. cclib and I’ve followed similar discussions with rdkit and other chemistry tools.

As I said, I’m good with suggesting 3.10 right now, but let’s hold off making “official policy” right now. I think we’ll get more information when we release Avogadro 2.0.

1 Like
8

Alright then, in that case I’ll target 3.10 for avo_xtb for now.

1 Like
9

Yes, cclib explicitly supports 3.7 and I think at least one person may be using 3.6. Some people want to use the default Python installs on their clusters, and a lot of admins are bad about moving away from old distros (CentOS 7). Buster (old-stable) is 3.7 and Ubuntu 20.04 is 3.8. I’d say I don’t like it, but IMO it’s not even the slightest pain point (been using typing.Union and typing.List/Dict/... forever), so the decision is easy.

3 Likes
10

A couple of times I used the AppImage in a computer pool running like “once set up, maintenance basically is reduced to is exchange of broken hardware by a volunteer” with some variant of Linux and programs which usually don’t need an update by the IT help desk – like Libre Office to prepare lab reports/presentations, a molecule editor (chemdoodle), etc. Where classes supported by Python anyway collect their libraries in virtual environments (which actually isn’t too bad, in some way). One of these pools offers Python only in versions 2.7.18 and 3.6.13 – and it supports pretty well the AppImage of version 1.99.

On the other hand, in Linux based computer pools which are more actively maintained, the help desk usually doesn’t launch an upgrade until the first point release of a LTS is published (for 24.04.1, this was only end of August) plus some \Delta t at their discretion.

11

In other words, it’s probably safer to go by “previous Ubuntu LTS” = 22.04.x, yes?

I think the moral of the story is we’ll get more information as Avogadro 2.0.0 is released and people let us know more. In the meantime, 3.10 is probably an okay baseline.

1 Like
12

Affirmative. Not only because schools tend to upgrade slowly, but with the general support of Ubuntu 22.04LTS/Jammy extending till 2027-06-01 (ref) in mind. Even Ubuntu LTS20.02/Fossa still enjoys general support till 2025-05-29 (loc. cit.).

1 Like