User Experience Concerns

Development
1

I have been thinking for a while about the current big things that stop people from using Avogadro 2, and I think that the two major contributors are

  1. Downloading releases instead of nightly builds
  2. Not having python installed for force fields and other plugins

I think that downloading the release version and not the nightly builds is almost always a problem, since the releases are usually months out of date, whereas the nightly builds are quite literally updated nightly. I see quite often on this forum too that the immediate recommendation is to just download the latest nightly build. There is the obvious issue that occasionally a nightly build introduces a new bug, but the odds that a nightly fixes more bugs than it introduces are high.

I think, and I imagine that @matterhorn103 agrees, that the Python problem is likely the single most pervasive issue for Av2. The LJ forcefield that currently is the default without Python is decidedly not designed for molecules, and when new users see that as the first thing when they try to use Avogadro it sets a bad precedent for the rest of the program (it also sees significant mention on r/cursedchemistry).

If the goal is to not bundle a Python interpreter, then I think it would be wise to throw a warning if someone tries to use the optimize geometry thing that the force fields require Python. If bundling Python is an option, then I think that throwing up a dialogue to ask the user if they would like to install Python (with a list of the features that are dependent on it) as well as the packages required is the best option. It means that users can still say no if they canā€™t afford the extra space (I mean come on though, a Python interpreter is like, a few MB at most) and if they already have Python installed. It could also run handy commands to fetch any required packages like numpy, ideally using something basic like pip.

Regardless of the solution, it needs to get done sooner than later because at this point some of the most critical features of the program need Python, and it is what likely >90% of the userbase use Avogadro for. Hopefully there can be some good discussion regarding this, and hopefully we can reach consensus quickly!

2

I think that boils down to two (and a half) things:

  1. More frequent releases (yes, itā€™s a good idea)
  2. A dialog to suggest installing Python (e.g., Windows) - which already exists (and points to a URL to miniforge)

Yes, we probably should have had some sort of release late-summer / early fall. And Iā€™ll start a thread about a more frequent ā€œmajor releaseā€ schedule (e.g. 2-3 times a year) in a bit.

As to the LJ as a default optimizerā€¦ it kinda isnā€™t. The default doesnā€™t require Python at all. On Mac and Windows, the default builds include GPLā€™ed plugins which use Open Babel for UFF, MMFF94, GAFF, etc - and no Python needed.

One thing thatā€™s true is that until a recent bug fix, if Avo2 detected that your molecule was charged or had a radical, it wouldnā€™t even try UFF. Now it will.

Now, as Iā€™ve mentioned, thereā€™s some benefit to having a separate implementation of UFF - since this would avoid the GPL and also be a bit faster (avoiding copying molecule information to Open Babel and back).

As to a Python install for Windowsā€¦ Thereā€™s an open feature request if someone wants to look at the installersā€¦ Consider how to package Python as an optional install for Windows Ā· Issue #1449 Ā· OpenChemistry/avogadrolibs Ā· GitHub

3

Maybe I just donā€™t quite understand what you mean here, are all of the Open Babel things just bundled up into an Avogadro installation, or is it calling to something else?

What was happening previously that required Python to run?

On a related note, why are the only available force fields GAFF, MMFF94, UFF, and LJ? I know there are things like GFNFF, GFN2, GFN1, etc. that exist as .py files in Av2, but they never show up.

4

Avogadro 1.99 and later include a script API to optimize with Python scripts. I included some examples, including Open Babel and GFN1, etc.

  • to optimize with Open Babel scripts, you need the openbabel Python package installed
  • to optimize with GFN-FF, GFN1, GFN2, you need the xtb-python package installed

These are now no longer bundled, because people found the warning messages confusing. Iā€™ll be releasing them as separate downloads with requirements.txt as well as a few other energy scripts (e.g., ANI, AIMNET, ASE, MACE, MatterSim, RDKit, etc.)

But it doesnā€™t require Python. The force field framework includes two backup options to access the Open Babel force fields:

  • optional GPL code that directly uses Open Babel
  • BSD-licensed code that reads from the obmm utility
5

I factory reset an old laptop of mine and installed Avogadro 2 (latest nightly), and it never popped up. I have a few guesses as to why.

The first is that it thinks that Python already exists. In the dialogue where you can configure Python settings, there are 3 paths already in the drop-down menu:

  1. C:/hostedtoolcache/windows/Python/3.11.9/x64/python3.exe (Unknown)
  2. C:/Users/{Username}/Appdata/Local/Microsoft/WindowsApps/python3.exe (Unknown)
  3. C:/Users/{Username}/Appdata/Local/Microsoft/WindowsApps/python.exe (Unknown)

The second two are from a path variable that is always set in Windows %USERPROFILE%\AppData\Local\Microsoft\WindowsApps, and points to a file with a bunch of executables, however running any of the executables there simply points the user to the Windows store download page for that thing.

The first thing there is much more interesting, and after some googling I think it comes from the way that the nightly builds are built. My best guess is that you (@ghutchis) use an Azure Virtual Machine to build Avogadro 2 nightly builds for Windows, and on that virtual machine there is a thing that Azure seems to do called Pipeline caching. I think then when you build Avogadro 2, it stores the variable PYTHON_EXECUTABLE as a string called pythonInterpreterPath, which then gets referenced in avogadrolibs/avogadro/qtgui/pythonscript.cpp (line 40) in the method PythonScript::setDefaultPythonInterpreter.

What I believe is happening as a consequence of this is that Avogadro never encounters a situation where the Python path is empty, thus it will never show the dialog box to install it.

1 Like
6

Okay, the ā€œcacheā€ part can be handled by this:

I have to ask about the WindowsApps paths ā€“ do those actually exist as a file thatā€™s executable?

In that case, do we filter them? If someone installs python from the App store does it show up at that path? (In other words, can we include code that explicitly ignores WindowsApps in the paths for python?)

7

They do exist, however they are just part of Windowsā€™s app execution aliases. They do actually execute, causing a command prompt to immediately open and close, which I imagine is a thing that just opens the microsoft store.

Annoyingly, yes. I took the laptop that was factory reset and installed Python from the Windows store, and the same path variable pointed to the Python executable there. I did notice that upon reopening Avogadro 2, the path that pointed there changed from

C:/Users/{Username}/Appdata/Local/Microsoft/WindowsApps/python3.exe (Unknown)

to

C:/Users/{Username}/Appdata/Local/Microsoft/WindowsApps/python3.exe (3.13.1)

So somewhere whatever program figures out the Python version knows that something changed. Perhaps checking if the version is Unknown versus a legitimate Python version would work?

Edit: So it seems that when I download Python that the environment variable directs to the AppData place, however the actual Python executable that is stored there is still a 0kB file that just points to the real Python interpreter under

C:/Program Files/WindowsApps/PythonSoftwareFoundation.Python.3.13_3.13.496.0_x64_qbz5n2kfra8p0

Iā€™m not sure if this actually helps at all, but if youā€™d like to just look for the real Python executable under that address, then you might be able to do that too and just ignore the AppData variable. That does seem a bit harder to do though, so maybe not the best idea.

1 Like
8

Hmm. I wonder if I check the size of the file? I guess it depends on whether it redirects or not.

Probably safer to try to get a version, even if that opens up the App Store.

(Actually, now that I think about it, opening the App Store isnā€™t a bad outcome either.)

9

Well the issue may arise that if you check the file size then Avogadro will have to check another place for a Python interpreter since itā€™ll avoid the 0 kB pointer files. Iā€™d say itā€™s probably fine to just check for the version, since the redirects work fine on Avogadro. And opening up the Windows Store isnā€™t a terrible idea, since at least itā€™ll get it working from an end-user standpoint.

10

Okay, the new pull request should now check every potential python for a version number before adding it to the paths.

Iā€™m pretty confident at this point, your example would return no paths.

(Itā€™s probably worth adding a global ā€œReset Settingsā€ command, which would also help testing these scenarios.)

11

It does indeed return no paths, but there are some bugs left to squash. First is that the popup window doesnā€™t actually do anything. It says something along the lines of ā€œDo you want to install Python?ā€ and provides an ā€œOkayā€ button (probably should also add a ā€œNoā€ button but thatā€™s not required), however the ā€œOkayā€ button doesnā€™t do anything. It didnā€™t open up any pages even though I recall you saying it should open a miniforge link.

The second issue is that the popup never pops up more than once. I even tried uninstalling Avogadro 2 and reinstalling it, I tried installing an older version and then reinstalling it, but nothing ever made the popup show again. That may be intended behavior, but it seems weird that it is so persistent that even removing Avogadro 2 entirely and reinstalling causes it to still not show up.

While I was working at this, I also realized that maybe including a README file with an Avogadro download might be useful, perhaps in addition to the pop-up there could be a link in the README to either the miniforge or to the Python website. Now, the README would contain other information besides all that, I am thinking especially that it should have a message for people downloading nightly builds since on Windows I think you didnā€™t get the code signing for nightlies, so it still pops up with a Windows Defender warning. It could also just include some of the generic information, like about how plugins require Python, and how some features are still in testing, so any questions should go to the forum. Iā€™ll take a crack at writing one that I think contains everything a new user could need and post it here when I get that done.

12

Installing / re-installing Avogadro wonā€™t change the popup, itā€™s stored in the settings, which IIRC is the Registry on Windows.

As to the ā€œOkayā€ button - it should open a URL: https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Windows-x86_64.exe

That URL is handled through QDesktopServices::openUrl(miniforge); which should open in a browser window. (Presumably that works for similar things in the Help menu?)

I can certainly change the code to warn every time itā€™s launched without a valid Pythonā€¦

If you want to try writing a README for Windows, that would be fine. (Iā€™m more inclined to include miniforge into the NSIS installer.)

Yeah, the problem is that the code signing requires an actual human to approve once the binary is uploaded to SignPath to sign.

I tested on a few Windows machines and evidently, Defender needs some people to agree before it trusts a new certificate. So even with the 1.100 release, there will be some delay before the certificate works for everyone. (Hopefully after that point, weā€™re good :crossed_fingers:)

13

I spent about an hour yesterday poking around the Registry in my old laptop before factory resetting it, and I was shocked at how many programs I had used and uninstalled that still had their settings in the Registry. Turns out, itā€™s darn near impossible to delete registry entries properly from within an uninstaller (darn you Windows).

I just checked, and deleting the OpenChemistry Registry folder does indeed cause the popup to reappear. After that, I re-checked, and the link will still not open. The link you attached does immediately download it when I click it on it from inside a browser. Perhaps there is something blocking the link from immediately downloading it when I click on it? If youā€™re curious, I tried changing my default browser from Firefox to Edge, no difference.

The items in the Help menu do work, which makes me think itā€™s an issue with downloading instantly instead of opening a webpage.

I think that is a good idea, saves some trouble down the line for most users.

14

Yes, thatā€™s probably right.

We should clearly create a page on https://two.avogadro.cc/ with a big link to the Windows installer and some descriptions about why you would want to install Python / Miniforge.

For example: conda-forge | community-driven packaging for conda | conda-forge | community-driven packaging for conda

15

That is a good solution. Iā€™d suggest adding in options for various package managers or just base Python for users who donā€™t want anything else on their machine, or perhaps users who are on devices that are operated by an organization that hasnā€™t approved miniforge or the like.

I personally just use regular old Python+Pip for everything since I donā€™t really see any need to do anything else, and it works just fine for what I want.

16

If you (or @matterhorn103) or anyone else wants to write up a page (or even just some Markdown somewhere) Iā€™ll be happy to put it up on Install ā€” Avogadro 1.99.0 documentation before the release.

I just want to keep it simple because this will pop up for lots of new users.

As Iā€™ve said, one advantage of miniforge / conda-forge is that they distribute a range of science packages as binaries, not available through pip. Agreed that some users donā€™t have full control over installs.

17

My main concern with the Python stuff is the fact that, as far as I can tell from the code, thereā€™s no deliberate invocation of a virtual environment. If a plugin has a dependency, the user will be asked if they want to install it, and if they do, it will be installed into the environment of the Python interpreter being used. This means that for many users Avogadro will install packages directly into their system environment, or into their base conda environment, which is not good practice.

On my system, installation of the plugins that require dependencies isnā€™t possible because the system Python is protected against installation of packages with pip, but I am not sure that this is the case on all Linux systems, especially the less cutting-edge distros. And while this means the userā€™s installation is safe, it also would be a confusing problem for the average user to solve. I also donā€™t know if conda prevents you from installing things with pip into its environments?

Furthermore, the dependency installation is done by Avogadro executing python -m pip ..., but not all Python installations include pip. This includes the Python interpreters used by uv (meaning that if I select a Python interpreter in the Python settings thatā€™s in a venv created by uv, Avogadro doesnā€™t have the power to install any packages into it) and is probably the case for virtual environments created by other programs too. Itā€™s also the case for the Python included in KDEā€™s Flatpak runtime, but I guess I can look at bundling pip with Avogadro to solve that.

I still think the best approach would be to declare Python as a hard dependency, bundle it with Avogadro on Windows and Mac, and only use the system Python on Linux.

But even if we donā€™t, I certainly think that the first time Avogadro launches and successfully finds a Python interpreter it should create a virtual environment for itself (something all Python installations are capable of) at <user_data>/OpenChemistry/Avogadro/.venv and always run plugins from within that virtual environment.

This way Avogadro doesnā€™t interfere with any other Python installations and neither does the user or the system interfere with Avogadroā€™s packages.

Users could then explicitly choose a different virtual environment from the Python settings dialog if they really wish, but I think very few users would want to.

18

We donā€™t need to install Python on Mac. In some cases it might be a bit old (e.g., 3.9.6 on my MacBook) but itā€™s included.

No, not necessarily. If a user has set a specific conda environment (or created one through Avogadro since it will prompt them) it installs into that.

The main problem is that the real ideal is a separate virtual environment for each plugin script. And the code sets that up on install and detects when it goes to run the script. But itā€™s a pain because the ā€œbest practicesā€ for virtual environments has been a rapidly moving target over the last ~3 years and because many chemistry tools also heavily use conda (e.g., xtb-python and tblite). So maybe pixi is a solution. etc. :man_shrugging:

Thatā€™s why the current code suggests installing miniforge and Avogadro will prompt you to create a conda environment.

I recognize this is not necessarily ā€œbest practices,ā€ but I think reflects the current state of users. Iā€™m open to other suggestions as we move forward.

As far as this current situation, if the dialog doesnā€™t directly download miniforge I think adding a page with some user-friendly suggestions / download buttons is a good place to start. We can tweak it to use uv or whatever else in the future.

And we should certainly run a user survey fairly soon.

19

Youā€™ve said this several times, but all my googling indicates this is not true. It was true a long time ago, but these days it seems you need to install the command line developer tools first, which is part of xcode. So while naturally you have this installed, the average user who doesnā€™t program wonā€™t. See e.g. this thread, or this one, or this article which explains how to install them.

Iā€™m fine with that but if you do that then you end up with the duplicated dependencies that you seemed exceptionally keen to avoid. A single venv for Avogadro would at least only have one set of packages but would be isolated from the wider environment.

Not sure Iā€™ve ever actually seen this prompt. Certainly on Linux Iā€™m not offered, it just tries to install into the system environment and then errors because my distro has added the externally managed environment marker.

There has been an influx of new tools, sure, but the way that virtual environments actually work is, as far as Iā€™m aware, universal across all of them and hasnā€™t changed, the necessary directory structure and the environment variables that are set are still the same. That means python -m venv is still a completely valid way of creating a virtual environment. Iā€™ve suggested uv in the past because it could do a lot of the dirty work for us, but we wouldnā€™t need to necessarily.

Sure, a Python setup guide is high on my priority list for the docs. I just would prefer if a setup guide wasnā€™t necessary at all because Avogadro handles everything without the user even knowing about it, just like the user doesnā€™t need to know about C++ build tools.

I donā€™t like that a user even has to be aware that this stuff is going on, thatā€™s all, I think it leaves a lot of users completely befuddled, as evidenced by the support requests.

20

Ideally yes. Iā€™d like to avoid duplicated dependencies. Iā€™m all for saving disk space. OTOH, I agree with your comment that users shouldnā€™t have to be aware that this stuff is going on. So the ideal default would be for separate virtual environments for plugins.

Letā€™s talk in a separate thread. As I said, it still doesnā€™t help that a variety of tools still expect conda, but perhaps we can sort out some options (e.g., pixi seems like it will figure out uv or conda-forge).

In this thread (and for the next release) I think we just create https://two.avogadro.cc/install/python.html or similar with brief user-friendly instructions. We can even have the link in avogadrolibs go to a particular section, e.g. #windows

Meh. Speaking of saving disk spaceā€¦

As far as setting up a conda environment: