New Plugin Infrastructure

Thanks to plenty of help from @matterhorn103 and @brockdyer03 the current HEAD and continuous build should be mostly stable again.

There are probably still bugs (just squashed one) - but you should be able to see plugins load dynamically. For example, the xtb-energy demo should load GFN-FF, GFN1, and GFN2 methods into the AutoOpt tool and Optimize Geometry automagically on install.

I’ve put in a few plugins of different types including porting cclib and rdkit for testing. (I think @matterhorn103 had some additional suggestions, so we’ll see if we can get a checklist together to finish up the loose ends.)

I’ll continue to make progress adding plugins, so if you have requests, please let me know:

Energy models:

  • OpenFF
  • Some ML potentials like MACE, ANI, AIMNET, etc.

Electrostatic models:

  • Antechamber (AM1BCC, ABCG2)
  • Grimme multicharge (CEH, EEQ-BC)

Commands

  • pymatgen - make slabs, calculate XRD
  • chiraltube
  • QupKake pKa predictions
  • TurtleMol
  • @matterhorn103 xtb and crest plugins
  • .…

Formats

Many of these should be fairly quick (famous last words) since I had prototypes for the previous script-file approach.

1 Like

Thanks for the efforts on the new plugin infrastructure!

I am however unable to install the generators plugin, I get the following message in the text box to the right:

Downloading avogenerators.zip to /home/javier/.local/share/OpenChemistry/Avogadro/plugins

Extracting avogenerators.zip…



Extraction complete (44 files)



Error: could not move extracted package to /home/javier/.local/share/OpenChemistry/Avogadro/plugins/avogenerators-143a06f3fc053a886fd47ab9d08f583997a4f009

And there is nothing in Avogadro/plugins directory. If I generate the avogenerators-143a06f3fc053a886fd47ab9d08f583997a4f009 folder, it is removed when I try to install the plugin (so it seems that Avogadro cleans the folder once the installation is unsuccessful).

Instead, the rest of plugins are properly installed (at least no error message), and the plugins folder is populated with the corresponding subfolders for each plugin.

Is it something I am doing wrong installing the generators? I am using the Linux AppImage. Thanks!

You shouldn’t need to install the generators. They should be included in the AppImage. Is that not happening?

I get a pop-up window when the app starts, asking if I want to install the generators. When I click yes, nothing seems to be installed, and I get this message in the terminal:

venv creation failed for "/home/javier/.local/share/OpenChemistry/Avogadro/plugins/plugin-demo" : ""
venv creation failed for "/tmp/.mount_AvogadWItLzM/usr/lib/avogadro2/plugins/avogenerators" : "Error: [Errno 30] Read-only file system: '/tmp/.mount_AvogadW
ItLzM/usr/lib/avogadro2/plugins/avogenerators/.venv'\n"
RPC server listening on: "/tmp/avogadro"
PackageManager: no avogadro-* entry in [project.scripts] in "/home/javier/.local/share/OpenChemistry/Avogadro/plugins/plugin-demo/pyproject.toml"

Orca generator appears in the list (under Inputs). When I select it I get in the terminal:

PackageManager: user-options JSON root is not an object: "/tmp/.mount_AvogadWItLzM/usr/lib/avogadro2/plugins/avogenerators/src/avogenerators/orca/options.js
on"

and the generator window contains nothing:

Moreover, I only have the generators in Avogadro/inputGenerators/avogenerators, but not in the Avogadro/plugins directory. So, when the latest nightly build, I cannot use the generators (this happens since some days, but I knew the app could be not stable for a while).

Moreover, I also find another different behavious of the app. When I open it from the terminal with an ouput:

avogadro2 job.out

it does not open job.out, while it did so previously (like one week ago or so)

Thanks, that’s helpful.

That sounds like a different issue. I’m assuming it opens job.out if you open from the dialog once it’s launched?

Hmm, seems like two problems:

  • it’s using the older generators (I’ll go check that)
  • the generators included in the AppImage are read-only so I’ll need a solution for installing the virtual environment for them (probably back to the ~/.local/ directories

Yes, it opens from the “Open” dialog. And sorry for mixing issues

Regarding the generators, they are shown up in the menu even if the old generators in Avogadro/inputGenerators/avogenerators folder is not present.

Yeah, it’s a different mechanism. I’ll need to code something to clean out the old plugin files.

I’ll take a look later today why it’s not installing completely. It’s working on my test builds, but not so much the releases.

Maybe there should only be one true plugin directory after all (the one under ~/.local/ on Linux) and the bundled plugins are not just installed to an environment there but are literally just copied across wholesale, and thereafter treated identically to other plugins?

1 Like

Yeah, it makes some sense - I just need to think if there are any issues with “locked-down” environments. Probably not.

And I fixed the issue @Javier_Cerezo reported with the generators showing up but not working. There was a slight change to the JSON syntax that wasn’t fixed on the C++ side. Should be working in the latest continuous release.

1 Like

Yes, the plugin works now, thanks!

However, I still find some related issues. Namely, the app keeps asking if I want to install the generators every time it’s opened (if I click on “No”, the generators do not show up). Moreover, the error message if I try to install the generators from the plugin manager persists (those reported above).

Finally, with the current structure, it is not clear if it is possible to override generators (i.e., modify the generator locally). I tried copying the avogenerator folder to the plugin directory, but the changes are not detected (maybe the folder tree inside generator has changed?)

With the AppImage. Yeah, that’ll require a bit more work. Right now, the AppImage will see the packages in the AppImage that aren’t registered and ask if you want to install them. But you can’t because the AppImage is read-only.

You can absolutely make local changes - that’s what I’m doing right now. :slight_smile:

You’d want something linked into /home/javier/.local/share/OpenChemistry/Avogadro/plugins based on your comments above.

What happens if there’s one version of avogenerators in the AppImage and one in the .local plugins directory?

It should defer to the .local one - that was the priority settings we used before.

But it’s also one reason I’m leaning towards not bundling avogenerators

1 Like

Can you be a bit more specific about which error messages? They should be fixed in the latest continuous build, so I’d like to see what else is a problem.

I was referring to this error, when I try to install the generators via the plugin manager:

Downloading avogenerators.zip to /home/javier/.local/share/OpenChemistry/Avogadro/plugins

Extracting avogenerators.zip…

Extraction complete (44 files)

Error: could not move extracted package to /home/javier/.local/share/OpenChemistry/Avogadro/plugins/avogenerators-143a06f3fc053a886fd47ab9d08f583997a4f009

Regarding the local installation of the avogenerators:

I tried copying the src/avogeneratorsfolder from the github repo ( GitHub - OpenChemistry/avogenerators: Python input file generators for Avogadro 2 · GitHub ) into the plugin directory, but these generators are not loaded. Which folder do I need to place in the plugin directory to override the generators bundled in the AppImage?

No, you want the whole avogenerators repo to be in your plugins directory. Avogadro is looking for the pyproject.toml among other things.

Let me see if I can get some better error messages on that “could not move extracted…” comment.

I also have a quick fix soon so it doesn’t continue to try to install into the read-only AppImage.

OK, I see. Anyway, with the patch reported in Unbundling generator scripts the avogadro generators are properly installed, as also confirmed in that post. Thanks!

I also checked that I can modify the scripts locally. However, if I click on Yes at start, the package is reinstalled and my local changes overwritten (not dramatic, I just wanted to note it).

This is actually a good thing in my opinion since it means the plugin detection/hashing is working properly.

Perhaps though the message could be changed to more clearly indicate what is actually happening? I like what the current message is (except for the missing space after the first sentence’s period), but I think that maybe subsequent messages should be something along the lines of

“Avogadro has detected that the input generators have been modified or removed, would you like to perform a clean installation?”

That way people who are tinkering with the input generators aren’t going to accidentally overwrite their modified versions, which is doubly important since the local generators don’t have source control.

Let me look carefully through that logic - it shouldn’t be constantly re-installing. The current HEAD should see that there’s a “generator” package in your plugins and not bother copying a new one.

The automated code review actually called out this point (i.e., it shouldn’t re-update if you’ve got something in your plugin directory) but I think it’s the correct behavior. For anyone modifying their plugins, it won’t overwrite those changes. And the plugin manager interface will offer updates whenever they appear.

Since I updated a bunch of things yesterday, are you using the latest continuous release?

I checked with the one from this commit: 6aa10c5a and the last NightlyBuild (I guess the are the same), and got the same behavior (the pop up window asking to install the generators appears every time I open the app)