I will shortly turn to the new plugin system - over the weekend, I started up some fuzzing tests: both for IO (e.g., reading file formats) and now for some of the key classes.
If you’re not familiar, there are a few kinds of software tests, for example:
- unit tests (i.e., does this function work correctly)
- functional tests (i.e., does this code do what the user asks, e.g. optimizing a geometry, etc.)
- integration tests (i.e., can I do several things together like draw a molecule, optimize, save to ORCA)
- static analysis (i.e., looking for common coding flaws, e.g. lint, clang-tidy, Codacy, CodeQL, .. now some LLMs)
- fuzz testing
The latter works by generating random data and shoving it at the code to see if (when) it breaks. It’s used a lot in IO and security testing because it exposes lots of bugs (e.g. OSS-Fuzz).
I had set up some minimal fuzz tests for common formats last year, but they weren’t run routinely - mostly when I fired up my Linux box.
They’re now running every night and on request via GitHub action. I’m sure I’ll find a bunch of bugs in CJSON, SDF, XYZ, etc. - so you’ll see a variety of updates for those.
I coded up some new tests, to throw at the molecule classes, unit cells, etc. hopefully to proactively find bugs and crashes before 2.0.
If you have some suggestions for parts of the code to test (e.g. soon parts of the plugin infrastructure), please let me know.