It's that time again - next Avogadro release

Development
1

Hi,

As I am sure many of you are aware it has been quite some time since we last
made a release and we have lots of new features, along with stability
enhancements, that are sat in trunk. I propose that we aim to make a release
on 16 May. This will give me a little time to integrate that as a snapshot in
KDE 4.1 as well as giving us all some time to get those features finished.

What I would like to know is what features you would like to get finished
before our next release? I plan on branching and tagging as before with a bug
fix only branch. I would like to get a more automated MinGW based Windows
build finished for this released but have currently stalled in my VM due to
formats not loading correctly. I have got the OpenBabel CMake build system
working quite nicely now though. Any help here would be appreciated.

I would like to get the orbital calculations working a little better, along
with tracking down as many crashers as possible. In that vein please file
bugs on crashers - I feel this is one area we really need to do some work and
I have become quite intimately acquainted with GDB over these last few weeks
trying to make that happen.

I will be trying to improve the way in which the engine configuration is
displayed and making a few UI changes over the next few days. There is a bug
in the text renderer that I can trigger sometimes where all the lighting gets
messed up - this would also be good to track down. I will finish moving
IsoGen to IsoSurface too - this has languished in my half finished local
changes in git for a while but should improve performance and API.

So, any objections to the proposed release date? Please point out any blocker
bugs or essential features for the next release. Let’s make this a great
release.

Thanks,

Marcus

2

Moin

So, any objections to the proposed release date? Please point out any blocker
bugs or essential features for the next release. Let’s make this a great
release.

A coordinated release with OpenBabel 2.2 would be cool :slight_smile:

Carsten

3

I gotta call timeout for a big suggestion and/or change.

OK. So in talking about OB3.0 we discussed the idea of unique
identifiers for each atom. I think there are two things here…

  1. Index is still OK and we should accept that index with change but
    it’s easy to look at an atom and say “atom 5” because it’s the one
    listed at 5.

  2. Id (Identifier). We need this RIGHT NOW in Avogadro because with our
    undo/redo architecture we’re starting to run into problems with drawing
    atoms. We can’t rely on Index numbers because they change WAY too much.
    We also can’t rely on pointers because through undo/redo and all that
    stuff, the pointer becomes invalid too easy.

I would bet you that there is a TON of bugs just coming from the
undo/redo architecture.

Thus, I have to suggest that we either disable undo/redo for the next
release OR we add a layer of identifiers on top of OpenBabel. This is
what I propose:

Either OpenBabel can add-in unique identifiers to it’s API which should
be fairly easy. You just need a vector that is always increasing of
pointers. The memory constraints will never be that big. A pointer is
roughly an unsigned integer on most architectures. Then, all you have
to do is allow creation of atoms with previous identifiers. That way if
you undo a draw, you delete ID105, then when you redo, you re-created
atom 105.

If this is something that OB isn’t interested in putting time into, I
think we could easily do the same thing in our Molecule wrapper class
and implement the same type of thing. For now, I don’t think it’s
important that ID’s are maintained after the program has exited, but for
the sake of undo/redo I guarantee this is causing a majority of our
crashes (bad pointers).

This might be a good place to mention it but I think OB might benefit
from using shared pointers in the future 3.0 but that is not my
immediate concern. Right now I want to fix some of these bugs with Avo
and dangling pointers.

–
Donald

(Tue, May 06, 2008 at 11:53:27AM +0200) Carsten Niehaus carsten.niehaus@gmail.com:

Moin

So, any objections to the proposed release date? Please point out any blocker
bugs or essential features for the next release. Let’s make this a great
release.

A coordinated release with OpenBabel 2.2 would be cool :slight_smile:

Carsten

This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
Don’t miss this year’s exciting event. There’s still time to save $100.
Use priority code J8TL2D2.
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone

Avogadro-devel mailing list
Avogadro-devel@lists.sourceforge.net
avogadro-devel List Signup and Options

4

Carsten Niehaus wrote:

Moin

So, any objections to the proposed release date? Please point out any blocker
bugs or essential features for the next release. Let’s make this a great
release.

A coordinated release with OpenBabel 2.2 would be cool :slight_smile:

I think we can at least get another OpenBabel beta out. As Avogadro is
beta I don’t see an issue with depending upon another beta release. I am
not sure OpenBabel 2.2 will be out in that time frame but I don’t think
it is something we should specifically delay a release for.

5

On Tue, May 6, 2008 at 10:23 AM, Marcus D. Hanwell marcus@cryos.org wrote:

Carsten Niehaus wrote:

Moin

So, any objections to the proposed release date? Please point out any blocker
bugs or essential features for the next release. Let’s make this a great
release.

A coordinated release with OpenBabel 2.2 would be cool :slight_smile:

I think we can at least get another OpenBabel beta out. As Avogadro is
beta I don’t see an issue with depending upon another beta release. I am
not sure OpenBabel 2.2 will be out in that time frame but I don’t think
it is something we should specifically delay a release for.

In terms of Avogadro support and bug reports I think the primary
requirement beyond just “it works” is that we have a single
corresponding OB “snapshot” (svn snapshot, beta, or full release) for
the Avogadro release. It’s much more difficult to help people or find
bugs if you have to account for various combinations of OB svn +
Avogadro. It also helps distro packagers since they’ll need to update
their OB packages.

-Jordan

6

On May 6, 2008, at 6:13 AM, Donald Ephraim Curtis wrote:

I gotta call timeout for a big suggestion and/or change.

Suggestions and/or changes are welcome. However, I don’t think it’s a
good idea for Open Babel to make such a large change in the timeframe
of 2.2 (which is intended to release by the end of the May). As is,
I’m worried we haven’t provided our usual level of testing for 2.2.0.

Certainly Open Babel will add this at some future time, but I would
say that Avogadro shouldn’t count on it ASAP.

Thus, I have to suggest that we either disable undo/redo for the next
release OR we add a layer of identifiers on top of OpenBabel. This is
what I propose:

There is, also, no reason that Avogadro can’t use smart pointers
internally.
http://doc.trolltech.com/4.4/qpointer.html
shared_ptr in GCC or Boost

That said, I don’t think there’s any reason Avogadro can’t include
permanent object IDs like you mention. This would likely remove many
undo/redo bugs.

OTOH, while there are bugs, I don’t come across them on a day-to-day
basis. I think we do have to be pragmatic sometimes. Is the current
SVN trunk better than the last release? Do we have regressions (i.e.,
new bugs that didn’t exist in 0.6.x?)?

These are questions we should answer first IMHO.

Cheers,
-Geoff

7

Well, here is the problem that smart pointers won’t resolve. If I draw
an atom, then draw a bonded atom off of that the bond has two options,
remember the pointer to the second atom drawn, or remember the index
numbers for each atom it is connected to. In the case of remembering
pointers, on an undo, the atom and bond get deleted. on redo when I
recreate the bond based on a pointer to the atom, that pointer is
invalid and I can never find the atom I am supposed to bond with.

On the other hand, if I use indexes with auto-add hydrogen this always
screws up the induces. For example: Draw two connected, then draw a
third connected to the second. the third atom you’ve drawn will be 6.
Then draw a bond between the third and the first, your third atom
becomes index 3.

Now the main problem I see with a unique index is aren’t we going to run
out of numbers sooner or later? Especially if auto-add hydrogens is
creating and deleting a bunch of hydrogens over and over again. The max
that any QList can hold (bad design by TT) is the max int which on my
systems is 2147483647. Although, a std::vector of pointers can hold up
to 2305843009213693951 which is longer than I expect any instance of
Avogadro will run.

Another idea I was bouncing around was if we have the draw tool hang on
to the atom it is using and not actually let Molecule delete, but rather
let it remove that atom from the molecule. I don’t know that OB
supports this. Anyways, the idea is that when an atom drawing gets
undone, it just hangs on to that atom, then when it gets redone it just
re-adds it rather than creating a new one, then the bond would be able
to hang on to it’s pointers. However, this would be problematic when we
start doing molecule duplications. Like “RemoveBond” does a complete
backup of the molecule and so if you drew two bonded molecules, then
deleted the bond, then undid that the pointers that your AddAtom/AddBond
commands had would not be valid to perform an undo because by undoing
the RemoveBond command you are replacing the Molecule with new atoms and
bonds… old pointers are invalid.

I apologies for my kinda rampant emails. I have not had a lot of time
to better think through a solution or how to explain the problem.
Hopefully some feedback can be given. I am working on just implementing
unique IDs in Avogadro.

I do agree that we should try to use more “smart pointers” in our code
at least where we have extensions/tools/engines using pointers.

I will be trying to resolve more crashes in the upcoming week. Is there
a better tool for debugging than GDB? It is so goddamn slow for startup
especially having to load all the plugins I just get tired of having to
restart after a crash too.

–
Donald

(Tue, May 06, 2008 at 03:43:47PM -0400) Geoffrey Hutchison geoff.hutchison@gmail.com:

On May 6, 2008, at 6:13 AM, Donald Ephraim Curtis wrote:

I gotta call timeout for a big suggestion and/or change.

Suggestions and/or changes are welcome. However, I don’t think it’s a
good idea for Open Babel to make such a large change in the timeframe
of 2.2 (which is intended to release by the end of the May). As is,
I’m worried we haven’t provided our usual level of testing for 2.2.0.

Certainly Open Babel will add this at some future time, but I would
say that Avogadro shouldn’t count on it ASAP.

Thus, I have to suggest that we either disable undo/redo for the next
release OR we add a layer of identifiers on top of OpenBabel. This is
what I propose:

There is, also, no reason that Avogadro can’t use smart pointers
internally.
http://doc.trolltech.com/4.4/qpointer.html
shared_ptr in GCC or Boost

That said, I don’t think there’s any reason Avogadro can’t include
permanent object IDs like you mention. This would likely remove many
undo/redo bugs.

OTOH, while there are bugs, I don’t come across them on a day-to-day
basis. I think we do have to be pragmatic sometimes. Is the current
SVN trunk better than the last release? Do we have regressions (i.e.,
new bugs that didn’t exist in 0.6.x?)?

These are questions we should answer first IMHO.

Cheers,
-Geoff

8

On May 6, 2008, at 5:13 PM, Donald Ephraim Curtis wrote:

Now the main problem I see with a unique index is aren’t we going to
run
out of numbers sooner or later? Especially if auto-add hydrogens is
creating and deleting a bunch of hydrogens over and over again. The
max
that any QList can hold (bad design by TT) is the max int which on my
systems is 2147483647.

That’s 2.1 billion. To take a conservative estimate, we might have
some large proteins (e.g., 100,000 or more) => around 21,000 actions.
Keep in mind, that Open Babel only recently added support for >65,000
atom molecules. Is it possible that we might want more? Yes.

Another idea I was bouncing around was if we have the draw tool hang
on
to the atom it is using and not actually let Molecule delete, but
rather
let it remove that atom from the molecule. I don’t know that OB
supports this.

Ah. Good point. That’s definitely something which can easily be
changed in OB. We’d add a new parameter to OBMol::DeleteAtom() or
similar to let the user actually delete the atom pointer. I’ll go do
this now.

I will be trying to resolve more crashes in the upcoming week. Is
there
a better tool for debugging than GDB? It is so goddamn slow for
startup
especially having to load all the plugins I just get tired of having
to
restart after a crash too.

There isn’t much you can do if the program crashes – you need to
restart the debugging run.

As far as improving speed during debugging, you can always delete some
of the plugins before running GDB.

Other debuggers? Apple actually uses GDB too, although the new Dtrace
framework is really helpful for some things. There’s also DDD (DDD - Data Display Debugger - GNU Project - Free Software Foundation (FSF)
)

Cheers,
-Geoff

9

Donald Ephraim Curtis wrote:

Well, here is the problem that smart pointers won’t resolve. If I draw
an atom, then draw a bonded atom off of that the bond has two options,
remember the pointer to the second atom drawn, or remember the index
numbers for each atom it is connected to. In the case of remembering
pointers, on an undo, the atom and bond get deleted. on redo when I
recreate the bond based on a pointer to the atom, that pointer is
invalid and I can never find the atom I am supposed to bond with.

I almost added unique atom identifiers in our Molecule class in pretty
much the way you described, but then found the guarded pointers
(QPointer) class and it fitted the bill for many of the crasher bugs I
was working on at the time. I haven’t been hitting the undo/redo support
too hard and so hadn’t spotted the crasher bugs with undo/redo. Do we
have bugs filed for this too? It would be good to make people aware of
these issues and recipes for reproducing. If you are working on this
then that sounds great and I will leave unique identifiers to you.

Hopefully the guarded pointer stuff I added in some tools will continue
to work, if not then let me know how to change it to work within the
modified Molecule class. It sounds to me like this stuff would live
quite happily inside our Molecule class and would allow us more freedom
to tweak it as needed.

On the other hand, if I use indexes with auto-add hydrogen this always
screws up the induces. For example: Draw two connected, then draw a
third connected to the second. the third atom you’ve drawn will be 6.
Then draw a bond between the third and the first, your third atom
becomes index 3.

Now the main problem I see with a unique index is aren’t we going to run
out of numbers sooner or later? Especially if auto-add hydrogens is
creating and deleting a bunch of hydrogens over and over again. The max
that any QList can hold (bad design by TT) is the max int which on my
systems is 2147483647. Although, a std::vector of pointers can hold up
to 2305843009213693951 which is longer than I expect any instance of
Avogadro will run.

It could be nice to have the very large size std::vector can support but
to be honest I would be shocked if QList was exhausted in a typical
session too :wink: It is always nice to have insanely big numbers to play
with if it isn’t too tough to utilise them.

Another idea I was bouncing around was if we have the draw tool hang on
to the atom it is using and not actually let Molecule delete, but rather
let it remove that atom from the molecule. I don’t know that OB
supports this. Anyways, the idea is that when an atom drawing gets
undone, it just hangs on to that atom, then when it gets redone it just
re-adds it rather than creating a new one, then the bond would be able
to hang on to it’s pointers. However, this would be problematic when we
start doing molecule duplications. Like “RemoveBond” does a complete
backup of the molecule and so if you drew two bonded molecules, then
deleted the bond, then undid that the pointers that your AddAtom/AddBond
commands had would not be valid to perform an undo because by undoing
the RemoveBond command you are replacing the Molecule with new atoms and
bonds… old pointers are invalid.

It sounds to me like unique identifiers are the way to go and like I say
I almost did it myself when I was on my crasher bug killing weekend, but
then found most of the bugs I was trying to fix were due to dangling
pointers. Unique identifiers would have worked there too rather than the
guarded pointers I used and I would not be averse to changing the
implementation there too if it turned out to be a more robust solution.

I apologies for my kinda rampant emails. I have not had a lot of time
to better think through a solution or how to explain the problem.
Hopefully some feedback can be given. I am working on just implementing
unique IDs in Avogadro.

I do agree that we should try to use more “smart pointers” in our code
at least where we have extensions/tools/engines using pointers.

I will be trying to resolve more crashes in the upcoming week. Is there
a better tool for debugging than GDB? It is so goddamn slow for startup
especially having to load all the plugins I just get tired of having to
restart after a crash too.

I use GDB and it works well for me although I agree the startup time can
be irritating. It is also useless when we get failed asserts etc as
there is no stack. Not sure if there is a decent solution to this other
than doing some exception checking in Avogadro so we can at least figure
out what functions are failing?

It would be good to make a release soon. I think trunk is a big
improvement over our last release, but if it was a matter of an extra
week to make Avogadro far more robust I would also be in favour of
delaying the release a little. I will be trying to figure out where some
of our rendering bugs are springing from - I suspect the text renderer
isn’t as robust as it could be but we shall see.

Thanks,

Marcus

10

Apart from the atom index undo/redo problems, better Python support
would also be a great improvement. Are we going to keep the current
boost.h to export our API in Python or is SWIG a better solution.
Ideally we should only expose some Avogadro specific functions (like
update(), …) and use openbabel’s python bindings for the rest. Is
anyone working on this?

Tim

11

On May 7, 2008, at 2:58 PM, Tim Vandermeersch wrote:

Apart from the atom index undo/redo problems, better Python support
would also be a great improvement. Are we going to keep the current
boost.h to export our API in Python or is SWIG a better solution.

The best solution may be SIP, which is used for PyQt, for example. One
concern I have is that we’re currently focusing on Python embedded in
libavogadro, while many people may want to wrap libavogadro into a
Python application.

I think, however, this discussion may be best postponed until after
0.8 is released.

-Geoff

12

It’s OK to discuss it a bit. I definitely don’t think there is much
time to change but I would like to chime in a bit.

My limited experience with SWIG and our libraries was not fun but in the
end it doesn’t matter what we use to “wrap” our libraries. We can embed
python farily easily no matter what we use to generate the python
interface. On the other hand, boost_python provides a lot of nice
template classes that make embedding very nice. There isn’t any reason
I can see that you can’t also generate python extensions using boost
python. The only difficulty may be in finding documentation on it. In
the end it’d be nice to use only a single library (less dependancies)
for both embedding and extending. Anyways, figuring out how to build a
shared library is all we need. Actually, now that I think about it,
python may be able to just load libavogadro given the correct boost
stuff. I don’t know enough. Love Boost.Python right now.

SIP could be cool if it’s easy to get embedding working and we could cut
boost out of the equation all together.

–
Donald

(Wed, May 07, 2008 at 04:20:34PM -0400) Geoffrey Hutchison geoff.hutchison@gmail.com:

On May 7, 2008, at 2:58 PM, Tim Vandermeersch wrote:

Apart from the atom index undo/redo problems, better Python support
would also be a great improvement. Are we going to keep the current
boost.h to export our API in Python or is SWIG a better solution.

The best solution may be SIP, which is used for PyQt, for example. One
concern I have is that we’re currently focusing on Python embedded in
libavogadro, while many people may want to wrap libavogadro into a
Python application.

I think, however, this discussion may be best postponed until after
0.8 is released.

-Geoff

This SF.net email is sponsored by the 2008 JavaOne(SM) Conference
Don’t miss this year’s exciting event. There’s still time to save $100.
Use priority code J8TL2D2.
http://ad.doubleclick.net/clk;198757673;13503038;p?http://java.sun.com/javaone

Avogadro-devel mailing list
Avogadro-devel@lists.sourceforge.net
avogadro-devel List Signup and Options

13

I agree about the wrapping.

What I do like about boost.python is the template classes they provide
which make embedding python much easier than if you had to do it
yourself. Python has a lot of shared pointer type objects (reference
counters etc) to handle their memory management, but when you embed it
you also have to handle all that stuff manually. Boost.python has
classes like object/handler that wrap around python’s C stuff to handle
this in a more C++ way.

To be honest, it’s not a huge deal if someone can get SWIG / SIP working
i’d be happy to fix up the embedded stuff.

–
Donald

(Thu, May 08, 2008 at 08:30:55AM -0400) Geoffrey Hutchison geoff.hutchison@gmail.com:

On May 7, 2008, at 5:29 PM, Donald Ephraim Curtis wrote:

My limited experience with SWIG and our libraries was not fun
…
On the other hand, boost_python provides a lot of nice
template classes that make embedding very nice. There isn’t any reason
I can see that you can’t also generate python extensions using boost
python.

My experience with SWIG and SIP are fairly good. There’s also the bonus
that they’re compile-time dependencies. I need to have SWIG for
generating Open Babel bindings, but users don’t. SWIG is also largely
automatic. You give it a bunch of headers and it goes to work.

Right now with Boost.Python, it’s manual. It’s actually worse than that –
it doesn’t handle overloading very well, so if you want to wrap
OBMol::SetTitle(char*) and OBMol::SetTitle(const std::string), you need to
do things like this:
void (OBMol::*SetTitle)(const char *) = &OBMol::SetTitle;

SWIG knows that std::string and char* should be the same thing in Python,
so it merges the two methods. It’s also smart enough to handle multiple
overloaded methods.

SIP looks like it’s custom-designed for Python and Qt/KDE. For example, the
PyKDE package is generated from the KDE headers. Like Boost.Python, it
requires manual coding. However, the format is close enough to a normal
header file that I think we could create a Perl or Python script which
saves most of the work.

Donald, perhaps you could explain what you like about Boost.Python?

Just my $0.02.
-Geoff

14

On May 7, 2008, at 5:29 PM, Donald Ephraim Curtis wrote:

My limited experience with SWIG and our libraries was not fun
…
On the other hand, boost_python provides a lot of nice
template classes that make embedding very nice. There isn’t any
reason
I can see that you can’t also generate python extensions using boost
python.

My experience with SWIG and SIP are fairly good. There’s also the
bonus that they’re compile-time dependencies. I need to have SWIG
for generating Open Babel bindings, but users don’t. SWIG is also
largely automatic. You give it a bunch of headers and it goes to work.

Right now with Boost.Python, it’s manual. It’s actually worse than
that – it doesn’t handle overloading very well, so if you want to
wrap OBMol::SetTitle(char*) and OBMol::SetTitle(const std::string),
you need to do things like this:
void (OBMol::*SetTitle)(const char *) = &OBMol::SetTitle;

SWIG knows that std::string and char* should be the same thing in
Python, so it merges the two methods. It’s also smart enough to handle
multiple overloaded methods.

SIP looks like it’s custom-designed for Python and Qt/KDE. For
example, the PyKDE package is generated from the KDE headers. Like
Boost.Python, it requires manual coding. However, the format is close
enough to a normal header file that I think we could create a Perl or
Python script which saves most of the work.

Donald, perhaps you could explain what you like about Boost.Python?

Just my $0.02.
-Geoff