Ok, nice. I assume the icons are the ones in avogadrolibs/avogadro/qtplugins/*tool/*.png for each tool. Are there vector graphic versions somewhere or only the pngs?
For the above .png files, I see there’s a 32x32 and a 64x64 version of each. Looking at the difference between screenshots on your system and mine here, it looks like on macOS the 64x64 versions are being used while on my system its just the 32x32 versions.
For what it’s worth, while you find the toolbar icons in my screenshots small, I feel they are actually a very appropriate size when you compare them to other toolbars. If you look at ChemDraw, Origin, classic Word/MS Office, LibreOffice, browsers, they all have similarly sized toolbar buttons. The iconography is typically simpler, however.