Mfcttrf

We have a program with many dependencies. Because the user may not want to use all of them, the program should skip over failed imports and only raise an error if absolutely necessary. e.g. if the user tries to call that class.

I have implemented a way of doing this but I'm not sure it is as good as it could be, it certainly seems somewhat inelegant. I presume there are also some overlooked bugs.

The important file structure is:

Program:
 - engines
 - __init__.py
 - base_engine.py
 - ...
 - psi4.py
 - rdkit.py
 - run.py

The base_engine.py contains the base class Engines which all the other engines (PSI4 in psi4.py, RDKit in rdkit.py ... ) inherit from, as well as a Default class.

If any class fails to import, Default is imported instead, but with the name of the class that failed to import (see the __init__.py file).

base_engine.py:

class Engines:
 """
 Engines base class containing core information that all other engines (PSI4, etc) will have.
 """

 def __init__(self, molecule):

 self.molecule = molecule


class Default:
 """
 If there is an import error, this class replaces the class which failed to be imported.
 Then, only if initialised, an import error will be raised notifying the user of a failed call.
 """

 def __init__(self, *args, **kwargs):
 # self.name is set when the failed-to-import class is set to Default.
 raise ImportError(
 f'The class self.name you tried to call is not importable; '
 f'this is likely due to it not being installed.')

__init__.py:

try:
 from .psi4 import PSI4
except ImportError:
 from .base_engine import Default as PSI4
 setattr(PSI4, 'name', 'PSI4')

try:
 from .rdkit import RDKit
except ImportError:
 from .base_engine import Default as RDKit
 setattr(RDKit, 'name', 'RDKit')

psi4.py:

from Program.engines.base_engine import Engines

# Example import that would fail
import abcdefg


class PSI4(Engines):
 """
 Writes and executes input files for psi4.
 """

 def __init__(self, molecule):

 super().__init__(molecule)

 def generate_input(self):

 ...

run.py:

from Program.engines import PSI4

PSI4('example_molecule').generate_input()

So now when classes are imported at the top of the run.py file, there is no problem even if there's an import error; if there's a failed import with PSI4 because abcdefg cannot be imported, Default is simply imported as PSI4 and given the name PSI4 via the setattr().

Then, only if that Default class is called the ImportError is raised and the user can see that the issue was with PSI4.

This seems to work quite well, even when there are multiple failed imports. We can also add extended error messages for each different failed import. Is this the best way of doing something like this, though? It gets quite messy since we have so many files in our engines package.

Please let me know if some relevant code has been omitted and I can add it back. Thanks for any help.

There is one issue with your approach which is, in the case of multiple import failure, only the last one can be properly reported:

>>> try:
... from .foo import Foo
... except ImportError:
... from .base_engine import Default as Foo
... setattr(Foo, 'name', 'Foo')
... 
>>> try:
... from .bar import Bar
... except ImportError:
... from .base_engine import Default as Bar
... setattr(Bar, 'name', 'Bar')
... 
>>> Bar()
Traceback (most recent call last):
…
ImportError: The class Bar you tried to call is not importable; this is likely due to it not being installed.
>>> Foo()
Traceback (most recent call last):
…
ImportError: The class Bar you tried to call is not importable; this is likely due to it not being installed.

You instead want to generate a new class with the proper message each time.

Something along the line should do:

def mock_missing(name):
 def init(self, *args, **kwargs):
 raise ImportError(
 f'The class name you tried to call is not importable; '
 f'this is likely due to it not being installed.')
 return type(name, (), '__init__': init)

Usage being:

try:
 from .foo import Foo
except ImportError:
 Foo = mock_missing('Foo')

That's an odd use of the word "default", which, etymologically speaking, means to remove failure, and generally refers to a fallback setting that "just works". I suggest calling it Missing instead.

This answer builds upon the solution presented by Mathias Ettinger

If you want to get rid of the try: ... except: ... boilerplate you could implement some kind of custom loader for your engines.

This is something I wrote quickly, which is likely far from perfect, but it works.

in __init__.py:

import importlib


def mock_missing(name, msg):
 def init(self, *args, **kwargs):
 raise ImportError(
 f'The class name you tried to call is not importable; '
 f'this is likely due to it not being installed. '
 f'Original reason: msg')
 return type(name, (), '__init__': init)


def try_load(engine, module):
 """Try to load an engine, return a mock class if it was not found

 Inspired by https://stackoverflow.com/a/10675081
 """
 try:
 module = importlib.import_module(module, __name__)
 return getattr(module, engine)
 except (ModuleNotFoundError, AttributeError) as ex:
 return mock_missing(engine, msg=str(ex))

To test it:

from engines import try_load

PSI4 = try_load("PSI4", ".psi4")
a = PSI4()

# module does not exist
try:
 PSI5 = try_load("PSI5", ".PSI5")
 b = PSI5()
 raise AssertionError("This should have failed!")
except ImportError as ex:
 print(str(ex))

# class does not exist in module
try:
 PSI6 = try_load("PSI6", ".psi4")
 c = PSI6()
 raise AssertionError("This should have failed!")
except ImportError as ex:
 print(str(ex))

This outputs:

The class PSI5 you tried to call is not importable; this is likely due to it not being installed. Original reason: No module named 'engines.psi5'
The class PSI6 you tried to call is not importable; this is likely due to it not being installed. Original reason: module 'engines.psi4' has no attribute 'PSI6'