Tackling Python’s project structure

I recently started playing with oTree (“a Django-based framework for implementing multiplayer decision strategy games.”) in order to code a school choice experiment. This lead me to dig deeper into Python.

One of the issue I got stuck on is probably the most basic issue of all : how to structure the folder and subfolders containing my Python code?

I was not looking for anything fancy here. The code for oTree experiment is mostly located in a file called  model.py and all I wanted was a way to

  • Define the Classes I use in model.py in separate files that I could later “import” in model.py, so that model.py does not become crazy long and unreadable
  • Group the files in which I define those classes into subfolders so that the folder containing model.py does not get overcrowded either.
  • E.g : because I am coding a school choice experiment, I wanted to have a subfolder containing all the solvers that I use to compute the final assignments based on preferences and priorities, and later import the corresponding classes in model.py in order to compute participants’ payoffs.

As basic as this problem may seem, the solution is not obvious (at least to me). Googeling  this kind of issues yields a ton of different solutions, and it’s easy to get lost.

The most understandable and functional solution I have found so far is : http://mikegrouchy.com/blog/2012/05/be-pythonic-__init__py.html

If we are to believe the title, this should also be a “Pythonic” solution, which means it should — hopefully — put you at peace with Python aesthetes (from what I understand, “Pythonic” means something like “in the spirit of Python” or “following the coding strandards which are considered good practice by Python’s community”, whatever that may mean).

Two warnings about the solution in the above links:

  • Although the solution claims to be “Pythonic”, I’ve often seen people argue against the use of imports of the form from subpackage import *. These people usually say that these tend to “clutter the namespace”. The truth is I have no idea what that refers to, so I don’t know whether the argument applies to the use of import *  described in the links. Anyways it is good to know that import * can sometimes freak people out, even if — like myself — you don’t quite understand why.
  • Suppose that following the solution described in the link, you’ve specified the __all__ variable in __init__ in the subpackage directory , and you run  from subpackage import * from file2.py (in the package directory).   Now say you want to access That_Class, one of the Classes from submodule2.py in the subpackage directory. Don’t be surprised if calling  That_Class() from file2.py returns the error NameError: name 'That_Class' is not defined. Indeed, you’ve only imported submodule2.py “as a single object”, and not all the Classes it contains individually. Therefore In order to call That_Class from file2.py, you will need to use submodule2.That_Class().