Database drivers become optional extras #1626

Motivation

DataJoint supports two production database backends, PostgreSQL and MySQL, but the two drivers are packaged differently:

• pymysql (MySQL) is a core dependency. Every pip install datajoint installs it, including installs that only ever connect to PostgreSQL.
• psycopg2-binary (PostgreSQL) is the optional datajoint[postgres] extra.

import datajoint also imports pymysql at load time, through the adapter registry, and imports psycopg2 too when it happens to be installed. A process that never connects to a database still loads both drivers.

Treating the backends alike means each driver is installed only when it is needed. It also leaves room for more backends later without adding a hard dependency for each one.

The change

• Every database driver becomes an optional extra:

  • pip install "datajoint[postgres]" installs psycopg2-binary
  • pip install "datajoint[mysql]" installs pymysql (same >=0.7.2 floor as today)

• A bare pip install datajoint installs no database driver, and import datajoint loads none.

• Each adapter imports its driver when a connection is made. When the driver is missing, connecting raises a DataJointError that names the extra, e.g.:

  The MySQL backend requires the pymysql driver, which is not installed. Install it with: pip install "datajoint[mysql]"
  

  The PostgreSQL adapter raises the same kind of error. It used to raise ImportError.

• The PostgreSQL adapter imports the psycopg2 module, so either the psycopg2 or the psycopg2-binary distribution satisfies it.

• The test dependency group and the test and dev extras install every driver.

• conda has no extras, so the conda-forge package keeps listing the drivers as run requirements.

Migration for MySQL users

• Install the MySQL extra: pip install "datajoint[mysql]". In a requirements file, write datajoint[mysql] in place of datajoint.
• An upgrade in place keeps a pymysql that is already installed. A fresh environment built from a requirements file that lists only datajoint has no driver, and the first connection says which extra to install.
• Code that caught ImportError from PostgreSQLAdapter() should catch DataJointError.