Apply local workaround for scmdata + numpy 2.x StringDtype bug (blocking Python 3.12 support) #6

Background

PR #4 added Python 3.12 compat fixes (distutils removal, numpy 2.x scalar handling) but deliberately did not declare 3.12 support in pyproject.toml classifiers because one test still fails on 3.12:

tests/unit/test_fair1x_utils.py::test_emissions_to_ignore
  -> scmdata/groupby.py:61: np.issubdtype on a pandas StringDtype  -> TypeError

This is reached via ScmRun.convert_unit, which is called from _scmdf_to_emissions._get_fair_col_unit_context. So the FaIR 1.6 adapter is effectively broken on Python 3.12 until the upstream bug is fixed.

Upstream issue: openscm/scmdata#318.

What this issue tracks

When we decide we need Python 3.12 support before the upstream scmdata fix is released, apply a local workaround so we are not blocked. Options:

1. Pin scmdata to the last known-good version (or to a version with the fix once released).
2. Cast StringDtype meta to object at the point where openscm-runner constructs an ScmRun from CSV, in _scmdf_to_emissions.HistoricalWorldEmms.values_fair_units. Pre-normalises the dtype so the eventual convert_unit -> groupby call inside scmdata does not trip the bug.
3. Avoid convert_unit in this adapter and do the unit conversion via openscm_units directly. Larger refactor; aligns with the longer-term direction of leaning less on scmdata.

Drop the workaround once scmdata#318 ships in a release we depend on.

When

Not urgent for current AR7 work since we are on Python 3.9 fine today and IIASA climate-assessment is likely on similar versions for the next year+. Revisit when:

• We need 3.12 for a real use case, or
• scmdata releases the fix (upstream issue is likely to sit for months given the project's maintenance state)

Related

• PR Python 3.12 compat: distutils removal and numpy 2.x scalar #4 (this fork): Python 3.12 compat: distutils + numpy 2.x scalar
• ScmRun.groupby fails on Python 3.12 / numpy 2.x  openscm/scmdata#318 (upstream): StringDtype + numpy 2.x in groupby